[Vaex] big data (.csv) covert to hdf5
2020. 9. 2. 13:53ㆍ분석 Python/Vaex
728x90
import vaex
for i, df in enumerate(vaex.from_csv('taxi.csv', chunk_size=100_000)):
df = df[df.passenger_count < 6]
df.export_hdf5(f'taxi_{i:02}.hdf5')vaex.readthedocs.io/en/latest/api.html#vaex.open
API documentation for vaex library — vaex 3.0.0 documentation
Parameters: x – expression or list of expressions, e.g. df.x, ‘x’, or [‘x, ‘y’] y – expression or list of expressions, e.g. df.x, ‘x’, or [‘x, ‘y’] limits – description for the min and max values for the expressions, e.g. ‘minma
vaex.readthedocs.io
728x90
'분석 Python > Vaex' 카테고리의 다른 글
| [Vaex] Join on Multiple Columns (0) | 2020.09.02 |
|---|---|
| [Vaex 1.0.0-beta.6] Groupby 사용해보기 (0) | 2020.08.29 |
| [Vaex 1.0.0-beta.6] Virtual Column 알아보기 (0) | 2020.08.29 |
| [Vaex 1.0.0-beta.6] Virtual column 생성 후 pandas로 변경하기 (0) | 2020.08.29 |
| [Vaex 1.0.0-beta.6] how to split data from str to list. (0) | 2020.08.29 |