Pandas read csv chunk

1 개요

Pandas read csv chunk

import pandas as pd
filename = 'mydata.csv'
chunksize = 100000
df = pd.DataFrame()
with pd.read_csv(filename, chunksize=chunksize) as reader:
  for chunk in reader:
    df = pd.concat([df,chunk])
    print(df.shape)
df.to_feather('mydata.feather')

2 같이 보기

• pd.read_csv()
• pd.concat()