I have 1000 CSV files. Each CSV file is between 1 and 500 MB and is formatted the same way (i.e. same column order). I have a header file for column headers, which match my DynamoDB table's column names. I need to import those files into a DynamoDB table. What's the best way / tool to do so?
I can concatenate those CSV files into a single giant file (I'd rather avoid to though), or convert them into JSON if needed. I am aware of the existence of BatchWriteItem so I guess a good solution would involve batch writing.
- The DynamoDB table has two columns: first_name, last_name
- The header file only contains:
- One CSV file looks like
John,Doe Bob,Smith Alice,Lee Foo,Bar