Had an application where I had to convert a ton of CSV logs to JSON format for ingestion by another system. I didn’t have good luck with Python, so I tried PHP. Posting this work here in the hopes it will help someone else. Keep in mind, this is a basic example and the following warning applies should this be considered for production use:
Stern Warning: This example assumes the source of the CSV files is doing any error handling before writing records in the CSV source files. It is also assumed that the source CSV files each have a KEY as the first row in each file so that fields in the data rows are properly represented. If this is not the case in your application, the below example will require further enhancement. Only use the example as shown if you are confident that your application/process creating the CSV source input files is doing proper data validation and handling!
The Code: (convert.php)
<?php
// php function to convert csv to json format. Takes 2 arguments:
// arg1 = input csv file arg2 = output json file
$fin = $argv[1];
$fout = $argv[2];
function csvToJson($fin,$fout) {
// File Handles
if (!($fp = fopen($fin, 'r'))) {
die("Can't open file...");
}
$fo = fopen($fout, 'a');
// Processing
$key = fgetcsv($fp,"0",",");
while ($row = fgetcsv($fp,"0",",")) {
$json = array_combine($key, $row);
$json = json_encode($json) . "\n";
fwrite($fo, $json);
}
// release file handles
fclose($fp);
fclose($fo);
}
// phone a friend
csvToJson($fin,$fout);
?>
To use this script, here’s an example from my shell:
for FILE in /var/convert/csv/*.csv; do php /var/convert/convert.php $FILE /var/convert/json/$(basename $FILE).json; done
If this helped you, or if you have suggestions for refinement, please let me know in the comments below! 🙂