The CSV for the above JSON will look like this:.

I have to write the data to the CSV file as described above.

As far I saw, the JSON data will be at most 7 level-nested so we could have something like:.

I can't change the structure of the JSON file.

I don't know where the JSON file comes from so I have to stick with it and process it as is.

Write_json_to_csv(flattened_json, CSV_PATH) W = csv.DictWriter(out_file, flattened_json.keys())įlattened_json = flattenjson(json_data, '.') Of the csv and the values.well, the values ^_^.įlattened_json (dict): Flattened JSON object. The JSON file that I'm trying to convert is 60 GB and I have 64GB of RAM.įlatten a simple JSON by prepending a delimiter to nested children.ĭelim (str): Delimiter for nested children (e.g: '.')ĭef write_json_to_csv(flattened_json, csv_path): (perhaps not load everything at once into memory - even if it's gonna be slower).

I'd like to improve the performance of my solution as much as possible. As you can see, my solution is not memory efficient since I'm reading all the file into memory. I won't describe the functionality in too much detail since I have reasonable docstrings for that. And as someone kindly commented and pointed out it appears i am iterating over keys instead of the different dictionaries, and this is where I get stuck because im not sure what to do? From my understanding it looks like the JSON is split into 3 different arrays with JSON object for each, but that does not appear to be the case according to the structure? I apologize for the length of the code, but I want some resemblance of context to what i am trying to accomplish.Simply take a JSON file as input and convert the data in it into a CSV file. Returns the following error: TypeError: string indices must be integers. The code im using to try to do this: f = open('output.json') 'delivery_point_barcode': '997059901010',Īfter storing the API output the trouble is trying to parse the returned output (Sample output) into a CSV format. R = requests.post(url, data=dict_ready, headers=headers) Here is my current code for reference: import pyodbcĬonn = nnect('DSN=DSN', autocommit=True) I am currently trying to convert a JSON output from an API request to a CSV format so i can store the results into our database.

Python json csv