0% found this document useful (0 votes)

113 views

How To Combine Multiple CSV Files With 8 Lines of Code PDF

Uploaded by

Kaleab Tekle

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

113 views

How To Combine Multiple CSV Files With 8 Lines of Code PDF

Uploaded by

Kaleab Tekle

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 6

8/17/2019 How to combine multiple CSV files with 8 lines of code

1 FEBRUARY 2019 / #PYTHON

How to combine multiple CSV

les with 8 lines of code
Ekapope Viriyakovithya
Love to automate routine stuff, former oil eld engineer. https://ekapope.github.io/

Why do you need this?

Manually copy-pasting is ne if you don’t have too many les to work
with.
https://www.freecodecamp.org/news/how-to-combine-multiple-csv-files-with-8-lines-of-code-265183e0854/ 1/6
8/17/2019 How to combine multiple CSV files with 8 lines of code

Menu
But imagine if you have 100+ les to concatenate — are you willing to
do it manually? Doing this repetitively is tedious and error-prone.

If all the les have the same table structure (same headers & number
of columns), let this tiny Python script do the work.

Step 1: Import packages and set the

working directory
Change “/mydir” to your desired working directory.

import os
import glob
import pandas as pd
os.chdir("/mydir")

Step 2: Use glob to match the pattern ‘csv’

Match the pattern (‘csv’) and save the list of le names in the
‘all_ lenames’ variable. You can check out this link to learn more about
regular expression matching.

https://www.freecodecamp.org/news/how-to-combine-multiple-csv-files-with-8-lines-of-code-265183e0854/ 2/6
8/17/2019 How to combine multiple CSV files with 8 lines of code

extension = 'csv'
all_filenames = [i for i in glob.glob('*.{}'.format(extension))]

Step 3: Combine all files in the list and

export as CSV
Use pandas to concatenate all les in the list and export as CSV. The
output le is named “combined_csv.csv” located in your working
directory.

#combine all files in the list

combined_csv = pd.concat([pd.read_csv(f) for f in all_filenames ])
#export to csv
combined_csv.to_csv( "combined_csv.csv", index=False, encoding='utf-8-sig

encoding = ‘utf-8-sig’ is added to overcome the issue when exporting

‘Non-English’ languages.

And…it’s done!

This article was inspired by my actual everyday problem, and the

coding structure is from a discussion on stackover ow. The completed
script for this how-to is documented on GitHub.

Thank you for reading. Please give it a try, have fun and let me know
your feedback!

If you like what I did, consider following me on GitHub, Medium, and

https://www.freecodecamp.org/news/how-to-combine-multiple-csv-files-with-8-lines-of-code-265183e0854/ 3/6
8/17/2019 How to combine multiple CSV files with 8 lines of code

Twitter. Make sure to star it on GitHub :P Menu

Show comments

Countinue reading about

Python
A Python project in 30 lines of code: how to set up an SMS notification when your
favorite Twitcher is streaming

How to classify butterflies with deep learning in Keras

An Introduction to Unit Testing in Python

See all 236 posts →

https://www.freecodecamp.org/news/how-to-combine-multiple-csv-files-with-8-lines-of-code-265183e0854/ 4/6
8/17/2019 How to combine multiple CSV files with 8 lines of code

#JAVASCRIPT

Matterhorn in Depth — Project Aspects Explained

ETHAN ARROWOOD 7 MONTHS AGO

#WEB DEVELOPMENT

10 tips for success when you’re learning to code

JESSICA CHAN 7 MONTHS AGO

freeCodeCamp is a donor-supported tax-exempt 501(c)(3) nonpro t organization (United States Federal

https://www.freecodecamp.org/news/how-to-combine-multiple-csv-files-with-8-lines-of-code-265183e0854/ 5/6
8/17/2019 How to combine multiple CSV files with 8 lines of code

Tax Identi cation Number: 82-0779546)

Menu
Our mission: to help people learn to code for free. We accomplish this by creating thousands of videos,
articles, and interactive coding lessons - all freely available to the public. We also have thousands of
freeCodeCamp study groups around the world.

Donations to freeCodeCamp go toward our education initiatives, and help pay for servers, services, and
staff. You can make a tax-deductible donation here.

Our Nonpro t Best Tutorials

About Python Tutorial

Donate Git Tutorial
Shop Linux Tutorial
Alumni Network JavaScript Tutorial
Open Source React Tutorial
Support HTML Tutorial
Sponsors CSS Tutorial
Academic Honesty SQL Tutorial
Code of Conduct Java Tutorial
Privacy Policy Angular Tutorial
Terms of Service WordPress Tutorial
Copyright Policy Bootstrap Tutorial

Best Examples Trending Reference

Python Example 2019 Web Developer Roadmap
JavaScript Example Linux Command Line Guide
React Example Git Reset and Git Revert
Linux Example Git Merge and Git Rebase
HTML Example JavaScript Array Map
CSS Example JavaScript Array Reduce
SQL Example JavaScript Date
Java Example JavaScript String Split
Angular Example CSS Flexbox Guide
jQuery Example CSS Grid Guide
Bootstrap Example Create a Linux Sudo User
PHP Example How to Set Up SSH Keys

https://www.freecodecamp.org/news/how-to-combine-multiple-csv-files-with-8-lines-of-code-265183e0854/ 6/6