Соединить excel файлы в один python

Normally, we’re working with Excel files, and we surely have come across a scenario where we need to merge multiple Excel files into one. The traditional method has always been using a VBA code inside excel which does the job but is a multi-step process and is not so easy to understand. Another method is manually copying long Excel files into one which is not only time-consume, troublesome but also error-prone. 

This task can be done easily and quickly with few lines of code in Python with the Pandas module. First, we need to install the module with pip. So let’s get the installation out of our way. 

Use the following command in the terminal:

pip install pandas

Method 1: Using dataframe.append()

Pandas dataframe.append() function is used to append rows of other dataframe to the end of the given dataframe, returning a new dataframe object. Columns not in the original dataframes are added as new columns and the new cells are populated with NaN value.

Syntax : DataFrame.append(other, ignore_index=False, verify_integrity=False, sort=None)

Parameters :

• other : DataFrame or Series/dict-like object, or list of these
• ignore_index : If True, do not use the index labels. default False.
• verify_integrity : If True, raise ValueError on creating index with duplicates. default False.
• sort : Sort columns if the columns of self and other are not aligned. default False.

Returns: appended DataFrame

Example:

Excel Used: FoodSales1-1, FoodSales2-1
 

Output :

Method 2: Using pandas.concat()

The pandas.concat() function does all the heavy lifting of performing concatenation operations along with an axis of Pandas objects while performing optional set logic (union or intersection) of the indexes (if any) on the other axes.

Syntax: concat(objs, axis, join, ignore_index, keys, levels, names, verify_integrity, sort, copy)

Parameters:

• objs: Series or DataFrame objects
• axis: axis to concatenate along; default = 0 //along rows
• join: way to handle indexes on other axis; default = ‘outer’
• ignore_index: if True, do not use the index values along the concatenation axis; default = False
• keys: sequence to add an identifier to the result indexes; default = None
• levels: specific levels (unique values) to use for constructing a MultiIndex; default = None
• names: names for the levels in the resulting hierarchical index; default = None
• verify_integrity: check whether the new concatenated axis contains duplicates; default = False
• sort: sort non-concatenation axis if it is not already aligned when join is ‘outer’; default = False
• copy: if False, do not copy data unnecessarily; default = True

Returns: a pandas dataframe with concatenated data.

Example:

In the last example, we worked on only two Excel files with a few rows. Let’s try merging more files each containing approximately 5000 rows and 7 columns. We have 5 files BankE, BankD, BankC, BankB, BankA having historical stock data for respective bank. Let’s merge them into a single ‘Bank_Stocks.xlsx’ file. Here we are using the pandas.concat() method.

Output :

Solution with openpyxl only (without a bunch of other dependencies).

This script should take care of merging together an arbitrary number of xlsx documents, whether they have one or multiple sheets. It will preserve the formatting.

There’s a function to copy sheets in openpyxl, but it is only from/to the same file. There’s also a function insert_rows somewhere, but by itself it won’t insert any rows. So I’m afraid we are left to deal (tediously) with one cell at a time.

As much as I dislike using for loops and would rather use something compact and elegant like list comprehension, I don’t see how to do that here as this is a side-effect show.

Credit to this answer on copying between workbooks.

#!/usr/bin/env python3

#USAGE
#mergeXLSX.py <a bunch of .xlsx files> ... output.xlsx
#
#where output.xlsx is the unified file

#This works FROM/TO the xlsx format. Libreoffice might help to convert from xls.
#localc --headless  --convert-to xlsx somefile.xls

import sys
from copy import copy

from openpyxl import load_workbook,Workbook

def createNewWorkbook(manyWb):
    for wb in manyWb:
        for sheetName in wb.sheetnames:
            o = theOne.create_sheet(sheetName)
            safeTitle = o.title
            copySheet(wb[sheetName],theOne[safeTitle])

def copySheet(sourceSheet,newSheet):
    for row in sourceSheet.rows:
        for cell in row:
            newCell = newSheet.cell(row=cell.row, column=cell.col_idx,
                    value= cell.value)
            if cell.has_style:
                newCell.font = copy(cell.font)
                newCell.border = copy(cell.border)
                newCell.fill = copy(cell.fill)
                newCell.number_format = copy(cell.number_format)
                newCell.protection = copy(cell.protection)
                newCell.alignment = copy(cell.alignment)

filesInput = sys.argv[1:]
theOneFile = filesInput.pop(-1)
myfriends = [ load_workbook(f) for f in filesInput ]

#try this if you are bored
#myfriends = [ openpyxl.load_workbook(f) for k in range(200) for f in filesInput ]

theOne = Workbook()
del theOne['Sheet'] #We want our new book to be empty. Thanks.
createNewWorkbook(myfriends)
theOne.save(theOneFile)

Tested with openpyxl 2.5.4, python 3.4.

Файлы к уроку:

• Для спонсоров Boosty
• Для спонсоров VK

• YouTube
• VK

Ссылки:

• Страница курса
• Плейлист YouTube
• Плейлист ВК

Описание

У нас есть несколько однообразных книг Excel. В каждой книге Excel находится несколько листов с единой структурой. Нам нужно одновременно объединить все книги и все листы в них в одну таблицу.

Решим эту задачу с помощью модуля pandas.

Решение

Сначала импортируем нужные модули.

# Импорт модулей
import pandas as pd
import os
import glob

Укажем директорию, в которой находятся файлы.

# Сменим директорию
os.chdir('data')

Создадим список книг для объединения.

# Список файлов Excel для объединения
xl_files = glob.glob('*.xlsx')

Создадим датафрейм, в который запишем таблицы.

# Читаем каждую книгу объединяем все листы в один датафрейм
combined = pd.DataFrame()

Читаем каждый файл и объединяем все таблицы.

# Цикл по файлам
for xl_file in xl_files:
    # Создать объект ExcelFile
    xl_file_obj = pd.ExcelFile(xl_file)
    # Цикл по листам
    for sheet_name in xl_file_obj.sheet_names:
        # Прочитать нужный лист книги
        data = pd.read_excel(xl_file_obj,
                             sheet_name=sheet_name)
        # Создадать столбец с названием книги
        data['workbook'] = xl_file
        # Создать столбец с названием листа
        data['sheet'] = sheet_name
        # Дописать в датафрейм combined
        combined = combined.append(data)

Запишем результат в книгу Excel.

combined.to_excel('sales_combined.xlsx',
                  index=False)

Примененные функции

• os.getcwd
• os.chdir
• glob.glob
• pandas.ExcelFile
• pandas.DataFrame
• pandas.read_excel
• pandas.DataFrame.append
• pandas.DataFrame.to_excel

Курс Python Практический