Таблицу excel в sqlite

Довольно часто возникают задачи заполнения базы данных из каких-либо внешних источников. В данном примере показано как можно наполнить базу данных (SQLlite) данными из xlsx файла. 

В разработке я использую PyCharm 2018.3.3 Professoinal. Python 3.7

Используемые библитеки sqlite3 — работа с базой данных, openpyxl — работа с excel

1. Создайте новый проект Python и добавьте .py файл

2. Добавьте в проект базу данных и сделайте тестовый connect. 

3. Программу можно разбить на три части:

1. Подключение к базе и создание таблицы

2. Чтение xlsx файла с данными

3. Запись в базу и закрытие соединения

Исходный файл, который необходимо переложить в SQL

После запуска программы у вас должна появиться таблица cars с точно такими же данными

import os
import sqlite3
import openpyxl


def export_to_sqlite():
    '''Экспорт данных из xlsx в sqlite'''

    # 1. Создание и подключение к базе

    # Получаем текущую папку проекта
    prj_dir = os.path.abspath(os.path.curdir)

    a = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

    # Имя базы
    base_name = 'auto.sqlite3'

    # метод sqlite3.connect автоматически создаст базу, если ее нет
    connect = sqlite3.connect(prj_dir + '/' + base_name)
    # курсор - это специальный объект, который делает запросы и получает результаты запросов
    cursor = connect.cursor()

    # создание таблицы если ее не существует
    cursor.execute('CREATE TABLE IF NOT EXISTS cars (brand text, model text, distance int , year int)')

    # 2. Работа c xlsx файлом

    # Читаем файл и лист1 книги excel
    file_to_read = openpyxl.load_workbook('Cars.xlsx', data_only=True)
    sheet = file_to_read['Sheet1']

    # Цикл по строкам начиная со второй (в первой заголовки)

    for row in range(2, sheet.max_row + 1):
        # Объявление списка
        data = []
        # Цикл по столбцам от 1 до 4 ( 5 не включая)
        for col in range(1, 5):
            # value содержит значение ячейки с координатами row col
            value = sheet.cell(row, col).value
            # Список который мы потом будем добавлять
            data.append(value)

    # 3. Запись в базу и закрытие соединения

        # Вставка данных в поля таблицы
        cursor.execute("INSERT INTO cars VALUES (?, ?, ?, ?);", (data[0], data[1], data[2], data[3]))

    # сохраняем изменения
    connect.commit()
    # закрытие соединения
    connect.close()


def clear_base():
    '''Очистка базы sqlite'''

    # Получаем текущую папку проекта
    prj_dir = os.path.abspath(os.path.curdir)

    # Имя базы
    base_name = 'auto.sqlite3'

    connect = sqlite3.connect(prj_dir + '/' + base_name)
    cursor = connect.cursor()

    # Запись в базу, сохранение и закрытие соединения
    cursor.execute("DELETE FROM cars")
    connect.commit()
    connect.close()


# Запуск функции
export_to_sqlite()

Input file

Our API accepts one or multiple Excel .XLSX files as input.

Max file size for web uploads: 50 GB
Register to upload big files via Amazon S3.

Output file

The API will return a ZIP archive that contains a .SQLITE database file.

Conversion methods

Using our Java tool

Download the

RebaseData client Java tool
.
To convert your database using RebaseData, run the following command:

                    java -jar client-0.0.5.jar convert --output-format=sqlite file.xlsx output-dir/
                

Using CURL

Replace file.xlsx with the path to the Excel file you want to convert.

The file output.zip will contain a .SQLITE file with your data or the error message if something went wrong.

curl -F files[]=@file.xlsx 'https://www.rebasedata.com/api/v1/convert?outputFormat=sqlite&errorResponse=zip' -o output.zip

How long does it take?

The conversion process takes between 15 seconds and multiple minutes. It depends on the size of the database.

You can also use your favourite tool

  • Read Excel using PHP
  • Read Excel using Python
  • Read Excel using Ubuntu

Why use RebaseData?

  • Strong security.
  • Convert online without installing Microsoft Excel.
  • Works with Windows, Mac and Linux.
  • Use an API to convert your databases.
  • Professional support.

Terms

  • We don’t guarantee for the success or correctness of the conversion
  • You are only allowed to convert your own database files
  • By using RebaseData, you agree to our general terms

Excel-to-sqlite

Convert excel documents to sqlite!

Dependencies

  • Sqlite3: 5.0.0
  • Xlsx: 0.16.4

Documentation / usage

Navigation

  • Main
    • Loading your excel
    • Sheet names
    • Read One Sheet
      • Get sheet data
      • Save sheet to sqlite
    • Read All Sheets
      • Get data of all sheets
      • Save whole excel
    • _xlsx

Firstly, load the package.

const excelToSqlite = require("excel-to-sqlite");

The export of excel-to-sqlite is one function, with one parameter: excelPath.

Loading your excel

The path must me absolute, like __dirname. This example uses path.

const path = require("path");
let excelPath = path.join(__dirname, "test.xlsx"); // File "test.xlsx" in the current directory
let excel = excelToSqlite(excelPath);

Sheet names

After you used the excelToSqlite function, you can use the property sheets to get a string array of all the sheet names.

let sheets = excel.sheets;
console.log(sheets);

Read One Sheet

To read one sheet, use the method readSheet afer you used the excelToSqlite function. It has one parameter: The name of the sheet.

The name of the sheet is case-sensitive!

let sheet = excel.readSheet("Sheet1"); // Read "Sheet1".

Data

To get the data of the current sheet, use the property data on sheet.

To get the json data, use the method getJSON.

let json = sheet.getJSON();

Output

[
  Sheet1: [
    {column_1_name: "value", column_2_name: "value", ...},
    {column_1_name: "value", column_2_name: "value", ...}
  ]
]

Save sheet to SQLITE

To save the current sheet to sqlite, use the saveTo method. It has one parameter: The name of the sqlite file.

Warning: When saving to a database, a table with the same name as the current sheet will get deleted.

To change the name of the table in the sqlite database, set sheet._name to another string before calling the saveTo function.

This function returns a promise, that will be resolved with a database object of sqlite3.

// Save without changing the name
sheet.saveTo("database.sqlite").then((database) => {
  console.log(`Sheet ${sheet._name} saved in "database.sqlite"!`);
});

// Change the name for the table and then save
sheet._name = "lol";
sheet.saveTo("database.sqlite").then((database) => {
  console.log(`Sheet ${sheet._name} saved in "database.sqlite"!`);
});

Read All Sheets

To read all the sheets in an excel, use the readAll method.

let sheets = excel.readAll();

Data

To get all the data of all the sheets, use the property data. To convert your data to JSON, use the method getJSON.

let data = sheets.data;
let json = sheets.getJSON();

Output:
See Output

Saving Whole Excel

To save the whole excel in sqlite, use the saveTo method. It has one parameter, the name of the database.

This function returns a promise, that will be resolved with a database object of sqlite3.

Warning: When saving to a database, a table with the same name as a sheet will get deleted.

sheets.saveTo("database.sqlite").then((database) => {
  console.log("Whole database saved in sqlite!");
});

_xlsx

The property _xlsx is the output of xlsx.readFile.

Improve Article

Save Article

Like Article

  • Read
  • Discuss
  • Improve Article

    Save Article

    Like Article

    A tiny, quick, self-contained, highly reliable, fully-featured serverless, zero-configuration, transactional SQL database engine is implemented by SQLite, an in-process C language library. The most popular database engine worldwide is SQLite. The public domain status of SQLite’s source code allows for its use for any purpose, whether they are public or private. All smartphones, the majority of laptops, and many other daily-used programmes include SQLite as standard equipment. The most widely used database in the world, SQLite has more uses than we can count, including some well-known programmes. So let’s first talk about why and How to connect Excel to SQLite.

    Why Connect Excel to SQLite?

    Numerous accounting software programmes are used by businesses to manage their finances. While some companies utilise commercial accounting or financial software, others develop their own internal systems using programmes like Microsoft Access or Visual Basic. Although the features of accounting applications vary widely, a back-end database is a characteristic that almost all of them employ to index and store data.

    The open-source SQLite engine is widely used in custom accounting software to manage data. You might want to export data from your business accounting tool for use in Microsoft Excel analysis if it connects to an SQLite database. You can use Excel’s built-in features to retrieve data if the application lacks built-in control to export it and make spreadsheets.

    How to Connect Excel to SQLite?

    • On the Ch-Werner.de website, download the SQLite ODBC driver. Get it installed and make sure whether this is done properly or not on your PC. 
    • Launch Excel from Microsoft. Open the worksheet you wish to add data from the SQLite database to or create a new one if necessary.
    • On the Excel ribbon menu, select “Data.” Over the “Get External Data” section label, click the “From Other Sources” button. When the drop-down selection appears, select “From Microsoft Query”.

    Selecting-data

    • Now click on Microsoft Query just as illustrated,

    Clicking-on-microsoft-query

    • Turn off the option to “Use the Query Wizard to Create/Edit” Queries. After selecting “New Data Source,” click “OK.”

    Selecting-new-data-source

    • Turn on “SQLite3 ODBC Driver” by selecting it. Press “OK.” The tables from your SQLite database are displayed in a new query window that appears. Click “Close” after you have highlighted and chosen the table holding the data you wish to import into Excel.

    Turning-on-SQLite3-odbc-driver

    • Click the Fields list’s “*” icon. Then, to import every field from the table into Excel, click the “>” button in the window’s centre. Choose the desired data filtering choices. Excel is instructed through filters to only get information from the SQLite data table that satisfies certain criteria. For instance, your SQLite data table likely has a column with the name “old telephone” or something similar if it contains a list of customers along with their addresses or other contact information. Applying a filter will allow you, for example, to only obtain data for clients with the area code “211”. In the “Column to Filter” window, select the “old telephone” or a field with a similar name. In the filter list, select “Contains”. Click “Next” after entering “211” in the field next to the “Contains” choice.

    clicking-on-fields-list

    • Either enable the ascending sort order option and click “Next” to sort the records in that direction, which is the default By clicking, the “Return Data to Microsoft Excel” option can be chosen. To end the fresh query window, click the “Finish” button.

    ending-fresh-query-window

    • In the Import Data window, click to make the “Table” and “Existing Worksheet” options active. To make the data table from the SQLite database display in a specific cell on the Excel spreadsheet, click the empty cell. Select “OK” from the menu. Based on any filters you applied to the query, Excel builds and presents a new table with the records it has retrieved from the SQLite data table.

    New-table-presented

    Everything is now set up correctly. As necessary, modify or format the new data table from the SQLite database. Delete your worksheet.

    Like Article

    Save Article

    Cover image for Creating an SQLite Database From Excel Workbook.

    A database is an organized collection of rows and columns with data that are related to each other. Sometimes you have an excel worksheet and would like to query it using SQL. This post takes you through using Pandas to create an SQLite database from an excel sheet. So let’s jump into it .

    Step 1:Load the dataset to Pandas

    To load the dataset from excel to pandas use the following commands:

       ```python 
          import pandas as pd 
          import sqlite3
          df=pd.read_excel('path to file')```
    

    Enter fullscreen mode

    Exit fullscreen mode

    Import pandas as pd tells python to bring the pandas data analysis library into your current environment. pd portion of the code tells python to give pandas the alias pd. This means you can use pandas by typing pd. Import sqlite3 provides an SQL-like interface to read,query and write SQL databases from python.df=pd.read_excel(‘path to file’) tells pandas to read the excel document in a specific location.

    Step 2: Create SQLite database using sqlite3.connect()

    The following sample code creates an empty database file called Day.db and assigns it to a variable db_conn

       ```db_conn=sqlite3.connect('Day.db')```
    

    Enter fullscreen mode

    Exit fullscreen mode

    This provides an interface for working with the SQLite database.

    Step 3:Create a cursor object to run the SQL code that will create the data tables.

    Cursor is an instance using which you can invoke methods that execute SQLite statements from the result sets of the queries i.e makes the connection for executing SQL queries on the SQLite database.

    ```cur=db_conn.cursor()```
    

    Enter fullscreen mode

    Exit fullscreen mode

    Step 4:Create a table

    This is a command used to create a table called DayofWeeek with new columns Day and Sum.The data type for each column is specified to the right of the column name.

    ```create_table="CREATE TABLE DayofWeek(Day TEXT,Sum INT)```
    

    Enter fullscreen mode

    Exit fullscreen mode

    The column names must match those stored in the dataframe.

    Step 6:Moving from pandas dataframe to SQL table

    .to_sql command is used to write records stored in a dataframe to a SQL database.DayofWeek specifies the the name of SQL table created.
    if_exists=’append’ inserts new values into the existing table DayofWeek.index=False to avoid bringing the dataframe index as a column.

    ```df.to_sql('DayofWeek',db_conn,if_exists='append', index=False)```
    

    Enter fullscreen mode

    Exit fullscreen mode

    You can run your sql commands on pandas for example:

    
    pd.read_sql("SELECT * FROM DayofWeek",db_conn)
    
    
    

    Enter fullscreen mode

    Exit fullscreen mode

    Like this post? Please share to your friends:
  • Таблице excel файл фамилия
  • Таблицу ms excel нельзя вставить в файл формата
  • Таблицах excel двумя массивами массивом
  • Таблицу excel для расчета точки безубыточности
  • Таблицами ms excel в openoffice org