Dbf to excel python

Русские Блоги

[Python] Python преобразует файлы dbf в Excel и получает максимальное количество строк и столбцов в Excel.

Каталог статей

Преобразование dbf в файл Excel (код завершен)

Возможное решение

Невозможные решения, найденные в Интернете

Первоначально я искал в Интернете метод преобразования базы данных dbf в Excel на Python, и я увидел много .
Я не экспериментировал, поэтому я искал код, который Excel считывает указанное количество строк. Я не ожидал, что после того, как Excel прочитает указанный диапазон строк, код для преобразования dbf в Excel будет реализован онлайн. Он не может пройти . и я вижу в Интернете тот же метод .

Это может быть проблема моей среды, использующей Python 3.6 для разработки в Windows.
Но вам нужно вызвать win32com, а это значит, что он не будет работать в Linux. Его можно разработать только в Windows.

Установить win32com

Необходимо установить win32com
Прямая установка не работает

Можно использовать следующее:

После завершения установки:

Сообщение об ошибке:

Python написать Excel

Считайте количество строк и столбцов в Excel

Считайте максимальное количество строк в Excel

openpyxl не поддерживает файлы xls

Сообщение об ошибке:

Итак, openpyxl не поддерживает xls .

Доступные методы панды.

Что можно использовать:
Используйте панды для выполнения

Итак, в приведенном выше коде, если имя листа записано как имя_листа, предупреждающей информации нет.

Прочтите количество столбцов

По-прежнему используйте информацию о модуле pandas:

Источник

rafaelferrero/dbf2xls

Use Git or checkout with SVN using the web URL.

Work fast with our official CLI. Learn more.

Sign In Required

Please sign in to use Codespaces.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio Code

Your codespace will open once ready.

There was a problem preparing your codespace, please try again.

Latest commit

Git stats

Files

Failed to load latest commit information.

readme.md

dbf2xls — DBF to XLS converter

1. pip install -r requirements.txt
2. set your DBF files into dbf-directory
3. execute dbf2xls.py

dbf2xls.py will create a sub-directory into dbf-directory called xls-directory and put your converted files in there.

About

Convert DBF files to Excel file

Resources

Stars

Watchers

Forks

Releases

Packages 0

Languages

© 2023 GitHub, Inc.

You can’t perform that action at this time.

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.

Источник

Use Git or checkout with SVN using the web URL.

Work fast with our official CLI. Learn more.

Sign In Required

Please sign in to use Codespaces.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio Code

Your codespace will open once ready.

There was a problem preparing your codespace, please try again.

Latest commit

Git stats

Files

Failed to load latest commit information.

README.md

Python binding for the CodeBase-for-DBF data table engine, plus tools for spreadsheet access and other functions

CodeBase Tools is a Python binding that encapsulates the low-level functioning of the CodeBase(tm) product. This is a formerly proprietary product released to Open Source as of September, 2018. Python CodeBase Tools provide the following capabilities:

• Creates, Opens, Reads, Writes and Indexes DBF tables in a format compatible with Visual FoxPro type DBF tables.
• Supports record, table, and table header locking compatible with that of Visual FoxPro, allowing hundreds of users to access the same tables simultaneously
• Optionally supports tables, fields, and index elements larger than the limits in Visual FoxPro (Maximum table size is 8GB)
• Recognizes a wide variety of fields including:
  • Character (Assumes Code Page 1252 — Windows European Language characters)
  • Character-binary (Makes no assumptions regarding language representation so suitable for Unicode data)
  • Memo (Large text up to 2GB)(Assumes Code Page 1252 — Windows European Language characters)
  • Memo-binary (Large text up to 2GB) (Makes no assumptions regarding language representation so suitable for Unicode data)
  • Number (decimal values up to 17 digits)
  • Integer (4-byte signed integers)
  • Boolean (Logical, True/False)
  • Currency (Fixed 4-decimal point exact representation of money values)
  • Date (Year, Month, Day)
  • Datetime (Year, Month, Day, Hour, Minute, Second)
  • Float (like Number)
  • Double (like Number)
  • General (treated like Memo-binary, OLE component content is not recognized)
• Allows creation of standard VFP-style CDX indexes (with multiple «tags» for different orderings), IDX indexes (with a single ordering), and the auto updating of indexes when records are changed.
• Provides for temporary indexes which remain synchronized with the data table while in use, and are then deleted automatically when closed.
• Supports copying DBF table data to Excel spreadsheet format with the ExcelTools modules (and the non-open-source commercial product LibXL, a purchased license for which is required.)
• Supports VFP-style CURSORTOXML and XMLTOCURSOR functions for rapid conversion of DBF data to generic XML
• Copies DBF tables to CSV and System Data (fixed field length) text files as well as importing those formats directly in a DBF table. For all functionality see the HTML documentation of this module.

Support for Python Versions

The .PY files in Python-CodeBase-Tools package are designed to be cross platform, functional on all versions of Python from 2.7 and up. However the compiled library .pyd file is specific to the Python version you are using. Currently, Python 2.7, 3.6, and 3.7 are supported. Support for earlier Python versions can be made available if interest is strong enough.

This package is copyright M-P Systems Services, Inc., and is released to Open Source under the GNU Lesser GPL V.3.0 license, a copy of which is found in this repository. The CodeBase-for-DBF module, is covered by this same license. The CodeBase package, including the core library c4dll.dll is copyright Sequiter, Inc., and is licensed under the GNU Lesser GPL v.3.0.

Integration with Visual FoxPro Applications

Источник

Работа с DBF в Питоне

Формат DBF, хоть и признан устаревшим, до сих пор широко используется в финансовых учреждениях. Он подходит для обмена табличными данными, такими, например, как отчеты и реестры платежей. Формат имеет простую логическую структуру, поэтому библиотеку для работы с ним можно написать самостоятельно. Классический DBF поддерживает основные типы данных — числа, строки, даты и булево. Благодаря этому дата, записанная в DBF, будет считата как дата, а не текстовая строка, которую будет необходимо парсить.

Для работы с файлами DBF в Питоне потребуется библиотека dbfpy. Это, кстати, не единственное решение, есть еще библиотеки pyDBF и dbf. Сам я работал только с первой из трех, функционал полность устраивает.

Приведу пару примеров: чтение и запись файла.

Запись нового файла немного сложнее, так как потребуется указать схему данных. Фрагмент кода из реального проекта:

Готово, файл создан.

Наверняка вы обратили внимание на кодирование поля payment.first_name . Это unocode-строка, и для того, чтобы записать ее в файл, необходимо сперва преобразовать в обычную строку в какой-то кодировке. Исторически сложилось так, что основной кодировкой для этого формата является OEM, она же DOS, она же cp866 . Многие программы, например, офисные Эксель и Акцесс работают с DBF-файлами именно в этой кодировке. Чтобы созданные вами файлы читались в сторонних программах, преобразовывайте перед записью строки с кириллицей в cp866 .

Работать с DBF можно как с SQL-базой через драйвер ODBC. При этом именем таблицы является имя файла. Доступны все CRUD-операции на записями. Склеить два DBF-файла оператором JOIN, к сожалению, нельзя. Для работы с ODBC-источниками скачайте библиотеку pyodbc.

Вставка, удаление и обновление записей реализуются аналогично согласно SQL-синтаксису:

Какой способ работы с DBF задействовать в проекте — решать вам. dbfpy не требует сторонних зависимостей, написана на чистом Питоне, может работать в сервисах типа App Engine, однако, не столь быстра, как pyodbc. Последняя, напротив, быстрее, но зависит от платформы и иногда глючит на Винде (у меня были проблемы с ошибками в DLL-библиотеках). Для небольших проектов рекомендую использовать dbfpy, в промышленных масштабах — pyodbc.

Комментарии из старого блога

04/08/14 Городецкий: привет! получаю invalid syntax при from dbfpy import dbf . Подскажи пожалуйста, что не так?

04/09/14 Иван Гришаев: Точка в конце не нужна. Или какая-то из букв кириллическая. Трейс должен показывать стрелочкой, где косяк.

01/09/15 Ole: Отличный пост

09/02/15 Дамир Аманов: в pyodbc как работать с кириллицой? в windows

09/03/15 Иван Гришаев: Насколько помню, кодировать строки в cp2151, если не получается с юникодом.

Нашли ошибку? Выделите мышкой и нажмите Ctrl/⌘+Enter

Привет. Выдаёт такие ошибки:
Traceback (most recent call last): from dbfpy import dbf — это ругается на «from dbfpy import dbf»
File «C:Users. AppDataLocalProgramsPythonPython37-32libsite-packagesdbfpydbf.py», line 280
print repr(_rec)

Либа под второй питон, а вас третий.

Спасибо.
Не можете подстазать что либо под третьего питона?

Быстрый поиск ничего не дает. Либо писать под второй, либо портировать библиотеку. Мне кажется, второе будет несложно, либа маленькая.

Красивая статья, кратко и ясно. Все работает, огромное спасибо!

Writing on programming, education, books and negotiations.

Источник

rnelsonchem/simpledbf

Use Git or checkout with SVN using the web URL.

Work fast with our official CLI. Learn more.

Sign In Required

Please sign in to use Codespaces.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio Code

Your codespace will open once ready.

There was a problem preparing your codespace, please try again.

Latest commit

Git stats

Files

Failed to load latest commit information.

README.rst

simpledbf is a Python library for converting basic DBF files (see Limitations) to CSV files, Pandas DataFrames, SQL tables, or HDF5 tables. This package is fully compatible with Python >=3.4, with almost complete Python 2.7 support as well. The conversion to CSV and SQL (see to_textsql below) is entirely written in Python, so no additional dependencies are necessary. For other export formats, see Optional Requirements. This code was designed to be very simple, fast and memory efficient for convenient interactive or batch file processing; therefore, it lacks many features, such as the ability to write DBF files, that other packages might provide.

Bug fixes, questions, and update requests are encouraged and can be filed at the GitHub repo.

This code is derived from an ActiveState DBF example that works with Python2 and is distributed under a PSF license.

• Pandas >= 0.15.2 (Required for DataFrame)
• PyTables >= 3.1 (with Pandas required for HDF tables)
• SQLalchemy >= 0.9 (with Pandas required for DataFrame-SQL tables)

The most recent release of simpledbf can be installed using pip or conda , if you happen to be using the Anaconda Python distribution.

The development version can be installed from GitHub:

As an alternative, this package only contains a single file, so in principle, you could download the simpledbf.py file from Github and put it in any folder of your choosing.

DBF File Limitations

This package currently supports a subset of dBase III through 5 DBF files. In particular, support is missing for linked memo (i.e. DBT) files. This is mostly due to limitations in the types of files available to the author. Feel free to request an update if you can supply a DBF file with an associated memo file. DBF version 7, the most recent DBF file spec, is not currently supported by this package.

Python 2 Support

Except for HDF file export, this code should work fine with Python >=2.7. However, HDF files created in Python3 are compatible with all Python2 HDF packages, so in principle, you could make any HDF files in a temporary Python3 environment. If you are using the Anaconda Python distribution (recommended), then you can make a small Python3 working environment as follows:

HDF file export is currently broken in Python2 due to a limitation in Pandas HDF export with unicode. This issue may be fixed future versions of Pandas/PyTables.

Load a DBF file

This module currently only defines a single class, Dbf5 , which is instantiated with a DBF file name, which can contain path info as well. An optional ‘codec’ keyword argument that controls the codec used for reading/writing files. The default is ‘utf-8’. See the documentation for Python’s codec standard library module for more codec options.

The Dbf5 object initially only reads the header information from the file, so you can inspect some of the properties. For example, numrec is the number of records in the DBF file, and fields is a list of tuples with information about the data columns. See the DBF file spec for info on the column type characters. The «DeletionFlag» column is always present as a check for deleted records; however, it is never exported during conversion.

The docstring for this object contains a complete listing of attributes and their descriptions.

The mem method gives an approximate memory requirement for processing this DBF file. (

2x the total file size, which could be wildly inaccurate.) In addition, all of the output methods in this object take a chunksize keyword argument, which lets you split up the processing of large files into smaller chunks to limit the total memory usage of the conversion process. When this keyword argument is passed into mem , the approximate memory footprint of the chunk will also be given, which can be useful when trying to determine the maximum chunksize your memory will allow.

Export the Data

The Ddb5 object behaves like Python’s file object in that it will be «exhausted» after export. To re-export the DBF data to a different format, first create a new Dbf5 instance using the same file name. This procedure is followed in the documentation below.

Note on Empty/Bad Data

This package attempts to convert most blank strings and poorly formatted values to an empty value of your choosing. This is controlled by the na keyword argument to all export functions. The default for CSV is an empty string (»), and for all other exports, it is ‘nan’ which converts empty/bad values to float(‘nan’) . NOTE The exception here is that float/int columns always use float(‘nan’) for all missing values for DBF->SQL->DataFrame conversion purposes. Pandas has very powerful functions for working with missing data, including converting NaN to other values (e.g. empty strings).

Use the to_csv method to export the data to a CSV file. This method requires the name of a CSV file as an input. The default behavior is to append new data to an existing file, so be careful if the file already exists. The chunksize keyword argument controls the frequency that the file buffer will be flushed, which may not be necessary. The na keyword changes the value used for missing/bad entries (default is »). The keyword header is a boolean that controls writing of the column names as the first row of the CSV file. The encoding of the resulting CSV file is determined by the codec that is set when opening the DBF file, see Loading.

If you are unhappy with the default CSV output of this module, Pandas also has very powerful CSV export capabilities for DataFrames.

To SQL (CSV-based)

Most SQL databases can create tables directly from local CSV files. The pure-Python to_textsql method creates two files: 1) a header-less CSV file containing the DBF contents, and 2) a SQL file containing the appropriate table creation and CSV import code. It is up to you to run the SQL file as a separate step. This function takes two mandatory arguments, which are simply the names of the SQL and CSV files, respectively. In addition, there are a number of optional keyword arguments as well. sqltype controls the output dialect. The default is ‘sqlite’, but ‘postgres’ is also accepted. table sets the name of the SQL table that will be created. By default, this will be the name of the DBF file without the file extension. You should escape quote characters («) in the CSV file. This is controlled with the escapeqoute keyword, which defaults to ‘»‘ . (This changes ‘»‘ in text strings to ‘»»‘, which the SQL server should ignore.) The chunksize , na , and header keywords are used to control the CSV file. See above.

Here’s an example for SQLite:

Here’s an example for Postgresql:

The to_dataframe method returns the DBF records as a Pandas DataFrame. If the size of the DBF file exceeds available memory, then passing the chunksize keyword argument will return a generator function. This generator yields DataFrames of len( na keyword changes the value used for missing/bad entries (default is ‘nan’ which inserts float(‘nan’) ).

Issue with DataFrame Chunksize

When a DataFrame is constructed, it attempts to determine the dtype of each column. If you chunk the DataFrame output, it turns out that the dtype for a column can change. For example, if one chunk has a column with all strings, the dtype will be np.object ; however, if in the next chunk that same column is full of float(‘nan’) , the resulting dtype will be set as float . This has some consequences for writing to SQL and HDF tables as well. In principle, this behavior could be changed, but it is currently non-trivial to set the dtypes for DataFrame columns on construction. Please file a PR through GitHub if this is a big problem.

To an SQL Table using Pandas

The to_pandassql method will transfer the DBF entries to an SQL database table of your choice using a combination of Pandas DataFrames and SQLalchemy. A valid SQLalchemy engine string argument is required to connect with the database. Database support will be limited to those supported by SQLalchemy. (This has been tested with SQLite and Postgresql.) Note, if you are transferring a large amount of data, this method will be very slow. If you have direct access to the SQL server, you might want to use the text-based SQL export instead.

This method accepts three optional arguments. table is the name of the table you’d like to use. If this is not passed, your new table will have the same name as the DBF file without file extension. Again, the default here is to append to an existing table. If you want to start fresh, delete the existing table before using this function. The chunksize keyword processes the DBF file in chunks of records no larger than this size. The na keyword changes the value used for missing/bad entries (default is ‘nan’ which inserts float(‘nan’) ).

To an HDF5 Table

The to_pandashdf method transfers the DBF entries to an HDF5 table of your choice. This method uses a combination of Pandas DataFrames and PyTables, so both of these packages must be installed. This method requires a file name string for the HDF file, which will be created if it does not exist. Again, the default behavior is to append to an existing file of that name, so be careful here. The HDF file will be created using the highest level of compression (9) with the ‘blosc’ compression lib. This saves an enormous amount of disk space, with little degradation of performance; however, this compression library is non-standard, which can cause problems with other HDF libraries. Compression options are controlled use the complib and complevel keyword arguments, which are identical to the ones described in the Pandas HDF compression docs.

This method uses the same optional arguments, and corresponding defaults, as to_pandassql (see above). A example with chunksize is shown below. In addition, a data_columns keyword argument is also available, which sets the columns that will be used as data columns in the HDF table. Data columns can be used for advanced searching and selection; however, there is some degredation of preformance for large numbers of data columns. See the Pandas data columns docs for a more detailed explanation.

See the chunksize issue for DataFrame export for information on a potential problem you may encounter with chunksize.

Batch file export is trivial using simpledbf. For example, the following code processes all DBF files in the current directory into separate tables in a single HDF file.

Источник