Oledb providers for excel

Провайдеры данных

Для работы с Excel 2003 (.Xls) можно использовать провайдер Microsoft Jet OLE DB 4.0.

SELECT * FROM OPENROWSET(
	'Microsoft.Jet.OLEDB.4.0', 
	'Excel 12.0;Database=d:tmpTimeSheets.xlsx;HDR=YES;IMEX=1', 
	'SELECT * FROM [Sheet1$]');

Для работы с Excel 2007 (.Xlsx) — Microsoft ACE OLEDB 12.0.

SELECT * FROM OPENROWSET (
    'Microsoft.ACE.OLEDB.12.0',
    'Excel 12.0;Database=d:tmpTimeSheets.xlsx;HDR=YES;IMEX=1',
    'SELECT * FROM [Sheet1$]');

В Windows 10 открыть настройки источников данных ODBC можно написав «Источники данных ODBC» или через Панель управления Администрирование.

Extended Properties

• HDR=YES|NO. HDR=YES означает, что первую строку листа, следует рассматривать как заголовки колонок. Т.о. значение из первой строки можно использовать как имена полей в sql запросах (любых: select, insert, update, delete).
• IMEX=1|3. 1 — открыть соединение для чтения. 3 — для записи.

Создание Linked Server в Sql Server для доступа к Excel

EXEC sp_addLinkedServer
    @server= N'XLSX_2010',
    @srvproduct = N'Excel',
    @provider = N'Microsoft.ACE.OLEDB.12.0',
    @datasrc = N'd:tmpTimeSheets.xlsx',
    @provstr = N'Excel 12.0; HDR=Yes';
GO

После создания связанного сервера можно будет просмотреть имена доступных листов.

Затем, чтобы обратиться к сервису:

SELECT * FROM OPENQUERY (XLSX_2010, 'Select * from [Sheet1$]')
или
SELECT * FROM [XLSX_2010]...[Лист1$]

Обращение к лиcтам, диапазонам, полям

Для обращения к листу из SQL запроса нужно использовать имя листа, например: [Sheet1$] или [Лист1$]. Обращение к диапазону: [Sheet1$A16:F16].

Вставка данных в произвольное место

Примеры указания диапазона при вставке строк (insert)

• [table1$B4:E20]
• [table1$S4:U]
• [table1$F:G]

При вставке должны выполняться следующие условия:

• Первая строчка указанного диапазона дожна входить в диапазон ячеек с данными. Чтобы создать на листе диапазон с данными достаточно в углах некоторого прямоугольного диапазона (в левом верхнем и правом нижнем) вписать значение (C4:I7 на скриншоте). Т.е. сама первая строчка указанного в insert диапазона данные содержать не обязана, достаточно, чтобы она просто входила в такой диапазон. Иначе возникнет ошибка "This table contains cells that are outside the range of cells defined in this spreadsheet"
• Хвост диапазона должен содержать пустые строки (хотя бы одну).

Пример: Дан лист, где заполнены только 2 ячейки: C4, I7. После выполнения команды INSERT INTO [table1$E6:G] VALUES(2, 'FF','2014-01-03') результат будет как на скриншоте. Поясним: строка E6:G6 является первой строкой диапазона для вставки. Она входит в заполненный диапазон C4:I7. Поэтому данные были вставлены на следующей пустой строке — 8. Из этого примера становится ясно, что через OleDb нельзя работать с несколькими независимыми диапазонами на одном листе, используя вставку (update будет работать).

Ошибки

• System.Data.OleDb.OleDbException (0x80004005): Operation must use an updateable query. Соединение открыто для чтение, при этом происходит попытка внести изменения (выполнить insert, update или delete). Решение: открыть соединение для записи, установив свойство провайдера в строке соединения IMEX=3 (см. выше).
• System.Data.OleDb.OleDbException (0x80004005): "This table contains cells that are outside the range of cells defined in this spreadsheet". Такая ошибка возникает при подпытке обновить (update) или вставить (insert) значения в диапазоне, в котором отсутствуют данные на листе.
  1. Если нужно произвести запись в определенные ячейки инструкцией update, то

Ссылки

• https://www.codeproject.com/Tips/705470/Read-and-Write-Excel-Documents-Using-OLEDB
• https://stackoverflow.com/questions/36987636/cannot-create-an-instance-of-ole-db-provider-microsoft-jet-oledb-4-0-for-linked
• https://stackoverflow.com/questions/26267224/the-ole-db-provider-microsoft-ace-oledb-12-0-for-linked-server-null
• http://www.ashishblog.com/importexport-excel-xlsx-or-xls-file-into-sql-server/
• https://yoursandmyideas.com/2011/02/05/how-to-read-or-write-excel-file-using-ace-oledb-data-provider/
• https://stackoverflow.com/questions/46373895/how-to-open-a-huge-excel-file-efficiently Несколько способов открыть большой Excel файл, в т.ч. с помощью OleDb.

• Sample tool in C# with XLSX (Office 2007) support — 24.99 KB
• Sample tool in C# — 24.61 KB
• Sample tool in VB.NET with XLSX (Office 2007) Support — 26.88 KB
• Sample tool for VB.NET — 26.28 KB
• Sample CreateWorkBook download in ASP.NET — 21.62 KB

Table of Contents

1. Introduction
2. Available Ways to work with Excel Workbooks
3. Background
4. Working with Excel Workbook
   1. Anatomy of ConnectionString
   2. Creating Excel Workbook
   3. Getting Schema Definition
   4. Retrieve Data By Worksheet Name
   5. Retrieve Data Using Range
   6. Manipulating Data (Insert / Update / Delete)
   7. Drop Excel Worksheet
5. Description and Usage of Sample Tool
6. Code Explanation and Usage Info
7. History

Introduction

Hi folks. It’s been a while since I wrote my last article. Meanwhile, I came across a lot of stuff, and want to share it with you. This article is regarding all we need to work with Excel through our programs.

While searching Google for this topic, I came across some of the links, but none of them gave a clear and concise idea of how to work with data in Excel in the easiest way from .NET. So I decided to jot down everything that may appear with this topic in this article.

To Work With Excel Workbooks, You Can Do Through Three Different Ways

You need 3^rd party library which acts as an interface between your program and the Excel.

1. You can make use of Excel InterOp Objects, but this requires you to have Excel installed in the development environment. This is a binding if you are going to make a product which is to be distributed.
2. You can use OleDb data providers for Excel which comes for free with Windows. But there is one limitation though, you can access only data using this technique. You cannot do formatting through this technique.
3. You can use XML to create Excel objects which will open in MSExcel correctly. This is easier, just you need to work with XML through programming. It also supports XML stylesheets. I will also try to discuss this in another article, for the time being, you may look into ExcelXMLDemo.

In this topic, I am going to discuss about the 3^rd method which is the most common one that we use while working with Excel.

Background

Excel is the most common and popular format of showing data to the client nowadays. After the most common one (PDF), you need to place another format which may show the reports to the client. Excel would be the right choice for that. Now we often come up with a requirement to generate the data in an Excel Workbook. Recently while developing, I got one requirement to dump some data in Excel sheet. Thus I thought of writing this one.

Another important requirement is to read data from MS Excel 2007 format, which is also an unusual task to learn the entire structure of Excel 2007 objects. Using MDac, one can easily work with both of them without changing any of the code whatsoever.

Working with Excel Workbook

The rows and columns of Excel workbook closely resemble the rows and columns of a database table. We can use MDac (Microsoft Data Access Tool) that comes free with Windows update to work with Excel worksheet. In case of Excel Workbooks, each worksheet acts as a table and each workbook is actually a database. You can create, insert drop Excel objects through OleDb data clients from your program.

Now Let Us See How the connectionstring Will Look Like

Normal ConnectionString: (work for xls files)

Provider=Microsoft.Jet.OLEDB.4.0;Data Source={0};Extended Properties="Excel 8.0;HDR=YES;""

Office 2007 ConnectionString : (work for xlsx files)

Provider=Microsoft.ACE.OLEDB.12.0;Data Source={0};
Extended Properties="Excel 12.0;HDR=YES;""

Here, Data Source will be placed with a proper filename like C:\test.xls or C:\test.xlsx. If you want to create a workbook, just place the one that is not existing and use Create Table to create a workbook.

The connectionstring has some parts:

1. Provider: It is the main oledb provider that is used to open the Excel sheet. This will be Microsoft.Jet.OLEDB.4.0 for Excel 97 onwards Excel file format and Microsoft.ACE.OLEDB.12.0 for Excel 2007 or higher Excel file format (One with xlsx extension)
2. Data Source: It is the entire path of the Excel workbook. You need to mention a dospath that corresponds to an Excel file. Thus, it will look like: Data Source=C:\testApp.xls".
3. Extended Properties (Optional): Extended properties can be applied to Excel workbooks which may change the overall activity of the Excel workbook from your program. The most common ones are the following:
   • HDR: It represents Header of the fields in the Excel table. Default is YES. If you don’t have fieldnames in the header of your worksheet, you can specify HDR=NO which will take the columns of the tables that it finds as f1,f2 etc.
   • ReadOnly: You can also open Excel workbook in readonly mode by specifying ReadOnly=true; By default, Readonly attribute is false, so you can modify data within your workbook.
   • FirstRowHasNames: It is the same as HDR, it is always set to 1 ( which means true) you can specify it as false if you don’t have your header row. If HDR is YES, provider disregards this property. You can change the default behaviour of your environment by changing the Registry Value [HKLMSoftwareMicrosoftJet4.0EnginesExcelFirstRowHasNames] to 00 (which is false)
   • MaxScanRows: Excel does not provide the detailed schema defination of the tables it finds. It need to scan the rows before deciding the data types of the fields. MaxScanRows specifies the number of cells to be scanned before deciding the data type of the column. By default, the value of this is 8. You can specify any value from 1 — 16 for 1 to 16 rows. You can also make the value to 0 so that it searches all existing rows before deciding the data type. You can change the default behaviour of this property by changing the value of [HKLMSoftwareMicrosoftJet4.0EnginesExcelTypeGuessRows] which is 8 by default. Currently, MaxScanRows is ignored, so you need only to depend on TypeGuessRows Registry value. Hope Microsoft fixes this issue to its later versions.
   • IMEX: (A Caution) As mentioned above, Excel will have to guess a number or rows to select the most appropriate data type of the column, a serious problem may occur if you have mixed data in one column. Say you have data of both integer and text on a single column, in that case, Excel will choose its data type based on majority of the data. Thus it selects the data for the majority data type that is selected, and returns NULL for the minority data type. If the two types are equally mixed in the column, the provider chooses numeric over text.

     For example, in your eight (8) scanned rows, if the column contains five (5) numeric values and three (3) text values, the provider returns five (5) numbers and three (3) null values.

     To work around this problem for data, set «IMEX=1» in the Extended Properties section of the connection string. This enforces the ImportMixedTypes=Text registry setting. You can change the enforcement of type by changing [HKLMSoftwareMicrosoftJet4.0EnginesExcelImportMixedTypes] to numeric as well.

     Thus if you look into the simple connectionstring with all of them, it will look like:

     Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\testexcel.xls;
     Extended Properties="Excel 8.0;HDR=YES;IMEX=1;MAXSCANROWS=15;READONLY=FALSE""

     or:

     Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\testexcel.xlsx;
     Extended Properties="Excel 12.0;HDR=YES;IMEX=1;MAXSCANROWS=15;READONLY=FALSE""

     We need to place extended properties into Quotes(«) as there are multiple number of values.

Can We Create Excel Workbook through this Technique?

If you are eager to know if we can create Excel workbook directly through OleDB, your answer is yes. The only thing that you need to do is to specify a non-existing file in the Data Source of the connectionstring.

string connectionstring = "Provider=Microsoft.Jet.OLEDB.4.0;
                          Data Source=c:\testexcel.xls;
                          Extended Properties"Excel 8.0;HDR=YES"";
string createTableScript = "CREATE TABLE newTable(a1 MEMO,a2 INT,a3 CHAR(255))";
using(conObj = new OleDbConnection(connectionstring))
{
   using (OleDbCommand cmd = new OleDbCommand(createTableScript, conObj)
   {
     if (this.Connection.State != ConnectionState.Open) this.Connection.Open();
     cmd.ExecuteNonQuery();
   }
}

This will create a new workbook with one worksheet if the datasource file (testexcel.xls) is not existing in the location.

To Retrieve Schema Information of Excel Workbook

You can get the worksheets that are present in the Excel workbook using GetOleDbSchemaTable. Use the following snippet:

DataTable dtSchema = null;
dtSchema = conObj.GetOleDbSchemaTable(
OleDbSchemaGuid.Tables, new object[] { null, null, null, "TABLE" });

Here dtSchema will hold the list of all workbooks. Say we have two workbooks: wb1, wb2. The above code will return a list of wb1, wb1$, wb2, wb2$. We need to filter out $ elements.

Selecting Data From a WorkBook (Specifying Range)

You can run a simple query to select Data from an Excel workbook. Say your workbook contains tables like w1, w2. Now, if write SELECT * FROM [w1] or SELECT * FROM 'w1', it will return you the whole datatable with all the data.

You Can Also Specify the Range of Selection, Just Write Query Like

SELECT * FROM [w1$A10:B10]

Thus it will select the data only from Excel Cell A10 : B10 Range.

string cmdText = "SELECT * FROM [w1$A10:B10]";
using(OleDbCommand cmd = new OleDbCommand(cmdText))
{
      cmd.Connection = this.Connection;
      OleDbDataAdapter adpt = new OleDbDataAdapter(cmd);
      DataSet ds = new DataSet();
      adpt.Fill(ds,"w1");
}

NOTE

A caution about specifying worksheets: The provider assumes that your table of data begins with the upper-most, left-most, non-blank cell on the specified worksheet. In other words, your table of data can begin in Row 3, Column C without a problem. However, you cannot, for example, type a worksheeet title above and to the left of the data in cell A1.

A caution about specifying ranges: When you specify a worksheet as your source, the provider adds new records below existing records in the worksheet as space allows. When you specify a range (named or unnamed), Jet also adds new records below the existing records in the range as space allows. However, if you re-query on the original range, the resulting recordset does not include the newly added records outside the range. Using MDAC, you cannot add new rows beyond the defined limits of the range, otherwise, you will receive Exception: "Cannot expand named range"

Running DML Statement

You can run any DML statement in the same way you do for other databases. Samples:

INSERT INTO [w1] VALUES('firsttextcol', 2, '4/11/2009', '10:20');

[We assume First Column is either memo or Char field, 2^nd col is int, 3^rd is Date, 4^th is Time data type]

DELETE FROM [w1] Where secondintcol=2;
UPDATE [w1] SET secondintcol = 3 where firsttextcol = 'firsttextcol';

We can use [] (Square brackets) to allow spaces within columnnames and tablenames as we do for databases.

Droping Excel WorkSheet

To Drop an Excel Worksheet, Just Use

Drop Table [w1]

This will drop the worksheet.

If this is the last worksheet, it will not delete the workbook file. You need to do it yourself.

Using the Sample Tool

I have added one sample application that demonstrates the problem. It includes one class called ExcelObject which allows you to work with Excel. You can use the code to work in your own application easily.

1. Choose Browse and select an xls file. If you want to create the workbook, just click on Create table to Create a table with workbook.

2. Click on Retrieve to get the Tables present in the workbook. These are mainly worksheets.

3. You can create tables using the window. Just write the column name and click on Insert. Specify Tablename and a new worksheet will be created for you.

4. Generate Insert statements from the dynamic screen.

5. You can use Go to get the data loaded into the Grid.

NOTE

This is just a demo application. You can use the Class associated with the application call functions to do your job easy.

Using the Code

The code for ExcelObject Class will be like this:

using System.IO;
using System.Data.OleDb;
using System.Text;
using System.Data;
using System.Windows.Forms;

public class ExcelObject
{
     private string excelObject = = "Provider=Microsoft.{0}.OLEDB.{1};Data Source={2};
                                     Extended Properties="Excel {3};HDR=YES"";
     private string filepath = string.Empty;
     private OleDbConnection con = null;

        public delegate void ProgressWork(float percentage);
        private event ProgressWork Reading;
        private event ProgressWork Writeing;
        private event EventHandler connectionStringChange;

        public event ProgressWork ReadProgress
        {
            add
            {
                Reading += value;
            }
            remove
            {
                Reading -= value;
            }
        }

        public virtual void onReadProgress(float percentage)
        {
            if (Reading != null)
                Reading(percentage);
        }

        public event ProgressWork WriteProgress
        {
            add{ Writeing += value; }
            remove{ Writeing -= value; }
        }

        public virtual void onWriteProgress(float percentage)
        {
            if (Writeing != null)
                Writeing(percentage);
        }

        public event EventHandler ConnectionStringChanged
        {
            add{ connectionStringChange += value; }
            remove { connectionStringChange -= value; }
        }

        public virtual void onConnectionStringChanged()
        {
            if (this.Connection != null && 
                !this.Connection.ConnectionString.Equals(this.ConnectionString))
            {
                if (this.Connection.State == ConnectionState.Open)
                    this.Connection.Close();
                this.Connection.Dispose();
                this.con = null;
            }
            if (connectionStringChange != null)
            {
                connectionStringChange(this, new EventArgs());
            }
        }
        
        public string ConnectionString
        {
            get
            {
                if (!(this.filepath == string.Empty))
                {
                   
                    FileInfo fi = new FileInfo(this.filepath);
                    if (fi.Extension.Equals(".xls"))
                    {
                        
                        return string.Format(this.excelObject, 
                                   "Jet", "4.0", this.filepath, "8.0");
                    }
                    else if (fi.Extension.Equals(".xlsx"))
                    {
                        
                        return string.Format(this.excelObject, 
                                   "Ace", "12.0", Me.filepath, "12.0");
                    }
                }
                else
                {
                    return string.Empty;
                }
            }
        }
        
        public OleDbConnection Connection
        {
            get
            {
                if (con == null)
                {
                    OleDbConnection _con = new OleDbConnection { 
                                ConnectionString = this.ConnectionString };
                    this.con = _con;
                }
                return this.con;
            }
        }

        public ExcelObject(string path)
        {
            this.filepath = path;
            this.onConnectionStringChanged();
        }
        
        public DataTable GetSchema()
        {
            DataTable dtSchema = null;
            if (this.Connection.State != ConnectionState.Open) this.Connection.Open();
            dtSchema = this.Connection.GetOleDbSchemaTable(
                   OleDbSchemaGuid.Tables, new object[] { null, null, null, "TABLE" });
            return dtSchema;
        }
        
        public DataTable ReadTable(string tableName)
        {
            return this.ReadTable(tableName, "");
        }

        public DataTable ReadTable(string tableName, string criteria)
        {
            try
            {
                DataTable resultTable = null;
                if (this.Connection.State != ConnectionState.Open)
                {
                    this.Connection.Open();
                    onReadProgress(10);
                    
                }
                string cmdText = "Select * from [{0}]";
                if (!string.IsNullOrEmpty(criteria))
                {
                    cmdText += " Where " + criteria;
                }
                OleDbCommand cmd = new OleDbCommand(string.Format(cmdText, tableName));
                cmd.Connection = this.Connection;
                OleDbDataAdapter adpt = new OleDbDataAdapter(cmd);
                onReadProgress(30);
                
                DataSet ds = new DataSet();
                onReadProgress(50);
                
                adpt.Fill(ds, tableName);
                onReadProgress(100);
                
                if (ds.Tables.Count == 1)
                {
                    return ds.Tables[0];
                }
                else
                {
                    return null;
                }
            }
            catch
            {
                MessageBox.Show("Table Cannot be read");
                return null;
            }
        }
        
        public bool DropTable(string tablename)
        {
            try
            {
                if (this.Connection.State != ConnectionState.Open)
                {
                    this.Connection.Open();
                    onWriteProgress(10);                    
                }
                string cmdText = "Drop Table [{0}]";
                using (OleDbCommand cmd = new OleDbCommand(
                         string.Format(cmdText, tablename), this.Connection))
                {
                    onWriteProgress(30);
                    
                    cmd.ExecuteNonQuery();
                    onWriteProgress(80);                    
                }
                this.Connection.Close();
                onWriteProgress(100);
                
                return true;
            }
            catch (Exception ex)
            {
                onWriteProgress(0);
                
                MessageBox.Show(ex.Message);
                return false;
            }
        }
        
        public bool WriteTable(string tableName, Dictionary<string, string> 
                                                             tableDefination)
        {
            try
            {
                using (OleDbCommand cmd = new OleDbCommand(
                this.GenerateCreateTable(tableName, tableDefination), this.Connection))
                {
                    if (this.Connection.State != ConnectionState.Open)
                    this.Connection.Open();
                    cmd.ExecuteNonQuery();
                    return true;
                }
            }
            catch
            {
                return false;
            }
        }
        
        public bool AddNewRow(DataRow dr)
        {
            using (OleDbCommand cmd = new OleDbCommand(
                          this.GenerateInsertStatement(dr), this.Connection))
            {
               cmd.ExecuteNonQuery();
            }
            return true;
        }
        
        private string GenerateCreateTable(string tableName, 
                            Dictionary<string, string> tableDefination)
        {
            StringBuilder sb = new StringBuilder();
            bool firstcol = true;
            sb.AppendFormat("CREATE TABLE [{0}](", tableName);
            firstcol = true;
            foreach (KeyValuePair<string, string> keyvalue in tableDefination)
            {
                if (!firstcol)
                {
                    sb.Append(",");
                }
                firstcol = false;
                sb.AppendFormat("{0} {1}", keyvalue.Key, keyvalue.Value);
            }

            sb.Append(")");
            return sb.ToString();
        }
        
        private string GenerateInsertStatement(DataRow dr)
        {
            StringBuilder sb = new StringBuilder();
            bool firstcol = true;
            sb.AppendFormat("INSERT INTO [{0}](", dr.Table.TableName);

            foreach (DataColumn dc in dr.Table.Columns)
            {
                if (!firstcol)
                {
                    sb.Append(",");
                }
                firstcol = false;

                sb.Append(dc.Caption);
            }

            sb.Append(") VALUES(");
            firstcol = true;
            for (int i = 0; i <= dr.Table.Columns.Count - 1; i++)
            {
                if (!object.ReferenceEquals(dr.Table.Columns[i].DataType, typeof(int)))
                {
                    sb.Append("'");
                    sb.Append(dr[i].ToString().Replace("'", "''"));
                    sb.Append("'");
                }
                else
                {
                    sb.Append(dr[i].ToString().Replace("'", "''"));
                }
                if (i != dr.Table.Columns.Count - 1)
                {
                    sb.Append(",");
                }
            }

            sb.Append(")");
            return sb.ToString();
        }
    }

After looking through the code you are clear that we are actually generating DDL and DML statements based on the Schema Definition. I know we can easily do this using OleDbCommandBuilder object, but I thought of making them myself. Functions exposed through this class are:

Methods

• GetSchema: It returns the Schema defination datatable of the currently selected xls file. You can call this if you have connected with an existing Excel Workbook.
• ReadTable: It automatically generates Select statement on the tablename passed and based on the criteria provided. It returns the DataTable of the currently selected Excel worksheet.
• DropTable: Drops the table name passed, and which results in actual deletion of one worksheet from the workbook. The Function returns true if successful.
• AddNewRow: This function creates an Insert statement and inserts a new row based on the DataRow passed in.

Properties

• ConnectionString: You can get connectionstring of the filepath passed.
• Connection: Returns OleDbConnection Object.

Events

• ReadProgress: It generates a callback to the calling procedure on the percentage of Read of the file. You can handle this event to get the percentage progress value.
• WriteProgress: Same as ReadProgress, only it is called during actual insert of data.
• ConnectionStringChanged: This event occurs if FileName is changed somehow or a new file is created.

I have also provided the same class in VB.NET for those people who wants it in VB.NET.

You can find both of them from here:

Version 1

• Download ExcelWrite_Csharp.zip — 24.61 KB
• Download ExcelWrite_VB.NET.zip — 26.28 KB

Version 2

• Download ExcelWrite_Csharp_V2.zip — 24.99 KB
• Download ExcelWrite_VBNET_V2.zip — 26.88 KB
• I have also added one example for ASP.NET users to dynamically create one Excel file and download it to clients.

You can find that from here:

• Download ExcelDownload.zip — 21.62 KB

History

• 7^th June, 2009: First release
  • Looking forward to updating the article with new things. Hope you like this article.
• 10^th June, 2009: Second release
  • Support for xlsx files (Office 2007 Files). Hope this would help you.