Excel odbc for linux

ProgressBlogs
ODBC drivers for Microsoft Access and Excel on Unix/Linux for SAS, SAP, Oracle, IBM



Posted on
January 02, 2013

0 Comments

From recent consulting projects, I have learned that analyzing data in Microsoft Office enables unique and valuable business insights across an organization.  In particular, these projects were based on ODBC connectivity to Excel/Access from applications such as SAS (SAS/Access interface to ODBC), Oracle Database Gateway for ODBC, SAP Data Services, IBM Cognos, Oracle Business Intelligence (OBIEE), and Tibco Spotfire.  This sort of competitive advantage reminds me of another megatrend, «Big Data».

Seriously, what does big data have to do with Microsoft Excel and Access?

Big data enables organizations to analyze data sets that were previously unavailable due to computing limitations.  With Microsoft Excel and Microsoft Access, the data is typically unavailable to many business intelligence (BI) initiatives since it cannot be easily accessed by Unix/Linux applications.   The commonality is that the analysis of both data sources have been overlooked for very different reasons; and I will share how you can gain additional competitive advantages by integrating Microsoft Office data using the DataDirect SequeLink ODBC drivers for Microsoft Access and Excel across AIX, Solaris, HP-UX and Linux.

Getting Started with Microsoft Access and Excel ODBC drivers for Unix/Linux

Watch this 13 minute video on installation and configuration using the trial download for the 64-bit SequeLink Client for ODBC to Microsoft Office on AIX:

Refer to the production documentation for complete details.

Who else is using DataDirect SequeLink?

  • Home Depot
  • Save the Children
  • The Winterthur Group
  • Northwestern Mutual
  • National Center for Genome Research (NCGR)
  • Mizuho Bank

Congratulations on becoming a Microsoft Office Data Scientist

Call us today at 1-800-876-3101 to speak live with a Solutions Consultant to learn more about integrating Microsoft Office data sources into your Unix/Linux platform.  Or you can download a free trial of the DataDirect SequeLink ODBC Socket today.

Sumit Sakar

Sumit Sarkar

Technology researcher, thought leader and speaker working to enable enterprises to rapidly adopt new technologies that are adaptive, connected and cognitive. Sumit has been working in the data access infrastructure field for over 10 years servicing web/mobile developers, data engineers and data scientists. His primary areas of focus include cross platform app development, serverless architectures, and hybrid enterprise data management that supports open standards such as ODBC, JDBC, ADO.NET, GraphQL, OData/REST. He has presented dozens of technology sessions at conferences such as Dreamforce, Oracle OpenWorld, Strata Hadoop World, API World, Microstrategy World, MongoDB World, etc.

Next:

  • Progress DataDirect Now Connects to Denodo

  • Progress DataDirect Achieves Google Cloud Ready—AlloyDB Designation

  • Top 5 Reasons to Use DataDirect with Salesforce

Comments

Comments are disabled in preview mode.

I’m looking for help in finding some workaround so that we can:

  1. use an Excel driver (preferably some kind of ODBC-JDBC bridge), and
  2. use SQL queries with the driver.

Do such facilities exist?

Jonathan Leffler's user avatar

asked Sep 3, 2013 at 13:56

user2743249's user avatar

I’ve used SQLSheet with some success in the past. It supports basic CREATE, INSERT and UPDATE.

answered Sep 3, 2013 at 14:19

DB5's user avatar

DB5DB5

13.4k6 gold badges65 silver badges71 bronze badges

With ODBC, you can summarise, and select just the data you need, in an Excel workbook before importing it into SQL Server. You can join data from different areas or worksheets. You can even get data from the result of a SQL Server SELECT statement into an Excel spreadsheet. Phil Factor shows how, and warns of some of the pitfalls.

Why Use ODBC?

It is reasonably easy to insert data from Excel into SQL Server, or the reverse, from any other ODBC database to any other, using PowerShell. The most important direction is from Excel to SQL Server, of course. It is quicker than automating Excel and you can do it without requiring a copy of Excel. It is neater than SSIS too, and more versatile. The most important thing, though, is that you can aggregate before you send the data. It is possible to do a lot of filtering and aggregation of data before it ever gets to SQL Server, since you can turn an existing Excel Workbook into a poor-man’s relational database, or even create one. This article will aim to show how this is done.

I always feel slightly awkward in talking about ODBC. It is a Once and Future technology, developed before its time, but now showing its value for processing large volumes of data, despite its quirks, poor documentation and lackluster support. If you use the ODBC driver, then your Excel workbook becomes a little SQL-based relational database. Worksheets, or areas within worksheets, become tables. There are some features missing, of course, but you can do joins between tables, filter rows to taste,  do aggregations and some string manipulations.  This means that you need pull far less data into SQL because you can do a lot of selection and pre-processing before the data gets anywhere near SQL server. If, for example, you only need the total, count, and variance of a day’s readings, then why on earth would you want to import more than those aggregated figures?  Even if you do, these aggregations, performed on the original data, can be used as a ‘reconciliation’ check that you’ve gulped all the data into their final destination without error.

 I also prefer to use ODBC and the sequential data reader to read data from Excel, or any other ODBC source, because it is fast; and I like to use the bulk copy library to insert  ODBC ‘reader’ data into a SQL Server table because it is extremely fast, so we’ll use that. When you have a large number of big spreadsheets to insert as a chore, then speed matters.

The ODBC Excel driver (ACE)

ODBC was conceived as a way of making it as easy to connect to a particular datasource such a relational database, text file, data document (e.g. XML), web-based data or spreadsheet

Currently, the state of the art in ODBC for Access and Excel is the Microsoft Access Database Engine 2010 Redistributable which can be downloaded here. This includes the more popular OLEDB drivers which run well in PowerShell too.  These drivers enable you to access a range of data files via SQL as if they were a relational database.  Formats include Access, CSV,  delimited, DBase and Excel 

For developing on a general-purpose 64-bit desktop computer, you’re likely to hit a very silly Microsoft muddle. Microsoft  recommends that you install the 32-bit version of Office 2010, even on 64-bit machines,  since many of the common Office Add-ins did not run in the 64-bit Office environment. This advice has become baked-in ‘best practice’.  If you are using 64-bit PowerShell, as most of us are, then you need to use the 64-bit version of the drivers. If you  only have the 32-bit Office on your machine, then it will already have the 32-bit drivers, which won’t be visible to 64-bit PowerShell, and won’t work.  You can’t install the 64 bit drivers when you already have the 32-bit drivers and I don’t think you can get anything good to happen by uninstalling the 32-bit drivers. Nope. All three (or four if you include Visual Studio) must be 64 bit. I gather that one way out of this Catch 22 is  to first install the 64-bit Office 2010 ODBC/OleDB drivers and after that the (32-bit) Office, but there is a better fix that involves tweaking the registry.  See this for the full frustrating story.

The ODBC Excel driver in ACE works with the latest Excel spreadsheet format up to 2010 as well as the old ones. I suspect that the latest version will work with Office 2013, though I haven’t yet tried it.

This driver  is valuable because of the flexibility it gives. It actually executes ODBC SQL, which is a subset of SQL92,  so you can put in column aliases, change the column order,  and filter out rows that you don’t want to import. In effect, it presents you with a SQL tables which can be named ranges, if it is an existing worksheet that you’ve added named ranges to.

Select * from MyNamedRange

More commonly, you can specify with a delimited worksheet name followed by a range, the range being a specification of the area of the worksheet just sufficient to enable the driver to find the data you want. If you leave out the range spec entirely, the entire worksheet becomes the table.

Select * from [MyWorksheet$]

If, for example, you wanted the data in the range from C3 to L8, you’d use the statement

Select * from [MyWorksheet$C3:M8]

In ODBC, if you specified, say,  row 8 as the end of the table, you can only select rows up to row 8, even if you have inserted more rows beyond that limit, as ODBC allows. If you use some flavours, such as  the old  MDAC ‘JET’ database engine,  then you cannot add new rows beyond the defined limits of a range, otherwise you will get the Exception: "Cannot expand named range" message

If you wanted to define your table as being between the columns C and L, starting at row 3 you’d use

Select * from [NameOfExcelSheet$C3:M]

If you do this, then there is no limit to the length of the table so you can insert as many rows as you like.  The ODBC provider adds new rows to the existing rows in the defined area as space allows

The dreaded connection string

Now, before we start doing interesting things with the ACE drivers, I ought to explain a bit about their connection strings. These contain the specification of the ODBC driver you wish to use, and the settings that you wish to transmit to the driver.

Ignoring, for the time being, the extended property settings, For Microsoft Office Access data, set the Connection String to

«Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ= MyPath/MyFile«

For Excel data, use

«Driver={Microsoft Excel Driver (*.xls, *.xlsx, *.xlsm, *.xlsb)};DBQ=MyPath/MyFile«

For dBASE data, use

«Driver={Microsoft Access dBASE Driver {*.dbf, *.ndx, *.mdx)};DBQ=MyPath/MyFile«

For text data, use

«Driver={Microsoft Access Text Driver {*.txt, *.csv);DBQ=MyPath«

But you’re likely to want some extended properties for the settings to add a few details about the way that the ODBC provider should tackle this particular connection. Because the defaults can be changed globally in the registry, it is rather better to specify these extended properties rather than to rely on the defaults.

These extended properties are only relevant for the driver that you’re using. They are not always reliable and are poorly documented by Microsoft. I’ll only mention the essentials.

The driver needs to know if the first row of the table holds the name of the column. “HDR=Yes;” indicates that the first row contains column names, not data. It will actually just use the first 64 characters of the header.   “HDR=No;” treats the first row as data, but then the columns are named F1  onwards and you’d want to  alias them in your SQL statements to give them meaningful column names.

The Excel ODBC doesn’t keep a detailed schema definition of the tables. (the Text and Access  drivers by contrast do)  The ODBC Excel  driver will try to make sense of the data it finds by testing it to see what datatype it can use for the result. It does so by testing a number of rows before doing the import, and you can specify how many rows it tests before deciding the data type of the column by using MaxScanRows in the extended properties.  By default the value of this is 8. You can specify any value from 1 – 16 for 1 to 16 rows. You can also make the value to 0 so that it searches all existing rows before deciding the data type, but this slows things down.

This is fine for a relational table but Excel often has mixed types in a column The ODBC Provider will try to return the data of the majority type, but return NULL values for the rest that won’t convert. If the two types are equally mixed in the column, the provider chooses numeric over text, and you lose all the text. Also, it will judge the length of the character datatype in the column from the first rows and if the first rows are less than 255 characters long it will truncate all the subsequent data to 255 characters even if cell values below are longer.

By setting the Import Mode (IMEX=1). You can force mixed data to be converted to text, but only when it finds mixed values on the rows that it checks.

You can also open the Excel workbook in read-only mode by specifying ReadOnly=true; By Default Readonly attribute is false, so you can modify data within your workbook. However, this will lock the entire workbook from access until you close the connection.

Let’s try it out.

Just so you can prove all this to yourself, I’ve supplied an Excel workbook that represents the old PUBS database that used to be distributed with SQL Server and Sybase. This means that you can use SQL from old examples that use PUBS and see what works. All you need to do is to convert the SQL Server version slightly by altering the names of the tables slightly to tell the driver that you want the entire worksheet of that name (the $ is the separator between the worksheet name and the range specification)

So let’s pop together a very simple test-rig to try things out in PowerShell. Be warned, I’ve set this up in read-write mode so it will update your spreadsheet in some circumstances (CUD). To play along, you’ll need to download my Excel version of the PUBS database and  alter the path to the excel file.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

set-psdebug -strict

$ErrorActionPreference = «stop»

$ExcelFilePath=‘MyPathpubs.xlsx’ #the full path of the excel workbook

if (!(Test-Path $ExcelFilePath))

{

Write-Error «Can’t find ‘$($ExcelFilePath)’. Sorry, can’t proceed because of this»

exit

}

try {

$Connection = New-Object system.data.odbc.odbcconnection

$Connection.ConnectionString = ‘Driver={Microsoft Excel Driver (*.xls, *.xlsx, *.xlsm, *.xlsb)};DBQ=’+$ExcelFilePath+‘; Extended Properties=»Mode=ReadWrite;ReadOnly=false; HDR=YES»‘

$Connection.Open()

}

catch

{

$ex = $_.Exception

Write-Error «whilst opening connection to $ExcelFilePath : Sorry, can’t proceed because of this»

exit

}

try {

$Query = New-Object system.data.odbc.odbccommand

$Query.Connection = $connection

$Query.CommandText = @’

SELECT title, SUM(qty) AS sales,

COUNT(*) AS orders

FROM [titles$] t

INNER JOIN [sales$] s ON t.title_id=s.title_id

WHERE title like ‘%?’

GROUP BY title

ORDER BY SUM(qty) DESC

‘@

$Reader = $Query.ExecuteReader([System.Data.CommandBehavior]::SequentialAccess) #get the datareader and just get the result in one gulp

}

catch

{

$ex = $_.Exception

Write-Error «whilst executing the query ‘$($Query.CommandText)’ $ex.Message Sorry, but we can’t proceed because of this!»

$Reader.Close()

$Connection.Close()

Exit;

}

Try

{

$Counter = $Reader.FieldCount #get it just once

$result=@() #initialise the empty array of rows

while ($Reader.Read()) {

$Tuple = New-Object -TypeName ‘System.Management.Automation.PSObject’

foreach ($i in (0..($Counter 1))) {

Add-Member `

-InputObject $Tuple `

-MemberType NoteProperty `

-Name $Reader.GetName($i) `

-Value $Reader.GetValue($i).ToString()

}

$Result+=$Tuple

}

$result | Format-Table

}

catch

{

$ex = $_.Exceptio

Write-Error «whilst reading the data from the datatable. $ex.Message»

}

$Reader.Close()

$Connection.Close()

All these work

Inner joins

SELECT logo, pr_info, pub_name, city, state, country

FROM [pub_info$] pif INNER JOIN [publishers$] p

ON p.pub_id=pif.pub_id

Left or right outer joins

SELECT  title, stor_id, ord_num, qty,ord_date

FROM [titles$] t LEFT OUTER JOIN  [sales$] s

ON t.title_id=s.title_id

Expressions using columns

SELECT  fname+‘ ‘+ minit+‘ ‘+lname AS name, job_desc

FROM [jobs$] d

INNER JOIN  [employee$] e

ON d.job_id=e.job_id

Simple GROUP BY expression

SELECT COUNT(*) FROM [sales$] GROUP BY stor_ID

More complex aggregation with ORDER BY clause and a WHERE clause

SELECT  title, SUM(qty) AS sales,

COUNT(*) AS orders

FROM [titles$] t

INNER JOIN  [sales$] s ON t.title_id=s.title_id

WHERE title like ‘%?’

GROUP BY title

ORDER BY SUM(qty) DESC

String functions

SELECT  title, left(notes,20)+‘…’ as [note] FROM [titles$]

UNION and UNION ALL

SELECT  au_fname FROM [authors$] UNION ALL SELECT lname FROM [employee$]

One could go on and on; even subqueries work, but I think I’ve made the point that there is far more power in this ODBC Excel driver than just the facility for pulling out raw data. The same is true of the TEXT driver for OLEDB. It will do all this as well. To conform with the minimum syntax for ODBC, a driver must be able to execute CREATE TABLE, DELETE FROM (searched), DROP TABLE, INSERT INTO, SELECT, SELECT DISTINCT, and UPDATE (searched). SELECT statements can have WHERE and ORDER BY clauses. ACE does a bit better than this, since even the text driver allows SELECT INTO, and SELECT statements allow GROUP BY and HAVING. 

Creating a spreadsheet

You can, of course use the ODBC driver to create an Excel spreadsheet and write data into it. Here is the simplest working demo I can write without blushing. Be careful to ensure that the spreadsheet doesn’t exist as the whole point of the demo is to prove to you that it can create an entire spreadsheet workbook with several worksheets.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

$ExcelFilePath=‘MyPathNew.xlsx’ #the full path of the excel workbook

$Header= $true # we want your first row to be column headers

try {

$Connection = New-Object system.data.odbc.odbcconnection

$TheConnectionString = ‘Driver={Microsoft Excel Driver (*.xls, *.xlsx, *.xlsm, *.xlsb)};DBQ=’+$ExcelFilePath+‘;Mode=ReadWrite;ReadOnly=false;Extended Properties=»HDR=’+«$(if ($Header){‘YES’}else{‘NO’})»+‘»‘

$Connection.ConnectionString=$TheConnectionString

$Connection.Open()

}

catch

{

$ex = $_.Exception

Write-Error «whilst opening connection to $ExcelFilePath using ‘$($TheConnectionString)’ : $ex.Message»

}

$Mycommand=$connection.CreateCommand()

$MyCommand.CommandText=«create table MyTable (MyColumn varchar, MyOtherColumn varchar)»

if ($Mycommand.ExecuteNonQuery() -eq -1)

{

$MyCommand.CommandText=«insert into MyTable (MyColumn, MyOtherColumn) select ‘myfirstRowCol’,’myFirstRowCol2′»

$rows=$Mycommand.ExecuteNonQuery()

«$rows rows inserted into worksheet MyTable»

}

$connection.Close()

Notice that I can’t create the table and do the insert in one batch as a command. One statement only can be used in the commandText.

Exploring your Excel metadata

You can find out what datatypes are available for any ODBC source, by using the OdbcConnection.GetSchema(string) method.

$Datatypes=$connection.GetSchema(‘DATATYPES’).TypeName

Which with my connection gives only the LOGICAL, CURRENCY, NUMBER, VARCHAR and DATETIME datatypes. More useful is..

$tables=$connection.GetSchema(‘TABLES’).Table_Name

 … that gives you a list of the available worksheets . The complete list, if you wish to peep at them, is

$connection.GetSchema(‘TABLES’)

$connection.GetSchema(‘DATATYPES’)

$connection.GetSchema(‘DataSourceInformation’)

$connection.GetSchema(‘Restrictions’)

$connection.GetSchema(‘ReservedWords’)

$connection.GetSchema(‘Columns’)

$connection.GetSchema(‘Indexes’)

$connection.GetSchema(‘Views’)

Hmm. This is beginning to look a bit more like a database. With the Columns MetadataCollection, you can find out as much as you’d ever want to know about the data that is available in the spreadsheet so if you want to read all the worksheets straight into SQL Server, this is a wide-open goal.

Creating Worksheets

Going back to the PUBS Excel database, let’s create a peoples table and populate it with both authors and salespeople. This has to be done in three gulps since the driver seems to dislike the idea of doing a batch, and it kicks when I try to UNION the two results.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

$ExcelFilePath=‘C:UsersAdministratorDocumentsPOSHScriptsPubs.xlsx’ #the full path of the excel workbook

$Header= $true # true if you want your first row to be read as column headers

if (!(Test-Path $ExcelFilePath))

{

Write-Error «Can’t find ‘$($ExcelFilePath)’.  Sorry, can’t proceed because of this»

exit

}

try {

$Connection = New-Object system.data.odbc.odbcconnection

$TheConnectionString = ‘Driver={Microsoft Excel Driver (*.xls, *.xlsx, *.xlsm, *.xlsb)};DBQ=’+$ExcelFilePath+‘;Mode=ReadWrite;ReadOnly=false;Extended Properties=»HDR=’+«$(if ($Header){‘YES’}else{‘NO’})»+‘»‘

$Connection.ConnectionString=$TheConnectionString

$Connection.Open()

}

catch

{

$ex = $_.Exception

Write-Error «whilst opening connection to  $ExcelFilePath using ‘$($TheConnectionString)’ : $ex.Message»

}

$Mycommand=$connection.CreateCommand()

$MyCommand.CommandText=

  CREATE TABLE people

    (Person varchar)

«@

if ($Mycommand.ExecuteNonQuery() -eq -1)

{$MyCommand.CommandText=

    INSERT into [people$](person)

       SELECT lname FROM [employee$]

«@

$rows=$Mycommand.ExecuteNonQuery()

$MyCommand.CommandText=

    INSERT into [people$](person)

       SELECT  au_fname FROM [authors$]

«@

$rows=$rows+$Mycommand.ExecuteNonQuery()

}

«$rows rows inserted into table»

$connection.Close()

You’ll find you can UPDATE, INSERT and DELETE data perfectly happily this way.  If you connect up a spreadsheet to a SQL Server database, then you can have a lot of fun copying entire databases into spreadsheets, and back again. Robyn and I show how to do this here.

The problem is in the Workbook you create. Whether you name it XLS or XSLX it produces an XLSX spreadsheet, in the latest zipped Office Open XML form.  The trouble is that, with my version of the driver,  I can only get  Excel to read it with the XLS filetype, since it says that there is an error if you try to open it as an .XLSX file. I suspect that the ODBC driver hasn’t been that well tested by Microsoft.

Getting data into SQL Server from Excel using PowerShell

Now, what about using PowerShell to copy the data, maybe filtered, sorted and aggregated, into SQL Server, using PowerShell and ODBC. In this direction we can save a lot of time by using the BCP library. We’ll now describe the routine.

We’ll keep this unpacked, as a script rather than a function, since this is designed to illustrate the process.

We’ll start by defining our credentials, preferences, sources and destinations. We’ll read in the data from and excel spreadsheet and then spit it out into SQL Server, creating a table if necessary. To create the destination table (some of these spreadsheets are rather wide and therefore easier to import automatically), we’ll need to examine the metadata, and to interpret this to the SQL Server equivalent, so we’ll do that. To use the BCP library, it is good to have an indication of progress so I’ll show how you do that.

I’ve provided the sample data so that you don’t have to scramble around to find something suitable. This is some climate data, which is handy for checking things like date conversion.

You will notice that although you can render numbers in a variety of ways, there is only one way of storing numbers in Excel, in the ‘NUMBER‘ datatype (the other datatypes in Excel are LOGICAL, CURRENCY, VARCHAR and DATETIME).  I’ve therefore had to specify the precision of numeric data, which is tough if you have some columns with integers and others with real decimal data with numbers after the decimal point (scale). Remember that this routine is just creating a staging table, not the final destination. All you need to do is to add your own statements to transfer the data to their final table with the CAST to the correct internal data type!

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

set-psdebug -strict

$ErrorActionPreference = «stop»

$ExcelFilePath = ‘MyPathCambridgeWeatherData.xlsx’ #the full path of the excel workbook

$Worksheet = ‘cambridgedata’ #this is the actual worksheet where the data is

$DataRange = » #e.g. ‘A2:M33’  this is the range of the cells that make up the table. leave blank to read the whole worksheet

# leave out the second row number to read all rows from the column range

$Header = $true # true if you want your first row to be read as column headers

# If you aren’t reading columns they are labelled F1..n. You can easily specify them

#$ColumnNames=»’2011» as year,[F1] as Day’

$ColumnNames = ‘*’

#If you dont have fieldnames in the header of your worksheet, you can specify $Header= $false and use F1..Fn instead.

$DestinationTable = ‘CambridgeClimateData’ #the name of the SQL Server table where you want to put the data

$Destinationinstance = ‘MyInstance’ #the name of the server or instance

$Destinationdatabase = ‘MyDataBase’ #the name of the datatabase where you want to put the data

$DestinationWindowsSecurity = $true #or $False if you aren’t using Windows security

$DestinationUserID = » #the name of the SQL Server user if not integrated security

$DeleteContentsOfTableBeforeCopy = $false

$PrecisionForNumericData = 1

if (!(Test-Path $ExcelFilePath))

{

    Write-Error «Can’t find ‘$($ExcelFilePath)’.  Sorry, can’t proceed because of this»

    exit

}

try

{

    $Connection = New-Object system.data.odbc.odbcconnection

    $TheConnectionString = ‘Driver={Microsoft Excel Driver (*.xls, *.xlsx, *.xlsm, *.xlsb)};DBQ=’ + $ExcelFilePath + ‘; Extended Properties=»READONLY=TRUE; HDR=’ + «$(if ($Header) { ‘YES’ }

        else { ‘NO’ })» + ‘»‘

    $Connection.ConnectionString = $TheConnectionString

    $Connection.Open()

}

catch

{

    $ex = $_.Exception

    Write-Error «whilst opening connection to  $ExcelFilePath using ‘$($TheConnectionString)’ :  $($ex.Message). Sorry, can’t proceed because of this»

    exit

}

# get the types via $Connection.GetSchema(‘DataTypes’)|select TypeName, DataType,SQLType

try

{

    $Query = New-Object system.data.odbc.odbccommand

    $Query.Connection = $connection

    $Query.CommandText = ‘Select’ + $columnNames + ‘ from [‘ + $Worksheet + ‘$’ + $DataRange + ‘]’

    $Reader = $Query.ExecuteReader([System.Data.CommandBehavior]::SequentialAccess) #get the datareader and just get the result in one gulp

}

catch

{

    $ex = $_.Exception

    Write-Error «whilst making the query ‘$($Query.CommandText)’ $ex.Message Sorry, but we can’t proceed because of this!»

    Exit;

}

$columns = $reader.GetSchemaTable() | select columnName, datatype

if ($DeleteContentsOfTableBeforeCopy) { $deletionScript = «ELSE DELETE from $DestinationTable « }

else { $deletionScript = » }

$CreateScript =

IF NOT EXISTS

(select TABLE_NAME from information_schema.tables

    where TABLE_NAME like ‘$DestinationTable’)

CREATE TABLE $DestinationTable (

«@

$CreateScript += $columns | foreach-object{ $datatype = «$($_.dataType)»; «`n`t[$($_.columnName.Trim())]  $(switch ($dataType) { ‘double'{ «numeric(18,$PrecisionForNumericData)» } ‘boolean'{ ‘int’ } ‘decimal'{ ‘Money’ } ‘datetime'{ ‘DateTime’ }

            default { ‘NVARCHAR(MAX)’ } }),» }

$CreateScript = $CreateScript.Substring(0, $CreateScript.Length 1) + «`n`t)`n $deletionScript»

$DestinationConnectionString = «Data Source=$Destinationinstance;Initial Catalog=$Destinationdatabase;$(

    if ($DestinationWindowsSecurity) { ‘integrated security=true’ }

    else { ‘User Id=’ + $DestinationUserID + ‘;Password=’ + «$(((Get-Credential $DestinationUserID).GetNetworkCredential()).Password)» + ‘;integrated security=false’ }

try

{

    #test to see if the table is there. If it isn’t, then create it. If it is, then delete the contents

    $SqlCommand = new-object (‘Data.SqlClient.SqlCommand’) $CreateScript, $DestinationConnectionString;

    $SqlCommand.Connection.Open();

    $handler = [System.Data.SqlClient.SqlInfoMessageEventHandler] { param ($sender,

            $event) Write-Host «Message: $($event.Message)» };

    $SqlCommand.Connection.add_InfoMessage($handler);

    $success = $SqlCommand.ExecuteNonQuery();

    #now squirt the data in using the bulk copy library.

    $bulkCopy = new-object («Data.SqlClient.SqlBulkCopy») $DestinationConnectionString

    $bulkCopy.DestinationTableName = $DestinationTable

    $bulkCopy.BatchSize = 5000 #The number of rows in each batch sent to the server

    $bulkcopy.NotifyAfter = 200 #The number of rows to copy before firing a notification

    $bulkCopy.BulkCopyTimeout = 0 #the number of seconds before a time-out

    $objectEvent = Register-ObjectEvent $bulkcopy SqlRowsCopied -Action { write-host «Copied $($eventArgs.RowsCopied) rows « }

    $bulkCopy.WriteToServer($reader) #copy all rows to the server

}

catch

{

    $ex = $_.Exception

    Write-Error «Whilst doing the bulk copy ‘$($Query.CommandText)’ $ex.Message Sorry, but we can’t proceed because of this!»

}

$Reader.Close()

$SqlCommand.Connection.Close()

$Connection.Close()

OK, but does it work with real data? Off to the Health and Social Care Information Centre for some realistic data in spreadsheet form. I’ve included some data just so you don’t have to go to the site to play along, but it is far better to use the latest version of this data from the site. I’m sure I don’t have to tell you how easy this is to do in a script via PowerShell.

$ExcelFilePath=‘MyPathhosp-epis-stat-admi-tot-ops-11-12-tab.xlsx’ #the full path of the excel workbook

$Worksheet=‘Total procedures’ #this is the actual worksheet where the data is

$DataRange=  ‘A16:J1509’  #e.g. ‘A2:M33’  this is the range of the cells that make up the table. leave blank to read the whole worksheet

Also

$DestinationTable=‘Hosp’ # or whatever you want. The name of the SQL Server table where you want to put the data

 …and

$PrecisionForNumericData=0

 Try it. Whoosh. In it goes. If you were doing this as a routine, you’d be wanting to wrap this script into a function with parameters by now, but you know how to do this already, I’m sure. I’m trying to give you the ‘workbench’ narrative here.

Writing to Excel from SQL Server.

The process of going from SQL Server to excel via ODBC is, I think, needlessly complicated, especially if you use parameterised queries (excellent for SQL Server but they add very little for writing to Excel).In this example, I’ll do the old and horrible approach of using insert statements. There are other ways, including even using a dataset, but this is the most obvious.

I’m not particularly happy with this sample because Excel whines a bit when it opens it, saying that it is in the wrong format, (which it is, but you try naming it XLSX) but it deigns to open it.

“The file you are trying to open, ‘MyExcelFile.xls’, is in a different format than specified by the file extension. Verify that the file is not corrupted and is from a trusted source before opening the file. Do you want to open the file now?”

More seriously, it complains that the numbers in the columns are ‘formatted as text’. It turns out that the data is saved in the correct format, but the next time the file is opened, all columns revert to varchar. Seasoned users of ODBC gets used to the bugs, but if anyone knows of a workaround to this, I’d be grateful.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

set-psdebug -strict

$ErrorActionPreference = «stop»

$Sourceinstance = ‘MyServerOrInstance’ #the name of the server or instance

$Sourcedatabase = ‘AdventureWorks’ #the name of the datatabase where you want to get the data

#here is where we put the SQL command to get the result from the database

$SelectStatementForDatabase =

SELECT ProductNumber, p.Name AS ProductName, color,  SafetyStockLevel, ReorderPoint, StandardCost, ListPrice,

NonDiscountSales = (OrderQty * UnitPrice), Discounts = ((OrderQty * UnitPrice) * UnitPriceDiscount)

FROM Production.Product AS p

INNER JOIN Sales.SalesOrderDetail AS sod

ON p.ProductID = sod.ProductID

where ((OrderQty * UnitPrice) * UnitPriceDiscount)>0

ORDER BY ProductName DESC;

«@

$SourceWindowsSecurity = $false #or $True if you are using Windows security

$SourceUserID = ‘SA’ #the name of the SQL Server user if not integrated security

$DestinationTable = ‘ProductWithDiscounts’

$DestinationExcelFilePath = ‘MyPathMyName.xls’ #the full path of the excel workbook

$DestinationHeader = $true # true if you want your first row to be read as column headers

#firstly, we create a connection string ‘on the fly’

#connect to the datanbase

#…and get the DataReader object

$SourceConnectionString = «Data Source=$Sourceinstance;Initial Catalog=$Sourcedatabase;$(

    if ($SourceWindowsSecurity) { ‘integrated security=true’ }

    else { ‘User Id=’ + $SourceUserID + ‘;Password=’ + «$(((Get-Credential $SourceUserID).GetNetworkCredential()).Password)» + ‘;integrated security=false’ })»

try

{

    #here we open a connection to the SQL Server source database

    $SqlCommand = new-object (‘Data.SqlClient.SqlCommand’)  $SelectStatementForDatabase, $SourceConnectionString;

    $SqlCommand.Connection.Open(); #we open the connection

    $handler = [System.Data.SqlClient.SqlInfoMessageEventHandler] {

        param ($sender,

            $event) Write-Host «Message: $($event.Message)»

    };

    $SqlCommand.Connection.add_InfoMessage($handler);

    $Reader = $SqlCommand.ExecuteReader([System.Data.CommandBehavior]::SequentialAccess) #get the datareader and just get the result in one gulp

}

catch

{

    $ex = $_.Exception

    Write-Error «whilst getting data from  $Sourceinstance  $Sourcedatabase ‘ : $ex.Message»

    exit

}

# excel has only the LOGICAL,CURRENCY,NUMBER,VARCHAR,DATETIME datatypes

# according to $connection.GetSchema(‘DATATYPES’).TypeName

# lets work out what the Excel datatype would be…

$columns = $reader.GetSchemaTable() | select columnName, datatype, @{

    name = ‘ExcelDatatype’; expression = {

        switch ($_.datatype)

        { { @(‘float’, ‘decimal’, ‘Numeric’) -contains $_ } { ‘Number’ } ‘bit’ { ‘logical’ } ‘int16’{ ‘Int’ } { @(‘smallmoney’, ‘money’) -contains $_ } { ‘currency’ } ‘DateTime’{ ‘datetime’ }

            default { ‘VarChar’ }

        }

    }

}

# now we need to create an equivalent worksheet in the Workbook.

#If there is no workbook, it will create it

$CreateScript =

CREATE TABLE $DestinationTable (

«@

$CreateScript += $columns | foreach-object{ «`n`t$($_.ColumnName.Trim())  $($_.ExcelDataType),» }

$CreateScript = $CreateScript.Substring(0, $CreateScript.Length 1) + «`n`t)»

# and make a columnlist for the insert statement.

$columnList = ‘[‘ + $columns[0].ColumnName + ‘]’

for ($ii = 1; $ii -le $columns.Length 1; $ii++) { $params += ‘,?’; $columnList += ‘ ,[‘ + $columns[$ii].ColumnName + ‘]’ }

try

{

    #to open the destination workbook or create it if not exist

    $Connection = New-Object system.data.odbc.odbcconnection

    $TheConnectionString = ‘Driver={Microsoft Excel Driver (*.xls, *.xlsx, *.xlsm, *.xlsb)};DBQ=’ + $DestinationExcelFilePath + ‘;Mode=ReadWrite;ReadOnly=false;Extended Properties=»HDR=’ + «$(if ($DestinationHeader) { ‘YES’ }

        else { ‘NO’ })» + ‘»‘

    $Connection.ConnectionString = $TheConnectionString

    $Connection.Open()

    $insertionCommand = $Connection.CreateCommand()

}

catch

{

    $ex = $_.Exception

    Write-Error «whilst opening connection to  $DestinationExcelFilePath using ‘$($TheConnectionString)’ : $ex.Message»

    exit

}

try

{

    #if the table doesn’t exist we create it.

    $CreateTableCommand = $Connection.CreateCommand()

    $CreateTableCommand.CommandText = $CreateScript

    if ($connection.GetSchema(‘TABLES’).Table_Name -notcontains $DestinationTable)

    { if ($CreateTableCommand.ExecuteNonQuery() -eq -1) { write-host «created table (worksheet) $DestinationTable» } }

}

catch

{

    $ex = $_.Exception

    Write-Error «couldn’t create table with command $CreateScript : $ex.Message»

    exit

}

$rows = 0

try

{

    #now we create each insert statement on the fly! Developers look away, please

    while ($Reader.Read())

    {

        $insertcommand = «INSERT INTO [$destinationTable» + ‘$] (‘ + «$columnList) VALUES(«

        for ($i = 0; $i -lt $Reader.FieldCount; $i++)

        {

            $insertcommand += «$(if ($i -eq 0) { » }

                else { ‘,’ }) $(if ($columns[$i].ExcelDataType -eq ‘VarChar’) { «‘$($reader.GetValue($i) -replace «‘«, «»«)'» }

                else { «$($reader.GetValue($i))» }) «

        }

        $insertioncommand.CommandText = $insertcommand + ‘)’

        $rows += $insertionCommand.ExecuteNonQuery()

    }

}

catch

{

    $ex = $_.Exception

    Write-Error «whilst writing to column $i of file  $DestinationExcelFilePath ‘ : $ex.Message»

}

#we report what we’ve done.

write-host «Wrote $rows rows of $($columns.count) columns to worksheet $destinationTable»

$Reader.Close()

$SqlCommand.Connection.Close()

$connection.Close()

CSV and Delimited ODBC Sources: Text AdventureWorks.

Although the ACE drivers are used more by people reading Excel files, I must emphasize that there are drivers for a number of other formats. It is pretty easy, for example, to turn a bunch of CSV files into a relational database. Just to prove it, I’ve created a CSV/Text version of AdventureWorks, together with its schema.ini. This was originally created in this article The TSQL of CSV: Comma-Delimited of Errors. With this text-based database, you can do a lot of the sample AdventureWorks SQL examples with only a minor modification.

Once you’ve installed the ACE drivers, you’ll can use a modified version of the routine I showed you or exploring the PUBS Excel database to play along.

All you have to do is to unzip Text Adventureworks into a new directory with the name of your database (AdventureWorks) and point your connection string at the directory by giving it the full path to the directory. I just altered two lines

#set the directory in which your database should go.  

$TextFilePath=‘MyPathToTheDirectoryTextAdventureWorks’ #the path to the database

… and

$Connection.ConnectionString=‘Driver={Microsoft Access Text Driver (*.txt, *.csv)};DBQ=’+$TextFilePath+»

Now you should be ready with your text-based relational database.

You can, of course, create tables and write to them using the INSERT statement.

create table [Log#csv] (MyInteger int,TheDate date TheMessage char(125))

…and do insert statements into it. You can SELECT INTO as well, which is new to me. I didn’t notice this in previous incarnations of this driver.

With CREATE statements, you can use ‘BIT, BYTE , LONGCHAR, CURRENCY, INTEGER, SMALLINT, REAL, FLOAT, CHAR or DATETIME

(Out of curiosity, the OLEDB driver allows  Long, Single, Double, Currency, DateTime , Bit, Byte, GUID, BigBinary, LongBinary, VarBinary, LongText, VarChar char and Decimal)

# You can list out the tables

$Connection.GetSchema(«tables»)|select table_name  

And the schema

$Connection.GetSchema(«columns»)|select tableName, ColumnName, cardinalPosition

 Here are a few of the SQL Statements that work

SELECT * into [gloves#csv]

FROM [Production_ProductModel#csv]

WHERE ProductModelID IN (3, 4)

 SELECT count(*) as [discounted]

 FROM [Production_Product#csv] AS p

 INNER JOIN [Sales_SalesOrderDetail#csv] AS sod

 ON p.ProductID = sod.ProductID

 where ((OrderQty * UnitPrice) * UnitPriceDiscount)>0

SELECT Name, ProductNumber, ListPrice AS Price

FROM [Production_Product#csv]

WHERE ProductLine = ‘R’

AND DaysToManufacture < 4

ORDER BY Name DESC

SELECT p1.ProductModelID

FROM [Production_Product#csv] AS p1

GROUP BY p1.ProductModelID

having p1.ProductModelID >100

SELECT p1.ProductModelID

FROM [Production_Product#csv] AS p1

GROUP BY p1.ProductModelID

HAVING MAX(p1.ListPrice) >= ALL

 (SELECT AVG(p2.ListPrice)

 FROM [Production_Product#csv] AS p2

 WHERE p1.ProductModelID = p2.ProductModelID)

SELECT top 50 SalesOrderID, SUM(LineTotal) AS SubTotal

FROM [Sales_SalesOrderDetail#csv]

GROUP BY SalesOrderID

ORDER BY SalesOrderID;

SELECT ProductModelID, Name

FROM [Production_ProductModel#csv]

WHERE ProductModelID IN (3, 4)

union all

SELECT ProductModelID, Name

FROM [Production_ProductModel#csv]

WHERE ProductModelID NOT IN (3, 4)

Conclusions

If only Microsoft put some energy into their whole range of ODBC drivers, including all the possible datastores that can be mapped to relational databases,  they’d be the obvious way of transferring data, and would put Microsoft in great shape for providing ‘big data’ solutions.. As it is, they are extraordinarily useful, but marred by  quirks and oddities.

For me, ODBC is the obvious way to  script  data from Excel or Access into SQL Server, for doing data imports.  

Introduction

ODBC is an open specification for providing application developers with a predictable API with which to access Data Sources. Data Sources include SQL Servers and any Data Source with an ODBC Driver.

With the need for an open-source implementation and compatibility with other operating systems, unixODBC was born. This project also has a graphical interface that you can use but its potential is in the binaries that offer compatibility with this implementation.

Why use ODBC?

The two major advantages of choosing to code an application to the ODBC API are;

Portable Data Access Code

The ODBC API, as outlined by X/Open and ISO, is availible on all major platforms. Microsoft platforms include many enhancements to this specification; these enhancements are also supported by unixODBC

Dynamic Data Binding

This allows the user or the system administrator to easily configure an application to use any ODBC compliant data source. This is perhaps the single biggest advantage of coding an application to the ODBC API and to purchase these applications. Dyamic binding allows the end-user to pick a data source, ie an SQL Server, and use it for all data applications without having to worry about recompiling the application. 

Installing ODBC on Ubuntu 21.04

Upgrading the system completely is the first step in performing this tutorial.

So, open a terminal and run the following commands:

$ sudo apt update && sudo apt upgrade

Once the installation has been completed, all the basic tools for compiling and building packages must be installed. This can easily done with the following command:

$ sudo apt install build-essential -y

This will install all the dependencies we need to install unixODBC on Ubuntu 21.04

The next step is to download the unixODBC source code file. At the time of writing, the latest stable version is 2.3.9

$ wget ftp://ftp.unixodbc.org/pub/unixODBC/unixODBC-2.3.9.tar.gz

Then decompress the generated file and go to the unixODBC folder.

$ tar xvzf unixODBC-2.3.9.tar.gz

There you prepare the file for compilation:

$ cd unixODBC-2.3.9/
$ ./configure --prefix=/usr/local/unixODBC

Then, using the make command to create the package

$ make

And finally, install it on the system by running the following command:

$  make install

When the process is finished you can go to the /usr/local/unixODBC/bin folder and see all the binaries that have been installed.

$ cd /usr/local/unixODBC/bin/
$ ls

So unixODBC is ready for battle. You can now implement it and connect to various databases such as those made in Microsoft SQL Server.

Conclusion

So In this post, we show you how to install ODBC on Ubuntu 21.04 through a free implementation. If you use this database access API a lot, it will serve you well.. It’s a very interesting database project that can be very useful to many professionals. So in this post, I’ve shown you how to install it from the source code.

This article is part of a series that includes Firebird, Postgresql, Microsoft SQLServer, Oracle RDBMS, HSQLDB, MariaDB, MongoDB, and Excel. The goal is to set up a self-standing environment for testing an ODBC extension for gawk presented here to be completed. Excepting the present part which deals with the ODBC Driver manager’s installation which is a prerequisite, each part can be used independently from the others.
The test system is a debian v11 (bullseye).

Installation of the ODBC driver Manager

There are 2 main implementations of ODBC for Linux: UnixODBC (http://www.unixodbc.org/) and iODBC (http://www.iodbc.org); we picked the former for no particular reason.
The driver manager can be installed as root through the standard package management tool in Debian:

[email protected]:~# apt-get install libodbc1
[email protected]:~# apt install unixodbc 

[email protected]:~# odbcinst -j
unixODBC 2.3.6
DRIVERS............: /etc/odbcinst.ini
SYSTEM DATA SOURCES: /etc/odbc.ini
FILE DATA SOURCES..: /etc/ODBCDataSources
USER DATA SOURCES..: /root/.odbc.ini
SQLULEN Size.......: 8
SQLLEN Size........: 8
SQLSETPOSIROW Size.: 8

The system-wide file /etc/odbcinst.ini stores the list of installed ODBC drivers and is maintained by root. It is the default one, but its location can be changed and set in the environment variable $ODBCINST.
Let’s create a working folder in user debian’s home directory, our test account:

$ id
uid=1000(debian) gid=1000(debian) groups=1000(debian)
$ mkdir ~/odbc4gawk
$ export workdir=~/odbc4gawk
$ mkdir $workdir
$ cd $workdir

As most of the ODBC drivers have their shared libraries installed in /usr/lib/x86_64-linux-gnu/odbc, this path must be added the $LD_LIBRARY_PATH environment variable of any user that must use ODBC, preferably in the ~/.bashrc file (if bash is the select shell):

export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/odbc:$LD_LIBRARY_PATH

We are now ready to install the ODBC drivers for each data source of interest.

Installation of the SQLite ODBC driver

SQLite along with its administrative tool isql are installed by the native package manager as root:

# apt install sqlite3
# apt install libsqliteodbc

Each time an ODBC driver is installed, it updates the file /etc/odbcinst.ini with driver-dependent information, which can later be checked using the above command odbcinst, e.g.:
List the installed ODBC drivers:

$ odbcinst -q -d
[SQLite3]

Query a particular driver:

$ odbcinst -q -d -n SQLite
SQLite3]
Description=SQLite3 ODBC Driver
Driver=libsqlite3odbc.so
Setup=libsqlite3odbc.so
UsageCount=1

As user debian, edit the file ~/.odbc.ini and add the SQLite data source’s details we will use:

$ cd $workdir 
$ mkdir sqlite
$ cd sqlite
$ vi ~/.odbc.ini
[mysqlitedb]
Description=My SQLite sample database
Driver=SQLite3
Database=/home/debian/odbc4gawk/sqlite/sampledb

Let’s check it:
List all the DSN currently defined in ~/.odbc.ini

$ odbcinst -q -s
[mysqlitedb]

List our new DSN:

$ odbcinst -q -s -n mysqlitedb
[mysqlitedb]
Description=My SQLite sample database
Driver=SQLite3
Database=/home/debian/odbc4gawk/sqlite/sampledb

which is the information we just entered above.

The ~/.odbc.ini file allows to hide a drivers’ private settings so that they don’t need to be specified later. When connecting to a data source, the name that is specified here between square brackets, the DSN, is enough. It is also possible to directly specify in-line all the parameters in this section when connecting programmatically but the code will need to be edited (and maybe recompiled) in case one them has to be changed, so using a DSN is preferable.
The default file location is the current user’s home directory but it can be changed and its full path name set in the environment variable $ODBCINI.

Let’s test the connection via ODBC using the command isql included with the ODBC driver manager:

$ isql mysqlitedb -v
+---------------------------------------+
| Connected!                            |
|                                       |
| sql-statement                         |
| help [tablename]                      |
| quit                                  |
|                                       |
+---------------------------------------+

The ODBC driver for SQLite does work.

To populate a database, start sqllite3:

$ /usr/bin/sqlite3
SQLite version 3.34.1 2021-01-20 14:10:07
Enter ".help" for usage hints.
Connected to a transient in-memory database.

Use “.open FILENAME” to reopen a persistent database.

sqlite> .open sampledb

The file sampledb has been created in the directory specified in ~/.odbc.ini.
Go to the following links, copy the SQL statements and paste them in sqllite3: table creation and insert data.

Still in sqlite3, check that the database has been populated by running the test SQL query listed above:

sqlite> SELECT
        c.country_name,
        c.country_id,
        l.country_id,
        l.street_address,
        l.city
FROM
        countries c
LEFT JOIN locations l ON l.country_id = c.country_id
WHERE
        c.country_id IN ('US', 'UK', 'CN');
Output:
China|CN|||
United Kingdom|UK|UK|8204 Arthur St|London
United Kingdom|UK|UK|Magdalen Centre, The Oxford Science Park|Oxford
United States of America|US|US|2014 Jabberwocky Rd|Southlake
United States of America|US|US|2011 Interiors Blvd|South San Francisco
United States of America|US|US|2004 Charade Rd|Seattle

Now, quit sqlite3 and re-run the same query from isql, the ODBC-based utility:

$ isql -v mysqlitedb
SQL> SELECT c.country_name, c.country_id, l.country_id, l.street_address, l.city FROM countries c LEFT JOIN locations l ON l.country_id = c.country_id WHERE c.country_id IN ('US', 'UK', 'CN')
+---------------------------+-------------+-----------+---------------------------------------------+----------------------+
| country_name              | country_id  | country_id| street_address                              | city                 |
+---------------------------+-------------+-----------+---------------------------------------------+----------------------+
| China                     | CN          |           |                                             |                      |
| United Kingdom            | UK          | UK        | 8204 Arthur St                              | London               |
| United Kingdom            | UK          | UK        | Magdalen Centre, The Oxford Science Park    | Oxford               |
| United States of America  | US          | US        | 2014 Jabberwocky Rd                         | Southlake            |
| United States of America  | US          | US        | 2011 Interiors Blvd                         | South San Francisco  |
| United States of America  | US          | US        | 2004 Charade Rd                             | Seattle              |
+---------------------------+-------------+-----------+---------------------------------------------+----------------------+
SQLRowCount returns 0
6 rows fetched

Note: the output has been shrunk a bit to reduce horizontal scrolling.

Everything looks good so far. Let’s now install the pyodbc ODBC module for python as root;

# apt install pip
# needed to compile pyodbc:
# apt install unixodbc-dev
# pip install pyodbc

As debian, execute the following python script with the following connection string:

import pyodbc 
cnxn = pyodbc.connect(DSN='mysqlitedb;UID=SA;PWD=admin2021!')
cursor = cnxn.cursor()  
cursor.execute("""SELECT
        c.country_name,
        c.country_id,
        l.country_id,
        l.street_address,
        l.city
FROM
        countries c
LEFT JOIN locations l ON l.country_id = c.country_id
WHERE
        c.country_id IN ('US', 'UK', 'CN')""")
row = cursor.fetchone() 
while row:
    print (row) 
    row = cursor.fetchone()
Output:
('China', 'CN', None, None, None)
('United Kingdom', 'UK', 'UK', '8204 Arthur St', 'London')
('United Kingdom', 'UK', 'UK', 'Magdalen Centre, The Oxford Science Park', 'Oxford')
('United States of America', 'US', 'US', '2014 Jabberwocky Rd', 'Southlake')
('United States of America', 'US', 'US', '2011 Interiors Blvd', 'South San Francisco')
('United States of America', 'US', 'US', '2004 Charade Rd', 'Seattle')Output:

Everything works as expected.
We have completed the steps to access an SQLite database via ODBC from isql and from the python module pyodbc. It was quite straightforward with that RDBMS. Once the gawk interface is completed, it should return the same output.
Turn now to the links below for the other data sources:
Firebird
HSQLDB
MariaDB
PostgreSQL
Oracle
Microsoft SQLServer for Linux
MongoDB
Excel

Post Views: 2,736

Like this post? Please share to your friends:
  • Excel odbc error 08001
  • Excel odbc driver x64 скачать
  • Excel objects properties and methods
  • Excel objects how to
  • Excel object workbook failed