From excel to oracle table

There are many different methods, depending 
upon the amount of data, the repetitiveness 
of the process, and the amount of programming 
I am willing to invest.

First, create the Oracle table, using the 
SQL CREATE TABLE statement to define the table's 
column lengths and types. Here's an example of a 
sqlplus 'CREATE TABLE' statement: 

CREATE TABLE SPECIES_RATINGS 
(SPECIES VARCHAR2(10),
COUNT NUMBER,
RATING VARCHARC2(1));

Then load the data using any of the following 
methods or an entirely new method you invent:

--------------------------------------------

First load method:

I use the SQL*Loader method.
You will need to save a copy of your spreadsheet 
in a text format like CSV or PRN. 

SQL*Loader Control file for CSV file:

load data
 infile 'c:datamydata.csv'
 into table emp
 fields terminated by "," optionally enclosed by '"'          
 ( empno, empname, sal, deptno )

There are some GUIs that have wizards to walk you through the
process (Enterprise Manager -> Maintenance -> Data Movement ->
Move Row Data -> Load Data from User Files) for the 
ad-hoc imports. Toad for Oracle has a SQL*Loader Wizard as
well. (DBA -> Data Import/Export -> SQL*Loader Wizard)

You can save your Excel data in PRN format if you are 
planning to use positional data (fixed length) in your 
control file.

SQL*Loader Control file for PRN file:

load data
 infile 'c:datamydata.prn'
 replace
 into table departments
 (  dept     position (02:05) char(4),
    deptname position (08:27) char(20)  )

Position(02:05) will give the 2nd to the 5th character 

Once I've gone through the EM or Toad wizard, I save 
the control file, tweak it as needed in a text editor, 
and reuse it in SQL*Plus scripts.

SQL*Loader is handy also since it allows you to 
skip certain data and call filter functions (i.e.
native functions as in DECODE() or TO_DATE() or 
user defined functions) in your control .ctl file.

You can load from multiple input files provided 
they use the same record format by repeating the 
INFILE clause. Here is an example:

LOAD DATA
  INFILE file1.prn
  INFILE file2.prn
  INFILE file3.prn
  APPEND
  INTO TABLE emp
  ( empno  POSITION(1:4)   INTEGER EXTERNAL,
    ename  POSITION(6:15)  CHAR,
    deptno POSITION(17:18) CHAR,
    mgr    POSITION(20:23) INTEGER EXTERNAL
  )

You can also specify multiple "INTO TABLE" clauses 
in the SQL*Loader control file to load into multiple 
tables.

LOAD DATA
 INFILE 'mydata.dat'
 REPLACE
 INTO TABLE emp
      WHEN empno != ' '
 ( empno  POSITION(1:4)   INTEGER EXTERNAL,
   ename  POSITION(6:15)  CHAR,
   deptno POSITION(17:18) CHAR,
   mgr    POSITION(20:23) INTEGER EXTERNAL
 )
 INTO TABLE proj
      WHEN projno != ' '
 (  projno POSITION(25:27) INTEGER EXTERNAL,
    empno  POSITION(1:4)   INTEGER EXTERNAL
 )

With SQL*Loader, you can selectively load only
the records you need (see WHEN clause), skip 
certain columns while loading data (see FILLER 
columns) and load multi-line records (see
CONCATENATE and CONTINUEIF)

Once you've created the control file, you need 
to start sql loader from the command line like this:
sqlldr username/password@connect_string control=ctl_file.ctl log=log.log 

You can create a batch file to call sqlldr.

For more examples, see
http://examples.oreilly.com/orsqlloader/

That's it for the versatile SQL*Loader.

--------------------------------------------

Second load method: 

In this scenario, I have full control of the 
spreadsheet, but less control of the data because 
users send me the spreadsheets back with data. 

I create another worksheet within the same Excel 
file, which has locked down INSERT statements 
referring back to the sheet with the data. When 
I receive the spreadsheet, I copy and paste the 
INSERT statements directly into SQL*Plus, or 
indirectly staging them in a SQL script.

Excel is a great tool for composing dynamic 
SQL statements dynamically. (see Excel functions)

--------------------------------------------

Third load method:

If you need a utility to load Excel data into 
Oracle, download quickload from sourceforge at 
http://sourceforge.net/projects/quickload 

--------------------------------------------

Fourth load method:

In theory, this should work.

Configure Generic Database connectivity (Heterogeneous Database HS)
Connect to the Excel spreadsheet from Oracle through ODBC.
Describe it (see DESC command) or 
CREATE TABLE AS SELECT col1, col2 FROM ExcelTable 
to make a copy and see what data types Oracle assigns
the columns by default.

http://www.e-ammar.com/Oracle_TIPS/HS/configuring_generic_database_con.htm

--------------------------------------------

References:

http://209.85.173.132/search?q=cache:GJN388WiXTwJ:www.orafaq.com/wiki/SQL*Loader_FAQ+Oracle+control+file+columns&cd=3&hl=en&ct=clnk&gl=us

http://forums.oracle.com/forums/thread.jspa?threadID=305918&tstart=0

http://techrepublic.com.com/5208-6230-0.html?forumID=101&threadID=223797&messageID=2245485

http://examples.oreilly.com/orsqlloader/

TL;DR – Skip to the video!

Hate GUIs, want to do this via the command-line?

THIS is your number one question – and it has been here on my blog since the day I posted it.

You have an Excel file – and you want that data put into a table. I’ll show you how, and we’ll document each step of the way with plenty of pictures.

You will be prepared to import data to an existing table from Excel with confidence after reading this post. Want to build a new table from Excel? We can do that, too.

Warning: This post has a LOT of pictures.

For our example I’ll be using the HR.EMPLOYEES table to create the XLS file for our import. We’ll use that Excel file to populate an empty copy of the EMPLOYEES table in another schema.

Step 0: The Empty Oracle Table and your Excel File

You have an Oracle table and you have one or more Excel files.

You do know how to view multiple objects at once in SQL Developer, right?

Step 1: Mouse-right click – Import Data

Step 2: Select your input (XLSX) file and verify the data

As you select the file, we’ll grab the first 100 rows for you to browse below. This ‘Preview Row Limit’ defines how many rows you can use to verify the IMPORT as we step through the wizard. You can increase it, but that will take more resources, so don’t go crazy.

Also, does your Excel file have column headers? Do we want to treat those as a row to the table? Probably not. If you uncheck the ‘Header’ flag, the column names will become a new row in your table – and probably fail to be inserted.

Sometimes your Excel file has multiple headers, or you may need to only import a certain subset of the spreadsheet. Use the ‘Skip Rows’ option to get the right data.

Step 3: Create a script or import automatically

For this exercise the ‘Insert’ method will be used. Each row processed in the Excel file will result in an INSERT statement executed on the table we’re importing to.

If you choose ‘Insert Script’, the wizard will end with an INSERT Script in your SQL Worksheet. This is a nice alternative if you want to customize the SQL, or if you need to debug/see why the ‘Insert’ method isn’t working.

Working with CSV? You’ll get even more methods – great for VERY LARGE data sets.

Step 4: Select the Excel Columns to be Imported

You may have an Excel file with 100 columns but your table only has 30. This is where you tell SQL Developer what columns are to be used for the import. You can also modify the column order, which may make the next step a bit easier.

Step 5: Map the Excel Columns to the Table Columns

If you’re not paying attention and just letting the wizard guide you home, then now is the time to wake up. There’s a good chance the column order of the Excel file won’t match the definition of your table. This is where you will tell SQL Developer what columns in the spreadsheet match up to what columns in the Oracle table.

And remember how we set that preview window to 100 rows? We’re peaking at the data, looking for problems as we try to fit it into your table column. If we find a problem, we’ll mark the columns with those ‘warning’ symbols.

I’ve polluted my Excel file with some values that I know won’t ‘fit.’ When these rows are encountered in the wizard, they’ll be rejected by the database – but the other rows will come in.

Let’s talk about DATES for a second.

And TIMESTAMPS too. In the excel file, you’re probably going to have some date/time fields you want to move into DATE or TIMESTAMP columns. SQL Developer is treating those value as strings – and YOU need to tell SQL Developer the DATE or TIMESTAMP format to use to be able to convert them.

Let’s look at HIREDATE.

See the ‘Format’ drop down selector? SQL Developer has defaulted the DATE format string to ‘DD-MON-RR’ – we try to guess based on the rows we’re looking at in that 100 preview window.

If we have guessed wrong, or were unable to figure it out, you’ll need to input this yourself. The Oracle Docs can help you define the correct DATE Format Model. If you see the little warning graphics next to your Date values in the Data panel, you might have the wrong format.

Step 6: Verify your settings and GO!

Click on the ‘Finish’ button.

If the Wizard runs into any problems doing the INSERTs, you’ll see this:

I’m going to say ‘Yes’ to ignore all the errors. But, if you need EVERY single row – you need to say ‘Cancel’ to start over. Then you can either fix your data in the Excel file, or make changes to your table so the data will fit/work.

After clicking ‘Yes’, we’ll get to the end of our story, and our data!

Step 7: See What Worked and What Didn’t

First, any bad news?

If there were rows rejected by the database, we’ll see those now.

Now let’s go look at our new table data!

Note the ‘Log’ panel. We’ll show you the file we worked with, and how much time it took to load the data.

Just a few more pointers when it comes to Excel

• Storing data in Excel can be…dangerous
• If you have a lot of data or if this will be a repeating process, consider External Tables or SQL*Loader instead
• If you like to put data in Oracle because you understand SQL better than Excel macros, then welcome to the club 😉
• Here’s 11 more tips on Importing Excel/CSV into your database

The Movie

Задача. У вас есть файл Excel – и вы хотите, чтобы эти данные были помещены в таблицу. Я покажу вам, как это делается, и мы задокументируем каждый шаг с большим количеством иллюстраций.

Прочитав этот пост, вы будете готовы с уверенностью импортировать данные в существующую таблицу из Excel. Хотите создать новую таблицу из Excel? Мы тоже можем это сделать.

В нашем примере я буду использовать таблицу HR.EMPLOYEES для создания XLS-файла для нашего импорта. Мы будем использовать этот файл Excel для заполнения пустой копии таблицы EMPLOYEES в другой схеме.

Шаг 0: Пустая таблица Oracle и ваш файл Excel

У вас есть таблица Oracle и один или несколько файлов Excel.

Шаг 1: Выбираем «Импорт данных» по правому щелчку мыши

Шаг 2: Выберите свой входной файл (XLSX) и проверьте данные

Когда вы выберете файл, мы захватим первые 100 строк для просмотра ниже. Этот «Предел строк предварительного просмотра» определяет, сколько строк вы можете использовать для проверки ИМПОРТА по мере прохождения мастера. Вы можете увеличить его, но это потребует больше ресурсов, так что не сходите с ума.

Кроме того, есть ли в вашем файле Excel заголовки столбцов? Хотим ли мы рассматривать их как строку к таблице? Скорее всего, нет. Если вы снимите флажок «Заголовок», имена столбцов станут новой строкой в вашей таблице – и, вероятно, не будут вставлены.

Иногда ваш файл Excel имеет несколько заголовков, или вам может потребоваться импортировать только определенное подмножество электронной таблицы. Используйте опцию «Пропустить строки», чтобы получить правильные данные.

Шаг 3: Создайте сценарий или импортируйте автоматически

Для этого упражнения будет использоваться метод «Вставки» (Insert). Каждая строка, обработанная в файле Excel, приведет к выполнению инструкции INSERT в таблице, в которую мы импортируем.

Если вы выберете «Вставить скрипт», мастер завершит работу скриптом ВСТАВКИ на вашем листе SQL. Это хорошая альтернатива, если вы хотите настроить SQL, или если вам нужно отладить/посмотреть, почему метод «Insert» не работает.

Шаг 4: Выберите столбцы Excel для импорта

У вас может быть файл Excel со 100 столбцами, но ваша таблица имеет только 30. Здесь вы даете команду SQL Developer, какие столбцы должны использоваться для импорта. Вы также можете изменить порядок столбцов, что может сделать следующий шаг немного проще.

Шаг 5: Сопоставьте столбцы Excel со столбцами таблицы

Если вы не обращаете внимания и просто позволяете Мастеру импорта делать всё по умолчанию, то сейчас самое время проснуться. Есть большая вероятность, что порядок столбцов файла Excel не будет соответствовать определению вашей таблицы. На этом шаге вы указываете SQL Developer-у, какие столбцы в электронной таблице совпадают с какими столбцами в таблице Oracle.

А помните, как мы установили это окно предварительного просмотра на 100 строк? Мы внимательно изучаем данные, ищем проблемы, пытаясь вписать их в столбец вашей таблицы. Если мы обнаружим проблему, мы пометим столбцы этими «предупреждающими» символами.

Я загрязнил свой файл Excel некоторыми намеренно ошибочными значениями, которые, как я знаю, не будут «подходить». Когда эти строки будут обнаружены в мастере, база данных их отклонит, но остальные строки будут введены.

Давайте на секунду поговорим о форматах даты и времени.

О TIMESTAMP тоже. В файле Excel у вас, вероятно, будут некоторые поля даты и времени, которые вы хотите переместить в столбцы формата  DATE или TIMESTAMP. SQL Developer обрабатывает эти значения как строки – и ВАМ нужно сообщить SQL Developer формат DATE или TIMESTAMP, чтобы иметь возможность их преобразовать.

Давайте посмотрим на HIREDATE.

Видите выпадающий селектор «Формат» (Format)? SQL Developer по умолчанию установил строку формата ДАТЫ в ‘DD-MON-RR’ – мы пытаемся угадать это на основе строк, которые мы рассматриваем в этом окне предварительного просмотра 100.

Если мы ошиблись в догадках или не смогли разобраться, вам нужно будет ввести это самостоятельно. Документы Oracle могут помочь вам определить правильную модель формата ДАТЫ. Если вы видите небольшое предупреждающее изображение рядом со значениями даты на панели данных, возможно, у вас неправильный формат.

Шаг 6: Проверьте свои настройки и ВПЕРЕД!

Нажмите на кнопку «Готово» (Finish).

Если Мастер столкнется с какими-либо проблемами при выполнении вставок, вы увидите следующее:

Я собираюсь сказать «Да», чтобы игнорировать все ошибки. Но если вам нужна КАЖДАЯ отдельная строка – вам нужно сказать «Отмена», чтобы начать все сначала. Затем вы можете либо исправить свои данные в файле Excel, либо внести изменения в свою таблицу, чтобы данные соответствовали/работали.

Нажав ‘Да», мы доберемся до конца нашей истории и наших данных!

Шаг 7: Посмотрите, Что сработало, а Что нет

Во-первых, есть ли плохие новости?

Если были строки, отклоненные базой данных, мы увидим их сейчас.

Теперь давайте посмотрим на наши новые табличные данные!

Обратите внимание на панель «Журнал» (Log). Там можно увиеть файл, с которым мы работали, и сколько времени потребовалось для загрузки данных.

Всё! Можно выдохнуть свободно. Импорт из файла Excel в таблицу базы данных Oracle завершен. В шапке статьи вы можете посмотреть видео инструкцию  — копию этого мануала на английском языке.

Are you trying to figure out how to load data from Excel into your database?

The problem: load data to Oracle

Many business analysts rely heavily upon Microsoft Excel as one of their primary sources for important business data. At some point they will need to load data into an Oracle database, and thus you will often hear them make requests such as the following:

• Copying Excel data to Oracle
• Importing Excel data to Oracle
• Loading Excel data to Oracle
• Transferring Excel data to Oracle

They key point is that regardless of how the business users ask, the request is essentially the same. Some very common ways they obtain such loading of the database data include:

Business user sending a formal data load request to their information systems people to perform and deliver the loaded Oracle table. However, this process can take time – sometimes more than the business user can wait.

Business user truly self-servicing by loading the database data themselves using a powerful tool like Quest®Toad® for Oracle, and then working with that data in Toad or some other database tool. This approach is quick and easy, plus Toad for Oracle integrates easily into the business analyst’s workflow.

In my recent blog titled “Copying database data to Microsoft Excel via Toad for Oracle” I showed how easily Toad for Oracle can export data into Excel — a favorite tool of many business analysts.

Toad for Oracle makes short work of this use case as well — how to load data from Excel into Oracle. For my example, I chose to use the Internet Movie Database (IMDB) and thus downloaded the movies title basic data. I then unzipped the file to obtain the 606 megabyte tab delimited text file named data.tsv. I then renamed that data file to movies.tsv so as not to forget what the file contains.

The solution to load data into Oracle

Many business analysts strongly favor the second choice of using Toad for Oracle to self-service and load data. In this blog I’ll review some common scenarios that facilitate easily copying Excel data to Oracle using Toad. The techniques that I’ll be showing work essentially the same in both of Toad’s primary interfaces for this task: the Schema Browser and Main-Menu—> Database —> Create —> Table. I’ll demonstrate using the Schema Browser, which is often the most common choice for this task.

Figure 1: Create a new table to load data

Now in Figure 2 below I’m inside the create table screen. First I chose to check the box for display advanced features, otherwise I would not see displayed the options which will be required. Second, I chose to open a file to read for the table’s column names. It uses the first row of data for that. Third, I chose my tab delimited text file. Finally, I pressed OK. Since my file was over 600 megabytes, it took a few seconds to process.

Figure 2: Method to populate the table columns to load data.

In Figure 3 below I now have all the columns for my table. Note that Toad has set the default datatype to VARCHAR2 for all columns. Moreover Toad also has set each column to be optional. You may need to modify these default choices as you know your data far better than Toad. Then press the OK button to create the table and load data.

Figure 3: Resulting population of table columns.

In Figure 4 below I have now created the table required to hold all the data for the IMDB movies data file. Remember, the file is over 600 megabytes in size, so the data load will likely take a few moments. But we’re more than half-way home at this point. Now I just need to import the data into my newly created table.

Figure 4: Resulting table creation.

In Figure 5 below I have now invoked Toad for Oracle’s Data Import wizard by opening the Right Hand Mouse (RHM) menu and select Import —> Data, or Main Menu —> Database—>Import—>Import Table Data. As you can now see, I am setup to initiate loading of the MOVIES table. However this is a multi-step wizard where we’ll need to specify additional information before the table can be loaded.

Figure 5: Page 1 of the data import wizard.

In Figure 6 below I am now on the second page of the data import wizard where I identify the file type and name. If you look back at Figure 2, you will see that we’ve answered two of the three key parameters. Note that we’ve yet to say that it’s a tab delimited file. Remember too that the first line in the file has the column names, so we’ll need to be able to tell Toad to skip over those.

Figure 6: Page 2 of the data import wizard.

In Figure 7 below I am now on the third page of the data import wizard where I identify the file as tab delimited text.

Figure 7: Page 3 of the data import wizard.

In Figure 8 below I am now on the fourth page of the data import wizard where I identify that Toad should start reading the table data at line 2 because the first row contains the column names. You should take your time on this page of the wizard to make sure you review the dates, times, and numbers settings based upon how your data is formatted in the text file.

Figure 8: Page 4 of the data import wizard.

In Figure 9 below I am now on the fifth page of the data import wizard where Toad has read a sample of the data rows and displays them for you to verify. Moreover, Toad displays a pop-up to see how you want to map the columns of data to the table columns. In this case it really does not matter. But had I created the columns in the table in a different order or reordered them, then the choice would have to be by matching the names.

Figure 9: Page 5 of the data import wizard.

In Figure 10 below I am now on the sixth page of the data import wizard where Toad displays the results of your selection from the prior page’s pop up where you specified the data file field to table column mapping. Again if I had reordered the columns in the table, I would have had to make changes on this page to properly align the fields to columns.

Figure 10: Page 6 of the data import wizard.

In Figure 11 below I am now on the seventh page of the data import wizard where Toad displays the field to column matching results performed upon the sample data with the column header row removed. Note that the columns displayed here are sized based upon the column name size. I’ll need to press the size cols to data button if I wish to be able to see the column values displayed in their entirety.

Figure 11: Page 7 of the data import wizard.

In Figure 12 below I am now on the eighth page of the data import wizard where Toad allows one to specify a host of parameters or options that control the actual database transaction processing. As you can see, I have said to perform batch array inserts of 500 rows, to truncate the table before loading, and to commit after every 500 rows. These choices will affect how long the actual data load takes. Remember that my data file is over 600 megabytes, so setting these parameters is critical. Now when I press the execute button, the data import process will begin.

Figure 12: Page 8 of the data import wizard.

In Figure 13 below I am now executing the data import process. The first time I ran this I got errors because the default column data types and size were VARCHAR(32) and the movies titles are much longer than that. I just chose to abort, increased the column lengths, and restarted the wizard. It then ran to completion without error. Did you note that even on my small VM running on a desktop PC that I’m getting loads of over 66,000 records per second!

Figure 13: Page 9 of the data import wizard.

Conclusion

If you are a business user who wants to copy from Excel to Oracle, use Toad. You can easily do automatic table creation and data loading as shown above. Thus you can self-service without requesting help from your information systems people. No other database tool makes copying data from Excel to Oracle quicker and easier than Toad. Plus no matter whether you’re using Toad freeware or the commercial version of Toad, the process is essentially the same.

Tags:

Toad for Oracle

Written by Bert Scalzo

In this article, we will explain different methods to load data from excel sheet into Oracle database.

Below are the methods that can be used.

• SQL DEVELOPER
•  SQLCL utility
•  TOAD
• SQLLOADER

Here we will explain how to do this with SQL developer and SQLCL utility.

USING SQLDEVELOPER TOOL:

SQL developer is a free tool provided by SQL developer is a free tool provided by Oracle for managing database with GUI interface.

You can download and install it on your desktop. Loading the excel sheet into a table using SQL developer is the easiest way.

Let’s say, we have an excel sheet names employee.xlsx as below.

So first create a table with the columns same as excel sheet.

SQL> show user
USER is "APP_USER"
SQL>
SQL>
SQL> CREATE TABLE EMPLOYEE
( EMPNAME VARCHAR2(100),
  EMPNO   NUMBER ,
  GRADE   VARCHAR2(10),
  SAL     NUMBER
);  2    3    4    5    6

Table created.
  

Once the table is created, connect to the database using SQL developer.

Select the table, which we created,(where data will be loaded).

Right click on the table and select Import Data.

Browse the xlsx file and set format to xls.

Import method -> Insert

We have successfully imported the excel sheet into the table. Let’s check in the database.

SQL> select count(*) from app_user.employee;

  COUNT(*)
----------
         4
		 

USING SQLCL  UTILITY:

SQLCL is a free utility provided by Oracle that provides an option to load CSV file to a database table easily.

For installation and use please refer – SQLCL Utility in Oracle:

But here we have an excel sheet which has the extension. .xlsx So 1st step is to convert this to.xlsx csv .

Steps to convert excel sheet to the csv file.

Open a notepad and paste the below connect and name it as XlsToCsv.vbs

Script reference – https://stackoverflow.com/questions/1858195/convert-xls-to-csv-on-command-line

if WScript.Arguments.Count < 2 Then
    WScript.Echo "Error! Please specify the source path and the destination. Usage: XlsToCsv SourcePath.xls Destination.csv"
    Wscript.Quit
End If
Dim oExcel
Set oExcel = CreateObject("Excel.Application")
Dim oBook
Set oBook = oExcel.Workbooks.Open(Wscript.Arguments.Item(0))
oBook.SaveAs WScript.Arguments.Item(1), 6
oBook.Close False
oExcel.Quit
WScript.Echo "Done"

Now in command prompt, go to the location where this executable file exists.

If it shows Done, then CSV file conversion is successful. Then copy this csv file to your database server.

Login with SQLCL utility and use LOAD command. Before that table structure needs to be created in the database.

SQL> CREATE TABLE EMPLOYEE
( EMPNAME VARCHAR2(100),
  EMPNO   NUMBER ,
  GRADE   VARCHAR2(10),
  SAL     NUMBER
);  2    3    4    5    6

Table created.
  

SYNTAX – LOAD  [ table_name ] [ csv file with path ]

  
SQL> select count(*) from app_user.employee;

  COUNT(*)
----------
         4

We loaded the CSV file successfully.