Basic Concepts

Sql *Loader

It is a high speed data loading utility supplied by Oracle that loads data from external files into table in an Oracle Database. It can accepts data in variety of formats, can perform transformation, filtering and can load into multiple tables from multiple files in same loading session
2

Requirements- For Sql *Loader

Tables to be loaded must already exists in the Database. SQL *Loader never creates tables, it loads existing tables. Table may be empty or may already contain data. Privileges: INSERT privilege DELETE privilege, when using REPLACE or TRUNCATE insert option
3

Files used in Sql *Loader

Control File - .ctl file Data File - .dat file Log File - .log file Bad File - .bad file Discard File - .dis file

Control File
It controls the Behavior of Sql *Loader Source of Data to be loaded Destination of Data to be loaded Filtering of Data before Loading Transformation of Data before Loading Relating the Data file fields to table columns
5

Log File

It is a record of SQL *Loaders activities during a load session Control, Data, Bad, Discard file Name. Values of several command line parameters. Detailed breakdown of the fields & datatypes in data file that was loaded. Error Message for records that cause error. Message indicating when records have been discarded. Summary of the Load (no: of records read from file, no: of rows rejected because of errors, no: of rows discarded & elapsed time of load)
6

Data File Contains the data to be loaded, it is optional Discard File Contains the data that are discarded by when clause in control file, it is optional. Bad File Contains the data which are not loaded due to some errors, it is not optional, if any one error occurred, SQL *Loader will create the bad file and write the offending input records into it.

SQL *Loader Overview

Input Data File Control File

Log File

SQL *Loader

Bad File

Discard File DataBase

8

Loading Method
Basically there are 3 Loading Methods Conventional Path Load Direct Path Load External Table Load( from Oracle 9i).

Conventional Path Load

Bind Array (Row Insert) is created by Sql *Loader based on field specification in control file Data from the Bind Array are then insert into the table by the DB server, if data satisfies with the corresponding column datatype
10

Direct Path Load

Parses the data according to the field specification in Control File Converts the data to Column datatype (not to field datatype in .ctl file) and builds Column array. Column array is passed to block formatter Newly formatted blocks are directly written directly to the database file bypassing most SQL Processing PARALLEL loading is possible.
11

PARALLEL Loading

Setup has to done, as the text file must be broken into several smaller files We can run several SQLLDR session in parallel for each broken data files It will increase the performance, reduce the time for loading

12

Advantage & Disadvantage of Direct Path Load

Advantage : Very High Speed Disadvantage : Cannot Load Object Types, Collection Types (Nested Table, VARRAY),LOB(CLOB,NCLOB,BLOB, BFILE)
13

Field Conversion
Field 1
DATA FILE Field 2

a a a
CHAR(5) Ctrl File Spec

b b b
CHAR(5)

SQL LOADER

BIND ARRAY

aaa
Table

bbb

DATABASE

Column 1

Column 2

SERVER

a a a_ _
CHAR(5) Column Data Type

bbb
VARCHAR(5)
14

Record Filtering
Record in Data file
Read in

Database

Control file

SQL *Loader Field Processing

Accepted

Bad File
Selected

Inserted

Discard Discarded File

SQL *Loader When-clause Evaluation

Database Server

15

16

Mapping between data fields in control file and columns of the table
SQL> desc sampledata; Name Null Type ------------------------------- -------- ---N1 VARCHAR2(25) N2 VARCHAR2(25) LOAD DATA INFILE '/u01/xxvis/data1.dat' replace into table sampledata fields terminated by X'09' ( n2, n1 ) 12 7891 1234 3456 0111 3333

SQL> select * from sampledata; N1 N2 ------------------------- ---3456 12 0111 7891 3333 1234
17

Data with in Control File

control1.ctl load data $ sqlldr control=control1.ctl userid=apps/secretone infile * into table sql_ldr1 REPLACE ( n position(1:2) ) Sql_ldr1 begindata N 3 3 4 4 5 5 6 6 7 7

18

Data in DATA file

data2.dat 111 222 666 333 444 666 555 666 666 777 888 666 999 000 666

$ sqlldr control=control2.ctl userid=apps/secretone Sql_ldr2: N1 N2 N3 111 222 666 333 444 666 555 666 666 777 888 666 999 000 666
19

control2.ctl load data infile 'data2.dat' insert into table sql_ldr2 replace ( n1 position(1:3), n2 position(5:7), n3 position(9:11) )

Appending to a table
Sql_ldr3 ( Before Loading): --------------------------------N1 N2 N3 ------ --------- --------111 222 666 333 444 666 555 666 666 777 888 666 999 0 666

data3.dat 111 222 666 333 444 666 555 666 666 777 888 666 999 000 666 control3.ctl load data infile 'data3.dat' insert into table sql_ldr3 append ( n1 position(1:3), n2 position(5:7), n3 position(9:11) )

Sql_ldr3 ( After Loading): --------------------------------N1 N2 N3 ------ --------- --------111 222 666 333 444 666 555 666 666 777 888 666 999 0 666 111 222 666 333 444 666 555 666 666 777 888 666 999 0 666

20

Loading the Selected data

data4.dat 111 222 333 444 111 222 333 444 111 222

$ sqlldr control=control4.ctl userid=apps/secretone

$ sqlldr control=control4.ctl userid=apps/secretone dicard = data4.dis control4.ctl

load data infile 'data4.dat' insert into table sql_ldr4 when n2 = '222' ( n1 position(1:3), n2 position(5:7) ) Sql_ldr4: N1 N2 --------- --------111 222 111 222 111 222 data4.dis 333 444 333 444

21

Field Separator
data5.dat 1111;2222; 3333;4444; 1111;2222; 3333;4444; 1111;2222;

$ sqlldr control=control5.ctl userid=apps/secretone

control5.ctl load data infile 'data5.dat' insert into table sql_ldr5 replace when n2='2222' fields terminated by ';' ( n1 integer external, n2 integer external )

Sql_ldr5

N1 N2 -------- --------1111 2222 1111 2222 1111 2222

22

Loading Multiple Data Files

data91.dat control9.ctl 11 22 $ sqlldr control=control2.ctl userid=apps/secretone load data 33 44 infile 'data91.dat' 55 66 Sql_ldr9 infile 'data92.dat' data92.dat insert into N1 N2 77 88 table sql_ldr9 11 22 99 00 replace 33 44 11 11 ( 55 66 n1 position(1:2), 77 88 n2 position(4:5) 99 0 ) 11 11

23

Loading into Multiple Tables

data10.dat 11 22 33 44 55 66

$ sqlldr control=control10.ctl userid=apps/secretone

control10.ctl load data infile 'data10.dat' insert into table sql_ldr101 replace (n1 position(1:2), n2 position(4:5)) into table sql_ldr102 replace (n1 position(1:2), n2 position(4:5))

Sql_ldr101 N1 11 33 55 N2 22 44 66

Sql_ldr102 N1 11 33 55 N2 22 44 66

24

Loading from Multiple Data files to Multiple Tables

data111.dat control11.ctl 11 22 $ sqlldr control=control11.ctl userid=apps/secretone load data 33 44 infile 'data111.dat' 55 66 infile 'data112.dat' Sql_ldr111 Sql_ldr101 data112.dat insert into table sql_ldr111 replace 77 88 99 11 (n1 position(1:2), N1 N2 N1 N2 99 99 11 22 11 22 n2 position(4:5)) 33 44 33 44 into 55 66 55 66 table sql_ldr112 replace 77 88 77 88 (n1 position(1:2), 99 11 99 11 n2 position(4:5)) 99 99 99 99
25

Skipping records
data2.dat 111 222 666 333 444 666 555 666 666 777 888 666 999 000 666

$ sqlldr control=control2.ctl userid=apps/secretone skip=2

control2.ctl load data infile 'data2.dat' insert into table sql_ldr2 replace ( n1 position(1:3), n2 position(5:7), n3 position(9:11) )

Sql_ldr2 N1 N2 N3 555 666 666 777 888 666 999 000 666

26

Using LOAD option

data14.dat aabbb ccddd $ sqlldr control=control14.ctl userid=apps/secretone LOAD = 2 eefff control14.ctl load data infile 'data14.dat' into table sql_ldr14 truncate/replace ( a1 char(2), a2 char(3) ) Sql_ldr14: A1 A2

aa cc

bbb ddd

27

Fixed Record Format

data12.dat 12,3456,7891,0111,2 control12.dat load data infile 'data12.dat' "fix 5" insert into table sql_ldr12 replace fields terminated by ',' ( n1 integer external, n2 integer external )

$ sqlldr control=control12.ctl userid=apps/secretone

Sql_ldr12: N1 N2

12 56 91

34 78 1

28

Variable Record Format

data1.dat 07aaa,bbb08ccc,dddd05ee,ff control1.ctl load data $ sqlldr control=control1.ctl userid=apps/secretone infile data1.dat var 02 Sql_ldr11: into table sql_ldr1 fields terminated by , A1 A2 ( aaa bbb a1 char, ccc dddd a2 char ee ff ); Desc sql_ldr1 a1 Varchar2(10) a2 Varchar2(10)

29

Stream Record Format

data2.dat aaa,bbb|ccc,dddd|eeee,ffff| Desc sql_ldr1 a1 Varchar2(10) a2 Varchar2(10)

Control2.ctl $ sqlldr control=control2.ctl userid=apps/secretone load data Sql_ldr12: infile data2.dat str | into table sql_ldr2 A1 A2 fields terminated by , aaa bbb ( ccc dddd a1 char, eeee ffff a2 char )
30

Command Line Parameters

BAD - Specifies the file in which all bad data is kept. The default filename is the control filename with a .bad extension. BINDSIZE - Size of the bind array in bytes. System dependent. CONTROL - Specifies the name of control file.

DATA - Specifies the file that contains all data to be loaded.

DIRECT - If set to TRUE Direct Path Load is used. Otherwise Conventional Path will be used. Default value is FALSE.

DISCARD - Specifies the file where all discarded data is kept

31

Command Line Parameters

DISCARDMAX The maximum no: of invalid records, which is not satisfying the when clause in control file, that may be encountered before SQL *Loader session is stop. Default value is ALL. ERRORS Specifies how many total errors can be encountered before SQL *Loader session is to stop. Default value is 50. LOAD Specifies the maximum no: of records to load before stopping. Default Value is ALL. LOG Specifies the log files name, where information on success or failure of the Loading session is reported. The Default filename is control_filename.log.

32

Command Line Parameters

PARALLEL If set to TRUE, loads are performed in parallel where possible. Default value is FALSE PARFILE An additional file that contains more parameter specification.

ROWS No: of rows to put in the path bind array, for Conventional path load. For Direct path load, ROWS specifies the no: of rows to read before a data save is performed. Default value is 64 in Conventional Path.
SKIP No: of record to skip before starting the load. This parameter is important for restarting a load process after stopping an earlier session. Useful in Recovery from failure.
33

Command Line Parameters

USERID Specifies the username and password for the user conducting the SQL *Loader session SILENT Allows to suppress various header and feedback messages that SQL *Loader normally displays during a Load session.

Keyword for use with SILENT parameter

DISCARDS Suppresses the discarded messages ERRORS Suppresses the error message FEEDBACK Suppresses the commit point reached messages. HEADER Suppresses the messages that SQL *Loader displays on the screen when we first launch the executable. PARTITION Suppresses the per-partition statistics, when loading a direct path load of a partitioned table.
34

35