OracleDBA

co.uk
Stuck in a bind…bind variables

au.fujitsu.com / www.oracledba.co.uk

Presented by
CONNOR MCDONALD
Senior Consultant
Fujitsu Consulting, Asia Pacific

Mar 2004
 OracleDBA

Bits and pieces

co.uk
OracleDBA

co.uk
AVAILABLE NOW COMING SOON

Connor McDonald
April 2004 2
www.oracledba.co.uk
 OracleDBA

Terminology 101 (a)

co.uk
ƒ Which of these is cursor?

delete from PRP_PROPERTY;

drop table MY_TABLE; select *

from VEHICLE

?
where VEH_ID > 1234;

declare
cursor C(p number) is
select * from NOMINAL
where nom_id = p;
begin
begin
MY_PROCEDURE(1,2,3);
for rec in C loop
end;
…
end loop;
end;

Connor McDonald
April 2004 3
www.oracledba.co.uk
 OracleDBA

Answer

co.uk
ƒ All of them!
ƒ Cursors merely a handle (pointer) to an area to run database
commands

Memory

SQL Plus Process

table defs
parameters
dependencies
results
settings

Server

Connor McDonald
April 2004 4
www.oracledba.co.uk
 OracleDBA

Cursor mechanics

co.uk
ƒ All cursors have three phases
ƒ OPEN
ƒ Please allocate me some memory and return me a pointer to it
ƒ PROCESS
ƒ Process the command associated with this cursor
ƒ CLOSE
ƒ Release the memory for re-use

ƒ Can be done explicitly within code

ƒ Implicit in many cases

Connor McDonald
April 2004 5
www.oracledba.co.uk
 OracleDBA

Terminology 101 (b)

co.uk
ƒ Latching
ƒ Semaphoring operation to control access to complex memory structures

Connor McDonald
April 2004 6
www.oracledba.co.uk
 OracleDBA

Terminology 101 (b)

co.uk
ƒ Latching
ƒ Semaphoring operation to control access to complex memory structures

Connor McDonald
April 2004 7
www.oracledba.co.uk
 OracleDBA

Terminology 101 (b)

co.uk
ƒ Latching
ƒ Semaphoring operation to control access to complex memory structures

Connor McDonald
April 2004 8
www.oracledba.co.uk
 OracleDBA

Terminology 101 (b)

co.uk
ƒ Latching
ƒ Semaphoring operation to control access to complex memory structures

ƒ Get latch
1

Connor McDonald
April 2004 9
www.oracledba.co.uk
 OracleDBA

Terminology 101 (b)

co.uk
ƒ Latching
ƒ Semaphoring operation to control access to complex memory structures

ƒ Get latch 2
ƒ Operation 1

Connor McDonald
April 2004 10
www.oracledba.co.uk
 OracleDBA

Terminology 101 (b)

co.uk
ƒ Latching
ƒ Semaphoring operation to control access to complex memory structures

ƒ Get latch
ƒ Operation
ƒ Release latch

Connor McDonald
April 2004 11
www.oracledba.co.uk
 OracleDBA

Terminology 101 (b)

co.uk
ƒ Latching
ƒ Competition means someone must wait

ƒ High CPU cost

ƒ Will CPU help? 2

1

"wait"

Connor McDonald
April 2004 12
www.oracledba.co.uk
 OracleDBA

Processing SQL

co.uk
open cursor

ƒ Syntax check
ƒ Semantics check
ƒ Optimization process cursor
ƒ Row source generation
ƒ Execution
ƒ Fetch

close cursor

Connor McDonald
April 2004 13
www.oracledba.co.uk
 OracleDBA

Processing SQL

co.uk
open cursor

ƒ Syntax check
ƒ Semantics check
ƒ Optimization process cursor
ƒ Row source generation
SQL> select *
ƒ Execution 2 frmo emp;
ƒ Fetch frmo emp
*
ERROR at line 2:
close
ORA-00923: FROM keyword not cursor
found where expected

Connor McDonald
April 2004 14
www.oracledba.co.uk
 OracleDBA

Processing SQL

co.uk
open cursor

ƒ Syntax check
ƒ Semantics check
ƒ Optimization process cursor
SQL> select nom_idd
ƒ Row source generation 2 from nominal;
select nom_idd
ƒ Execution *
ƒ Fetch ERROR at line 1:
ORA-00904: invalid column name

close cursor

Connor McDonald
April 2004 15
www.oracledba.co.uk
 OracleDBA

Processing SQL

co.uk
PLAN
open cursor
-------------------------------------
SELECT STATEMENT
ƒ Syntax check TABLE ACCESS BY INDEX ROWID NOMINAL
ƒ Semantics check INDEX RANGE SCAN NOM_PK

ƒ Optimization process cursor

ƒ Row source generation
ƒ Execution
ƒ Fetch

close cursor

Connor McDonald
April 2004 16
www.oracledba.co.uk
 OracleDBA

Processing SQL

co.uk
open cursor

ƒ Syntax check
ƒ Semantics check
ƒ Optimization process cursor
ƒ Row source generation
ƒ Execution
ƒ Fetch

close cursor

NOM_PK NOMINAL

Connor McDonald
April 2004 17
www.oracledba.co.uk
 OracleDBA

Processing SQL

co.uk
open cursor
ƒ Syntax check
ƒ Semantics check close cursor
ƒ Optimization process cursor
ƒ Row source generation
ƒ Execution
ƒ Fetch

close cursor

Connor McDonald
April 2004 18
www.oracledba.co.uk
 OracleDBA

Processing SQL

co.uk
open cursor
ƒ Syntax check
ƒ Semantics check close cursor
ƒ Optimization process cursor
ƒ Row source generation
ƒ Execution
ƒ Fetch

close cursor

Connor McDonald
April 2004 19
www.oracledba.co.uk
 OracleDBA

Processing SQL

co.uk
ƒ What we really wanted to do
ƒ Execute
ƒ (maybe) Fetch
ƒ A lot of preliminary overhead to do this
ƒ This is PARSING
ƒ If execution times are short, parsing might be the dominant factor for
ƒ CPU consumption
ƒ Response time
ƒ Latching problems
ƒ Lookup objects definitions, privileges, synonym translation etc

Connor McDonald
April 2004 20
www.oracledba.co.uk
 OracleDBA

More latching than you think

co.uk
SQL> select name, gets
2 from v$latch
3 where name like 'library%' or name like 'shared%';

NAME GETS
------------------------------ ----------
library cache 10619
library cache pin 6271
library cache pin allocation 3204
shared pool 14321

SQL> select 0 from dual;

0
----------
0

SQL> select name, gets

2 from v$latch
3 where name like 'library%' or name like 'shared%';

NAME GETS
------------------------------ ----------
library cache 10675 = 56
library cache pin 6288 = 17
library cache pin allocation 3216 = 12
shared pool 14363 = 42

Connor McDonald
April 2004 21
www.oracledba.co.uk
 OracleDBA

Impossible to avoid?

co.uk
ƒ If I need to run 11 SQL's
ƒ surely must parse 11 times ?

select surname, firstname from nominal where nom_id = 123

select * from vehicle where veh_id = 4567
select * from address where locality = 17
select pr_id from prp_property where reference = 'X123G'
select * from vehicle where veh_id = 23
select surname, firstname from nominal where nom_id = 123
select * from address where locality = 256
select * from organisation where officer = 'PD71986'
select surname, firstname from nominal where nom_id = 7632
select * from vehicle where veh_id = 4567
select sug_id from vw_ims_crime where cr_id = 24

Connor McDonald
April 2004 22
www.oracledba.co.uk
 OracleDBA

The Library Cache

co.uk
ƒ Memory structure
ƒ Records previously executed SQL
ƒ Optimizer information
ƒ Row source information
ƒ Reused with LRU algorithm
ƒ Exact pattern matching
ƒ Other checks still needed
ƒ Optimizer settings
ƒ Memory parameters
ƒ Synonym translation
ƒ etc

Connor McDonald
April 2004 23
www.oracledba.co.uk
 OracleDBA

The Library Cache

co.uk
Program Library Cache

select surname, firstname from nominal where nom_id = 123

select * from vehicle where veh_id = 4567

select * from address where locality = 17

select pr_id from prp_property where reference = 'X123G'
select * from vehicle where veh_id = 23
select surname, firstname from nominal where nom_id = 123
select * from address where locality = 256
select * from organisation where officer = 'PD71986'
select surname, firstname from nominal where nom_id = 7632
select * from vehicle where veh_id = 4567
select sug_id from vw_ims_crime where cr_id = 24

Two full parses avoided exposed as V$SQL

Connor McDonald
April 2004 24
www.oracledba.co.uk
 OracleDBA

"Nearly the same" SQL

co.uk
ƒ Likelihood of reuse appears quite low
ƒ Some queries by definition should be run the same
ƒ eg Primary key lookup
select surname, firstname from nominal where nom_id = 123

select surname, firstname from nominal where nom_id = 7632

ƒ Can we assign a "representative" value?

ƒ We use a place-holder, or BIND VARIABLE

Parse this…. Then …"bind and execute"

select surname, firstname
? = 123
from nominal
? = 7632
where nom_id = ?

Connor McDonald
April 2004 25
www.oracledba.co.uk
 OracleDBA

The Library Cache with binds

co.uk
Program Library Cache

select surname, firstname from nominal where nom_id = ?

select * from vehicle where veh_id = ?

select * from address where locality = ?

select pr_id from prp_property where reference = '?'
select * from vehicle where veh_id = ?
select surname, firstname from nominal where nom_id = ?
select * from address where locality = ?
select * from organisation where officer = '?'
select surname, firstname from nominal where nom_id = ?
select * from vehicle where veh_id = ?
select sug_id from vw_ims_crime where cr_id = ?

ƒFive full parses avoided

Connor McDonald
April 2004 26
www.oracledba.co.uk
 OracleDBA

Two identical SQL's ?

co.uk
ƒ Will two identical statements always be shared?
ƒ No!
ƒ Resolution differences
ƒ User1: select * from nominal;
ƒ User2: select * from nominal;
ƒ Bind variable differences
Number
ƒ User1: select * from nominal where nom_id = :v1
ƒ User2: select * from nominal where nom_id = :v1
varchar2(6)

ƒ User1: select * from nominal where nom_id = :v1 varchar2(4)

ƒ User2: select * from nominal where nom_id = :v1 varchar2(60)
ƒ V$SQL_SHARED_CURSOR
ƒ Always take care with data type definition in Oracle

Connor McDonald
April 2004 27
www.oracledba.co.uk
 OracleDBA

We're still parsing…

co.uk
Hard Soft
Parse Parse
ƒ Syntax check
ƒ Semantics check
ƒ Library cache check
ƒ Optimization
ƒ Row source generation
ƒ Execution
ƒ Fetch

Connor McDonald
April 2004 28
www.oracledba.co.uk
 OracleDBA

The cost of parsing…

co.uk
SQL> set timing on
SQL> declare
2 c number;
3 begin
4 c := dbms_sql.open_cursor;
5 for i in 1 .. 1000 loop Hard
6 dbms_sql.parse(c,'select * from vw_ims_crime
7 where add_id = '||i , dbms_sql.native); Parse
8 end loop;
9 dbms_sql.close_cursor(c);
10 end;
11 /
Elapsed: 00:01:31.00

SQL> declare
2 c number;
3 begin
4 c := dbms_sql.open_cursor;
5 for i in 1 .. 1000 loop
6 dbms_sql.parse(c,'select * from vw_ims_crime Soft
7 where add_id = :x' , dbms_sql.native);
8 dbms_sql.bind_variable(c,':x',i); Parse
9 end loop;
10 dbms_sql.close_cursor(c);
11 end;
12 /
Elapsed: 00:00:00.02

Connor McDonald
April 2004 29
www.oracledba.co.uk
 OracleDBA

Can we do better?

co.uk
ƒ A soft parse is good but
ƒ Still does syntax, semantic check = LATCHING
ƒ Idea?
ƒ When we close a cursor,
ƒ the next time we run it, the SQL may no longer be valid – must reparse
ƒ Why not when we process a cursor
ƒ Hold onto cursor handle ("pseudo-close")
ƒ Create a local map of all cursors used so far in in this session
ƒ Whenever a cursor reused, don't even parse, just re-run
ƒ Track dependencies from cursor to objects in library cache
ƒ Issues
ƒ How to keep track of every cursor we've ever run in session?
ƒ Might run out of memory?
ƒ How to be informed when dependencies change?

Connor McDonald
April 2004 30
www.oracledba.co.uk
 OracleDBA

Theoretical solution

co.uk
Process Library Cache

cursor1 object1

cursor2 object2
Program
cursor3
object3
cursor4
object4
cursor5
object5

Connor McDonald
April 2004 31
www.oracledba.co.uk
 OracleDBA

Sounds hard…but

co.uk
ƒ It comes for free!!!
begin
select *
into …
from vw_ims_crime
where add_id = ln_my_plsql_variable
end;

ƒ All static SQL in PL/SQL

ƒ automatically uses bind variables (ln_my_plsql_variable)
ƒ automatically holds all cursors open (even if you close them)
ƒ is automatically optimal processing of SQL (with respect to parsing)
ƒ does not automatically write good SQL for you /

Connor McDonald
April 2004 32
www.oracledba.co.uk
 OracleDBA

PL/SQL cursor caching

co.uk
Session 1 Session 2
declare begin
cursor c_emp is select *
select * from emp; into …
begin from nominal
open c_emp; where …
fetch c_emp end;
into …;
close c_emp;
end;

select * from emp

select * from nominal where …

open_cursors = 1000
(database setting)

Connor McDonald
April 2004 33
www.oracledba.co.uk
 OracleDBA

Risks with dynamic SQL

co.uk
ƒ In its simplest form (concatenation)
ƒ High probability of HARD PARSE every time
execute immediate 'select * from nominal
where nom_id = '||my_var;

ƒ Used with bind variables

ƒ SOFT PARSE every time
ƒ No cursor caching ever with dynamic SQL

execute immediate 'select * from nominal

where nom_id = :v1' using my_var;

ƒ Is dynamic SQL bad?

ƒ No, but inappropriate use can cripple a system

Connor McDonald
April 2004 34
www.oracledba.co.uk
 OracleDBA

So always use bind variables?

co.uk
ƒ No!
ƒ Recall the motivation for bind variables
ƒ Encourage sharing of nearly the same cursors in library cache
ƒ The aim is to reduce parsing overheads
ƒ If a query takes
ƒ 0.5 seconds to parse
ƒ 200 seconds to run
then parsing (and hence bind variables) are not the issue

Connor McDonald
April 2004 35
www.oracledba.co.uk
 OracleDBA

Dynamic SQL Examples

co.uk
lc_sql := 'select nom_date_from
from nominal
where nom_id = ' || ln_nom_id

ƒ Yes should change to bind variables

ƒ All access by NOM_ID is "identical"
ƒ Parse overhead a large factor of execution time

Connor McDonald
April 2004 36
www.oracledba.co.uk
 OracleDBA

Dynamic SQL Examples

co.uk
lc_sql := 'select nom_date_from
from nominal
where surname = '|| lc_search_name

ƒ No – do not change to bind variables

ƒ Access by SURNAME is not identical
ƒ "SMITH" far more costly than "PZANSKI"

Connor McDonald
April 2004 37
www.oracledba.co.uk
 OracleDBA

Dynamic SQL Examples

co.uk
lc_sql := 'select nom_date_from
from nominal
where current_ind = ''Y''
and nom_id = ' || ln_nom_id

ƒ Yes should change to bind variables

ƒ All access by NOM_ID is "identical"
ƒ Parse overhead a large factor of execution time
ƒ "Y" for CURRENT_IND does not need changing
ƒ Only two distinct values
ƒ Minimal sharing benefit anyway

Connor McDonald
April 2004 38
www.oracledba.co.uk
 OracleDBA

SQL Examples

co.uk
select …
from gen_links g, ims_incident i
where g.rc_mast_table_id = 41156
and i.mast_id = i.inc_id

ƒ No changes required
ƒ 41156 is a constant
ƒ never changes for this query
ƒ does not impact capacity for sharing
ƒ RC_MAST_TABLE_ID
ƒ skewed distribution
ƒ different access paths might be appropriate

Connor McDonald
April 2004 39
www.oracledba.co.uk
 OracleDBA

SQL Examples

co.uk
lc_sql := 'select '||lc_snap_date_col
'from '||lc_snap_tab_name
'where '||lc_snap_col||' = :n';

ƒ Was dynamic really needed?

ƒ Only 5 snapshot tables in IMS
ƒ Why not convert to 5 static SQL's?
ƒ Eliminates soft parse
ƒ Consider "Dynamic at build time, static at run time"

select nom_date_from select add_date_from

from nominal from address etc
where nom_id = my_plsql_var where add_id = my_plsql_var

Connor McDonald
April 2004 40
www.oracledba.co.uk
 OracleDBA

Summary

co.uk
ƒ Use static SQL
ƒ Avoid dynamic SQL unless absolutely necessary
ƒ "Dynamically build static SQL"
ƒ For high use, fast execution SQL
ƒ Must use bind variables
ƒ Take advantage of PL/SQL cursor caching
ƒ (no SQL in triggers)
ƒ For long running, low frequency SQL
ƒ Bind variables not required
ƒ Give optimizer maximum information
ƒ Examples: searches, ad-hoc query, data warehouses etc

Connor McDonald
April 2004 41
www.oracledba.co.uk