26/08/13

Performing raw SQL queries | Django documentation | Django

PerformingrawSQLqueries
WhenthemodelqueryAPIsdontgofarenough,youcanfallbacktowritingrawSQL.Djangogivesyoutwo waysofperformingrawSQLqueries:youcanuseM a n a g e r . r a w ( ) toperformrawqueriesandreturn modelinstances,oryoucanavoidthemodellayerentirelyandexecutecustomSQLdirectly.

Performingrawqueries
Ther a w ( ) managermethodcanbeusedtoperformrawSQLqueriesthatreturnmodelinstances:

M a n a g e r . r a w (raw_query,params=None,translations=None)
ThismethodmethodtakesarawSQLquery,executesit,andreturnsa d j a n g o . d b . m o d e l s . q u e r y . R a w Q u e r y S e t instance.ThisR a w Q u e r y S e t instancecanbeiteratedover justlikeannormalQuerySettoprovideobjectinstances. Thisisbestillustratedwithanexample.Supposeyouvegotthefollowingmodel:
c l a s sP e r s o n ( m o d e l s . M o d e l ) : f i r s t _ n a m e=m o d e l s . C h a r F i e l d ( . . . ) l a s t _ n a m e=m o d e l s . C h a r F i e l d ( . . . ) b i r t h _ d a t e=m o d e l s . D a t e F i e l d ( . . . )

YoucouldthenexecutecustomSQLlikeso: > > >f o rpi nP e r s o n . o b j e c t s . r a w ( ' S E L E C T*F R O Mm y a p p _ p e r s o n ' ) : . . . p r i n t ( p ) J o h nS m i t h J a n eJ o n e s Ofcourse,thisexampleisntveryexcitingitsexactlythesameasrunningP e r s o n . o b j e c t s . a l l ( ) . However,r a w ( ) hasabunchofotheroptionsthatmakeitverypowerful. Modeltablenames WheredthenameoftheP e r s o n tablecomefrominthatexample? Bydefault,Djangofiguresoutadatabasetablenamebyjoiningthemodelsapplabelthe nameyouusedinm a n a g e . p ys t a r t a p p tothemodelsclassname,withanunderscore betweenthem.IntheexampleweveassumedthattheP e r s o n modellivesinanappnamed m y a p p ,soitstablewouldbem y a p p _ p e r s o n . Formoredetailscheckoutthedocumentationforthed b _ t a b l e option,whichalsoletsyou manuallysetthedatabasetablename.

Warning NocheckingisdoneontheSQLstatementthatispassedinto. r a w ( ) .Djangoexpectsthat thestatementwillreturnasetofrowsfromthedatabase,butdoesnothingtoenforcethat.If thequerydoesnotreturnrows,a(possiblycryptic)errorwillresult.

Mappingqueryfieldstomodelfields
r a w ( ) automaticallymapsfieldsinthequerytofieldsonthemodel. Theorderoffieldsinyourquerydoesntmatter.Inotherwords,bothofthefollowingqueriesworkidentically: > > >P e r s o n . o b j e c t s . r a w ( ' S E L E C Ti d ,f i r s t _ n a m e ,l a s t _ n a m e ,b i r t h _ d a t eF R O Mm y a p p _ p e r s o n ' )

https://docs.djangoproject.com/en/dev/topics/db/sql/

1/5

26/08/13

Performing raw SQL queries | Django documentation | Django

. . . > > >P e r s o n . o b j e c t s . r a w ( ' S E L E C Tl a s t _ n a m e ,b i r t h _ d a t e ,f i r s t _ n a m e ,i dF R O Mm y a p p _ p e r s o n ' ) . . .

Matchingisdonebyname.ThismeansthatyoucanuseSQLsA S clausestomapfieldsinthequeryto modelfields.SoifyouhadsomeothertablethathadP e r s o n datainit,youcouldeasilymapitinto P e r s o n instances: > > >P e r s o n . o b j e c t s . r a w ( ' ' ' S E L E C Tf i r s tA Sf i r s t _ n a m e , . . . l a s tA Sl a s t _ n a m e , . . . b dA Sb i r t h _ d a t e , . . . p ka si d , . . . F R O Ms o m e _ o t h e r _ t a b l e ' ' ' ) Aslongasthenamesmatch,themodelinstanceswillbecreatedcorrectly. Alternatively,youcanmapfieldsinthequerytomodelfieldsusingthet r a n s l a t i o n s argumenttor a w ( ) . Thisisadictionarymappingnamesoffieldsinthequerytonamesoffieldsonthemodel.Forexample,the abovequerycouldalsobewritten: > > >n a m e _ m a p={ ' f i r s t ' :' f i r s t _ n a m e ' ,' l a s t ' :' l a s t _ n a m e ' ,' b d ' :' b i r t h _ d a t e ' ,' p k ' :' i d ' > > >P e r s o n . o b j e c t s . r a w ( ' S E L E C T*F R O Ms o m e _ o t h e r _ t a b l e ' ,t r a n s l a t i o n s = n a m e _ m a p )

Indexlookups
r a w ( ) supportsindexing,soifyouneedonlythefirstresultyoucanwrite: > > >f i r s t _ p e r s o n=P e r s o n . o b j e c t s . r a w ( ' S E L E C T*f r o mm y a p p _ p e r s o n ' ) [ 0 ] However,theindexingandslicingarenotperformedatthedatabaselevel.Ifyouhaveabigamountof P e r s o n objectsinyourdatabase,itismoreefficienttolimitthequeryattheSQLlevel: > > >f i r s t _ p e r s o n=P e r s o n . o b j e c t s . r a w ( ' S E L E C T*f r o mm y a p p _ p e r s o nL I M I T1 ' ) [ 0 ]

Deferringmodelfields
Fieldsmayalsobeleftout: > > >p e o p l e=P e r s o n . o b j e c t s . r a w ( ' S E L E C Ti d ,f i r s t _ n a m eF R O Mm y a p p _ p e r s o n ' ) TheP e r s o n objectsreturnedbythisquerywillbedeferredmodelinstances(seed e f e r ( ) ).Thismeans thatthefieldsthatareomittedfromthequerywillbeloadedondemand.Forexample: > > >f o rpi nP e r s o n . o b j e c t s . r a w ( ' S E L E C Ti d ,f i r s t _ n a m eF R O Mm y a p p _ p e r s o n ' ) : . . . p r i n t ( p . f i r s t _ n a m e ,#T h i sw i l lb er e t r i e v e db yt h eo r i g i n a lq u e r y . . . p . l a s t _ n a m e )#T h i sw i l lb er e t r i e v e do nd e m a n d . . . J o h nS m i t h J a n eJ o n e s Fromoutwardappearances,thislookslikethequeryhasretrievedboththefirstnameandlastname. However,thisexampleactuallyissued3queries.Onlythefirstnameswereretrievedbytheraw()query thelastnameswerebothretrievedondemandwhentheywereprinted.

ThisdocumentisforDjango'sdevelopmentversion,whichcanbesignificantly differentfrompreviousreleases.Forolderreleases,usetheversionselectorfloating Thereisonlyonefieldthatyoucantleaveouttheprimarykeyfield.Djangousestheprimarykeyto inthebottomrightcornerofthispage. identifymodelinstances,soitmustalwaysbeincludedinarawquery.AnI n v a l i d Q u e r y exceptionwill

beraisedifyouforgettoincludetheprimarykey.

https://docs.djangoproject.com/en/dev/topics/db/sql/

2/5

26/08/13

Performing raw SQL queries | Django documentation | Django

Addingannotations
Youcanalsoexecutequeriescontainingfieldsthatarentdefinedonthemodel.Forexample,wecoulduse PostgreSQLsage()functiontogetalistofpeoplewiththeiragescalculatedbythedatabase: > > >p e o p l e=P e r s o n . o b j e c t s . r a w ( ' S E L E C T* ,a g e ( b i r t h _ d a t e )A Sa g eF R O Mm y a p p _ p e r s o n ' ) > > >f o rpi np e o p l e : . . . p r i n t ( " % si s% s . "%( p . f i r s t _ n a m e ,p . a g e ) ) J o h ni s3 7 . J a n ei s4 2 . . . .

Passingparametersintor a w ( )
Ifyouneedtoperformparameterizedqueries,youcanusethep a r a m s argumenttor a w ( ) : > > >l n a m e=' D o e ' > > >P e r s o n . o b j e c t s . r a w ( ' S E L E C T*F R O Mm y a p p _ p e r s o nW H E R El a s t _ n a m e=% s ' ,[ l n a m e ] )

p a r a m s isalistordictionaryofparameters.Youlluse% s placeholdersinthequerystringforalist,or % ( k e y ) s placeholdersforadictionary(wherek e y isreplacedbyadictionarykey,ofcourse),regardlessof yourdatabaseengine.Suchplaceholderswillbereplacedwithparametersfromthep a r a m s argument. Note DictionaryparamsnotsupportedwithSQLite DictionaryparamsarenotsupportedwiththeSQLitebackendwiththisbackend,youmust passparametersasalist.

Warning Donotusestringformattingonrawqueries! Itstemptingtowritetheabovequeryas: > > >q u e r y=' S E L E C T*F R O Mm y a p p _ p e r s o nW H E R El a s t _ n a m e=% s '%l n a m e > > >P e r s o n . o b j e c t s . r a w ( q u e r y )

Dont. Usingthep a r a m s argumentcompletelyprotectsyoufromSQLinjectionattacks,acommon exploitwhereattackersinjectarbitrarySQLintoyourdatabase.Ifyouusestringinterpolation, soonerorlateryoullfallvictimtoSQLinjection.Aslongasyouremembertoalwaysusethe p a r a m s argumentyoullbeprotected.

ChangedinDjango1.6: InDjango1.5andearlier,youcouldpassparametersasdictionarieswhenusingPostgreSQLorMySQL,although thiswasntdocumented.NowyoucanalsodothiswhemusingOracle,anditisofficiallysupported.

ExecutingcustomSQLdirectly
SometimesevenM a n a g e r . r a w ( ) isntquiteenough:youmightneedtoperformqueriesthatdontmap cleanlytomodels,ordirectlyexecuteU P D A T E ,I N S E R T ,orD E L E T E queries. Inthesecases,youcanalwaysaccessthedatabasedirectly,routingaroundthemodellayerentirely.

https://docs.djangoproject.com/en/dev/topics/db/sql/

3/5

26/08/13

Performing raw SQL queries | Django documentation | Django

Theobjectd j a n g o . d b . c o n n e c t i o n representsthedefaultdatabaseconnection.Tousethedatabase connection,callc o n n e c t i o n . c u r s o r ( ) togetacursorobject.Then,call c u r s o r . e x e c u t e ( s q l ,[ p a r a m s ] ) toexecutetheSQLandc u r s o r . f e t c h o n e ( ) or c u r s o r . f e t c h a l l ( ) toreturntheresultingrows. Forexample: f r o md j a n g o . d bi m p o r tc o n n e c t i o n

d e fm y _ c u s t o m _ s q l ( s e l f ) : c u r s o r=c o n n e c t i o n . c u r s o r ( )

c u r s o r . e x e c u t e ( " U P D A T Eb a rS E Tf o o=1W H E R Eb a z=% s " ,[ s e l f . b a z ] ) c u r s o r . e x e c u t e ( " S E L E C Tf o oF R O Mb a rW H E R Eb a z=% s " ,[ s e l f . b a z ] ) r o w=c u r s o r . f e t c h o n e ( )

r e t u r nr o w
ChangedinDjango1.6: InDjango1.5andearlier,afterperformingadatachangingoperation,youhadtocall t r a n s a c t i o n . c o m m i t _ u n l e s s _ m a n a g e d ( ) toensureyourchangeswerecommittedtothedatabase.Since Djangonowdefaultstodatabaselevelautocommit,thisisntnecessaryanylonger.

Notethatifyouwanttoincludeliteralpercentsignsinthequery,youhavetodoubletheminthecaseyou arepassingparameters: c u r s o r . e x e c u t e ( " S E L E C Tf o oF R O Mb a rW H E R Eb a z=' 3 0 % ' " ) c u r s o r . e x e c u t e ( " S E L E C Tf o oF R O Mb a rW H E R Eb a z=' 3 0 % % 'a n di d=% s " ,[ s e l f . i d ] )

Ifyouareusingmorethanonedatabase,youcanused j a n g o . d b . c o n n e c t i o n s toobtaintheconnection (andcursor)foraspecificdatabase.d j a n g o . d b . c o n n e c t i o n s isadictionarylikeobjectthatallowsyou toretrieveaspecificconnectionusingitsalias: f r o md j a n g o . d bi m p o r tc o n n e c t i o n s c u r s o r=c o n n e c t i o n s [ ' m y _ d b _ a l i a s ' ] . c u r s o r ( ) #Y o u rc o d eh e r e . . . Bydefault,thePythonDBAPIwillreturnresultswithouttheirfieldnames,whichmeansyouendupwitha l i s t ofvalues,ratherthanad i c t .Atasmallperformancecost,youcanreturnresultsasad i c t byusing somethinglikethis:
d e fd i c t f e t c h a l l ( c u r s o r ) : " R e t u r n sa l lr o w sf r o mac u r s o ra sad i c t " d e s c=c u r s o r . d e s c r i p t i o n r e t u r n[ d i c t ( z i p ( [ c o l [ 0 ]f o rc o li nd e s c ] ,r o w ) ) f o rr o wi nc u r s o r . f e t c h a l l ( ) ]

Hereisanexampleofthedifferencebetweenthetwo: > > >c u r s o r . e x e c u t e ( " S E L E C Ti d ,p a r e n t _ i df r o mt e s tL I M I T2 " ) ; > > >c u r s o r . f e t c h a l l ( ) ( ( 5 4 3 6 0 9 8 2 L ,N o n e ) ,( 5 4 3 6 0 8 8 0 L ,N o n e ) ) > > >c u r s o r . e x e c u t e ( " S E L E C Ti d ,p a r e n t _ i df r o mt e s tL I M I T2 " ) ; > > >d i c t f e t c h a l l ( c u r s o r ) [ { ' p a r e n t _ i d ' :N o n e ,' i d ' :5 4 3 6 0 9 8 2 L } ,{ ' p a r e n t _ i d ' :N o n e ,' i d ' :5 4 3 6 0 8 8 0 L } ]

https://docs.djangoproject.com/en/dev/topics/db/sql/

4/5

26/08/13

Performing raw SQL queries | Django documentation | Django

Connectionsandcursors
c o n n e c t i o n andc u r s o r mostlyimplementthestandardPythonDBAPIdescribedinPEP249except whenitcomestotransactionhandling. IfyourenotfamiliarwiththePythonDBAPI,notethattheSQLstatementinc u r s o r . e x e c u t e ( ) uses placeholders," % s " ,ratherthanaddingparametersdirectlywithintheSQL.Ifyouusethistechnique,the underlyingdatabaselibrarywillautomaticallyescapeyourparametersasnecessary. AlsonotethatDjangoexpectsthe" % s " placeholder,notthe" ? " placeholder,whichisusedbytheSQLite Pythonbindings.Thisisforthesakeofconsistencyandsanity.

Managers

Databasetransactions

20052013DjangoSoftwareFoundationunlessotherwisenoted.DjangoisaregisteredtrademarkoftheDjangoSoftware Foundation.LinuxWebhostinggraciouslyprovidedbyMediaTemple.

https://docs.djangoproject.com/en/dev/topics/db/sql/

5/5