Professional Documents
Culture Documents
Oracle CHARACTERSET LOCALE PDF
Oracle CHARACTERSET LOCALE PDF
December 2005
Oracle Database Globalization Support Guide, 10g Release 2 (10.2)
B14225-02
Contributors: Dan Chiba, Winson Chu, Claire Ho, Gary Hua, Simon Law, Geoff Lee, Peter Linsley,
Qianrong Ma, Keni Matsuda, Meghna Mehta, Valarie Moore, Shige Takeda, Linus Tanaka, Makoto Tozawa,
Barry Trute, Ying Wu, Peter Wallack, Chao Wang, Huaqing Wang, Simon Wong, Michael Yau, Jianping Yang,
Qin Yu, Tim Yu, Weiran Zhang, Yan Zhu
The Programs (which include both the software and documentation) contain proprietary information; they
are provided under a license agreement containing restrictions on use and disclosure and are also protected
by copyright, patent, and other intellectual and industrial property laws. Reverse engineering, disassembly,
or decompilation of the Programs, except to the extent required to obtain interoperability with other
independently created software or as specified by law, is prohibited.
The information contained in this document is subject to change without notice. If you find any problems in
the documentation, please report them to us in writing. This document is not warranted to be error-free.
Except as may be expressly permitted in your license agreement for these Programs, no part of these
Programs may be reproduced or transmitted in any form or by any means, electronic or mechanical, for any
purpose.
If the Programs are delivered to the United States Government or anyone licensing or using the Programs on
behalf of the United States Government, the following notice is applicable:
U.S. GOVERNMENT RIGHTS Programs, software, databases, and related documentation and technical data
delivered to U.S. Government customers are "commercial computer software" or "commercial technical data"
pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As
such, use, duplication, disclosure, modification, and adaptation of the Programs, including documentation
and technical data, shall be subject to the licensing restrictions set forth in the applicable Oracle license
agreement, and, to the extent applicable, the additional rights set forth in FAR 52.227-19, Commercial
Computer Software—Restricted Rights (June 1987). Oracle Corporation, 500 Oracle Parkway, Redwood City,
CA 94065
The Programs are not intended for use in any nuclear, aviation, mass transit, medical, or other inherently
dangerous applications. It shall be the licensee's responsibility to take all appropriate fail-safe, backup,
redundancy and other measures to ensure the safe use of such applications if the Programs are used for such
purposes, and we disclaim liability for any damages caused by such use of the Programs.
Oracle, JD Edwards, PeopleSoft, and Retek are registered trademarks of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective owners.
The Programs may provide links to Web sites and access to content, products, and services from third
parties. Oracle is not responsible for the availability of, or any content provided on, third-party Web sites.
You bear all risks associated with the use of such content. If you choose to purchase any products or services
from a third party, the relationship is directly between you and the third party. Oracle is not responsible for:
(a) the quality of third-party products or services; or (b) fulfilling any of the terms of the agreement with the
third party, including delivery of products or services and warranty obligations related to purchased
products or services. Oracle is not responsible for any loss or damage of any sort that you may incur from
dealing with any third party.
Contents
Preface ............................................................................................................................................................... xv
Intended Audience.................................................................................................................................... xv
Documentation Accessibility ................................................................................................................... xv
Structure ..................................................................................................................................................... xvi
Related Documents .................................................................................................................................. xvii
Conventions .............................................................................................................................................. xvii
iii
Writing Direction ........................................................................................................................ 2-3
What Characters Does a Character Set Support? .......................................................................... 2-3
ASCII Encoding........................................................................................................................... 2-4
How are Characters Encoded? ......................................................................................................... 2-6
Single-Byte Encoding Schemes ................................................................................................. 2-7
Multibyte Encoding Schemes.................................................................................................... 2-7
Naming Convention for Oracle Character Sets ............................................................................. 2-8
Length Semantics ..................................................................................................................................... 2-8
Choosing an Oracle Database Character Set.................................................................................... 2-10
Current and Future Language Requirements............................................................................. 2-11
Client Operating System and Application Compatibility......................................................... 2-11
Character Set Conversion Between Clients and the Server ...................................................... 2-12
Performance Implications of Choosing a Database Character Set........................................... 2-12
Restrictions on Database Character Sets...................................................................................... 2-12
Restrictions on Character Sets Used to Express Names ..................................................... 2-13
Database Character Set Statement of Direction .......................................................................... 2-13
Choosing Unicode as a Database Character Set ......................................................................... 2-13
Choosing a National Character Set............................................................................................... 2-14
Summary of Supported Datatypes ............................................................................................... 2-14
Changing the Character Set After Database Creation.................................................................... 2-15
Monolingual Database Scenario ........................................................................................................ 2-15
Character Set Conversion in a Monolingual Scenario ............................................................... 2-16
Multilingual Database Scenarios....................................................................................................... 2-17
Restricted Multilingual Support ................................................................................................... 2-17
Unrestricted Multilingual Support .............................................................................................. 2-18
iv
NLS_TIMESTAMP_FORMAT ............................................................................................... 3-18
NLS_TIMESTAMP_TZ_FORMAT ........................................................................................ 3-19
Calendar Definitions ............................................................................................................................ 3-19
Calendar Formats ............................................................................................................................ 3-20
First Day of the Week .............................................................................................................. 3-20
First Calendar Week of the Year ............................................................................................ 3-20
Number of Days and Months in a Year ................................................................................ 3-21
First Year of Era........................................................................................................................ 3-21
NLS_CALENDAR........................................................................................................................... 3-22
Numeric and List Parameters.............................................................................................................. 3-22
Numeric Formats ............................................................................................................................ 3-23
NLS_NUMERIC_CHARACTERS................................................................................................. 3-23
NLS_LIST_SEPARATOR ............................................................................................................... 3-24
Monetary Parameters............................................................................................................................ 3-24
Currency Formats ........................................................................................................................... 3-25
NLS_CURRENCY ........................................................................................................................... 3-25
NLS_ISO_CURRENCY................................................................................................................... 3-26
NLS_DUAL_CURRENCY ............................................................................................................. 3-27
Oracle Support for the Euro .......................................................................................................... 3-27
NLS_MONETARY_CHARACTERS............................................................................................. 3-28
NLS_CREDIT ................................................................................................................................... 3-28
NLS_DEBIT ...................................................................................................................................... 3-29
Linguistic Sort Parameters................................................................................................................... 3-29
NLS_SORT ....................................................................................................................................... 3-29
NLS_COMP...................................................................................................................................... 3-30
Character Set Conversion Parameter ................................................................................................. 3-31
NLS_NCHAR_CONV_EXCP ........................................................................................................ 3-31
Length Semantics .................................................................................................................................. 3-31
NLS_LENGTH_SEMANTICS ....................................................................................................... 3-31
v
Explicit Conversion of Datetime Datatypes ................................................................................ 4-11
Datetime SQL Functions...................................................................................................................... 4-12
Datetime and Time Zone Parameters and Environment Variables ............................................. 4-13
Datetime Format Parameters......................................................................................................... 4-13
Time Zone Environment Variables............................................................................................... 4-14
Daylight Saving Time Session Parameter.................................................................................... 4-14
Choosing a Time Zone File.................................................................................................................. 4-15
Upgrading the Time Zone File............................................................................................................ 4-17
Setting the Database Time Zone ........................................................................................................ 4-18
Setting the Session Time Zone ........................................................................................................... 4-19
Converting Time Zones With the AT TIME ZONE Clause........................................................... 4-20
Support for Daylight Saving Time .................................................................................................... 4-21
Examples: The Effect of Daylight Saving Time on Datetime Calculations............................. 4-21
vi
Specify NOT NULL in a WHERE Clause If the Column Was Not Declared NOT NULL.......
5-18
Example: Setting Up a French Linguistic Index .................................................................. 5-19
Searching Linguistic Strings ............................................................................................................... 5-19
SQL Regular Expressions in a Multilingual Environment ........................................................... 5-19
Character Range '[x-y]' in Regular Expressions.......................................................................... 5-20
Collation Element Delimiter '[. .]' in Regular Expressions ........................................................ 5-20
Character Class '[: :]' in Regular Expressions .............................................................................. 5-21
Equivalence Class '[= =]' in Regular Expressions....................................................................... 5-21
Examples: Regular Expressions .................................................................................................... 5-21
vii
The NVARCHAR2 Datatype..................................................................................................... 7-4
The NCLOB Datatype ................................................................................................................ 7-5
Implicit Datatype Conversion Between NCHAR and Other Datatypes.................................... 7-5
Exception Handling for Data Loss During Datatype Conversion .............................................. 7-5
Rules for Implicit Datatype Conversion ......................................................................................... 7-6
SQL Functions for Unicode Datatypes............................................................................................ 7-7
Other SQL Functions ......................................................................................................................... 7-8
Unicode String Literals...................................................................................................................... 7-8
NCHAR String Literal Replacement ............................................................................................... 7-9
Using the UTL_FILE Package with NCHAR Data..................................................................... 7-10
OCI Programming with Unicode ....................................................................................................... 7-10
OCIEnvNlsCreate() Function for Unicode Programming ........................................................ 7-10
OCI Unicode Code Conversion..................................................................................................... 7-12
Data Integrity............................................................................................................................ 7-12
OCI Performance Implications When Using Unicode........................................................ 7-12
OCI Unicode Data Expansion ................................................................................................ 7-13
Setting UTF-8 to the NLS_LANG Character Set in OCI ............................................................ 7-14
Binding and Defining SQL CHAR Datatypes in OCI................................................................ 7-14
Binding and Defining SQL NCHAR Datatypes in OCI............................................................. 7-15
Handling SQL NCHAR String Literals in OCI ........................................................................... 7-16
Binding and Defining CLOB and NCLOB Unicode Data in OCI ............................................ 7-17
Pro*C/C++ Programming with Unicode ........................................................................................... 7-17
Pro*C/C++ Data Conversion in Unicode.................................................................................... 7-18
Using the VARCHAR Datatype in Pro*C/C++.......................................................................... 7-18
Using the NVARCHAR Datatype in Pro*C/C++ ...................................................................... 7-19
Using the UVARCHAR Datatype in Pro*C/C++ ...................................................................... 7-19
JDBC Programming with Unicode..................................................................................................... 7-20
Binding and Defining Java Strings to SQL CHAR Datatypes .................................................. 7-20
Binding and Defining Java Strings to SQL NCHAR Datatypes ............................................... 7-21
Using the SQL NCHAR Datatypes Without Changing the Code............................................ 7-22
Using SQL NCHAR String Literals in JDBC ............................................................................... 7-22
Data Conversion in JDBC............................................................................................................... 7-23
Data Conversion for the OCI Driver ..................................................................................... 7-23
Data Conversion for Thin Drivers ......................................................................................... 7-23
Data Conversion for the Server-Side Internal Driver ......................................................... 7-24
Using oracle.sql.CHAR in Oracle Object Types ......................................................................... 7-24
oracle.sql.CHAR....................................................................................................................... 7-24
Accessing SQL CHAR and NCHAR Attributes with oracle.sql.CHAR .......................... 7-26
Restrictions on Accessing SQL CHAR Data with JDBC............................................................ 7-26
Character Integrity Issues in a Multibyte Database Environment ................................... 7-26
ODBC and OLE DB Programming with Unicode........................................................................... 7-27
Unicode-Enabled Drivers in ODBC and OLE DB ...................................................................... 7-27
OCI Dependency in Unicode......................................................................................................... 7-28
ODBC and OLE DB Code Conversion in Unicode..................................................................... 7-28
OLE DB Code Conversions .................................................................................................... 7-29
ODBC Unicode Datatypes ............................................................................................................. 7-29
OLE DB Unicode Datatypes .......................................................................................................... 7-30
viii
ADO Access ..................................................................................................................................... 7-30
XML Programming with Unicode...................................................................................................... 7-31
Writing an XML File in Unicode with Java ................................................................................. 7-31
Reading an XML File in Unicode with Java ................................................................................ 7-32
Parsing an XML Stream in Unicode with Java ........................................................................... 7-32
ix
oracle.i18n.lcsd ................................................................................................................................ 8-41
oracle.i18n.net .................................................................................................................................. 8-41
oracle.i18n.servlet............................................................................................................................ 8-41
oracle.i18n.text ................................................................................................................................. 8-42
oracle.i18n.util.................................................................................................................................. 8-42
GDK for PL/SQL Supplied Packages ................................................................................................ 8-42
GDK Error Messages ............................................................................................................................ 8-43
x
Additional Problems Caused by Data Truncation.............................................................. 11-2
Character Set Conversion Issues................................................................................................... 11-3
Replacement Characters that Result from Using the Export and Import Utilities......... 11-3
Invalid Data That Results from Setting the Client's NLS_LANG Parameter Incorrectly ........
11-4
Changing the Database Character Set of an Existing Database................................................... 11-5
Migrating Character Data Using a Full Export and Import...................................................... 11-6
Migrating a Character Set Using the CSALTER Script.............................................................. 11-6
Using the CSALTER Script in an Oracle Real Application Clusters Environment ........ 11-7
Migrating Character Data Using the CSALTER Script and Selective Imports ...................... 11-7
Migrating to NCHAR Datatypes........................................................................................................ 11-8
Migrating Version 8 NCHAR Columns to Oracle9i and Later................................................. 11-8
Changing the National Character Set........................................................................................... 11-9
Migrating CHAR Columns to NCHAR Columns ...................................................................... 11-9
Using the ALTER TABLE MODIFY Statement to Change CHAR Columns to NCHAR
Columns 11-9
Using Online Table Redefinition to Migrate a Large Table to Unicode ........................ 11-10
Tasks to Recover Database Schema After Character Set Migration .......................................... 11-11
xi
Example: Command-Line Method ...................................................................................... 12-18
Database Character Set Scanner Messages......................................................................... 12-19
Single Table Scan: Examples........................................................................................................ 12-19
Example: Parameter-File Method ........................................................................................ 12-19
Example: Command-Line Method ...................................................................................... 12-19
Database Character Set Scanner Messages......................................................................... 12-19
Example: Parameter-File Method ........................................................................................ 12-20
Example: Command-Line Method ...................................................................................... 12-20
Database Character Set Scanner Messages......................................................................... 12-20
Column Scan: Examples............................................................................................................... 12-20
Example: Parameter-File Method ........................................................................................ 12-21
Example: Command-Line Method ...................................................................................... 12-21
Database Character Set Scanner Messages......................................................................... 12-21
Database Character Set Scanner Reports........................................................................................ 12-21
Database Scan Summary Report................................................................................................. 12-21
Database Size .......................................................................................................................... 12-22
Database Scan Parameters .................................................................................................... 12-22
Scan Summary ........................................................................................................................ 12-23
Data Dictionary Conversion Summary .............................................................................. 12-24
Application Data Conversion Summary ............................................................................ 12-25
Application Data Conversion Summary Per Column Size Boundary ........................... 12-25
Distribution of Convertible Data Per Table ....................................................................... 12-25
Distribution of Convertible Data Per Column................................................................... 12-26
Indexes To Be Rebuilt ............................................................................................................ 12-26
Truncation Due To Character Semantics............................................................................ 12-26
Character Set Detection Result............................................................................................. 12-27
Language Detection Result................................................................................................... 12-27
Database Scan Individual Exception Report............................................................................. 12-27
Database Scan Parameters .................................................................................................... 12-27
Data Dictionary Individual Exceptions .............................................................................. 12-28
Application Data Individual Exceptions ............................................................................ 12-28
How to Handle Convertible or Lossy Data in the Data Dictionary .......................................... 12-29
Storage and Performance Considerations in the Database Character Set Scanner................ 12-31
Storage Considerations for the Database Character Set Scanner ........................................... 12-31
CSM$TABLES......................................................................................................................... 12-31
CSM$COLUMNS ................................................................................................................... 12-31
CSM$ERRORS ........................................................................................................................ 12-32
Performance Considerations for the Database Character Set Scanner.................................. 12-32
Using Multiple Scan Processes............................................................................................. 12-32
Setting the Array Fetch Buffer Size ..................................................................................... 12-32
Optimizing the QUERY Clause ........................................................................................... 12-32
Suppressing Exception and Convertible Log .................................................................... 12-32
Recommendations and Restrictions for the Database Character Set Scanner...................... 12-33
Scanning Database Containing Data Not in the Database Character Set ...................... 12-33
Scanning Database Containing Data from Two or More Character Sets....................... 12-33
Database Character Set Scanner CSALTER Script........................................................................ 12-33
Checking Phase of the CSALTER Script ................................................................................... 12-34
xii
Updating Phase of the CSALTER Script.................................................................................... 12-35
Database Character Set Scanner Views........................................................................................... 12-35
CSMV$COLUMNS ....................................................................................................................... 12-36
CSMV$CONSTRAINTS ............................................................................................................... 12-36
CSMV$ERRORS ............................................................................................................................ 12-37
CSMV$INDEXES........................................................................................................................... 12-37
CSMV$TABLES ............................................................................................................................. 12-37
Database Character Set Scanner Error Messages .......................................................................... 12-38
13 Customizing Locale
Overview of the Oracle Locale Builder Utility................................................................................ 13-1
Configuring Unicode Fonts for the Oracle Locale Builder ....................................................... 13-1
Font Configuration on Windows........................................................................................... 13-2
Font Configuration on Other Platforms ............................................................................... 13-2
The Oracle Locale Builder User Interface .................................................................................... 13-2
Oracle Locale Builder Windows and Dialog Boxes ................................................................... 13-3
Existing Definitions Dialog Box............................................................................................. 13-3
Session Log Dialog Box ........................................................................................................... 13-4
Preview NLT Tab Page ........................................................................................................... 13-4
Open File Dialog Box............................................................................................................... 13-5
Creating a New Language Definition with the Oracle Locale Builder....................................... 13-6
Creating a New Territory Definition with the Oracle Locale Builder ........................................ 13-9
Customizing Time Zone Data ..................................................................................................... 13-15
Customizing Calendars with the NLS Calendar Utility.......................................................... 13-15
Displaying a Code Chart with the Oracle Locale Builder........................................................... 13-16
Creating a New Character Set Definition with the Oracle Locale Builder .............................. 13-20
Character Sets with User-Defined Characters .......................................................................... 13-20
Oracle Character Set Conversion Architecture......................................................................... 13-21
Unicode 4.0 Private Use Area...................................................................................................... 13-21
User-Defined Character Cross-References Between Character Sets...................................... 13-22
Guidelines for Creating a New Character Set from an Existing Character Set.................... 13-22
Example: Creating a New Character Set Definition with the Oracle Locale Builder.......... 13-23
Creating a New Linguistic Sort with the Oracle Locale Builder................................................ 13-26
Changing the Sort Order for All Characters with the Same Diacritic ................................... 13-29
Changing the Sort Order for One Character with a Diacritic ................................................. 13-31
Generating and Installing NLB Files .............................................................................................. 13-33
Deploying Custom NLB Files on Other Platforms ....................................................................... 13-34
Upgrading Custom NLB Files from Previous Releases of Oracle ............................................. 13-35
Transportable NLB Data .................................................................................................................... 13-35
A Locale Data
Languages ................................................................................................................................................. A-1
Translated Messages ............................................................................................................................... A-3
Territories .................................................................................................................................................. A-4
Character Sets........................................................................................................................................... A-5
Recommended Database Character Sets ....................................................................................... A-6
xiii
Other Character Sets ......................................................................................................................... A-8
Character Sets that Support the Euro Symbol ............................................................................ A-13
Client-Only Character Sets ............................................................................................................ A-14
Universal Character Sets ................................................................................................................ A-15
Character Set Conversion Support ............................................................................................... A-16
Subsets and Supersets..................................................................................................................... A-16
Language and Character Set Detection Support ............................................................................. A-18
Linguistic Sorts ...................................................................................................................................... A-20
Calendar Systems .................................................................................................................................. A-22
Time Zone Names.................................................................................................................................. A-23
Obsolete Locale Data ............................................................................................................................ A-29
Obsolete Linguistic Sorts................................................................................................................ A-29
Obsolete Territories......................................................................................................................... A-29
Obsolete Languages ........................................................................................................................ A-30
New Names for Obsolete Character Sets..................................................................................... A-30
AL24UTFFSS Character Set Desupported ................................................................................... A-31
Updates to the Oracle Language and Territory Definition Files.............................................. A-31
Index
xiv
Preface
This manual describes Oracle globalization support for the database. It explains how
to set up a globalization support environment, choose and migrate a character set,
customize locale data, do linguistic sorting, program in a global environment, and
program with Unicode.
This preface contains these topics:
■ Intended Audience
■ Documentation Accessibility
■ Structure
■ Related Documents
■ Conventions
Intended Audience
Oracle Database Globalization Support Guide is intended for database administrators,
system administrators, and database application developers who perform the
following tasks:
■ Set up a globalization support environment
■ Choose, analyze, or migrate character sets
■ Sort data linguistically
■ Customize locale data
■ Write programs in a global environment
■ Use Unicode
To use this document, you need to be familiar with relational database concepts, basic
Oracle server concepts, and the operating system environment under which you are
running Oracle.
Documentation Accessibility
Our goal is to make Oracle products, services, and supporting documentation
accessible, with good usability, to the disabled community. To that end, our
documentation includes features that make information available to users of assistive
technology. This documentation is available in HTML format, and contains markup to
facilitate access by the disabled community. Standards will continue to evolve over
time, and Oracle is actively engaged with other market-leading technology vendors to
xv
address technical obstacles so that our documentation can be accessible to all of our
customers. For additional information, visit the Oracle Accessibility Program Web site
at
http://www.oracle.com/accessibility/
Structure
This document contains:
xvi
Chapter 10, "OCI Programming in a Global Environment"
This chapter describes globalization considerations for OCI programming.
Glossary
The glossary contains definitions of globalization support terms.
Related Documents
Many of the examples in this book use the sample schemas of the seed database, which
is installed by default when you install Oracle. Refer to Oracle Database Sample Schemas
for information on how these schemas were created and how you can use them
yourself.
Printed documentation is available for sale in the Oracle Store at
http://oraclestore.oracle.com/
To download free release notes, installation documentation, white papers, or other
collateral, please visit the Oracle Technology Network (OTN). You must register online
before using OTN; registration is free and can be done at
http://otn.oracle.com/membership/
If you already have a username and password for OTN, then you can go directly to the
documentation section of the OTN Web site at
http://otn.oracle.com/documentation/
Conventions
This section describes the conventions used in the text and code examples of this
documentation set. It describes:
■ Conventions in Text
■ Conventions in Code Examples
■ Conventions for Windows Operating Systems
xvii
Conventions in Text
We use various conventions in text to help you more quickly identify special terms.
The following table describes those conventions and provides examples of their use.
The following table describes typographic conventions used in code examples and
provides examples of their use.
xviii
Convention Meaning Example
| A vertical bar represents a choice of two or {ENABLE | DISABLE}
more options within brackets or braces. [COMPRESS | NOCOMPRESS]
Enter one of the options. Do not enter the
vertical bar.
... Horizontal ellipsis points indicate either:
■ That we have omitted parts of the CREATE TABLE ... AS subquery;
code that are not directly related to the
example SELECT col1, col2, ... , coln FROM
employees;
■ That you can repeat a portion of the
code
. Vertical ellipsis points indicate that we SQL> SELECT NAME FROM V$DATAFILE;
. have omitted several lines of code not NAME
. directly related to the example. ------------------------------------
/fsl/dbs/tbs_01.dbf
/fs1/dbs/tbs_02.dbf
.
.
.
/fsl/dbs/tbs_09.dbf
9 rows selected.
Other notation You must enter symbols other than acctbal NUMBER(11,2);
brackets, braces, vertical bars, and ellipsis acct CONSTANT NUMBER(4) := 3;
points as shown.
Italics Italicized text indicates placeholders or CONNECT SYSTEM/system_password
variables for which you must supply DB_NAME = database_name
particular values.
UPPERCASE Uppercase typeface indicates elements SELECT last_name, employee_id FROM
supplied by the system. We show these employees;
terms in uppercase in order to distinguish SELECT * FROM USER_TABLES;
them from terms you define. Unless terms DROP TABLE hr.employees;
appear in brackets, enter them in the order
and with the spelling shown. However,
because these terms are not case sensitive,
you can enter them in lowercase.
lowercase Lowercase typeface indicates SELECT last_name, employee_id FROM
programmatic elements that you supply. employees;
For example, lowercase indicates names of sqlplus hr/hr
tables, columns, or files. CREATE USER mjones IDENTIFIED BY ty3MU9;
Note: Some programmatic elements use a
mixture of UPPERCASE and lowercase.
Enter these elements as shown.
xix
Convention Meaning Example
File and directory File and directory names are not case c:\winnt"\"system32 is the same as
names sensitive. The following special characters C:\WINNT\SYSTEM32
are not allowed: left angle bracket (<), right
angle bracket (>), colon (:), double
quotation marks ("), slash (/), pipe (|), and
dash (-). The special character backslash (\)
is treated as an element separator, even
when it appears in quotes. If the file name
begins with \\, then Windows assumes it
uses the Universal Naming Convention.
C:\> Represents the Windows command C:\oracle\oradata>
prompt of the current hard disk drive. The
escape character in a command prompt is
the caret (^). Your prompt reflects the
subdirectory in which you are working.
Referred to as the command prompt in this
manual.
Special characters The backslash (\) special character is C:\>exp scott/tiger TABLES=emp
sometimes required as an escape character QUERY=\"WHERE job='SALESMAN' and
for the double quotation mark (") special sal<1600\"
character at the Windows command C:\>imp SYSTEM/password FROMUSER=scott
prompt. Parentheses and the single TABLES=(emp, dept)
quotation mark (') do not require an escape
character. Refer to your Windows
operating system documentation for more
information on escape and special
characters.
HOME_NAME Represents the Oracle home name. The C:\> net start OracleHOME_NAMETNSListener
home name can be up to 16 alphanumeric
characters. The only special character
allowed in the home name is the
underscore.
xx
Convention Meaning Example
ORACLE_HOME In releases prior to Oracle8i release 8.1.3, Go to the ORACLE_BASE\ORACLE_
and ORACLE_ when you installed Oracle components, all HOME\rdbms\admin directory.
BASE subdirectories were located under a top
level ORACLE_HOME directory that by
default used one of the following names:
■ C:\orant for Windows NT
■ C:\orawin98 for Windows 98
This release complies with Optimal
Flexible Architecture (OFA) guidelines. All
subdirectories are not under a top level
ORACLE_HOME directory. There is a top
level directory called ORACLE_BASE that
by default is C:\oracle. If you install the
latest Oracle release on a computer with no
other Oracle software installed, then the
default setting for the first Oracle home
directory is C:\oracle\orann, where nn
is the latest release number. The Oracle
home directory is located directly under
ORACLE_BASE.
All directory path examples in this guide
follow OFA conventions.
Refer to Oracle Database Platform Guide for
Windows for additional information about
OFA compliances and for information
about installing Oracle products in
non-OFA compliant directories.
xxi
xxii
What's New in Globalization Support?
This section describes new features of globalization support and provides pointers to
additional information.
xxiii
The support for all SQL functions and operators to honor the NLS_SORT setting is
now available using the new NLS_COMP mode LINGUISTIC. This feature ensures
all SQL string comparisons are consistent, and that they follow the linguistic
convention as specified in the NLS_SORT parameter.
xxiv
See Also: Chapter 8, "Oracle Globalization Development Kit"
■ Regular Expressions
This release supports POSIX-compliant regular expressions to enhance search and
replace capability in programming environments such as UNIX and Java. In SQL,
this new functionality is implemented through new functions that are regular
expression extensions to existing SQL functions such as LIKE, REPLACE, and
INSTR. This implementation supports multilingual queries and is locale-sensitive.
■ Locale Variants
In previous releases, Oracle defined language and territory definitions separately.
This resulted in the definition of a territory being independent of the language
setting of the user. In this release, some territories can have different date, time,
number, and monetary formats based on the language setting of a user. This type
of language-dependent territory definition is called a locale variant.
■ NLS_LENGTH_SEMANTICS
NLS_LENGTH_SEMANTICS is now supported as an environment variable.
xxv
xxvi
1
Overview of Globalization Support
Multilingual
Database
ta s e
Fr Da
Da ane
en ta
ch
p
Ja
See Also:
■ Chapter 10, "OCI Programming in a Global Environment"
■ Chapter 3, "Setting Up a Globalization Support Environment"
See Also:
■ Chapter 6, "Supporting Multilingual Databases with Unicode"
■ Chapter 7, "Programming with Unicode"
■ "Enabling Multilingual Support with Unicode Datatypes" on
page 6-6
Language Support
The database enables you to store, process, and retrieve data in native languages. The
languages that can be stored in a database are all languages written in scripts that are
encoded by Oracle-supported character sets. Through the use of Unicode databases
and datatypes, the Oracle database supports most contemporary languages.
Additional support is available for a subset of the languages. The database can, for
example, display dates using translated month names or how to sort text data
according to cultural conventions.
When this manual uses the term language support, it refers to the additional
language-dependent functionality (for example, displaying dates or sorting text), not
to the ability to store text of a specific language.
For some of the supported languages, Oracle provides translated error messages and a
translated user interface for the database utilities.
See Also:
■ Chapter 3, "Setting Up a Globalization Support Environment"
■ "Languages" on page A-1 for a complete list of Oracle language
names and abbreviations
■ "Translated Messages" on page A-3 for a list of languages into
which Oracle messages are translated
Territory Support
The database supports cultural conventions that are specific to geographical locations.
The default local time format, date format, and numeric and monetary conventions
depend on the local territory setting. Setting different NLS parameters allows the
database session to use different cultural settings. For example, you can set the euro
(EUR) as the primary currency and the Japanese yen (JPY) as the secondary currency
for a given database session even when the territory is defined as AMERICA.
See Also:
■ Chapter 3, "Setting Up a Globalization Support Environment"
■ "Territories" on page A-4 for a list of territories that are
supported by the Oracle server
See Also:
■ Chapter 3, "Setting Up a Globalization Support Environment"
■ Chapter 4, "Datetime Datatypes and Time Zone Support"
■ Oracle Database SQL Reference
Calendars Feature
Many different calendar systems are in use around the world. Oracle supports seven
different calendar systems: Gregorian, Japanese Imperial, ROC Official (Republic of
China), Thai Buddha, Persian, English Hijrah, and Arabic Hijrah.
See Also:
■ Chapter 3, "Setting Up a Globalization Support Environment"
■ "Calendar Systems" on page A-22 for a list of supported
calendars
Linguistic Sorting
Oracle provides linguistic definitions for culturally accurate sorting and case
conversion. The basic definition treats strings as sequences of independent characters.
The extended definition recognizes pairs of characters that should be treated as special
cases.
Strings that are converted to upper case or lower case using the basic definition always
retain their lengths. Strings converted using the extended definition may become
longer or shorter.
See Also:
■ Chapter 2, "Choosing a Character Set"
■ "Character Sets" on page A-5 for a list of supported character
sets
Character Semantics
Oracle provides character semantics. It is useful for defining the storage requirements
for multibyte strings of varying widths in terms of characters instead of bytes.
See Also:
■ Chapter 13, "Customizing Locale"
■ "Customizing Calendars with the NLS Calendar Utility" on
page 13-15
Unicode Support
You can store Unicode characters in an Oracle database in two ways:
■ You can create a Unicode database that enables you to store UTF-8 encoded
characters as SQL CHAR datatypes.
■ You can support multilingual data in specific columns by using Unicode
datatypes. You can store Unicode characters into columns of the SQL NCHAR
datatypes regardless of how the database character set has been defined. The
NCHAR datatype is an exclusively Unicode datatype.
This chapter explains how to choose a character set. It includes the following topics:
■ Character Set Encoding
■ Length Semantics
■ Choosing an Oracle Database Character Set
■ Changing the Character Set After Database Creation
■ Monolingual Database Scenario
■ Multilingual Database Scenarios
The computer industry uses many encoded character sets. Character sets differ in the
following ways:
■ The number of characters available
■ The available characters (the character repertoire)
■ The scripts used for writing and the languages they represent
■ The code values assigned to each character
■ The encoding scheme used to represent a character
Oracle supports most national, international, and vendor-specific encoded character
set standards.
Writing Direction
Most Western languages are written left to right from the top to the bottom of the
page. East Asian languages are usually written top to bottom from the right to the left
of the page, although exceptions are frequently made for technical books translated
from Western languages. Arabic and Hebrew are written right to left from the top to
the bottom.
Numbers reverse direction in Arabic and Hebrew. Although the text is written right to
left, numbers within the sentence are written left to right. For example, "I wrote 32
books" would be written as "skoob 32 etorw I". Regardless of the writing direction,
Oracle stores the data in logical order. Logical order means the order that is used by
someone typing a language, not how it looks on the screen.
Writing direction does not affect the encoding of a character.
The following CHAR and VARCHAR characters are represented in all Oracle database
character sets and can be transported to any platform:
■ Uppercase and lowercase English characters A through Z and a through z
■ Arabic digits 0 through 9
■ The following punctuation marks: % ‘ ' ( ) * + - , . / \ : ; < > = ! _ & ~ { } | ^ ? $ # @ "
[]
■ The following control characters: space, horizontal tab, vertical tab, form feed
If you are using characters outside this set, then take care that your data is supported
in the database character set that you have chosen.
Setting the NLS_LANG parameter properly is essential to proper data conversion. The
character set that is specified by the NLS_LANG parameter should reflect the setting for
the client operating system. Setting NLS_LANG correctly enables proper conversion
from the client operating system character encoding to the database character set.
When these settings are the same, Oracle assumes that the data being sent or received
is encoded in the same character set as the database character set, so character set
validation or conversion may not be performed. This can lead to corrupt data if
conversions are necessary.
During conversion from one character set to another, Oracle expects client-side data to
be encoded in the character set specified by the NLS_LANG parameter. If you put other
values into the string (for example, by using the CHR or CONVERT SQL functions), then
the values may be corrupted when they are sent to the database because they are not
converted properly. If you have configured the environment correctly and if the
database character set supports the entire repertoire of character data that may be
input into the database, then you do not need to change the current database character
set. However, if your enterprise becomes more global and you have additional
characters or new languages to support, then you may need to choose a character set
with a greater character repertoire. Oracle Corporation recommends that you use
Unicode databases and datatypes in these cases.
See Also:
■ Chapter 6, "Supporting Multilingual Databases with Unicode"
■ Oracle Database SQL Reference for more information about the
CHR and CONVERT SQL functions
■ "Displaying a Code Chart with the Oracle Locale Builder" on
page 13-16
ASCII Encoding
Table 2–2 shows how the ASCII character is encoded. Row and column headings
denote hexadecimal digits. To find the encoded value of a character, read the column
number followed by the row number. For example, the code value of the character A is
0x41.
Character sets have evolved to meet the needs of users around the world. New
character sets have been created to support languages besides English. Typically, these
new character sets support a group of related languages based on the same script. For
example, the ISO 8859 character set series was created to support different European
languages. Table 2–3 shows the languages that are supported by the ISO 8859 character
sets.
Character sets evolved and provided restricted multilingual support. They were
restricted in the sense that they were limited to groups of languages based on similar
scripts. More recently, universal character sets have been regarded as a more useful
solution to multilingual support. Unicode is one such universal character set that
encompasses most major scripts of the modern world. The Unicode character set
supports more than 94,000 characters.
bit can be used to indicate whether that byte is a single-byte character or the first
byte of a double-byte character.
■ Shift-sensitive variable-width multibyte encoding schemes
Some variable-width encoding schemes use control codes to differentiate between
single-byte and multibyte characters with the same code values. A shift-out code
indicates that the following character is multibyte. A shift-in code indicates that
the following character is single-byte. Shift-sensitive encoding schemes are used
primarily on IBM platforms. Note that ISO-2022 character sets cannot be used as
database character sets, but they can be used for applications such as a mail server.
The parts of the names in angle brackets are concatenated. The optional S or C is used
to differentiate character sets that can be used only on the server (S) or only on the
client (C).
Length Semantics
In single-byte character sets, the number of bytes and the number of characters in a
string are the same. In multibyte character sets, a character or code point consists of
one or more bytes. Calculating the number of characters based on byte lengths can be
difficult in a variable-width character set. Calculating column lengths in bytes is called
byte semantics, while measuring column lengths in characters is called character
semantics.
Character semantics is useful for defining the storage requirements for multibyte
strings of varying widths. For example, in a Unicode database (AL32UTF8), suppose
that you need to define a VARCHAR2 column that can store up to five Chinese
characters together with five English characters. Using byte semantics, this column
requires 15 bytes for the Chinese characters, which are three bytes long, and 5 bytes for
the English characters, which are one byte long, for a total of 20 bytes. Using character
semantics, the column requires 10 characters.
The following expressions use byte semantics:
■ VARCHAR2(20 BYTE)
■ SUBSTRB(string, 1, 20)
Note the BYTE qualifier in the VARCHAR2 expression and the B suffix in the SQL
function name.
The following expressions use character semantics:
■ VARCHAR2(10 CHAR)
■ SUBSTR(string, 1, 10)
Note the CHAR qualifier in the VARCHAR2 expression.
The NLS_LENGTH_SEMANTICS initialization parameter determines whether a new
column of character datatype uses byte or character semantics. The default value of the
parameter is BYTE. The BYTE and CHAR qualifiers shown in the VARCHAR2 definitions
should be avoided when possible because they lead to mixed-semantics databases.
Instead, set NLS_LENGTH_SEMANTICS in the initialization parameter file and define
column datatypes to use the default semantics based on the value of NLS_LENGTH_
SEMANTICS.
Byte semantics is the default for the database character set. Character length semantics
is the default and the only allowable kind of length semantics for NCHAR datatypes.
The user cannot specify the CHAR or BYTE qualifier for NCHAR definitions.
Consider the following example:
CREATE TABLE employees
( employee_id NUMBER(4)
, last_name NVARCHAR2(10)
, job_id NVARCHAR2(9)
, manager_id NUMBER(4)
, hire_date DATE
, salary NUMBER(7,2)
, department_id NUMBER(2)
) ;
When the NCHAR character set is AL16UTF16, last_name can hold up to 10 Unicode
code points. When the NCHAR character set is AL16UTF16, last_name can hold up to
20 bytes.
Figure 2–2 shows the number of bytes needed to store different kinds of characters in
the UTF-8 character set. The ASCII characters requires one byte, the Latin and Greek
characters require two bytes, the Asian character requires three bytes, and the
supplementary character requires four bytes of storage.
ASCII
Latin
ASCII
Asian
Supplementary character
ASCII
Latin
Greek
Characters
Byte Storage 63 C3 91 74 E4 BA 9C F0 9D 84 9E 64 C3 B6 D0 A4
for UTF-8
1 2 1 3 4 1 2 2
byte bytes byte bytes bytes byte bytes bytes
See Also:
■ "SQL Functions for Different Length Semantics" on page 9-5 for
more information about the SUBSTR and SUBSTRB functions
■ "Length Semantics" on page 3-31 for more information about
the NLS_LENGTH_SEMANTICS initialization parameter
■ Chapter 6, "Supporting Multilingual Databases with Unicode"
for more information about Unicode and the NCHAR datatype
■ Oracle Database SQL Reference for more information about the
SUBSTRB and SUBSTR functions and the BYTE and CHAR
qualifiers for character datatypes
After the database is created, you cannot change the character sets, with some
exceptions, without re-creating the database.
Consider the following questions when you choose an Oracle character set for the
database:
■ What languages does the database need to support now?
■ What languages does the database need to support in the future?
■ Is the character set available on the operating system?
■ What character sets are used on clients?
■ How well does the application handle the character set?
■ What are the performance implications of the character set?
■ What are the restrictions associated with the character set?
The Oracle character sets are listed in "Character Sets" on page A-5. They are named
according to the languages and regions in which they are used. Some character sets
that are named for a region are also listed explicitly by language.
If you want to see the characters that are included in a character set, then:
■ Check national, international, or vendor product documentation or standards
documents
■ Use Oracle Locale Builder
This section contains the following topics:
■ Current and Future Language Requirements
■ Client Operating System and Application Compatibility
■ Character Set Conversion Between Clients and the Server
■ Performance Implications of Choosing a Database Character Set
■ Restrictions on Database Character Sets
■ Choosing a National Character Set
■ Summary of Supported Datatypes
See Also:
■ "UCS-2 Encoding" on page 6-3
■ "Choosing a National Character Set" on page 2-14
■ "Changing the Character Set After Database Creation" on
page 2-15
■ Appendix A, "Locale Data"
■ Chapter 13, "Customizing Locale"
system, you can create and run a database with a Japanese character set. However,
when an application in the client operating system accesses the database, the client
operating system must be able to support the database character set with appropriate
fonts and input methods. For example, you cannot insert or retrieve Japanese data on
the English Windows operating system without first installing a Japanese font and
input method. Another way to insert and retrieve Japanese data is to use a Japanese
operating system remotely to access the database server.
set as the database character set. Currently, only the AL16UTF16 character set cannot
be used as a database character set.
For a list of supported string formats and character sets, including LOB data (LOB,
BLOB, CLOB, and NCLOB), see Table 2–7 on page 2-14.
your systems today in Unicode offers many advantages in usability, compatibility, and
extensibility. Oracle Database's comprehensive support enables you to deploy
high-performing systems faster and more easily while utilizing the advantages of
Unicode. Even if you do not need to support multilingual data today or have any
requirement for Unicode, it is still likely to be the best choice for a new system in the
long run and will ultimately save you time and money as well as give you competitive
advantages. See Chapter 6, "Supporting Multilingual Databases with Unicode" for
more information about Unicode.
Table 2–7 lists the SQL datatypes that are supported for abstract datatypes.
You can create an abstract datatype with the NCHAR attribute as follows:
SQL> CREATE TYPE tp1 AS OBJECT (a NCHAR(10));
Type created.
SQL> CREATE TABLE t1 (a tp1);
Table created.
See Also:
■ Chapter 11, "Character Set Migration"
■ Oracle Database Upgrade Guide for more information about
exporting and importing data
■ Oracle Streams Concepts and Administration for information
about using Streams to change the character set of a database
while the database remains online
Japanese
Server
(JA16EUC)
Unix
(JA16EUC)
You can also use a multitier architecture. Figure 2–4 shows an application server
between the database server and the client. The application server and the database
server use the same character set in a monolingual scenario. The server, the application
server, and the client use the JA16EUC character set.
Japanese Browser
Server
(JA16EUC)
Application
Server
(JA16EUC)
Japanese
Server
(JA16EUC)
Unix
(JA16EUC)
Character
Conversion
Windows
(JA16SJIS)
When a target character set does not contain all of the characters in the source data,
replacement characters are used. If, for example, a server uses US7ASCII and a
German client uses WE8ISO8859P1, then the German character ß is replaced with ?
and ä is replaced with a.
Replacement characters may be defined for specific characters as part of a character set
definition. When a specific replacement character is not defined, a default replacement
character is used. To avoid the use of replacement characters when converting from a
client character set to a database character set, the server character set should be a
superset of all the client character sets.
Figure 2–6 shows that data loss occurs when the database character set does not
include all of the characters in the client character set. The database character set is
US7ASCII. The client's character set is WE8MSWIN1252, and the language used by the
client is German. When the client inserts a string that contains ß, the database replaces
ß with ?, resulting in lost data.
American
Database
Server
(US7ASCII)
Character
Conversion
German
Windows
(WE8MSWIN1252)
If German data is expected to be stored on the server, then a database character set that
supports German characters should be used for both the server and the client to avoid
data loss and conversion overhead.
When one of the character sets is a variable-width multibyte character set, conversion
can introduce noticeable overhead. Carefully evaluate your situation and choose
character sets to avoid conversion as much as possible.
Swedish
(WE8ISO8859P1)
Western
European
Server
Character
Conversion
French German
(WE8ISO8859P1) (WE8DEC)
Figure 2–8 shows a server that uses the AL32UTF8 Oracle character set that is based on
the Unicode UTF-8 character set.
French German
Client Client
(WE8ISO8859P1) (WE8DEC)
Character Character
Conversion Conversion
Unicode
Database
(AL32UTF8)
Character Character
Conversion Conversion
Japanese Japanese
Client Client
(JA16EUC) (JA16SJIS)
French
Client
Browser
(UTF-8)
German
Client
Application
Server
(UTF-8)
Japanese
(UTF-8) Client
Browser
The database, the application server, and each client use the AL32UTF8 character set.
This eliminates the need for character conversion even though the clients are French,
German, and Japanese.
This chapter tells how to set up a globalization support environment. It includes the
following topics:
■ Setting NLS Parameters
■ Choosing a Locale with the NLS_LANG Environment Variable
■ NLS Database Parameters
■ Language and Territory Parameters
■ Date and Time Parameters
■ Calendar Definitions
■ Numeric and List Parameters
■ Monetary Parameters
■ Linguistic Sort Parameters
■ Character Set Conversion Parameter
■ Length Semantics
■ In SQL functions
You can use NLS parameters explicitly to hardcode NLS behavior within a SQL
function. This practice overrides the default values that are set for the session in
the initialization parameter file, set for the client with environment variables, or
set for the session by the ALTER SESSION statement. For example:
TO_CHAR(hiredate, 'DD/MON/YYYY', 'nls_date_language = FRENCH')
Table 3–1 shows the precedence order of the different methods of setting NLS
parameters. Higher priority settings override lower priority settings. For example, a
default value has the lowest priority and can be overridden by any other method.
Table 3–2 lists the available NLS parameters. Because the SQL function NLS
parameters can be specified only with specific functions, the table does not show the
SQL function scope.
The NLS_LANG parameter has three components: language, territory, and character set.
Specify it in the following format, including the punctuation:
NLS_LANG = language_territory.charset
For example, if the Oracle Installer does not populate NLS_LANG, then its value by
default is AMERICAN_AMERICA.US7ASCII. The language is AMERICAN, the territory
is AMERICA, and the character set is US7ASCII. The values in NLS_LANG and other
NLS parameters are case-insensitive.
Each component of the NLS_LANG parameter controls the operation of a subset of
globalization support features:
■ language
Specifies conventions such as the language used for Oracle messages, sorting, day
names, and month names. Each supported language has a unique name; for
example, AMERICAN, FRENCH, or GERMAN. The language argument specifies
default values for the territory and character set arguments. If the language is not
specified, then the value defaults to AMERICAN.
■ territory
Specifies conventions such as the default date, monetary, and numeric formats.
Each supported territory has a unique name; for example, AMERICA, FRANCE, or
CANADA. If the territory is not specified, then the value is derived from the
language value.
■ charset
Specifies the character set used by the client application (normally the Oracle
character set that corresponds to the user's terminal character set or the OS
character set). Each supported character set has a unique acronym, for example,
US7ASCII, WE8ISO8859P1, WE8DEC, WE8MSWIN1252, or JA16EUC. Each
language has a default character set associated with it.
NLS_LANG = FRENCH_CANADA.WE8ISO8859P1
NLS_LANG = JAPANESE_JAPAN.JA16EUC
Note that illogical combinations can be set but do not work properly. For example, the
following specification tries to support Japanese by using a Western European
character set:
NLS_LANG = JAPANESE_JAPAN.WE8ISO8859P1
Because the WE8ISO8859P1 character set does not support any Japanese characters,
you cannot store or display Japanese data if you use this definition for NLS_LANG.
The rest of this section includes the following topics:
■ Specifying the Value of NLS_LANG
■ Overriding Language and Territory Specifications
■ Locale Variants
See Also:
■ Appendix A, "Locale Data" for a complete list of supported
languages, territories, and character sets
■ Your operating system documentation for information about
additional globalization settings that may be necessary for your
platform
Then the query shown in Example 3–1 returns the following output:
LAST_NAME HIRE_DAT SALARY
------------------------- -------- ----------
Note that the date format and the number format have changed. The numbers have
not changed, because the underlying data is the same.
Note: You cannot modify the setting for the client character set
with the ALTER SESSION statement.
The ALTER SESSION statement modifies only the session environment. The local
client NLS environment is not modified, unless the client explicitly retrieves the new
settings and modifies its local environment.
See Also:
■ "Overriding Default Values for NLS_LANGUAGE and NLS_
TERRITORY During a Session" on page 3-13
■ Oracle Database SQL Reference
Locale Variants
Before Oracle Database 10g, Oracle defined language and territory definitions
separately. This resulted in the definition of a territory being independent of the
language setting of the user. In Oracle Database 10g, some territories can have different
date, time, number, and monetary formats based on the language setting of a user.
This type of language-dependent territory definition is called a locale variant.
For the variant to work properly, both NLS_TERRITORY and NLS_LANGUAGE must be
specified.
Table 3–3 shows the territories that have been enhanced to support variations.
UTF-8 WIN32 clients. Instead, the NLS_LANG setting should reflect the code page of
the client. For example, on an English Windows client, the code page is 1252. An
appropriate setting for NLS_LANG is AMERICAN_AMERICA.WE8MSWIN1252.
Setting NLS_LANG correctly allows proper conversion from the client operating system
character set to the database character set. When these settings are the same, Oracle
assumes that the data being sent or received is encoded in the same character set as the
database character set, so character set validation or conversion may not be performed.
This can lead to corrupt data if the client code page and the database character set are
different and conversions are necessary.
See Also: Oracle Database Installation Guide for 32-Bit Windows for
more information about commonly used values of the NLS_LANG
parameter in Windows
OCINlsGetInfo() Function
User applications can query client NLS settings with the OCINlsGetInfo() function.
NLS_LANGUAGE
Property Description
Parameter type String
Parameter scope Initialization parameter and ALTER SESSION
Default value Derived from NLS_LANG
Range of values Any valid language name
Messages used by the server are stored in binary-format files that are placed in the
$ORACLE_HOME/product_name/mesg directory, or the equivalent for your
operating system. Multiple versions of these files can exist, one for each supported
language, using the following filename convention:
<product_id><language_abbrev>.MSB
For example, the file containing the server messages in French is called oraf.msb,
because ORA is the product ID (<product_id>) and F is the language abbreviation
(<language_abbrev>) for French. The product_name is rdbms, so it is in the
$ORACLE_HOME/rdbms/mesg directory.
If NLS_LANG is specified in the client environment, then the value of NLS_LANGUAGE
in the initialization parameter file is overridden at connection time.
Messages are stored in these files in one specific character set, depending on the
language and the operating system. If this character set is different from the database
character set, then message text is automatically converted to the database character
set. If necessary, it is then converted to the client character set if the client character set
is different from the database character set. Hence, messages are displayed correctly at
the user's terminal, subject to the limitations of character set conversion.
The language-specific binary message files that are actually installed depend on the
languages that the user specifies during product installation. Only the English binary
message file and the language-specific binary message files specified by the user are
installed.
The default value of NLS_LANGUAGE may be specific to the operating system. You can
alter the NLS_LANGUAGE parameter by changing its value in the initialization
parameter file and then restarting the instance.
All messages and text should be in the same language. For example, when you run an
Oracle Developer application, the messages and boilerplate text that you see originate
from three sources:
■ Messages from the server
■ Messages and boilerplate text generated by Oracle Forms
■ Messages and boilerplate text generated by the application
NLS_LANGUAGE determines the language used for the first two kinds of text. The
application is responsible for the language used in its messages and boilerplate text.
The following examples show behavior that results from setting NLS_LANGUAGE to
different values.
NLS_TERRITORY
Property Description
Parameter type String
Parameter scope Initialization parameter and ALTER SESSION
Default value Derived from NLS_LANG
Range of values Any valid territory name
NLS_TERRITORY specifies the conventions for the following default date and numeric
formatting characteristics:
■ Date format
■ Decimal character and group separator
■ Local currency symbol
■ ISO currency symbol
■ Dual currency symbol
■ First day of the week
■ Credit and debit symbols
■ ISO week flag
■ List separator
The value specified for NLS_TERRITORY in the initialization parameter file is the
default for the instance. For example, to specify the default as France, the parameter
should be set as follows:
NLS_TERRITORY = FRANCE
When the territory is FRANCE, numbers are formatted using a comma as the decimal
character.
You can alter the NLS_TERRITORY parameter by changing the value in the
initialization parameter file and then restarting the instance. The default value of NLS_
TERRITORY can be specific to the operating system.
If NLS_LANG is specified in the client environment, then the value of NLS_TERRITORY
in the initialization parameter file is overridden at connection time.
The territory can be modified dynamically during the session by specifying the new
NLS_TERRITORY value in an ALTER SESSION statement. Modifying NLS_
TERRITORY resets all derived NLS session parameters to default values for the new
territory.
To change the territory to France during a session, issue the following ALTER
SESSION statement:
ALTER SESSION SET NLS_TERRITORY = France;
The following examples show behavior that results from different settings of NLS_
TERRITORY and NLS_LANGUAGE.
Note that the currency symbol has changed from $ to €. The numbers have not
changed because the underlying data is the same.
Note that the currency symbol changed from € to $ because the territory changed from
Germany to America.
Note the language of the month abbreviations and the decimal character.
Note that the language of the month abbreviations is German and the decimal
character is a period.
The behavior of the NLS_LANG environment variable implicitly determines the
language environment of the database for each session. When a session connects to a
database, an ALTER SESSION statement is automatically executed to set the values of
the database parameters NLS_LANGUAGE and NLS_TERRITORY to those specified by
the language and territory arguments of NLS_LANG. If NLS_LANG is not defined,
then no implicit ALTER SESSION statement is executed.
When NLS_LANG is defined, the implicit ALTER SESSION is executed for all instances
to which the session connects, for both direct and indirect connections. If the values of
NLS parameters are changed explicitly with ALTER SESSION during a session, then
the changes are propagated to all instances to which that user session is connected.
Date Formats
Different date formats are shown in Table 3–4.
NLS_DATE_FORMAT
Property Description
Parameter type String
Parameter scope Initialization parameter, environment variable, and ALTER
SESSION
Default value Derived from NLS_TERRITORY
Range of values Any valid date format mask
The NLS_DATE_FORMAT parameter defines the default date format to use with the
TO_CHAR and TO_DATE functions. The NLS_TERRITORY parameter determines the
default value of NLS_DATE_FORMAT. The value of NLS_DATE_FORMAT can be any
valid date format mask. For example:
NLS_DATE_FORMAT = "MM/DD/YYYY"
To add string literals to the date format, enclose the string literal with double quotes.
Note that when double quotes are included in the date format, the entire value must
be enclosed by single quotes. For example:
NLS_DATE_FORMAT = '"Date: "MM/DD/YYYY'
You should see the following output if today's date is February 12, 1997:
CURRDATE
---------
12 II 1997
The value of NLS_DATE_FORMAT is stored in the internal date format. Each format
element occupies two bytes, and each string occupies the number of bytes in the string
plus a terminator byte. Also, the entire format mask has a two-byte terminator. For
example, "MM/DD/YY" occupies 14 bytes internally because there are three format
elements (month, day, and year), two 3-byte strings (the two slashes), and the two-byte
terminator for the format mask. The format for the value of NLS_DATE_FORMAT
cannot exceed 24 bytes.
You can alter the default value of NLS_DATE_FORMAT by:
■ Changing its value in the initialization parameter file and then restarting the
instance
■ Using an ALTER SESSION SET NLS_DATE_FORMAT statement
If a table or index is partitioned on a date column, and if the date format specified by
NLS_DATE_FORMAT does not specify the first two digits of the year, then you must use
the TO_DATE function with a 4-character format mask for the year.
For example:
TO_DATE('11-jan-1997', 'dd-mon-yyyy')
NLS_DATE_LANGUAGE
Property Description
Parameter type String
Parameter scope Initialization parameter, environment variable, ALTER SESSION,
and SQL functions
Default value Derived from NLS_LANGUAGE
Range of values Any valid language name
The NLS_DATE_LANGUAGE parameter specifies the language for the day and month
names produced by the TO_CHAR and TO_DATE functions. NLS_DATE_LANGUAGE
overrides the language that is specified implicitly by NLS_LANGUAGE. NLS_DATE_
LANGUAGE has the same syntax as the NLS_LANGUAGE parameter, and all supported
languages are valid values.
NLS_DATE_LANGUAGE also determines the language used for:
■ Month and day abbreviations returned by the TO_CHAR and TO_DATE functions
■ Month and day abbreviations used by the default date format (NLS_DATE_
FORMAT)
■ Abbreviations for AM, PM, AD, and BC
When numbers are spelled in words using the TO_CHAR function, the English spelling
is always used. For example, enter the following SELECT statement:
SQL> SELECT TO_CHAR(TO_DATE('12-Oct.-2001'),'Day: ddspth Month') FROM DUAL;
Time Formats
Different time formats are shown in Table 3–5.
NLS_TIMESTAMP_FORMAT
Property Description
Parameter type String
Parameter scope Initialization parameter, environment variable, and ALTER
SESSION
Default value Derived from NLS_TERRITORY
Range of values Any valid datetime format mask
NLS_TIMESTAMP_FORMAT defines the default date format for the TIMESTAMP and
TIMESTAMP WITH LOCAL TIME ZONE datatypes. The following example shows a
value for NLS_TIMESTAMP_FORMAT:
NLS_TIMESTAMP_FORMAT = 'YYYY-MM-DD HH:MI:SS.FF'
NLS_TIMESTAMP_TZ_FORMAT
Property Description
Parameter type String
Parameter scope Initialization parameter, environment variable, and ALTER
SESSION
Default value Derived from NLS_TERRITORY
Range of values Any valid datetime format mask
The following example of the TO_TIMESTAMP_TZ function uses the format value that
was specified for NLS_TIMESTAMP_TZ_FORMAT:
SQL> SELECT TO_TIMESTAMP_TZ('2000-08-20, 05:00:00.55 America/Los_Angeles',
'yyyy-mm-dd hh:mi:ss.ff TZR') FROM DUAL;
See Also:
■ Oracle Database SQL Reference for more information about the
TO_TIMESTAMP_TZ function and the ALTER SESSION
statement
■ "Choosing a Time Zone File" on page 4-15 for more information
about time zones
Calendar Definitions
This section includes the following topics:
■ Calendar Formats
■ NLS_CALENDAR
Calendar Formats
The following calendar information is stored for each territory:
■ First Day of the Week
■ First Calendar Week of the Year
■ Number of Days and Months in a Year
■ First Year of Era
Table 3–7 First ISO Week of the Year: Example 1, January 1998
Mo Tu We Th Fr Sa Su ISO Week
- - - 1 2 3 4 First ISO week of 1998
Table 3–7 (Cont.) First ISO Week of the Year: Example 1, January 1998
Mo Tu We Th Fr Sa Su ISO Week
5 6 7 8 9 10 11 Second ISO week of 1998
12 13 14 15 16 17 18 Third ISO week of 1998
19 20 21 22 23 24 25 Fourth ISO week of 1998
26 27 28 29 30 31 - Fifth ISO week of 1998
Table 3–8 shows an example in which January 1 occurs in a week that has three or
fewer days in the first calendar week of the year. The week containing January 1 is the
53rd ISO week of 1998, and the following week is the first ISO week of 1999.
Table 3–8 First ISO Week of the Year: Example 2, January 1999
Mo Tu We Th Fr Sa Su ISO Week
- - - - 1 2 3 Fifty-third ISO week of 1998
4 5 6 7 8 9 10 First ISO week of 1999
11 12 13 14 15 16 17 Second ISO week of 1999
18 19 20 21 22 23 24 Third ISO week of 1999
25 26 27 28 29 30 31 Fourth ISO week of 1999
The first calendar week of the year is determined by the NLS_TERRITORY parameter.
NLS_CALENDAR
Property Description
Parameter type String
Parameter scope Initialization parameter, environment variable, ALTER SESSION,
and SQL functions
Default value Gregorian
Range of values Any valid calendar format name
Many different calendar systems are in use throughout the world. NLS_CALENDAR
specifies which calendar system Oracle uses.
NLS_CALENDAR can have one of the following values:
■ Arabic Hijrah
■ English Hijrah
■ Gregorian
■ Japanese Imperial
■ Persian
■ ROC Official (Republic of China)
■ Thai Buddha
Numeric Formats
The database must know the number-formatting convention used in each session to
interpret numeric strings correctly. For example, the database needs to know whether
numbers are entered with a period or a comma as the decimal character (234.00 or
234,00). Similarly, applications must be able to display numeric information in the
format expected at the client site.
Examples of numeric formats are shown in Table 3–9.
Numeric formats are derived from the setting of the NLS_TERRITORY parameter, but
they can be overridden by the NLS_NUMERIC_CHARACTERS parameter.
NLS_NUMERIC_CHARACTERS
Property Description
Parameter type String
Parameter scope Initialization parameter, environment variable, ALTER SESSION,
and SQL functions
Default value Default decimal character and group separator for a particular
territory
Range of values Any two valid numeric characters
This parameter specifies the decimal character and group separator. The group
separator is the character that separates integer groups to show thousands and
millions, for example. The group separator is the character returned by the G number
format mask. The decimal character separates the integer and decimal parts of a
number. Setting NLS_NUMERIC_CHARACTERS overrides the values derived from the
setting of NLS_TERRITORY.
Any character can be the decimal character or group separator. The two characters
specified must be single-byte, and the characters must be different from each other.
The characters cannot be any numeric character or any of the following characters:
plus (+), hyphen (-), less than sign (<), greater than sign (>). Either character can be a
space.
SQL statements can include numbers represented as numeric or text literals. Numeric
literals are not enclosed in quotes. They are part of the SQL language syntax and
always use a dot as the decimal character and never contain a group separator. Text
literals are enclosed in single quotes. They are implicitly or explicitly converted to
numbers, if required, according to the current NLS settings.
The following SELECT statement formats the number 4000 with the decimal character
and group separator specified in the ALTER SESSION statement:
SELECT TO_CHAR(4000, '9G999D99') FROM DUAL;
NLS_LIST_SEPARATOR
Property Description
Parameter type String
Parameter scope Environment variable
Default value Derived from NLS_TERRITORY
Range of values Any valid character
Monetary Parameters
This section includes the following topics:
■ Currency Formats
■ NLS_CURRENCY
■ NLS_ISO_CURRENCY
■ NLS_DUAL_CURRENCY
■ NLS_MONETARY_CHARACTERS
■ NLS_CREDIT
■ NLS_DEBIT
Currency Formats
Different currency formats are used throughout the world. Some typical ones are
shown in Table 3–10.
NLS_CURRENCY
Property Description
Parameter type String
Parameter scope Initialization parameter, environment variable, ALTER SESSION,
and SQL functions
Default value Derived from NLS_TERRITORY
Range of values Any valid currency symbol string
NLS_CURRENCY specifies the character string returned by the L number format mask,
the local currency symbol. Setting NLS_CURRENCY overrides the setting defined
implicitly by NLS_TERRITORY.
$003,878.40
$021,586.20
NLS_ISO_CURRENCY
Property Description
Parameter type String
Parameter scope Initialization parameter, environment variable, ALTER SESSION,
and SQL functions
Default value Derived from NLS_TERRITORY
Range of values Any valid string
NLS_ISO_CURRENCY has the same syntax as the NLS_TERRITORY parameter, and all
supported territories are valid values.
NLS_DUAL_CURRENCY
Property Description
Parameter type String
Parameter scope Initialization parameter, environmental variable, ALTER SESSION,
and SQL functions
Default value Derived from NLS_TERRITORY
Range of values Any valid symbol
Customers who must retain their obsolete local currency symbol can override the
default for NLS_DUAL_CURRENCY or NLS_CURRENCY by defining them as parameters
in the initialization file on the server and as environment variables on the client.
It is not possible to override the ISO currency symbol that results from the value of
NLS_ISO_CURRENCY.
NLS_MONETARY_CHARACTERS
Property Description
Parameter type String
Parameter scope Environment variable
Default value Derived from NLS_TERRITORY
Range of values Any valid character
NLS_CREDIT
Property Description
Parameter type String
Parameter scope Environment variable
Default value Derived from NLS_TERRITORY
Range of values Any string, maximum of 9 bytes (not including null)
NLS_CREDIT sets the symbol that displays a credit in financial reports. The default
value of this parameter is determined by NLS_TERRITORY. For example, a space is a
valid value of NLS_CREDIT.
NLS_DEBIT
Property Description
Parameter type String
Parameter scope Environment variable
Default value Derived from NLS_TERRITORY
Range of values Any string, maximum or 9 bytes (not including null)
NLS_DEBIT sets the symbol that displays a debit in financial reports. The default
value of this parameter is determined by NLS_TERRITORY. For example, a minus sign
(-) is a valid value of NLS_DEBIT.
This parameter can be specified only in the client environment.
It can be retrieved through the OCIGetNlsInfo() function.
NLS_SORT
Property Description
Parameter type String
Parameter scope Initialization parameter, environment variable, ALTER SESSION,
and SQL functions
Default value Derived from NLS_LANGUAGE
Range of values BINARY or any valid linguistic sort name
NLS_SORT specifies the type of sort for character data. It overrides the default value
that is derived from NLS_LANGUAGE.
NLS_SORT contains either of the following values:
NLS_SORT = BINARY | sort_name
NLS_SORT = German
You can alter the default value of NLS_SORT by doing one of the following:
■ Changing its value in the initialization parameter file and then restarting the
instance
■ Using an ALTER SESSION statement
See Also:
■ Chapter 5, "Linguistic Sorting and String Searching"
■ Oracle Database SQL Reference for more information about the
ALTER SESSION statement
■ "Linguistic Sorts" on page A-20 for a list of linguistic sort names
NLS_COMP
Property Description
Parameter type String
Parameter scope Initialization parameter, environment variable, and ALTER
SESSION
Default value BINARY
Range of values BINARY , LINGUISTIC, or ANSI
See Also:
■ "Using Linguistic Sorts" on page 5-2
■ "Using Linguistic Indexes" on page 5-17
NLS_NCHAR_CONV_EXCP
Property Description
Parameter type String
Parameter scope Initialization parameter and ALTER SESSION
Default value FALSE
Range of values TRUE or FALSE
Length Semantics
This section includes the following topic:
■ NLS_LENGTH_SEMANTICS
NLS_LENGTH_SEMANTICS
Property Description
Parameter type String
Parameter scope Environment variable, initialization parameter, and ALTER
SESSION
Default value BYTE
Range of values BYTE or CHAR
By default, the character datatypes CHAR and VARCHAR2 are specified in bytes, not
characters. Hence, the specification CHAR(20) in a table definition allows 20 bytes for
storing character data.
This works well if the database character set uses a single-byte character encoding
scheme because the number of characters is the same as the number of bytes. If the
database character set uses a multibyte character encoding scheme, then the number of
bytes no longer equals the number of characters because a character can consist of one
or more bytes. Thus, column widths must be chosen with care to allow for the
maximum possible number of bytes for a given number of characters. You can
overcome this problem by switching to character semantics when defining the column
size.
NLS_LENGTH_SEMANTICS enables you to create CHAR, VARCHAR2, and LONG columns
using either byte or character length semantics. NCHAR, NVARCHAR2, CLOB, and NCLOB
columns are always character-based. Existing columns are not affected.
You may be required to use byte semantics in order to maintain compatibility with
existing applications.
NLS_LENGTH_SEMANTICS does not apply to tables in SYS and SYSTEM. The data
dictionary always uses byte semantics.
Note that if the NLS_LENGTH_SEMANTICS environment variable is not set on the
client, then the client session defaults to the value for NLS_LENGTH_SEMANTICS on
the database server. This enables all client sessions on the network to have the same
NLS_LENGTH_SEMANTICS behavior. Setting the environment variable on an
individual client enables the server initialization parameter to be overridden for that
client.
See Also:
■ "Length Semantics" on page 2-8
■ Oracle Database Concepts for more information about length
semantics
■ YEAR
■ MONTH
■ DAY
■ HOUR
■ MINUTE
■ SECOND
TIMESTAMP WITH TIME ZONE also includes these fields:
■ TIMEZONE_HOUR
■ TIMEZONE_MINUTE
■ TIMEZONE_REGION
■ TIMEZONE_ABBR
TIMESTAMP WITH LOCAL TIME ZONE does not store time zone information
internally, but you can see local time zone information in SQL output if the TZH:TZM
or TZR TZD format elements are specified.
The following sections describe the datetime datatypes and interval datatypes in more
detail:
■ Datetime Datatypes
■ Interval Datatypes
See Also: Oracle Database SQL Reference for the valid values of the
datetime and interval fields. Oracle Database SQL Reference also
contains information about format elements.
Datetime Datatypes
This section includes the following topics:
■ DATE Datatype
■ TIMESTAMP Datatype
■ TIMESTAMP WITH TIME ZONE Datatype
■ TIMESTAMP WITH LOCAL TIME ZONE Datatype
■ Inserting Values into Datetime Datatypes
■ Choosing a TIMESTAMP Datatype
DATE Datatype
The DATE datatype stores date and time information. Although date and time
information can be represented in both character and number datatypes, the DATE
datatype has special associated properties. For each DATE value, Oracle stores the
following information: century, year, month, date, hour, minute, and second.
You can specify a date value by:
■ Specifying the date value as a literal
■ Converting a character or numeric value to a date value with the TO_DATE
function
A date can be specified as an ANSI date literal or as an Oracle date value.
An ANSI date literal contains no time portion and must be specified in exactly the
following format:
DATE 'YYYY-MM-DD'
Alternatively, you can specify an Oracle date value as shown in the following example:
TO_DATE('1998-DEC-25 17:30','YYYY-MON-DD HH24:MI','NLS_DATE_LANGUAGE=AMERICAN')
The default date format for an Oracle date value is derived from the NLS_DATE_
FORMAT and NLS_DATE_LANGUAGE initialization parameters. The date format in the
example includes a two-digit number for the day of the month, an abbreviation of the
month name, the last two digits of the year, and a 24-hour time designation. The
specification for NLS_DATE_LANGUAGE is included because 'DEC' is not a valid value
for MON in all locales.
Oracle automatically converts character values that are in the default date format into
date values when they are used in date expressions.
If you specify a date value without a time component, then the default time is
midnight. If you specify a date value without a date, then the default date is the first
day of the current month.
Oracle DATE columns always contain fields for both date and time. If your queries use
a date format without a time portion, then you must ensure that the time fields in the
DATE column are set to midnight. You can use the TRUNC (date) SQL function to
ensure that the time fields are set to midnight, or you can make the query a test of
greater than or less than (<, <=, >=, or >) instead of equality or inequality (= or !=).
Otherwise, Oracle may not return the query results you expect.
See Also:
■ Oracle Database SQL Reference for more information about the
DATE datatype
■ "NLS_DATE_FORMAT" on page 3-15
■ "NLS_DATE_LANGUAGE" on page 3-16
■ Oracle Database SQL Reference for more information about
literals, format elements such as MM, and the TO_DATE function
TIMESTAMP Datatype
The TIMESTAMP datatype is an extension of the DATE datatype. It stores year, month,
day, hour, minute, and second values. It also stores fractional seconds, which are not
stored by the DATE datatype.
Specify the TIMESTAMP datatype as follows:
TIMESTAMP [(fractional_seconds_precision)]
See Also:
■ Oracle Database SQL Reference for more information about the
TIMESTAMP datatype
■ "NLS_TIMESTAMP_FORMAT" on page 3-18
■ "NLS_DATE_LANGUAGE" on page 3-16
Two TIMESTAMP WITH TIME ZONE values are considered identical if they represent
the same instant in UTC, regardless of the TIME ZONE offsets stored in the data. For
example, the following expressions have the same value:
TIMESTAMP '1999-01-15 8:00:00 -8:00'
TIMESTAMP '1999-01-15 11:00:00 -5:00'
You can replace the UTC offset with the TZR (time zone region) format element. The
following expression specifies US/Pacific for the time zone region:
TIMESTAMP '1999-01-15 8:00:00 US/Pacific'
To eliminate the ambiguity of boundary cases when the time switches from Standard
Time to Daylight Saving Time, use both the TZR format element and the corresponding
TZD format element. The TZD format element is an abbreviation of the time zone
region with Daylight Saving Time information included. Examples are PST for
US/Pacific standard time and PDT for US/Pacific daylight time. The following
specification ensures that a Daylight Saving Time value is returned:
TIMESTAMP '1999-10-29 01:30:00 US/Pacific PDT'
If you do not add the TZD format element, and the datetime value is ambiguous, then
Oracle returns an error if you have the ERROR_ON_OVERLAP_TIME session parameter
set to TRUE. If ERROR_ON_OVERLAP_TIME is set to FALSE (the default value), then
Oracle interprets the ambiguous datetime as Standard Time.
The default date format for the TIMESTAMP WITH TIME ZONE datatype is
determined by the value of the NLS_TIMESTAMP_TZ_FORMAT initialization
parameter.
See Also:
■ Oracle Database SQL Reference for more information about the
TIMESTAMP WITH TIME ZONE datatype
■ "TIMESTAMP Datatype" on page 4-3 for more information
about fractional seconds precision
■ "Support for Daylight Saving Time" on page 4-21
■ "NLS_TIMESTAMP_TZ_FORMAT" on page 3-19
■ Oracle Database SQL Reference for more information about
format elements
■ Oracle Database SQL Reference for more information about
setting the ERROR_ON_OVERLAP_TIME session parameter
See Also:
■ Oracle Database SQL Reference for more information about the
TIMESTAMP WITH LOCAL TIME ZONE datatype
■ "TIMESTAMP Datatype" on page 4-3 for more information
about fractional seconds precision
■ "NLS_TIMESTAMP_FORMAT" on page 3-18
Create a table table_dt with columns c_id and c_dt. The c_id column is of
NUMBER datatype and helps to identify the method by which the data is entered. The
c_dt column is of DATE datatype.
SQL> CREATE TABLE table_dt (c_id NUMBER, c_dt DATE);
Insert the date as a TIMESTAMP literal. Oracle drops the time zone information.
SQL> INSERT INTO table_dt VALUES(3, TIMESTAMP '2003-01-01 00:00:00 US/Pacific');
C_ID C_DT
---------- --------------------
1 01-JAN-2003 00:00:00
2 01-JAN-2003 00:00:00
3 01-JAN-2003 00:00:00
4 01-JAN-2003 00:00:00
Create a table table_ts with columns c_id and c_ts. The c_id column is of
NUMBER datatype and helps to identify the method by which the data is entered. The
c_ts column is of TIMESTAMP datatype.
SQL> CREATE TABLE table_ts(c_id NUMBER, c_ts TIMESTAMP);
Insert the same date and time as a TIMESTAMP WITH TIME ZONE literal. Oracle
converts it to a TIMESTAMP value, which means that the time zone information is
dropped.
SQL> INSERT INTO table_ts VALUES(3, TIMESTAMP '2003-01-01 2:00:00 -08:00');
C_ID C_TS
---------- -----------------------------
1 01-JAN-03 02:00:00.000000 AM
2 01-JAN-03 02:00:00.000000 AM
3 01-JAN-03 02:00:00.000000 AM
Note that the three methods result in the same value being stored.
Example 4–3 Inserting Data into the TIMESTAMP WITH TIME ZONE Datatype
Set the timestamp format.
SQL> ALTER SESSION SET NLS_TIMESTAMP_TZ_FORMAT='DD-MON-RR HH:MI:SSXFF AM TZR';
Create a table table_tstz with columns c_id and c_tstz. The c_id column is of
NUMBER datatype and helps to identify the method by which the data is entered. The
c_tstz column is of TIMESTAMP WITH TIME ZONE datatype.
SQL> CREATE TABLE table_tstz (c_id NUMBER, c_tstz TIMESTAMP WITH TIME ZONE);
Insert the same date and time as a TIMESTAMP literal. Oracle converts it to a
TIMESTAMP WITH TIME ZONE literal, which means that the session time zone is
appended to the TIMESTAMP value.
SQL> INSERT INTO table_tstz VALUES(2, TIMESTAMP '2003-01-01 2:00:00');
Insert the same date and time as a TIMESTAMP WITH TIME ZONE literal.
SQL> INSERT INTO table_tstz VALUES(3, TIMESTAMP '2003-01-01 2:00:00 -8:00');
C_ID C_TSTZ
---------- ------------------------------------
1 01-JAN-03 02:00.00:000000 AM -07:00
2 01-JAN-03 02:00:00.000000 AM -07:00
3 01-JAN-03 02:00:00.000000 AM -08:00
Note that the time zone is different for method 3, because the time zone information
was specified as part of the TIMESTAMP WITH TIME ZONE literal.
Example 4–4 Inserting Data into the TIMESTAMP WITH LOCAL TIME ZONE Datatype
Consider data that is being entered in Denver, Colorado, U.S.A., whose time zone is
UTC-7.
SQL> ALTER SESSION SET TIME_ZONE='-07:00';
Create a table table_tsltz with columns c_id and c_tsltz. The c_id column is
of NUMBER datatype and helps to identify the method by which the data is entered.
The c_tsltz column is of TIMESTAMP WITH LOCAL TIME ZONE datatype.
SQL> CREATE TABLE table_tsltz (c_id NUMBER, c_tsltz TIMESTAMP WITH LOCAL TIME ZONE);
Insert the same data as a TIMESTAMP WITH LOCAL TIME ZONE literal.
SQL> INSERT INTO table_tsltz VALUES(2, TIMESTAMP '2003-01-01 2:00:00');
Insert the same data as a TIMESTAMP WITH TIME ZONE literal. Oracle converts the
data to a TIMESTAMP WITH LOCAL TIME ZONE value. This means the time zone
that is entered (-08:00) is converted to the session time zone value (-07:00).
SQL> INSERT INTO table_tsltz VALUES(3, TIMESTAMP '2003-01-01 2:00:00 -08:00');
C_ID C_TSLTZ
---------- ------------------------------------
1 01-JAN-03 02.00.00.000000 AM
2 01-JAN-03 02.00.00.000000 AM
3 01-JAN-03 03.00.00.000000 AM
Note that the information that was entered as UTC-8 has been changed to the local
time zone, changing the hour from 2 to 3.
Interval Datatypes
Interval datatypes store time durations. They are used primarily with analytic
functions. For example, you can use them to calculate a moving average of stock
prices. You must use interval datatypes to determine the values that correspond to a
particular percentile. You can also use interval datatypes to update historical tables.
This section includes the following topics:
■ INTERVAL YEAR TO MONTH Datatype
■ INTERVAL DAY TO SECOND Datatype
■ Inserting Values into Interval Datatypes
year_precision is the number of digits in the YEAR datetime field. Accepted values
are 0 to 9. The default value of year_precision is 2.
Interval values can be specified as literals. There are many ways to specify interval
literals.The following is one example of specifying an interval of 123 years and 2
months.The year precision is 3.
day_precision is the number of digits in the DAY datetime field. Accepted values
are 0 to 9. The default is 2.
fractional_seconds_precision is the number of digits in the fractional part of
the SECOND datetime field. Accepted values are 0 to 9. The default is 6.
The following is one example of specifying an interval of 4 days, 5 hours, 12 minutes,
10 seconds, and 222 thousandths of a second. The fractional second precision is 3.
INTERVAL '4 5:12:10.222' DAY TO SECOND(3)
Interval values can be specified as literals. There are many ways to specify interval
literals.
interval (INTERVAL DAY TO SECOND and INTERVAL YEAR TO MONTH) data. You
can maintain the most precision in arithmetic operations by using a timestamp
datatype with an interval datatype.
You can use NUMBER constants in arithmetic operations on date and timestamp values.
Oracle internally converts timestamp values to date values before doing arithmetic
operations on them with NUMBER constants. This means that information about
fractional seconds is lost during operations that include both date and timestamp
values. Oracle interprets NUMBER constants in datetime and interval expressions as
number of days.
Each DATE value contains a time component. The result of many date operations
includes a fraction. This fraction means a portion of one day. For example, 1.5 days is
36 hours. These fractions are also returned by Oracle built-in SQL functions for
common operations on DATE data. For example, the built-in MONTHS_BETWEEN SQL
function returns the number of months between two dates. The fractional portion of
the result represents that portion of a 31-day month.
Oracle performs all timestamp arithmetic in UTC time. For TIMESTAMP WITH LOCAL
TIME ZONE data, Oracle converts the datetime value from the database time zone to
UTC and converts back to the database time zone after performing the arithmetic. For
TIMESTAMP WITH TIME ZONE data, the datetime value is always in UTC, so no
conversion is necessary.
See Also:
■ Oracle Database SQL Reference for detailed information about
datetime and interval arithmetic operations
■ "Datetime SQL Functions" on page 4-12 for information about
which functions cause implicit conversion to DATE
Datetime Comparisons
When you compare date and timestamp values, Oracle converts the data to the more
precise datatype before doing the comparison. For example, if you compare data of
TIMESTAMP WITH TIME ZONE datatype with data of TIMESTAMP datatype, Oracle
converts the TIMESTAMP data to TIMESTAMP WITH TIME ZONE, using the session
time zone.
The order of precedence for converting date and timestamp data is as follows:
1. DATE
2. TIMESTAMP
3. TIMESTAMP WITH LOCAL TIME ZONE
4. TIMESTAMP WITH TIME ZONE
For any pair of datatypes, Oracle converts the datatype that has a smaller number in
the preceding list to the datatype with the larger number.
Table 4–2 describes additional datetime functions. For more detailed descriptions, refer
to Oracle Database SQL Reference.
See Also:
■ "Date and Time Parameters" on page 3-15 for more information,
including examples
■ "NLS_DATE_FORMAT" on page 3-15
■ "NLS_TIMESTAMP_FORMAT" on page 3-18
■ "NLS_TIMESTAMP_TZ_FORMAT" on page 3-19
See Also:
■ "Choosing a Time Zone File" on page 4-15
■ "Setting the Session Time Zone" on page 4-19
Note: If you are already using the default time zone file, then it is
not practical to change to the smaller time zone file because the
database may contain data with time zones that are not part of the
smaller time zone file.
Oracle's time zone data is derived from the public domain information available at
ftp://elsie.nci.nih.gov/pub/. Oracle's time zone data may not reflect the
most recent data available at this site.
You can obtain a list of time zone names and time zone abbreviations from the time
zone file that is installed with your database by entering the following statement:
SELECT tzname, tzabbrev FROM V$TIMEZONE_NAMES;
For the default time zone file, this statement results in output similar to the following:
TZNAME TZABBREV
-------------------- ----------
Africa/Algiers LMT
Africa/Algiers PMT
Africa/Algiers WET
Africa/Algiers WEST
Africa/Algiers CET
Africa/Algiers CEST
Africa/Cairo LMT
Africa/Cairo EET
Africa/Cairo EEST
Africa/Casablanca LMT
Africa/Casablanca WET
Africa/Casablanca WEST
Africa/Casablanca CET
...
W-SU LMT
W-SU MMT
W-SU MST
W-SU MDST
W-SU S
W-SU MSD
W-SU MSK
W-SU EET
W-SU EEST
WET LMT
WET WEST
WET WET
There are 6 time zone abbreviations associated with the Africa/Algiers time zone, 3
abbreviations associated with the Africa/Cairo time zone, and 4 abbreviations
associated with the Africa/Casablanca time zone. The following table shows the time
zone abbreviations and their meanings.
Note that an abbreviation can be associated with more than one time zone. For
example, CET is associated with both Africa/Algiers and Africa/Casablanca, as well
as time zones in Europe.
If you want a list of time zones without repeating the time zone name for each
abbreviation, use the following query:
SELECT UNIQUE tzname FROM V$TIMEZONE_NAMES;
For the default time zone file, this results in output similar to the following:
TZNAME
--------------------
Africa/Algiers
Africa/Cairo
Africa/Casablanca
Africa/Ceuta
...
US/Pacific
US/Pacific-New
US/Samoa
UTC
W-SU
WET
The default time zone file contains more than 350 unique time zone names. The small
time zone file contains more than 180 unique time zone names.
See Also:
■ "Customizing Time Zone Data" on page 13-15
■ "Time Zone Names" on page A-23 for a list of valid Oracle time
zone names
Before upgrading, you can create a table tztab_back (note that column y here is
defined as VARCHAR2 to preserve the original value):
CREATE TABLE tztab_back(x NUMBER PRIMARY KEY, y VARCHAR2(256));
INSERT INTO tztab_back
SELECT x, TO_CHAR(y, 'YYYY-MM-DD HH24.MI.SSXFF TZR')
FROM tztab;
After upgrading, you need to update the data in the table tztab using the value in
tztab_back, as in the following:
UPDATE tztab t SET t.y =
(SELECT to_timestamp_tz(t1.y,'YYYY-MM-DD HH24.MI.SSXFF TZR')
FROM tztab_back t1
WHERE t.x=t1.x);
Or you can use the Export utility to export your data before the upgrade and then
import your data again after the upgrade. See the comments in the utltzuv2.sql
script for more information.
Although the transition rule changes may affect data of TIMESTAMP WITH LOCAL
TIME ZONE datatype, there is no way to upgrade the data. The data cannot be
upgraded because this type does not preserve the original time zone/region associated
with the data.
Time zone regions in Brazil and Israel may have frequent transition rules changes,
perhaps as often as every year. Use the time zone offset instead of the time zone region
name to avoid storing inconsistent data.
Customers using time zone regions that have been updated in version 2 of the time
zone files are required to update all Oracle9i Database clients and databases that will
communicate with an Oracle Database 10g server. This ensures that all environments
will have the same version of the time zone file, version 2. This is not a requirement for
other customers, but Oracle still recommends that you do so. Users who need to
update their time zone files to version 2 can find the following information on
OracleMetaLink (http://metalink.oracle.com):
■ readme.txt contains the list of time zone regions that have changed from
version 1 to version 2.
■ Actual time zone files for version 2 for the Oracle9i Database release.
■ utltzuv2.sql script that must be run on the server side to find out if the
database already has a column of type TIMESTAMP WITH TIME ZONE. It contains
time zones that have changed from version 1 to version 2.
Oracle Database 10g clients that communicate with Oracle Database 10g servers
automatically get version 2 of the time zone file, so there is no need to download the
new time zone file.
You can change the database time zone by using the SET TIME_ZONE clause of the
ALTER DATABASE statement. For example:
ALTER DATABASE SET TIME_ZONE='05:00';
ALTER DATABASE SET TIME_ZONE='Europe/London';
The ALTER DATABASE SET TIME_ZONE statement returns an error if the database
contains a table with a TIMESTAMP WITH LOCAL TIME ZONE column and the
column contains data.
The change does not take effect until the database has been shut down and restarted.
You can find out the database time zone by entering the following query:
SELECT dbtimezone FROM DUAL;
Note: Setting the session time zone does not affect the value
returned by the SYSDATE and SYSTIMESTAMP SQL function.
SYSDATE returns the date and time of the operating system on
which the database resides, taking into account the time zone of the
database server's operating system that was in effect when the
database was started.
You can change the time zone for a specific SQL session with the SET TIME_ZONE
clause of the ALTER SESSION statement.
You can find out the current session time zone by entering the following query:
SELECT sessiontimezone FROM DUAL;
See Also: "Time Zone Names" on page A-23 for a list of valid time
zones
Example 4–6 Comparing Daylight Saving Time Calculations Using TIMESTAMP WITH
TIME ZONE and TIMESTAMP
SELECT orderdate1 + INTERVAL '8' HOUR, orderdate2 + INTERVAL '8' HOUR
FROM global_orders;
ORDERDATE1+INTERVAL'8'HOUR ORDERDATE2+INTERVAL'8'HOUR
-------------------------- --------------------------
29-OCT-00 07.24.54.000000 AM 29-OCT-00 06.24.54.000000 AM AMERICA/NEW_YORK
This example shows the effect of adding 8 hours to the columns. The time period
includes a Daylight Saving Time boundary (a change from Daylight Saving Time to
standard time). The orderdate1 column is of TIMESTAMP datatype, which does not
use Daylight Saving Time information and thus does not adjust for the change that
took place in the 8-hour interval. The TIMESTAMP WITH TIME ZONE datatype does
adjust for the change, so the orderdate2 column shows the time as one hour earlier
than the time shown in the orderdate1 column.
Example 4–7 Comparing Daylight Saving Time Calculations Using TIMESTAMP WITH
LOCAL TIME ZONE and TIMESTAMP
The TIMESTAMP WITH LOCAL TIME ZONE datatype uses the value of TIME_ZONE
that is set for the session environment. The following statements set the value of the
TIME_ZONE session parameter and create a global_orders table. The global_
orders table has one column of TIMESTAMP datatype and one column of TIMESTAMP
WITH LOCAL TIME ZONE datatype.
ALTER SESSION SET TIME_ZONE='America/New_York';
CREATE TABLE global_orders ( orderdate1 TIMESTAMP(0),
orderdate2 TIMESTAMP(0) WITH LOCAL TIME ZONE );
INSERT INTO global_orders VALUES ( '28-OCT-00 11:24:54 PM',
'28-OCT-00 11:24:54 PM' );
Because a time zone region is associated with the datetime value for orderdate2, the
Oracle server uses the Daylight Saving Time rules for the region. Thus the output is
the same as in Example 4–6. There is a one-hour difference between the two
calculations because Daylight Saving Time is not calculated for the TIMESTAMP
datatype, and the calculation crosses a Daylight Saving Time boundary.
Example 4–8 Daylight Saving Time Is Not Calculated for Regions That Do Not Use
Daylight Saving Time
Set the time zone region to UTC. UTC does not use Daylight Saving Time.
ALTER SESSION SET TIME_ZONE='UTC';
The times are the same because Daylight Saving Time is not calculated for the UTC
time zone region.
This chapter explains sorting and searching for strings in an Oracle environment.
This chapter contains the following topics:
■ Overview of Oracle's Sorting Capabilities
■ Using Binary Sorts
■ Using Linguistic Sorts
■ Linguistic Sort Features
■ Case-Insensitive and Accent-Insensitive Linguistic Sorts
■ Performing Linguistic Comparisons
■ Using Linguistic Indexes
■ Searching Linguistic Strings
■ SQL Regular Expressions in a Multilingual Environment
When characters used in other languages are present, a binary sort usually does not
produce reasonable results. For example, an ascending ORDER BY query returns the
character strings ABC, ABZ, BCD, ÄBC, when Ä has a higher numeric value than B in the
character encoding scheme. A binary sort is not usually linguistically meaningful for
Asian languages that use ideographic characters.
Table 5–1 Sample Glyphs and Their Major and Minor Sort Values
Glyph Major Value Minor Value
a 15 5
A 15 10
ä 15 15
Ä 15 20
b 20 5
See Also:
■ "Canonical Equivalence" on page 5-7
■ "Supplementary Characters" on page 6-2
■ Characters ä and A are equal on the primary level and different on the secondary
and tertiary levels.
■ The primary and secondary level orders for the dash character - is 0. That is, it is
ignored on the primary and secondary levels. If a dash is compared with another
character whose primary level order is nonzero, for example, u, then no result for
the primary level is available because u is not compared with anything. In this
case, Oracle finds a difference between - and u only at the tertiary level.
The following list has been sorted on the primary level (resume comes before
resumes) and on the secondary level (strings without diacritics come before strings
with diacritics) and on the tertiary level (lower case comes before upper case):
resume
Resume
résumé
Résumé
resumes
Resumes
résumés
Résumés
Base Letters
Base letters are defined in a base letter table, which maps each letter to its base letter.
For example, a, A, ä, and Ä all map to a, which is the base letter. This concept is
particularly relevant for working with Oracle Text.
Ignorable Characters
Some characters can be ignored in a linguistic sort. These characters are called
ignorable characters. There are two kinds of ignorable characters: diacritics and
punctuation.
Examples of ignorable diacritics are:
■ ^, so that rôle is treated the same as role
■ The umlaut, so that naïve is treated the same as naive
An example of an ignorable punctuation character is the dash character -. If it is
ignored, then multi-lingual can be treated that same as multilingual and
e-mail can be treated the same as email.
Contracting Characters
Sorting elements usually consist of a single character, but in some locales, two or more
characters in a character string must be considered as a single sorting element during
sorting. For example, in traditional Spanish, the string ch is composed of two
characters. These characters are called contracting characters in multilingual linguistic
sorting and special combination letters in monolingual linguistic sorting.
Do not confuse a composed character with a contracting character. A composed
character like á can be decomposed into a and ', each with their own encoding. The
difference between a composed character and a contracting character is that a
composed character can be displayed as a single character on a terminal, while a
contracting character is used only for sorting, and its component characters must be
rendered separately.
Expanding Characters
In some locales, certain characters must be sorted as if they were character strings. An
example is the German character ß (sharp s). It is sorted exactly the same as the string
SS. Another example is that ö sorts as if it were oe, after od and before of. These
characters are known as expanding characters in multilingual linguistic sorting and
special letters in monolingual linguistic sorting. Just as with contracting characters,
the replacement string for an expanding character is meaningful only for sorting.
Context-Sensitive Characters
In Japanese, a prolonged sound mark that resembles an em dash — represents a length
mark that lengthens the vowel of the preceding character. The sort order depends on
the vowel that precedes the length mark. This is called context-sensitive sorting. For
example, after the character ka, the — length mark indicates a long a and is treated the
same as a, while after the character ki, the — length mark indicates a long i and is
treated the same as i. Transliterating this to Latin characters, a sort might look like
this:
kaa
ka— -- kaa and ka— are the same
kai -- kai follows ka- because i is after a
kia -- kia follows kai because i is after a
kii -- kii follows kia because i is after a
ki— -- kii and ki— are the same
Canonical Equivalence
Canonical equivalence is an attribute of a multilingual sort and describes how
equivalent code point sequences are sorted. If canonical equivalence is applied in a
particular linguistic sort, then canonically equivalent strings are treated as equal.
One Unicode code point can be equivalent to a sequence of base letter code points plus
diacritic code points. This is called the Unicode canonical equivalence. For example, ä
equals its base letter a and an umlaut. A linguistic flag, CANONICAL_EQUIVALENCE =
TRUE, indicates that all canonical equivalence rules defined in Unicode 4.0 need to be
applied in a specific linguistic sort. Oracle-defined linguistic sorts include the
appropriate setting for the canonical equivalence flag. You can set the flag to FALSE to
speed up the comparison and ordering functions if all the data is in its composed form.
For example, consider the following strings:
■ äa (a umlaut followed by a)
■ a¨b (a followed by umlaut followed by b)
■ äc (a umlaut followed by c)
If CANONICAL_EQUIVALENCE=FALSE, then the sort order of the strings is:
a¨b
äa
äc
See Also: "Creating a New Linguistic Sort with the Oracle Locale
Builder" on page 13-26 for more information about setting the
canonical equivalence flag
See Also: "Creating a New Linguistic Sort with the Oracle Locale
Builder" on page 13-26 for more information about setting the
reverse secondary flag
See Also: "Creating a New Linguistic Sort with the Oracle Locale
Builder" on page 13-26 for more information about setting the
SWAP_WITH_NEXT flag
Special Letters
Special letters is a term used in monolingual sorts. They are called expanding
characters in multilingual sorts.
Upper
-----
GROSSE
WORD
------------
GROSSE
Große
große
WORD
------------
GROSSE
WORD
------------
GROSSE
Große
große
See Also:
■ "NLS_SORT" on page 3-29
■ "NLS_COMP" on page 3-30
Example 5–1 Linguistic Sort Using Base Letters, Diacritics, Punctuation, and Case
Information
The following list has been sorted using information about base letters, diacritics,
punctuation, and case:
blackbird
black bird
black-bird
Blackbird
Black-bird
blackbîrd
bläckbird
black-bird and Black-bird have the same value in the sort, because the only
different between them is case. They could appear interchanged in the list.
Blackbird and blackbird also have the same value in the sort and could appear
interchanged in the list.
LETTER
-----------
ä
a
A
Z
The default value of NLS_SORT is BINARY. Use the following statement to do a binary
sort of the characters in table test2:
SELECT * FROM test2 ORDER BY letter;
The following table shows the sort orders that result from setting NLS_SORT to
BINARY, BINARY_CI, and BINARY_AI.
You can use binary sorts for better performance when the character set is US7ASCII or
another character set that has the same sort order as the binary sorts.
The following table shows the sort orders that result from German sorts for the table.
A German sort places lowercase letters before uppercase letters, and ä occurs before Z.
When the sort ignores both case and diacritics (GERMAN_AI), ä appears with the other
characters whose base letter is a.
Note that a binary sort results in À voir being at the end of the list.
Perform a monolingual German sort by using the NLSSORT function with the NLS_
SORT parameter set to german.
SQL> SELECT * FROM test2 ORDER BY NLSSORT(name, 'NLS_SORT=german');
The output from the German sort shows the new character string last in the list of
entries because the characters are not recognized in a German sort.
Perform a multilingual sort by entering the following statement:
SQL> SELECT * FROM test2 ORDER BY NLSSORT(name, 'NLS_SORT=generic_m');
The output shows the new character string after Diet, following ISO sorting rules.
See Also:
■ "The NLSSORT Function" on page 9-7
■ "NLS_SORT" on page 3-29 for more information about setting
and changing the NLS_SORT parameter
ENAME
----------------------
Mc Calla
MCAfee
McCoye
Mccathye
McCafeé
5 rows selected
ENAME
----------------------
McCoye
1 row selected
ENAME
----------------------
McCoye
Mccathye
2 rows selected
ENAME
----------------------
McCoye
Mccathye
McCafeé
3 rows selected
ENAME
----------------------
McAfee
Mcafee
McCoy
3 rows selected
However, with a case-insensitive setting, McAfee and Mcafee are the same:
SQL> ALTER SESSION SET NLS_COMP=LINGUISTIC;
SQL> ALTER SESSION SET NLS_SORT=BINARY_CI;
SQL> SELECT DISTINCT ename FROM emp2;
ENAME
----------------------
McAfee
McCoy
2 rows selected
In this example, either McAfee or Mcafee could be returned from the DISTINCT
operation. There is no guarantee exactly which one will be picked.
ENAME
----------------------
Cindy
Chad
Clara
3 rows selected
In a database session with the linguistic rule set to traditional Spanish, XSPANISH, ch
is treated as one character. So the letter c in Chad is different than the letter C in Cindy
and Clara:
SQL> ALTER SESSION SET NLS_COMP=LINGUISTIC;
SQL> ALTER SESSION SET NLS_SORT=XSPANISH;
SQL> SELECT ename FROM emp3 WHERE ename LIKE 'C%';
ENAME
----------------------
Cindy
Clara
2 rows selected
Changes
---------------------
charatter
/*The NOT NULL in the CREATE TABLE statement ensures that the index is used*/
After the index has been created, enter a SELECT statement similar to the following:
SELECT * FROM my_table ORDER BY name;
It returns the result much faster than the same SELECT statement without an index.
The rest of this section contains the following topics:
■ Linguistic Indexes for Multiple Languages
■ Requirements for Using Linguistic Indexes
See Also:
■ Oracle Database Concepts
■ Oracle Database SQL Reference for more information about
function-based indexes
Oracle chooses the index based on the NLS_SORT session parameter or the
arguments of the NLSSORT function specified in the ORDER BY clause. For
example, if the NLS_SORT session parameter is set to FRENCH, then Oracle uses
french_index. When it is set to GERMAN, Oracle uses german_index.
■ Build a single linguistic index for all languages. This requires a language column
(LANG_COL in "Example: Setting Up a French Linguistic Index" on page 5-19) to be
used as a parameter of the NLSSORT function. The language column contains
NLS_LANGUAGE values for the data in the column on which the index is built. The
following example builds a single linguistic index for multiple languages. With
this index, the rows with the same values for NLS_LANGUAGE are sorted together.
CREATE INDEX i ON t (NLSSORT(col, 'NLS_SORT=' || LANG_COL));
Queries choose an index based on the argument of the NLSSORT function specified
in the ORDER BY clause.
■ Build a single linguistic index for all languages using one of the multilingual
linguistic sorts such as GENERIC_M or FRENCH_M. These indexes sort characters
according to the rules defined in ISO 14651. For example:
CREATE INDEX i on t (NLSSORT(col,
'NLS_SORT=GENERIC_M');
Specify NOT NULL in a WHERE Clause If the Column Was Not Declared NOT NULL
When you want to use the ORDER BY column_name clause with a column that has a
linguistic index, include a WHERE clause like the following:
WHERE NLSSORT(column_name) IS NOT NULL
This WHERE clause is not necessary if the column has already been defined as a NOT
NULL column in the schema.
Note: The SQL functions MAX( ) and MIN( ), and also the LIKE
operator, cannot use linguistic indexes when NLS_COMP is set to
LINGUISTIC.
See Also:
■ "Linguistic Sort Features" on page 5-5
■ "SQL Regular Expressions in a Multilingual Environment" on
page 5-19
See Also:
■ Oracle Database Application Developer's Guide - Fundamentals for
more information about regular expression syntax
■ Oracle Database SQL Reference for more information about
REGEX SQL functions
See Also:
■ Oracle Database Application Developer's Guide - Fundamentals for
more information about regular expression syntax
■ Oracle Database SQL Reference for more information about
REGEX SQL functions
Overview of Unicode
Dealing with many different languages in the same application or database has been
complicated and difficult for a long time. To overcome the limitations of existing
character encodings, several organizations began working on the creation of a global
character set in the late 1980s. The need for this became even greater with the
development of the World Wide Web in the mid-1990s. The Internet has changed how
companies do business, with an emphasis on the global market that has made a
universal character set a major requirement. A global character set needs to fulfill the
following conditions:
■ Contain all major living scripts
■ Support legacy data and implementations
■ Be simple enough that a single implementation of an application is sufficient for
worldwide use
A global character set should also have the following capabilities:
■ Support multilingual users and organizations
■ Conform to international standards
■ Enable worldwide interchange of data
This global character set exists, is in wide use, and is called Unicode.
What is Unicode?
Unicode is a universal encoded character set that enables information from any
language to be stored using a single character set. Unicode provides a unique code
value for every character, regardless of the platform, program, or language.
The Unicode standard has been adopted by many software and hardware vendors.
Many operating systems and browsers now support Unicode. Unicode is required by
standards such as XML, Java, JavaScript, LDAP, and WML. It is also synchronized
with the ISO/IEC 10646 standard.
Oracle started supporting Unicode as a database character set in Oracle Database 7. In
Oracle Database 10g, Unicode support has been expanded. Oracle Database 10g,
Release 2 supports Unicode 4.0.
Supplementary Characters
The first version of Unicode was a 16-bit, fixed-width encoding that used two bytes to
encode each character. This allowed 65,536 characters to be represented. However,
more characters need to be supported, especially additional CJK ideographs that are
important for the Chinese, Japanese, and Korean markets.
Unicode 4.0 defines supplementary characters to meet this need. It uses two 16-bit
code points (also known as supplementary characters) to represent a single character.
This enables an additional 1,048,576 characters to be defined. The Unicode 4.0
standard defines 45,960 supplementary characters.
Adding supplementary characters increases the complexity of Unicode, but it is less
complex than managing several different encodings in the same configuration.
Unicode Encodings
The Unicode standard encodes characters in different ways: UTF-8, UCS-2, and
UTF-16. Conversion between different Unicode encodings is a simple bit-wise
operation that is defined in the Unicode standard.
This section contains the following topics:
■ UTF-8 Encoding
■ UCS-2 Encoding
■ UTF-16 Encoding
■ Examples: UTF-16, UTF-8, and UCS-2 Encoding
UTF-8 Encoding
UTF-8 is the 8-bit encoding of Unicode. It is a variable-width encoding and a strict
superset of ASCII. This means that each and every character in the ASCII character set
is available in UTF-8 with the same code point values. One Unicode character can be 1
byte, 2 bytes, 3 bytes, or 4 bytes in UTF-8 encoding. Characters from the European
scripts are represented in either 1 or 2 bytes. Characters from most Asian scripts are
represented in 3 bytes. Supplementary characters are represented in 4 bytes.
UTF-8 is the Unicode encoding supported on UNIX platforms and used for HTML and
most Internet browsers. Other environments such as Windows and Java use UCS-2
encoding.
The benefits of UTF-8 are as follows:
■ Compact storage requirement for European scripts because it is a strict superset of
ASCII
■ Ease of migration between ASCII-based characters sets and UTF-8
See Also:
■ "Supplementary Characters" on page 6-2
■ Table B–2, " Unicode Character Code Ranges for UTF-8
Character Codes" on page B-2
UCS-2 Encoding
UCS-2 is a fixed-width, 16-bit encoding. Each character is 2 bytes. UCS-2 is the
Unicode encoding used by Java and Microsoft Windows NT 4.0. UCS-2 supports
characters defined for Unicode 3.0, so there is no support for supplementary
characters.
The benefits of UCS-2 over UTF-8 are as follows:
■ More compact storage for Asian scripts because all characters are two bytes
■ Faster string processing because characters are fixed-width
■ Better compatibility with Java and Microsoft clients
UTF-16 Encoding
UTF-16 encoding is the 16-bit encoding of Unicode. UTF-16 is an extension of UCS-2
because it supports the supplementary characters by using two UCS-2 code points for
each supplementary character. UTF-16 is a strict superset of UCS-2.
One character can be either 2 bytes or 4 bytes in UTF-16. Characters from European
and most Asian scripts are represented in 2 bytes. Supplementary characters are
represented in 4 bytes. UTF-16 is the main Unicode encoding used by Microsoft
Windows 2000.
The benefits of UTF-16 over UTF-8 are as follows:
■ More compact storage for Asian scripts because most of the commonly used Asian
characters are represented in two bytes.
■ Better compatibility with Java and Microsoft clients
See Also:
■ "Supplementary Characters" on page 6-2
■ Table B–1, " Unicode Character Code Ranges for UTF-16
Character Codes" on page B-1
You can create a Unicode database that enables you to store UTF-8 encoded characters
as SQL CHAR datatypes (CHAR, VARCHAR2, CLOB, and LONG).
If you prefer to implement Unicode support incrementally or if you need to support
multilingual data only in certain columns, then you can store Unicode data in either
the UTF-16 or UTF-8 encoding form in SQL NCHAR datatypes (NCHAR, NVARCHAR2,
and NCLOB). The SQL NCHAR datatypes are called Unicode datatypes because they are
used only for storing Unicode data.
The following sections explain how to use the two Unicode solutions and how to
choose between them:
■ Enabling Multilingual Support with Unicode Databases
■ Enabling Multilingual Support with Unicode Datatypes
■ How to Choose Between a Unicode Database and a Unicode Datatype Solution
■ Comparing Unicode Character Sets for Database and Datatype Solutions
Note: Specify the database character set when you create the
database.
The encoding used in the SQL NCHAR datatypes is the national character set specified
for the database. You can specify one of the following Oracle character sets as the
national character set:
■ AL16UTF16
This is the default character set for SQL NCHAR datatypes. The character set
encodes Unicode data in the UTF-16 encoding. It supports supplementary
characters, which are stored as four bytes.
■ UTF8
When UTF8 is specified for SQL NCHAR datatypes, the data stored in the SQL
datatypes is in UTF-8 encoding.
You can specify the national character set for the SQL NCHAR datatypes when you
create a database using the CREATE DATABASE statement with the NATIONAL
Table 6–4 Character Set Advantages and Disadvantages for a Unicode Database Solution
Database
Character Set Advantages Disadvantages
AL32UTF8 ■ Supplementary characters are stored in 4 ■ You cannot specify the length of SQL CHAR types in
bytes, so there is no data conversion number of UCS-2 code points for supplementary
when supplementary characters are characters. Supplementary characters are treated as
retrieved and inserted if the client setting one code point rather than the standard two code
is UTF-8. points.
■ The storage for supplementary characters ■ The binary order for SQL CHAR columns is different
requires less disk space in AL32UTF8 from the binary order of SQL NCHAR columns when
than in UTF8. the data consists of supplementary characters. As a
result, CHAR columns and NCHAR columns do not
always have the same sort for identical strings.
UTF8 ■ You can specify the length of SQL CHAR ■ Supplementary characters are stored as 6 bytes
types in number of UCS-2 code points. instead of the 4 bytes defined by Unicode 4.0. As a
result, Oracle has to convert data for
■ The binary order of the SQL CHAR
supplementary characters if the client setting is
columns is always the same as the binary
UTF-8.
order of the SQL NCHAR columns when
the data consists of the same
supplementary characters. As a result,
CHAR columns and NCHAR columns have
the same sort for identical strings.
UTFE ■ This is the only Unicode character set for ■ Supplementary character are stored as 8 bytes (two
the EBCDIC platform. 4-byte sequences) instead of the 5 bytes defined by
the Unicode standard. As a result, Oracle has to
■ You can specify the length of SQL CHAR
convert data for those supplementary characters.
types in number of UCS-2 code points.
■ The binary order of the SQL CHAR ■ UTFE is not a standard encoding in the Unicode
columns is always the same as the binary standard. As a result, clients requiring standard
order of the SQL NCHAR columns when UTF-8 encoding must convert data from UTFE to
the data consists of the same the standard encoding when data is retrieved and
supplementary characters. As a result, inserted.
CHAR columns and NCHAR columns have
the same sort for identical strings.
Table 6–5 contains advantages and disadvantages of different character sets for a
Unicode datatype solution. The Oracle character sets that can be national character sets
are AL16UTF16 and UTF8. The default is AL16UTF16.
Table 6–5 Character Set Advantages and Disadvantages for a Unicode Datatype Solution
National
Character Set Advantages Disadvantages
AL16UTF16 ■ Asian data in AL16UTF16 is usually more ■ European ASCII data requires more disk
compact than in UTF8. As a result, you save disk space to store in AL16UTF16 than in UTF8.
space and have less disk I/O when most of the If most of your data is European data, then
multilingual data stored in the database is Asian it uses more disk space than if it were UTF8
data. data.
■ It is usually faster to process strings encoded in ■ The maximum lengths for NCHAR and
the AL16UTF16 character set than strings encoded NVARCHAR2 are 1000 and 2000 characters,
in UTF8 because Oracle processes most characters which is less than the lengths for NCHAR
in an AL16UTF16 encoded string as fixed-width (2000) and NVARCHAR2 (4000) in UTF8.
characters.
■ The maximum length limits for the NCHAR and
NVARCHAR2 columns are 1000 and 2000
characters, respectively. Because the data is
fixed-width, the lengths are guaranteed.
UTF8 ■ European data in UTF8 is usually more compact ■ Asian data requires more disk space to
than in AL16UTF16. As a result, you save disk store in UTF8 than in AL16UTF16. If most
space and have better response time when most of of your data is Asian data, then disk space
the multilingual data stored in the database is usage is not less efficient than when the
European data. character set is AL16UTF16.
■ The maximum lengths for the NCHAR and ■ Although you can specify larger length
NVARCHAR2 columns are 2000 and 4000 characters limits for NCHAR and NVARCHAR, you are
respectively, which is more than those for NCHAR not guaranteed to be able to insert the
(1000) and NVARCHAR2 (2000) in AL16UTF16. number of characters specified by these
Although the maximum lengths of the NCHAR and limits. This is because UTF8 allows
NVARCHAR2 columns are larger in UTF8, the variable-width characters.
actual storage size is still bound by the byte limits
■ It is usually slower to process strings
of 2000 and 4000 bytes, respectively. For example,
encoded in UTF8 than strings encoded in
you can store 4000 UTF8 characters in an
AL16UTF16 because UTF8 encoded strings
NVARCHAR2 column if all the characters are single
consist of variable-width characters.
byte, but only 4000/3 characters if all the
characters are three bytes.
■ The database is very large and the scheduled downtime is short. Fast migration of
the database to Unicode is vital. Because the database is in US7ASCII, the easiest
and fastest way of enabling the database to support Unicode is to switch the
database character set to UTF8 by running the CSALTER script. No data
conversion is required because US7ASCII is a subset of UTF8.
■ Because most of the code is written in Java and PL/SQL, changing the database
character set to UTF8 is unlikely to break existing code. Unicode support is
automatically enabled in the application.
■ Because the application supports French, German, and Japanese, there are few
supplementary characters. Both AL32UTF8 and UTF8 are suitable.
Example 6–5 Unicode Solution with a Unicode Database and Unicode Datatypes
A Japanese company wants to develop a new Java application. The company expects
that the application will support as many languages as possible in the long run.
■ In order to store documents as is, the company decided to use the BLOB datatype
to store documents of multiple languages.
■ The company may also want to generate UTF-8 XML documents from the
relational data for business-to-business data exchange.
■ The back-end has Windows applications written in C/C++ using ODBC to access
the Oracle database.
In this case, the typical solution is to create a Unicode database using AL32UTF8 as the
database character set and use the SQL NCHAR datatypes to store multilingual data.
The national character set should be set to AL16UTF16. The reasons for this solution
are as follows:
■ When documents of different languages are stored in BLOB format, Oracle Text
requires the database character set to be one of the UTF-8 character sets. Because
the applications may retrieve relational data as UTF-8 XML format (where
supplementary characters are stored as four bytes), AL32UTF8 should be used as
the database character set to avoid data conversion when UTF-8 data is retrieved
or inserted.
■ Because applications are new and written in both Java and Windows C/C++, the
company should use the SQL NCHAR datatype for its relational data. Both Java and
Windows support the UTF-16 character datatype, and the length of a character
string is always measured in the number of characters.
■ If most of the data is for Asian languages, then AL16UTF16 should be used with
the SQL NCHAR datatypes because AL16UTF16 offers better storage efficiency.
When you use CHAR and VARCHAR2 datatypes for storing multilingual data, the
maximum length specified for each column is, by default, in number of bytes. If the
database needs to support Thai, Arabic, or multibyte languages such as Chinese and
Japanese, then the maximum lengths of the CHAR, VARCHAR, and VARCHAR2 columns
may need to be extended. This is because the number of bytes required to encode these
languages in UTF8 or AL32UTF8 may be significantly larger than the number of bytes
for encoding English and Western European languages. For example, one Thai
character in the Thai character set requires 3 bytes in UTF8 or AL32UTF8. In addition,
the maximum column lengths for CHAR, VARCHAR, and VARCHAR2 datatypes are 2000
bytes, 4000 bytes, and 4000 bytes respectively. If applications need to store more than
4000 bytes, then they should use the CLOB datatype.
You can also create a view to select the data of the current language. For example:
ALTER TABLE scott.product_information ADD (language_id VARCHAR2(50)):
Creating Multilexers
The first step in creating the multilexer is the creation of language-specific lexer
preferences for each language supported. The following example creates English,
German, and Japanese lexers with PL/SQL procedures:
ctx_ddl.create_preference('english_lexer', 'basic_lexer');
ctx_ddl.set_attribute('english_lexer','index_themes','yes');
ctx_ddl.create_preference('german_lexer', 'basic_lexer');
ctx_ddl.set_attribute('german_lexer','composite','german');
ctx_ddl.set_attribute('german_lexer','alternate_spelling','german');
ctx_ddl.set_attribute('german_lexer','mixed_case','yes');
ctx_ddl.create_preference('japanese_lexer', 'JAPANESE_VGRAM_LEXER');
After the language-specific lexer preferences are created, they need to be gathered
together under a single multilexer preference. First, create the multilexer preference,
using the MULTI_LEXER object:
ctx_ddl.create_preference('global_lexer','multi_lexer');
Now add the language-specific lexers to the multilexer preference using the add_
sub_lexer call:
To create an index for this table, use the multilexer preference and specify the name of
the language column:
CREATE INDEX globalx ON globaldoc(text)
indextype IS ctxsys.context
parameters ('lexer
global_lexer
language
column
language');
You can put word-processing or spreadsheet documents into the table and specify
binary in the format column. For documents in HTML, XML and text format, you
can put them into the table and specify text in the format column.
Because there is a column in which to specify the character set, you can store text
documents in different character sets.
When you create the index, specify the names of the format and character set columns:
CREATE INDEX globalx ON globaldoc(text)
indextype is ctxsys.context
parameters ('filter inso_filter
lexer global_lexer
language column language
You can use the charset_filter if all documents are in text format. The charset_
filter converts data from the character set specified in the charset column to the
database character set.
This chapter describes how to use Oracle's database access products with Unicode. It
contains the following topics:
■ Overview of Programming with Unicode
■ SQL and PL/SQL Programming with Unicode
■ OCI Programming with Unicode
■ Pro*C/C++ Programming with Unicode
■ JDBC Programming with Unicode
■ ODBC and OLE DB Programming with Unicode
■ XML Programming with Unicode
Figure 7–1 shows how the database access products can access the database.
· OLE DB
· ODBC Pro*C/C++
· Oracle Data Provider
for .NET JDBC
Oracle
Net
Oracle
PL/SQL
Oracle Net on TCP/IP
SQL
Java
The Oracle Call Interface (OCI) is the lowest level API that the rest of the client-side
database access products use. It provides a flexible way for C/C++ programs to access
Unicode data stored in SQL CHAR and NCHAR datatypes. Using OCI, you can
programmatically specify the character set (UTF-8, UTF-16, and others) for the data to
be inserted or retrieved. It accesses the database through Oracle Net.
Oracle Pro*C/C++ enables you to embed SQL and PL/SQL in your programs. It uses
OCI's Unicode capabilities to provide UTF-16 and UTF-8 data access for SQL CHAR
and NCHAR datatypes.
The Oracle ODBC driver enables C/C++, Visual Basic, and VBScript programs
running on Windows platforms to access Unicode data stored in SQL CHAR and NCHAR
datatypes of the database. It provides UTF-16 data access by implementing the
SQLWCHAR interface specified in the ODBC standard specification.
The Oracle Provider for OLE DB enables C/C++, Visual Basic, and VBScript programs
running on Windows platforms to access Unicode data stored in SQL CHAR and NCHAR
datatypes. It provides UTF-16 data access through wide string OLE DB datatypes.
The Oracle Data Provider for .NET enables programs running in any .NET
programming environment on Windows platforms to access Unicode data stored in
SQL CHAR and NCHAR datatypes. It provides UTF-16 data access through Unicode
datatypes.
Oracle JDBC drivers are the primary Java programmatic interface for accessing an
Oracle database. Oracle provides the following JDBC drivers:
■ The JDBC OCI driver that is used by Java applications and requires the OCI
library
■ The JDBC thin driver, which is a pure Java driver that is primarily used by Java
applets and supports the Oracle Net protocol over TCP/IP
■ The JDBC server-side thin driver, a pure Java driver used inside Java stored
procedures to connect to another Oracle server
■ The JDBC server-side internal driver that is used inside the Oracle server to access
the data in the database
All drivers support Unicode data access to SQL CHAR and NCHAR datatypes in the
database.
The PL/SQL and SQL engines process PL/SQL programs and SQL statements on
behalf of client-side programs such as OCI and server-side PL/SQL stored procedures.
They allow PL/SQL programs to declare CHAR, VARCHAR2, NCHAR, and NVARCHAR2
variables and to access SQL CHAR and NCHAR datatypes in the database.
The following sections describe how each of the database access products supports
Unicode data access to an Oracle database and offer examples for using those
products:
■ SQL and PL/SQL Programming with Unicode
■ OCI Programming with Unicode
■ Pro*C/C++ Programming with Unicode
■ JDBC Programming with Unicode
■ ODBC and OLE DB Programming with Unicode
See Also:
■ Oracle Database SQL Reference
■ PL/SQL User's Guide and Reference
For example, if the national character set is UTF8, then the maximum byte length is 30
characters times 3 bytes for each character, or 90 bytes.
The national character set, which is used for all NCHAR datatypes, is defined when the
database is created. The national character set can be either UTF8 or AL16UTF16. The
default is AL16UTF16.
The maximum column size allowed is 2000 characters when the national character set
is UTF8 and 1000 when it is AL16UTF16. The actual data is subject to the maximum
byte limit of 2000. The two size constraints must be satisfied at the same time. In
PL/SQL, the maximum length of NCHAR data is 32767 bytes. You can define an NCHAR
variable of up to 32767 characters, but the actual data cannot exceed 32767 bytes. If
you insert a value that is shorter than the column length, then Oracle pads the value
with blanks to whichever length is smaller: maximum character length or maximum
byte length.
value in the column exactly as you specify it, if the value does not exceed the column's
maximum length. Oracle does not pad the string value to the maximum length.
The maximum column size allowed is 4000 characters when the national character set
is UTF8 and 2000 when it is AL16UTF16. The maximum length of an NVARCHAR2
column in bytes is 4000. Both the byte limit and the character limit must be met, so the
maximum number of characters that is actually allowed in an NVARCHAR2 column is
the number of characters that can be written in 4000 bytes.
In PL/SQL, the maximum length for an NVARCHAR2 variable is 32767 bytes. You can
define NVARCHAR2 variables up to 32767 characters, but the actual data cannot exceed
32767 bytes.
The following statement creates a table with one NVARCHAR2 column whose
maximum length in characters is 2000 and maximum length in bytes is 4000.
CREATE TABLE table2 (column2 NVARCHAR2(2000));
character type conversion. When this parameter is set to TRUE, any SQL statements
that result in data loss return an ORA-12713 error and the corresponding operation is
stopped. When this parameter is set to FALSE, data loss is not reported and the
unconvertible characters are replaced with replacement characters. The default value
is TRUE. This parameter works for both implicit and explicit conversion.
In PL/SQL, when data loss occurs during conversion of SQL CHAR and NCHAR
datatypes, the LOSSY_CHARSET_CONVERSION exception is raised for both implicit
and explicit conversion.
Example 7–1 Populating the Customers Table Using the TO_NCHAR Function
The TO_NCHAR function converts the data at run time, while the N function converts
the data at compilation time.
INSERT INTO customers VALUES (1000,
TO_NCHAR('John Smith'),N'500 Oracle Parkway',sysdate);
Example 7–2 Selecting from the Customer Table Using the TO_CHAR Function
The following statement converts the values of name from characters in the national
character set to characters in the database character set before selecting them according
to the LIKE clause:
SELECT name FROM customers WHERE TO_CHAR(name) LIKE '%Sm%';
Example 7–3 Selecting from the Customer Table Using the TO_DATE Function
Using the N function shows that either NCHAR or CHAR data can be passed as
parameters for the TO_DATE function. The datatypes can mixed because they are
converted at run time.
DECLARE
ndatestring NVARCHAR2(20) := N'12-SEP-1975';
ndstr NVARCHAR2(50);
BEGIN
SELECT name INTO ndstr FROM customers
WHERE (birthdate)> TO_DATE(ndatestring, 'DD-MON-YYYY', N'NLS_DATE_LANGUAGE =
AMERICAN');
END;
The space character ' ' is converted to the corresponding character in the NCHAR
character set and then padded to the right of name until the total display length
reaches 100.
OCIEnv *envhp;
status = OCIEnvNlsCreate((OCIEnv **)&envhp,
(ub4)0,
(void *)0,
(void *(*) ()) 0,
(void *(*) ()) 0,
(void(*) ()) 0,
(size_t) 0,
(void **)0,
(ub2)OCI_UTF16ID, /* Metadata and SQL CHAR character set */
(ub2)OCI_UTF16ID /* SQL NCHAR character set */);
The Unicode mode, in which the OCI_UTF16 flag is used with the OCIEnvCreate()
function, is deprecated.
When OCI_UTF16ID is specified for both SQL CHAR and SQL NCHAR character sets, all
metadata and bound and defined data are encoded in UTF-16. Metadata includes SQL
statements, user names, error messages, and column names. Thus, all inherited
operations are independent of the NLS_LANG setting, and all metatext data parameters
(text*) are assumed to be Unicode text datatypes (utext*) in UTF-16 encoding.
To prepare the SQL statement when the OCIEnv() function is initialized with the
OCI_UTF16ID character set ID, call the OCIStmtPrepare() function with a
(utext*) string. The following example runs on the Windows platform only. You
may need to change wchar_t datatypes for other platforms.
const wchar_t sqlstr[] = L"SELECT * FROM ENAME=:ename";
...
OCIStmt* stmthp;
sts = OCIHandleAlloc(envh, (void **)&stmthp, OCI_HTYPE_STMT, 0,
NULL);
status = OCIStmtPrepare(stmthp, errhp,(const text*)sqlstr,
wcslen(sqlstr), OCI_NTV_SYNTAX, OCI_DEFAULT);
To bind and define data, you do not have to set the OCI_ATTR_CHARSET_ID attribute
because the OCIEnv() function has already been initialized with UTF-16 character set
IDs. The bind variable names also must be UTF-16 strings.
/* Inserting Unicode data */
OCIBindByName(stmthp1, &bnd1p, errhp, (const text*)L":ename",
(sb4)wcslen(L":ename"),
(void *) ename, sizeof(ename), SQLT_STR, (void
*)&insname_ind,
(ub2 *) 0, (ub2 *) 0, (ub4) 0, (ub4 *)0,
OCI_DEFAULT);
OCIAttrSet((void *) bnd1p, (ub4) OCI_HTYPE_BIND, (void *)
&ename_col_len,
(ub4) 0, (ub4)OCI_ATTR_MAXDATA_SIZE, errhp);
...
/* Retrieving Unicode data */
OCIDefineByPos (stmthp2, &dfn1p, errhp, (ub4)1, (void *)ename,
(sb4)sizeof(ename), SQLT_STR, (void *)0, (ub2 *)0,
(ub2*)0, (ub4)OCI_DEFAULT);
Data Integrity
You can lose data during conversion if you call an OCI API inappropriately. If the
server and client character sets are different, then you can lose data when the
destination character set is a smaller set than the source character set. You can avoid
this potential problem if both character sets are Unicode character sets (for example,
UTF8 and AL16UTF16).
When you bind or define SQL NCHAR datatypes, you should set the OCI_ATTR_
CHARSET_FORM attribute to SQLCS_NCHAR. Otherwise, you can lose data because the
data is converted to the database character set before converting to or from the
national character set. This occurs only if the database character set is not Unicode.
This method is the simplest and quickest way, but it may not be accurate and can
waste memory. It is applicable to any character set combination. For example, for
UTF-16 data binding and defining, the following example calculates the client buffer:
ub2 csid = OCI_UTF16ID;
oratext *selstmt = "SELECT ename FROM emp";
counter = 1;
...
OCIStmtPrepare(stmthp, errhp, selstmt, (ub4)strlen((char*)selstmt),
OCI_NTV_SYNTAX, OCI_DEFAULT);
OCIStmtExecute ( svchp, stmthp, errhp, (ub4)0, (ub4)0,
(CONST OCISnapshot*)0, (OCISnapshot*)0,
OCI_DESCRIBE_ONLY);
OCIParamGet(stmthp, OCI_HTYPE_STMT, errhp, &myparam, (ub4)counter);
OCIAttrGet((void*)myparam, (ub4)OCI_DTYPE_PARAM, (void*)&col_width,
(ub4*)0, (ub4)OCI_ATTR_DATA_SIZE, errhp);
...
maxenamelen = (col_width + 1) * sizeof(utext);
cbuf = (utext*)malloc(maxenamelen);
...
OCIDefineByPos(stmthp, &dfnp, errhp, (ub4)1, (void *)cbuf,
(sb4)maxenamelen, SQLT_STR, (void *)0, (ub2 *)0,
(ub2*)0, (ub4)OCI_DEFAULT);
OCIAttrSet((void *) dfnp, (ub4) OCI_HTYPE_DEFINE, (void *) &csid,
(ub4) 0, (ub4)OCI_ATTR_CHARSET_ID, errhp);
OCIStmtFetch(stmthp, errhp, 1, OCI_FETCH_NEXT, OCI_DEFAULT);
...
If bound buffers are of the utext datatype, then you should add a cast (text*)
when OCIBind() or OCIDefine() is called. The value of the OCI_ATTR_
MAXDATA_SIZE attribute is usually determined by the column size of the server
character set because this size is only used to allocate temporary buffer space for
conversion on the server when you perform binding operations.
■ Call OCIBind() or OCIDefine() with the NLS_LANG character set specified as
UTF8 or AL32UTF8.
UTF8 or AL32UTF8 can be set in the NLS_LANG environment variable. You call
OCIBind() and OCIDefine() in exactly the same manner as when you are not
using Unicode. Set the NLS_LANG environment variable to UTF8 or AL32UTF8
and run the following OCI program:
...
oratext ename[100]; /* enough buffer size for ENAME */
...
/* Inserting Unicode data */
OCIBindByName(stmthp1, &bnd1p, errhp, (oratext*)":ENAME",
(sb4)strlen((char *)":ENAME"), (void *) ename, sizeof(ename),
SQLT_STR, (void *)&insname_ind, (ub2 *) 0, (ub2 *) 0,
(ub4) 0, (ub4 *)0, OCI_DEFAULT);
OCIAttrSet((void *) bnd1p, (ub4) OCI_HTYPE_BIND, (void *) &ename_col_len,
(ub4) 0, (ub4)OCI_ATTR_MAXDATA_SIZE, errhp);
...
/* Retrieving Unicode data */
OCIDefineByPos (stmthp2, &dfn1p, errhp, (ub4)1, (void *)ename,
(sb4)sizeof(ename), SQLT_STR, (void *)0, (ub2 *)0, (ub2*)0,
(ub4)OCI_DEFAULT);
...
Note that, when the NCHAR literal replacement is turned on, OCIStmtPrepare and
OCIStmtPrepare2 will transform N' literals with U' literals in the SQL text and
store the resulting SQL text in the statement handle. Thus, if the application uses OCI_
ATTR_STATEMENT to retrieve the SQL text from the OCI statement handle, the SQL
text will return U' instead of N' as specified in the original text.
See the Oracle Database SQL Reference for information regarding environment variables.
The amtp parameter is the data length in number of Unicode code units. The offset
parameter indicates the offset of data from the beginning of the data column. The
csid parameter must be set for UTF-16 data.
To read UTF-16 data from CLOB columns, call OCILobRead() as follows:
...
ub2 csid = OCI_UTF16ID;
err = OCILobRead(ctx->svchp, ctx->errhp, lobp, &amtp, offset, (void *) buf,
(ub4)BUFSIZE , (void *) 0, (sb4 (*)()) 0, (ub2)csid,
(ub1) SQLCS_IMPLICIT);
The data length is always represented in the number of Unicode code units. Note one
Unicode supplementary character is counted as two code units, because the encoding
is UTF-16. After binding or defining a LOB column, you can measure the data length
stored in the LOB column using OCILobGetLength(). The returning value is the data
length in the number of code units if you bind or define as UTF-16.
err = OCILobGetLength(ctx->svchp, ctx->errhp, lobp, &lenp);
If you are using an NCLOB, then you must set OCI_ATTR_CHARSET_FORM to SQLCS_
NCHAR.
EXEC SQL SELECT ename, address INTO :ename, :address FROM emp;
/* ename is NULL-terminated */
printf(L"ENAME = %s, ADDRESS = %.*s\n", ename, address.len, address.arr);
...
}
When you use the VARCHAR datatype or native text datatype in a Pro*C/C++
program, the preprocessor assumes that the program intends to access columns of SQL
CHAR datatypes instead of SQL NCHAR datatypes in the database. The preprocessor
generates C/C++ code to reflect this fact by doing a bind or define using the SQLCS_
IMPLICIT value for the OCI_ATTR_CHARSET_FORM attribute. As a result, if a bind or
define variable is bound to a column of SQL NCHAR datatypes in the database, then
implicit conversion occurs in the database server to convert the data from the database
character set to the national database character set and vice versa. During the
conversion, data loss occurs when the database character set is a smaller set than the
national character set.
main()
{
...
/* Change to STRING datatype: */
EXEC ORACLE OPTION (CHAR_MAP=STRING) ;
utext ename[20] ; /* unsigned short type */
uvarchar address[50] ; /* Pro*C/C++ uvarchar type */
EXEC SQL SELECT ename, address INTO :ename, :address FROM emp;
/* ename is NULL-terminated */
wprintf(L"ENAME = %s, ADDRESS = %.*s\n", ename, address.len,
address.arr);
...
}
When you use the UVARCHAR datatype or native utext datatype in Pro*C/C++
programs, the preprocessor assumes that the program intends to access SQL NCHAR
datatypes. The preprocessor generates C/C++ code by binding or defining using the
SQLCS_NCHAR value for OCI_ATTR_CHARSET_FORM attribute. As a result, if a bind or
define variable is bound to a column of a SQL NCHAR datatype, then an implicit
conversion of the data from the national character set occurs in the database server.
However, there is no data lost in this scenario because the national character set is
always a larger set than the database character set.
You can define the target SQL columns by specifying their datatypes and lengths.
When you define a SQL CHAR column with the datatype and the length, JDBC uses
this information to optimize the performance of fetching SQL CHAR data from the
column. The following is an example of defining a SQL CHAR column.
OraclePreparedStatement pstmt = (OraclePreparedStatement)
conn.prepareStatement("SELECT ename, empno from emp");
pstmt.defineColumnType(1,Types.VARCHAR, 3);
pstmt.defineColumnType(2,Types.INTEGER);
ResultSet rest = pstmt.executeQuery();
String name = rset.getString(1);
int id = reset.getInt(2);
You can define the target SQL NCHAR columns by specifying their datatypes, forms of
use, and lengths. JDBC uses this information to optimize the performance of fetching
SQL NCHAR data from these columns. The following is an example of defining a SQL
NCHAR column.
OraclePreparedStatement pstmt = (OraclePreparedStatement)
conn.prepareStatement("SELECT ename, empno from emp");
pstmt.defineColumnType(1,Types.VARCHAR, 3,
OraclePreparedStatement.FORM_NCHAR);
pstmt.defineColumnType(2,Types.INTEGER);
ResultSet rest = pstmt.executeQuery();
String name = rset.getString(1);
int id = reset.getInt(2);
To define a SQL NCHAR column, you need to specify the datatype that is equivalent to
a SQL CHAR column in the first argument, the length in number of characters in the
second argument, and the form of use in the fourth argument of
defineColumnType().
You can bind or define a Java string against an NCHAR column without explicitly
specifying the form of use argument. This implies the following:
■ If you do not specify the argument in the setString() method, then JDBC
assumes that the bind or define variable is for the SQL CHAR column. As a result, it
tries to convert them to the database character set. When the data gets to the
database, the database implicitly converts the data in the database character set to
the national character set. During this conversion, data can be lost when the
database character set is a subset of the national character set. Because the national
character set is either UTF8 or AL16UTF16, data loss would happen if the database
character set is not UTF8 or AL32UTF8.
■ Because implicit conversion from SQL CHAR to SQL NCHAR datatypes happens in
the database, database performance is degraded.
In addition, if you bind or define a Java string for a column of SQL CHAR datatypes but
specify the form of use argument, then performance of the database is degraded.
However, data should not be lost because the national character set is always a larger
set than the database character set.
With this property specified, the Oracle JDBC drivers assume the presence of the form
of use argument for all bind and define operations in the application.
If you have a database schema that consists of both the SQL CHAR and SQL NCHAR
columns, then using this flag may have some performance impact when accessing the
SQL CHAR columns because of implicit conversion done in the database server.
conversion before being processed. The database, in turn, converts the SQL statements
from UTF-8 to the database character set. The database, in turn, converts the SQL
statement to the database character set. For Java string bind and define variables, the
conversion paths shown in Table 7–6 are taken for the thin driver.
oracle.sql.CHAR
The oracle.sql.CHAR class has a special functionality for conversion of character
data. The Oracle character set is a key attribute of the oracle.sql.CHAR class. The
Oracle character set is always passed in when an oracle.sql.CHAR object is
Each character set that Oracle supports has a unique predefined OracleId. The
OracleId can always be referenced as a character set specified as Oracle_
character_set_name_CHARSET where Oracle_character_set_name is the
Oracle character set.
2. Construct an oracle.sql.CHAR object. Pass to the constructor a string (or the
bytes that represent the string) and the CharacterSet object that indicates how
to interpret the bytes based on the character set. For example:
String mystring = "teststring";
...
oracle.sql.CHAR mychar = new oracle.sql.CHAR(teststring, mycharset);
The oracle.sql.CHAR class has multiple constructors: they can take a string, a
byte array, or an object as input along with the CharacterSet object. In the case
of a string, the string is converted to the character set indicated by the
CharacterSet object before being placed into the oracle.sql.CHAR object.
The server (database) and the client (or application running on the client) can use
different character sets. When you use the methods of this class to transfer data
between the server and the client, the JDBC drivers must convert the data between the
server character set and the client character set.
The Java class corresponding to this object type can be constructed as follows:
public class person implement SqlData
{
oracle.sql.CHAR name;
oracle.sql.CHAR address;
oracle.sql.NUMBER age;
// SqlData interfaces
getSqlType() {...}
writeSql(SqlOutput stream) {...}
readSql(SqlInput stream, String sqltype) {...}
}
The oracle.sql.CHAR class is used here to map to the NAME attributes of the Oracle
object type, which is of VARCHAR2 datatype. JDBC populates this class with the byte
representation of the VARCHAR2 data in the database and the CharacterSet object
corresponding to the database character set. The following code retrieves a person
object from the employees table:
TypeMap map = ((OracleConnection)conn).getTypeMap();
map.put("PERSON_TYPE", Class.forName("person"));
conn.setTypeMap(map);
. . .
. . .
ResultSet rs = stmt.executeQuery(
"SELECT PERSON FROM EMPLOYEES");
rs.next();
person p = (person) rs.getObject(1);
oracle.sql.CHAR sql_name = p.name;
oracle.sql.CHAR sql_address=p.address;
String java_name = sql_name.getString();
String java_address = sql_address.getString();
The getString() method of the oracle.sql.CHAR class converts the byte array
from the database character set or national character set to UTF-16 by calling Oracle's
Java data conversion classes and returning a Java string. For the rs.getObject(1)
call to work, the SqlData interface has to be implemented in the class person, and
the Typemap map has to be set up to indicate the mapping of the object type PERSON_
TYPE to the Java class.
characters used by Java clients to Oracle database character set characters, and vice
versa. Character data that makes a round trip from the Java Unicode character set to
the database character set and back to Java can suffer some loss of information. This
happens when multiple Unicode characters are mapped to a single character in the
database character set. An example is the Unicode full-width tilde character (0xFF5E)
and its mapping to Oracle's JA16SJIS character set. The round-trip conversion for this
Unicode character results in the Unicode character 0x301C, which is a wave dash (a
character commonly used in Japan to indicate range), not a tilde.
Figure 7–2 shows the round-trip conversion of the tilde character.
Oracle database
Character Set
Java Unicode (JA16SJIS) Java Unicode
.. .. ..
0x301C
. . .
.. .. 0x8160 .. 0x301C
. . .
0xFF5E .. .. 0xFF5E
. .
This issue is not a bug in Oracle's JDBC. It is an unfortunate side effect of the
ambiguity in character mapping specifications on different operating systems.
Fortunately, this problem affects only a small number of characters in a small number
of Oracle character sets such as JA16SJIS, JA16EUC, ZHT16BIG5, and KO16KS5601.
The workaround is to avoid making a full round-trip with these characters.
Oracle provides ODBC and OLE DB products for Windows platforms only. For Unix
platforms, contact your vendor.
You must specify the datatype for inserting and updating operations.
The datatype of the ODBC client buffer is given when you call SQLGetData but not
immediately. Hence, SQLFetch does not have the information.
Because the ODBC driver guarantees data integrity, if you perform implicit bindings,
then redundant conversion may result in performance degradation. Your choice is the
trade-off between performance with explicit binding or usability with implicit binding.
Microsoft's ODBC 3.5 specification defines three Unicode datatype identifiers for the
SQL_C_WCHAR, SQL_C_WVARCHAR, and SQL_WLONGVARCHAR clients; and three
Unicode datatype identifiers for servers SQL_WCHAR, SQL_WVARCHAR, and SQL_
WLONGVARCHAR.
For binding operations, specify datatypes for both client and server using
SQLBindParameter. The following is an example of Unicode binding, where the
client buffer Name indicates that Unicode data (SQL_C_WCHAR) is bound to the first
bind variable associated with the Unicode column (SQL_WCHAR):
SQLBindParameter(StatementHandle, 1, SQL_PARAM_INPUT, SQL_C_WCHAR,
SQL_WCHAR, NameLen, 0, (SQLPOINTER)Name, 0, &Name);
Table 7–9 represents the datatype mappings of the ODBC Unicode datatypes for the
server against SQL NCHAR datatypes.
The OLESTR macro works exactly like an "L" modifier to indicate the Unicode string. If
you need to allocate Unicode data buffer dynamically using OLESTR, then use the
IMalloc allocator (for example, CoTaskMemAlloc). However, using OLESTR is not
the normal method for variable length data; use wchar_t* instead for generic string
types. BSTR is similar. It is a string with a length prefix in the memory location
preceding the string. Some functions and methods can accept only BSTR Unicode
datatypes. Therefore, BSTR Unicode string must be manipulated with special
functions like SysAllocString for allocation and SysFreeString for freeing
memory.
Unlike ODBC, OLE DB does not allow you to specify the server datatype explicitly.
When you set the client datatype, the OLE DB driver automatically performs data
conversion if necessary.
Table 7–10 illustrates OLE DB datatype mapping.
ADO Access
ADO is a high-level API to access database with the OLE DB and ODBC drivers. Most
database application developers use the ADO interface on Windows because it is
easily accessible from Visual Basic, the primary scripting language for Active Server
Pages (ASP) for the Internet Information Server (IIS). To OLE DB and ODBC drivers,
ADO is simply an OLE DB consumer or ODBC application. ADO assumes that OLE
DB and ODBC drivers are Unicode-aware components; hence, it always attempts to
manipulate Unicode data.
// create a file
File file = new File( "myfile.xml" );
Application Server
Instance 2
Japanese
Locale Oracle
Unicode
Server B Database
HTTP
Oracle Net
Each application server is configured for the locale that it serves. This deployment
model assumes that one instance of an Internet application runs in the same locale as
the application in the middle tier.
The Internet applications access a back-end database in the native encoding used for
the locale. The following are advantages of deploying monolingual Internet
applications:
■ The support of the individual locales is separated into different servers so that
multiple locales can be supported independently in different locations and that the
workload can be distributed accordingly. For example, customers may want to
support Western European locales first and then support Asian locales such as
Japanese (Japan) later.
■ The complexity required to support multiple locales simultaneously is avoided.
The amount of code to write is significantly less for a monolingual Internet
application than for a multilingual Internet application.
The following are disadvantages of deploying monolingual Internet applications:
■ Extra effort is required to maintain and manage multiple servers for different
locales. Different configurations are required for different application servers.
■ The minimum number of application servers required depends on the number of
locales the application supports, regardless of whether the site traffic will reach the
capacity provided by the application servers.
Browsers Customer
Database
ISO-8859-1
English
Locale
Shift-JIS
Server
Multilingual
Japanese Application with Unicode
Locale Oracle
Dynamic Locale Unicode
Switching Database
UTF-8
Application Server
Instance
Hebrew
Locale
UTF-8
HTTP
Oracle Net
Thai
Locale
To support multiple locales in a single application instance, the application may need
to do the following:
■ Dynamically detect the locale of the users and adapt to the locale by constructing
HTML pages in the language and cultural conventions of the locale
■ Process character data in Unicode so that data in any language can be supported.
Character data can be entered by users or retrieved from back-end databases.
■ Dynamically determine the HTML page encoding (or character set) to be used for
HTML pages and convert content from Unicode to the page encoding and the
reverse.
The following are major advantages of deploying multilingual Internet application:
■ Using a single application server configuration for all application servers
simplifies the deployment configuration and hence reduces the cost of
maintenance.
■ Performance tuning and capacity planning do not depend on the number of
locales supported by the Web site.
■ Introducing additional locales is relatively easy. No extra machines are necessary
for the new locales.
■ Testing the application across different locales can be done in a single testing
environment.
■ This model can support multilingual content within the same instance of the
application. For example, a web page containing Japanese, Chinese, English and
Arabic data can be easily supported in this model.
The disadvantage of deploying multilingual Internet applications is that it requires
extra coding during application development to handle dynamic locale detection and
Unicode, which is costly when only one or two languages need to be supported.
Deploying multilingual Internet applications is more appropriate than deploying
monolingual applications when Web sites support multiple locales.
Locale Determination
To be locale-aware or locale-sensitive, Internet applications need to be able to
determine the preferred locale of the user.
Monolingual applications always serve users with the same locale, and that locale
should be equivalent to the default runtime locale of the corresponding programming
environment.
Multilingual applications can determine a user locale dynamically in three ways. Each
method has advantages and disadvantages, but they can be used together in the
applications to complement each other. The user locale can be determined in the
following ways:
■ Based on the user profile information from a LDAP directory server such as the
Oracle Internet Directory or other user profile tables stored inside the database
The schema for the user profile should include preferred locale attribute to
indicate the locale of a user. This way of determining a locale user does not work if
a user has not been logged on before.
■ Based on the default locale of the browser
Get the default ISO locale setting from a browser. The default ISO locale of the
browser is sent through the Accept-Language HTTP header in every HTTP
request. If the Accept-Language header is NULL, then the desired locale should
default to English. The drawback of this approach is that the Accept-Language
header may not be a reliable source of information for the locale of a user.
■ Based on user selection
Allow users to select a locale from a list box or from a menu, and switch the
application locale to the one selected.
The Globalization Development Kit provides an application framework that enables
you to use these locale determination methods declaratively.
Locale Awareness
To be locale-aware or locale-sensitive, Internet applications need to determine the
locale of a user. After the locale of a user is determined, applications should:
■ Construct HTML content in the language of the locale
■ Use the cultural conventions implied by the locale
Locale-sensitive functions, such as date, time, and monetary formatting, are built into
various programming environments such as Java and PL/SQL. Applications may use
them to format the HTML pages according to the cultural conventions of the locale of
a user. A locale is represented differently in different programming environments. For
example, the French (Canada) locale is represented in different environments as
follows:
■ In the ISO standard, it is represented by fr-CA where fr is the language code
defined in the ISO 639 standard and CA is the country code defined in the ISO 3166
standard.
■ In Java, it is represented as a Java locale object constructed with fr, the ISO
language code for French, as the language and CA, the ISO country code for
Canada, as the country. The Java locale name is fr_CA.
J2EE User
Application
GDK
Framework for J2EE
LDAP
The functionalities offered by GDK for Java can be divided into two categories:
■ The GDK application framework for J2EE provides the globalization framework
for building J2EE-based Internet application. The framework encapsulates the
complexity of globalization programming, such as determining user locale,
maintaining locale persistency, and processing locale information. It consists of a
set of Java classes through which applications can gain access to the framework.
These associated Java classes enable applications to code against the framework so
that globalization behaviors can be extended declaratively.
■ The GDK Java API offers development support in Java applications and provides
consistent globalization operations as provided in Oracle database servers. The
API is accessible and is independent of the GDK framework so that standalone
Java applications and J2EE applications that are not based on the GDK framework
are able to access the individual features offered by the Java API. The features
provided in the Java API include data and number formatting, sorting, and
handling character sets in the same way as the Oracle Database.
Note: The GDK Java API is certified with JDK versions 1.3 and
later with the following exception: The character set conversion
classes depend on the java.nio.charset package, which is
available in JDK 1.4 and later.
GDK for Java is contained in nine .jar files, all in the form of orai18n*jar. These
files are shipped with the Oracle Database, in the $ORACLE_HOME/jlib directory. If
the application using the GDK is not hosted on the same machine as the database, then
the GDK files must be copied to the application server and included into the
CLASSPATH to run your application. You do not need to install the Oracle Database
into your application server to be able to run the GDK inside your Java application.
GDK is a pure Java library that runs on every platform. The Oracle client parameters
NLS_LANG and ORACLE_HOME are not required.
The initial, non-GDK HelloWorld Web application simply prints a "Hello World!"
message, along with the current date and time in the top right hand corner of the page.
The following code shows the original HelloWorld JSP source code for the preceding
image.
The following code example shows the corresponding Web application descriptor file
for the HelloWorld message.
The HelloWorld JSP code in Example 8–1 is only for English-speaking users. Some of
the problems with this code are as follows:
■ There is no locale determination based on user preference or browser setting.
■ The title and the heading are included in the code.
■ The date and time value is not localized based on any locale preference.
■ The character encoding included in the code is for Latin-1.
The GDK framework can be integrated into the HelloWorld code to make it a global,
multilingual application. The preceding code can be modified to include the following
features:
■ Automatic locale negotiation to detect the user's browser locale and serve the
client with localized HTML pages. The supported application locales are
configured in the GDK configuration file.
■ Locale selection list to map the supported application locales. The list can have
application locale display names which are the name of the country representing
the locale. The list will be included on the Web page so users can select a different
locale.
■ GDK framework and API for globalization support for the HelloWorld JSP. This
involves selecting display strings in a locale-sensitive manner and formatting the
date and time value.
In addition, the user will be able to change the language by selecting a supported
locales from the locale selection list. The following tasks describe how to modify the
application:
■ Task 1: Enable the Hello World Application to use the GDK Framework
■ Task 2: Configure the GDK Framework for Hello World
■ Task 3: Enable the JSP or Java Servlet
■ Task 4: Create the Locale Selection List
■ Task 5: Build the Application
Task 1: Enable the Hello World Application to use the GDK Framework
In this task, the GDK filter and a listener are configured in the Web application
deployment descriptor file, web.xml. This allows the GDK framework to be used with
the HelloWorld application. Example 8–3 shows the GDK-enabled web.xml file.
<listener>
<listener-class>oracle.i18n.servlet.listener.ContextListener</listener-class>
</listener>
<session-config>
<session-timeout>35</session-timeout>
</session-config>
<mime-mapping>
<extension>html</extension>
<mime-type>text/html</mime-type>
</mime-mapping>
<mime-mapping>
<extension>txt</extension>
<mime-type>text/plain</mime-type>
</mime-mapping>
</web-app>
<!-- The supported application locales for the Hello World Application -->
<application-locales>
<locale>de-CH</locale>
<locale default="yes">en-US</locale>
<locale>zh-CN</locale>
</application-locales>
<locale-determine-rule>
<locale-source>oracle.i18n.servlet.localesource.UserInput</locale-source>
<locale-source>oracle.i18n.servlet.localesource.HttpAcceptLanguage</locale-source>
</locale-determine-rule>
<message-bundles>
<resource-bundle name="default">com.oracle.demo.Messages</resource-bundle>
</message-bundles>
</gdkapp>
The file must be configured for J2EE applications. The following tags are used in the
file:
■ <page-charset>
The page encoding tag specifies the character set used for HTTP requests and
responses. The UTF-8 encoding is used as the default because many languages can
be represented by this encoding.
■ <application-locales>
Configuring the application locales in the gdkapp.xml file makes a central place
to define locales. This makes it easier to add and remove locales without changing
source code. The locale list can be retrieved using the GDK API call
ApplicationContext.getSupportedLocales.
■ <locale-determine-rule>
The language of the initial page is determined by the language setting of the
browser. The user can override this language by choosing from the list. The
<body>
<div style="color: blue;" align="right">
<% Date currDate= new Date(System.currentTimeMillis()); %>
<%=localizer.formatDateTime(currDate, OraDateFormat.LONG)%>
</div>
<hr/>
<div align="left">
<form>
<select name="locale" size="1">
<%= getCountryDropDown(request)%>
</select>
<input type="submit" value="<%= localizer.getMessage("changeLocale") %>">
</input>
</form>
</div>
<h1><%= localizer.getMessage("helloWorld") %></h1>
</body>
</html>
Figure 8–5 shows the HelloWorld application that has been configured with the zh-CN
locale as the primary locale for the browser preference. The HelloWorld string and
page title are displayed in Simplified Chinese. In addition, the date is formatted in the
zh-CN locale convention. This example allows the user to override the locale from the
locale selection list.
When the locale changes or is initialized using the HTTP Request Accept-Language
header or the locale selection list, the GUI behaves appropriately for that locale. This
means the date and time value in the upper right corner is localized properly. In
addition, the strings are localized and displayed on the HelloWorld page.
The GDK Java Localizer class provides capabilities to localize the contents of a Web
page based on the automatic detection of the locale by the GDK framework.
The following code retrieves an instance of the localizer based on the current
HTTPServletRequest object. In addition, several imports are declared for use of the
GDK API within the JSP page. The localizer retrieves localized strings in a
locale-sensitive manner with fallback behavior, and formats the date and time.
<%@page contentType="text/html;charset=UTF-8"%>
<%@page import="java.util.*, oracle.i18n.servlet.*" %>
<%@page import="oracle.i18n.util.*, oracle.i18n.text.*" %>
<%
Localizer localizer = ServletHelper.getLocalizerInstance(request);
%>
The following code retrieves the current date and time value stored in the currDate
variable. The value is formatted by the localizer formatDateTime method. The
OraDateFormat.LONG parameter in the formatDateTime method instructs the
localizer to format the date using the locale's long formatting style. If the locale of the
incoming request is changed to a different locale with the locale selection list, then the
date and time value will be formatted according to the conventions of the new locale.
No code changes need to be made to support newly-introduced locales.
div style="color: blue;" align="right">
The HelloWorld JSP can be reused for any locale because the HelloWorld string and
title are selected in a locale-sensitive manner. The translated strings are selected from a
resource bundle.
The GDK uses the OraResourceBundle class for implementing the resource bundle
fallback behavior. The following code shows how the Localizer picks the HelloWorld
message from the resource bundle.
The default application resource bundle Messages is declared in the gdkapp.xml file.
The localizer uses the message resource bundle to pick the message and apply the
locale-specific logic. For example, if the current locale for the incoming request is
"de-CH", then the message will first be looked for in the messages_de_CH bundle. If it
does not exist, then it will look up in the Messages_de resource bundle.
<h1><%= localizer.getMessage("helloWorld") %></h1>
The locale selection list is constructed from the HTML code generated by the
getCountryDropDown method. The method converts the configured application
locales into localized country names.
A call is made to the ServletHelper class to get the ApplicationContext object
associated with the current request. This object provides the globalization context for
an application, which includes information such as supported locales and
configuration information. The getSupportedLocales call retrieves the list of
locales in the gdkapp.xml file. The configured application locale list is displayed as
options of the HTML select. The OraDisplayLocaleInfo class is responsible for
providing localization methods of locale-specific elements such as country and
language names.
An instance of this class is created by passing in the current locale automatically
determined by the GDK framework. GDK creates requests and response wrappers for
HTTP request and responses. The request.getLocale() method returns the GDK
determined locale based on the locale determination rules.
The OraDsiplayLocaleInfo.getDisplayCountry method retrieves the localized
country names of the application locales. An HTML option list is created in the
ddOptBuffer string buffer. The getCountryDropDown call returns a string containing
the following HTML values:
<option value="en_US" selected>United States [en_US]</option>
<option value="zh_CN">China [zh_CN]</option>
<option value="de_CH">Switzerland [de_CH]</option>
In the preceding values, the en-US locale is selected for the locale. Country names are
generated are based on the current locale.
Example 8–6 shows the code for constructing the locale selection list.
if (currentLocale.getCountry().equals(""))
{
// Since the Country was not specified get the Default Locale
// (with Country) from the GDK
OraLocaleInfo oli = OraLocaleInfo.getInstance(currentLocale);
currentLocale = oli.getLocale();
}
OraDisplayLocaleInfo odli =
OraDisplayLocaleInfo.getInstance(currentLocale);
for (int i=0;i<appLocales.length; i++)
{
ddOptBuffer.append("<option value=\"" + appLocales[i] + "\"" +
(appLocales[i].getLanguage().equals(currentLocale.getLanguage()) ? "
selected" : "") +
">" + odli.getDisplayCountry(appLocales[i]) +
" [" + appLocales[i] + "]</option>\n");
}
return ddOptBuffer.toString();
}
%>
Request Response
ApplicationContext
Configuration
File
LocalSource
Localizer
defining it in the GDK application configuration file. There is no code change required
when you add or remove a locale from the list of supported application locales.
The following list gives you some idea of the extent to which you can define the
globalization support in the GDK application configuration file:
■ You can add and remove a locale from the list of supported locales.
■ You can change the way the user locale is determined.
■ You can change the HTML page encoding of your application.
■ You can specify how the translated resources can be located.
■ You can plug a new LocaleSource object into the framework and use it to detect
a user locale.
This section includes the following topics:
■ Making the GDK Framework Available to J2EE Applications
■ Integrating Locale Sources into the GDK Framework
■ Getting the User Locale From the GDK Framework
■ Implementing Locale Awareness Using the GDK Localizer
■ Defining the Supported Application Locales in the GDK
■ Handling Non-ASCII Input and Output in the GDK Framework
■ Managing Localized Content in the GDK
For a J2EE application to use the GDK application framework defined by the
corresponding GDK application configuration file, the GDK Servlet filter and the GDK
context listener must be defined in the web.xml file of the application. The web.xml
file should be modified to include the following at the beginning of the file:
<web-app>
<!-- Add GDK filter that is called after the authentication -->
<filter>
<filter-name>gdkfilter</filter-name>
<filter-class>oracle.i18n.servlet.filter.ServletFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>gdkfilter</filter-name>
<url-pattern>*.jsp</url-pattern>
</filter-mapping>
<listener>
<listener-class>oracle.i18n.servlet.listener.ContextListener</listener-class>
</listener>
</web-app>
Examples of the gdkapp.xml and web.xml files can be found in the $ORACLE_
HOME/nls/gdk/demo directory.
The GDK application framework supports Servlet container version 2.3 and later. It
uses the Servlet filter facility for transparent globalization operations such as
determining the user locale and specifying the character set for content files. The
ContextListener instantiates GDK application parameters described in the GDK
application configuration file. The ServletFilter overrides the request and
response objects with a GDK request (ServletRequestWrapper) and response
(ServletResponseWrapper) objects, respectively.
If other application filters are used in the application to also override the same
methods, then the filter in the GDK framework may return incorrect results. For
example, if getLocale returns en_US, but the result is overridden by other filters,
then the result of the GDK locale detection mechanism is affected. All of the methods
that are being overridden in the filter of the GDK framework are documented in Oracle
Globalization Development Kit Java API Reference. Be aware of potential conflicts when
using other filters together with the GDK framework.
The GDK application framework provides seamless support for predefined locale
sources, such as user input locale, HTTP language preference, user profile locale
preference in the database, and the application default locale. You can incorporate the
locale sources to the framework by defining them under the
<locale-determine-rule> tag in the GDK application configuration file as
follows:
<locale-determine-rule>
<locale-source>oracle.i18n.servlet.localesource.UserInput</locale-source>
<locale-source>oracle.i18n.servlet.localesource.HTTPAcceptLanguage</locale-source>
</locale-determine-rule>
The GDK framework uses the locale source declaration order and determines whether
a particular locale source is available. If it is available, then it is used as the source,
otherwise, it tries to find the next available locale source for the list. In the preceding
example, if the UserInput locale source is available, it is used first, otherwise, the
HTTPAcceptLanguage locale source will be used.
Custom locale sources, such as locale preference from an LDAP server, can be easily
implemented and integrated into the GDK framework. You need to implement the
LocaleSource interface and specify the corresponding implementation class under
the <locale-determine-rule> tag in the same way as the predefined locale
sources were specified.
The LocaleSource implementation not only retrieves the locale information from the
corresponding source to the framework but also updates the locale information to the
corresponding source when the framework tells it to do so. Locale sources can be
read-only or read/write, and they can be cacheable or noncacheable. The GDK
framework initiates updates only to read/write locale sources and caches the locale
information from cacheable locale sources. Examples of custom locale sources can be
found in the $ORACLE_HOME/nls/gdk/demo directory.
The getLocale() method returns the Locale that represents the current locale. This
is similar to invoking the HttpServletRequest.getLocale() method in JSP or
Java Servlet code. However, the logic in determining the user locale is different,
because multiple locale sources are being considered in the GDK framework.
Alternatively, you can get a Localizer object that encapsulates the Locale object
determined by the GDK framework. For the benefits of using the Localizer object,
see "Implementing Locale Awareness Using the GDK Localizer" on page 8-21.
Localizer localizer = ServletHelper.getLocalizerInstance(request);
Locale loc = localizer.getLocale();
The locale detection logic of the GDK framework depends on the locale sources
defined in the GDK application configuration file. The names of the locale sources are
registered in the application configuration file. The following example shows the
locale determination rule section of the application configuration file. It indicates that
the user-preferred locale can be determined from either the LDAP server or from the
For example, when the user is authenticated in the application and the user locale
preference is stored in an LDAP server, then the LDAPUserSchema class connects to
the LDAP server to retrieve the user locale preference. When the user is anonymous,
then the HttpAcceptLanguage class returns the language preference of the web
browser.
The cache is maintained for the duration of a HTTP session. If the locale source is
obtained from the HTTP language preference, then the locale information is passed to
the application in the HTTP Accept-Language header and not cached. This enables
flexibility so that the locale preference can change between requests. The cache is
available in the HTTP session.
The GDK framework exposes a method for the application to overwrite the locale
preference information persistently stored in locale sources such as the LDAP server or
the user profile table in the database. This method also resets the current locale
information stored inside the cache for the current HTTP session. The following is an
example of overwriting the preferred locale using the store command.
<input type="hidden"
name="<%=appctx.getParameterName(LocaleSource.Parameter.COMMAND)%>"
value="store">
To discard the current locale information stored inside the cache, the clean command
can be specified as the input parameter. The following table shows the list of
commands supported by the GDK:
Command Functionality
store Updates user locale preferences in the available locale sources with the
specified locale information. This command is ignored by the read-only
locale sources.
clean Discards the current locale information in the cache.
Note that the GDK parameter names can be customized in the application
configuration file to avoid name conflicts with other parameters used in the
application.
<locale>ja</locale>
<locale default="yes">en-US</locale>
</application-locales>
When the GDK framework detects the user locale, it verifies whether the locale that is
returned is one of the supported locales in the application configuration file. The
verification algorithm is as follows:
1. Retrieve the list of supported application locales from the application
configuration file.
2. Check whether the locale that was detected is included in the list. If it is included
in the list, then use this locale as the current client's locale.
3. If there is a variant in the locale that was detected, then remove the variant and
check whether the resulting locale is in the list. For example, the Java locale de_
DE_EURO has a EURO variant. Remove the variant so that the resulting locale is
de_DE.
4. If the locale includes a country code, then remove the country code and check
whether the resulting locale is in the list. For example, the Java locale de_DE has a
country code of DE. Remove the country code so that the resulting locale is de.
5. If the detected locale does not match any of the locales in the list, then use the
default locale that is defined in the application configuration file as the client
locale.
By performing steps 3 and 4, the application can support users with the same language
requirements but with different locale settings than those defined in the application
configuration file. For example, the GDK can support de-AT (the Austrian variant of
German), de-CH (the Swiss variant of German), and de-LU (the Luxembourgian
variant of German) locales.
The locale fallback detection in the GDK framework is similar to that of the Java
Resource Bundle, except that it is not affected by the default locale of the Java VM.
This exception occurs because the Application Default Locale can be used during the
GDK locale fallback operations.
If the application-locales section is omitted from the application configuration file,
then the GDK assumes that the common locales, which can be returned from the
OraLocaleInfo.getCommonLocales method, are supported by the application.
request and response objects. The GDK framework does not support the scenario
where the incoming character set is different from that of the outgoing character set.
The GDK application framework supports the following scenarios for setting the
character sets of the HTML pages:
■ A single local character set is dedicated to the whole application. This is
appropriate for a monolingual Internet application. Depending on the properties
of the character set, it may be able to support more than one language. For
example, most Western European languages can be served by ISO-8859-1.
■ Unicode UTF-8 is used for all contents regardless of the language. This is
appropriate for a multilingual application that uses Unicode for deployment.
■ The native character set for each language is used. For example, English contents
are represented in ISO-8859-1, and Japanese contents are represented in Shift_JIS.
This is appropriate for a multilingual Internet application that uses a default
character set mapping for each locale. This is useful for applications that need to
support different character sets based on the user locales. For example, for mobile
applications that lack Unicode fonts or Internet browsers that cannot fully support
Unicode, the character sets must to be determined for each request.
The character set information is specified in the GDK application configuration file.
The following is an example of setting UTF-8 as the character set for all the application
pages.
<page-charset>UTF-8</page-charset>
The default mappings are derived from the LocaleMapper class, which provides the
default IANA character set for the locale name in the GDK Java API.
Table 8–2 lists the mappings between the common ISO locales and their IANA
character sets.
Table 8–2 Mapping Between Common ISO Locales and IANA Character Sets
ISO Locale NLS_LANGUAGE Value NLS_TERRITORY Value IANA Character Set
ar-SA ARABIC SAUDI ARABIA WINDOWS-1256
de-DE GERMAN GERMANY WINDOWS-1252
en-US AMERICAN AMERICA WINDOWS-1252
en-GB ENGLISH UNITED KINGDOM WINDOWS-1252
el GREEK GREECE WINDOWS-1253
es-ES SPANISH SPAIN WINDOWS-1252
fr FRENCH FRANCE WINDOWS-1252
fr-CA CANADIAN FRENCH CANADA WINDOWS-1252
iw HEBREW ISRAEL WINDOWS-1255
ko KOREAN KOREA EUC-KR
Table 8–2 (Cont.) Mapping Between Common ISO Locales and IANA Character Sets
ISO Locale NLS_LANGUAGE Value NLS_TERRITORY Value IANA Character Set
ja JAPANESE JAPAN SHIFT_JIS
it ITALIAN ITALY WINDOWS-1252
pt PORTUGUESE PORTUGAL WINDOWS-1252
pt-BR BRAZILIAN BRAZIL WINDOWS-1252
PORTUGUESE
tr TURKISH TURKEY WINDOWS-1254
nl DUTCH THE NETHERLANDS WINDOWS-1252
zh SIMPLIFIED CHINESE CHINA GBK
zh-TW TRADITIONAL CHINESE TAIWAN BIG5
The locale to character set mapping in the GDK can also be customized. To override
the default mapping defined in the GDK Java API, a locale-to-character-set mapping
table can be specified in the application configuration file.
<locale-charset-maps>
<locale-charset>
<locale>ja</locale><charset>EUC-JP</charset>
</locale-charset>
</locale-charset-maps>
The previous example shows that for locale Japanese (ja), the GDK changes the
default character set from SHIFT_JIS to EUC-JP.
The GDK framework simplifies the retrieval of text strings from the resource bundles.
Localizer.getMessage() is a wrapper to the resource bundle.
<% Localizer.getMessage ("Welcome") %>
Instead of specifying the base class name as getBundle() in the application, you can
specify the resource bundle in the application configuration file, so that the GDK
automatically instantiates a ResourceBundle object when a translated text string is
requested.
<message-bundles>
<resource-bundle name="default">resource</resource-bundle>
</message-bundles>
This configuration file snippet declares a default resource bundle whose translated
contents reside in the "resource" Java bundle class. Multiple resource bundles can be
specified in the configuration file. To access a nondefault bundle, specify the name
parameter in the getMessage method. The message bundle mechanism uses the
OraResourceBundle GDK class for its implementation. This class provides the
special locale fallback behaviors on top of the Java behaviors. The rules are as follows:
■ If the given locale exactly matches the locale in the available resource bundles, it
will be used.
■ If the resource bundle for Chinese in Singapore (zh_SG) is not found, it will fall
back to the resource bundle for Chinese in China (zh_CN) for Simplified Chinese
translations.
■ If the resource bundle for Chinese in Hong Kong (zh_HK) is not found, it will fall
back to the resource bundle for Chinese in Taiwan (zh_TW) for Traditional Chinese
translations.
■ If the resource bundle for Chinese in Macau (zh_MO) is not found, it will fall back
to the resource bundle for Chinese in Taiwan (zh_TW) for Traditional Chinese
translations.
■ If the resource bundle for any other Chinese (zh_ and zh) is not found, it will fall
back to the resource bundle for Chinese in China (zh_CN) for Simplified Chinese
translations.
■ The default locale, which can be obtained by the Locale.getDefault()
method, will not be considered in the fallback operations.
For example, assume the default locale is ja_JP and the resource handle for it is
available. When the resource bundle for es_MX is requested, and neither resource
bundle for es or es_MX is provided, the base resource bundle object that does not have
a local suffix is returned.
The usage of the OraResourceBundle class is similar to the
java.util.ResourceBundle class, but the OraResearchBundle class does not
instantiate itself. Instead, the return value of the getBundle method is an instance of
the subclass of the java.util.ResourceBundle class.
/ja/index.html
The URL rewrite definitions are defined in the GDK application configuration file:
<url-rewrite-rule fallback="yes">
<pattern>(.*)/(a-zA-Z0-9_\]+.)$</pattern>
<result>$1/$A/$2</result>
</url-rewrite-rule>
The pattern section defined in the rewrite rule follows the regular expression
conventions. The result section supports the following special variables for replacing:
■ $L is used to represent the ISO 639 language code part of the current user locale
■ $C represents the ISO 3166 country code
■ $A represents the entire locale string, where the ISO 639 language code and ISO
3166 country code are connected with an underscore character (_)
■ $1 to $9 represent the matched substrings
For example, if the current user locale is ja, then the URL for the welcome.jpg
image file is rewritten as image/ja/welcome.jpg, and welcome.html is changed
to html/ja/welcome.html.
Both ServletHelper.rewriteURL()and Localizer.getMessage() methods
perform consistent locale fallback operations in the case where the translation files for
the user locale are not available. For example, if the online help files are not available
for the es_MX locale (Spanish for Mexico), but the es (Spanish for Spain) files are
available, then the methods will select the Spanish translated files as the substitute.
number formatting and linguistic sorting in the middle tier, the GDK Java API allows
developers to eliminate expensive programming logic in the database, hence
improving the overall application performance by reducing the database load in the
database server and the unnecessary network traffic between the application tier and
the database server.
The GDK Java API also offers advance globalization functionalities, such as language
and character set detection, and the enumeration of common locale data for a territory
or a language (for example, all time zones supported in Canada). These are
globalization features that are not available in most programming platforms. Without
the GDK Java API, developers must write business logic to handle them inside an
application.
The following are the key functionalities of the GDK Java API:
■ Oracle Locale Information in the GDK
■ Oracle Locale Mapping in the GDK
■ Oracle Character Set Conversion (JDK 1.4 and Later) in the GDK
■ Oracle Date, Number, and Monetary Formats in the GDK
■ Oracle Binary and Linguistic Sorts in the GDK
■ Oracle Language and Character Set Detection in the GDK
■ Oracle Translated Locale and Time Zone Names in the GDK
■ Using the GDK for E-Mail Programs
locale = LocaleMapper.getJavaLocale("AMERICAN","AMERICA");
locale = LocaleMapper.getJavaLocale("TRADITONAL CHINESE", "");
The LocaleMapper class can also return the most commonly used e-mail character
set for a specific locale on both Windows and UNIX platforms. This is useful when
developing Java applications that need to process e-mail messages.
See Also: "Using the GDK for E-Mail Programs" on page 8-33
Oracle Character Set Conversion (JDK 1.4 and Later) in the GDK
The GDK Java API contains a set of character set conversion classes APIs that enable
users to perform Oracle character set conversions. Although Java JDK is already
equipped with classes that can perform conversions for many of the standard
character sets, they do not support Oracle-specific character sets and Oracle's
user-defined character sets.
In JDK 1.4, J2SE introduced an interface for developers to extend Java's character sets.
The GDK Java API provides implicit support for Oracle's character sets by using this
plug-in feature. You can access the J2SE API to obtain Oracle-specific behaviors.
Figure 8–7 shows that the GDK character set conversion tables are plugged into J2SE in
the same way as the Java character set tables. With this pluggable framework of J2SE,
the Oracle character set conversions can be used in the same way as other Java
character set conversions.
J2SE
Runtime
Because the java.nio.charset Java package is not available in JDK versions before
1.4, you must install JDK 1.4 or later to use Oracle's character set plug-in feature.
The GDK character conversion classes support all Oracle character sets including
user-defined characters sets. It can be used by Java applications to properly convert to
and from Java's internal character set, UTF-16.
Oracle's character set names are proprietary. To avoid potential conflicts with Java's
own character sets, all Oracle character set names have an X-ORACLE- prefix for all
implicit usage through Java's API.
The following is an example of Oracle character set conversion:
// Converts the Chinese character "three" from UCS2 to JA16SJIS
Just as with other Java character sets, the character set facility in
java.nio.charset.Charset is applicable to all of the Oracle character sets. For
example, if you wish to check whether the specified character set is a superset of
another character set, then you can use the Charset.contains method as follows:
Charset cs1 = Charset.forName("x-oracle-US7ASCII");
Charset cs2 = Charset.forName("x-oracle-WE8WINDOWS1252");
// true if WE8WINDOWS1252 is the superset of US7ASCII, otherwise false.
boolean osc = cs2.contains(cs1);
For a Java application that is using the JDBC driver to communicate with the database,
the JDBC driver provides the necessary character set conversion between the
application and the database. Calling the GDK character set conversion methods
explicitly within the application is not required. A Java application that interprets and
generates text files based on Oracle's character set encoding format is an example of
using Oracle character set conversion classes.
// Obtain the numeric value 1234567.89 using the default number format
// for the Locale en_IN (English_India)
onf = OraNumberFormat.getCurrencyInstance(locale);
nm = onf.format(new Double(1234567.89));
cname = "GENERIC_M";
ocol = OraCollator.getInstance(cname);
List result = getCollationKeys(source, ocol);
{
karr.add(ocol.getCollationKey(source[i]));
}
Collections.sort(karr); // sorting operation
return karr;
}
The following are examples of using the LCSDetector class to enable language and
character set detection:
// This example detects the character set of a plain text file "foo.txt" and
// then appends the detected ISO character set name to the name of the text file
// This example shows how to use the LCSDector class to detect the language and
// character set of a byte array
int offset = 0;
LCSDetector led = new LCSDetector();
/* loop through the entire byte array */
while ( true )
{
bytes_read = led.detect(byte_input, offset, 1024);
if ( bytes_read == -1 )
break;
offset += bytes_read;
}
LCSDResultSet res = led.getResult();
import java.util.Properties;
import javax.mail.Message;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
import javax.mail.internet.MimeUtility;
/**
* Email send operation sample
*
* javac -classpath orai18n.jar:j2ee.jar EmailSampleText.java
* java -classpath .:orai18n.jar:j2ee.jar EmailSampleText
*/
public class EmailSampleText
{
public static void main(String[] args)
{
send("localhost", // smtp host name
"your.address@your-company.com", // from email address
"You", // from display email
"somebody@some-company.com", // to email address
"Subject test zh CN", // subject
"Content •4E02 from Text email", // body
new Locale("zh", "CN") // user locale
);
}
public static void send(String smtp, String fromEmail, String fromDispName,
String toEmail, String subject, String content, Locale locale
)
{
// get the list of common email character sets
final String[] charset = LocaleMapper.getIANACharSetFromLocale(LocaleMapper.
EMAIL_WINDOWS,
locale
);
// pick the first one for the email encoding
final String contentType = "text/plain; charset=" + charset[0];
try
{
Properties props = System.getProperties();
props.put("mail.smtp.host", smtp);
// here, set username / password if necessary
Session session = Session.getDefaultInstance(props, null);
MimeMessage mimeMessage = new MimeMessage(session);
mimeMessage.setFrom(new InternetAddress(fromEmail, fromDispName,
charset[0]
)
);
mimeMessage.setRecipients(Message.RecipientType.TO, toEmail);
mimeMessage.setSubject(MimeUtility.encodeText(subject, charset[0], "Q"));
// body
mimeMessage.setContent(content, contentType);
mimeMessage.setHeader("Content-Type", contentType);
mimeMessage.setHeader("Content-Transfer-Encoding", "8bit");
mimeMessage.setSentDate(new Date());
Transport.send(mimeMessage);
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
locale-charset-maps
This section enables applications to override the mapping from the language to the
default character set provided by the GDK. This mapping is used when the
page-charset is set to AUTO-CHARSET.
For example, for the en locale, the default GDK character set is windows-1252.
However, if the application requires ISO-8859-1, this can be specified as follows:
<locale-charset-maps>
<locale-charset>
<locale>en</locale>
<charset>ISO_8859-1</charset>
</locale-charset>
</locale-charset-maps>
The locale name is comprised of the language code and the country code, and they
should follow the ISO naming convention as defined in ISO 639 and ISO 3166,
respectively. The character set name follows the IANA convention.
Optionally, the user-agent parameter can be specified in the mapping table to
distinguish different clients.
<locale-charset>
<locale>en,de</locale>
<user-agent>^Mozilla⁄4.0</user-agent>
<charset>ISO-8859-1</charset>
</locale-charset>
The previous example shows that if the user-agent value in the HTTP header starts
with Mozilla/4.0 (which indicates older version of web clients) for English (en) and
German (de) locales, then the GDK sets the character set to ISO-8859-1.
Multiple locales can be specified in a comma-delimited list.
page-charset
This tag section defines the character set of the application pages. If this is explicitly set
to a given character set, then all pages use this character set. The character set name
must follow the IANA character set convention.
<page-charset>UTF-8</page-charset>
See Also:
■ "locale-charset-maps" on page 8-35
■ "Handling Non-ASCII Input and Output in the GDK
Framework" on page 8-23
application-locales
This tag section defines a list of the locales supported by the application.
<application-locales>
<locale default="yes">en-US</locale>
<locale>de</locale>
<locale>zh-CN</locale>
</application-locales>
If the language component is specified with the * country code, then all locale names
with this language code qualify. For example, if de-* (the language code for German)
is defined as one of the application locales, then this supports de-AT (German-
Austria), de (German-Germany), de-LU (German-Luxembourg), de-CH
(German-Switzerland), and even irregular locale combination such as de-CN
(German-China). However, the application can be restricted to support a predefined
set of locales.
It is recommended to set one of the application locales as the default application locale
(by specifying default="yes") so that it can be used as a fall back locale for
customers who are connecting to the application with an unsupported locale.
locale-determine-rule
This section defines the order in which the preferred user locale is determined. The
locale sources should be specified based on the scenario in the application. This section
includes the following scenarios:
■ Scenario 1: The GDK framework uses the accept language at all times.
<locale-source>oracle.i18n.servlet.localesource.HTTPAcceptLanguage</locale-source>
■ Scenario 2: By default, the GDK framework uses the accept language. After the
user specifies the locale, the locale is used for further operations.
<locale-source>oracle.i18n.servlet.localesource.UserInput</locale-source>
<locale-source>oracle.i18n.servlet.localesource.HTTPAcceptLanguage</locale-source>
■ Scenario 3: By default, the GDK framework uses the accept language. After the
user is authenticated, the GDK framework uses the database locale source. The
database locale source is cached until the user logs out. After the user logs out, the
accept language is used again.
<db-locale-source
data-source-name="jdbc/OracleCoreDS"
locale-source-table="customer"
user-column="customer_email"
user-key="userid"
language-column="nls_language"
territory-column="nls_territory"
timezone-column="timezone"
>oracle.i18n.servlet.localesource.DBLocaleSource</db-locale-source>
<locale-source>oracle.i18n.servlet.localesource.HttpAcceptLanguage</locale-source>
Note that Scenario 4 uses the custom database locale source. If the user profile schema
is complex, such as user profile information separated into multiple tables, then the
custom locale source should be provided by the application. Examples of custom
locale sources can be found in the $ORACLE_HOME/nls/gdk/demo directory.
locale-parameter-name
The tag defines the name of the locale parameters that are used in the user input so
that the current user locale can be passed between requests.
Table 8–3 shows the parameters used in the GDK framework.
The parameter names are used in either the parameter in the HTML form or in the
URL.
message-bundles
This tag defines the base class names of the resource bundles used in the application.
The mapping is used in the Localizer.getMessage method for locating translated
text in the resource bundles.
<message-bundles>
<resource-bundle>Messages</resource-bundle>
<resource-bundle name="newresource">NewMessages</resource-bundle>
</message-bundles>
url-rewrite-rule
This tag is used to control the behavior of the URL rewrite operations. The rewriting
rule is a regular expression.
<url-rewrite-rule fallback="no">
<pattern>(.*)/([^/]+)$</pattern>
<result>$1/$L/$2</result>
</url-rewrite-rule>
If the localized content for the requested locale is not available, then it is possible for
the GDK framework to trigger the locale fallback mechanism by mapping it to the
closest translation locale. By default the fallback option is turned off. This can be
turned on by specifying fallback="yes".
For example, suppose an application supports only the following translations: en, de,
and ja, and en is the default locale of the application. If the current application locale
is de-US, then it falls back to de. If the user selects zh-TW as its application locale,
then it falls back to en.
A fallback mechanism is often necessary if the number of supported application
locales is greater than the number of the translation locales. This usually happens if
multiple locales share one translation. One example is Spanish. The application may
need to support multiple Spanish-speaking countries and not just Spain, with one set
of translation files.
Multiple URL rewrite rules can be specified by assigning the name attribute to
nondefault URL rewrite rules. To use the nondefault URL rewrite rules, the name must
be passed as a parameter of the rewrite URL method. For example:
<img src="<%=ServletHelper.rewriteURL("images/welcome.gif", request) %>">
<img src="<%=ServletHelper.rewriteURL("US.gif", "flag", request) %>">
The first rule changes the "images/welcome.gif" URL to the localized welcome
image file. The second rule named "flag" changes the "US.gif" URL to the user's
country flag image file. The rule definition should be as follows:
<url-rewrite-rule fallback="yes">
<pattern>(.*)/([^/]+)$</pattern>
<result>$1/$L/$2</result>
</url-rewrite-rule>
<url-rewrite-rule name="flag">
<pattern>US.gif/pattern>
<result>$C.gif</result>
</url-rewrite-rule>
■ The page character set for the en and de locales when using an Internet Explorer
client is windows-1252.
■ The page character set for the en, de, and fr locales on other web browser clients
is iso-8859-1.
■ The page character sets for all other locales are the default character set for the
locale.
■ The user locale is determined by the following order: user input locale and then
Accept-Language.
■ The localized contents are stored in their appropriate language subfolders. The
folder names are derived from the ISO 639 language code. The folders are located
in the root directory of the application. For example, the Japanese file for
/shop/welcome.jpg is stored in /ja/shop/welcome.jpg.
<?xml version="1.0" encoding="utf-8"?>
<gdkapp
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="gdkapp.xsd">
<!-- Language to Character set mapping -->
<locale-charset-maps>
<locale-charset>
<locale>ja</locale>
<charset>UTF-8</charset>
</locale-charset>
<locale-charset>
<locale>en,de</locale>
<user-agent>^Mozilla\/[0-9\. ]+\(compatible; MSIE [^;]+; \)</user-agent>
<charset>WINDOWS-1252</charset>
</locale-charset>
<locale-charset>
<locale>en,de,fr</locale>
<charset>ISO-8859-1</charset>
</locale-charset>
</locale-charset-maps>
■ oracle.i18n.net
■ oracle.i18n.servlet
■ oracle.i18n.text
■ oracle.i18n.util
oracle.i18n.lcsd
Package oracle.i18n.lcsd provides classes to automatically detect and recognize
language and character set based on text input. It supports the detection of both plain
text and HTML files. Language is based on ISO; encoding is based on IANA or Oracle
character sets. It includes the following classes:
■ LCSDetector: Contains methods to automatically detect and recognize language
and character set based on text input.
■ LCSDResultSet: The LCSDResultSet class is for storing the result generated by
LCSDetector. Methods in this class can be used to retrieve specific information
from the result.
■ LCSDetectionInputStream: Transparently detects the language and encoding
for the stream object.
■ LCSDetectionReader: Transparently detects the language and encoding and
converts the input data to Unicode.
■ LCSDetectionHTMLInputStream: Extends the LCSDetectionInputStream
class to support the language and encoding detection for input in HTML format.
■ LCSDetectionHTMLReader: Extends the LCSDetectionReader class to
support the language and encoding detection for input in HTML format.
oracle.i18n.net
Package oracle.i18n.net provides Internet-related data conversions for
globalization. It includes the following classes:
■ CharEntityReference: A utility class to escape or unescape a string into
character reference or entity reference form
■ CharEntityReference.Form: A form parameter class that specifies the
escaped form
oracle.i18n.servlet
Package oracle.i18n.Servlet enables JSP and JavaServlet to have automatic
locale support and also returns the localized contents to the application. It includes the
following classes:
■ ApplicationContext: An application context class that governs application
scope operation in the framework
■ Localizer: An all-in-one object class that enables access to the most commonly
used globalization information
■ ServletHelper: A delegate class that bridges between Java servlets and
globalization objects
oracle.i18n.text
Package oracle.i18n.text provides general text data globalization support. It
includes the following classes:
■ OraCollationKey: A class which represents a String under certain rules of a
specific OraCollator object
■ OraCollator: A class to perform locale-sensitive string comparison, including
linguistic collation and binary sorting
■ OraDateFormat: An abstract class to do formatting and parsing between
datetime and string locale. It supports Oracle datetime formatting behavior.
■ OraDecimalFormat: A concrete class to do formatting and parsing between
number and string locale. It supports Oracle number formatting behavior.
■ OraDecimalFormatSymbol: A class to maintain Oracle format symbols used by
Oracle number and currency formatting
■ OraNumberFormat: An abstract class to do formatting and parsing between
number and string locale. It supports Oracle number formatting behavior.
■ OraSimpleDateFormat: A concrete class to do formatting and parsing between
datetime and string locale. It supports Oracle datetime formatting behavior.
oracle.i18n.util
Package oracle.i18n.util provides general utilities for globalization support. It
includes the following classes:
■ LocaleMapper: Provides mappings between Oracle locale elements and
equivalent locale elements in other vendors and standards
■ OraDisplayLocaleInfo: A translation utility class that provides the
translations of locale and attributes
■ OraLocaleInfo: An Oracle locale class that includes the language, territory, and
collator objects
■ OraSQLUtil: An Oracle SQL Utility class that includes some useful methods of
dealing with SQL
GDK-03005 Binary sort is not available for the specified character set .
Cause: Binary sorting for the specified character set is not supported.
Action: Check the Globalization Support Guide for a character set that supports
binary sort.
Action: Write a separate exception handler for the invalid character, or call the
withReplacement method so that the invalid character can be replaced with a valid
replacement character.
GDK-05011 The Julian date precludes the use of the day of the year.
Cause: Both the Julian date and the day of the year were specified.
Action: Remove either the Julian date or the day of the year.
GDK-05022 The era format code is not valid with this calendar.
Cause: An invalid era format code was specified for the calendar.
Action: Remove the era format code or use anther calendar that supports the era.
GDK-05030 The date format pattern ends before converting entire input string.
Cause: An incomplete date format pattern was specified.
Action: Rewrite the format pattern to cover the entire input string.
GDK-05032 The day of the year conflicts with the Julian date.
Cause: An incompatible day of year was specified for the Julian date.
Action: Make sure that the Julian date and the day of the year are not in conflict.
GDK-05034 The day of the month conflicts with the Julian date.
Cause: An incompatible day of the month was specified for the Julian date.
Action: Make sure that the Julian date and the day of the month are not in
conflict.
GDK-05035 The day of the week conflicts with the Julian date.
Cause: An incompatible day of the week was specified for the Julian date.
Action: Make sure that the Julian date and the day of week are not in conflict.
GDK-05037 The minutes of the hour conflicts with the seconds in the day.
Cause: The specified minutes of the hour and the seconds in the day were not
compatible.
Action: Make sure the minutes of the hour and the seconds in the day are not in
conflict.
GDK-05038 The seconds of the minute conflicts with the seconds in the day.
Cause: The specified seconds of the minute and the seconds in the day were not
compatible.
Action: Make sure the seconds of the minute and the seconds in the day are not in
conflict.
GDK-05040 Input value not long enough for the date format
Cause: Too many format codes were specified.
Action: Remove unused format codes or specify a longer value.
GDK-05041 A full year must be between -4713 and +9999, and not be 0.
Cause: An illegal year was specified.
Action: Specify the year in the specified range.
GDK-05047 A day of the month must be between 1 and the last day of the month.
Cause: An illegal day of the month was specified.
Action: Make sure that the day of the month is in the specified range.
GDK-05048 A day of year must be between 1 and 365 (366 for leap year).
Cause: An illegal day of the year was specified.
Action: Make sure that the day of the year is in the specified range.
GDK-05062 The numeric value does not match the length of the format item.
Cause: The numeric value did not match the length of the format item.
Action: Correct the input date or turn off the FX or FM format modifier. When the
FX and FM format codes are specified for an input date, then the number of digits
must be exactly the number specified by the format code. For example, 9 will not
match the format code DD but 09 will.
GDK-05070 The specified intervals and datetimes were not mutually comparable.
Cause: The specified intervals and datetimes were not mutually comparable.
Action: Specify a pair of intervals or datetimes that are mutually comparable.
GDK-05078 The specified field was not found in the datetime or interval.
Cause: The specified field was not found in the datetime or interval.
Action: Make sure that the specified field is in the datetime or interval.
Cause: Reserved.
Action: Reserved.
Action: Make sure the string includes the hexadecimal character in the form of
&x[0-9A-Fa-f]+;.
GDK-05207 Invalid class of the object, key, in the user-defined locale to charset
mapping"
Cause: The class of key object in the user-defined locale to character set mapping
table was not java.util.Locale.
Action: When you construct the Map object for the user-defined locale to
character set mapping table, specify java.util.Locale for the key object.
GDK-05208 Invalid class of the object, value, in the user-defined locale to charset
mapping
Cause: The class of value object in the user-defined locale to character set
mapping table was not java.lang.String.
Action: When you construct the Map object for the user-defined locale to
character set mapping table, specify java.lang.String for the value object.
GDK-05212 The rewriting rule must be a String array with three elements.
Cause: The rewriting rule parameter was not a String array with three elements.
Action: Make sure the rewriting rule parameter is a String array with three
elements. The first element represents the match pattern in the regular expression,
the second element represents the result pattern in the form specified in the
JavaDoc of ServletHelper.rewriteURL, and the third element represents the
Boolean value "True" or "False" that specifies whether the locale fallback operation
is performed or not.
GDK-05213 Invalid type for the class of the object, key, in the user-defined
parameter name mapping
Cause: The class of key object in the user-defined parameter name mapping table
was not java.lang.String.
Action: When you construct the Map object for the user-defined parameter name
mapping table, specify java.lang.String for the key object.
GDK-05214 The class of the object, value, in the user-defined parameter name
mapping, must be of type \"java.lang.String\".
Cause: The class of value object in the user-defined parameter name mapping
table was not java.lang.String.
Action: When you construct the Map object for the user-defined parameter name
mapping table, specify java.lang.String for the value object.
GDK-05216 The attribute \"var\" must be specified if the attribute \"scope\" is set.
Cause: Despite the attribute "scope" being set in the tag, the attribute "var" was
not specified.
Action: Specify the attribute "var" for the name of variable.
GDK-05220 No corresponding Oracle character set exists for the IANA character set.
GDK-05222 Invalid type for the class of the object, key, in the user-defined message
bundle mapping.
Cause: The class of key object in the user-defined message bundle mapping table
was not "java.lang.String."
Action: When you construct the Map object for the user-defined message bundle
mapping table, specify java.lang.String for the key object.
GDK-05223 Invalid type for the class of the object, value, in the user-defined
message bundle mapping
Cause: The class of value object in the user-defined message bundle mapping
table was not "java.lang.String."
Action: When you construct the Map object for the user-defined message bundle
mapping table, specify java.lang.String for the value object.
GDK-06004 A character set filter and a language filter cannot be set at the same
time.
Cause: A character set filter and a language filter were set at the same time in a
LCSDetector object.
GDK-06005 Reset is necessary before LCSDetector can work with a different data
source.
Cause: The reset method was not invoked before a different type of data source
was used for a LCSDetector object.
Action: Call LCSDetector.reset to reset the detector before switching to detect
other types of data source.
Explicitly specifying the optional NLS parameters for these functions enables the
functions to be evaluated independently of the session's NLS parameters. This feature
can be important for SQL statements that contain numbers and dates as string literals.
For example, the following query is evaluated correctly if the language specified for
dates is AMERICAN:
SELECT last_name FROM employees WHERE hire_date > '01-JAN-1999';
Such a query can be made independent of the current date language by using a
statement similar to the following:
SELECT last_name FROM employees
WHERE hire_date > TO_DATE('01-JAN-1999','DD-MON-YYYY',
'NLS_DATE_LANGUAGE = AMERICAN');
In this way, SQL statements that are independent of the session language can be
defined where necessary. Such statements are necessary when string literals appear in
SQL statements in views, CHECK constraints, or triggers.
All character functions support both single-byte and multibyte characters. Except
where explicitly stated, character functions operate character by character, rather than
byte by byte.
The rest of this section includes the following topics:
■ Default Values for NLS Parameters in SQL Functions
■ Specifying NLS Parameters in SQL Functions
■ Unacceptable NLS Parameters in SQL Functions
For example:
'NLS_DATE_LANGUAGE = AMERICAN'
Table 9–1 shows which NLS parameters are valid for specific SQL functions.
Table 9–1 (Cont.) SQL Functions and Their Valid NLS Parameters
SQL Function Valid NLS Parameters
TO_CHAR NLS_DATE_LANGUAGE
NLS_NUMERIC_CHARACTERS
NLS_CURRENCY
NLS_ISO_CURRENCY
NLS_DUAL_CURRENCY
NLS_CALENDAR
TO_NCHAR NLS_DATE_LANGUAGE
NLS_NUMERIC_CHARACTERS
NLS_CURRENCY
NLS_ISO_CURRENCY
NLS_DUAL_CURRENCY
NLS_CALENDAR
NLS_UPPER NLS_SORT
NLS_LOWER NLS_SORT
NLS_INITCAP NLS_SORT
NLSSORT NLS_SORT
The following examples show how to use NLS parameters in SQL functions:
TO_DATE ('1-JAN-99', 'DD-MON-YY',
'nls_date_language = American')
■ NLS_LANGUAGE
■ NLS_TERRITORY
■ NLS_DATE_FORMAT
NLS_DATE_FORMAT and NLS_TERRITORY_FORMAT are not accepted as parameters
because they can interfere with required format masks. A date format must always be
specified if an NLS parameter is in a TO_CHAR or TO_DATE function. As a result, NLS_
DATE_FORMAT and NLS_TERRITORY_FORMAT are not valid NLS parameters for the
TO_CHAR or TO_DATE functions. If you specify NLS_DATE_FORMAT or NLS_
TERRITORY_FORMAT in the TO_CHAR or TO_DATE function, then an error is returned.
NLS_LANGUAGE can interfere with the session value of NLS_DATE_LANGUAGE. If you
specify NLS_LANGUAGE in the TO_CHAR function, for example, then its value is
ignored if it differs from the session value of NLS_DATE_LANGUAGE.
char is the value to be converted. source_char_set is the source character set and
dest_char_set is the destination character set. If the source_char_set parameter
is not specified, then it defaults to the database character set.
See Also:
■ Oracle Database SQL Reference for more information about the
CONVERT function
■ "Character Set Conversion Support" on page A-16 for character
set encodings that are used only for the CONVERT function
Table 9–2 How the SUBSTR Functions Calculate the Length of a String
Function Calculation Method
SUBSTR Calculates the length of a string in characters based on the length semantics associated with the
character set of the datatype. For example, AL32UTF8 characters are calculated in UCS-4
characters. UTF8 and AL16UTF16 characters are calculated in UCS-2 characters. A supplementary
character is counted as one character in AL32UTF8 and as two characters in UTF8 and
AL16UTF16. Because VARCHAR and NVARCHAR may use different character sets, SUBSTR may
give different results for different datatypes even if two strings are identical. If your application
requires consistency, then use SUBSTR2 or SUBSTR4 to force all semantic calculations to be
UCS-2 or UCS-4, respectively.
SUBSTRB Calculates the length of a string in bytes.
SUBSTR2 Calculates the length of a string in UCS-2 characters, which is compliant with Java strings and
Windows client environments. Characters are represented in UCS-2 or 16-bit Unicode values.
Supplementary characters are counted as two characters.
SUBSTR4 Calculates the length of a string in UCS-4 characters. Characters are represented in UCS-4 or
32-bit Unicode values. Supplementary characters are counted as one character.
SUBSTRC Calculates the length of a string in Unicode composed characters. Supplementary characters and
composed characters are counted as one character.
The LENGTH and INSTR functions calculate string length in the same way, according to
the character or number added to the function name.
The following examples demonstrate the differences between SUBSTR and SUBSTRB
on a database whose character set is AL32UTF8.
For the string Fußball, the following statement returns a substring that is 4
characters long, beginning with the second character:
SELECT SUBSTR ('Fußball', 2 , 4) SUBSTR FROM DUAL;
SUBS
----
ußba
For the string Fußball, the following statement returns a substring 4 bytes long,
beginning with the second byte:
SELECT SUBSTRB ('Fußball', 2 , 4) SUBSTRB FROM DUAL;
SUB
---
ußb
If the value CHAR_CS is entered for text, then the function returns the ID of the
server's database character set. If the value NCHAR_CS is entered for text, then the
function returns the ID of the server's national character set. The function returns NULL
if text is not a recognized name.
Example 9–1 Specifying a German Sort with the NLS_SORT Session Parameter
ALTER SESSION SET NLS_SORT = GERMAN;
SELECT * FROM table1
ORDER BY column1;
The WHERE clause uses binary comparison when NLS_COMP is set to BINARY, but this
can be overridden by using the NLSSORT function in the WHERE clause.
Setting the NLS_COMP session parameter to LINGUISTIC causes the NLS_SORT value
to be used in the WHERE clause.
The rest of this section contains the following topics:
■ NLSSORT Syntax
■ Comparing Strings in a WHERE Clause
NLSSORT Syntax
There are four ways to use NLSSORT:
■ NLSSORT(), which relies on the NLS_SORT parameter
■ NLSSORT(column1, 'NLS_SORT=xxxx')
■ NLSSORT(column1, 'NLS_LANG=xxxx')
■ NLSSORT(column1, 'NLS_LANGUAGE=xxxx')
The NLS_LANG parameter of the NLSSORT function is not the same as the NLS_LANG
client environment setting. In the NLSSORT function, NLS_LANG specifies the
abbreviated language name, such as US for American or PL for Polish. For example:
SELECT * FROM table1
ORDER BY NLSSORT(column1, 'NLS_LANG=PL');
Note that NLSSORT must be on both sides of the comparison operator. For example:
SELECT column1 FROM table1 WHERE NLSSORT(column1) > NLSSORT('B');
If a German linguistic sort has been set, then the statement does not return strings
beginning with Ä because Ä comes before B in the German alphabet. If a Swedish
linguistic sort has been set, then strings beginning with Ä are returned because Ä
comes after Z in the Swedish alphabet.
abbreviations "Lu" and "Ma" can be specified for the French "Lundi" and "Mardi",
respectively.
For Roman numerals, you can specify either upper case or lower case, using RN or rn,
respectively. The number being converted must be an integer in the range 1 to 3999.
the national character set for the NCLOB datatype. The loading takes place on the
server because BFILE data is not supported on the client.
■ Use DBMS_LOB.LOADBLOBFROMFILE to load BLOB columns.
■ Use DBMS_LOB.LOADCLOBFROMFILE to load CLOB and NCLOB columns.
See Also:
■ PL/SQL Packages and Types Reference
■ Oracle Database Application Developer's Guide - Large Objects
See Also:
■ Oracle Call Interface Programmer’s Guide
■ "OCI Programming with Unicode" on page 7-10
lmsgen Utility
Purpose
The lmsgen utility converts text-based message files (.msg) into binary format (.msb)
so that Oracle messages and OCI messages provided by the user can be returned to
OCI functions in the desired language.
Messages used by the server are stored in binary-format files that are placed in the
$ORACLE_HOME/product_name/mesg directory, or the equivalent for your
operating system. Multiple versions of these files can exist, one for each supported
language, using the following filename convention:
<product_id><language_abbrev>.msb
For example, the file containing the server messages in French is called oraf.msb,
because ORA is the product ID (<product_id>) and F is the language abbreviation
(<language_abbrev>) for French. The value for product_name is rdbms, so it is in
the $ORACLE_HOME/rdbms/mesg directory.
Syntax
LMSGEN text_file product facility [language] [-i indir] [-o outdir]
Parameter Value
product myapp
facility imp
language AMERICAN
text_file impus.msg
The lmsgen utility converts the text message file (impus.msg) into binary format,
resulting in a file called impus.msb. The directory $ORACLE_HOME/myapp/mesg
must already exist.
% lmsgen impus.msg myapp imp AMERICAN
This chapter discusses character set conversion and character set migration. It includes
the following topics:
■ Overview of Character Set Migration
■ Changing the Database Character Set of an Existing Database
■ Migrating to NCHAR Datatypes
■ Tasks to Recover Database Schema After Character Set Migration
Data Truncation
When the database is created using byte semantics, the sizes of the CHAR and
VARCHAR2 datatypes are specified in bytes, not characters. For example, the
specification CHAR(20) in a table definition allows 20 bytes for storing character data.
When the database character set uses a single-byte character encoding scheme, no data
loss occurs when characters are stored because the number of characters is equivalent
to the number of bytes. If the database character set uses a multibyte character set,
then the number of bytes no longer equals the number of characters because a
character can consist of one or more bytes.
During migration to a new character set, it is important to verify the column widths of
existing CHAR and VARCHAR2 columns because they may need to be extended to
support an encoding that requires multibyte storage. Truncation of data can occur if
conversion causes expansion of data.
Table 11–1 shows an example of data expansion when single-byte characters become
multibyte characters through conversion.
The first column of Table 11–1 shows selected characters. The second column shows
the hexadecimal representation of the characters in the WE8MSWIN1252 character set.
The third column shows the hexadecimal representation of each character in the
AL32UTF8 character set. Each pair of letters and numbers represents one byte. For
example, ä (a with an umlaut) is a single-byte character (E4) in WE8MSWIN1252, but
it becomes a two-byte character (C3 A4) in AL32UTF8. Also, the encoding for the euro
symbol expands from one byte (80) to three bytes (E2 82 AC).
If the data in the new character set requires storage that is greater than the supported
byte size of the datatypes, then you need to change your schema. You may need to use
CLOB columns.
■ When CHAR data contains characters that expand after migration to a new
character set, space padding is not removed during database export by default.
This means that these rows will be rejected upon import into the database with the
new character set. The workaround is to set the BLANK_TRIMMING initialization
parameter to TRUE before importing the CHAR data.
See Also: Oracle Database Reference for more information about the
BLANK_TRIMMING initialization parameter
Replacement Characters that Result from Using the Export and Import Utilities
The Export and Import utilities can convert character sets from the original database
character set to the new database character set. However, character set conversions can
sometimes cause data loss or data corruption. For example, if you are migrating from
character set A to character set B, then the destination character set B should be a
superset of character set A. The destination character, B, is a superset if it contains all
the characters defined in character set A. Characters that are not available in character
set B are converted to replacement characters, which are often specified as ? or ¿ or as
a character that is related to the unavailable character. For example, ä (a with an
umlaut) can be replaced by a. Replacement characters are defined by the target
character set.
Figure 11–1 shows an example of a character set conversion in which the copyright
and euro symbols are converted to ? and ä is converted to a.
To reduce the risk of losing data, choose a destination character set with a similar
character repertoire. Migrating to Unicode can be an attractive option because
AL32UTF8 contains characters from most legacy character sets.
Invalid Data That Results from Setting the Client's NLS_LANG Parameter
Incorrectly
Another character set migration scenario that can cause the loss of data is migrating a
database that contains invalid data. Invalid data usually occurs in a database because
the NLS_LANG parameter is not set properly on the client. The NLS_LANG value should
reflect the client operating system code page. For example, in an English Windows
environment, the code page is WE8MSWIN1252. When the NLS_LANG parameter is set
properly, the database can automatically convert incoming data from the client
operating system. When the NLS_LANG parameter is not set properly, then the data
coming into the database is not converted properly. For example, suppose that the
database character set is AL32UTF8, the client is an English Windows operating
system, and the NLS_LANG setting on the client is AL32UTF8. Data coming into the
database is encoded in WE8MSWIN1252 and is not converted to AL32UTF8 data
because the NLS_LANG setting on the client matches the database character set. Thus
Oracle assumes that no conversion is necessary, and invalid data is entered into the
database.
This can lead to two possible data inconsistency problems. One problem occurs when
a database contains data from a character set that is different from the database
character set but the same code points exist in both character sets. For example, if the
database character set is WE8ISO8859P1 and the NLS_LANG setting of the Chinese
Windows NT client is SIMPLIFIED CHINESE_CHINA.WE8ISO8859P1, then all
multibyte Chinese data (from the ZHS16GBK character set) is stored as multiples of
single-byte WE8ISO8859P1 data. This means that Oracle treats these characters as
single-byte WE8ISO8859P1 characters. Hence all SQL string manipulation functions
such as SUBSTR or LENGTH are based on bytes rather than characters. All bytes
constituting ZHS16GBK data are legal WE8ISO8859P1 codes. If such a database is
migrated to another character set such as AL32UTF8, then character codes are
converted as if they were in WE8ISO8859P1. This way, each of the two bytes of a
ZHS16GBK character are converted separately, yielding meaningless values in
AL32UTF8. Figure 11–2 shows an example of this incorrect character set replacement.
Database Server
(WE8ISO8859P1)
Simplified Chinese
Windows NT
(WE8ISO8859P1)
The second possible problem is having data from mixed character sets inside the
database. For example, if the data character set is WE8MSWIN1252, and two separate
Windows clients using German and Greek are both using WE8MSWIN1252 as the
NLS_LANG character set, then the database contains a mixture of German and Greek
characters. Figure 11–3 shows how different clients can use different character sets in
the same database.
Database Server
(WE8MSWIN1252)
0xE4 0xF6
= 0xE4 = 0xE4
= 0xF6 = 0xF6
For database character set migration to be successful, both of these cases require
manual intervention because Oracle cannot determine the character sets of the data
being stored. Incorrect data conversion can lead to data corruption, so perform a full
backup of the database before attempting to migrate the data to a new character set.
character set conversion problems and truncation of data. This step is called data
scanning.
Data scanning identifies the amount of effort required to migrate data into the new
character encoding scheme before changing the database character set. Some examples
of what may be found during a data scan are the number of schema objects where the
column widths need to be expanded and the extent of the data that does not exist in
the target character repertoire. This information helps to determine the best approach
for converting the database character set.
Incorrect data conversion can lead to data corruption, so perform a full backup of the
database before attempting to migrate the data to a new character set.
There are three approaches to converting data from one database character set to
another if the database does not contain any of the inconsistencies described in
"Character Set Conversion Issues" on page 11-3. A description of methods to migrate
databases with such inconsistencies is out of the scope of this documentation. For
more information, contact Oracle Consulting Services for assistance.
The approaches are:
■ Migrating Character Data Using a Full Export and Import
■ Migrating a Character Set Using the CSALTER Script
■ Migrating Character Data Using the CSALTER Script and Selective Imports
See Also: Chapter 12, "Character Set Scanner Utilities" for more
information about data scanning
See Also: Oracle Database Utilities for more information about the
Export and Import utilities
data dictionary and sample schemas are stored in CLOB columns. Customers may also
store data in CLOB fields. When the database character set is multibyte, then CLOB data
is stored in a format that is compatible with UCS-2 data. When the database character
set is single-byte, then CLOB data is stored using the database character set. Because
the CSALTER script converts data only in CLOB columns in the data dictionary and
sample schemas that were created by Oracle, any other CLOB columns that are created
must be first exported and then dropped from the schema before the CSALTER script
can be run.
To change the database character set, perform the following steps:
1. Shut down the database, using either a SHUTDOWN IMMEDIATE or a SHUTDOWN
NORMAL statement.
2. Do a full backup of the database, because the CSALTER script cannot be rolled
back.
3. Start up the database.
4. Run the Database Character Set Scanner utility.
CSSCAN /AS SYSDBA FULL=Y...
Note that the CSALTER script does not perform any user data conversion. It only
changes the character set metadata in the data dictionary.Thus, after the CSALTER
operation, Oracle will behave as if the database was created using the new character
set.
See Also:
■ "Migrating Character Data Using the CSALTER Script and
Selective Imports" on page 11-7
■ "Database Character Set Scanner CSALTER Script" on
page 12-33
Migrating Character Data Using the CSALTER Script and Selective Imports
Another approach to migrating character data is to perform selective exports followed
by rescanning and running the CSALTER script. This approach is most common when
the subset character set is single-byte and the migration is to a multibyte character set.
In this scenario, user-created CLOBs must be converted because the encoding changes
from the single- byte character set to a UCS-2-compatible format which Oracle uses for
storage of CLOBs regardless of the multibyte encoding. The Database Character Set
Scanner identifies these columns as convertible. It is up to the user to export these
columns and then drop them from the schema, rescan, and, if the remaining data is
clean, run the CSALTER script. When these steps have been completed, then import the
CLOB columns to the database to complete migration.
See Also:
■ Oracle Database Utilities for a description of export and import
procedures
■ Oracle Database Upgrade Guide for NCHAR migration information
See Also: Chapter 12, "Character Set Scanner Utilities" for the
syntax of the CSALTER script
Using the ALTER TABLE MODIFY Statement to Change CHAR Columns to NCHAR
Columns
The ALTER TABLE MODIFY statement can be used to change table column definitions
from the CHAR datatypes to NCHAR datatypes. It also converts all of the data in the
column from the database character set to the NCHAR character set. The syntax of the
ALTER TABLE MODIFY statement is as follows:
ALTER TABLE table_name MODIFY (column_name datatype);
If indexes have been built on the migrating column, then dropping the indexes can
improve the performance of the ALTER TABLE MODIFY statement because indexes
are updated when each row is updated.
The maximum column lengths for NCHAR and NVARCHAR2 columns are 2000 and 4000
bytes. When the NCHAR character set is AL16UTF16, the maximum column lengths for
NCHAR and NVARCHAR2 columns are 1000 and 2000 characters, which are 2000 and
4000 bytes. If this size limit is violated during migration, then consider changing the
column to the NCLOB datatype instead.
2. Create an empty interim table in the same schema as the table that is to be
redefined. Create it with NCHAR datatypes as the attributes. For example, enter a
statement similar to the following:
CREATE TABLE int_emp(
empno NUMBER(4),
ename NVARCHAR2(10),
job NVARCHAR2(9),
mgr NUMBER(4),
hiredate DATE,
sal NUMBER(7,2),
deptno NUMBER(2),
org NVARCHAR2(10));
3. Start the online table redefinition. Enter a command similar to the following:
DBMS_REDEFINITION.START_REDEF_TABLE('scott',
'emp',
'int_emp',
'empno empno,
to_nchar(ename) ename,
to_nchar(job) job,
mgr mgr,
hiredate hiredate,
sal sal,
deptno deptno,
to_nchar(org) org');
If you are migrating CLOB columns to NCLOB columns, then use the TO_NCLOB
SQL conversion function instead of the TO_NCHAR SQL function.
4. Create triggers, indexes, grants, and constraints on the interim table. Referential
constraints that apply to the interim table (the interim table is a parent or child
table of the referential constraint) must be created in DISABLED mode. Triggers
that are defined on the interim table are not executed until the online table
redefinition process has been completed.
5. You can synchronize the interim table with the original table. If many DML
operations have been applied to the original table since the online redefinition
began, then execute the DBMS_REDEFINITION.SYNC_INTERIM_TABLE
procedure. This reduces the time required for the DBMS_
REDEFINITION.FINISH_REDEF_TABLE procedure. Enter a command similar to
the following:
DBMS_REDEFINITION.SYNC_INTERIM_TABLE('scott', 'emp', 'int_emp');
When this procedure has been completed, the following conditions are true:
■ The original table is redefined so that it has all the attributes, indexes,
constraints, grants, and triggers of the interim table.
■ The referential constraints that apply to the interim table apply to the
redefined original table.
7. Drop the interim table. Enter a statement similar to the following:
DROP TABLE int_emp;
The character set scanner utilities are tools for detecting and verifying valid and
invalid data. The Language and Character Set File Scanner supports text files, while
the Database Character Set Scanner scans data inside the database.
This chapter introduces the Language and Character Set File Scanner and the Database
Character Set Scanner. The topics in this chapter include:
■ The Language and Character Set File Scanner
■ The Database Character Set Scanner
■ Scan Modes in the Database Character Set Scanner
■ Installing and Starting the Database Character Set Scanner
■ Database Character Set Scanner Parameters
■ Database Character Set Scanner Sessions: Examples
■ Database Character Set Scanner Reports
■ How to Handle Convertible or Lossy Data in the Data Dictionary
■ Storage and Performance Considerations in the Database Character Set Scanner
■ Database Character Set Scanner CSALTER Script
■ Database Character Set Scanner Views
■ Database Character Set Scanner Error Messages
language successfully is statistically reduced. Both plain text and HTML files are
accepted. If the format is known, you should set the FORMAT parameter to improve
accuracy.
This section includes the following topics:
■ Syntax of the LCSSCAN Command
■ Examples: Using the LCSSCAN Command
■ Getting Command-Line Help for the Language and Character Set File Scanner
■ Supported Languages and Character Sets
■ LCSSCAN Error Messages
RESULTS
The RESULTS parameter is optional.
Property Description
Default value 1
Minimum value 1
Maximum value 3
Purpose The number of language and character set pairs that are returned.
They are listed in order of probability. The comparative weight of
the first choice cannot be quantified. The recommended value for
this parameter is the default value of 1.
FORMAT
The FORMAT paramater is optional.
Property Description
Default Value text
Purpose This parameter identifies the type of file to be scanned. The possible
values are html, text, and auto.
BEGIN
The BEGIN parameter is optional.
Property Description
Default value 1
Minimum value 1
Maximum value Number of bytes in file
Property Description
Purpose The byte of the input file where LCSSCAN begins the scanning
process. The default value is the first byte of the input file.
END
The END parameter is optional.
Property Description
Default value End of file
Minimum value 3
Maximum value Number of bytes in file
Purpose The last byte of the input file that LCSSCAN scans. The default value
is the last byte of the input file.
FILE
The FILE parameter is required.
Property Description
Default value None
Purpose Specifies the name of a text file to be scanned
Example 12–1 Specifying Only the File Name in the LCSSCAN Command
LCSSCAN FILE=example.txt
In this example, the entire example.txt file is scanned because the BEGIN and END
parameters have not been specified. One language and character set pair will be
returned because the RESULTS parameter has not been specified.
In this example, the entire example.html file is scanned because the BEGIN and END
parameters have not been specified. The scan will strip HTML tags before the scan,
thus results are more accurate. One language and character set pair will be returned
because the RESULTS parameter has not been specified.
Example 12–3 Specifying the RESULTS and BEGIN Parameters for LCSSCAN
LCSSCAN RESULTS=2 BEGIN=50 FILE=example.txt
The scanning process starts at the 50th byte of the file and continues to the end of the
file. Two language and character set pairs will be returned.
Example 12–4 Specifying the RESULTS and END Parameters for LCSSCAN
LCSSCAN RESULTS=3 END=100 FILE=example.txt
The scanning process starts at the beginning of the file and ends at the 100th byte of
the file. Three language and character set pairs will be returned.
Example 12–5 Specifying the BEGIN and END Parameters for LCSSCAN
LCSSCAN BEGIN=50 END=100 FILE=example.txt
The scanning process starts at the 50th byte and ends at the 100th byte of the file. One
language and character set pair will be returned because the RESULTS parameter has
not been specified.
Getting Command-Line Help for the Language and Character Set File Scanner
To obtain a summary of the Language and Character Set File Scanner parameters,
enter the following command:
LCSSCAN HELP=y
The resulting output shows a summary of the Language and Character Set Scanner
parameters.
■ Can the data cells be successfully converted to the new character set?
■ Will the post-conversion data fit into the current column size?
The Database Character Set Scanner reads and tests for data in CHAR, VARCHAR2,
LONG, CLOB, NCHAR, NVARCHAR2, NCLOB and VARRAY columns as well as nested
tables. The Database Character Set Scanner does not perform post-conversion column
size testing for LONG, CLOB, and NCLOB columns.
User Scan
The Database Character Set Scanner reads and verifies character data of all tables
belonging to the specified user and reports on the effects on the tables of changing the
character set.
Table Scan
The Database Character Set Scanner reads and verifies the character data of the
specified tables, and reports the effects on the tables of changing the character set.
Column Scan
The Database Character Set Scanner reads and verifies the character data of the
specified columns, and reports the effects on the tables of changing the character set.
Ensure that there is sufficient storage space available in the assigned tablespace before
scanning the database. The amount of space required depends on the type of scan and
the nature of the data in the database.
On UNIX platforms, run csminst.sql using these commands and SQL statement:
% cd $ORACLE_HOME/rdbms/admin
% sqlplus sys/password as sysdba
SQL> START csminst.sql
In an interactive session, the Database Character Set Scanner prompts you for the
values of the following parameters:
FULL/TABLE/USER
TOCHAR
ARRAY
PROCESS
If you want to specify other parameters, then use the Database Character Set
Parameter file or specify the parameter values in the CSSCAN command.
You can add comments to the parameter file by preceding them with the pound sign
(#). All characters to the right of the pound sign are ignored.
The following is an example of a parameter file:
USERID=system/manager
USER=HR # scan HR tables
TOCHAR=al32utf8
ARRAY=4096000
PROCESS=2 # use two concurrent scan processes
FEEDBACK=1000
The resulting output shows a summary of the Database Character Set Scanner
parameters.
ARRAY
Property Description
Default value 1024000
Minimum value 4096
Maximum value Unlimited
Purpose Specifies the size in bytes of the array buffer used to fetch data. The
size of the array buffer determines the number of rows fetched by
the Database Character Set Scanner at any one time.
The following formula estimates the number of rows fetched at one time for a given
table:
rows fetched =
(ARRAY value) / [(sum of all the CHAR and VARCHAR2 column sizes) +
(number of CLOB columns * 4000) + (number of VARRAY columns * 4000)]
For example, suppose table A contains two CHAR columns (5 bytes and 10 bytes), two
VARCHAR2 columns (100 bytes and 200 bytes), and one CLOB column. If
ARRAY=1024000 (the default), then the number of rows fetched is calculated as
follows:
1024000/[5 + 10 + 100 + 200 + (1*4000) + (0*4000)] = 237.3
The Database Character Set Scanner can fetch 23 rows of data at one time from table A.
If the sum in the denominator exceeds the value of the ARRAY parameter, then the
Database Character Set Scanner fetches only one row at a time. Tables with LONG
columns are fetched only one row at a time.
This parameter affects the duration of a database scan. In general, the larger the size of
the array buffer, the shorter the duration time. Each scan process allocates the specified
size of array buffer.
BOUNDARIES
Property Description
Default value None
Purpose Specifies the list of column boundary sizes that are used for an
application data conversion summary report. This parameter is
used to locate the distribution of the application data for the CHAR,
VARCHAR2, NCHAR, and NVARCHAR2 datatypes.
For example, if you specify a BOUNDARIES value of (10, 100, 1000), then the
application data conversion summary report produces a breakdown of the CHAR data
into the following groups by their column length, CHAR(1..10), CHAR(11..100)
and CHAR(101..1000). The behavior is the same for the VARCHAR2, NCHAR, and
NVARCHAR2 datatypes.
CAPTURE
Property Description
Default value N
Range of values Y or N
Purpose Indicates whether to capture the information on the individual
convertible rows, as well as the default of storing only the exception
rows. Information regarding the convertible rows is written to the
CSM$ERRORS table if the CAPTURE parameter is set to Y. It records the
data that needs to be converted during the conversion to the target
character set. When CAPTURE is set to Y, the data dictionary
CONVERTIBLE data cells are also listed in the database scan individual
exception report scan.err. With CAPTURE set to Y, the amount of time
required to complete the scan can increase and more storage space may
be required.
COLUMN
Property Description
Default value None
Purpose Specifies the names of the columns to be scanned
When this parameter is specified, the Database Character Set Scanner scans the
specified columns. You can specify the following when you specify the name of the
column:
■ schemaname specifies the names of the user's schema from which to scan the table
■ tablename specifies the name of the table from which to scan the column
■ columnname specifies the name of the column to be scanned
For example, the following command scans the columns LASTNAME and FIRSTNAME
in the hr sample schema:
CSSCAN system/manager COLUMN=(HR.EMPLOYEES.LASTNAME, HR.EMPLOYEES.FIRSTNAME) ...
EXCLUDE
Property Description
Default value None
Purpose Specifies the names of the tables to be excluded from the scan
When this parameter is specified, the Database Character Set Scanner excludes the
specified tables from the scan. You can specify the following when you specify the
name of the table:
■ schemaname specifies the name of the user's schema from which to exclude the
table
■ tablename specifies the name of the table or tables to be excluded
For example, the following command scans all of the tables that belong to the hr
sample schema except for the employees and departments tables:
CSSCAN system/manager USER=HR EXCLUDE=(HR.EMPLOYEES , HR.DEPARTMENTS) ...
FEEDBACK
Property Description
Default value None
Minimum value 100
Maximum value 100000
Purpose Specifies that the Database Character Set Scanner should display a
progress meter in the form of a dot for every N number of rows
scanned
For example, if you specify FEEDBACK=1000, then the Database Character Set Scanner
displays a dot for every 1000 rows scanned. The FEEDBACK value applies to all tables
being scanned. It cannot be set for individual tables.
FROMCHAR
Property Description
Default value None
Purpose Specifies the current character set name for CHAR, VARCHAR2,
LONG, and CLOB datatypes in the database. By default, the Database
Character Set Scanner assumes the character set for these datatypes
to be the database character set.
Use this parameter to override the default database character set definition for CHAR,
VARCHAR2, LONG, and CLOB data in the database.
FROMNCHAR
Property Description
Default value The current database character set.
Purpose Specifies the current national database character set name for
NCHAR, NVARCHAR2, and NCLOB datatypes in the database. By
default, the Database Character Set Scanner assumes the character
set for these datatypes to be the database national character set.
Use this parameter to override the default database character set definition for NCHAR,
NVARCHAR2, and NCLOB data in the database.
FULL
Property Description
Default value N
Range of values Y or N
Purpose Indicates whether to perform the full database scan (that is, to scan
the entire database including the data dictionary). Specify FULL=Y
to scan in full database mode.
HELP
Property Description
Default value N
Range of values Y or N
Purpose Displays a help message with the descriptions of the Database
Character Set Scanner parameters
LASTRPT
Property Description
Default value N
Range of values Y or N
Purpose Indicates whether to regenerate the Database Character Set Scanner
reports based on statistics gathered from the previous database scan
If LASTRPT=Y is specified, then the Database Character Set Scanner does not scan the
database, but creates the report files using the information left by the previous
database scan session instead.
If LASTRPT=Y is specified, then only the USERID, BOUNDARIES, and LOG parameters
take effect.
LCSD
Property Description
Default value N
Range of values Y or N
Purpose Indicates whether to apply language and character set detection
during scanning
If LCSD=Y is specified, then the Database Character Set Scanner (CSSCAN) performs
language and character set detection on the data cells categorized by the LCSDDATA
parameter. The accuracy of the detection depends greatly on the size and the quality of
the text being analyzed. The ideal case is literary text of one single language with no
spelling or grammatical errors. Data cells that contain a mixture of languages or
character sets or text such as addresses and names can yield poor results. When
CSSCAN cannot determine the most likely language and character set, it may return up
to three most likely languages and character sets for each cell. In some cases it may
return none. CSSCAN ignores any data cells with less than 10 bytes of data and
returns UNKNOWN for their language and character set.
The language and character set detection is a statistically-based technology, so its
accuracy varies depending on the quality of the input data. The goal is to provide
CSSCAN users with additional information about unknown data inside the database. It
is important for CSSCAN users to review the detection results and the data itself before
migrating the data to another character set.
Note that language and character set detection can affect the performance of the
Database Character Set Scanner, depending on the amount of data that is being
analyzed.
LCSDDATA
Property Description
Default value LOSSY
Range of values LOSSY, TRUNCATION, CONVERTIBLE, CHANGELESS, ALL
Purpose Specifies the scope of the language and character set detection. The
default is to apply the detection to only the LOSSY data cells.
This parameter takes effect only when LCSD=Y is specified. For example, if LCSD=Y
and LCSDDATA=LOSSY, CONVERTIBLE, then the Database Character Set Scanner
tries to detect the character sets and languages of the data cells that are either LOSSY
or CONVERTIBLE. Data that is classified as CHANGELESS and TRUNCATION will not be
processed. Setting LCSDDATA=ALL results in language and character set detection for
all data cells scanned in the current session.
After language and character set detection has been applied to CONVERTIBLE and
TRUNCATION data cells, some data cells may change from their original classification
to LOSSY. This occurs when the character set detection process determines that the
character set of these data cells is not the character set specified in the FROMCHAR
parameter.
LOG
Property Description
Default value scan
Purpose Specifies a base file name for the following Database Character Set
Scanner report files:
■ Database Scan Summary Report file, whose extension is .txt
■ Individual Exception Report file, whose extension is .err
■ Screen log file, whose extension is .out
By default, the Database Character Set Scanner generates the three text files,
scan.txt, scan.err, and scan.out in the current directory.
MAXBLOCKS
Property Description
Default value None
Minimum value 1000
Maximum value Unlimited
Purpose Specifies the maximum block size for each table, so that large tables
can be split into smaller chunks for the Database Character Set
Scanner to process
For example, if the MAXBLOCKS parameter is set to 1000, then any tables that are
greater than 1000 blocks in size are divided into n chunks, where
n=CEIL(table block size/1000).
Dividing large tables into smaller pieces is beneficial only when the number of
processes set with PROCESS is greater than 1. If the MAXBLOCKS parameter is not set,
then the Database Character Set Scanner attempts to split up large tables based on its
own optimization rules.
PARFILE
Property Description
Default value None
Purpose Specifies the name for a file that contains a list of Database
Character Set Scanner parameters
PRESERVE
Property Description
Default value N
Range of values Y or N
Purpose Indicates whether to preserve the statistics gathered from the
previous scan session
If PRESERVE=Y is specified, then the Database Character Set Scanner preserves all of
the statistics from the previous scan. It adds (if PRESERVE=Y) or overwrites (if
PRESERVE=N) the new statistics for the tables being scanned in the current scan
request.
PROCESS
Property Description
Default value 1
Minimum value 1
Maximum value 32
Purpose Specifies the number of concurrent scan processes to utilize for the
database scan
QUERY
Property Description
Default value None
Purpose Applies a filter to restrict the data to be scanned by specifying a
clause for a SELECT statement, which is applied to all tables and
columns in the scanner session
The value of the query parameter is a string that contains a WHERE clause for a SELECT
statement that will be applied to all tables and columns listed in the TABLE and
COLUMN parameters.
Only one query clause is allowed per scan session. The QUERY parameter is only
applicable when performing table or column scans. The parameter will be ignored
when performing a Full database or a User scan. QUERY can be applied to multiple
tables and columns scans, however, the identical WHERE clause will be appended to all
specified tables and columns.
For example, the following command scans the employess who were hired within the
last 30 days:
CSSCAN system/manager TABLE=HR.EMPLOYEES QUERY= 'hire_date > SYSDATE - 180' ...
Note that the WHERE clause is not required inside the QUERY parameter. CSSCAN will
automatically remove the WHERE clause if it is found to be the first five characters in
the QUERY parameter.
SUPPRESS
Property Description
Default value Unset (results in unlimited number of rows)
Minimum value 0
Maximum value Unlimited
Purpose Specifies the maximum number of data exceptions being logged for
each table
The Database Character Set Scanner inserts information into the CSM$ERRORS table
when an exception is found in a data cell. The table grows depending on the number
of exceptions reported.
This parameter is used to suppress the logging of individual exception information
after a specified number of exceptions are inserted for each table. For example, if
SUPPRESS is set to 100, then the Database Character Set Scanner records a maximum
of 100 exception records for each table.
TABLE
Property Description
Default value None
Purpose Specifies the names of the tables to scan
You can specify the following when you specify the name of the table:
■ schemaname specifies the name of the user's schema from which to scan the table
■ tablename specifies the name of the table or tables to be scanned
For example, the following command scans the employees and departments tables
in the HR sample schema:
CSSCAN system/manager TABLE=(HR.EMPLOYEES, HR.DEPARTMENTS)
TOCHAR
Property Description
Default value None
Purpose Specifies a target database character set name for the CHAR,
VARCHAR2, LONG, and CLOB data
TONCHAR
Property Description
Default value None
Purpose Specifies a target database character set name for the NCHAR,
NVARCHAR2, and NCLOB data
If you do not specify a value for TONCHAR, then the Database Character Set Scanner
does not scan NCHAR, NVARCHAR2, and NCLOB data.
USER
Property Description
Default value None
Purpose Specifies the owner of the tables to be scanned
If the USER parameter is specified, then the Database Character Set Scanner scans all
tables belonging to the specified owner. For example, the following statement scans all
tables belonging to HR:
CSSCAN system/manager USER=hr ...
USERID
Property Description
Default value None
Purpose Specifies the username and password (and optional connect string)
of the user who scans the database. If you omit the password, then
the Database Character Set Scanner prompts you for it
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.0.0 - Production
With the Partitioning and Data Mining options
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.0.0 - Production
With the Partitioning and Data Mining options
Copyright (c) 1983, 2005 Oracle Corporation. All rights reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.0.0 - Production
With the Partitioning and Data Mining options
Copyright (c) 1983, 2005 Oracle Corporation. All rights reserved.
The following example shows how to scan a single table to see the effect of migrating
it to WE8MSWIN1252. Before scanning, a query is run against the table to limit the
rows that will be scanned. This example assumes the current database character set is
in US7ASCII. Language and character set detection is performed on the LOSSY data
cells.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.0.0 - Production
With the Partitioning and Data Mining options
Copyright (c) 1983, 2005 Oracle Corporation. All rights reserved.
set is in US7ASCII. Language and character set detection is performed on the LOSSY
data cells.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.0.0 - Production
With the Partitioning and Data Mining options
Copyright (c) 1983, 2005 Oracle Corporation. All rights reserved.
Database Size
This section reports on the current database size as well as identifying the amount of
potential data expansion after the character set migration.
The following output is an example.
Tablespace Used Free Total Expansion
------------------------- --------------- --------------- --------------- ---------------
SYSTEM 206.63M 143.38M 350.00M 588.00K
SYSAUX 8.25M 131.75M 140.00M .00K
------------------------- --------------- --------------- --------------- ---------------
Total 214.88M 275.13M 490.00M 588.00K
TOCHAR AL32UTF8
Scan NCHAR data? NO
Array fetch buffer size 1024000
Number of processes 2
Capture convertible data? NO
Charset Language Detections Yes
Charset Language Parameter LOSSY
------------------------------ ------------------------------------------------
Scan Summary
This section indicates the feasibility of the database character set migration. There are
two basic criteria that determine the feasibility of the character set migration of the
database. One is the condition of the data dictionary and the other is the condition of
the application data.
The Scan Summary section consists of two status lines: one line reports on the data
dictionary, and the other line reports on the application data.
The following is sample output from the Scan Summary:
All character type data in the data dictionary are convertible to the new character set
Some character type application data are not convertible to the new character set
Table 12–1 shows the types of status that can be reported for the data dictionary and
application data.
Table 12–1 Possible Status of the Data Dictionary and Application Data
Data Dictionary Status Application Data Status
All character-type data in the data dictionary remains All character-type application data remains the same in
the same in the new character set. the new character set.
All character-type data in the data dictionary is All character-type application data is convertible to the
convertible to the new character set. new character set.
Some character-type data in the data dictionary is not Some character-type application data is not convertible to
convertible to the new character set. the new character set.
When all data remains the same in the new character set, it means that the encoding
values of the original character set are identical in the target character set. For example,
ASCII data is stored using the same binary values in both WE8ISO8859P1 and AL32
UTF8. In this case, the database character set can be migrated using the CSALTER
script.
If all the data is convertible to the new character set, then the data can be safely
migrated using the Export and Import utilities. However, the migrated data will have
different encoding values in the target character set.
See Also:
■ "Database Scan Individual Exception Report" on page 12-27 for
more information about non-convertible data
■ "Migrating a Character Set Using the CSALTER Script" on
page 11-6
■ "Migrating Character Data Using a Full Export and Import" on
page 11-6
The data dictionary can be safely migrated using the CSALTER script.
If the numbers of data cells recorded in the Convertible, Truncation, and Lossy
columns are zero, then no data conversion is required to migrate the data dictionary
from the FROMCHAR character set to the TOCHAR character set.
If the numbers in the Truncation and Lossy columns are zero and some numbers in
the Convertible columns are not zero, then all data in the data dictionary is
convertible to the new character set. However, it is dangerous to convert the data in
the data dictionary without understanding their impact on the database. The CSALTER
script can convert some of the convertible cells in the data dictionary. The message that
follows the conversion summary table indicates whether this conversion can be
supported by the CSALTER script.
If the numbers in the Lossy column are not zero, then there is data in the data
dictionary that is not convertible. Therefore, it is not feasible to migrate the current
database to the new character because the export and import processes cannot convert
the data into the new character set. For example, you might have a table name with
invalid characters or a PL/SQL procedure with a comment line that includes data that
cannot be mapped to the new character set. These changes to schema objects must be
corrected manually before migration to a new character set.
If the numbers in the Truncation column are not zero, then the export and import
process would truncate the data.
See Also:
■ "Database Character Set Scanner CSALTER Script" on
page 12-33
■ "How to Handle Convertible or Lossy Data in the Data
Dictionary" on page 12-29
HR.EMPLOYEES 1 0 0
OE.CUSTOMERS 2 0 0
PM.ONLINE_MEDIA 13 0 0
PM.PRINT_MEDIA 7 0 1
SYS.EXTERNAL_TAB$ 1 0 0
SYS.METASTYLESHEET 80 0 0
---------------------------- ---------------- ---------------- ----------------
Indexes To Be Rebuilt
This generates a list of all the indexes that are affected by the database character set
migration. These can be rebuilt after the data has been imported. The following output
is an example.
USER.INDEX on USER.TABLE(COLUMN)
--------------------------------------------------------------------------------------
HR.EMP_NAME_IX on HR.EMPLOYEES(FIRST_NAME)
HR.EMP_NAME_IX on HR.EMPLOYEES(LAST_NAME)
OE.CUST_EMAIL_IX on OE.CUSTOMERS(CUST_EMAIL)
--------------------------------------------------------------------------------------
-------------------------------------------------- ----------------
HR.EMPLOYEES|FIRST_NAME 1
See Also:
■ "Application Data Individual Exceptions" on page 12-28 for
more information about exceptions
■ "How to Handle Convertible or Lossy Data in the Data
Dictionary" on page 12-29
The values Ährenfeldt and órâclë8™ exceed the column size (10 bytes) because
each of the characters Ä, ó, â, and ë occupies one byte in WE8ISO8859P1 but two bytes
in AL32UTF8. The value órâclë8™ has lossy conversion to AL32UTF8 because the
trademark sign ™ (code 153) is not a valid WE8ISO8859P1 character. It is a
WE8MSWIN1252 character, which is a superset of WE8ISO8859P1.
You can view the data that has an exception by issuing a SELECT statement:
SELECT name FROM user1.test WHERE ROWID='AAAA2fAAFAABJwQAAu';
You can modify the data that has the exception by issuing an UPDATE statement:
UPDATE user1.test SET name = 'Oracle8 TM'
WHERE ROWID='AAAA2fAAFAABJwQAAu';
If the language and character set detection option is enabled, then CSSCAN attempts to
provide the most probable languages and character sets for the data cells specified by
the LCSDDATA parameter.
The following is an example of an Individual Exception Report that illustrates
language and character set detection results for lossy data cells when changing the
database character set from US7ASCII to AL32UTF8.
User: USER2
Table: TEST
Column: NAME
Type: VARCHAR2(30)
Number of Exceptions: 2
Max Post Conversion Data Size: 11
ROWID Exception Type Size Cell Data(first 30 bytes) Language & Character Set
------------------ ------------------ ----- --------------------------- ------------------------
AAAA2fAAFAABJwQAAt lossy conversion C'est français (French,UTF8)
AAAA2fAAFAABJwQAAu lossy conversion Ciò è italiana (Italian,WE8MSWIN1252)
------------------ ------------------ ----- --------------------------- ------------------------
See Also:
■ "Data Truncation" on page 11-1
■ "Character Set Conversion Issues" on page 11-3
For single-byte character sets, CLOB data is stored in the database character set
encoding. For multibyte character sets, CLOB data is stored in an internal Oracle
format which is compatible with UCS-2. For example, the byte representation of
the string 'ABC' stored inside a VARCHAR2 column in a US7ASCII database remains
unchanged when migrated to AL32UTF8. The same string stored inside a CLOB
column doubles in size and is stored completely differently. When migrating from
a single-byte character set to a multibyte character set, CLOB data is never
CHANGELESS.
■ Migrating a database to a character set that is a superset in the sense that it
contains all of the characters of the original character set, but the binary values of
some characters is not the same in the two character sets
This is similar to user application data whose data cells need to be manually
converted to the new character set. A common cause of this is that the user 's
object definitions (such as table names, column names, package names and
package bodies) were created using non-ASCII characters. These are typically
characters or symbols that are part of the user's native language.
The easiest approach to migrating convertible data dictionary data is to create a new
database in the target character set and then re-create all of the data dictionary and
schema definitions by exporting and importing. However, this method means creating
a new database.
If you want to migrate the existing database instead of building a new one, then the
CSALTER script is the simplest way to migrate convertible CLOB data inside the data
dictionary and to change the existing database character set to the target character set.
For data dictionary CONVERTIBLE data cells that are not CLOB data, you must find the
schema objects containing the convertible data. Then you can choose to do one of the
following:
■ Amend the object definitions (such as removing non-ASCII characters from
comments inside a package body) so that the data cells become CHANGELESS
■ Drop these objects from the database schema altogether and then re-create them
after the database character set has been migrated to the target character set
LOSSY dictionary data cells require further examination of the data dictionary to see
whether the current FROMCHAR character set is the actual character set of the database.
If it is, you have to correct these object definitions (such as removing the offending
characters from comments inside a package body) so that the data cells become
CHANGELESS and they can be migrated safely to the target character set.
Three SQL scripts are included with the Database Character Set Scanner to help users
to locate the CONVERTIBLE and LOSSY data cells in the data dictionary:
■ analyze_source.sql
■ analyze_histgrm.sql
■ analyze_rule.sql
The scripts are stored in the $ORACLE_HOME/nls/csscan/sql directory. They
perform SQL SELECT operations on the SYS.SOURCE$, SYS.HISTGRM$ and
SYS.RULE$ data dictionary tables so that the offending data dictionary objects can be
identified.
The following example shows output from the analyze_source.sql script:
SQL> @$ORACLE_HOME/nls/csscan/sql/analyze_source.sql
Table: SYS.SOURCE$
Error: CONVERTIBLE DATA
no rows selected
Table: SYS.SOURCE$
Error: EXCEPTIONAL DATA
CSM$TABLES
The Database Character Set Scanner enumerates all tables that need to be scanned into
the CSM$TABLES table.
You can look up the number of tables (to get an estimate of how large CSM$TABLES
can become) in the database by issuing the following SQL statement:
SELECT COUNT(*) FROM DBA_TABLES;
CSM$COLUMNS
The Database Character Set Scanner stores statistical information for each column
scanned into the CSM$COLUMNS table.
You can look up the number of character type columns (to get an estimate of how large
CSM$COLUMNS can become) in the database by issuing the following SQL statement:
CSM$ERRORS
When exceptions are detected with cell data, the Database Character Set Scanner
inserts individual exception information into the CSM$ERRORS table. This information
then appears in the Individual Exception Report and facilitates identifying records to
be modified if necessary.
If your database contains a lot of data that is signaled as Exceptional or
Convertible (when the parameter CAPTURE=Y is set), then the CSM$ERRORS table
can grow very large. You can prevent the CSM$ERRORS table from growing
unnecessarily large by using the SUPPRESS parameter.
The SUPPRESS parameter applies to all tables. The Database Character Set Scanner
suppresses inserting individual Exceptional information after the specified number
of exceptions is inserted. Limiting the number of exceptions to be recorded may not be
useful if the exceptions are spread over different tables.
To run the CSALTER script, start SQL*Plus and connect to the database whose
character set is to be migrated. Note that the Database Character Set Scanner must be
run before the CSALTER script. Then enter the following command:
sqlplus> @@CSALTER.PLB
4. It checks whether all CLOB columns in the Sample Schemas created by Oracle are
changeless or convertible. Convertible CLOB columns occur when migrating from
a single-byte character set to a multibyte character set. The tables that belong to
the following schemas are part of the Sample Schemas:
HR
OE
SH
PM
5. It checks whether the CLOB dataype is the only datatype that contains convertible
data in the data dictionary and Sample Schemas. It checks that all other users'
tables have no convertible data for all datatypes including the CLOB datatype.
Because the CSALTER script is meant to be run only when the current database is a
proper subset of the new database, all data should be changeless with the possible
exception of the CLOB data. When migrating from a single-byte character set to a
multibyte character set, user-created CLOB data requires conversion and must first
be exported and deleted from the schema. The database must be rescanned in
order to run the CSALTER script. Cells of all other datatypes that are reported to be
convertible or subject to truncation must be corrected before the Database
Character Set Scanner is rerun.
See Also:
■ "Subsets and Supersets" on page A-16
■ "Migrating a Character Set Using the CSALTER Script" on
page 11-6 for more information about the CSALTER script and
CLOB data
2. Start up the database and open it, because CSALTER requires an open database.
STARTUP OPEN
4. Shut down the database with either the IMMEDIATE or the NORMAL option.
5. Start up the database.
■ CSMV$INDEXES
■ CSMV$TABLES
CSMV$COLUMNS
This view contains statistical information about columns that were scanned.
CSMV$CONSTRAINTS
This view contains statistical information about constraints that were scanned.
CSMV$ERRORS
This view contains individual exception information for cell data and object
definitions.
CSMV$INDEXES
This view contains individual exception information for indexes.
CSMV$TABLES
This view contains information about database tables to be scanned. The Database
Character Set Scanner enumerates all tables to be scanned into this view.
Cause: SELECT statement that checks if the specified user exists in the database
failed.
Action: This is an internal error. Contact Oracle Support Services.
This chapter shows how to customize locale data. It includes the following topics:
■ Overview of the Oracle Locale Builder Utility
■ Creating a New Language Definition with the Oracle Locale Builder
■ Creating a New Territory Definition with the Oracle Locale Builder
■ Displaying a Code Chart with the Oracle Locale Builder
■ Creating a New Character Set Definition with the Oracle Locale Builder
■ Creating a New Linguistic Sort with the Oracle Locale Builder
■ Generating and Installing NLB Files
n is the next available sequence number to assign to the Arial Unicode MS font in the
font list. Java Runtime searches the font mapping list for each virtual font and uses the
first font available on your system.
After you edit the font.properties file, restart the Oracle Locale Builder.
In a Windows operating system, start the Oracle Locale Builder from the Start menu as
follows: Start > Programs > Oracle-OraHome10 > Configuration and Migration
Tools > Locale Builder. You can also start it from the DOS prompt by entering the
%ORACLE_HOME%\nls\lbuilder directory and executing the lbuilder.bat
command.
When you start the Oracle Locale Builder, the screen shown in Figure 13–1 appears.
want to start with, then click its displayed name. For example, you can open the
AMERICAN language definition file as shown in Figure 13–2.
Choosing AMERICAN opens the lx00001.nlb file. An NLB file is a binary file that
contains the settings for a specific language, territory, character set, or linguistic sort.
Language and territory abbreviations are for reference only and cannot be opened.
The following restrictions apply when choosing names for locale objects such as
languages:
■ Names must contain only ASCII characters
■ Names must start with a letter
■ Language, territory, and character set names cannot contain underscores
The valid range for the Language ID field for a user-defined language is 1,000 to
10,000. You can accept the value provided by Oracle Locale Builder or you can specify
a value within the range.
Figure 13–7 shows how to set month names using the Month Names tab page.
All names are shown as they appear in the NLT file. If you choose Yes for
capitalization, then the month names are capitalized in your application, but they do
not appear capitalized in the Month Names tab page.
Figure 13–8 shows the Day Names tab page.
You can choose day names for your user-defined language. All names are shown as
they appear in the NLT file. If you choose Yes for capitalization, then the day names
are capitalized in your application, but they do not appear capitalized in the Day
Names tab page.
Figure 13–9 shows the Common Info tab page.
You can display the territories, character sets, Windows character sets, and linguistic
sorts that have associations with the current language. In general, the most
appropriate or the most commonly used items are displayed first. For example, with a
language of FRENCH, the common territories are FRANCE, BELGIUM, CANADA,
and DJIBOUTI, while the character sets for supporting French are WE8ISO8859P1,
WE8MSWIN1252, AL32UTF8, and WE8ISO8859P15. As WE8MSWIN1252 is more
common than WE8ISO8859P1 in a Windows environment, it is displayed first.
The valid range for Territory ID for a user-defined territory is 1000 to 10000.
Figure 13–11 shows settings for calendar formats in the Calendar tab page.
Monday is set as the first day of the week, and the first week of the calendar year is set
as an ISO week. Figure 13–11 displays a sample calendar.
See Also:
■ "Calendar Formats" on page 3-20 for more information about
choosing the first day of the week and the first week of the
calendar year
■ "Customizing Calendars with the NLS Calendar Utility" on
page 13-15 for information about customizing calendars
themselves
When you choose a format from a list, Oracle Locale Builder displays an example of
the format. In this case, the Short Date Format is set to DD-MM-YY. The Short Time
Format is set to HH24:MI:SS. The Oracle Date Format is set to DD-MM-YY. The Long
Date Format is set to fmDay, Month dd, yyyy. The TimeStamp Timezone Format
is not set.
You can also enter your own formats instead of using the selection from the
drop-down menus.
See Also:
■ "Date Formats" on page 3-15
■ "Time Formats" on page 3-17
■ "Customizing Time Zone Data" on page 13-15
A period has been chosen for the Decimal Symbol. The Negative Sign Location is
specified to be on the left of the number. The Numeric Group Separator is a comma.
The Number Grouping is specified as 3 digits. The List Separator is a comma. The
Measurement System is metric. The Rounding Indicator is 4.
You can enter your own values instead of using values in the lists.
When you choose a format from a list, Oracle Locale Builder displays an example of
the format.
Figure 13–14 shows settings for currency formats in the Monetary tab page.
The Local Currency Symbol is set to $. The Alternative Currency Symbol is the euro
symbol. The Currency Presentation shows one of several possible sequences of the
local currency symbol, the debit symbol, and the number. The Decimal Symbol is the
period. The Group Separator is the comma. The Monetary Number Grouping is 3.
The Monetary Precision, or number of digits after the decimal symbol, is 3. The
Credit Symbol is +. The Debit Symbol is -. The International Currency Separator is a
blank space, so it is not visible in the field. The International Currency Symbol (ISO
currency symbol) is USD. Oracle Locale Builder displays examples of the currency
formats you have selected.
You can enter your own values instead of using the lists.
You can display the common languages and time zones for the current territory. For
example, with a territory of CANADA, the common languages are ENGLISH,
CANADIAN FRENCH, and FRENCH. The common time zones are
America/Montreal, America/St_Johns, America/Halifax, America/Winnipeg,
America/Regina, America/Edmonton, and America/Vancouver.
The rest of this section contains the following topics:
■ Customizing Time Zone Data
■ Customizing Calendars with the NLS Calendar Utility
See Also: "Choosing a Time Zone File" on page 4-15 for more
information about the contents of the time zone files and how to
install the smaller time zone file
or deviation days in the future. To add this information without waiting for a new
release of the Oracle database server, you can use an external file that is automatically
loaded when the calendar functions are executed.
Calendar data is first defined in a text file. The text definition file must be converted
into binary format. You can use the NLS Calendar Utility (lxegen) to convert the text
definition file into binary format.
The name of the text definition file and its location for the lxegen utility are
hard-coded and depend on the platform. On UNIX platforms, the file name is
lxecal.nlt. It is located in the $ORACLE_HOME/nls directory. A sample text
definition file is included in the $ORACLE_HOME/nls/demo directory.
The lxegen utility produces a binary file from the text definition file. The name of the
binary file is also hard-coded and depends on the platform. On UNIX platforms, the
name of the binary file is lxecal.nlb. The binary file is generated in the same
directory as the text file and overwrites an existing binary file.
After the binary file has been generated, it is automatically loaded during system
initialization. Do not move or rename the file.
Invoke the calendar utility from the command line as follows:
% lxegen
See Also:
■ Operating system documentation for the location of the files on
your system
■ "Calendar Systems" on page A-22
Click Show Existing Definitions. Highlight the character set you wish to display.
Figure 13–17 shows the Existing Definitions combo box with US7ASCII highlighted.
Click Open to choose the character set. Figure 13–18 shows the General tab page when
US7ASCII has been chosen.
Figure 13–18 General Tab Page When US7ASCII Has Been Chosen
Click the Character Data Mapping tab. Figure 13–19 shows the Character Data
Mapping tab page for US7ASCII.
Click View CodeChart. Figure 13–20 shows the code chart for US7ASCII.
It shows the encoded value of each character in the local character set, the glyph
associated with each character, and the Unicode value of each character in the local
character set.
If you want to print the code chart, then click Print Page.
Creating a New Character Set Definition with the Oracle Locale Builder
You can customize a character set to meet specific user needs. You can extend an
existing encoded character set definition. User-defined characters are often used to
encode special characters that represent the following:
■ Proper names
■ Historical Han characters that are not defined in an existing character set standard
■ Vendor-specific characters
■ New symbols or characters that you define
This section describes how Oracle supports user-defined characters. It includes the
following topics:
■ Character Sets with User-Defined Characters
■ Oracle Character Set Conversion Architecture
■ Unicode 4.0 Private Use Area
■ User-Defined Character Cross-References Between Character Sets
■ Guidelines for Creating a New Character Set from an Existing Character Set
■ Example: Creating a New Character Set Definition with the Oracle Locale Builder
The Oracle character sets listed in Table 13–2 contain predefined ranges that support
user-defined characters.
The following table shows how the character is encoded in different character sets.
In Oracle, all character sets are defined in terms of Unicode 4.0 code points. That is,
each character is defined as a Unicode 4.0 code value. Character conversion takes place
transparently to users by using Unicode as the intermediate form. For example, when
a JA16SJIS client connects to a JA16EUC database, the character shown in Figure 13–21
has the code point value 889F when it is entered from the JA16SJIS client. It is
internally converted to Unicode (with code point value 4E9C) and then converted to
JA16EUC (code point value B0A1).
Guidelines for Creating a New Character Set from an Existing Character Set
By default, the Oracle Locale Builder generates the next available character set ID for
you. You can also choose your own character set ID. Use the following format for
naming character set definition NLT files:
lx2dddd.nlt
If you derive a new character set from an existing Oracle character set, then Oracle
Corporation recommends using the following character set naming convention:
<Oracle_character_set_name><organization_name>EXT<version>
■ EXT specifies that this character set is an extension to the JA16EUC character set
■ 1 specifies the version
Example: Creating a New Character Set Definition with the Oracle Locale Builder
This section shows how to create a new character set called MYCHARSET with 10001
for its Character Set ID. The example uses the WE8ISO8859P1 character set and adds
10 Chinese characters.
Figure 13–22 shows the General tab page for MYCHARSET.
Click Show Existing Definitions and choose the WE8ISO8859P1 character set from
the Existing Definitions dialog box.
The ISO Character Set ID and Base Character Set ID fields are optional. The Base
Character Set ID is used for inheriting values so that the properties of the base
character set are used as a template. The Character Set ID is automatically generated,
but you can override it. The valid range for a user-defined character set ID is 8000 to
8999 or 10000 to 20000.
The ISO Character Set ID field remains blank for user-defined character sets.
In this example, the Base Character Set ID field remains blank. However, you can
specify a character set to use as a template. The settings in the Type Specification tab
page must match the type settings of the base character set that you enter in the Base
Character Set ID field. If the type settings do not match, then you will receive an error
when you generate your custom character set.
Figure 13–23 shows the Type Specification tab page.
Open the Character Data Mapping tab page. Highlight the character that you want to
add characters after in the character set. In this example, the 0xff local character
value is highlighted.
You can add one character at a time or use a text file to import a large number of
characters. In this example, a text file is imported. The first column is the local
character value. The second column is the Unicode value. The file contains the
following character values:
88a2 963f
88a3 54c0
88a4 611b
88a5 6328
88a6 59f6
88a7 9022
88a8 8475
88a9 831c
88aa 7a50
88ab 60aa
Choose File > Import > User-Defined Characters Data.
Figure 13–25 shows that the imported characters are added after 0xff in the character
set.
Settings for the flags are automatically derived. SWAP_WITH_NEXT is relevant for
Thai and Lao sorts. REVERSE_SECONDARY is for French sorts. CANONICAL_
EQUIVALENCE determines whether canonical rules are used. In this example,
CANONICAL_EQUIVALENCE is checked.
The valid range for Collation ID (sort ID) for a user-defined sort is 1000 to 2000 for
monolingual collation and 10000 to 11000 for multilingual collation.
See Also:
■ Figure 13–30, "Canonical Rules Dialog Box" for more
information about canonical rules
■ Chapter 5, "Linguistic Sorting and String Searching"
This example customizes the linguistic sort by moving digits so that they sort after
letters. Complete the following steps:
1. Highlight the Unicode value that you want to move. In Figure 13–27, the \x0034
Unicode value is highlighted. Its location in the Unicode Collation Sequence is
called a node.
2. Click Cut. Select the location where you want to move the node.
3. Click Paste. Clicking Paste opens the Paste Node dialog box, shown in
Figure 13–28.
■ The Paste Node dialog box enables you to choose whether to paste the node after
or before the location you have selected. It also enables you to choose the level
(Primary, Secondary, or Tertiary) of the node in relation to the node that you want
to paste it next to.
Select the position and the level at which you want to paste the node.
In Figure 13–28, the After button and the Primary button are selected.
■ Click OK to paste the node.
Use similar steps to move other digits to a position after the letters a through z.
Figure 13–29 shows the resulting Unicode Collation Sequence tab page after the digits
0 through 4 have been moved to a position after the letters a through z.
Changing the Sort Order for All Characters with the Same Diacritic
This example shows how to change the sort order for characters with diacritics. You
can do this by changing the sort for all characters containing a particular diacritic or by
changing one character at a time. This example changes the sort of each character with
a circumflex (for example, û) to be after the same character containing a tilde.
Verify the current sort order by choosing Tools > Canonical Rules. This opens the
Canonical Rules dialog box, shown in Figure 13–30.
Figure 13–30 shows how characters are decomposed into their canonical equivalents
and their current sorting orders. For example, û is represented as u plus ^.
In the Oracle Locale Builder collation window (shown in Figure 13–26), click the
Non-Spacing Characters tab. If you use the Non-Spacing Characters tab page, then
changes for diacritics apply to all characters. Figure 13–31 shows the Non-Spacing
Characters tab page.
Figure 13–31 Changing the Sort Order for All Characters with the Same Diacritic
Select the circumflex and click Cut. Click Yes in the Removal Confirmation dialog box.
Select the tilde and click Paste. Choose After and Secondary in the Paste Node dialog
box and click OK.
Figure 13–32 illustrates the new sort order.
Figure 13–32 The New Sort Order for Characters with the Same Diacritic
Figure 13–33 Changing the Sort Order of One Character with a Diacritic
Choose After and Primary in the Insert New Node dialog box. Enter the Unicode code
point value of ä. The code point value is \x00e4. Click OK.
Figure 13–34 shows the resulting sort order.
Note that the -p option preserves the timestamp of the original file.
2. In Oracle Locale Builder, choose Tools > Generate NLB or click the Generate NLB
icon in the left side bar.
3. Click Browse to find the directory where the NLT file is located. The location
dialog box is shown in Figure 13–35.
Do not try to specify an NLT file. Oracle Locale Builder generates an NLB file for
each NLT file.
4. Click OK to generate the NLB files.
Figure 13–36 illustrates the final notification that you have successfully generated
NLB files for all NLT files in the directory.
5. Copy the lx1boot.nlb file into the path that is specified by the ORA_NLS10
environment variable. For example, on a UNIX platform, enter a command similar
to the following:
% cp /directory_name/lx1boot.nlb $ORA_NLS10/lx1boot.nlb
6. Copy the new NLB files into the ORA_NLS10 directory. For example, on a UNIX
platform, enter commands similar to the following:
% cp /directory_name/lx22710.nlb $ORA_NLS10
% cp /directory_name/lx52710.nlb $ORA_NLS10
This appendix lists the languages, territories, character sets, and other locale data
supported by the Oracle server. It includes these topics:
■ Languages
■ Translated Messages
■ Territories
■ Character Sets
■ Language and Character Set Detection Support
■ Linguistic Sorts
■ Calendar Systems
■ Time Zone Names
■ Obsolete Locale Data
You can obtain information about character sets, languages, territories, and linguistic
sorts by querying the V$NLS_VALID_VALUES dynamic performance view.
See Also: Oracle Database Reference for more information about the
data that can be returned by this view
Languages
Languages in Table A–1 provide support for locale-sensitive information such as the
following:
■ Day and month names and their abbreviations
■ Symbols for equivalent expressions for A.M., P.M., A.D., and B.C.
■ Default sorting sequence for character data when the ORDER BY SQL clause is
specified
■ Writing direction (left to right or right to left)
■ Affirmative and negative response strings (for example, YES and NO)
By using Unicode databases and datatypes, you can store, process, and retrieve data
for almost all contemporary languages, including many that do not appear in
Table A–1.
Translated Messages
Oracle error messages have been translated into the languages which are listed in
Table A–2.
Territories
Table A–3 lists the territories supported by the Oracle server.
Character Sets
Oracle-supported character sets are listed in the following sections according to three
broad categories.
■ Recommended Database Character Sets
■ Other Character Sets
■ Client-Only Character Sets
In addition, common character set subset/superset combinations are listed. Some
character sets can only be used with certain data types. For example, the AL16UTF16
character set can only be used as an NCHAR character set, and not as a database
character set.
Also documented in the comment section are other unique features of the character set
that may be important to users or your database administrator. For example, the
information includes whether the character set supports the euro currency symbol,
whether user-defined characters are supported, and whether the character set is a
strict superset of ASCII. (You can use the CSALTER script to migrate an existing
database to a new character set, only if all of the schema data is a strict subset of the
new character set.)
The following is the key for the comment column of the character set tables:
SB: single-byte encoding
MB: multibyte encoding
FIXED: fixed-width multibyte encoding
ASCII: strict superset of ASCII
EURO: euro symbol supported
UDC: user-defined characters supported
Oracle does not document individual code page layouts. For specific details about a
particular character set, its character repertoire, and code point values, you can use
Oracle Locale Builder. Otherwise, you should refer to the actual national, international,
or vendor-specific standards.
Table A–5 lists the recommended and most commonly used EBCDIC-based Oracle
database character sets. The list is ordered alphabetically within their respective
language group.
Table A–7 lists the other EBCDIC-based Oracle character sets. The list is ordered
alphabetically within their language groups.
Table A–8 (Cont.) Character Sets that Support the Euro Symbol
Character Set Name Hexadecimal Code Value of the Euro Symbol
WE8EBCDIC1140 9F
WE8EBCDIC1140C 9F
WE8EBCDIC1145 9F
WE8EBCDIC1146 9F
WE8EBCDIC1148 9F
WE8EBCDIC1148C 9F
WE8EBCDIC924 9F
WE8ISO8859P15 A4
WE8MACROMAN8 DB
WE8MACROMAN8S DB
WE8MSWIN1252 80
WE8PC858 DF
ZHS32GB18030 A2E3
ZHT16HKSCS A3E1
ZHT16HKSCS31 A3E1
ZHT16MSWIN950 A3E1
You can use these character sets as the source_char_set or dest_char_set in the
CONVERT function.
See Oracle Database SQL Reference for more information about the CONVERT function
and "The CONVERT Function" on page 9-4.
US7ASCII is a special case because so many other character sets are supersets of it.
Table A–12 lists supersets for US7ASCII.
Table A–13 Languages and Character Sets Supported by CSSCAN, LCSSCAN, and GDK
Language Character Sets
Arabic AL16UTF16, AL32UTF8, AR8ISO8859P6, AR8MSWIN1256, UTF8
Bulgarian AL16UTF16, AL32UTF8, CL8ISO8859P5, CL8MSWIN1251, UTF8
Catalan AL16UTF16, AL32UTF8, US7ASCII, UTF8, WE8ISO8859P1, WE8ISO8859P15,
WE8MSWIN1252
Croatian AL16UTF16, AL32UTF8, EE8ISO8859P2, EE8MSWIN1250, UTF8
Czech AL16UTF16, AL32UTF8, EE8ISO8859P2, EE8MSWIN1250, UTF8
Danish AL16UTF16, AL32UTF8, US7ASCII, UTF8, WE8ISO8859P1, WE8ISO8859P15,
WE8MSWIN1252
Dutch AL16UTF16, AL32UTF8, US7ASCII, UTF8, WE8ISO8859P1, WE8ISO8859P15,
WE8MSWIN1252
English AL16UTF16, AL32UTF8, US7ASCII, UTF8, WE8ISO8859P1, WE8ISO8859P15,
WE8MSWIN1252
Estonian AL16UTF16, AL32UTF8, NEE8IOS8859P4, UTF8
Finnish AL16UTF16, AL32UTF8, US7ASCII, UTF8, WE8ISO8859P1, WE8ISO8859P15,
WE8MSWIN1252
French AL16UTF16, AL32UTF8, US7ASCII, UTF8, WE8ISO8859P1, WE8ISO8859P15,
WE8MSWIN1252
German AL16UTF16, AL32UTF8, US7ASCII, UTF8, WE8ISO8859P1, WE8ISO8859P15,
WE8MSWIN1252
Greek AL16UTF16, AL32UTF8, EL8ISO8859P7, EL8MSWIN1253, UTF8
Hebrew AL16UTF16, AL32UTF8, IW8ISO8859P8, IW8MSWIN1255, UTF8
Hungarian AL16UTF16, AL32UTF8, EE8ISO8859P2, EE8MSWIN1250, UTF8
Italian AL16UTF16, AL32UTF8, US7ASCII, UTF8, WE8ISO8859P1, WE8ISO8859P15,
WE8MSWIN1252
Japanese AL16UTF16, AL32UTF8, ISO2022-JP, JA16EUC, JA16SJIS, UTF8
Korean AL16UTF16, AL32UTF8, ISO2022-KR, KO16KSC5601, KO16MSWIN949, UTF8
Malay AL16UTF16, AL32UTF8, US7ASCII, UTF8, WE8ISO8859P1, WE8ISO8859P15,
WE8MSWIN1252
Norwegian AL16UTF16, AL32UTF8, US7ASCII, UTF8, WE8ISO8859P1, WE8ISO8859P15,
WE8MSWIN1252
Polish AL16UTF16, AL32UTF8, EE8ISO8859P2, EE8MSWIN1250, UTF8
Portuguese AL16UTF16, AL32UTF8, US7ASCII, UTF8, WE8ISO8859P1, WE8ISO8859P15,
WE8MSWIN1252
Romanian AL16UTF16, AL32UTF8, EE8ISO8859P2, EE8MSWIN1250, UTF8
Table A–13 (Cont.) Languages and Character Sets Supported by CSSCAN, LCSSCAN, and GDK
Language Character Sets
Russian AL16UTF16, AL32UTF8, CL8ISO8859P5, CL8KOI8R, CL8MSWIN1251, UTF8
Simplified Chinese AL16UTF16, AL32UTF8, HZ-GB-2312, UTF8, ZHS16GBK, ZHS16CGB231280
Slovak AL16UTF16, AL32UTF8, EE8ISO8859P2, EE8MSWIN1250, UTF8
Spanish AL16UTF16, AL32UTF8, US7ASCII, UTF8, WE8ISO8859P1, WE8ISO8859P15,
WE8MSWIN1252
Swedish AL16UTF16, AL32UTF8, US7ASCII, UTF8, WE8ISO8859P1, WE8ISO8859P15,
WE8MSWIN1252
Thai AL16UTF16, AL32UTF8, TH8TISASCII, UTF8
Traditional Chinese AL16UTF16, AL32UTF8, UTF8, ZHT16MSWIN950
Turkish AL16UTF16, AL32UTF8, TR8MSWIN1254, UTF8, WE8ISO8859P9
Linguistic Sorts
Oracle offers two kinds of linguistic sorts, monolingual and multilingual. In addition,
monolingual sorts can be extended to handle special cases. These special cases
(represented with a prefix X) typically mean that the characters are sorted differently
from their ASCII values. For example, ch and ll are treated as a single character in
XSPANISH.
All of the linguistic sorts can be also be performed as case-insensitive or
accent-insensitive by appending _CI or _AI to the linguistic sort name.
Table A–14 lists the monolingual linguistic sorts supported by the Oracle server.
See Also: Table A–1, " Oracle Supported Languages" on page A-2
for a list of the default sort for each language
Table A–15 lists the multilingual linguistic sorts available in Oracle. All of them
include GENERIC_M (an ISO standard for sorting Latin-based characters) as a base.
Multilingual linguistic sorts are used for a specific primary language together with
Latin-based characters. For example, KOREAN_M sorts Korean and Latin-based
characters, but it does not collate Chinese, Thai, or Japanese characters.
Calendar Systems
By default, most territory definitions use the Gregorian calendar system. Table A–14
lists the other calendar systems supported by the Oracle server.
Figure A–1 shows how March 27, 1998 appears in Japanese Imperial.
Obsolete Territories
Table A–19 contains territories that have been desupported in Oracle Database 10g.
Obsolete Languages
Table A–20 contains languages that have been desupported in Oracle Database 10g.
Steps 2 and 3 will need to be repeated for all 10g database clients that need to revert
back to the Oracle9i definition files.
Oracle Corporation strongly recommends that customers use the Oracle Database 10g
locale definition files; Oracle9i locale definition files will be desupported in a future
release.
Table B–1 Unicode Character Code Ranges for UTF-16 Character Codes
Types of Characters First 16 Bits Second 16 Bits
ASCII 0000-007F -
European (except ASCII), Arabic, Hebrew 0080-07FF -
Iindic, Thai, certain symbols (such as the euro symbol), Chinese, Japanese, 0800-0FFF -
Korean
1000 - CFFF
D000 - D7FF
F900 - FFFF
Private Use Area #1 E000 - EFFF -
F000 - F8FF
Supplementary characters: Additional Chinese, Japanese, and Korean D800 - D8BF DC00 - DFFF
characters; historic characters; musical symbols; mathematical symbols
D8CO - DABF DC00 - DFFF
DAC0 - DB7F DC00 - DFFF
Private Use Area #2 DB80 - DBBF DC00 - DFFF
DBC0 - DBFF DC00 - DFFF
Table B–2 contains code ranges that have been allocated in Unicode for UTF-8
character codes.
Table B–2 Unicode Character Code Ranges for UTF-8 Character Codes
Types of Characters First Byte Second Byte Third Byte Fourth Byte
ASCII 00 - 7F - - -
European (except ASCII), Arabic, Hebrew C2 - DF 80 - BF - -
Indic, Thai, certain symbols (such as the E0 A0 - BF 80 - BF -
euro symbol), Chinese, Japanese, Korean
E1 - EC 80 - BF 80 - BF
ED 80 - 9F 80 - BF
EF A4 - BF 80 - BF
Private Use Area #1 EE 80 - BF 80 - BF -
EF 80 - A3 80 - BF
Supplementary characters: Additional F0 90 - BF 80 - BF 80 - BF
Chinese, Japanese, and Korean characters;
F1 - F2 80 - BF 80 - BF 80 - BF
historic characters; musical symbols;
mathematical symbols F3 80 - AF 80 - BF 80 - BF
Private Use Area #2 F3 B0 - BF 80 - BF 80 - BF
F4 80 - 8F 80 - BF 80 - BF
UTF-16 Encoding
As shown in Table B–1, UTF-16 character codes for some characters (Additional
Chinese/Japanese/Korean characters and Private Use Area #2) are represented in two
units of 16-bits. These are supplementary characters. A supplementary character
consists of two 16-bit values. The first 16-bit value is encoded in the range from
0xD800 to 0xDBFF. The second 16-bit value is encoded in the range from 0xDC00 to
0xDFFF. With supplementary characters, UTF-16 character codes can represent more
than one million characters. Without supplementary characters, only 65,536 characters
can be represented. Oracle's AL16UTF16 character set supports supplementary
characters.
UTF-8 Encoding
The UTF-8 character codes in Table B–2 show that the following conditions are true:
■ ASCII characters use 1 byte
■ European (except ASCII), Arabic, and Hebrew characters require 2 bytes
■ Indic, Thai, Chinese, Japanese, and Korean characters as well as certain symbols
such as the euro symbol require 3 bytes
■ Characters in the Private Use Area #1 require 3 bytes
■ Supplementary characters require 4 bytes
■ Characters in the Private Use Area #2 require 4 bytes
Oracle's AL32UTF8 character set supports 1-byte, 2-byte, 3-byte, and 4-byte values.
Oracle's UTF8 character set supports 1-byte, 2-byte, and 3-byte values, but not 4-byte
values.
accent
A mark that changes the sound of a character. Because the common meaning of accent
is associated with the stress or prominence of the character's sound, the preferred
word in Oracle Database Globalization Support Guide is diacritic.
See also diacritic.
AL16UTF16
The default Oracle character set for the SQL NCHAR data type, which is used for the
national character set. It encodes Unicode data in the UTF-16 encoding.
See also national character set.
AL32UTF8
An Oracle character set for the SQL CHAR data type, which is used for the database
character set. It encodes Unicode data in the UTF-8 encoding.
See also database character set.
ASCII
American Standard Code for Information Interchange. A common encoded 7-bit
character set for English. ASCII includes the letters A-Z and a-z, as well as digits,
punctuation symbols, and control characters. The Oracle character set name is
US7ASCII.
base letter
A character without diacritics. For example, the base letter for a, A, ä, and Ä is a.
See also diacritic.
binary sorting
Ordering character strings based on their binary coded values.
byte semantics
Treatment of strings as a sequence of bytes.
See also character semantics and length semantics.
Glossary-1
canonical equivalence
A basic equivalence between characters or sequences of characters. For example, ç is
equivalent to the combination of c and ,. They cannot be distinguished when they are
correctly rendered.
case
Refers to the condition of being uppercase or lowercase. For example, in a Latin
alphabet, A is the uppercase glyph for a, the lowercase glyph.
case conversion
Changing a character from uppercase to lowercase or vice versa.
character
A character is an abstract element of text. A character is different from a glyph, which
is a specific representation of a character. For example, the first character of the
English upper-case alphabet can be displayed as A, A, A, and so on. These forms are
different glyphs that represent the same character. A character, a character code, and a
glyph are related as follows:
character --(encoding)--> character code --(font)--> glyph
For example, the first character of the English uppercase alphabet is represented in
computer memory as a number. The number is called the encoding or the character
code. The character code for the first character of the English uppercase alphabet is
0x41 in the ASCII encoding scheme. The character code is 0xc1 in the EBCDIC
encoding scheme.
You must choose a font to display or print the character. The available fonts depend
on which encoding scheme is being used. The character can be printed or displayed as
A, A, or A, for example. The forms are different glyphs that represent the same
character.
See also character code and glyph.
character code
A character code is a number that represents a specific character. The number depends
on the encoding scheme. For example, the character code of the first character of the
English uppercase alphabet is 0x41 in the ASCII encoding scheme, but it is 0xc1 in the
EBCDIC encoding scheme.
See also character.
character semantics
Treatment of strings as a sequence of characters.
See also byte semantics and length semantics.
character set
A collection of elements that represent textual information for a specific language or
group of languages. One language can be represented by more than one character set.
Glossary-2
A character set does not always imply a specific character encoding scheme. A
character encoding scheme is the assignment of a character code to each character in a
character set.
In this manual, a character set usually does imply a specific character encoding
scheme. Therefore, a character set is the same as an encoded character set in this
manual.
character string
An ordered group of characters.
A character string can also contain no characters. In this case, the character string is
called a null string. The number of characters in a null string is 0 (zero).
character classification
Information provides details about the type of character associated with each character
code. For example, a character can be uppercase, lowercase, punctuation, or control
character.
code point
The numeric representation of a character in a character set. For example, the code
point of A in the ASCII character set is 0x41. The code point of a character is also called
the encoded value of a character.
See also Unicode code point.
code unit
The unit of encoded text for processing and interchange. The size of the code unit
varies depending on the character encoding scheme. In most character encodings, a
code unit is 1 byte. Important exceptions are UTF-16 and UCS-2, which use 2-byte
code units, and wide character, which uses 4 bytes.
See also character encoding scheme.
collation
Ordering of character strings according to rules about sorting characters that are
associated with a language in a specific locale. Also called linguistic sort.
See also linguistic sort, monolingual linguistic sort, multilingual linguistic sort,
accent-insensitive linguistic sort, case-insensitive linguistic sort.
Glossary-3
data scanning
The process of identifying potential problems with character set conversion and
truncation of data before migrating the database character set.
diacritic
A mark near or through a character or combination of characters that indicates a
different sound than the sound of the character without the diacritical mark. For
example, the cedilla in façade is a diacritic. It changes the sound of c.
EBCDIC
Extended Binary Coded Decimal Interchange Code. EBCDIC is a family of encoded
character sets used mostly on IBM systems.
encoded value
The numeric representation of a character in a character set. For example, the code
point of A in the ASCII character set is 0x41. The encoded value of a character is also
called the code point of a character.
font
An ordered collection of character glyphs that provides a graphical representation of
characters in a character set.
globalization
The process of making software suitable for different linguistic and cultural
environments. Globalization should not be confused with localization, which is the
process of preparing software for use in one specific locale.
glyph
A glyph (font glyph) is a specific representation of a character. A character can have
many different glyphs. For example, the first character of the English uppercase
alphabet can be printed or displayed as A, A, A, and so on. These forms are different
glyphs that represent the same character.
See also character.
ideograph
A symbol that represents an idea. Chinese is an example of an ideographic writing
system.
ISO
International Organization for Standards. A worldwide federation of national
standards bodies from 130 countries. The mission of ISO is to develop and promote
standards in the world to facilitate the international exchange of goods and services.
Glossary-4
ISO 8859
A family of 8-bit encoded character sets. The most common one is ISO 8859-1 (also
known as ISO Latin1), and is used for Western European languages.
ISO 14651
A multilingual linguistic sort standard that is designed for almost all languages of the
world.
See also multilingual linguistic sort.
ISO/IEC 10646
A universal character set standard that defines the characters of most major scripts
used in the modern world. In 1993, ISO adopted Unicode version 1.1 as ISO/IEC
10646-1:1993. ISO/IEC 10646 has two formats: UCS-2 is a 2-byte fixed-width format,
and UCS-4 is a 4-byte fixed-width format. There are three levels of implementation, all
relating to support for composite characters:
■ Level 1 requires no composite character support.
■ Level 2 requires support for specific scripts (including most of the Unicode scripts
such as Arabic and Thai).
■ Level 3 requires unrestricted support for composite characters in all languages.
ISO currency
The 3-letter abbreviation used to denote a local currency, based on the ISO 4217
standard. For example, USD represents the United States dollar.
ISO Latin1
The ISO 8859-1 character set standard. It is an 8-bit extension to ASCII that adds 128
characters that include the most common Latin characters used in Western Europe.
The Oracle character set name is WE8ISO8859P1.
See also ISO 8859.
length semantics
Length semantics determines how you treat the length of a character string. The length
can be treated as a sequence of characters or bytes.
See also character semantics and byte semantics.
linguistic index
An index built on a linguistic sort order.
linguistic sort
An ordering of strings based on requirements from a locale instead of the binary
representation of the strings.
See also multilingual linguistic sort and monolingual linguistic sort.
locale
A collection of information about the linguistic and cultural preferences from a
particular region. Typically, a locale consists of language, territory, character set,
linguistic, and calendar information defined in NLS data files.
Glossary-5
localization
The process of providing language-specific or culture-specific information for software
systems. Translation of an application's user interface is an example of localization.
Localization should not be confused with globalization, which is the making software
suitable for different linguistic and cultural environments.
monolingual support
Support for only one language.
multibyte
Two or more bytes.
When character codes are assigned to all characters in a specific language or a group of
languages, one byte (8 bits) can represent 256 different characters. Two bytes (16 bits)
can represent up to 65,536 different characters. Two bytes are not enough to represent
all the characters for many languages. Some characters require 3 or 4 bytes.
One example is the UTF8 Unicode encoding. In UTF8, there are many 2-byte and
3-byte characters.
Another example is Traditional Chinese, used in Taiwan. It has more than 80,000
characters. Some character encoding schemes that are used in Taiwan use 4 bytes to
encode characters.
See also single byte.
multibyte character
A character whose character code consists of two or more bytes under a certain
character encoding scheme.
Note that the same character may have different character codes under different
encoding schemes. Oracle cannot tell whether a character is a multibyte character
without knowing which character encoding scheme is being used. For example,
Japanese Hankaku-Katakana (half-width Katakana) characters are one byte in the
JA16SJIS encoded character set, two bytes in JA16EUC, and three bytes in UTF8.
See also single-byte character.
Glossary-6
multilingual linguistic sort
An Oracle sort that evaluates strings on three levels. Asian languages require a
multilingual linguistic sort even if data exists in only one language. Multilingual
linguistic sorts are also used when data exists in several languages.
NLB files
Binary files used by the Locale Builder to define locale-specific data. They define all of
the locale definitions that are shipped with a specific release of the Oracle database
server. You can create user-defined NLB files with Oracle Locale Builder.
See also Oracle Locale Builder and NLT files.
NLS
National Language Support. NLS allows users to interact with the database in their
native languages. It also allows applications to run in different linguistic and cultural
environments. The term is somewhat obsolete because Oracle supports global users at
one time.
NLSRTL
National Language Support Runtime Library. This library is responsible for providing
locale-independent algorithms for internationalization. The locale-specific information
(that is, NLSDATA) is read by the NLSRTL library during run-time.
NLT files
Text files used by the Locale Builder to define locale-specific data. Because they are in
text, you can view the contents.
null string
A character string that contains no characters.
replacement character
A character used during character conversion when the source character is not
available in the target character set. For example, ? is often used as Oracle's default
replacement character.
Glossary-7
script
A collection of related graphic symbols that are used in a writing system. Some scripts
can represent multiple languages, and some languages use multiple scripts. Examples
of scripts include Latin, Arabic, and Han.
single byte
One byte. One byte usually consists of 8 bits. When character codes are assigned to all
characters for a specific language, one byte (8 bits) can represent 256 different
characters.
See also multibyte.
single-byte character
A single-byte character is a character whose character code consists of one byte under
a specific character encoding scheme. Note that the same character may have different
character codes under different encoding schemes. Oracle cannot tell which character
is a single-byte character without knowing which encoding scheme is being used. For
example, the euro currency symbol is one byte in the WE8MSWIN1252 encoded
character set, two bytes in AL16UTF16, and three bytes in UTF8.
See also multibyte character.
supplementary characters
The first version of Unicode was a 16-bit, fixed-width encoding that used two bytes to
encode each character. This allowed 65,536 characters to be represented. However,
more characters need to be supported because of the large number of Asian
ideograms.
Unicode 3.1 defines supplementary characters to meet this need. It uses two 16-bit
code units (also known as surrogate pairs) to represent a single character. This allows
an additional 1,048,576 characters to be defined. The Unicode 3.1 standard added the
first group of 44,944 supplementary characters.
surrogate pairs
See also supplementary characters.
syllabary
Provide a mechanism for communicating phonetic information along with the
ideographic characters used by languages such as Japanese.
UCS-2
A 1993 ISO/IEC standard character set. It is a fixed-width, 16-bit Unicode character
set. Each character occupies 16 bits of storage. The ISO Latin1 characters are the first
256 code points, so it can be viewed as a 16-bit extension of ISO Latin1.
UCS-4
A fixed-width, 32-bit Unicode character set. Each character occupies 32 bits of storage.
The UCS-2 characters are the first 65,536 code points in this standard, so it can be
viewed as a 32-bit extension of UCS-2. This is also sometimes referred to as ISO-10646.
Glossary-8
Unicode
Unicode is a universal encoded character set that allows you information from any
language to be stored by using a single character set. Unicode provides a unique code
value for every character, regardless of the platform, program, or language.
Unicode database
A database whose database character set is UTF-8.
Unicode datatype
A SQL NCHAR datatype (NCHAR, NVARCHAR2, and NCLOB). You can store Unicode
characters in columns of these datatypes even if the database character set is not
Unicode.
UTFE
A Unicode 3.0 UTF-8 Oracle database character set with 6-byte supplementary
character support. It is used only on EBCDIC platforms.
UTF8
The UTF8 Oracle character set encodes characters in one, two, or three bytes. It is for
ASCII-based platforms. The UTF8 character set supports Unicode 3.0 and it is
compliant to the CESU-8 standard. Although specific supplementary characters were
not assigned code points in Unicode until version 3.1, the code point range was
allocated for supplementary characters in Unicode 3.0. Supplementary characters are
treated as two separate, user-defined characters that occupy 6 bytes.
UTF-8
The 8-bit encoding of Unicode. It is a variable-width encoding. One Unicode character
can be 1 byte, 2 bytes, 3 bytes, or 4 bytes in UTF-8 encoding. Characters from the
European scripts are represented in either 1 or 2 bytes. Characters from most Asian
scripts are represented in 3 bytes. Supplementary characters are represented in 4 bytes.
The Oracle character set that supports UTF-8 is AL32UTF8.
UTF-16
The 16-bit encoding of Unicode. It is an extension of UCS-2 and supports the
supplementary characters defined in Unicode 3.1 by using a pair of UCS-2 code points.
One Unicode character can be 2 bytes or 4 bytes in UTF-16 encoding. Characters
(including ASCII characters) from European scripts and most Asian scripts are
represented in 2 bytes. Supplementary characters are represented in 4 bytes. The
Oracle character set that supports UTF-16 is AL16UTF16.
wide character
A fixed-width character format that is useful for extensive text processing because it
allows data to be processed in consistent, fixed-width chunks. Wide characters are
intended to support internal character processing.
Glossary-9
Glossary-10
Index
Index-1
character repertoire, 2-2 choosing between a Unicode database and Unicode
character semantics, 2-8, 3-31 datatypes, 6-7
character set client operating system
conversion, 13-21 character set compatibility with
data loss applications, 2-11
during conversion, 2-12 CLOB and NCLOB data
detecting with Globalization Development binding and defining in OCI, 7-17
Kit, 8-32 CLOBs
national, 7-4 creating indexes, 6-16
character set conversion code chart
between OCI client and database server, 7-12 displaying and printing, 13-16
parameters, 3-31 code point, 2-2
character set definition collation
customizing, 13-23 customizing, 13-26
guidelines for editing files, 13-22 column parameter
naming files, 13-22 Database Character Set Scanner, 12-10
character set migration compatibility
CSALTER script, 11-6 client operating system and application character
identifying character data conversion sets, 2-11
problems, 11-5 composed characters, 5-6
postmigration tasks, 11-11 context-sensitive characters, 5-6
scanning character data, 11-5 contracting characters, 5-6
character sets contracting letters, 5-8
AL16UTF16, 6-4 control characters, encoding, 2-3
AL24UTFFSS, 6-4 conversion
AL32UTF8, 6-4 between character set ID number and character set
Asian, A-6, A-7 name, 9-6
changing after database creation, 2-15 CONVERT SQL function, 9-4
choosing, 11-1 character sets, A-16
choosing a character set for a Unicode convert time zones, 4-20
database, 6-9 convertible data
choosing a national character set, 6-9 data dictionary, 12-29
conversion, 2-12, 2-16, 9-4 converting character data
conversion using OCI, 10-5 CONVERT SQL function, 9-4
customizing, 13-20 converting character data between character
data loss, 11-3 sets, 9-4
encoding, 2-1 Coordinated Universal Time, 4-4, 4-5
European, A-8, A-11 creating a database with Unicode datatypes, 6-6
ISO 8859 series, 2-5 creating a Unicode database, 6-6
Middle Eastern, A-14 CSALTER script, 11-6, 11-7
migrating and the data dictionary, 12-29 checking phase, 12-34
migration, 11-1, 11-2 running, 12-33
naming, 2-8 updating phase, 12-35
national, 6-6, 7-4 CSM$COLUMNS table, 12-31
restrictions on character sets used to express CSM$ERRORS table, 12-32
names, 2-13 CSM$TABLES table, 12-31
supersets and subsets, A-16 CSMIG user, 12-7
supported, A-5 csminst.sql script
supporting different character repertoires, 2-3 running, 12-7
universal, A-15 CSMV$COLUMNS view, 12-36
UTFE, 6-4 CSMV$CONSTRAINTS view, 12-36
character snational, 2-14 CSMV$ERROR view, 12-37
character type conversion CSMV$INDEXES view, 12-37
error reporting, 3-31 CSMV$TABLES view, 12-37
characters currencies
available in all Oracle database character sets, 2-3 formats, 3-25
context-sensitive, 5-6 CURRENT_DATE SQL function, 4-12
contracting, 5-6 CURRENT_TIMESTAMP SQL function, 4-12
user-defined, 13-20 customizing time zone data, 13-15
choosing a character set, 11-1
Index-2
D lastrpt parameter, 12-13, 12-14
maxblocks parameter, 12-14
data conversion
online help, 12-8
in Pro*C/C++, 7-18
performance, 12-32
OCI driver, 7-23
preserve parameter, 12-15
ODBC and OLE DB drivers, 7-28
query parameter, 12-15
thin driver, 7-23
restrictions, 12-33
Unicode Java strings, 7-23
scan modes, 12-6
data dictionary
suppress parameter, 12-16
changing character sets, 12-29
table parameter, 12-16
convertible or lossy data, 12-29
tochar parameter, 12-16
data dictionary views
user parameter, 12-17
NLS_DATABASE_PARAMETERS, 3-8
userid parameter, 12-17
NLS_INSTANCE_PARAMETERS, 3-8
utility, 12-5
NLS_SESSION_PARAMETER, 3-8
views, 12-35
data expansion
Database Scan Summary Report, 12-21
during character set migration, 11-2
database schemas
during data conversion, 7-13
designing for multiple languages, 6-12
data inconsistencies causing data loss, 11-4
database time zone, 4-18
data loss
datatype conversion
caused by data inconsistencies, 11-4
data loss and exceptions, 7-5
during character set conversion, 2-12
implicit, 7-6
during character set migration, 11-3
SQL functions, 7-7
during datatype conversion
datatypes
exceptions, 7-5
abstract, 2-14
during OCI Unicode character set
DATE, 4-2
conversion, 7-12
datetime, 4-1
from mixed character sets, 11-5
inserting values into datetime datatypes, 4-5
data truncation, 11-1
inserting values into interval datatypes, 4-10
restrictions, 11-2
interval, 4-1, 4-9
database character set
INTERVAL DAY TO SECOND, 4-10
character data conversion, 11-6
INTERVAL YEAR TO MONTH, 4-9
choosing, 2-10
supported, 2-14
compatibility between client operating system and
TIMESTAMP, 4-3
applications, 2-11
TIMESTAMP WITH LOCAL TIME ZONE, 4-5
performance, 2-12
TIMESTAMP WITH TIME ZONE, 4-4
Database Character Set Scanner, 12-12
date and time parameters, 3-15
analyse_histgrm.sql script, 12-30
DATE datatype, 4-2
analyse_rule.sql script, 12-30
date formats, 3-15, 9-9
analyse_source.sql script, 12-30
and partition bound expressions, 3-16
array parameter, 12-9
dates
boundaries parameter, 12-10
ISO standard, 3-20, 9-10
capture parameter, 12-10
NLS_DATE_LANGUAGE parameter, 3-17
column parameter, 12-10
datetime datatypes, 4-1
CSM$COLUMNS table, 12-31
inserting values, 4-5
CSM$ERRORS table, 12-32
datetime format parameters, 4-13
CSM$TABLES table, 12-31
Daylight Saving Time
CSMV$COLUMNS view, 12-36
Oracle support, 4-21
CSMV$CONSTRAINTS view, 12-36
daylight saving time session parameter, 4-14
CSMV$ERROR view, 12-37
days
CSMV$INDEXES view, 12-37
format element, 3-17
CSMV$TABLES view, 12-37
language of names, 3-17
Database Scan Summary Report, 12-21
DB_TZ database time zone, 4-19
error messages, 12-38
DBMS_LOB PL/SQL package, 9-10
exclude parameter, 12-11
DBMS_LOB.LOADBLOBFROMFILE
feedback parameter, 12-11
procedure, 9-11
fromnchar parameter, 12-12
DBMS_LOB.LOADCLOBFROMFILE
full parameter, 12-12
procedure, 9-11
help parameter, 12-12
DBMS_REDEFINITION.CAN_REDEF_TABLE
Individual Exception Report, 12-27
procedure, 11-10
invoking, 12-7
Index-3
DBTIMEZONE SQL function, 4-12 IW, 9-10
dest_char_set parameter, A-16 IY, 9-10
detecting language and character sets L, 9-10
Globalization Development Kit, 8-32 month, 3-17
detection RM, 9-9
supported languages and character sets, A-18 RN, 9-10
diacritic, 5-1 format masks, 3-23, 9-9
dynamic performance views formats
V$NLS_PARAMETERS, 3-8 currency, 3-25
V$NLS_VALID_VALUES, 3-8 date, 3-15, 4-14
numeric, 3-23
time, 3-17
E
FROM_TZ SQL function, 4-12
encoding fromchar parameter, 12-12
control characters, 2-3 Database Character Set Scanner, 12-12
ideographic writing systems, 2-3 fromnchar parameter
numbers, 2-3 Database Character Set Scanner, 12-12
phonetic writing systems, 2-3 full parameter
punctuation, 2-3 Database Character Set Scanner, 12-12
symbols, 2-3
encoding schemes
7-bit, 2-7
G
8-bit, 2-7 GDK
fixed-width, 2-7 application configuration file, 8-18
multibyte, 2-7 GDK application configuration file, 8-35
shift-sensitive variable-width, 2-8 example, 8-39
shift-sensitive variable-width multibyte, 2-8 GDK application framework for J2EE, 8-16
single-byte, 2-7 GDK components, 8-7
variable-width, 2-7 GDK error messages, 8-43
environment variables GDK Java API, 8-27
ORA_SDTZ, 4-14, 4-19 GDK Java supplied packages and classes, 8-40
ORA_TZFILE, 4-14 GDK Localizer object, 8-21
error messages gdkapp.xml application configuration file, 8-35
languages, A-3 gdkapp.xml GDK application configuration
translation, A-3 file, 8-18
ERROR_ON_OVERLAP_TIME session getString() method, 7-25
parameter, 4-14 getStringWithReplacement() method, 7-25
euro Globalization Development Kit, 8-1
Oracle support, 3-27 application configuration file, 8-35
exclude parameter character set conversion, 8-29
Database Character Set Scanner, 12-11 components, 8-7
expanding characters, 5-8 defining supported application locales, 8-22
characters e-mail programs, 8-33
expanding, 5-6 error messages, 8-43
EXTRACT (datetime) SQL function, 4-12 framework, 8-16
integrating locale sources, 8-19
Java API, 8-27
F
Java supplied packages and classes, 8-40
feedback parameter locale detection, 8-20
Database Character Set Scanner, 12-11 Localizer object, 8-21
fixed-width multibyte encoding schemes, 2-7 managing localized content in static files, 8-26
fonts managing strings in JSPs and Java servlets, 8-25
Unicode, 13-1 non_ASCII input and output in an HTML
Unicode for UNIX, 13-2 page, 8-23
Unicode for Windows, 13-2 Oracle binary and linguistic sorts, 8-31
format elements, 9-10 Oracle date, number, and monetary formats, 8-30
C, 9-10 Oracle language and character set detection, 8-32
D, 9-10 Oracle locale information, 8-28
day, 3-17 Oracle locale mapping, 8-28
G, 9-10 Oracle translated locale and time zone
Index-4
names, 8-33 JDBC programming
supported locale resources, 8-19 Unicode, 7-20
globalization features, 1-4 JDBC Server Side internal driver
globalization support and Unicode, 7-3
architecture, 1-1 JDBC Server Side thin driver
Greenwich Mean Time, 4-4, 4-5 and Unicode, 7-3
guessing the language or character set, 12-1 JDBC thin driver
and Unicode, 7-3
H
help parameter L
Database Character Set Scanner, 12-12 language
detecting with Globalization Development
Kit, 8-32
I
language abbreviations, A-1
IANA character sets Language and Character Set File Scanner, 12-1
mapping with ISO locales, 8-24 language definition
ideographic writing systems, encoding, 2-3 customizing, 13-6
ignorable characters, 5-6 overriding, 3-6
implicit datatype conversion, 7-6 language support, 1-4
indexes languages
creating for documents stored as CLOBs, 6-16 error messages, A-3
creating for multilingual document search, 6-15 languages and character sets
creating indexes for documents stored as supported by LCSSCAN, A-18
BLOBs, 6-16 LAST_DAY SQL function, 4-12
partitioned, 9-9 lastrpt parameter
Individual Exception Report, 12-27 Database Character Set Scanner, 12-13, 12-14
initialization parameters LCSCCAN
NLS_DATE_FORMAT, 4-14 error messages, 12-4
NLS_TIMESTAMP_FORMAT, 4-14 LCSSCAN, 12-1
NLS_TIMESTAMP_TZ_FORMAT, 4-14 supported languages and character sets, 12-4,
INSTR SQL functions, 7-8, 9-5 A-18
Internet application LCSSCAN command
locale BEGIN parameter, 12-2
determination, 8-6 END parameter, 12-3
monolingual, 8-2 examples, 12-3
multilingual, 8-2, 8-4 FILE parameter, 12-3
interval datatypes, 4-1, 4-9 HELP parameter, 12-4
inserting values, 4-10 online help, 12-4
INTERVAL DAY TO SECOND datatype, 4-10 RESULTS parameter, 12-2
INTERVAL YEAR TO MONTH datatype, 4-9 syntax, 12-2
ISO 8859 character sets, 2-5 length semantics, 2-8, 3-31
ISO locales LENGTH SQL functions, 9-5
mapping with IANA character sets, 8-24 LIKE conditions in SQL statements, 9-6
ISO standard LIKE2 SQL condition, 9-6
date format, 9-10 LIKE4 SQL condition, 9-6
ISO standard date format, 3-20, 9-10 LIKEC SQL condition, 9-6
ISO week number, 9-10 linguistic sort definitions
IW format element, 9-10 supported, A-20
IY format element, 9-10 linguistic sorts
accent-insensitive, 5-8
J BINARY, 5-11
BINARY_AI, linguistic sorts
Java BINARY_CI, 5-11
Unicode data conversion, 7-23 case-insensitive, 5-8
Java strings controlling, 9-9
binding and defining in Unicode, 7-20 customizing, 13-26
JDBC drivers characters with diacritics, 13-29, 13-31
form of use argument, 7-22 levels, 5-4
JDBC OCI driver list of defaults, A-2
and Unicode, 7-3
Index-5
parameters, 3-29 multilingual document search
list parameter, 3-22 creating indexes, 6-15
lmsgen utility, 10-6 multilingual Internet application, 8-4
loading external BFILE data into LOBs, 9-10 multilingual linguistic sorts
LOBs example, 5-13
loading external BFILE data, 9-10 supported, A-22
storing documents in multiple languages, 6-14 multilingual support
locale, 3-3 restricted, 2-18
dependencies, 3-6 unrestricted, 2-18
detection multiple languages
Globalization Development Kit, 8-20 designing database schemas, 6-12
of Internet application storing data, 6-13
determining, 8-6 storing documents in LOBs, 6-14
variant, 3-6
locale information
N
mapping between Oracle and other
standards, 10-3 N SQL function, 7-7
locale-charset-map, 8-35 national character set, 2-14, 6-6, 7-4
locale-determine-rule, 8-36 NCHAR columns
LocaleMapper class, 8-33 migrating from version 8 to Oracle9i and
locale-parameter-name, 8-37 later, 11-8
LOCALTIMESTAMP SQL function, 4-12 NCHAR datatype, 7-4
lossy data creating abstract datatype, 2-15
data dictionary, 12-29 migrating, 11-8
lxegen utility, 13-16 migration, 11-8
NCHR SQL function, 7-9
NCLOB datatype, 7-5
M NEW_TIME SQL function, 4-12
maxblocks parameter NEXT_DAY SQL function, 4-12
Database Character Set Scanner, 12-14 NLB data
message-bundles, 8-38 transportable, 13-35
migrating a character set NLB file, 13-4
CSALTER script, 11-6 NLB files, 13-1
migrating character sets in Real Application generating and installing, 13-33
Clusters, 11-7 NLS Calendar Utility, 13-15
migration NLS parameters
CHAR columns to NCHAR columns, 11-9 default values in SQL functions, 9-2
character sets, 11-1 list, 3-2
to NCHAR datatypes, 11-8 setting, 3-1
version 8 NCHAR columns to Oracle9i and specifying in SQL functions, 9-2
later, 11-8 unacceptable in SQL functions, 9-3
mixed character sets NLS Runtime Library, 1-1
causing data loss, 11-5 NLS_CALENDAR parameter, 3-22
monetary parameters, 3-24 NLS_CHARSET_DECL_LEN SQL function, 9-7
monolingual Internet application, 8-2 NLS_CHARSET_ID SQL function, 9-6
monolingual linguistic sorts NLS_CHARSET_NAME SQL function, 9-6
example, 5-12 NLS_COMP parameter, 3-30, 9-8
supported, A-20 NLS_CREDIT parameter, 3-28
months NLS_CURRENCY parameter, 3-25
format element, 3-17 NLS_DATABASE_PARAMETERS data dictionary
language of names, 3-17 view, 3-8
MONTHS_BETWEEN SQL function, 4-12 NLS_DATE_FORMAT initialization parameter, 4-14
multibyte encoding schemes, 2-7 NLS_DATE_FORMAT parameter, 3-15
fixed-width, 2-7 NLS_DATE_LANGUAGE parameter, 3-16
shift-sensitive variable-width, 2-8 NLS_DEBIT parameter, 3-29
variable-width, 2-7 NLS_DUAL_CURRENCY parameter, 3-27
multilexers NLS_INITCAP SQL function, 5-8, 9-1
creating, 6-15 NLS_INSTANCE_PARAMETERS data dictionary
multilingual data view, 3-8
specifying column lengths, 6-12 NLS_ISO_CURRENCY parameter, 3-26
Index-6
NLS_LANG parameter, 3-3 OCI_ATTR_MAXDATA_SIZE attribute, 7-13
choosing a locale, 3-3 OCI_UTF16ID character set ID, 7-10
client setting, 3-7 OCIBind() function, 7-14
examples, 3-5 OCICharSetConversionIsReplacementUsed(), 10-5
OCI client applications, 7-14 OCICharSetConvert(), 10-5
specifying, 3-5 OCICharsetToUnicode(), 10-5
UNIX client, 3-7 OCIDefine() function, 7-14
Windows client, 3-7 OCIEnvNlsCreate(), 7-10, 10-2
NLS_LANGUAGE parameter, 3-9 OCILobRead() function, 7-17
NLS_LENGTH_SEMANTICS parameter, 2-9 OCILobWrite() function, 7-17
NLS_LIST_SEPARATOR parameter, 3-31 OCIMessageClose(), 10-6
NLS_LOWER SQL function, 5-8, 5-9, 9-1 OCIMessageGet(), 10-6
NLS_MONETARY_CHARACTERS parameter, 3-28 OCIMessageOpen(), 10-6
NLS_NCHAR_CONV_EXCP parameter, 3-31 OCIMultiByteInSizeToWideChar(), 10-4
NLS_NUMERIC_CHARACTERS parameter, 3-23 OCIMultiByteStrCaseConversion(), 10-5
NLS_SESSION_PARAMETERS data dictionary OCIMultiByteStrcat(), 10-4
view, 3-8 OCIMultiByteStrcmp(), 10-4
NLS_SORT parameter, 3-29, 5-18 OCIMultiByteStrcpy(), 10-4
NLS_TERRITORY parameter, 3-11 OCIMultiByteStrlen(), 10-5
NLS_TIMESTAMP_FORMAT initialization OCIMultiByteStrncat(), 10-4
parameter, 4-14 OCIMultiByteStrncmp(), 10-4
NLS_TIMESTAMP_FORMAT parameter OCIMultiByteStrncpy(), 10-4
parameters OCIMultiByteStrnDisplayLength(), 10-5
NLS_TIMESTAMP_FORMAT, 3-18, 3-19 OCIMultiByteToWideChar(), 10-4
NLS_TIMESTAMP_TZ_FORMAT initialization OCINlsCharSetIdToName(), 10-3
parameter, 4-14 OCINlsCharSetNameTold(), 10-2
NLS_UPPER SQL function, 5-8, 5-9, 9-1 OCINlsEnvironmentVariableGet(), 10-3
NLSRTL, 1-1 OCINlsGetInfo(), 10-2
NLSSORT SQL function, 9-1, 9-7 OCINlsNameMap(), 10-3
syntax, 9-8 OCINlsNumericInfoGet(), 10-3
NLT files, 13-1 OCIUnicodeToCharset(), 10-5
numbers, encoding, 2-3 OCIWideCharDisplayLength(), 10-4
numeric formats, 3-23 OCIWideCharInSizeToMultiByte(), 10-4
SQL masks, 9-10 OCIWideCharIsAlnum(), 10-5
numeric parameters, 3-22 OCIWideCharIsAlpha(), 10-5
NUMTODSINTERVAL SQL function, 4-12 OCIWideCharIsCntrl(), 10-5
NUMTOYMINTERVAL SQL function, 4-13 OCIWideCharIsDigit(), 10-5
NVARCHAR datatype OCIWideCharIsGraph(), 10-5
Pro*C/C++, 7-19 OCIWideCharIsLower(), 10-5
NVARCHAR2 datatype, 7-4 OCIWideCharIsPrint(), 10-5
OCIWideCharIsPunct(), 10-5
OCIWideCharIsSingleByte(), 10-5
O
OCIWideCharIsSpace(), 10-5
obsolete locale data, A-30 OCIWideCharIsUpper(), 10-5
OCI OCIWideCharIsXdigit(), 10-5
binding and defining CLOB and NCLOB data in OCIWideCharMultibyteLength(), 10-4
OCI, 7-17 OCIWideCharStrCaseConversion(), 10-4
binding and defining SQL NCHAR OCIWideCharStrcat(), 10-4
datatypes, 7-15 OCIWideCharStrchr(), 10-4
setting the character set, 10-2 OCIWideCharStrcmp(), 10-4
SQL CHAR datatypes, 7-14 OCIWideCharStrcpy(), 10-4
OCI and Unicode, 7-2 OCIWideCharStrlen(), 10-4
OCI character set conversion, 7-12 OCIWideCharStrncat(), 10-4
data loss, 7-12 OCIWideCharStrncmp(), 10-4
performance, 7-12 OCIWideCharStrncpy(), 10-4
OCI client applications OCIWideCharStrrchr(), 10-4
using Unicode character sets, 7-14 OCIWideCharToLower(), 10-4
OCI data conversion OCIWideCharToMultiByte(), 10-4
data expansion, 7-13 OCIWideCharToUpper(), 10-4
OCI_ATTR_CHARSET_FORM attribute, 7-12 ODBC Unicode applications, 7-29
Index-7
OLE DB Unicode datatypes, 7-30 NLS_ISO_CURRENCY, 3-26
online table redefinition NLS_LANG, 3-3
migrating from CHAR to NCHAR, 11-9, 11-10 NLS_LANGUAGE, 3-9
operating system NLS_LIST_SEPARATOR, 3-31
character set compatibility with NLS_MONETARY_CHARACTERS, 3-28
applications, 2-11 NLS_NCHAR_CONV_EXCP, 3-31
ORA_NLS10 environment variable, 1-2 NLS_NUMERIC_CHARACTERS, 3-23
ORA_SDTZ environment variable, 4-14, 4-19 NLS_SORT, 3-29
ORA_TZFILE environment variable, 4-14 NLS_TERRITORY, 3-11
Oracle Call Interface and Unicode, 7-2 numeric, 3-22
Oracle Data Provide for .NET and Unicode, 7-3 setting, 3-1
Oracle Language and Character Set Detection Java time and date, 3-15
classes, 8-32 time zone, 3-18, 3-19
Oracle Locale Builder partitioned
choosing a calendar format, 13-10 indexes, 9-9
choosing currency formats, 13-13 tables, 9-9
choosing date and time formats, 13-11 performance
displaying code chart, 13-16 choosing a database character set, 2-12
Existing Definitions dialog box, 13-3 during OCI Unicode character set
fonts, 13-2 conversion, 7-12
Open File dialog box, 13-5 phonetic writing systems, encoding, 2-3
Preview NLT screen, 13-4 PL/SQL and SQL and Unicode, 7-3
restrictions on names for locale objects, 13-6 preserve parameter
Session Log dialog box, 13-4 Database Character Set Scanner, 12-15
starting, 13-2 primary level sort, 5-4
Oracle ODBC driver and Unicode, 7-2 Private Use Area, 13-21
Oracle OLE DB driver and Unicode, 7-2 Pro*C/C++
Oracle Pro*C/C++ and Unicode, 7-2 data conversion, 7-18
oracle.i18n.lcsd package, 8-41 NVARCHAR datatype, 7-19
oracle.i18n.net package, 8-41 UVARCHAR datatype, 7-19
oracle.i18n.servlet package, 8-41 VARCHAR datatype, 7-18
oracle.i18n.text package, 8-42 punctuation, encoding, 2-3
oracle.i18n.util package, 8-42
oracle.sql.CHAR class
Q
character set conversion, 7-24
getString() method, 7-25 query parameter
getStringWithReplacement() method, 7-25 Database Character Set Scanner, 12-15
toString() method, 7-25
ORDER BY clause, 9-9 R
OS_TZ local operating system time zone, 4-19
overriding language and territory definitions, 3-6 Real Application Clusters
database character set migration, 11-7
REGEXP SQL functions, 5-19
P regular expressions
page-charset, 8-36 character class, 5-21
parameters character range, 5-20
BLANK_TRIMMING, 11-3 collation element delimiter, 5-20
calendar, 3-19 equivalence class, 5-21
character set conversion, 3-31 examples, 5-21
linguistic sorts, 3-29 multilingual environment, 5-19
methods of setting, 3-2 replacement characters
monetary, 3-24 CONVERT SQL function, 9-4
NLS_CALENDAR, 3-22 restricted multilingual support, 2-17, 2-18
NLS_COMP, 3-30 restrictions
NLS_CREDIT, 3-28 data truncation, 11-2
NLS_CURRENCY, 3-25 passwords, 11-2
NLS_DATE_FORMAT, 3-15 space padding during export, 11-3
NLS_DATE_LANGUAGE, 3-16 usernames, 11-2
NLS_DEBIT, 3-29 reverse secondary sorting, 5-7
NLS_DUAL_CURRENCY, 3-27 ROUND (date) SQL function, 4-12
Index-8
RPAD SQL function, 7-8 NLS_CHARSET_NAME, 9-6
NLS_INITCAP, 5-8, 9-1
NLS_LOWER, 5-8, 5-9, 9-1
S
NLS_UPPER, 5-8, 5-9, 9-1
scan modes NLSSORT, 9-1, 9-7
Database Character Set Scanner, 12-6 NUMTODSINTERVAL, 4-12
full database scan, 12-6 NUMTOYMINTERVAL, 4-13
single table scan, 12-6 ROUND (date), 4-12
user tables scan, 12-6 RPAD, 7-8
scan.err file, 12-21 SESSIONTIMEZONE, 4-13
scan.out file, 12-18, 12-19, 12-20, 12-21 specifying NLS parameters, 9-2
scan.txt file, 12-21 SUBSTR, 9-5
searching multilingual documents, 6-15 SUBSTR2, 9-5
searching string, 5-19 SUBSTR4, 9-5
secondary level sort, 5-4 SUBSTRB, 9-5
session parameters SUBSTRC, 9-5
ERROR_ON_OVERLAP, 4-14 SYS_EXTRACT_UTC, 4-13
session time zone, 4-19 SYSDATE, 4-13
SESSIONTIMEZONE SQL function, 4-13 SYSTIMESTAMP, 4-13
setFormOfUse() method, 7-22 TO_CHAR, 9-1
shift-sensitive variable-width multibyte encoding TO_CHAR (datetime), 4-13
schemes, 2-8 TO_DATE, 7-7, 9-1
single-byte encoding schemes, 2-7 TO_DSINTERVAL, 4-13
sorting TO_NCHAR, 7-7
reverse secondary, 5-7 TO_NUMBER, 9-1
specifying nondefault linguistic sorts, 3-29, 3-30 TO_TIMESTAMP, 4-13
source_char_set parameter, A-16 TO_TIMESTAMP_TZ, 4-13
space padding TO_YMINTERVAL, 4-13
during export, 11-3 TRUNC (date), 4-12
special combination letters, 5-6, 5-8 TZ_OFFSET, 4-13
special letters, 5-6, 5-8 unacceptable NLS parameters, 9-3
special lowercase letters, 5-8 UNISTR, 7-9
special uppercase letters, 5-8 SQL NCHAR datatypes
SQL CHAR datatypes, 2-10 binding and defining in OCI, 7-15
OCI, 7-14 SQL statements
SQL conditions LIKE conditions, 9-6
LIKE2, 9-6 strict superset, 6-2
LIKE4, 9-6 string comparisons
LIKEC, 9-6 WHERE clause, 9-8
SQL functions string literals
ADD_MONTHS, 4-12 Unicode, 7-8
CONVERT, 9-4 string manipulation using OCI, 10-3
CURRENT_DATE, 4-12 strings
CURRENT_TIMESTAMP, 4-12 searching, 5-19
datatype conversion, 7-7 SUBSTR SQL function, 9-5
DBTIMEZONE, 4-12 SUBSTR SQL functions, 9-5
default values for NLS parameters, 9-2 SUBSTR, 9-5
EXTRACT (datetime), 4-12 SUBSTR2, 9-5
FROM_TZ, 4-12 SUBSTR4, 9-5
INSTR, 7-8, 9-5 SUBSTRB, 9-5
LAST_DAY, 4-12 SUBSTRC, 9-5
LENGTH, 9-5 SUBSTR4 SQL function, 9-5
LOCALTIMESTAMP, 4-12 SUBSTRB SQL function, 9-5
MONTHS_BETWEEN, 4-12 SUBSTRC SQL function, 9-5
N, 7-7 superset, strict, 6-2
NCHR, 7-9 supersets and subsets, A-16
NEW_TIME, 4-12 supplementary characters, 5-3, 6-2
NEXT_DAY, 4-12 linguistic sort support, A-22
NLS_CHARSET_DECL_LEN, 9-7 supported datatypes, 2-14
NLS_CHARSET_ID, 9-6 supported territories, A-4
Index-9
suppress parameter group separator, 3-23
Database Character Set Scanner, 12-16 language for dates, 3-16
surrogate pairs, 6-2 spelling of days and months, 3-17
syllabary, 2-3 TO_DATE SQL function, 7-7, 9-1
symbols, encoding, 2-3 default date format, 3-15, 4-14
SYS_EXTRACT_UTC SQL function, 4-13 format masks, 9-9
SYSDATE SQL function, 4-13 language for dates, 3-16
effect of session time zone, 4-19 spelling of days and months, 3-17
sys.sys_tzuv2_temptab table, 4-17 TO_DSINTERVAL SQL function, 4-13
SYSTIMESTAMP SQL function, 4-13 TO_NCHAR SQL function, 7-7
TO_NUMBER SQL function, 9-1
format masks, 9-9
T
TO_TIMESTAMP SQL function, 4-13
table parameter TO_TIMESTAMP_TZ SQL function, 4-13
Database Character Set Scanner, 12-16 TO_YMINTERVAL SQL function, 4-13
tables tochar parameter
partitioned, 9-9 Database Character Set Scanner, 12-16
territory toString() method, 7-25
dependencies, 3-6 transportable NLB data, 13-35
territory definition, 3-11 TRUNC (date) SQL function, 4-12
customizing, 13-9 TZ_OFFSET SQL function, 4-13
overriding, 3-6 TZABBREV, 4-15
territory support, 1-4, A-4 TZNAME, 4-15
territory variant, 3-6
tertiary level sort, 5-4
Thai and Laotian character rearrangement, 5-8
U
tilde, 7-27 UCS-2 encoding, 6-3
time and date parameters, 3-15 Unicode, 6-1
time zone binding and defining Java strings, 7-20
abbreviations, 4-15 character code assignments, B-1
data source, 4-15 character set conversion between OCI client and
database, 4-18 database server, 7-12
effect on SYSDATE SQL function, 4-19 code ranges for UTF-16 characters, B-1
environment variables, 4-14 code ranges for UTF-8 characters, B-1
names, 4-15 data conversion in Java, 7-23
parameters, 3-18, 3-19 encoding, 6-2
session, 4-19 fonts, 13-1
time zone file JDBC OCI driver, 7-3
choosing, 4-15 JDBC programming, 7-20
default, 4-15 JDBC Server Side internal driver, 7-3
time zones JDBC Server Side thin driver, 7-3
converting, 4-20 JDBC thin driver, 7-3
customizing, 13-15 mode, 7-10
upgrading definitions, 4-17 ODBC and OLE DB programming, 7-27
TIMESTAMP datatype, 4-3 Oracle Call Interface, 7-2
when to use, 4-8 Oracle Data Provide for .NET, 7-3
TIMESTAMP datatypes Oracle ODBC driver, 7-2
choosing, 4-8 Oracle OLE DB driver, 7-2
timestamp format, 3-18 Oracle Pro*C/C++, 7-2
TIMESTAMP WITH LOCAL TIME ZONE Oracle support, 6-4
datatype, 4-5 parsing an XML stream with Java, 7-32
when to use, 4-8 PL/SQL and SQL, 7-3
TIMESTAMP WITH TIME ZONE datatype, 4-4 Private Use Area, 13-21
when to use, 4-8 programming, 7-1
timezlrg.dat file, 13-15 reading an XML file with Java, 7-32
timezone.dat file, 13-15 string literals, 7-8
TO_CHAR (datetime) SQL function, 4-13 UCS-2 encoding, 6-3
TO_CHAR SQL function, 9-1 UTF-16 encoding, 6-3
default date format, 3-15, 4-14 UTF-8 encoding, 6-2
format masks, 9-9 writing an XML file with Java, 7-31
Index-10
XML programming, 7-31
Unicode database, 6-5
case study, 6-10
choosing a character set, 6-9
using with Unicode datatypes (case study), 6-11
when to use, 6-7
Unicode datatypes, 6-6
case study, 6-11
choosing a national character set, 6-9
using with a Unicode database (case study), 6-11
when to use, 6-8
UNISTR SQL function, 7-9
unrestricted multilingual support, 2-18
url-rewrite-rule, 8-39
US7ASCII
supersets, A-17
user parameter
Database Character Set Scanner, 12-17
user-defined characters, 13-20
adding to a character set definition, 13-25
cross-references between character sets, 13-22
userid parameter
Database Character Set Scanner, 12-17
UTC, 4-4, 4-5
UTF-16 encoding, 6-3, B-2
UTF8 character set, 6-5, A-15
UTF-8 encoding, 6-2, B-2
UTFE character set, 6-4, 6-5, A-15
UTL_FILE package, using with NCHAR, 7-10
UTL_I18N PL/SQL package, 8-42
UTL_LMS PL/SQL package, 8-43
utltzuv2.sql script, 4-17
UVARCHAR datatype
Pro*C/C++, 7-19
V
V$NLS_PARAMETERS dynamic performance
view, 3-8
V$NLS_VALID_VALUES dynamic performance
view, 3-8
VARCHAR datatype
Pro*C/C++, 7-18
variable-width multibyte encoding schemes, 2-7
version 8 NCHAR columns
migrating to Oracle9i and later, 11-8
W
wave dash, 7-27
WHERE clause
string comparisons, 9-8
X
XML
parsing in Unicode with Java, 7-32
reading in Unicode with Java, 7-32
writing in Unicode with Java, 7-31
XML programming
Unicode, 7-31
Index-11
Index-12