Practical database design, Part 1

Keys and datatypes

Level: Intermediate

Philipp Janert (janert@ieee.org), Software Project Consultant, IEEE.org

27 May 2003

What are the best choices when designing the schema for a relational database? What is the rationale in
deciding in favor of one and against some other alternative? Given the amount of vendor-specific
recommendations, it is all too easy to overlook basic relational database fundamentals. In this first of two
parts, author Philipp K. Janert talks about simple and complex datatypes, and about primary and foreign
keys -- the plumbing that holds the entire database together.

In this series, I discuss some general best practices that I have found to be particularly helpful. Nothing in it is specific
to any one vendor's product and everything should, therefore, be applicable, regardless of which database
implementation is being used. In this frist part of the article, I want to talk about both simple and complex datatypes, and
about primary and foreign keys, which are the plumbing that holds the entire database together.

Primary keys and related matters

A relational database (DB) stores two kinds of information -- data and plumbing. Data comprises the customer names,
inventory numbers, item descriptions, and so on, that the application uses. Plumbing refers to the primary and foreign
keys that the DB needs to find database records and relate them to one another.

Basic plumbing
For the purpose of data modeling, the plumbing should be largely transparent. In fact, purist DB lore makes no
distinction between data and plumbing. However, you will see that it is more efficient for administration and
maintenance, as well as in terms of runtime performance, to have some additional fields to serve as DB keys.

Every table must have a primary key: an attribute or combination of attributes that are guaranteed to be unique and not-
null. It is generally helpful to introduce a surrogate key -- a table attribute which has no business meaning, but simply
serves as unique identifier for each record in the table. This is the plumbing that I have been referring to.
The requirements for a primary key are very strict. It must:
Exist
Be unique
Not change over time
Surrogate keys help to mitigate the fact that real business data never reliably fulfills these requirements. Not every
person has a Social Security Number (think of those outside the U.S.), people change their names, and other important
information.

Business data might also simply be bad -- glitches in the Social Security Administration's system may lead to different
persons having the same Social Security Number. A surrogate key helps to isolate the system from such problems.

The second reason that surrogate keys are favorable has to do with efficiency and ease of maintenance, since you can
choose the most efficient datatype for the surrogate key. Furthermore, the surrogate key typically is a single field (not a
compound key), which simplifies the schema (particularly when the key is used in other tables as a foreign key).

Every table should have a dedicated column to serve as this table's primary key. This column may be called id or pk
(or possibly <table_name>_id or <table_name>_pk). Most databases are tuned for queries on integers, so
it makes sense to use this datatype as primary key. Many databases, including Postgres and Oracle, also provide a special
serial or sequence integer type, which generates a sequence of unique integers. Declaring a column to be of this
type guarantees that a unique key is generated for each inserted row.

Foreign keys are table attributes, the values of which are the primary keys of another table. It often makes sense to label
foreign key columns explicitly, for instance, by adopting a naming convention such as
<other_table_name>_fk. A referential integrity constraint (references) should be declared as part of the
CREATE statement when creating the table.

It bears repeating that the surrogate keys discussed earlier are part of the plumbing only -- their existence does not
obviate the modeling requirement to be able to form a primary key from the business data alone. Such a business data
candidate key is a subset of all attributes, the values of which are never null, and each combination of values is unique.
As a check on correct data modeling, such a candidate key must exist and should be documented for every table.

Strictly speaking, you may not always find a candidate key among the business data. Imagine a table recording the first
and last name for each user, but having no further attributes. Now assume that there are two different persons, both of
whom have the first name "Joe" and last name "Blow." In such a case, there exists no combination of table attributes
that can form a suitable candidate key.

The underlying problem here is whether you are talking about the uniqueness of datasets or about the uniqueness of the
underlying entities -- users, in this example. It is generally more intuitive, in particular to developers used to object-
oriented analysis, to model the uniqueness of the underlying entities. Surrogate keys as discussed earlier can help to
achieve this.

Alternate keys and visible identifiers

As part of the plumbing, the surrogate key has no need to ever be visible outside the DB. In particular, it should never
be revealed to the user. This allows the DB administrator to change the representation of the keys at will if necessary. If
a business need arises for providing the user with a unique identifier to a particular dataset, this identifier should be
considered real business data and kept separate from the plumbing. For instance, an additional column called
VisibleAccountNumber or the like can be introduced. Of course, this attribute should be non-null and unique so
that it forms an alternative candidate key (an alternate key). Having a separate column for visible identifiers also makes
it possible to generate and format the values for this attribute in a user-friendly way so that it is easy to read over the
phone to a customer support person, for instance.

A borderline case is when the identifier is not directly visible, but may still be accessible to the user. Examples include
hidden fields in Web pages in which an identifier is shuttled to the client to be used as a parameter in the following
request. Although there is no need for the user to handle the identifier, a malicious user may read and attempt to spoof it.
Using the numerical values of a primary key directly, in principle, allows any attacker to walk the entire table!

Defences against this problem include either encrypting and decrypting the value of the primary key, or protecting the
key by appending a Message Authentication Code (MAC). An alternative is to use a hard-to-spoof visible identifier
attribute for the table, such as the hash of the record's primary key or creation timestamp. (Of course, the uniqueness of
this attribute must be assured.)
Whether the key is visible to the application (as opposed to the end user) depends on the specifics of the project. Using a
numeric type directly carries the key's database representation straight into the application code and should be avoided to
prevent coupling. In small-scale developments, a String representation of the key's value may be acceptable (all
datatypes that can be stored in a DB must be able to be serialized).
But a better solution is a simple wrapper object that adds very little complexity, but provides strong decoupling of the
database keys' representation from their interfaces. A danger exists in making the wrapper object too smart. The
intention with surrogate keys is to make them simple and efficient for the database to handle. Settings from a database
value and possibly from a String, comparing with another key object, and possibly serializing are all the methods
that are required. Smarts, such as the ability to verify the contents based on a checksum calculation, suggest that this
object probably belongs to the business data domain (like the visible record identifiers, introduced earlier).

The problem of the Universally Unique Identifier

A final consideration concerns the possible need for a Universally Unique Identifier (UUID). The short answer is that
relational databases do not require UUIDs at all. In fact, the entire UUID concept is somewhat unrelated to relational
database management. Relational database keys -- the plumbing -- need only be unique per table, which can be achieved
database management. Relational database keys -- the plumbing -- need only be unique per table, which can be achieved
by using an auto-incrementing datatype such as the serial type mentioned earlier.

UUIDs can have some technical difficulties. To ensure uniqueness, all UUIDs must be generated by a centralized service
-- which leads to scalability problems and can become a single point of failure. (The scalability issue can be mitigated by
a stratified approach in which a central master is used to give out seeds to several slaves, which in turn generate the final
identifiers in batches, and so on.) To represent a UUID in a database, use either a string attribute or a compound key
comprising several integer columns. Both approaches are significantly slower than operations based on keys made up of
long integers. Compound keys also increase the complexity of the DB schema when used as foreign keys.

In the end, whether or not a record in a database needs to have a truly globally unique ID is dictated by the business
rules, not the database architecture. Some records may already contain some form of UUID (merchandise items, for
instance, typically possess a Universal Product Code as barcode). Some other records, possibly corresponding to
principal business entities, may otherwise already contain a unique identifier as part of their business data (such as the
combination of timestamp and account name for a ledger entry). If this is not the case, a UUID can be generated and
stored alongside the business data for those records that require it. In any case, UUIDs should be considered part of the
business data -- not of the plumbing.

Even if (and this is a big if) the object-relational mapping approach chosen requires every business object to have a
persistent, unique ID, there is no need to base the internal workings of the underlying relational database engine on this
fact.

In summary, I argue to keep business data apart from the database's internal plumbing. Building a relational database
around UUIDs breaks this principle by using attributes, which, in the end, are really part of the business data, as internal
infrastructure. (For a totally different point of view on this issue and a careful discussion of the problems involved in
generating UUIDs in a scalable fashion, see Scott Ambler's paper, "Mapping objects to relational databases," in
Resources.)

Datatypes
The SQL standard defines a number of standard datatypes and most DB vendors support some additional ones which are
specific to their own product. In the absence of truly compelling reasons to the contrary, avoid such extensions for the
sake of portability.

Strings and numbers

In general, numerical types pose few problems -- just select one that is large enough to support the necessary range of
values.

The attempt to find the optimal width of a string column is usually not worth the effort. You can avoid a lot of
confusion later on by making all text messages of type varchar(n) and limiting yourself to a few standard string
lengths and introducing aliases for them, such as: 32 bytes ("Label"), 256 bytes ("Note"), and 4k ("Text").

Even if other business requirements restrict the maximum length of certain fields to specific values, the DB schema is
arguably not the best place to enforce these rules. By the time the data reaches the DB, it is too late to do anything about
it (except reject it). Individual restrictions, stemming from business rules and requirements, should be enforced by the
business logic layer, which handles user interaction and input validation. On the other hand, maintenance of the DB
schema is considerably simplified if it is restricted to a handful of different string attributes.

Limit the use of fixed-width strings to codes of all sorts (as opposed to variable-length strings for real text). Keep in
mind however, that many seemingly fixed-length codes do actually become wider over time. The prudent DB designer
tries to avoid anything similar to the Y2K problem for new development work.

Time and money

A type to record timestamps (date/time combinations) is always necessary and is, fortunately, covered by the SQL
standard. No fully satisfying way exists to record a monetary value, however.
standard. No fully satisfying way exists to record a monetary value, however.

Saving monetary values and treating them in program code as a floating-point values always leads to round-off errors.
Recording the value as an exact integer of the smallest currency subdivision (such as "cent" for US dollars, as well as
for Euros and other appropriate currencies) may not be sufficient either. Many values carry more digits behind the
decimal point than the two for which actual coins exist (just visit your local gas station). A choice of decimal with 5 to 9
digits should work, though.

It goes without saying that no monetary value should ever be recorded without also recording the currency -- even if you
think your application will never handle anything but US dollars. Consider setting up a currency table and relating it to
the monetary values using foreign keys rather than embedding currency information directly. This helps with
internationalization (different currency names and symbols), as well as with formatting issues.

Booleans and the System Types Table

The use of the type bool anywhere in the design is a hint to rethink this particular module again. Very few attributes
are truly limited to only two values -- even a gender column has a malicious tendency to morph towards (at least) three
states -- male, female, and unknown. Allowing nulls only masks the real problem. A more flexible approach to type
codes is needed.

In many places in the database, attributes determine a record's type in some way. The GenderType mentioned above is
such an instance. Other examples may include ItemType (such as Merchandise, Freight, Packaging, Insurance),
PaymentType (Cash, Check, MoneyOrder, CreditCard, Barter), and things like StoreType, MembershipType, DeviceType,
and many more. This also includes instances in which you want to store the type of an object in some applicable object
model.

For each type, you need some form of documentation telling you not only the name of the type, but also the
characteristics associated with it; for instance, you may want to know what permissions each UserType entails. What
better place to keep this information than in the database itself?

Any record that has some form of type information associated with it should contain a type code column, which itself is
a foreign key referencing a type code table. The foreign key constraint ensures that no records with invalid types exist.
The type code table might have the following attributes:
typeCode_pk
label (unique mnemonic, such as varchar(32))
description (varchar(256) should be sufficient)
uri (pointing to additional resources, where necessary)
codeGroup_fk
Additional attributes are, of course, conceivable -- such as a three-letter code or a visible numeric code.

The codeGroup_fk attribute serves to organize related type codes. For instance, all subscriber types might form a
group. The codeGroup_fk attribute is a foreign key into a separate code group table. However, realizing that a code
group is nothing but a type code itself, the relationship can be made recursive so that codeGroup_fk references
typeCode_pk. Not only does this make the additional type code table unnecessary, but it also makes it possible to
order groups in an arbitrarily deep hierarchical structure. It is best to keep the type of code system relatively simple and
straightforward.

Complex datatypes
Finally, there are some common but complex datatypes -- such as phone numbers, postal addresses, contact information,
and credit cards -- that occur in almost every database schema. Typically, such records need to be accessed from many
tables in the database. In a typical eCommerce system, for instance, it might be necessary to store contact information
for users, suppliers, warehouses, and admins.

Rather than including those attributes in the respective user, supplier, or other record. (and thereby repeating those
columns throughout the database), it makes sense to set up a single table for the contact information that is referenced
through foreign keys by all other tables. This has two immediate benefits:
 It is easier to later change the cardinality of the relationships.
It localizes any future changes to the complex datatype.
Anticipating the attributes that will likely be required for each of those complex types is something of an art. My
suggestion is to try to strive for completeness from the outset rather than being forced to change the schema each time
an additional field becomes necessary.

A sampling of possible attributes for postal addresses includes:

Department
Company
Mail Stop
Address Line 1
Address Line 2
Address Line 3
City
State
Postal Code ("Zip")
Country
Full contact information might include the following attributes:
Title
First Name
Middle Name (or Initial)
Last Name
Suffix (such as jr. or sr.)
Home address
Work address
Home Phone
Work Phone
Cell Phone
Fax
Pager
Email
Finally, phone numbers should never be considered flat numbers. In fact, they break down into the following fields:
CountryCode
AreaCode (Number Plan Area)
ExchangeCode (Prefix)
LineNumber (Suffix)
Extension
In a phone number such as 987-1234, the prefix is the 987 and the suffix is the 1234. The extension is the only part of
the phone number that is optional. It is probably sufficient to use char(4) for all columns, but one might consider
char(6) to be on the safe side. Note that area codes in the US are limited to three digits, but this is not true for other
countries.

Sensitive data should be kept in encrypted form. Even if the database system itself is compromised, the data is still
protected from misuse. The most famous example of this kind of data management is the Unix password system which
only stores hashes of the users' passwords rather than the passwords themselves. Some data, such as credit card numbers,
needs to be encrypted in a recoverable fashion; however, a one-way encryption (as for the Unix password file) will not
do. This leads to the problem of encryption key management -- clearly, it should not be stored in the DB, along with the
secrets, but supplied at boot-time, for instance.

Summary
In this article, I've discussed some general best practices when designing relational databases, including:
The benefits of maintaining additional table attributes without any business meaning to serve as surrogate keys.
The recommendation not to base the internal workings of the database on Universally Unique Identifiers.
The use of a centralized type codes facility to express attributes with finite, predetermined ranges of values.
Some considerations in designing complex datatypes to be used throughout the database schema.
In the next half of this article, I'll cover database normalization and some additional uses for a database within a project,
like the use of history tables and event logs.

Resources
Learn about the importance of the surrogate primary key when designing stable, flexible, and well-performing
enterprise databases in Mike Lonigro's article, "The Case for the Surrogate Key."

Read the book, Translucent Databases , as Peter Wayner offers a better, deeper protection paradigm for databases,
one that works by scrambling the data with encryption algorithms, using the minimal amount of encryption that
ensures the database is still highly functional.

Search developerWorks' vast library for more than 1,000 articles on database design.

Discover how DB2 Relational Connect helps with information integration by allowing several databases to be
viewed and manipulated as one.

About the author

Philipp K. Janert is a Software Project Consultant, server programmer, and architect. His specific interest is the
identification, establishment, and transmission of software engineering's best practices. He maintains the
www.BeyondCode.org Web site and his articles have appeared in IEEE Software, Linux Journal, and on the O'Reilly
Network site. He holds a Ph.D. in Theoretical Physics from the University of Washington in Seattle. You can contact
the author at janert@ieee.org.

Share this....
Digg this story del.icio.us Slashdot it!
Practical database design, Part 2
Normalization, history, and event logging

Level: Intermediate

Philipp Janert (janert@ieee.org), Software Project Consultant, IEEE.org

24 Jun 2003

What are the best choices when designing the schema for a relational database? What is the rationale in
deciding in favor of one and against some other alternative? Given the amount of vendor-specific
recommendations, it is all too easy to overlook basic relational database fundamentals. In this second of
two parts, author Philipp K. Janert presents an introduction to database normalization and to the five
Normal Forms. He also discusses other possible uses for a database in a project, for example as a repository
for configurational or logging data.

In the first half of this article, I began to discuss some general best practices that I have found to be particularly helpful.
Again, none of it is specific to any one vendor's product and everything should, therefore, be applicable, regardless of
which database implementation is being used. In this concluding article, I'll attempt to give an approachable introduction
to the topic of database normalization and to the five Normal Forms. I'll also discuss other possible uses for a database
in a project, such as a repository for configurational or logging data, for example.

Normalization
No discussion of relational database (DB) design is complete without a section on normalization. A normalized DB
schema avoids certain anomalies when inserting, updating, or deleting data and, therefore, helps to keep consistent data
in the database.

However, the absence of anomalies is only the tangible result of a deeper benefit of normalization -- namely the correct
identification and modeling of entities. The insert, update, and delete anomalies I've just referred to are the consequences
of the redundancy introduced by improper or inadequate separation between distinct entities. The normalization
procedure is, therefore, not just a technical chore to be done out of principle, but it can actively help to improve the
understanding of the business domain.
Regrettably, the treatment of normalization is often prohibitively formal, and it suffers from a special, rather non-
intuitive terminology. This is unfortunate since the outcome of a normalization procedure often evokes the reaction that
it all is nothing more than common sense. I will try to offer explanations of expressions that you are likely to encounter
in the literature as they come up in the following discussion.

Overview
Normalization is a process in which an initial DB design is transformed, or decomposed, into a different, but equivalent,
design. The resulting schema is equivalent to the original one in the sense that no information is lost when going from
one to the other.
The normalization procedure consists of a sequence of projections -- that is, some attributes are extracted from one table
to form a new one. In other words, tables are split up vertically. The decomposition is lossless, only if you can restore
the original table by joining its projections.

Through such non-loss decompositions it is possible to transform an original schema into a resulting one that satisfies
certain conditions, known as Normal Forms:
The First Normal Form (1NF) addresses the structure of an isolated table.
The Second (2NF), Third (3NF), and Boyce-Codd (BCNF) Normal Forms address one-to-one and one-to-many
relationships.
The Fourth (4NF) and Fifth (5NF) Normal Forms deal with many-to-many relationships.
These Normal Forms form a hierarchy in such a way that a schema in a higher normal form automatically fulfills all the
criteria for all of the lower Normal Forms.

The Fifth Normal Form is the ultimate normal form with respect to projections and joins -- it is guaranteed to be free of
anomalies that can be eliminated by taking projections.
In the following discussion, any mention of keys refers to the conceptual keys formed from business data, not to any
plainly technical surrogate keys which might have been defined.

First Normal Form

A table is said to be in First Normal Form (1NF), if all entries in it are scalar-valued. Relational database tables are 1NF
by construction since vector-valued entries are forbidden. Vector-valued data (that is, entries which have more than one
value in each row) are referred to as repeating groups.

The following relation violates 1NF because the SupplierID forms a repeating group (here and in the following
examples and text, primary key fields are in bold):
{ PartID
D, Supplier1ID, Supplier2ID, Supplier3ID }

Repeating groups indicate a one-to-many relationship -- in other words, a relationship which in relational databases is
treated using foreign keys. Note that the problem of repeating groups cannot be solved by adding any number of fields to
a record; even if the number of elements of the vector-valued data was fixed, finite, and predetermined, searching for a
value in all these parallel fields is prohibitively cumbersome.

To achieve 1NF, eliminate repeating groups by creating separate tables for each set of related data.

To demonstrate the typical anomalies that occur in tables that are only 1NF, consider the following example:
{ CustomerID
D, O r d e r I D
D, CustomerAddress, OrderDate }

Note the following problems:

Insert: It is not possible to add a record for a customer who has never placed an order.
Update: To change the address for a customer, this change has to be repeated for all of the customer's existing
orders.
Delete: Deleting the last order for a customer loses all information about the customer.
Functional dependency
The Second and Third Normal Forms address dependencies among attributes, specifically between key and non-key
fields.

By definition, a key uniquely determines a record: Knowing the key determines the values of all the other attributes in
the table row, so that given a key, the values of all the other attributes in the row are fixed.
This kind of relationship can be formalized as follows. Let X and Y be attributes (or sets of attributes) of a given
relationship. Then Y is functionally dependent on X if, whenever two records agree on their X-values, they must also
agree on their Y-values. In this case, X is called the determinant and Y is called the dependent. Since for any X there
must be a single Y, this relationship represents a single-valued functional dependency. If the set of attributes in the
determinant is the smallest possible (in the sense that after dropping one or more of the attributes from X, the remaining
set of attributes does no longer uniquely determine Y), then the dependency is called irreducible.

Note that functional dependency is a semantic relationship: It is the business logic of the problem domain, represented
by the relation, which determines whether a certain X determines Y.

Second Normal Form

A table is in Second Normal Form (2NF) if every non-key field is a fact about the entire key. In other words, a table is
2NF if it is 1NF and all non-key attributes are functionally dependent on the entire primary key (that is, the dependency
is irreducible).
is irreducible).

Clearly, 2NF is only relevant when the key is composite (that is, consisting of several fields). The following example
describes a table which is not 2NF since the WarehouseAddress attribute depends only on WarehouseID but
not on PartID:
{ PartID
D, W a r e h o u s e I D
D, Quantity, WarehouseAddress }

To achieve 2NF, create separate tables for sets of values that apply to multiple records and relate these tables through
foreign keys. The determinants of the initial table become the primary keys of the resulting tables.

Third Normal Form

A relation is in Third Normal Form (3NF) if it is 2NF and none of its attributes is a fact about another non-key field. In
other words, no non-key field functionally depends on any other non-key field. (Such indirect dependencies are known
as transitive dependencies.)

The following example violates 3NF since the Location is functionally dependent on the DepartmentID:
{ EmployeeID
D, DepartmentID, Location }

To achieve 3NF, eliminate fields that do not depend on the key from the original table and add them to the table whose
primary key is their determinant.

To summarize the normalization procedure up to and including Third Normal Form:

Every field in a record must depend on The Key (1NF), the Whole Key (2NF), and Nothing But The Key
(3NF).

Boyce-Codd Normal Form

Boyce-Codd Normal Form (BCNF) is an extension of 3NF in the case with two or more candidate keys which are
composite and overlapping (that is, they have at least one field in common). If these conditions are not fulfilled, 3NF
and BCNF are equivalent. A table is BCNF if, and only if its only determinants are candidate keys.
In the following table, both {SupplierID, PartID}, as well as {SupplierName, PartID}, are
candidate keys. The table is not BCNF since it contains two determinants (SupplierID and SupplierName)
which are not candidate keys. (SupplierID and SupplierName are determinants, since they determine each
other.)
{ SupplierID, PartID, SupplierName, Quantity }

However, either of the following decompositions is BCNF:

{ SupplierID
D, SupplierName }
{ SupplierID
D, P a r t I D
D, Quantity }

or
{ SupplierName
e, SupplierID }
{ SupplierName
e, P a r t I D
D, Quantity }

To achieve BCNF, remove the determinants which are not candidate keys.

Many-to-many relationships and higher Normal Forms

Fourth and Fifth Normal Forms apply to situations involving many-to-many relationships. In relational databases, many-
to-many relationships are expressed through cross-reference tables.
As an example, consider a case of class enrollment. Each student can be enrolled in one or more classes and each class
can contain one or more students. Clearly, there is a many-to-many relationship between classes and students. This
relationship can be represented by a Student/Class cross-reference table:
{ StudentID
D, C l a s s I D }

The key for this table is the combination of StudentID and ClassID. To avoid violation of 2NF, all other
information about each student and each class is stored in separate Student and Class tables, respectively.

Note that each StudentID determines not a unique ClassID, but a well-defined, finite set of values. This kind of
behavior is referred to as multi-valued dependency of ClassID on StudentID.

Fourth Normal Form

A table is in Fourth Normal Form (4NF) if it is 3NF and it does not represent two or more independent many-to-many
relationships.

Consider an example with two many-to-many relationships, between students and classes and between classes and
teachers. Also, a many-to-many relationship between students and teachers is implied. However, the business rules do
not constrain this relationship in any way -- the combination of StudentID and TeacherID does not contain any
additional information beyond the information implied by the student/class and class/teacher relationships.
Consequentially, the student/class and class/teacher relationships are independent of each other -- these relationships
have no additional constraints. The following table is, then, in violation of 4NF:
{ StudentID
D, C l a s s I D
D, T e a c h e r I D }

As an example of the anomalies that can occur, realize that it is not possible to add a new class taught by some teacher
without adding at least one student who is enrolled in this class.

To achieve 4NF, represent each independent many-to-many relationship through its own cross-reference table.

Fifth Normal Form

A table is in Fifth Normal Form (5NF) if it is 4NF and its information content cannot be reconstructed from several
tables containing fewer attributes.

Consider again the student/class/teacher example, but now assume that there is an additional relationship between
students and teachers. The previous example table is now 4NF, since all the relationships it describes are interrelated.
However, it is not 5NF, since it can be reconstructed from three cross-reference tables, each representing one of the
three many-to-many relationships:
{ StudentIDD, C l a s s I D }
{ ClassID
D, TeacherID }
{ TeacherIDD, S t u d e n t I D }

To achieve 5NF, isolate interrelated many-to-many relationships, introducing the required number of new tables to
represent all business domain constraints.

Normalization in context
In practice, many databases are de-normalized to greater or lesser degree. The reason most often stated has to do with
performance -- a de-normalized database may require fewer joins and can, therefore, be faster for retrievals.

While this reasoning may be true, the usual caveats against premature optimization apply here as well as everywhere
else. First, you should determine sufficiently that a performance problem exists and that the proposed de-normalization
improves it before introducing a conceptually suboptimal design.

Furthermore, a de-normalized schema can be harder to update. The additional integrity checks that are necessary in this
case may offset the performance gains for queries obtained through denormalization.
Finally, it should be noted that dealing with many-to-many relationships raises some issues that cannot be fully resolved
through normalization (Chris Date's article, "Normalization is no Panacea," in Resources covers this topic).

History tables and event logging

Besides holding the data that is necessary to support the primary business purpose of the system under construction, the
DB is also a possible location to record information that is useful primarily for internal technical purposes, such as
adminstration and maintenance of the system itself.

History tables
In a production system, you may desire to preserve the history of changes to the data in the live database. This can be
achieved through the use of history (or backup) tables, and the appropriate INSERT, DELETE, and UPDATE triggers.

Each table in the DB should have a history table, mirroring the entire history of the primary table. If entries in the
primary table are to be updated, the old contents of the record are first copied to the history table before the update is
made. In the same way, deleted records in the primary table are copied to the history table before being deleted from the
primary one. The history tables always have the name of the corresponding primary one, but with _Hist appended.

Entries to the history table are always appended at the end. The history table, therefore, grows strictly monotonically in
time. It will become necessary to periodically spool ancient records to tape for archiving. Such records may, as a result,
not be immediately available for recall.

The attributes of the history table should agree exactly with the attributes of the primary table. In addition, the history
table records the date and type of the change to the primary table. The type is one of the following: Create, Update, or
Delete.

Changes to the structure of the primary table affect the history table. When an attribute is added to the primary table, it
is added to the history table as well. When an attribute is deleted from the primary table, the corresponding attribute is
not deleted from the history table. Instead, this field is left blank (NULL) in all future records. Consequentially, the
history table not only grows in length over time, but also in width.

Note that the choice to use such a history mechanism affects neither the overall DB layout, nor applications that access
only the primary tables. During development, you can probably dispense with recording changes in this way and leave
the creation of the history tables and the necessary triggers until installation time.

Event logging for fun and profit

A database can be used as an event logger. The notion of event is broad, ranging from common debugging and system
specific runtime information, to events which are specific to the business domain. Possible candidates for events to be
logged to the database include:
Transactions making changes to persistent data
Transactions crossing component boundaries
Errors and exceptions
Dispatching of messages to the user
Events involving financial transactions
State changes to business entities
An EventLog table to log such information contains at least these fields to record:
Timestamp
EventType (a type code)
Details (a descriptive string)
Optionally, it may identify an owner or originator of the event. The owner concept can either identify a logged-in user or
admin, but it may as well describe a part or module of the system itself. In applications dealing with financial
transactions, additional (optional) fields identifying the from- and to-accounts can be useful.
transactions, additional (optional) fields identifying the from- and to-accounts can be useful.

System config tables

Finally, it is possible to use the database as centralized storage for configurational data. Usually this information is kept
distributed in miscellaneous plain-text files, such as start-up scripts or property files. The database can provide a single,
managed storage facility for such information.

Besides start-up parameters, which are usually supplied to the system at boot-time, one may also think of properties that
are required at runtime, such as localized strings and messages.

Lastly, the database is a possible place to keep system documentation. This is most useful, of course, for information
that is naturally in tabular form (rather than free text), such as lists of assigned port numbers or shared memory keys, for
instance. But this approach is not limited to codes. A data dictionary, defining the permissible values for each field, is a
necessity on any non-trivial project. This also can be made accessible to all developers and administrators by storing it in
the database.

In any case, the data is stored in simple key/value pairs. Additional table attributes can contain comments or pointers
(URLs) to relevant offline documentation.

The primary advantage to keeping such information in the database is that the database provides a central repository for
all relevant information, as opposed to the typical approach in which data is scattered over miscellaneous files.

Summary
In this article I've covered database normalization and the five Normal Forms. In the normalization process, an original
database design is transformed into an equivalent one, which avoids certain anomalies when inserting, updating, or
deleting records. Proper normalization also helps to identify entities correctly. I also discussed the possible use of a
database as a central repository for logging information or configurational data.

Resources
Check out Barry Wise's "Database Normalization and Design Techniques"which gives a simple and accessible
introduction to the Normal Forms and the normalization process.

Read William Kent's "A Simple Guide to Five Normal Forms in Relational Database Theory" for a thorough
primer on the five Normal Forms.

Look at C.J. Date's article, "Normalization is no Panacea," which discusses some problems that cannot be solved
by normalization alone, since (in his own words): "It's always important to understand the limitations of any
technology on which we rely heavily."

Avail yourself of this white paper on Simple Conditions for Guaranteeing Higher Normal Forms in Relational
Databases.
 See the results of last year's survey on Winning Database Configurations.

About the author

Philipp K. Janert is a Software Project Consultant, server programmer, and architect. His specific interest is the
identification, establishment, and transmission of software engineering's best practices. He maintains the
www.BeyondCode.org Web site and his articles have appeared in IEEE Software, Linux Journal, and on the O'Reilly
Network site. He holds a Ph.D. in Theoretical Physics from the University of Washington in Seattle. You can contact
the author at janert@ieee.org.

Share this....
Digg this story del.icio.us Slashdot it!