Data Validity Testing
Data Validity Testing
1
Why back end testing is so important
A back end is the engine of any client/server system. If the back end malfunctions, it may
cause system deadlock, data corruption, data loss and bad performance. Many front ends
log on to a single SQL server. A bug in a back end may put serious impact on the whole
system. Too many bugs in a back end will cost tremendous resources to find and fix bugs
and delay the system developments.
It is very likely that many tests in a front end only hit a small portion of a back end. Many
bugs in a back end cannot be easily discovered without direct testing.
Why back end testing is so important
A back end is the engine of any client/server system. If the back end malfunctions, it may
cause system deadlock, data corruption, data loss and bad performance. Many front ends
log on to a single SQL server. A bug in a back end may put serious impact on the whole
system. Too many bugs in a back end will cost tremendous resources to find and fix bugs
and delay the system developments.
It is very likely that many tests in a front end only hit a small portion of a back end. Many
bugs in a back end cannot be easily discovered without direct testing.
Component testing will be done early in the development cycle. Integration and system tests
(including interfaces to front ends and nightly processes) are performed after the component
tests pass. Regression testing will be performed continuously throughout the project until it
is finished. The back end usually does not have an independent beta test, as it only
exercised by the front end during the beta test period. The last step is to deliver users a
quality product
2
application’s functionality. The test focus is on functionality of input and output but not on the
implementation and structure. Different projects may have different ways to break down.
Boundary testing
Many columns have boundary conditions. For example, in a column for percentages, the
value cannot be less than zero and cannot be greater than 100%. We should find out these
types of boundary conditions and test them.
Stress testing
It involves subjecting a database to heavy loads. For incidence, many users heavily access
the same table that has a large number of records. To simulate this situation, we need to
start as many machines as possible and run the tests over and over.
STRUCTURAL BACK END TESTS
Although not all databases are the same, there are a set of test areas that will be covered in
all test specifications.
Based on structure, a SQL database can be divided into three categories: database
schema, stored procedures, and triggers. Schema includes database design, tables, table
columns, column types, keys, indices, defaults, and rules. Stored procedures are
constructed on the top of a SQL database. The front end talks to APIs in DLL. The APIs
communicate a SQL database through those stored procedures. Triggers are a kind of
stored procedures. They are the "last line of defense" to protect data when data is about to
be inserted, updated or deleted.
Test Coverage Criterion: “EACH AND EVERY ITEM IN SCHEMA MUST BE TESTED AT
LEAST ONCE”
1.1 Databases and devices
Verify the following things and find out the differences between specification and actual
databases
• Database names
• Data device, log device and dump device
• Enough space allocated for each database
• Database option setting (i.e. trunc. option)
3
• Whether a rule is bound to correct table columns
• Whether access privileges are granted to correct groups
• Foreign keys
• Column data types between a foreign key column and a column in other table
• Indices, clustered or nonclustered; unique or not unique
Return values:
• Whether a stored procedure returns values
• When a failure occurs, nonzero must be returned.
Error messages:
• Make stored procedure fail and cause every error message to occur at least once
• Find out any exception that doesn’t have a predefined error message
Others:
• Whether a stored procedure grants correct access privilege to a group/user
• See if a stored procedure hits any trigger error, index error, and rule error
• Look into a procedure code and make sure major branches are test covered.
2.2 Integration tests of procedures
• Group related stored procedures together. Call them in particular order
• If there are many sequences to call a group of procedures, find out equivalent classes and
run tests to cover every class.
4
• Make invalid calling sequence and run a group of stored procedures.
• Design several test sequences in which end users are likely to do business and do stress
tests
Trigger tests
Test Coverage Criterion: “EACH AND EVERY TRIGGER AND TRIGGER ERROR MUST
BE TESTED AT LEAST ONCE”
1. Updating triggers
Verify the following things and compare them with design specification
5
4. Integration tests of SQL server
Integration tests should be performed after the above component testing is done. It should
call stored procedures intensively to select, update, insert and delete records in different
tables and different sequences. The main purpose is to see any conflicts and incompatibility.
• Conflicts between schema and triggers
• Conflicts between stored procedures and schema
• Conflicts between stored procedures and triggers
Functional tests more focus on functionality and features of a back end. Test cases can be
different from project to project. But many projects have things in common. The following
section discusses the common areas. We encourage testers to add project-specific test
cases in the functional test design.
How to divide back end on function basis
It is not a good idea to test a server database as a single entity at initial stage. We have to
divide it into functional modules. If we can not do the partition, either we do not know that
project deep enough or the design is not modulized well. How to divide a server database is
largely dependent on features of a particular project.
METHOD 1: We may ask ourselves what the features of a project are. For each major
feature, pick up portion of schema, triggers and stored procedures that implement the
function and make them into a functional group. Each group can be tested together. For
6
example, the Forecast LRS project had four services: forecast, product lite, reporting, and
system. This was the key for functional partitioning:
METHOD 2: If the border of functional groups in a back end is not obvious, we may watch
data flow and see where we can check the data: Start from the front end. When a service
has a request or saves data, some stored procedures will get called. The procedures will
update some tables. Those stored procedures will be the place to start testing and those
tables will be the place to check test results.
1. Test functions and features
Test Coverage Criterion: “EACH AND EVERY FUNCTION OR FEATURE MUST BE
TESTED AT LEAST ONCE”
The following areas should be tested:
• Every feature no matter major or minor
• For updating functions, make sure data is updated following application rules
• For insertion functions, make sure data is inserted following application rules
• For deletion functions, make sure data is deleted correctly
• Think about if those functions make any sense to us. Find out nonsense, invalid logic, and
any bugs.
• Check for malfunctioning
• Check for interoperations
• Error detection
• Error handling
• See if error messages are clear and right.
• Find out time-consuming features and provide suggestions to developers
Checking data integrity and consistency
This is a really important issue. If a project does not guarantee data integrity and
consistency, we have obligation to ask for redesign. We have to check the minimum things
below:
• Find out data protection mechanisms for a project and evaluate them to see if they are
secure
• Data validation before insertion, updating and deletion.
• Triggers must be in place to validate reference table records
• Check major columns in each table and see if any weird data exist. (Nonprintable
characters in name field, negative percentage, and negative number of PSS phone calls per
month, empty product and so on)
• Generate inconsistent data and insert them into relevant tables and see if any failure
occurs
• Try to insert a child data before inserting its parent’s data.
• Try to delete a record that is still referenced by data in other table
• If a data in a table is updated, check whether other relevant data is updated as well
• Make sure replicated servers or databases are on sync and contain consistent information
7
• Email validation
• SQL user login (user id, password, host name)
• NT server login
• Database access privilege (the sysusers table)
• Database security hierarchy
• Table access privilege (if ‘select’ is allowed.)
• Table data access control
• Training account (maybe no password is required)
There are more test cases here:
• Simulate front end login procedure and check if a user with correct login information can
login
• Simulate front end login procedure and check if a user with incorrect login information fail
to login
• Check concurrent logins (make many users login at the same time.)
• Try to login when a time-consuming query is running to see how long login will take to
succeed
• Check for any security-restrict functions and see they are working proper
See any data view restriction in place, such as, a user can see his data and the data of
people who report to him.
4. Stress Testing
We should do stress tests on major functionality. Get a list of major back end
functions/features for a project. Find out corresponding stored procedures and do the
following things:
Test Coverage Criterion: “EACH AND EVERY MAJOR FUNCTION OR FEATURE MUST
BE INCLUDED IN STRESS TESTING”
• Write test scripts to try those functions in random order but every function must be
addressed at least once in a full cycle.
• Run test scripts over and over for a reasonable period
• Make sure log execution results and errors.
• Analyze log files and look for any deadlock, failure out of memory, data corruption, or
nothing changed.
5. Test a back end via a front end
Sometimes back end bugs can be found by front end testing, specially data problem. The
following are minimum test cases:
• Make queries from a front end and issue the searches (It hits SELECT statements or query
procedures in a back end)
• Pick up an existing record, change values in some fields and save the record. (It involves
UPDATE statement or update stored procedures, update triggers.)
• Push FILE - NEW menu item or the NEW button in a front end window. Fill in information
and save the record. (It involves INSERT statements or insertion stored procedures,
deletion triggers.)
• Pick up an existing record, click on the DELETE or REMOVE button, and confirm the
deletion. (It involves DELETE statement or deletion stored procedures, deletion triggers.)
8
• Repeat the first three test cases with invalid data and see how the back end handles them.
benchmark testing
When a system does not have data problems or user interface bugs, system performance
will get much attention. The bad system performance can be found in benchmark testing.
Four issues must be included: