Content Server 6.5 SP2 Full-Text Indexing Deployment and Administration Guide

EMC ® Documentum ®
Content Server
Version 6.5 SP2
Full-text Indexing Deployment and Administration

Guide
P/N 300-010-042 A01
EMC Corporation
Corporate Headquarters:
Hopkinton, MA 01748-9103
1-508-435-1000
www.EMC.com
Copyright © 1992 - 2009 EMC Corporation. All rights reserved.
Published September 2009
EMC believes the information in this publication is accurate as of its publication date. The information is subject to change
without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS
OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY
DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.
For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.
All other trademarks used herein are the property of their respective owners.
Table of Contents
Preface ................................................................................................................................. 9
Part 1 Overview ............................................................................................................. 11

Chapter 1 Overview ..................................................................................................... 13
Introduction ..................................................................................................... 13
Features ........................................................................................................... 14
Using grammatical normalization ................................................................. 14
Architectural overview ..................................................................................... 15
Indexing process overview ................................................................................ 16
Part 2 Deploy ................................................................................................................. 19

Chapter 2 Prepare for Installation ................................................................................ 21
Pre-installation procedure ................................................................................. 21
Preinstallation checklist .................................................................................... 25
Chapter 3 Install ........................................................................................................... 27

Overview ......................................................................................................... 27
Install a high-availability deployment ................................................................ 28
Installing the index server and the index agent configuration program................. 30
Configure indexing and querying ...................................................................... 32
Enabling synonym searches .......................................................................... 33
Disabling indexing of specific object types ...................................................... 33
Rendition formats to index ............................................................................ 34
Configuring format objects to specify which renditions are indexed ................. 34
Supported formats and mime_types .......................................................... 35
Configuring TRY_FTDQL_FIRST query retries on failure ................................ 35
Forcing permissions checks against the repository properties .......................... 36
Configuring batched returns for non-FTDQL queries ...................................... 36
Configuring the index agent .............................................................................. 36
Modifying the IndexAgent.xml file to map file stores .......................................... 38
Reviewing the installation log files..................................................................... 39
Installing the index server in a multinode configuration ...................................... 40
Installing the Indexing Software in a multinode environment .............................. 40
Installing a multinode configuration .............................................................. 40
Configuring the index agent .......................................................................... 42
Modifying the indexagent.xml file to map file stores ................................... 43
Configuring directed routing ......................................................................... 44
Creating full-text collections ...................................................................... 45
Mapping file stores to collections ............................................................... 46
Mapping collections to index columns ....................................................... 47
EMC Documentum Content Server Version 6.5 SP2 Full-text Deployment and Administration Guide 3
Table of Contents
Adding a node ............................................................................................. 49

Removing a node.......................................................................................... 51
Chapter 4 Upgrade ....................................................................................................... 53

Overview ......................................................................................................... 53
Adding the full-text indexing system after upgrading from a 5.3
repository ....................................................................................................... 53
Upgrading a full-text system on a 5.3 SP2 or SP3 repository without
the December 2006 Hotfix ................................................................................. 54
Upgrading from the December 2006 Fulltext Hotfix or 5.3 SP4 or later ................. 56
Chapter 5 Uninstall ...................................................................................................... 59

Uninstall order ................................................................................................. 59
Deleting an index agent .................................................................................... 59
Deleting the index agent configuration program ................................................. 60
Deleting an index server ................................................................................... 61
Deleting a full-text Index ................................................................................... 61
Part 3 Administer ........................................................................................................... 63

Chapter 6 Manage a Full-text Index ............................................................................. 65
Managing and administering the index agent and index server ........................... 65
The index agent JMX URL ................................................................................. 65
Managing the index queue ................................................................................ 66
Resubmitting individual objects .................................................................... 67
Resubmitting all failed queue items ............................................................... 67
Removing queue items by status ................................................................... 67
Removing queue items.................................................................................. 68
Viewing queue items associated with an object ............................................... 68
Creating a new indexing queue item .............................................................. 68
Limitations of full-text indexing in high-availability configurations ................. 69
Prune and missing destroy events.............................................................. 69
Save events not generated during load operations ...................................... 69
The “State of the Index” job ............................................................................... 69
Arguments ................................................................................................... 70
Using the matchsysobjversion and matchallversion arguments ........................ 71
Job report and generated files ........................................................................ 71
Turning indexing on and off .............................................................................. 72
Turning off all indexing ................................................................................ 72
Turning off content indexing ......................................................................... 73
Suspending and resuming indexing ................................................................... 73
Suspending and resuming an index server in a single-node
configuration................................................................................................ 73
Suspending and resuming an index server in a multinode
configuration................................................................................................ 73
Reindexing a repository .................................................................................... 74
Creating a new index ........................................................................................ 74
Pointing a repository to a previously created index ............................................. 75
Troubleshooting indexing timeouts .................................................................... 75
Chapter 7 Manage Full-text Indexing Components ...................................................... 77
4 EMC Documentum Content Server Version 6.5 SP2 Full-text Deployment and Administration Guide
Table of Contents
Administration tools ......................................................................................... 77

The Index Agent Admin Tool ........................................................................ 78
Starting and stopping the full-text indexing system ............................................ 78
Starting and stopping the index agent ............................................................ 79
The dm_FTIndexAgentBoot job ..................................................................... 80
Starting and stopping the index server ........................................................... 80
Enabling and disabling index agents .............................................................. 81
Viewing or modifying index agent properties ..................................................... 81
Viewing index server properties ........................................................................ 82
Reviewing the log files ...................................................................................... 82
Index agent log files ...................................................................................... 83
Index server log files ..................................................................................... 83
Application server log files ............................................................................ 83
Administration operations ................................................................................ 84
Enabling thesaurus searching ........................................................................ 84
Creating the synonym file ......................................................................... 85
Importing the synonym file ....................................................................... 85
Logging ................................................................................................... 86
Log sample .......................................................................................... 87
Obtaining a list of indexable formats .............................................................. 89
Tracing full-text query operations .................................................................. 89
Enabling tracing for the index agent............................................................... 89
If a node fails in a high-availability configuration............................................ 90
Cleaning up old log files ............................................................................... 90
Large file rejection error .................................................................................... 90
Increasing capacity ........................................................................................... 91
Increasing indexing capacity ......................................................................... 91
Increasing the number of exporter threads in the index agent .......................... 91
Chapter 8 Back up and Restore Full-text Indexes ........................................................ 93

Overview ......................................................................................................... 93
Configure the system for backup and restore .................................................. 94
Perform a cold backup ...................................................................................... 94
Restore an index at a particular point in time ...................................................... 96
Chapter 9 Verify Full-text Index Completeness and Accuracy ..................................... 99

Overview ......................................................................................................... 99
Modifying the parameter file ........................................................................... 100
Additional parameters ................................................................................ 101
Running the index verification tool .................................................................. 103
Accuracy verification acceptable failures ...................................................... 105
Resubmitting objects to the index agent ........................................................... 105
Part 4 Reference .......................................................................................................... 109

Appendix A Components .............................................................................................. 111
Index server ................................................................................................... 111
Index server processes ................................................................................ 111
Index server modes..................................................................................... 112
Index agent ................................................................................................... 112
Index agent processes ................................................................................. 113
Index agent modes ..................................................................................... 114
Table of Contents
Normal mode ........................................................................................ 114

Migration mode ..................................................................................... 115
File mode ............................................................................................... 115
Full-text index ................................................................................................ 115
Partitions ................................................................................................... 116
Collections ................................................................................................. 116
Directed routing ..................................................................................... 116
Repository objects and properties supporting full-text indexing......................... 117
Fulltext index object .................................................................................... 119
FT index agent config object ........................................................................ 119
FT engine config object ............................................................................... 119
Location objects .......................................................................................... 119
Supporting properties of other objects.......................................................... 119
The a_full_text property .......................................................................... 120
The fulltext_location property ................................................................. 120
Initialization files ........................................................................................ 120
Fulltext entries in the server.ini file .......................................................... 120
The dmfulltext.ini file ............................................................................. 121
Appendix B Sample Output of ftintegrity Utility ............................................................ 123

Appendix C Indexed and Non-Indexed Characters ....................................................... 127
Indexed characters .......................................................................................... 127
Non-indexed special characters ....................................................................... 127
Appendix D Supported and Unsupported Formats ....................................................... 129

Appendix E Supported Languages ............................................................................... 137
Table of Contents
List of Figures
Figure 1. Full-text indexing components .............................................................................. 16

Figure 2. Mapping file stores to collections to index columns ................................................. 45
Figure 3. Fulltext indexing object relationships ................................................................... 118
Table of Contents
List of Tables
Table 1. Required environment variables ............................................................................ 23

Table 2. Checklist for Full-text Indexing ............................................................................. 25
Table 3. “State of the Index” job arguments......................................................................... 70
Table 4. Syntax of ImportDictionary.py script ..................................................................... 86
Table 5. ftintegrity parameters ......................................................................................... 101
Table 6. Supported document formats .............................................................................. 129
Table 7. Unsupported document formats .......................................................................... 135
Table 8. Supported languages .......................................................................................... 137
Preface
Purpose of the manual

This manual contains information and instructions for installing, upgrading, and maintaining the
full-text indexing system (including multiple nodes) used with EMC Documentum Content Server. It
describes decisions you need to make and requirements that you need to meet before you install the
full-text indexing software. It also provides step-by-step instructions for installing and upgrading
the software in several different configurations.
Intended audience
This manual is intended for the person installing Content Server and the full-text indexing software.
Typically, a system administrator installs the software.
Related documentation
• The Content Server Installation Guide contains information on installing Content Server.
• The Documentum Administrator online help system contains instructions for managing the index
queue and for starting and stopping the index server and index agent, in basic, consolidated,
and high-availability deployments. No Documentum Administrator support is provided for
multinode deployments.
• The EMC Documentum Search Development Guide contains complete information on querying.
Revision history
The following revisions have been made to this document:
Revision History
Date Description
July 2008 Initial publication for version 6.5.
Preface
Date Description
March 2009 Initial publication for version 6.5 SP1.5.
September 2009 Initial publication for version 6.5 SP2.
Part 1
Overview
Overview
Chapter 1
Overview
These topics are included:

• Introduction, page 13
• Features, page 14
• Architectural overview, page 15
• Indexing process overview, page 16
Introduction
A repository can contain millions of objects. Locating objects containing particular values can be a
challenge, whether you are trying to find a phrase, a date, or a name. You can make the task of
locating objects somewhat easier by carefully structuring folders to storing related objects together.
You can also use the Documentum Query Language (DQL) to rapidly locate objects whose metadata
contains the values for which you are searching. However, to meet this daunting challenge, full-text
indexes will enable you to quickly search the content of every indexable file as well as the properties
of every indexable object anywhere in your repositories.
The full-text indexing system:
• Creates indexes of all the words in every indexable object’s properties and content file in a
repository, and
• Enables searches to be made against those indexes and quickly return results.
Note: Phonetic searching is not supported.
An indexable object is an object that is a SysObject, SysObject subtype, or, optionally, a lightweight
SysObject (LWSO). (Because the SysObject type is the parent type of the most commonly used objects,
most of your relevant objects are indexable.) All properties of SysObject and SysObject subtype
objects are indexed automatically. A content file (which can reside in any storage area) is indexed
only if its associated object’s a_full_text property value is “TRUE” and the content file’s format is a
supported file format.
Note: Unless otherwise required for clarity, any usage of the term, “full-text indexing”, will be
simplified to “indexing”; for example, a full-text index will be referred to as an “index”.
Overview
Features
The full-text indexing system supports these major features:
• Wide variety of file formats
See Appendix D, Supported and Unsupported Formats.
• Indexes every object’s properties
• Wide variety of characters
See Appendix C, Indexed and Non-Indexed Characters.
• Wide variety of languages
All standard Unicode character sets are supported and no special configuration is necessary.
By default, content files and properties in all supported languages (including two right-to-left
languages, Hebrew and Arabic) are indexed.
See Appendix E, Supported Languages.
• Grammatical normalization (also known as “lemmatization”)
Enabling grammatical normalization forces all grammatical forms (such as singular and plural) of
a word to be indexed and includes all other grammatical forms of a word in a search. For example,
a query on “cat” would return objects containing not only “cat”, but also those containing “cats”.
See Using grammatical normalization , page 14.
• Thesaurus
You can specify in a thesaurus all of the synonyms to include in a search that only contains one of
the synonyms. For example, if the thesaurus specifies “feline”, “polecat”, “cougar”, and “bobcat”
as synonyms of “cat”, then a search on the term “cat” will also include the synonyms, “feline”,
“polecat”, “cougar”, and “bobcat”.
See Enabling thesaurus searching, page 84.
Using grammatical normalization

Grammatical normalization is a process that the full-text indexing system performs when indexing
and when processing a query if the feature is enabled. The process ensures that queries return
matches for terms and grammatical matches. For example, a query on ’cat’ would return objects
containing ’cat’ and those containing ’cats’. Grammatical normalization is not used for wildcard
searches on property values. The Search Development Guide contains a full description of how
grammatical normalization behaves during indexing and querying.
You can choose the combinations of parts of speech you want to be normalized. The following
combinations are offered:
• Nouns
• Nouns and adjectives
• Nouns, adjectives, and verbs
• Nouns and verbs
Overview
Grammatical normalization is enabled during index server installation. Enabling grammatical

normalization loads specific dictionaries into the index server host’s memory. By default, grammatical
normalization is enabled and the dictionaries for Japanese, Korean, and English are loaded. You
can choose to load additional dictionaries to support other languages. However, the memory
requirements for the additional dictionaries might result in lower search and indexing performance.
Enabling grammatical normalization increases memory use by the indexing process. You cannot
change the grammatical normalization option after installation. To change the grammatical
normalization option, you must uninstall the indexing software and delete the index, then reinstall
the software and index the entire repository again.
Architectural overview
Full-text indexing in a repository is controlled by three software components:
• Content Server, which does the following:
— Manages the objects in a repository
— Generates the events that trigger full-text indexing operations
— Queries the full-text indexes
— Returns query results to client applications
Note: Although Content Server supports full-text indexing by default, Content Server itself
does not create nor maintain full-text indexes. You need to install the full-text indexing software
components, which create and maintain the full-text indexes.
• The index agent, which exports documents from a repository and prepares them for indexing
• The index server, which is a third-party server product that creates and maintains the full-text
index for a repository. The index server also receives full-text queries from Content Server and
responds to those queries.
The index server’s operations are processor- and memory-intensive. Therefore, EMC
Documentum recommends that you install the index server on a host other than the Content
Server host. You can install multiple index servers on a single UNIX or Linux host, provided the
directories, ports, and environment variables for each index server are configured properly. You
can install only one index server on a Windows host.
Note: DQL, the Content Server query language, supports querying against full-text indexes. You
can run queries against full-text indexes only or against full-text indexes and the metadata tables.
For information about the types of queries supported by DQL, refer to the Search Development
Guide.
Figure 1, page 16, illustrates the relationships among the Content Server, index agent, and index
server. For indexing, the Content Server sends metadata and content for indexing to the index agent,
which in turn packages that information as Documentum Full-text XML (DFTXML) and sends it to
the index server. DFTXML is an XML format that contains the object’s properties and the location
of the object’s content file. The index server processes the information and updates the index. For
queries, Content Server sends the query to the index server, and the index server’s query server
process returns the results to Content Server.
Overview
Figure 1. Full-text indexing components
Indexing process overview

The full-text index is updated on a continuous basis, provided that all software components are
running. No special administrative tasks are required to ensure that the index is updated and current.
The indexing process begins when an object is added to the repository, removed from the repository,
or updated in a way that might affect the full-text index. The process in response to the changed
object is:
1. Content Server generates a queue item and adds it to the work queue for the index agent.
2. The index agent acquires the queue item from its work queue.
3. The index agent retrieves the object associated with the queue item from the repository and
creates a DFTXML representation of the object.
If you are using distributed content and a content file is located at a remote component of the
distributed store, the content file is copied to the distributed store component located at the
primary site.
4. The index agent sends the DFTXML representation of the object to the index server.
5. The index server retrieves the content file associated with the object from the repository and
creates its own representation of the content in the file and properties of the object. This internal
representation is called FIXML.
Note: The full-text indexing system rejects for indexing any FIXML file larger than 10MB.
6. The index server notifies the index agent that the FIXML is created and the object will be indexed,
and the index agent destroys the queue item for the object.
7. The index server indexes the content file and its properties, adding its information to the full-text
index.
The object is now searchable.
Overview
The indexing process is not destructive to existing content or attributes in the repository. The content
files and object attributes are read, but not modified, during the indexing process.
Overview
Part 2
Deploy
Deploy
Chapter 2
Prepare for Installation
Use the information in this chapter to prepare your installation for the full-text indexing component
installation.
• Pre-installation procedure, page 21
• Preinstallation checklist, page 25
Pre-installation procedure
1. Install Content Server and configure a repository.
Use the instructions in the Content Server Installation Guide.
2. Host names
You need to identify the host where the index server and index agent are installed by a fully
qualified domain name (FDQN). For example, the host name isolde.documentum.com is
acceptable, but an IP address, for example, 172.04.8.275 is not acceptable.
3. Ports to use for the index agent
The index agent runs in the application server container. When an index agent instance is
configured, you need to designate two ports for the index agent and application server to use.
The default ports for the first index agent on a host are 9200 and 9080. The default for any other
index agents are 90220 for index agent 2, 9240 for index agent 3, and so on. If the index agent is
on the Content Server host, ensure that the ports are not the ports used for the application server
instance in which the Java method server and ACS server run.
4. Ports to use for the index server
The index server requires a contiguous range of 4000 free ports. You must designate which ports
to use during installation. The default range is from 13000 to 17000.
5. Index server operating system and host
Note: Installing the index server installs Java on the host. The installed version of Java is the same
version as the Java version installed with Content Server.
You need to install the index server on a supported operating system. EMC Documentum
recommends using a host on which a clean installation of the operating system has been
performed.
Install the index server on a disk partition separate from the system partition and larger than
the system partition.
Caution: Do not install the index server on VMware in a production environment.
Constraints on third-party software on the index server host — The following restrictions
apply to the index server host:
• Do not run network security scanning software on a host where the index server is installed.
Network security scanners might lock index server processes, which can create intermittent
search and indexing failures.
• Do not run backup utilities while the index server is running.
Backup utilities might lock indexing processes.
• Do not run antivirus software on the %FASTSEARCH% directory, where the index server and
indexes are stored.
Antivirus software interferes with index server startup and proper functioning. Antivirus
software might quarantine log and other frequently-changed files.
EMC Documentum recommends testing any third-party monitoring tools on a development
system before the tools are deployed to a production system where the index server is installed.
Windows host requirements for the index server — The following restrictions apply to
Windows hosts:
• Do not run the Windows Index System on the index server host.
• On 32-bit Windows hosts, do not set the /3GB option in the boot.ini file.
• Disable automatic Windows updates on index server hosts.
• Do not install the index server on a domain controller.
Note: If a 5.2.x repository is running on a Windows host and you are performing a pre-upgrade
index migration, you must install the index agent and index server on a host other than the
Content Server host. For more information, refer to Chapter 4, Upgrade.
6. Host time settings
Set the time zone on the host where the index server runs to Greenwich Mean Time (GMT) or
Universal Time Coordinated (UTC). On Windows hosts, clear Automatically adjust clock for
daylight saving changes.
7. Ensuring correct network configuration
If you are installing the indexing software on a host other than the Content Server host, ensure
that the domain name service (DNS) entries for the two machines are correct so that they are able
to locate each other on the network.
To verify the DNS entries:
a. On the index server host, look up the Content Server host:
nslookup FQDN_of_Content_Server_host
where FQDN_of_Content_Server_host is the FQDN of the Content Server host.
This returns one or more IP addresses for the Content Server host.
b. Use the first IP address returned in step 1 for a reverse lookup:
nslookup IP_address_returned
The correct return value is the same FQDN you entered in step 1.
c. If the two nslookup commands do not return the correct values, update the DNS servers used
by the two hosts to reflect the correct FQDNs.
d. If necessary, on Windows with more than one network card, update the host files to ensure
that the correct IP address for each host is listed first.
e. If the nslookup commands succeeded and return the correct values, ping the index server
host from the Content Server host to ensure that it responds to the pin and to ensure that the
IP address that responds to the ping is the IP address defined in the ftengine config object.
8. Index agent and index server installation account
You need to install the index agent and index server under the same user name under which you
installed Content Server (the Content Server installation owner). If you are installing the index
agent and index server on a host other than the Content Server host, ensure that the user exists
on that host. Refer to “Installation Owner Account” in the correct section for your operating
system, in Chapter 3, “Preparing for Installation,” of the Content Server Installation Guide, for more
information on the installation owner account.
9. Environment variables on UNIX and Linux hosts
You must set the following environment variables in the installation owner’s environment on
UNIX and Linux hosts before installing the index agent and index server:
Table 1. Required environment variables
Environment Variable Description Required Values

DOCUMENTUM The directory in which the Any directory in the
indexing software is installed installation owner’s
environment
DOCUMENTUM_SHARED The directory in which the Any directory in the
Documentum Foundation installation owner’s
Classes (DFC) are installed environment
LD_LIBRARY_PATH, The index server library $DOCUMENTUM/fulltext/
SHLIB_PATH, or LIBPATH location IndexServer/lib
$DOCUMENTUM/fulltext/
fast
$DOCUMENTUM_SHARED/
dfc
$DOCUMENTUM_SHARED/
IndexAgents/ftintegrity
FASTSEARCH Location of the index server $DOCUMENTUM/fulltext/
IndexServer
DISPLAY Controls the display localhost:0.0
Environment Variable Description Required Values

LC_ALL C
JAVA_HOME The home directory for the Any directory in the
Java installation on the host installation owner’s
environment
10. Ensuring that the index server environment is correct on UNIX and Linux hosts
The index server installation includes a script that sets required environment variables for
running the index server. The script is setupenv.sh or setupenv.csh, depending on the shell from
which you run, and it is located in the indexserver_install_dir/bindirectory. You can source this
script to ensure that the environment variables are correct.
11. The deprecated DFC_DATA environment variable on UNIX hosts
The DFC_DATA environment variable was deprecated after the 5.1 EMC Documentum
release, but it is still used by Documentum installers for backward compatibility. If you are
installing the indexing software on a UNIX host where older EMC Documentum software
required setting DFC_DATA, the installer uses the value of DFC_DATA to create the /config
directory ($DFC_DATA/config). However, the startupIndexAgent.sh script expects to
find the $DOCUMENTUM_SHARED variable set and expects the /config directory to be
$DOCUMENTUM_SHARED/config.
If the /config directory is not $DOCUMENTUM_SHARED/config, edit the startupIndexAgent.sh
script so that it points to the valid /config directory path on the index agent host. Replace these
lines:
CLASSPATH=$DOCUMENTUM_SHARED/dctm.jar:$DOCUMENTUM_SHARED/config:
$DOCUMENTUM_SHARED/dfc/dfc.jar:$DOCUMENTUM_SHARED/dfc/dfcbase.jar:
$DOCUMENTUM_SHARED/dfc/log4j.jar
with:
CLASSPATH=$DOCUMENTUM_SHARED/dctm.jar:$DOCUMENTUM/config:
$DOCUMENTUM_SHARED/dfc/dfc.jar:$DOCUMENTUM_SHARED/dfc/dfcbase.jar:
$DOCUMENTUM_SHARED/dfc/log4j.jar
12. Installing the index server on Windows hosts

The Windows file cscript.exe is required for running the index server installer. Do not delete
cscript.exe from a Windows host on which you are installing the index server.
Preinstallation checklist
Use following the checklist to ensure that you have performed all required tasks before installing
or upgrading the full-text indexing software.
Table 2. Checklist for Full-text Indexing
Task For More Information Completed?

Determine which full-text Documentum System Planning
indexing configuration to use. Guide
Determine the port numbers Pre-installation procedure,
the index agent uses. page 21,
Determine the base port the Pre-installation procedure,
index server uses. page 21,
Ensure that 4,000 ports above Pre-installation procedure,
the base port number are page 21,
unused and available for the
index server, for example, if the
base port number is 3,000, port
numbers from 3,000 to 7,000
must be available.
Determine the user accounts Pre-installation procedure,
to use for installing the index page 21,
agent and index server.
Ensure that the user accounts Network administrators
are set up.
Ensure that the disk space and • Documentum System Planning
memory requirements for the Guide
indexing software are met.
• Chapter 2, Prepare for
Installation
For a Content Server upgrade, Chapter 4, Upgrade

decide whether to migrate
the indexes before or after
the server and repository are
upgraded.
Determine where the full-text Documentum System Planning
indexes are to be created and Guide
stored.
Ensure that any additional Chapter 2, Prepare for
hosts required by the index Installation
agent and index server are
configured.
Task For More Information Completed?

Mount the drive where the • Hardware and operating
content files are located to system vendor
make the content accessible to documentation
the index servers.
• Documentum System Planning
Guide
Determine for which languages Using grammatical
grammatical normalization is normalization , page 14
required.
Chapter 3
Install

• Overview, page 27
• Install a high-availability deployment, page 28
Review this section before you install a high-availability configuration or if you are converting an
existing installation to high availability.
• Installing the index server and the index agent configuration program, page 30
• Configuring the index agent, page 36
• Modifying the IndexAgent.xml file to map file stores, page 38
• Reviewing the installation log files, page 39
• Installing the index server in a multinode configuration, page 40
• Installing the Indexing Software in a multinode environment, page 40
Overview
This chapter contains instructions for installing the full-text indexing software and creating
full-text indexes, whether you are upgrading from an earlier Documentum release or creating new
repositories.
The high-level procedure for installing the full-text indexing server and full-text indexing components
for a new installation is:
1. Install the index server and index agent configuration program.
Use the instructions in Installing the index server and the index agent configuration program,
page 30.
2. Configure the index agent.
Configuring the index agent, page 36 has instructions.
3. Start the index agent in normal mode.
To install a consolidated deployment, install the index agent and configure an index agent for each
repository, then index each repository.
Install
To install a high availability configuration, you need to install the indexing software on the first host,
perform steps on the Content Server host, install the indexing software on the second host, and
perform additional steps on the Content Server host.
Install a high-availability deployment

Follow these instructions to install full-text indexing in a high-availability configuration (whether you
are installing a new indexing system or adding high availability to an existing basic configuration).
Note: High-availability indexing is supported with consolidated configurations.
1. If you do not already have an existing full-text indexing system, install the full-text indexing
software on the first host and configure an index agent.
Note: On Windows, you need to log on as the same user, in the same domain, as the user who
installed the Content Server installation.
Use the instructions in Installing the index server and the index agent configuration program,
page 30 and Configuring the index agent, page 36.
2. Log on to the Content Server host as the installation owner.
3. Ensure that no users are connected to the repository.
4. Shut down the first index agent.
5. Navigate to the directory where the create_fulltext_objects_ha.ebs script is located:
• On Windows, the %DM_HOME%\install\admin folder
• On UNIX or Linux, the $DM_HOME/install/admin directory
6. If you are upgrading the system, run the following command:
dmbasic -f create_fulltext_objects_ha.ebs
-e HACleanupBeforeUpgradeStep --
repository_name
Superuser_name Superuser_password
where repository_name is the name of the repository, Superuser_name is the user name of a user
with superuser privileges in the repository, and Superuser_password is the superuser’s password
7. Run the create_fulltext_objects_ha.ebs script using this syntax, where repository_name is the name
of the repository, Superuser_name is the user name of a user with Superuser privileges in the
repository, and Superuser_password is the Superuser’s password:
-e HAPreInstallStep -- repository_name
8. Install the full-text indexing software on the second indexing host and configure an index agent,
using the instructions in Installing the index server and the index agent configuration program,
page 30 and Configuring the index agent, page 36.
Do not start the new index agent. The repository now contains two full-text index objects, two
ft index agent config objects, and two ft engine config objects.
9. Log on to the Content Server host as the Content Server installation owner.
Install
10. Navigate to the directory where the create_fulltext_objects_ha.ebs script is located:

• On Windows, the %DM_HOME%\install\admin folder
• On UNIX or Linux, the $DM_HOME/install/admin directory
11. Run the create_fulltext_objects_ha.ebs script using this syntax, where repository_name is the name
of the repository, Superuser_name is the user name of a user with superuser privileges in the
repository, and Superuser_password is the superuser’s password:
-e HAPostInstallStep -- repository_name
12. Restart the Content Server.

Note: The index agent restarts only if the index agent’s application server instance is running.
13. Confirm that the required objects have been created and updated:
a. In Documentum Administrator, connect to the repository as a user with superuser privileges.
b. Execute the following DQL statement to verify that there are two fulltext index objects, each
associated with one of the two indexes and that each has an associated ftengine config object:
SELECT r_object_id,index_name,ft_engine_id,is_standby
from dm_fulltext_index
The query returns two rows, one with an index name ending in 00, which represents the
primary index, and one with an index name ending in 01, which represents the standby index.
c. Execute the following query to verify that correct ftengine config object is associated with
the primary index:
SELECT r_object_id,object_name FROM dm_ftengine_config
The object ID in the returned row with the object name that ends in 00 matches the
ft_engine_id value returned in the previous step for the primary index.
d. Execute the following DQL statement to verify that there is an ftindex agent config object for
each index and that the queue user is correctly set for the second index:
select r_object_id,object_name,index_name,queue_user
from dm_ftindex_agent_config
The object_name records the name of the index agent and index_name is the name of the
index. The name of the primary index ends in 00. The name of the standby index ends in 01.
The queue user for the primary index is dm_fulltext_index_user, and for the standby index,
the queue user is dm_fulltext_index_user_01.
14. Start the index agents on both hosts.
Starting the index agents will create the indexes.
15. Verify the indexes on both hosts.
Use ftintegrity or the State of the Index job to verify the indexes.
Install
Installing the index server and the index agent

configuration program
Use these instructions to install the index agent or the index server software. The same installation
program is used for both components. You can install either or both of the components on a
particular host.
Installing the index server also installs Java on the index server host. The Java version level is the
same as the version level of Java installed with Content Server.
The installer installs the index agent configuration program, which you use to configure an index
agent instance. If you do not configure the index agent immediately after installing the configuration
program, you can do so at a later time.
To install the index server and the index agent configuration program:
1. On HP-UX, you must set the following parameter values:
• Set maxdsiz or data seg size at 2 GB (0x80000000)
• Enable Largefiles
• Set maxusers to 256 or higher
• Set max_thread_proc to 256 or higher
• Set maxfiles to 1024 or higher
2. Ensure that the repository (for which you are installing the index server and index agent)
is running.
3. Log on to the index server and index agent host as the Content Server installation owner.
On Windows, this means you need to log on as the same user, in the same domain, as the user
who installed the Content Server installation.
4. Copy the installation files from the EMC Documentum download site or distribution CDs to
a temporary location on the host.
5. Start the installation program.
• On Windows, double-click fulltextWinSuiteSetup.exe.
• On UNIX and Linux, type
% fulltextoperatingsystemSuiteSetup.bin
and press Enter, where operatingsystem is the operating system on which you are installing.
A Welcome dialog box is displayed.
6. Click Next.
The license agreement dialog box is displayed.
7. Click I accept the terms of the license agreement and click Next.
A dialog box is displayed that lists the products you can install.
8. Choose the products to install.
• To install the index agent, check Documentum Index Agent Configuration Program.
• To install the index server, check Index Server.
Install
9. Click Next.
10. Indicate whether to install the developer documentation and click Next.
11. If required, install DFC.
a. On Windows, accept the default installation directory or type a different directory name
and click Next.
This is typically C:\Program Files\Documentum. On UNIX and Linux, the DFC directories
are determined by environment variables set before installation.
b. On Windows, accept the default user directory or type the name of a different directory
and click Next.
This is typically C:\Documentum.
Note: If the DOCUMENTUM environment variable is set on the host, you need to accept the
default directories. The installation program uses the value of the DOCUMENTUM variable to
generate the default directories and will not allow you to change the directories.
12. If a dfc.properties file does not exist on the host, provide connection information.
Note: If you are installing the software to create an index for a repository prior to version
5.3 for migration purposes, the connection information is in a dmcl.ini file on the host, not
a dfc.properties file.
a. Type the name of a host where a connection broker is running.
b. Type the port number used by the connection broker.
c. Click Next.
13. Read the explanatory information about full-text indexing, then click OK.
14. To install the index server, complete these steps.
a. Accept the default index server installation directory or type the name of a new directory,
then click Next.
default directory. The installation program uses the value of the DOCUMENTUM variable to
generate the default directory and will not allow you to change the directory.
b. On Windows, type the password for the account you used to log on, then click Next.
The installer verifies the password.
c. Type the base port number for the index server, then click Next.
The index server requires 4,000 available ports in sequence; for example, if the base port
you designate is 3000, the index server uses ports 3000 through 7000. Do not use a port
in ephemeral range.
The default base port is 13000.
Note: You cannot change the chosen ports unless you remove and reinstall the index server.
d. Check the check box to enable support for grammatical normalization.
Choosing the parts of speech to be indexed can reduce the size of the indexes and the disk
space required for maintaining the indexes. You can enable grammatical normalization only
for the languages listed on the dialog box. If you enable grammatical normalization, it is
enabled by default for Japanese and Korean and you cannot disable it.
Install
Note: Content files in languages that are not chosen or that are not available for normalization
are still indexed.
e. Choose the languages for which to perform grammatical normalization and the parts of
speech to be indexed by selecting the appropriate check boxes.
The recommended choice is to normalize only nouns.
f. Accept the default directory for the full-text indexes or type the name of a different directory,
then click Next.
The default on Windows is %DOCUMENTUM%. The default on UNIX or Linux is
$DOCUMENTUM. If you choose another directory, the name cannot contain any blank
spaces. The installer creates the directory \data\fulltext (/data/fulltext on UNIX or Linux)
under the location you designate.
On HP-UX (B.11.23 U 9000/800), you cannot install the Index Server in a directory that
contains an “_d” in the directory path.
default directory. The installation program uses the value of the DOCUMENTUM variable to
generate the default directory and will not allow you to change the directory.
A summary dialog box is displayed, listing the products that will be installed.
15. Click Next.
The products are installed and a panel is displayed indicating success when the installation is
completed.
16. Ensure that the index server starts.
• On Windows, select Yes, restart my computer., then click Next.
— If the computer does not restart automatically, click Start > Shutdown > Restart and restart
the computer manually.
— If the index server does not automatically start, click Start > Programs > Administrative
Tools > Services and start the FAST InStream service.
— If the system restarts, the index server starts automatically as a Windows service.
• On UNIX and Linux, navigate to the $DOCUMENTUM/fulltext/IndexServer/bin directory
(the installation location), type startup.sh, and press Enter.
The index server is started.
Caution: Do not run backup utilities while the index server is running, because they may lock
indexing processes.
Configure indexing and querying

This section describes how to configure indexing and querying behaviors, to help ensure that users’
searches find the objects they want.
Install
Enabling synonym searches

Perform the following procedure if you want to execute synonym searches against the index.
Note: You also need to generate the synonyms to be searched. This procedure only enables searching
after the synonyms are generated.
To enable synonym searches:

1. Log on to the index server host as the installation owner.
2. Navigate to %FASTSEARCH%\etc\config_data\QRServer\webcluster\etc\qrserver
($FASTSEARCH/etc/config_data/QRServer/webcluster/etc/qrserver).
3. Open the qtf-config.xml file in an editor.
4. Insert the following entry after the <instance-re name=”tokenize”/> entry:
<instance-ref name="synonymy"/>
5. Save the file.
Disabling indexing of specific object types

By default, all instances of dm_sysobject or its subtypes are indexed. However, you can disable
indexing of instances of specific object types in repositories at version 5.3 SP5 or later.
A type is registered for indexing if any of its supertypes are registered for indexing. You cannot turn
off indexing for an object type if any of its supertypes are indexed. Because dm_sysobject and all of its
subtypes are indexed (registered for indexing) by default, if you want to disable indexing for one or
more subtypes, you need to turn off indexing for dm_sysobject and then enable indexing specifically
for those subtypes that you want indexed, leaving those you do not want to index unregistered.
Use Documentum Administrator to disable or enable indexing. You need to connect as a superuser.
The Properties page for each object type capable of being indexed has a check box named Registered
for indexing. That check box indicates whether the object type is currently registered for indexing.
The field is enabled or disabled depending on whether you can change the type’s indexing status.
To remove or add registration for indexing for an object type:

1. Start Documentum Administrator as a superuser.
2. Select the Types node.
3. Navigate to the object type whose registration you want to change and open its Properties page.
This needs to be dm_sysobject or one of its subtypes.
4. To stop indexing of instances of the type, clear Register for indexing.
5. To resume indexing of instances of the type, select Register for indexing.
Install
Rendition formats to index

Documents have content files in many formats. Some documents have primary content that is not
indexable, but also have indexable renditions of that content. Other documents have indexable
primary content and indexable renditions. By setting properties within the format objects for the
various formats, you can direct Content Server to index all indexable renditions of a document or
only preferred renditions. For instructions on setting format object properties to configure which
renditions are indexed, refer to Configuring format objects to specify which renditions are indexed,
page 34.
Configuring format objects to specify which renditions

are indexed
Objects can have multiple content files in different formats; also known as renditions. Some of the
renditions may not be indexable. By setting properties within the format objects for the various
formats, you can direct Content Server to index all indexable renditions of an object or the preferred
renditions only. See Configuring format objects to specify which renditions are indexed, page 34.
Properties of the format object determine which formats are indexable and which content files in
indexable formats are indexed. If the value of the can_index property of a content file’s format object
is set to TRUE, the content file is indexable. If the primary content of an object is not in an indexable
format, you can ensure that the content file is indexed by creating a rendition in an indexable format.
The format_class property of the format object may be set to values that determine which formats
are indexed:
• ft_always
All renditions in formats whose format_class property is set to ft_always are indexed. For
example, if a document has renditions in Microsoft Word and PDF formats and the format_class
property for both formats is set to ft_always, both renditions are indexed.
• ft_preferred
If a document has multiple renditions in indexable formats and one is in a format whose
format_class property is set to ft_preferred, the rendition in that format is indexed rather than any
renditions in other formats, with the exception that any formats whose format_class property is
set to ft_always are also indexed. If a document has more than one rendition whose format_class
property is set to ft_preferred, the first rendition processed for indexing is indexed and the other
renditions are not. You cannot determine in advance which rendition is processed for indexing.
EMC Documentum recommends that for any document, only one rendition is in a format whose
format_class property is set to ft_preferred.
If a document has renditions in four different formats, of which the format_class of one is set
to ft_preferred and the format_class of the other three is set to ft_always, all four renditions
are indexed.
No default value for the format_class property exists. You need to set it manually to designate
whether a format is always indexed or is the preferred format for indexing.
By default, the first content file in a format whose can_index property is set to true is indexed. Other
renditions of the object are not indexed. If the primary content of an object is not in an indexable
Install
format, create a rendition in an indexable format. Appendix D, Supported and Unsupported Formats,
contains a complete list of indexable formats.
If the content file associated with a SysObject exists in a no-indexable format, its properties are
still indexed. To index the content, create a rendition of the SysObject in an indexable format. Use
Documentum Content Transformation Services or third-party client applications to create the
rendition.
Supported formats and mime_types
Appendix D, Supported and Unsupported Formats lists the formats considered indexable by the
index server.
Some formats in the appendix are not represented in the repository by a format object. The
formats.cvs file, which is located in $DM_HOME/install/tools, contains a complete list of supported
mime_types and the formats with which they are associated. If a supported mime_type is not
represented by a format object, create a format object in the repository and map the supported
mime_type to the format.
Configuring TRY_FTDQL_FIRST query retries on failure

In DQL, users can include the TRY_FTDQL_FIRST hint to queries, which instructs Content Server
to first execute a query as an FTDQL query and, if the query fails with certain full-text error codes,
re-execute the query as a standard query.
By default, only three error codes, 1105, 1012, and 1017, are caught by Content Server, and cause the
server to re-execute the query as a standard query. You can add additional error codes to the list
of recognized error codes if you want queries with this hint to be retried as standard queries for a
larger number of errors.
To add additional error codes:

1. Set the param_name property in the dm_ftengine_config object to:
ftquery_retry_on_queryserver_errorcodes
2. Set the param_value property at the corresponding index position to the list of error codes
representing the additional errors you wish to catch.
The error codes are the four-digit codes returned in the error message. For example, here is an
error message with the four-digit error code highlighted in bold:
[DM_FULLTEXT_E_SEARCH_NEW_FAIL]error:"dmFTSearchnew
failed with error: QRServer Error (1012):
Resource limit exceeded, Error from QRServer, error code: -2"
Separate multiple errors codes with commas.
Install
Forcing permissions checks against the repository

properties
When a search is conducted, the returned results are filtered so that users receives only those results
for which they have the appropriate permissions. By default, the permission checking is against the
acl_name and acl_domain values stored in the index. However, you might want to conduct the
permission checking against the values stored in the repository for these properties, to ensure that the
values are the most current values.
To force the index server to check permissions against the ACL specified by the acl_name and
acl_domain properties in the repository, set the acl_check_db parameter to TRUE (or true) in the
dm_ftengine_config object. Set the parameter name in the param_name property and the value (true)
in the param_value property. The name and value need to be set at the same index position within
these repeating properties.
Configuring batched returns for non-FTDQL queries

Queries that contain a SEARCH clause but are not FTDQL queries are processed using a temporary
table. Content Server populates a temporary table with the results returned by the SEARCH clause
and then filters the results for security and matches to any other conditions specified in the query.
Because the full-text engine may return duplicate hits for an object, Content Server also processes
the full-text results within each batch to remove duplicates. (However, if there are duplicate rows
across batches, those duplicates are not removed.)
If the SEARCH clause has returned a large number of results, processing them might take some time.
To improve performance, the results are populated into the temporary table and are processed in
batches except when the query contains any of the following:
• An aggregate function, such as count() or sum().
• A UNION, IN DOCUMENT, IN ASSEMBLY, or ORDER BY clause
• A SEARCH clause in a subquery
In these cases, all results are populated into the temporary table for processing as a single batch.
To set the batch size, use the dm_ftengine_config object’s temp_table_batch_size parameter. The
default batch size is 20000. To disable the batching feature, set this parameter to 0 (zero). The
parameter name is set in param_name property and the value is set in the param_value property.
These are repeating properties, so you must set the name and value at the same index position within
the property. The value is an integer number representing the number of results in each batch.
You must reinitialize Content Server after setting the parameter.
Configuring the index agent

The index agent configuration program configures the index agent to process documents for a
particular repository and to pass the documents to the correct index server instance for indexing. Use
these instructions to run the index agent configuration program.
Install
To configure the index agent:

1. Ensure that the index server associated with the index agent is running.
To check that the index server is running, check the index server log file for a message that says:
fnet: engine up
2. Start the index agent configuration program.

• On Windows, after the host reboots and you log on as the installation owner, click Start >
Programs > Documentum > Index Agent Configuration Program.
• On UNIX and Linux, navigate to $DOCUMENTUM_SHARED/IndexAgents and start the
configuration program for your operating system:
— On AIX, IndexAgent_Configuration_Program.aix
— On Solaris, IndexAgent_Configuration_Program.bin
— On HP-UX, IndexAgent_Configuration_Program.hp
— On Linux, IndexAgent_Configuration_Program.linux
3. Click Next.
4. On Windows, type the installation owner’s password, then click Next.
5. Specify the port used to communicate with the application server for administration purposes,
then click Next.
The default port, 9020, is for the first index agent on the host. If the index agent is on the Content
Server host, the port number cannot be the port number used by the Java method server or
Site Caching Services.
6. Select the repository for which the index agent will prepare documents, then click Next.
The dropdown list contains the repositories that project to the connection brokers listed in the
dfc.properties file on the host.
7. Specify the user name and password for the superuser account that the index agent will use to
connect to the repository.
Use this user name and password later to access the Index Agent Admin Tool.
8. Indicate whether to run the index agent in normal mode or migration mode.
If the index agent is installed to run against a pre-5.3 repository for migration purposes, choose to
run the index agent in migration mode. Otherwise, choose normal mode.
9. Type the name of the host where the index server for this index agent is running and the base
port number for the index server, then click Next.
If you are configuring the index agent for the second indexing installation in a high-availability
deployment, ensuring that you point the index agent to the second index server host, not the first
index server host. The installer defaults to the first index server host.
A summary dialog box is displayed.
10. Click Next.
On Windows, the index agent is created and the application server is started.
Install
11. If you are on UNIX or Linux, navigate to $DOCUMENTUM_SHARED/domains/DctmDomain

and type startupIndexAgentN.sh, where Nis the number corresponding to the new index agent
instance, to start the index agent and its application server instance.
12. To create additional index agents, select the check box and click Next, then complete steps 4–10
again.
13. To exit from the configuration program, click Finish.
14. To complete the installation process:
• If you are mapping the file stores, complete the instructions in Modifying the IndexAgent.xml
file to map file stores, page 38.
• If you are installing a high-availability configuration and you have installed the index
server and configured the index agent on the first host, return to Install a high-availability
deployment, page 28 and continue from Step 2.
• If you are installing a high-availability configuration and you have installed the index server
and configured the index agent on the second host, return to Install a high-availability
deployment, page 28 and continue from Step 9.
Do not start the index agent on the second host.
Modifying the IndexAgent.xml file to map file

stores
If you have shared or mounted the drives containing the repository’s file stores and installed the
indexing software, you need to edit the index agent configuration file manually to indicate that the
drives are shared. The changes depend on whether the file system paths to the content are identical
on the Content Server host and index server host.
In a distributed configuration, all content located in distributed stores is moved to the primary
site, using the Surrogate Get method, and indexed at the primary site. You cannot map the remote
components of a distributed store. Similarly, you cannot map an encrypted store or an external store.
To modify the IndexAgent.xml file and map the file stores:

1. On the index agent host, navigate to C:\Documentum\jboss4.2.
0\domains\DctmDomain\upload\IndexAgent1\IndexAgent1.war\WEB-INF\classes
(on Windows) or $DOCUMENTUM_SHARED/jboss4.2.0/domains/DctmDomain/upload/
IndexAgent1/IndexAgent1.war/WEB-INF/classes (on UNIX).
If you installed multiple index agents on this host, replace the number 1 in these paths with the
appropriate index agent number.
2. Open the IndexAgent.xml file in a text editor.
3. If the paths to the content files are identical on the Content Server host and index server host,
locate the <exporter></exporter> element and change the value of the <all_filestores_local>
element to true:
<all_filestores_local>true</all_filestores_local>
Install
If the paths to the content files are different, do not modify the value of <all_filestores_local>, but
instead, create a file store map within the <exporter> element.
For example, if Content Server is on a host called Dandelion where filestore_01 is physically
located in the directory /Dandelion/Documentum/data/repository_name/content_storage_01 and
the index agent and index server on a host from which the drive on the Content Server host is
shared as /mappingtoDandelion/repository_name/content_storage_01, create an alias as follows:
<local_filestore_map>
<local_filestore>
<store_name>filestore_01</store_name>
<local_mount>/mappingtoDandelion/
repository_name
/content_storage_01</local_mount>
</local_filestore>
<!-- and so on for each filestore --!>
</local_filestore_map>
If you are indexing content stored on a NAS device or a Windows 2003 Server host, you may see
the following error message in the dmi_queue_item’s message attribute:
DocumentRetriever :ERROR Retrieval error: Couldn’t open file
<file path/name> ERROR Processor error status:
DataNotAvailable Not read permission
To resolve this error, edit the <local_mount> element or elements in the IndexAgent.xml file that
reference the storage area or areas on the NAS device. Add two back slashes immediately after
the opening <local_mount> element. For example, assume the following references a storage
are on an NAS device:
<local_mount>\\100.2.4.32\share3\c\data_for_example
\content_storage_1</local_mount>
After editing, it is now:
<local_mount>\\\\100.2.4.32\share3\c\data_for_example
4. Save the IndexAgent.xml file.

5. Restart the index agent.
Reviewing the installation log files

On any host where you installed the full-text indexing components, an installer log is generated.
Navigate to the directory from which you ran the installation and examine install.log for errors and
warnings.
Install
Installing the index server in a multinode

configuration
You install the index server in a multinode configuration by running the index server installation
program on the host machine for each node. The installation program refers to a configuration file
named InstallProfile.xml to determine which index server processes to install on the current host.
The major steps required to install the index server in a multinode configuration are:
1. Create a single, global copy of the InstallProfile.xml configuration file that defines the complete
configuration.
The InstallProfile.xml file lists the host machines for each node and specifies which index server
processes to install on each host. See .
2. Run the index server installation program on each host machine, as described in Installing a
multinode configuration, page 40.
3. Install an index agent to provide communication with Content Server.
The index agent is usually installed on the host machine where the index server’s administrative
processes are running, or on the Content Server host. See Configuring the index agent, page 42.
4. If you plan to use directed routing, configure the index agent and index server to provide the
desired mapping of documents to index columns.
See Configuring directed routing, page 44.
Installing the Indexing Software in a multinode

environment
This chapter contains instructions for installing a multinode indexing deployment and configuring it
to use directed routing. It contains the following topics:
• Installing a multinode configuration, page 40
• Configuring the index agent, page 42
• Configuring directed routing, page 44
• Adding a node, page 49
• Removing a node, page 51
Installing a multinode configuration

Before installing the software for a multinode configuration, review the information in and create an
InstallProfile.xml configuration file suitable for your deployment.
Install
To install a multinode configuration:

1. Log on as the Content Server installation owner to the host machine that will serve as the
administrative node.
The administrative node is identified in the of the InstallProfile.xml file. It is the host identified by
the <configuration-service> element.
2. Copy the InstallProfile.xml configuration file you created to the index server installation program
directory.
3. Run the index server installation program.
• On Windows, double-click fulltextWinSuiteSetup.exe.
• On UNIX and Linux, type
% fulltextoperatingsystemSuiteSetup.bin
and press Enter, where operatingsystem is the operating system on which you are installing.
4. Select index server installation only.
The installation program detects the InstallProfile.xml file and asks you to confirm that you
require multinode installation.
5. Click Yes.
The correct index server processes are installed and configured on the host.
6. After the installation is completed, ensure that the index server starts.
• On Windows, select Yes, restart my computer., then click Next.
— If the computer does not restart automatically, click Start > Shutdown > Restart and restart
the computer manually.
— If the index server does not automatically start, click Start > Programs > Administrative
Tools > Services and start the FAST InStream service.
— If the system restarts, the index server starts automatically as a Windows service.
• On UNIX and Linux, run $DOCUMENTUM/fulltext/jboss4.2.0/IndexServer/bin/startup.sh.
The index server is started.
7. Install and configure an index agent as described in Configuring the index agent, page 42.
8. To configure directed routing, perform the steps described in Configuring directed routing,
page 44 on the administrative host.
9. Repeat Step 1 through Step 6 on each additional node.
10. Confirm that the index server processes are running correctly.
• On the administrative node, type dsadmin listmodules at a command prompt.
The running processes, their version numbers, host name, and port numbers are displayed.
• On each nonadministration node, use the nctrl command to verify that the processes are
running correctly:
$ cd $FASTSEARCH/bin
$ ../setupenv.sh
$ nctrl sysstatus
Each running module name is listed, as well as the process name, process ID, and status.
Install
Configuring the index agent

The index agent configuration program configures the index agent to process documents for a
particular repository and to pass the documents to the correct index server instance for indexing.
By default, the index agent automatically starts when the configuration program completes. You do
not want the index agent to start automatically when installing a multinode implementation, because
there may be additional configuration steps that need to be completed before the index agent starts.
(See Modifying the IndexAgent.xml file to map file stores, page 38 and Configuring directed routing,
page 44.) The procedure below includes steps to prevent the index agent from starting automatically.
To configure the index agent:

1. Log on to the host machine as the Content Server installation owner.
2. Ensure that the index server is running.
3. Navigate to the directory %DOCUMENTUM_SHARED\IndexAgents (on Windows) or
$DOCUMENTUM_SHARED/IndexAgents (on UNIX).
4. Use a text editor to create a file named setup.ini with these two lines as its content:
[INDEX_AGENT]
AUTO_START_INDEX_AGENT=false
5. Start the index agent configuration program using the appropriate command for your platform.
• Windows: IndexAgent_Configuration_Program.exe –config setup.ini
• AIX: IndexAgent_Configuration_Program.aix –config setup.ini
• Solaris: IndexAgent_Configuration_Program.bin –config setup.ini
• HP-UX: IndexAgent_Configuration_Program.hp –config setup.ini
• On Linux, IndexAgent_Configuration_Program.linux –config setup.ini
6. Click Next.
7. On Windows, type in the installation owner’s password, then click Next.
8. Type in the port used to communicate with application server for administration purposes,
then click Next.
The default port, 9200, is for the first index agent on the host. The default port for any other index
agents are 9220 for index agent 2, 9240 for index agent 3, and so on.
9. Select the repository for which the index agent will prepare documents, then click Next.
The drop-down list contains the repositories that project to the connection brokers listed in
the dfc.properties file on the host.
10. Type in the user name and password for the Superuser account that the index agent will use to
connect to the repository.
11. Indicate whether to run the index agent in normal mode or migration mode.
Install
12. Type in the host where the index server for this index agent is running and the base port number
for the index server, then click Next.
A summary dialog box is displayed.
13. Click Next.
14. To exit from the configuration program, click Finish.
15. To complete the installation process:
• If you are mapping the file stores, complete the instructions in Modifying the IndexAgent.xml
file to map file stores, page 38.
• If you are using directed routing, complete the instructions in Configuring directed routing,
page 44.
16. On Windows, to start the index agent manually, start the IndexAgent service to start IndexAgent
instance.
On Linux, to start the index agent manually, run $DOCUMENTUM_SHARED/jboss4.2.0/server/
startIndexAgent1.sh.
Modifying the indexagent.xml file to map file stores
If you have shared or mounted the drives containing the repository’s file stores and installed the
indexing software, the index agent configuration file needs to be manually edited to indicate that the
drives are shared. The changes depend on whether the file system paths to the content are identical
on the Content Server host and index server host.
Using shared or mounted drives improves performance, because the index agent does not need to
copy documents from Content Server to a staging area. See the Documentum System Planning Guide
for more information about sharing file store drives.
To modify the indexagent.xml file and map the file stores:

1. On the index agent host, navigate to C:\Documentum\jboss4.2.0\server\DctmServer_
IndexAgent1\deploy\IndexAgent1.war\WEB-INF\classes (on Windows) or
$DOCUMENTUM_SHARED/jboss4.2.0/server/DctmServer_IndexAgent1/deploy/
IndexAgent1.war/WEB-INF/classes (on UNIX).
2. Open the indexagent.xml file in a text editor.
3. If the paths to the content files are identical on the Content Server host and index server host,
locate the <exporter></exporter> element and change the value of the <all_filestores_local>
element to true:
<all_filestores_local>true</all_filestores_local>
4. If the paths to the content files are different, do not modify the value of <all_filestores_local>, but
instead, create a file store map within the <exporter> element.
For example, if Content Server is on a host called Dandelion where filestore_01 is physically
located in the directory /Dandelion/Documentum/data/repository_name/content_storage_01 and
Install
the index agent and index server on a host from which the drive on the Content Server host is
shared as /mappingtoDandelion/repository_name/content_storage_01, create an alias as follows:
<local_filestore_map>
<local_filestore>
<store_name>filestore_01</store_name>
<local_mount>/mappingtoDandelion/
repository_name
/content_storage_01</local_mount>
</local_filestore>
<!-- and so on for each filestore --!>
</local_filestore_map>
If you are indexing content stored on a NAS device or a Windows 2003 Server host, you may see
the following error message in the dmi_queue_item’s message attribute:
DocumentRetriever :ERROR Retrieval error: Couldn’t open file
<file path/name> ERROR Processor error status:
DataNotAvailable Not read permission
To resolve this error, edit the <local_mount> element or elements in the IndexAgent.xml file that
reference the storage area or areas on the NAS device. Add two backslashes immediately after
the opening <local_mount> element. For example, assume the following references a storage
are on an NAS device:
<local_mount>\\100.2.4.32\share3\c\data_for_example
After editing, it is now:
<local_mount>\\\\100.2.4.32\share3\c\data_for_example
5. Save the indexagent.xml file.

Configuring directed routing

Directed routing is a means of distributing documents to specific index server nodes based on which
Content Server file store contains their content files; see the Documentum System Planning Guide
for a conceptual overview.
There are four major steps involved in configuring directed routing:
1. Configure Content Server to direct documents to specific file stores based on business rules. See
“Allocating content to storage areas” in the Content Server Administration Guide for information
about creating content assignment policies.
2. Create a full-text collection corresponding to each index server node. See Creating full-text
collections, page 45.
3. Update the index agent configuration file (indexagent.xml) to associate particular file stores with
particular full-text collections. See Mapping file stores to collections, page 46.
4. Create a routing configuration file on the index server administrative node to associate each
collection with an index column. See Mapping collections to index columns, page 47.
Install
Figure 2. Mapping file stores to collections to index columns
Creating full-text collections
All documents submitted for indexing are assigned to a collection, which is a logical set of data to
which the index server applies the same indexing rules. In a basic implementation, all documents
are typically assigned to a single default collection that is created automatically as part of the index
server installation process.
With directed routing, you want the index server to treat documents differently depending on which
file stores contain their content files — specifically, you want the index server to route them to
different nodes for inclusion in different columns. To enable the index server to route documents to
different nodes, you need to assign the documents to different collections.
The first step is to create additional collections. You create one collection for each node.
To create a full-text collection:

1. Log on to the administrative node as the Content Server installation owner.
2. Enable access to the FAST InStream administration tool.
• On Windows, start the Fast InStream Web Server service.
• On UNIX, navigate to the directory $DOCUMENTUM/fulltext/IndexServer/bin and enter
these commands:
source setupenv.sh
httpd -f $DOCUMENTUM/fulltext/IndexServer/etc/httpd/httpd.conf
Install
3. Launch the FAST InStream administration tool by entering this URL:

http://localhost:port+3000
In this URL, port is the base port number assigned to the index server during installation. The
default base port is 13000, in which case the URL would be http://localhost:16000.
The FAST InStream administration tool appears.
4. Click the Create Collection link.
5. Enter a name and description for the collection. Note the name for future reference. You will
need it in later steps when you update the indexagent.xml file and the routing.cfg file. The
name is case sensitive.
6. Click Next. The page for selecting a cluster appears. The value webcluster is automatically
selected, since it is the only option.
7. Click Next.
8. Select DFTXML (webcluster) from the Available Pipelines drop-down list, then click the add
selected button that appears to the right of the drop-down list.
9. Click Next.
A message appears informing you that no data sources exist.
10. Click OK.
11. Repeat steps 4 to 10 to create additional collections.
In most cases, you want to create one collection for each index server node.
Mapping file stores to collections
Once you have created the necessary full-text collections, you associate them with file stores in the
indexagent.xml configuration file, which resides on the index agent host machine.
To configure the index agent for directed routing:

1. Shut down the index agent if it is running.
2. On the index agent host, navigate to C:\Documentum\jboss4.2.0\server\DctmServer_
IndexAgent1\deploy\IndexAgent1.war\WEB-INF\classes (on Windows) or
$DOCUMENTUM_SHARED/jboss4.2.0/server/DctmServer_IndexAgent1
/deploy/IndexAgent1.war/WEB-INF/classes (on UNIX).
3. Open the indexagent.xml file in a text editor.
4. Locate the closing tag of the <indexer> element.
Place the cursor just before the </indexer> tag.
5. Insert an element with this format.
<partition_config>
<default_partition>
<collection_name>name_of_default_collection</collection_name>
</default_partition>
Install
<partition>
<storage_name>name_of_file_store</storage_name>
<collection_name>name_of_collection</collection_name>
</partition>
</partition_config>
The name_of_default_collection needs to match the value of the <fds_collection> element that
appears later in the indexagent.xml file. It is the name of the collection created during installation.
Include a <partition> element for each file store whose documents you want to assign to a specific
collection. The name_of_file_store needs to match the file store name from the repository, and
the name_of_collection needs to match the collection name you specified when you created the
collection in Creating full-text collections, page 45.
For example, the <partition_config> element below assigns documents from filestore_01,
filestore_02, filestore_03, and filestore_04 to three collections (repb01, repb02, and repb03).
Documents from other file stores are assigned to repb01, which is designated as the default
partition. Notice also that filestore_03 and filestore_04 are assigned to the same collection.
<partition_config>
<default_partition>
<collection_name>repb01</collection_name>
</default_partition>
<partition>
<storage_name>filestore_01</storage_name>
</partition>
<partition>
</partition>
<partition>
</partition>
<partition>
</partition>
</partition_config>
</indexer>
6. Save and close the indexagent.xml file.

Mapping collections to index columns
When using directed routing, the index agent assigns documents to collections based on which file
store contains their content files. The index server needs to route the documents to different nodes
based on which collection they belong to. To configure the index server to do this, you create a routing
configuration file (routing.cfg) and update the Status Server configuration file (NodeConf.xml) so
that it refers to the routing configuration file.
To configure the index server for directed routing:

1. Log on to the index server administrative node as the Content Server installation owner.
The administrative node is the node on which the Status Server process is running.
Install
2. Shut down the index server if it is running.

3. Navigate to the directory %DOCUMENTUM%\fulltext\IndexServer\etc (on Windows) or
$DOCUMENTUM/fulltext/IndexServer/etc (on UNIX).
4. Open the CSConfig.xml file in a text editor.
The CSConfig.xml file defines the configuration of the index server nodes, including the
definitions of the index columns on each node.
5. Record the partition_id value for each index column.
The file includes a <column> element for each index column. The <column> element identifies
the machine hosting the column and also sets a numerical ID value for the column. Note the
numerical ID, which is the value of the partition_id parameter. You will need it in Step 8.
For example, this <column> element sets the partition_id for host1 to 0:
<column hostname="host1.emc.com" port="15674" mode="NORMAL"
partition_id="0"
ft_mode="0" docapiport="15500">
</column>
6. Close the CSConfig.xml file.

7. Use a text editor to create a file named routing.cfg in the current directory.
8. Enter the collection to column mapping in the following format:
collection_name=partition_id
On each line, collection_name is the name of a collection you created in Creating full-text
collections, page 45, and partition_id is the ID of the index column to which you want to route
documents from this collection. For example:
repb01=0
repb02=1
repb03=2
9. Save the routing.cfg file.

10. Open the NodeConf.xml file in a text editor.
11. Add a parameter to the Status Server configuration entry, providing the name and location of
the routing.cfg file.
Add the text shown in bold below:
<executable>statusserver</executable>
<parameters>>--type=single --db-dir=$FASTSEARCH/data/status
--db-mem=1 -ORBendPointNoListen giop:tcp:host1:$PORT
-ORBendPointNoPublish
giop:tcp::$PORT --collection-routing-file=
$FASTSEARCH/etc/routing.cfg
12. Save the NodeConf.xml file.

13. Restart all index server processes on all nodes.
14. Use the nctrl command on each node to verify that the processes are running correctly:
$ cd $FASTSEARCH/bin
$ . .setupenv.sh
$ nctrl sysstatus
A list of index server processes and their status is displayed.
Install
Adding a node
Use these instructions to add an additional node to an existing multinode configuration.
To add a node to a multinode configuration:

1. Edit a copy of the existing InstallProfile.xml file to include information about the additional node.
See for information about editing this file.
2. Shut down all running index server nodes and the index agent.
3. Log in to the host machine for the new node as the Content Server installation owner.
4. Install the new index server node.
a. Download the software distribution package for the correct platform to a temporary area
on the host machine.
b. Unzip or untar the software distribution package.
c. Copy the InstallProfile.xml configuration file you created at step 1 to the same temporary area.
d. Run the index server installation program, fulltextoperatingsystemSuiteSetup.
e. Select index server installation only. The installation program detects the InstallProfile.xml
file and asks you to confirm that you require multinode installation.
f. Click Yes. The correct index server processes are installed and configured on the host.
5. Log on to the administrative node host as the Content Server installation owner.
6. Edit the CSConfig.xml file on the administration node to add information defining the new node.
a. Navigate to the directory %DOCUMENTUM%\fulltext\IndexServer\etc (on Windows) or
$DOCUMENTUM/fulltext/IndexServer/etc (on UNIX).
b. Open the CSConfig.xml file in a text editor.
c. Within the <modules> element, add a new <module> element for the new host machine. The
<module> element has this format:
<module hostname="host_name" port=
"base_port_number" />
For example:
<module host="hostnew.emc.com" port="15674" />
d. Within the <search_clusters><cluster> element, add a new <column> element for the new
node. The <column> element has this format:
<column host="fully_qualified_host_name"
port="base_port_number" mode="NORMAL"
partition_id="partition_number"
ft_mode="0" docapiport="15500">
The partition_number should be the integer specified as part of the <search-engine-column>
ID in the of the IndexProfile.xml file; if the IndexProfile.xml file includes
<search-engine-column id="col4">, the partition_number for the column is 4. For
example:
Install
<column host="host0.emc.com" port="15674"

mode="NORMAL" partition_id="4" ft_mode="0" docapiport=
"15500>
e. Save and close the CSConfig.xml file.

7. If you are using directed routing, update the routing rules as necessary.
Updating the rules may require changes to routing.cfg, to indexagent.xml, or both. You may also
need to create additional collections. See Configuring directed routing, page 44 for details.
8. If you are using round-robin routing, rebalance the index across the newly expanded set of nodes.
With round-robin routing, each index column should have approximately the same number of
index entries. When you add a node, you need to redistribute the existing entries.
a. Backup the FIXML files on each node to a local temporary directory. The FIXML
files are in the directory %DOCUMENTUM%\data\fulltext\fixml on Windows and
$DOCUMENTUM/data/fulltext/fixml on UNIX.
b. Delete the FIXML files and the index columns on each node.
c. Restart the index server nodes, beginning with the administrative node.
d. Configure each defined full-text collection to use the migration pipeline. To do so, launch the
FAST InStream administration tool by entering this URL:
http://localhost:base_port+3000
The base_port is the index server’s base port number; if the index server is using the default
base port 13000, then the value for the URL is 16000. From the Collection Overview screen,
click the edit button that appears to the right of the collection information row, choose Edit
Pipeline, and select Migration from the Available Pipelines drop-down list. Repeat this
step for each defined collection.
e. Suspend indexing on each node. Enter this command to suspend indexing:
rtsadmin localhost port webcluster
partition_id 0 suspendindexing
The port is the index server’s base port number plus the constant 3099. If the index server
is using the default base port 13000, then the port value is 16099. The first integer after
’webcluster’ is the partition_id of the column hosted on the node.
f. Rebuild the index by running the FIXML feeder on each node. Enter this command:
cobra fixmlfeeder.py -i path_to_temp_dir
The path_to_temp_dir is the path to the directory where you backed up the FIXML files
at step a.
g. When the feeding is complete, use the FAST InStream administration tool to reset all
collections to use the DFTXML (webcluster) pipeline (see step d above).
h. Resume indexing on each node. Enter this command to resume indexing:
rtsadmin localhost port webcluster
partition_id 0 resetindex
9. Restart the index server on each node and restart the index agent.
Install
Removing a node
Use these instructions to remove a node from an existing multinode configuration.
To remove a node from a multinode configuration:

1. Shut down all nodes, and shut down the administration node last.
2. Update the CSConfig.xml file on the administration to remove the node.
3. Update routing.cfg on the administration node to remove the node from the collection.
4. Remove the NameService files from the omniorb log directory $FASTSEARCH/var/log/omniorb/
5. Update the $FASTSEARCH/etc/searchrc-1.xml file on each search node adjoining the node being
removed.
6. Update the following two directories that contain references to the node being removed:
$FAST_SEARCG/var/searchctrl/etc/enginesrc_[portNumber] and engine/data_[portNumber]
7. On Windows, run the uninstaller on the node being removed. On UNIX, manually remove the
$FASTSEARCH installation.
8. Restart the remaining node, first the administration node, then the other nodes.
Install
Chapter 4
Upgrade
This chapter contains the following topics:

• Adding the full-text indexing system after upgrading from a 5.3 repository , page 53
• Upgrading a full-text system on a 5.3 SP2 or SP3 repository without the December 2006 Hotfix,
page 54
• Upgrading from the December 2006 Fulltext Hotfix or 5.3 SP4 or later, page 56
Overview
This chapter provides the instructions for upgrading the full-text indexing components.
If you are using full-text indexing, are upgrading from 5.3 SP4 or later, you do not need to rebuild
your full-text indexes to migrate to version 6.0 or 6.5. If you are using full-text indexing, are
upgrading from 5.3 SP2 or SP3, and have applied the Get Well 4.3.1 hot fix, you do not need to
rebuild your full-text indexes to migrate to Documentum 6. If you are upgrading from 5.3 or 5.3 SP1,
you will need to rebuild your full-text indexes.
In a consolidated deployment, a single index server provides indexing services to multiple
repositories. In any indexing configuration, the indexing software and Content Servers must have the
same version number. Therefore, to upgrade a consolidated deployment, the indexing software and
all repositories must be upgraded simultaneously.
Adding the full-text indexing system after

upgrading from a 5.3 repository
This section contains general guidelines on adding the full-text indexing system after upgrading
from a 5.3 repository.
To add a full-text indexing system after upgrading from a 5.3 repository:

1. Install the full-text indexing software.
2. Configure an index agent in migration mode.
Upgrade
3. Create the full-text index.

4. Use the Index Agent Admin Tool to change the index agent from migration mode to normal mode.
Upgrading a full-text system on a 5.3 SP2 or

SP3 repository without the December 2006
Hotfix
This section contains general guidelines on upgrading from a 5.3 SP2 or 5.3 SP3 release that has not
had the Fulltext Hotfix of December 2006 installed.
If you previously installed the 5.3 SP2 or 5.3 SP3 Fulltext Hotfix or previously upgraded to 5.3 SP4 or
later, use the instructions in Upgrading from the December 2006 Fulltext Hotfix or 5.3 SP4 or later,
page 56. If you are unsure whether the earlier Hotfix release has been installed, check the Content
Server version number:
• On Windows, navigate to C:\Documentum\product\5.3\bin and enter:
documentum.exe -version
• On UNIX, navigate to $DM_HOME/bin and enter:

documentum -version
If you have installed the December 2006 Fulltext Hotfix, the version numbers will be:
• On 5.3 SP2: 5.3.0.229
• On 5.3 SP3: 5.3.0.325
Caution: Do not use this procedure if the deployment is a multinode deployment. Contact
Professional Services (or qualified third-party integrators) for instructions on upgrading a
multinode deployment.
To upgrade the full-text indexing components:

1. Log on as the Content Server installation owner.
Note: On Windows, this means that you must log on as the same user, in the same domain, as the
user who installed Content Server and the full-text indexing software.
2. Stop the index agents and index server.
If you did not stop these processes when you upgraded Content Server, stop them now. Make
sure all index server processes are stopped.
3. Delete all index agent instances.
Use the Index Agent Configuration Program to delete the index agents. See Deleting an index
agent, page 59.
4. Uninstall the Index Agent Configuration Program.
See Deleting the index agent configuration program, page 60.
5. Uninstall the index server while preserving the index itself.
Upgrade
The uninstaller will present a panel that asks if you wish to delete the existing index. The default
is “no”, which preserves the index. Accept the default.
Note: This step asks you to preserve the index so that the FIXML files are not deleted. Removing
the index at this stage removes the FIXML files also. In subsequent steps, you will remove the
actual index entries and regenerate the index using the FIXML files.
6. Manually delete the IndexServer directory.
Note: This step is only necessary if the index server is on a different host than Content Server. If
the index server is on the same host as Content Server, uninstalling the index server will also
remove the IndexServer directory in the following location:
• On Windows: C:\Documentum\fulltext\IndexServer
• On UNIX: $DOCUMENTUM/fulltext/IndexServer
7. Manually delete the following directory:

• On Windows: C:\Documentum\data\fulltext\index
• On UNIX: $DOCUMENTUM/data/fulltext/index
Caution: Do not delete the fixml directory.

8. Install the index server and the index agent configuration program.
Follow the instructions in Installing the index server and the index agent configuration program,
page 30 .
9. Shut down and restart Content Server.
10. Start the index server.
After you start the index server, it will recreate the index using the FIXML files in
C:\Documentum\data\fulltext\index ($DOCUMENTUM/fulltext/IndexServer). To determine
the status of the index server, check the all.log file for the entry: fnet: engine up. When that entry
is present, the server has completed reindexing.
Note: On a moderately powered host and assuming there is FIXML representing approximately 1
million documents to reindex, the process can take up to 10 hours.
The all.log file is in C:\Documentum\fulltext\IndexServer\var\log\all.log (Windows) or
$DOCUMENTUM/fulltext/IndexServer/var/log/all.log (UNIX).
11. After the reindexing is complete configure the index agents.
Use the instructions in the Configuring the index agent, page 36.
Upgrade
Upgrading from the December 2006 Fulltext

Hotfix or 5.3 SP4 or later
Use this procedure if you previously installed the December 2006 Fulltext Hotfix release or 5.3 SP4 or
later. If you are unsure whether the December Hotfix release has been installed, check the Content
Server version number:
• On Windows, navigate to C:\Documentum\product\5.3\bin and enter:
documentum.exe -version
• On UNIX, navigate to $DM_HOME/bin and enter:

documentum -version
If you have installed the December 2006 Fulltext hotfix, the version numbers will be:
• On 5.3 SP2: 5.3.0.229
• On 5.3 SP3: 5.3.0325
Caution: Do not use this procedure if the deployment is a multinode deployment. Contact
Professional Services (or qualified third-party integrators) for instructions on upgrading a
multinode deployment.
To upgrade the full-text indexing components:

1. Log on as the Content Server installation owner.
Note: On Windows, this means that you need to log on as the same user, in the same domain, as
the user who installed Content Server and the full-text indexing software.
2. Stop the index agents and index server.
If you did not stop these processes when you upgraded Content Server, stop them now. Make
sure all index server processes are stopped.
3. Delete all index agent instances.
Use the Index Agent Configuration Program to delete the index agents. Refer to Deleting an
index agent, page 59, for instructions.
4. Uninstall the Index Agent Configuration Program.
Refer to Deleting the index agent configuration program, page 60, for instructions.
5. Uninstall the index server while preserving the index itself.
The uninstaller will present a panel that asks if you want to delete the existing index.
Note: The default is “no”, which preserves the index. Accept the default to avoid having to
rebuild the index.
6. Manually delete the IndexServer directory.
Note: This step is only necessary if the index server is on a different host than Content Server. If
the index server is on the same host as Content Server, uninstalling the index server will also
remove the IndexServer directory in the following location:
• On Windows: C:\Documentum\fulltext\IndexServer
• On UNIX: $DOCUMENTUM/fulltext/IndexServer
Upgrade
7. Install the index server and the index agent configuration program.
Follow the instructions in Installing the index server and the index agent configuration program,
page 30 .
8. Shut down and restart Content Server.
10. Configure the index agents.
Use the instructions in the Configuring the index agent, page 36.
Upgrade
Chapter 5
Uninstall

• Uninstall order, page 59
• Deleting an index agent, page 59
• Deleting the index agent configuration program, page 60
• Deleting an index server, page 61
• Deleting a full-text Index, page 61
.
Uninstall order
Use a particular order to uninstall Content Server, a repository, the index agent, and the index server.
To uninstall an index agent, the repository its servers need to be running. To uninstall an index server,
the repository needs to be shut down. If the index server is on the Content Server host, additional
issues arise because of shared libraries in the software installations.
To uninstall a multinode configuration, stop all running processes on all hosts.
Uninstall the software components in this order:
1. Shut down and uninstall the index agent.
2. Shut down the repository.
3. Shut down and uninstall the index server.
4. Delete the repository, if required.
5. Uninstall the Content Server software, if required.
6. Uninstall the Index Agent Configuration Program, if required.
Deleting an index agent

1. Log on to the index agent host as the user who installed the index agent.
2. Stop the index agent.
Uninstall
Use the instructions in Starting and stopping the index agent, page 79 or in Documentum
Administrator online help, depending on which repository version the index agent is running
against.
3. Start the Index Agent Configuration Program.
• On Windows, click Start > Programs > Documentum > Index Agent Configuration Program.
• On UNIX and Linux, navigate to $DOCUMENTUM_SHARED/IndexAgents and start the
configuration program for your operating system:
— On AIX, IndexAgent_Configuration_Program.aix
— On Solaris, IndexAgent_Configuration_Program.bin
— On HP-UX, IndexAgent_Configuration_Program.hp
— On Linux, IndexAgent_Configuration_Program.linux
4. Click Next.
5. Select Delete index agent and click Next.
6. Read the information and click Next.
The index agent is deleted.
7. To run the configuration program again, check the check box.
8. Click Next.
9. If you checked the check box to run the configuration program again, skip to Step 5; otherwise,
the program exits.
The index agent software and configuration program are still on the host.
Deleting the index agent configuration program

1. Log on to the host as the user who installed the software.
2. If you are on Windows:
a. Click Start > Settings > Add/Remove Programs.
b. Select Documentum Index Agent Configuration Program.
c. Click Change/Remove.
The uninstaller starts.
3. If you are on UNIX or Linux, navigate to $DOCUMENTUM_SHARED/_uninst/IndexAgents
and type uninstall.bin.
The uninstaller starts.
4. Read the information screen and click Next.
An information screen displays the location where the index agent software is installed.
5. Click Next.
The software is uninstalled.
Uninstall
6. Click Finish.
Deleting an index server

1. Ensure that the repository served by the index server is stopped.
Refer to the Content Server Installation Guide for information on stopping repositories.
2. Log on to the host where the index server is installed as the installation owner.
3. Use the instructions in Starting and stopping the index agent, page 79 or in Documentum
Administrator online Help to stop the index server.
4. If you are on Windows:
a. Click Start > Settings > Control Panel > Add/Remove Programs.
b. Select Documentum Index Server.
c. Click Change/Remove.
The uninstaller starts and a welcome dialog box is displayed.
5. If you are on UNIX or Linux, navigate to $DOCUMENTUM/_uninst/IndexServer and type
uninstall.bin.
The uninstaller starts and a welcome dialog box is displayed.
6. Click Next.
An information dialog box is displayed with information about where the index server is installed.
7. Click Next.
The software is uninstalled and a summary dialog box is displayed.
8. Click Finish.
Deleting a full-text Index

To delete a full-text index, run the uninstaller and delete the index agent and index server. A dialog
box provides the option to delete the index as well.
Uninstall
Part 3
Administer
Administer
Chapter 6
Manage a Full-text Index
This chapter contains the information you need to create and verify the accuracy and completeness
of the full-text index. The chapter contains the following topics:
• Managing and administering the index agent and index server, page 65
• Managing the index queue, page 66
• The “State of the Index” job, page 69
• Turning indexing on and off, page 72
• Suspending and resuming indexing, page 73
• Reindexing a repository, page 74
• Troubleshooting indexing timeouts, page 75
• Creating a new index, page 74
• Pointing a repository to a previously created index, page 75
• Troubleshooting indexing timeouts, page 75
If you are installing a high-availability configuration, create and verify the index on both hosts.
Managing and administering the index agent

and index server
The index agent and index server are managed through Documentum Administrator. Depending on
what administration task you wish to perform, you use either the Index Management node or the
resource agent for the index agent and index server.
The index agent JMX URL

The format of the JMX URL used by the index agent resource agent in Documentum Administrator is:
service:jmx:rmi:///jndi/rmi://host:port/indexagent
where host is the machine hosting the index agent and port is the port used to communicate with
the index agent.
Managing the index queue

In 5.3 and later repositories, creating, versioning, or deleting a SysObject or SysObject subtype creates
a queue item indicating that the full-text indexes needs to be updated to account for the changes. The
index agent reads items from the queue and ensures that the required index updates take place.
If the repository’s indexing system runs in a high-availability configuration, with multiple index
agents and index servers, each index agent-server pair supports its own index. Creating, versioning,
or deleting a SysObject or SysObject subtype creates a queue item for each pair and each index
is updated.
Documentum Administrator provides the interface for viewing the current index queue and
managing the items in the index queue.
The Indexing Management > Index Queue page lists index queue items for the current repository.
To sort the queue items, click the Object ID, Object Name, Object Type, Task Status, Acquired by,
or Creation Date links. The Object Name and Object ID columns list the object name and object ID
of the object to be indexed, not the index queue item. If you click the Info icon for a queue item, the
properties of the object to be indexed are displayed, not the queue item’s properties.
If the indexing system is in a high-availability configuration, the name of each index is displayed
at the top of this page and only the queue items for one index at a time are displayed. To change
which index’s queue items are displayed, click the name of index.
To change the number of queue items displayed, select a different number in the Show Items
dropdown list.
By default, the list page displays failed queue items. To filter the queue items by status, choose
the appropriate status on the dropdown list:
• Indexing Failed, which is the default status displayed
If indexing failed, information about the error is displayed in red under the queue item’s name
and other properties.
• All, which displays all current queue items in the repository
• Indexing in Progress, which indicates that the object is being processed by the index agent or
index server
• Awaiting Indexing, which indicates that the index agent has not yet acquired the queue item and
started the indexing process
• Warning, which indicates that the index agent encountered a problem when it attempted to start
the indexing process for the object
If indexing generated a warning, information about the problem is displayed in red under the
queue item’s name and other properties.
Queue items that have failed indexing can be resubmitted individually, or all failed queue items can
be resubmitted with one command.
The procedures in this section cover the following topics:
• Resubmitting individual objects, page 67
• Resubmitting all failed queue items, page 67
• Removing queue items by status, page 67
• Removing queue items, page 68

• Viewing queue items associated with an object, page 68
• Creating a new indexing queue item, page 68
Resubmitting individual objects

You can resubmit individual objects for indexing.
To resubmit individual objects:

1. Connect to the repository as a user who has system administrator or superuser privileges.
2. Click Administration > Indexing Management > Index Queue.
3. If the repository’s indexing system is installed in a high-availability configuration, ensure that the
index queue for the correct index is selected.
4. Choose an object.
5. Click Tools > Resubmit Queue Item.
Resubmitting all failed queue items

You can resubmit for indexing all documents that failed indexing. This operation executes the
MARK_FOR_RETRY administration method. If the indexing system is installed in a high-availability
configuration, all failed queue items for all indexes are resubmitted.
To resubmit all objects that failed indexing:

3. Click Tools>Resubmit all failed queue items.
Removing queue items by status

Use these instructions to remove index queue items by status.
To remove queue items by status:

4. Click Tools > Remove Queue Items by Status.
5. Select the correct status.
Removing queue items

Use these instructions to remove queue items from the indexing queue. If a queue item has already
been acquired by the index agent, you cannot remove it from the indexing queue.
To remove queue items:

4. Select the queue items.
5. Click Tools > Remove queue items.
Viewing queue items associated with an object

From a repository’s cabinets, you can view the index queue items associated with a particular object.
To view the queue items associated with an object:

2. Navigate to the object in the repository’s cabinets.
3. Select the object.
4. Click Tools > View Queue Items.
The queue items are displayed for the selected index queue.
5. If the repository’s indexing system is installed in a high-availability configuration, optionally
click the links for each index queue.
Creating a new indexing queue item

You can create a queue item to submit a particular SysObject for indexing.
To create a queue item to submit a particular object for indexing:

2. Navigate to the object in the repository’s cabinets.
3. Select the object.
4. Click File > New > Create queue item.
The index queue and new queue item are displayed.

5. If the repository’s indexing system is installed in a high-availability configuration, optionally
click the links for each index queue.
Limitations of full-text indexing in high-availability

configurations
The following sections describe current limitations in high-availability full-text indexing
configurations.
Prune and missing destroy events
A prune method is used to delete multiple object versions from a version tree. The method generates
a destroy event for each version that is deleted. Content Server can only generate these destroy events
for the default full-text indexing queue users. The events cannot be generated for the second indexing
queue user in a high-availability configuration.
This means that when a prune method is used, the index entries for the objects destroyed cannot be
removed from the non-default index. However, false positive hits on the index for such objects are
filtered from query results, and are not returned to end users or applications generating the queries.
Save events not generated during load operations
Load operations do not generate save events for objects created during the load; special events are
generated instead. The events cannot be queued to the second full-text user.
The “State of the Index” job

Configuring a repository installs a job called “State of the Index”. This job is implemented as a Java
method. Running the job generates reports that provide the following information:
• Index completeness information, similar to ftintegrity, but also comparing version stamps on
documents
• Status information about the index server, including disk space usage, node statistics, and process
statuses
• Information about the total number of objects with content correctly indexed, the total number
of objects with content that had some failure during indexing, and the total number of objects
with no content
If you have configured a custom filter to ignore specific object types when indexing, you can include
the -usefilter argument to ensure that the job also uses that filter to exclude objects of those types
from the generated reports.
The job does recognize object types chosen for exclusion from indexing through Documentum
Administrator. Objects of those types are not included in the generated report.
Execute the job from Documentum Administrator.
Arguments
The following table lists the arguments for the job.
Table 3. “State of the Index” job arguments
Argument Datatype Description

-batchsize value integer Number of objects to be retrieved from the index in
each batch.
The default value is 1000.

-writetodb_threshold integer Determines how the object comparison is conducted.
value If the total number of returned objects is less than
the value of this argument, or its default value, the
comparison is conducted in memory. If the total
number is greater, the comparison is conducted in
the database.
The default value is 1000000.

Note: Conducting the comparison in memory
consumes memory and in a limited system, may
result in an out-of-memory error.
-usefilter value Boolean Determines whether a filter is used during the object
comparisons. The default value is F.
If a custom filter is used for indexing, set this

argument to T to use the filter when generating the
job results. If this is set to T but the system does not
find a filter , the job ignores this argument, but logs a
message in the job report.
Note: You can create custom filters to stop indexing
of specific object types. See Custom Full-Text Index
Filter Software Environment.
-dumpfailedid value Boolean Indicates whether to generate the docids-failedids.txt
file, which records the object IDs of objects with
content that experienced some failure during content
indexing.
T means to generate the file. F means do not generate

that file.
The default is F.
Argument Datatype Description

-serverbase value Boolean Whether to collect information about objects with
content and objects without content from one
collection in the index or all collections. T means to
collect information from all collections. F means to
collect information from only one collection.
The default is F.
-matchsysobjversion Boolean If set to T, the job performs the version stamp
comparison only for SysObjects.
The default is F.
-matchallversion Boolean If set to T, the job performs the version stamp
comparison for SysObjects and lightweight objects.
The default is F.
In addition, the job is installed with the -queueperson and -windowinterval arguments set. The
-queueperson and -windowinterval arguments are standard arguments for administration jobs and
are explained in the Content Server Administration Guide.
Using the matchsysobjversion and matchallversion

arguments
These two arguments control the version stamp comparison operations conducted by the utility. If
both arguments are false (defaults), the output files generated by the utility do not contain version
information for the monitored objects.
If you set -matchsysobjversion to T, the utility compares version stamps only for SysObjects.
Lightweight objects are recorded in the docids-common-misVstamp.txt file with a value of -1.
If you set -matchallversions to T, the utility compares version stamps for both SysObjects and
lightweight objects.
Job report and generated files

The job generates a job report, FTStateofIndexDoc.txt and four results files. The FTStateofIndexDoc.txt
contains information about the job execution, similar to the job reports generated by other
administration jobs. The four results files are:
• docids-common.txt
This file contains the object IDs and i_vstamp values of all objects in the index and the repository
and having identical i_vstamp values in both places.
• docid-common-misVstamp.txt
This file records all objects in the index and the repository with identical object IDs but
nonmatching i_vstamp values. For each object, it records the object’s object ID, i_vstamp value in
the repository, and i_vstamp value in the index.
• docids-dctm-only.txt
This report contains the object IDs and i_vstamp values of objects in the repository but not in
the index.
• docids-index-only.txt
This report contains the object IDs and i_vstamp values of objects in the index but not in the
repository.
In addition to the report and the four standard result files, the report may generate additional result
files depending on the arguments defined for the job run:
• If -dumpfailedid is set to T, the job generates a file called docids-failedids.txt. This file contains the
object ID of all objects whose content experienced some failure during indexing.
• If -usefilter is set to T, the job generates two additional reports.
— docids-filtered.txt, which records the objects that have been filtered from the repository for
indexing. The objects are identified by object ID and i_vstamp values.
— docids-error-inindex.txt, which records the object ID and i_vstamp values of any objects in
the index that should have been filtered out before indexing.
The report and result files are in %DOCUMENTUM%\dba\log\sessionID\sysadmin
($DOCUMENTUM/dba/log/sessionID/sysadmin).
Turning indexing on and off

The following guidelines apply to turning indexing on and off:
• You can turn off indexing of both properties and content files.
Turning off all indexing, page 72, contains instructions for turning off indexing.
• You can turn off content indexing but leave properties indexed.
Turning off content indexing, page 73, contains instructions for turning off content indexing.
• You cannot turn off only property indexing.
• You can disable the creation of the full-text index by shutting down the index agent and index
server for a repository, but queue items are still generated by the events that trigger indexing.
Turning off all indexing

To turn off all full-text indexing operations in a repository, connect as a superuser, then unregister
the Save, Checkin, Destroy, Readonlysave, and MoveContent notifications for the full-text user
(dm_fulltext_index_user). This disables the generation of queue items for Save, Checkin, Destroy,
Readonlysave, and MoveContent events.
Turning off content indexing

To turn off content indexing, set the a_full_text property of the SysObject to FALSE.
Setting a_full_text to FALSE turns off content indexing, but does not turn off indexing of the object’s
properties.
Suspending and resuming indexing

Suspending indexing stops the index server from processing the FIXML files generated by the
indexing process. However, the generation of FIXML files does not stop. The process proceeds
through the generation of the FIXML files, but is suspended at that point. When indexing is resumed,
the index server resumes processing the FIXML files and updating the index.
An index server must be running when you suspend its indexing functions.
Suspending and resuming an index server in a

single-node configuration
Use the following command to suspend an index server:
rtsadmin adminhost port
webcluster 0 0 suspendindexing
To resume indexing, use the following command:

webcluster 0 0 resetindex
where adminhost is the name of the index server’s administrative host. The port is the index server’s
base port number plus the constant 3099. If the index server is using the default base port, then
the port value is 16099.
Suspending and resuming an index server in a

multinode configuration
In a multinode configuration, you issue one suspend or resume command for each node. For
example, if there are two nodes in the configuration, you use the following commands:
rtsadmin adminlhost port
To resume indexing on those nodes, use the following commands:

where adminhost is the name of the index server’s administrative host. The port is the index server’s
base port number plus the constant 3099. If the index server is using the default base port, then
the port value is 16099.
The first integer after ’webcluster’ represents a column, and each node has one column. The
commands to suspend and reset indexing reference the nodes through the column numbers. Column
numbers start with 0 and increment by one for each columns, or node. The second integer must
always be 0.
To determine how many nodes are running, if needed, you can use the dsadmin tool to find all the
configured nodes. You can use grep (UNIX) or findstr (Windows) to limit the returned values to
Indexer modules. For example:
$ dsadmin listmodules | grep Indexer
[2006-12-11T21:47:17Z]: INFO dsadmin RTS Indexer 3.0.98-Release
mice.mylabs.com 15674
[2006-12-11T21:47:17Z]: INFO dsadmin RTS Indexer 3.0.98-Release
cats.mylabs.com 15674
Count the number of returned rows to determine how many indexing nodes are running. This
example return indicates that two nodes are running.
Reindexing a repository
An existing index may become corrupted because of disk failure or corruption, host failure, or the
unexpected shutdown of the index server. When the index is corrupted, you might want to reindex
the repository.
To reindex a repository use an index agent running in migration mode to resubmit all qualifying
repository objects for indexing. Reindexing the repository rebuilds the existing index by replacing
entries in the index. EMC Documentum recommends running the index server in suspended mode if
you reindex. For information on suspended mode, refer to Index server modes, page 112.
Creating a new index

If you need to delete an existing index and create a new index for a repository, use the following
high-level instructions.
To create a new index:

1. Delete the existing full-text indexing software and index, using the instructions in .
Ensure that you delete the existing index.
2. Reinstall the software, creating an index agent in migration mode, and reindex the repository,
using the instructions in Chapter 6, Manage a Full-text Index.
Pointing a repository to a previously created

index
If an index is created with the index agent in normal mode, the repository, index agent, and index
server are automatically associated with one another.
If an index is created with the index agent in migration mode, a normal mode index agent needs to
be created to update the index. If the repository is version 5.2.5.x, upgrade the repository, then
configure a normal mode index agent. If the repository is version 5.3 or later, configure the normal
mode index agent. Configuring the normal mode index agent associates the indexing software
components, the index, and the repository.
Troubleshooting indexing timeouts

Occasionally, the processing time for a particularly complex file, for example, a complex spreadsheet,
might be long and cause an indexing timeout. Use the procedure in this section if you are
experiencing a timeout while indexing a file.
To modify index agent and index server timeout values:

1. Connect to the host where the indexing software is installed as the installation owner.
2. Navigate to %DOCUMENTUM_SHARED\jboss4.2.
0\domains\DctmDomain\upload\IndexAgent1\
IndexAgent1.war\WEB-INF\classes (on Windows) or
$DOCUMENTUM_SHARED/jboss4.2.0/domains/DctmDomain/upload/IndexAgent1/
IndexAgent1.war/WEB-INF/classes (on UNIX).
3. Open the IndexAgent.xml file in a text editor.
4. In the <indexagent_instance> element, locate the <runaway_thread_timeout> and
<runaway_item_timeout> elements.
5. Increase the default value.
6. In the <exporter> element, locate the <runaway_timeout> element.
8. In the <indexer> element, locate the <runaway_timeout> element.
10. In the <fast_indexer> element, locate the <fds_callback_wait_time> element.
12. Save the IndexAgent.xml file.
13. Stop the index server.
Use the index agent admin tool.
14. Navigate to $DOCUMENTUM/fulltext/IndexServer/etc/processors/ (UNIX or Linux) or
%DOCUMENTUM%\fulltext\IndexServer\etc\processors\
15. Open the Format.xml file in a text editor.

16. Locate the <load module=”processorts.Format” class=”StellentConverter”/> element.
17. Increase the timeout value to 3000:
<param name="Timeout" value="3000" type="int"/>
18. Save the Format.xml file.

19. Navigate to $DOCUMENTUM/fulltext/IndexServer/etc/ (UNIX or Linux) or
%DOCUMENTUM%\fulltext\IndexServer\etc\.
20. Open the NodeConf.xml file in a text editor.
21. In the content distributor section, ensure that the value of the --operation-timeout parameter is
6000.
22. In the Config Server section, ensure that the last entry in the <parameters> element is -p 1200:
<parameters>-P $PORT -s no -M -ORBendPointNoListen
giop:tcp:pleearth:$PORT1
-ORBendPointNoPublish giop:tcp::$PORT1 -p 1200<parameters>
23. Save the NodeConf.xml file.

24. Restart the index server.
Chapter 7
Manage Full-text Indexing Components
This chapter contains information on starting and stopping the index agent and index server,
disabling index agents, viewing index server properties, and viewing or modifying index agent
properties. You need to have system administrator or superuser privileges to perform these tasks.
Topics in this chapter are:
• Administration tools, page 77
• Starting and stopping the full-text indexing system, page 78
• Viewing or modifying index agent properties, page 81
• Viewing index server properties, page 82
• Reviewing the log files, page 82
• Administration operations, page 84
• Large file rejection error, page 90
• Increasing capacity, page 91
Administration tools
To administer a Version 6 or later full-text indexing system, use Documentum Administrator.
Depending on the administration task you want to perform, use either the Index Management node
or the resource agent for the index agent and index server.
Note: Using console mode in RemoteDesktop for the administration of the index server is not
supported.
If the repository has a high-availability indexing configuration running, in which multiple, redundant
index agents and index servers are installed and multiple, redundant indexes are maintained, the
paired index agents and index servers are displayed in Documentum Administrator, listed under
the name of the index. The default names are:
• For the first index, repositoryname_ftindex_01
• For the first index server, FAST Fulltext Engine Configuration
• For the first and all subsequent index agents, hostname_IndexAgent1
• For the next index, repositoryname_ftindex_01_ha
• For the next index server, FAST Fulltext Engine Configuration - 2
The Index Agent Admin Tool

The Index Agent Admin Tool manages the index agent and index server on installations prior to
version 6. This tool is used primarily during migration procedures, when migrating from a version
prior to 5.3 to version 6. However, on 5.3 installations, you can use this tool to map file stores, monitor
the indexing process, and stop or start the index agent and index server on a particular host. On a 5.3
installation, you can also use Documentum Administrator to stop or start index agents and index
servers, to manage the full-text indexing queue, and to view index server logs.
For administration tasks in Version 6, use Documentum Administrator.
The Index Agent Admin Tool is installed when the index agent and index server are installed. The
tool is a JSP page accessible at the URL
http://hostname:portno/IndexAgentN/login.jsp
where hostname is the name of the host where the index agent is running, portno is the port where the
index agent is listening, and N is the number designating a particular index agent instance.
Starting and stopping the full-text indexing

system
Start the software components in this order:
1. Start the Content Servers
3. Start the index agent.
By default, at server startup, Content Server checks whether the index agent associated with the
repository is started. If the index agent is already running, it remains running. If the index agent is
not running and the start_index_agents parameter in the server.ini file is set to TRUE (the default
value), the server starts the index agent. If the application server instance in which the index agent
runs is not running, Content Server cannot start the index agent.
If Content Server is shut down while the index agent is running, the index agent continues to run,
waiting for Content Server.
Shut down the software in this order:
1. Shut down the index agent.
2. Shut down the index server.
3. Shut down the Content Server.
If the software is stopped and started in the correct order, indexing continues after a restart, regardless
of whether the index agent is in migration mode or normal mode. If the index agent is in migration
mode, however, EMC Documentum recommends monitoring the position in the list of object IDs of
the high-water mark queue item to ensure that indexing has not completed. A migration-mode index
agent that is started against a repository in which indexing is completed might reindex the entire
repository. The high-water mark is the queue item with the object name “Fulltext re-index high water
mark” and it traverses the repository in ascending order of object IDs.
Starting and stopping the index agent

Use these instructions to start or stop an index agent. When the index agent is running in normal
mode, use Documentum Administrator to start and stop it. When the index agent is running in
migration mode, you need to use the Index Agent Tool to start or stop it. The Index Agent Admin
Tool, page 78, has instructions on accessing the Index Agent Tool.
Stop the index agent before you stop its associated index server. If you try to stop the index server
first, Documentum Administrator automatically stops the index agent as well. The JSP interface
warns you to stop the index agent before you stop the index server.
If the index server is stopped and you try to start its associated index agent, you are asked whether
you want to also start the index server. If the status of the index server is Not Responding, you
cannot start or stop the index server, but you can start or stop the associated index agent.
You cannot start a disabled index agent, and it does not start automatically when its associated
Content Server is started. You need to enable the index agent first. For information on enabling a
disabled index agent, refer to Enabling and disabling index agents, page 81. If the index agent’s status
is Not Responding, examine the host on which it is installed and ensure that the software is running.
Stopping or starting an index agent does not stop or start the application server process in which the
index agent runs. You need to stop the application server processes manually. However, if you stop
an application server process, you need to restart it before the index agent can be started.
Caution: Stopping the index agent interrupts full-text indexing operations, including updates
to the index and queries to the index. An index agent that is stopped does not pick up index
queue items or process documents for indexing.
To start an index agent:

1. Start the application server:
• On Windows, start the application server using its service. The service name is IndexAgentN,
where N is the number of the index agent.
• On UNIX, start the application server by running the following script:
$DOCUMENTUM_SHARED/jboss4.2.0/domains/DctmDomain/
startIndexAgentN.sh
2. Log on to Documentum Administrator, connecting to the repository as a user who has system
administrator or superuser privileges.
3. Click Administration > Indexing Management > Index Agents and Index Servers.
4. Select the correct index agent.
5. To start the index agent, click Tools > Start.
6. Confirm that you want the index agent started.
The index agent’s status changes to running.
To stop an index agent:

4. To stop the index agent, click Tools > Stop.
5. Confirm that you want the index agent stopped.
The index agent’s status changes to stopped.
6. Stop the application server:
• On Windows, stop the application server using its service. The service name is IndexAgentN,
• On UNIX, stop the application server by running the following script:
$DOCUMENTUM_SHARED/jboss4.2.0/domains/DctmDomain/
stopIndexAgentN.sh
The dm_FTIndexAgentBoot job

The dm_FTIndexAgentBoot job is installed when you install the full-text indexing components. The
job is set to the active state when the index agent is configured. The job periodically checks the state
of the index agent and if the agent is not running, restarts the index agent.
Starting and stopping the index server

Use these instructions to stop a running index server or start an index server that is stopped. Use
Documentum Administrator to start and stop the index server. If you stop an index server, its
associate index agent is also stopped, and you are informed that the index agent will be stopped as
well.
If the index server’s status is Not Responding, attempt to retrieve and examine the log files, using the
instructions in Reviewing the log files, page 82. If you cannot retrieve the log files, examine the host
on which the index server is installed and determine whether it is running.
When you stop the index server, wait a few minutes before attempting to restart it. Stopping or
starting an index agent does not stop or start the application server process in which the index
server runs.
Caution: Stopping the index server interrupts full-text indexing operations, including updates
to the index and queries to the index.
To start or stop an index server using Documentum Administrator:

3. Select the correct index server.

4. To start the index server, click Tools > Start.
5. To stop the index server, click Tools > Stop.
6. Confirm that you want the index server started or stopped.
The index server’s status changes to running or stopped.
Enabling and disabling index agents

You cannot start a disabled index agent, and it does not start automatically when its associated
Content Server is started. You can disable an index agent whether it is running or stopped. To start a
disabled index agent that is not running, you need to enable the index agent first.
You might want to disable an index agent if the host on which the index server is installed has had a
hardware failure or if the index agent itself has failed.
To enable or disable an index agent:

4. Click Tools > Enable or Tools > Disable.
5. Confirm that you want the index agent enabled or disabled.
The index agent’s status changes to the selected state.
6. If you enabled the index agent, restart it.
Viewing or modifying index agent properties

Use these instructions to view the properties of an index agent.
To view or modify index agent properties:

3. Click the Info icon for the index agent.
4. You can modify the Exporter Thread Count and Polling Interval index agent properties; however,
EMC Documentum recommends that you do not modify the default values.
Note: All other properties are read-only.

• Exporter Thread Count
This is the number of concurrent exporter threads run by the index agent. The default
value is 3. If you change the exporter thread count, you need to restart the index agent for
the change to take effect.
• Polling Interval
This is the frequency, in seconds, at which the index agent polls for queue items. The default
value is 60.
5. Click OK to save the changes or Cancel to exit without saving.
Viewing index server properties

To view the properties of an index server, click the Info icon next to the name of the server in
Documentum Administrator (Administration > Indexing Management > Index Agents and Index
Servers).
Note: You cannot modify the properties of an index server.
The following properties are displayed:
• The index server name — Object name of the configuration object
• The name of the host on which the index server is running
• The base port number (default is 13000) used by the index server.
• Collection name — name of an index collection, which typically contains the index data for a
particular repository.
• Description — description of the collection, including the repository for which the collection
contains the index data
• Cluster — name of the search cluster (default is “webcluster”) to which the collection belongs
• Pipeline — document processing pipeline (default is DFTXML) for a particular collection
For 5.3 repositories with the indexing software installed, only one row is displayed because a 5.3
index server manages only one collection. In 5.3 SP1 and later repositories, an index server can
manage multiple collections.
Reviewing the log files

The index agent, index server, and the application server instance for each index agent generate log
files.
Index agent log files

You can find the index agent log file in the following location:
• For Windows platforms:
C:\Documentum\jboss4.2.0\domains\DctmDomain\servers\
DctmServer_IndexAgentN_hostname\logs\IndexAgentN.log
• For UNIX platforms:
$DOCUMENTUM_SHARED/jboss4.2.0/domains/DctmDomain/servers/
DctmServer_IndexAgentN_hostname/logs/IndexAgentN.log
where hostname is the name of the host on which the agent is running and N is number of the
particular index agent. For example, if the index agent is the first index agent on the host named
LondonOne, which is a Windows host, the log file is:
DctmServer_IndexAgent1_LondonOne\logs\IndexAgent1.log
The numbers disambiguating the index agents are used when there are multiple index agents running
against a single index server. For example, in a consolidated indexing deployment, in which more
than one repository is indexed by a single index server, you might have three index agents running,
with each corresponding to a particular repository.
Index server log files

To view the logs files for an index server, use Documentum Administrator.
To view the log files of an index server:

3. Select an index server.
4. Click View > Get Index Server Logs.
A zip file containing the logs is produced.
5. Save the zip file to the local drive.
6. Unzip the compressed file and view the logs.
Application server log files

You can find the application server log file in the following location:
• For Windows platforms:
DctmServer_IndexAgentN_hostname
\logs\DctmServer_IndexAgentN_hostname.log
• For UNIX platforms:
$DOCUMENTUM_SHARED/jboss4.2.0/domains/DctmDomain/servers/
DctmServer_IndexAgentN_hostname
/logs/DctmServer_IndexAgentN_hostname.log
where hostname is the name of the host on which the agent is running and N is number of the index
agent running in that application server instance. For example, if the index agent is the first index
agent on the host named LondonOne, which is a Windows host, the associated application server
log file is:
DctmServer_IndexAgent1_LondonOne\
logs\DctmServer_IndexAgent1_LondonOne.log
Administration operations
Use the instructions in this section to perform configuration and administrative tasks.
Note: Using console mode in Remote Desktop for the administration of the index server is not
supported.
Enabling thesaurus searching

Thesaurus searching allows users to define which words are returned by a particular search.
A synonym file contains the definitions. Typically, entries in the file contain words with similar
meanings (synonyms). For example, you can define “auto” and “automobile”as synonym entries
that need to be returned when the word “car” is the search term. However, an entry can contain
unrelated words or create user-defined associations.
A synonym file may also containing spelling variations and abbreviations. For example, “favour”is
the British spelling of “favor” and “fvr” is an abbreviation of “favor”.
Thesaurus searching is supported for only one language per index server installation. All languages
listed in Appendix E, Supported Languages, are supported for thesaurus searching. The default
language for thesaurus searching is English.
This section contains instructions for creating a synonym file and importing its contents to create
a synonym dictionary. Once the synonym file is imported and the synonym dictionary is created,
thesaurus searching is enabled. User searches return not only the searched-for word, but also any
synonyms defined in the synonym file.
If you upgraded from an EMC Documentum version prior to 5.3, you also can use an existing Verity
thesaurus file to create the synonym dictionary.
You can enable thesaurus searching at any time. You can also add entries to an existing synonym
dictionary.
Creating the synonym file
The synonym file defines which words you want to be returned by a particular search. Create a text
file with synonym entries in the following format:
term=[[spelling variations],
[abbreviations],[[synonyms][]]]
The file may have any name. Save the file in the UTF-8 code page.
The brackets in each entry are literal characters that separate spelling variations, abbreviations, and
synonyms. The empty brackets at the end of the synonym entry are required.
The following examples contain only synonyms:
car=[[],[],[[automobiles,autos],[]]]
cars=[[],[],[[automobiles,autos],[]]]
auto=[[],[],[[automobiles,autos],[]]]
autos=[[],[],[[automobiles,autos],[]]]
The following example contains a spelling variation, an abbreviation, and synonyms:

favor=[[favour],[fvr],[[prefer,privilege],[]]]
No limitations exist on the length of an entry or the number of entries in the synonym file.
Importing the synonym file
You can import a synonym file created as described in Creating the synonym file, page 85 or you can
import an existing Verity thesaurus file.
To import the synonym file, connect to the index server host as the index server installation owner
and run the ImportDictionary.py script. The script is executed by Cobra, which is included in the
index server installation. The following conditions need to be met to run ImportDictionary.py:
• The FASTSEARCH environment variable must be set ($FASTSEARCH on UNIX,
%FASTSEARCH% on Windows).
• The script must be run from the $FASTSEARCH/bin directory (UNIX) or %FASTSEARCH%\bin
folder (Windows).
• The synonym file needs to be found in the $FASTSEARCH/bin directory (UNIX) or
%FASTSEARCH%\bin folder (Windows).
The syntax is
cobra ImportDictionary.py [-I filename -M
verity|fast -L language -T transaction_type]
or
cobra ImportDictionary.py [--importfile=filename
--mode=verity|fast --language=
language --transactiontype=transaction_type]
The arguments are described in Table 4, page 86.
Table 4. Syntax of ImportDictionary.py script
Argument Description
-I filenameor --importfile The name of the synonym file. May be any arbitrary name.
filename
-M verity|fast or Whether an existing Verity thesaurus file or a new index server
--modeverity|fast synonym file is being imported. Allowable values are verity and fast. If
the argument is not included, the default value is fast.
-L or --language The language of the synonym file. Allowable values are the names
of the languages listed in Appendix E, Supported Languages. If the
argument is not included, the default value is English.
-T transaction_type The allowable values are:
or --transactiontype • createnew, which instructs the index server to create a new synonym
transaction_type dictionary using the terms in the file filename.
• insertentries, which instructs the index server to add entries from

the synonym file filename to an existing synonym dictionary.
• deletedictionary, which instructs the index server to delete

the existing dictionary. Do not provide a file name if you run
ImportDictionary.py to delete an existing synonym dictionary. If
you do not provide an argument, the English dictionary is deleted
by default.
For example, to import a new file called MySynonyms.txt:
cobra ImportDictionary.py -I MySynonyms.txt -M fast -L English -T
createnew
To use an existing Verity thesaurus file called VeritySynonyms:

cobra ImportDictionary.py -I VeritySynonyms -M verity -L English -T
createnew
To add new synonyms from a file MoreSynonyms.txt to an existing synonym dictionary:

cobra ImportDictionary.py -I MoreSynonyms.txt -M fast -L English -T
insertentries
To delete an existing synonym dictionary:

cobra ImportDictionary.py -T deletedictionary
Logging
Running the ImportDictionary.py script generates a log file in $FASTSEARCH/bin (UNIX) or

%FASTSEARCH%\bin (Windows). The log file is called error.log. It is generated each time the
ImportDictionary.py script runs, regardless of whether errors are reported. The information in the
log file is also displayed on your console or monitor while the script is running. The log file contains
information on:
• Which options were used to run the script
• The name of the synonym file
• Whether required files are present in the index server installation

• The actions performed by the script
• When any errors occurred
• How to correct the errors
• The name of the synonym dictionary that is created
• Which index server processes create the synonym dictionary
Log sample
The following is a sample log:
Information : Logging started

Services [’storageservice’, ’j2ee’] will be started
IMPORTANT!!!:The current File C:\Documentum\fulltext\IndexServer
\etc\NodeConf.xml
is being modified as C:\Documentum\fulltext\IndexServer\etc\
NodeConf.xml.origRestore
it back before rerunning the script if script fails Launching
File Transformer
C:\Documentum\fulltext\IndexServer\bin\cobra with args [’convert.py’,
’veritysyn.txt’, ’FASTFile’, ’en’]
##########################################################
Information : Launching application C:\Documentum\fulltext\

IndexServer\bin\cobra
The argument list is [’cobra’, ’convert.py’, ’veritysyn.txt’,
’FASTFile’, ’en’]
Successfully generated the transformed file FASTFile
Copying NodeConf.xml.mod as C:\Documentum\fulltext\IndexServer
\etc\NodeConf.xml
to start services
##########################################################

IndexServer\bin\nctrl
The argument list is [’nctrl’, ’reloadcfg’]
##########################################################

The argument list is [’nctrl’, ’start’, ’storageservice’, ’j2ee’]
##########################################################

The argument list is [’nctrl’, ’sysstatus’]
##########################################################

The dictionary app is C:\Documentum\fulltext\IndexServer\bin\dictman
The documentum_synonym_en dictionary will be generated
The DictmanCmdFile command file will be used by dictman
The Dictionary documentum_synonym_en exists
The importFile FASTFile
##########################################################
Starting Dictionary documentum_synonym_en import
Creating Dictman command file DictmanCmdFile
##########################################################
Information : File DictmanCmdFile created

Writing strings
open /documentum_synonym_en
thesimport /documentum_synonym_en FASTFile
close /documentum_synonym_en
compile /documentum_synonym_en
quit
Generated DictmanCmdFile command file for Dictman
##########################################################

IndexServer\bin\dictman
The argument list is [’dictman’, ’-u’, ’root’, ’-f’, ’DictmanCmdFile’]
imported entries successfully into Dictionary documentum_synonym_en
Stopping [’qrserver’, ’search-1’] services
##########################################################

The argument list is [’nctrl’, ’start’, ’qrserver’, ’search-1’]
##########################################################

##########################################################

The argument list is [’nctrl’, ’stop’, ’qrserver’, ’search-1’]
##########################################################

The argument list is [’nctrl’, ’stop’, ’qrserver’, ’search-1’]
Error : Application C:\Documentum\fulltext\IndexServer\bin\nctrl
returned
with error code -5
Error : Launch failed for application C:\Documentum\fulltext\
Error : Application C:\Documentum\fulltext\IndexServer\bin\
nctrl returned
with error code -5
Obtaining a list of indexable formats

Use the following DQL query to obtain a list of content formats that are marked as indexable:
SELECT "name", "description" FROM "dm_format"
WHERE "can_index"=true
Tracing full-text query operations

Use the MODIFY_TRACE administration method from Documentum Administrator to turn on
tracing for full-text querying operations. Two tracing levels are available: None, in which tracing is
turned off, and All, in which all Content Server and full-text messages resulting from queries are
logged. The fttrace_repository. log file contains messages generated by turning tracing on using
MODIFY_TRACE.
When tracing is turned on, FTDQL queries are logged with the following format:
FTDQL Execution. ft_convertible = T|F, converted_ft_query =
%s, ft_search_type = %s
When ft_convertible is TRUE, converted_ft_query contains the converted full-text equivalent of the
WHERE clause. If a where clause was not specified in the query’s select statement, converted_ft_query
is empty. The ft_search_type indicates how the query was executed against the index.
When tracing is turned on and non-FTDQL queries are run, the SQL statements created for the
DQL queries are logged to the full-text log file.
Enabling tracing for the index agent

Use these instructions to enable tracing for the index agent.
To enable tracing for the index agent:

1. On the index agent host, navigate to the $DOCUMENTUM_SHARED/config directory
(%DOCUMENTUM%\config on Windows).
2. Open the $DOCUMENTUM_SHARED/jboss4.2.0/domains/DctmDomain/servers/DctmServer_
IndexAgent<N>_
<hostname>/log, where <N> is the number of the index agent for which you are enabling tracing
and <hostname> is the name of the host server.
3. Locate the following text:
log4j.category.com.documentum=INFO
4. Replace INFO with DEBUG:

log4j.category.com.documentum=DEBUG
5. Save the file.
If a node fails in a high-availability configuration

In a high-availability configuration, one index is defined as the default index and the other index
is defined as the standby index. The value of the is_standby attribute of the fulltext index objects
determines which is the default index. If is_standby is set to FALSE, the index represented by the
fulltext object is the default index. If is_standby is set to TRUE, the index is the standby index.
If a node fails, indexing operations continue for the remaining index. If the remaining index is the
default index, querying operations continue. If the remaining index is the standby index, querying
operations stop. Use Documentum Administrator to change the value of the is_standby attribute of
the two fulltext index objects:
• Change the default index to the standby index by setting the value to TRUE
• Change the standby index to the default index by setting the value to FALSE
Restart any running Content Servers after setting the values. Queries are then directed to the index
that is active.
Cleaning up old log files

Periodically, the indexing system automatically archives the log files for the
index agent and index server. The archived files are stored as a zip file in the
C:\Documentum\fulltext\IndexServer\var\log\archive (Windows) or as a tar file
$DOCUMENTUM/fulltext/IndexServer/var/log/archive (UNIX) directory. These archived files are
not removed automatically. They remain in the archive directory until you remove them manually.
Large file rejection error

The full-text engine is configured to reject for indexing any FIXML file larger than 10MB. The rejection
is logged in the dmi_queue_item and in the index agent log file. The error message is:
DOCUMENT_ERROR Module :FIXMLFilter - Error: Process(
objectID) failure: docproc.ProcessorStatus.NotPassing’
where objectID is the object ID of the document containing the rejected content.
Increasing capacity
Use the information in this section if you require increased indexing or querying capacity.
Increasing indexing capacity

If the primary requirement is to increase indexing capacity—the ability to index a greater number of
documents in a particular time period—several strategies are available:
• Adding index server nodes (in a multinode configuration) increases indexing capacity.
In multinode configurations, documents are routed sequentially to different nodes, so that a
particular node handles a fraction of the indexing. In a four-node configuration, each node will
index about one-fourth of the data.
• Add docprocessor (procserver) processes on each host.
This increases the speed with which FIXML is processed.
• Adding index agents.
Increasing the number of exporter threads in the index

agent
The number of index agent exporter threads is configurable. Increasing the number of threads
increases the speed at which the index agent acquires queue items. The default value is 3. EMC
Documentum recommends configuring the value to 12 or fewer threads.
Chapter 8
Back up and Restore Full-text Indexes

• Perform a cold backup, page 94
• Restore an index at a particular point in time, page 96
Overview
An index installation backup consists of these required directories and files:
• FIXML files — Represent the content and properties of repository objects. These files are used by
the index server to create the binary index files.
• Binary index files — Represent the final index against which queries are executed.
• Configuration files, including custom files — Contain configuration specifications for a
specific index server installation. For example, index agent and multinode installation profile
configuration files. Some example of options that system-wide configuration files set are:
grammatical normalization, specific rendition formats to index, processing of batched returns for
queries that are not FTDQL, a thesaurus to implement thesaurus searching.
• Status server data files — Contain state and routing information for all documents in the indexing
system.
You can create only a cold, full backup of an index because all of an index’s files are changed
periodically over the course of a day; that is, the static copy of an index is completely replaced by its
dynamic copy. The dynamic copy is constantly being updated (as a result of additions or deletions of
content and objects in the associated repository); whereas the static copy (against which queries run)
remains unchanged until the dynamic copy replaces it.
In order to perform a cold backup, the entire indexing system is shut down. During the backup, no
content is indexed nor are query results returned during the backup. If you cannot perform a cold
backup because your backup window (that is, the time that is available for the indexing system to
be backed up) is smaller than the time is takes for the backup, then consider a high availability
configuration.
For more information about full-text indexing high availability configurations, see the Documentum
System Planning Guide.
Index backups require approximately three times more storage than the actual index size, because
of the static and dynamic copies of the index.
When your full-text index installation fails, you recover it by restoring it to a previously consistent
state (that is, a particular point in time) from your backups. Restoring to a particular point in time
is also known as a “point-in-time recovery”. If you have partitioned your full-text index storage
areas or have a multinode configuration, then you may only need to recover those storage areas or
nodes that have failed.
If the associated Content Server has also failed, then you will need to coordinate your full-text index
recovery with the Content Server recovery.
Configure the system for backup and restore

Follow these best practices to configure the indexing system for optimal backup and restore
operations:
• In a disaster recovery configuration, perform the backup at the remote replication site—not the
primary site—because you will be assured of an index backup that is consistent with the Content
Server backup.
• If you use SRDF/A (Symmetrix Remote Data Facility/Asynchronous), you will have to wait for
the indexing server to complete indexing and for the index to be replicated to the remote site.
Before backing up the indexing installation at the remote site, you must wait at least two times
as long as it takes for the content to replicate to the remote site (which is sometimes called
“replication data latency”).
Perform a cold backup

To perform a cold backup:
• Shut down all administrator, index, and search processes
• Back up each index and FIXML directory and contents
• Back up the status server data directory and contents
1. Stop index agents.

2. Validate the processes running on each node.
For example, execute this command:
nctrl sysstatus
which generates a list of processes running. For example:
Status for node : host1
FASTSEARCH: /h1/dmadmin/dctm/fulltext/IndexServer
Disk status: 11% used - 440.51 GB free - 51.91 GB used - 492.42 GB total
Module Name Process Name PID Status

-------------------------------------------------------
Web Server httpd Running
J2EE j2ee 2411 Running

Name Service nameservice 1576 Running
Document Processor procserver_2 2316 Running
Status Server statusserver 2072 Running
Content Distributor contentdistributor 1756 Running
Log Server logserver 1068 Running
RTS Top Dispatcher topfdispatch 1960 Running
Node Controller nctrl 1548 Running
RTS Indexer indexer 1812 Running
QRServer qrserver 2028 Running
RTS Search search-1 1880 Running
Storage Service storageservice User suspended
Config Server configserver 1696 Running
3. Stop administrator and all indexing and search processes on each node. For example:
• On Windows:
net stop FastDSService
• On Unix and Linux:
cd $DOCUMENTUM/fulltext/IndexServer/bin
./setupenv.sh
./shutdown.sh
4. Back up FIXML directories on all nodes (see the trsrearchrc.xml file).

Typically, the directory is /fast/data/fulltext/fixml
5. Back up binary index directories on all nodes (see the trsrearchrc.xml file).
Typically, the directory is /fast/data/fulltext/index
6. Back up the status server files in the $DOCUMENTUM/fulltext/IndexServer/data/status
directory.
The status server stores state and routing information, such as what indexer column holds a
particular document, for all documents in the system. This information is persisted under
$FASTSEARCH/data/status. If this directory structure is backed up, the status server can be
easily restored to its previous state.
7. Back up the configuration files in the $DOCUMENTUM/fulltext/IndexServer/etc directory
on all nodes.
This directory should include these files:
• CSConfig.xml
• routing.cfg
• NodeState.xml
• NodeConf.xml
• IndexAgent.xml (backup of this file is optional.)
8. Back up the CUSTOM files in these directories on each node:
• $DOCUMENTUM/fulltext/IndexServer/var
• $DOCUMENTUM/fulltext/IndexServer/resources/dictionaries
9. Restart administrator and all indexing and search processes on each node. For example:
• On Windows:
net start FastDSService

./setupenv.sh
./startup.sh
Restore an index at a particular point in time

If you have partitioned your full-text index storage areas or have a multinode configuration, then only
use this procedure for nodes that have failed and/or the nodes that are associated with the storage
areas that have failed. You will have to re-install the full-text indexing software on a new machine for
any node that has experienced a hardware failure. You will have to establish a new storage area for
any one that has experienced a hardware failure.
The
1. Stop all index agents (if they were not already stopped) on each failed node.
2. Stop administrator and all indexing and search processes (if they were not already stopped)
on each failed node. For example:
• On Windows:
net stop FastDSService

./setupenv.sh
./shutdown.sh
3. Determine your point-in-time recovery.

4. Recover the configuration files in $DOCUMENTUM/fulltext/IndexServer/etc directory on
the each node, if necessary.
5. Recover CUSTOM files in $DOCUMENTUM/fulltext/IndexServer/var directory
and/or $DOCUMENTUM/fulltext/IndexServer/resources/dictionaries directory, if
necessary.
6. Recover FIXML and/or binary indexes to directories on all nodes depending on your point-in-time
recovery decisions. (See rtsrearchrc.xml file for locations)
7. Restart the admin and all indexing and search processes on each node as follows:
• On Windows, execute this command:
net start FASTDSService
• On Unix and Linux, execute these commands:

./setupenv.sh
./startup.sh
8. Validate the processes running on each node by executing this command.

nctrl sysstatus
The status of each process will be displayed. For example:

Status for node : host1
FASTSEARCH: /h1/dmadmin/dctm/fulltext/IndexServer
Disk status: 11% used - 440.51 GB free - 51.91 GB used - 492.42 GB total
Module Name Process Name PID Status

-------------------------------------------------------
Web Server httpd Running
J2EE j2ee 2411 Running
Name Service nameservice 1576 Running
Status Server statusserver 2072 Running
Content Distributor contentdistributor 1756 Running
Log Server logserver 1068 Running
RTS Top Dispatcher topfdispatch 1960 Running
Node Controller nctrl 1548 Running
RTS Indexer indexer 1812 Running
QRServer qrserver 2028 Running
RTS Search search-1 1880 Running
Storage Service storageservice User suspended
Config Server configserver 1696 Running
9. Run State of Index job to create a report on non-indexed objects.

10. Run dm_FTCreateEvents job or manually recreate any full-text events desired.
11. Verify that all documents are restored to the node by using Matching Engines and Collections
in the FAST administration page.
12. Run the ftintegrity tool to check the index.
a. Run the ftintegrity utility.
See Chapter 9, Verify Full-text Index Completeness and Accuracy.
b. Resubmit any documents that were not indexed.
See Resubmitting objects to the index agent, page 105.
Note: You can also query to find all objects created between the last backup and failure and
resubmit those objects for indexing.
Chapter 9
Verify Full-text Index Completeness
and Accuracy

• Modifying the parameter file, page 100
• Running the index verification tool, page 103
• Resubmitting objects to the index agent, page 105.
Overview
Use the ftintegrity utility to verify the completeness or accuracy or both of an index. Use this tool after
completing a recovery of an index, an index migration, or adding an index to an existing repository.
When run in completeness mode, the utility generates three reports:
• res-comp-common.txt
This file contains the object IDs of all documents that are found in both the index and in the
repository.
• res-comp-dctmonly.txt
This file contains the object IDs of all indexable documents that are found in the repository but
not in the index.
Note: The ftintegrity tool is not aware of object types that you have chosen not to index, through
either a custom filter or the specification of selective indexing in Documentum Administrator. If
you have chosen not to index one or more object types, the object IDs of instances of those types
will still appear in this file.
• res-comp-fastonly.txt
This file contains the object IDs of documents found in the index, but not in the repository.
When running in accuracy mode, you can check a subset of documents or all documents in the index.
For each document the utility performs the following operations:
1. Gets metadata from the repository
2. Chooses a random metadata item and generates a random test for that item.
Verify Full-text Index Completeness and Accuracy
For example, it might generate the query: subject like “%foo”, r_content_szie>4000
3. Runs a test program named “iftdql’ to find that document.
The utility is installed when the index agent is configured. Each index agent instance has an
associated parameter file, called ftintegrity.params.txt. This file identifies the repository on which the
utility operates. It also accepts parameters, which you must complete before running the utility, that
identify the user name and password used to connect to that repository. The utility is found in:
• On Windows
Drive:\Program Files\Documentum\IndexAgents\ftintegrity
• On UNIX and Linux

$DOCUMENTUM_SHARED/IndexAgents/ftintegrity
The utility is run from the command line. Before running the utility, you must modify its associated
parameter file to add the user name and password. For instructions on modifying the file, refer to
Modifying the parameter file, page 100. For instructions on running the utility, refer to Running
the index verification tool, page 103.
After you run the utility, you may wish to resubmit documents for indexing if the utility reports
missing documents. For resubmission instructions, refer to Resubmitting objects to the index agent,
page 105.
Modifying the parameter file

Before running the index verification tool, modify the parameter file to include login information for
the repository for which you created the indexes.
Caution: The instruction below save a Superuser name and password to the file system in a
plain text parameter file. For security reasons, you may wish to remove that information from
the file after running the ftintegrity tool. It is recommended that you save the parameter file in a
location accessible only to the repository Superuser and installation owner.
To modify the parameter file:

1. Navigate to the parameter file location.
• On Windows:
C:\Documentum\bea9.2\domains\DctmDomain\upload\
IndexAgentN\IndexAgentN.war
• On UNIX and Linux:

$DOCUMENTUM_SHARED/bea9.2/domains/DctmDomain/
upload/IndexAgentN/IndexAgentN.war

2. Open the ftintegrity.params.txt file in a text editor.
The first line is
-D repositoryname
where repositoryname is the repository for which you created a new index.
3. Add the following two lines immediately after the first line
-U username
-P password
where username is the user name of the Superuser whose account was used to install the index
agent and password is the Superuser’s password.
4. Save the ftintegrity.params.txt file to %DOCUMENTUM%\fulltext\IndexServer\bin (Windows)
or $DOCUMENTUM/fulltext/IndexServer/bin (UNIX).
Additional parameters
Table 5, page 101 lists the full set of parameters it is possible to provide on the command line or
in the parameter file for ftintegrity.
Table 5. ftintegrity parameters
Parameter Description
-h Prints list of parameters with short descriptions
-i filename Reads the command line parameters from the specified file. In the
file, each parameter must be on a separate line.
-m [b|a|c] Set the mode for the operation. Values are:
• b, meaning perform both accuracy and completeness checks
• a, meaning perform accuracy checks only
• c, meaning perform completeness checks only

-D repository Identifies the repository against which to compare the index
This is a required parameter.

-U user Identifies the user as whom to connect to the repository

-P password Password for the specified user

-b port Index server base port
This may be specified in any mode.
If the default value was accepted when installing the index server,
this value is 13000.
-f host The fully qualified domain name of the host on which the index
server resides
-c collection Identifies a collection to be checked. If this is not specified, all
collections in the index are checked.

-l directory Identifies the working directory. All relative paths should be relative
to this directory.

-F path Identifies the file from which to read object IDs.
This is used only in completeness mode.

-T path Identifies the file to which to save object IDs.
This is used only in completeness mode.

-d number Specifies the number of random documents to test. Setting this to
0 means to test all documents. However, if the number of objects
in the repository is large, it is recommended that you set this to a
non-zero value. Exhaustive accuracy queries cause the utility to run
very slowly.
This is used only in accuracy mode.

-q number Specifies the number of queries to generate for each document.

-s path Identifies the file containing selected document object IDs for which
to test

-n Directs the utility to test only for the document ID and version
number. No queries are generated.

-X path Specifies the path to the query plug-in mapping file

-I path Specifies the command line for the DumpIDs program. The program
records r_object_id values in a file used by ftintegrity. This argument
is set automatically.

-A path Specifies the command line for the DumpAttrs program. The
program records property values in a file used by ftintegrity. This
argument is set automatically.
-Q path Specifies the command line for the iftdql program. This program is
a small standalone tool that allows you to run Content Server-like
queries without the Content Serer to test full-text.
For example:
C:\Program
Files\Documentum\IndexAgents\ftintegrity\iftdql.exe

-L path Identifies the location of the query plug-in object

-t path Specifies the location of the query conversion trace file

-w number Defines the number of queries to wait for before logging progress

-r number Specifies the number of seconds to wait before retrying a failed query
Running the index verification tool

Use these instructions for running the index verification tool. If you are on UNIX or Linux, the tool
can be started by running the run_ftintegrity.sh script, which sets the required environment variables.
The run_ftintegrity.sh script is located in $DOCUMENTUM/fulltext/IndexServer/bin.
Note: Regardless of whether you use the script or command line, you must pass the same arguments.
To run the index verification tool:

1. Navigate to %DOCUMENTUM%\fulltext\IndexServer\bin (Windows) or $DOCUMENTUM/
fulltext/IndexServer/bin (UNIX).
2. To verify both completeness and accuracy, open a command prompt and type the following
command and press Enter:
cobra ftintegrity.py -i ftintegrity.params.txt -m b
will appear in the generated res-comp-dctmonly.txt file.
3. To verify completeness only, open a command prompt and type the following command and
press Enter:
cobra ftintegrity.py -i ftintegrity.params.txt -m c
will appear in the generated res-comp-dctmonly.txt file.
To verify accuracy only and query all indexed objects, open a command prompt and type the
following command and press Enter:
cobra ftintegrity.py -i ftintegrity.params.txt -m a
4. To verify accuracy only and query a subset of indexed objects, open a command prompt and
type the following command and press Enter:
cobra ftintegrity.py -i ftintegrity.params.txt -m a -d integer
where integer is the number of objects in the subset.

5. Verify that any accuracy verification failures are acceptable.
See Accuracy verification acceptable failures, page 105.
If the integrity tool runs successfully, the screen output is similar to this sample:
E:\Documentum\fulltext\IndexServer\bin>cobra ftintegrity.py -i
ftintegrity.params.txt -m b
Wed Mar 23 13:33:38 2005 [PROGRESS]: Completeness Test Started
Wed Mar 23 13:33:47 2005 [PROGRESS]: Fetched 10000 docids from Documentum
Wed Mar 23 13:34:00 2005 [PROGRESS]: Fetched all 31261 docids from Documentum
Wed Mar 23 13:34:00 2005 [INFO ]: Get Documentum IDs: 21.870
Wed Mar 23 13:34:00 2005 [DEBUG ]: Got 31261 docids from Dctm
Wed Mar 23 13:34:00 2005 [INFO ]: Get Number of Documents in
Collection: 0.241 s
Wed Mar 23 13:34:00 2005 [INFO ]: Number of documents in collection
documentum: 502
Wed Mar 23 13:34:01 2005 [PROGRESS]: Fetched all 502 docids from index server
Wed Mar 23 13:34:01 2005 [INFO ]: Get index server IDs: 0.771 s
Wed Mar 23 13:34:01 2005 [DEBUG ]: Got 502 docids from index server
Wed Mar 23 13:34:01 2005 [INFO ]: Transform index server ID file: 0.005 s
Wed Mar 23 13:34:01 2005 [INFO ]: Transform Documentum ID file: 0.270 s
Wed Mar 23 13:34:01 2005 [INFO ]: Compare IDs: 0.271 s
Wed Mar 23 13:34:01 2005 [PROGRESS]: Completeness Test Finished
Wed Mar 23 13:34:01 2005 [PROGRESS]: Accuracy Test Started
Wed Mar 23 13:34:01 2005 [DEBUG ]: Getting Dctm Ids from res-comp-common.txt
Wed Mar 23 13:34:01 2005 [PROGRESS]: Fetched all 502 docids from Documentum
Wed Mar 23 13:34:01 2005 [PROGRESS]: Starting test of 0 documents
Wed Mar 23 13:34:01 2005 [PROGRESS]: Launching iftdql: C:\PROGRA~1\DOCUME~1
\INDEXA~1\FTINTE~1\iftdql C:\PROGRA~1\DOCUME~1\INDEXA~1\FTINTE~1
Wed Mar 23 13:34:10 2005 [PROGRESS]: Tested 100 of 502 documents,
0 failures. Estimated completion in 0.62 minutes
Wed Mar 23 13:34:24 2005 [INFO ]: Total documents: 502
Wed Mar 23 13:34:24 2005 [INFO ]: Documents tested: 502
Wed Mar 23 13:34:24 2005 [INFO ]: Failures: 0
Wed Mar 23 13:34:24 2005 [INFO ]: Statistical Information:

Wed Mar 23 13:34:24 2005 [INFO ]: Min Max
Avg StdDev N Name
Wed Mar 23 13:34:24 2005 [INFO ]: 0.02 5.01
0.03 0.22 502 Dump Attributes
Wed Mar 23 13:34:24 2005 [INFO ]: 0.00 0.00
0.00 0.00 502 Build Query
Wed Mar 23 13:34:24 2005 [INFO ]: 0.01 0.37
0.01 0.03 502 Execute Query
Wed Mar 23 13:34:24 2005 [INFO ]: 0.02 5.38
0.04 0.24 502 Test One Document
Wed Mar 23 13:34:24 2005 [PROGRESS]: Accuracy Test Finished
E:\Documentum\fulltext\IndexServer\bin>
Additional samples of the tool’s output are in Appendix B, Sample Output of ftintegrity Utility.
Accuracy verification acceptable failures

Although running the accuracy verification provides a high degree of confidence that the index is
accurate, verification is not exhaustive. Consequently, there may be some acceptable failures. For
example, the following causes of failures may be acceptable:
• Missing r_object_id values.
This may occur if an object ID was not indexed. However, ftintegrity tool is not aware of object
types that you have chosen not to index, through either a custom filter or the specification
of selective indexing in Documentum Administrator. If you have chosen not to index one
or more object types, the object IDs of instances of those types will appear in the generated
res-comp-dctmonly.txt file.
• vstamp mismatches
The object in the index may not yet reflect any changes made in the repository.
• Transient failures due to intermittent resource problems experienced by the index server
• Failure to dump property values, possibly due to an inactive Content Server
• Metadata query failures related indexing of special characters
Resubmitting objects to the index agent

When the ftintegrity tool is run, the completeness check produces a text file of object IDs
corresponding to objects that are found in the repository but not in the full-text index. The file is
called res-comp-dctmonly.txt. It is located in the Documentum/fulltext/indexserver/bin directory.
Use the file to resubmit objects to the index agent for indexing.
Because the ftintegrity tool is not aware of object types that you may have chosen not to index,
through either a custom filter or the specification of selective indexing in Documentum Administrator,
the object IDs of instances of those types will appear in the generated res-comp-dctmonly.txt file.
However, when you use the file to resubmit objects for indexing, the objects represented by those
object IDs are ignored. If you are using a custom filter to define the nonindexed types, the operation
will generate queue items for those objects, but they are not indexed and the queue items are deleted
later in the process.
Objects may be resubmitted when the index agent is running in either migration mode or normal
mode. The file of object IDs is designated in the file_name parameter of the indexagent.xml
configuration file for a particular index agent. By default, the file_name parameter is set to the file
name ids.txt. The indexagent.xml file is in
C:\Documentum\bea9.2\domains\DctmDomain\upload\IndexAgentN\IndexAgentN.war\WEB-
INF\classes (on Windows) or $DOCUMENTUM_SHARED/bea9.2/domains/DctmDomain/upload/
IndexAgentN/IndexAgentN.war/WEB-INF/classes (on UNIX), when N is the number of the index
agent.
The index agent periodically checks for the existence of ids.txt. If the file is found, the objects are
resubmitted for indexing.
The file designated in the file_name parameter can have any arbitrary name. For example, the output
file re-comp-dctmonly.txt might be renamed to resubmit.txt or to the default ids.txt. The name
may be a simple name (for example, ids.txt) or a fully-qualified name (for example, C:\Program
Files\Documentum\MyFiles\ids.txt). If the file has a simple name, the index agent tries to resolve
the name by following the CLASSPATH. If you change the name from the default name, you must
modify the indexagent.xml file and substitute the name you assign the file in the file_name parameter.
A particular file must be made available to one index agent only. If multiple index agents are
installed on a host, at installation time the indexagent.xml files for all index agents name the ids.txt
file in the file_name parameter.
For example, if the file is located in C:\Program Files\Documentum\bea9.
2\domains\DctmDomain\upload\IndexAgent1\IndexAgent1.war\WEB-INF\classes, only that
instance will find the file.
When the index agent has read the entire file and submitted all objects for indexing, the input file
is renamed to original_filename.done. For example, if the input file is ids.txt, when all objects are
submitted, it is renamed ids.txt.done.
To resubmit objects to the index agent:

1. Run the ftintegrity tool.
2. Navigate to $DOCUMENTUM//fulltext/indexserver/bin
(%DOCUMENTUM\fulltext\indexerver\bin).
3. Copy the res-comp-dctmonly.txt file to indexagentinstalldirectory/IndexAgents/
IndexAgentN/webapps/IndexAgentN/WEB-INF/classes (drive:\Program
Files\Documentum\IndexAgents\IndexAgentN\webapps\IndexAgentN\WEB-INF\classes).
4. Rename the res-comp-dctmonly.txt file to ids.txt or to an arbitrary name you choose.
5. If you rename res-comp-dctmonly.txt to any name other than ids.txt, you must modify the
file_name parameter in the indexagent.xml file for the particular index agent to reflect the new
name.
The objects are automatically resubmitted for indexing.
Part 4
Reference
Reference
Appendix A
Components
This chapter contains the following topics:

• Index server, page 111
• Index agent , page 112
• Full-text index, page 115
• Repository objects and properties supporting full-text indexing, page 117
Index server
The index server has two functions: it creates full-text indexes and responds to full-text queries
from Content Server.
An index server node is any physical host on which an index server instance runs, regardless of
whether multiple instances of the index server’s individual software process are running.
The index server is represented in the repository by the ft engine config object (dm_ftengine_config).
There is one ft engine config object for each fulltext index object.
Index server processes

The index server consists of five groups of processes that have different functions. In a basic
configuration, these processes all run on the same host. In a multinode configuration, these processes
are run on different hosts to improve performance. The processes are:
• Administrative services
The index server’s administrative services are the Status Server, which monitors the progress of a
document through the index server’s internal processes from when the document is acquired from
the index agent to the time the document is indexed, and the content distributor, which accepts
DFTXML from the index agent and routes it to a document processor.
• Document processors
Document processors (also sometimes called procservers) extract indexable content from
content files, convert DFTXML to FIXML (a format that is used directly by the indexer), and
merge the indexable content with the metadata during the DFTXML conversion process.
Components
Document processors are the largest consumer of CPU power in the index server. In multinode
configurations, a document processor communicates with the indexers on all other hosts in the
configuration.
• Indexer
The indexer creates the searchable full-text index from the intermediate FIXML format. It consists
of two processes. The frtsobj process interfaces with the document processor and spawns different
findex processes as necessary to build the index from FIXML.
• Query and results servers
The QR server is a permanently running process that accepts queries from Content Server,
passes queries to the fsearch processes, and merges the results when there are multiple fsearch
processes running. In a multinode configuration, the QR Server is installed on the node where
administrative processes are running.
• Search servers
Search servers locate items in the index as specified in a query. The fsearchctrl, fdispatch, and
fsearch processes work together to fulfil requests from the qrserver process.
In a single-node deployment, all of these processes reside on one host.
In a multinode deployment, copies of the document processors, the Indexer, and the search servers
reside on each node. The administrative processes and the query and results servers reside on only
one of the nodes.
Index server modes

The index server can run in continuous mode or in a special mode called suspended mode. In
suspended mode, FIXML is generated for any updates to the index but not integrated into the index.
When the index server is taken out of suspended mode, the index is updated.
Running in suspended mode speeds up the indexing process. Use suspended mode in these
circumstances:
• When creating a new index as part of migrating from a pre-5.3 Content Server
• When reindexing a repository
• When indexing a large volume of documents
Use the index agent JSP page to put the index server in suspended mode.
Index agent
The index agent is a multithreaded Java application running in the application server servlet
container. Each index agent runs in its own application server instance, and each index agent is
associated with only one repository.
Install the index agent and the index server on the same host.
Components
Certain operations on SysObjects in the repository generate queue items indicating that the object and
any associated content files must be submitted for full-text indexing. When the operation occurs,
Content Server generates an event that is stored as a queue item for the dm_fulltext_user user. The
index agent reads the queue items, then exports the content files and metadata from the repository
and prepares the documents and metadata for indexing by converting them to DFTXML.
The events that generate a queue item for the dm_fulltext_user user are:
• dm_checkin
• dm_destroy
• dm_move_content
This occurs when a content is moved to another storage area using MIGRATE_CONTENT.
• dm_readonlysave
This event occurs when Content Server sets a property of an immutable object.
• dm_save
An index agent in normal mode is represented by an ft index agent config object. The properties of
the ft index agent config object primarily record status information about the index agent, including
the mode in which the index agent is running and when the index agent began processing queue
items. The properties also record configuration information about the index agent, such as the
number of queue items processed in a single batch, the number of exporter threads, and the time
interval at which the index agent polls the repository for queue items. This information may be
viewed using Documentum Administrator. For more information about the ft index agent config
object, refer to the Documentum Object Reference Manual.
An index agent in migration mode is represented by an XML configuration file, IndexAgent.xml, on
the index agent host. Do not modify the parameters in the configuration file unless you are enabling
file store mapping. Mapping file stores for improved indexing performance is documented in the
Documentum System Planning Guide.
Index agent modes, page 114, contains complete information on the modes in which the index agent
runs.
Index agent processes

When running in normal mode, the index agent processes index queue items and the SysObjects
associated with the queue items. The index queue consists of dmi_queue_items that are queued to the
dm_fulltext_index_user. The queue items are generated by Save, Saveasnew, Checkin, Destroy, or
Branch events performed on a SysObject or SysObject subtype in the repository. The value of the
queue item’s task_state property changes depending on where the SysObject is in the indexing
process:
1. Empty string — The initial task_state is an empty string, indicating that the associated SysObject
needs to be indexed and the queue item is available for processing by the index agent.
2. Acquired — When the index agent acquires the queue item and begins to process the SysObject,
the task_state is updated to Acquired.
3. Done — When the SysObject is successfully indexed by the index server, the task_state is updated
to Done and the queue item is deleted from the repository.
Components
No further updates are made to the queue item.

4. Warning — If the index agent encounters a problem in processing the SysObject to which the
queue item refers, the task_state is changed to Warning.
This indicates that the index server has made a partial update to the index. The index agent does
not further update the queue item or perform further operations on the SysObject. The queue
item remains in the queue.
5. Failed — If indexing fails and the index server is unable to add information about the SysObject
to the index, the task_state is updated to Failed.
The index agent does no further processing on the SysObject and the queue item remains in the
queue. To determine which objects failed indexing, use Documentum Administrator.
By default, the index agent acquires queue items in batches of 1,000. If the index agent shuts down
unexpectedly after acquiring queue items, the queue items remain acquired by that index agent.
When the index agent is restarted, the task_state property for those acquired queue items is reset to ’ ’
(an empty string) and the queue items are processed correctly.
If the index agent acquires multiple queue items for a particular SysObject, it processes only one of
the queue items. For example, if an object is saved several times, the most recent version is indexed. If
an object is saved and then deleted, only the deletion is processed.
Managing the index queue, page 66, contains complete information on managing the index queue.
Index agent modes

An index agent may run in one of three operational modes:
• normal — In normal mode, the index agent processes index queue items and prepares the
SysObjects associated with the queue items for indexing.
• migration — In migration mode, the index agent processes all SysObjects in a repository
sequentially in r_object_id order and prepares them for indexing. A high-water mark queue item
is used to mark the index agent’s progress in the repository.
• file — In file mode, a file is used to submit a list of objects IDs to the index agent when a new
index is created and index verification determines which objects are missing from the index.
The index agent runs in either migration mode or normal mode against a 5.3 repository and in
migration mode against a 5.2.5 repository. Both modes are compatible with file mode, which is used
for submitting a list of object IDs to the index agent for indexing.
Normal mode
Content Servers 5.3 and later generate a queue item when an event such as a check-in or save
requires that a new or modified object must be indexed. In normal mode, the index agent reads
the queue item, prepares the object for indexing, and updates the queue item. When the index
agent successfully submits the object for indexing, the index agent deletes the queue item from the
repository. If the object is not submitted successfully, the queue item remains in the repository and
Components
the error or warning generated by the attempt to index the object is stored in the queue item. The
index agent can run in normal mode only against a 5.3 or later repository.
An index agent in normal mode and an index agent in migration mode cannot simultaneously
update the same index.
Migration mode
In migration mode, the index agent prepares all indexable objects in a repository for indexing
in object ID order. The high-water mark records the ID of the most recent object indexed. The
index agent reads the value in the queue item, exports the next batch of indexable objects from
the repository, and updates the queue item. The index agent can run in migration mode to create
new indexes against a 5.2.5.xor 5.3.x repository.
For example, if you need to move the indexing component installation to a new host, you can install
the components on the new host and create a new index there in migration mode, while the existing
indexing component installation maintains the existing index in normal mode.
In migration mode, the index agent prepares all indexable objects for indexing in object ID order. A
single queue item records the ID of the most recent object indexed.
File mode
File mode is used for submitting a list of object IDs to the index agent for indexing.
File mode can be used when the index agent is in any operational mode. However, it is most useful
immediately after a new index is created. After a new index is created, the ftintegrity tool is run to
determine whether all indexable objects in the repository are included in the index. The ftintegrity
tool produces a list of object IDs of SysObjects that are not included in the index. The list is used to
submit the objects to the index agent for indexing.
Full-text index
An index is made up of nodes. Depending on your deployment configuration, an index may have a
single node or multiple nodes.
Each node has one column. A column is a physical set of processes and partitions that contain a
portion of the indexed data. Each column has three partitions and the processes that search those
partitions.
The data in each column is unique. That is, a document’s indexed data resides on only one node. A
column may contain data from one repository or multiple repositories.
Components
Partitions
A partition is the smallest physical grouping of documents within a column. By default, each column
has three partitions: a small partition, a medium partition, and a large partition. The documents most
recently submitted for indexing are initially indexed into the small partition. The index server moves
index data from the small partition to the medium partition, and finally to the large partition. The
index data for all documents eventually is in the large partition.
While new documents are being added to a partition, the partition is offline and in the process of
being rebuilt. New documents are not added to the index while the partition is offline. A copy of the
partition remains online and searchable. When the offline partition is completely rebuilt, it becomes
online and searchable, and the online copy is taken offline and newly-processed FIXML is indexed.
Collections
All documents in an index belong to a collection, a logical set of data. The full-text indexing engine
uses collections to segregate data logically. If the index stores data from only one repository in a
single-node deployment, the data is typically in one collection.
In consolidated deployments, in which multiple repositories are indexed in the same index, the
indexed data from each repository is associated with a separate collection.
In a multinode deployment, a particular collection might be spread out over multiple nodes or it
might be stored on only one node. The default behavior in a multinode deployment is to store
the collections across nodes. However, you can configure the system to use directed routing. This
behavior requires you to store content files in particular storage areas, which are mapped to particular
collections, and in turn, those collections are mapped to particular nodes. Which content files go to
which storage area is a business decision.
Directed routing
Directed routing refers to the ability to direct index data to particular nodes. Using a multinode
deployment with directed routing is recommended if you need to separately back up and restore
individual collections or if you have a very large repository (in this release, 20 million or more
documents).
Directed routing is implemented by mapping each file store storage area to a particular collection,
and then mapping each collection to a particular column. Any arbitrary number of storage areas may
be mapped to a particular collection. The mapping may be modified after installation to add more
collections, though the index agent must be restarted after the mapping is modified.
Use directed routing only in the following circumstances:
• If large amounts of the data will be unchanging (never updated) and you want to reduce the
amount of data to index.
• If the full-text indexing system is growing and you need to incrementally add hardware to the
system with little or no need to reorganize the indexed data.
Components
Directed routing allows you to fill a node and move on to newly added nodes. If you were
not using directed routing in these situations, the system would perform additional work to
redistribute the data across all of the new nodes.
Configuring directed routing is a manual process requiring EMC Documentum Professional Services
(or qualified third-party integrators).
Repository objects and properties supporting

full-text indexing
Full-text indexing is supported in the repository by objects representing the full-text index, the index
server, and the index agent. Properties of the server config object, SysObject, and queue item also
support full-text indexing.
Fulltext indexing is supported in the repository by the following objects:
• fulltext index object, representing the fulltext index
• ft index agent config object, representing the index agent
• ft engine config object, representing the index server
The full-text index object is created at repository configuration time. In a basic full-text indexing
configuration, the ft index agent config object and ft engine config object are created in the repository
when the index agent configuration program creates a normal-mode index agent.
In a high-availability configuration, you need to create a fulltext index object, ft index agent config
object, and ft engine config object to represent the second indexing software installation and the index
maintained by the second installation. You need to set certain attribute values in the objects created to
ensure that the index is updated correctly. In addition, you need to create a second fulltext index user
(dm_fulltext_index_user_01), and you need to register the Save, Checkin, Destroy, Readonlysave, and
MoveContent events for that user. A script is provided to create the required objects, set the attribute
values, and register the events for the second fulltext index user.
Figure 3, page 118, shows the objects involved and the relationships among them:
Components
Figure 3. Fulltext indexing object relationships
Refer to Install a high-availability deployment, page 28, for high-level information on installing a
high-availability configuration and on the objects that must be created and attribute values updated
after installation.
Components
Fulltext index object

A fulltext index object (dm_fulltext_index) represents an index associated with a repository. Its
properties provide information such as the code page in which Content Server represents the
following:
• The strings sent to the query plug-in
• The index name
• The object ID of the index server for the index
• The name of the location object containing the location of the query plug-in
A fulltext index object is created when the first normal mode index agent is configured for the
repository. The only property of the fulltext index object that may be updated is the ft_engine_id
property, which contains the object ID of the ft engine config object representing the index server.
FT index agent config object

The ft index agent config object represents a normal-mode index agent configured for the repository.
Its properties record status information and configuration information about the index agent.
FT engine config object

The ft engine config object represents the index server for the repository.
Location objects
Each fulltext index object points to a location object that represents the location of the dmfulltext.ini
file.
Supporting properties of other objects

The following properties of other objects support full-text indexing:
• a_full_text
• fulltext_location
Configuring format objects to specify which renditions are indexed, page 34, contains information
on how the setting of the format_class property of the format object controls which renditions are
indexed.
Components
The a_full_text property
The a_full_text property is defined for the SysObject type and is inherited by all SysObject subtypes.
It is a Boolean property that controls whether an object’s associated content files are indexed.
When a_full_text is TRUE, content files are indexed whenever a Save, Checkin, Destroy,
Readonlysave, or MoveContent operation generates an index queue item for the object. Any changes
to the object’s content are added to the index.
The a_full_text property is set to TRUE whenever a SysObject is created. Users with system
administrator or superuser privileges can change the a_full_text setting.
The fulltext_location property
The value of the fulltext_location property of the server config object contains the name of the location
object that identifies the full-text configuration file, dmfulltext.ini.
Initialization files
This section describes the entries in the server initialization file that support full-text indexing and the
dedicated full-text initialization file.
Fulltext entries in the server.ini file
Two parameters in the server.ini file govern aspects of full-text indexing:

• max_ftacl_cache_size
Content Server caches ACL information on objects to evaluate security on the results returned by
full-text queries. The max_ftacl_cache_size key defines the number of elements cached.
The default value is -1 (no limit set). If the value is set to 0, no security information is cached.
The value may be set to any integer greater than -1. EMC Documentum recommends not to
change the default value.
• start_index_agents
The start_index_agents parameter defines whether the index agents associated with a particular
repository are started at Content Server startup. The default value is TRUE. Changing the value to
FALSE means that no index agent is available to process index queue items in the repository until
you manually start an index agent.
Components
The dmfulltext.ini file
The dmfulltext.ini file is created when the server is installed. It contains information used by Content
Server to find the index agent plug-in.
Components
Appendix B
Sample Output of ftintegrity Utility
The following report is the output of a failed completeness and accuracy run of the ftintegrity utility:
29 C% cobra ftintegrity.py -i ftintegrity.params.txt -m b
Mon Feb 07 15:48:37 2005 [PROGRESS]: Completeness Test Started
Mon Feb 07 15:48:38 2005 [PROGRESS]: Fetched all 11 docids from
Documentum
Mon Feb 07 15:48:38 2005 [WARNING ]: Suspiciously low document
count. Following
errors could be due to invalid result from the command below.
Mon Feb 07 15:48:38 2005 [WARNING ]: Please run it to validate
correctness
Mon Feb 07 15:48:38 2005 [WARNING ]: C:\PROGRA~1\DOCUME~1
\java\1449C1~1.2_0\bin\
java -classpath C:\PROGRA~1\DOCUME~1\INDEXA~1\INDEXA~1\webapps\
INDEXA~1\WEB-INF\
lib\server-impl.jar;C:\PROGRA~1\DOCUME~1\dctm.jar;C:\Documentum\
config com.docum
entum.server.impl.fulltext.ftintegrity.FTDumpIDs -m:0
-D:waasdfasdfasfasfasdfasf
-U:wongw -P:groucho2
Mon Feb 07 15:48:38 2005 [INFO ]: Get Documentum IDs: 0.988 s
Mon Feb 07 15:48:38 2005 [DEBUG ]: Got 11 docids from Dctm
Mon Feb 07 15:48:38 2005 [INFO ]: Get Number of Documents in
Collection: 0.28
1 s
Mon Feb 07 15:48:38 2005 [INFO ]: Number of documents in
collection documentu
m: 1351
Mon Feb 07 15:48:39 2005 [PROGRESS]: Fetched all 1351 docids from FAST
Mon Feb 07 15:48:39 2005 [INFO ]: Get FAST IDs: 0.780 s
Mon Feb 07 15:48:39 2005 [DEBUG ]: Got 1351 docids from fast
Mon Feb 07 15:48:39 2005 [INFO ]: Transform FAST ID file: 0.084 s
Mon Feb 07 15:48:39 2005 [INFO ]: Transform Documentum ID file:
0.026 s
Mon Feb 07 15:48:39 2005 [INFO ]: Compare IDs: 0.064 s
Mon Feb 07 15:48:39 2005 [PROGRESS]: Completeness Test Finished
Mon Feb 07 15:48:39 2005 [PROGRESS]: Accuracy Test Started
Mon Feb 07 15:48:39 2005 [DEBUG ]: Getting Dctm Ids
from res-comp-common.txt
Mon Feb 07 15:48:39 2005 [FATAL ]: Failed to read any docids from
Documentum.
The command below failed.
type res-comp-common.txt
Terminated.
The following report is the output of a successful completeness and accuracy run of the ftintegrity tool:
30 C% cobra ftintegrity.py -i ftintegrity.params.txt -m b
Mon Feb 07 15:50:29 2005 [PROGRESS]: Completeness Test Started
Mon Feb 07 15:50:35 2005 [PROGRESS]: Fetched all 1351 docids from
Documentum
Mon Feb 07 15:50:35 2005 [INFO ]: Get Documentum IDs: 5.223 s
Mon Feb 07 15:50:35 2005 [DEBUG ]: Got 1351 docids from Dctm
Mon Feb 07 15:50:35 2005 [INFO ]: Get Number of Documents in
Collection: 0.21
9 s
Mon Feb 07 15:50:35 2005 [INFO ]: Number of documents in
collection documentu
m: 1351
Mon Feb 07 15:50:36 2005 [PROGRESS]: Fetched all 1351 docids from FAST
Mon Feb 07 15:50:36 2005 [INFO ]: Get FAST IDs: 0.819 s
Mon Feb 07 15:50:36 2005 [DEBUG ]: Got 1351 docids from fast
Mon Feb 07 15:50:36 2005 [INFO ]: Transform FAST ID file: 0.114 s
Mon Feb 07 15:50:36 2005 [INFO ]: Transform Documentum ID file:
0.099 s
Mon Feb 07 15:50:36 2005 [INFO ]: Compare IDs: 0.136 s
Mon Feb 07 15:50:36 2005 [PROGRESS]: Completeness Test Finished
Mon Feb 07 15:50:36 2005 [PROGRESS]: Accuracy Test Started
Mon Feb 07 15:50:36 2005 [DEBUG ]: Getting Dctm Ids from
res-comp-common.txt
Mon Feb 07 15:50:36 2005 [PROGRESS]: Fetched all 1351 docids
from Documentum
Mon Feb 07 15:50:36 2005 [PROGRESS]: Starting test of 0 documents
Mon Feb 07 15:50:36 2005 [PROGRESS]: Launching iftdql: C:\PROGRA~1\
DOCUME~1\INDE
XA~1\FTINTE~1\iftdql C:\PROGRA~1\DOCUME~1\INDEXA~1\FTINTE~1
\FASTQueryPlugin.dll
Mon Feb 07 15:51:03 2005 [PROGRESS]: Tested 100 of 1351 documents,

0 failures.
Estimated completion in 5.63 minutes
0 failures.
0 failures.
0 failures.
0 failures.
0 failures.
0 failures.
0 failures.
0 failures.
0 failures.
0 failures.
0 failures.

0 failures.
Mon Feb 07 15:53:34 2005 [INFO ]: Total documents: 1351
Mon Feb 07 15:53:35 2005 [INFO ]: Documents tested: 1351
Mon Feb 07 15:53:35 2005 [INFO ]: Failures: 0
Mon Feb 07 15:53:35 2005 [INFO ]: Statistical Information:
Mon Feb 07 15:53:35 2005 [INFO ]: Min Max
Avg StdDev
N Name
Mon Feb 07 15:53:35 2005 [INFO ]: 0.01 6.80
0.03 0.19
1351 Dump Attributes
Mon Feb 07 15:53:35 2005 [INFO ]: 0.00 0.00
0.00 0.00
1351 Build Query
Mon Feb 07 15:53:35 2005 [INFO ]: 0.01 1.16
0.10 0.10
1351 Execute Query
Mon Feb 07 15:53:35 2005 [INFO ]: 0.02 7.04
0.13 0.22
1351 Test One Document
Appendix C
Indexed and Non-Indexed Characters

• Indexed characters, page 127
• Non-indexed special characters, page 127
Indexed characters
The following Unicode characters are indexed and are searchable:
• Alphabetic characters
• Numeric characters
• Extender characters
Extender characters extend the value or shape of a preceding alphabetic character. These are
typically length and iteration marks.
• Custom characters enclosing Chinese, Japanese, and Korean letters and months
These are derived from a number of custom character ranges that have bidirectional properties,
falling in the 3200-32FF range. The specific character ranges are:
— 3200-3243
— 3260-327B
— 327F-32B0
— 32C0-32CB
— 32D0-32FE
For a detailed list of UTF-8 special characters see http://doc.infosnel.nl/extreme_utf-8.html.
Non-indexed special characters

The index server does not index the following special characters and instead replaces them with
white spaces in the index:
!@#$%^_,.&;:()-+=<
Indexed and Non-Indexed Characters
For example, when the email address MyName@company.com is indexed, it appears as “MyName
company com” in the index. The text is treated as three words. Documents returned by a search for
MyName@company.com are treated as if they contain the words “MyName company com”.
Note: Because the index treats that email address as three words, the document containing
MyName@company.com would also be returned if the user conducted a phrase search on “MyName
company com”.
If a special character is included in a query, it is removed. For example, querying on Richard+Dodd

would return a document containing the text Richard=Dodd because the + and = signs are both
replaced by a blank space. If a search term includes an accent or diacritical mark, the search returns
all matching words with or without the accent or diacritical mark.
Appendix D
Supported and Unsupported Formats
The formats listed in Table 6, page 129 are supported for full-text indexing.
Note: Arabic and Hebrew — For non-binary formats, only logical text representation is supported;
visual text representation is not supported. For those binary formats listed below that support
right-to-left text in the native format, support is provided for indexing Hebrew and Arabic text, with
the exception that PDF files cannot be indexed.
Table 6. Supported document formats
Document format Version

Access Versions through 2.0
Adobe Acrobat (PDF) Versions 2.1, 3.0 – 7.0; You cannot search
scanned PDFs unless the OCR equipment
generated searchable PDFs.
Adobe FrameMaker (MIF) Version 6.0
Adobe FrameMaker graphics (FMV) Vector/raster through 5.0
Adobe Illustrator Versions through 7.0, 9.0
Adobe Photoshop (PSD) Version 4.0
Ami Draw (SDW) Ami Draw
ANSI Text 7 & 8 bit
ASCII Text 7 & 8 bit
AutoCAD Drawing Versions 2.5 - 2.6, 9.0 - 14.0, 2000i and 2002
AutoCAD Interchange and Native Drawing formats DXF and DWG
AutoShade Rendering (RND) Version 2.0
Binary Group 3 Fax All versions
Bitmap (BMP, RLE, ICO, CUR, OS/2 DIB & WARP) All versions
CALS Raster (GP4) Type I and Type II
Computer Graphics Metafile (CGM) ANSI, CALS NIST version 3.0
Corel Clipart format (CMX) Versions 5 through 6
Corel Draw (CDR with TIFF header) Versions 2.x – 9.x

Corel Draw (CDR) Versions 3.x – 8.x
Corel/Novell Presentations Versions through 11.0
DataEase Version 4.x
dBASE Versions through 5.0
dBXL Version 1.3
DEC WPS Plus (DX) Versions through 4.0
DEC WPS Plus (WPL) Versions through 4.1
DisplayWrite 2 & 3 (TXT) All versions
DisplayWrite 4 & 5 Versions through Release 2.0
EBCDIC
Enable Versions 3.0, 4.0 and 4.5
Enable Versions 3.0, 4.0 and 4.5
Encapsulated PostScript (EPS) TIFF header only
Executable (EXE, DLL)
Executable (Windows) NT
First Choice Through 3.0
First Choice Versions through 3.0

First Choice Versions through 3.0
FoxBase Version 2.1
Framework 3.0
Framework Version 3.0
Framework Version 3.0
Freelance (Windows) Versions through Millennium 9.6
Freelance for OS/2 Versions through 2.0
GEM Paint (IMG) No specific version
Graphics Environment Mgr (GEM) Bitmap & vector
Graphics Interchange Format (GIF) No specific version
GZIP All versions
Hangul Version 97
Harvard Graphics (Windows) Windows versions
Harvard Graphics for DOS Versions 2.x & 3.x
Hewlett Packard Graphics Language (HPGL) Version 2
HTML through 3.0 (some limitations)
IBM FFT All versions
IBM Graphics Data Format (GDF) Version 1.0

IBM Picture Interchange Format (PIF) Version 1.0
IBM Revisable Form Text All versions
IBM Writing Assistant 1.01
Initial Graphics Exchange Spec (IGES) Version 5.1
JFIF (JPEG not in TIFF format) All versions
JPEG (including EXIF) All versions
JustSystems Ichitaro Versions 5.0, 6.0, 8.0 – 12.0
JustWrite Versions through 3.0
Kodak Flash Pix (FPX) All versions
Kodak Photo CD (PCD) Version 1.0
Legacy Versions through 1.1
Lotus 1-2-3 (DOS & Windows) Versions through 5.0
Lotus 1-2-3 (OS/2) Versions through 2.0
Lotus 1-2-3 Charts (DOS & Windows) Versions through 5.0
Lotus 1-2-3 for SmartSuite Versions 97 – Millennium 9.6
Lotus AMI/AMI Professional Versions through 3.1
Lotus Manuscript Version 2.0
Lotus PIC All versions
Lotus Snapshot All versions
Lotus Symphony Versions 1.0,1.1 and 2.0
Lotus Word Pro Versions 96 through Millennium Edition
9.6, text only
LZA Self Extracting Compress All versions
LZH Compress All versions
Macintosh PICT1 & PICT2 Bitmap only
MacPaint (PNTG) No specific version
MacWrite II Version 1.1
MASS11 Versions through 8.0
Micrografx Designer (DRW) Versions through 3.1
Micrografx Designer (DSF) Windows 95, version 6.0
Micrografx Draw (DRW) Versions through 4.0
Microsoft Binder Versions 7.0-97 (only on Windows)
Microsoft Excel (Mac) Versions 3.0 – 4.0, 98, 2001
Microsoft Excel (Windows) Versions 2.2 through 2003
Microsoft Excel Charts Versions 2.x - 7.0

Microsoft Multiplan Version 4.0
Microsoft Outlook Folder (PST) Versions 97, 98, 2002, and 2002
Microsoft Outlook Message (MSG) All versions. The body of the message is
indexed and attachments to messages are
indexed.
Microsoft PowerPoint (Mac) Versions 4.0 through 2001
Microsoft PowerPoint (Windows) Versions 3.0 through 2003
Microsoft Project Versions 98 - 2002 (text only)
Microsoft Rich Text Format (RTF) All versions
Microsoft Word Versions through 6.0
Microsoft Word Versions through 2003
Microsoft Word (Mac) Versions 3.0 – 4.0, 98, 2001
Microsoft WordPad All versions
Microsoft Works Versions through 2.0

Microsoft Works Versions through 4.0
Microsoft Works (DOS) Versions through 2.0
Microsoft Works (DOS) Versions through 2.0
Microsoft Works (Mac) Versions through 2.0
Microsoft Works (Windows) Versions through 4.0
Microsoft Works (Windows) Versions through 4.0
Microsoft Write Versions through 3.0
MIME Text Mail
Mosaic Twin Version 2.5
MultiMate Versions through 4.0
Navy DIF All versions
Nota Bene Version 3.0
Novell Perfect Works Version 2.0
Novell Perfect Works Version 2.0
Novell Perfect Works (Draw) Version 2.0
Novell WordPerfect Versions through 6.1
Novell WordPerfect Versions 1.02 through 3.0
Novell/Corel WordPerfect Versions through 11.0
Office Writer Versions 4.0 - 6.0

OS/2 PM Metafile (MET) Version 3.0
Paint Shop Pro 6 (PSP) Windows only, versions 5.0 – 6.0
Paradox (DOS) Versions through 4.0
Paradox (Windows) Versions through 1.0
PC Paintbrush (PCX and DCX) All versions
PC-File Letter Versions through 5.0
PC-File+ Letter Versions through 3.0
Personal R:BASE Version 1.0
PFS: Professional Plan Version 1.0
PFS: Write Versions A, B and C
Portable Bitmap (PBM) All versions
Portable Graymap (PGM) No specific version
Portable Network Graphics (PNG) Version 1.0
Portable Pixmap (PPM) No specific version
Postscript (PS) Level II
Professional Write Versions through 2.1
Professional Write Plus Version 1.0
Progressive JPEG. No specific version
Q&A Versions through 2.0
Q&A Version 2.0
Q&A Write Version 3.0
Quattro Pro (DOS) Versions through 5.0
Quattro Pro (Windows) Versions through 11.0
R:BASE 5000 Versions through 3.1
R:BASE System V Version 1.0
Reflex Version 2.0
Samna Word Versions through Samna Word IV+
SmartWare II Version 1.02
Sprint Versions through 1.0
Star Office/Open Office Calc Star Office Versions 5.2, 6.x, and 7.x; Open
Office version 1.1 (text only)
Star Office/Open Office Draw Star Office 5.2, 6.x, and 7.x; Open Office
version 1.1 (text only)

Star Office/Open Office Writer Star Office Versions 5.2, 6.x, and 7.x; Open
Office version 1.1 (text only)
StarOffice / Open Office Impress StarOffice 5.2, 6.x, and 7.x; Open Office 1.1
(text only)
Sun Raster (SRS) No specific version
SuperCalc 5 Version 4.0
Text Mail (MIME)
TIFF Versions through 6
TIFF CCITT Group 3 & 4 Versions through 6
Total Word Version 1.2
Truevision TGA (TARGA) Version 2
Unicode Text All versions
UNIX Compress
UNIX TAR
UUEncode
vCard Version 2.1
Visio Versions 5, 2000 and 2002
Visio (preview) Version 4
Volkswriter 3 & 4 Versions through 1.0
VP Planner 3D Version 1.0
Wang PC (IWP) Versions through 2.6
WBMP No specific version
Windows Enhanced Metafile (EMF) No specific version
Windows Metafile (WMF) No specific version
WML Version 5.2

WordMARC Versions through Composer Plus
WordPerfect Graphics (WPG & WPG2) Versions through 2.0, 7 and 10
WordStar Versions through 7.0
WordStar Version 1.0
WordStar 2000 Versions through 3.0
XML All versions
X-Windows Bitmap (XBM) x10 compatible
X-Windows Dump (XWD) x10 compatible
X-Windows Pixmap (XPM) x10 compatible

XyWrite Versions through III Plus
ZIP PKWARE versions through 2.04g. Files
inside a ZIP file are not indexed.
The formats listed in Table 7, page 135are not supported for full-text indexing. Only those formats
listed in Table 6, page 129 are supported.
Table 7. Unsupported document formats

Applix Graphics 4.3, 4.4
Applix Presents 4.3, 4.4
Applix Spreadsheets 4.2, 4.3, 4.4
Applix Words 4.2, 4.3, 4.4
Folio Flat File 3.1
Fujitsu Oasys 7.0
Microsoft Outlook Express eml format No specific version
Microsoft Windows Animated Cursor No specific version
Microsoft Windows Cursor/Icon No specific version
SGI RGB No specific version
Appendix E
Supported Languages
This appendix lists the languages supported for full-text indexing.
Table 8. Supported languages
Language Code Language Code

Afrikaans af Italian 1 it
Albanian sq Japanese na
Arabic ar Kazahk kk
Armenian hy Kirghiz ky
Azeri az Korean ko
Bangla bn Kurdish ku
Basque eu Latin la
Bosnian bs Latvian lv
Breton br Letzeburgesch lb
Bulgarian bg Lithuanian lt
Byelorussian by Macedonian mk
Catalan ca Malay ms
Chinese_simplified 2 zh_cn Maltese mt
Chinese_traditional 2 zh_tw Maori mi
Croatian hr Mongolian mn
Czech cs Norwegian_Bokmaal nb
1
Danish da Norwegian_Nynorsk nn
1
1. Grammatical normalization is supported for this language. Using grammatical normalization ,

page 14 briefly describes grammatical normalization.
2. There is no dictionary for Chinese because that language is not an inflected language and
therefore grammatical normalization is not an operation that can be performed on Chinese text.
Supported Languages
Language Code Language Code

Dutch nl Polish 1 pl
English 1 en Portuguese 1 pt
Esperanto eo Rhaeto_Romance rm
(Romansch)
Estonian et Romanian ro
Faeroese fo Russian 1 ru
Farsi fa Sami_Northern se
Filipino (Tagalog) tl Serbian sr
Finnish fi Slovak sk
French 1 fr Slovenian sl
Frisian fy Spanish 1 es
Galician gl Swahili sw
Georgian ka Swedish 1 sv
German 1 de Tamil ta
Greek el Thai th
Greenlandic kl Turkish tr
Hausa ha Ukrainian uk
Hebrew he Urdu ur
Hindi hi Uzbek uz
Hungarian 1 hu Vietnamese vi
Icelandic is Welsh cy
Indonesian id Yiddish yi
Irish_Gaelic ga Zulu zu
1. Grammatical normalization is supported for this language. Using grammatical normalization ,

page 14 briefly describes grammatical normalization.
2. There is no dictionary for Chinese because that language is not an inflected language and
therefore grammatical normalization is not an operation that can be performed on Chinese text.
Index
/3GB switch constraint, 22 DFTXML, 111 to 112

diacritical marks, in full-text index, 127
dictionaries, synonym, 84
A directed routing
a_full_text property, 120 configuring, 44
accent marks, in full-text index, 127 directory constraint for HP-UX, 32
accuracy testing disabling indexing for object types, 33
confidence in results, 105 dm_FTCreateEvents, 74
acl_check_db configuration parameter, 36 dm_FTIndexAgentBoot job, 80
administration dm_fulltext_user user
Index Agent Admin Tool, 78 events registered for, 113
antivirus software, 22 dmfulltext.ini file, 121
application server, 112 DNS entries, 22
log file location, 83 DocProcessor, described, 111
Arabic documents
limitations, 129 routing to nodes, 116
Documentum Administrator, 77
C drives, sharing, 38, 43
Chinese grammatical normalization, 137
collections E
routing to columns, 116 environment variables, 23 to 24
collections, defined, 45, 116 error for large file rejection, 90
column, defined, 115 events initiating indexing, 113
consolidated deployments exporter threads
installing, 27 default number of, 91
upgrading, 53
Content Distributor, described, 111
content files F
full-text indexing constraint, 16 file store storage areas
Content Server full-text indexing and, 113
index agent at server startup, 78 findex process, described, 112
shutdown, 78 fixml
startup, 78 rejection error, 90
creating FIXML, 111
indexing queue items, 68 format objects
cscript.exe file, 24 format_class property, 34
using to control indexing, 34
format_class property, 34
D formats
deleting an index, 61 indexable, 34, 129
DFC_DATA variable (deprecated), 24
Index
listing indexable, 89 controlling by format, 34

non-indexable, 135 creating events, 74
frtsobj process, described, 112 creating queue items, 68
ft engine config objects, 119 deleting an index, 61
ft engine config parameters described, 13, 16
temp_table_batch_size, 36 DFC_DATA environment variable
ft index agent config object, 113 (deprecated), 24
ft index agent config objects, 119 DFTXML, 112
ftengine config directed routing, configuring, 44
acl_check_db parameter, 36 disabling by object types, 33
ftinetgrity utility disabling content indexing, 73
parameters, list of, 101 environment variables, 23 to 24
ftintegrity tool, 115 events used by, 113
running, 103 file store mapping, 38, 43, 113
ftintegrity utilitiy, 99 format_class property, 34
ftintegrity utility formats, 34
accuracy confidence, 105 supported, 129
ftintegrity.params.txt, 100 unsupported, 135
sample output, 123 ft engine config objects, 119
ftintegrity.params.txt, 100 ft index agent config object, 113
ftquery_retry_on_queryserver_errorcodes ft index agent config objects, 119
configuration parameter, 35 ftintegrity tool, 115
full-text index, 13, 21 ftintegrity.params.txt, 100
See also full-text indexing; fulltext index objects, 119
preinstallation requirements fulltext_location property, 120
accuracy, verifying, 99 grammatical normalization, 14
benefits of use, 13 host names, 21
completeness, verifying, 99 host requirements, 22
content of, 13 ImportDictionary.py script, 85, 87
dmfulltext.ini file, 121 index agent, 15, 112
nodes, 115 index agent, role in, 15, 112
replacing existing, 74 index agent, starting, 78
VMware constraint, 22 index queue, 66
full-text indexes index server, 15, 111
acl_check_db parameter, 36 index server, role in, 15
batched returns, 36 indexable content, 13
collections, 45, 116 indexable formats, 34, 89
columns, 115 installation account, 23
deleting, 61 installation order, 27
languages, supported, 137 installing the index agent configuration
partitions, 116 program, 30
State of the Index job, 69 installing the index server, 30
full-text indexing languages supported, 14
a_full_text property, 120 languages, supported, 137
administration, 77 large files, constraint on, 16
antivirus software, 22 location objects, 119
associating an index with a log files, 39
repository, 75 network configuration, 22
consolidated deployments, 53 new repository, 27
Content Server, 15 parts of speech, 14
Index
performance, 38, 43 host requirements, 22

processing queue items, 113 HP-UX directory constraint, 32
properties supporting, 117 HP-UX parameter values required to
queries, 15 install, 30
query plug-in, 15
queue item processing, 113
queue items, 66, 68
I
reindexing, 74 ImportDictionary.py script, 85
relocating software components, 115 index agent
required ports for index agent, 21 configuration file, 113
required ports for index server, 21 configuring, 36, 42
resubmitting objects, 105 Content Server shutdown, 78
routing, directed, 116 described, 15, 112
searchable characters, 127 DFC_DATA environment variable
server.ini file entries, 120 (deprecated), 24
sharing drives, 38, 43 exporter threads, configuring number
software components, 15 of, 91
software installation, 27 file mode, 114 to 115
submitting objects for indexing, 115 ft index agent config object, 113
suspended mode, 112 installation options, 112
suspending and resuming, 73 installing configuration program, 30
synonym file, 85 log file location, 83
SysObjects, 13 migration mode, 113 to 115
thesaurus, 14 modes, 114
thesaurus searching, 84 multiple queue items, 114
timeouts, 75 normal mode, 113 to 114
turning on or off, 72 processing queue items, 113
unsearchable characters, 127 queue item processing, 113 to 114
upgrading, 53 representation in repository, 119
verifying indexes, 103 required ports, 21
full-text queries resource agent URL, 65
tracing, 89 resubmitting objects for indexing, 105
fulltext index objects, 119 role in full-text indexing, 15, 112
fulltext_location property, 120 role in indexing process, 16
start_index_agents server.ini key, 78
starting, 78
G timeouts, 75
grammatical normalization, 14 tracing, 89
unexpected shutdowns, 114
uninstalling, 59
H uninstalling components, 59
Hebrew Index Agent Admin Tool, 78
limitations, 129 index agent configuration program
high-availability deployments deleting, 60
failures, 90 index agents
installing, 28 disabled, 81
limitations, 69 dm_FTIndexAgentBoot job, 80
prune methods, 69 enabling, 81
Save events, 69 modifying properties, 81
host names, 21 properties, 81
Index
starting, 79 VMware constraint, 22

stopping, 79 installation accounts, 23
index queue, 66 installation logs, 39
index routing installation order
index server configuration new repositories, 27
requirements, 47
index server
antivirus software, 22
J
component processes, 111 jobs
configuration options, 15 dm_FTIndexAgentBoot, 80
constraints on Windows hosts, 22 State of the Index, 69
Content Distributor, 111
deleting, 61 L
described, 15 large files
disk partition requirement, 22 rejection error, 90
DocProcessor, 111 location objects
findex process, 112 full-text indexing, 119
host time zone settings, 22 log files, 39
Indexer, 112 application server, 83
installing, 30 archived, 90
Java installation, 21 index agent, 83
nodes, 111 index server, 83
QR Server, 112
query retries, configuring, 35
query subsystem, 112 M
representation in repository, 119 MARK_FOR_RETRY administration
required ports, 21 method, 67
role in full-text indexing, 15 modifying
role in indexing process, 16 index agent properties, 81
Status Server, 111
suspended mode, 112
timeouts, 75
N
index servers network configuration, 22
log file location, 83 nodes, 115
properties, 82 nodes, described, 111
starting, 80 nonsearchable characters, 127
stopping, 80
indexable formats, 34, 89 O
indexagent.xml file, 38, 43
object types, disabling indexing of, 33
Indexer, described, 112
install directory constraint for HP-UX, 32
installation P
basic deployments, 27 partitions, described, 116
consolidated deployments, 27 parts of speech, normalization choices
constraints on Windows hosts, 22 for, 14
high-availability deployments, 28 permissions, checking against
index agent configuration program, 30 repository, 36
index server, 30 phonetic searching, 13
log files, 39 planning considerations
upgrading 5.3 full-text system, 54 grammatical normalization, 14
Index
ports full-text indexing installation, 27

index agent, 21 reindexing, 74
index server, 21 resource agent URL for index agent, 65
preinstallation requirements resuming suspended indexing, 73
environment variables, 23 to 24 right-to-left languages
full-text indexing, 21 limitations, 129
host names, 21
HP-UX parameter values, 30
index agent, 21
S
index server, 21 searchable characters, 127
operating system and host, 21 searching
upgrading, 53 phonetic, 13
user accounts, 23 server.ini file
procserver. See DocProcessor full-text indexing entires, 120
properties special characters, 127
index servers, 82 Status of the Index job, 69
punctuation marks, in full-text index, 127 Status Server, described, 111
suspended mode, index server, 112
suspending indexing, 73
Q synonym file, 85
QR Server, described, 112 synonym searches, enabling, 33
qtf-config.xnl file, modifying, 33 synonyms, 84
query plug-in, 15 SysObjects
query processing indexing, 13
batches, configuring use of, 36
query results
acl_check_db parameter, 36
T
query retries, configuring, 35 temp_table_batch_size (ft engine config
query subsystem, described, 112 parameter), 36
querying temporary table for query results, 36
configuring permission checking, 36 thesaurus searching, 84
synonym searches, enabling, 33 thesaurus, configuring, 14
queue items time zone settings on host, 22
full-text indexing, 113 timeouts
task_state, 113 full-text indexing, 75
queue items, indexing tracing
creating, 68 full-text queries, 89
described, 66 index agent, 89
removing by status, 67 TRY_FTDQL_FIRST DQL hint, configuring
removing individual, 68 retries, 35
resubmitting failed, 67
resubmitting individual objects, 67 U
status, 66 uninstalling full-text indexing software, 59
viewing, 68 UNIX and Linux installation, 23 to 24
upgrading
R consolidated deployments, 53
reindexing repositories, 74 upgrading 5.3 full-text system, 54
repositories URL for index agent resource agent, 65
associating with a full-text index, 75 utilities
ftintegrity, 99
Index
V W
verification of completeness and Windows host requirement, 22
accuracy, 99
viewing
index server properties, 82

Content Server 6.5 SP2 Full-Text Indexing Deployment and Administration Guide

Uploaded by

Document Informationclick to expand document information

Document Informationclick to expand document information

Copyright:

Available Formats

Content Server 6.5 SP2 Full-Text Indexing Deployment and Administration Guide

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Content Server 6.5 SP2 Full-Text Indexing Deployment and Administration Guide

Uploaded by

Copyright:

Available Formats

EMC ® Documentum ®

Full-text Indexing Deployment and Administration

Part 1 Overview ............................................................................................................. 11

Part 2 Deploy ................................................................................................................. 19

Chapter 3 Install ........................................................................................................... 27

Adding a node ............................................................................................. 49

Chapter 4 Upgrade ....................................................................................................... 53

Chapter 5 Uninstall ...................................................................................................... 59

Part 3 Administer ........................................................................................................... 63

Chapter 7 Manage Full-text Indexing Components ...................................................... 77

Administration tools ......................................................................................... 77

Chapter 8 Back up and Restore Full-text Indexes ........................................................ 93

Chapter 9 Verify Full-text Index Completeness and Accuracy ..................................... 99

Part 4 Reference .......................................................................................................... 109

Normal mode ........................................................................................ 114

Appendix B Sample Output of ftintegrity Utility ............................................................ 123

Appendix D Supported and Unsupported Formats ....................................................... 129

Figure 1. Full-text indexing components .............................................................................. 16

Table 1. Required environment variables ............................................................................ 23

Purpose of the manual

These topics are included:

Using grammatical normalization

Grammatical normalization is enabled during index server installation. Enabling grammatical

Figure 1. Full-text indexing components

Indexing process overview

Caution: Do not install the index server on VMware in a production environment.

Table 1. Required environment variables

Environment Variable Description Required Values

Environment Variable Description Required Values

12. Installing the index server on Windows hosts

Table 2. Checklist for Full-text Indexing

Task For More Information Completed?

For a Content Server upgrade, Chapter 4, Upgrade

Task For More Information Completed?

These topics are included:

Install a high-availability deployment

10. Navigate to the directory where the create_fulltext_objects_ha.ebs script is located:

12. Restart the Content Server.

Installing the index server and the index agent

Configure indexing and querying

Enabling synonym searches

To enable synonym searches:

5. Save the file.

Disabling indexing of specific object types

To remove or add registration for indexing for an object type:

Rendition formats to index

Configuring format objects to specify which renditions

Supported formats and mime_types

Configuring TRY_FTDQL_FIRST query retries on failure

To add additional error codes:

Forcing permissions checks against the repository

Configuring batched returns for non-FTDQL queries

Configuring the index agent

To configure the index agent:

2. Start the index agent configuration program.

11. If you are on UNIX or Linux, navigate to $DOCUMENTUM_SHARED/domains/DctmDomain

Modifying the IndexAgent.xml file to map file

To modify the IndexAgent.xml file and map the file stores: