Configuring Distributed Processing for FTK 3.

White Paper

Version: 3.2 th Published: October 6 , 2010

TableofContents
What is Distributed Processing? ..............................................................................................................................................1

What Additional Software Is Needed to Set Up and Use Distributed Processing? ....................................................................1 Distributed Processing Install Summary Checklist...................................................................................................................1 Performance and Operational Considerations in a Distributed Environment: ...........................................................................2 Environment Setup ..................................................................................................................................................................4 User/Service Account Setup:....................................................................................................................................................4 Mirrored Account Setup ............................................................................................................................................................5 Creating Mirrored User Accounts In Windows XP: ..................................................................................................................5 Creating Mirrored User Accounts In Windows Vista: ..............................................................................................................6 Setting Up Shared Folders .......................................................................................................................................................7 How to Share a Folder in Windows XP: ....................................................................................................................................7 How to Share a Folder in Windows Vista: .................................................................................................................................9 Sample Machine Configuration Template: ..............................................................................................................................10 Installing the Distributed Processing Engines ........................................................................................................................10 Configuring FTK to Utilize Distributed Worker Machines ........................................................................................................12 Testing Worker Nodes ............................................................................................................................................................13 Creating a Case Using Distributed Processing .......................................................................................................................13 Frequently Asked Questions and Troubleshooting ..................................................................................................................15

What is Distributed Processing?

Distributed processing is a functionality that exists within the FTK 3 application that allows users to create a distributed processing cluster with up to four total nodes (workers) 1 local and 3 distributed. These additional processing nodes (workers) will function together in a cluster to increase productivity and decrease overall processing time. How does distributed processing work? Evidence processing tasks assigned to the engine by the user are called Jobs. The FTK application submits the job to the processing engine. Each job is divided into small packets called Work Units. Each work unit is handed to a service called ADProcessor.exe (and ADIndexer, if youve chosen to index), which actually does the work. There are two components in distributed processing: 1. Processing Manager: The Processing Manager embedded in FTK manages Jobs and Work distribution. It also handles status updates and Job progress. 2. Processing Engine: The Processing Engine manages the processing resources of a particular computer/node. Every machine that participates in a processing cluster runs a Processing Engine. It decides how many jobs can be concurrently processed by that node. The processing engine also manages the ADProcessor.exe/ADIndexer.exe that actually do the processing work.

What Additional Software Is Needed to Set Up and Use Distributed Processing?

These additional components are needed for distributed processing with FTK 3, assuming an existing installation. For hardware requirements and system optimization, please refer to the Systems Specifications Guide. .NET 3.5 Service Pack 1 (on the Application ISO, or if connected to the Internet, will attempt to download) o Windows 2008 R2 requires that you manually install 3.5sp1 using the "Roles and Features" tool. AccessData Processing Engine installation executable

This document assumes an existing installation of FTK 3.1.x or newer. The Evidence Processing Engine IS NOT TO BE INSTALLED in distributed mode on the FTK examiner machine.

Distributed Processing Install Summary Checklist

FTK 3 Installed and able to process data. Microsoft .NET 3.5 Service Pack 1 on all nodes in the distributed processing cluster. (Windows 2008 R2 requires that you manually install 3.5sp1 using the "Roles and Features" tool.) Reliable high speed network connectivity between distributed processing nodes (workers) and FTK examiner machine. Reliable high speed network connectivity between Case folder, evidence folder, and Database. The evidence path and the case path must be shared with read / write permissions across the distributed network. This means they have a UNC path (\\servername\share\dir). The name of the Oracle database machine must begin with an alpha character. This requirement is due to a peculiarity of the Oracle database. Document IP address information for all machines in the cluster. Install Distributed Process Engines (DPE) to distributed nodes. Consider disabling Windows auto-update to prevent unannounced machine reboots and turn off Microsoft indexing. You may need to modify anti-virus and/or Windows firewall functionality to ensure distributed communication. Add distributed worker machines to the FTK examiner machine (via FTK interface).

1|Page

BEFORE BEGINNING!
It is important to understand the distributed worker nodes must have the Evidence Processing Engine installed in distributed mode and must not be installed locally with the FTK application. The Evidence Processing Engine installed local to the FTK application system must NOT be installed as distributed. The two cannot co-exist on the same machine. This means that a standalone FTK 3.x AccessData Evidence Processing Engine (EP) cannot participate as a node in a distributed processing cluster and cannot have the Distributed Processing Engine (DPE) installed on it.

Performance and Operational Considerations in a Distributed Environment:

In testing distributed processing improves performance across almost all ranges of processing groups. Processing groups made up of 16+ core machines see improvement, as well as processing groups made up of single-core Pentium 4 machines. 1. The machines that store the evidence and case folder become a bottleneck. Processing evidence is very disk IO intensive. As a result evidence should be stored on fast drives. With many large machines in a processing group, it is possible that the file sharing service in Windows will run out of kernel memory and fail to provide the evidence data across the network. AccessData support can assist in troubleshooting this problem. 2. The machine that runs the Processing Manager may become a bottleneck during the discovery phase. Discovery is the process of enumerating all the actual files in a piece of evidence. Information about these files is stored in the database and the Distributed Processing Engines work on processing them. This discovery phase always runs in the Processing Engine located on the same machine as the Processing Manager. Since it produces much of the work that other Processing Engines work on, it needs to be one of the fastest machines (CPU speed) in the processing group. 3. Distributed processing produces a lot of network traffic. There is control traffic between the engine components, but primarily the network is used to read evidence and write results to the case folder and database. It is very easy to saturate a gigabit network for extended periods of time while processing a large image. Please use the fastest network technologies available to you, at a minimum 100 Mb switched. We strongly recommend that the Case folder and image location are on separate drives. 4.

2|Page

Sample Configuration 1

This environment represents a standard FTK installation on a single examiner machine with evidence files, case files and the database residing together with 3 distributed processing nodes for a total of four nodes in the cluster.

Sample Configuration 2

This high performance configuration represents a four node distributed processing cluster with 1 local worker EP (evidence Processing engine) and three distributed processing workers (DPE). In addition the database is on a dedicated server with evidence/case folder on a separate source as well.

3|Page

Environment Setup
The following configuration steps assume a Windows XP or Vista operating system and existing network connectivity between all machines. Any of the FTK 3 supported operating systems can be used, but only XP and Vista are outlined this document. For Windows 2003/2008/7 the requirements are the same. Depending on your environment and operational policies, consider working with IT for assistance creating an infrastructure with the appropriate permission. Using the above environment setup, gather the following information from each node/machine: 1. Take note of all IP Addresses in the cluster: Node 1 (FTK/Processing Manager): Node 2 (DPE): Node 3 (DPE): Node 4 (DPE): To obtain IP address:
1. Click on the Start or Windows button and select the "Run" option. 2. In the Run box, type the command "cmd" as illustrated below.

3. In the resulting command box, utilize the "ipconfig /all" command to determine the IP address of the network connection adaptor (if unfamiliar with network addressing, consult IT or the network administrator for assistance with this). 4. Take note of the IP address for all nodes that are part of the cluster.

User/Service Account Setup:

Every node in the cluster requires read/write access to the case and evidence folders. Here are two approaches to ensure the service account is setup properly. Non Domain: One way to ensure that the DPE nodes can properly communicate with the FTK examiner machine hosting the cases and evidence folders is to have all machines logged in to Windows with mirrored user accounts. This means all users log in with the same user name and password. Create Administrator privileged user accounts with the same user name and account passwords on all machines in the environment via the following steps: See the FAQ #8, #9 and #10 at the end of this document for important notes regarding using mirrored accounts vs. individual accounts and installing in a domain environment. Domain: Another way to ensure that the distributed worker nodes can properly communicate with the FTK examiner node hosting the cases and evidence folders is to create a domain user account such as FTKService to be used for the distributed processing service. Once that service account has been created, it should be added to the local administrators group on all nodes that are part of the cluster.
4|Page

Mirrored Account Setup Creating Mirrored User Accounts In Windows XP / Server 2003:
1. 2. 3. 4. 5. 6. 7. Click on the Start or Windows button and select the "Control Panel" option. In the Control Panel, select the "User Accounts" application as illustrated (#1). In User Accounts, select "Create a new account" as illustrated (#2). In User Accounts, name the new account as illustrated (#3). In User Accounts, pick the account type of "Administrator" as illustrated (#4). Once the account is created, select Change an account and choose the user you just created. Then select Create a password, and create your password.

NOTE: Mirrored account MUST have a password. Accounts without passwords will not have sharing enabled. Figure 1: Figure 2:

Figure 3:

Figure 4:

5|Page

Creating Mirrored User Accounts In Windows Vista / Win 7:

1. 2. 3. 4. 5. Click on the Start or Windows button and select the "Control Panel" option. In the Control Panel, select the "User Accounts" application as illustrated (#1). In User Accounts, select "Manage another account" as illustrated (#2). In Manage Accounts, select "Create New Account" as illustrated (#3). In Create New Account, name the account and select the account type of "Administrator" as illustrated (#4). 6. Once the account is created, select Change an account and choose the user you just created. 7. Then select Create a password, and create your password.

NOTE: Mirrored account MUST have a password. Accounts without passwords will not have sharing enabled.
Figure1:

Figure2:

Figure3:

Figure4:

6|Page

Setting Up Shared Folders

NOTE: Any node (worker) participating in the cluster requires read/write access to the cases folder and evidence folder(s) on whichever machine/data source they reside. This access must be unimpeded, and these folders must be shared with read /write permissions. 1. Create a folder for the evidence if you do not have one already. 2. Create a folder for Cases if you do not already have one. 3. Share the evidence and case folders following the directions below or following your normal procedures.

How to Share a Folder in Windows XP:

NOTE: In order for the options listed in the sharing steps on the following pages to be available, you must first turn off Simple File Sharing, which is illustrated below:

7|Page

How to Share a Folder in Windows XP (continued):

1. 2. 3. 4. 5. 6. 7. 8. 9. Enable Network File Sharing if not already enabled. Locate your case and evidence folders in your networked environment. Right click on the folders and select "Sharing and Security...". In the next window, select "Share this folder" as illustrated (#1) [This option will not be displayed if Simple File Sharing is still enabled.] Click the "Permissions" box (#1). Add the appropriate user. Highlight the user in either mirrored account or domain service account. Then select the box to allow "Full Control" as illustrated (#2). This will automatically select "Change" and "Read" as well. Click the "Apply" button on all windows as steps are completed.

Figure1:

Figure2:

SHARED!

8|Page

How to Share a Folder in Windows Vista / Win 7:

1. Enable Network File Sharing if not already enabled. You will find the option in the Sharing and Discovery section of the Network and Sharing Center of the Windows Control Panel. 2. Locate your case and evidence folders in your networked environment. 3. Right click on the folder and select "Properties". 4. In the next window, select "Sharing" tab as illustrated (#1). 5. In the next window, click the "Advanced Sharing" box as illustrated (#1). 6. In the next window, select the "Share this folder" option as illustrated (#2). 7. Add the appropriate user. 8. Highlight the user either mirrored account or domain service account. 9. Click the "Permissions" box (#2) and select the desired user. 10. Select the box to allow "Full Control" as illustrated (#3). This will automatically select "Change" and "Read" as well. 11. Click the "Apply" button on all windows as steps are completed.
Figure1:

Figure2:

Figure3:

SHARED!

9|Page

Sample Machine Configuration Template:

At this stage, the environment should be in a state where all machines can properly communicate with each other using mirrored administrative accounts or a domain service account. IP addresses have been documented. Use this template to assist documenting that information if desired: IP Addresses of nodes in the cluster:

Node 1 (FTK/Processing Manager): Node 2 (DPE): Node 3 (DPE): Node 4 (DPE):

UNC Path to Evidence: For example: (\\servername\share\dir) UNC Path to Case Folder: For example: (\\servername\share\dir) Service Account/Mirrored User: NOTE: Make sure the user/service account has read / write access to the evidence and case folder.

Installing the Distributed Processing Engines

Copy the AccessData Distributed Processing Engine (DPE) installer to the distributed worker machines. On each machine in the cluster, follow these installation steps: 1. Launch the AccessData Processing Engine installer (see See figure #1 on next page). (Depending on your current machine configuration you may also have to install the .NET 3.5 SP1 and / or the Microsoft Windows Installer 4.5). 2. Click "Next" when prompted as illustrated (See figure #2 on next page). 3. Accept the "License Agreement" - click "Next" as illustrated (See figure #3 on next page). 4. Select the Install as distributed processing engine check box. Note the Destination Folder and click "Next" as illustrated (See figure #4 on next page). 5. Input the Administrative account name and password for the distributed machine. This must be the account that has read / write access to the shared case and evidence folders. If the machine is on a domain, enter the domain name - leave "Domain" blank if none is present - click "Next" as illustrated (See figure #5 on next page). 6. Click "Install" to begin installation (See figure #6 on next page) and "Finish" when complete.

10|Page

Figure1

Figure2

Figure3

Figure4

Figure5

11|Page

Configuring FTK to Utilize Distributed Worker Machines

When all distributed worker machines have been configured and distributed processing engines have been installed, they can be added to the FTK client. To configure FTK to use the distributed processing workers: 1. Log in with application administrator permissions. 2. In the FTK Management window, select the menu option: "Tools \ Processing Engine Configuration" 3. In the "Add engine" section, input the IP addresses of the distributed machines as illustrated (#2). 4. Click "Add" after inputting each distributed machine IP address - each machine will populate in the "Computer Name/IP" list as the Add button is clicked. 5. Close the Processing Engine Configuration window when complete.
Figure1:

as illustrated (#1).

Figure2:

12|Page

Testing Worker Nodes

Now is a good time to test accessing the case and evidence folder via UNC path from each of the worker nodes that are part of the cluster. When logging on with the proper username or service account and trying to access the evidence of a case folder, if the UNC path is not successful, then there is a permission issue that should be resolved prior to starting processing. The easiest way to test this is by doing the following: 1.) Reboot all machines in the cluster to clear any existing authentication. 2.) From each of the remote processing nodes, log on as the newly created mirrored account (or domain account). 3.) Click on the Start or Windows button and select the Run option. 4.) In the Run Box type \\<servername\share> where servername is the hostname of the machine hosting the evidence and share is the name of the evidence share. 5.) You should now have access to that location WITHOUT having to enter a username or password. If you are prompted for credentials, you will need to resolve that prior to continuing. It is not a bad idea to create a simple text file at this location just to ensure you have write permissions. 6.) Repeat steps 3 through 5 for the case folder path and on each of the remaining processing nodes.

Creating a Case Using Distributed Processing

1. Define the Case Folder Directory. a. It is a requirement that the "Case Folder Directory" be a UNC path to its location on the hosting machine in the distributed environment. Remember, this folder must be shared with read / write permissions (configured earlier in this document). Utilize the three dot ellipse on the "New Case Options" console to navigate to and test the UNC path of the shared Case Folder Directory as illustrated (#1). b. Choose your processing options.

Figure1 c. Press OK.

13|Page

2. Add evidence to the case. a. In the "Manage Evidence" console, it is a requirement that the evidence "Path" be a UNC path to its location on the hosting machine in the distributed environment. Remember, this folder must be shared with read / write permissions (configured earlier in this document). When selecting the evidence to add to the case, be sure to browse to the evidence location via the network path when adding that evidence. In the "Manage Evidence" console, validate the UNC path of the shared evidence location as illustrated (#2).

Figure2

3. Add each piece of evidence and ensure it has the proper path. 4. Press OK to initiate processing.

NOTE: The distributed machines will not immediately commence working on a case. New processes named "ADProcessor.exe" and "ADIndexer.exe" will spawn on the distributed worker machines and can be observed in the distributed worker machine process lists via the Windows Task Manager. To view these process, you must select Show all processes from all users as illustrated below. As the FTK examiner machine begins to enumerate (discover) items in the case, the enumeration builds to a point where the items are pushed in to the Oracle database. At that stage, the enumerated items become "things to do" and will be assigned out to the distributed worker machines previously configured in the FTK client. As these "item pushes" continue and as the case "discovery" increases, the distributed worker machines continue to function until case processing is complete.

14|Page

Frequently Asked Questions and Troubleshooting

1.) Q: How can I tell the difference between a normal Evidence Processing Engine installation and a Evidence Processing Engine in distributed mode installation? A: Check the Windows Services management console snap-in. If the AccessData Processing Engine Service is listed, the Evidence Processing Engine on that system was installed as distributed. If the service is not listed, the engine was installed in normal mode. 2.) Q: The Distributed Processing Engine installation fails or the installed service fails to start because it needs appropriate logon rights - how is this overcome? A: Follow these steps:
Right click on the "My Computer" icon in Windows XP (the "Computer" icon in Windows Vista). Select "Manage" from the right click menu. In the Computer Management Console, expand "Services and Applications" and click on "Services" as illustrated (#1). The machine service list will populate. Locate the "AccessData Processing Engine Service" and view the "Properties" dialog box by double clicking on the AccessData Processing Engine Service (#1). In the Properties window, select the "Log On" tab. Verify the correct Administrative logon credentials for the service (#2). Stop and Restart this service after performing any credential update (#1).

-- OR -If the failure was during installation, click "Retry" on the installation window.

NOTE: When deploying, the credentials used by this service need to be part of the Administrator control group and must have read / write access to the shared case and evidence folders.

Figure1

Figure2

15|Page

3.) Q: What is the option to "Maintain UI performance when processing"?

A: You can make either processing performance or UI performance the priority. By default, FTK is configured to optimize processing speed by creating indexes later in the process. This can cause searching for items while the case is still processing to be slow or unresponsive. You can change this under Tools > Processing Engine Config, with the check box at the bottom. If you are using distributed processing, a registry change needs to be made on the processing nodes as well. A .REG file called ProcessWithIndexes.reg has been provided that can be run on the distributed processing computers that will make these registry changes for you. This is located in the FTK folder on the Application disk. Alternatively, the following two registry keys can be added to the distributed processing computers (remove these to undo the modification):
HKEY_LOCAL_MACHINE\SOFTWARE\AccessData\Products\Forensic Toolkit\3.0\ProcessWithIndexes (dword = 00000001) HKEY_LOCAL_MACHINE\SOFTWARE\AccessData\Products\Forensic Toolkit\3.0\UsePlainBuffers (value = ON)

4.) Q: What if a distributed machine does not have the .NET 3.5 SP1?

A: The installer will try and download it from the Internet or read it from the Application ISO. However, Windows 2008 R2 machines require that you manually install 3.5sp1 using the "Roles and Features" tool. A: Remove: removes a worker from the list. Enable: enables a worker. Disable: disables a worker (it will not receive work until re-enabled).

5.) Q: In the FTK interface, what do the Processing Engine Configuration Options mean?

6.) Q: Distributed processes spawn in a distributed manner, but don't engage. What is wrong?

A: The most common problem in this scenario is pointing the case and / or evidence paths to local machine locations rather than the proper, fully shared, UNC path locations. Remember also, that a threshold of items must be enumerated and pushed in to the database before distributed workers will assist the FTK examiner machine. Also, check the entered IP addresses to ensure proper connectivity between the FTK examiner machine and distributed worker machines. Make sure all machines are in the same WORKGROUP or Domain when possible. A: Allow a firewall exception to the port used by the distributed worker machines (default = 34097). A: Yes, as long as proper name resolution is occurring on the network.

7.) Q: What other things can be done to fix issues preventing distributed processing? 8.) Q: Can machine names be used instead of machine IP addresses? 9.) Q: Is it necessary to use mirrored accounts?

A: In a non-domain environment, this is the easiest way to facilitate machine communication. In a domain, a service account can be used. A: Yes, but they may not receive work until the next push of enumerated items.

10.) Q: Can distributed worker machines be added to FTK while a case is processing? 11.) Q: What if the distributed network environment is using stand-alone machines to separately host the Oracle database and evidence files?

A: The Oracle database configuration would have presumably been taken care of during the FTK examiner installation and the evidence files on the evidence host machine would need to be configured according to this document - SHARED to the distributed worker machines, with full read / write access permissions (bet you never heard that before?).

16|Page

12.) Q: Do I have to use four workers in the cluster to take advantage of distributed processing?
A: No, you can use 1, 2, or 3 additional nodes in the cluster for a total of up to four.

13.) Q: Is it necessary for the distributed worker machine accounts to be logged on for a distributed worker machine engine to process data?

A: No, as long as the Distributed Processing Service Engine is installed with administrative credentials, and those credentials have full read / write permissions to the cases and evidence files folders on the FTK examiner machine.

Inthisgraphic,ADProcessorisrunningasa service(Ken)thatisnottheloggedonservice user(Alice).

14.) Q: What if the existing network for distributed implementation is domain-based?

A: Add these steps to your installation process: When installing the Distributed Processing Engines, list your domain name at this step of the installation:

Besuretoinstalltheenginewithadomain baseduseraccount. Ensure the domainbased user account is a member of the administrator group on the localmachine Ensurethatdomainbaseduseraccounthas read/writepermissionstothecaseand evidencefolder.

TIP: Having trouble getting started? The JobInformation.log file is where database connection errors and similar items are logged. PLEASE CONTACT ACCESSDATA SUPPORT WITH FURTHER QUESTIONS 801.377.5410 and select Option 5 800. 658.5199 (N. America) support@accessdata.com >

17|Page