Getting Started Guide for Windows HPC Server 2008 Beta 2

Microsoft Corporation Published: May 2008

Abstract
This guide provides information for deploying an HPC cluster using the Beta 2 release of Windows HPC Server 2008. It includes step-by-step instructions to deploy and configure the head node, add compute nodes to the cluster, and verify that your cluster deployment has been successful. It also includes an overview of some of the new technologies available in Windows HPC Server 2008 Beta 2.

Copyright Information
This document supports a preliminary release of a software product that may be changed substantially prior to final commercial release, and is the confidential and proprietary information of Microsoft Corporation. It is disclosed pursuant to a non-disclosure agreement between the recipient and Microsoft. This document is provided for informational purposes only and Microsoft makes no warranties, either express or implied, in this document. Information in this document, including URL and other Internet Web site references, is subject to change without notice. The entire risk of the use or the results from the use of this document remains with the user. Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, e-mail address, logo, person, place, or event is intended or should be inferred. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation. Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. 2008 Microsoft Corporation. All rights reserved. Microsoft, Active Directory, Excel, Windows, Windows PowerShell, Windows Server, and Windows Vista are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. All other trademarks are property of their respective owners.

Contents
Getting Started Guide for Windows HPC Server 2008 Beta 2 ........................................................ 5 What's New in Windows HPC Server 2008 Beta 2 ...................................................................... 5 Compatibility with Previous Versions ........................................................................................... 9 Additional Considerations ............................................................................................................ 9 Checklist: Deploy an HPC Cluster - Overview .............................................................................. 10 Step 1: Prepare for Your Deployment ........................................................................................... 10 Checklist: Prepare for Your Deployment .................................................................................... 10 1.1. Review the System Requirements ...................................................................................... 11 1.2. Decide How to Add Compute Nodes to Your Cluster ......................................................... 13 1.3. Choose the Active Directory Domain for Your Cluster ........................................................ 13 1.4. Choose a User Account for Installation and Diagnostics .................................................... 14 1.5. Choose a Network Topology for Your Cluster .................................................................... 14 1.6. Prepare for Multicast (Optional) .......................................................................................... 15 Step 2: Deploy the Head Node ...................................................................................................... 15 Checklist: Deploy the Head Node .............................................................................................. 15 2.1. Install Windows Server 2008 on the Head Node Computer ............................................... 16 2.2. Join the Head Node Computer to a Domain ....................................................................... 16 2.3. Install Microsoft HPC Pack 2008 on the Head Node Computer ......................................... 16 Step 3: Configure the Head Node ................................................................................................. 17 Checklist: Configure the Head Node .......................................................................................... 17 3.1. Configure the HPC Cluster Network ................................................................................... 18 3.2. Provide Installation Credentials........................................................................................... 19 3.3. Configure the Naming of New Nodes ................................................................................. 20 3.4. Create a Node Template ..................................................................................................... 21 3.5. Add Drivers for the Operating System Images (Optional) .................................................. 22 3.6. Add or Remove Users (Optional) ........................................................................................ 23 Step 4: Add Compute Nodes to the Cluster .................................................................................. 23 4.1. Deploy Compute Nodes from Bare Metal ........................................................................... 24 4.2. Add Compute Nodes by Importing a Node XML File .......................................................... 25 4.3. Add Preconfigured Compute Nodes ................................................................................... 25 4.4. Monitor Deployment Progress............................................................................................. 27 Step 5: Run Diagnostic Tests on the Cluster ................................................................................ 28 Step 6: Run a Test Job on the Cluster .......................................................................................... 29 Checklist: Run a Test Job on the Cluster ................................................................................... 29 6.1. Create a Job Template........................................................................................................ 29 6.2. Create and Submit a Job .................................................................................................... 30

6.3. Create and Submit a Job Using the Command-Line Interface (Optional) .......................... 31 6.4. Create and Submit a Job Using the HPC PowerShell (Optional) ....................................... 32 HPC Cluster Security ..................................................................................................................... 33 Groups and Authorized Operations............................................................................................ 33 HPC Cluster Credentials ............................................................................................................ 35 Application Firewall Exceptions.................................................................................................. 37 Access for Managed Code Applications .................................................................................... 38 Important Features in Windows HPC Server 2008 Beta 2 ............................................................ 38 The Service Oriented Application (SOA) Programming Model and Runtime ............................ 38 Access Your HPC Cluster Using a Web Service ....................................................................... 39 HPC Cluster Reporting ............................................................................................................... 40 Using the HPC PowerShell ........................................................................................................ 41 Additional Resources ..................................................................................................................... 42 Appendix 1: HPC Cluster Networking ........................................................................................... 43 HPC Cluster Networks ............................................................................................................... 43 Supported HPC Cluster Network Topologies ............................................................................. 44 HPC Network Services ............................................................................................................... 52 Windows Firewall Configuration ................................................................................................. 53 Appendix 2: Creating a Node XML File ......................................................................................... 54 Benefits of Using a Node XML File for Deployment................................................................... 54 How to Create a Node XML File ................................................................................................ 55 Sample Node XML File .............................................................................................................. 61

[This is prerelease documentation and is subject to change in future releases.]

Getting Started Guide for Windows HPC Server 2008 Beta 2

This document provides procedures and guidance for installing a high performance computing cluster using the Beta 2 release of Windows HPC Server 2008.

What's New in Windows HPC Server 2008 Beta 2

The names used to refer to this product and product components have changed in this version of the product. The changes are summarized as follows: Windows Server 2008 HPC Edition has replaced Windows Server 2003, Compute Cluster Edition. Windows HPC Server 2008 has replaced Windows Compute Cluster Server 2003. Microsoft HPC Pack 2008 has replaced Microsoft Compute Cluster Pack 2003. Important In this beta release, some text that is displayed on the user interfaces might not exactly match references made to that text, or might not reflect the naming conventions used in this document.

Feature Overview
Feature Compute Cluster Server 2003 Windows HPC Server 2008 Beta 2

Administrative tools

Cluscfg command-line tool for configuration. Clusrun command-line tool for running commands remotely on one or more nodes. Compute Cluster Administrator snap-in enabled simpler completion of essential tasks such as configuring cluster networking topology, setting user permissions, and monitoring jobs.

Cluscfg command-line tool for configuration. Clusrun command-line tool for running commands remotely on one or more nodes. The administrator console in Windows HPC Server 2008 Beta 2 (HPC Cluster Manager) is now based on Microsoft System Center. The new interface integrates all aspects of cluster management and simplifies the completion of essential tasks such as the configuration of cluster
5

[This is prerelease documentation and is subject to change in future releases.]

Feature

Compute Cluster Server 2003

Windows HPC Server 2008 Beta 2

Remote Installation Services (RIS) is used when deploying nodes with the automatic method.

networking, setting of user permissions, and monitoring of jobs and the operational health of the cluster. New features, such as node templates that utilize Windows Deployment Services, significantly improve the deployment of compute nodes. Cluster administrators can create node groups to view and manage collections of nodes. A new heat map view feature enables cluster administrators to view cluster health at a glance. New cluster and node diagnostic testing. Built-in extensible reporting. Ability to configure two head nodes for high availability so that jobs can continue running in the case of head node failures, and new jobs can be submitted within a short period of time after a failure. Support for cluster updates, including the ability to specify when nodes are updated so that running jobs are not interrupted. Support for all Windows Compute Cluster Server 2003 (CCS 2003) command-line interface (CLI) commands. New user interface for managing jobs (HPC Job Manager), with improved support for parametric commands. PowerShell commands for
6

Cluster user tools

Command-line interface Compute Cluster Job Manager snap-in made submission and tracking of user jobs simple.

[This is prerelease documentation and is subject to change in future releases.]

Feature

Compute Cluster Server 2003

Windows HPC Server 2008 Beta 2

scheduling and managing jobs. Windows HPC Server 2008 Beta 2 is fully compatible with existing CCS 2003 commands and also offers full support of PowerShell. More than 130 command-line tools enable the automation of system administration tasks. Networking Wizard-driven configuration of network services such as Dynamic Host Configuration Protocol (DHCP), Internet Connection Sharing (ICS), Remote Installation Services (RIS) and firewall settings. Custom job view filtering. New wizard-driven configuration of network services such as DHCP, network address translation (NAT), Domain Name System (DNS), and firewall settings. Support for NetworkDirect, a new Remote Direct Memory Access (RDMA) networking interface built for speed and stability. Programming interface identical to MPICH2 Improved cluster efficiency to shorten solution time Integrated with NetworkDirect, a new, high-speed Remote Direct Memory Access (RDMA) networking interface for Windows New implementation of shared memory communications that provides better support for nodes with many cores. Resource matching allows jobs to specify the types of resources on which they need to run Job templates for setting constraints and requirements on
7

Message Passing Interface (MPI)

Programming interface identical to MPICH2

Job scheduling and job policies

"First-come, first-served" scheduling Backfilling of jobs Exclusive scheduling License Scheduling

[This is prerelease documentation and is subject to change in future releases.]

Feature

Compute Cluster Server 2003

Windows HPC Server 2008 Beta 2

jobs, including the ability to route jobs to specific sets of resources. Job templates also simplify enduser job submission by enabling cluster administrators to specify default job values. Multilevel resource allocation allows jobs to take better advantage of multi-core systems by requesting resources at a granularity appropriate for their performance characteristics. Adaptive allocation of running jobs allows jobs with multiple tasks to shrink and grow as resources become available and work is completed. Pre-emption allows high priority jobs to start sooner by taking resources away from lower priority jobs.

Developer tools

.NET (C#) and COM application programming interface (API) for the CCS 2003 Job Scheduler Microsoft Message Passing Interface (MS-MPI)

Supports for all COM APIs in CCS 2003 Service Oriented Application (SOA) programming platform Scalable Microsoft HPC Pack 2008 API with eventing support MS-MPI events are now integrated with Event Tracing for Windows. This enhancement helps you to troubleshoot and fine tune the performance of your applications without having to use special trace or debug builds. For more information about the new MS-MPI tracing capability, see the
8

[This is prerelease documentation and is subject to change in future releases.]

Feature

Compute Cluster Server 2003

Windows HPC Server 2008 Beta 2

Windows HPC Server 2008 Software and Driver Development Kit (SDK). This SDK is available for download at the Windows HPC Server 2008 site on Connect (http://go.microsoft.com/fwlink/?Li nkID=119579). Customizable default job template

Compatibility with Previous Versions

The following list describes compatibility between Windows HPC Server 2008 Beta 2 and Windows Compute Cluster Server 2003: Windows HPC Server 2008 Beta 2 provides API-level compatibility for applications integrated with Windows Compute Cluster Server 2003. These applications might, however, require changes to run on Windows Server 2008. If you encounter problems running your application on Windows Server 2008, you should consult your software vendor. Windows HPC Server 2008 Beta 2 supports job submission from Windows Compute Cluster Server 2003 clients, including the command-line tools, the Compute Cluster Job Manager, and the COM APIs. The Windows HPC Server 2008 Beta 2 client tools, including the cluster administration console (HPC Cluster Manager), the job scheduling console (HPC Job Manager), the command-line tools, and the APIs cannot be used to manage or submit jobs to a Windows Compute Cluster Server 2003 cluster. Clusters that have Windows Compute Cluster Server 2003 nodes and Windows HPC Server 2008 Beta 2 nodes are not supported.

Additional Considerations
A side-by-side installation of Windows HPC Server 2008 Beta 2 and Windows Compute Cluster Server 2003 on the same computer is not supported. This includes the Windows HPC Server 2008 Beta 2 client utilities. The upgrade of a Windows Compute Cluster Server 2003 head node to a Windows HPC Server 2008 Beta 2 head node is not supported. The installation of HPC Pack 2008 adds the following server roles to the head node: DHCP Windows Deployment Services
9

[This is prerelease documentation and is subject to change in future releases.]

File Services Network Policy and Access Services, which enables Routing and Remote Access so that NAT services can be provided to the cluster nodes.

Checklist: Deploy an HPC Cluster - Overview

The following checklist describes the overall process of deploying a Windows HPC Server 2008 Beta 2 cluster. Each task in the checklist is linked to the section in this document that describes the specific steps to perform the task.
Task Reference

Before you start deploying your HPC cluster, review the list of prerequisites and initial considerations. Deploy the head node by installing Windows Server 2008 and Microsoft HPC Pack 2008. Configure the head node by following the steps in the configuration to-do list.

Step 1: Prepare for Your Deployment

Step 2: Deploy the Head Node

Step 3: Configure the Head Node

Add nodes to the cluster by deploying them Step 4: Add Compute Nodes to the Cluster from bare metal, by importing an XML file, or by manually configuring them Run diagnostic tests to verify that the deployment of the cluster was successful Run some basic jobs on the cluster to verify that the cluster is operational Step 5: Run Diagnostic Tests on the Cluster

Step 6: Run a Test Job on the Cluster

Step 1: Prepare for Your Deployment

The first step in the deployment of your HPC cluster is to make important decisions, such as deciding how you will be adding nodes to your cluster, and choosing a network topology for your cluster. The following checklist describes the steps involved in preparing for your deployment.

Checklist: Prepare for Your Deployment

10

[This is prerelease documentation and is subject to change in future releases.]

Task

Reference

Review the list of system requirements to ensure that you have all the necessary hardware and software components to deploy an HPC cluster. Decide if you will be adding compute nodes to your cluster from bare metal, from preconfigured nodes, or from an XML file. Choose the Active Directory domain to which you will join the head node and compute nodes of your HPC cluster. Choose an existing domain account with enough privileges to perform installation and diagnostics tasks. Choose how the nodes in your cluster will be connected, and how the cluster will be connected to your enterprise network. If you will be deploying nodes from bare metal and would like to multicast the operating system image that you will be using during deployment, configure your network switches appropriately.

1.1. Review the System Requirements

1.2. Decide How to Add Compute Nodes to Your Cluster

1.3. Choose the Active Directory Domain for Your Cluster

1.4. Choose a User Account for Installation and Diagnostics

1.5. Choose a Network Topology for Your Cluster

1.6. Prepare for Multicast (Optional)

1.1. Review the System Requirements

The following sections list the hardware and software requirements for Windows HPC Server 2008 Beta 2.

Hardware Requirements
Hardware requirements for Windows HPC Server 2008 Beta 2 are very similar to those for the 64-bit editions of Windows Server 2008. Note For more information about installing Windows Server 2008, including system requirements, see http://go.microsoft.com/fwlink/?LinkID=119578.

11

[This is prerelease documentation and is subject to change in future releases.]

Processor (x64-based):
Minimum: 1.4 GHz Recommended: 2 GHz or faster

RAM:
Minimum: 512 MB Recommended: 2 GB or more

Available disk space:

Minimum: 50 GB Recommended: 80 GB or more

Drive:
DVD-ROM drive

Network adapters:
The number of network adapters on the head node and on the compute nodes depends on the network topology that you choose for your cluster. For more information, see Appendix 1: HPC Cluster Networking.

Software Requirements
The following list outlines the software requirements for the head node and the compute nodes in a Window HPC Server 2008 Beta 2 cluster: One of the 64-bit editions of Windows Server 2008. Microsoft HPC Pack 2008 Beta 2, which can be downloaded from the Windows HPC Server 2008 site on Connect (http://go.microsoft.com/fwlink/?LinkID=119579).

To enable users to submit jobs to your HPC cluster, you can install the utilities included with Microsoft HPC Pack 2008 Beta 2 on client computers. Those client computers must be running any of the following operating systems: Windows XP Professional with Service Pack 3 or later (x86- or x64-based) Windows Vista Enterprise, Windows Vista Business, Windows Vista Home, and Windows Vista Ultimate Windows Server 2003 Standard Edition or Windows Server 2003 Enterprise Edition with Service Pack 2 or later (x86- or x64-based) Windows Server 2003, Compute Cluster Edition Windows Server 2003 R2 Standard Edition or Windows Server 2003 R2 Enterprise Edition (x86- or x64-based)
12

[This is prerelease documentation and is subject to change in future releases.]

1.2. Decide How to Add Compute Nodes to Your Cluster

There are three ways to add compute nodes to your cluster: From bare metal. The operating system and all the necessary HPC cluster components are automatically installed on each compute node as it is added to the cluster. No manual installation of the operating system or other software is required. Add preconfigured compute nodes. The compute nodes are already running one of the 64bit editions of the Windows Server 2008 operating system, and Microsoft HPC Pack 2008 is manually installed on each node. Import a node XML file. An XML file that contains a list of all the nodes that will be deployed is used. This XML file can be used to add nodes from bare metal or from preconfigured nodes. For more information about node XML files, see Appendix 2: Creating a Node XML File.

The following is a list of things to take into consideration when choosing how to add nodes to your HPC cluster: When deploying nodes from bare metal, Windows HPC Server 2008 Beta 2 automatically generates computer names for your compute nodes. During the configuration process, you will be required to specify the naming convention to use when automatically generating computer names for the new nodes. Compute nodes are assigned their computer name in the order that they are deployed. If you want to add compute nodes from bare metal and assign computer names in a different way, you can use a node XML file. For more information about node XML files, see Appendix 2: Creating a Node XML File. If you want to add preconfigured nodes to your cluster, you will need to install one of the 64bit editions of the Windows Server 2008 operating system on each node (if not already installed), as well as Microsoft HPC Pack 2008.

1.3. Choose the Active Directory Domain for Your Cluster

The head node and the compute nodes in your HPC cluster must be members of an Active Directory domain. Before deploying your cluster, you must choose the Active Directory domain that you will use for your HPC cluster. If you do not have an Active Directory domain to which you can join your cluster, or if you prefer not to join an existing domain, you can install Active Directory Domain Services on the head node. For more information about installing Active Directory Domain Services on a computer running Windows Server 2008, see http://go.microsoft.com/fwlink/?LinkID=119580.

13

[This is prerelease documentation and is subject to change in future releases.]

Important If you choose to install Active Directory Domain Services on the head node, consult with your network administrator about the correct way to isolate the new Active Directory domain from the enterprise network, or how to join the new domain to an existing Active Directory forest.

1.4. Choose a User Account for Installation and Diagnostics

During the configuration process of your HPC cluster, you must provide credentials for a domain user account that will be used for installation and diagnostics. You must choose an existing account or create a new account, before starting your cluster deployment. The user account that you choose: Must be a domain account with enough privileges to create Active Directory Domain Services accounts for the compute nodes. If part of your deployment requires access to resources on the enterprise network, the account must have the necessary permissions to access those resources. If you want to restart nodes remotely from the cluster administration console (HPC Cluster Manager), the account must be a member of the local Administrators group on the head node. This requirement is only necessary if you do not have Intelligent Platform Management Interface (IPMI) tools that you can use to remotely restart the compute nodes.

1.5. Choose a Network Topology for Your Cluster

Windows HPC Server 2008 Beta 2 supports five cluster topologies. These topologies are distinguished by how the compute nodes in the cluster are connected to each other and to the enterprise network. The five supported cluster topologies are: Topology No. 1: Compute Nodes Isolated on a Private Network Topology No. 2: All Nodes on Enterprise and Private Networks Topology No. 3: Compute Nodes Isolated on Private and Application Networks Topology No. 4: All Nodes on Enterprise, Private, and Application Networks Topology No. 5: All Nodes on an Enterprise Network

For more information about each network topology, see Appendix 1: HPC Cluster Networking. When you are choosing a network topology: Decide which cluster network in the topology that you have chosen will serve as the enterprise network, the private network, and the application network.

14

[This is prerelease documentation and is subject to change in future releases.]

Do not have the network adapter that is connected to the enterprise network on the head node in automatic configuration (that is, the IP address for that adapter does not start with: 169.254). That adapter must have a valid IP address, dynamically or manually assigned (static). If you choose a topology that includes a private network, and you are planning to add nodes to your cluster from bare metal, ensure that there are no active DHCP or Pre-boot Execution Environment (PXE) servers on that network. Please contact your system administrator to determine if IPSec is enforced on your domain through Group Policy. If IPSec is enforced on your domain through Group Policy, you may have problems during deployment. A workaround is to make your head node an IPSec boundary server so that compute nodes are allowed to talk to the head node at PXE boot.

1.6. Prepare for Multicast (Optional)

If you will be deploying nodes from bare metal and would like to multicast the operating system image that you will be using during deployment, we recommend that you prepare for multicast by: Enabling Internet Group Management Protocol (IGMP) snooping on your network switches, if this feature is available. This will help to reduce multicast traffic. Disabling Spanning Tree Protocol (STP) on your network switches, if this feature is enabled. Note For more information about these settings, contact your network administrator or your networking hardware vendor.

Step 2: Deploy the Head Node

The next step in the deployment of your HPC cluster is to deploy the head node. The following checklist describes the steps involved in deploying the head node.

Checklist: Deploy the Head Node

Task Reference

Install one of the 64-bit editions of the Windows Server 2008 operating system on the computer that will act as the head node. Join the computer that will act as the head node to a Microsoft Active Directory Domain. Install Microsoft HPC Pack 2008 on the

2.1. Install Windows Server 2008 on the Head Node Computer

2.2. Join the Head Node Computer to a Domain

2.3. Install Microsoft HPC Pack 2008 on the

15

[This is prerelease documentation and is subject to change in future releases.]

Task

Reference

computer that will act as the head node, using the installation media or from a network location.

Head Node Computer

2.1. Install Windows Server 2008 on the Head Node Computer

To deploy the head node of your HPC cluster, you must start by installing one of the 64-bit editions of the Windows Server 2008 operating system on the computer that will act as the head node. For more information about installing Windows Server 2008, including system requirements, see http://go.microsoft.com/fwlink/?LinkID=119578. Important We strongly recommend that you perform a clean installation of Windows Server 2008 before installing Microsoft HPC Pack 2008. If you want to install Microsoft HPC Pack 2008 on an existing installation of Windows Server 2008, remove all server roles first and then follow the procedures in this guide.

2.2. Join the Head Node Computer to a Domain

As described in the Step 1: Prepare for Your Deployment section, the head node must be a member of an Active Directory domain. After you have installed Windows Server 2008 on the head node, manually join the head node to an existing Active Directory domain.

2.3. Install Microsoft HPC Pack 2008 on the Head Node Computer
After Windows Server 2008 is installed on the head node computer, and the head node is joined to an Active Directory domain, you can install Microsoft HPC Pack 2008 on the head node. To install Microsoft HPC Pack 2008 on the head node computer 1. To start the Microsoft HPC Pack 2008 installation wizard on the computer that will act as the head node, run setup.exe from the HPC Pack 2008 installation media or from a network location. 2. On the Getting Started page, click Next. 3. On the Microsoft Software License Terms page, read or print the software license terms in the license agreement, and accept or reject the terms of that agreement. If you accept the terms, click Next.
16

[This is prerelease documentation and is subject to change in future releases.]

4. On the Select Installation Type page, click Create New Compute Cluster, and then click Next. 5. On the Basic New Compute Cluster page, click Create new instance using SQLExpress, and then click Next. 6. On the Select Installation Location page, click Next. 7. On the Install Required Components page, click Install. 8. On the Installation Complete page, click Close.

Step 3: Configure the Head Node

After you have deployed the head node of your HPC cluster, you must configure the head node by following the configuration to-do list in HPC Cluster Manager.

Checklist: Configure the Head Node

The following checklist includes the items in the configuration to-do list in HPC Cluster Manager that you need to complete in order to configure your head node.
Task Reference

Configure the cluster network by using the Network Configuration wizard. Specify which credentials to use for system configuration and when adding new nodes to the cluster. Specify the naming convention to use when generating names automatically for new compute nodes. Create a template that defines the steps to follow when configuring a compute node. If you will be deploying compute nodes from bare metal and those nodes require special device drivers, add drivers for the operating system images that you created for your node template on the previous task. If you will be giving access to the cluster to other members of your organization, add or

3.1. Configure the HPC Cluster Network

3.2. Provide Installation Credentials

3.3. Configure the Naming of New Nodes

3.4. Create a Node Template

3.5. Add Drivers for the Operating System Images (Optional)

3.6. Add or Remove Users (Optional)

17

[This is prerelease documentation and is subject to change in future releases.]

Task

Reference

remove users or administrators for your cluster.

3.1. Configure the HPC Cluster Network

The HPC cluster network configuration is the first step in the configuration process of your head node. The HPC cluster network is configured by following the Network Configuration wizard in HPC Cluster Manager. When configuring the HPC cluster network, you must choose the network topology that you have selected for your cluster, as described in Step 1: Prepare for Your Deployment. Important Before you start configuring the HPC cluster network in HPC Cluster Manager, ensure that the head node and the computers that you will add as compute nodes to the cluster are physically connected according to the network topology that you have chosen for your cluster. Also, ensure that you are able to identify to which network each one of the network adapters in the head node is connected. To configure the HPC cluster network 1. If HPC Cluster Manager is not already open on the head node, open it. Click Start, point to All Programs, click Microsoft HPC Pack, and then click HPC Cluster Manager. 2. To start the Network Configuration wizard, in the To do list, click Configure your network. 3. On the Network Topology Selection page, click the topology that you have chosen for your cluster, and then click Next. 4. On the Enterprise Network Adapter Selection page, in the Network adapter list, click the name of the network adapter that is physically connected to your enterprise network, and then click Next. Important To ensure that you are selecting the correct network adapter, use the information displayed on this wizard page after you select a network adapter from the list. Use each adapters IP address, domain information, and MAC address as a reference. 5. If you chose topology number 5 for your cluster, jump to step 9 in this procedure. Otherwise, repeat step 4 for the private network adapter. 6. On the Private Network Configuration page, type a static IP address and a subnet mask for the head node. Optionally, select network services for that network: a. To give access to resources on the enterprise network to compute nodes that are connected to this network, select the Enable network address translation (NAT)
18

[This is prerelease documentation and is subject to change in future releases.]

on the head node check box. b. To enable DHCP services for the nodes connected to this network, select the Enable DHCP and define a scope check box, and then type the starting and ending IP addresses for the DHCP scope. If the Gateway and DNS server IP addresses have not been automatically detected, type each of these addresses. Note For more information about enabling NAT and DHCP on your cluster network, see HPC Network Services in Appendix 1: HPC Cluster Networking. 7. Click Next after you are done configuring the private network. 8. Repeat steps 4, 6, and 7 for the application network adapter. Click Next after you are done configuring the application network. 9. On the Firewall Setup page, select the firewall setting for the cluster: a. To apply firewall settings automatically to head nodes and compute nodes on each network, click ON for that network. b. To disable the firewall on a network, click OFF. c. If you do not want to change any firewall settings, click Do not manage firewall settings. Note For more information about firewall settings for your cluster, see Windows Firewall Configuration in Appendix 1: HPC Cluster Networking. 10. On the Review page, verify your settings and click Configure. If you want to change any of the settings, navigate to the appropriate wizard page by clicking it on the navigation pane or by clicking Previous. 11. After the network configuration process is completed, on the Configuration Summary page, review the list of configuration items. If you want to save a report of the network configuration, click Save the configuration report. 12. To close the wizard, click Finish.

3.2. Provide Installation Credentials

Installation credentials must be provided in order to configure new compute nodes. These credentials will be used when installing the operating system, applications, and when adding nodes to Active Directory Domain Services (AD DS). Also, these same credentials will be used when running diagnostic tests on the cluster nodes. To provide installation credentials 1. In the To do list, click Provide installation credentials. 2. Type the user name, including the domain (DOMAIN\User), and then the password for
19

[This is prerelease documentation and is subject to change in future releases.]

the domain user account you will use to deploy compute nodes and to run diagnostic tests. Important The account must be a domain account with enough privileges to create AD DS accounts for the compute nodes. If part of your deployment requires access to resources on the enterprise network, the account should have the necessary permissions to access those resources. If you want to restart nodes remotely from the cluster administration console (HPC Cluster Manager), the account must be a member of the local Administrators group on the head node. This requirement is only necessary if you do not have Intelligent Platform Management Interface (IPMI) tools that you can use to remotely restart the compute nodes.

3. To save the specified credentials, click OK.

3.3. Configure the Naming of New Nodes

If you deploy compute nodes from bare metal, and you are not using a node XML file to import nodes to the cluster, Windows HPC Server 2008 Beta 2 will automatically generate computer names for the new nodes that are being deployed. You need to specify how those names will be generated, by defining a naming series. The naming series is defined by selecting a root name and the starting number that will accompany that name. The starting number is enclosed in percentage signs (%). For example: ClusterNode%1000%. When you deploy compute nodes from bare metal, nodes will be named in sequence, as they become available. For example, if you deploy three nodes after specifying the following naming series: ClusterNode-%100%, those nodes will be assigned these names: ClusterNode-100 ClusterNode-101 ClusterNode-102 Important Compute node names are limited to 15 characters. When specifying the compute node naming series, take into account the number of compute nodes in your deployment and ensure that the series that you specify will not generate names that exceed 15 characters. For example, if your deployment will consist of 1,000 compute nodes, and your starting number is 1, your root name cannot have more than 12 characters; otherwise, your node number 1,000 will need a name that consists of 16 characters.

20

[This is prerelease documentation and is subject to change in future releases.]

To specify the compute node naming series 1. In the To do list, click Configure the naming of new nodes. 2. Type the naming series that you want to use. The preview helps you to see an example of how the naming series will be applied to the names of the compute nodes. Note You cannot specify a compute node naming series that consists only of numbers. 3. To save the compute node naming series that you have specified, click OK.

3.4. Create a Node Template

Node templates are new in Windows HPC Server 2008 Beta 2. They define the necessary tasks for configuring and adding compute nodes to your cluster. With a node template, you can deploy an operating system image, add specific drivers and software to compute nodes, or simply add a preconfigured node to your cluster. Because you might have more than one type of compute node, or you may be adding compute nodes to your cluster in different ways, you can create different templates that apply to different nodes or situations. You can create two types of node templates: With an operating system image. This type of template includes a step to deploy an operating system on the compute nodes. Use this type of template when adding compute nodes from bare metal. Without an operating system image. This type of template is used to add preconfigured compute nodes to the cluster, or to update existing nodes.

The type of template that you create for the initial deployment of your HPC cluster depends on how you decided to add compute nodes to your cluster. For more information, see 1.2. Decide How to Add Compute Nodes to Your Cluster. Important To complete the following procedure, you will need the installation media for one of the 64-bit editions of Windows Server 2008, or you must have the installation files available on a network location that is accessible from the head node computer. To create a node template 1. In the To do list, click Create a node template. 2. On the Specify Template Name page, type a descriptive name for the template, and then click Next. 3. If you will be adding compute nodes to your cluster from bare metal: a. On the Select Deployment Type page, click With operating system, and then click Next.
21

[This is prerelease documentation and is subject to change in future releases.]

b. On the Select Operating System Image page, click Add Image. c. On the Add Operating System Image window, click Create a new operating system image, and then type or browse to the location of the Windows setup file for one of the 64-bit editions of Windows Server 2008.

d. Type a descriptive name for the new operating system image, and then click OK. e. After the image is added, in the Image Name list, click the image that you want to use with the template. f. Optionally, specify if you want to multicast the operating system image during deployment. For more information, see 1.6. Prepare for Multicast (Optional) in Step 1: Prepare for Your Deployment.

g. Optionally, specify if you want to include a product key to activate the operating system on the compute nodes, and then type the product key that should be used. h. Click Next to continue. i. j. On the Specify Local Administrator Password for Compute Node page, click Use a specific password, and then type and confirm the password that you want to use. Click Next to continue, and then jump to step 5 in this procedure.

4. If you will be adding preconfigured compute nodes to your cluster, on the Select Deployment Type page, click Without operating system, and then click Next. 5. On the Specify Windows Updates page, specify if you want to add a step in the template to download and install updates using Microsoft Update or the enterprise Windows Server Update Services (WSUS). Also, you can specify specific updates to be added to the template. Click Next to continue. 6. On the Review page, click Create.

3.5. Add Drivers for the Operating System Images (Optional)

If you will be deploying compute nodes from bare metal and those nodes require special device drivers, you will need to add those drivers during the configuration process of your head node. Drivers must be in the .inf format, and must be available to all operating system images in the image store. To add drivers for the operating system images 1. In Configuration, click To Do. 2. In the To do list, click Add drivers. 3. On the Manage Drivers window, click Add. 4. Type or browse to the location of the setup information file for the driver that you want to add (.inf format), and then click Open.
22

[This is prerelease documentation and is subject to change in future releases.]

5. Repeat step 4 for all the drivers that you want to add. 6. After you are done adding drivers, click Close.

3.6. Add or Remove Users (Optional)

If you will be giving access to the cluster to other members of your organization, you need to add them as users or administrators to your cluster. Also, you can remove users or administrators that were added by default during installation. To add or remove users for the cluster 1. In Configuration, click To Do. 2. In the To do list, click Add or remove users. 3. To add a user to the cluster: a. In the Actions pane, click Add User. b. Type the user name of the user that you want to add, and then click Check Names. For more information, on the Select Users or Groups window, click examples. c. Repeat the previous step for all users that you want to add. d. After you are done adding users, click OK. 4. To add an administrator to the cluster: a. In the Actions pane, click Add Administrator. b. Type the user name of the administrator that you want to add, and then click Check Names. For more information, on the Select Users or Groups window, click examples. c. Repeat the previous step for all administrators that you want to add 5. To remove a user or administrator, select it on the Users list, and then click Remove.

Step 4: Add Compute Nodes to the Cluster

Windows HPC Server 2008 Beta 2 simplifies the deploying process of compute nodes by providing automatic node imaging, automatic naming of nodes, and other capabilities to streamline deployment tasks. Also, it provides tools to monitor the progress of your deployment. Important Unlike previous versions of Windows HPC Server 2008, the default in Windows HPC Server 2008 Beta 2 is to respond only to PXE requests that come from existing compute nodes. This default setting is automatically changed when you use the Add Node wizard to add nodes from bare metal. Also, you can manually change this setting in the Options menu, under Deployment Settings.
23

[This is prerelease documentation and is subject to change in future releases.]

After creating a node template, you can use the Add Node wizard to add compute nodes to your HPC cluster. There are three ways by which you can add compute nodes to your cluster: 4.1. Deploy Compute Nodes from Bare Metal 4.2. Add Compute Nodes by Importing a Node XML File 4.3. Add Preconfigured Compute Nodes

4.1. Deploy Compute Nodes from Bare Metal

The following procedure describes how to add compute nodes to your HPC cluster from bare metal, by using a node template that includes a step to deploy an operating system image. Important To complete this procedure, you must have a template that includes a step to deploy an operating system image. If you do not have a template that includes a step to deploy an operating system image, create one by following the steps in 3.4. Create a Node Template in Step 3: Configure the Head Node. Important Before turning on a compute node for this procedure, verify in the configuration of the BIOS of that computer that the compute node will boot from the network adapter that is connected to the private network, instead of booting from the local hard drive or another device, and that PXE boot is enabled for that network adapter. To deploy compute nodes from bare metal 1. If HPC Cluster Manager is not already open on the head node, open it. Click Start, point to All Programs, click Microsoft HPC Pack, and then click HPC Cluster Manager. 2. In Configuration, click To Do. 3. In the To do list, click Add compute nodes. 4. On the Select Deployment Method page, click Deploy compute nodes from bare metal using an operating system image, and then click Next. 5. On the Select New Nodes page, in the Node template list, click the name of a node template that includes a step to deploy an operating system image. 6. Turn on the computers that you want to add as compute nodes to your cluster. Computers will be listed on the Add Node wizard as they contact the head node during PXE boot. They will be named using the naming series that you specified during the configuration process of the head node. For more information, see 3.3. Configure the Naming of New Nodes. 7. When all computers that you have turned on are listed, click Select all, and then click Deploy. 8. On the Completing the Add Node Wizard page, if you will be deploying more nodes,
24

[This is prerelease documentation and is subject to change in future releases.]

click Continue responding to all PXE requests. If you will not be deploying more nodes, click Respond only to PXE requests that come from existing compute nodes. 9. To track deployment progress, select the Go to Node Management to track progress check box, and then click Finish. 10. During the deployment process of a compute node, its state is set to Provisioning. When the deployment process is complete, the state changes to Offline. To bring online the nodes that have finished deploying: a. In Node Management, under By State, click Offline. b. Select all the nodes that you want to bring online. To select all the nodes that are currently offline, on the list of offline nodes, click any node and then press CTRL+A. c. In the Actions pane, click Bring Online.

4.2. Add Compute Nodes by Importing a Node XML File

The following procedure describes how to add compute nodes by importing a node XML file. Important In order to complete this procedure, you must have a valid node XML file. For more information, see Appendix 2: Creating a Node XML File. To add compute nodes by importing a node XML file 1. If HPC Cluster Manager is not already open on the head node, open it. Click Start, point to All Programs, click Microsoft HPC Pack, and then click HPC Cluster Manager. 2. In Configuration, click To Do. 3. In the To do list, click Add compute nodes. 4. On the Select Deployment Method page, click Import compute nodes from a node XML file, and then click Next. 5. On the Select New Nodes page, type or browse to the location of the node XML file, and then click Import. 6. On the Completing the Add Node Wizard page, click Finish.

4.3. Add Preconfigured Compute Nodes

The following procedure describes how to add preconfigured compute nodes to your HPC cluster. It first describes how to install HPC Pack 2008 on the computers that will act as compute nodes, and then how to add the preconfigured compute nodes to the cluster.

25

[This is prerelease documentation and is subject to change in future releases.]

Important The computers that you will add to your cluster as preconfigured compute nodes must already be running one of the 64-bit editions of the Windows Server 2008 operating system. For more information about installing Windows Server 2008, including system requirements, see http://go.microsoft.com/fwlink/?LinkID=119578. Important To complete this procedure, you must have a template that does not include a step to deploy an operating system image. If you do not have a template that does not include a step to deploy an operating system image, create one by following the steps in 3.4. Create a Node Template in Step 3: Configure the Head Node. To install Microsoft HPC Pack 2008 on a compute node computer 1. To start the HPC Pack 2008 installation wizard on the computer that will act as a compute node, run setup.exe from the HPC Pack 2008 installation media or from a network location. 2. On the Getting Started page, click Next. 3. On the Microsoft Software License Terms page, read or print the software license terms in the license agreement, and accept or reject the terms of that agreement. If you accept the terms, click Next. 4. On the Select Installation Type page, click Join Existing Compute Cluster, and then click Next. 5. On the Join Cluster page, type the computer name of the head node on your cluster, and then click Next. 6. On the Select Installation Location page, click Next. 7. On the Install Required Components page, click Install. 8. On the Installation Complete page, click Close. When HPC Pack 2008 is installed on all the compute nodes that you want to add to your cluster, follow the steps in the Add Node wizard on the compute node to add the preconfigured nodes to your cluster. To add preconfigured compute nodes to your cluster 1. If HPC Cluster Manager is not already open on the head node, open it. Click Start, point to All Programs, click Microsoft HPC Pack, and then click HPC Cluster Manager. 2. In Configuration, click To Do. 3. In the To do list, click Add compute nodes. 4. On the Select Deployment Method page, click Add compute nodes that have already been configured, and then click Next.
26

[This is prerelease documentation and is subject to change in future releases.]

5. Turn on all the preconfigured nodes that you want to add to your cluster. 6. When all the preconfigured nodes are turned on, on the Before Deploying page, click Next. 7. On the Select New Nodes page, in the Node template list, click the name of a node template that does not include a step to deploy an operating system image. 8. Select the preconfigured compute nodes that you want to add to your cluster. To select all the preconfigured compute nodes, click Select all. 9. To add the selected compute nodes to your cluster, click Add. 10. On the Completing the Add Node Wizard page, click Respond only to PXE requests that come from existing compute nodes. 11. To track deployment progress, select the Go to Node Management to track progress check box. 12. To add the preconfigured nodes to your cluster, click Finish. 13. During the deployment process of a compute node, its state is set to Provisioning. When the deployment process is complete, the state changes to Offline. To bring online the nodes that have finished deploying: a. In Node Management, under By State, click Offline. b. Select all the nodes that you want to bring online. To select all nodes that are currently offline, on the list of offline nodes, click any node and then press CTRL+A. c. In the Actions pane, click Bring Online.

4.4. Monitor Deployment Progress

You can monitor the progress of the deployment process of compute nodes in Node Management. You can also see detailed information for each deployment operation, and any errors that may have occurred. To monitor deployment progress 1. If HPC Cluster Manager is not already open on the head node, open it. Click Start, point to All Programs, click Microsoft HPC Pack, and then click HPC Cluster Manager. 2. To see information about the deployment operations: a. In Node Management, click Operations. b. To see more information about a specific operation, click that operation on the list of operations. The details pane will list the log entries for that operation. 3. To see the list of compute nodes that are currently being deployed: a. In Node Management, under By State, click Provisioning. b. To see the list of operations related to the deployment of a specific node, double-click that node, and then click the Operations tab.
27

[This is prerelease documentation and is subject to change in future releases.]

4. To bring online the nodes that have finished deploying: a. In Node Management, under By State, click Offline. b. Select all the nodes that you want to bring online. To select all nodes that are currently offline, on the list of offline nodes, click any node and then press CTRL+A. c. In the Actions pane, click Bring Online. 5. To see the list of nodes that were not deployed successfully: a. In Node Management, under By Health, click Provisioning Failed. b. To see the list of operations related to the deployment failure of a specific node, click that node on the list of nodes, and then click View operations in the details pane (Properties tab). The pivoted view will list all the operations related to that node. c. To see more information about a specific operation, click that operation on the list of operations. The details pane will list the log entries for that operation.

If you realize that the template that you assigned to a node is incorrect, or that you assigned it to the wrong node, you can cancel the assignment of the node template. To cancel the assignment of a node template 1. In Node Management, under By State, click Provisioning. 2. Click the node for which you want to cancel the template assignment, and then click Cancel in the details pane (Properties tab). The state of the node will be changed to Unknown, and the health to Provisioning Failed.

Related Documents
For more information about monitoring, see Step-by-Step Guide for Monitoring in Windows HPC Server 2008 Beta 2 (HPCSxSMonitoring.doc).

Step 5: Run Diagnostic Tests on the Cluster

After you have configured your head node and added all compute nodes to the cluster, you should run diagnostic tests to validate cluster functionality and troubleshoot any configuration issues. To run diagnostic tests on the cluster 1. If HPC Cluster Manager is not already open on the head node, open it. Click Start, point to All Programs, click Microsoft HPC Pack, and then click HPC Cluster Manager. 2. In Configuration, click To Do. 3. In the To do list, click Validate your cluster (under Diagnostics).
28

[This is prerelease documentation and is subject to change in future releases.]

4. On the Run Diagnostics window, click Run all functional test, click All nodes, and then click OK. 5. To see the progress of the diagnostic tests and the test results, in Diagnostics, click Test Results. 6. To see detailed information about a test, double-click the test. To expand the information in a section of the test results, click the down arrow for that section.

Related Documents
For more information about running diagnostics on your HPC cluster, see Step-by-Step Guide for Diagnostics in Windows HPC Server 2008 Beta 2 (HPCSxSDiagnostics.doc).

Step 6: Run a Test Job on the Cluster

After you have finished deploying your cluster, you can run a simple test job to verify that your cluster is fully functional. The following checklist describes the steps involved in running a simple test job on your cluster.

Checklist: Run a Test Job on the Cluster

Task Reference

Create a job template by running the Generate Job Template wizard in HPC Cluster Manager. Create and submit a basic job in HPC Cluster Manager. Create and submit a basic job by using the HPC command-line tools. Create and submit a basic job by using the cmdlets in the HPC PowerShell.

6.1. Create a Job Template

6.2. Create and Submit a Job

6.3. Create and Submit a Job Using the Command-Line Interface (Optional) 6.4. Create and Submit a Job Using the HPC PowerShell (Optional)

6.1. Create a Job Template

Job templates simplify the job management of your HPC cluster by helping you to limit the kinds of jobs that can be submitted to your cluster, what resources are assigned to jobs, and which users can submit jobs. HPC Cluster Manager includes the Generate Node Template wizard to help you create basic job templates.
29

[This is prerelease documentation and is subject to change in future releases.]

To create a simple job template 1. If HPC Cluster Manager is not already open on the head node, open it. Click Start, point to All Programs, click Microsoft HPC Pack, and then click HPC Cluster Manager. 2. In Configuration, click Job Templates. 3. In the Actions pane, click New. 4. On the Welcome page, type Test Template for the name of the new job template, and optionally a description. Click Next to continue. 5. On the Job Run Times page, select the Jobs submitted to this template may not run for longer than check box, and then click Next without changing any settings. This will limit all jobs that are submitted using this template to run for no longer than 1 minute. 6. On the Job Priorities page, click Next without changing any settings. This will run jobs that are submitted using this template with Normal priority. 7. On the Project Names page, click Next without changing any settings. This will allow jobs from any project to be submitted using this template. 8. On the Node Groups page, click Next without changing any settings. This will allow jobs that are submitted using this template to run on any node group. 9. On the Finish page, click Finish.

6.2. Create and Submit a Job

This section describes how to submit a job in HPC Cluster Manager that: Displays a directory of the files in the C:\Program Files folder of a compute node in your cluster Uses the job template that you created in the previous section, which limits to 1 minute the maximum duration of time that a job can run Runs at low priority To create and submit a job 1. In Job Management, in the Actions pane, click New Job. 2. In Job Details, specify the following job parameters: a. In the Job name box, type Directory Contents. b. In the Job template list, click Test Template (the template that you created in section 6.1. Create a Job Template). c. When you are prompted if you want to change the job template for the job, click Yes. d. In the Priority list, click Lowest. 3. To add a task, click Task List, and then specify the following task parameters: a. To add a new basic task to the job, click Add. b. In the Task name box, type a name for the new task.
30

[This is prerelease documentation and is subject to change in future releases.]

c.

In the Command line box, type dir.

d. In the Work directory box, type c:\Program Files. e. To add this task, click Save. 4. To limit the job so that it only runs on a specific compute node in your HPC cluster, click Resource Selection, and then specify the following resource parameters: a. Select the Run this job only on nodes in the following list check box. b. Select the check box for one of the nodes in your HPC cluster. 5. To submit the job, click Submit. 6. If you are prompted to enter your credentials, type your user name and password, and then click OK. 7. To see the progress and the results of the job that you submitted: a. In Job Management, click All Jobs. b. In the list of jobs, click the job that you submitted. c. When the state of the job is Finished, in the lower pane, double-click the task that you created in step 3.

d. In the Task Properties window, in the Results tab, the Output box will display the directory of c:\Program Files for the compute node that you selected in step 4. e. If you want to copy the results to the clipboard, click Copy output to clipboard.

6.3. Create and Submit a Job Using the Command-Line Interface (Optional)
You can create and submit a job similar to the job that you created and submitted in the previous section, using the command-line interface tools that are included with Windows HPC Server 2008 Beta 2. To create and submit a job using the command-line interface 1. Open a Command Prompt window. Click Start, point to All Programs, click Accessories, and then click Command Prompt. 2. To create a new job, type the following command: job new /jobname:"Directory Contents" /priority:"Lowest" /RunTime:0:0:1 /requestednodes:<ComputeNodeName> Where <ComputeNodeName> is the name of a compute node in your HPC cluster. 3. To add a task to the job, type the following command: job add <JobID> /workdir:"C:\Program Files" dir Where <JobID> is the identification number for the job, as displayed on the commandline interface after typing the command in step 2.
31

[This is prerelease documentation and is subject to change in future releases.]

4. To submit the job, type the following command: job submit /id: <JobID> Where <JobID> is the identification number for the job, as displayed on the commandline interface after typing the command in step 2. 5. If you are prompted to enter your credentials, type your password, and then type ENTER.

6.4. Create and Submit a Job Using the HPC PowerShell (Optional)
You can also create and submit the same job that you created and submitted in the previous section, using the HPC PowerShellTM. Note For more information about the HPC PowerShell, see Using the HPC PowerShell in Important Features in Windows HPC Server 2008 Beta 2. To create and submit a job using the HPC PowerShell 1. On the head node, click Start, point to All Programs, and then click Microsoft HPC Pack. 2. Right-click HPC PowerShell, and then click Run as administrator. 3. If you are prompted by Windows PowerShell if you want to run the ccppsh.format.ps1xml script, type A and then press ENTER. 4. To create a new job, type the following cmdlet: $j = New-HpcJob -Name "Directory Contents" -Priority Lowest RunTime "0:0:1" -RequestedNodes <ComputeNodeName> Where <ComputeNodeName> is the name of a compute node in your HPC cluster. 5. To add a task to the job, type the following cmdlet: $j | Add-HpcTask -WorkDir "C:\Program Files" -CommandLine "dir" 6. To submit the job, type the following cmdlet: $j | Submit-HpcJob 7. If you are prompted to enter your credentials, type your password, and then type ENTER. Note You can also type all three cmdlets in one line: New-HpcJob -Name "Directory Contents" -Priority Lowest RunTime "0:0:1" -RequestedNodes <ComputeNodeName> | AddHpcTask -WorkDir "C:\Program Files" CommandLine "dir" | Submit-HpcJob
32

[This is prerelease documentation and is subject to change in future releases.]

Where <ComputeNodeName> is the name of a compute node in your HPC cluster.

Related Documents
For more information about creating and submitting jobs, see Step-by-Step Guide for Job Submission in Windows HPC Server 2008 Beta 2 (HPCSxSJobSubmission.doc). For more information about the job scheduler configuration, see Step-by-Step Guide for Configuring Job Submission and Scheduling Policies in Windows HPC Server 2008 Beta 2 (HPCSxSJobSchedulerConfig.doc).

HPC Cluster Security

Security is a primary focus of Windows HPC Server 2008 Beta 2. The goal is to protect resources of both the user and the HPC cluster as a whole. This section describes the following aspects of security in Windows HPC Server 2008 Beta 2: Groups and Authorized Operations. Describes the differences between a cluster administrator and a cluster user, and what each user is authorized to do on the cluster. HPC Cluster Credentials. Describes how user credentials are handled, how they are entered into the cluster, and how long they persist in the system. Application Firewall Exceptions. Describes how to open ports for customer applications so that they can run on the cluster. Access Managed Code Applications. Describes how to change code access security settings to allow a compute node to access a managed code application that is available on a remote shared resource.

Groups and Authorized Operations

AD DS is a prerequisite to running Windows HPC Server 2008 Beta 2 because the authentication process for users and computers relies entirely on the services provided by AD DS.

Groups
Windows HPC Server 2008 Beta 2 uses the local settings for users and groups on the head node to assign administrator and user rights on the cluster. Local users and groups on the head node include the Administrators group and the Users group. When the head node is added to an Active Directory domain, the Domain Admins group is added to the Administrators group, and the Domain Users group is added to the Users group. All memberships in the Domain Users and Domain Admins groups are automatically propagated to all compute nodes and secondary head nodes on the cluster, as part of the configuration process. To add a user to the cluster, you

33

[This is prerelease documentation and is subject to change in future releases.]

add the domain account for the user to the Users group, if it is not already part of the Domain Users group. Note The user account must be a domain account. A local user account on the head node is not sufficient. Also, installation credentials are provided during cluster configuration and used to install software on compute nodes. To have the necessary permissions to add an Active Directory object (for example, a computer account), and to reboot compute nodes remotely, the user account that is associated with the installation credentials must be a member of the Domain Admins group. If the user account is not a member of the Domain Admins group, but a member of the Domain Users group instead, the domain administrator must give that user account specific permissions to add Active Directory objects, or the installation process will fail. The following table lists memberships and where they are originated.
Membership Origin

Local Users and Groups: Administrators Local Users and Groups: Administrators: Domain Admins Local Users and Groups: Users Local Users and Groups: Users: Domain Users Local Users and Groups: Users: Authenticated Users

Windows Server 2008 Active Directory domain

Windows Server 2008 Active Directory domain Active Directory domain

Authorized Operations
The following table lists Job Scheduler operations and which of these are authorized for members of the Users and Administrators groups.
Job Scheduler Operation User Administrator

List jobs for every user List all compute nodes View own tasks Submit jobs for every user Cancel own job

Yes Yes Yes Yes Yes

Yes Yes Yes Yes Yes

34

[This is prerelease documentation and is subject to change in future releases.]

Job Scheduler Operation

User

Administrator

Modify jobs of other users View tasks for every user Run the clusrun command-line tool Note

No No No

Yes Yes Yes

You can provide more detailed permissions to submit jobs and use shared sessions by creating job templates. User and group rights for SOA applications and for the HPC Basic Profile Web service are the same as those for the Job Scheduler.

The following table lists HPC cluster management operations and which of these are authorized for members of the Users and Administrators groups.
HPC Management Operation User Administrator

Cluster configuration Apply a template to a node Add a node to the cluster Run diagnostic tests on the cluster Create a computer account in Active Directory to be used during deployment Restart a node remotely Note

No No No No Yes (required or installation will fail) No

Yes Yes Yes Yes Yes

Yes

Windows HPC Server 2008 Beta 2 stops unauthorized computers from being added to the cluster and compute nodes. If an unauthorized node is detected, it is marked as Unknown until a cluster administrator adds that node to the cluster by applying a node template to it.

HPC Cluster Credentials

This section describes how HPC cluster credentials are handled, how they are provided, and how long they are stored.

35

[This is prerelease documentation and is subject to change in future releases.]

How Cluster Credentials are Handled

Job submission credentials are sent from the client computer to the head node across a .NET Remoting secure channel where they are encrypted and sent back to the client computer. The user can decide to cache the encrypted credentials locally on the client computer for later usage. The encryption of credentials on the head node is done using an Advanced Encryption Standard (AES) cryptography provider that is compliant with the Federal Information Processing Standards (FIPS), more specifically: FIPS 140-2 level 1. The encryption key is created and stored under a secured registry key in the Service Control Manager (SCM) service record for each management and scheduler service. The registry key is added to the Access Control List (ACL) in a way that only the service has access to this data as the local administrator. Caution Users who are members of the local Administrators group, or the local System account can take ownership of the key and override the discretionary access control list (DACL). When a job is ready to run on compute nodes, the Job Scheduler decrypts the credentials using the same key and sends the credentials over a secure .NET Remoting channel to the compute nodes. The compute nodes then create a user logon token based on the credentials received, which are deleted once the job initialization is done. On the head node, the encrypted password stays in the Job Scheduler database until the job record is deleted. Jobs are automatically deleted by the Job Scheduler five days after completion, although this time interval can be changed as part of the cluster configuration. The credentials can be used for scheduling jobs in the future. Installation credentials are stored in the SQL database for the cluster, and are never stored on a client computer. The same encryption key that is used for job submission is used for installation credentials.

How Cluster Credentials are Provided

Credentials for the Job Scheduler are provided .in the following ways: Submission of jobs using the job submit command on a command line Submission of jobs using HPC Cluster Manager or HPC Job Manager Using the cluscfg setcreds command on a command line Using the Submit-HpcJob cmdlet on the HPC PowerShell Using the Set-HpcJobCredential cmdlet on the HPC PowerShell Using the Job Scheduler API Using HPC Basic Profile File Stagging credentials in an XML file Using the user credentials interface for the HPC Basic Profile Web service

Credentials for cluster management are provided .in the following ways:
36

[This is prerelease documentation and is subject to change in future releases.]

Installation credentials input on HPC Cluster Manager Installation credentials input on HPC PowerShell

How Much Time Credentials Are Stored

On the head node, the encrypted password stays in the Job Scheduler database until the job record is deleted. Jobs are automatically deleted by the Job Scheduler five days after completion. You can configure this interval in two ways: By changing the time period that jobs are stored in the Job Scheduler configuration by using HPC Cluster Manager By using the cluscfg setparams command with the ttlcompletedjobs parameter on a command line. To change the time period that jobs are stored using HPC Cluster Manager 1. If HPC Cluster Manager is not already open on the head node, open it. Click Start, point to All Programs, click Microsoft HPC Pack, and then click HPC Cluster Manager. 2. On the Options menu, click Job Scheduler Configuration, and then click the Job History tab. 3. In the Days to keep jobs in history box, type or select the number of days that you want to keep jobs in history. 4. To save your changes, click OK. To change the time period that jobs are stored using a command line 1. Open a Command Prompt window on the head node. Click Start, point to All Programs, click Accessories, and then click Command Prompt. 2. Type cluscfg setparams ttlcompletedjobs=<period>; where <period> is the number of days that you want to keep jobs in history. The value that you type for <period> must be number between 1 and 99. Important If credentials expire or are renewed before a scheduled job starts running, when a job is still running, or when a job is about to write results on a shared network resource, that job will fail.

Application Firewall Exceptions

By default, Windows Firewall is disabled on the private and application networks to provide the best performance and manageability experience. Normally, this state does not present a security problem because administrators for whom intra-node security is a priority will isolate the private

37

[This is prerelease documentation and is subject to change in future releases.]

and application networks behind the head node, and enable NAT if access to the enterprise network is required. Even when Windows Firewall is turned on, Windows HPC Server 2008 Beta 2 opens ports and application exceptions to enable internal services to run. It is the responsibility of the system administrator to create Windows Firewall exceptions for the executables of client applications.

Access for Managed Code Applications

You need to provide access to the cluster Administrators and Users groups to the remote installation folder where applications are located and accessed by the compute nodes. You also need to modify code security settings to allow managed code applications to be trusted. You can do this by using the Code Access Security Policy tool (caspol.exe). For more information about the Code Access Security Policy tool, see http://go.microsoft.com/fwlink/?LinkId=119687.

Important Features in Windows HPC Server 2008 Beta 2

The Service Oriented Application (SOA) Programming Model and Runtime
High performance computing applications use a cluster to solve a single computational problem or a single set of closely related computational problems. These applications are classified as either message intensive or embarrassingly parallel. An embarrassingly parallel problem is one for which no particular effort is needed to decompose the problem into a very large number of parallel tasks, and there is no dependency or communication between those parallel tasks. The Service Oriented Application (SOA) programming model can be used for building, deploying, running, and managing interactive high-performance computing applications, which are embarrassingly parallel.
Example Application Example Task

Monte Carlo problems that simulate the behavior of various mathematical or physical systems. Monte Carlo methods are used in physics, physical chemistry, economics, and related fields. BLAST searches

Predicting the price of a financial instrument.

Gene matching

38

[This is prerelease documentation and is subject to change in future releases.]

Example Application

Example Task

Genetic algorithms Ray tracing Digital content creation Microsoft Excel add-in calculations

Evolutionary computational meta-heuristics Computational physics and rendering Rendering frames Calling add-in functions

Related Documents
For more information about the SOA programming model and the Job Scheduler in Windows HPC Server 2008 Beta 2, see http://go.microsoft.com/fwlink/?LinkID=119581. For more information about building a simple a simple SOA application, see Step-by-Step Guide for Building, Deploying, and Running SOA-based Applications in Windows HPC Server 2008 Beta 2 (HPCSxSSOAApp.doc). For more information about managing the SOA application infrastructure, see Step-by-Step Guide for Managing SOA Application Infrastructure in Windows HPC Server 2008 Beta 2 (HPCSxSSOAManaging.doc).

Access Your HPC Cluster Using a Web Service

Why use a Web service?
Windows HPC Server 2008 Beta 2 is designed to operate within enterprise environments where desktops and servers that are running Windows are part of a single (or federated) Active Directory domain. From their Windows-based desktops, users are able to use the graphical and command-line tools provided with Windows HPC Server 2008 Beta 2 to submit and manage jobs on a remote HPC cluster. However, in some enterprise environments users may use desktops that are not running Windows, from which they may wish to access a high performance computing cluster, or have an existing application in place for scheduling jobs, which they may wish to use to manage their computing workload across multiple clusters, including a Windows HPC Server 2008 Beta 2 cluster. To support these and other cross-platform interoperability scenarios, a Web service interface has been developed within the high performance computing community to provide secure remote access to HPC clusters through the different workload managers (also known as job schedulers). Microsoft, in collaboration with customers and vendors, has been actively involved in the Open Grid Forum to develop an HPC Basic Profile Web service. This Web service was created in order to provide interoperability between different job schedulers and job submission clients, and to
39

[This is prerelease documentation and is subject to change in future releases.]

provide the basis for building tools and applications that can access high performance computing resources directly.

Community Adoption and Support

There is an increasing number of vendors (including Microsoft), and of open source projects that have adopted the HPC Basic Profile specification and its extensions. At the time of writing this document, the following vendors have announced plans to release support for the HPC Basic Profile in their products: Altair Engineering, Inc., with PBS Professional (http://go.microsoft.com/fwlink/?LinkID=119582) Platform Computing Corporation, with Platform LSF (http://go.microsoft.com/fwlink/?LinkID=119583)

As mentioned before, there are several open source projects that use the HPC Basic Profile; among them: GridSAM (http://go.microsoft.com/fwlink/?LinkID=119584) BES++ (http://go.microsoft.com/fwlink/?LinkID=119585)

Extensions continue to be developed in response to community needs. More information can be obtained from the HPC Profile working group within the OGF (http://go.microsoft.com/fwlink/?LinkID=119586)

Related Documents
For more information about the HPC Basic Profile Web service and how to configure it for accessing your HPC cluster remotely, see Step-by-Step Guide for Configuring the Basic Profile Web Service in Windows HPC Server 2008 Beta 2 (HPCSxSBasicProfileConfig.doc). For more information about using a C# client to access the HPC Basic Profile Web service, see Step-by-Step Guide for Using the Basic Profile Web Service from C# in Windows HPC Server 2008 Beta 2 (HPCSxSUseBasicProfile.doc).

HPC Cluster Reporting

HPC cluster reporting provides long-term, aggregated data that helps you to understand how resources in your cluster are being utilized, and to monitor the health of your cluster. With HPC cluster reporting, you can: Determine the availability of computing resources in your cluster, by generating a Node Availability report. Understand how much of your cluster has been used, by generating a Job Resource Usage report.

40

[This is prerelease documentation and is subject to change in future releases.]

Determine how many jobs have been processed by the cluster, by generating a Job Throughput report. Determine how long jobs had to wait in the queue before they were processed, by generating a Job Turnaround report.

These reports can be generated in HPC Cluster Manager, in Charts and Reports. Also, in Charts and Reports you can monitor the current status and performance of your cluster by reviewing charts with real-time, aggregated data for node state, job throughput, network usage, and other cluster resources.

Related Documents
For more information about HPC cluster reporting, see Step-by-Step Guide for Reporting in Windows HPC Server 2008 Beta 2 (HPCSxSReporting.doc).

Using the HPC PowerShell

The HPC PowerShell, built on Microsoft Windows PowerShell technology, provides a powerful command-line interface and a scripting platform to enable the automation of administrative tasks. The HPC PowerShell is installed by default on the head node, and can also be installed on a client computer as part of the utilities available with Windows HPC Server 2008 Beta 2. To start the HPC PowerShell on the head node 1. Click Start, point to All Programs, and then click Microsoft HPC Pack. 2. Right-click HPC PowerShell, and then click Run as administrator. 3. If you are prompted by Windows PowerShell to choose if you want to run the ccppsh.format.ps1xml script, type A and then press ENTER. To start the HPC PowerShell on a client computer 1. Click Start, point to All Programs, click Microsoft HPC Pack, and then click HPC PowerShell. 2. If you are prompted by Windows PowerShell to choose if you want to run the ccppsh.format.ps1xml script, type A and then press ENTER. You can also add the HPC PowerShell snap-in from Windows PowerShell. To add the HPC PowerShell from Windows PowerShell 1. Click Start, point to All Programs, click Windows PowerShell 1.0, and then click Windows PowerShell. 2. In the Windows PowerShell, type Add-PsSnapin Microsoft.ComputeCluster.

41

[This is prerelease documentation and is subject to change in future releases.]

Add the HPC PowerShell Snap-In to Your Windows PowerShell Profile

If you have a Windows PowerShell profile, you can add the HPC PowerShell snap-in to it so that it is available in every PowerShell session under your user name. For more information about Windows PowerShell profiles, see http://go.microsoft.com/fwlink/?LinkID=119587. To add the HPC PowerShell snap-in to your Windows PowerShell profile 1. Open Windows PowerShell. Click Start, point to All Programs, click Windows PowerShell 1.0, and then click Windows PowerShell. 2. To edit your profile in Notepad, type notepad $profile. 3. Type the following cmdlet as a new line in the profile: Add-PsSnapin Microsoft.ComputeCluster. 4. To save the profile, in the File menu, click Save. 5. To close Notepad, in the File menu, click Exit.

Run HPC PowerShell Scripts

One of the many benefits of using HPC PowerShell is the ability to run scripts. For more information about running Windows PowerShell scripts, see http://go.microsoft.com/fwlink/?LinkID=119588.

View Help in HPC PowerShell

In-context help is available for the HPC PowerShell cmdlets: To see a list of the cmdlets that are available in the HPC PowerShell, type Get-Command PSSnapin Microsoft.ComputeCluster. To view basic help information for a specific cmdlet, type Get-Help <cmdlet>; where <cmdlet> is an HPC PowerShell cmdlet. To view detailed information for a specific cmdlet, type Get-Help <cmdlet> -Detailed; where <cmdlet> is an HPC PowerShell cmdlet

Additional Resources
An updated version of this Getting Started guide is available online, on the Windows HPC Server 2008 Library on TechNet: http://go.microsoft.com/fwlink/?LinkId=118024. The Windows HPC Server Library on TechNet: http://go.microsoft.com/fwlink/?LinkId=119594.
42

[This is prerelease documentation and is subject to change in future releases.]

The release notes for Windows HPC Server 2008 Beta 2: http://go.microsoft.com/fwlink/?LinkId=117922. The Windows HPC Server 2008 site on Connect: http://go.microsoft.com/fwlink/?LinkID=119579.

Appendix 1: HPC Cluster Networking

Windows HPC Server 2008 Beta 2 supports five cluster topologies designed to meet a wide range of user needs and performance, scalability, manageability, and access requirements. These topologies are distinguished by how the compute nodes in the cluster are connected to each other and to the enterprise network. Depending on the network topology that you choose for your cluster, certain network services, such as DHCP and network address translation (NAT), can be provided by the head node to the compute nodes. You must choose the network topology that you will use for your cluster well in advance of setting up an HPC cluster. In this section: HPC Cluster Networks Supported HPC Cluster Network Topologies HPC Network Services Windows Firewall Configuration

HPC Cluster Networks

The following table lists and describes the networks to which an HPC cluster can be connected.
Network name Description

Enterprise network

An organizational network to which the head node is connected and optionally the compute nodes. The enterprise network is often the network that most users in an organization log on to when performing their job. All intra-cluster management and deployment traffic is carried on the enterprise network unless a private network (and optionally, an application network) also connects the cluster nodes. A dedicated network that carries intra-cluster communication between nodes. This network carries management, deployment, and
43

Private network

[This is prerelease documentation and is subject to change in future releases.]

Network name

Description

application traffic if no application network exists. Application network A dedicated network, preferably with high bandwidth and low latency. These characteristics are important so that this network can perform latency-sensitive tasks, such as carrying parallel MPI application communication between compute nodes

Supported HPC Cluster Network Topologies

There are five cluster topologies supported by Windows HPC Server 2008 Beta 2: Topology No. 1: Compute Nodes Isolated on a Private Network Topology No. 2: All Nodes on Enterprise and Private Networks Topology No. 3: Compute Nodes Isolated on Private and Application Networks Topology No. 4: All Nodes on Enterprise, Private, and Application Networks Topology No. 5: All Nodes on an Enterprise Network

Topology No. 1: Compute Nodes Isolated on a Private Network

The following image illustrates how the head node and the compute nodes are connected to the cluster networks in this topology:

The following table lists and describes details about the different components in this topology:

44

[This is prerelease documentation and is subject to change in future releases.]

Component

Description

Network adapters

The head node has two network adapters. Each compute node has one network adapter. The head node is connected to both an enterprise network and to a private network. The compute nodes are connected only to the private network. The private network carries all communication between the head node and the compute nodes, including deployment, management and application traffic (for example, MPI communication). The default configuration for this topology is NAT enabled on the private network in order to provide the compute nodes with address translation and access to services and resources on the enterprise network. DHCP is enabled by default on the private network to assign IP addresses to compute nodes. If a DHCP server is already installed on the private network, then both NAT and DHCP will be disabled by default. The default configuration on the cluster has the firewall turned ON for the enterprise network and turned OFF for the private network. Cluster performance is more consistent because intra-cluster communication is routed onto the private network Network traffic between compute nodes and resources on the enterprise network (such as databases and file servers) pass through the head node. For this reason, and depending on the amount of traffic, this might impact cluster performance.
45

Traffic

Network Services

Security

Considerations when selecting this topology

[This is prerelease documentation and is subject to change in future releases.]

Component

Description

Compute nodes are not directly accessible by users on the enterprise network. This has implications when developing and debugging parallel applications for use on the cluster.

Topology No. 2: All Nodes on Enterprise and Private Networks

The following image illustrates how the head node and the compute nodes are connected to the cluster networks in this topology:

The following table lists and describes details about the different components in this topology:
Component Description

Network adapters

The head node has two network adapters. Each compute node has two network adapters. All nodes in cluster are connected to both the enterprise network and to a dedicated private cluster network. Communication between nodes, including deployment, management, and application traffic, is carried on the private network in this topology. Traffic from the enterprise network can be routed directly to a compute node.

Traffic

46

[This is prerelease documentation and is subject to change in future releases.]

Component

Description

Network Services

The default configuration for this topology has DHCP enabled on the private network, to provide IP addresses to the compute nodes. NAT is not required in this topology because the compute nodes are connected to the enterprise network, so this option is disabled by default. The default configuration on the cluster has the firewall turned ON for the enterprise network and OFF for the private network. This topology offers more consistent cluster performance because intra-cluster communication is routed onto a private network. This topology is well suited for developing and debugging applications because all compute nodes are connected to the enterprise network. This topology provides easy access to compute nodes by users on the enterprise network. This topology provides faster access to enterprise network resources by the compute nodes.

Security

Considerations when selecting this topology

Topology No. 3: Compute Nodes Isolated on Private and Application Networks

The following image illustrates how the head node and the compute nodes are connected to the cluster networks in this topology:

47

[This is prerelease documentation and is subject to change in future releases.]

The following table lists and describes details about the different components in this topology:
Component Description

Network adapters

The head node has three network adapters: one for the enterprise network, one for the private network, and a highspeed adapter that is connected to the application network. Each compute node has two network adapters, one for the private network and another for the application network. The private network carries deployment and management communication between the head node and the compute nodes. Jobs running on the cluster use the highperformance application network for crossnode communication. The default configuration for this topology has both DHCP and NAT enabled for the private network, to provide IP addressing and address translation for compute nodes. DHCP is enabled by default on the application network, but not NAT. If a DHCP is already installed on the private network, then both NAT and DHCP will be disabled by default. The default configuration on the cluster has
48

Traffic

Network Services

Security

[This is prerelease documentation and is subject to change in future releases.]

Component

Description

the firewall turned ON for the enterprise network and turned OFF on the private and application networks. Considerations when selecting this topology This topology offers more consistent cluster performance because intra-cluster communication is routed onto the private and application networks. Compute nodes are not directly accessible by users on the enterprise network in this topology.

Topology No. 4: All Nodes on Enterprise, Private, and Application Networks

The following image illustrates how the head node and the compute nodes are connected to the cluster networks in this topology:

The following table lists and describes details about the different components in this topology:
Component Description

Network adapters

The head node has three network adapters. All compute nodes have three network adapters. The network adapters are for the enterprise
49

[This is prerelease documentation and is subject to change in future releases.]

Component

Description

network, the private network, and a high speed adapter for the high performance application network. Traffic The private cluster network carries only deployment and management traffic. The application network carries latencysensitive traffic, such as MPI communication between nodes. Network traffic from the enterprise network reaches the compute nodes directly. The default configuration for this topology has DHCP enabled for the private and application networks to provide IP addresses to the compute nodes on both networks. NAT is disabled for the private and application networks because the compute nodes are connected to the enterprise network. The default configuration on the cluster has the firewall turned ON for the enterprise network and turned OFF on the private and application networks. This topology offers more consistent cluster performance because intra-cluster communication is routed onto a private and application network. This topology is well suited for developing and debugging applications because all cluster nodes are connected to the enterprise network. This topology provides easy access to compute nodes by users on the enterprise network. This topology provides faster access to enterprise network resources by the compute nodes.
50

Network Services

Security

Considerations when selecting this topology

[This is prerelease documentation and is subject to change in future releases.]

Topology No. 5: All Nodes on an Enterprise Network

The following image illustrates how the head node and the compute nodes are connected to the cluster networks in this topology:

The following table lists and describes details about the different components in this topology:
Component Description

Network adapters

The head node has one network adapter. All compute nodes have one network adapter. All nodes are on the enterprise network. All traffic, including intra-cluster, application, and enterprise traffic, is carried over the enterprise network. This maximizes access to the compute nodes by users and developers on the enterprise network. This topology does not require NAT or DHCP because the compute nodes are connected to the enterprise network. The default configuration on the cluster has the firewall turned ON for the enterprise network. This topology offers easy access to compute nodes by users on the enterprise network. Access of resources on the enterprise network by individual compute nodes is
51

Traffic

Network Services

Security

Considerations when selecting this topology

[This is prerelease documentation and is subject to change in future releases.]

Component

Description

faster. This topology, like topologies 2 and 4, is well suited for developing and debugging applications because all cluster nodes are connected to the enterprise network. This topology provides easy access to compute nodes by users on the enterprise network. This topology provides faster access to enterprise network resources by the compute nodes. Because all nodes are connected only to the enterprise network, you cannot use Windows Deployment Services to deploy compute node images using the new deployment tools in Windows HPC Server 2008 Beta 2.

HPC Network Services

Depending on the network topology that you have chosen for your HPC cluster, the following network services can be provided by the head node to the compute nodes connected to the different cluster networks: Network Address Translation (NAT) Dynamic Host Configuration Protocol (DHCP) server

This section describes these HPC network services.

Network Address Translation (NAT)

Network address translation (NAT) provides a method for translating Internet Protocol version 4 (IPv4) addresses of computers on one network into IPv4 addresses of computers on a different network. Enabling NAT on the head node enables compute nodes on the private or application networks to access resources on the enterprise network. You do not need to enable NAT if you have another server providing NAT or routing services on the private or application networks. Also, you do not need NAT if all nodes are connected to the enterprise network.

52

[This is prerelease documentation and is subject to change in future releases.]

DHCP Server
A DHCP server assigns IP addresses to network clients. Depending on the detected configuration of your HPC cluster and the network topology that you choose for your cluster, the compute nodes will receive IP addresses from either the head node running DHCP, or from a dedicated DHCP server on the private network, or via DHCP services coming from a server on the enterprise network.

Windows Firewall Configuration

Windows HPC Server 2008 Beta 2 opens firewall ports on the head node and compute nodes to enable internal services to run. By default, Windows Firewall is enabled only on the enterprise network, and disabled on the private and application networks to provide the best performance and manageability experience. Important If you have applications that require access to the head node or to the cluster nodes on specific ports, you will have to manually open those ports in Windows Firewall.

Firewall Ports Required by Windows HPC Server 2008 Beta 2

The following table lists all the ports that are opened by Windows HPC Server 2008 Beta 2 for communication between cluster services on the head node and the compute nodes.
Port Number (TCP) Required By

5969

Required by the client tools on the enterprise network to connect to the HPC Job Scheduler Service on the head node. Used by the HPC Management Service on the compute nodes to communicate with the HPC SDM Service on the head node. Used for communication between the HPC Management Service on the compute nodes and the HPC Job Scheduler Service on the head node. Used for communication between ExecutionClient.exe on the compute nodes and the HPC Management Service on the head node. ExecutionClient.exe is used during the deployment process of a compute node. It performs tasks such as imaging the computer, installing all the necessary HPC components, and joining the computer to the domain. Used for communication between the client application on the enterprise network and the services provided by the WCF Broker node. Used by the HPC Job Scheduler Service on the head node to communicate with the HPC Node Manager Service on the compute
53

9892, 9893

5970

9794

9087, 9088, 9089

1856

[This is prerelease documentation and is subject to change in future releases.]

Port Number (TCP)

Required By

nodes. 8677 Used for communication between the HPC MPI Service on the head node and the HPC MPI Service on the compute nodes. Used for management services traffic coming from the compute nodes to the head node or WCF broker node. Used for communication between the HPC command-line tools on the enterprise network and the HPC Job Scheduler Service on the head node. Used by the remote node service on the enterprise network to enumerate nodes in a node group, or to bring a node online or take it offline. Used by HPC Cluster Manager on the enterprise network to communicate with the HPC Job Scheduler Service on the head node. Used by the clients on the enterprise network to connect to the HPC Basic Profile Web service on the head node.

6729

5800

5801

5999

443

Appendix 2: Creating a Node XML File

A node XML file contains a list of compute nodes that you want to add to your cluster. This list includes: When adding compute nodes from bare metal, a hardware identification parameter for each compute node, such as the System Management BIOS (SMBIOS) GUID or the Media Access Control address (MAC) address. When adding preconfigured nodes that are already running one of the 64-bit editions of the Windows Server 2008 operating system, and Microsoft HPC Pack 2008 has been installed, the computer name is sufficient for identification purposes. Other properties, such as the physical location of each compute node and the Windows product key that should be used to activate the operating system.

Benefits of Using a Node XML File for Deployment

The following list outlines some of the benefits of using a node XML file when adding compute nodes to your cluster:

54

[This is prerelease documentation and is subject to change in future releases.]

You can pre-stage a PXE deployment of compute nodes for your HPC cluster by importing a node XML file with a list of all the computers that you will be adding to the cluster. The compute nodes can be deployed both from bare metal or preconfigured nodes. Preconfigured nodes that are added to your HPC cluster using a node XML file do not need to be manually approved into the cluster. This makes the deployment process more efficient and streamlined. Importing a node XML file is a simple and efficient way for you to associate properties with compute nodes. Examples of properties that can be associated with compute nodes are: location (including data center, rack, and chassis), a Windows product key, node templates, or tags that are used to automatically create node groups. You can give specific computer names (NetBIOS names) to compute nodes that are deployed from bare metal, without having to worry about powering them on in a specific order. Using a node XML file, computer names will already be associated with a specific SMBIOS GUID or MAC address (or both).

How to Create a Node XML File

The node XML file can be created in any XML editor or text editor, but it must follow a specific schema. Also, a node XML file can be created from an HPC cluster that is already configured, by exporting it from HPC Cluster Manager.

The Node XML File Schema

The node XML file is based on an XML Schema definition language (XSD) file: NodeConfigurationFile.xsd. This XSD file is available on the head node, in the Bin folder of the installation path for Windows HPC Server 2008 Beta 2. For example, if you are using the default installation path, the XSD file is available here: C:\Program Files\Microsoft HPC Pack\Bin\NodeConfigurationFile.xsd The following table lists and describes the attributes and elements that are defined in the node XML file schema:
Attribute, Element or Element:Attribute Required Description

Location

No

Optional element. Contains attributes with information about the location of the compute node. Optional attribute of the Location element. Specifies the name of the data center where the compute node is located.
55

Location: DataCenter

No

[This is prerelease documentation and is subject to change in future releases.]

Attribute, Element or Element:Attribute

Required

Description

Location: Rack

No

Optional attribute of the Location element. Specifies the name or number of the server rack where the compute node is located. Optional attribute of the Location element. Specifies the name or number of the chassis used for the compute node. Optional element. This element is required when deploying compute nodes from bare-metal. Contains attributes with information about the node template that will be used to deploy the compute node. Required attribute of the Template element. This attribute is required only when a Template element is included. Specifies the name of the node template that will be used to deploy the compute node. If the specified note template name does not exist on the head node, the deployment will fail. If you are deploying compute nodes from bare metal, this attribute must specify the name of a node template that includes a step to deploy an operating system image, or your deployment will fail. Optional attribute of the Template element. Specifies if the node is a preconfigured node (True), or not (False).

Location: Chassis

No

Template

No

Template: Name

Yes

Template: Provisioned

No

56

[This is prerelease documentation and is subject to change in future releases.]

Attribute, Element or Element:Attribute

Required

Description

MacAddress

No

Optional element. Specifies the MAC address of the network adapter used by the compute node. If you are deploying compute nodes from bare metal, you must specify this element or the MachineGuid parameter, or the deployment will fail. You must also specify this element if the cluster nodes in your system have SMBIOS GUIDs that are not unique (that is, two or more nodes in the node XML file have the same value for the MachineGuid parameter). There can be multiple instances of this element, if the compute node uses more than one adapter. Ensure that you specify only those MAC addresses that exist in the compute node. Specifying a MAC address that does not exist in a compute node, might cause the import of that node to fail. Optional element. Specifies the name of the node group to which the compute node should be added during deployment. There can be multiple instances of this element, if the compute node should be added to more than one node group. Required attribute. Specifies the computer name (NetBIOS name) of the compute node. If you are deploying compute nodes from bare metal, this attribute specifies the computer name that will be assigned to the node during deployment. If you are deploying preconfigured
57

Tag

No

Name

Yes

[This is prerelease documentation and is subject to change in future releases.]

Attribute, Element or Element:Attribute

Required

Description

nodes, this attribute specifies the current computer name of the compute node. If the specified name is that of a preconfigured node that has already been added to the cluster (that is, it is not in the Unknown state), the node XML file will fail to import. Optional attribute. Specifies the Active Directory domain to which the compute node should be added. If this attribute is not specified, the Active Directory domain of the head node is used. Optional attribute. Specifies information required for Intelligent Platform Management Interface (IPMI) integration. You only need this attribute if you are using IPMI tools to manage power on your cluster. Optional attribute. Specifies the SMBIOS GUID of the computer where the compute node is deployed. If you are deploying compute nodes from bare metal, you must specify this parameter or the MacAddress element, or the node XML file will fail to import. Optional attribute. Specifies the Windows product key that will be used to activate the operating system on the compute node. The product key is used during the activation task of a node template that includes a step to deploy an operating
58

Domain

No

ManagementIpAddress

No

MachineGuid

No

ProductKey

No

[This is prerelease documentation and is subject to change in future releases.]

Attribute, Element or Element:Attribute

Required

Description

system image. The product key that you specify must match the edition of the operating system in the image that is used by the node template.

Creating a Node XML File for Deployment from Bare Metal

When creating a node XML file for a deployment from bare metal, you will need a hardware identification parameter for each compute node. This parameter can be the SMBIOS GUID or the MAC address of the computer. When creating a node XML file for deployment from bare metal: Specify the MAC address of a compute node in the MacAddress attribute for that compute node. Specify the SMBIOS GUID of a compute node in the MachineGuid attribute for that compute node. If both the SMBIOS GUID and MAC address of a compute node are specified, the SMBIOS GUID is used. If for some reason you do not have access to the SMBIOS GUID of a node, you can use only the MAC address. Ensure that you specify only those MAC addresses that exist in each compute node. Specifying a MAC address that does not exist in a compute node, might cause the import of that node to fail. You must specify a node template for each compute node listed, and that node template must include a step to deploy an operating system image. If you do not specify a node template or if you specify a node template that does not include a step to deploy an operating system image, the deployment will fail. For more information about node templates, see 3.4. Create a Node Template. Ensure that the node template names that are specified in the node XML file match the names of the node templates listed on the head node. Specify any location information that you want to be attached to the node. If you want nodes to be automatically added to specific node groups during deployment, specify the Tag attribute with the name of the node group for each compute node. If you are using a retail Windows product key, you can specify it in the node XML file. If your IPMI integration requires a BMC IP address for each compute node, it can be added to the node XML file.

59

[This is prerelease documentation and is subject to change in future releases.]

Creating a Node XML File for Deployment from Preconfigured Compute Nodes
When creating a node XML file for a deployment from preconfigured compute nodes, you only need to specify the computer name of each compute node. You can also specify the SMBIOS GUID or the MAC address of the computer, but it is not necessary. When creating a node XML file for deployment from preconfigured compute nodes: Specify the computer name of the compute node in the Name attribute for each compute node. If a node template is not specified for a compute node in the node XML file, the compute node will be listed in the Unknown state in Node Management. To add that compute node to the cluster, you will need to manually assign a template to the node. For more information about node templates, see 3.4. Create a Node Template. Ensure that the node template names that are specified in the node XML file match the names of the node templates listed on the head node. The node templates that are specified in the node XML file do not have to include a step to deploy an operating system image. If the node template specified in the node XML file does not include a step to deploy an operating system image, the node will be added to the cluster successfully, but you will not be able to reimage the node later on. If the node template specified in the node XML file does include a step to deploy an operating system image, the deployment process will skip that step when adding the preconfigured compute node to the cluster. To specify that a compute node is preconfigured, specify a True value for the Provisioned attribute of that node. Specify any location information that you want to be attached to the node. If you want nodes to be automatically added to specific node groups during deployment, specify the Tag attribute with the name of the node group for each compute node. If you are using a retail Windows product key, you can specify it in the node XML file. If your IPMI integration requires a BMC IP address for each compute node, it can be added to the node XML file.

Creating a Node XML File from a Preconfigured HPC Cluster

If you have an HPC cluster that is already configured, you can export this information to an XML file and use it if you need to rebuild the cluster with exactly the same compute node names and properties. To create a node XML file from a preconfigured HPC cluster 1. On the head node, open HPC Cluster Manager. Click Start, point to All Programs, click
60

[This is prerelease documentation and is subject to change in future releases.]

Microsoft HPC Pack, and then click HPC Cluster Manager. 2. In Node Management, click Nodes. 3. Select all the nodes that you want to include in the node XML file. To select all nodes in your cluster, on the list of nodes, click any node and then press CTRL+A. 4. On the Actions pane, click Export Node XML. 5. Type a name for the node XML file and then click Save.

Sample Node XML File

<?xml version="1.0" encoding="utf-8" standalone="yes" ?> <Nodes xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://schemas.microsoft.com/HpcNodeConfigurationFile/2007/12"> <Node Name="ComputeNodeName1" Domain="CONTOSO" MachineGuid="{4c4c4544-0038-5710-804b-c6c04f464331}"> <Location DataCenter="Data Center 1" Rack="2" Chassis="1" /> <Template Name="Default ComputeNode Template" Provisioned="True" /> <MacAddress>00301B445F02</MacAddress> <MacAddress>001B2104EDF5</MacAddress> <Tag>ComputeNodes</Tag> <Tag>Rack2</Tag> </Node> </Nodes>

61