FUJITSU Server PRIMEQUEST 2000 Series Design Guide: C122-A014-01EN

FUJITSU Server PRIMEQUEST 2000 Series
Design Guide
C122-A014-01EN
Copyright 2014 FUJITSU LIMITED
Table of Contents
Preface
P.4
1.1 Product Lineup

1.2 Specifications
1.3 Supported OS
1.4 Component Names
1.5 Basic Configuration
1.6 Features
P.8
P.9
P.10
P.11
P.12
P.19
2.1 Operator Panel (OPL)

2.2 System Board (SB)
2.3 Management Board (MMB)
2.4 IO Unit (IOU)
2.5 Disk Unit (DU)
2.6 PCI Box
2.7 Power Supply Unit (PSU)
2.8 FAN Unit (FANU)
2.9 Peripherals Configuration
P.27
P.28
P.40
P.43
P.46
P.47
P.51
P.52
P.53
n 1. PRIMEQUEST Overview
n 2. Basic Hardware Configuration
PRIMEQUEST 2000 Series Design Guide
Table of Contents (Cont'd)

n 3. Hardware Configuration Design
3.1 Key Points for MMB Design

3.2 Key Points for Hardware Partitioning Design
3.3 Key Points for I/O Configuration Design
3.4 Key Points for Cluster Configurations
P.58
P.59
P.83
P.101
4.1 Key Points for Console Operation

4.2 Key Points for Server Operation Management
4.3 Key Points for Backup and Restoration Configurations
4.4 Key Points for NTP Operation
P.108
P.110
P.112
P.114
n 4. Operation Management Design
Table of Contents (Cont'd)

n 5. OS/Support
5.1 Arrangement Information
P.116
Appendix A Bundled Software

Appendix B OS Specifications
P.118
P.120
n Appendices
Preface
n About This Manual
n Contents of This Manual
This manual describes the concepts of system design and notes for users of the
PRIMEQUEST 2000 series.
For details on the operation of the main unit and other information, refer to the manuals
for the PRIMEQUEST 2000 series main unit.
For details on the operating environment of software, refer to the website of each
software program.
For details on virtualization, refer to the document of each virtualization software program.
For details on SAN boot, refer to "PRIMEQUEST 1000/2000 Series SAN Boot Environment
Configuration Manual".
n Organization of Manuals
The following figure shows the organization of manuals for the PRIMEQUEST 2000
series.
Design Guide
Linux Design Guide
Design
Linux Installation Guide
Installation
Linux Operation Guide
Operation
Preface
n Symbols in This Manual
The following table shows the meaning of the symbol used in this manual.
Symbol
Meaning
Indicates the page or document to refer to.
n Abbreviations in This Manual

Name
Abbreviation
PRIMEQUEST 2400E
4 socket model
PRIMEQUEST 2800B
8 socket model
PRIMEQUEST 2800E
PRIMEQUEST 2000 series or PRIMEQUEST
Microsoft Windows Server 2012 R2 Standard

Microsoft Windows Server 2012 R2 Datacenter
Windows Server 2012
Microsoft Windows Server 2012 Standard
Windows
Microsoft Windows Server 2012 Datacenter

Microsoft Windows Server 2008 R2 Standard
Microsoft Windows Server 2008 R2 Enterprise
Windows Server 2008
Microsoft Windows Server 2008 R2 Datacenter

Red Hat Enterprise Linux 6 (for Intel64)
RHEL6 (for Intel64)
VMware vSphere 5
VMware
RHEL6, RHEL, or Linux
Preface
Name
Abbreviation
PRIMEQUEST 2000 Series Hardware Installation Manual
Hardware Installation Manual
PRIMEQUEST 2000 Series General Description
General Description
PRIMEQUEST 2000 Series Installation Manual
Installation Manual
PRIMEQUEST 2000 Series Administration Manual
Administration Manual
PRIMEQUEST 2000 Series Tool Reference
Tool Reference
PRIMEQUEST 2000 Series Linux Design Guide -Red Hat Enterprise Linux-
Linux Design Guide
PRIMEQUEST 2000 Series Linux Installation Guide -Red Hat Enterprise Linux-
Linux Installation Guide
Microsoft, Windows, and Windows Server are the registered trademarks or trademarks of Microsoft Corporation in the United States and/or other
countries.
Linux is a registered trademark of Linus Torvalds.
Red Hat is a registered trademark of Red Hat, Inc. in the United States and other countries.
SUSE is a trademark or registered trademark of Novell, Inc. in the United States and other countries.
Intel, Itanium, and Xeon are trademarks of Intel Corporation in the United States and/or other countries.
VMware and other VMware product names are registered trademarks or trademarks of VMware, Inc. in the United States and/or other countries.
NetVault is a registered trademark of Dell, Inc. in the U.S.A. and/or other countries.
Oracle and Java are the registered trademarks of Oracle Corporation and its subsidiaries/related companies in the United States and other countries.
PRIMECLUSTER is a registered trademark of Fujitsu Limited.
Other company names and product names are the trademarks or registered trademarks of their respective owners.
1. PRIMEQUEST Overview
This chapter describes the overview of PRIMEQUEST.
For details, refer to "General Description".
1.1 Product Lineup

Core servers that combine the reliability of a mainframe and
the cost efficiency of an open server to support mission critical operations
For Enterprise
For Business
PRIMEQUEST 2400E
PRIMEQUEST 2800E
Up to 2 partitions
Up to 4 CPUs and 60 cores
Up to 4 partitions
Up to 8 CPUs and 120 cores
PRIMEQUEST 2800B
Up to 1 partition
8 CPUs and 120 cores
Highly-reliable servers that are based on the latest Intel architecture

1.2 Specifications
Item
CPU
PRIMEQUEST 2000 series
Purpose
Business
Model name
2800B
Enterprise
2400E
Type
Number of cores (*1)
Maximum SMP (sockets/cores) (*1)

Maximum number of partitions
6/12/15
8/120
4/60
8/120
Supported DIMMs
Memory
Maximum memory capacity (*1)
Maximum internal disk capacity

Maximum number of PCI slots (*1)
2800E
Intel Xeon Processor E7 v2 Family
8GB/16GB/32GB/64GB
12TB (64GB DIMM x 192)
12.8TB
(DU 1.6TB x 8)
25.6TB
(SB 1.6TB x 8, DU 1.6TB x 8)
38.4TB
(SB 1.6TB x 16, DU 1.6TB x 8)
Up to 16 (internal only)
16 (internal only ) / up to 56
16 (internal only ) / up to 56
Height
10U
Reliability, Availability, and Serviceability (RAS)

Memory Mirror
Supported
Memory Spare
Supported
Flexible I/O
Not supported
Supported
Reserved SB
Not supported
Supported
Dynamic Reconfiguration
Not supported
Supported (RHEL)
RAID
Hardware RAID / Software RAID
Redundancy
Memory, IOU, PSU, FAN, HDD, SAS-SSD, PCI Express card, MMB, and power input system
IOU (in a redundant configuration), PSU (in a redundant configuration), FAN, HDD (in a RAID configuration),
SAS-SSD (in a RAID configuration), PCI Express card (when a PCI Box is equipped), and MMB (in a duplex configuration)
Hot plug
Original dump function
Cluster
sadump (RHEL)
Intra-cabinet
Supported
Inter-cabinet
Supported
Remote operation
WOL, PXE, video redirection, and console redirection
(*1) The maximum number of CPUs, the maximum number of cores, and the maximum memory capacity that can be configured in a system vary depending on
the type and version of the OS. For details, refer to Appendix B OS Specifications. The maximum number of PCI slots depends on the type and the number of
IO Units.
For details, refer to "Chapter 1 Product Overview" in "General Description".
1.3 Supported OS
For the latest information on the supported OSs, refer to the following URL:
http://www.fujitsu.com/global/services/computing/server/primequest/products/software/
10
1.4 Component Names

System Board
(SB)
Power Supply Unit

(PSU)/FAN
IO Unit (IOU)
Management Board
(MMB)
Disk Unit (DU)
Front
(View without the front panel)
11
Rear
1.5 Basic Configuration

n Layout and Internal Connection Configuration for 2800B
"1.5.1 Basic Configuration (2800B)"
n Layout and Internal Connection Configuration for 2400E

"1.5.2 Basic Configuration (2400E)"
n Layout and Internal Connection Configuration for 2800E

"1.5.3 Basic Configuration (2800E)"
12
1.5.1 Basic Configuration (2800B) (1/2)

n 2800B Layout
Basic components
Component
Max.
installable
number
Installed as
standard
SB
(must be
ordered)
IOU
(must be
ordered)
MMB
PSU
(must be
ordered)
FAN
No
DU
No
No
No
No
Front
Rear
13
No
AC cable
(must be
ordered)
1.5.1 Basic Configuration (2800B) (2/2)

n Internal Connection Configuration for 2800B
QPI: QuickPath Interconnect
Maximum numbers of ports and cards

for network connection
Component
IOU onboard LAN ports

PCI Express slots on IOUs
Maximum
numbers of
ports and
cards
8 ports
16 cards (*1)
(*1) Three cards per 10GbE IO Unit and

four cards per 1GbE IO Unit
For details on the configuration, refer to "General Description".

14
1.5.2 Basic Configuration (2400E) (1/2)

n 2400E Layout
Basic components
Component
Max.
installable
number
Installed as
standard
No
SB
(must be
ordered)
IOU
(must be
ordered)
MMB
PSU
(must be
ordered)
FAN
No
DU
No
AC cable
No
No
Front
Rear
15
No
(must be
ordered)

n Internal Connection Configuration for 2400E

Component

PCI Express slots on PCI Boxes
Maximum
numbers of
ports and cards
8 ports
16 cards (*1)
48 cards

* The number of available PCI Box slots
depends on which type of IO Unit is
selected.

16

n 2800E Layout
Basic components
Component
Max.
installable
number
Installed as
standard
SB
(must be
ordered)
IOU
(must be
ordered)
MMB
PSU
(must be
ordered)
FAN
No
DU
No
No
No
No
Front
Rear
No
AC cable
17
(must be
ordered)

n Internal Connection Configuration for 2800E

Maximum
numbers of
ports and
cards
Component
8 ports
16 cards (*1)
PCI Express slots on PCI Boxes
48 cards

* The number of available PCI Box slots
depends on which type of IO Unit is
selected.

18
1.6 Features (1/7)

n Hardware Partitioning (PPAR) [2400E/2800E Only]
PPAR creates multiple separate blocks (partitions) in one cabinet and operates
an independent system in each partition to allow flexible system operation.
A different OS can be operated in each partition. Configuration example: 8 socket model
Partition C
Partition A
Partition B
The OS in a partition can be rebooted or
Work A
Work C
Work B
shut down separately from other partitions.
The partitions can be configured flexibly by
QPI network
using flexible I/O and Reserved SBs.
SB#1
SB#3
SB#2
SB#0
For details, refer to "3.2 Key Points
for Hardware Partitioning Design".
Multiple works can be constructed in a single
cabinet, and multiple servers that are
IOU #2
IOU #3
IOU #0
IOU #1
operated separately from each other
can be integrated in the cabinet.
If a failure occurs in a partition,
the hardware can protect other partitions
from being affected by the failure.
PCI Express
switch
PCI Express
card
19
PCI Express
switch
PCI Express
card
PCI Express
switch
PCI Express
card
PCI Express
switch
PCI Express
card
1.6 Features (2/7)

n Memory Mirror Function
For details, refer to "2.2 System Board (SB)".
n The Full Mirror Mode (which allows mirroring for all the memory that is installed in a
PPAR block) and the Partial Mirror Mode (which allows mirroring for only the
memory on the Home SB in a PPAR block) are supported.
n Memory Spare Function

This function ensures uninterrupted system operation by proactively monitoring
memory to detect a precursor of a problem at the earliest possible time. Data is copied
if a precursor of a problem is detected.
n Flexible I/O [2400E/2800E Only]

n Partitioning does not have any limitations on the placement of SBs and IOUs.
n Flexible I/O enables the Reserved SB function by using alternative SBs.
20
1.6 Features (3/7)

n Reserved SB Function [2400E/2800E Only]
For details, refer to "3.2 Key Points for Hardware Partitioning Design".
This function automatically disconnects a failed SB, incorporates a spare SB that has
been configured in advance, and starts PPAR to continue system operation.
n Degradation Function
If redundantly configured hardware fails, this function isolates the failed component
and configures the system with the remaining hardware resources to continue system
operation.
n Dynamic Reconfiguration [2400E/2800E Only]

Dynamic Reconfiguration links with RHEL to enable the addition or the removal of
hardware resources such as CPUs, memory, and I/O in a partition without stopping the
PPAR system.
Resources are added or removed in units of SBs or IOUs with the PCI Box LH modules.
* Dynamic Reconfiguration as a hardware and firmware function of PRIMEQUEST
supports online removal of SBs. However, as of when this manual is published, no
OS vendor states availability of this function.
21
1.6 Features (4/7)

n RAID
For details, refer to "3.3 Key Points for I/O Configuration Design".
Software RAID and hardware RAID ensures the reliability and availability of the
system.
n Redundancy
By duplicating or multiplexing internal components, even if memory or other hardware
fails, it is possible to prevent the system operation from being stopped.
n Hot Plug
A component that is stopped due to a failure can be replaced without stopping the
operation of the partition.
n Original Dump Function

For details, refer to "3.3 Key Points for I/O Configuration Design".
This function collects system information when a problem occurs in the OS.
22
1.6 Features (5/7)

n Cluster System
For details, refer to "3.4 Key Points for Cluster Configurations".
Partition redundancy that is based on an inter-cabinet cluster or an intracabinet cluster is supported.
n Boot Devices
The following boot types are supported:
Internal HDD/SAS-SSD (RAID card)
HDD/SAS-SSD installed in a JX40 (RAID card)
iSCSI-SAN boot
FC-SAN boot
FCoE boot
23
1.6 Features (6/7)

n Remote Operation Function
n Wake On LAN (WOL) Function
This function enables the power of partitions to be controlled remotely.
Both onboard LANs on IOUs and LAN cards are supported.
n Preboot eXecution Environment (PXE) Function

This function enables operations such as installing and updating the OS to be managed from
a server in the network.
Only onboard LANs on IOUs are supported.
n Console Functions
Video Redirection Connection
This function remotely performs KVM operations from a console that is connected to a LAN.
This allows a remote storage function (in which FD/CD/DVD drives on the console PC can be
used as the drives for the PRIMEQUEST).
Text Console Redirection (for Maintenance)
This function directs the serial output from a partition to a terminal via a LAN.
n USB Port Disable Function

The front USB ports on the Home SB can be enabled or disabled in the BIOS menu.
24
1.6 Features (7/7)

n Green Compliance and Power Saving
n Power Saving Function
This function automatically adjusts the system operation so that the power
consumption does not exceed the specified maximum value.
n Energy-Saving Measures
Power-efficient devices (LV-DIMM and SAS-SSD) are available.
A highly-efficient power supply unit (80 PLUS platinum certified) is available.
n Optimized Cooling
Cooling is optimized by finely controlling the FAN speed for each cooling group.
n Reduction of Power Consumption

Power capping for PRIMEQUEST can automatically lower system performance so as
to keep power consumption under pre-determined value.
n Compliant with RoHS Directive (Revision <B> in September 2010)
25
2. Basic Hardware Configuration

This chapter describes the basic configuration of each
component.
For details, refer to "General Description".
26
2.1 Operator Panel (OPL)

The panel for displaying the system status
n System LED Control
A device (I2C LED Driver) is installed for controlling the system LEDs that display system
status.
n Suction Temperature Sensor

A sensor for monitoring suction temperature of the main unit is installed.
n System FRU
A system FRU (I2C EEPROM) for storing the information of the main unit is installed.
System LEDs
27
2.2 System Board (SB)

A board for installing CPUs and memory. Internal storage can be
installed in models that are more higher-end than the 2400E.
n Features
Up to four SBs can be installed in a cabinet
(or up to two SBs for a 2 or 4 socket model).
At least one CPU and a set of memory modules (two modules) must be installed on an SB.
Up to two CPUs, 24 sets of memory modules (48 modules), one RAID card, four 2.5-inch
internal hard disk drives (HDDs) or internal solid state drives (SAS-SSDs) can be installed on
an SB.
Hot Plugging can be performed for internal HDDs and SAS-SSDs (in a RAID configuration).
RAID0, RAID1, RAID1E, RAID5, RAID6, and RAID10 are supported.
A hot spare disk is supported.
n External Ports
The following ports are supported for the local KVM for a partition.
(The local KVM cannot be connected permanently.
Use the video redirection function.)
When multiple SBs are used to configure partitions,
only the external ports of the Home SB are available.
USB 1.1/USB 2.0 x 4 (USB Type A)
VGA port
VGA x 1 (D-sub 15pin)

USB ports
28
2.2.1 CPU (1/4)

Intel Xeon processors are used.
A QPI interface and memory controllers are integrated in these processors.
It is possible to scale up the configuration by connecting CPUs via the QPI.
CPUs Supported by the PRIMEQUEST 2000 Series
Performance
Number of
cores
2800B
2400E
2800E
Intel Xeon Processor E7-8890V2
2.80GHz/37.5MB
15
Yes
Yes
2.80GHz/37.5MB
15
Yes
2.50GHz/37.5MB
15
Yes
Yes
2.50GHz/37.5MB
15
Yes
2.30GHz/30.0MB
15
Yes
Yes
2.30GHz/30.0MB
15
Yes
3.00GHz/30.0MB
12
Yes
Yes
2.30GHz/24.0MB
12
Yes
Yes
2.30GHz/24.0MB
12
Yes
3.40GHz/37.5MB
Yes
Yes
Yes
Product number
Yes: Installable
* For the latest information on the supported CPUs, contact your sales representative.
29
-: Not installable
2.2.1 CPU (2/4)

n QuickPath Interconnect (QPI)
A high-speed system bus delivering up to 8.0GT/s (bandwidth 32GB/s)
The QPI connects a CPU and a chipset or connects CPUs to each other.
n CPU Installation Conditions

For details, refer to "Appendix G Component Mounting Conditions" in
"Administration Manual".
CPUs must be installed from CPU#0 on an SB in the order of the slot numbers.
SBs in which no CPU is installed cannot be incorporated in a PPAR block.
At least one set of memory modules (two modules) must be installed for a CPU.
n Conditions for Installing Different CPUs

Different types of CPUs can be installed in different PPAR blocks.
(The CPUs must be supported in the PRIMEQUEST model.)
30
2.2.1 CPU (3/4)

nConditions for Installing Different CPUs (1/2)
Different types of CPUs can be installed
in different PPAR blocks of a cabinet (*).
Partition#0
Different CPUs can

be installed in
different PPAR
blocks.
Partition#1
SB#0
CPU
Partition#0
SB#1
CPU
CPU
SB#0
CPU
CPU
Flexible I/O
IOU#0
SB#1
CPU
CPU
CPU
Different CPUs
cannot be
installed in the
same PPAR block.
Flexible I/O
IOU#1
IOU#0
IOU#1
(*) The CPUs must be supported in the PRIMEQUEST model.
31
2.2.1 CPU (4/4)

nConditions for Installing Different CPUs (2/2)
Different types of CPUs cannot be installed
in the same PPAR block of a cabinet.
Partition#0
Partition#0
SB#0
Different types of
CPUs cannot be
installed in the same
PPAR block.
CPU
CPU
Flexible I/O
Flexible I/O
IOU#0
Reserved SB
Partition#0
SB#0
SB#1
CPU
IOU#0
This installation is not

allowed because
PPAR#0 is switched
from SB#0 to SB#2 if
a failure occurs in
SB#0.
SB#0
CPU
CPU
Partition#1
SB#1
CPU
CPU
SB#2
CPU
CPU
Reserved SB
Partition#0
SB#0
CPU
IOU#1
CPU
SB#1
CPU
Flexible I/O
CPU
SB#2
CPU
CPU
CPU
Flexible I/O
IOU#0
IOU#0
32
2.2.2 Memory (1/7)

Supported memory types are 8GB, 16GB, 32GB, and 64GB DDR3 DIMM.
n Install a set of two DIMMs.
Install a set of four DIMMs to use the Memory Mirror function

(Refer to 2.2.2 Memory (4/7).)
Install a set of six memory modules to use the Memory Spare function
When either Dynamic Reconfiguration is used or one PPAR block is configured with eight CPUs
and the Segment Mode is set as the PCI Address mode.
Memory Slot Arrangement
n For one CPU, six sets of memory modules can be installed

in the memory slots on the SB and another six sets of
memory modules can be installed on an optional extended
memory board.
n Installed memory operates only when the relevant CPU is operating.
n Installation Group
Install memory modules on a set basis in two memory slots with

memory slot numbers whose third digits have a difference of 3.
Installation Group (1 CPU and Normal Mode)

Installation group
Memory slot numbers
0A0, 0A3
0A1, 0A4
0A2, 0A5
0B0, 0B3
0B1, 0B4
0B2, 0B5
24
1D2, 1D5
: Memory slots on an extended memory board

33
2.2.2 Memory (2/7)

n Memory Installation Conditions
For one CPU, 1 to 12 sets of memory modules can be installed.
Memory must be added on a set basis.
(Add memory on a two-set basis when the Memory Mirror function is used or on a three-set basis when the
Memory Spare function is used.)
When only one set of memory modules is installed, if one module fails, the SB is disconnected from the
PPAR block. Therefore, it is recommended to install at least two sets of memory modules.
Only the same memory modules can be installed to the same memory channel of each Memory Buffer.
Different memory modules can be installed for different memory channels.
A "memory channel" is a DDR3 interface

connected to a Memory Buffer.
"Same memory channel"

The same memory modules
must be installed.
"Same memory channel"

The same memory modules
must be installed.
* MB: Memory Buffer
When the Memory Mirror function is used, install the same configuration of memory (in size and the
number of modules) in multiple memory channels that are used for mirroring.
When the Memory Spare function is used, install the same memory modules in all six slots of the
same Memory Buffer.
34
2.2.2 Memory (3/7)

n Memory Installation Order
When installing memory modules in the memory slots whose slot numbers have the same first and second digits, install the
modules in the order of the third digit pairs, i.e., (0 and 3), (1 and 4), and (2 and 5).
Refer to the table in the following "Example of Installation Order".
There are multiple patterns in installation order depending on whether the Memory Mirror function or Memory Spare
function is used or not.
For details on installation order, refer to "G.2.1 DIMM mounting sequence" in "Administration Manual and the
following slides.
[Example of Installation Order] 1 CPU and Normal Mode (The same type of memory is shown in the same color.)
CPU#
MB#
35
Memory
slot#
Installation
order
0D5
0D4
0D3
0D2
0D1
0D0
0C5
0C4
0C3
0C2
0C1
0C0
0B5
0B4
0B3
0B2
0B1
0B0
0A5
0A4
0A3
0A2
0A1
0A0
12
8
4
12
8
4
10
6
2
10
6
2
11
7
3
11
7
3
9
5
1
9
5
1
2.2.2 Memory (4/7)

DIMM installation order for each mode
CPU#0
Slot
No.
Normal
Mode
Mirror
Mode
Spare
Mode
CPU#1
0A0
0A3
0B0
0B3
0C0
0C3
0D0
0C3
1A0
1A3
1B0
1B3
1C0
1C3
1D0
1D3
0A1
0A4
0B1
0B4
0C1
0C4
0D1
0C4
1A1
1A4
1B1
1B4
1C1
1C4
1D1
1D4
0A2
0A5
0B2
0B5
0C2
0C5
0D2
0C5
1A2
1A5
1B2
1B5
1C2
1C5
1D2
1D5
12
12
10
10
14
14
13
13
11
11
15
15
16
16
20
20
18
18
22
22
17
17
21
21
19
19
23
23
10
10
10
10
11
11
11
11
(Slots with the same symbols must have

the same type of DIMM installed.)
Install DIMMs at the one time in the slots with the same installation order number.
When only one CPU is installed, install the DIMMs in the slots starting from the smallest slot number on CPU#0.
Install DIMMs in pairs of two.
Refer to the next slide when one PPAR block is configured with eight CPUs (Segment) or the Dynamic Reconfiguration
function is used.
36
2.2.2 Memory (5/7)

DIMM installation order for each mode (when one of the following conditions apply)
Dynamic Reconfiguration is used
One PPAR block is configured with eight CPUs when the Segment Mode is set as the PCI Address mode
CPU#0
Slot
No.
Normal
Mode
Mirror
Mode
Spare
Mode
CPU#1
0A0
0A3
0B0
0B3
0C0
0C3
0D0
0C3
1A0
1A3
1B0
1B3
1C0
1C3
1D0
1D3
0A1
0A4
0B1
0B4
0C1
0C4
0D1
0C4
1A1
1A4
1B1
1B4
1C1
1C4
1D1
1D4
0A2
0A5
0B2
0B5
0C2
0C5
0D2
0C5
1A2
1A5
1B2
1B5
1C2
1C5
1D2
1D5
(Slots with the same symbols must have

the same type of DIMM installed.)
Install DIMMs at the one time in the slots with the same installation order number.
When Dynamic Reconfiguration is used or one PPAR block is configured with eight CPUs, two CPUs are installed in an SB.
When only one CPU is installed and the Dynamic Reconfiguration is enabled, the IOU- Dynamic Reconfiguration requires
memory installation for CPU#0 in the above order.
Install DIMMs in pairs of two.
37
2.2.2 Memory (6/7)

n Conditions of Installing Different Memory Modules
16GB memory (8GB single-rank (*1) DIMM x 2) and 32GB memory (16GB dual-rank (*1) DIMM
x 2) can be installed in the same partition.
Different types of memory modules other than the above combination cannot be installed
together in the same partition.
Different types of memory modules can be installed in a different partition on a different SB.
Partition#0
64GB
(32GB DIMM x 2)
16GB
(8GB DIMM x 2)
Partition#0
64GB
(32GB DIMM x 2)
Partition#1
32GB
(16GB DIMM x 2)
16GB
(8GB DIMM x 2)
(*1) Memory rank: A unit of data input and output by a memory controller from the DRAM
chips on a DIMM
Memory access performance increases as the rank number increases.
38
2.2.2 Memory (7/7)

n Memory Mirror
Mirroring is performed between the memory group in MB#xA and MB#xB and the memory
group in MB#xC and MB#xD that are controlled by the same CPU.
Memory Mirror is available even if only one CPU is installed in an SB.
n Memory Spare
When multiple DIMM pairs are configured, a DIMM rank is used as a spare of other ranks.
If the number of correctable errors that occur in a specific time period in a rank exceeds a
threshold, the contents of the memory are copied to the rank that has been reserved as a
spare and the DIMM rank in which the errors occurred is disconnected.
* Memory Mirror and Memory Spare cannot be enabled together.
Memory Mirror
Memory Spare
Mirroring is available only between the same types of memory.

Single rank
Intra-CPU
mirroring
DIMM pair
DIMM pair
DIMM pair
Rank 0 Rank 0
Rank 0 Rank 0
Rank 0 Rank 0
DIMM A
DIMM C
DIMM E
DIMM B
DIMM D
Spare
DIMM F
* The available memory capacity is two-thirds of the total capacity of the installed memory.
Dual rank
DIMM pair
DIMM pair
DIMM pair
Rank 0 Rank 0
Rank 0 Rank 0
Rank 0 Rank 0
Rank 1 Rank 1
Rank 1 Rank 1
Rank 1 Rank 1
DIMM A
DIMM C
DIMM E
DIMM B
DIMM D
Spare
DIMM F
* The available memory capacity is five-sixths of the total capacity of the installed memory.
39
2.3 Management Board (MMB) (1/3)

A system control board that is equipped with an independent
dedicated processor
n Features
A redundant configuration can be achieved
by installing two boards.
Hot swap is possible only in a redundant configuration.
The dedicated CPU operates as a dedicated host for system management.
A Web server that is included in an MMB provides a graphical user
interface.
An MMB can be operated from a Web browser (GUI) on a remote PC, and it
is not necessary to prepare an operation management server separately.
Hardware management and monitoring function
Consolidated management and monitoring of the power supply, fan, and
hardware of each partition
Hardware setup (configuration change) function
Power control in the cabinet, sensor monitoring, partition configuration
setting, Memory Mirror mode setting, and reset process
Collection and display of operation logs and error logs
40

High-Level Security
An original protocol is used to perform internal communication.
Server management interfaces (Web Server and SNMP) are managed collectively.
It is possible to add user definitions that allow operating only specific partitions.
This function prevents wrong operations and malicious operations on partitions.
SB
(Partition)
SB
(Partition)
Management and monitoring
MMB (One board is included in the default configuration. Another MMB can be added for redundancy.)
41

nExternal Ports
User port x 2
Used to connect to the management LAN for operation management of
PRIMEQUEST.
Duplication is possible.
Port for maintenance x 1

Used by field engineers for maintenance.
COM port for an MMB console

Used for initial setting.
Port for
maintenance
User ports
COM port for an MMB

console
42
2.4 IO Unit (IOU) (1/3)

A unit that is equipped with PCI Express slots and LAN ports
There are two types of IOUs: 1GbE and 10GbE.
n Features
An IOU connects components and external devices to an SB.
Up to four IOUs can be installed in a cabinet.
The number of PCI Express slots that can be installed in each IOU is as follows:
- 1GbE IO Unit : 4
- 10GbE IO Unit : 3
To connect a PCI Box, install a PCI Box connection card in the specified PCI Express
slot on the IOU.
1GbE IO Units and 10GbE IO Units can be installed together in the same cabinet.
43
2.4 IO Unit (IOU) (2/3)

n 1GbE IO Unit
A 1GbE IO Unit has two 1000BASE-T LAN ports.
Up to four PCI cards with low profile bracket sizes can be
installed.
n 10GbE IO Unit
A 10GbE IO Unit has two 10GBASE-T LAN ports.
Up to three PCI cards can be installed.
(If the bracket size of the PCI card is full height, up to
two cards can be installed.)
1GbE IO Unit
10GbE IO Unit
44
2.4 IO Unit (IOU) (3/3)

nIOU Configuration Examples
[Example with LAN
Cards]
Addition of LAN ports in
a cluster configuration
[Example with FC Cards]

Connection to an
external storage device
(ETERNUS)
[Example with a SAS Array

Controller Card]
Connection to an external hard
disk cabinet (JX40)
JX40
[Example with a SAS Card]

Connection to a backup
cabinet (PRIMERGY SX05)
PRIMERGY SX05
ETERNUS
45
2.5 Disk Unit (DU)

A unit in which internal storage is installed
n Features
Up to two units can be installed in a cabinet.
Up to four 2.5-inch internal hard disk drives (HDDs) or
internal solid state drives (SAS-SSDs) can be installed
in a unit.
Hot Plugging can be performed for internal HDDs and SAS-SSDs
Example with one RAID Card
Example with two RAID Cards
(in a RAID configuration).
RAID0, RAID1, RAID1E, RAID5, RAID6, and
RAID10 are supported.
A hot spare disk is supported.
Up to two RAID cards can be installed
in a unit.
PCI Express Slot#0
(RAID Card)
HDD#0
HDD#1
HDD#2
HDD#0
HDD#1
HDD#3
SAS
SATA
PCI Express Slot#0

(RAID Card)
46
HDD#2
SAS
SATA
HDD#3
For details on RAID configurations,

refer to "3.3.4 HDD Configuration".
From
IOU#1/IOU#3
PCI Express Slot#0

(RAID Card)
From
IOU#0/IOU#2
From
IOU#0/IOU#2
2.6 PCI Box (1/4)

An extended I/O cabinet that is equipped with 12 PCI Express slots, each
having eight PCI Express Gen3 (8Gbps) lanes
n
Features
Up to four PCI Boxes can be connected to a cabinet.
(No PCI Box can be connected to 2800B.)
The PCI Box connection card on an IOU is used to connect a PCI Box.
PCI Hot Plug (PHP) for PCI Express cards is supported.
A PCI Box can be divided into two partitions (PCI Box LH modules), each having six slots.
PCI Boxes, power supply units and power cables for PCI Boxes need to be obtained separately.
The PHP function is not

available when virtualization
software is used.
PCI Box LH
IOU connection
port
Six PCI Express

(eight lanes) slots
PHP
PEXU: Abbreviation of a PCI Express Unit
PCI Box LH
LNKC: Abbreviation of a LINK Card. These cards

are installed in PCI Boxes and are used
to connect to IOUs.
For details on the power supply configuration for a PCI Box, refer to "3.3.7 Power Supply Redundancy".
47
2.6 PCI Box (2/4)

n IOU combinations and the available number of PCI slots
Model
2800B
2400E
2800E
Number of
1GbE IO Units
Number of
10GbE IO Units
Number of PCNCs
Number of
PCI Boxes
(2400E/2800E only)
Available number of
slots
4
3
3
2
2
2
1
1
1
1
0
0
0
0
0
1
0
2
1
0
3
2
1
0
4
3
2
1
8
7
6
6
5
4
5
4
3
2
4
3
2
1
4
3.5
3
3
2.5
2
2.5
2
1.5
1
2
1.5
1
0.5
56
50
42
44
36
28
38
30
22
14
32
24
16
8
48
2.6 PCI Box (3/4)

nPCI Box Connection Conditions
Different cabinets cannot be connected to a PCI Box (PCI Boxes cannot be shared
by servers).
Straight Connection
Cross Connection
Connection to Different PCI Boxes
Connection from 1GbE IO Units and

10GbE IO Units
Connection from Different Cabinets to a PCI Box
* The connection configuration depends on the type of IO Unit that is to be installed.

49
2.6 PCI Box (4/4)

n A Configuration for Redundancy of PCI Boxes and PCI Box LH modules
The effects of a failed PCI Box are taken into account.
PRIMEQUEST
PCI Box #0
PCI Box LH #0
Failure
PCI Box #1
PCI Box LH #1
PCI Box LH #1
PCI Box LH #1
Partition#0
SB
Stop
IOU
Start in a
degraded state
Partition#1
SB
Stop
IOU
Start in a
degraded state
Redundancy of the PCI Boxes and PCI Box LH

modules
The PCI Box is

disconnected
by a reboot.
n A Configuration for Failure Isolation of PCI Boxes

PRIMEQUEST
Partition#0
SB
PCI Box #0
PCI Box LH #0
PCI Box #1
PCI Box LH #1
PCI Box LH #0 PCI Box LH #1
IOU
Not
affected
Partition#1
SB
IOU
Stop
Failure
The PCI Box cannot be used

until it is repaired.
Advantage
Even if one PCI Box fails, the failed
PCI Box can be disconnected by a
reboot and the system can be
started using the other PCI Box.
Disadvantage
Both partitions stop when a PCI Box
fails.
The failure in partition#1 is isolated from partition#0.
Advantage
This configuration provides failure
isolation. Even if the PCI Box for a
partition fails, the other partition is
not affected.
Disadvantage
The PCI Box for the relevant partition
cannot be used until the PCI Box is
repaired.
For details on redundancy of PCI Express cards and connection destinations, refer to "3.3.5 Network Configuration".
50
2.7 Power Supply Unit (PSU)

A power supply unit (with FANs installed) that converts AC200V or AC100V to DC12V
For details on connecting this unit, refer to "2.3 Power cable connection" in "Hardware
Installation Manual".
n Up to six PSUs can be installed in a cabinet.
For details on the number of PSUs that must be installed, refer to "4.5.1 PSU Configuration"
in "General Description".
n The following three types of units are available

(but cannot be installed together).
High-efficiency power supply unit (200V): 80 PLUS Platinum certified

Power supply unit (200V): 80 PLUS Silver certified
Power supply unit (100V): 80 PLUS Silver certified (2800B and 2800E do not support
100V.)
n Power Supply Configuration

Input
voltage
100V
Power feeding
method
Primary power
feed
Redundancy
Non-redundant
Redundant
2400E
2800B/2800E
4+0
Not supported
4+1
Not supported
Dual power feed

200V
Primary power
feed
Not supported
Non-redundant
Redundant
Dual power feed

3+0
3+0
3+1
3+1
3x2
3x2
51
2.8 FAN Unit (FANU)

A unit that contains only a FAN and can be installed in a PSU slot in which no PSU
is installed.
n A total of six units between the PSUs and the FANUs must be installed.
For details on the number of FANUs that must be installed and the
installation slots, refer to "4.5.1 PSU Configuration" in "General
Description".
A total of six units

between the PSUs and
the FANUs are installed.
52
2.9 Peripherals Configuration (1/4)

n Console Unit
A PC is required to connect a console.
For details, refer to "4.1 Key Points for Console Operation".
53

n Rack
For details on racks that can be installed and the use of rack products, contact your
sales representative.
n How to Select Racks

(1) Check whether or not
racks must be secured
to the floor.
Racks
must be
secured.
* Fujitsu recommends setting rack stabilizer against earthquakes

with 80 gal or more.
(3) Select a rack securing
method.
(2) Select racks.
Rack securing method

using an earthquakeresistant kit
Rack without a stabilizer
(4) Arrange options.

Earthquake-resistant
kit
- Four sides (for a basic rack)
- Two sides
(for an additional rack)
Rack securing method

using level feet
None
Racks
do not
need to be
secured.
Rack with a stabilizer
None
Pullout type
54

nNotes on Using Racks
When installing devices in a rack, basically install heavier devices in the
lower spaces to lower the center of gravity of the rack and stabilize the
rack.
Attach a blank panel to the front of each unused space of the rack.
For PRIMEQUEST, it is recommended to
install one PRIMEQUEST unit per rack.
(In some cases, two units can be installed.)
Rack External View
PRIMERGY and ETERNUS can also be

installed.
PRIMERGY
PRIMEQUEST
x 1 unit
(recommended)
ETERNUS
For details on devices that can be installed, visit our website at

http://www.fujitsu.com/global/services/computing/server/primequest/products/peripheral/.
55

nRack Installation Requirements
A. Allowable spacing between posts: 685 to 790mm
B. Distance from the inside of the front door to the outside
of the front rack posts: 60mm or more
C. Distance from the outside of the front rack posts to the inside
of the rear door: 860mm or more
A
Rear door
Front door
Front
Posts
Posts
For details on rack installation conditions, refer to "Hardware Installation Manual".

56
3. Hardware Configuration Design

This chapter describes the key points for designing hardware
configurations.
57
3.1 Key Points for MMB Design

n When focusing on reliability, install two MMBs to create a
redundant configuration
n Considerations Before Setting an MMB
n IP address, host name, subnet mask, and gateway address
n User accounts
n Name of the PRIMEQUEST system (this is also used as the SNMP system
name.)
n Management VLAN (IP addresses that can be switched between MMBs)
environment
For details, refer to "2.2 Key Points for the MMB Settings" in
"Linux Installation Guide" and "3.3 Connection and Setting of MMB"
in "Installation Manual".
58
3.2 Key Points for Hardware Partitioning Design

Key Points for Configuring PPAR blocks
n Partitioning Design Rules
For details, refer to "3.2.1 Partitioning Design Rules".
n Key points for partitioning design
n Partition granularity and configuration conditions
n The total numbers of SBs and CPUs per partition
n Functions That are Used for Partition Configuration

For details, refer to "3.2.2 Functions That are Used for Partition Configuration".
n Home SB
n Reserved SB function
n Dynamic Reconfiguration function
n Switching Reserved SB Rules

For details, refer to "3.2.3 Switching Reserved SB Rules".
n Configuration Example
For details, refer to "3.2.4 Configuration Example".
n Guidelines for selecting a partition configuration
n Configuration examples for each model
59
3.2.1 Partitioning Design Rules (1/2)

n One or more usable SBs and one or more usable IOUs are required in one
partition.
n For important works, it is recommended to set them in a partition with the
lowest partition number and also incorporate SBs with lower SB numbers in the
partition.
n It is recommended to specify the SB with the lowest SB number as the Home SB.
n When including DUs and PCI Boxes in a partition, include the IOU for
connecting these components in the same partition.
n Note that the performance and the number of software licenses may vary if
Reserved SB switching occurs or the system configuration is changed.
Partition Granularity and Configuration Rules
Component name
Minimum
Maximum
Configuration conditions
SB
1 board
4 boards
None
IOU
1 unit
4 units
None
Half of a DU: RAID card slot x 1, HDD bay x 2
0 units
The number of IOUs
Must be connected to an IOU
PCI Box LH (half of a PCI Box: LINKC x 1, PCI

Express slot x 6)
0 units
The number of IOUs
Must be connected to an IOU
* If an upper component is degraded, the lower components also cannot be used.

(If an IOU is degraded, the DUs and PCI Boxes that are connected to the IOU are also degraded.)
60
3.2.1 Partitioning Design Rules (2/2)

n The Total Numbers of SBs and CPUs per Partition
n When multiple SBs are used to configure a partition, two CPUs must be
installed in each SB.
n In a default configuration, an SB with two CPUs and an SB with one CPU in
one partition (2SB-3CPU) cannot be installed.
Partition configuration
(Number of SBs per partition)
Total number of
CPUs
2800B
2400E
2800E
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Three SBs
Yes
Yes
Four SBs
Yes
Yes
One SB
Two SBs
Yes: Possible
-: Not possible
61
3.2.2 Functions That are Used for Partition Configuration - Home SB
n Home SB
The main SB among the SBs that are used to configure a PPAR block.
n Functions That Are Enabled on the Home SB

PCH
Onboard I/O is enabled, and USB ports and a VGA port can be used only on the
Home SB.
Reference clock source
The clock source of the Home SB becomes the clock source in the partition.
n Selection of the Home SB

Specify the Home SB by using the MMB Web-UI.
If the Home SB is not specified directly, the first SB that is incorporated in the
partition is specified as the Home SB.
* If the Home SB is removed from the partition or is degraded, the SB with the lowest
number in the partition is specified as the Home SB.
62
3.2.2 Functions That are Used for Partition Configuration - Reserved SB (1/6)
n Reserved SB Function
n This function automatically disconnects a failed SB, incorporates a spare SB that has been
implemented in advance, and restarts the PPAR block.
The SB resources are not degraded, and the partition can be recovered quickly.
In a partition that has only one SB, even if the SB fails (or it is degraded), the partition can be recovered.
n During operation of a partition, an SB in the partition can be specified as a Reserved SB.

An SB that is used in a partition of a test environment during normal operation can be incorporated in a
partition of the production environment as a Reserved SB if a failure occurs.
n 2800B is not supported.

Operation Example in Which an SB in a Test Environment Partition Is Changed to a Reserved SB
Partition#1
Partition#1
SB#0
SB#1
SB#2
SB#3
SB#0
SB#1
SB#2
SB#3
IOU#0
IOU#1
IOU#2
IOU#3
IOU#0
IOU#1
IOU#2
IOU#3
: Home SB
: Reserved SB
Test environment partition
Test environment partition

(Cannot be operated)
63
3.2.2 Functions That are Used for Partition Configuration - Reserved SB (2/6)
n Wait Time for Reserved SB Switching

From 0 to 99 minutes can be set for the shutdown wait time when a Reserved SB
that is used in another partition is switched by using the MMB Web-UI (the
default setting is 10 minutes).
If the partition cannot be shut down after a specified time is passed, the
Reserved SB is forced to be activated.
n Save of Partition Configuration Information

Before Reserved SB switching is performed, the partition configuration
information for the entire cabinet is automatically saved to the MMB.
The partition configuration information for the last time a Reserved SB was
operated is saved.
The saved information can be viewed from the MMB Web-UI.
64
3.2.2 Functions That are Used for Partition Configuration - Reserved SB

(3/6)
n Reserved SB Setting Rules (1/4)
Consider the priority of partitions and do not specify SBs in different partitions with each other as
Reserved SBs when multiple partitions are used or do not perform any loop settings for Reserved SBs.
Any SB that does not belong to the local partition can be set as a
Reserved SB.
OK
Reserved SB
SB#0
CPU
SB#1
CPU
IOU#0
OK
Reserved SB
CPU
CPU
SB#0
CPU
The SBs that are incorporated in a partition can also be set as Reserved SBs.
For a single partition, multiple Reserved SBs can be set.
Reserved SB
CPU
SB#1
CPU
CPU
SB#2
CPU
CPU
CPU
CPU
CPU
IOU#1
Partition#0
Partition#1
Reserved SB
SB#0
CPU
CPU
IOU#0
CPU
Reserved SB
SB#1
SB#2
CPU
CPU
CPU
IOU#0
Partition#0
CPU
OK
Reserved SB
SB#3
CPU
CPU
For a partition that consists of one SB, an SB with one or two CPUs
can be set as a Reserved SB.
OK
SB#0
CPU
IOU#0
Partition#1
SB#2
SB#1
CPU
IOU#1
Partition#0
OK
Reserved SB
SB#2
CPU
An SB can be set as a Reserved SB for multiple partitions.
Partition#0
65

(4/6)
For a partition that consists of multiple SBs, only an SB with two
CPUs can be set as a Reserved SB.
NG
OK
CPU
SB#1
CPU
CPU
NG
OK
Reserved SB
SB#0
Only an SB that has the same DIMM installation conditions as the

partition can be set as a Reserved SB.
Reserved SB
SB#2
SB#3
CPU
CPU
CPU
Reserved SB
SB#0
CPU
8GB
IOU#0
SB#1
8GB
8GB
SB#2
8GB
32GB
Reserved SB
SB#3
32GB
16GB
16GB
IOU#0
Partition#0
Partition#0
Reserved SB settings that make a configuration violate the CPU installation rules after
an SB is replaced with a Reserved SB are not supported.
Reserved SB settings that make a configuration violate the rules for installing different
types of memory modules after an SB is replaced with a Reserved SB are not supported.
Only an SB that satisfies the "Conditions for Installing Different

CPUs" in regard to partitions can be set as the Reserved SB.
NG
OK
Reserved SB
SB#0
CPU
SB#1
CPU
CPU
SB#2
CPU
CPU
Reserved SB
SB#3
CPU
CPU
CPU
IOU#0
Partition#0
Different types
Reserved SB settings that make a configuration violate the rules for installing different
types of CPUs after an SB is replaced with a Reserved SB are not supported.
66

(5/6)
n In some SB configurations, the amounts of resources (the number of CPUs and/or the number of
memory modules (memory capacity)) decrease or increase after switching.
It is necessary to check the estimated performance and the estimated number of software licenses in
advance.
* If a volume license or package product is used in a Windows OS or if the SB being used is not an SB that was purchased together with the
enable kit, license authentication may be demanded after Reserved SB switching occurs.
SB#0
CPU
IOU#0
Partition#0
Failure
CPU
CPU
1CPU
Partition#1
SB#0
IOU#0
CPU
SB#2
IOU#1
Partition#0
CPU
Reserved SB
SB#1
CPU
A Reserved SB (CPU x 2) is set for the following

partitions:
Parition#0 (CPU x 2)
Parition#1 (CPU x 1)
SB#1
SB#2
CPU
CPU
CPU
Failed
When the Reserved SB is incorporated in partition#1,

the number of CPUs increases.
Two CPU licenses are required for partition#1.
CPU
IOU#1
Partition#1
2CPU
67

(6/6)
n When using Reserved SBs, use NTP for time synchronization that is performed after switching.
n After the Home SB is switched, the connections of the external (VGA and USB) ports must be changed to
the switched Home SB.
68
3.2.2 Functions That are Used for Partition Configuration

-Dynamic Reconfiguration
n Dynamic Reconfiguration Function

This function allows changing the hardware configuration of a partition while
the OS is running, without a restart.
This function can be used in the following conditions.
n Supported Models
2400E and 2800E
n Supported OSs
RHEL 6.4 and later
n Supported Functions (for RHEL6.4)

Hot addition:
Hot addition of SBs and IOUs with the PCI Box LH modules is possible with
supported OSs and models.
Hot removal:
Hot removal of IOUs with the PCI Box LH modules is possible with supported OSs
and models.
69
3.2.3 Switching Reserved SB Rules (1/5)

n Switching Conditions
An SB that is incorporated in a partition is switched to a Reserved SB when one of the following
events occurs (*1).
n
n
n
n
n
n
n
SB degradation
CPU core degradation
DIMM degradation (even if one DIMM fails)
Detection of partial corruption in the memory of the mirroring pair (*2)
Detection of QPI lane degradation (*2)
Detection of SMI2 lane switchover (*2)
Detection of PCI Express lane/speed degradation (between a CPU and an IOU) (*2)
(*1) Actual Reserved SB switching occurs when the partition is started after a condition for Reserved SB switching
is met.
(*2) Because the partition operation continues, Reserved SB switching occurs when the partition is rebooted
manually.
n When the Home SB is Switched

n If an SB that is not the Home SB is degraded, the Home SB is not changed.
n When the Home SB is switched to a Reserved SB, the SB with the lowest SB number among SBs
including Reserved SBs is changed to the Home SB.
For details on switching conditions, refer to "3.2.3 Reserved SB" in "Administration Manual".
70

n When an SB Is Set as a Reserved SB for Multiple Partitions
If multiple SBs in different partitions fail simultaneously, use the Reserved SB
for the partition with the lowest partition number among the partitions with
the failed SBs.
[Example] An SB Is Set as a Reserved SB for Two Partitions.
(If SB#0 and SB#2 Fail Simultaneously)
Partition#1
Partition#0
Partition#1
Partition#0
H
R R
SB#0
SB#1
SB#2
SB#3
SB#0
SB#1
SB#2
SB#3
IOU#0
IOU#1
IOU#2
IOU#3
IOU#0
IOU#1
IOU#2
IOU#3
Not switched to
the Reserved SB
H
: Home SB
: Reserved SB
The SB with the lowest SB

number is changed to the
new Home SB.
* The setting of SB#3 as the Reserved SB for partition#1 is

automatically cancelled when Reserved SB switching occurs.
71

n When Multiple Reserved SBs Are Set for a Partition
When there are Reserved SBs that do not belong to any partition, the Reserved
SB with the highest SB number is used first.
[Example] Multiple SBs (SB#2 and SB#3) Are Set as Reserved SBs for Partition#0.
(If SB#1 Fails)
Partition#1
Partition#1
Partition#0
R R
Partition#0
R R
R R
SB#0
SB#1
SB#2
SB#3
SB#0
SB#1
SB#2
SB#3
IOU#0
IOU#1
IOU#2
IOU#3
IOU#0
IOU#1
IOU#2
IOU#3
: Home SB
: Reserved SB
72

n When Multiple Reserved SBs Are Set for Multiple Partitions
The priority order of switching is determined by the following conditions.
n The partition with the lowest partition number is selected as the priority
partition for which a Reserved SB is used.
n The Reserved SB with the highest SB number is used first.
[Example] Two SBs Are Set as Reserved SBs for Two Partitions.
(If SB#0 and SB#1 Fail Simultaneously)
Partition#1
SB#0
IOU#0
Partition#1
Partition#0
R R
R R
Partition#0
SB#1
SB#2
SB#3
SB#0
SB#1
SB#2
SB#3
IOU#1
IOU#2
IOU#3
IOU#0
IOU#1
IOU#2
IOU#3
: Home SB
: Reserved SB
73

n When Each Reserved SB Is Incorporated in a Partition
n If One or More Partitions Are Powered Off
Among the powered-off Reserved SBs, the Reserved SB with the highest SB number is used first.
n If All Partitions Are Powered On

The Reserved SB with the highest SB number is used first.
=> Then, the partition with this Reserved SB is shut down.
[Example] When the Reserved SBs (SB#1, SB#2, and SB#3) for Partition#0 Belong to Other Partitions
(If SB#0 Fails)
Partition#0
Partition#1
Partition#2
R R
Partition#1
Partition#3
R R R
Partition#0
Partition#3
R R R
SB#0
SB#1
SB#2
SB#3
SB#0
SB#1
SB#2
SB#3
IOU#0
IOU#1
IOU#2
IOU#3
IOU#0
IOU#1
IOU#2
IOU#3
Powered off
Powered off
Powered off
Powered off
Partition#2
: Home SB
: Reserved SB
Cannot be
used
74
3.2.4 Configuration Example - Guidelines

n Prerequisites
This section assumes that the following items are predetermined.
The number of partitions
The number of CPUs and the number of cores
The lowest memory capacity
n Guidelines for Model Selection

n When I/O scalability is highly important and the Dynamic Reconfiguration function is
used, 2400E or 2800E is recommended.
n When I/O redundancy is highly important, a 2400E model or higher is recommended.
n When higher CPU performance per partition is required, 2800E is recommended.
Item
A guideline of the number of cores
per partition
A guideline of the number of
partitions
Work details
2800B
2400E
2800E
6 to 120 cores
6 to 60 cores
6 to 120 cores
1 partition
1 to 2 partitions
2 partitions or more
Aggregation of PC servers and

higher reliability
75
Core system reconstruction (open MF), UNIX server replacement,

and DB server/ERP infrastructure
3.2.4 Configuration Example - Pattern Overview

n Examples of Determining Partition Configurations
n Focusing on Reducing the Downtime at a Failure (A or B)
For a cluster configuration, a few seconds to a few minutes (a guideline for a twonode configuration)
When Reserved SBs are used, switching time + OS startup time (approximately 10
minutes or longer)
n Focusing on Reliability (Redundancy) (C, D, or E)
Introduce redundancy in CPUs and memory.
n Focusing on the Cost (F)
Configuration Examples
Component or function
Pattern examples
A
Cabinet of the
same model x 2
2800E x 1
2800E x 1
2800E x 1
2400E x 1
2800B x 1
Any
24 - 240
120
120
90
45
90
Memory Mirror
Yes
Yes
Yes
Yes
Yes
Yes
Cluster configuration
Yes
Yes
Yes
Yes
Yes
Cabinets
Number of partitions
Number of CPU cores (*1)
Reserved SB
(*1) Includes the number of CPU cores for Reserved SB

Yes: Supported, - : Not supported

76
3.2.4 Configuration Example - Cluster

(A) Example of a Cluster Configuration (Inter-Cabinet Cluster)
nMemory Mirror configuration
nBy clustering cabinet#0 and cabinet#1, if a failure occurs in
one node, the other node is used and the work continues.
Clustering
Cabinet#0
Cabinet#1
* Cabinet#0 and cabinet#1 have the same configuration.

77
3.2.4 Configuration Example - 2800E

(B) Example of a Configuration with Four SBs, Three Partitions, and
Some Reserved SBs
nEight CPUs are installed in total.
nSB#3 is set as the Reserved SB for partition#0, partition#1, and
partition#2 to reduce the downtime at a failure.
R R
SB#0
SB#1
SB#2
SB#3
IOU#0
IOU#1
IOU#2
IOU#3
P#0
P#1
: Home SB
P#2
: Reserved SB
78

(C) Example of a Configuration with Four SBs and Three Partitions
nEight CPUs are installed in total.
nThree works are performed in different partitions.
SB#0
IOU#0
P#0
R R
SB#3
SB#1
SB#2
IOU#1
IOU#2
IOU#3
P#1
P#2
79

(D) Example of a Configuration with Three SBs and Two Partitions
nSix CPUs are installed in total.
nPartition#0 contains two SBs to achieve redundancy for
reliability.
SB#0
IOU#0
P#0
SB#1
SB#2
IOU#1
IOU#2
IOU#3
P#1
80

(E) Example of a Configuration with Two SBs and Two Partitions
nThree CPUs are installed in total.
SB#0
SB#1
IOU#0
IOU#1
P#0
IOU#2
IOU#3
P#1
81
3.2.4 Configuration Example - 2800B

(F) Example of a Configuration with Three SBs and One Partition
nSix CPUs are installed in total.
SB#0
SB#1
SB#2
IOU#0
IOU#1
IOU#2
IOU#3
P#0
82
3.3 Key Points for I/O Configuration Design

n Boot Type Selection
For details, refer to "3.3.1 Boot Type Selection".
n Guidelines for Selecting Internal Storage

For details, refer to 3.3.2 Guidelines for Selecting Internal Storage".
n Obtaining RAID Cards and FRUs

For details, refer to 3.3.3 Obtaining RAID Cards and FBUs".
n HDD Configuration
For details, refer to "3.3.4 HDD Configuration".
n Guidelines for Selecting a RAID Configuration for HDDs

Software RAID
Hardware RAID
SAN boot
n Memory Dump Function

n Reliable Configuration for Internal HDDs
n Network Configuration
For details, refer to "3.3.5 Network Configuration".
n Network Types and Usage

n Network Redundancy
n I/O Space Settings for PCI Devices

For details, refer to "3.3.6 Setting I/O Space for PCI Devices".
I/O space assignment setting
n Power Supply Redundancy

For details, refer to "3.3.7 Power Supply Redundancy".
Examples of primary power feed and dual power feed
83
3.3.1 Boot Type Selection

Boot type
Internal HDD or SAS-SSD (*1)
Boot devices
DU
Boot operation in which the boot type is used

Start an OS that is installed in an internal HDD.
Internal disk of an SB
External HDD cabinet
JX40 with a SAS card for connection
Start an OS that is installed in an external HDD cabinet.
FC boot in an external file unit

(SAN environment)
FC card
Start an OS that is installed in an external disk array device.
FCoE boot in an external file unit

(SAN environment)
CNA card
iSCSI boot in an external file unit

(SAN environment)
LAN card, and LAN port on an IOU
Preboot eXecution Environment (PXE)
LAN port on an IOU
Start an OS through remote operations.
Remote storage device (*2)
Internal USB port (for video redirection or

remote storage) of an SB
Start an OS from a medium that is set in a remote storage device.
(*1) For details on the internal storage in the cabinet, refer to "3.3.2 Guidelines for Selecting Internal Storage.
(*2) The following remote storage devices can be used.
Internal CD/DVD drive in a PC
Internal floppy disk drive in a PC
USB CD/DVD drive
USB floppy disk drive
n It is recommended to use a redundant HDD (hardware RAID or software RAID) as a boot HDD.
n When an internal disk is used as a boot disk, RAID is configured by using ServerView Installation
Manager (SVIM) to install an OS to the disk.
84
3.3.2 Guidelines for Selecting Internal Storage

n RAID Cards or HDDs/SAS-SSDs in DUs and SBs
Installed in DUs
(not shared between
partitions)
Installed in DUs
(shared between partitions)
Item
Supported models
All models
Installed in SBs
2400E, 2800E
Number of RAID cards
Up to 1
Number of HDDs
Up to 2 per RAID card
UP to 4
Up to 4
When a Reserved SB
switching occurs
Intra-cabinet cluster
configuration
Connection
source
Internal storage is automatically connected to the switched partition
The number of partitions that are used (up

No partitions need to be stopped at the same time because
to 2) must be stopped at the same time
internal storage is not shared between the partitions
when maintenance is being performed
Shared
Not shared
DU#0
RAID#0
HDD #2/3
DU#0
RAID#0
HDD #0/1/2/3
IOU#1
DU#0
RAID#1
HDD #0/1
DU#0
RAID#1
HDD #0/1/2/3
IOU#2
DU#1
RAID#0
HDD #2/3
DU#1
RAID#0
HDD #0/1/2/3
DU#1
RAID#1
HDD #0/1
DU#1
RAID#1
HDD #0/1/2/3
IOU#0
IOU#3
Internal storage is isolated

(unavailable)
SB #m
SB #n
SB #m
SB #n
IOU #0 or 2
IOU #1 or 3
IOU #0 or 2
IOU #1 or 3
DU
(LDU#1)
RAID
card
#0
RAID
card
#1
HDD#2
HDD#0
HDD#3
85
HDD#1
RAID
card
RAID
#0
card
#1
DU
RAID
card
RAID
#1
card
#0
HDD#0
HDD#0
HDD#1
HDD#1
HDD#2
HDD#2
HDD#3
HDD#3
3.3.3 Obtaining RAID Cards and FBUs

n RAID Cards in SBs
n An SAS array controller card installation kit is required
n Obtain FBUs (Flash Backup Unit) (optional)
n RAID Cards in DUs

n Obtain FBUs (optional) for each RAID card
n RAID Cards in 1GbE IO Units (for JX40 connection)

n Obtain FBUs (optional) for each RAID card (installed in Slot#1 and Slot#2)
n Obtain FBUs (optional) (different from the FBUs for SBs/DUs)
n Obtain an FBU installation kit (this kit uses Slot#0 exclusively)
FBUs cannot be connected to RAID cards that are installed in 10GbE IO Units, PCI
Boxes, or Slot#3 on 1GbE IO Units.
86
3.3.4 HDD Configuration (1/5)

n Guidelines for Selecting a RAID Configuration for HDDs
Redundancy with "SAN boot + multipath driver" is recommended for reliability.
Software RAID
Item
Hardware RAID
SAN boot + multipath driver
OS standard
PRIMECLUSTER GDS
Inside of an SB or DU (*1)
External hard disk cabinet (JX40)
(*2)
Supported with RHEL
Supported
Supported
Supported
Relatively slow
Relatively slow
Supported
Supported
Supported
Card duplication
Supported
Supported
Not supported
Not supported
Supported
Path duplication
Supported
Supported
Not supported
Not supported
Supported
Depends on the software
0, 1
0, 1, 1E, 5, 6, 10
Available at the RAID levels that are

supported by the SAS array controller card
that is connected to JX40
Available at the RAID level of the

storage device being used
Not supported
Supported
Supported
Supported
Supported
OS
High-speed access
RAID level
Device duplication at
boot time
(*1) For details, refer to "3.3.2 Guidelines for Selecting Internal Storage.
(*2) For details on the supported OSs or virtualization software, refer to the related documents of the OS or virtualization software.
n Notes (1/2)
If an unpredictable power supply problem occurs, data in the cache (*) that is equipped in a
hardware RAID or disk may be lost.
To avoid such a problem, make sure to consider installing a redundant power supply or dual
power feed option as well as a CVCF or UPS.
(*) A hardware RAID uses the cache on the hardware when Write Back or Always Write Back is specified. A
disk uses the cache on the hardware when the disk cache is enabled.
87

n Notes (2/2)
When internal HDDs are used, the hardware RAID controller may fail to fully detect failures on a hard disk,
and a system slowdown may occur (excluding RHEL 6.4 or later).
[Additional Notes]
If intermittent media errors occur on a HDD, a slowdown may occur on rare occasions.
This problem may occur for example due to many scratches in a wide area on the HDD. The RAID
controller tries to replace the media error area and salvage data using RAID parity. Therefore, if
intermittent media errors occur over a long time, the salvage processing takes a longer time and the
conventional media access processing is delayed.
If there are restrictions in operation, the following options are recommended.
- Use ETERNUS.
- Replace the HDD with a SAS-SSD.
When using an internal SSD for a PCI slot, note the following points.
The SSD cannot be used as a boot drive.
When a failed SSD is replaced, the integrity of the data in the drive cannot be guaranteed. Reliability
design including recovery from a regular backup and software RAID configuration is required to prepare
for possible risks.
When using an internal HDD in an SB, note the following points.
When the Reserved SB function is used, the HDD that is used as a boot drive and the data disks cannot
be started or operated.
88

n Software RAID
RHEL standard RAID function
For details, refer to "Linux Design Guide".
Mirroring with PRIMECLUSTER GDS is supported.
For details on support information, refer to "Linux Design Guide".
n Hardware RAID
OS
OS
GDS
RAID CARD
HDD
HDD
HDD
HDD
Hardware RAID
Software RAID
Hard disk redundancy for disk units and SBs
A RAID card is included by default in a DU to support RAID configurations.
RAID 0, RAID 1, RAID 1E, RAID 5, RAID 6, and RAID 10 can be configured.
Hot swap of internal HDDs is supported (excluding RAID 0 configurations).
A RAID management tool ServerView RAID Manager" is supported. (This tool is provided with the
server main unit by default).
Hard disk redundancy on an external file unit
A RAID configuration can be created in the file unit on an external storage device (such as ETERNUS).
For details, refer to the manual for the file unit.
RAID 0, RAID 1, RAID 1E, RAID 5, RAID 6, RAID 10, RAID 50, and RAID 60 can be configured.
Hot swap of internal HDDs is supported (excluding RAID 0 configurations).
A RAID management tool "ServerView RAID Manager" is supported. (This tool is provided with the
server main unit as standard).
For details, refer to "MegaRAID SAS User's Guide" and "ServerView RAID Manager User's Guide".
n SAN Boot
An OS is started from a boot disk that is set in an external storage.
For details, refer to "PRIMEQUEST 1000/2000 Series SAN Boot Environment Configuration Manual".
89

n Memory Dump Function for Windows (OS Standard Function)
If a fatal system error occurs, data in the physical memory is output to a file.
n Dump Types and Estimated Sizes
Dump type
Dump file size
Description
Complete memory dump
Records all data that exists in the physical memory when the
system stops.
Physical memory size + 300MB
Kernel memory dump
Records the information only in the kernel memory space.
Depends on the memory space during system operation.
Minimum memory dump
Records the minimum information that is useful for

identifying the problem.
128KB or 256KB
Automatic memory dump
Records information that is equivalent to the conventional kernel memory dump.

The following points are different from the kernel memory dump.
l The initial value of the paging file size is smaller than the kernel memory dump size.
l If part of the information in the kernel memory space cannot be recorded, the page file size is automatically extended after
the next startup. (If part of the information in the kernel memory space cannot be recorded, acquisition of memory dump
information may fail.)
Mechanism for Acquiring Dump Information

1.
2.
If a system failure occurs, dump information is output to the paging file in the boot volume.
After the OS is restarted, a tentative paging file is created, and the acquired dump information is
moved to the specified save destination.
Work
Windows
STOP error
Acquire memory dump

information
Store to a save
destination
For details, refer to "10.4.3 Setting up the dump environment (Windows)" in "Administration Manual".
90

n Memory Dump Function for RHEL
If a fatal system error occurs, data in the physical memory is output to a file.
n Dump Types and Estimated Sizes

kdump (OS standard function)
Internal disk => OK

External disk => OK
Cluster sharing is not allowed.
Save data to a disk

that is not
managed with GDS
(EXT3 file system)
Acquisition area
for kdump
When creating a dedicated disk
partition (such as /var/crash) for
saving dump data, increase the
acquisition area size for kdump by
10% in consideration of the file
system management area.
Acquisition
Dump acquisition area

(Dump device for sadump)
(Temporary data storage
area)
sadump (a dedicated tool for PRIMEQUEST)
Error
Error
Installed memory capacity + 512MB

* Required
Alternative area that is used
if a device error occurs
(Installed memory capacity + 512MB)
Save data to a disk

that is not managed
with GDS
(RAW device)
N areas
(Installed memory capacity + 1GB) / 0.9

(1st generation) * Required
* Non-Fujitsu storage devices cannot be used for

dump areas for sadump.
(Installed memory capacity + 1GB) / 0.9

(2nd generation)
91
3.3.5 Network Configuration (1/5)

n Network Types
Connect with different types of networks for various intended purposes to achieve security and load
balancing.
On the external switching hub device for connecting to the
management LAN for MMBs, select [Disable] for the spanning tree
protocol function.
n Operation LAN
The LAN that is used by the customer
n Management LAN
Operation LAN
The LAN for connecting each partition and MMB to external

servers such as the operation management server
Cluster LAN
n Cluster LAN
(Cluster interconnect or private network)
The LAN that allows mutual status monitoring
between nodes in a cluster configuration
Use a subnet that is different from the one
for other LANs (such as the operation LAN).
For details, refer to
"3.4 Key Points for Cluster Configurations".
n Internal LAN
Cabinet
Partition
PCH
Internal LAN
MMB#0
User
port
REMCS Port for

port maintenance
MMB#1
User
port
REMCS
Port for
port maintenance
The LAN for connecting each partition to MMBs

for internal communication
n Maintenance LAN
Management
LAN
Maintenance LAN
The LAN that is used for maintenance

For connecting a maintenance terminal
(port for maintenance)
Maintenance
terminal
Management
server
* Specify different subnets for the management LAN, cluster LAN, internal LAN, and maintenance LAN.
* When counting the number of LAN ports that are required for a partition, consider that two ports are used for the management LAN.
92

n Redundancy of Internal Ports
Concepts of Reliable Configurations
When configuring partitions with multiple components, install internal ports on
different components.
Example of IOUs
IOU#0
Separate IOUs from each other.
IOU#1
PCI Express
Slots
LAN
Partition A
PCI Express
Slots
LAN
Partition B
93

n Configuration That Focuses on Redundancy
Multiple channel ports on a PCI Express card are used effectively.
- FC card (Single)
=> FC card (multiple channels)
- Unused 1000Base-T ports are used.
IOU
Advantage: The numbers of extended I/O cabinets and PCI Boxes can
be reduced.
Disadvantage: If the PCI Express card fails, multiple ports cannot be
used.
* Because the card is duplicated, the system can continue to operate safely.
PCI Express card
1000Base-T (Dual)
FC card (Dual)
1000Base-T (Dual)
Incorrect Example
Duplicated
Duplicated
Duplicated
Duplicated
1000Base-T (Dual)
Duplicated
This configuration is not correct because the duplication does

not make sense if the PCI Express card fails.
FC card (Dual)
n Configuration That Focuses on Hot Swap

Multiple channel ports on a PCI Express card are used effectively.
IOU
PCI Box
PCI Express card
1000Base-T (Dual) Not used

PCI Box LH
#0
FC card (Single)
Duplicated
Duplicated
Empty
PCI Box connection card

PCI Box LH
#1
1000Base-T (Dual) Not used

FC card (Single)
Empty
94

n Network Configuration Redundancy with an Onboard LAN
A network configuration can be duplicated by using an Onboard LAN.
IOU
Onboard
Can be duplicated
Can be duplicated
LAN card
LAN card
Can be duplicated
PCI Box
LAN card
LAN card
* When an onboard LAN is used, before maintenance is performed on an IOU,

all partitions that use the IOU must be stopped.
n Redundant Configuration for the Management LAN

n The management LAN can be recovered when an error occurs by creating a redundant
configuration of the user ports on the MMBs that are connected to the management LAN with
tools such as the bonding driver, PRIMECLUSTER GLS, and Intel PROSet.
n When MMBs are duplicated but user ports are not, the active MMB must be switched manually
if an MMB fails.
For details, refer to "1.3.3 Redundant configuration of the management LAN" in "Administration Manual".
95

n Infiniband Network Configuration Redundancy
IOU
IB card
IB card
IOU
Can be duplicated
Can be duplicated
PCI Box
IB card
Can be duplicated
Can be duplicated
IB card
IB card
n Replacement of an Ethernet Environment with Infiniband Cards

n Infiniband cards with a wide bandwidth of 40Gbps or 56Gbps can accommodate more networks
than when a 10GbE environment is used.
n Infiniband cards can be used for the paths to synchronize databases.
Use of Infiniband cards with PCIeSSDs can provide better effects within the network.
96
3.3.6 Setting I/O Space for PCI Devices (1/2)

To perform a boot in a legacy BIOS environment, it is necessary to assign the I/O space
to the boot device.
n The size of the I/O space is 64KB for all models.
n A 4KB area of the I/O space can be assigned per device.
n The assignment is possible for up to 15 devices (that is, 16 devices in total including
one device that is reserved for a PCH).
The following devices can be assigned:
- Onboard LANs
- RAID cards in SBs
- RAID cards in DUs
- PCI Express slots on IOUs
- PCI Boxes
97
3.3.6 Setting I/O Space for PCI Devices (2/2)

n Select one of the following two assignment types and set the assignment in the
"Device Manager menu" of the UEFI.
* When auto assignment is used for some devices and forced assignment is used for other
devices in the same system, auto assignment is performed after forced assignment is set
(and a system reset is performed).
Forced assignment of the I/O space
Reserve an I/O resource for each of devices (including onboard devices such as IOUs) for I/O
resources and PCI Express cards that are included in a partition.
This type of assignment is useful for assigning I/O space to specific devices and cards if
there are more than 16 devices that require the I/O space.
Auto assignment of the I/O space
This type of assignment is easy to use and can be used when the partition configuration is
relatively small and 16 or less devices require the I/O space.
For details, refer to 3.4 [Device Manager] Menu" in "Tool Reference".

98
3.3.7 Power Supply Redundancy

nExamples of 200V (for 2800B/2800E)
Primary Power Feed
Without Redundant Power Supply (3+0)
PSU
PSU
Primary Power Feed

With Redundant Power Supply (3+1)
PSU
PSU
Failure
If a PSU fails, the entire
PSU
PSU
Failure
PSU
Redundant
PSU
Using a redundant PSU

prevents the system from
going down.
system goes down.
PSU
PSU
CB
CB
CB
CB
PSU
AC power
supply
(#1
system)
Using dual power feed

prevents the system
from going down.
Without Redundant Power Supply (4+0)
PSU
PSU
Customer's distribution panel

(AC200V)
nExamples of 100V (for 2400E)

PSU
PSU
Power failure
Customer's distribution panel

(AC200V)
Customer's distribution panel (AC200V)
PSU
CB
AC power supply
(#0 system)
AC power
supply
(#0
system)
CB
CB
CB
CB
CB
CB
CB
CB
AC power supply
(#0 system)
Dual Power Feed (3x2)
With Redundant Power Supply (4+1)
PS
PSU
Failure
PSU
PSU
PSU
Failure
Redundant
PSU
Using a redundant PSU

prevents the system from
going down.
If a PSU fails, the entire

system goes down.
CB
CB
CB
CB
CB
CB
CB
CB
CB
AC power supply
(#0 system)
AC power supply
(#0 system)
* When using a redundant configuration or dual power feed for the main unit, also use the same power supply configuration for PCI Boxes.
For details on PSU configurations, refer to "2.7 Power Supply Unit (PSU)".
99
3.3.8 Guidelines for Selecting a 200V Power Distribution Box

n AC200V Power Distribution Box Brief Specifications
U number
Voltage
Input
Output
Output current
1U
200V
NEMA L6-30P
IEC320 C19
24A/BOX
16A per box
0U
200V
NEMA L6-30P
IEC320 C19
24A/BOX
16A per box
n Guidelines on the Number of AC200V Power Distribution Boxes

Model
200V non-redundant
configuration
200V redundant
configuration
200V dual power feed

configuration
2400E
2800E
2800E/2800B (*1) (4,500VA or less)
(*1) Calculate the required number of power distribution boxes according to the maximum apparent power. The maximum
apparent power can be calculated by using the Power Calculator ("Tool").
As the above example shows, if the maximum apparent power is 4.500VA or less , the number of power distribution boxes can
be reduced.
100
3.4 Key Points for Cluster Configurations (1/4)

Cluster operation with cluster operation software is supported to increase the system availability.
For details, refer to "Chapter 1 Product Overview" in "General Description".
n Relationship Between Redundant Configurations and Reliability
Disaster recovery
(Countermeasures against
disaster)
Inter-cabinet cluster
server failures)
Intra-cabinet cluster
partition failures)
Firm reliability of the cabinet

l Component redundancy
(PSU, FAN, MMB, IOU, and so on)
Reliability
Flexible I/O + spare SB

CPU/SB failures)
Redundancy of the power

supply, fan, and so on
operation failure)
Memory Mirror/
Memory Spare
memory failures)
CPU
CPU
DIMM
DIMM
CPU
CPU
DIMM
DIMM
Improved reliability of SBs and QPI network

l Flexible I/O mechanism
l Reserved SB (spare SB)
Firm reliability in the middle layer and application layer

l Inter-cabinet cluster (for securing work continuity)
l Intra-cabinet cluster (for improving work continuity)
* Cluster software
Windows => Windows Server Failover Clustering (WSFC)
RHEL => PRIMECLUSTER
Cost for Installation and Operation

101

n Cluster System Overview
A dedicated LAN for inter-node communication is used to connect between nodes.
Private network is used for Windows.
Cluster interconnect is used for Linux.
Inter-node communication is performed periodically to monitor for problems.
There are two types of cluster systems: inter-cabinet cluster and intra-cabinet
cluster.
For detail, refer to "3.4.1 Inter-Cabinet Cluster" and "3.4.2 Intra-Cabinet
Cluster".
Windows
Clustering with Windows Server Failover Clustering (OS standard function) is
recommended.
* A server for Active Directory (AD server) is required separately.
Linux
Clustering with PRIMECLUSTER is recommended.
102

n Possible Combinations of PRIMEQUEST Cluster Configurations (1/2)
Inter-Cabinet Cluster
Clustering between different models is not supported.
Each node consists of the same model in the same configuration
(in the number of CPUs, CPU frequency, memory capacity, HDD
capacity, and number of HDDs).
Intra-Cabinet Cluster
All nodes have the same configuration (in the number of CPUs,
CPU frequency, memory capacity, HDD capacity, and number of
HDDs).
If an IOU fails, the redundancy of the system is lost and
components such as the midplane become single points of failure.
103

n Possible Combinations of PRIMEQUEST Cluster Configurations (2/2)
For OracleDB, availability as the cluster management software for DB software is
shown.
OS
Cluster software
PRIMECLUSTER
(excluding RAC)
RHEL
PRIMECLUSTER (RAC)
Oracle Clusterware
(including RAC)
WSFC (excluding RAC)
Windows
WSFC (RAC)
Oracle Clusterware
(including RAC)
Configuration
2800B
2400E
2800E
Intra-cabinet
No
Yes*
Yes
Inter-cabinet
Intra-cabinet/
inter-cabinet
Intra-cabinet
Yes*
Yes*
Yes*
No
No
No
No
Yes*
Yes
Inter-cabinet
Yes*
Yes*
Yes*
Intra-cabinet
No
Yes*
Yes
Inter-cabinet
Intra-cabinet/
inter-cabinet
Intra-cabinet
Yes*
Yes*
Yes*
No
No
No
No
Yes*
Yes
Inter-cabinet
Yes*
Yes*
Yes*
Yes*: Supported and recommended (IOU redundancy configuration is allowed)

Yes: Supported and recommended
No: Not supported
RAC: Oracle Real Application Clusters

WSFC: Windows Server Failover Clustering
When RAC is used, PRIMECLUSTER and WSFC cannot be used together with RAC. Use Oracle Clusterware.
When Oracle SE-RAC is used, a cluster configuration is possible only for 2400E. For 2800E, Oracle SE-RAC cannot be used, but Oracle EE-RAC can
be used.
When Oracle SE-RAC is used, only an intra-cluster configuration is possible for 2400E.
When Oracle EE-RAC is used, an inter-cluster configuration is possible.
104
3.4.1 Inter-Cabinet Cluster

n A cluster configuration for communication between PRIMEQUEST nodes
(cabinets)
Windows
AD server
Private network
Public network (operation LAN)
The private network is configured

separately from the public network
(operation LAN).
Cabinet#0
Partition#0
Cabinet#1
Partition#0
Cluster service
Cluster service
Teaming
Cluster interconnect
Operation LAN
Linux
The cluster interconnect is configured
separately from the operation LAN
and management LAN.
Cabinet#0
Partition#0
Cluster software
Cluster software
BMC
Cabinet#1
Partition#0
MMB#0
MMB#1
(Active)
(Standby)
BMC
MMB#0
MMB#1
(Active)
(Standby)
Management LAN
105
3.4.2 Intra-Cabinet Cluster

n A cluster configuration for communication between nodes (partitions) that
are created in a cabinet
n By configuring both active and standby systems in a cabinet, availability of
the entire system including the software layer is ensured.
Windows
Linux
Cluster interconnect
Operation LAN
AD server
Private network
Public network
(Operation LAN)
Partition#X
Partition#Y
Cluster service
Cluster service
Partition#X
Partition#Y
Cluster software
Cluster software
BMC
BMC
Clustering
(PRIMECLUSTER)
Clustering
(WSFC)
MMB#0
MMB#1
(Active)
(Standby)
Management LAN
106
4. Operation Management Design

This chapter describes the key points for designing operation
management.
For details, refer to "Administration Manual".
107
4.1 Key Points for Console Operation (1/2)

n Functions of Consoles
Connection
type
Console type
Intended purposes
Video redirection
Control and
management of the
OS in a partition
(using images)
Connection
target
Management
LAN
Remarks
Remote
Text console
Maintenance
It is difficult to acquire operation logs or

automate operations (log acquisition can be
performed by using an application with a log
acquisition function).
A PC in which JRE is installed must be connected
to the management LAN.
If the resolution on the server is 800 x 600, part
of the screen may not be displayed or trails of the
mouse pointer may appear.
Management
LAN
n Action That Is Required When Reserved SB Switching Occurs
If switching by the Reserved SB function occurs, the console connection must be

changed.
108
4.1 Key Points for Console Operation (2/2)

n Console Connection Configuration
Operation LAN
: SW redundancy
: Redundancy with teaming
(PRIMECLUSTER GLS etc.)
IOU
IOU
: Disabled on the standby side
Video redirection
Text console
Remote storage
Management LAN
Maintenance LAN
109
4.2 Key Points for Server Operation Management (1/2)

Management PC
Hardware management of a PRIMEQUEST server

through the MMB Web-UI
(agentless management)
- Hardware status monitoring
- Hardware setting
- Power supply operation (turning on/off) etc. on a partition basis
MMB
SVAS
BMC
BIOS
Partition
Operation target: One PRIMEQUEST server

Operation management server
GUI client
Hardware management of multiple PRIMEQUEST and

PRIMERGY servers through ServerView Suite
ServerView
Operations Manager
http://www.fujitsu.com/global/services/computing/server/primequest/technology/server-view.html
- Consolidated management of multiple PRIMEQUEST servers in an integrated

screen and operability
- Functions for supporting the entire system life cycle that are provided to the
managed servers
- Power supply operation and reboot operation for a partition that can be
performed from a GUI
PRIMERGY
MMB
MMB
PRIMERGY
MMB
SVAS
SVAS
SVAS
SV RAID
SV RAID
SV RAID
Operation target: Multiple PRIMEQUEST and PRIMERGY servers

Integrated management of different models through

ServerView Resource Orchestrator V3 (optional)
Operation management client

ServerView Resource
Orchestrator
http://www.fujitsu.com/global/services/software/ror/
Storage
MMB
SV Agent
PRIMERGY
MMB
SV Agent
Network
Management target: Resources
SVAS: ServerView Agentless Service

SV RAID: ServerView RAID Manager

110
- A single screen for displaying server information in which different operations

and monitoring between platforms are integrated
- Server operations (start, stop, and reboot operations) through the screen
(Automation with a CLI is also possible.)
- Functions for supporting the entire operation of virtual environment
(Integrated management of physical servers and virtual servers, simplified and
visualized operation of virtual servers, and simplified maintenance)
4.2 Key Points for Server Operation Management (2/2)

n Operation Management of the Entire System with Systemwalker
Centric Manager
Operation management including work management, network management, and storage
management
n Failure statuses can be checked easily through an integrated monitoring screen.
n Functions of ServerView Suite can be used through web call linkage.
n Duplicated operation management server
If a problem occurs on the primary management server, monitoring can be continued by switching to the secondary management
server.
n Operations on the monitoring screen are restricted to prevent unintended operations and unauthorized access.
Operation management client
Systemwalker Centric
Manager (Manager)
Manager (Client)
Storage
MMB
Manager (Agent)
PRIMERGY
MMB
Manager (Agent)
Network
Management target: Resources
For details, refer to "Appendix J Systemwalker Centric Manager Linkage" in "Administration Manual".
111
4.3 Key Points for Backup and Restoration Configurations (1/2)

n Backup and Restoration of Configuration Information
For details, refer to 7.1 Backing Up and Restoring Configuration Information" in "Administration
Manual".
n Backup Mechanism
Configuration information is stored in the System FRU (I2C EEPROM) on
the OPL.
(This processing is performed regularly or when MMB firmware is
updated.)
The same information is stored also in the flash memory of the MMB.
When duplicated MMBs are used, the same information is always stored in
both MMBs.
n Backup Contents
Firmware setting information
Device identification information
112
4.3 Key Points for Backup and Restoration Configurations (2/2)

n Backup and Restoration of System Disks
n OS Standard Commands and Tools
OS
Command or tool name
Windows
Linux
Yes
Yes
Windows Server Backup

dump and restore commands
Yes: Available
-: Not available
n Optional Products
Tool name
OS
Options that can be added
Windows (*1)
Linux
ServerView Resource Orchestrator V3
Yes
Yes
SystemcastWizard Professional
Yes
Yes
Yes
Yes (*2)
Yes (*2)
PRIMECLUSTER GDS Snapshot

ETERNUS SF AdvancedCopy Manager
Yes
NetVault Backup 9 BMR Offline Only Client

NetVault Backup 9 BMR for NVBU Server
NetVault Backup 9 BMR Client for Linux
NetVault Backup 9 for Windows BMR Client for Windows
NetVault Backup 9 for Windows
Yes
NetVault Backup 9 for Windows BMR Offline Only Client

NetVault Backup 9 for Windows BMR Client for Windows
NetVault Backup 9 for Windows BMR for NVBU Server
NetVault Backup 9 BMR Client for Linux
CA ARCserve Backup for Windows
Yes
CA ARCserve Backup for Windows Disaster Recovery Option
NetVault Backup 9
(*1) For details on support information for Windows Server 2012, visit our web site:
http://www.fujitsu.com/global/services/computing/server/primequest/products/software/.
Yes: Available, -: Not available
(*2) Only for a SAN boot configuration with an ETERNUS disk storage system.
* Floppy disk drives cannot be connected to the PRIMEQUEST 2000 series. Select a method that does not require direct connection of a floppy disk drive.
* The software support information is as of February 2014. For the latest software information, contact your sales representative.
For details, refer to "Linux Design Guide".

113
4.4 Key Points for NTP Operation

n The NTP should be used when Reserved SBs are used or when a partition is
configured with two or more SBs
n Time Setting Targets
For details on settings, refer to "Installation Manual" and "4.4 Key Points for Time Correction"
in "Linux Design Guide".
n MMB
The MMB works as an NTP client that is connected external NTP servers to synchronize time.
n Each partition
Each partition works as an NTP client that is connected to external NTP servers.
NTP server
NTP server
(Primary)
NTP server
(Other)
(Secondary)
Partition#0
NTP
client
System
clock
PCH
RTC
MMB
NTP
client
* To ensure the stability of the NTP operation, specify multiple NTP

servers from each NTP client.
* Select NTP servers that have a high-precision clock in the Internet
or intranet.
System
clock
MMB
RTC
BMC
RTC
114
5. OS/Support
This chapter describes information on the OS.
115
5.1 Arrangement Information

n For details on the types of OSs and support, contact your sales
representative.
n For details on OS specifications, refer to "Appendix B OS
Specifications" and the website of each vendor.
116
Appendices
117
Appendix A Bundled Software (1/2)

n Software Bundled with Hardware
The "ServerView Suite" DVD is bundled with the main unit by default.
For details, refer to "3.3 Bundled software" in "General Description".
The tools below can be installed all together by using ServerView Installation
Manager (SVIM).
Tool
Windows
RHEL
SUSE
DSNAP
Yes
Software Support Guide
Yes
ServerView Operations
Manager
Yes
Yes
Yes
ServerView Agentless Service
Yes
Yes
Yes
ServerView Agent
Yes
Yes
Yes
ServerView RAID Manager
Yes
Yes
Yes
Dynamic Reconfiguration
utility
Yes
Yes: Supported, -: Not supported
118
Appendix A Bundled Software (2/2)

n Drivers
n Windows
Drivers are provided with ServerView Suite.
The latest versions can be downloaded from the following URL:
http://www.fujitsu.com/global/services/computing/server/primequest/
n Related Software
n Server Management Software
For details, refer to "1.5.3 Server management software" in "General
Description".
119
Appendix B OS Specifications (1/6)

The following OS specification information is applied to the PRIMEQUEST 2000
series.
For details on the specifications including maximum values on each OS, refer to
the information that is provided by each vendor.
nWindows Server 2012: Comparison of Specifications

OS type
R2 Datacenter / R2 Standard
Maximum number of sockets
CPU
Memory
Datacenter / Standard
8
Maximum number of cores
120
Maximum number of logical CPUs
240
Maximum memory capacity
4TB
MBR format (Primary + Extended): 3+124, 4+0
GPT format: 128+0
Number of partitions in a disk

Disk (*1)
MBR format: 2TB

GPT format: 256TB (Upper limit in the file system)
Maximum LUN capacity

Maximum number of PCI slots
PCI
Firmware
All slots can be used.
PCI Hot Plug
PCI Box is supported. No IOU is supported. (Specification of the main unit)
UEFI support
Supported
(*1) The upper limits of the file system size and the file size in each file system are as follows:
FAT Maximum size: 2TB, maximum file size: 2TB
NTFS Maximum size: 16TB (4KB cluster)/256TB (64KB cluster) (implementation restrictions, the logical maximum value is 16EB)
Maximum file size: 16TB (implementation restrictions, the logical maximum value is 2^64-1 clusters)
ReFS, which is the latest file system for Windows Server 2012, can be used for data areas, but ReFS is not recommended because its use has not been proven well.
120

nWindows Server 2008 R2 : Comparison of Specifications for Each
Version
OS type
R2 Standard (64bit)
CPU
R2 Enterprise (64bit)
4 (2 system boards)
8 (4 system boards)
60
120

Memory
R2 Datacenter (64bit)
120
32GB
2TB (*2)
2TB (*2)
MBR format (Primary + Extended): 3+124, 4+0

GPT format: 128+0

Disk (*1)
MBR format: 2TB

GPT format: 256TB (Upper limit in the file system)
PCI
Firmware
PCI Hot Plug
No IOU is supported. (Specification of the main unit)
UEFI support
Supported
(*1) The upper limits of the file system size and the file size in each file system are as follows:
FAT Maximum size: 2TB, maximum file size: 2TB
NTFS Maximum size: 16TB (4KB cluster)/256TB (64KB cluster) (implementation restrictions, the logical maximum value is 16EB)
Maximum file size: 16TB (implementation restrictions, the logical maximum value is 2^64-1 clusters)
(*2) To install 1TB or more memory, Windows Server 2008 R2 SP1 must be applied. If SP1 is not available, apply KB980598. For details, refer to the KB information that is available
from Microsoft (http://support.microsoft.com/kb/980598/).
121

n RHEL: Comparison of Specifications
The information in this page is as of February 2014. For the latest information, refer to the following URL:
https://hardware.redhat.com/list.cgi?product=Red+Hat+Hardware+Certification&quicksearch=PrimeQuest
OS type
RHEL6 (for Intel64)
8 (4 system boards)

CPU
Memory
120
240
12TB
MBR format (Primary + Extended): 4+0, 3+1 (up to 11 logical partitions in the extended partition)
GPT format: 15
The ext4 file system (MAX16TB)

Disk
MBR format: 2TB (* The maximum value for MBR)

GPT format: 240TB (* ext4 x 15 partitions equivalent)
Maximum number of volumes
18,278 (* Red Hat tested up to 8,192.)
SCSI common layer: 512/4,294,967,296

Maximum number of LUNs (default
value/maximum value) (*1)
Fusion-MPT SAS (mptsas driver): 16,896/16,896

Fusion-MPT SCSI (mptspi driver): 256/256
Emulex Fibre Channel (lpfc driver): 256/65,536
MegaRAID SAS (megaraid_sas driver): 8/8

PCI Hot Plug
PCI
MSI support (OS/driver)

(Common between x86 and Intel64)
Firmware

Kernel: MSI and MSI-X are supported. / FusionMPT: MSI is supported, but MSI-X is not supported. INTx is used by default.
LPFC: MSI and MSI-X are supported. INTx is used by default. / e1000e: MSI and MSI-X are supported. MSI is used by default.
igb: MSI and MSI-X are supported. MSI-X is used by default. / ixgbe: MSI and MSI-X are supported. MSI-X is used by default.
* To specify an option other than the default for a driver, use the option parameter of the driver.
UEFI support
Supported
(*1) - In actual operation, the values are restricted by the maximum number of volumes.
- The number of LUNs for each of the lower-level drivers (mptsas, mptspi, lpfc, and megaraid_sas) cannot exceed the value that is set for the SCSI common layer.
- To specify a non-default value, use the option parameter of each driver.
122

n SUSE: Comparison of Specifications
The information in this page is as of February 2014. For the latest information, refer to the following URL:
https://www.suse.com/products/server/technical-information/#System
OS type
SLES 11 SP3 for AMD64 & Intel64
CPU
4,096
Memory
12TB
16TB
Maximum file system size (EXT3)

Disk
2TB
Maximum file size (EXT3)

PCI
Firmware
PCI Hot Plug
UEFI support
Supported
123

n VMware vSphere 5.1: Comparison of Specifications
The configuration maximums vary depending on actual environment.
For details, refer to "Configuration Maximums for VMware vSphere 5.1" in the following URL:
http://www.vmware.com/pdf/vsphere5/r51/vsphere-51-configuration-maximums.pdf
Edition of vSphere 5
CPU
Memory
Essentials
Essentials Plus
Maximum number of logical

CPUs
Standard
Enterprise
Enterprise Plus
32Way
64Way
160
Number of vCPUs per virtual

machine
8Way
2TB
Maximum number of LUNs
256
Maximum volume capacity
64TB
Disk (in VMFS)
PCI
Firmware
Maximum number of FC-HBAs
Maximum number of physical

NIC ports
For details, refer to "Networking Maximums" in

"Configuration Maximums for VMware vSphere 5.1".
UEFI support
Not Supported
124

n VMware vSphere 5.5: Comparison of Specifications
The configuration maximums vary depending on actual environment.
For details, refer to "Configuration Maximums for VMware vSphere 5.5" in the following URL:
http://www.vmware.com/pdf/vsphere5/r55/vsphere-55-configuration-maximums.pdf
Edition of vSphere 5
CPU
Memory
Essentials
Essentials Plus
Standard
Maximum number of logical

CPUs
320
Number of vCPUs per virtual

machine
64Way
4TB
Maximum number of LUNs
256
Maximum volume capacity
64TB
Enterprise
Enterprise Plus
Disk (in VMFS)
PCI
Firmware
Maximum number of FC-HBAs
Maximum number of physical

NIC ports
For details, refer to "Networking Maximums" in

"Configuration Maximums for VMware vSphere 5.5".
UEFI support
Supported
125
Revision Record
Edition
01
Revised location
Date
2014-02-18
Description
All pages
Created the first edition
126
Conditions of Use
n Copyright, Trademark Rights, and Other Intellectual Property
Rights
The contents (text, images, audio, etc.) in this manual are protected by
copyright, trademark rights, and other intellectual property rights. These
contents may be printed and downloaded for personal use only. However,
permission from Fujitsu Limited or the relevant right holder is required for any
other use, such as reuse on your own website or uploading to another server.
n Warranty Restrictions
Fujitsu Limited does not guarantee the accuracy, merchantability, or
applicability to your purpose of these contents, and shall bear no liability for any
damages that are incurred as a result of the use of these contents. These
contents are subject to change or removal without notice.
If you have any questions, contact us at "Contact About Products"
(http://www.fujitsu.com/global/contact/computing/PRMQST_index.html).
All rights reserved.
C122-A014-01EN
2014.02
127

FUJITSU Server PRIMEQUEST 2000 Series Design Guide: C122-A014-01EN

Uploaded by

Copyright:

Available Formats

FUJITSU Server PRIMEQUEST 2000 Series Design Guide: C122-A014-01EN

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

FUJITSU Server PRIMEQUEST 2000 Series Design Guide: C122-A014-01EN

Uploaded by

Copyright:

Available Formats

FUJITSU Server PRIMEQUEST 2000 Series

1.1 Product Lineup

2.1 Operator Panel (OPL)

n 2. Basic Hardware Configuration

PRIMEQUEST 2000 Series Design Guide

Copyright 2014 FUJITSU LIMITED

Table of Contents (Cont'd)

3.1 Key Points for MMB Design

4.1 Key Points for Console Operation

n 4. Operation Management Design

PRIMEQUEST 2000 Series Design Guide

Copyright 2014 FUJITSU LIMITED

Table of Contents (Cont'd)

5.1 Arrangement Information

Appendix A Bundled Software

PRIMEQUEST 2000 Series Design Guide

Copyright 2014 FUJITSU LIMITED

Linux Design Guide

PRIMEQUEST 2000 Series Design Guide

Linux Installation Guide

Linux Operation Guide

Copyright 2014 FUJITSU LIMITED

n Abbreviations in This Manual

PRIMEQUEST 2000 series or PRIMEQUEST

Microsoft Windows Server 2012 R2 Standard

Windows Server 2012

Microsoft Windows Server 2012 Standard

Microsoft Windows Server 2012 Datacenter

Windows Server 2008

Microsoft Windows Server 2008 R2 Datacenter

RHEL6 (for Intel64)

PRIMEQUEST 2000 Series Design Guide

RHEL6, RHEL, or Linux

Copyright 2014 FUJITSU LIMITED

PRIMEQUEST 2000 Series Hardware Installation Manual

Hardware Installation Manual

PRIMEQUEST 2000 Series General Description

PRIMEQUEST 2000 Series Installation Manual

PRIMEQUEST 2000 Series Administration Manual

PRIMEQUEST 2000 Series Tool Reference

Linux Design Guide

Linux Installation Guide

Copyright 2014 FUJITSU LIMITED

PRIMEQUEST 2000 Series Design Guide

Copyright 2014 FUJITSU LIMITED

1.1 Product Lineup

Highly-reliable servers that are based on the latest Intel architecture

Copyright 2014 FUJITSU LIMITED

PRIMEQUEST 2000 series

Maximum SMP (sockets/cores) (*1)

Maximum memory capacity (*1)

Maximum internal disk capacity

Intel Xeon Processor E7 v2 Family

6TB (64GB DIMM x 96)

12TB (64GB DIMM x 192)

Reliability, Availability, and Serviceability (RAS)

Hardware RAID / Software RAID

WOL, PXE, video redirection, and console redirection