Datacenter

DATA CENTER SOLUTIONS
How Intelligent
Data Centre Infrastructures
Help Manage Resources,
Services and Costs.
Who is Raritan?
A global leader in data center management solutions
• Founded in 1985 in Somerset, NJ USA (HQ)
• Acquired by Legrand in Sept 2015
• Offices in Australia, Canada, China, France,
Germany, India, Japan, Netherlands,
Singapore, Taiwan, UK
• Products sold and supported in over 76
countries and installed in over 50,000
critical data centers
History of Innovation
Our solutions help IT professionals gain
more insight into data center operations
and manage smarter in more than one way.
We help maximize uptime, optimize
efficiency and allow for strategic decision‐
making based on reliable data points.
Raritan is always looking to push technology
further and innovation is at our core.
With 47 patents granted and 30 more
applications pending, we make sure you
always have the latest and future‐proof
technology.
“Last Mile” Example Issues:
LEGRAND GROUP: The last mile
Dry transformers
Capacitor banks
Switch boards
UPS
Cable management
Structured cabling
Busbar systems
Aisle containment
Patch & server racks
Co‐Location cabinets
Row based cooling
Power distribution
KVM / Serial
Monitoring
Agenda of today’s Seminar
“Last Mile” Example Issues:
PDU:
1/ Circuit Breaker Trip Coordination
• 2/ Sufficient Circuit Breaker Trip Detection and Alarming Overlooked in Design
• 3/ Improper Feed Sizing for Blade Servers and Chassis‐Based Networking Gear
• 4/ Residual Current Monitoring
• 5/ Outlet malfunction ‐ trip Analyzation
• 6/ Human error minimization
• 7/ Equipment failure & redundancy
• 8/Outlet switching benefit and risk
ATS:
• 9/ Application
10/ Technologies
• 11/ Insufficient Switching Time
Circuit Breaker Trip Coordination
Why metering ?
The “Buckaroo Effect”
Avoid inefficiencies based
on fear & uncertainty
Avoid over/under capacity
with accurate data!
Capacity Planning
HP Proliant DL380G5 HP Proliant DL380G6

Name plate 700 Watt Name plate 500 Watt
Source: http://www.spec.org/power_ssj2008/
Breaker Coordination Must Extend into the Rack
EXAMPLE 1
Common scope of breaker coordination protects against cascading failure…
Critical Facility Design Scope: End‐to‐End
As IT loads become more sophisticated…
… the “last mile” of the power chain:
from the distribution board,
to the cabinet, to the equipment…
… pose new challenges for clients
EXAMPLE 1
… but must consider cabinet components to be fully effective!
EXAMPLE 1
Most common trip event = faulty server power supply.
EXAMPLE 1
MCCB Trip Curve Fuse Melt Curve
100
10
TIME IN SECONDS
1
.1
.01
1
.9
800
100
10
RMS SYMMETRICAL CURRENT IN AMPERES
5kAIC typical; Finger-safe cylindrical fuse holder;

Typical 20kAIC+
Circuit Breakers types
• IEC 60950‐1 Safety Standard requires use overcurrent protectors (OCP)
‐ Thermal‐Magnetic Breakers
‐ Magnetic‐Hydraulic Breakers
‐ Hydraulic‐Magnetic Breakers
‐ Fast‐blow fuses per branch or per outlet
Standardized trip delay curves: Type B (fastest), C or D (slowest).
Recommended are Type D
to manage inrush current and because of temperature derating / harsh environment
2 Pole for UL Models and 1 Pole Breakers (For EU / VDE)
Regulatory Approved OCP
Approved Circuit Breakers
– UL‐489 (USA)
– CSA C22.2 #5 (Canada)
– EN 60934 VDE 0642 (Europe/International)
Approved Fuses
– UL‐248 (USA)
– CSA C22.2 #248 (Canada)
– IEC 60127‐1 (International)
NOT Approved Devices
– UL‐1077
(“supplemental” button breakers found on multi‐outlet tap boxes)
– UL‐489A
(DC rated for communication circuits)
Circuit Breaker Mechanism Types
Thermal Magnetic
 Most common type. Used in all commercial/residential panelboards.
 Standardized trip delay curves.
 Thermal element (bimetallic strip) handles time delayed trips
(currents <=600% breaker rating).
 Magnetic element (iron core coil) handles instantaneous trip short circuits.
 Must be derated if used at high ambient temperature (i.e. rack PDU)
Hydraulic Magnetic
 Used where high ambient temperature is concern (rack PDU)
 Non‐standardized vendor specific trip delay curves.
 Variable magnetic element. An air coil core containing a movable,
viscous damped spring loaded iron slug.
 No derating at high ambient temperature.
 Slower to trip compared to thermal magnetic for short circuits.
Derating of Thermal Magnetic Breakers
• All thermal magnetic breakers must be
derated when operated at high (>40°C)
temperatures.
• Graph shows our Moeller thermal magnetic
breaker must be derated to 95% at 50°C
(20A breaker = 19A @ 50°C) and
cannot be used in a 60°C rated PDU.
Circuit Breaker Trip Delays
Thermal magnetic delays are standardized: A,B,C, or D. For data centers, type D is used.
Hydraulic magnetic trip delays are not standardized. Type 62 are used by most vendors
Thermal Magnetic Delay Sensata Type 62 Delay
At 600% delay is 0.02
to 2 seconds.
Circuit Breaker Trip Coordination
When short occurs, only closest up‐stream breaker should trip.
Short in rack should trip PDU breaker ‐ not panel breaker protecting the PDU.
 Panel main & branch CB manufacturer/type.
 PDU and panel breakers are different manufacturer & type.
Current ratings are close (PDU 16A, panel 32A).
PDU hydraulic‐magnetic are slower than
panel thermal‐magnetic.
 Some customers test and complain
panel trips before PDU breaker.
Highly dependent on panel breaker
manufacturer and current capacity of circuit.
Insufficient Trip Breaker Alerting in Power Chain
EXAMPLE 2
• Most modern data centre builds equipped with
branch circuit monitoring per pole;
• For same reasons as in previous example,
granularity is insufficient;
• Clients often do not realize until too late;
EXAMPLE 2
EXAMPLE 2
Very important to differentiate between
zero amperes and zero volts!
Proper Feed Sizing for High Density Blade Chassis
EXAMPLE 3
• Prevalence of blade servers increasing every year;
• Increased confusion regarding power
interconnects required to maintain true 2N;
• Issue compounded when clients solely consider
power capacity guidelines of RPP / distribution
panel feed;
e.g. Cisco UCS 5108 e.g. 415V, 3phase WYE; 32a Supply
• 6U height; • 23,000VA / ~22,400 Watts;
• ~1800W typical; • In theory, should support up
~2300W peak;
to (22400 ÷ 2300) = 9 chassis;
• 4x Power Supplies,
up to 2500W each; Let’s try seven (48U rack)…
EXAMPLE 3
First Connect
Six Chassis
(14.2kVA)
Peak = 4.9A per
connection;
Peak (Failure Mode)
= 9.8A on one plug
EXAMPLE 3
9.8A
Even with
B‐side down,
each circuit breaker at
61% load.
Very safe.
EXAMPLE 3
9.8A
Failure Mode:
B-Side power down
and some A-side
power supplies fail.
Still safe.
EXAMPLE 3
Add 7th Chassis.
Need to share
breakers.
EXAMPLE 3
14.7A
Still safe so far.
Two breakers
load to 14.7A
14.7A if B‐side power down.
9.8A
9.8A
EXAMPLE 3
14.7A
Not truly redundant!
One bad power supply
(during B‐side maintenance)
14.7A
can shut down 3 chassis
9.8A
9.8A
EXAMPLE 3
14.7A
14.7A
9.8A
9.8A
EXAMPLE 3
14.7A
9.8A
9.8A
EXAMPLE 3
14.7A
9.8A
9.8A
EXAMPLE 3
14.7A
9.8A
9.8A
EXAMPLE 3
19.6A
9.8A
9.8A
EXAMPLE 3
9.8A
9.8A
EXAMPLE 3
9.8A
9.8A
Violates best practice concept
of isolated failure domains.
EXAMPLE 3
(One Possible Solution)
• “This cabinet provides two
redundant 23kVA power feeds.”
• Does not necessarily translate
into, “Can implement 23kVA of load.”
• 7 chassis @ ~16.5kVA
did not work!
• Apparent maximum =
6 chassis @ 36U (14.1kVA);
• Wastes 40% of power capacity;
• Wastes 25% of rack space
(assuming 48U cabinet);
Metering Accuracy
Metering Accuracy
ISO/IEC 62053‐21 = +/‐1%
Circuit Breaker Residual
Outlet Metering Inlet level Metering
Status Monitoring Current Monitoring
Metering Accuracy ‐ Squelch
Raritan
IEC 62053
0.2% 1200:1 A
1% - Class 1
0.3% 2400:1 A
Metering Accuracy – Phase shift
IEC 62053
1% - Class 1
Metering Accuracy icw. Circuit Breakers
 Detect the Root‐cause of unplanned “suspected-Trip-CauseOutlet Xx”

outages per branch
 Gain time on device testing in case of
circuit breaker trip event 1
Circuit Breaker Trip
 Leverage instant alerting to limit Event
 business impact and improve MTTR 3 2
CB Peak current sensor

 Flags the outlet responsible
for tripping the breaker
Residual Current Metering
Residual Current Monitoring (RCM)
Residual Current Device (RCD)
Residual Current Monitoring
Basic electric theory says sum of currents in a
closed loop = zero.
When leakage occurs sum of currents does not equal 0.
Sensor is a current transformer with inlet phase
& neutral wires passing through it.
Risks of current leakage: avoid downtime
• Proactive detection of a leakage current in high‐availability facility.
• Regulatory testing required by law in some regions
Germany, Austria, UK moving towards EN 50600
• Permanent monitoring, automated saves cost
Residual Current: causes?
• Old or damaged cable isolation
• Leaking capacitators
• Failing power supplies
• IEC‐60950‐1 compliance
Residual Current: Sensor options
RCM Type A RCM Type B
Detects AC leakage and is Detects AC and DC leakage and is
sensitive down to 6mA sensitive down to 30mA.
leakage. Neutral Monitoring
UL/IEC 60950‐1 + IEC 62020 standard Compliance tolerates leakage up to 3mA (Type A) and 30mA (Type B)
Residual Current: Testing
Avoid false positive/negative readings, ignore current measured under 3mA
 Useful for testing the alerting system
 Ensures Sensor Readings reliability
Minimize Human Error
Human Error: Data
• CNET 22%
• Uptime Institute 70%
Human Error: Color
• Easily Identify Power Feeds
• Critical and non‐critical Feeds
• Corporate identity
• Phase Marking
• Alternating Load Balancing Circuits
(L1/L2, L2/L3, L3/L1)
Human Error: Locking
Outlet locking
- PDU side
- Server side
Equipment failure and redundancy
PDU redundancy
Additional features
- Temperature sensors
- Smoke detection
- Door‐lock solutions
- USB webcam security
- Cascading/ Daisy chain
- Modem/ SMS notification
PDU redundancy: Power Sharing
Alert
Feed A Power
Down on
feed B*
In case of a feed
outage, your PX
controller stays
powered and Power Share
sends alerts
Alert
This Feed
Feed B Power is
Down
Feed A1Feed B1 Feed A2Feed B2 Feed A3Feed B3
Rack 1 Rack 2 Rack 3
Regular Cascading
Power Loss
Regular Cascading With Power Sharing
Power Loss
Power Sharing ON
Regular Cascading With Power Sharing
Networking Cascading
Eliminate daisy‐chains
single points of failure
Protocols / Options
- Modbus daisy chain
- Modbus ring
- Master / Slave setup
- SNMP
Switch
PDU redundancy: Replaceable controllers
Outlet switching benefit and risk
Why outlet switching
• Remote reboot / Lights out
• Outlet sequencing
• Load Shedding, UPS
• Security
• In‐rush current
• Graceful shutdown
Outlet switching: Relays
• Non‐latching
• 0.5W – 1.0W ON status
• Default state generally ON
• Non configurable power‐on
• Latching
• 0.0W ON status, only consumes when changing state
• User configurable power‐on relay: pre‐outage state
or power cycle
Relays: (near) Zero‐Crossing
Synchronize relay switching > Cheap components
Automatic Transfer Switch (ATS)
When do you need an ATS
Switching times
• Switching power supplies (SMPS) frequently cited with 15ms+ holdup time (one cycle at 50Hz);
Example: HP DL360 G9 Power Supply
• But reality is not deterministic.
• Can depend on load, capacitance, ambient heat,
AC power curve, etc.;
• Real‐world experience: switchover for ~12ms –
non‐zero probability of server reboot;
Cost of outage
Avg. cost of downtime:
2010: $5.617
2016: $8.851 (+57%)
Avg. downtime:
84 minutes
Transfer Switch
Technical considerations
Switch Technology
Switch point
Power Quality
ATS (Automatic Transfer Switch)
– Electromechanical Relay
Stromquelle 1
– Advantage:
• Power loss free transfers
– Disadvantage
• Risk of Arc Welding
• Slow Stromquelle 2
STS (Static Transfer Switch)
– Thyristor / SCR
Stromquelle 1
– Advantage
• Very Fast switching
– Disadvantage
• Energy consuming transfer
Stromquelle 2
HTS (Hybrid Transfer Switch)
– Thyristor / SCR + Relay
– Advantage: Fast Switching Stromquelle 1
Stromquelle 2
Switch Point
– Make‐before‐Break
Closed transition
– Break‐before‐Make
Open transition
Power Quality
EN50600: Power Quality according to EN50160
• Voltage tolerance +‐10%
• Frequency tolerance +‐0,5Hz
• Unbalance
– Voltage
– Phase
• Harmonic Distorsion
Power Quality
Causes of downtime
1% < U < 90%
– Short downtime (1ms – 1min)
Lightning, failing breakers, Arc flashes

‐ Black Out – planned or unplanned
– Voltage drop
– Longterm failure >3 minutes
‐ Brown Out
– Ie. Overload
– Rare in the region due to UPS/ Gen.set
Questions?
THANK YOU
Shukran Jazilan
‫شكرا جزي‬
Backup: Environmental
80

Datacenter

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Datacenter

Uploaded by

Copyright:

Available Formats

DATA CENTER SOLUTIONS

HP Proliant DL380G5 HP Proliant DL380G6

5kAIC typical; Finger-safe cylindrical fuse holder;

 Detect the Root‐cause of unplanned “suspected-Trip-CauseOutlet Xx”

CB Peak current sensor

RCM Type A RCM Type B

Rack 1 Rack 2 Rack 3

Rack 1 Rack 2 Rack 3

Rack 1 Rack 2 Rack 3

Lightning, failing breakers, Arc flashes

You might also like