Is DC the power to solve heat problems?

(continued from previous page)

Several power and cooling problems now afflict those who run data centers--large rooms filled with the computing, networking and storage equipment at the heart of large-scale information technology operations.

Processors throw off vastly more waste heat than before, a situation aggravated by the way they're packed in more densely. On top of that, the appetite for equipment shows no signs of abating, as computerization penetrates ever more deeply into people's lives. The problem has become severe enough that installing computers now often requires discussions not just with IT staff but also with those in charge of the power and air conditioning for a building.

"I think distributing DC power in a single rack or cabinet is a viable alternative."

--Bob Sullivan, data center expert, Uptime Institute

"There's a merge coming between IT and facilities. You can't just talk to the IT guy. You have to talk to facilities as well," said Ron Mann, director of engineering in HP's Infrastructure Group.

Historically, a standard 6-foot-tall rack of computing gear consumed 2 to 4 kilowatts of power and threw off a corresponding amount of waste heat, Mann said. Today, 7 to 10 kilowatts is more common, and 15 to 30 kilowatts can be found. For perspective, 15 kilowatts is the same amount of power used by 150 100-watt light bulbs.

Using DC doesn't help the problem of hot, power-hungry processors, but it does get rid of power supplies and their cooling fans in a server. They don't go away entirely, of course, but typically are relocated at the top of a rack where their hot waste heat will rise and not be drawn into server cooling intakes.

Although HP isn't as aggressive a DC advocate as Rackable, it does use the approach to power its blade servers, using bulk power supplies within a rack. Putting a larger power supplies inside the rack is more efficient than having smaller ones in each server, Mann said. For example, four 5.25-inch-thick power supply units can power 96 blade servers with dual Opteron processors from Advanced Micro Devices, he said.

Sun's Bechtolsheim is unconvinced of the merits of DC, though. The crux of his argument is that DC requires two conversions: one from outside AC to 48-volt DC for distribution within the building, and a second, within servers, from 48 volts to 12 volts. Even if each conversion is 90 percent efficient, wasting 10 percent of power as heat, the combination makes the overall process 81 percent efficient.

By contrast, a single AC-DC conversion with the server uses power supplies rated at 90 percent efficiency, Bechtolsheim said.

Tim Dougherty, director of blade strategy at IBM, doesn't always agree with Sun, but this time is an exception.

"It's one of those things that always looks better on paper than it turns out to be," he said. Sure, there are DC advocates, but they can't sidestep power conversion problems altogether, he said: "All they do is move the problem somewhere else in your computer room."

[Seaspray0]

Perspective

I have worked on industrial sized unninterruptable power supplies and can attest to some of the problems dealing with DC. Power is a multiple of voltage and wattage. To transmit the same power over a lower voltage (i.e. 48 volts vs 120 volts) you have almost two and a half times the current, whether it be DC or AC. That's why distribution lines from a power plant run on such high voltage. You can transmit an huge amount of power with a low current if you increase the voltage to several thousand volts. DC also does not translate to another voltage like you can do with AC. Transformers do not work with DC at all. To step DC up or down is very ineffient and requires specialized electronics. AC motors are more efficient, smaller, and cheaper to make. The use of 3 phase power makes AC even more attractive.

This is all great, but computers (and just about all electronics) need DC. Since you must convert AC to DC, it would make sense to do all of it all at once. A large scale DC rectifier being fed by 3 phase AC will be way more efficient than the single phase rectifiers used now. The hard part is distributing the DC once you generate it. Power cables will need to handle 2 1/2 times the current for 48 volts versus 120 volts. You'll also experience an increase in corrosion at the electrical joints caused by current continuously traveling in one direction rather than alternating (battery posts also exibit this type of corrosion). You'll also need computers that are addapted to running on DC rather than AC.

The benefits? Some improvement in efficiency but mostly heat dissapation. By removing one of the major heat causing components, you are lowering the amount of heat accross all the racks. The DC rectifier will generate heat, of course, but this heat is centralized in one unit and easier to deal with (may even be able to locate the rectifier outside and not have to provide any cooling for it at all).

[Phil Somerset]

Modifying computers

In the middle of your post, you talk of the need to adapt computers to run on DC, where earlier you say that just about all electronics need DC. The second statement is true. Of the first, I'd point out that just about all electronics consist of circuitry that uses DC bundled with a power supply that converts AC to DC. Modifications would mostly consist of removing the power supply and providing input(s) for DC voltage(s).

[Phil Somerset]

Modifying computers

In the middle of your post, you talk of the need to adapt computers to run on DC, where earlier you say that just about all electronics need DC. The second statement is true. Of the first, I'd point out that just about all electronics consist of circuitry that uses DC bundled with a power supply that converts AC to DC. Modifications would mostly consist of removing the power supply and providing input(s) for DC voltage(s).

[Phil Somerset]

Modifying computers

In the middle of your post, you talk of the need to adapt computers to run on DC, where earlier you say that just about all electronics need DC. The second statement is true. Of the first, I'd point out that just about all electronics consist of circuitry that uses DC bundled with a power supply that converts AC to DC. Modifications would mostly consist of removing the power supply and providing input(s) for DC voltage(s).

[timfox123]

They never got it

If you want to distribute DC with low loss you
need voltage (I^2R). I would like to see an Air
cooled AC to DC converter on top of the building
which takes out the AC for this. Also explore
using this waste heat to heat the building's hot
water. They also need to explore putting a blade
in a sealed "can" and putting cooling fluid
around it. Again the waste energy can be
recovered. This all would have to be done by a
professional that does just this but look at the
long term savings. I know I am going to get
flamed due to saying "put fluid and high voltage
DC together" but it would have to be done right.
One server rack stays in place for how long? A
good server rack says there for a long time, just
the computer processors and the likes change, the
rack infrastructure stays and the next computer
runs on electric also.

[jdscott]

Reality Check

The assumption is that AC is brought straight off the street and into the server so there is only one conversion. Not likely, most companies use a UPS; the AC is converted to DC, put onto the same bus as the batteries to maintain a static charge, then inverted to AC sent to a 208/110 transformer, then sent to the server which converts the AC to DC. Regardless of the status of the AC, the UPS unit continues to convert / invert so that the load is never interrupted.

DC on the other hand is converted from AC, and sent directly to the equipment. Batteries are connected without the need for expensive electronics. Multiple rectifiers and battery stings can be attached for unparalleled uptime compared to a complicated UPS system with phase timing, logic circuits, static bypass, etc.

[bforbes]

UPS is the key

You're right! The fact that every server is running on a UPS is the missing piece that they didn't consider. Batteries are inherently DC so the incoming AC to a UPS gets converted to DC and then back to AC before it gets converted back to DC by the server power supply.

You can distribute DC in the datacenter (with the disadvantages outlined above) or with AC to a UPS at the bottom of the rack which would deliver DC to the servers in the rack. Yes a decentralized UPS is more of a pain/cost etc etc, but it's a trade off decentralized UPS vs DC wiring.

If the UPS outputs DC (basically just simple batteries with a charging circuit as pointed out above) you only have AC->DC(high voltage)->DC (12V/5V/3V) instead of AC->DC(high voltage)->AC->DC (12V/5V/3V)

Plus, show me an actual server that uses a 90% efficient AC power supply! I haven't seen any...

[pj-mckay]

Do the maths firsr though please...

Power = Voltage x Current, therefore you have a huge increase in current when you go DC, and that current leads to losses at junctions, in the form of heat which leads to corrosion/deformation, which leads to more heat, and the cycle continues till total breakdown. We've stuck with AC distribution for a reason.. efficiency. Any half baked idea to run huge bus bars through a room had better get some serious home work first. Incidentally; have you seen photos of folk that have had their wedding bands caught around the 14.4volts in a car battery. It may be low voltage, but the high current leads to massive burns and trauma. That could be us next!

[Phil Somerset]

Good Point!

The initial AC-to-DC conversion when coupled with a bank of batteries becomes a de facto UPS. Hang some big capacitors off it, and you've eliminated almost all crosstalk, signal coupling, etc. Now all we've got to deal with is the problem of distributing hefty DC currents as noted in the article and previous posts.

It's been correctly pointed out that currents are higher with lower voltages, and that higher currents cause increased I^^2 * R (I Squared R) losses and potential terminal corrosion. Why not distribute the DC as a higher voltage and reduce it to needed voltages at the destination? If you have a rack that has four banks of blade servers, bring in 192 volta and split the voltage four ways (with additional regulation circuitry to account for unequal loads).

[heystoopid]

Oh well

Oh well, go visit a telephone central exchange, if possible and observe how the professionals, power large scale installed rack equipment, and note the size of the emergency backup batteries, that allow reserve changeover time before the emergency generators kick in and come upto speed in the event of AC power failure!

For they use DC powered bus on the rack mount telephone exchange equipment!

Seriously poor reporting

Any discussion of AC or DC current in the Computer Room without talking about the UPS/battery backup situation and the inherent dangers of running DC vs AC is pointless.

Heat is only one part of the equation.

[Earl Benser]

No significantt danger in running DC....

......especially at computer voltages.

[Mark Shapiro]

DC clearly has costs, but there are other advantages

Previous comments about DC's disadvantages were quite
instructive. Low voltage requires higher current, and
correspondingly higher resistance losses, heat, and corrosion at
the terminals. Safety is clearly critical, and 3-phase AC still
sounds best for motors.

Since UPS backup would be simpler (according to B Forbes
comment) that could tip the scales to DC for datacenters. But I
would also like to see a standard, say 12 V DC, for all the DC
electronics we have at home: PCs, phones, PDAs, music players,
TVs, stereos. Think of all the incompatible plug adapters we
would avoid. And you could have standard plugs for planes,
trains, and automobiles.

Yet another advantage: fuel cells and solar cells produce DC. A
big part of their cost is the inverter. If we cut out all the
conversions, how much do we save?

[bforbes]

12V DC home distribution

Some people do distribute 12V around the home (especially if they have solar power), however it's an extra set of wires to run and outlets to have. Maybe the simple way to get going would be to have UPSes or big power strips, output 12V (with cigarette lighter connections) so you could have a common dc connector for all those phones/mp3/players etc. Now if we could just get all the manufacturers to agree on the end of the plug that goes in to the device!

[jbass29503]

DC vs AC

Using DC in large Data Center would make them more effiecent then AC not just from the reduced heat but from the fact that there would be no harmonics from the switched power supplies that created the heat to begin with, and why not give the server the kind power of power that it uses anyways, the power supply just converts it to DC, In my opion it is the PC manaufacture that stop supporting DC servers due to the fact that there are more consumers (including business offices) with only AC power that buy the servers/workstations.

[bforbes]

DC power supplies $$$

You're right a big reason we don't see servers with option for DC power supplies is volumes. So many people have AC only, that the server manufacturers, if they offer a DC supply charge huge $$$ for it. Cisco and (ironically given the article) Sun, and a some of the blade manufacturers, as well as industrial/telecom computers (Ziatech/PTI, Force/Solectron, etc.) offer DC supplies (usually powered of -48V)

[203129769353146603573853850462]

Has nothing to do with Edison

Edison's issue was over long-distance distribution.

Poor comparison.

[Mark Shapiro]

For DC to be effective, there must be a standard

The absolutely critical need is for a universal standard. Just like
120V, 60 cycle AC with the ubiquitous three prong plug, a 12V
DC (or even 12V/5V/3V) standard would lower cost, and waste,
dramatically.

The cigarette lighter is a de facto standard already for cars and
boats, but I'm sure that IEEE, or some other relevant standards
body, could do even better.

For starters, imagine a small brick that you plug into a 120 V
outlet. In turn it has 2, 4, or more standard receptacles, one for
every gizmo you own. For a few bucks more you throw in a
battery and have a UPS for your computer. Better yet, the
receptacles start to appear in cars, planes, taxis, coffee shops,
etc. Plus, fuel cells and solar cells no longer have to drag
around huge inverters.

This can't happen quickly, but it can't happen at all without a
universal standard. How do we start?

[Earl Benser]

An exercise in the obvious....

... computers run on DC power. Utilities supply AC power. A power
converter is required to change AC to DC. Power converters do
work and so generate heat. Using an external power converter
reduces the heat created within the computer.

Now, any one ready to move on to Second Grade????

[Mark Shapiro]

Fuel cells, which create DC power, look even better...

Here's why fuel cells are looking better and better for DC server
farms.

They produce DC power directly. They are efficient, reliable,
clean, quiet. For example, UTC has a 200 KW model. It has 3
main parts: a natural gas reformer to convert gas to hydrogen,
the fuel cell stacks, and an inverter to convert DC to AC.

Does this suggest a match made in heaven? Just drop the
inverter, and supply the DC output directly to each rack. Since
the voltage of each stack is a function of the number of cells
stacked in serial connection, the voltage can easily be tuned to
12 VDC.

The question is, who has the clout to bring server makers
together with fuel cell makers to make this happen? (Answer:
Google.)

[Maccess]

Another DC advantage

Another DC advantage is power reliability as there are more alternative power sources. Fuel cells, as you've mentioned, solar cells, etc. can all be configured for 12V. UPSs don't have to deal with the inefficient process of inverting 12V to make it 110/220V only to have the server PS bring it back down to 12V/5V

I have set up an Internet Cafe in a rural area in South East Asia. We used a 12V system. LCDs take a 12V input, so it was easy. About the only thing you can't run on a 12V line is Airconditioning, Laser Printers and Dry Powder Photocopiers.

Everything else: the computer, LCD monitor, Inkjet printer, network equipment (not all support 12V, but many do, just choose the ones which run off 12V).

We have 12V back up batteries. When the mains go down, which happens quite often, we have power. No airconditioning, but we have running computers.

We are also exploring using solar cells for charging the batteries, and maybe even fuel cells.

[Maccess]

Another DC advantage

Another DC advantage is power reliability as there are more alternative power sources. Fuel cells, as you've mentioned, solar cells, etc. can all be configured for 12V. UPSs don't have to deal with the inefficient process of inverting 12V to make it 110/220V only to have the server PS bring it back down to 12V/5V

I have set up an Internet Cafe in a rural area in South East Asia. We used a 12V system. LCDs take a 12V input, so it was easy. About the only thing you can't run on a 12V line is Airconditioning, Laser Printers and Dry Powder Photocopiers.

Everything else: the computer, LCD monitor, Inkjet printer, network equipment (not all support 12V, but many do, just choose the ones which run off 12V).

We have 12V back up batteries. When the mains go down, which happens quite often, we have power. No airconditioning, but we have running computers.

We are also exploring using solar cells for charging the batteries, and maybe even fuel cells.

In a pinch we can even use the power from a running car's alternator.

[eugened]

2 more considerations

1. There are too many different DC voltages involved to make a single voltage distribution practical. DC-to-DC converters generate waste heat too. Might be more optimal to look at a low voltage, say 48V 400Hz (was/is aircraft standard) so that power conversion throws off less waste heat.

2. Look under your desk and count the number of power bricks you find. When you consider all the offices and homes, these probably waste more power than data centres. So the best solution might be to have a low voltage (electronic devices only) reticulation system for offices and homes. Might be ideal to use the above 48V 400Hz standard.

Most important of all, it needs IEEE standardisation, preferably aligned with PoE (Power Over Ethernet) standards to create an electronics, communications and computing devices power source standard. Implemented as a single unit incorporating a UPS and having multiple single voltage outlets (standardised plug/socket) would provide welcome relief to clutter and wasted power.

[mancfrank]

Holistic approach required

Andy Bechtolsheim's comments do not take in to account the whole story. The server has cooling, the power supply has cooling, take the power supply cooling outside of the space of the server cooling and the cooling requirement becomes less stringent. One of the ways telecomms company achieve their high (>99.999%) uptime is the reliability of their DC power supplies.
It's not the solution for every data centre but I'm convinced for large server farms this could be more efficient overall.

[Earl Benser]

Please don't call it 'holistic'....

... maybe it actually is, but there is so much BS in the market place
that uses the term 'holistic' that 'holistic' tends to signify fraud
more than anything else. Another case of a good word being
destroyed by bad people.

[mulca_g]

DC Efficiency is better, but real win is replacement of ACUPS

Contrary to being a myth, the math regarding DC efficiency is actually very simple and quite compelling. When systems are evaluated in terms of power conversion efficiency, it's important you start from the processor, and then work your way back to the utility, taking into account all conversion losses. As opposed to what is stated in your article, typical ATX-compatible, "silver box" AC-DC power supplies used in servers these days rarely provide much better than 70% overall operating efficiency. In fact, some redundant models and some older models, where the average power draw can be well below 40% of the unit's rated power, are considerably lower than this. In reaction to these shortcomings of traditional AC-DC multiple output computer power supplies, there is presently a utility industry sponsored initiative (www.80plus.org) to bring these supplies up to 80%. There are very few choices that achieve this yet. In contrast, the latest products available for the DC approach provide an AC-to-48V efficiency of 92%, followed by an ATX compatible DC-DC efficiency of 94%, for a net of greater than 86%.

There are servers that utilize a single 12V output from the AC-DC power supply, and best in class efficiency for these units is in the high 80's. However, to these conversion losses must also be added the losses of DC-DC converters on the microprocessor board that convert 12V to 5V, 3.3V and other voltages. The net effect is to bring overall server power conversion efficiency back down to less than 85% for an AC-DC approach. Many of these 12V only approaches locate the AC-to-12V converter a few feet from the data processing apparatus, and often lose a few percent more inefficiency through the high current/low voltage DC power distribution wires.

Along with improved efficiency, a DC approach also removes significant heat from the server chassis. This, along with the superior reliability of lower voltage circuits, will provide a significant reliability boost to large-scale systems.

While AC-UPS-fed cabinets with an intermediate DC bus will provide efficiency gains, the real home run with DC power comes when you consider the data center from a overall system perspective. Replacing the central AC-UPS with a DC-UPS distributed amongst the data processing equipment will significantly reduce both up-front and ongoing operating costs. Contrary to the traditional telecommunications approach, where the DC plant is segregated to a remote part of the facility and large conductors are bussed throughout the facility to delver the power, a distributed DC-UPS approach distributes AC throughout the facility and converts this to an intermediate 48VDC bus within the server cluster. This can be done either within the cabinet, or shared amongst a number of cabinets.

Uninterruptible power is achieved through connection of a 48V battery across the intermediate DC bus. Battery backup power is provided to service only the local cluster load for the brief amount of time required to switch from one AC power source to an alternate AC power source. (Similar to what is done within AC-UPS systems, in the event of a utility AC mains power failure, sustaining power is provided through either an alternate utility feed, or motor-generators located at the facility.) Consequently, in the distributed DC-UPS approach the battery typically only requires a modest amount of space. For example, a 10 kW cluster can be serviced by a battery that fits within 2U of rack space.

A Distributed DC-UPS approach eliminates the need for a facility-level AC UPS, and all the cost, inefficiencies and headaches these add to the system. Rather than "sidestepping power conversion problems" and moving them "somewhere else in your computer room", DC power will free the computer room from the paradigm that all server equipment must be AC powered, and the tyranny of the AC-UPS this carries with it.