Entropy Revisited, Gorilla and All 

David Siminovitchsiminovitch@hg.uleth.caPeter T. Landsbergptl@maths.soton.ac.ukAllen

Nussbaumnussbaum@ece.umn.eduRichard H. TourinBenjamin
Crowellcrowell00@lightandmatter.comElliott H. Lieblieb@princeton.eduJakob
Yngvasonyngvason@thor.thp.univie.ac.at 

Citation: Physics Today 53, 10, 11 (2000); doi: 10.1063/1.1325179 

View online: http://dx.doi.org/10.1063/1.1325179 
View Table of Contents: http://physicstoday.scitation.org/toc/pto/53/10 
Published by the American Institute of Physics 

Articles you may be interested in 

A Fresh Look at Entropy and the Second Law of Thermodynamics 
Physics Today 53, (2007); 10.1063/1.883034
 L ETTERS 
Entropy Revisited, Gorilla and All 
T he article by Elliott H. Lieb and
 
Jakob Yngvason (PHYSICS TODAY, April, page 32) was indeed a fresh look at entropy and the second law  of
thermodynamics, but I believe it did not do justice to Constantin Carathéodory who, in his axiomatic development of
thermodynamics, was the first to replace traditional statements of the second law (heat engines, cyclic processes). He is
1

men tioned once in the article, but in a context that does not recognize his seminal role in laying an axiomatic
foundation for thermodynamics (beginning more than 90 years ago!). The mathematical complexities of his formulation
of the second law not only obscured the physical simplicity of his idea, but for some time they were a significant
impediment to the use of his methods by physicists. Hans Buch dahl’s exposition of the Carathéodory formulation
2

played an extremely important role in popularizing the axiomatic development begun by Carathéodory, so that by the
late 1960s, there were several undergradu ate textbooks incorporating his approach. Although the authors do give
3,4

Buchdahl credit, there is even more reason to credit Carathéodory: He started the whole movement! 
References 
1. C. Carathéodory, Math. Ann. 67, 355 (1909); Sitzungsber. K. Preuss. Akad. Wiss. 39 (1925). 
2. H. A. Buchdahl, Amer. J. Phys. 17, 41, 44, 212 (1949).  
3. H. A. Buchdahl, The Concepts of Classi cal Thermodynamics, Cambridge U. Press, London (1966). 
4. C. J. Adkins, Equilibrium Thermody namics, McGraw-Hill, London (1968). DAVID SIMINOVITCH 
(siminovitch@hg.uleth.ca) 
The University of Lethbridge 
Lethbridge, Alberta, Canada 

I n connection with the interesting

article on entropy by Lieb and Yng 
Letters submitted for publication should be sent to Letters, PHYSICS TODAY, American Center for
Physics, One Physics Ellipse, College Park, MD 20740- 3843, or by email to ptletter@aip.org (using your
surname as “Subject”). Please include your affiliation, mailing address, and daytime phone number. We
reserve the right to edit letters. 
vason, I should like to make a few comments to broaden the back ground and possibly to provide a slight change of
emphasis. 
  On page 34, the authors rightly emphasize extensivity as normally essential for the second law of thermo dynamics.
One must not overlook, however, the possibility of gravitation al interactions. Their effect is that, when the system is
doubled, its energy does not double, because the gravita tional interaction enters as an “extra.” The same applies to the
entropy. 
Furthermore, for large enough masses, objects become unstable and collapse (see, for example, Chandra 
sekhar’s limit for white dwarfs), which incidentally causes a problem with the concept of the “thermody namic limit.”
Also, similar systems can merge with a resulting decrease of entropy.   1
   On page 36, the positivity of the specific heats is noted. One might observe here that the effect of gravi tation can
lead to negative specific heats. See for example, ref. 2. 
These effects are reminders that “systems of particles interacting by long-range forces ought to be ruled out on page
1 of any book on (‘normal’) thermodynamics.” Relativistic ther modynamics is of course finding a way around these
3

difficulties. But “normal” thermodynamics rules these possibilities out (usually implicitly).   Regarding the symbol for
adiabatic accessibility, appropriately stressed on page 34 of the Lieb and Yngvason article, some credit might be given
to Constantin Carathéodory for an early use of this concept. His work need no longer be avoided on grounds of
mathematical complica tion, since Carathéodory’s principle can now be related to Kelvin’s princi ple, mentioned on
page 33 of the article, rather simply. Proceedings of the first international conference on thermodynamics provide
4 5

further citations. 
  To the (perhaps rhetorical) ques tion raised on page 32 of the article as to whether statistical mechanics 
is essential to the second law, the answer is presumably “No,” since the main work of Sadi Carnot and Rudolph
Clausius preceded that of Willard Gibbs. 
References 
1. P. T. Landsberg, R. B. Mann, Class. Quantum Grav. 10, 2373 (1993). 2. R. D. Sorkin, R. M. Wald, Z. Z. Jiu, 
Gen. Rel. Grav. 13, 1127 (1981). See also P. T. Landsberg, R. P. Woodward, J. Stat. Phys. 73, 361 (1993). 
3. P. T. Landsberg, in The Study of Time III, J. T. Fraser, N. Lawrence, D. Park, eds., Springer, New York (1978),  p. 118. P. T.
Landsberg, Seeking Ulti mates, Institute of Physics, Bristol, UK (2000), p. 121. 
4. P. T. Landsberg, Nature 201, 485 (1964). 5. P. T. Landsberg, Pure Appl. Chem. 22, 215 (1970). Also published as a sepa rate
volume, Proceedings of the Interna tional Conference on Thermodynamics, Butterworths, London (1970). 
PETER T. LANDSBERG 
(ptl@maths.soton.ac.uk) 
University of Southampton 
Southampton, UK 

T he article by Lieb and Yngvason

 
discusses aspects of entropy and the second law of thermodynamics that are not well known or men tioned in the
commonly used texts. Most of the article is devoted to a complicated presentation of the prop erties of entropy,
concluding with the statement that they have developed an axiomatic foundation for thermo dynamics and eliminated
the intu itive but hard-to-define terms such as “hot,” “cold,” and “heat” from its development. This desirable goal,
however, was achieved in a much simpler and more transparent manner by an old friend, the late Herbert B. Callen of
the physics department at the University of Pennsylvania. His book shows how a postulatory foun dation of 1

thermodynamics makes entropy a necessary variable for the understanding of thermal, chemical, mechanical, and
electrical processes. It is interesting that the authors cred it Peter T. Landsberg for beginning the movement to use the
second law as a pillar of physics in its own right, but no interaction between Callen and Landsberg is mentioned in the
article. 
To show the simplicity and beauty of Callen’s approach, a brief summary of his ideas follows. He first points out
that conservation of energy is not sufficient to explain many physical phenomena: A body at a uniform tem perature
does not spontaneously develop a gradient nor does a homo geneous chemical system separate

© 2000 American Institute of Physics, S-0031-9228-0010-220-0

OCTOBER 2000 PHYSICS TODAY 11 

into components. An extensive vari able, the entropy S, is needed to explain why. Callen states that   Entropy depends
on the internal energy U, volume t, number of con stituents N, and charge q of a sys tem through a relation of the
general form S = S(U, t, N, q), which may be inverted, in principle, to give U = U(S, t, N, q). 
  Entropy is defined only for equi librium states. 
  The entropy of a system for which all differences in temperature, pres sure, concentration, and so forth  are allowed
to equalize will go to a maximum. 
Differentiating the expression for U then leads to the absolute temper ature, the negative of the pressure, and the
chemical and electrostatic potentials. With those definitions in hand, the first law of thermodynam ics follows, and
includes thermal, mechanical, chemical, and electro static energy. It is then a simple matter to show that conservation
of energy combined with the entropy maximizing postulate leads to well known equilibrium conditions, such as
uniformity of temperature, chemi cal potential, or electrostatic poten tial. If N and q are constant, then dU = T dS – P dt,
which we may rewrite as dU = dQ – dW, where +dQ is the amount of heat that enters a body and +dW is the amount of
work done by the body. This is the primi tive form of the first law, and the relation dS = dQ/T can be applied to a
Carnot cycle to verify that all changes in an isolated system lead  to an increase in the entropy. As the entropy of a 2

system 
increases, so does a measure of its organization known as the disorder W. The connection between entropy and
disorder can be expressed as S = k log W  e
 where k is Boltzmann’s constant. An ingenious single combinatorial expression by Cowan, plus the use of
3

distinguishability and the exclu sion principle, shows that maximiz ing S with Lagrange multipliers leads to the
appropriate statistical distribution: Maxwell–Boltzmann, Fermi–Dirac, or Bose–Einstein. I agree with the authors, then,
that the Gibbs–Boltzmann approach is clearly the wrong direction to go, but it also follows that entropy can be
described and explained without all the complex mathematics. 
References 
1. H. B. Callen, Thermodynamics, Wiley, New York (1960). 
2. G. Joos, Theoretical Physics, G. E. 
Stechert, New York (1934). 
3. R. D. Cowan, Am. J. Phys. 25, 463 (1957). 
ALLEN NUSSBAUM 
(nussbaum@ece.umn.edu) 

University of Minnesota, Minneapolis I enjoyed the article on entropy and

 
the second law of thermodynamics by Lieb and Yngvason. I have tried with little success to tell physics pro fessors
that statistical mechanics is unnecessary for understanding the second law. One professor called me a chemist,
apparently his idea of a hopeless ignoramus! 
Nevertheless, I find Lieb and Yng vason’s gorilla model confusing, and remote from practical, physical aspects of
thermodynamics. The clearest, most self-evident, and most useful explanation of entropy seems to me to fall out of the
temperature– entropy diagrams used routinely by mechanical and chemical engineers. The area below the cycle on
such a diagram is simply the portion of process energy that is unavailable for conversion to mechanical work. And the
reason for the unavailability is simple: A thermodynamic process needs an energy inhomogeneity (typi cally a
temperature gradient) to drive it, and most of the process energy is used to maintain the gradient, thus becoming
unavailable for conversion to work—for example, by dissipation in cooling water. By dissipating ener gy, the process
tends to destroy the gradient and stop itself, so energy is continuously supplied to the process cycle at a high level to
maintain the gradient, and continuously dumped at a low level into a homogeneous energy sea, where there are no
gradi ents to act as process drivers. Entropy is the measure of the diminution of energy inhomogeneity needed to drive
processes, and the formalism of ther modynamics enables engineers to cal culate the entropy change of a partic ular
process. 
The “disorder” of statistical 
mechanics corresponds to the homo geneity of the sea of dissipated ener gy, but we don’t need to know this to 
understand the physical reality of entropy and the second law. 
RICHARD H. TOURIN 
New York City 

L ieb and Yngvason’s article is

 
indeed a breath of fresh air on a quantity that is so often described unintelligibly by undergraduate text books as a
measure of “disorder.” 
I am not convinced, however, that their approach to entropy is “inde pendent of any statistical model—or 
even of atoms.” Their definition of entropy is not an operational one. To determine which of two systems has the
greater entropy, we must imagine an infinite number of hypothetical processes that might transform one into the other.
Only if we find such a process do we get a definite answer.  If we do not know of the existence  of atoms, then it is far
from obvious how to compare the entropies of a grasshopper and a flower. 
To operationalize Lieb and Yngva son’s definition, we would have to find a generic strategy for comparing
entropies, rather than creating meth ods ad hoc. Such a strategy exists, but it depends on the existence of atoms: We can
take advantage of the additive property of entropy to subdi vide a system into smaller and small er parts until we arrive
at subsys tems simple enough that we can find their entropy by inspection. Only the existence of atoms gives us reason
to believe that this process of subdivi sion will terminate with subsystems that contain small enough numbers of atoms
to be analyzed trivially. 
It would perhaps be more accurate to refer to Lieb and Yngvason’s defini tion as a metadefinition that states the general
properties that should be possessed by a true operational defini tion of entropy. It would then be a non trivial task to
arrive at the appropri ate operational definition in the case of, say, a truly continuous system. BENJAMIN CROWELL 
(crowell00@lightandmatter.com) Fullerton College 
Fullerton, California 

L IEB AND YNGVASON REPLY:

  We

are happy that our article attract ed the interest of so many letter writers and we appreciate their com ments. Yes, we
would have wished to include more references, but space limitations gave us little room for scholarship. However, our
long paper discusses the history and relationships of the various formula tions of the second law; we hoped that readers
1

would consult that paper. The recent paper of Jos Uffink is also valuable. 
2
 As mentioned by Siminovitch and Landsberg, and by us in our article, Carathéodory is one of the founding fathers, and
it is true that he was the first to emphasize the idea of a relation among states of macroscopic systems based on
adiabatic accessi bility. We regarded his ideas, which date to 1909, as well known, but per haps the connection could
have been clearer. However, although his work motivated Landsberg and the others

12 OCTOBER 2000 PHYSICS TODAY 

we mentioned, his conceptual frame work is different in important respects from the circle of ideas that led to the work
of Giles and then to our work. 
Our work is logically divided into two parts. The first shows that entropy comes out of little more than the list of
pairs of states X and Y such that one can go from X to Y without doing more to the surround ings than moving a weight.
It is this fact that speaks for our approach (and Giles’s book). Quasistatic paths in state space are not needed; calcu lus
and the sophisticated mathemat ics of differential forms are also unnecessary in this part. It is not even necessary to
parametrize equi librium states by coordinates such as energy and volume. Nevertheless, entropy emerges together with
a spe cific formula that determines the entropy function uniquely, except for the choice of units. Carathéodory, on the
other hand, makes essential use of coordinates and differentials in state space, and entropy and temper ature appear to
be more delicate con structs than they naturally are. Likewise, we do not have to intro duce extraneous physical
principles and heat engines, as in the older approach of Carnot and successors. 
What we must introduce in the first part is the notion of adiabatic accessibility, and since we do not have the help of
coordinates, we must do this empirically and without mentioning heat and temperature. This is where the gorilla enters
as a metaphor for the rest of the universe and its possible action on the system under discussion. Unlike Tourin, we do
not find this confusing. Rather, we consider it to be an essential clar ification of the kinds of processes with which the
second law deals. 
Many formulations of the second law have been made, and choosing among them is largely a question of taste. We
submit, however, that our assumptions are easier to under stand than those of Kelvin, Planck, Clausius, and
Carathéodory. Callen’s approach, mentioned by Nussbaum, is excellent (as is the work of Tisza and ultimately Gibbs,
on which he relies), but Callen starts where we leave off. In his approach one postu lates the existence and basic proper
ties of entropy and then runs with the ball. We seek to answer the prior question: Where does entropy—and its
properties, especially its temporal increase—come from? Our answer is that it comes from a relation among
macroscopic equilibrium states 
whose simple properties are, except for one, so obvious that they are taken for granted and rarely men tioned
explicitly.  
The one nonobvious property of equilibrium states is the “compari son hypothesis,” which states that, given any
two states X and Y of a 
system, either X is adiabatically accessible from Y or Y is adiabatical ly accessible from X. The second half of our work,
which goes beyond Giles, Landsberg, and the others we mentioned, turns this hypothesis into a fact, with the aid of
some rea sonable assumptions. Here, for the first time, we describe states by means of energy and volume, in the usual
way, and make some contact with Carathéodory’s approach. We introduce an important assumption that is similar to,
but significantly narrower than, Carathéodory’s main one: For every state X there is anoth er state, Y, somewhere, and
not nec essarily nearby, that is adiabatically accessible from X, but not the other way around. Otherwise, our approach is
mathematically and physically dif ferent from Carathéodory’s. It is not necessary to derive the entropy func tion afresh,
because that was done  in the first part. We simply have  to derive the simpler, nonquantita tive fact that any two states
are comparable. 
The uniqueness of entropy implies that it can be measured conveniently without using the original formula that
established its existence. This answers the objections of Crowell and Tourin; you can find the differ ence of entropy
between two states in the old-fashioned way by measur ing specific heats, compressibilities, and so on. The uniqueness
guaran tees that all experiments will answer the question in the same way. Once we have the existence and unique ness
of entropy, all the techniques of the mechanical and chemical engi neers mentioned by Tourin are at our disposal.  
We disagree with Crowell’s com ment that one can find the entropy of a macroscopic system by cutting it into tiny
subsystems. If, by this, he means that reduction to submeso scopic sizes will simplify thermody namics, we have to
demur. Clearly, additivity and scaling do not hold down to the atomic level because of surface effects. Even if, at some
suf ficiently small size, entropy can be computed from Boltzmann’s formula, the law of its temporal increase may have
to be replaced by an as yet unknown version of the law, perhaps 
a statistical one. This is a fascinat ing and largely unexplored area; see refs. 3 and 4 for additional remarks. 
“Extensivity,” or additivity, can break down on large scales as well, in the presence of gravitational interactions, as
mentioned by Lands berg. This is correct, well known, and worthy of emphasis. However, it lies outside the realm of
the labora tory physics under discussion. 
Nussbaum’s remark about the Gibbs–Boltzmann approach being “the wrong direction” could be misin terpreted to
imply that we have an anti-Gibbs–Boltzmann bias. This would be unfortunate since we do value statistical mechanics
as does any physicist. We said, and we main tain, that the second law, as under stood for equilibrium states of
 macroscopic systems, does not require statistical mechanics, or any other particular mechanics, for its existence. It does
require certain properties of macroscopic systems, and statistical mechanics is one model that, hopefully, can give those
properties, such as irreversibility. One should not confuse the exis tence, importance, and usefulness of the Boltzmann-
Gibbs-Maxwell theo ry with its necessity on the macro scopic level as far as the second law is concerned. Another way
to make the point is this: If the statistical mechanics of atoms is essential for the second law, then that law must imply
something about atoms and their dynamics. Does the second law prove the existence of atoms in the way that light
scattering, for exam ple, tells us what Avogadro’s number has to be? Does the law distinguish between classical and
quantum mechanics? The answer to these and similar questions is “No,” and if there were a direct connection, the late
19th-century wars about the existence of atoms would have been won much sooner. Alas, there is no such direct
connection that we are aware of, despite the many examples in which atomic constants make an appearance at the
macroscopic level— Planck’s radiation formula, the Sack ur–Tetrode equation, stability of matter with Coulomb forces,
and so on. The second law, however, is not such an example.  
As our title read, this is a fresh look at the second law of thermody namics. It is no more obligatory than any other
approach, but happily some readers and colleagues have found it useful.  
continued on page 106

14 OCTOBER 2000 PHYSICS TODAY 

LETTERS (continued from page 15)References 

1. E. H. Lieb, J. Yngvason, Phys. Rep. 310, 1 (1999). Also available at 
http://xxx.arXiv.org/abs/cond-mat/ 9708200. See especially Section 1B. 2. J. Uffink, in Studies in History and Philosophy of
Modern Physics, Perga mon, Exeter, UK (in press). Also avail able at http://xxx.arXiv.org/abs/cond mat/0005327. 
3. E. H. Lieb, Physica A 263, 491 (1999). 4. E. H. Lieb, J. Yngvason, in Visions in Mathematics, Towards 2000, GAFA, Geom. Funct.
Anal. Special Volume (1) 334 (in press). Also available at 
http://www.ma.utexas.edu/mp_arc bin/mpa?yn=00-332. 
ELLIOTT H. LIEB 
(lieb@princeton.edu) 
Princeton University 
Princeton, New Jersey 
JAKOB YNGVASON 
(yngvason@thor.thp.univie.ac.at) University of Vienna 
Vienna, Austria 

Optimal Vision: 
Blurring and Aliasing I n his article “Retinal Imaging and
 
Vision at the Frontiers of Adaptive Optics” (PHYSICS TODAY, January, page 31), Donald T. Miller shows that it is
possible to improve the res olution, contrast, and clarity of reti nal images by correcting for defects in the eye’s optics.
We agree that “the best retinal image quality is obtained with the largest physiological pupil diameter (8 mm) and with
full correc tion of all ocular aberrations.” Howev er, we disagree with Miller’s sugges tion that the quality of vision may
be improved similarly (to achieve “super normal vision”) if the eye’s optics could be “corrected” with “adaptive optics”
to produce the performance of an aberration-free 8-mm lens. 
The angular spacing between reti nal photoreceptors, as Miller states, “represents a neural limitation to visual
resolution.” In terms of com munication theory, this spacing determines the sampling passband of the eye that,
analogous to the bandwidth of a communication chan nel, sets an upper bound on the high est spatial frequencies that
the eye can convey to the higher levels of the brain. The preferred modulation transfer function (MTF)—or spatial
frequency response—of the eye’s optics relative to this sampling pass band is inescapably a compromise between
blurring and aliasing. Because the MTF decreases smoothly with increasing frequency, aliasing can be substantially
decreased only at 
the cost of blurring and vice versa. If blurring and aliasing are prop erly accounted for in terms of their effect on the
information rate that the eye conveys to the higher levels of the brain, then it is the MTF of the 3-mm lens rather than
that of the 8-mm lens that, in normal day light, maximizes this rate for the 50 cycles/degree sampling passband of the
eye. Hence, communication the ory and evolution converge, under appropriate conditions, toward the same optical
1

design. And why not? It seems unlikely that evolution would have missed the opportunity to improve our vision if it
could have done so merely by permitting the pupil to be wider than 3 mm during normal daylight conditions. 
Reference 
1. F. O. Huck, C. L. Fales, D. J. Jobson, Z. Rahman, Opt. Eng. 34, 795 (1995). F. O. Huck, C. L. Fales, Z. Rahman, Philos. Trans.
Roy. Soc. London, A354, 2193-2248 (1996). F. O. Huck, C. L. Fales, Z. Rahman, Visual Communica tion: An Information Theory
Approach, Kluwer Academic, Norwell, Mass., 1997. FRIEDRICH O. HUCK 
 (f.o.huck@larc.nasa.gov) 
CARL L. FALES 
NASA Langley Research Center Hampton, Virginia 

M ILLER REPLIES: Friedrich Huck

 
and Carl Fales raise valid con cerns about realizing supernormal vision. These concerns, however, are also
expressed in my article. The neural system will ultimately limit the degree of supernormal vision that may be achieved
after the aberra tions in the eye are corrected. In my article I state, “In an eye with perfect optics, visual performance
becomes constrained by neural factors, specifi cally the spacing between retinal photoreceptors, which represents a
neural limitation to visual resolution that is only slightly higher than the normal optical limit.” I go on to say that this
would lead to aliasing, which would degrade vision. 
Optimal vision then becomes a compromise between blurring and aliasing. But what constitutes opti mal vision and
what compromise is appropriate for achieving it? These are difficult questions that the vision community continues to
address. Currently, our understanding of the limits placed on vision by the retina and visual pathways of the brain are
not sufficient to provide universal answers to these questions. The search for answers is further compli cated because
visual performance is heavily task-dependent. Visual per 
formance for some specialized tasks will probably decline with adaptive optics. For example, when observers viewed a
steady point source through adaptive optics, it some times appeared green and sometimes red, depending on which
photorecep tor type the light was stimulating. With more natural stimuli, however, subjects have regularly experienced
a strikingly crisp appearance consis tent with the supernormal quality of the retinal image. For everyday vision, the
penalty of aliasing may be outweighed by the reward of heightened contrast sensitivity and detection acuity.  
Huck and Fales’s application of communication theory unfortunately relies on the superficial analogy of the eye as
an electronic video camera. It ignores much of the neural process ing of the image and does not take into account the
type of visual task. A rigorous application of this theory would require a deeper understanding of the visual system than
we present ly have. It is perhaps for these rea sons that the approach of Huck and Fales does not predict the enhanced
vision already experienced with adap tive optics. Ultimately, the extent to which vision will be improved by cor recting
ocular aberrations will be determined in the laboratory. 
DONALD T. MILLER 
(dtmiller@indiana.edu) 
Indiana University, Bloomington 

Moore’s Law and the Future of Computing J oel Birnbaum and R. Stanley
 
Williams of Hewlett-Packard com ment on Moore’s law (P HYSICS TODAY, January, page 38) and dis cuss the
projections of its theoreti cally anticipated validity until 2012, or even 2020. However, it is more a matter of
practicable engineering and technology than it is of theoreti cal limits of the physical theory. 
Interestingly, the researchers of Intel Corp see the whole develop ment rather more pessimistically. For instance,
David Papworth, Intel Fellow, suggests that Moore’s law won’t survive beyond 2004–5. He noted at a VLSi Circuits
Symposium in Hawaii in 1998 that by using two or three times as many transistors to increase performance 1.8 times,
progress continued apace but at cost: Power consumption has doubled or tripled in each generation. Papworth
concluded that, after 2004, density increases will slow down.   1

His colleague Paul Packan, com 

106 OCTOBER 2000 PHYSICS TODAY