From particles to waves, part 2b

 So far, I’ve discussed the meaning of periodic reflective boundary conditions for the 1-D wave equation and looked at the general solution, or what might be called the necessary generic form of the solution, to this equation as it applies to a small disturbance (y-direction small-amplitude motion) on a taut string having linear density σ and tension τ. This solution is written as

y(x,t) = f(x-ct) + g(x+ct)

or (see end of previous post)

y(x,t) = f(t-x/c) + g(t+x/c),

where c = (τ/σ)^1/2 is the speed of the disturbance, which is actually two disturbances when both terms in either of these equations are nonzero—the f-disturbance travelling to the right and the g-disturbance travelling to the left.

I’ll first look at how we get from the generic traveling wave solution to a traveling sine wave solution for the infinitely long string, then look at the form the generic solution takes for the semi-infinite string (where there’s a boundary at one end), and finally look at what happens when the traveling wave (the “incident wave”) on a semi-infinite string is a sine wave.  Part 3 (the next, and last, post on this subject) will be a return to the string fixed at both ends and a look at what happens to the generic form of the solution and the sine wave solution in that case. The beginning of Schrödinger’s wave mechanics in 1926 will also be discussed in Part 3. 

(The titles of Erwin's first four wave mechanics papers were "Quantization as an Eigenvalue Problem" Parts 1, 2, 3, 4. Since many people, myself included, have eigenproblems understanding the proper eigencontext of the prefix "eigen," I will include some eigencomments in my Part 3.)

The generic form of the 1-D wave equation solution above is derived without consideration of initial conditions or boundary conditions and has a unique speed associated with it, but no single frequency or wavelength like we might expect from a wave. But, as you probably realize, if it did have a frequency and wavelength in place of speed—if we just tried to substitute λν = c or ω/k = c into the generic solution—it wouldn’t be a generic solution anymore because f and g would have to be pure sine waves having their own frequencies ν or ω, and wave numbers k,  with f and g necessarily expressible, for instance, as Acos(kx ± ωt + φ), where A, k, ω, and φ are constants, although we can have a superposition of sine waves of different amplitudes and frequencies. 

Dudley Towne says in Section 1.7 of Wave Phenomena, where he’s still considering waves on a string of infinite length:

 

Contrary to the impression which may be created by the fact that waves of sinusoidal form are the most frequently cited examples of waves, it is to be noted that neither wavelength nor periodicity is an essential characteristic of a wave. An initial waveform of any desired shape determines an allowable solution to the wave equation.

 

After making this cautionary statement, Towne gives several reasons why sinusoidal waves are fundamental in the study of wave phenomena: 1) a sine wave is “one of the simplest analytic functions which is bounded on an infinite interval,” 2) it represents a pure tone in sound and a “spectral color” of light, and 3) it “achieves an overwhelming importance through Fourier’s theorem.” Dudley also mentions dispersion: “In some contexts involving the phenomenon of dispersion, the wave equation is not satisfied except for waves of sinusoidal form, and then only when a wave propagation velocity appropriate to the given frequency is substituted.”

 

Sinusoidal Progressive Wave on an Infinite String

For the 1-D wave equation, how do we get to a traveling sine wave solution, y(x,t) = Acos(kx±ωt + φ), from the generic traveling wave solution, y(x,t) = f(x-ct) + g(x+ct)? The same way we always get from the general to the particular when dealing with differential equations: initial conditions and boundary conditions. For the infinite string, there are no boundaries, so we just have initial conditions. And for partial differential equations like the 1-D wave equation, it’s really initial functions we have to deal with, a subject I’ll mention again further down in this post (see the paragraph below that starts with “Now”).

Right now, I just want to particularize the generic solution to the case of a traveling sine wave. That means we start with an initial sinusoidal disturbance on our infinitely long string, courtesy of Towne’s Section 1.7, “Description of a Progressive Sinusoidal Wave,” where he says

 

An initial waveform which is sinusoidal is described by the function

y(x, 0) = Re {ye^ikx}

where k and y are given constants. If the complex amplitude y is written in the form

y = y_me^iφ,

the real amplitude y_m determines the maximum displacement of the string from the x-axis and the phase φ determines the position of the curve with respect to translation parallel to the x-axis. … When the quantity (kx) increases by 2π, the function y(x,0) goes through one cycle of its values. The corresponding increment in x is, of course, the wavelength, λ. Thus k(x  + λ) = kx + 2π, or kλ = 2π, or k = 2π/λ. The parameter k is referred to as the wave number and may be thought of as the number of waves contained in a distance of 2π meters.

 

Towne chooses φ = 0, which makes the initial waveform a cosine function—but let’s keep the solution in exponential form for the moment. Next, we declare the initial waveform to be in motion in the positive x direction, making g(x+ct) in the generic solution of the wave equation identically equal to zero. The travelling wave solution is now

 

y(x,t) = f(x-ct) =  Re {y_me^ik(x-ct)}.

 

Towne says, “Note that if we focus on a particular particle of the string, say the particle [located at] x = x₁, the motion of this particle as a function of time is sinusoidal.” In a footnote, he says “in any progressive wave the curve which describes the history of a single particle is of the same geometric form as the wave profile of the string.” Then he continues on about the motion being sinusoidal, but I won’t quote him from this paragraph, except to say this is where he brings angular frequency into the argument: as the wave travels, the particles of the string undergo simple harmonic motion, with angular frequency ω = kc. Making this substitution in the above equation, we get

 

y(x,t) = Re {y_me^i(kx-ωt)} = y_mcos(kx-ωt).

 

Thus, starting from the generic form y(x,t) = f(x-ct) + g(x+ct), we’ve arrived at our desired expression for a traveling sinusoidal wave.

Now I’ll mention the use of “initial functions,” as discussed in Towne’s last section in Chapter One, “Initial Conditions Applied to the Case of a String of Infinite Length,” which, by the way, he says “may be omitted without loss of continuity.” (Dr. Rolleigh didn't omit this subject in his notes.)  I’ll omit the derivations but state the main idea. Since the wave equation is a 2^nd order partial differential equation, knowing the initial positions and initial velocities of all the particles on the string—the initial wave profile and the initial velocity profile—allows the functions f and g to be uniquely determined. As Towne says at the beginning of Chapter Three (Boundary Value Problems, where we’re about to go now), “the solution is uniquely determined for an infinite string by the requirements that y(x,t) and dy/dt reduce at t=0 to given functions.”

 

Sinusoidal Progressive Wave on Semi-Infinite String

The semi-infinite string extends from minus infinity on the x-axis to x=0, where it’s tied or fixed or anchored to a solid wall—the boundary. The 1-D wave equation must be satisfied within the interval – ∞ < x < 0. Towne chooses the form of the generic solution to be

y(x,t) = f(t-x/c) + g(t+x/c)                 (3-1)

“to simplify algebraic manipulations,” he says. I’ll just use two pages from the book to show the general waveform case and the sinusoidal waveform case. The boundary condition is y(0,t) = 0.

y(x,t) to vanish, regardless of the value of t. The fixed end is a node, and the spacing between successive nodes is a half-wavelength. Unlike the progressive wave, the sinusoidal waveform does not undergo translation parallel to the x-axis. This type of motion is referred to as a standing wave.

d) The amplitude of the simple harmonic motion of an individual particle depends on its location. The particles halfway between the nodes have the largest amplitude for their motion. These halfway positions are referred to as antinodes.

 

Before moving on to the string having boundaries at both ends, I’d like to note a couple of things about the semi-infinite string.

First, as you can surmise from the picture of the sine wave(s) above, there is no restriction on the frequency (or wavelength)  the standing wave can have. This is shown by the standing wave solution itself,

 

y(x,t) = 2y_msin(kx) sin(ωt),

 

where k and ω can take on a continuous range of values. (And now that we do have a sinusoidal wave, we necessarily have the relation ω/k = c = (τ/σ)^1/2.) This range of continuous values is in contrast to waves propagating in a “confined region,” such as on a string fixed at both ends, where the allowed frequencies are determined by the dimensions of the region, such as the length of the string.  This latter case has a discretely-infinite set of frequencies that are characteristic of the dimensions of the limited region--a periodicity in the frequency space and k-space. In the former case, there is a continuously-infinite number of possible sinusoidal standing wave frequencies that can be formed.

Secondly, the standing wave solution is a product of a function only of x and a function  only of t, something I’ll be returning to in Part 3—and something probably familiar to you from the usual “separation of variables” technique. I don’t like to invoke this rather abstruse technique unnecessarily, and neither does Dudley Towne, so I’ll follow his Wave Phenomena Chapter 15 discussion in Part 3.

From Particles to Waves, Part 2a

In part one, the emphasis was on the meaning of periodic reflective boundary conditions in the case of the one-dimensional wave equation. This is also called a rigid or fixed boundary condition. Other possible b.c.'s for the string are the free and the circular conditions. (The circular b.c. for a string is apparently the same as the more general periodic b.c., which I mistakenly said in earlier versions of these posts is the name for what is really a reflective b.c.) In Part 3, the final part, the emphasis will be on solutions to the wave equation for reflective boundaries, but first it may be helpful to be reminded of what the “most general solution” is to the 1-D wave equation with no boundary or initial conditions prescribed. 

This generic solution is most easily found by using a "difference of two squares" method to factor the equation. I first saw this method used in some notes written by Prof. Richard Rolleigh for a second-semester classical mechanics class I took at Hendrix College. (Yeah, two semesters of classical mechanics were required, and ditto for classical E&M.) The overall subject of Dr. Rolleigh's 14-page handout is "Classical Mechanics of Continuous Media." The key idea is that the 1-D wave equation, 

∂ ²y/∂x² — (1/c²) ∂²y/∂t² = 0,                                          (1-4)    

(see the middle part of Wikipedia's Wave Equation entry) can be factored into

(∂ /∂x + (1/c) ∂ /∂t)( ∂ /∂x – (1/c) ∂ /∂t)) y(x,t) = 0. 

The factored form of the equation makes it simpler to look for the necessary relation between x and t that makes the left-hand side identically equal to zero: in the first factor we have the requirement that the first partial of y with respect to x equals (1/c) times the first partial of y with respect to t , and in the second factor that the first partial of y with respect to x is (-1/c) times the first partial with respect to t. Since the factors commute with each other, an expression satisfying either requirement is a solution.

The simplest relations between x and t that satisfy these requirements are x-ct and x+ct. (Below we’ll see, courtesy of Dudley Towne, that these are the only relations between x and t that fit the 1-D wave equation as a function of position. We’ll also see [Exercise 2] how these relations can be re-written as a more useful time dependence.) We are thus looking for any functions f(x-ct) and g(x+ct). To see for yourself how this works, try each of these in the factored form of the equation. Hint: you should write these as f(u) and g(v), where u=x-ct and v=x+ct, and then do the chain rule partial derivatives.

One more standard thing needs to be pointed out before I reproduce three pages from Chapter 1 of Towne’s book: The functional form f(x-ct) represents a waveform travelling to the right on the string, and the form g(x+ct) represents a (possibly different) waveform traveling to the left. To show this for f(x-ct) we can move time ahead by Δt, so that distance is changed by Δx = cΔt, giving

f(x+cΔt, t+Δt) = f(x+cΔt – c(t+Δt)) =  f(x+cΔt – ct - cΔt)) =f(x-ct)).

The waveform’s vertical displacement is unchanged, meaning the waveform has moved to the right at speed c. (How would you do the same calculation for g(x+ct)?)

Now for the Towne pages, wherein we see how the sum (superposition!) of f(x,t) and g(x,t) given above provides the necessary form of the one dimensional wave equation solution. Prior to this page, Towne has shown f(x-ct) to be a solution, by doing what I mentioned above as a chain rule hint for using f(u).

…pressed by the general solution. We are not as fortunate in the case of the two-and three-dimensional wave equations, for which such a convenient form of general solution does not exist."

Towne’s general description of the motion of the string, and of the variables, constants, and assumptions involved in the 1-D wave equation, are also worth reproducing here, and the same goes for his first two textbook problems in Chapter 1, which I’ll give as exercises for the interested reader:

 

The motion of the string can be specified by a function y(x,t) which is a function of the two independent variables x and t. Thus, for example, the graph of y versus x determined by the equation y = y(x, t₁) depicts the shape of the string at fixed time t₁, and the graph of y versus t  determined by the equation y = y(x₁,t) specifies the transverse displacement of the single particle located at x₁ as a function of the time. The latter graph is sometimes referred to as the history of the particle at x₁. … Assume that the string in the undisturbed configuration has a uniform linear mass density, σ, and is under uniform tension T. Also assume that any changes in either of these quantities which occur during the motion of the string are sufficiently small so that they may be neglected. … The equation [1-4] is a valid representation of the physical conditions in the system only so long as the inclination of the string remains everywhere small. [Meaning the slope is small, so ∂y /∂x << 1. Not really what's shown in the drawings above.] … The particles of the string are moving in a transverse direction, whereas the waveform propagates along the string. The “propagation” is one of form, but not of substance.

 

 

Exercises

1. Which of the following are solutions to the one-dimensional wave equation for transverse waves on a string? [For use in (c), the wave speed, or propagation speed of the waveform in the x direction, is given by c = (T/σ)^1/2.]

 

a) x²-2xct+c²t²

b) 10(x²- c²t²)

c) σx² + Tt²

d) {sin[(x-ct)³]}^1/3

e) 2x-3ct 

f) 10(sin x)(cos ct)

 

2. If h(u) is an arbitrary twice-differentiable function of u, show by direct calculation that y(x,t) = h(t + x/c) satisfies the one-dimensional wave equation. What relation does this solution have to the general solution written as y(x,t) = f(x-ct) + g(x+ct)?

From particles to waves, Part One

Here’s a reminder of where we’ve been recently and where we’re going next.

We’ve been touring through thermodynamics and statistical mechanics with Herb Callen and Harry Robertson, and in my previous post also with Mark Whittle, discussing bounded regions of space containing macroscopically homogeneous systems of particles.

More specifically, as Robertson says, we’ve looked at "the simple systems that are treated in terms of statistical thermophysics [and these] are made up of atoms or molecules, the spatial distribution of which is described by probability densities that are constant (or periodic on the scale of a crystal lattice) over the volume of the system."

Next, we’re going to be dealing with the probability densities of quantum mechanics and comparing the Callen and Robertson thermodynamics postulates with quantum mechanics postulates.

As a prelude to doing that, we’re going to now look at Dudley Towne’s discussion of “Waves confined to a limited region,” which is the title of Chapter 15  in his book Wave Phenomena, published in 1967, the year in which A Serious Man takes place. (Our beleaguered Professor Gopnick might have had a copy of it. He definitely has a copy of Dirac’s Principles of Quantum Mechanics on his desk when he hangs up on Sy Ableman after finding the failing student’s bribe money in the envelope.)

Specifically, we’re going from discussing atoms and molecules in some kind of container to discussing waves in some kind of container, where “container” means a bounded region of space.  There are three parts to the discussion (the latter two parts will be coming along soon).

Part One: one-dimensional waves in a "box"

First we’ll consider waves on a string of length l, the classic classical-mechanical one-dimensional wave problem. Two dimensions of space, y and x, are needed to describe the motion, but it’s nevertheless called one-dimensional motion because the string only moves up and down (a particle on the string can move only in the ±y direction). Towne says that for the one-dimensional wave equation (with c as wave speed),

∂ ²y/∂x² — (1/c²) ∂²y/∂t² = 0

and boundary conditions y(0,t) = y(l,t) = 0, “an important property of the general solution to this problem is that any function satisfying all three of these conditions is necessarily periodic in time.”

For most of us, or at least for me, this seems to be the same old statement that  sinusoidal waves with certain frequencies, the standing wave or resonant frequencies, satisfy these three conditions.

Aye, but Professor Towne is saying more than that!

Sine waves on a string or in a box are not just periodic in time, they’re also periodic in space. Towne’s statement only says “necessarily periodic in time,” and he immediately gives a non-sinusoidal example of such a confined wave:

Consider a short pulse which starts out near the center of the string and travels in the positive x-direction. This pulse is reflected with inversion at the fixed end x = l and is reinverted upon reflection at x = 0. When the pulse returns to the original position, traveling in the same direction that it was initially, the string is in identically the same condition as it was initially, and the process repeats itself. The period T is therefore the length of time required for the pulse to travel twice the length of the string: T = 2l/c. If the disturbance on the string is the superposition of any number of pulses traveling in either direction, they all repeat themselves in this interval and the entire motion is periodic.

Towne’s discussion should clear up any confusion you or I might have had about the meaning of periodic boundary conditions. The waves that meet the above three conditions may be periodic in space, but they don’t have to be. They do have to be periodic in time, and this is one example of where the name “periodic boundary conditions” comes from.*

That’s really all I want to point out here in Part One. But the example of the pulse on a string is worth thinking about in terms of its analogy to the quantum mechanical particle in a box, and its relation to a classical or quantum harmonic oscillator, and the fact that such a pulse can be expressed as a Fourier series (yeah, the pulse needs a specific profile in order to do that). Also: do the coefficients in a Fourier series expansion have any relation to the components of a Hilbert space vector?

Postscript 30 October 2022

Here’s an unexpected example of particles confined to a limited region that uses periodic boundary conditions in one dimension. I learned about this example from page 273 of the recently published textbook by Robert Swendsen, An Introduction to Statistical Mechanics and Thermodynamics (Second Edition, 2020). We’ll be coming back to Swendsen later, because he’s a former student of Herbert Callen and gives a new version of Callen’s postulates of thermodynamics, with four “essential” postulates and three “optional” postulates. The optional ones aren’t applicable to all thermodynamic systems but simplify the analysis when they are applicable, Swendsen says.

The free expansion of a classical gas is the example of particles confined to a limited region that so unexpectedly uses periodic boundary conditions in one dimension (well, maybe it meets your expectations, but I hadn't expected it). It’s in Swendsen’s chapter on irreversibility, Chapter 22, and comes from a 1958 Physical Review paper by Harry Lloyd Frisch: “An approach to equilibrium,” Phys. Rev., 109, 22-9 (1958). “Following Frisch’s 1958 paper,” Swendsen says, “we will eliminate the difficulties involved in describing collisions with the walls at x= 0 and x= L by mapping the problem onto a box of length 2L with periodic boundary conditions.” Yeah, I guess you can't call it a free expansion when the particles collide with walls. More about that later!

Post-postscript 25 November 2022

I wasn't aware until now that "periodic boundary conditions" of a more general nature are used in simulations and computer games. Now I know. Swendsen's discussion of Frisch's use of periodic boundary conditions for the free expansion of of an ideal gas is a simple example of a simulation.

*Oops, I recently realized, after writing the above post-postscript, that I'm wrong about this being a case of periodic boundary conditions, since the boundary conditions of a string fixed at both ends are reflective b.c.'s. Sorry for causing further confusion. Our man Towne didn't use the term "periodic boundary conditions," by the  way, so don't blame him. I just made the assumption that b.c.'s resulting in necessarily periodic-in-time wave motion on the string would be called periodic b.c.'s. 

Early Universe as an example of a "simple system"

Knowing when the first stars were formed, soon after the Big Bang, and understanding how they produced the building blocks of the first galaxies is an important scientific question and one of the primary science goals of JWST. We know that the elements that are needed for life and modern technology, such as carbon, silicon and gold, were ultimately created in early stars—but we don't currently have a good understanding of how this happened.  From the article "James Webb telescope: How it could uncover some of the universe's best-kept secrets," by Martin Barstow.

Before moving on to the different kinds of postulates that can be used to quantify and justify quantum mechanics, I'll take a brief detour to an unexpected and rather spectacular example of a simple thermodynamic system: the universe in its early stages of existence, just before it cooled enough (to about 3000 K) for atoms to be formed. This was 380,000 years after the Big Bang, when the signature of the cosmic microwave background radiation (CMB) we observe today was left behind.

As I mentioned previously, our main man Harry Robertson defines a simple system as a “bounded region of space that is macroscopically homogeneous,” and says of the possible boundaries for such a system that they can be material boundaries or “described by a set of mathematical surfaces in space.” 

Mark Whittle, an astronomy professor at the University of Virginia, in Lecture 16 of “Cosmology: The History and Nature of Our Universe,” gives the conditions that make the early universe simple:

1. The young universe is almost perfectly homogeneous (“it’s a smooth gas”).

2. It contains (relatively) simple components: light, electrons, nuclei, and dark matter. “And furthermore, the light, electrons, and nuclei are all tied tightly together into a single coherent photon-baryon gas.” So the early universe was a photon-baryon gas plus dark matter.

3. The gas of nuclei, electrons and light is in thermal equilibrium.

4. The deviation from exact homogeneity—the lumpiness—is very slight, so it’s considered to be in the “linear regime,” and the physics of how the lumpiness grows in time is simple (linear, not nonlinear).

The only other condition we need in order for the early universe to fit the criteria for a simple system is a boundary—and this is more of a problem.  In Lecture 15, “Primordial Sound—Big Bang Acoustics,” Mark Whittle says, “Although the universe has no spatial boundary, it is bounded in time. At a given time, e.g., the [time of formation of the] CMB, regions of a specific size are caught at maximum or minimum compression or rarefaction, and these specific region sizes create the strongest patches on the CMB, giving the harmonics … .”

The slight deviations in the density of the almost-homogeneous gas of electromagnetic radiation, simple nuclei (protons+neutrons), and electrons result in acoustic waves in the gas that bounce in and out of the denser regions. These waves each have a fundamental frequency (about 50 times lower than the low end of human hearing range) and harmonics. And what good are harmonics? Different harmonics make different musical instruments playing the same note (the fundamental) sound different. The size and shape of an object and the components that make up the object can be roughly determined by what harmonics it produces.

Studying the harmonics may not be able to give a unique or exact size, shape, and material composition of a vibrating object, but given a limited set of possible components in the material and other information, the harmonics give the best possible educated guess. 

In his intro to Lecture 16, Whittle says, “Different objects make different sounds, and this is also true for the Universe: If the Universe had different properties, its primordial sound would be different. Cosmologists have been extremely successful at measuring many properties of the Universe by comparing computer calculations of the primordial sound with the sound of the real Universe. The match is in fact so good that it essentially proves that the Hot Big Bang Theory is valid and robust.”

Don’t forget that the CMB radiation is black-body radiation. It’s in the microwave region of the electromagnetic spectrum today because it’s been redshifted by cosmic expansion from its original black-body spectrum centered in the infrared region.  The infrared photons were what mainly gave the photon-baryon gas its pressure and temperature, and the small fluctuations in the density of this gas—the rarefactions and compressions—are what it is measurable today as former sound waves frozen in time on the microwave background. These fluctuations tell us when and how the first stars and galaxies formed and what the universe is made of. We now know the presence of dark matter is necessary for stars and galaxies to form, but we still don’t know what dark matter is.

By the way, Astronomy magazine's cover story this month is called "Assembling the Universe."  So there's an example of how it's better to call even a "simple system" an assembly rather than a system. See my previous post for more on this subject, under the sub-heading "An aside related to vocabulary."

For a short summary of the big bang and its consequences, see this "early universe" website.

For a short summary of what homogeneous means, see this Organic Valley blog post on homogenized milk. A carton of homogenized milk sitting in a refrigerator at 277 K is another example of a simple thermodynamic system in equilibrium. 

Thermo Postulates of Callen and Robertson, etc.

 (see my previous post for a general comparison of the C and R textbooks)

Callen:

I.    There exist particular states (called equilibrium states) of simple systems that, macroscopically, are characterized completely by the internal energy U, the volume V, and the mole numbers N₁, N₂, …, N_r of the chemical components.

II.   There exists a function (called the entropy S) of the extensive parameters of any composite system, defined for all equilibrium states and having the following property: The values assumed by the extensive parameters in the absence of an internal constraint are those that maximize the entropy over the manifold of constrained equilibrium states.

III.  The entropy of a composite system is additive over the constituent subsystems. The entropy is continuous and differentiable and is a monotonically increasing function of the energy. [Callen then adds: Several mathematical consequences follow immediately.]

IV.  The entropy of any system vanishes in the state for which (∂U/∂S) = 0, with volume and all chemical potentials held constant. That is, the entropy goes to zero at the zero of absolute temperature.

 

Robertson:

1.  The macroscopic thermodynamic equilibrium states of simple systems are completely specified in terms of the extensive parameters (U, V, N_i in the present context), where the N_i are the component populations, with all the information contained in the entropy function S(U,V,N_i), called the fundamental relation.

2.  The entropy of a simple system is a continuous, differentiable, homogeneous-first-order function of the extensive parameters, monotone increasing in U.

3.  The entropy of a composite system S({U_i, V_i, {N_i}}) is the sum of the entropies of the constituent subsystems:

S({U_i, V_i, {N_i}}) = S₁({U₁, V₁, N_1i }) + S₁({U₂, V₂, N_2i }) + … .

4.  The equilibrium state of a composite system when a constraint is removed maximizes the total entropy over the set of all possible constrained equilibrium states compatible with the constraint and the permitted range of extensive parameters.

5.  The entropy of a simple system approaches zero when

(∂U/∂S)_{V, {N}} → 0

 --------

Like Callen, Robertson uses the idea of equilibrium states of simple macroscopic systems as a starting point, with equilibrium sort of implicitly taken to be any state that can be characterized completely by a constant internal (average) energy, constant volume, and constant “component populations” (Robertson) or “mole numbers of the chemical components” (Callen).

Notice that C and R both use a completeness specification. The word “completeness” resonates a little bit here with the “complete set of commuting observables” in quantum mechanics. We could say an equilibrium state in thermodynamics is characterized by—or exists because of— the existence of stationary values of a complete set of extensive parameters,  which are U, V and {N_i} in the entropy representation.

Also notice that a fundamental relation such as the monotonically increasing entropy function S(U,V,N_i) in thermodynamics is described similarly to the wave function in quantum mechanics, in that both are said to contain all the information about the system.

An aside related to vocabulary

The word “system” should be used with some humility and caution, rather like the word “universe”. An ideal isolated system in thermodynamics is a universe unto itself (if you don’t tamper with it), while, conversely, the universe is a system unto itself.  What are they really? Models. Mainly, “system” is a very broadly used word in science and engineering and it can close off creative thinking rather than promoting it. Some people—Darwin and Fowler in their 1922 and 1923 papers, and Schrödinger in his little Statistical Thermodynamics book—have chosen to use the word “assembly” instead of “system” when discussing Boltzmann’s ideal gas and Planck’s ideal electromagnetic resonators. These authors also use the word “system,” but they refer to the individual molecules or Planck resonators/vibrators/oscillators as the systems that make up the assembly under consideration.  Thus, in their view, an assembly is macroscopic and must be assembled, and its “component population” is made of N identical (sub)microscopic systems that each possess mechanical and maybe electromagnetic energy (KE, PE). The assembly itself then has some overall thermal energy distribution. A more complicated assembly would be made up of a set {N_i} of different types of systems.

Now back to (thermodynamic) systems analysis

But I will continue talking about thermodynamic “systems” and their constituents since this is the usual terminology.

Before he provides the above postulates (Chapter 2, p. 66), Robertson describes a simple system (his bold emphasis) as “a bounded region of space that is macroscopically homogeneous.” He goes on to say: “That we regard a system as simple may imply nothing more than that we have not examined it on a fine enough scale. The failure to do so may be one of choice, or lack of it. We may usually choose to regard an obviously complex system as simple if its internal behavior is not involved in the problem at hand … the simple systems that are treated in terms of statistical thermophysics are made up of atoms or molecules, the spatial distribution of which is described by probability densities that are constant (or periodic on the scale of a crystal lattice) over the volume of the system.” Robertson then discusses the nature of possible boundaries of simple systems, such as their being either material or “described by a set of mathematical surfaces in space,” or diathermal (allowing thermal contact) or adiabatic (preventing thermal contact), or restrictive to matter flow in various degrees (semipermeable, open, closed), and whether they allow transfer of energy via a work process (such as a movable piston).

I’ve discussed Robertson’s and Callen’s statements of the postulates of thermodynamics in this post in order to prepare for my next post, where I’ll compare these postulates with those of quantum mechanics and also, mainly, try to figure out why we don’t normally see the square of the wavefunction or the squares of the complex quantum mechanical superposition coefficients used as probabilities in the Shannon expression for entropy. Meanwhile, here’s a blog post on that subject: Wavefunction entropy. [The comparison of thermo and quantum postulates wasn't my next post. As of December 16, 2023, I still haven't managed to get to it. Later!]

 

Postscript, March 22: Callen's postulate II and Robertson's corresponding postulate (No. 4) are too abstract to be understood, at least for me, without an example.  Callen gives a general sort of example, and also gives a problem at the end of the section (1.10) to further illustrate the example.

But first let's consider the problem that these particular postulates of C and R are supposed to solve. It's el problema grande of thermodynamics, which as Callen states it (and I've previously quoted) is: The single, all-encompassing problem of thermodynamics is the determination of the equilibrium state that eventually results after the removal of internal constraints in a closed, composite system.

Example: (page 26) "Given two or more simple systems, they may be considered as constituting a single composite system. The composite system is termed closed if it is surrounded by a wall that is restrictive with respect to the total energy, the total volume, and the total mole numbers of each component of the composite system." ... (page 28) "In the case of two systems separated by a diathermal wall we might wish to predict the manner in which the total energy U distributes between the two systems. We then consider the composite system with the internal diathermal wall replaced by an adiabatic wall and with particular values of U⁽¹⁾ and U⁽²⁾, with U⁽¹⁾ + U⁽²⁾ = U. For each such constrained equilibrium state there is an entropy of the composite system, and for some particular values of U⁽¹⁾ and U⁽²⁾ this entropy is a maximum. These, then, are the values of U⁽¹⁾ and U⁽²⁾ that obtain in the presence of the diathermal wall, or in the absence of the adiabatic constraint."

Problem 1.10-3: "The fundamental equation of system A is

S = (R²/v₀ θ)^1/3(NVU)^1/3

and similarly for System B. The two systems are separated by a rigid, impermeable, adiabatic wall. System A has a volume of 9x10^-6 m³ and a mole number of 3 moles. System B has a volume of 4x10^-6 m³ and a mole number of 2 moles. The total energy of the composite system is 80 J. Calculate and Plot the entropy as a function of U_A/(U_A + U_B) . If the internal wall is now made diathermal and the system is allowed to come to equilibrium, what are the internal energies of the individual systems? (R², v₀ , and θ are constants.)"

Post-postscript, March 25: (The red text above is what I left out or wrote wrongly in my initial post. The red text below is what I re-wrote on March 29.) Non-numerical solution to our Problem 1.10-3: The given constraint U = U_A + U_B applies to the composite system with either the adiabatic wall or the diathermal wall. The composite system entropy sum S = S_A + S_B applies when the adiabatic wall is in place and subsystems A and B are energetically distinct, AND when the diathermal wall is in place with the particular values of U_A and U_B found from maximizing S = S_A + S_B.  These are the thermal equilibrium values with the diathermal wall in place.

We have a continuum of different values for S_A and S_B that satisfy the sum S = S_A + S_B with the adiabatic wall in place, and these are Callen’s and Robertson’s “constrained equilibrium states” over which we want to maximize S. Using the energy constraint to write total entropy in terms of system A’s energy, and using constants k_A and k_B as stand-ins for all the alphabetic and numerical constants given in the problem,

S  =  S_A + S_B   =  k_A U_A ^1/3 + k_B U_B ^1/3

=  k_A U_A ^1/3 + k_B (U – U_A)^1/3

dS/dU_A =  (k_A /3) U_A ^-2/3 –  (k_B /3)(U – U_A)^-2/3  = 0,

(not checked yet for min instead of max) resulting in

U_A = U/[1 +  (k_B / k_A)^3/2]

and

U_B = U/[1 +  (k_A / k_B )^3/2].

The ratios are easy to calculate, with alphabetic constants and numerical exponents canceling:  k_A / k_B  = 27/8.  Plotting the normalized relation "entropy as a function of U_A/(U_A + U_B)" is left to the intrepid reader for the moment.

Robertson’s "uncertainty" vs Callen’s "disorder"

Harry Robertson may take Herb Callen’s place as our main man of statistical mechanics. Robertson is the author of Statistical Thermophysics (1993). I don’t see this textbook cited very often, but I remember when I ran across it on the “New Books” shelf in the Physics-Math-Astronomy library at UT-Austin. I liked it enough to make some photocopies from it right away, but I never bought a copy of the book myself until a few weeks ago. Based on what I’ve studied in it so far, I think it’s the best graduate-level statistical mechanics text I've seen.  And maybe it’s also one of the most overlooked, although Mark Loewe mentions it in his 1995 notes I included in my previous post (see last page of notes, last footnote, about the “surprise” function--more about that later). One reason instructors might not use Robertson's book as a text is that complete solutions to the problems are given in the book. Instructors would have to assign different problems if they wanted to grade students on problem solving.

Callen uses the idea of the disorder in a probability distribution {p_i} – he labels it {f_i} but I’m using Robertson’s notation—whereas Robertson uses the idea of the uncertainty in information content of such a distribution.  They both write the entropy (a lá Claude Shannon) as

S = –k∑ p_i ln p_i .

For the special case (the subject of this post) of having N possible outcomes, the maximum entropy occurs when each outcome has a probability 1/N. For Callen, this is the maximum disorder for the N outcomes. My problem with Callen’s “disorder” interpretation of entropy is that I don’t see why this equiprobability of outcomes should be called the maximum disorder.  Callen’s discussion of this, at the beginning of Chapter 17, doesn’t agree with my intuition of what disorder is. Equiprobability seems very ordered!

Robertson’s discussion of how the {p_i} should be interpreted in the N outcomes case is worth quoting from the beginning of his section on Information Theory, on page 3:

 

It has long been realized that the assignment of probabilities to a set of events represents information, in a loose sense, and that some probability sets represent more information than do others. For example, if we know that one of the probabilities, say p₂, is unity, and therefore all the others are zero, then we know that the outcome of the experiment to determine y_i will give y₂. Thus we have complete information. On the other hand, if we have no basis whatever for believing that any event y_i is more or less likely than any other, then we obviously have the least possible information about the outcome of the experiment.

 

Having “no basis whatever for believing” that any one event is more, or less, probable than any other means assigning all the events the same probability, and this equiprobability of events is something I can understand intuitively as giving the least amount of information. And this is the “information-theoretic maximized uncertainty” that Robertson uses in place of Callen’s maximum disorder.

Robertson subscribes to what is often called the subjective assignment of probabilities, while Callen sticks with the frequency-of-occurrence assignment of probabilities. Both men use Claude Shannon’s 1948 information theory formulation to define entropy (see above equation), but their interpretations of what the {p_i} represent are very different.

Callen wants to use only frequencies of occurrence as a measure of probabilities, as in, for example (my example), the objectively calculable and measurable frequencies of various sums-of-dots appearing on the upward faces of many dice tossed simultaneously many times. Robertson, on the other hand, is a follower of Edwin Jaynes’ 1957 re-interpretation of Shannon’s information theory as a subjective-probabilities theory.  Lots of controversy is involved in that interpretation. (The dice-throwing example probabilities are not uncertain enough to even need a subjective-probabilities approach.)

In spite of not agreeing with the subjective interpretation, Callen gives a great discussion (p. 380) of the subjectiveness in the general meaning of the word “disorder” before he introduces Shannon’s solution to the problem of the meaning of disorder. As one example of subjective disorder, Callen says a pile of bricks appears to be very disordered, but it may be “the prized creation of a modern artist,” and thus may not be any more disordered than a brick wall once the artist’s intention is understood.

But Callen then says this sort of apparent subjectiveness in the idea of disorder is removed by Shannon’s definition of the type of disorder used in information theory.  “The problem solved by Shannon,” Callen claims, “is the definition of a quantitative measure of the disorder associated with a given distribution {p_i}.” By “quantitative measure” he means the entropy expression above, and there’s no controversy over that. The controversy is about how the set {p_i} can legitimately be determined.

That’s as far as I’ll go on the subject at the moment. I only wanted to say how much better the idea of minimal information (and thus maximal uncertainty in information) is than the idea of maximal disorder when the case of equiprobability of outcomes is being described.

P.S. (15 October 2021)  Maybe it occurred to you that there are two different ideas of "maximum" being discussed here? After a week of pondering these and related concepts--like uncertainty and probability in quantum theory compared with uncertainty and probability as discussed above in relation to the expression for entropy--it's finally occurred to me that we have a particular case of known equiprobable outcomes as a case of maximum entropy, and we also have the general case that involves the entropy function with unknown or arbitrary p_i 's and we want to find it's maximum and in the process find the p_i themselves for this case, which would usually be a thermodynamic equilibrium case.  

Robertson distinguishes the particular case from the general cases at the end of his Information Theory section, prior to his introducing the idea of Maximum Uncertainty in the next section: "For a given number of possible events n, it is easily shown that S is a maxima when the events are all equally probable ... . The next problem to be examined is that of assigning the p_i's on the basis of our knowledge. This problem leads to the conceptual foundation of the approach to statistical mechanics used in the present development." The "present development" means Robertson's textbook.

The particular case is the subject of the first Problem at the end of Robertson's first chapter. In this problem, he gives the constraint of the probabilities summing to unity as the only constraint to use in maximizing the entropy expression, meaning one Lagrange unknown multiplier rather than the usual two--try it! You only need the above entropy expression and the constraint

∑ p_i = 1,

where the sum goes from 1 to n. (Or 1 to N, or however you want to label the number of possible outcomes.)

Callen's and Robertson's textbooks are actually very complementary in the different subjects, and levels of subjects, and interpretation of the subject itself, that they cover. Callen's book is an undergrad text, the best one in my opinion, and it covers more thermodynamics than Robertson's book, which is intended to be a graduate-level text (the best one in my opinion) with statistical mechanics as its primary subject rather than macroscopic thermodynamics. The difference in their interpretations has been the subject of this little essay, thank you.