Statistical Mechanics

How to Predict Macroscopic Behaviour from Microscopic Laws

1. General Formalism and Definitions

1.1 States

Macrostate – State of a system specified by a few macroscopically observable quantities, e.g. 1kg of
water at 1atm pressure and 300K. Typical quantities are: Energy/Temperature, Volume/Pressure,
Number of particles (U, T, p, V, N)

Microstate – Full description of the system at some fundamental level, e.g. Positions and
momentum of all atoms in a classical system (r1, r2, …, v1, v2, …) or a wavefunction, ψ, in quantum
mechanics.

The aim: To deduce properties of the macroscopic from knowledge of the microscopic.

The problem: A system of NA ~1023 particles require us to consider ~ 10 N A microstates. We never

know what microstate a real system is in.

The solution: Consider an ensemble; many copies of the system, all in the same macrostate but
typically in different microstates. Take an average over the ensemble.

1.2 Assumptions

We assume:

1. Conservation of Energy
2. Existence of Microstates
3. Principle of equal a priori probabilities (PEAPP). For any closed system at eqm, all accessible
microstates are equally likely.

Closed: No exchange of energy, particle or anything else with the outside world.

Eqm: No flow of heat/particles etc.

Accessible: Compatible with macroscopic constraints (e.g. fixed volume)

Equally Likely: Occur equally often in the ensemble (i.e. with equal probability)

1.3 Example

System of QHO’s. A QHO has certain allowed volumes of the energy, En = (n + ½)ℏω, n = 0, 1, 2

The microstate of the QHO is specifired by n , the number of quanta in the oscillator.

Consider a system of N identical QHO’s, all exchanging energy with each other.

⇒The microstate is specified by the number of each quanta in each oscillator, (n1, n2, …, nN). The
macrostate is given by the total energy, U, or equivalently the total number of quanta, M
e.g. Macrostate with M = 1, system with N = 5

5 possible macrostates:

n1 = 1, n2 = 0, n3 = 0, etc.

n1 = 0, n2 = 1, n3 = 0, etc.

etc.

if M=2

15 possible states, 5 with ni = 2, and 10 with ni, nj = 1

For M quanta in N oscillators there are:

 N  M  1! possible microstates

M !( N  1)!

Let Ω(E, N) be be the number of microstates with energy E. PEAPP says that in an ensemble with N =
500 all of the 10299 microstates occur equally as often.

N M Ω(E, N)
5 1 5
5 5 126
5 50 ~3x105
50 50 ~5x1028
500 500 ~10299

Density of States

So far the total energy has been Mℏω for M as an integer or half integer

But more often spacing is not constant…

Smooth out distribution by saying the number of states with

ˆ ( E ) E
energy between E and E + δE is 
δE is an element of energy big enough that in a smooth function but small compared to the
precision of measurement of the energy E. We use Ω and ̂ interchangeably on this course, do not
worry.

1.4 Partitions

Partitions are a way to relate the temperature of a system to its microstates. System of N QHO’s,
energy E. Divide (partition) the system into two parts. N1 + N2 = N

Part 1 has energy E1 and Part 2 has energy E2 = E – E1

Question: What is the most likely value of E1?

Count how many microstates of the whole system give energy E1, for part 1. PEAPP most likely value
is the one with the most microstates.

ˆ ( E , N ) possible microstates.
If part 1 has energy E1 then it has  1 1

ˆ ( E , N ) possible microstates.
Part 2 has energy E2 therefore it has  2 2

The whole system has total  ( E1 , N1 ) ( E2 , N2 ) with E2  E  E1

Maximise total over E1, or better maximise ln total  ln ( E1 , N1 )  ln ( E2 , N2 )

d
ln total  0
dE1
d d
 ln ( E1 , N1 )    ln ( E2 , N 2 )   0
dE1 dE2

d
Both parts of the system have the same value of dE
ln ( E )

Very sharp point at E1=U1, the most likely value.

For large systems, E1 is almost always very close to U1

Parts 1 and 2 are in thermal eqm: they must have the same temperature therefore we can define
temperature by:

1 d  1 d
 ln ( E, N ) or if : T ln ( E, N )
kBT dE E kB dE

Note: we never used the fact that we were dealing with harmonic oscillators. The system was made
of QHO’s. This final result is a general definition of temperature in statistical mechanics.

1.4b) Other Partitions

System with total energy E, total volume V and N particles in total.

What is the most likely value of V1? Need to know

( E,V , N ) , number of microstates now depends on the
volume, V.

Repeating previous argument, at eqm use PEAPP and find

 
T1=T2 and ln ( E1 ,V1 , N1 )  ln ( E2 , V2 , N 2 )
V1 V2

 
Define:  ln ( E ,V , N )
kBT V

Most likely value of N1? Repeat

argument and define chemical
potential:

 
 ln ( E ,V , N )
kBT N

As before; At mechanical eqm, V1 is almost always very close to the most likely value. At diffuse eqm
N1 is almost always very close to its most likely value.

S 1 S p S 
Recall from thermodynamics:  ;  ; 
U NV T V NU T N UV T

Consistency of statistical mechanics with thermodynamics shows that:

S (U ,V , N )  kB ln (U ,V , N )
Boltzmann’s formula for the entropy depends only on counting microstates.

Thermodynamic Derivatives

dU  Tds  pdV   dN
Recall: 1 p 
dS  dU  dV  dN
T T T

Recognising that S(U,V,N) is a function of 3 variables so:

S S S
ds  dU  dV  dN
U NV V NU N UV

U U U
Also U ( S ,V , N ) : dU  dS  dV  dN
S NV V SN N SV

Chemical Potential

This is the intensive variable conjugate to the number of particles. It is defined by:

 dS
  kBT ln ( E,V , N )  T
N dN

As with heat flowing from higher temperatures from lower temperatures, chemical potential flows
from high μ to low μ
1.5 Open System

Recall PEAPP applies to closed systems with fixed energy E. More often we know the temperature of
a system but not its energy (this is an open system).

How likely are different microstates in such a system?

Em

k BT
Microstate “m” occurs with probability proportional to e where Em is the energy of the
microstate.

The energy of the whole system E.

If the small system has energy Em the bath must have energy E - Em

The number of microstates available to the bath is:

G(m)  bath ( E  Em )

Using a Taylor Expansion:


ln bath ( E  Em )  ln bath ( E )  Em ln bath ( E )  ...
E
 1
ln bath ( E )  
E k BT
Em
 ln bath ( E  Em )  ln bath ( E ) 
k BT
Em

G (m)  bath ( E )e k BT

Every microstate of the whole system is equally likely, So the probability of finding the small system
in microstate, m, is proportional to the number of microstates accessible to the bath, G(m):

Em Em
1  
P(m)  G(m)  e kBT z  e kBT

Z
Z is called the partition function. It depends on the temperature and the microstates of the system.

Note: For the bath we only assumed that Ωbath exists so the result is valid for any bath. Boltzmann
distribution holds for any open system of temperature, T.

Ensemble Specifics

1. In a closed system all copies of the system have the same energy, E, therefore PEAPP applies
and all microstates are equally likely, this is known as a microcanonical ensemble.
2. Take an open system at temp, T (with a fixed number of particles). Each copy of the system
has a different energy in general therefore PEAPP does not apply, instead the microstate
probabilities are:
Em
1  k BT
e <<< The canonical system
Z
3. An open system at temperature, T, exchanging particles with its environment (e.g. a droplet
of water) have a probability function of:
Em   N m En   N
1  
P(m)  e k BT
(T ,  )   e k BT

(T ,  ) n

(T ,  ) is the grand partition function.

This is the grand canonical ensemble


Recall:   kBT ln ( E ,V , N )
N

Example

Calculate the average energy <E> of a system in the canonical ensemble.

Em
1  1
 E   Em P(m)   Em e kBT   Em e   Em
m m Z m Z

 
 m
e  Em   Em e   Em
m

 
 E   ln Z  k BT 2 ln Z
 T

1.6 Connection of Statistical Mechanics with Thermodynamics

We have already seen the link for entropy: S  kB ln 

Another important connection is the free energy (Helmholtz), F(T,V,N)

F (T ,V , N )  kBT ln Z (T ,V , N )

We can connect thermodynamics to statistical mechanics using either of these relations, for large
enough systems these two relations are approximately equivalent.
Combining what we know so far we can find:

S  kB  P(m) ln P(m) This is the Gibbs definition of the entropy in open systems.
m

The Boltzmann-Gibbs definitions are equal in a microcanonical ensemble and equivalent for large
enough systems.

Laws of Thermodynamics


Zeroth Law: Equivalent to partition arguments, systems with equal values of ln ( E ) are all at
E
eqm with each other

First Law: In thermodynamics, energy is conserved and can be changed by heat or work. In statistical
mechanics, heat changes the occurrences of microstates and work changes the microstates
themselves.

Second Law: Total entropy change is greater than 0 for spontaneous processes. The only exception
to this is for reversible processes in which entropy is equal to 0. In statistical mechanics this means
that systems move towards macrostates with more accessible microstates.

Third Law (NON EXAMINABLE): In thermodynamics it says that as temperature tend to “0” the
entropy approaches a constant. In statistical mechanics this would mean Ω(E)=0 for E<E0 where E0 is
the ground state energy.
2. Some Examples and Practical Rules

What can we do with statistical mechanics?

For real systems we cannot usually count all the microstates or calculate the partition function of a
system exactly. Instead we consider “model systems” that we can do calculations with (e.g. an ideal
gas). We then compare our calculations with experiments and computer simulations to see if the
model system helps us to understand the real system.

2.1 Very Simple Model of a Rubber Band

We want to explain why rubber bands get stiffer on heating.

Take a chain of N links, each link points either left or right

Let the number of links pointing right be, N+, and pointing left, N-

Then it follows that L=a(N+-N-)

N!
For a given L, there are microstates (N+ being the links going right from N)
N ! N !

Entropy is then given by:

 N! 
S  k B ln   k B ln  
 N ! N ! 
N N  N  N 
 k B N   ln      ln    
 N  N  N  N 

The first law for the rubber band states that:

dU  TdS  FdL
dS
F  T
dL U

dS
To find we are required to use expressions for N+ and N-.
dL
It follows that:

1 L 1 L
N    N   and N    N  
2 a 2 a
kT  L   L 
F  B ln 1    ln 1  
2a   Na   Na  
For L  Na
k BT  L 
F  
a2  N 

This agrees to Hooke’s Law, It is an intensive force acting upon the band and the band gets stiffer
with increasing temperature (although this model is very simplified)

Simple Magnetic Model (Paramagnet)

If we put atoms in magnetic fields, then unpaired electrons like to align their spins with the field:

Up spins have lower energy than down spins and are more likely for B>0,  is the magnetic moment
of the electron.

Microstate: The state of each spin (total N spins)

Macrostate: The total energy or total magnetisation must be specified, E   B( Nd  Nu ) ,

M   ( Nu  N d )

Properties of the material can then be calculated by two routes, either by finding entropy and
proceeding as before or by calculation of the partition function.
Calculating the Partition Function

If N=1 (one atom) there are two states, up and down.

E B B
 
Z1  e
States
kT
e kT
e kT

For N atoms
 E1  E2  E3 ... EN 

ZN  
All States
e kT

E1 E2 EN
  
 
E1   B
e kT
 
E2   B
e kT
 ...  
EN   B
e kT

  Z1 
N

The rule when combining systems, if Etotal = E1 + E2 + … + EN is that ZN = Z1 x Z2 x … x ZN

Now to calculate the free energy:

F  k BT ln Z
 k BT ln( Z1  Z 2  ...  Z N )
N N
 k BT  ln Z i   Fi
i 1 i 1

For the paramagnet:

 ktB 
B

F   Nk BT ln  e  e kt 
 

The average energy of 1 atom is given by:

E
1  kTM
E1  
States Z
e EM
B B

 Be kt
 (  B)e kt
 B B

 e kt
e kt

 B 
   B tanh  
 kt 

For N atoms: U=N<E1> or Magnetisation M = N<M1>

As a result we find:

At low temperatures (or large B), all spins are up, entropy is zero and energy is minimised. However
for high temperatures (or B~0) spins are half up and half down, energy is large and entropy is
maximised. (NkBln2)

Quantum Particle in a Box

Box has a volume of V = L x L x L

Quantum mechanics says that the values of the energy are:

2
E (k ) 
2
k with m being the particle mass
2m

k=
L
 n , n , n  where n , n , n
x y z x y z are integers (non 0)

How many states have: kB  k  k0   k ?

Let each state be a point in “k-space”

kz comes out of the screen…

The points form a grid, we can use this to approximate the amount of points within kb of the origin:

4 k03 1 1
  
3   3 8
 
L
To find out the points between k0 and δk:

1 1 k02 L3
 k  4 k02    k
   8 2
3 2

 
L

For entropy we want to find the points with energy between E & E + δE

In General:

g (k ) k states between k & k   k

then
(E) E states between E & E   E
dk
( E )  g ( k )
dE

In this case:

2
k2 dE 2
k 1
E so  and k  2mE
2m dk 2
dk L3 m mL3
( E )  g ( k )  2 k2 2  2 2 k
dE 2 k 2
Substituting in k
3
L3  2m  2 1
(E)= 2  2  E 2
4  

Partition Function

Em E
 
Z1  e
states
kT
  dE(e)e kT

let x 2  E
3
 mk T  2
Z1   B 2  L3
 2 

Free Energy

3  2 2 
F  kbT ln Z  k BT ln  2 
2  mkBTL 

Average Energy

 3
 E  kBT 2 ln Z  kBT
T 2
Pressure

dF
p  k BT
dV T

A Note about Degeneracy

If more than one microstate in a system has the same energy we say that the energy level is
degenerate, in fact if there are g microstates then the level has a degeneracy, g. In general we can
check with:

Ei

Z 
allowed
gi e kT
where gi is degeneracy of level i
energies
Ei

Also for electrons, microstates come in pairs, the result of this is that the density of states gets
multiplied by 2 as long as the up and down spin states have the same energy.
3. Ideal Quantum Systems with More Than One Particle

Realistic systems (solid, liquid, gas etc.) consist of many quantum particles (atoms, molecules etc.).
We will consider some examples of such systems, showing how to count microstates and how we
may assume a classical description/

3.1 Identical Particles Compared to Distinguishable Particles

Assume, 1, All particles are identical, 2, the energy of the system is the sum of the energies of each
individual particle. To solve the problem, solve the Schrödinger equation for just one particle, find
the energy levels, Ei, and their degeneracies, gi.

For many (N) particles we define the microstates of the N-particle system in terms of the 1 particle
system. For distinguishable particles, a microstate, M, might be: particle 1 in state m1, particle 2 in
state m2 … particle N in state mN.

mi refers to the microstate of the 1 particle system.

But for identical/indistinguishable particles a microstate, M, must be defined as (e.g.): state m1 has 1
particle, state m2 has no particles etc. defined for all m states.

When we apply PEAPP, there are two ways of defining the microstates that lead to different
conclusions. This means we MUST choose the correct definition for the situation (in the majority of
cases this will be the identical particle definition.)

3.2 Fermions and Bosons

Quantum particles are either fermions or bosons. For fermions, each state, m, contains at most one
particle (due to the Pauli Exclusion Principle). For bosons, each state, m, may contain any number of
particles. Usually in these systems it is convenient to use the grand canonical state.

If M is a microstate of the many particle system with NM particles and total energy EM then the
probability of finding this microstate is:

1   EM   NM 
1

P ( M )  e k BT

where    
1
kBT
 EM   NM 
M

Note: NM  n m and EM   nm Em with nm being the number of particles in state m.

3.3 Fermionic Systems

Consider just one state, m, of the 1-particle system, in the N-particle system, state m may contain 0
or 1 fermions.

For this state:

1
  Em   
m  1  e k BT

The probability that state m contains 1 fermion is then:

1
1   Em   
1   Em    e K BT
1
P(nm  1)  e K BT
  1
 
1
 Em    
1
 Em   
1 e k BT
e k BT

 nFD ( Em )

NFD(Em) is called the Fermi-Dirac distribution function – arranging n fermions in a non-degenerate

energy level with energy E.

Note: Fermions have ‘spin’, often the up and down states have the same energy in which case g = 2,
two fermions may have the same energy if they have different spins, or if there is some other
degeneracy in the system.

Luckily for ideal systems, the grand partition function is the same as the product of all the
microstates of the 1-particle system. This means that the state occupancies are given by nFD(E) even
if we have many microstates of the 1-particle system. For ideal systems each state can be treated
separately.
3.4 Bosonic States

If state m contains n bosons then:

1
  n Em   
m   e k BT

n 0

This formula is a geometric progression (See any C1 book from A-level maths):

1
 1
  Em   
1 e k BT

The average number of particles in state m is:

n
1   Em   

 n
ne k BT

Notice that this is the same as:


kBT log 


Taking the derivative:

1
 n  1
 nBE ( E )
( Em   )
e k BT
1

With nBE(E) being the Bose-Einstein distribution function, giving the average number of particles in a
bosonic energy level.
3.5 Combining nx(E) with Density of State (E)

Instead of one state m, consider many states, with density of state (E)

We have the average number of particles, and average energy in the whole system given by:

 N   dE( E )nx ( E ) ,  E   dE( E ) Enx ( E )

There is an exception to this, bosons at low temperature.

In section 2.3 we showed that for an ideal gas:

3
V  2m  2 12
( E )  2  2  E (neglecting spin, if it’s an electron in a box then a factor of 2 is introduced)
4  

So we can plug this into the formulae for <N> and <E> to get the number of particles as a function of
(m, V, T, μ) and hence average the energy as a function of (m, V, T, μ or density ρ)

3.6 Examples of gases of Fermion and Bosons

1. The Classical Limit: If Em – μ >> kBT then:

Em  

nx ( E )  e k BT
for both fermions and bosons, this is known as the classical limit.

N
For an ideal gas this happens when the density,    nQ
V

nQ is the quantum concentration:

3
 mk T  2
nQ   B 2 
 2 

The estimate of the quantum uncertainty in particles positions, λ, can be written as:

1
 2 2  2
  
 mk BT 

In the classical limit we can easily work with a fixed number of particles, N, but for identical
(indistinguishable) particles the partition for N particles is:

1
ZN   Z1 
N

Where Z1 = nQV, the partition δn of 1-particle in a box of volume V.

2. Blackbody radiation (an ideal gas of photons)

Consider photons in a box, the microstates are the same as those of a quantum particle in a box.

States are labelled by a wave-vector:


k (nx , ny , nz )
L

The number of states with k between k0 and k0 + δk is g(k)δk, with:

k 2 L3
g (k ) 
2n 2

For photons, E   and   ck , We want the density of states:

dk g (k )
( E )  g (k ) 
dE c

Each microstate has degeneracy, Z1, to take care of polarisation so:

E 2 L3
( E )  , this is known as the photon density of states
n 2 ( c )3

Photons are bosons. They can be created and destroyed spontaneously, this means that μ = 0

So the total energy of the system is:


 E   dE( E ) EnBE ( E )
0

E 
dE  d 
 E  L3  d U ( )
3
U ( ) 
 k T 
 c  e B  1
2 3
 
 

This is the Planck Distribution

3.S Summary

1. Take care counting microstates if you have identical quantum particles, remember that
“particle A in state a, particle B in state b” is not a different state to “particle B in state a,
particle A in state b”
2. The average number of particles in an energy level with energy E is:
1 1
nFD ( E )  E 
for fermions and nBE ( E )  E 
for bosons.
1 e k BT
1 e k BT

E 

3. The classical limit at high temperatures: nFD ( E )  nBE ( E )  e k BT
 1
N
The equivalent statement for an ideal gas is the number density:    nQ
V
3
 mk T  2
nQ   B 2 
 2 
Another equivalent statement is that the quantum uncertainty in particle positions <<
particle spacing.
4. Statistical Mechanics in Practice

For the purposes of this course, there are three important kinds of statistical mechanics calculation
you need to be able to do. We will discuss these three methods in this section.

4.1 Microcanonical

Route 1: When you know the value of the energy of a system, but not it’s temperature the usual
method is:

1. Calculate the number of microstates that are available (Ω(E)) Where E is the energy.
2. Calculate the entropy through S  kB log 
3. Use derivatives to calculate physical quantities (see below)

**DIAGRAM**

PEAPP states all states with energy E are equally likely in a closed system.

Examples (PS1)

The route can be modified. In PS1-Q4 we counted the number of states for a crystal with m
vacancies. In the second section, we considered a number of states of a rubber band with length L.
Generally we calculate entropy by counting states for these types of questions.

Derivatives of the Entropy

For a system of N particles in a volume V, with energy E, the number of states Ω and the entropy S
depend on (E, V, N). As discussed in the previous sections, we can obtain the temperature T,
pressure p and chemical potential μ using:

1  S   S   S 
  , p T   ,   T  
T  E V , N  V  E , N  N  E ,V

So if you know Ω as a function of (E, V, N) you can calculate (T, p, μ) by taking derivatives of S.

You can remember the above formulae from:

dU  TdS  pdV   dN

1 p 
dS  dU  dV  dN
T T N
4.2 Boltzmann Distributions

Route 2: When you know the temperature of a system but not its energy you can calculate the
probability of finding the system in a particular microstate.

If the energy of the microstate is Em then this probability is:

Em Em
1  
P(m)  e kBT
Z
where Z  
All States m
e k BT

If there are gi microstates with energy Ei then it follows that the probability that the system has
energy Ei is:

Ei Ei
1  
P( Ei )  gi e kBT
Z
where Z  
All Energies Ei
gi e k BT

Example (See PS2-Q2)

Calculate P(m2)/P(m1) where m2 state has energy E2 and m1 has E1.

E2
1  k BT E1  E2
e
P(m2 ) Z
 E1
 e kBT ( 0.33 for PS2-Q2)
P(m1 ) 1  k T
e B
Z

In this case there are two states with energy E1 and eight states with energy E2. If m1 is one of the
states with energy E1 and m2 one of the states with E2 then P(m2)/P(m1)=0.33, but the probability of
having energy P(E2) = 8P(m2) So:

E2
8  k BT E1  E2
e
P( E2 ) Z
 E1
 4e kBT  1.2
P( E1 ) 2  k T
e B
Z
4.3 Free Energies

Route 3: When you know the temperature of a system but not its energy, you can calculate the free
energy from the partition function Z. This lets you derive the macroscopic quantities as in route one,
the method is:

1. Write down the partition function as a sum

2. Calculate the sum and then use F  kBT log Z to calculate the free energy
3. Use derivatives of F or log Z to calculate pressure, energy, etc. (see below)

Examples

PS2-Q4 and PS3-Q1, also paramagnet, and particle in a box examples from previous sections.

Derivatives of F and log Z

WE use derivatives of F in a similar way to the derivatives of S.

For a system of N particles in a volume V, at temperature T, the partition function Z and the free
energy F depend on (N, V, T). As discussed in the lectures, we obtain the entropy S, pressure p and
chemical potential μ using:

 F   F   F 
S    , p    ,   
 T V , N  V T , N  N V ,T
Also
  log Z 
U  E  k BT 2  
 T  N ,V

By knowing Z we can calculate S, and thus all the other physical quantities
4.S Other Helpful Hints

In addition to familiarity with the types of calculation given here, the most important features of
statistical mechanics that come up are:

 Definitions: microstates, macrostates, PEAPP, density of states.

 How to count microstates in systems with many identical (indistinguishable) particles
 Physical interpretations of Fermi-Dirac and Bose-Einstein distribution functions nFD(E) and
nBE(E).
 Properties of ideal gases of classical particles, fermions, photons and other bosons.

Extra Information on Counting Microstates of Identical Particles (PS4-Q2)

E0 E1
 
1 Particle: Z  e k BT
 2e kBT

E0  E1 2 E1
 
2 Identical Fermions (same spin): Z  2e e
k BT k BT

Probability of being in state with 1 fermion in E1 and 1 in E2:

E0  E1

k BT
e
P ( m) 
Z
 Ea  Eb

2 Identical Bosons: Z  
All States
e k BT