16 September 2013

Degrees of Freedom

How many degrees of freedom do you have in your life?  Well, if you can’t answer that, can you say how many constraints you’re subject to?  The number of constraints would tell you how many degrees of freedom you subtract from the original or unconstrained state you started off in.  But freedom versus constraint is a difficult thing to get a hold of in the realm of human activity, so let’s go to the realm of physics.
 
I first encountered degrees of freedom and constraints in a two-semester junior-level Newtonian mechanics course I took at Hendrix College, in the 1978-79 academic year.  Richard Rolleigh was the professor, and a good one.  I was not that good of a student, except in the first three months of the course.  The textbook is a pretty good one:  Mechanics, by Keith R. Symon.  Chapter 9 in the book discusses generalized coordinates, Lagrange’s equations, systems subject to constraints, constants of the motion and ignorable coordinates, Hamilton’s equations, and Liouville’s theorem. 
 
In section 9.4, Symon says this about degrees of freedom:  “The number of independent ways in which a mechanical system can move without violating any constraints which may be imposed is called the number of degrees of freedom of the system.    For example, a single particle moving in space has three degrees of freedom, but if it is constrained to move along a certain curve, it only has one.  A system of N free particles has 3N degrees of freedom, a rigid body has 6 degrees of freedom (3 translational and 3 rotational), and a rigid body constrained to rotate about an axis has one degree of freedom.”
 
 
The concepts of constraints and degrees of freedom are also used in statistics.  Let’s see what professors Box, Hunter, and Hunter (BH&H) have to say on the subject.  First jog your memory and recall that each deviation for a sample is the difference in an observational value y and the sample average ŷ. (This symbol is ‘y-hat’, which in physics is used to indicate the  unit vector in the y direction, but the symbol set on my computer doesn’t seem to have ‘y-bar’, so I’ll use ŷ instead.)  The deviation for a population is the difference in y and the mean η.   That is, in the case of a sample average, the deviation is yŷ, and in the case of the population mean, the deviation is y η.  Add up the deviations for a given sample or a given population and what do you get?  That’s right, zero.
 
Here’s the BH&H paragraph that discusses why n-1 is used for calculating the sample standard deviation, but first they have to define the variance:
 
“The deviations of n observations from their sample average must sum to zero.  This requirement, that Σ(y-ŷ) = 0, constitutes a linear constraint on the deviations or residuals y1 – ŷ, y2 – ŷ, …, yn – ŷ used for calculating s2 = Σ(y- ŷ)2/(n-1).  It implies that any n-1 of them completely determine the other [one].   The n residuals y- ŷ [and hence their sum of squares Σ(y- ŷ)2 and the sample variance, Σ(y- ŷ)2/(n-1)] are therefore said to have n-1 degrees of freedom.  In this book the number of degrees of freedom is denoted by the Greek letter ν (nu).”
 
Allright!  So in the People Aren’t Perfect lab, we have ν = n – 1 = 10 – 1 = 9 degrees of freedom.  BH&H say this:  “The loss of one degree of freedom is associated with the need to replace the unknown population mean η by ŷ, the sample average derived from the data.” 
 
Yep, and as you just noticed, there’s this thing called the variance, which the standard deviation comes from by taking the square root.  The population variance is σ2 = Σ(y-η)2/N,  and its standard deviation is σ =[Σ(y-η)2/N]1/2.  The sample variance is s2 = Σ(y-ŷ)2/(n-1) and its standard deviation is  s = [Σ(y-ŷ)2/(n-1)]1/2.

Would you believe this is the same expression as the one at the bottom of page 1 of the People Are Not Perfect* lab handout?  Different symbology is the only difference.

*Neither are machines!  [But what about equations?]