In this post, I’m going to look at the methodology that produces our monthly employment reports because with today’s report, it’s particularly timely. As mentioned in the first post, the employment situation report for January 2011 is a real muddle, full of apparent contradiction. The obvious contradiction is 36,000 new jobs (an incredibly weak number) vs. a 0.4 point drop in unemployment rate from 9.4 to 9.0% (an encouraging number if we can believe it).  But once you read the full report, there’s even more confusion to be had. The report says that only 36,000 jobs were created but it also elsewhere says 117,000 more people were employed. Huh? Well it turns out there’s reasons. First, words matter – a job is not the same thing as an employed person (people can have two jobs). Second, while all the data is published each month as a single “employment situation report”, in fact the data being reported are estimates, not exact counts. This is not a monthly employment census, it’s a survey-based estimate/analysis. And the estimates depend crucially on five inputs:

1. A sample survey of households (people) done by BLS.
2. A sample survey of businesses that employ people, also done by BLS.
3. Monthly estimates of the demographics and population of the nation provided by the Census Bureau.
4. Econometric models that the BLS uses that forecasts the number and size of new businesses created each month and the number that failed. A model is used since reliable actual data isn’t available for a long time after the month it happens.
5. The seasonal adjustment method.  In trying to seasonally adjust the data, this month’s SA data is in effect dependent upon the seasonal pattern exhibited in the past. If this year is different from the past, a temporary distortion enters.

This month we’ve got a near perfect storm of noise and confusion coming from at least 4 of these inputs.

First, the two surveys sharply disagreed. This could be a fluke occurence, an outlier of sampling, or it could indicate beginnings of a trend such as the number of jobs held person changing (normally stable), or it could be..who knows?  Garth Brazelton at Reviving Economics tries to explain through his exasperation:

…Typically always true from November – January imop.
But this one is particularly f***ed up.

News is reporting that according to BLS only 36,000 jobs were added in January (about 1/4 of what was expected).

Meanwhile, BLS reports the unemployment rate falls by another 0.4% – 2 months in a row – to 9.0%. …

In any case, it is important to note that BLS reports 2 surveys:
The first is the household survey (which surveys people, duh) – that is where we get the unemployment rate.
The other survey is for businesses and the government – that is where news organizations usually report jobs gained or lost.  (36,000 more employed in Jan. compared to Dec. 2010)

The unemployment rate simply cannot be compared to establishment based employment payroll changes because the two numbers come from 2 completely different surveys, measuring slightly different things:

Reported change in civilian employment from Dec. 2010 to Jan. 2011 (private): 117,000  (HOUSEHOLD SURVEY)

Reported change in total private non-farm employment from Dec. 2010 to Jan. 2011: 36,000  (ESTABLISHMENT SURVEY)

The unemployment rate comes from the survey of the former statistic, not the latter.  Media does everyone a dis-service by not pointing this out.

So, you know what you should do with all these news reports and the January statistics?  Throw them in the garbage, and wait for February and March.

Unlike Garth, I don’t think we need to throw the report in the garbage (see my post here), but we need to think critically about it.  Calculated Risk explains:

This data comes from two separate surveys. The unemployment rate comes from the Current Population Survey (CPS: commonly called the household survey), a monthly survey of about 60,000 households. The payroll jobs number comes from Current Employment Statistics (CES: establishment survey), a sample of approximately “140,000 businesses and government agencies representing approximately 410,000 worksites”.

These are very different surveys: the CPS gives the total number of employed (and unemployed including the alternative measures), and the CES gives the total number of positions (excluding some categories like the self-employed, and a person working two jobs counts as two positions).

A couple of key concepts (from the BLS):

The CES employment series are estimates of nonfarm wage and salary jobs, not an estimate of employed persons; an individual with two jobs is counted twice by the payroll survey. The CES employment series excludes employees in agriculture and private households and the self-employed.

And the CPS:

The CPS estimate of employment is for the total number of employed persons. Included are categories of workers that are not covered by the Current Employment Statistics (CES) survey: self-employed persons, private household workers, agriculture workers, unpaid family workers, and workers on leave without pay during the reference period. Multiple jobholders are counted once in the estimate of total employed.

Unemployed persons include those who did not have a job during the reference week, had actively looked for work in the prior 4 weeks, and were available for work. Actively looking for work includes activities such as contacting a possible employer, contacting an employment agency or employment center, having a job interview, sending out resumes, filling out job applications, placing or answering job advertisements, and checking union or professional registers.

In January the headline CES number showed a gain of 36,000 non-farm jobs (by the definitions above). The CPS showed an increase of 117,000 employed people.

These two surveys are almost always different, and both are useful for understanding the employment situation. 

Second, another problem confounding things this month is what’s called the “benchmark adjustment”.  Put very simply, the survey data is used to establish some proportions and ratios among those sampled. Then these ratios and proportions are scaled up to national numbers by multiplying by the latest population estimates from the Census bureau. But the Census bureau monthly estimates of population and demographics are themselves estimates that get revised yearly and of course, definitively every ten years. So once each year, the BLS has to adjust it’s data series on employment and jobs to keep it in line with the annual revision of population from Census. This year, these “benchmark adjustments” were very large (around 500,000).  Again I turn to Calculated Risk:

On Feb 4th, with the release of the January employment report, the BLS will make the following three changes / revisions:

1) Annual Benchmark revision to the Establishment Survey Data

With the release of January 2011 data on February 4, 2011, the Current Employment Statistics survey will introduce revisions to nonfarm payroll employment, hours, and earnings data to reflect the annual benchmark adjustments for March 2010 and updated seasonal adjustment factors. Not seasonally adjusted data beginning with April 2009 and seasonally adjusted data beginning with January 2006 are subject to revision.

Last October the BLS released the preliminary annual benchmark revision of minus 366,000 payroll jobs. Usually the preliminary estimate is pretty close to the final benchmark estimate. 

Finally, we have the issue of seasonality and winter. The data series on employment and labor force must be seasonally adjusted to be meaningful and informative since there’s a huge month-to-month swing in jobs that’s very predictable: summer hiring for kids, Christmas season hiring, July shut-downs in some industries, etc. The raw, non-seasonally adjusted data (NSA) just changes too wildly each month to be easily interpreted. So seasonal adjustment (SA) is necessary. But SA can distort numbers if this year’s seasonal change happens at a slightly different time than it normally has in the past.  This year, Christmas and New Years were on a weekend. More importantly, much of the nation (fortunately not Southeast Michigan, IMO) was wracked by snow and winter storms that were out-of-normal. So the severe winter and snow may have distorted the numbers. After all, snow doesn’t just keep kids home from school. It also keeps job hunters from being active. It keeps people out of stores and causes retailers to temporarily lay-off workers (i.e. tell’em not to come to work). It keeps businesses from hiring (temporarily) since you can’t hire somebody when both interviewer and interviewee are stuck in a snow drift. Old man winter could have had an impact on the numbers, but we have no certain way of knowing. This is particularly true since the surveys do not happen during the entire month. Instead the two surveys (CPS and CES) are done during a few particular days during mid-month. If that’s during a blizzard, well, you can guess the impact. Right now a lot of commentators (see  are willing to write-off the bad numbers in this report to winter snow. They assume the numbers will bounce back next month.  Maybe. Maybe not. We’ll see.  Meanwhile, let’s see what conclusions we can take away from report in the next post.