|
The Trouble With Surveys
A New Reason to Question the Official Unemployment Rate
David Leonhardt, NY Times, August 26, 2014 http://www.nytimes.com/2014/08/26/upshot/a-new-reason-to-question-the-official-unemployment-rate.html
...The [unemployment] report tries to estimate employment in a big
country – and to do so quickly, to give policy makers, business
executives and everyone else a sense of how the economy is performing.
It’s a tough task.
And it has become tougher, because Americans are less willing
to respond to surveys than they used to be.
A
new academic paper suggests that the unemployment rate appears
to have become less accurate over the last two decades, in part
because of this rise in nonresponse. In particular, there seems
to have been an increase in the number of people who once would
have qualified as officially unemployed and today are considered
out of the labor force, neither working nor looking for work.
The trend obviously matters for its own sake: It suggests that
the official unemployment rate – 6.2 percent in July –
understates the extent of economic pain in the country today.
That makes intuitive sense. Wage growth is weak, and Americans
are pretty dissatisfied with the economy, according to other surveys.
The new paper is a reminder that the unemployment rate deserves
less attention than it often receives.
Yet the research also relates to a larger phenomenon. The declining
response rate to surveys of almost all kinds is among the biggest
problems in the social sciences. It’s complicating our ability
to understand how people live and what they believe. “It’s
a huge issue,” says Alan Krueger, a Princeton economist
and one of the new paper’s three authors. (Mr. Krueger,
who recently spent two years as the chairman of President Obama’s
Council of Economic Advisers, founded the Princeton University
Survey Research Center in the 1990s.)
Why are people less willing to respond? The rise of caller ID
and the decline of landlines play a role. But they’re not
the only reasons. Americans’ trust in institutions –
including government, the media, churches, banks, labor unions
and schools – has fallen in recent decades. People seem
more dubious of a survey’s purpose and more worried about
intrusions into their privacy than in the past.
“People are skeptical – Is this a real survey? What
they are asking me?” Francis Horvath, of the Labor Department,
says.
In 1997, the response rate to a typical telephone poll was a
healthy 36 percent, according to Pew. By 2012, it had fallen to
9 percent. Fortunately, many surveys appear to be doing a good
job of weighting the answers of people who do respond, to make
up for those who don’t. Still, the long-term reasons for
concern are clear: People who are more likely to avoid polls,
such as anyone born after, say, 1980, are different from those
who answer them.
The response rate of the Labor Department’s monthly jobs
survey is far higher (about 89 percent) than that of a political
poll, but it has also fallen (from 96 percent in the 1980s). Not
surprisingly, the people who do not respond have different experiences
in the job market than those who do.
The trouble with the unemployment rate revolves around a technical
concept known as “rotation-group bias,” explain the
paper’s three authors, Mr. Krueger, Alexandre Mas and Xiaotong
Niu. The government surveys people for four consecutive months,
gives them eight months off and then surveys them for four more
months. This pattern allows the Labor Department to track people’s
experiences for more than a year in a way that is less burdensome
than 16 months of monthly surveys would be.
Over time, the kinds of answers that people give — or the
kinds of people who respond — change. In later months as
part of the survey panel, people who aren’t working are
less likely to report being available to work and having looked
for a job in the previous four weeks, which is the definition
for unemployment. The differences are big, too.
The unemployment rate in the first half of 2014 among people
in the first month of being interviewed was 7.5 percent. Among
people in the final month of being interviewed, it was only 6.1
percent. Because the Labor Department weights later panelists
– for whom there is historical data – more heavily,
the official unemployment rate during this period was 6.5 percent.
That number seems too low. The authors note that the higher jobless
rate among early-month panelists correlates more strongly with
some other economic indicators than the rate among later-month
panelists.
If you’re tempted to blame President Obama for this situation,
...you should dig into the data. The problem has existed, and
been growing, for decades. A redesign
of the survey in 1994, to move it from paper-based to computer-based,
seems to be one cause. The full reasons aren’t clear, but
something about the redesign seems to have changed the way people
answer questions. The gap in jobless rates between early and later
panelists starting spiking in 1994 and is now about twice as large
as it was then.
Of course, survey response rates have also been dropping over
that same period. And there may indeed be an Obama effect here.
The response rate to the unemployment survey fell noticeably in
2009, shortly after Mr. Obama took office, when the Tea Party
was forming. Since then, it has continued falling more steeply
than pre-2009. In the Obama era, some Americans simply trust the
government less.
Tea Party aside, the main factor is technology. It’s a
major cause of today’s response-rate problems – but
it’s also the solution.
For decades, survey research has revolved around the telephone,
and it’s worked very well. But Americans’ relationship
with their phones has radically changed. It’s no surprise
that survey research will have to as well.
Maybe people in coming years will answer questions about their
employment status (and political views) by text message on their
iPhones – or through Google glasses. Or maybe they’ll
do so on their televisions, when they once would have been watching
commercials.
In the future, we are unlikely to live in a country in which
information is scant. We are certain to live in one in which information
is collected in different ways. The transition is under way, and
the federal government is among those institutions that will need
to adapt.
See also from 2006 "Study Finds that Labor Department Overstates Share of Working Americans By 1.4 Percentage Points"
|
