Tag Archives: employment

Credibility, statistics, and the legal profession

Posted on by

Elie Mystal at Above the Law has this take on a recent story involving credibility, statistics, and the legal profession:

This week, the law schools at Columbia, NYU, and Fordham have come under fire for their allegedly inflated employment statistics. A story in the New York Post specifically called out the top New York-area law schools for shady reporting of graduate outcomes when it comes to graduates employed by the schools….

I want to take a step back and look at what we’re really fighting about here: some of the best law schools in New York City have put out a statistic about how many graduates get jobs, and the New York Post and a bunch of other people immediately called “bulls**t.” Think about that. Even if the law schools can somehow convince people that, technically, their published information isn’t riddled with lies, we’re living in a world where such data can be assumed to be false absent a long and detailed explanation and discussion from the law schools. When somebody notices a discrepancy between a school’s numbers and what’s in the newspaper, we assume the school was full of crap, not that the newspaper got it wrong.

I suppose this isn’t very flattering to either newspapers or law schools.  Perhaps Americans now trust journalists more than lawyers (or at least legal educators)?

Commenting a few months ago on a scandal within academic sociology, Brian suggested several approaches to dealing with uncertain statistics:

This reminds me of Joel Best’s recommendations regarding dealing with statistics. One common option is to simply trust all statistics. Numbers look authoritative, often come from experts, and they can be overwhelming. Just accepting them can be easy. At the other pole is the common option of saying that all statistics are simply interpretation and are manipulated so we can’t trust any of them. No numbers are trustworthy. Neither approaches are good options but they are relatively easy options. The better route to go when dealing with scientific studies is to have the basic skills necessary to understand whether they are good studies or not and how the process of science works [emphasis added].

Brian’s point is a good one.  Unfortunately, it’s not possible to implement his “third way” here because the root problem is the lack of raw information rather than the inability to duplicate experimental/study results.  The question is not, in the theoretical abstract:  how many law students will get jobs when the economy is in condition X?  The question is rather:  as a matter of historical fact, how many 2010 law school graduates (or 2011, or 2009, or whatever) actually had jobs by date Y?

Elie faults the American Bar Association, which oversees and accredits law schools, for the current disaster of unreliable data:

The ABA is supposed to represent lawyers and law schools to the public. It’s supposed to relegate them so that the public can trust that moral and ethical standards are being upheld and enforced. And on that scale, the ABA has been an unmitigated failure. It’s done a disservice to all law schools. Nobody can trust any law school because the ABA has failed to impose effective oversight over all of them.

That’s tragic. A society is supposed to be proud of its institutions of higher learning, but the ABA has robbed us of that pride in our nation’s law schools. We no longer get to feel like our justice system is populated by people trained to the highest ethical standards, because we can’t even trust our law schools to tell us the truth about how many people got hired.

If the numbers published by law schools under the oversight of the ABA are unreliable, it goes without saying that it’s very difficult to derive these numbers through other means, especially in a form that allows for legitimate comparisons between schools and over time.  There are workarounds, of course, like journalistic attempts to compile and/or verify employment statistics independently of the law schools.  But those are obviously imperfect solutions, as The Economist recently noted in its analysis of the (surprisingly analogous) problem of Argentinan inflation statistics:

Since 2007 Argentina’s government has published inflation figures that almost nobody believes. These show prices as having risen by between 5% and 11% a year. Independent economists, provincial statistical offices and surveys of inflation expectations have all put the rate at more than double the official number (see article). The government has often granted unions pay rises of that order….

We [The Economist] hope that we can soon revert to an official consumer-price index for Argentina. That would require INDEC to be run by independent statisticians working unhindered. Until then, readers are better served by a credible unofficial figure than a bogus official one.

Unfortunately, for the foreseeable future, I think we’re going to need to start seeing a lot more credible unofficial figures out there, both for Argentinian inflation and for law school placement statistics.

How jobless Americans are spending their time

Posted on by

Some new research suggests that unemployed Americans are doing a variety of things:

One study last year found that much of the extra time gets spent sleeping and watching TV–leading to news reports that the jobless “frittered away” their time. Another analysis–this one released in January and co-written by Princeton economist Alan Krueger, who was announced Monday as the White House’s pick to serve as the chief economic adviser to President Obama–pointed in the same direction. It found that people tend to devote fewer hours to job searches the longer they’ve been unemployed, and that sleep–especially “sleep in the morning hours”–increases as joblessness goes on. Together, the studies appeared to create a picture of the unemployed as lazy and unproductive.But a sophisticated new analysis (pdf) complicates that picture. In a paper written for the National Bureau of Economic Research, Mark A. Aguiar, Erik Hurst, and Loukas Karabarbounis, using data from the American Time Use Survey, found that the jobless do spend about 30 percent of their extra time–the time they would otherwise have spent working–sleeping or watching TV, and another 20 percent on other leisure activities. But around 35 percent is spent doing unpaid but nonetheless important work, like child-care and housework. And other investments–things like education, health-care, and volunteer work –account for another 10 percent.

The notion advanced by some that jobless benefits are being used to support a life of leisure is, at best, simplistic.

But as Nancy Folbre, an economics professor at the University of Massachusetts, Amherst, notes, there’s a limit to how much useful unpaid work the jobless can do. “They lack the capital, land, tools and skills needed to flexibly shift from wage employment to production for their own use.,” she writes. “Even when they can make a partial shift, their productivity is likely to be lower in unpaid work than paid work.”

I’m a little surprised by the quote from an economist at the end: unpaid work still needs to be done by someone whether they currently have the skills for it or not. Perhaps she is referring to longer-term issues: do the unemployed go back to work (perhaps by changing fields or getting educated in new areas) or do they adjust to a life of unpaid work? In the meantime, there is a transition that has to be made. But I can imagine that some people would see this quote and wonder what this means for people who have always done unpaid work, particularly mothers.

Another way to interpret the earlier study that the unemployed enjoy a life of leisure is that this is due to feelings of restlessness and perhaps even depression.

In general, I find time use studies to be quite interesting. When you ask people general questions about how they spend their time, like how long they spend at work, the numbers can be quite inflated. The better studies require logs or diaries and ask questions about recent time periods where memories will not be as distorted. Here is how the American Time Use Study describes some of its methodology (starting at page 11 of this document):

The ATUS sample is randomized by day, with 50 percent of the sample reporting about
weekdays, Monday through Friday, and 50 percent reporting about Saturday and
Sunday. Designated persons must report about their activities on their designated day,
without any substitution of days…

The ATUS interview is a combination of structured questions and conversational
interviewing. It consists of four major topics: the household roster, the time diary, the
summary questions, and a section related to information collected in the eighth CPS
interview. The portion of the interview relating to the CPS is divided into four sections:
labor force status, looking for work, industry and occupation, and earnings and school
enrollment. These questions are used to update or confirm time-sensitive CPS data or
to fill in missing CPS data. Each section is described below in more detail…

For all parts of the interview except the collection of the time-use diary data (in
section 4, above), interviewers read scripted text on the CATI screen and enter the
reported responses.

For the time-use diary, the interviewer uses conversational interviewing rather than
asking scripted questions. This is a more flexible interviewing technique designed to
allow the respondent to report on his or her activities comfortably and accurately. This
technique also allows interviewers to use methods to guide respondents through memory lapses, to probe in a nonleading way for the level of detail required to code activities, and to redirect respondents who are providing unnecessary information. As each activity is reported, the interviewer records the verbatim responses on a new activity line. The interviewers are trained to ensure that the respondent reports
activities (and activity durations) actually done on the previous (diary) day, not
activities done on a “usual” day. Interviewers do this by placing continual emphasis on
the word “yesterday” throughout the interview.

This study relies on both a diary and asking questions about yesterday.

500 to 1

Posted on by

I contemplated the effects of technological changes on law jobs several weeks ago when I posted a link to news reports about IBM’s Watson winning Jeopardy.  The New York Times has written what essentially amounts to a follow-up article, and it’s eye opening:

Quantifying the employment impact of these new technologies [that help automate the legal discovery process] is difficult. Mike Lynch, the founder of Autonomy, is convinced that “legal is a sector that will likely employ fewer, not more, people in the U.S. in the future.” He estimated that the shift from manual document discovery to e-discovery would lead to a manpower reduction in which one lawyer would suffice for work that once required 500 and that the newest generation of software, which can detect duplicates and find clusters of important documents on a particular topic, could cut the head count by another 50 percent. [emphasis added]

To be sure, 500:1 may just be the talking point of a businessman who is trying to sell his particular solution. Nonetheless, it seems clear that technology like Mr. Lynch’s is already fundamentally altering the economics of the legal profession.  We probably are headed towards a future with fewer lawyers (at least, ones performing discovery-related tasks).

What are some of the broader economic implications?  The NYTimes piece also quotes from  David H. Autor, an economics professor at the Massachusetts Institute of Technology:

“There is no reason to think that technology creates unemployment,” Professor Autor said. “Over the long run we find things for people to do. The harder question is, does changing technology always lead to better jobs? The answer is no.”

Law jobs in Jeopardy

Posted on by

There’s been a lot of talk this week about Watson’s appearance on Jeopardy! — and its win.  Now, the pundits are trying to digest what the implications will be now that Watson has already been hired as a physician’s assistant.

What, specifically, does this mean for lawyers?  Robert C. Weber, a senior VP and general counsel at IBM, breaks it down for us over at over at the National Law Journal:

Imagine a new kind of legal research system that can gather much of the information you need to do your job — a digital associate, if you will. With the technology underlying Watson, called Deep QA, you could have a vast, self-contained database loaded with all of the internal and external information related to your daily tasks, whether you’re preparing for litigation, protecting intellectual property, writing contracts or negotiating an acquisition. Pose a question and, in milliseconds, Deep QA can analyze hundreds of millions of pages of content and mine them for facts and conclusions — in about the time it takes to answer a question on a quiz show.

But won’t this mean fewer jobs for lawyers?  Oh no, reassures Mr. Weber:

Deep QA won’t ever replace attorneys; after all, the essence of good lawyering is mature and sound reasoning, and there’s simply no way a machine can match the knowledge and ability to reason of a smart, well-educated and deeply experienced human being. But the technology can unquestionably extend our capabilities and help us perform better.

Humanity — I mean — lawyers win, huh?  This is great!  Where can I put Watson to work?

The technology might even come in handy, near real-time, in the courtroom. If a witness says something that doesn’t seem credible, you can have an associate check it for accuracy on the spot.

Wait a minute — I thought you said that we’ll always need lawyers?  But if using Watson/Deep QA is just as easy as running a Google search against a witness on the stand, why do you need to have an associate perform it?  Associates are expensive, or, at least, used to be.  Why not a paralegal?  Why not someone even cheaper, with even less training?  Are you sure it has to be an actual lawyer?  (Besides, Weber also tells us that “We’re pretty sure [Watson] would do quite well in a multistate bar exam!”)

Perhaps when he said Watson “won’t ever replace attorneys,” Mr. Weber meant that Watson won’t ever replace someone like himself:  a successful, established, general counsel at a Fortune 500.  You know, the sort of person who passes off his “research” to an “associate.”  Or whomever.  Or whatever.

I’m not buying it, Weber (neither is Above the Law, for whatever that’s worth).  Watson is going to put a lot of lawyers out on the street, which is precisely the conclusion that Andy Kessler comes to over at the Wall Street Journal.  In Kessler’s colorful employment taxonomy, lawyers are classified as “sponges”:

Sponges are those who earned their jobs by passing a test meant to limit supply. According to [the WSJ], 23% of U.S. workers now need a state license….All this does is legally bar others from doing the same job, so existing workers can charge more and sponge off the rest of us.

But eDiscovery is the hottest thing right now in corporate legal departments. The software scans documents and looks for important keywords and phrases, displacing lawyers and paralegals who charge hundreds of dollars per hour to read the often millions of litigation documents. Lawyers, understandably, hate eDiscovery.

We can argue whether this is a good for society overall (or not).  But come on, Weber.  Don’t say that Watson “won’t ever replace attorneys” when what you really mean is that “I personally am going to be able to keep my job.”

The sun never sets on legal un(der)employment

Posted on by

John Flood, a U.K. legal scholar and sociologist, comments on the well-documented travails of recent U.S. law graduates, noting that their U.K. counterparts are facing similar difficulties as globalization changes the practice of law the world over:

What we’ve seen in the UK is a disjunct between the numbers of law students coming into the academy and the numbers of jobs available. For many the problem is that the academy is producing too many law graduates and should be more sensitive to job availibility rates….[T]here is also a big rise in the use of paralegals and I don’t mean those trained to be paralegals. Rather the unemployed would-be lawyers are turning to paralegaling in the hope that a training contract might open up while they are there.

What will entrench the stratification of the market is the opening up (de- and re-regulation) of the legal services market that’s now taking place. Fewer jobs will need to be done by fully-qualified lawyers. They can instead be carried out by a range of people qualified for certain legal and quasi-legal tasks. This is where corporatized law meets Tesco Law. [Tesco is a U.K.-based retailer similar to Wal-Mart.]

The US legal profession still thinks it can maintain a headlock on the control of the profession. How long for? At the expense of a cheap shot, [Egyptian President] Mubarek is finding a 30-year rule coming to end; [former British Prime Minister] Tony Blair only lasted for 10 years before he was ejected. Permanent monopoly becomes increasingly hard to justifiy, especially in a global market.

Flood also references a recent Above the Law article, which noted that Thomson Reuters recently

announced that it was exploring the sale of BAR/BRI, its bar exam prep business, and purchasing Pangea3, a legal process outsourcing company. That’s a strong message that they think there’s more of a future in hiring people to do low-end legal work, work that technically doesn’t constitute “practicing law” under legal ethics rules, than in training the practitioners of the future.

I’d like to see a quantitative analysis backing up some of Flood’s assertions, but his general points are well taken:

  • There are more lawyers than jobs.
  • Many law jobs do not, objectively speaking, require lawyers.
  • Much legal work can be done at a distance–even across international borders–as a back-office service.
  • In the long- (and maybe even the short-) run, the established legal cartels are no match for these forces of globalization.

The importance of having meaningful work

Posted on by

Recent research suggests that the satisfaction individuals derive from work is not based just on a paycheck but rather on the meaning found in even doing menial tasks:

In several recent studies, social scientists have zeroed in on why paychecks alone can’t explain the link between work and well-being. The evidence shows that people can find meaning in seemingly insignificant jobs and that even trivial tasks make us far happier than no tasks at all.

“We become very dedicated to things it would be hard to be dedicated to if we were perfectly rational,” says behavioral scientist Dan Ariely, author of “The Upside of Irrationality,” published in June. “It turns out you can give people lots of meaning in lots of ways, even small ones.”…

The findings suggest that, although people often yield to idleness, deep down they seek excuses to stay busy, because busyness is happiness. However much Sisyphus rued his meaningless job, the authors conclude, he would have been even more miserable with no job at all.

Interesting findings that would have profound implications for the workplace.

Some quick questions:

1. Do these researchers argue that these benefits of working are linked to human nature or is it a conditioned response based on culture and other factors?

2. What are the long-term consequences of people having no work? If work is meaningful, what happens if people cannot work for different reasons (health, unemployment, other possibilities)?

3. How many workplaces (or what percentage) explicitly talk to employees about the meaningfulness of their individual work?