Reflections of a Bar Exam Skeptic

May 26th, 2017 / By

Robert Anderson has posted a thoughtful comment on the bar exam in which he dubs me a “bar exam skeptic.” I accept the label with pride: I have been deeply skeptical of the bar exam for years. I first wrote about the exam in 2001, when the national pass rate for first-time takers was a relatively high 77% (see p. 23). My skepticism today, with a national pass rate of 69%, is no greater or smaller. As I wrote recently, it’s time to convene a National Task Force to examine our bar admissions process.

Who Cares About the Bar Exam?

As Professor Anderson rightly observes, decanal concerns about the bar exam have risen as pass rates have fallen. That’s human nature. The content and scoring of the bar exam are boring subjects for alumni gatherings, graduation speeches, or law review submissions. Few legal educators spontaneously write about setting cut scores, scaling essay questions, or equating test scores over time. The bar exam is like plumbing: most people take it for granted until something goes wrong.

But now the bar exam pipes are leaking and people are paying attention. The leak doesn’t mean we should patch things up just to revive pass rates; the bar exam should measure competence, not admit a predetermined number of lawyers. But now that people are paying attention, this is a good time to consider whether we’re using the right type of filter and piping in our rather antiquated system.

(more…)

, No Comments Yet

Our Broken Bar Exam

May 4th, 2017 / By

The bar exam is broken: it tests too much and too little. On the one hand, the exam forces applicants to memorize hundreds of black-letter rules that they will never use in practice. On the other hand, the exam licenses lawyers who don’t know how to interview a client, compose an engagement letter, or negotiate with an adversary.

This flawed exam puts clients at risk. It also subjects applicants to an expensive, stressful process that does little to improve their professional competence. The mismatch between the exam and practice, finally, raises troubling questions about the exam’s disproportionate racial impact. How can we defend a racial disparity if our exam does not properly track the knowledge, skills, and judgment that new lawyers use in practice?

We can’t. In the language of psychometricians, our bar exam lacks “validity.” We haven’t shown that the exam measures the quality (minimal competence to practice law) that we want to measure. On the contrary, growing evidence suggests that our exam is invalid: the knowledge and skills tested by the exam vary too greatly from the ones clients require from their lawyers.

We cannot ignore the bar exam’s invalidity any longer. Every legal educator should care about this issue, no matter how many of her students pass or fail the exam. The bar exam defines the baseline of our profession. If the exam tests the wrong things, we have a professional obligation to change it.

*   *   *

For the rest of this essay, please see aalsnews. I discuss the concept of exam validity, our lack of agreement on “minimal competence,” and how educators and practitioners could work together to solve these serious problems.

, View Comments (3)

Law School Deans Ask For Extension On Exploitation

January 18th, 2017 / By

Originally published on Above the Law.

Laptop in classic libraryMore than 90 law school deans have asked their accreditor to halt new standards that would hold schools accountable for very low bar passage rates.

Last October, the Council of the ABA Section of Legal Education & Admissions to the Bar approved two new standards to stop exploitative admissions and retention practices. At a time when demand for law school decreased significantly, a minority of law schools began admitting swaths of students who, after three or more years of legal education, were not adequately equipped to pass the bar exam.

Why would a law school choose to do this? To keep tuition dollars flowing.
(more…)

, View Comment (1)

Council Approves New Bar Passage Standard

October 21st, 2016 / By

The Council of the ABA’s Section of Legal Education and Admissions to the Bar has approved a hotly debated proposal to tighten the accreditation standard governing bar passage rates. When the new standard takes effect, schools will have to demonstrate that seventy-five percent of graduates who choose to take a bar exam pass that exam within two years.

Opponents of the standard argued that it might reduce racial and ethnic diversity in the legal profession. Council members, however, largely rejected that argument. Raymond Pierce, former dean of the North Carolina Central University School of Law, distinguished between programs that give students “an opportunity” and those offering “a false chance.”

For more, see this story.

, View Comment (1)

ABA Moves Forward on Revised Accreditation Standards

September 10th, 2016 / By

The ABA Section of Legal Education’s Standards Review and Data Policy Committee voted unanimously today to recommend that the Section’s Council approve revisions to Standards 501 and 316.

This comes on the heels of a multi-month notice and comment period, which saw a number of comments about the revisions.

The committee recommended that the revised standards be adopted as proposed.

By taking this action, the committee acknowledges that its primary responsibilities are protecting the public and students, not law schools.

, No Comments Yet

The True Cost of the Georgia Bar Exam Error

September 8th, 2016 / By

To many, late October signals nothing more than fall in full swing, pumpkins, or costumes. In late May, we look forward to the Memorial Day holiday and long weekends. Yet, the last weekend of every October and May, Georgia bar takers anxiously await exam results. Some stalk the postman. Most spend the day refreshing a webpage, hoping and praying their name appears on the public pass list.

The stages of grief—denial, anger, bargaining, depression, and acceptance—are experienced by one who fails a state bar exam. Imagine discovering that a family member is alive after grieving their death for ten months. This week, 90 Georgia bar takers—45 from July 2015 and 45 from February 2016—were informed that the thing they grieved was, in fact, alive. Though their names failed to appear on that very public pass list, they indeed passed the Georgia bar exam.
(more…)

, No Comments Yet

The Latest Change in the MBE

September 5th, 2016 / By

In the memo announcing results from the July 2016 MBE, Erica Moeser also notified law school deans about an upcoming change in the test. For many years the 200-question exam has included 190 scored items and 10 pre-test questions. Starting in February 2017, the numbers will shift to 175 scored items and 25 pre-test ones.

Pre-testing is an important feature of standardized exams. The administrator uses pre-test answers to gauge a question’s clarity, difficulty, and usefulness for future exams. When examinees answer those questions, they improve the design of future tests.

From the test-taker’s perspective, these pre-test questions are indistinguishable from scored ones. Like other test-makers, NCBE scatters its pre-test questions throughout the exam. Examinees answer each question without knowing whether it is a “real” item that will contribute to their score or a pre-test one that will not.

So what are the implications of NCBE’s increase in the number of pre-test items? The shift is relatively large, from 10 questions (5% of the exam) to 25 (12.5% of the exam). I have three concerns about this change: fair treatment of human research subjects, reliability of the exam, and the possible impact on bar passage rates. I’ll explore the first of these concerns here and turn to the others in subsequent posts.

(more…)

, No Comments Yet

Surprise: MBE Scores Rise in 2016

August 31st, 2016 / By

Erica Moeser, President of the National Conference of Bar Examiners, sent a memo to law school deans today. The memo reported the welcome, but surprising, news that the national mean score on the MBE was higher in July 2016 than in July 2015. Last year, the national mean was just 139.9. This year, it’s 140.3.

That’s a small increase, but it’s nonetheless noteworthy. LSAT scores for entering law students have been falling for several years. The drop between fall 2012 and fall 2013 was quite noticeable: Seventy percent of ABA-accredited law schools experienced a drop in the 25th percentile score of their entering class. At 19 schools, that score fell 3 points. At another five, it was 4 points.

LSAT scores correlate with MBE scores, so many observers expected July 2016 MBE scores to be lower than those recorded in 2015. Moeser, for example, has repeatedly stressed the link between LSAT scores and MBE ones. She recently declared: “What would surprise me is if LSAT scores dropped and bar pass rates didn’t go down.”

Moeser just received that surprise: Students who began law school in fall 2013 had lower LSAT scores than those who began a year earlier. The former students, however, beat the latter on the MBE after graduation.

So What Happened?

Unpacking this news will take more time and data. Moeser mentions in her memo that the mean MBE score increased in 22 jurisdictions, fell in 26, and remained stable in two. Teasing apart the jurisdictions will provide insights. School-specific results will be even more informative in exploring why the overall score rose.

For now, I offer four hypotheses in descending order of likelihood (from my perspective):

(more…)

, View Comments (3)

The ABA Intends To Hold Law Schools Accountable

March 16th, 2016 / By

The good news keeps coming for law school reform advocates. The ABA Section of Legal Education and Admissions to the Bar has taken its next affirmative step towards holding law schools accountable for their exploitative admissions and retention choices.

Soon, the Council for the Section of Legal Education will publish the proposed ABA accreditation standard changes for public comment. The Council will assess any new information it obtains and consider approving the new standards in October. Although the Council is the final authority for law school accreditation, the ABA House of Delegates will vote in February. The process allows the House a formal but non-binding say in new standards.

Let’s review the proposals. (more…)

, View Comment (1)

The Latest Issue of the Bar Examiner

March 15th, 2016 / By

The National Conference of Bar Examiners (NCBE) has released the March 2016 issue of their quarterly publication, the Bar Examiner. The issue includes annual statistics about bar passage rates, as well as several other articles. For those who lack time to read the issue, here are a few highlights:

Bar-Academy Relationships

In his Letter from the Chair, Judge Thomas Bice sounds a disappointingly hostile note towards law students. Quoting Justice Edward Chavez of the New Mexico Supreme Court, Bice suggests that “those who attend law school have come to have a sense of entitlement to the practice of law simply as a result of their education.” Against this sentiment, he continues, bar examiners “are truly the gatekeepers of this profession.” (P. 2)

NCBE President Erica Moeser, who has recently tangled with law school deans, offers a more conciliatory tone on her President’s Page. After noting the importance of the legal profession and the challenges facing law schools, she concludes: “In many ways, we are all in this together, and certainly all of us wish for better times.” (P. 5)

(more…)

, View Comment (1)

About Law School Cafe

Cafe Manager & Co-Moderator
Deborah J. Merritt

Cafe Designer & Co-Moderator
Kyle McEntee

ABA Journal Blawg 100 HonoreeLaw School Cafe is a resource for anyone interested in changes in legal education and the legal profession.

Around the Cafe

Subscribe

Enter your email address to receive notifications of new posts by email.

Categories

Recent Comments

Recent Posts

Monthly Archives

Participate

Have something you think our audience would like to hear about? Interested in writing one or more guest posts? Send an email to the cafe manager at merritt52@gmail.com. We are interested in publishing posts from practitioners, students, faculty, and industry professionals.

Past and Present Guests