Our Broken Bar Exam

May 4th, 2017 / By

The bar exam is broken: it tests too much and too little. On the one hand, the exam forces applicants to memorize hundreds of black-letter rules that they will never use in practice. On the other hand, the exam licenses lawyers who don’t know how to interview a client, compose an engagement letter, or negotiate with an adversary.

This flawed exam puts clients at risk. It also subjects applicants to an expensive, stressful process that does little to improve their professional competence. The mismatch between the exam and practice, finally, raises troubling questions about the exam’s disproportionate racial impact. How can we defend a racial disparity if our exam does not properly track the knowledge, skills, and judgment that new lawyers use in practice?

We can’t. In the language of psychometricians, our bar exam lacks “validity.” We haven’t shown that the exam measures the quality (minimal competence to practice law) that we want to measure. On the contrary, growing evidence suggests that our exam is invalid: the knowledge and skills tested by the exam vary too greatly from the ones clients require from their lawyers.

We cannot ignore the bar exam’s invalidity any longer. Every legal educator should care about this issue, no matter how many of her students pass or fail the exam. The bar exam defines the baseline of our profession. If the exam tests the wrong things, we have a professional obligation to change it.

*   *   *

For the rest of this essay, please see aalsnews. I discuss the concept of exam validity, our lack of agreement on “minimal competence,” and how educators and practitioners could work together to solve these serious problems.

  • Scott_Fruehwald


    I enjoyed reading your article, and I wrote about it on the Legal Skills Prof Blog. One question: You wrote, “The primary reason we don’t test bar candidates on these skills is that law schools don’t stress them.” Isn’t it the other way around? If the bar exam tested on these competencies, law schools would teach them. As you pointed out, the bar exam sets minimum competencies for the legal profession. If we changed the bar exam, we would change legal education.

    • DeborahMerritt

      It was definitely too short of a statement to communicate my full meaning! My research suggests that, although deans complain frequently about the bar, schools have had more influence over the bar than they admit. In the early days, schools were quite influential in determining the nature of the bar–examiners really had no model other than what law schools did. Professors have always been involved in drafting and grading questions (even during the MBE-dominated era) and they have had surprising influence over state supreme courts–especially when it comes to excluding requirements for “practice” oriented admissions requirements. See, e.g., the demise of the recent California proposal for 15 hours of clinical experience. The latter wasn’t a proposal to modify the exam, but I think deans would protest just as vigorously (and probably with as much success) if it had been.
      I absolutely agree that, if the bar tested practice skills, schools would teach them. But for exactly that reason, most schools will resist inclusion of those skills on the bar. It’s possible that we are at a magic moment in which schools desperately want to change the bar’s heavy reliance on memorization (which is hurting their grads, admissions, rank, and possibly accreditation) and, as a quid pro quo, would accept increased testing (and thus teaching) of practice-oriented skills. The concept of validity would require that compromise: testing skills is valid, which testing memorization is not.
      I’m not sure if this moment will support that change, but I’m eager to try.

  • eddie davidson

    I couldn’t agree more. Well stated. While the rest of the world is driven by Silicon Valley, the legal profession remains enamored of coal oil and clipper ships. It’s probably too late to do much about this. If the law were personified in a a male image, it would have 3 names and muttonchops. No matter, I appreciate those who recognize the flaws. Thank you.

About Law School Cafe

Cafe Manager & Co-Moderator
Deborah J. Merritt

Cafe Designer & Co-Moderator
Kyle McEntee

ABA Journal Blawg 100 HonoreeLaw School Cafe is a resource for anyone interested in changes in legal education and the legal profession.

Around the Cafe


Enter your email address to receive notifications of new posts by email.


Recent Comments

Recent Posts

Monthly Archives


Have something you think our audience would like to hear about? Interested in writing one or more guest posts? Send an email to the cafe manager at merritt52@gmail.com. We are interested in publishing posts from practitioners, students, faculty, and industry professionals.

Past and Present Guests