Stephen Carter on the Bar Exam

July 26th, 2022 / By Deborah J. Merritt

Eminent Yale Professor Stephen Carter has penned a thoughtful critique of the bar exam. Professor Carter notes the exam’s similarities to the LSAT, which some law schools have abandoned as an admissions requirement. In addition to their shared affection for multiple choice questions, the LSAT and bar exam both constrain the diversity of our profession. Despite the bar exam’s disproportionate racial impact, Professor Carter notes, the exam has never been properly validated. Here, he cites a column I wrote in 2017 for the AALS Newsletter.

As I wrote then, state bar examiners and NCBE designed the bar exam around a definition of minimum competence that they “felt in their bones.” NCBE did not conduct a practice analysis of the knowledge and skills that new lawyers need until 2012. That analysis supported some of the doctrinal subjects that NCBE was testing, but not the depth of memorization required by the exam. The analysis also confirmed that skills like researching the law, fact gathering, negotiating, and interviewing were essential for law practice–all skills conspicuously absent from the bar exam.

NCBE conducted another practice analysis in 2019, which once again exposed numerous flaws in the exam. My own research, conducted with Logan Cornett and IAALS (the Institute for the Advancement of the American Legal System), reached a similar conclusion: the written bar exam tests both too much and too little. It restricts admission to the profession (especially of people of color) without adequately protecting the public.

NCBE is developing a new exam that will better serve the goals of licensing, but that exam won’t be ready until 2026. And it may still demand more memorization than new lawyers need while omitting critical skills like legal research. Lawyers don’t memorize the millions of state, local, and national rules that govern our society; they master threshold concepts and research techniques that allow them to find the rules they need. No matter how improved, NCBE’s bar exam is likely to remain an artificial barrier to entry into the legal profession.

How else can we license lawyers? Professor Carter suggests wider use of Wisconsin’s diploma privilege–licensing all graduates of ABA-accredited law schools. Here I part ways with him. If law schools taught law students all of the ways they need to think like a lawyer, I might agree. But most law schools persist in the illusion that 3 years of reading judicial opinions (or, for many students, 1-2 semesters of reading judicial opinions followed by 4-5 semesters of downloading case squibs and course outlines from Quimbee and other sources) teaches students to “think like lawyers.”

The traditional law school curriculum shies away from the more complex thinking required to gather facts related to legal principles, interview clients and witnesses, negotiate letter matters, and counsel clients. Law school classes teach students two-dimensional thinking, while law practice requires thinking in four dimensions.

Fortunately, it is possible to improve both legal education and licensing by adopting an experiential education path to licensing. New Hampshire adopted this approach through its Daniel Webster Scholars Program. Oregon’s Supreme Court has approved a similar path in principle, and a committee is fleshing out details. These pathways assure that future lawyers learn all of the knowledge and skills they need to protect clients; they also keep the final licensing decision in the hands of bar examiners, rather than law school professors.

How do these programs work? How do they achieve reliability and fairness in a feasible manner? I’ll address those issues in future posts. But for an overview, see this research guide that I coauthored with Logan Cornett.

Permalink

Public Comment on the Bar Exam

July 25th, 2022 / By Deborah J. Merritt

David Lat is hosting a “Notice and Comment on the Bar Exam” at Original Jurisdiction. Several commentators have offered thoughtful insights. Here’s mine:

One of the many problems with the bar exam is that it doesn’t test the knowledge and skills that new lawyers really need. The exam was designed based on “gut instincts” about that knowledge and skills but, as often is the case, gut instincts were wrong. The problem was compounded by well intentioned efforts to create a national exam, which has resulted in candidates memorizing a vast number of federal or “consensus” rules that they will never use in practice.

But now we have good evidence about the knowledge and skills that new lawyers actually use. NCBE’s recent practice analysis offers some insights, and the Building a Better Bar study (which I coauthored with Logan Cornett) offers more. NCBE is now building a better exam around those studies, but a written exam can’t capture many of the skills that are essential to lawyering–and it’s hard to capture the type of knowledge most new lawyers use in a uniform national exam.

States like Oregon, California, Minnesota, and Utah are considering much better alternatives, with Oregon in the lead. It is possible to assess lawyering knowledge and skills through either law school coursework (including clinics and other experiential work) or post-graduate supervised practice. For both of these pathways, bar examiners would make the final decision based on portfolios of work product and assessments from professors or others. And it is feasible to construct both of these pathways with sufficient reliability, fairness, and validity.

The benefits of change? Cheaper pathways to licensure for candidates, better protection of the public, and (most likely, given the stereotype threat that affects high stakes testing) a more diverse profession. What stands in the way? Outdated ideas about how lawyers think and work, legal education’s reluctance to embrace more experiential education, our profession’s reluctance to innovate, and good old fashioned protectionism (the bar exam may exclude more lawyers than these alternatives would).

It’s time to honor our avowed commitments to open the profession to all qualified candidates, protect the public, and increase diversity. The bar exam is not achieving those goals.

Permalink