More on the Bar Exam: Correlation and Competence
May 31st, 2017 / By Deborah J. Merritt
Derek Muller has identified an intriguing study of alternative ways to assess bar applicants. In 1980, the California bar examiners worked with a research team to explore the desirability of testing a wider range of lawyering skills on the bar exam. The researchers designed a two-day supplement to the bar exam and invited all July test-takers to participate in the supplemental exercise. More than 4,000 test-takers volunteered and, using appropriate sampling methods, the researchers chose 500 to participate. A few volunteers were unable to complete the exercise due to illness, so the final sample included 485 bar examinees.
These examinees completed the supplemental exercises in August 1980, shortly after taking the regular July exam. For two days, the examinees interviewed clients, drafted discovery plans, prepared letters, wrote trial briefs, cross-examined witnesses, and made arguments to mock juries. Each day’s work involved 5-6 tasks focused on a single client matter. Professional actors played the role of clients, and the researchers developed elaborate protocols for scoring the exercises.
How did results on the supplemental exam compare to those on the conventional test?
» Read the full text for More on the Bar Exam: Correlation and Competence
Reflections of a Bar Exam Skeptic
May 26th, 2017 / By Deborah J. Merritt
Robert Anderson has posted a thoughtful comment on the bar exam in which he dubs me a “bar exam skeptic.” I accept the label with pride: I have been deeply skeptical of the bar exam for years. I first wrote about the exam in 2001, when the national pass rate for first-time takers was a relatively high 77% (see p. 23). My skepticism today, with a national pass rate of 69%, is no greater or smaller. As I wrote recently, it’s time to convene a National Task Force to examine our bar admissions process.
Who Cares About the Bar Exam?
As Professor Anderson rightly observes, decanal concerns about the bar exam have risen as pass rates have fallen. That’s human nature. The content and scoring of the bar exam are boring subjects for alumni gatherings, graduation speeches, or law review submissions. Few legal educators spontaneously write about setting cut scores, scaling essay questions, or equating test scores over time. The bar exam is like plumbing: most people take it for granted until something goes wrong.
But now the bar exam pipes are leaking and people are paying attention. The leak doesn’t mean we should patch things up just to revive pass rates; the bar exam should measure competence, not admit a predetermined number of lawyers. But now that people are paying attention, this is a good time to consider whether we’re using the right type of filter and piping in our rather antiquated system.
Our Broken Bar Exam
May 4th, 2017 / By Deborah J. Merritt
The bar exam is broken: it tests too much and too little. On the one hand, the exam forces applicants to memorize hundreds of black-letter rules that they will never use in practice. On the other hand, the exam licenses lawyers who don’t know how to interview a client, compose an engagement letter, or negotiate with an adversary.
This flawed exam puts clients at risk. It also subjects applicants to an expensive, stressful process that does little to improve their professional competence. The mismatch between the exam and practice, finally, raises troubling questions about the exam’s disproportionate racial impact. How can we defend a racial disparity if our exam does not properly track the knowledge, skills, and judgment that new lawyers use in practice?
We can’t. In the language of psychometricians, our bar exam lacks “validity.” We haven’t shown that the exam measures the quality (minimal competence to practice law) that we want to measure. On the contrary, growing evidence suggests that our exam is invalid: the knowledge and skills tested by the exam vary too greatly from the ones clients require from their lawyers.
We cannot ignore the bar exam’s invalidity any longer. Every legal educator should care about this issue, no matter how many of her students pass or fail the exam. The bar exam defines the baseline of our profession. If the exam tests the wrong things, we have a professional obligation to change it.
* * *
For the rest of this essay, please see aalsnews. I discuss the concept of exam validity, our lack of agreement on “minimal competence,” and how educators and practitioners could work together to solve these serious problems.
Law School Deans Ask For Extension On Exploitation
January 18th, 2017 / By Kyle McEntee
Originally published on Above the Law.
More than 90 law school deans have asked their accreditor to halt new standards that would hold schools accountable for very low bar passage rates.
Last October, the Council of the ABA Section of Legal Education & Admissions to the Bar approved two new standards to stop exploitative admissions and retention practices. At a time when demand for law school decreased significantly, a minority of law schools began admitting swaths of students who, after three or more years of legal education, were not adequately equipped to pass the bar exam.
Why would a law school choose to do this? To keep tuition dollars flowing.
(more…)
» Read the full text for Law School Deans Ask For Extension On Exploitation
Council Approves New Bar Passage Standard
October 21st, 2016 / By Deborah J. Merritt
The Council of the ABA’s Section of Legal Education and Admissions to the Bar has approved a hotly debated proposal to tighten the accreditation standard governing bar passage rates. When the new standard takes effect, schools will have to demonstrate that seventy-five percent of graduates who choose to take a bar exam pass that exam within two years.
Opponents of the standard argued that it might reduce racial and ethnic diversity in the legal profession. Council members, however, largely rejected that argument. Raymond Pierce, former dean of the North Carolina Central University School of Law, distinguished between programs that give students “an opportunity” and those offering “a false chance.”
For more, see this story.
ABA Moves Forward on Revised Accreditation Standards
September 10th, 2016 / By Kyle McEntee
The ABA Section of Legal Education’s Standards Review and Data Policy Committee voted unanimously today to recommend that the Section’s Council approve revisions to Standards 501 and 316.
This comes on the heels of a multi-month notice and comment period, which saw a number of comments about the revisions.
The committee recommended that the revised standards be adopted as proposed.
By taking this action, the committee acknowledges that its primary responsibilities are protecting the public and students, not law schools.
The True Cost of the Georgia Bar Exam Error
September 8th, 2016 / By Renee Allen
To many, late October signals nothing more than fall in full swing, pumpkins, or costumes. In late May, we look forward to the Memorial Day holiday and long weekends. Yet, the last weekend of every October and May, Georgia bar takers anxiously await exam results. Some stalk the postman. Most spend the day refreshing a webpage, hoping and praying their name appears on the public pass list.
The stages of grief—denial, anger, bargaining, depression, and acceptance—are experienced by one who fails a state bar exam. Imagine discovering that a family member is alive after grieving their death for ten months. This week, 90 Georgia bar takers—45 from July 2015 and 45 from February 2016—were informed that the thing they grieved was, in fact, alive. Though their names failed to appear on that very public pass list, they indeed passed the Georgia bar exam.
(more…)
» Read the full text for The True Cost of the Georgia Bar Exam Error
The Latest Change in the MBE
September 5th, 2016 / By Deborah J. Merritt
In the memo announcing results from the July 2016 MBE, Erica Moeser also notified law school deans about an upcoming change in the test. For many years the 200-question exam has included 190 scored items and 10 pre-test questions. Starting in February 2017, the numbers will shift to 175 scored items and 25 pre-test ones.
Pre-testing is an important feature of standardized exams. The administrator uses pre-test answers to gauge a question’s clarity, difficulty, and usefulness for future exams. When examinees answer those questions, they improve the design of future tests.
From the test-taker’s perspective, these pre-test questions are indistinguishable from scored ones. Like other test-makers, NCBE scatters its pre-test questions throughout the exam. Examinees answer each question without knowing whether it is a “real” item that will contribute to their score or a pre-test one that will not.
So what are the implications of NCBE’s increase in the number of pre-test items? The shift is relatively large, from 10 questions (5% of the exam) to 25 (12.5% of the exam). I have three concerns about this change: fair treatment of human research subjects, reliability of the exam, and the possible impact on bar passage rates. I’ll explore the first of these concerns here and turn to the others in subsequent posts.
Surprise: MBE Scores Rise in 2016
August 31st, 2016 / By Deborah J. Merritt
Erica Moeser, President of the National Conference of Bar Examiners, sent a memo to law school deans today. The memo reported the welcome, but surprising, news that the national mean score on the MBE was higher in July 2016 than in July 2015. Last year, the national mean was just 139.9. This year, it’s 140.3.
That’s a small increase, but it’s nonetheless noteworthy. LSAT scores for entering law students have been falling for several years. The drop between fall 2012 and fall 2013 was quite noticeable: Seventy percent of ABA-accredited law schools experienced a drop in the 25th percentile score of their entering class. At 19 schools, that score fell 3 points. At another five, it was 4 points.
LSAT scores correlate with MBE scores, so many observers expected July 2016 MBE scores to be lower than those recorded in 2015. Moeser, for example, has repeatedly stressed the link between LSAT scores and MBE ones. She recently declared: “What would surprise me is if LSAT scores dropped and bar pass rates didn’t go down.”
Moeser just received that surprise: Students who began law school in fall 2013 had lower LSAT scores than those who began a year earlier. The former students, however, beat the latter on the MBE after graduation.
So What Happened?
Unpacking this news will take more time and data. Moeser mentions in her memo that the mean MBE score increased in 22 jurisdictions, fell in 26, and remained stable in two. Teasing apart the jurisdictions will provide insights. School-specific results will be even more informative in exploring why the overall score rose.
For now, I offer four hypotheses in descending order of likelihood (from my perspective):
The ABA Intends To Hold Law Schools Accountable
March 16th, 2016 / By Kyle McEntee
The good news keeps coming for law school reform advocates. The ABA Section of Legal Education and Admissions to the Bar has taken its next affirmative step towards holding law schools accountable for their exploitative admissions and retention choices.
Soon, the Council for the Section of Legal Education will publish the proposed ABA accreditation standard changes for public comment. The Council will assess any new information it obtains and consider approving the new standards in October. Although the Council is the final authority for law school accreditation, the ABA House of Delegates will vote in February. The process allows the House a formal but non-binding say in new standards.
Let’s review the proposals. (more…)
» Read the full text for The ABA Intends To Hold Law Schools Accountable
About Law School Cafe
Cafe Manager & Co-Moderator
Deborah J. Merritt
Cafe Designer & Co-Moderator
Kyle McEntee
Law School Cafe is a resource for anyone interested in changes in legal education and the legal profession.
Around the Cafe
Subscribe
Categories
Recent Comments
- on Scholarship Advice
- on ExamSoft: New Evidence from NCBE
- on COVID-19 and the Bar Exam
- on Women Law Students: Still Not Equal
- on Ranking Academic Impact
Recent Posts
- GPT-4 on Legal Education and Licensing
- The Bot Takes the Bar Exam
- The Rule Against Perpetuities
- Don’t Disparage
- Stephen Carter on the Bar Exam
Monthly Archives
Participate
Have something you think our audience would like to hear about? Interested in writing one or more guest posts? Send an email to the cafe manager at merritt52@gmail.com. We are interested in publishing posts from practitioners, students, faculty, and industry professionals.