COVID-19 and the Bar Exam
March 23rd, 2020 / By Deborah J. Merritt
The news about COVID-19 gets worse by the hour. People are dying. The virus is spreading. Health care workers lack protective gear. Businesses are closed. We are sheltered at home. In the midst of these life-or-death matters, it seems mundane to worry about the July 2020 bar exam.
But we need to plan for the July bar, just as we need to think about every other part of life affected by the pandemic. Jurisdictions almost certainly will not be able to administer the exam in its usual format–to hundreds of test-takers gathered in large rooms or arenas. What should they do instead?
Over the weekend, I joined with a team of researchers to propose alternatives for licensing the Class of 2020. Our short paper is available at
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3559060. We don’t have all the answers, but we think it is critical to start the conversation. Decisions about the July bar must be made during the next few weeks.
If you have comments or additional suggestions, please let us know. As we state at the end of the paper: These are unprecedented times and we must work together to meet them. Our society must first work to secure its health, but intense legal needs will follow soon thereafter. We can’t afford to leave the talented members of the Class of 2020 sitting on the sidelines as those needs erupt. It is time to call all hands on deck.
Ranking Academic Impact
February 17th, 2020 / By Deborah J. Merritt
Paul Heald and Ted Sichelman have published a new ranking of the top U.S. law schools by academic impact. Five distinguished scholars comment on their ranking in the same issue of Jurimetrics Journal in which the ranking appears. But neither the authors of this ranking nor their distinguished commentators notice a singular result: The Heald/Sichelman rankings include a law school that does not exist.
According to Heald and Sichelman, Oregon State ranks 53d among U.S. law schools for its SSRN downloads; 35th for its citations in the Hein database; and 46th in a combined metric. Oregon State, however, does not have a law school. The University of Oregon has a law school, but it appears separately in the Heald/Sichelman rankings. So Heald and Sichelman have not simply fumbled the name of Oregon’s only public law school.
Instead, it appears that my own law school (Ohio State) has been renamed Oregon State. I can’t be sure without seeing Heald and Sichelman’s underlying data; even the “open” database posted in Dropbox refers to the nonexistent Oregon State. But Ohio State, currently tied for 34th in the US News survey, seems conspicuously absent from the Heald/Sichelman ranking.
I’m sure that my deans will contact Heald and Sichelman to request a correction–assuming that Oregon State actually is Ohio State. Oregon State Law School’s administrators probably will not complain. They can’t celebrate either, of course, because they don’t exist. But apart from that correction, let’s ruminate on this error. What does it have to say about rankings?
Reliability
A mistake like this obviously raises doubts about the reliability of the Heald/Sichelman ranking. If an error of this magnitude exists, what other errors lurk in the data? Even if you like the Heald/Sichelman method, how do you know it was carried out faithfully?
Some errors plague any type of large quantitative study, but an error of this nature is unusual. One of the key rules of quantitative analysis is to step back from the data periodically to ask if the patterns make sense. Surprising results may represent genuine, novel insights–but they can also be signs of underlying errors.
Heald and Sichelman studied the correlations between their rankings and several other measures. Didn’t they notice that one school produced a missing value? And when discussing schools that had highly discrepant rankings, didn’t they notice that one school in their scheme did not appear at all in other ranking schemes?
It’s possible, of course, that Heald and Sichelman misnamed Ohio State throughout their database so that they compared Oregon State’s Heald/Sichelman rank with the same misnamed school’s US News rank. Or perhaps the error slipped in near the end when they or an assistant changed “Ohio” to “Oregon” in the article’s spreadsheets.
Quantitative researchers who have their hands deeply in the data, however, should catch errors like this. Even after Heald and Sichelman banish Oregon State from their ranking, I will retain doubts about the reliability of their data. And my doubts about the reliability of other rankings, no matter how “scientific,” have been aroused.
For What Purpose?
Heald and Sichelman’s error is troubling, but I am equally concerned about the failure of any of their readers to spot the mistake. How could five commentators, as well as numerous other readers and workshop participants, blithely skim over the nonexistent Oregon State? Even if they weren’t familiar with Oregon’s law schools, weren’t they surprised to see Oregon State ranked 35th for Hein citations? The three schools in that state currently appear as 83d, 104th, and in the unranked fourth tier of the US News survey. Shouldn’t someone have noticed the surprising strength of Oregon State’s faculty?
I suspect that no one noticed the presence of Oregon State because most faculty read rankings primarily to see where their own school ranks. That’s what I did: I was curious where my own faculty ranked and, when Ohio State was absent, I looked more closely. It was only then that I noticed a law school that doesn’t exist.
But if that is the use of these academic impact rankings, to give faculty comfort or angst about where their law school ranks, are these rankings worth producing? They require a great deal of work and number crunching, as Heald and Sichelman make clear. Even with their presumably careful work, a substantial mistake occurred. Is the pay-off (including mistakes) worth the effort?
More worrisome, I think these rankings will harm the legal profession and its clients. Legal educators are key stewards of the legal profession. We are the profession’s primary gatekeepers: Few people become lawyers without first earning our diplomas. We are also responsible for giving students the foundation they need to serve clients competently and ethically.
Rankings of academic impact almost certainly will incentivize schools to invest still more of their resources in faculty scholarship—which, in turn, will raise tuition, reduce student discounts, and/or divert money from preparing students for their essential professional roles.
Scholarship is part of our commitment to the profession, clients, and society, but only one part. Over the last 20 years, I have seen law schools shift increasing resources to scholarship, while reducing teaching loads and raising tuition rapaciously. We produced excellent scholarship before 2000–scholarship that created fields like critical race theory, law and economics, feminist theory, and social science analyses of law-related issues. There is much still to explore, but why does today’s scholarship demand so many more resources? And will rankings further accelerate that trend?
Women Law Students: Still Not Equal
January 5th, 2020 / By Deborah J. Merritt
The Law School Survey of Student Engagement (LSSSE) released its annual report just before the holiday break. This year’s report, titled “The Cost of Women’s Success,” explores the gendered nature of law students’ experience.
I had the honor of contributing a Foreword to the LSSSE report. Summing up the report’s findings, I wrote:
Two law students, a woman and a man, sit side-by-side in class. From the podium, they look similar: both concentrate intently on the professor, take notes, and listen to classmates’ comments. But, as this LSSSE report reveals, their broader law school experiences likely diverge in meaningful ways.
The man is more likely to have a parent who was a lawyer; he is also more likely to have a parent who attended college. When the professor pauses for questions, the man is more likely to raise his hand. If the man and women are Latinx, the gender difference in classroom participation will be particularly stark.
After class, the man is more likely to exercise, read for pleasure, and pursue other leisure activities. The woman is more likely to attend a student organization meeting, email a professor, or speak to an advisor about her career plans.
At the end of the day, the woman is less likely than the man to get a full night’s sleep: half of LSSSE’s women respondents report that they average no more than five hours of sleep a night. And when the woman wakes to face another demanding day, she is less likely to find institutional support for her burdens.
Nor do the woman’s challenges end with graduation. She is more likely than the man to shoulder high debt as she enters the workplace. Those differences, like others noted in this report, sharpen at the intersection of gender and race. Sixteen percent of Latinas borrow more than $200,000 to attend law school, compared to 12% of Latinos and 4.3% of White men.
Despite these differences, women succeed in law school. Among LSSSE respondents, women’s reported grades exceeded those of men. That was true for women overall, as well as within each racial or ethnic group. As the report’s title suggests, however, women succeed at a cost: less sleep, fewer wellness activities, and more debt.
In this new year, law schools need to look more deeply at the gender differences that color our students’ experience. Those of us who stand at the podium (faculty and administrators) see the equal numbers of men and women sitting before us. We pride ourselves on that equity without probing below the surface. Feminist scholars, including some student authors, have continued to illuminate the gendered nature of legal education. Now LSSSE adds to that literature through the voices of more than 18,000 law students.
What’s Your Story?
October 6th, 2018 / By Deborah J. Merritt
I’ve been attending the SALT Teaching Conference, hosted by Penn State Law in the aptly named Happy Valley, Pennsylvania. It was a great conference, with many thought-provoking ideas: I hope to share several of them over the coming days.
Here’s my first pass-along idea: Mariela Olivares from Howard University’s School of Law told us that she asks students in her Immigration Law course to write their personal immigration stories. When did members of their families arrive in the United States? Was the immigration voluntary or forced? What challenges did they face? What opportunities? Mariela allows students to choose whether these stories are confidential (for her eyes only) or can be shared with the rest of the class.
What a great way to engage students in the course content! As the course proceeds, students can reflect on how the laws affected their own family’s experience–and how that experience might differ under contemporary regulations. Even a class of 20 students will generate a rich set of stories that, if students are willing to share, could illuminate many corners of the course content.
This is also a wonderful way to build empathy in a doctrinal classroom. Empathy begins with self knowledge, and Mariela’s exercise requires students to confront their own history and feelings about the immigration system. Then, as students share their stories with others, they can begin to experience the system from a variety of perspectives.
I think it would be easy to expand this technique to almost every course in the curriculum. A Torts professor could ask students to write about an incident in which they or a family member suffered a physical or emotional injury. As with immigration stories, the exercise probably would generate stories relevant to every legal principle covered in the course. Which injuries could have been addressed by the tort system? Which ones were left out? Why? If the student/family member did not seek redress, why not?
Even courses about procedural rules could incorporate stories. Next time I teach Evidence, I may begin the semester by asking students to write about an incident in which a piece of evidence contributed to a decision they made–and they later discovered that the evidence was false or misleading. I won’t be looking for stories about the courtroom, but about everyday life. I suspect that the everyday stories may help me illuminate the problems with character evidence, hearsay, eyewitness identifications, and other evidence challenges.
So what’s your story? And how could personal stories kick off a course that you teach?
NALP Employment Data
August 2nd, 2018 / By Deborah J. Merritt
The National Association for Law Placement (NALP) has just released data about employment outcomes for the Class of 2017. More than two-thirds of graduates (68.8%) found full-time, long-term jobs requiring bar admission. According to NALP’s figures, that’s “higher than the rate measured before the recession.” The boost in employment outcomes, however, rests largely on the decline in JD class sizes. Between 2013 and 2017, the graduating class size fell by more than 25%.
Employment outcomes thus offer a mixed picture. On the one hand, as NALP’s Executive Director James Leipold writes, “we are closer than at any time since the recession to having the number of law school graduates more closely match the number and kind of jobs available.” Graduates are also obtaining more of the lawyering jobs they prefer; as Leipold notes, the percentage of graduates taking JD Advantage jobs has fallen, “suggest[ing] that despite the growth of new JD Advantage opportunities in areas like compliance, many law graduates prefer bar passage required jobs if they can be found.”
On the other hand, as Leipold also stresses, these positive employment outcomes rest on “a smaller [graduating] class and not more jobs.” Indeed, the Class of 2017 “secured fewer private practice jobs than any class since 1996.” The “unemployment rate ten months after graduation still remains much higher than it should be” and “the actual number of jobs obtained was flat or went down in virtually every sector.” (more…)
Now They Just Need Jobs
July 31st, 2018 / By Deborah J. Merritt
Legal education is regaining some of its luster: The National Law Journal reports that applications for this year’s entering class increased 8% over last year. The news for next year is even better: LSAT-takers increased 30% this summer compared to last year. But observers, including LSAC’s president Kellye Testy, urge caution. The entry-level job market remains relatively flat, with fewer 2017 graduates finding long-term, full-time positions requiring bar admission in 2017 (23,114) than in 2011 (24,149). Those employment levels don’t accommodate our current, reduced class sizes–much less an expanded class.
Integrating employment data with admissions is a tricky business, as I and several others note in a recent ABA Journal article. On the one hand, it is worrisome for schools to charge tuition to students who are unlikely to find jobs that will fully use their expensive degrees. On the other hand, limiting admissions to reduce the supply of lawyers can raise prices for consumers (although lawyers, unfortunately, are not known for their competitive, cost-saving innovations).
However your school strikes this balance, this is a good time to consider how we can improve employment prospects for current and future students. Here are my top five ideas. Some may help expand the market for entry-level lawyers. Others could give your students an employment edge over those from other schools. (more…)
You Too Can Check Out Quimbee
July 30th, 2018 / By Deborah J. Merritt
Since I posted about Quimbee, several colleagues have asked if there are ways to check out this new study aid. That’s easy: just sign up for a free 7-day trial. The trial really is free. You don’t need to enter credit card information; nor will Quimbee hound you to purchase the service when the week ends. I signed up for a free trial before writing my blog post and, other than a polite email noting that my trial was about to expire, Quimbee did nothing to pressure me for money. Nor have I gotten spam since my trial ended. So, if you wonder what your students are reading on Quimbee, go ahead and check the site out.
Can We Teach Students to Read Cases?
July 24th, 2018 / By Deborah J. Merritt
I suggested in my last three posts that law students don’t learn how to read judicial opinions as carefully and thoughtfully as they should. Can we fix this? Can we modify legal education so that JD’s develop stronger case-reading skills? Solving this problem is important in itself: most lawyers interpret appellate opinions at least some of the time. Considering how to fix this problem, furthermore, can shed light on other pedagogical challenges. (more…)
» Read the full text for Can We Teach Students to Read Cases?
But Can They Read Cases?
July 8th, 2018 / By Deborah J. Merritt
I recently suggested that the case method fails to achieve one of its central goals: teaching students how to read and synthesize judicial opinions effectively. I identified three reasons for this shortfall: the format of law school exams, a growing emphasis on teaching doctrine, and the impact of contemporary study aids. But is it true? Are law students failing at case analysis?
An empirical study led by education scholar Dorothy Evensen suggests that they are. Evensen collaborated with Laurel Oates, an internationally recognized expert on legal analysis, and two other empiricists (James Stratman and Sarah Zappe) to examine the case reading skills of more than 300 students at five different law schools. The four published their study ten years ago, but it is just as relevant today. Let’s take a look at the study’s method, findings, and import. (more…)
Quimbee
July 2nd, 2018 / By Deborah J. Merritt
What is Quimbee? It’s a database of more than 13,800 case briefs summarizing the cases presented in almost 200 casebooks. The covered casebooks include all of the bar subjects plus many more: antitrust, copyright, counterterrorism, cyberspace, education law, health, insurance, oil and gas, sports law, and others. The briefs seem thoughtful, complete, and well organized. If users discover an error, Quimbee invites them to submit a correction.
About two dozen law schools (including heavy hitters like Yale, Berkeley, and Northwestern) have signed up for Quimbee; students at those schools use Quimbee for free. Students at other schools can access Quimbee for a modest fee: just $15 per month for all of the site’s case briefs. For $24 per month, students can use the briefs, flashcards, video lessons, multiple choice questions, essays with model answers, and professional development courses.
Even the ABA has lent its endorsement; students who elect Quimbee’s platinum subscription receive an ABA premium membership as part of the deal. For a one-time payment of $499, these students get three full years’ of access to Quimbee’s case briefs and other services, downloads of course outlines, and the ABA membership. That’s a law school career of case briefs and other study aids (plus a bit of networking) for the price of two casebooks.
Plenty of other publishers have developed attractive study-aid subscriptions. There are also websites that offer case briefs and law school outlines for little or no charge. I focus here on Quimbee because it’s a good example of the resources that today’s law students use. These easily tapped sources raise the question: How much pedagogic value does the case method deliver when most students are using canned case briefs and course outlines? (more…)
About Law School Cafe
Cafe Manager & Co-Moderator
Deborah J. Merritt
Cafe Designer & Co-Moderator
Kyle McEntee
Law School Cafe is a resource for anyone interested in changes in legal education and the legal profession.
Around the Cafe
Subscribe
Categories
Recent Comments
- on Scholarship Advice
- on ExamSoft: New Evidence from NCBE
- on COVID-19 and the Bar Exam
- on Women Law Students: Still Not Equal
- on Ranking Academic Impact
Recent Posts
- The Bot Takes a Bow
- Fundamental Legal Concepts and Principles
- Lay Down the Law
- The Bot Updates the Bar Exam
- GPT-4 Beats the Bar Exam
Monthly Archives
Participate
Have something you think our audience would like to hear about? Interested in writing one or more guest posts? Send an email to the cafe manager at merritt52@gmail.com. We are interested in publishing posts from practitioners, students, faculty, and industry professionals.