The Bot Updates the Bar Exam

October 30th, 2023 / By Deborah J. Merritt

** Update 11/15/23: NCBE has deleted the sample question I discuss here from their site. If you want to see the original question, you can find it through the Wayback Machine at this link.

In my last post, I discussed a sample bar exam question on which GPT-4 provided more helpful answers than the choices on the bar exam. As I continued my exploration, I found another question on which GPT-4 revealed that NCBE is testing an outdated minority rule of law.

The question is the eighth multiple-choice question on the NextGen site. The question posits a landlord and tenant who agree upon a commercial tenancy beginning July 1. When the tenant arrives at the premises on that date, it finds that the previous tenant has not vacated the premises–and does not intend to vacate. I prompted GPT-4 with NCBE’s question (including its answer choices) and received this response:

---

The answer would likely be D. The tenant had not notified the landlord before bringing the eviction action.

Here’s a breakdown of the choices:

A. By suing the previous tenant for possession, the tenant elected that remedy in lieu of a suit against the landlord.

• This is not necessarily the most convincing argument. While it’s true that the tenant elected to pursue the remedy of eviction against the prior tenant, it doesn’t mean they waived their rights against the landlord.

B. The landlord had delivered the legal right of possession to the tenant.

• This is a tricky choice. While it’s true that the landlord and tenant orally agreed to a commercial tenancy, the landlord might have failed to deliver actual possession if the previous tenant was still occupying the premises. However, the landlord did give the tenant the legal right to possess the property, even if actual possession wasn’t immediately available. This could be a valid argument, but it’s not as strong as choice D.

C. The tenant failed to timely vacate as required to sue for constructive eviction.

• This choice doesn’t make sense in this context. Constructive eviction refers to a situation where a tenant is forced to leave a property due to the landlord’s actions (or failures to act), which make the premises uninhabitable. The tenant in this scenario was trying to gain possession, not vacate.

D. The tenant had not notified the landlord before bringing the eviction action.

• This seems like the strongest answer. Before taking any legal action, it’s typically expected that the aggrieved party (in this case, the tenant) will notify the party at fault (the landlord) to give them an opportunity to rectify the situation. The tenant didn’t inform the landlord of the issue until after evicting the previous tenant, meaning the landlord wasn’t given a chance to address the issue or find a solution.

Therefore, the most likely explanation if the court finds for the landlord would be D.

---

This response intrigued me because, according to NCBE, the correct answer is B. I decided to explore the inconsistency between NCBE’s question writers and GPT-4, so I told GPT-4: “I want to understand more about choice B. Are there states in which it is sufficient for a landlord to deliver legal possession without actual possession?” GPT-4 responded:

---

Yes, there’s a distinction between legal possession and actual possession, and different jurisdictions handle this distinction differently.

1. Legal Possession vs. Actual Possession:
   • Legal Possession: Refers to the tenant’s right to possess a property. When a lease is signed (or an oral agreement is made, as long as it’s legally enforceable), the tenant typically gains legal possession of the property. This means that even if the tenant cannot actually use or occupy the property, they still have the legal rights to it.
   • Actual Possession: Refers to the physical occupancy of a property. When a tenant moves into a property and starts using it, they have taken actual possession.
2. Legal Possession Sufficient in Some Jurisdictions:
   • In some jurisdictions, when a landlord grants a lease to a tenant, they are only obligated to deliver legal possession, not actual possession. In these jurisdictions, if a prior tenant holds over (i.e., doesn’t vacate at the end of their lease), the new tenant’s primary remedy might be against the holdover tenant, not the landlord. The reasoning is that the landlord has done their part by granting the right to possess the property, and any dispute over actual possession is between the new tenant and anyone unlawfully occupying the property.
   • However, in many other jurisdictions, landlords have a duty to deliver both legal and actual possession. If they don’t, they can be in breach of the lease.
3. Implication in the Given Scenario:
   • If the jurisdiction in question is one where delivering legal possession is sufficient, then choice B (“The landlord had delivered the legal right of possession to the tenant.”) could be the correct reason for the court to find in favor of the landlord. In this scenario, the landlord gave the tenant the legal right to the property, and any issue with the holdover tenant would be for the new tenant to resolve.
4. Variations by State:
   • Landlord-tenant law can vary significantly from one state to another. So, it’s important to consult the specific laws and precedents of a given jurisdiction to understand the obligations and rights of both landlords and tenants.

If you are dealing with a specific state or jurisdiction, I can try to provide more detailed information about its stance on the delivery of possession.

---

I know that NCBE aims to test majority or consensus rules, rather than minority ones, on its exams, so I asked GPT-4 if it knew which rule (actual or legal possession) a majority of states follow. GPT-4 answered:

---

• In the U.S., the majority rule is that landlords are obligated to deliver actual possession of the premises to the tenant at the beginning of the lease term. If the landlord fails to do so, they can be held in breach of the lease.

• This rule is more protective of tenants, ensuring that when they sign a lease (or enter into a valid oral agreement for shorter-term tenancies) and perhaps move their belongings, business equipment, etc., they are assured of actually being able to use and occupy the space without interference from a holdover tenant or any other unauthorized occupant. After all, one of the primary reasons for entering into a lease is to have a physical space to live or conduct business.

• However, as with all legal matters, nuances can exist. While the majority rule requires actual possession, there are states or jurisdictions where only legal possession is required, and the burden is placed on the new tenant to deal with holdovers.

• But again, the majority trend leans toward requiring landlords to deliver actual possession. If you’re dealing with a specific issue or jurisdiction, it would be essential to consult local landlord-tenant statutes and case law or seek legal counsel for detailed guidance.

---

I know better than to trust GPT-4, which is trained on the internet rather than on legal databases, as an authoritative source on legal principles. In my previous posts, I have illustrated simply how GPT-4 could help a new lawyer identify legal issues in a client problem, understand the general scope of those issues, and generate questions to ask a client about the problem. I would not rely on GPT-4 to tell me whether a rule is the majority rule–or whether a particular state follows that rule.

So I turned to WestLaw to unravel this knotty question of whether a majority of states allow the landlord to deliver only legal possession to a tenant. WestLaw quickly confirmed that GPT-4 was correct. An ALR annotation collecting cases suggests that eleven states allow the landlord to deliver only legal possession, while twenty require the landlord to deliver actual possession together with legal possession. Two thoughtful student notes affirm that the requirement of actual possession is very much the majority rule, with one (Heiser) referring to a “mass exodus” away from the rule that legal possession suffices. (See the end of this post for citations.)

Even the state that originated the more landlord-friendly rule, New York, discarded it by statute in 1962. New York’s Real Property Law Article 7, section 233-a now provides: “In the absence of an express provision to the contrary, there shall be implied in every lease of real property a condition that the lessor will deliver possession at the beginning of the term.”

If you’ve followed me down this rabbit hole of real property law, you’ve learned: (1) At least for this rule of law, GPT-4 accurately identified the majority and minority rules. It was also able to explain those rules concisely. (2) NCBE is using, as one of the few sample questions it has released for the NextGen exam, a question that tests an outdated, minority rule. I alerted a contact at NCBE about this situation in mid-September, but the question is still on the sample questions site.

What do these lessons teach us about using AI in entry-law practice? And what do they suggest about the bar exam? I will explore both those questions in upcoming posts. Spoiler alert on the second question: It’s easy to declare, “ha, NCBE is wrong!” but the lesson I draw from this is deeper and more complex than that.

References:

Implied covenant or obligation to provide lessee with actual possession, 96 A.L.R.3d 1155 (Originally published in 1979, updated weekly).

Christopher Wm. Sullivan, Forgotten Lessons from the Common Law, the Uniform Residential Landlord and Tenant Act, and the Holdover Tenant, 84 Wash. U.L. Rev. 1287 (2006).

Matthew J. Heiser, What’s Good for the Goose Isn’t Always Good for the Gander: The Inefficiencies of A Single Default Rule for Delivery of Possession of Leasehold Estates, 38 Colum. J.L. & Soc. Probs. 171 (2004).

AI, Bar Exam, GPT-4, NCBE