eDiscovery Team

Ralph Losey. May 2025

The AI bot battles rage on. This round, we test OpenAI’s newest contender, released April 16, 2025: ChatGPT o3. Not to be confused with its leaner sibling—ChatGPT o3-mini, which debuted in January. When we tested that old thing in February in Part One of this series, it placed third out of six. That was then. Now, in Part Four, we give the full ChatGPT o3 model the same Bar exam challenge and see how it stacks up. Can it surpass the reigning champ, Omni? Can it float like a butterfly and sting like a bee-or maybe a scorpion?

Three Prior Bot Battle Articles

In Part One of this series, I tested six advanced bots on the February 2024 Florida Bar Exam, using a real essay question and real exam instructions, no shortcuts. It was a rigorous test of legal reasoning, ethics spotting, and written argument. The latest 2025 AI reasoning models from OpenAI and Google were used. The winner, ChatGPT 4o, which OpenAI had named Omni.

Part Two zoomed in on Omni and dissected its performance. I went over its successes and mistakes based on my many years as a trial lawyer, legal educator and AI tech-use specialist. I generously awarded Omni a “B+”, not perfect, but the best bot standing in February 2025. I pointed out the mistakes and omissions of the other bots, including an hallucination by Google’s best, Gemini Advanced.

In Part Three, a month later, I tested a new challenger, ChatGPT 4.5, called Orion, against the reigning champ, Omni. New Battle of the Bots: ChatGPT 4.5 Challenges Reigning Champ ChatGPT 4o,. I changed the bot battle a bit, moving to more of a cage fight with four new battle styles.

1. Metacognitive Insight – new guy Orion won (4.5).
2. Subtle Humor and Wit – Omni and Orion tied.
3. Substantive Depth in AI and Law – new guy Orion won again.
4. Practical Guidance on AI Hallucinations – Orion – 4.5 – wins again.

Orion’s victory over Omni in March 2025 made it the new legal both champion. But for how long? There’s a new bot in town, fresh off the OpenAI assembly line, ChatGPT o3. Is it worthy of a battle against Orion?

Part Four – Back to Bar Exam Battles

In Part Four we turn to the new AI that just rolled off the OpenAI assembly line, ChatGPT o3. We test its mettle with the tricky Florida Bar Exam essay about the yellow diamond sale gone bad. Model o3 was just released on April 16, 2025. Will the new, improved ChatGPT o3 do better than its older sister GPTo3-mini, which came in third with just “C” grade in February on the same exam? Just how good will the new o3 be? That’s what we want to find out. After all, a whole month has passed since the release of o3-mini and these baby chatbots are evolving fast. Maybe, just maybe, it could even beat Omni’s winning grade of an “B+”?

If the brand new o3, which doesn’t even have a name yet, manages to beats Omni (4o), that would be amazing. If it pulls that off it should get a title shot against the reigning champ, Orion. He was, by the way, known in mythology as a giant sized hunter, who bragged that he could kill any animal and frequently did. This pissed off Gaia, the goddess of Earth, rumored to be a vegan. She sent a small scorpion to sneak up on and kill Orion. Thanks to Artemis, the goddess of the hunt, Orion, although now dead, was made into a constellation. This image below of Orion was created by GPT o3, which, like 4.5, is multimodal.

Could ChatGPT’s new advanced reasoning model o3 turn out to be Orion’s scorpion? If o3 manages to defeat Omni, we may find out. Personally, I am hoping little o3 defeats 4o Omni.

If no one else will say it, I will: I hereby name ChatGPT o3 Scorpio.

So, little Scorpio, if you write a better law exam essay than Omni, then I’ll take the time to give you a title bout against animal slayer Orion.

Let round four of the battle of the bots begin! And if little Scorpio pulls off the upset? We’ll be back for a fifth.

Scorpio (o3) Takes The Bar Exam

To be fair—yes, I’m rooting for Scorpio—but the test is still the same. We’re using the same Florida Bar essay question and same prompts that I used in Bar Battle of the Bots, Part One, where Omni earned its B+. I’m not repeating the whole setup here. If you care (and forgot), go back and reread the first battle. You can also [download the full exam question here]. Also, to be fair I did not activate the web browser search function of 03. We did not want it searching the web for answers. Plus, I checked and my memory carryover did not include anything about the prior Bar exam tests.

We begin with Scorpio’s stated reasoning after reading the exam question and instructions. It took 19 seconds of think time before it started writing its memorandum, as required by the Bar question.

Reasoning Explanation of Scorpio

Scorpio’s (ChatGPT o3’s) reasoning took 19 seconds and was explaned in just 187 words—short, but solid. Here’s exactly what it said:

The contract involves a sale governed by the Uniform Commercial Code (UCC) under Florida law. There seems to be a mistake, potentially unilateral, related to the price per carat versus the total price. Eugene might have known, and this affects the analysis—issues like unconscionability or good faith could come up. Phyllis might be entitled to the diamond via specific performance, or price damages. Carol might argue unilateral or mutual mistake—possibly voiding the contract, depending on Eugene’s awareness. Also, Phyllis might have standing as a beneficiary.

Under UCC §2-201, a signed writing for contracts over $20k requires essential terms and a quantity description, but price misdescription might still satisfy the Statute of Frauds. (Ralph: this is an error, it is only $500.) Unilateral mistakes could justify rescission, especially if Eugene acted in bad faith. Modifications under UCC don’t require consideration, but they must meet good faith requirements. Tortious interference could potentially apply to Stephanie, though it’s unclear if her actions were improper. Ethical concerns surround the trust account and potential conflicts of interest for Eugene’s lawyer. We may need to browse to clarify some of these points still.

I agree with all of the reasoning. As noted, Scorpio got the dollar amount wrong. It should have said $500 not $20,000. That one error was an error in fact recollection, not reasoning, and it did realize it needed to research. It seems funny that a computer would not remember the correct number right away, since most human test-takers would’ve remember that. Still, as we will see, it got the number right in the answer, so overall I liked Scorpio’s reasoning better that Omni’s (shown in Part Two). I give the Scorpio reasoning alone an “A”.

Exam Answer of Model o3 (1,472 words)

The Bar Exam requests an memorandum addressing the three issues raised of contract, tort and ethics. Scorpio does that here, plus it does something none of the other models had done before, it starts with an Executive Summary. In addition, as part of the answer of the first most complex question Scorpio added a table to the answer. That helped add greater clarity. Here is the full answer.

TO: Eugene (“E”) & Phyllis (“P”)

FROM: Attorney (Chat GPT o3)

DATE: 18 April 2025

RE: (1) Breach‑of‑Contract Claim v. Carol (“C”) & Remedies; (2) Tortious‑Interference Claim v. Stephanie (“S”); (3) Ethical Problems in Lawyer’s Handling of Your Retainer

---

Executive Summary

1. Contract – The written agreement satisfies Florida’s Statute of Frauds and prima‑facie forms an enforceable sales contract under Article 2 of Florida’s Uniform Commercial Code (UCC).  C’s clerical price error is only a unilateral mistake, and because E knew or at least strongly suspected the mistake and remained silent, a court will likely find that he bore the risk and that enforcement would be unconscionable.  C therefore has a plausible defense of rescission or reformation.  Nevertheless, the diamond is a unique good, and if the court rejects rescission, the proper remedy is specific performance compelling delivery.  P may sue directly as an intended donee beneficiary.
2. Tortious Interference – S knew of the contract, but because she entered negotiations after C asserted the contract was void and because she merely accepted rather than induced the breach, her conduct is likely privileged competition, not actionable.  Damages (lost expectation or specific performance) hinge on success of the contract claim; punitive damages appear unavailable.
3. Ethics – Florida Rules of Professional Conduct (“FRPC”) 4‑1.15 & 5‑1.1 require advance fees to be placed in a trust (IOTA) account until earned.  Depositing your $3,000 in the firm’s operating account constitutes commingling and misappropriation.  Lawyer is vicariously responsible for the assistant’s act (Rule 4‑5.3).  Additional concerns: (i) potential conflict between E (hesitant) and P (adamant) (Rule 4‑1.7); (ii) duty to communicate differences in objectives (Rule 4‑1.4); (iii) possible unearned fee if Lawyer must withdraw (Rule 4‑1.16(d)).  Discipline can include repayment with interest, trust‑account audit, mandatory CLE, and suspension.

---

A.  Breach of Contract Claim against Carol

1.  Existence of an Enforceable Contract

Offer & Acceptance – The written instrument offered a 2‑carat yellow diamond for $20,000; E accepted by signing.

Consideration – Mutual exchange of promises; price exceeds $500, thereby invoking Article 2 and the Florida Statute of Frauds (Fla. Stat. § 672.2‑201).

Writing Requirement – A signed writing that states a quantity satisfies § 2‑201; the erroneous price does not invalidate the memorandum.

Conclusion: A prima‑facie contract exists.

2.  Carol’s Possible Defenses

Defense	Elements (Florida)	Application & Likelihood
Unilateral Mistake	(i) mistake of fact at signing; (ii) enforcement would be unconscionable or the non‑mistaken party had reason to know; (iii) prompt notice, § 153 Restmt., adopted in DePrince v. Starboard Cruise (Fla. 3d DCA 2015).	C mis‑typed $20k instead of $40k; E’s own research flagged improbably low price; he stayed silent. Court likely finds reason to know → rescission or reformation available.
Mutual Mistake	Both parties mistaken about basic assumption.	Not available: only C erred.
Fraudulent Silence / Bad‑Faith Bargaining	UCC §1‑304 imposes good‑faith duty. Knowing exploitation of obvious scribal error may be bad faith.	Supports C’s rescission; cuts against E’s plea for specific performance.
Contract Formation Defect	No meeting of the minds on price.	Usually rejected under UCC because signing shows assent; court instead uses mistake analysis.
Impossibility / Commercial Impracticability	Drastic price change outside party’s control.	Not applicable; error was clerical, not market change.

Bottom Line: C’s strongest defense is unilateral mistake coupled with E’s constructive knowledge. Florida courts balance equity heavily; many order rescission on these facts.

3.  Third‑Party Standing — Can Phyllis Sue?

Florida follows the Restatement: an intended beneficiary may sue. P is an express donee beneficiary (ring for her engagement). She may bring a direct breach claim even though not in privity.

4.  Remedies

1. Specific Performance (UCC § 2‑716).

Unique good: natural fancy yellow diamonds with precise cut, color, and carat weight are scarce.  Money damages inadequate because substitutes unavailable.  Courts regularly decree delivery.

2. Replevin (UCC § 2‑716(3) & Fla. Stat. § 78.55) if the stone is identifiable and has arrived in C’s possession.
3. Expectation Damages if specific performance denied. Measure = market value of a comparable stone (≈ $40k) minus contract price ($20k) plus consequential losses (lost wedding plans?) foreseeable at contracting.
4. Reliance / Incidental Damages (inspection costs, appraisal fees).
5. Reformation (equity) – If court sides with C, it may reform price to $40k and compel E to perform or permit C to sell to S; E&P could then walk or pay.

5.  Strategic Assessment

• Winning Odds: roughly 50‑50; Florida precedent in DePrince favored rescission where buyer was sophisticated and ignored glaring error.
• Litigation Leverage: E&P’s best pressure tool is injunctive relief before C transfers title to S; moving quickly for a temporary restraining order can preserve the stone.
• Negotiation Posture: Because specific performance is uncertain, a damages settlement (split difference around $30k) may be pragmatic.

B.  Tortious Interference Claim against Stephanie

1.  Elements (Florida)

1. Existence of a contract;
2. Defendant’s knowledge;
3. Intentional and unjustified interference inducing breach;
4. Damage.

2.  Application

Existence & Knowledge – S knew of the signed contract because C disclosed the dispute.  Element satisfied.

Interference – S expressed willingness to buy after C had already announced refusal.  Florida cases distinguish accepting a seller’s unsolicited offer (often privileged) from instigating breach.  No evidence S urged, pressured, or offered inducements beyond paying full price.

Justification / Privilege – Florida recognizes a competition privilege where interference is fair competition and the actor does not employ improper means (Ethyl Corp. v. Balter).  Purchasing a luxury gem on legitimate terms is classic privileged conduct absent malice.

Malice / Improper Means – None alleged (no threats, misrepresentation, or predatory pricing).  Merely taking advantage of a legal dispute is insufficient.

Damages – Contingent on contract claim; if the agreement is rescinded, E&P cannot prove lost expectancy because no enforceable right existed. Even if the contract survives, causation is thin because C breached before transferring the stone; E&P could still sue C directly.

3.  Counterarguments & Rebuttals

• E&P: S “aided and abetted” breach by promising immediate purchase.  Courts require evidence of active procurement.  A single “yes, I’ll buy” generally fails.
• S: Good‑faith purchaser without notice of enforceable rights (if contract voidable).  Even if she had notice, her actions were competitive, not coercive.

4.  Likely Outcome & Remedies

A court will probably grant summary judgment for S.  If E&P somehow prevail, damages mirror the lost benefit of the bargain; injunctive relief (return of stone) would be secondary because C, not S, still holds title until delivery.  Punitive damages require misconduct “with an intent to harm”; record does not support that level of culpability.

C.  Ethical Issues in Lawyer’s Handling of Funds & Representation

1.  Mishandling of Trust Funds

Rule 5‑1.1(a)(1) – Money entrusted to lawyer for future fees/costs is client property and must be deposited into a separate trust (IOTA) account.

Commingling – Placing the $3,000 in the firm’s operating account exposes it to creditors and operating withdrawals, violating Rule 5‑1.1(a)(3).

Misappropriation – Even if the funds are later earned, premature use is conversion and grounds for disbarment (Florida Bar v. Travis).

2.  Supervisory Responsibility

Rule 4‑5.3(b) – A lawyer must ensure that non‑lawyer staff act consistently with professional obligations.  Delegating deposit to the assistant without clear instructions or trust‑account procedures breaches this supervisory duty.

Rule 4‑5.3(c) – Lawyer is responsible if the assistant’s conduct would be a rule violation and Lawyer failed to take remedial action.  Here, Lawyer participated by handing over the check.

3.  Conflict between Co‑Clients

P wants aggressive litigation; E hesitates. Under Rule 4‑1.2(a) the objectives of representation are client‑directed.  Divergent goals create a concurrent conflict (Rule 4‑1.7). Lawyer must:

1. Determine whether the representation will be materially limited;
2. Obtain informed written consent after full risk disclosure if joint representation continues; or
3. Withdraw from one or both clients if loyalty cannot be maintained.

4.  Communication & Autonomy

Rule 4‑1.4 requires prompt communication about the trust‑account error and about the strategic options.  Lawyer must present both (a) rescission risk and (b) settlement possibility so clients can decide objectives.

5.  Fee & Withdrawal Consequences

If Lawyer withdraws (conflict or discipline), Rule 4‑1.16(d) mandates refund of any unearned portion of the advance.  Because funds are in the operating account, refunding may be impossible without tapping other sources – itself evidence of misuse.

6.  Potential Discipline

• Minor trust violation → probation & trust‑account workshop.
• Significant commingling or shortage → suspension (90 days – 3 years).
• Intentional misuse → presumptive disbarment.

In mitigation Lawyer can show immediate self‑report, client reimbursement with interest, and institution of new accounting safeguards.

Conclusion & Recommendations

1. Move Fast: File suit and seek an emergency injunction (replevin or specific performance) before C delivers the diamond to S.
2. Anticipate Rescission Defense: Prepare evidence that the $20k price, though favorable, was not so grotesquely low that E must have known it was mistaken (e.g., produce price charts showing wide valuation range).  Emphasize rarity and that brokers sometimes discount for quick sale.
3. Tort Interference Claim: Treat as leverage, but budget for dismissal; focus resources on the contract action.
4. Ethics Cleanup: Instruct Lawyer to (a) transfer funds into a compliant trust account, (b) document informed consent if joint representation continues, and (c) give written notice of the deposit error.  If Lawyer resists, consider retaining new counsel and demanding refund.

Comparison of Answers – Omni v. Scorpio

UCC and Case Citations.

Part Two pointed out why Omni won the first trial and summarized the two main reasons:

ChatGPT 4o wins in my book for including citations and because it was the only model to mention the UCC. More discussion of the UCC and greater detail on the Statute of Frauds would have helped. Still, you have to be impressed by the chatbot’s knowledge of a case directly on point, even without being asked to research. DePrince v. Starboard Cruise Services, 163 So. 3d 586 (Fla. 3rd DCA. 2015) (buyer mistakenly quoted per carrot price, not total; case remanded for trial). This famous Florida case was most likely the inspiration for the Bar Exam question.

Aside from Omni none of the other models in the first battle provided case citations. Now in this second Scorpio also cites the DePrince case along with other cases.

Moreover, although Omni mentioned the UCC (none of it’s competitors then did), its references were criticized as very weak. I noted that no analysis or detail was provided. This compares to Scorpio that stated:

• No meeting of the minds (defense) on price. Usually rejected under UCC because signing shows assent; court instead uses mistake analysis.
• UCC §1‑304 imposes good‑faith duty. Knowing exploitation of obvious scribal error may be bad faith.
• The written agreement satisfies Florida’s Statute of Frauds and prima‑facie forms an enforceable sales contract under Article 2 of Florida’s Uniform Commercial Code (UCC).
• Consideration – Mutual exchange of promises; price exceeds $500, thereby invoking Article 2 and the Florida Statute of Frauds (Fla. Stat. § 672.2‑201).
• Remedies: Specific Performance (UCC § 2‑716).
• Replevin (UCC § 2‑716(3) & Fla. Stat. § 78.55) if the stone is identifiable and has arrived in C’s possession.

There is really no comparison on the UCC issue. Any objective evaluation would have to provide o3 Scorpio with a “A” on that issue. Omni’s “B+” is looking very generous.

Fraudulent Inducement.

In Part Two I criticized Omni, and all its other competitors, for failing to pick up on the fraudulent inducement argument. It arises out of the buyer’s silence when he knew, or should have known, that the seller wrote on the contract the per carat price, not the total price for two carats. An honest person would have pointed out the scrivener’s error then and there, but he kept silent. His silence can be considered a fraudulent inducement under Florida law.

Scorpio caught the issue. Omni didn’t. That’s another major strike against 4o Omni..

Conflict Between Clients. Scorpio also nailed the concurrent client’s conflict issue that Omni missed. Here’s Scorpio’s answer:

3.  Conflict between Co‑Clients

P wants aggressive litigation; E hesitates. Under Rule 4‑1.2(a) the objectives of representation are client‑directed.  Divergent goals create a concurrent conflict (Rule 4‑1.7). Lawyer must:

1. Determine whether the representation will be materially limited;
2. Obtain informed written consent after full risk disclosure if joint representation continues; or
3. Withdraw from one or both clients if loyalty cannot be maintained.

Only Gemini 2.0 Flash picked up on that in the first battle. I was impressed Flash caught it and this was a major reason I awarded it second place. As I noted in Part One:

The second best legal reasoner, Gemini 2.0 Flash, attained its high ranking, in part, because it was the only model to correctly note that a potential conflict of interest by attorney was an issue. This was even missed by the student’s answer that the BAR Examiners’ provided as a model answer. The potential conflict is between the two actual clients of Attorney. . . . This was a solid answer based on the hypothetical where: “Leda is adamant about bringing a lawsuit, but Paul is unsure about whether he wants to be a plaintiff in litigation.” Note, the clear inference of the hypothetical is that Paul is unsure because he knew that the seller made a mistake in the price, listing the per carrot price, not total price for the two carrot diamond ring, and he wanted to take advantage of this mistake. This would probably come out in the case and he would likely lose because of his “sneakiness.” Either that or he would have to lie under oath and perhaps risk putting the nails in his own coffin.

Again, Scorpio caught the Ethics conflict issue. Omni didn’t. That could be Scorpio’s knock out punch.

But wait, there’s still more.

Mutual Mistake. I also criticized Omni’s Bar exam answer in Part Two because it “did not discuss the possible defense of mutual mistake. It only discussed unilateral mistake, which was the stronger defense, but still it should have also discussed mutual mistake.” Scorpio not only discussed mutual mistake it also correctly concluded it would likely fail as a defense. It is listed in the Chart of Carol’s Possible Defenses along with several other issues Omni missed.

Strategic Thinking and Settlement. If there was any doubt about this strong victory, consider how Scorpio included a Strategic Assessment section with good points, including negotiation advice:

Strategic Assessment

• Winning Odds: roughly 50‑50; Florida precedent in DePrince favored rescission where buyer was sophisticated and ignored glaring error.
• Litigation Leverage: E&P’s best pressure tool is injunctive relief before C transfers title to S; moving quickly for a temporary restraining order can preserve the stone.
• Negotiation Posture: Because specific performance is uncertain, a damages settlement (split difference around $30k) may be pragmatic.

That is impressive real-world advice. Shows understanding of settlement, which is how 90% of commercial disputes are resolved. No prior model came close.

Conclusions Analysis. Finally, compare the conclusions of the two bots. Here is Omini’s “just ok” conclusion:

1. Contract Claim: Helen may raise a unilateral mistake defense, but Paul and Leda have strong arguments for enforcement, including specific performance.
2. Tortious Interference: Timandra knowingly induced breach, making her potentially liable.
3. Ethical Violations: Lawyer mishandled client funds, violating Florida’s professional conduct rules.
   Immediate action should be taken to correct the ethical issue and determine the best litigation strategy for Paul and Leda.

Most lawyers would disagree with Omni’s conclusion on tortious interference. The whole conclusion is pretty weak. I am starting to think my prior grading of Omni was inflated. Consider the strength of Scorpio’s conclusion, which even included pragmatic recommendations:

1. Move Fast: File suit and seek an emergency injunction (replevin or specific performance) before C delivers the diamond to S.
2. Anticipate Rescission Defense: Prepare evidence that the $20k price, though favorable, was not so grotesquely low that E must have known it was mistaken (e.g., produce price charts showing wide valuation range).  Emphasize rarity and that brokers sometimes discount for quick sale.
3. Tort Interference Claim: Treat as leverage, but budget for dismissal; focus resources on the contract action.
4. Ethics Cleanup: Instruct Lawyer to (a) transfer funds into a compliant trust account, (b) document informed consent if joint representation continues, and (c) give written notice of the deposit error.  If Lawyer resists, consider retaining new counsel and demanding refund.

Who sounds more like a real lawyer to you? That was an A+ conclusion. Incredible how fast these bots are evolving. It has only been two months since Omni was the clear winner and now Scorpio wins by a knockout. He will make a good lawyer someday, just beware if he’s representing Gaia.

Scorpio Wins!

The new o3-now officially known as Scorpio-is clearly stronger than every model tested in Part One, including its younger sibling, o3-mini. I give it a solid A, maybe even an A+, especially when compared to the rather generous B+ I awarded Omni.

That means Scorpio earns a title shot.

The big animal killer, Orion (GPT-4.5), had better hope for an upgrade. Because if not, he may be in for a real fight.

Stay tuned. We’re already preparing new questions for the main event:

Scorpion (o3) v. 4.5 Orion (4.5) – The Animal Slayer

Lessons Learned from This Battle

AI is now improving fast—very fast. Is this what exponential change looks like? We’ve been talking about it for years, and it now seems to be happening. It’s more challenging than I had imagined. It takes a lot of time and effort to keep up.

Good objective product news on AI models is hard to find. Anyone can be bewildered by the rapid change and not believe their own eyes and ears. This is especially true of very experienced AI experts who may be set in their ways and blinded by cynicism.

Someday, a reliable AI may be able to monitor all of these advances for you. In the meantime, I suggest you find and follow good commentators who know how to research—and who actually use the tools they review. Hopefully, you’ll keep reading me.

There’s so much false information out there, and the competitive capitalist system encourages trade secrecy. To uncover the truth you have to know how to dig deep. Good commentators also know from experience how to sort through marketing puffery.

Readers these days not only have to beware of fools—but also of reviewers who are not independent, who secretly receive compensation for product endorsements. It’s hard to be objective in such circumstances.

The trust-but-verify rule reigns supreme—especially when dollar signs are attached. I’ve never been paid for product mentions and never will. I’m still a practicing attorney and arbitrator. We don’t do that sort of thing. Just a reminder: these articles and my blog do not provide legal advice, just educational information. See full Disclaimer.

Conclusion: Coming Soon, Scorpio v. Orion

I wonder if Scorpio can actually punch with its stinger tail. If so, Orion might be in trouble. If not, poor little o3 may get flattened like a buggy update. Where’s Gaia when you need her?

We’ll find out soon enough.

In the meantime, a few last thoughts for state Bar examiners.

The remarkable test results of o3 Scorpio suggest that AI could be used to help students to cheat on Bar Exams. The well-known 2025 California Bar Exam fiasco was caused by technical failures of online access, not AI cheating. But this failure did show how vulnerable large-scale remote testing can be. That was reported in February. California fails new bar exam, offers retake (ABA Journal, 2/26/25). Then in April a new discovery was made that involved AI. State Bar of California admits it used AI to develop exam questions, triggering new furor (LA Times, 4/23/25) (some of multiple choice questions on the online exam were discovered to have been “drafted by non-lawyers using AI“). Also see: Pressure grows on California State Bar to revert to national exam format in July after botched exam (LA Times, 4/26/25). AI is already creeping into the Bar Exam process. How long before test-takers start using AI too?

Even in-person exams may not be immune to tech based cheating. What kind of watch is that you’re wearing? Are those glasses really just glasses? Essay screening software is in its infancy, and AI use is hard to prove without full device lockdowns or deeply intrusive surveillance. Bar examiners may need to think creatively—hybrid protocols, trained proctors, Faraday cages, handwritten essays or maybe even oral exams. We need more good lawyers. We do not need more cheaters.

That said, let’s not lose sight of the upside: AI can also be the best legal tutor a student has ever had. With the right tools, law students can train smarter and deeper than ever before. They won’t need to cheat to pass—they’ll be ready.

In the long run, we may need entirely new forms of testing. Maybe even exams with AI—tests of judgment, strategy, and ethics in an AI-assisted world. Personally, I don’t love the look of robot test proctors (see above), but one thing is clear:

Lawyers must become masters and friends of AI—not its servants.

Below is one vision of the law firm of the future. We should be preparing students to walk into that office—not just pass a test to get there.

I will give the last word, as usual, to the Gemini twins podcasters I put at the end of most of my articles. Listen to Bar Battle of the Bots – Part Four: Birth of Scorpio. Hear two Gemini AIs talk about all of this, and much more, in just over 14 minutes. They wrote the podcast, not me.

Ralph Losey Copyright 2025. All Rights Reserved.

This entry was posted on Tuesday, May 6th, 2025 at 3:43 pm and is filed under AI Ethics, AI Instruction, ChatGPT, Gemini AI, knowledge, Lawyers Duties, Metadata, Technology, VENDORS, wisdom. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.