California Bar Reveals Use of AI in Creating Questions for Challenging February Exam
The Role of AI in California’s Recent Bar Exam Controversy
Background on the Situation
In an unexpected announcement, the State Bar of California revealed that several multiple-choice questions on a recently administered bar exam were created with the assistance of artificial intelligence. This revelation has raised concerns and criticism from various stakeholders, particularly regarding the integrity and reliability of the examination process.
February 2025 Bar Exam Issues
The February 2025 bar exam was marred by numerous technical issues. Many candidates experienced significant difficulties, such as:
- Online platforms crashing, which hindered test-takers from starting their exams.
- Problems with saving and submitting essays, leading to incomplete applications.
- Lagging screens and error messages that disrupted the testing experience.
- Inability to copy and paste text, causing additional frustrations for examinees.
These challenges led to numerous complaints from candidates, prompting the State Bar to reconsider the scoring of the exam results.
Examination Question Development
A notable aspect of this situation is the methodology behind the creation of exam questions. According to a presentation by the State Bar:
- Out of the 171 scored multiple-choice questions, 100 were developed by Kaplan Exam Services.
- 48 questions came from a first-year law students’ exam, raising questions about their relevance.
- Importantly, 23 questions were crafted by ACS Ventures, the State Bar’s psychometrician, utilizing AI technology.
This reliance on AI to draft exam questions has drawn mixed reactions from legal educators and professionals.
Reactions from Legal Experts
Experts have expressed profound concern over the use of AI in developing exam questions. Mary Basick, an assistant dean at the University of California, Irvine, expressed her disbelief, stating, “Having the questions drafted by non-lawyers using artificial intelligence is just unbelievable.” Such sentiments highlight the apprehension surrounding the reliance on AI for such critical assessments.
Katie Moran, an associate professor specializing in bar exam preparation at the University of San Francisco, pointed out that the State Bar’s admissions raise significant ethical questions. She stated, “The State Bar has admitted they employed a company to have a non-lawyer use AI to draft questions that were given on the actual bar exam. They then paid that same company to assess and ultimately approve of the questions on the exam, including the questions the company authored.” This raises concerns about conflicts of interest and the overall validity of the exam.
Ensuring Exam Validity
In response to the controversy, Leah Wilson, the executive director of the State Bar, attempted to reassure the public, asserting confidence in the validity of the multiple-choice questions. She emphasized that the questions were designed to accurately and fairly assess the legal competence of test-takers. However, the reliance on AI has sparked a debate about the future of standardized testing in the legal field.
Potential Adjustments to Exam Results
Following the October 2024 exam debacle and the misuse of AI, the State Bar is planning to ask the California Supreme Court for adjustments to the exam scores of those who participated in the February exam. This move indicates an acknowledgment of the complications and possible unfair advantages or disadvantages caused by the flawed testing conditions.
Conclusion: The Path Forward
As the legal community grapples with these developments, it remains to be seen how the State Bar will handle these changes and what measures will be put in place to prevent similar issues in the future. The role of technology, especially artificial intelligence, in high-stakes testing like the bar exam is likely to be a focal point in discussions about legal education and licensure moving forward.
