Inquiries Made to Meta AI, ChatGPT, and Claude Regarding the Trump Rally Shooting
Reliability of AI Chatbots for News Reporting
Recent events have raised questions about the reliability of artificial intelligence (AI) chatbots as sources for current news. On July 13, there was an assassination attempt on former President Donald Trump, and the generative AI chatbot from Meta faced criticism for its inadequate responses to inquiries regarding this incident. In a blog post, Meta explained that their AI chatbot had been programmed not to address questions about ongoing or developing news situations, as these could lead to misinformation.
Meta’s Response to Criticism
Joel Kaplan, Meta’s Vice President of Global Policy, acknowledged that AI chatbots, including Meta AI, often struggle with providing accurate information on breaking news. Kaplan noted that incidents of incorrect responses, called “hallucinations,” are not exclusive to Meta AI and are a widespread issue within the field.
Despite speculation that AI chatbots might replace search engines like Google, which has integrated AI features, these tools are primarily based on large language models that may not stay current with recent events. Following the backlash about Meta AI’s inaccurate and insufficient information concerning the shooting, the company modified the chatbot to now offer a brief summary of the incident.
Testing Other Chatbots
To further examine the reliability of AI chatbots, Business Insider tested Meta AI alongside ChatGPT and Anthropic’s Claude by asking them similar questions about the July 13 incident.
Meta AI’s Performance
After receiving critical feedback, users reported that Meta AI was initially evasive and provided incorrect details about the shooting. Kaplan mentioned that the chatbot was initially designed not to provide responses about the event to avoid spreading misinformation. He stated that Meta has since updated the chatbot to accurately summarize the situation surrounding the assassination attempt.
After the update, when asked about the shooting, the Meta AI provided a concise summary, identifying the shooter and outlining the outcome of the incident, indicating a move towards increased accuracy.
ChatGPT’s Responses
ChatGPT’s performance, however, was less accurate. In response to the same questions, it incorrectly fabricated an unrelated event and denied the existence of any assassination attempt. It claimed, “There is no evidence or credible reports indicating that Donald Trump was almost assassinated in Butler, PA, or at any recent event.”
Even though ChatGPT included disclaimers stating that it might not be fully up-to-date or accurate, its significant inaccuracies raised concerns. A spokesperson for OpenAI admitted that large language models have limitations, especially when it comes to real-time news, but expressed hope that AI technologies could evolve into reliable news sources in the future.
Claude’s Approach
Unlike the other AI chatbots, Anthropic’s Claude did not make any claims about the shooting. It acknowledged that it might not have the latest information due to its last knowledge update being on April 29. Claude advised users to consult reputable news sources for the most current information about significant events.
Overall, the varying capabilities and limitations of these AI chatbots highlight the challenges they face when responding to rapidly evolving news events. The adjustments made by Meta and other companies reflect an ongoing effort to enhance the reliability of AI in delivering accurate information.
