My Experience with Grok 3 Reveals It’s Not Worth the Increased Cost
Grok 3: A Closer Look at xAI’s Latest AI Chatbot
Earlier this week, xAI launched Grok 3, which the company touts as its most sophisticated artificial intelligence (AI) to date. With new reasoning capabilities and a feature called DeepSearch, they’ve branded it as the "world’s smartest AI." Elon Musk has praised Grok 3, claiming it outshines other AI models. So, does it truly live up to these ambitious proclamations?
Pricing and Value Proposition
Grok 3 comes at a price. Beyond a limited free trial, users are required to purchase a subscription, which costs either $40 per month for the X Premium+ or $30 for the SuperGrok package. Given these costs, many users are left wondering if Grok 3 offers enough value to justify the investment.
From personal testing and expert reviews, it appears that Grok 3 lacks groundbreaking advancements compared to existing AI models. While it has certainly evolved from its predecessor, Grok 2, many users find it doesn’t offer a compelling enough experience for the price.
Performance and Technical Specifications
Improvements Over Grok 2
Grok 3 is said to be more powerful, having been trained on 200,000 Nvidia H100 GPUs and utilizing over ten times the computational power of its predecessor. This increase in capability translates to faster responses suitable for daily tasks. However, while typical replies are prompt, the Think feature that provides more detailed answers often requires up to two minutes to generate responses.
Despite its processing power, Grok 3 still experiences "hallucinations"—a common issue among AI chatbots where incorrect or nonsensical information is presented as facts.
Benchmarks vs. Real-World Use
In internal tests conducted by xAI, Grok 3 reportedly performs better than most competitors, only trailing behind OpenAI’s upcoming o3 model. However, benchmarks alone do not fully represent user satisfaction. A truly great AI must also provide a well-rounded, mature experience that goes beyond mere numbers.
User Experiences and Expert Opinions
Varied Performance in Real Tasks
Field tests of Grok 3 revealed mixed results. AI expert Theo Browne evaluated its coding abilities and found Grok 3 lacking, as it struggled to execute a coding task without issues. Conversely, Andrej Karpathy noted that while Grok 3 demonstrated some competence, its performance was rather average and didn’t surpass existing solutions.
Users have reported that Grok can handle light tasks like researching products or learning about general topics. However, for more complex inquiries, other AI models often provide better results.
The “Based” Controversy
Musk emphasized Grok 3’s ability to deliver "based" opinions, a term often used to describe unfiltered, straightforward responses. Initial tests, however, revealed a more measured tone than anticipated. When users asked provocative questions, Grok returned balanced, sometimes even cautious, analyses rather than bold claims or judgments.
Interaction Examples
- When questioned about news outlets, Grok provided a neutral perspective, describing one popular tech publication as occasionally "niche" without offering any provocative critiques.
- In political inquiries, responses leaned towards general economic issues rather than engaging in heated debate.
While some may appreciate this nuanced approach, others expecting more spirited responses might find Grok’s demeanor uninspiring.
DeepSearch Capabilities
Grok’s DeepSearch feature aims to compete with similar tools from rivals like Perplexity AI by offering in-depth report generation. In user trials for trip planning and product searches, Perplexity often outperformed Grok, especially in terms of presentation and usability. Grok did redeem itself in some areas, such as generating suggestions for alternatives during a travel query.
Comparisons with Competitors
- Travel Planning: Both AI tools provided similar itineraries, but Perplexity delivered a more user-friendly layout.
- Shopping Recommendations: Grok failed to suggest locally available products, while Perplexity offered choices relevant to the user’s location.
Despite its faster response times, Grok’s DeepSearch feature seems to lack the same level of effectiveness and versatility found in established competitors.
Subscription Drawbacks
In terms of monetary value, Grok’s subscription model raises eyebrows. The $40 monthly fee is notably higher than the industry standard, which hovers around $20 for competitors like Gemini Advanced and ChatGPT Plus. As xAI continues to refine Grok 3, users may find better options that offer similar or superior capabilities without the hefty price tag. Currently, many users are hesitant to invest heavily in a service that might not surpass free or more affordable alternatives.
