Europe’s Response to DeepSeek: Collaborative Models, Unified Infrastructure, and Specialized Knowledge
The Emergence of AI in Europe
Introduction to DeepSeek
The introduction of DeepSeek in February 2023 has ignited conversations regarding Europe’s position in the global AI arena. While this large language model (LLM) has gained attention due to its competitive pricing and efficiency, its open-source nature and reliance on data stored in China have raised significant security alarms. In response, Italy has entirely banned DeepSeek, followed by restrictions from other European nations concerning its usage by government entities and contractors.
New Initiative: OpenEuroLLM Project
In the same month, the OpenEuroLLM project was launched with the objective of enhancing Europe’s AI competitiveness and digital independence. This initiative seeks to lower barriers for developing and enhancing AI products in the region.
This comes at a crucial time as stakeholders have been pondering which European companies can rival DeepSeek. Mistral AI from France and Aleph Alpha from Germany are frequently mentioned contenders. However, Aleph Alpha has transitioned from foundational LLMs to providing AI infrastructures tailored for various industries and government clients.
Both Mistral AI and Aleph Alpha are involved in the OpenEuropeLLM consortium. This group comprises 20 notable European research institutions and companies and operates under the coordination of Jan Hajič from Charles University in Czechia and co-leader Peter Sarlin from AMD Silo AI in Finland. The goal of this consortium is to create a series of effective, multilingual foundation models aimed at fulfilling demands in commercial, industrial, and public service sectors.
Notably, these models are designed in accordance with Europe’s strict regulatory standards, ensuring that they align with European values. The development process also emphasizes collaboration with open-source and open-science communities, allowing for fine-tuning to meet specific industry and public sector needs.
The Role of Smaller Language Models (SLMs)
Anita Schjøll Abildgaard, CEO of Iris.ai, advocates for a shift towards smaller, domain-specific models, termed SLMs. She believes that embracing these models and fostering open-source collaborations is crucial for advancing Europe’s AI vision.
As the demand for data center power in Europe could triple by 2030, exploring alternative AI approaches that do not rely on large models consuming extensive energy will be essential.
Abildgaard points out that while initiatives like the InvestAI plan and OpenEuroLLM indicate that progress is being made, there is still a significant need for stronger commitment and clarity regarding funding allocations and project timelines.
The Advantages of Small Language Models
SLMs rely on foundational models to produce high-quality training data for their systems. Therefore, for Europe to maintain competitiveness, it’s fundamental to possess both foundational and SLM capabilities.
SLMs are often more economical and can be tailored to better meet the specific requirements of businesses. Many actual business cases don’t call for the extensive capabilities of large models. Victor Botev, CTO and co-founder of Iris.ai, notes that distilling necessary knowledge into smaller models tends to be a more efficient and cost-effective strategy.
SLMs have particular effectiveness in areas that demand rapid decision-making, such as agent-based workflows where multiple small models work together. They can also be particularly beneficial in specialized fields like healthcare and chemistry.
Botev highlights the risk of "catastrophic forgetting" when training large models on complex tasks, such as understanding DNA structures. Smaller models allow for specialization without this drawback, making them a safer choice.
Besides, SLMs consume significantly less energy. Botev showcases a striking difference in resource requirements: a 1 billion parameter model can need up to 60,000 times fewer computational resources than a 400 billion parameter model. This aligns with Europe’s sustainability goals.
Leveraging Transparency and Collaboration
Botev emphasizes the burgeoning landscape of open-source collaboration. For instance, projects like DeepSeek have made open-sourcing techniques to build small models from large ones accessible, with over 10,000 distilled models now available on platforms like Hugging Face. Additionally, reinforcement learning is emerging as a viable method for customization.
Abildgaard advocates for European startups to adopt a similar approach to open-access publishing. She proposes that companies receiving EU funding should be encouraged to convert part of their work, especially foundational models, into open-source assets to foster greater collaboration and transparency.
The Need for More SLMs in Europe
Despite the noted advantages, the development of SLMs in Europe has been somewhat slow. One noteworthy company is Malted AI based in Scotland, which specializes in converting the outputs of large models into more manageable smaller ones. Their technology allows businesses to utilize SLMs that address specific domain challenges while achieving substantial cost savings.
Rather than solely focusing on scalability, Europe has the opportunity to lead by promoting smart, open, and sustainable AI solutions. By investing in foundational models along with the supporting ecosystems, Europe can create an environment that encourages agile development, cooperation, and finetuning. The path ahead likely involves nurturing the smallest, most effective AI models available.
