Researchers Aim to Create a More Open Version of DeepSeek’s AI Reasoning Model
Hugging Face’s Response to DeepSeek’s R1 AI Model
Less than a week after DeepSeek introduced its R1 "reasoning" AI model, researchers from Hugging Face have embarked on a project to replicate this model. They view this endeavor as a quest for "open knowledge."
The Open-R1 Initiative
Leandro von Werra, the head of research at Hugging Face, along with a team of engineers, has initiated the Open-R1 project. This initiative aims to duplicate DeepSeek’s R1 model and make all related components freely available, including the data necessary for training the model.
The motivation behind this project stems from concerns regarding DeepSeek’s "black box" approach. While technically R1 has an open license allowing for its deployment without major restrictions, it doesn’t qualify as "open source" in the conventional sense. Many of the instruments involved in crafting R1 remain undisclosed, making it challenging for other researchers to understand and replicate its workings.
The Need for Transparency
Elie Bakouch, a Hugging Face engineer involved in the Open-R1 project, criticized the lack of transparency from DeepSeek. He stated, “The R1 model is impressive, but there’s no open dataset, experiment details, or intermediate models available, which makes replication and further research difficult.” By fully open-sourcing R1’s architecture, Hugging Face believes they can enhance the model’s potential and foster deeper research.
DeepSeek and R1’s Prominence
DeepSeek, a Chinese AI lab partially backed by a quantitative hedge fund, launched R1 to considerable acclaim. The model has shown impressive performance, surpassing OpenAI’s o1 reasoning model in various benchmarks. As a reasoning model, R1 can effectively verify its outputs, helping to mitigate common mistakes made by typical AI models. Though reasoning models generally take longer to generate results, they prove to be more reliable in fields like physics, science, and mathematics.
R1 gained notable attention when DeepSeek integrated it into a chatbot application that quickly climbed the Apple App Store rankings. The rapid development of R1 has prompted discussions among analysts regarding the U.S.’s position in the AI landscape.
The Drive for Open Knowledge
The focus of the Open-R1 project is not solely on competing with U.S. AI innovations but rather on enhancing openness in AI model training. Bakouch emphasized that the absence of training code or instructions accompanying R1 hinders in-depth studies and understanding of the model’s behavior. He stated, “Having control over the dataset and process is critical for deploying a model responsibly in sensitive areas. It also helps with understanding and addressing biases in the model."
Path to Replication
Hugging Face aims to replicate R1 within a few weeks by leveraging its Science Cluster, which boasts 768 Nvidia H100 GPUs. The team intends to create datasets similar to those that DeepSeek used for R1. They are also seeking support from the AI community on platforms like Hugging Face and GitHub, where the Open-R1 project is being hosted.
Von Werra noted, “We need to ensure that we implement the algorithms and recipes correctly… it’s a community effort that benefits from multiple perspectives.”
Community Engagement and Interest
The Open-R1 project has generated significant interest, gathering 10,000 stars on GitHub within just three days. This metric reflects users’ appreciation and acknowledgment of the project’s potential.
If successful, the Open-R1 initiative could empower researchers to develop the next generation of open-source reasoning models. Bakouch expressed hope that Open-R1 will not only replicate R1 but also serve as a stepping stone for creating even better models in the future. “Rather than being a zero-sum game, open source development benefits everyone, including frontier labs and model providers, as they can all utilize the same innovations.”
While there are concerns regarding the potential misuse of open-source AI, Bakouch believes the advantages outweigh the risks. Once the R1 model is replicated, anyone with access to the necessary resources can create their own R1 variant, thereby expanding the technology’s reach. He emphasizes the importance of recent open-source releases, stating that they are shifting the narrative in AI from being dominated by a few labs to a more collaborative and accessible approach.
