This news is very important for the world of Artificial Intelligence, as ChatGPT is one of the most advanced language models in the world. And now, with the release of Dolly, others can create and develop their own open source language models.
This means that the AI technology is becoming increasingly accessible and democratic. Now, more people can contribute to the evolution of AI and develop solutions that meet needs of society.
The creation of a open source language model it also means more people can use it to develop chatbots, virtual assistants and others AI solutions for your companies or personal projects.
O Dolly launch is a testament to the potential of open source technology to change the way AI is created and used. With more companies and developers working together, we can expect even more significant advances in the coming years.
Collaboration and transparency driven by open source language models
A recent example of this trend is the Dolly LLM, an open source large-scale language model that seeks to provide broader access to AI technology. With this initiative, the objective is to prevent companies from being afraid of sharing confidential data with third parties that control AI technology.
One of the main reasons driving the movement of Open Source AI is the fear that companies may be reluctant to share sensitive information with a third party that controls the technology. This could harm innovation and the evolution of the sector.
With the increasing popularity of open source large-scale language models, greater transparency is possible, collaboration and information sharing. This can contribute to the development of AI in a faster, more efficient and safer way.
The initiative to make AI open source may have a major impact on society and the economy, promoting greater democratization of technology and encouraging innovation around the world. It is an important step towards making AI more accessible and allowing more people to contribute to the evolution of technology.
Dolly: the new open source creation based on language models
The foundation for Dolly's success is LLaMA, an open-source language model with 65 billion parameters created by Meta. LLaMA is trained on publicly available data, which allows for constant improvement. According to an article in Weights & Biases, Calls it can outperform many major language models despite being smaller.
With the use of LLaMA and the creation of Dolly, the artificial intelligence in chatGPT can reach a new level. These technologies enable more effective and natural communication with users, in addition to allowing the model to constantly evolve and adapt to new challenges. Creating open-source language models trained on public data is an important trend in artificial intelligence, and Dolly is a prime example of the positive results this approach can bring.
The importance of training data quality in improving natural language processing
This article explains that when using expert-written instructions for new tasks, the GPT-3 language model can be tuned more efficiently than when using existing public datasets.
They performed human evaluations and found that the SELF-INSTRUCT method outperformed other methods, leaving only a 5% difference compared to the best existing method.
Now, a new tool called Dolly demonstrates that it is possible to create a large, useful language model with a smaller but high-quality dataset. It works by modifying a 6 billion parameter open source EleutherAI model, adding instruction following features such as text generation and brainstorming, which were not present in the original model. This was possible thanks to the use of Alpaca data.
The most surprising thing is that you don't need to have the latest or greatest model to get instruction following skills, as the model used has only 6 billion parameters, compared to GPT-175's 3 billion.
This discovery is encouraging for those who work with natural language processing, as it suggests that the quality of the dataset is more important than the size. And this opens the door to smaller but higher quality language models, which can lead to a significant improvement in the quality of natural language processing.
Conclusion
Furthermore, initiatives such as mozilla.ai demonstrate a growing movement toward using open source software to make AI more accessible and transparent. With these initiatives, we believe that artificial intelligence will become increasingly present in our lives, bringing significant benefits to society.