Enterprise software company Databricks has released a new wide language model called Dolly, an open-source version of ChatGPT . The name Dolly is a tribute to the famous sheep of the same name, the first cloned mammal.
This news is very important for the world of Artificial Intelligence , as ChatGPT is one of the most advanced programming language models in the world. And now, with the launch of Dolly, other people can create and develop their own open-source programming language models.
This means that AI technology is becoming increasingly accessible and democratic. Now, more people can contribute to the evolution of AI and develop solutions that meet the needs of society.
Creating an open-source programming language model also means that more people can use it to develop chatbots , virtual assistants, and other AI solutions for their businesses or personal projects.
The launch of Dolly is proof of the potential of open-source technology to change how AI is created and used. With more companies and developers working together, we can expect even more significant advancements in the coming years.
Collaboration and transparency driven by open source language models
Large-scale, open-source programming languages are becoming increasingly prevalent in the world of Artificial Intelligence (AI) , aiming to provide broader access to the technology so that it is not monopolized and controlled by large corporations.
A recent example of this trend is Dolly LLM , an open-source, large-scale programming language model that seeks to provide broader access to AI technology . The goal of this initiative is to prevent companies from being afraid to share confidential data with third parties who control AI technology.
One of the main reasons driving the open-source AI movement is the fear that companies may be reluctant to share sensitive information with a third party that controls the technology. This could hinder innovation and the evolution of the sector.
With the growing popularity of large-scale, open-source programming language models, greater transparency, collaboration, and information sharing are possible . This can contribute to the development of AI in a faster, more efficient, and safer way.
The initiative to make AI open source can have a major impact on society and the economy , promoting greater democratization of technology and encouraging innovation worldwide. It is an important step towards making AI more accessible and allowing more people to contribute to the evolution of the technology.
Dolly: the new open source creation based on language models
The creation of Dolly is a significant contribution to the evolution of language models in artificial intelligence. Developed from the open-source model created by the EleutherAI research institute and the Alpaca model from Stanford University, Dolly stands out for being able to outperform many of the most advanced language models, such as OpenAI GPT-3, DeepMind's Gopher, and DeepMind's Chinchilla.
The foundation of Dolly's success is LLaMA, an open-source programming language model with 65 billion parameters created by Meta. LLaMA is trained on publicly available data, allowing for its constant improvement. According to an article in Weights & Biases, LLaMA can outperform many leading programming language models, despite being smaller.
With the use of LLaMA and the creation of Dolly, artificial intelligence in chatGPT can reach a new level. These technologies enable more effective and natural communication with users, as well as allowing the model to constantly evolve and adapt to new challenges. The creation of open-source language models trained on public data is an important trend in the field of artificial intelligence, and Dolly is an excellent example of the positive results that this approach can bring.
The importance of training data quality in improving natural language processing
The importance of creating high-quality datasets is undeniable. And recently, an academic research paper titled "SELF-INSTRUCT: Aligning Language Model with Self-Generated Instructions PDF" highlighted a way to create high-quality, automatically generated question-and-answer training data that is better than limited public data.
This article explains that by using instructions written by experts in new tasks, the GPT-3 language model can be tuned more efficiently than by using existing public datasets.
They performed human evaluations and found that the SELF-INSTRUCT method outperformed other methods, leaving only a 5% difference compared to the best existing method.
Now, a new tool called Dolly demonstrates that it's possible to create a large and useful language model with a smaller, but high-quality, dataset. It works by modifying an open-source model with 6 billion parameters from EleutherAI, adding instruction-following features like text generation and brainstorming that weren't present in the original model. This was made possible by using Alpaca data.
The most surprising thing is that you don't need the latest or largest model to achieve instruction-following capabilities , since the model used has only 6 billion parameters, compared to the 175 billion of GPT-3.
This discovery is encouraging for those working with natural language processing , as it suggests that the quality of the dataset is more important than its size. This opens the door to smaller, but higher-quality language models, which could lead to a significant improvement in the quality of natural language processing.
Conclusion
In summary, the Dolly tool represents a major advance in the democratization of artificial intelligence. Thanks to this new approach to creating language models with smaller but high-quality datasets, more people can have access to this powerful technology.
Furthermore, initiatives like Mozilla.ai demonstrate a growing movement towards the use of open-source software to make AI more accessible and transparent. With these initiatives, we believe that artificial intelligence will become increasingly present in our lives, bringing significant benefits to society.
