Big data analytics firm Databricks last week debuted Dolly, an open-source large language model (LLM), that is a cheaper alternative to the popular ChatGPT.
Dolly, named after the sheep that was the first cloned mammal, is claimed to be as magical as ChatGPT despite being trained on far less data in 30 minutes on a single machine.
Databricks evaluated Dolly on the instruction-following capabilities described in the InstructGPT paper that ChatGPT is based on and found that it exhibits many of the same qualitative capabilities, including text generation, brainstorming and open Q&A.
The model underlying Dolly only has 6 billion parameters, compared to 175 billion in GPT-3, and is two years old, making it particularly surprising that it works so well.
“This suggests that much of the qualitative gains in state-of-the-art models like ChatGPT may owe to focused corpora of instruction-following training data, rather than larger or better-tuned base models,” said Databricks in a recently published blogpost.
Democratising AI
In offering an off-the-shelf open-source AI model, Databricks hopes to bring the technology into the hands of small enterprises who otherwise could not afford it.
“With 30 bucks, one server and three hours, we’re able to teach [Dolly] to start doing human-level interactivity,” said Databricks CEO Ali Ghodsi.
“We’re in the earliest days of the democratisation of AI for the enterprise, and much work remains to be done, but we believe the technology underlying Dolly represents an exciting new opportunity for companies that want to cheaply build their own instruction-following models.”
Ali Ghodsi, Databricks
Databricks pointed out there are many reasons a company would prefer to run their AI model in-house rather than sending data to a centralised LLM provider that serves a proprietary model behind an API.
“For many companies, the problems and datasets most likely to benefit from AI represent their most sensitive and proprietary intellectual property and handing it over to a third party may be unpalatable,” said Databricks’ blog post.
“Furthermore, organisations may have different tradeoffs in terms of model quality, cost, and desired behaviour. Databricks believes that ML users are best served long term by directly controlling and owning their models,” it added.
The release of Dolly is the first in a series of announcements Databricks is making that focus on helping every organisation harness the power of large language models.
