Microsoft explores small language model (SML) alternative to ChatGPT

Despite its multibillion-dollar investment in OpenAI and its coveted ChatGPT pre-generative large language models (LLM), Microsoft is also working internally to create alternatives that now include a cheaper “small language model” (SLM) solution.

How much does ChatGPT cost?

According to a report from the Information, Microsoft’s exploration of supplemental and alternative solutions to OpenAI LLM’s has now surfaced a new team dedicated to creating a SLM that costs less and requires fewer resources to operate.

By most accounts Microsoft is losing money running ChatGPT as its underlying platform for its various Copilot projects, powered by OpenAI’s GPT-3 and 3.5 Turbo language models. The Motley Fool claims that Microsoft lost an average of $20 per user during the first few months of 2023, and some users cost the company an average loss of over $80 per month.

The publication Interesting Engineering estimates that ChatGPT was costing OpenAI some $700,000 per day in computing costs alone, based on the processing needed to run GPT-3.5. A third source, Gizmodo reports that OpenAI’s losses reached $540 million since the development of ChatGPT, and that its future was uncertain as recent as the end of October 2023.

Beyond the sunk costs of embedding the popular LLM in Copilot, Microsoft and OpenAI also have a structured partner deal that comes with a 10-year deadline or repayment timetable that would divest the company’s largest-single-shareholder position and free up space for other entities or consortiums to manage OpenAI in ways outside of Microsoft’s current control.

Microsoft’s GenAI team

Understandably Microsoft wants to be in the driver seat of such a large financial investment in pre-generative artificial intelligence and its new GenAI team is tasked with creating an SML will be charged with providing a similarly functioning conversational and pre-generative platform to ChatGPT but whose carbon footprint is smaller, and ultimately costs less.

Microsoft is bundling the GenAI team under its flagship Azure cloud business to funnel the sector’s healthy profits into research from its top developers who have been drafted from other parts of the company. GenAI consists of Microsoft heavy hitters such as CVP Misha Bilenko as team leader and Microsoft Research vets Jianfeng Gao, Michel Galley, and Chris Brockett who have worked on DiabloGPT and DeepSpeed projects.

While Microsoft was able to get Bing Chat which ultimately became Copilot, out to the door to users and customers first, it is in enviable position of watching Google manage and maintain its own homebrewed Bard services powered by its latest Gemini LLM, and this new GenAI team looks to be the company’s answer to future AI applications from Microsoft.