LongMem is Microsoft's answer for unlimited context length

Microsoft has been at the forefront of AI education, and AI tools, in general. Copilot is coming to Windows 11 very soon, as part of Windows updates. AI is making its way into Microsoft Teams, with an intelligent recap helping people work easier.

Bing is also one of the most exciting AI tools around, and its features allow you to optimize both your work and the way you navigate the Internet.

But earlier this week, Microsoft also unveiled that Orca 13B will soon become open source. Orca 13B is a small AI model that performs at the same quality and speed as the other AI models, such as ChatGPT.

Now, in the battle of AIs and AI research, Microsoft is coming up with LongMem, a hypothetical framework and AI language model.

According to this Reddit thread, LongMem allows for unlimited context length at a reduced GPU, and CPU memory usage. All of these are done at a faster speed.

Microsoft Research proposes new framework, LongMem, allowing for unlimited context length along with reduced GPU memory usage and faster inference speed. Code will be open-sourced
byu/llamaShill inLocalLLaMA

Is LongMem the Microsoft answer for unlimited context length?

Well, to understand what it means to have an unlimited context length, we first need to understand what does context length mean?

Context length refers to the number of tokens (words, signs, etc) permitted by a model, its input and output, and yours as well.

For example, ChatGPT has a limited number of tokens which means its context length is limited as well. As soon as ChatGPT passes that limit, then all the continuous interaction that you had with it up to that point, will lose all significance. Or better said, it will reset.

So if you’re starting a conversation with ChatGPT on the subject of Windows, and that conversation lasts more than the limited context length, then the AI tool will lose the context, and will either start to deviate from the subject, or the whole conversation will reset.

An unlimited context length will make sure that won’t happen, and the AI model will continue to answer you on the subject, while also learning and adapting information as you talk to it about Windows.

That means the AI model will also personalize the conversation according to your input, hence the need for the context to have an unlimited length.

How does LongMem work?

Microsoft promises just that with their new research on the framework, LongMem. LongMem would enable large language models to memorize long-term contexts and utilize long-term memory at the reduced CPU power.

The framework consists of a frozen large language model as the memory encoder, a residual side network as the memory retriever and reader, and a cached memory bank that stores key-value pairs from past contexts.

In the research done by Microsoft, experiments show that LongMem outperforms baselines on long-text language modeling, long-context understanding, and memory-augmented in-context learning tasks. Plus, long-term memory allows it to utilize more demonstration examples for better learning.

And the good news is that LongMem will be open-source. So you will be able to study it and learn how to implement the framework in your own AI model. You can check its GitHub profile here.

And if you’re interested to read the whole research paper on the framework, head over here and check it out.

What do you think about this new technological breakthrough? Will it revolutionize the way AI models work? Be sure to let us know your thoughts and opinions below.