The first NPU for Windows is here, as Microsoft and Intel debut the Direct ML-powered processor

The era of the NPU is finally here: Microsoft and Intel debut the first NPU for Windows, and it is powered by Direct ML, an API that should greatly simplify the deployment and development of AI across the Windows ecosystem.

We are thrilled to announce our collaboration with Intel®, one of our key partners, to bring the first Neural Processing Unit (NPU) powered by DirectML on Windows. AI is transforming the world, driving innovation and creating value across industries. NPUs are critical components in enabling amazing AI experiences for developers and consumers alike.

Microsoft

The Direct ML-powered Windows NPU (Neural Processing Unit) will run on the latest Intel® Core™ Ultra processors to deliver new possibilities for AI on Windows, according to Microsoft. We say it’s just in time, as recently it was reported that the next version of Windows will extensively rely on NPUs to deliver AI experiences to users.

According to the press release published by Microsoft, the Direct ML-powered Windows NPU will enter a developer preview in early 2024, and it will gradually expand to a general preview over that year.

So what does this mean for the regular Windows user? Well, for now, it’s not that significant as NPUs are not intended for regular commercial users, but that will change in a short time. Let us explain.

Windows NPU: What is an NPU?

NPUs have been the subject of discussion ever since AI gained popularity, and for good reasons, as these special processors use AI to speed up data processing with high accuracy and less power usage.

In other words, NPU is a microprocessor built to enable machine learning (ML) workloads that are computationally intensive, do not require graphics interactions, and provide efficient power consumption, as Microsoft states.

An NPU is much faster than the standard CPU or GPU, and it can perform millions of operations in a matter of seconds, making it the perfect processor for extensive workloads, such as complex calculations in deep learning, accuracy, and so on.

For this very reason, an NPU is perfect for industrial environments and industries where a large quantity of information has to be processed, such as transportation, hardware manufacturing, automatization, healthcare, or finance.

Some real-life applications of an NPU-based operation would be:

• Self-driving cars, as in the case of the transportation industry.
• Accurately scanning medical images and concluding accurate diagnostics and treatments based on it, when used in healthcare.
• Calculating risk management and detecting millions of threats, when employed in financial environments.

And the list could go on and on, as these are some of the most obvious examples. An NPU is capable of such things by using a process called inference. This means the NPU is trained on data to recognize various objects and subjects. In simpler words, the NPU gains an awareness of its environment.

The inference process happens when new input is given to the NPU, and using its trained awareness, the processor will then analyze the new input and offer new perspectives. The excellent aspect of the inference process is that it is meant to be built as factually as possible, to begin with.

So this means that the perspectives offered by the NPU will also be factually correct, and will exist within the right methodologies.

Actually, Microsoft has been one of the companies that have been working extensively on training AI models to develop, recognize, and offer new perspectives from a scientific point of view, using real-life methodologies.

Just this year, the Redmond-based tech giant has been investing a lot of resources into various AI research projects, and many of them stand out: LongMem, Gorrila AI, Kosmos-2, and many more models that use the existing research to push boundaries and come up with yet-to-be-discovered methodologies of their own.

In time, as AI becomes more performant, and will eventually reach a state of AGI, NPUs can employ the technology to build supercomputers capable of solving complex issues, and come up with ways to address humanity’s biggest concerns.

It does sound like a take from a science fiction film, but it’s just a matter of time until this happens. An NPU-based device with AGI capabilities could be able, theoretically, to find a cure for serious diseases in a matter of months, as it could run billions of research studies in a short span of time.

For the standard commercial user, an NPU-based device would be useful as long as it could be targeted to complete a certain task. Many companies currently have developed their own NPU capabilities that are aimed at doing exactly that.

Take, for instance, the example of Intel’s Neural Network Processor (NNP), which is NPU processor capable of delivering strong performances to customer devices. There are countless other NPUs on the market, currently, such as:

• TPU by Google
• NNP, Myriad, EyeQ by Intel
• NVDLA by Nvidia
• AWS Inferentia by Amazon
• Ali-NPU by Alibaba
• Kunlun by Baidu
• Sophon by Bitmain
• MLU by Cambricon
• IPU by Graphcore
• Ascend by Huawei
• Neural Engine by Apple
• Neural Processing Unit (NPU) by Samsung

It’s still very early to say what these NPUs are capable of, but theoretically, they can transform every device into a mini-supercomputer. Is this what Microsoft intends to do by releasing DirectML-based NPUs for Windows?

Maybe, but we’ll have to wait. Fortunately, 2024 is right ahead of us.