Windows PCs Can Now Team Up Over USB4 to Run AI Models That Won't Fit on One Machine


Microsoft has unveiled a clever way to run massive AI models without relying on expensive server hardware. Its new RDMA over USB4 technology lets multiple Windows PCs combine their memory and GPU power to run a single AI model. The idea is simple but interesting. Connect two or more compatible PCs using USB4 cables, and they can work together as a small local AI cluster. In other words, no dedicated RDMA network cards, switches, or third-party drivers are required.

Microsoft wants your Windows PCs to work together for AI

Microsoft announced RDMA over USB4 yesterday via a new Microsoft USB Blog. The technology uses Remote Direct Memory Access, or RDMA, to move data directly between connected systems with lower latency and reduced CPU overhead.

Windows already supports RDMA through NetworkDirect, but traditionally required specialized Ethernet or InfiniBand adapters. As we previously reported, Microsoft is exploring larger memory configurations for Windows AI PCs, including a reported roadmap featuring up to 512GB of RAM. The company has also been expanding local AI models and on-device APIs across Windows.

Microsoft is now extending that capability to USB4 connections, making multi-PC AI setups more accessible to developers. The early performance numbers are encouraging. Microsoft says RDMA over USB4 reduced small-message round-trip latency to around 28–50 microseconds. That’s roughly four times lower than Ethernet UDP over the same USB4 connection.

AMD and Intel show massive local AI models

Microsoft tested the technology with AMD and Intel hardware. Three AMD Ryzen AI Max mini PCs, each featuring 128GB of unified memory, ran Qwen3-235B-A22B at approximately 18.5 tokens per second.

AMD separately measured 21.98 tokens per second for decoding. Intel also demonstrated GPT-OSS-120B across three and four PCs featuring Core Ultra Series 3 processors. Its results reached 49.4 and 54.2 tokens per second, respectively, at a 1K-token context.

These figures come from Microsoft and its hardware partners, not independent testing. Still, they demonstrate how connected PCs could handle models that exceed individual system memory.

Developer preview is coming to Windows Insiders

The technology will reach through the Windows Insider Experimental Channel (Future Platforms). Microsoft says the required Windows build is coming soon, while its open-source NetworkDirect provider and llama.cpp testbed are already available.

It should be noted that every PC needs sufficient USB4 ports, direct cables, and compatible hardware. Microsoft recommends 40Gbps or faster connections, with 80Gbps offering additional bandwidth. The preview also requires test signing and Secure Boot to be disabled. Daisy-chaining PCs is discouraged, and dropped connections must be re-established by applications. You can check the detailed blog post here.

More about the topics: AI, microsoft

Readers help support Windows Report. We may get a commission if you buy through our links. Tooltip Icon

Read our disclosure page to find out how can you help Windows Report sustain the editorial team. Read more

User forum

0 messages