Microsoft Says Automation Bug Caused Azure and Microsoft 365 Outage

The Azure and Microsoft 365 outage on July 23 resulted from an automation bug during routine network maintenance, Microsoft confirmed.

The disruption began at 10:44 AM ET and primarily affected customers whose connections relied on infrastructure linked to the West US Azure region. Microsoft tracked the Microsoft 365 incident as MO1437424.

Automation bug removed critical network routes

Microsoft said the outage started when engineers submitted a routine network device maintenance request.

An automated system converted that request into incorrect instructions. A software bug expanded the scope of the maintenance operation and included more network devices than intended.

The resulting change removed IP routes from several devices connecting the West US datacenter to Microsoft’s wider network.

Microsoft initially detected the problem as large-scale routing instability across its infrastructure.

West US traffic suffered connectivity failures

The missing routes disrupted traffic entering and leaving the West US region. Traffic that remained entirely within the region continued operating normally.

Affected customers experienced connection failures, increased latency, slow-loading applications, and problems accessing Microsoft services.

The incident affected both Azure products and Microsoft 365 applications that depended on the impacted network paths.

Microsoft 365 services affected by the outage

OneDrive access became intermittent, while SharePoint users encountered error messages when opening content.

Microsoft Teams chat performance degraded, and some images failed to load. The Microsoft 365 Admin Center also loaded slowly or remained inaccessible.

Other affected services included:

• Power Automate, where flows failed to load
• Copilot Chat, which experienced delays and failed actions
• Microsoft Loop, where users could not open pages
• Microsoft Graph
• Windows 365
• Microsoft Defender

Microsoft tracked the Microsoft 365 disruption under incident ID MO1437424.

Azure, Power BI, and security services also failed

The routing problem affected a broad range of Microsoft cloud products, including Fabric, Power BI, Power Apps, and Copilot Studio.

Customers also reported problems with Azure Virtual Desktop, Microsoft Sentinel, Azure App Service, Cosmos DB, Azure Kubernetes Service, Azure Databricks, Azure Firewall, and VPN Gateway.

Several other Azure services experienced connectivity problems or delayed recovery as Microsoft restored network routes.

Microsoft Defender investigations faced delays

The outage also affected parts of Microsoft Defender.

Some customers experienced delays when requesting assistance from Microsoft Defender Experts. Threat Explorer and Advanced Hunting investigations could fail during the incident.

Automated security workflows, investigation processes, and remediation actions also encountered problems.

Microsoft completed the rollback at 2:26 PM ET

Microsoft first redirected traffic through alternative network paths. This improved access for some customers, although several services remained unavailable or degraded.

Engineers identified the faulty maintenance change and began rolling it back at 1:45 PM ET.

Microsoft completed the rollback at 2:26 PM ET and confirmed that the Microsoft 365 incident had been resolved.

Some Azure products required additional time to recover after the network configuration returned to normal. Microsoft reported that all affected services had fully recovered by 3:41 PM ET.

Microsoft will review maintenance safeguards

Microsoft is reviewing the automated process that converts maintenance requests into network device instructions.

The investigation will examine validation systems, scope controls, and safety checks designed to prevent maintenance operations from affecting unintended infrastructure.

Microsoft expects to publish a final Post Incident Review after completing its internal investigation, typically within 14 days.

Via BleepingComputer