Back to overview
Degraded

ZA North Azure Connectivity

Jul 29 at 09:30am UTC
Affected services
Voice - ZA North (JNB)

Resolved
Jul 29 at 03:53pm UTC

As we have now received confirmation from Microsoft the issue is deemed resolved from their side, we are closing this issue. We will await further RCA from Azure, but their current provided Root Cause is an unexpected increase in traffic - most likely a large scale DDoS attack given the number of regions affected.

Update from Microsoft (MO406867)

Current status: Our monitoring telemetry indicates that the issue has recovered. We're continuing to actively monitor the environment and perform a set of proactive changes to prevent this type of issue from reoccurring.

Scope of impact: This issue is specific to a subset of traffic routed through infrastructure in South Africa, Brazil, North America, Canada, Japan and Korea. This issue is also causing impact for users located in regions surrounding the infrastructure hosted in those locations.

Root cause: An unexpected increase in user requests resulted in network congestion.

Updated
Jul 29 at 02:29pm UTC

Update from Microsoft (MO406867)

Current status: Our monitoring telemetry indicates that the issue has recovered. We're continuing to actively monitor the environment and perform a set of proactive changes to prevent this type of issue from reoccurring.

Scope of impact: This issue is specific to a subset of traffic routed through infrastructure in South Africa, Brazil, North America, Canada, Japan and Korea. This issue is also causing impact for users located in regions surrounding the infrastructure hosted in those locations.

Updated
Jul 29 at 02:07pm UTC

We continue to see no further issues, but are awaiting for an official confirmation of resolution from Microsoft

Updated
Jul 29 at 01:40pm UTC

We have seen zero packet loss and latency spikes for past 10 minutes.

We will keep issue open pending official confirmation of resolution

Updated
Jul 29 at 01:30pm UTC

Latest update from Microsoft

We've determined an unexpected spike in traffic is causing network congestion for some customers, resulting in the impact seen. We are currently applying multiple mitigation strategies to alleviate the connectivity issues. An update will be provided in the next 60 minutes or as events warrant.

Updated
Jul 29 at 01:00pm UTC

Further update from Microsoft (MO406867)

We’ve identified a potential networking issue resulting in impact. We’re reviewing service monitoring telemetry to verify that this is the root cause, before formulating a plan for remediation.

Updated
Jul 29 at 12:50pm UTC

We are seeing a change in pattern of the issue - previously we would see periodic packet loss and prolonged latency spikes.

We are currently seeing shorter periods of packet loss and no latency spikes. This should lead to an improvement in call quality.

Microsoft confirm issue began at 10am SAST - we are still waiting on an update as to a fix. However we hope this change in pattern indicates a solution is in progress.

We fully appreciate the severity of this issue, and will await a full RCA from Azure in due course.

Updated
Jul 29 at 12:08pm UTC

Update from Microsoft

We've identified a potential root cause and are currently investigating mitigation options. An update will be provided in the next 60 minutes or as events warrant.

Created
Jul 29 at 09:30am UTC

Azure ZA North Region is experiencing periodic periods of increased latency to the public internet. This will affect call quality.

We await an official announcement with an ETA from Azure.