Throttling challenges in Office 365 (429)

[bcameron1231]

Category

• Question
• Typo
• Bug
• Additional article idea

Expected or Desired Behavior

Improved Throttling. I obviously won't expect throttling to go away, but it feels like something has changed in the past couple of weeks.

Observed Behavior

I'm working in an Office 365 tenant of about 30k users.

It seems there may have been some changes to the throttling behaviors in Office 365. For the past few weeks, I have been working on a migration into Office 365. A few weeks ago, I migrated close to 1m files with just few issues.

This past week, I've been running reports on some of the document libraries in the environment in Office 365... and I get immediately throttled. I can't even run a report on a library with 1700 items in it, without being throttled...

I have more files that I need to migrate into this environment, which will be nearly impossible due to the current throttling issues.

Here is a Twitter thread with lots of comments from people who are also seeing increased throttling in the last couple of weeks. Seems to be a common issue.

https://twitter.com/Beau__Cameron/status/1181992491018403840?s=20

Have there been any changes to the throttling behaviors in Office365?

[msft-github-bot]

Thank you for reporting this issue. We will be triaging your incoming issue as soon as possible.

[hittoday]

We are seeing the same thing have been throttled after opening a ticket with MS we found that the account being throttled was making only 4-5% of overall calls the support engineer considered this a low value. We had previously tested similar throughput a couple of months ago and did not get throttled. this is a bit of a moving target and better communication would be ideal.

[MRuimerman]

We are seeing increased throttling on Graph calls. We added some comments on an open ticket for this one already, but it is getting a little quiet there: #4573

[MikeBatchelor365]

We've had the same since end of August. We had hoped it was a temporary issue but seems to have got worse, so have raised issue with Microsoft. We've been getting '503 Server Unavailable' and not 429 Throttling responses however.

[fthorild]

Same here, some LOB apps (CSOM/Sharepoint REST) running since a few years are being throttled. Office 365 Tenant with 35k users.

Our scenario is mostly a background job running every 5 minutes and it is not implemented with decorated traffic, this was implemented years ago. Why I joined this thread is because it showed up now all of a sudden. It was before my time and I would’ve suggested other patterns if was introduced today, in fact I will ask them to make a change.

[VesaJuvonen]

Can all of you provide additional details on the exact scenario. What kind of operation was performed? Provisioning, migration, automation, Azure etc? This helps on understanding things on our side, so all details would be highly appreciated.

[hittoday]

Provisioning new sites

[MRuimerman]

Loading a page with webparts that do graph calls (only fetching data). We have a ticket open with Microsoft support for this.

[JeremyKelley]

There are two resources everyone should have handy when talking about throttling just to make sure you're prepared for the case where throttling kicks in.

The first is https://aka.ms/spo429 this page will tell you everything you need to know about the SharePoint Online throttling mechanism.

The second resource is https://aka.ms/scanguidance. The scanning guidance will give you the best practices for creating high volume apps. In particular this gives the best guidance on how to process document libraries using the Delta query API.

Some other general points to remember:

1. There are two different throttle responses, 429 means your app has used up its available quota for calling the service and it needs to wait before more calls can be made. If you're processing libraries and getting 429's definitely check out the scan guidance for instructions. 503 means the server is under pressure and needs to return to normal before apps can call it. Generally if one app is getting 503s then other apps will be as well. 503 can clear on its own over time or may be an indication of broader issues. If you're getting continued 503s the tenant owner should call support for help.

2. If you're working with permissions definitely read the scan guidance for how to use Delta query to retrieve permission hierarchies. Other legacy methods are not going to be as optimized for this scenario and can easily result in throttling.