避免在 SharePoint Online 中受限或遭屏蔽Avoid getting throttled or blocked in SharePoint Online

了解 SharePoint Online 中的限制,以及如何避免受限或遭屏蔽。Find out about throttling in SharePoint Online, and learn how to avoid being throttled or blocked. 随附可用来简化任务的示例客户端对象模型 (CSOM) 和 REST 代码。Includes sample client-side object model (CSOM) and REST code you can use to make your task easier.

这听起来是不是很熟悉?举例来说,你正在运行 CSOM 过程以迁移 SharePoint Online 中的文件,但一直受限制,甚至更糟糕的是,你被阻止了。接下来会发生什么?应怎样做才能阻止这一切?Does this sound familiar? You're running a CSOM process - for example, to migrate files in SharePoint Online - but you keep getting throttled. Or even worse, you get blocked. What's going on and what can you do to make it stop?

什么是限制?What is throttling?

SharePoint Online 使用限制来维护 SharePoint Online 服务的最佳性能和可靠性。限制会限制用户操作或并发调用数量(通过脚本或代码),以防止资源的过度使用。SharePoint Online uses throttling to maintain optimal performance and reliability of the SharePoint Online service. Throttling limits the number of user actions or concurrent calls (by script or code) to prevent overuse of resources.

也就是说,很少有用户会在 SharePoint Online 中受到限制。That said, it is rare for a user to get throttled in SharePoint Online. 该服务非常强大,其设计旨在处理高容量信息。The service is robust, and it is designed to handle high volume. 如果你被限制,99% 都是因为自定义代码。If you do get throttled, 99% of the time it is because of custom code. 这并不意味着没有其他限制方式,只是它们不太常见。That doesn't mean that there aren't other ways to get throttled, just that they're less common. 例如,你启动了 10 台计算机,并在这 10 台计算机上运行一个同步客户端。For example, you spin up 10 machines and have a sync client going on all 10. 每台计算机同步 1 TB 的内容。On each sync 1 TB of content. 这可能会使您被限制。This would likely get you throttled.

限制如何发生

在 SharePoint Online 中被限制时会发生什么?What happens when you get throttled in SharePoint Online?

当用户超过使用限制时,SharePoint Online 会限制该用户帐户发出的任何进一步请求,但通常只是短时间内。在用户限制的有效期内,将限制该用户的所有用户操作。When a user exceeds usage limits, SharePoint Online throttles any further requests from that user account for a short period. All user actions are throttled while the throttle is in effect.

  • 对于用户直接在浏览器中执行的请求,SharePoint Online 会将您重定向到限制信息页面,请求将失败。For requests that a user performs directly in the browser, SharePoint Online redirects you to the throttling information page, and the requests fail.
  • 对于所有其他请求(包括 CSOM 或 REST 调用),SharePoint Online 将返回 HTTP 状态代码 429(“请求过多”)或 503(“服务器太忙”),且请求将失败。For all other requests, including CSOM or REST calls, SharePoint Online returns HTTP status code 429 ("Too many requests") or 503 ("Server Too Busy") and the requests will fail.

如果有问题的进程仍然超过使用限制,SharePoint Online 可能会完全阻止该进程;在这种情况下,你将看不到任何成功的请求,Microsoft 会在 Office 365 消息中心向你提供有关此阻止的通知。If the offending process continues to exceed usage limits, SharePoint Online might completely block the process; in this case, you won't see any successful requests and Microsoft will notify you of the block in the Office 365 Message Center.

应用程序限制Application Throttling

除了按用户帐户限制外,还会对每个应用程序实施限制。In addition to throttling by user account, limits are also applied to each application. SharePoint Online 中的每个应用程序都有自己的可用资源,但针对同一租户运行的多个应用程序最终会共享同一资源存储桶,并且在极少数情况下会导致速率受限。Every application in SharePoint Online has its own available resources, but multiple applications running against the same tenant ultimately share from the same resource bucket and in rare occurrences can cause rate limiting. 在这些情况下,SharePoint Online 将尝试通过后台活动确定交互式用户请求的优先级。In these cases, SharePoint Online will attempt to prioritize interactive user requests over background activities.

备注

请注意,使用应用程序限制时,AppOnly 身份验证和委派身份验证之间存在差异。Notice that there is difference between AppOnly authentication and Delegated authentication when making use of Applications when it comes to throttling. 使用委派方法时,限制是基于用户而不是使用的应用程序进行的。When making use of the delegated approach, throttling is applied on an user-basis and not the used application. 当使用 AppOnly 身份验证方式时,将基于应用程序进行限制。When making use of the AppOnly way of authenticating, throttling is applied on application-basis.

SharePoint Online 中常见的限制场景Common throttling scenarios in SharePoint Online

导致 SharePoint Online 中出现用户限制最常见的原因是以太高的频率执行太多操作的客户端对象模型 (CSOM) 或代表性状态传输 (REST) 代码。The most common causes of per-user throttling in SharePoint Online are client-side object model (CSOM) or Representational State Transfer (REST) code that performs too many actions too frequently.

  • 突发通信Sporadic traffic

    必须优化针对 SharePoint Online 的恒定负载或重复的复杂查询,以降低影响。Constant load or repetitive complex queries against SharePoint Online must be optimized for low impact. 对于批量处理文件的应用程序,如果未能遵循扫描该类应用程序的最佳做法,则可能会导致限制。Failing to follow best practices for scanning applications that process files in bulk will likely result in throttling. 这些应用包括同步引擎、备份提供程序、搜索索引器、分类引擎、数据丢失防护工具以及任何其他试图推理全部数据并对其应用更改的工具。These apps include sync engines, backup providers, search indexers, classification engines, data loss prevention tools, and any other tool, which attempts to reason over the entirety of data and apply changes to it.

    例如,将文件迁移到 SharePoint Online 后,可运行自定义 CSOM 或 REST 脚本以更新文件中的元数据。For example, after migrating files to SharePoint Online, you run a custom CSOM or REST script to update metadata on the files. CSOM/REST 脚本将以很高的频率更新大量文件,从而触发限制。The CSOM/REST script is updating a large number of files at a high frequency, which triggers throttling. 类似地,使用 REST 服务的自动完成 UI 小组件,在每次最终用户操作过程中对列表进行的调用次数过多也可能导致限制,具体取决于同时消耗资源的其他操作。Similarly, an autocomplete UI widget using REST services, making too many calls to lists during each end-user operation, may also cause throttling, depending on what other operations are consuming resources at the same time.

    个别限制

  • 无比拥挤的通信Overwhelming traffic

    单个进程在很长一段时间内持续大大超出限制。A single process dramatically exceeds throttling limits, continually, over a long time period.

    • 您使用 Web 服务构建同步用户配置文件属性的工具。该工具将根据您的业务线 (LOB) 人力资源 (HR) 系统中的信息更新用户配置文件属性。该工具以太高的频率发出调用。You used web services to build a tool to synchronize user profile properties. The tool updates user profile properties based on information from your line-of-business (LOB) human resources (HR) system. The tool makes calls at too high a frequency.

    • 在 SharePoint Online 上运行负载测试脚本但受到限制。You're running a load-testing script on SharePoint Online and you get throttled. SharePoint Online 上不允许进行负载测试。Load testing isn't allowed on SharePoint Online.

    • 例如,您在 SharePoint Online 上对您的工作组网站进行了自定义设置,方法是在主页上添加了一个状态指示器。此状态指示器频繁更新,这会导致页面向 SharePoint Online 服务发出太多调用,从而触发限制。You customized your team site on SharePoint Online, for example, by adding a status indicator on the Home page. This status indicator updates frequently, which causes the page to make too many calls to the SharePoint Online service - this triggered throttling.

      稳定的限制

为什么不能告诉我确切的限制?Why can't you just tell me the exact throttling limits?

设置和发布确切的限制上限听起来很简单,但事实上它会导致更严格的限制。Setting and publishing exact throttling limits sounds straightforward, but in fact it would result in more restrictive limits. 我们会持续监控 SharePoint Online 上的资源使用情况。We continually monitor resource usage on SharePoint Online. 根据使用情况,我们会对阈值进行微调,以便用户可以使用最大数量的资源,而不会降低 SharePoint Online 的可靠性和性能。Depending on usage, we fine-tune thresholds so users can consume the maximum number of resources without degrading the reliability and performance of SharePoint Online.

这就是 CSOM 或 REST 代码需遵循 Retry-After HTTP 标头值的重要原因所在;这使得你的代码在任何给定的日期都能尽可能快地运行,并且如果它达到限制上限,它可以使代码“恰量”回退。That's why it's so important for your CSOM or REST code to honor the Retry-After HTTP header value; this lets your code run as fast as possible on any given day, and it lets your code back off "just enough" if it hits throttling limits. 本文稍后部分的代码示例将向你展示如何使用 Retry-After HTTP 标头。The code samples later in this article show you how to use the Retry-After HTTP header.

使用仅具有 Sites.Read.All 权限的仅应用身份验证时的搜索查询数量限制Search query volume limits when using app-only authentication with Sites.Read.All permission

在 SharePoint 和 OneDrive 中,我们处理数十亿个文档,并使客户能够每秒发出大量查询。In SharePoint and OneDrive, we process multiple billions of documents and enable our customers to issue large query volumes per second. 当你使用具有仅应用身份验证的 SharePoint Online 搜索 API 且该应用具有 Sites.Read.All 权限(或更高权限)时,该应用将被注册具有完全权限,并被允许查询你的所有 SharePoint Online 内容(包括用户的私有 ODB 内容)。When you are using SharePoint Online search APIs with app-only authentication and the app having Sites.Read.All permission (or stronger), the app will be registered with full permissions and is allowed to query all your SharePoint Online content (including user’s private ODB content).

我们希望告知客户,使用此类权限的 SharePoint Online 搜索查询将被限制为 25 个查询 (QPS)。We want to let our customers know that SharePoint Online search queries using such permission will be throttled at 25 QPS. 搜索查询将以 429 响应返回,你可以在 2 分钟后重试该查询。The search query will return with a 429 response and you can retry the query after 2 minutes. 等待 429 恢复时,你应确保使用相似的“仅应用”权限暂停对服务提出的所有搜索查询请求。When waiting for 429 recovery, you should ensure to pause all search query requests you may be making to the service using similar app-only permission. 在收到限制响应的同时拨打其他电话会延长你的应用限制的时间。Making additional calls while receiving throttle responses will extend the time it takes for your app to become unthrottled.

在扩展系统规模时,我们意识到加强系统的效率以使其高效运行并保护系统的重要性,因此也需要进行这种更改。As we scale our system, we realize the importance of hardening the system to run it efficiently and also to protect the system and hence this change. 预计这一变化将于 8 月到 2020 年秋季推广到租户。This change is expected to roll out to tenants starting Aug into the Fall of 2020.

处理限制的最佳实践Best practices to handle throttling

  • 减少每个请求的操作数Reduce the number of operations per request
  • 减少调用频率Reduce the frequency of calls
  • 修饰流量以体现自己是谁(有关详细信息,请参阅下面有关流量修饰最佳做法的部分)Decorate your traffic so we know who you are (see section on traffic decoration best practice more on that below)
  • 利用 Retry-After HTTP 标头Leverage the Retry-After HTTP header

如果你遇到限制,我们需要利用 Retry-After HTTP 标头来确保最小延迟,直到限制被移除。If you do run into throttling, we require leveraging the Retry-After HTTP header to ensure minimum delay until the throttle is removed.

Retry-After HTTP 是处理受限制最快速的方法,因为 SharePoint Online 会动态确定重试的正确时间。The Retry-After HTTP header is the fastest way to handle being throttled because SharePoint Online dynamically determines the right time to try again. 换句话说,主动重试将对你不利,因为即使调用失败,它们仍会计入你的使用限制。In other words, aggressive retries work against you because even though the calls fail, they still accrue against your usage limits. 遵循 Retry-After HTTP 标头将确保最短的延迟。Following the Retry-After HTTP header will ensure the shortest delay.

有关 SharePoint Online 活动监视方式的详细信息,请参阅诊断 SharePoint Online 的性能问题For information about ways to monitor your SharePoint Online activity, see Diagnosing performance issues with SharePoint Online.

有关 Microsoft 云限制的更广泛讨论,请参阅限制模式For a broader discussion of throttling on the Microsoft Cloud, see Throttling Pattern.

如何修饰 http 流量以免受限?How to decorate your http traffic to avoid throttling?

为了确保和保持高可用性,一些流量可能会受到限制。To ensure and maintain high-availability, some traffic may be throttled. 当系统健康受到威胁时,就会出现限制。一种适用于限制的条件是流量修饰,这可直接影响流量的优先级。Throttling happens when system health is at stake and one of the criteria used for throttling is traffic decoration, which impacts directly on the prioritization of the traffic. 良好修饰的流量优先于未正确修饰的流量。Well-decorated traffic will be prioritized over traffic, which isn't properly decorated.

未修饰流量的定义是什么?What is definition of undecorated traffic?

  • 如果对 SharePoint Online 执行的 CSOM 或 REST API 调用中没有 AppID/AppTitle 和用户代理字符串,那么流量就未经修饰。Traffic is undecorated if there's no AppID/AppTitle and User Agent string in CSOM or REST API call to SharePoint Online. 用户代理字符串应采用如下所述的特定格式。The User Agent string should be in a specific format as described below.

有哪些建议?What are the recommendations?

  • 如果已创建应用,建议注册并使用 AppID 和 AppTitle。这样,可确保为今后解决任何问题提供最佳总体体验和最佳途径。If you've created an application, the recommendation is to register and use AppID and AppTitle – This will ensure the best overall experience and best path for any future issue resolution. 还请包括以下步骤中定义的用户代理字符串信息。Include also the User Agent string information as defined in following step.

    备注

    有关创建 Azure AD 应用程序的信息,请参考 Microsoft 标识文档,例如快速入门:向 Microsoft 标识平台注册应用程序页面。Refer to the Microsoft identity documentation, such as the Quickstart: Register an application with the Microsoft identity platform page, for information on creating an Azure AD application.

  • 务必使用以下命名约定,在对 SharePoint 执行的 API 调用中添加用户代理字符串Make sure to include User Agent string in your API call to SharePoint with following naming convention

类型Type 用户代理User Agent 说明Description
ISV 应用ISV Application ISV|CompanyName|AppName/VersionISV|CompanyName|AppName/Version 标识为 ISV,并添加公司名称、应用名称(用竖线符隔开),再添加版本号(用斜线符隔开)Identify as ISV and include Company Name, App Name separated by a pipe character and then adding Version number separated with a slash character
企业应用Enterprise application NONISV|CompanyName|AppName/VersionNONISV|CompanyName|AppName/Version 标识为 NONISV,并添加公司名称、应用名称(用竖线符隔开),再添加版本号(用斜线符隔开)Identify as NONISV and include Company Name, App Name separated by a pipe character and then adding Version number separated with a slash character
  • 若要生成自己的 JavaScript 库来调用 SharePoint Online API,请务必将用户代理信息添加到 http 请求中,并在合适、可能的情况下将 Web 应用程序还是注册为应用程序。If you're building your own JavaScript libraries, which are used to call SharePoint Online APIs, make sure that you include the User Agent information to your http request and potentially register your web application also as an Application, where suitable.

备注

用户代理字符串格式应遵循 RFC2616。因此,请按照上述说明操作,正确使用分隔符。Format of the user agent string is expected to follow RFC2616, so please follow up on the above guidance on the right separators. 也可以追加包含所请求信息的现有用户代理字符串。its also fine to append existing user agent string with the requested information.

备注

如果开发的是在浏览器中执行的前端组件,大部分新式浏览器不允许覆盖用户代理字符串,因此无需实施此操作。If you are developing front end components executing in the browser, most of modern browsers don't allow overwriting the user agent string and you don't need to implement this.

使用客户端对象模型 (CSOM) 时通过用户代理修饰流量的示例Example of decorating traffic with User agent when using Client Side Object Model (CSOM)

// Get access to source site
using (var ctx = new ClientContext("https://contoso.sharepoint.com/sites/team"))
{
  //Provide account and pwd for connecting to SharePoint Online
  var passWord = new SecureString();
  foreach (char c in pwd.ToCharArray()) passWord.AppendChar(c);
  ctx.Credentials = new SharePointOnlineCredentials("contoso@contoso.onmicrosoft.com", passWord);

  // Add our User Agent information
  ctx.ExecutingWebRequest += delegate (object sender, WebRequestEventArgs e)
  {
      e.WebRequestExecutor.WebRequest.UserAgent = "NONISV|Contoso|GovernanceCheck/1.0";
  };

  // Normal CSOM Call with custom User-Agent information
  Web site = ctx.Web;
  ctx.Load(site);
  ctx.ExecuteQuery();
}

使用 REST API 时通过用户代理修饰流量的示例Example of decorating traffic with User agent when using REST APIs

下面的示例采用 C# 格式,但建议也对 SharePoint Online 页面中使用的 JavaScript 库使用类似的用户代理信息。Following sample is in C# format, but the similar User Agent information is recommended to be used even for the JavaScript libraries used in the SharePoint Online pages.

HttpWebRequest endpointRequest = (HttpWebRequest) HttpWebRequest.Create(sharepointUrl.ToString() + "/_api/web/lists");
endpointRequest.Method = "GET";
endpointRequest.UserAgent = "NONISV|Contoso|GovernanceCheck/1.0";
endpointRequest.Accept = "application/json;odata=nometadata";
endpointRequest.Headers.Add("Authorization", "Bearer " + accessToken);
HttpWebResponse endpointResponse = (HttpWebResponse)endpointRequest.GetResponse();

CSOM 代码示例:ExecuteQueryWithIncrementalRetry 扩展方法CSOM Code sample: ExecuteQueryWithIncrementalRetry extension method

备注

需要使用 SharePoint Online CSOM 版本 16.1.8316.1200(2018 年 12 月版)或更高版本。You'll need to use SharePoint Online CSOM version 16.1.8316.1200 (December 2018 version) or higher.

在静态类中添加此扩展方法,并使用 ExecuteQueryWithIncrementalRetry 而不是 ExecuteQuery 来使代码处理限制请求。Add this extension method in a static class and use ExecuteQueryWithIncrementalRetry instead of ExecuteQuery to make your code handle throttling requests.

public static void ExecuteQueryWithIncrementalRetry(this ClientContext clientContext, int retryCount, int delay)
{
  int retryAttempts = 0;
  int backoffInterval = delay;
  int retryAfterInterval = 0;
  bool retry = false;
  ClientRequestWrapper wrapper = null;
  if (retryCount <= 0)
    throw new ArgumentException("Provide a retry count greater than zero.");
  if (delay <= 0)
    throw new ArgumentException("Provide a delay greater than zero.");

  // Do while retry attempt is less than retry count
  while (retryAttempts < retryCount)
  {
    try
    {
      if (!retry)
      {
        clientContext.ExecuteQuery();
        return;
      }
      else
      {
        //increment the retry count
        retryAttempts++;

        // retry the previous request using wrapper
        if (wrapper != null && wrapper.Value != null)
        {
          clientContext.RetryQuery(wrapper.Value);
          return;
        }
        // retry the previous request as normal
        else
        {
          clientContext.ExecuteQuery();
          return;
        }
      }
    }
    catch (WebException ex)
    {
        var response = ex.Response as HttpWebResponse;
        // Check if request was throttled - http status code 429
        // Check is request failed due to server unavailable - http status code 503
        if (response != null && (response.StatusCode == (HttpStatusCode)429 || response.StatusCode == (HttpStatusCode)503))
        {
          wrapper = (ClientRequestWrapper)ex.Data["ClientRequest"];
          retry = true;

          // Determine the retry after value - use the `Retry-After` header when available
          string retryAfterHeader = response.GetResponseHeader("Retry-After");
          if (!string.IsNullOrEmpty(retryAfterHeader))
          {
            if (!Int32.TryParse(retryAfterHeader, out retryAfterInterval))
            {
              retryAfterInterval = backoffInterval;
            }
          }
          else
          {
            retryAfterInterval = backoffInterval;
            }

          // Delay for the requested seconds
          Thread.Sleep(retryAfterInterval * 1000);

          // Increase counters
          backoffInterval = backoffInterval * 2;
        }
        else
        {
          throw;
        }
    }
  }
  throw new MaximumRetryAttemptedException($"Maximum retry attempts {retryCount}, has be attempted.");
}

[Serializable]
public class MaximumRetryAttemptedException : Exception
{
  public MaximumRetryAttemptedException(string message) : base(message) { }
}

如果我在 SharePoint Online 中被阻止,该怎么办?What should you do if you get blocked in SharePoint Online?

阻止是限制最极端的形式。Blocking is the most extreme form of throttling. 我们很少阻止租户,除非我们检测到可能威胁 SharePoint Online 服务整体运行状况的长期、过多的流量。We rarely ever block a tenant, unless we detect long-term, excessive traffic that may threaten the overall health of the SharePoint Online service. 我们应用阻止的目的是,防止因过多的流量降低 SharePoint Online 的性能和可靠性。We apply blocks to prevent excessive traffic from degrading the performance and reliability of SharePoint Online. 阻止发生在应用程序或用户级别,它会阻止有问题的进程运行,直到你解决问题为止。A block - which is placed at the app or user level - prevents the offending process from running until you fix the problem. 如果我们阻止了你的订阅,你必须采取操作修改有问题的进程,然后才能将阻止移除。If we block your subscription, you must take action to modify the offending processes before the block can be removed.

如果我们阻止了你的订阅,我们将在 Office 365 消息中心向你提供有关此阻止的通知。If we block your subscription, we will notify you of the block in the Office 365 Message Center. 该消息将描述导致阻止的原因,提供有关如何解决违规问题的指南,并告诉你应该联系谁以移除阻止。The message describes what caused the block, provides guidance on how to resolve the offending issue, and tells you who to contact to get the block removed.

另请参阅See also