FTC to CAM – Advance HTTP remote operations for SP2013

SharePoint remote APIs are really powerful and can be used for modifying broad spectrum of the different options in site or in list level. There are however still some limitations compared to the functionalities which have been available using server side APIs. These limitations are being addressed gradually in the client side object model or in other remote techniques, but as a short term workaround we can quite often used so called HTTP remote operations patterns to achieve the needed functionality.

These needed functionalities could some something like following as a short example list

  • Enable additional languages in site level
  • Adjust audit logging capabilities in site collection level
  • Adjust advance list settings
  • Enable site request access functionality for some email address
  • Control regional settings in site level
  • Activate Sandbox solutions


DISCLAIMER: Microsoft does NOT recommend to use this pattern, since code can get broken with any future Office365 updates which will impact the html dom structure. This pattern is only to be used when native remote operation is not available and you have business critical need for required action. You should consider this route only when you are willing to maintain the code or when the operation is only for on-time execution. Microsoft will not take any responsibility of the updates to the Office365, which could break up this pattern. 

Update on 12th of June 2015 - Cookie name for authentication has been changed from FedAuth to SPOIDCRL. This change has to be done for the code to make provided code sample to work again. Notice that many of the settings mentioned above (like regional settings or language settings) are currently natively supported with Office 365 CSOM. See following blog post for more details: Latest API updates in Client Side Object Model (March 2015 CU for SP2013).

Introduction to advance HTTP remote operations

HTTP remote operations pattern is based on the model that we mimic what end users are doing while they operate SharePoint using broser. This means that we mimic following events using HttpWebRequest and HttpWebResponse objects.

  1. Authenticated users requests the page from SharePoint
  2. User modifies html controls in the browser
  3. Modified set of properties are posted to server using HTTP Post operations

This means that we’ll need to first take care of the authentication, which is easy and straight forward within on-premises or with Office365-Dedicated, but does require additional considerations with Office365. In on-premises, we can simply use either default credentials of the process or impersonate with Network credentials as follows.

  1: HttpWebRequest request = (HttpWebRequest)WebRequest.Create(this.TargetSiteUrl);
  2: NetworkCredential credential = new NetworkCredential(this.User, this.Password, this.Domain);
  3: CredentialCache credentialCache = new CredentialCache();
  4: credentialCache.Add(new Uri(this.TargetSiteUrl), "NTLM", credential);
  5: request.Credentials = credentialCache;

With Office365, we’ll however need to do authentication slightly differently and to delegate authentication for the HttpWebRequest, we’ll need to set the FedAuth cookie for the http request. Here’s the reference code for doing that. In provided code example this is done automatically for the Office365 authentication style.

  1: // Convert password to secure string and create MSO Creds
  2: HttpWebRequest request = (HttpWebRequest)WebRequest.Create(this.TargetSiteUrl);
  3: var spoPassword = new SecureString();
  4: foreach (char c in this.Password)
  5: {
  6:     spoPassword.AppendChar(c);
  7: }
  8: SharePointOnlineCredentials Credentials = new SharePointOnlineCredentials(this.User, spoPassword);
  9: Uri tenantUrlUri = new Uri(this.TargetSiteUrl);
  10: string authCookieValue = Credentials.GetAuthenticationCookie(tenantUrlUri);
  11: // Create fed auth Cookie and set that to http request properly to access Office365 site
  12: Cookie fedAuth = new Cookie()
  13: {
  14:     Name = "FedAuth",
  15:     Value = authCookieValue.TrimStart("SPOIDCRL=".ToCharArray()), 
  16:     Path = "/",
  17:     Secure = true,
  18:     HttpOnly = true,
  19:     Domain = new Uri(this.TargetSiteUrl).Host
  20: };
  21: // Hookup authentication cookie to request
  22: request.CookieContainer = new CookieContainer();
  23: request.CookieContainer.Add(fedAuth);

After we have done the initial request of the page, we’ll need to store the request cookie view state and view validation values for posting and gather other elements which are posted back to server for adjusting the configuration. Following shows what elements are required for enabling access request.

  1: public override void SetPostVariables()
  2: {
  3:     // Set operation specific parameters
  4:     this.PostParameters.Add("__EVENTTARGET", "ctl00$PlaceHolderMain$ctl01$RptControls$btnSubmit");
  5:     this.PostParameters.Add("ctl00%24PlaceHolderMain%24ctl00%24txtEmail", this.EmailAddresses);
  6:     if (this.EnableAccessRequests)
  7:     {
  8:         // Present if we enable access request.
  9:         this.PostParameters.Add("ctl00%24PlaceHolderMain%24ctl00%24chkRequestAccess", "on");
  10:     }
  11: }

After the post parameters are collected, we can simply post in the information back to server in proper format. For additional details, please have a closer look on the code example and referenced video.

Video of the demo

Following video walks through the code and explains how provided example was implemented. Notice that used code in the demo is also available for download from following link.



Video will be soon available for download from my SkyDrive if needed for offline purposes.