Pipeline

Create or Update

Creates a new pipeline, or updates the content of an existing pipeline.

Request

The Create or Update Pipeline request may be constructed as follows. HTTPS is recommended:

HTTP Verb Request URI HTTP Version
PUT https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName}/datapipelines/{PipelineName}?api-version={api-version} HTTP/1.1

URI Parameters

URI Parameter Required Description
SubscriptionID Yes Your Azure subscription ID.
ResourceGroupName Yes A unique name for the resource group that hosts your Azure data factory.
DataFactoryName Yes Name for the data factory that you want to create your pipeline in.
PipelineName Yes Name of pipeline you want to create.
Api-Version Yes Specifies the version of the protocol used to make this request.

Request Headers

The following table describes the request headers:

Request Header Required Description
x-ms-client-request-id Yes The operation id for this request.

Request Body

Request body depends on your pipeline property. For more information about pipelines, see Pipelines and Activities in Azure Data Factory article for details.

The format of the request body is as follows:


{  
    "name": <name>,  
    "properties": {  
        "description": <description>,  
        "activities": <activity>  
    }  
}  

The following table describes the elements of the request body.

Element Name Required Description
name Yes Name of the pipeline.
description Yes Text describing what the activity is used for.
activity Yes Activity in the pipeline. A pipeline can have one or more activities.

Response

The response includes an HTTP status code, a set of response headers, and a response body.

Status Code

  • 200 (OK) - if the request completed successfully.

  • 202 (Accepted) - if the request was accepted.

  • 400 (Bad Request) - if the request body fails validation.

  • 404 (NotFound) - if the subscription or resource group does not exist.

  • 412 (Precondition Failed) - if the condition specified by If-Match header failed.

  • 501 (Not Implemented) - if validate is not implemented

Response Headers

The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.

Response Header Description
x-ms-request-id A unique identifier for the current operation, service generated.
x-ms-ratelimit-remaining-subscription-writes The remaining limit for current subscription.
x-ms-correlation-request-id Specifies the tracing correlation Id for the request; the resource provider must log this so that end-to-end requests can be correlated across Azure.
x-ms-routing-request-id Location+DateTime+correlation-request-ID
Date A UTC date/time value generated by the service that indicates the time at which the response was initiated.
Location URI of the object you want to create or get.

Response Body


{  
    "name": <name>,  
    "id": <ID>,  
    "properties": {  
        "description": <description>,  
        "activities": <activity>,  
        "isPaused": <Status>,  
        "provisioningState": <ProvisioningState>,  
        "hubName": <HubName>  
    }  
}  

The following table describes the elements of the response body.

Element Name Description
name Specifies the name of pipeline.
ID Specifies the identifying URL of the Pipeline.
Description Text describing what the activity is used for.
Activity Activity in the pipeline. A pipeline can have one or more activities.
Status Specifies whether this pipeline is paused or not.
Provisioningstate Specifies the current provisioning state of the pipeline. When a pipeline is successfully created, the value of the element is Succeeded.
Hubname The hub which contains your pipeline.

Sample Request and Response

Example URI:

PUT: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/SPRestDataFactory/datapipelines/ADFTutorialPipeline?api-version=2015-10-01  

The request is sent with the following headers:

x-ms-client-request-id        : 00000000-1111-1111-1111-000000000000  

and the following body:


{  
  "name": "ADFTutorialPipeline",  
  "properties": {  
    "description": "Copy data from a blob to Azure SQL table",  
    "activities": [  
      {  
        "name": "CopyFromBlobToSQL",  
        "description": "Push Regional Effectiveness Campaign data to Azure SQL database",  
        "type": "Copy",  
        "inputs": [  
          {  
            "name": "EmpTableFromBlob"  
          }  
        ],  
        "outputs": [  
          {  
            "name": "EmpSQLTable"  
          }  
        ],  
        "typeProperties": {  
          "source": {  
            "type": "BlobSource"  
          },  
          "sink": {  
            "type": "SqlSink"  
          }  
        },  
        "Policy": {  
          "concurrency": 1,  
          "executionPriorityOrder": "NewestFirst",  
          "style": "StartOfInterval",  
          "retry": 0,  
          "timeout": "01:00:00"  
        },  
        "scheduler": {  
          "frequency": "Hour",  
          "interval": 1  
        }  
      }  
    ],  
    "start": "2015-12-09T00:00:00Z",  
    "end": "2015-12-10T00:00:00Z",  
    "isPaused": false  
  }  
}  

After the request has been sent, the following response is returned.

Header:


Status Code:  
Created  

Headers:  
Pragma                        : no-cache  
x-ms-request-id               : 00000000-1111-1111-1111-000000000000  
x-ms-ratelimit-remaining-subscription-writes: 11999  
x-ms-correlation-request-id   : 00000000-1111-2222-1111-000000000000  
x-ms-routing-request-id       : WESTUS:20141203T213927Z: 00000000-1111-2222-1111-000000000000  
Strict-Transport-Security     : max-age=31536000; includeSubDomains  
Cache-Control                 : no-cache  
Date                          : Wed, 03 Dec 2014 21:39:27 GMT  
Location                      : https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/test/datapipelines/pipelinetest?api-version=2015-10-01  
Server                        : Microsoft-IIS/8.5  
X-Powered-By                  : ASP.NET  
client-tracking-id            : 51  

The response includes the following XML body:


Body:  
{  
  "name": "ADFTutorialPipeline",  
  "id":  
"/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/ADF/providers/Microsoft.DataFactory/datafactories/  
SPRestDataFactory/datapipelines/ADFTutorialPipeline",  
  "properties": {  
    "description": "Copy data from a blob to Azure SQL table",  
    "activities": [  
      {  
        "type": "Copy",  
        "typeProperties": {  
          "source": {  
            "type": "BlobSource"  
          },  
          "sink": {  
            "type": "SqlSink",  
            "writeBatchSize": 0,  
            "writeBatchTimeout": "00:00:00"  
          }  
        },  
        "inputs": [  
          {  
            "name": "EmpTableFromBlob"  
          }  
        ],  
        "outputs": [  
          {  
            "name": "EmpSQLTable"  
          }  
        ],  
        "policy": {  
          "timeout": "01:00:00",  
          "concurrency": 1,  
          "executionPriorityOrder": "NewestFirst",  
          "style": "StartOfInterval"  
        },  
        "scheduler": {  
          "frequency": "Hour",  
          "interval": 1  
        },  
        "name": "CopyFromBlobToSQL",  
        "description": "Push Regional Effectiveness Campaign data to Azure SQL database"  
      }  
    ],  
    "start": "2015-12-09T00:00:00Z",  
    "end": "2015-12-10T00:00:00Z",  
    "isPaused": false,  
    "runtimeInfo": {  
      "deploymentTime": "2016-02-22T22:40:24.9768561Z",  
      "activePeriodSetTime": "2016-02-22T22:40:25.7707093Z",  
      "activityPeriods": {  
        "copyFromBlobToSQL": {  
          "start": "2015-12-09T00:00:00Z",  
          "end": "2015-12-10T00:00:00Z"  
        }  
      }  
    },  
    "id": "00000000-0000-0000-0000-000000000000",  
    "provisioningState": "Succeeded",  
    "hubName": "sprestdatafactory_hub",  
    "pipelineMode": "Scheduled"  
  }  
}  

See Also

Pipelines and Activities in Azure Data Factory

Delete

Deletes the specified pipeline.

Request

The Delete Pipeline request may be constructed as follows (HTTPS is recommended):

HTTP Verb Request URI HTTP Version
DELETE https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName}/datapipelines/{PipelineName}?api-version={api-version} HTTP/1.1

URI Parameters

URI Parameter Required Description
SubscriptionID Yes Your Azure subscription ID.
ResourceGroupName Yes The unique name for the resource group that hosts your Azure data factory.
DataFactoryName Yes Name for the data factory that you want to delete your pipeline.
PipelineName Yes Name of pipeline you want to delete.
Api-Version Yes Specifies the version of the protocol used to make this request.

Request Headers

The following table describes the request headers.

Request Header Required Description
x-ms-client-request-id Yes The operation ID for this request.

Request Body

None.

Response

The response includes an HTTP status code, a set of response headers, and a response body.

Status Code

  • 200 (OK) - if the request was completed successfully.

  • 400 (Bad Request) - if request body fails validation.

  • 404 (NotFound) - if subscription or resource group does not exist.

  • 412 (Precondition Failed) - if condition specified by If-Match header failed.

  • 501 (Not Implemented) - if validate is not implemented

Response Headers

The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.

Response Header Description
x-ms-request-id A unique identifier for the current operation, service generated.
x-ms-ratelimit-remaining-subscription-writes The remaining limit for current subscription.
x-ms-correlation-request-id Specifies the tracing correlation Id for the request; the resource provider must log this so that end-to-end requests can be correlated across Azure.
x-ms-routing-request-id Location+DateTime+correlation-request-ID
Date A UTC date/time value generated by the service that indicates the time at which the response was initiated.

Response Body

None.

Sample Request and Response

Example URI:

DELETE: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/test/datapipelines/pipelinetest?api-version=2015-10-01  

The request is sent with the following headers.

x-ms-client-request-id        : 00000000-1111-1111-1111-000000000000  

After the request has been sent, the following response is returned.


Status Code:  
OK  

Headers:  
Pragma                        : no-cache  
x-ms-request-id               : 00000000-1111-1111-1111-000000000000  
x-ms-ratelimit-remaining-subscription-writes: 11999  
x-ms-correlation-request-id   : 00000000-1111-2222-1111-000000000000  
x-ms-routing-request-id       : WESTUS:20141203T213927Z: 00000000-1111-2222-1111-000000000000  
Strict-Transport-Security     : max-age=31536000; includeSubDomains  
Cache-Control                 : no-cache  
Date                          : Wed, 03 Dec 2014 21:39:27 GMT  
Server                        : Microsoft-IIS/8.5  
X-Powered-By                  : ASP.NET  
client-tracking-id            : 79  

See Also

Pipelines and Activities in Azure Data Factory

Get

Gets information about the specified pipeline.

Request

The Get Pipeline request may be constructed as follows (HTTPS is recommended):

HTTP Verb Request URI HTTP Version
GET https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName}/datapipelines/{PipelineName}?api-version={api-version} HTTP/1.1

URI Parameters

URI Parameter Required Description
SubscriptionID Yes Your Azure subscription ID.
ResourceGroupName Yes A unique name for the resource group that hosts your Azure data factory.
DataFactoryName Yes Name for the data factory that you want to find your pipeline.
PipelineName Yes Name of pipeline you want to find.
Api-Version Yes Specifies the version of the protocol used to make this request.

Request Headers

The following table describes the request headers.

Request Header Required Description
x-ms-client-request-id Yes The operation id for this request.

Request Body

None.

Response

The response includes an HTTP status code, a set of response headers, and a response body.

Status Code

  • 200 (OK) - if the request was completed successfully.

  • 400 (Bad Request) - if the request body fails validation.

  • 404 (NotFound) - if the subscription or resource group does not exist.

  • 412 (Precondition Failed) - if the condition specified by If-Match header failed.

Response Headers

The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.

Response Header Description
x-ms-request-id A unique identifier for the current operation, service generated.
x-ms-ratelimit-remaining-subscription-writes The remaining limit for current subscription.
x-ms-correlation-request-id Specifies the tracing correlation Id for the request; the resource provider must log this so that end-to-end requests can be correlated across Azure.
x-ms-routing-request-id Location+DateTime+correlation-request-ID
Date A UTC date/time value generated by the service that indicates the time at which the response was initiated.

Response Body

Response body depends on your pipeline property, for more information about pipelines, see Pipelines and Activities.


{  
    "name": <name>,  
    "id": <ID>,  
    "properties": {  
        "description": <description>,  
        "activities": <activity>,  
        "isPaused": <Status>,  
        "provisioningState": <ProvisioningState>,  
        "hubName": <HubName>  
    }  
}  

The following table describes the elements of the response body.

Element Name Description
name Specifies the name of pipeline.
ID Specifies the identifying URL of the Pipeline.
Description Text describing what the activity is used for.
Activity Activity in the pipeline. A pipeline can have one or more activities.
Status Specifies whether this pipeline is paused or not
Provisioningstate Specifies the current provisioning state of the pipeline. When a pipeline is successfully created, the value of the element is Succeeded.
Hubname The hub which contains your pipeline.

Sample Request and Response

Example URI:

GET: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/SPRestDataFactory/datapipelines/ADFTutorialPipeline?api-version=2015-10-01  

The request is sent with the following headers.

x-ms-client-request-id        : 00000000-1111-1111-1111-000000000000  

After the request has been sent, the following response is returned.

Header:


Status Code:  
OK  

Headers:  
Pragma                        : no-cache  
x-ms-request-id               : 00000000-1111-1111-1111-000000000000  
x-ms-ratelimit-remaining-subscription-writes: 11999  
x-ms-correlation-request-id   : 00000000-1111-2222-1111-000000000000  
x-ms-routing-request-id       : WESTUS:20141203T213927Z: 00000000-1111-2222-1111-000000000000  
Strict-Transport-Security     : max-age=31536000; includeSubDomains  
Cache-Control                 : no-cache  
Date                          : Wed, 03 Dec 2014 21:39:27 GMT  
Server                        : Microsoft-IIS/8.5  
X-Powered-By                  : ASP.NET  

The response includes the following XML body.


{  
  "name": "ADFTutorialPipeline",  
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/ADF/providers/Microsoft.DataFactory/datafactories/SPRestDataFactory/datapipelines/ADFTutorialPipeline",  
  "properties": {  
    "description": "Copy data from a blob to Azure SQL table",  
    "activities": [  
      {  
        "type": "Copy",  
        "typeProperties": {  
          "source": {  
            "type": "BlobSource"  
          },  
          "sink": {  
            "type": "SqlSink",  
            "writeBatchSize": 0,  
            "writeBatchTimeout": "00:00:00"  
          }  
        },  
        "inputs": [  
          {  
            "name": "EmpTableFromBlob"  
          }  
        ],  
        "outputs": [  
          {  
            "name": "EmpSQLTable"  
          }  
        ],  
        "policy": {  
          "timeout": "01:00:00",  
          "concurrency": 1,  
          "executionPriorityOrder": "NewestFirst",  
          "style": "StartOfInterval"  
        },  
        "scheduler": {  
          "frequency": "Hour",  
          "interval": 1  
        },  
        "name": "CopyFromBlobToSQL",  
        "description": "Push Regional Effectiveness Campaign data to Azure SQL database"  
      }  
    ],  
    "start": "2015-12-09T00:00:00Z",  
    "end": "2015-12-10T00:00:00Z",  
    "isPaused": false,  
    "runtimeInfo": {  
      "deploymentTime": "2016-02-22T22:40:24.9768561Z",  
      "activePeriodSetTime": "2016-02-22T22:40:25.7707093Z",  
      "activityPeriods": {  
        "copyFromBlobToSQL": {  
          "start": "2015-12-09T00:00:00Z",  
          "end": "2015-12-10T00:00:00Z"  
        }  
      }  
    },  
    "id": "00000000-0000-0000-0000-000000000000",  
    "provisioningState": "Succeeded",  
    "hubName": "sprestdatafactory_hub",  
    "pipelineMode": "Scheduled"  
  }  
}  

See Also

Pipelines and Activities in Azure Data Factory

List

Lists information about all linked services in a data factory.

Request

The List Pipeline request may be constructed as follows (HTTPS is recommended):

HTTP Verb Request URI HTTP Version
GET https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName}/datapipelines?api-version={api-version} HTTP/1.1

URI Parameters

URI Parameter Required Description
SubscriptionID Yes Your Azure subscription ID.
ResourceGroupName Yes A unique name for the resource group that hosts your Azure data factory Service.
DataFactoryName Yes Name for the data factory that you want to find your pipeline.
Api-Version Yes Specifies the version of the protocol used to make this request.

Request Headers

The following table describes the request headers.

Request Header Required Description
x-ms-client-request-id Yes The operation id for this request.

Request Body

None.

Response

The response includes an HTTP status code, a set of response headers, and a response body.

Status Code

  • 200 (OK) - if the request completed successfully.

  • 400 (Bad Request) - if the request body fails validation.

  • 404 (NotFound) - if the subscription or resource group does not exist.

  • 412 (Precondition Failed) - if the condition specified by If-Match header failed.

Response Headers

The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.

Response Header Description
x-ms-request-id A unique identifier for the current operation, service generated.
x-ms-ratelimit-remaining-subscription-writes The remaining limit for current subscription.
x-ms-correlation-request-id Specifies the tracing correlation Id for the request. The resource provider must log this so that end-to-end requests can be correlated across Azure.
x-ms-routing-request-id Location+DateTime+correlation-request-ID
Date A UTC date/time value generated by the service that indicates the time at which the response was initiated.

Response Body


{  
    "value": [  
    {  
        "name": <name>,  
        "id": <ID>,  
        "properties": {  
            "description": <description>,  
            "activities": <activity>,  
            "isPaused": <Status>,  
            "provisioningState": <ProvisioningState>,  
            "hubName": <HubName>  
        }  
    },  
    {  
        "name": <name2>,  
        "id": <ID>,  
        "properties": {  
            "description": <description>,  
            "activities": <activity>,  
            "isPaused": <Status>,  
            "provisioningState": <ProvisioningState>,  
            "hubName": <HubName>  
        }  
    }  
    ]  
}  

The following table describes the elements of the response body.

Element Name Description
name Specifies the name of pipeline.
ID Specifies the identifying URL of the Pipeline.
Description Text describing what the activity is used for.
Activity Activity in the pipeline, a pipeline can have one or more activities.
Status Specifies whether this pipeline is paused or not.
Provisioningstate Specifies the current provisioning state of the pipeline. When a pipeline is successfully created, the value of the element is Succeeded.
Hubname The hub which contains your pipeline.

See Pipelines and Activities in Azure Data Factory for more details.

If there are more than 100 records, you will get 100 records at a time with a nextLink:

{
    "value": [  
    {  
        "name": <name>,  
        "id": <ID>,  
        "properties": {  
            "description": <description>,  
            "activities": <activity>,  
            "isPaused": <Status>,  
            "provisioningState": <ProvisioningState>,  
            "hubName": <HubName>  
        }  
    },  
    {  
        "name": <name2>,  
        "id": <ID>,  
        "properties": {  
            "description": <description>,  
            "activities": <activity>,  
            "isPaused": <Status>,  
            "provisioningState": <ProvisioningState>,  
            "hubName": <HubName>  
        }  
    }  
    ] ,
    "nextLink": "https://management.azure.com/subscriptions/{subscription ID>/resourcegroups/{Resource group name>/providers/Microsoft.DataFactory/datafactories/{data factory name>/datapipelines?api-version=2015-10-01&%24skipToken=MSExNiFSR0YwWVVGeWRHbG1ZV04wCjEhNTIhUkVGZlpUSXlaR1UyTXpndFpURTJNeTAwWXpZNUxUaGhNR0l0Wm1VMU1ETmpOakF4T0RObA%3d%3d"
}  

You send the next request using the nextLink URL to get the next set of records. You may need to wrap & with double quotes ("") in the nextLink URL. Only 100 records are returned at a time.

Sample Request and Response

Example URI:

GET: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/SPRestDataFactory/datapipelines?api-version=2015-10-01  

The request is sent with the following headers.

x-ms-client-request-id        : 00000000-1111-1111-1111-000000000000  

After the request has been sent, the following response is returned.


Status Code:  
OK  

Headers:  
Pragma                        : no-cache  
x-ms-request-id               : 00000000-1111-1111-1111-000000000000  
x-ms-ratelimit-remaining-subscription-writes: 799985  
x-ms-correlation-request-id   : 00000000-1111-2222-1111-000000000000  
x-ms-routing-request-id       : WESTUS:20141203T213927Z: 00000000-1111-2222-1111-000000000000  
Strict-Transport-Security     : max-age=31536000; includeSubDomains  
Cache-Control                 : no-cache  
Date                          : Wed, 03 Dec 2014 21:39:27 GMT  
Server                        : Microsoft-IIS/8.5  
X-Powered-By                  : ASP.NET  

The response includes the following XML body.


{  
  "value": [  
    {  
      "name": "FirstPipeline",  
      "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/ADF/providers/Microsoft.DataFactory/datafactories/SPRestDataFactory/datapipelines/FirstPipeline",  
      "properties": {  
        "activities": [  
          {  
            "type": "Copy",  
            "typeProperties": {  
              "source": null,  
              "sink": null,  
              "translator": null  
            },  
            "inputs": [  
              {  
                "name": "InputTable1"  
              }  
            ],  
            "outputs": [  
              {  
                "name": "OutputTable1"  
              }  
            ],  
            "scheduler": {  
              "frequency": "Hour",  
              "interval": 1  
            },  
            "name": "First pipeline"  
          }  
        ],  
        "start": "2015-12-09T00:00:00Z",  
        "end": "2015-12-10T00:00:00Z",  
        "isPaused": false,  
        "runtimeInfo": {  
          "deploymentTime": "2016-02-22T22:40:24.9768561Z",  
          "activityPeriods": {  
            "copyFromBlobToSQL": {  
              "start": "2015-12-09T00:00:00Z",  
              "end": "2015-12-10T00:00:00Z"  
            }  
          }  
        },  
        "id": "c5326985-fd60-4f46-bbae-130d318759c6",  
        "provisioningState": "Succeeded",  
        "pipelineMode": "Scheduled"  
      }  
    },  
{  
  "value": [  
    {  
      "name": "SecondPipeline",  
      "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/ADF/providers/Microsoft.DataFactory/datafactories/SPRestDataFactory/datapipelines/SecondPipeline",  
      "properties": {  
        "activities": [  
          {  
            "type": "Copy",  
            "typeProperties": {  
              "source": null,  
              "sink": null,  
              "translator": null  
            },  
            "inputs": [  
              {  
                "name": "InputTable2"  
              }  
            ],  
            "outputs": [  
              {  
                "name": "OutputTable2"  
              }  
            ],  
            "scheduler": {  
              "frequency": "Hour",  
              "interval": 1  
            },  
            "name": "Second pipeline"  
          }  
        ],  
        "start": "2015-12-09T00:00:00Z",  
        "end": "2015-12-10T00:00:00Z",  
        "isPaused": false,  
        "runtimeInfo": {  
          "deploymentTime": "2016-02-22T22:40:24.9768561Z",  
          "activityPeriods": {  
            "copyFromBlobToSQL": {  
              "start": "2015-12-09T00:00:00Z",  
              "end": "2015-12-10T00:00:00Z"  
            }  
          }  
        },  
        "id": "c5326985-fd60-4f46-bbae-130d318759c6",  
        "provisioningState": "Succeeded",  
        "pipelineMode": "Scheduled"  
      }  
    }  
  ]  
}  

See Also

Pipelines and Activities in Azure Data Factory

List Activity Windows

Lists the first page of activity window instances associated with a pipeline along with a link to the next page.

Request

The Get Activity window request may be constructed as follows (HTTPS is recommended):

HTTP Verb Request URI HTTP Version
POST https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName/datapipelines/{PipelineName}/activitywindows?api-version={api-version} HTTP/1.1

URI Parameters

URI Parameter Required Description
SubscriptionID Yes Your Azure subscription ID.
ResourceGroupName Yes Name of the resource group that hosts your Azure data factory.
DataFactoryName Yes Name of the data factory.
PipelineName Yes Name of the pipeline.
Api-Version Yes Specifies the version of the protocol used to make this request.

Request Headers

The following table describes the request headers.

Request Header Required Description
x-ms-client-request-id Yes The operation id for this request.

Request Body

Element name Description Required
Filter The activity window filter specified using Azure Search filter grammar. See https://msdn.microsoft.com/library/azure/dn798921.aspx for details. No
OrderBy Order the response by specified activity window properties. This is a list of comma separated properties. For example: WindowStart, PercentComplete. By befault, the order is ascending order (ASC). Specify DESC if you want the list to be ordered in descending order. No
Top The number of activity windows to be listed. No
RunStart The start time of activity window run. No
RunEnd The end time of activity window run. No
WindowStart The start time of activity window. No
WindowEnd The end time of activity window. No
WindowState The state of activity window. Must be one of the values specified in this article. No
WindowSubstate The substate of activity window. Must be one of the values specified in this article. No

Response

The response includes an HTTP status code, a set of response headers, and a response body.

Status Code

  • 200 (OK) - if the request completed successfully.

  • 400 (Bad Request) - if the request body fails validation.

  • 404 (NotFound) - if the subscription or resource group does not exist.

Response Headers

The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.

Response Header Description
x-ms-request-id A unique identifier for the current operation, service generated.
x-ms-ratelimit-remaining-subscription-writes The remaining limit for current subscription.
x-ms-correlation-request-id Specifies the tracing correlation Id for the request. The resource provider must log this so that end-to-end requests can be correlated across Azure.
x-ms-routing-request-id Location+DateTime+correlation-request-ID
Date A UTC date/time value generated by the service that indicates the time at which the response was initiated.

Response Body


{  
    "value": {  
    "activityWindows": [  
      {  
        "resourceGroupName": "<resource group name>",  
        "dataFactoryName": "<data factory name>",  
        "pipelineName": "<pipeline name>",  
        "activityName": "<Pipeline activity name.>",  
        "linkedServiceName": "<linked service name>",  
        "activityType": "<The activity type>",  
        "runAttempts": "<Number of activity run attempts.>",  
        "runStart": "<start time of the last run>",  
        "runEnd": "<end time of the last run>",  
        "duration": "<duration of the activity window>",  
        "percentComplete": "<percent completion of activity window>",  
        "windowStart": "<window start time>",  
        "windowEnd": "<window end time>",  
        "windowState": "<window state>",  
        "windowSubstate": "<window substate>",  
        "inputDatasets": "[<input dataset names>]",  
        "outputDatasets": [  
          "<output dataset names>"  
        ],  
        "inputDatasetIds":["<input dataset ids>"],  
        "outputDatasetIds": [  
          "<output dataset ids>" 
        ]  
      }  
    ],  
    "lastUpdate":"<The approximate time for the last activity windows>",  
    "aggregates": {}  
  },  
    "nextLink": "<Link to next page>"  
}  

Sample Request and Response

Example URI:

POST:https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/ADF/providers/Microsoft.DataFactory/datafactories/dfName/datapipelines/pipelineName/activitywindows?api-version=2015-10-01  

The request is sent with the following headers and the body.

Header:

x-ms-client-request-id        : 00000000-1111-1111-1111-000000000000  

Body:

{
  "top": 2
}

After the request has been sent, the following response is returned.


Status Code:  
OK  

Headers:  
Pragma                        : no-cache  
x-ms-request-id               : 00000000-1111-1111-1111-000000000000  
x-ms-ratelimit-remaining-subscription-writes: 799985  
x-ms-correlation-request-id   : 00000000-1111-2222-1111-000000000000  
x-ms-routing-request-id       : WESTUS:20141203T213307Z: 00000000-1111-2222-1111-000000000000  
Strict-Transport-Security     : max-age=31536000; includeSubDomains  
Cache-Control                 : no-cache  
Date                          : Wed, 03 Dec 2014 21:33:07 GMT  
Server                        : Microsoft-IIS/8.5  
X-Powered-By                  : ASP.NET  

The response includes the following XML body.


"value": {  
    "activityWindows": [  
      {  
        "resourceGroupName": "rgName",  
        "dataFactoryName": "dfName",  
        "pipelineName": "pipelineName",  
        "activityName": "DA_WikipediaClickEvents",  
        "linkedServiceName": "LS_Wikipedia",  
        "activityType": "Copy",  
        "runAttempts": 1,  
        "runStart": "2016-01-26T21:41:48.636Z",  
        "runEnd": "2016-01-26T21:41:48.886Z",  
        "duration": "00:00:00.0000250",  
        "percentComplete": 100,  
        "windowStart": "2016-01-25T01:00:00Z",  
        "windowEnd": "2016-01-25T02:00:00Z",  
        "windowState": "Pending",  
        "windowSubstate": "Validating",  
        "inputDatasets": ["DA_WikipediaEvents"],  
        "outputDatasets": [  
           "dsName"  
        ],  
        "inputDatasetIds": ["308F5E07-5B0C-4AE9-95EB-9CEF26CFDD21"],  
        "outputDatasetIds": [  
          "1e4a0e42-5df3-4a7e-9860-7f75d306ff75"  
        ]  
      },
      {  
        "resourceGroupName": "rgName",  
        "dataFactoryName": "dfName",  
        "pipelineName": "pipelineName",  
        "activityName": "DA_WikipediaClickEvents",  
        "linkedServiceName": "LS_Wikipedia",  
        "activityType": "Copy",  
        "runAttempts": 1,  
        "runStart": "2016-01-26T21:41:48.636Z",  
        "runEnd": "2016-01-26T21:41:48.886Z",  
        "duration": "00:00:00.0000250",  
        "percentComplete": 100,  
        "windowStart": "2016-01-25T02:00:00Z",  
        "windowEnd": "2016-01-25T03:00:00Z",  
        "windowState": "Pending",  
        "windowSubstate": "Validating",  
        "inputDatasets": ["DA_WikipediaEvents"],  
        "outputDatasets": [  
           "dsName"  
        ],  
        "inputDatasetIds": ["308F5E07-5B0C-4AE9-95EB-9CEF26CFDD21"],  
        "outputDatasetIds": [  
          "1e4a0e42-5df3-4a7e-9860-7f75d306ff75"  
        ]  
      }  
    ],  
    "lastUpdate": "2016-01-26T21:43:45.6837301+00:00",  
    "aggregates": {}  
  },  
    "nextLink": "https://localhost:86/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/rgName/datafactories/dfName/datapipelines/pipelineName/activitywindows/skip=FG154H?ApiVerion01012015"}  

Use the nextLink URL to get the next page of results.

List Activity Windows by Activity

Lists the first page of activity window instances associated with an activity along with a link to the next page.

Request

The Get Activity window request may be constructed as follows (HTTPS is recommended):

HTTP Verb Request URI HTTP Version
POST https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName/datapipelines/{PipelineName}/activities/{ActivityName}/activitywindows?api-version={api-version} HTTP/1.1

URI Parameters

URI Parameter Required Description
SubscriptionID Yes Your Azure subscription ID.
ResourceGroupName Yes Name of the resource group that hosts your Azure data factory.
DataFactoryName Yes Name of the data factory.
PipelineName Yes Name of the pipeline.
ActivityName Yes Name of the activity.
Api-Version Yes Specifies the version of the protocol used to make this request.

Request Headers

The following table describes the request headers.

Request Header Required Description
x-ms-client-request-id Yes The operation id for this request.

Request Body

Element name Description Required
Filter The activity window filter specified using Azure Search filter grammar. See https://msdn.microsoft.com/library/azure/dn798921.aspx for details. No
OrderBy Order the response by specified activity window properties. This is a list of comma separated properties. For example: WindowStart, PercentComplete. By befault, the order is ascending order (ASC). Specify DESC if you want the list to be ordered in descending order. No
Top The number of activity windows to be listed. No
RunStart The start time of activity window run. No
RunEnd The end time of activity window run. No
WindowStart The start time of activity window. No
WindowEnd The end time of activity window. No
WindowState The state of the activity window. No
WindowSubstate The state of the activity window. No

Response

The response includes an HTTP status code, a set of response headers, and a response body.

Status Code

  • 200 (OK) - if the request completed successfully.

  • 400 (Bad Request) - if the request body fails validation.

  • 404 (NotFound) - if the subscription or resource group does not exist.

Response Headers

The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.

Response Header Description
x-ms-request-id A unique identifier for the current operation, service generated.
x-ms-ratelimit-remaining-subscription-writes The remaining limit for current subscription.
x-ms-correlation-request-id Specifies the tracing correlation Id for the request. The resource provider must log this so that end-to-end requests can be correlated across Azure.
x-ms-routing-request-id Location+DateTime+correlation-request-ID
Date A UTC date/time value generated by the service that indicates the time at which the response was initiated.

Response Body


{  
    "value": {  
    "activityWindows": [  
      {  
        "resourceGroupName": "<resource group name>",  
        "dataFactoryName": "<data factory name>",  
        "pipelineName": "<pipeline name>",  
        "activityName": "<Pipeline activity name.>",  
        "linkedServiceName": "<linked service name>",  
        "activityType": "<The activity type>",  
        "runAttempts": "<Number of activity run attempts.>",  
        "runStart": "<start time of the last run>",  
        "runEnd": "<end time of the last run>",  
        "duration": "<duration of the activity window>",  
        "percentComplete": "<percent completion of activity window>",  
        "windowStart": "<window start time>",  
        "windowEnd": "<window end time>",  
        "windowState": "<window state>",  
        "windowSubstate": "<window substate>",  
        "inputDatasets": "[<input dataset names>]",  
        "outputDatasets": [  
          "<output dataset names>"  
        ],  
        "inputDatasetIds":["<input dataset ids>"],  
        "outputDatasetIds": [  
          "<output dataset ids>" 
        ]  
      }  
    ],  
    "lastUpdate":"<The approximate time for the last activity windows>",  
    "aggregates": {}  
  },  
    "nextLink": "<Link to next page>"  
}  

Sample Request and Response

Example URI:

POST:https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/ADF/providers/Microsoft.DataFactory/datafactories/dfName/datapipelines/pipelineName/activities/activityName/activitywindows?api-version=2015-10-01  

The request is sent with the following header and the body:

Header:

x-ms-client-request-id        : 00000000-1111-1111-1111-000000000000  

Body:

{
  "top": 2
}

After the request has been sent, the following response is returned.


Status Code:  
OK  

Headers:  
Pragma                        : no-cache  
x-ms-request-id               : 00000000-1111-1111-1111-000000000000  
x-ms-ratelimit-remaining-subscription-writes: 799985  
x-ms-correlation-request-id   : 00000000-1111-2222-1111-000000000000  
x-ms-routing-request-id       : WESTUS:20141203T213307Z: 00000000-1111-2222-1111-000000000000  
Strict-Transport-Security     : max-age=31536000; includeSubDomains  
Cache-Control                 : no-cache  
Date                          : Wed, 03 Dec 2014 21:33:07 GMT  
Server                        : Microsoft-IIS/8.5  
X-Powered-By                  : ASP.NET  

The response includes the following XML body.


"value": {  
    "activityWindows": [  
      {  
        "resourceGroupName": "rgName",  
        "dataFactoryName": "dfName",  
        "pipelineName": "pipelineName",  
        "activityName": "activityName",  
        "linkedServiceName": "LS_Wikipedia",  
        "activityType": "Copy",  
        "runAttempts": 1,  
        "runStart": "2016-01-26T21:41:48.636Z",  
        "runEnd": "2016-01-26T21:41:48.886Z",  
        "duration": "00:00:00.0000250",  
        "percentComplete": 100,  
        "windowStart": "2016-01-25T01:00:00Z",  
        "windowEnd": "2016-01-25T02:00:00Z",  
        "windowState": "Pending",  
        "windowSubstate": "Validating",  
        "inputDatasets": ["DA_WikipediaEvents"],  
        "outputDatasets": [  
           "dsName"  
        ],  
        "inputDatasetIds": ["308F5E07-5B0C-4AE9-95EB-9CEF26CFDD21"],  
        "outputDatasetIds": [  
          "1e4a0e42-5df3-4a7e-9860-7f75d306ff75"  
        ]  
      },
        {  
        "resourceGroupName": "rgName",  
        "dataFactoryName": "dfName",  
        "pipelineName": "pipelineName",  
        "activityName": "activityName",  
        "linkedServiceName": "LS_Wikipedia",  
        "activityType": "Copy",  
        "runAttempts": 1,  
        "runStart": "2016-01-26T21:41:48.636Z",  
        "runEnd": "2016-01-26T21:41:48.886Z",  
        "duration": "00:00:00.0000250",  
        "percentComplete": 100,  
        "windowStart": "2016-01-25T02:00:00Z",  
        "windowEnd": "2016-01-25T03:00:00Z",  
        "windowState": "Pending",  
        "windowSubstate": "Validating",  
        "inputDatasets": ["DA_WikipediaEvents"],  
        "outputDatasets": [  
           "dsName"  
        ],  
        "inputDatasetIds": ["308F5E07-5B0C-4AE9-95EB-9CEF26CFDD21"],  
        "outputDatasetIds": [  
          "1e4a0e42-5df3-4a7e-9860-7f75d306ff75"  
        ]  
      }
    ],  
    "lastUpdate": "2016-01-26T21:43:45.6837301+00:00",  
    "aggregates": {}  
  },  
    "nextLink": "https://localhost:86/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/rgName/datafactories/dfName/datapipelines/pipelineName/activities/activityName/activitywindows/skip=FG154H?ApiVerion01012015"}  

Use the nextLink URL to get the next page of results.

Resume

Resumes a suspended pipeline in data factory.

Request

The Resume Pipeline request may be constructed as follows (HTTPS is recommended):

HTTP Verb Request URI HTTP Version
POST https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName}/datapipelines/{PipelineName}/resume?api-version={api-version} HTTP/1.1

URI Parameters

URI Parameter Required Description
SubscriptionID Yes Your Azure subscription ID
ResourceGroupName Yes A unique name for the resource group that hosts your Azure data factory.
DataFactoryName Yes Name for the data factory that you want to find your pipeline.
PipelineName Yes Name of pipeline you want to resume.
Api-Version Yes Specifies the version of the protocol used to make this request.

Request Headers

The following table describes the request headers.

Request Header Required Description
x-ms-client-request-id Yes The operation id for this request.

Request Body

None.

Response

The response includes an HTTP status code, a set of response headers, and a response body.

Status Code

  • 200 (OK) - if the request was completed successfully.

  • 400 (Bad Request) - if the request body fails validation.

  • 404 (NotFound) - if the subscription or resource group does not exist.

Response Headers

The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.

Response Header Description
x-ms-request-id A unique identifier for the current operation, service generated.
x-ms-ratelimit-remaining-subscription-writes The remaining limit for current subscription.
x-ms-correlation-request-id Specifies the tracing correlation Id for the request; the resource provider must log this so that end-to-end requests can be correlated across Azure.
x-ms-routing-request-id Location+DateTime+correlation-request-ID
Date A UTC date/time value generated by the service that indicates the time at which the response was initiated.

Response Body

None.

Sample Request and Response

Example URI:

POST: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/test/datapipelines/pipelinetest/resume?api-version=2015-10-01  

The request is sent with the following headers.

x-ms-client-request-id        : 00000000-1111-1111-1111-000000000000  

After the request has been sent, the following response is returned.


Status Code:  
OK  

Headers:  
Pragma                        : no-cache  
x-ms-request-id               : 00000000-1111-1111-1111-000000000000  
x-ms-ratelimit-remaining-subscription-writes: 11995  
x-ms-correlation-request-id   : 00000000-1111-2222-1111-000000000000  
x-ms-routing-request-id       : WESTUS:20141203T213927Z: 00000000-1111-2222-1111-000000000000  
Strict-Transport-Security     : max-age=31536000; includeSubDomains  
Cache-Control                 : no-cache  
Date                          : Wed, 03 Dec 2014 21:39:27 GMT  
Server                        : Microsoft-IIS/8.5  
X-Powered-By                  : ASP.NET  

See Also

Pipelines and Activities in Azure Data Factory

Set Active Period

Configures the active period for a data pipeline.

Request

The Set Slice Status request may be constructed as follows (HTTPS is recommended):

HTTP Verb Request URI HTTP Version
POST https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName}/datapipelines/{PipelineName}/update?start={StartDateTime}&end={EndDateTime}&autoResolve=false&forceRecalc=false&api-version={api-version} HTTP/1.1

URI Parameters

Insert subsection body here.

URI Parameter Required Description
SubscriptionID Yes Your Azure subscription ID.
ResourceGroupName Yes A unique name for the resource group that hosts your Azure data factory.
DataFactoryName Yes Name for the data factory that you want to set your pipeline.
PipelineName Yes Name of table you want to create.
StartDateTime Yes The datetime when slices start.
EndDateTime Yes The datetime when slices end. The default datetime is 48 hours after start datetime.
autoResolve No Indicates that this request uses auto-resolve or not.
forceRecalc No Indicates that this request uses force recalculate or not.
Api-Version Yes Specifies the version of the protocol used to make this request.

Request Headers

The following table describes the request headers.

Request Header Required Description
x-ms-client-request-id Yes The operation id for this request.

Request Body

None.

Response

The response includes an HTTP status code, a set of response headers, and a response body.

Status Code

  • 200 (OK) - if the request completed successfully.

  • 400 (Bad Request) - if the request body fails validation.

  • 404 (NotFound) - if the subscription or resource group does not exist.

Response Headers

The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.

Response Header Description
x-ms-request-id A unique identifier for the current operation, service generated.
x-ms-ratelimit-remaining-subscription-writes The remaining limit for current subscription.
x-ms-correlation-request-id Specifies the tracing correlation Id for the request. The resource provider must log this so that end-to-end requests can be correlated across Azure.
x-ms-routing-request-id Location+DateTime+correlation-request-ID
Date A UTC date/time value generated by the service that indicates the time at which the response was initiated.
Location URI of the object you want to create or get.

Response Body

None.

Sample Request and Response

Example URI:

POST: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/test/datapipelines/pipelinetest/update?start=2014-12-02T00:00:00.0000000&end=2014-12-02T01:00:00.0000000& autoResolve=false&forceRecalc=false&api-version=2015-10-01  

The request is sent with the following headers.

x-ms-client-request-id        : 00000000-1111-1111-1111-000000000000  

After the request has been sent, the following response is returned.


Status Code:  
OK  

Headers:  
Pragma                        : no-cache  
x-ms-request-id               : 00000000-1111-1111-1111-000000000000  
x-ms-ratelimit-remaining-subscription-writes: 799985  
x-ms-correlation-request-id   : 00000000-1111-2222-1111-000000000000  
x-ms-routing-request-id       : WESTUS:20141203T213307Z: 00000000-1111-2222-1111-000000000000  
Strict-Transport-Security     : max-age=31536000; includeSubDomains  
Cache-Control                 : no-cache  
Date                          : Wed, 03 Dec 2014 21:33:07 GMT  
Server                        : Microsoft-IIS/8.5  
X-Powered-By                  : ASP.NET  

See Also

Pipelines and Activities in Azure Data Factory

Suspend

Pauses an existing running pipeline.

Request

The Suspend Pipeline request may be constructed as follows (HTTPS is recommended):

HTTP Verb Request URI HTTP Version
POST https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName}/datapipelines/{PipelineName}/pause?api-version={api-version} HTTP/1.1

URI Parameters

URI Parameter Required Description
SubscriptionID Yes Your Azure subscription ID.
ResourceGroupName Yes A unique name for the resource group that hosts your Azure data factory.
DataFactoryName Yes Name for the data factory that you want to find your pipeline.
PipelineName Yes Name of pipeline you want to suspend.
Api-Version Yes Specifies the version of the protocol used to make this request.

Request Headers

The following table describes the request headers.

Request Header Required Description
x-ms-client-request-id Yes The operation id for this request.

Request Body

None.

Response

The response includes an HTTP status code, a set of response headers, and a response body.

  • 200 (OK) - if the request completed successfully.

  • 400 (Bad Request) - if the request body fails validation.

  • 404 (NotFound) - if the subscription or resource group does not exist.

Response Headers

The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.

Response Header Description
x-ms-request-id A unique identifier for the current operation, service generated.
x-ms-ratelimit-remaining-subscription-writes The remaining limit for current subscription.
x-ms-correlation-request-id Specifies the tracing correlation Id for the request. The resource provider must log this so that end-to-end requests can be correlated across Azure.
x-ms-routing-request-id Location+DateTime+correlation-request-ID
Date A UTC date/time value generated by the service that indicates the time at which the response was initiated.

Response Body

None.

Sample Request and Response

Example:.

POST: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/test/datapipelines/pipelinetest/pause?api-version=2015-10-01  

The request is sent with the following headers.

x-ms-client-request-id        : 00000000-1111-1111-1111-000000000000  

After the request has been sent, the following response is returned.


Status Code:  
OK  

Headers:  
Pragma                        : no-cache  
x-ms-request-id               : 00000000-1111-1111-1111-000000000000  
x-ms-ratelimit-remaining-subscription-writes: 11995  
x-ms-correlation-request-id   : 00000000-1111-2222-1111-000000000000  
x-ms-routing-request-id       : WESTUS:20141203T213927Z: 00000000-1111-2222-1111-000000000000  
Strict-Transport-Security     : max-age=31536000; includeSubDomains  
Cache-Control                 : no-cache  
Date                          : Wed, 03 Dec 2014 21:39:27 GMT  
Server                        : Microsoft-IIS/8.5  
X-Powered-By                  : ASP.NET  

See Also

Pipelines and Activities in Azure Data Factory