Pipeline version 1 (V1)
Note
This reference documentation applies to Azure Data Factory version 1 (V1).
Create or Update
Creates a new pipeline, or updates the content of an existing pipeline.
Request
The Create or Update Pipeline request may be constructed as follows. HTTPS is recommended:
| HTTP Verb | Request URI | HTTP Version |
|---|---|---|
| PUT | https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName}/datapipelines/{PipelineName}?api-version={api-version} | HTTP/1.1 |
URI Parameters
| URI Parameter | Required | Description |
|---|---|---|
| SubscriptionID | Yes | Your Azure subscription ID. |
| ResourceGroupName | Yes | A unique name for the resource group that hosts your Azure data factory. |
| DataFactoryName | Yes | Name for the data factory that you want to create your pipeline in. |
| PipelineName | Yes | Name of pipeline you want to create. |
| Api-Version | Yes | Specifies the version of the protocol used to make this request. |
Request Headers
The following table describes the request headers:
| Request Header | Required | Description |
|---|---|---|
| x-ms-client-request-id | Yes | The operation id for this request. |
Request Body
Request body depends on your pipeline property. For more information about pipelines, see Pipelines and Activities in Azure Data Factory article for details.
The format of the request body is as follows:
{
"name": <name>,
"properties": {
"description": <description>,
"activities": <activity>
}
}
The following table describes the elements of the request body.
| Element Name | Required | Description |
|---|---|---|
| name | Yes | Name of the pipeline. |
| description | Yes | Text describing what the activity is used for. |
| activity | Yes | Activity in the pipeline. A pipeline can have one or more activities. |
Response
The response includes an HTTP status code, a set of response headers, and a response body.
Status Code
200 (OK) - if the request completed successfully.
202 (Accepted) - if the request was accepted.
400 (Bad Request) - if the request body fails validation.
404 (NotFound) - if the subscription or resource group does not exist.
412 (Precondition Failed) - if the condition specified by If-Match header failed.
501 (Not Implemented) - if validate is not implemented
Response Headers
The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.
| Response Header | Description |
|---|---|
| x-ms-request-id | A unique identifier for the current operation, service generated. |
| x-ms-ratelimit-remaining-subscription-writes | The remaining limit for current subscription. |
| x-ms-correlation-request-id | Specifies the tracing correlation Id for the request; the resource provider must log this so that end-to-end requests can be correlated across Azure. |
| x-ms-routing-request-id | Location+DateTime+correlation-request-ID |
| Date | A UTC date/time value generated by the service that indicates the time at which the response was initiated. |
| Location | URI of the object you want to create or get. |
Response Body
{
"name": <name>,
"id": <ID>,
"properties": {
"description": <description>,
"activities": <activity>,
"isPaused": <Status>,
"provisioningState": <ProvisioningState>,
"hubName": <HubName>
}
}
The following table describes the elements of the response body.
| Element Name | Description |
|---|---|
| name | Specifies the name of pipeline. |
| ID | Specifies the identifying URL of the Pipeline. |
| Description | Text describing what the activity is used for. |
| Activity | Activity in the pipeline. A pipeline can have one or more activities. |
| Status | Specifies whether this pipeline is paused or not. |
| Provisioningstate | Specifies the current provisioning state of the pipeline. When a pipeline is successfully created, the value of the element is Succeeded. |
| Hubname | The hub which contains your pipeline. |
Sample Request and Response
Example URI:
PUT: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/SPRestDataFactory/datapipelines/ADFTutorialPipeline?api-version=2015-10-01
The request is sent with the following headers:
x-ms-client-request-id : 00000000-1111-1111-1111-000000000000
and the following body:
{
"name": "ADFTutorialPipeline",
"properties": {
"description": "Copy data from a blob to Azure SQL table",
"activities": [
{
"name": "CopyFromBlobToSQL",
"description": "Push Regional Effectiveness Campaign data to Azure SQL database",
"type": "Copy",
"inputs": [
{
"name": "EmpTableFromBlob"
}
],
"outputs": [
{
"name": "EmpSQLTable"
}
],
"typeProperties": {
"source": {
"type": "BlobSource"
},
"sink": {
"type": "SqlSink"
}
},
"Policy": {
"concurrency": 1,
"executionPriorityOrder": "NewestFirst",
"style": "StartOfInterval",
"retry": 0,
"timeout": "01:00:00"
},
"scheduler": {
"frequency": "Hour",
"interval": 1
}
}
],
"start": "2015-12-09T00:00:00Z",
"end": "2015-12-10T00:00:00Z",
"isPaused": false
}
}
After the request has been sent, the following response is returned.
Header:
Status Code:
Created
Headers:
Pragma : no-cache
x-ms-request-id : 00000000-1111-1111-1111-000000000000
x-ms-ratelimit-remaining-subscription-writes: 11999
x-ms-correlation-request-id : 00000000-1111-2222-1111-000000000000
x-ms-routing-request-id : WESTUS:20141203T213927Z: 00000000-1111-2222-1111-000000000000
Strict-Transport-Security : max-age=31536000; includeSubDomains
Cache-Control : no-cache
Date : Wed, 03 Dec 2014 21:39:27 GMT
Location : https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/test/datapipelines/pipelinetest?api-version=2015-10-01
Server : Microsoft-IIS/8.5
X-Powered-By : ASP.NET
client-tracking-id : 51
The response includes the following XML body:
Body:
{
"name": "ADFTutorialPipeline",
"id":
"/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/ADF/providers/Microsoft.DataFactory/datafactories/
SPRestDataFactory/datapipelines/ADFTutorialPipeline",
"properties": {
"description": "Copy data from a blob to Azure SQL table",
"activities": [
{
"type": "Copy",
"typeProperties": {
"source": {
"type": "BlobSource"
},
"sink": {
"type": "SqlSink",
"writeBatchSize": 0,
"writeBatchTimeout": "00:00:00"
}
},
"inputs": [
{
"name": "EmpTableFromBlob"
}
],
"outputs": [
{
"name": "EmpSQLTable"
}
],
"policy": {
"timeout": "01:00:00",
"concurrency": 1,
"executionPriorityOrder": "NewestFirst",
"style": "StartOfInterval"
},
"scheduler": {
"frequency": "Hour",
"interval": 1
},
"name": "CopyFromBlobToSQL",
"description": "Push Regional Effectiveness Campaign data to Azure SQL database"
}
],
"start": "2015-12-09T00:00:00Z",
"end": "2015-12-10T00:00:00Z",
"isPaused": false,
"runtimeInfo": {
"deploymentTime": "2016-02-22T22:40:24.9768561Z",
"activePeriodSetTime": "2016-02-22T22:40:25.7707093Z",
"activityPeriods": {
"copyFromBlobToSQL": {
"start": "2015-12-09T00:00:00Z",
"end": "2015-12-10T00:00:00Z"
}
}
},
"id": "00000000-0000-0000-0000-000000000000",
"provisioningState": "Succeeded",
"hubName": "sprestdatafactory_hub",
"pipelineMode": "Scheduled"
}
}
See Also
Pipelines and Activities in Azure Data Factory
Delete
Deletes the specified pipeline.
Request
The Delete Pipeline request may be constructed as follows (HTTPS is recommended):
| HTTP Verb | Request URI | HTTP Version |
|---|---|---|
| DELETE | https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName}/datapipelines/{PipelineName}?api-version={api-version} | HTTP/1.1 |
URI Parameters
| URI Parameter | Required | Description |
|---|---|---|
| SubscriptionID | Yes | Your Azure subscription ID. |
| ResourceGroupName | Yes | The unique name for the resource group that hosts your Azure data factory. |
| DataFactoryName | Yes | Name for the data factory that you want to delete your pipeline. |
| PipelineName | Yes | Name of pipeline you want to delete. |
| Api-Version | Yes | Specifies the version of the protocol used to make this request. |
Request Headers
The following table describes the request headers.
| Request Header | Required | Description |
|---|---|---|
| x-ms-client-request-id | Yes | The operation ID for this request. |
Request Body
None.
Response
The response includes an HTTP status code, a set of response headers, and a response body.
Status Code
200 (OK) - if the request was completed successfully.
400 (Bad Request) - if request body fails validation.
404 (NotFound) - if subscription or resource group does not exist.
412 (Precondition Failed) - if condition specified by If-Match header failed.
501 (Not Implemented) - if validate is not implemented
Response Headers
The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.
| Response Header | Description |
|---|---|
| x-ms-request-id | A unique identifier for the current operation, service generated. |
| x-ms-ratelimit-remaining-subscription-writes | The remaining limit for current subscription. |
| x-ms-correlation-request-id | Specifies the tracing correlation Id for the request; the resource provider must log this so that end-to-end requests can be correlated across Azure. |
| x-ms-routing-request-id | Location+DateTime+correlation-request-ID |
| Date | A UTC date/time value generated by the service that indicates the time at which the response was initiated. |
Response Body
None.
Sample Request and Response
Example URI:
DELETE: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/test/datapipelines/pipelinetest?api-version=2015-10-01
The request is sent with the following headers.
x-ms-client-request-id : 00000000-1111-1111-1111-000000000000
After the request has been sent, the following response is returned.
Status Code:
OK
Headers:
Pragma : no-cache
x-ms-request-id : 00000000-1111-1111-1111-000000000000
x-ms-ratelimit-remaining-subscription-writes: 11999
x-ms-correlation-request-id : 00000000-1111-2222-1111-000000000000
x-ms-routing-request-id : WESTUS:20141203T213927Z: 00000000-1111-2222-1111-000000000000
Strict-Transport-Security : max-age=31536000; includeSubDomains
Cache-Control : no-cache
Date : Wed, 03 Dec 2014 21:39:27 GMT
Server : Microsoft-IIS/8.5
X-Powered-By : ASP.NET
client-tracking-id : 79
See Also
Pipelines and Activities in Azure Data Factory
Get
Gets information about the specified pipeline.
Request
The Get Pipeline request may be constructed as follows (HTTPS is recommended):
| HTTP Verb | Request URI | HTTP Version |
|---|---|---|
| GET | https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName}/datapipelines/{PipelineName}?api-version={api-version} | HTTP/1.1 |
URI Parameters
| URI Parameter | Required | Description |
|---|---|---|
| SubscriptionID | Yes | Your Azure subscription ID. |
| ResourceGroupName | Yes | A unique name for the resource group that hosts your Azure data factory. |
| DataFactoryName | Yes | Name for the data factory that you want to find your pipeline. |
| PipelineName | Yes | Name of pipeline you want to find. |
| Api-Version | Yes | Specifies the version of the protocol used to make this request. |
Request Headers
The following table describes the request headers.
| Request Header | Required | Description |
|---|---|---|
| x-ms-client-request-id | Yes | The operation id for this request. |
Request Body
None.
Response
The response includes an HTTP status code, a set of response headers, and a response body.
Status Code
200 (OK) - if the request was completed successfully.
400 (Bad Request) - if the request body fails validation.
404 (NotFound) - if the subscription or resource group does not exist.
412 (Precondition Failed) - if the condition specified by If-Match header failed.
Response Headers
The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.
| Response Header | Description |
|---|---|
| x-ms-request-id | A unique identifier for the current operation, service generated. |
| x-ms-ratelimit-remaining-subscription-writes | The remaining limit for current subscription. |
| x-ms-correlation-request-id | Specifies the tracing correlation Id for the request; the resource provider must log this so that end-to-end requests can be correlated across Azure. |
| x-ms-routing-request-id | Location+DateTime+correlation-request-ID |
| Date | A UTC date/time value generated by the service that indicates the time at which the response was initiated. |
Response Body
Response body depends on your pipeline property, for more information about pipelines, see Pipelines and Activities.
{
"name": <name>,
"id": <ID>,
"properties": {
"description": <description>,
"activities": <activity>,
"isPaused": <Status>,
"provisioningState": <ProvisioningState>,
"hubName": <HubName>
}
}
The following table describes the elements of the response body.
| Element Name | Description |
|---|---|
| name | Specifies the name of pipeline. |
| ID | Specifies the identifying URL of the Pipeline. |
| Description | Text describing what the activity is used for. |
| Activity | Activity in the pipeline. A pipeline can have one or more activities. |
| Status | Specifies whether this pipeline is paused or not |
| Provisioningstate | Specifies the current provisioning state of the pipeline. When a pipeline is successfully created, the value of the element is Succeeded. |
| Hubname | The hub which contains your pipeline. |
Sample Request and Response
Example URI:
GET: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/SPRestDataFactory/datapipelines/ADFTutorialPipeline?api-version=2015-10-01
The request is sent with the following headers.
x-ms-client-request-id : 00000000-1111-1111-1111-000000000000
After the request has been sent, the following response is returned.
Header:
Status Code:
OK
Headers:
Pragma : no-cache
x-ms-request-id : 00000000-1111-1111-1111-000000000000
x-ms-ratelimit-remaining-subscription-writes: 11999
x-ms-correlation-request-id : 00000000-1111-2222-1111-000000000000
x-ms-routing-request-id : WESTUS:20141203T213927Z: 00000000-1111-2222-1111-000000000000
Strict-Transport-Security : max-age=31536000; includeSubDomains
Cache-Control : no-cache
Date : Wed, 03 Dec 2014 21:39:27 GMT
Server : Microsoft-IIS/8.5
X-Powered-By : ASP.NET
The response includes the following XML body.
{
"name": "ADFTutorialPipeline",
"id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/ADF/providers/Microsoft.DataFactory/datafactories/SPRestDataFactory/datapipelines/ADFTutorialPipeline",
"properties": {
"description": "Copy data from a blob to Azure SQL table",
"activities": [
{
"type": "Copy",
"typeProperties": {
"source": {
"type": "BlobSource"
},
"sink": {
"type": "SqlSink",
"writeBatchSize": 0,
"writeBatchTimeout": "00:00:00"
}
},
"inputs": [
{
"name": "EmpTableFromBlob"
}
],
"outputs": [
{
"name": "EmpSQLTable"
}
],
"policy": {
"timeout": "01:00:00",
"concurrency": 1,
"executionPriorityOrder": "NewestFirst",
"style": "StartOfInterval"
},
"scheduler": {
"frequency": "Hour",
"interval": 1
},
"name": "CopyFromBlobToSQL",
"description": "Push Regional Effectiveness Campaign data to Azure SQL database"
}
],
"start": "2015-12-09T00:00:00Z",
"end": "2015-12-10T00:00:00Z",
"isPaused": false,
"runtimeInfo": {
"deploymentTime": "2016-02-22T22:40:24.9768561Z",
"activePeriodSetTime": "2016-02-22T22:40:25.7707093Z",
"activityPeriods": {
"copyFromBlobToSQL": {
"start": "2015-12-09T00:00:00Z",
"end": "2015-12-10T00:00:00Z"
}
}
},
"id": "00000000-0000-0000-0000-000000000000",
"provisioningState": "Succeeded",
"hubName": "sprestdatafactory_hub",
"pipelineMode": "Scheduled"
}
}
See Also
Pipelines and Activities in Azure Data Factory
List
Lists information about all linked services in a data factory.
Request
The List Pipeline request may be constructed as follows (HTTPS is recommended):
| HTTP Verb | Request URI | HTTP Version |
|---|---|---|
| GET | https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName}/datapipelines?api-version={api-version} | HTTP/1.1 |
URI Parameters
| URI Parameter | Required | Description |
|---|---|---|
| SubscriptionID | Yes | Your Azure subscription ID. |
| ResourceGroupName | Yes | A unique name for the resource group that hosts your Azure data factory Service. |
| DataFactoryName | Yes | Name for the data factory that you want to find your pipeline. |
| Api-Version | Yes | Specifies the version of the protocol used to make this request. |
Request Headers
The following table describes the request headers.
| Request Header | Required | Description |
|---|---|---|
| x-ms-client-request-id | Yes | The operation id for this request. |
Request Body
None.
Response
The response includes an HTTP status code, a set of response headers, and a response body.
Status Code
200 (OK) - if the request completed successfully.
400 (Bad Request) - if the request body fails validation.
404 (NotFound) - if the subscription or resource group does not exist.
412 (Precondition Failed) - if the condition specified by If-Match header failed.
Response Headers
The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.
| Response Header | Description |
|---|---|
| x-ms-request-id | A unique identifier for the current operation, service generated. |
| x-ms-ratelimit-remaining-subscription-writes | The remaining limit for current subscription. |
| x-ms-correlation-request-id | Specifies the tracing correlation Id for the request. The resource provider must log this so that end-to-end requests can be correlated across Azure. |
| x-ms-routing-request-id | Location+DateTime+correlation-request-ID |
| Date | A UTC date/time value generated by the service that indicates the time at which the response was initiated. |
Response Body
{
"value": [
{
"name": <name>,
"id": <ID>,
"properties": {
"description": <description>,
"activities": <activity>,
"isPaused": <Status>,
"provisioningState": <ProvisioningState>,
"hubName": <HubName>
}
},
{
"name": <name2>,
"id": <ID>,
"properties": {
"description": <description>,
"activities": <activity>,
"isPaused": <Status>,
"provisioningState": <ProvisioningState>,
"hubName": <HubName>
}
}
]
}
The following table describes the elements of the response body.
| Element Name | Description |
|---|---|
| name | Specifies the name of pipeline. |
| ID | Specifies the identifying URL of the Pipeline. |
| Description | Text describing what the activity is used for. |
| Activity | Activity in the pipeline, a pipeline can have one or more activities. |
| Status | Specifies whether this pipeline is paused or not. |
| Provisioningstate | Specifies the current provisioning state of the pipeline. When a pipeline is successfully created, the value of the element is Succeeded. |
| Hubname | The hub which contains your pipeline. |
See Pipelines and Activities in Azure Data Factory for more details.
If there are more than 100 records, you will get 100 records at a time with a nextLink:
{
"value": [
{
"name": <name>,
"id": <ID>,
"properties": {
"description": <description>,
"activities": <activity>,
"isPaused": <Status>,
"provisioningState": <ProvisioningState>,
"hubName": <HubName>
}
},
{
"name": <name2>,
"id": <ID>,
"properties": {
"description": <description>,
"activities": <activity>,
"isPaused": <Status>,
"provisioningState": <ProvisioningState>,
"hubName": <HubName>
}
}
] ,
"nextLink": "https://management.azure.com/subscriptions/{subscription ID>/resourcegroups/{Resource group name>/providers/Microsoft.DataFactory/datafactories/{data factory name>/datapipelines?api-version=2015-10-01&%24skipToken=MSExNiFSR0YwWVVGeWRHbG1ZV04wCjEhNTIhUkVGZlpUSXlaR1UyTXpndFpURTJNeTAwWXpZNUxUaGhNR0l0Wm1VMU1ETmpOakF4T0RObA%3d%3d"
}
You send the next request using the nextLink URL to get the next set of records. You may need to wrap & with double quotes ("") in the nextLink URL. Only 100 records are returned at a time.
Sample Request and Response
Example URI:
GET: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/SPRestDataFactory/datapipelines?api-version=2015-10-01
The request is sent with the following headers.
x-ms-client-request-id : 00000000-1111-1111-1111-000000000000
After the request has been sent, the following response is returned.
Status Code:
OK
Headers:
Pragma : no-cache
x-ms-request-id : 00000000-1111-1111-1111-000000000000
x-ms-ratelimit-remaining-subscription-writes: 799985
x-ms-correlation-request-id : 00000000-1111-2222-1111-000000000000
x-ms-routing-request-id : WESTUS:20141203T213927Z: 00000000-1111-2222-1111-000000000000
Strict-Transport-Security : max-age=31536000; includeSubDomains
Cache-Control : no-cache
Date : Wed, 03 Dec 2014 21:39:27 GMT
Server : Microsoft-IIS/8.5
X-Powered-By : ASP.NET
The response includes the following XML body.
{
"value": [
{
"name": "FirstPipeline",
"id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/ADF/providers/Microsoft.DataFactory/datafactories/SPRestDataFactory/datapipelines/FirstPipeline",
"properties": {
"activities": [
{
"type": "Copy",
"typeProperties": {
"source": null,
"sink": null,
"translator": null
},
"inputs": [
{
"name": "InputTable1"
}
],
"outputs": [
{
"name": "OutputTable1"
}
],
"scheduler": {
"frequency": "Hour",
"interval": 1
},
"name": "First pipeline"
}
],
"start": "2015-12-09T00:00:00Z",
"end": "2015-12-10T00:00:00Z",
"isPaused": false,
"runtimeInfo": {
"deploymentTime": "2016-02-22T22:40:24.9768561Z",
"activityPeriods": {
"copyFromBlobToSQL": {
"start": "2015-12-09T00:00:00Z",
"end": "2015-12-10T00:00:00Z"
}
}
},
"id": "c5326985-fd60-4f46-bbae-130d318759c6",
"provisioningState": "Succeeded",
"pipelineMode": "Scheduled"
}
},
{
"value": [
{
"name": "SecondPipeline",
"id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/ADF/providers/Microsoft.DataFactory/datafactories/SPRestDataFactory/datapipelines/SecondPipeline",
"properties": {
"activities": [
{
"type": "Copy",
"typeProperties": {
"source": null,
"sink": null,
"translator": null
},
"inputs": [
{
"name": "InputTable2"
}
],
"outputs": [
{
"name": "OutputTable2"
}
],
"scheduler": {
"frequency": "Hour",
"interval": 1
},
"name": "Second pipeline"
}
],
"start": "2015-12-09T00:00:00Z",
"end": "2015-12-10T00:00:00Z",
"isPaused": false,
"runtimeInfo": {
"deploymentTime": "2016-02-22T22:40:24.9768561Z",
"activityPeriods": {
"copyFromBlobToSQL": {
"start": "2015-12-09T00:00:00Z",
"end": "2015-12-10T00:00:00Z"
}
}
},
"id": "c5326985-fd60-4f46-bbae-130d318759c6",
"provisioningState": "Succeeded",
"pipelineMode": "Scheduled"
}
}
]
}
See Also
Pipelines and Activities in Azure Data Factory
List Activity Windows
Lists the first page of activity window instances associated with a pipeline along with a link to the next page.
Request
The Get Activity window request may be constructed as follows (HTTPS is recommended):
| HTTP Verb | Request URI | HTTP Version |
|---|---|---|
| POST | https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName/datapipelines/{PipelineName}/activitywindows?api-version={api-version} | HTTP/1.1 |
URI Parameters
| URI Parameter | Required | Description |
|---|---|---|
| SubscriptionID | Yes | Your Azure subscription ID. |
| ResourceGroupName | Yes | Name of the resource group that hosts your Azure data factory. |
| DataFactoryName | Yes | Name of the data factory. |
| PipelineName | Yes | Name of the pipeline. |
| Api-Version | Yes | Specifies the version of the protocol used to make this request. |
Request Headers
The following table describes the request headers.
| Request Header | Required | Description |
|---|---|---|
| x-ms-client-request-id | Yes | The operation id for this request. |
Request Body
| Element Name | Description | Required |
|---|---|---|
| Filter | The activity window filter specified using Azure Search filter grammar. See https://msdn.microsoft.com/library/azure/dn798921.aspx for details. | No |
| OrderBy | Order the response by specified activity window properties. This is a list of comma separated properties. For example: WindowStart, PercentComplete. By befault, the order is ascending order (ASC). Specify DESC if you want the list to be ordered in descending order. | No |
| Top | The number of activity windows to be listed. | No |
| RunStart | The start time of activity window run. | No |
| RunEnd | The end time of activity window run. | No |
| WindowStart | The start time of activity window. | No |
| WindowEnd | The end time of activity window. | No |
| WindowState | The state of activity window. Must be one of the values specified in this article. | No |
| WindowSubstate | The substate of activity window. Must be one of the values specified in this article. | No |
Response
The response includes an HTTP status code, a set of response headers, and a response body.
Status Code
200 (OK) - if the request completed successfully.
400 (Bad Request) - if the request body fails validation.
404 (NotFound) - if the subscription or resource group does not exist.
Response Headers
The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.
| Response Header | Description |
|---|---|
| x-ms-request-id | A unique identifier for the current operation, service generated. |
| x-ms-ratelimit-remaining-subscription-writes | The remaining limit for current subscription. |
| x-ms-correlation-request-id | Specifies the tracing correlation Id for the request. The resource provider must log this so that end-to-end requests can be correlated across Azure. |
| x-ms-routing-request-id | Location+DateTime+correlation-request-ID |
| Date | A UTC date/time value generated by the service that indicates the time at which the response was initiated. |
Response Body
{
"value": {
"activityWindows": [
{
"resourceGroupName": "<resource group name>",
"dataFactoryName": "<data factory name>",
"pipelineName": "<pipeline name>",
"activityName": "<Pipeline activity name.>",
"linkedServiceName": "<linked service name>",
"activityType": "<The activity type>",
"runAttempts": "<Number of activity run attempts.>",
"runStart": "<start time of the last run>",
"runEnd": "<end time of the last run>",
"duration": "<duration of the activity window>",
"percentComplete": "<percent completion of activity window>",
"windowStart": "<window start time>",
"windowEnd": "<window end time>",
"windowState": "<window state>",
"windowSubstate": "<window substate>",
"inputDatasets": "[<input dataset names>]",
"outputDatasets": [
"<output dataset names>"
],
"inputDatasetIds":["<input dataset ids>"],
"outputDatasetIds": [
"<output dataset ids>"
]
}
],
"lastUpdate":"<The approximate time for the last activity windows>",
"aggregates": {}
},
"nextLink": "<Link to next page>"
}
Sample Request and Response
Example URI:
POST:https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/ADF/providers/Microsoft.DataFactory/datafactories/dfName/datapipelines/pipelineName/activitywindows?api-version=2015-10-01
The request is sent with the following headers and the body.
Header:
x-ms-client-request-id : 00000000-1111-1111-1111-000000000000
Body:
{
"top": 2
}
After the request has been sent, the following response is returned.
Status Code:
OK
Headers:
Pragma : no-cache
x-ms-request-id : 00000000-1111-1111-1111-000000000000
x-ms-ratelimit-remaining-subscription-writes: 799985
x-ms-correlation-request-id : 00000000-1111-2222-1111-000000000000
x-ms-routing-request-id : WESTUS:20141203T213307Z: 00000000-1111-2222-1111-000000000000
Strict-Transport-Security : max-age=31536000; includeSubDomains
Cache-Control : no-cache
Date : Wed, 03 Dec 2014 21:33:07 GMT
Server : Microsoft-IIS/8.5
X-Powered-By : ASP.NET
The response includes the following XML body.
"value": {
"activityWindows": [
{
"resourceGroupName": "rgName",
"dataFactoryName": "dfName",
"pipelineName": "pipelineName",
"activityName": "DA_WikipediaClickEvents",
"linkedServiceName": "LS_Wikipedia",
"activityType": "Copy",
"runAttempts": 1,
"runStart": "2016-01-26T21:41:48.636Z",
"runEnd": "2016-01-26T21:41:48.886Z",
"duration": "00:00:00.0000250",
"percentComplete": 100,
"windowStart": "2016-01-25T01:00:00Z",
"windowEnd": "2016-01-25T02:00:00Z",
"windowState": "Pending",
"windowSubstate": "Validating",
"inputDatasets": ["DA_WikipediaEvents"],
"outputDatasets": [
"dsName"
],
"inputDatasetIds": ["308F5E07-5B0C-4AE9-95EB-9CEF26CFDD21"],
"outputDatasetIds": [
"1e4a0e42-5df3-4a7e-9860-7f75d306ff75"
]
},
{
"resourceGroupName": "rgName",
"dataFactoryName": "dfName",
"pipelineName": "pipelineName",
"activityName": "DA_WikipediaClickEvents",
"linkedServiceName": "LS_Wikipedia",
"activityType": "Copy",
"runAttempts": 1,
"runStart": "2016-01-26T21:41:48.636Z",
"runEnd": "2016-01-26T21:41:48.886Z",
"duration": "00:00:00.0000250",
"percentComplete": 100,
"windowStart": "2016-01-25T02:00:00Z",
"windowEnd": "2016-01-25T03:00:00Z",
"windowState": "Pending",
"windowSubstate": "Validating",
"inputDatasets": ["DA_WikipediaEvents"],
"outputDatasets": [
"dsName"
],
"inputDatasetIds": ["308F5E07-5B0C-4AE9-95EB-9CEF26CFDD21"],
"outputDatasetIds": [
"1e4a0e42-5df3-4a7e-9860-7f75d306ff75"
]
}
],
"lastUpdate": "2016-01-26T21:43:45.6837301+00:00",
"aggregates": {}
},
"nextLink": "https://localhost:86/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/rgName/datafactories/dfName/datapipelines/pipelineName/activitywindows/skip=FG154H?ApiVerion01012015"}
Use the nextLink URL to get the next page of results.
List Activity Windows by Activity
Lists the first page of activity window instances associated with an activity along with a link to the next page.
Request
The Get Activity window request may be constructed as follows (HTTPS is recommended):
| HTTP Verb | Request URI | HTTP Version |
|---|---|---|
| POST | https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName/datapipelines/{PipelineName}/activities/{ActivityName}/activitywindows?api-version={api-version} | HTTP/1.1 |
URI Parameters
| URI Parameter | Required | Description |
|---|---|---|
| SubscriptionID | Yes | Your Azure subscription ID. |
| ResourceGroupName | Yes | Name of the resource group that hosts your Azure data factory. |
| DataFactoryName | Yes | Name of the data factory. |
| PipelineName | Yes | Name of the pipeline. |
| ActivityName | Yes | Name of the activity. |
| Api-Version | Yes | Specifies the version of the protocol used to make this request. |
Request Headers
The following table describes the request headers.
| Request Header | Required | Description |
|---|---|---|
| x-ms-client-request-id | Yes | The operation id for this request. |
Request Body
| Element Name | Description | Required |
|---|---|---|
| Filter | The activity window filter specified using Azure Search filter grammar. See https://msdn.microsoft.com/library/azure/dn798921.aspx for details. | No |
| OrderBy | Order the response by specified activity window properties. This is a list of comma separated properties. For example: WindowStart, PercentComplete. By befault, the order is ascending order (ASC). Specify DESC if you want the list to be ordered in descending order. | No |
| Top | The number of activity windows to be listed. | No |
| RunStart | The start time of activity window run. | No |
| RunEnd | The end time of activity window run. | No |
| WindowStart | The start time of activity window. | No |
| WindowEnd | The end time of activity window. | No |
| WindowState | The state of the activity window. | No |
| WindowSubstate | The state of the activity window. | No |
Response
The response includes an HTTP status code, a set of response headers, and a response body.
Status Code
200 (OK) - if the request completed successfully.
400 (Bad Request) - if the request body fails validation.
404 (NotFound) - if the subscription or resource group does not exist.
Response Headers
The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.
| Response Header | Description |
|---|---|
| x-ms-request-id | A unique identifier for the current operation, service generated. |
| x-ms-ratelimit-remaining-subscription-writes | The remaining limit for current subscription. |
| x-ms-correlation-request-id | Specifies the tracing correlation Id for the request. The resource provider must log this so that end-to-end requests can be correlated across Azure. |
| x-ms-routing-request-id | Location+DateTime+correlation-request-ID |
| Date | A UTC date/time value generated by the service that indicates the time at which the response was initiated. |
Response Body
{
"value": {
"activityWindows": [
{
"resourceGroupName": "<resource group name>",
"dataFactoryName": "<data factory name>",
"pipelineName": "<pipeline name>",
"activityName": "<Pipeline activity name.>",
"linkedServiceName": "<linked service name>",
"activityType": "<The activity type>",
"runAttempts": "<Number of activity run attempts.>",
"runStart": "<start time of the last run>",
"runEnd": "<end time of the last run>",
"duration": "<duration of the activity window>",
"percentComplete": "<percent completion of activity window>",
"windowStart": "<window start time>",
"windowEnd": "<window end time>",
"windowState": "<window state>",
"windowSubstate": "<window substate>",
"inputDatasets": "[<input dataset names>]",
"outputDatasets": [
"<output dataset names>"
],
"inputDatasetIds":["<input dataset ids>"],
"outputDatasetIds": [
"<output dataset ids>"
]
}
],
"lastUpdate":"<The approximate time for the last activity windows>",
"aggregates": {}
},
"nextLink": "<Link to next page>"
}
Sample Request and Response
Example URI:
POST:https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/ADF/providers/Microsoft.DataFactory/datafactories/dfName/datapipelines/pipelineName/activities/activityName/activitywindows?api-version=2015-10-01
The request is sent with the following header and the body:
Header:
x-ms-client-request-id : 00000000-1111-1111-1111-000000000000
Body:
{
"top": 2
}
After the request has been sent, the following response is returned.
Status Code:
OK
Headers:
Pragma : no-cache
x-ms-request-id : 00000000-1111-1111-1111-000000000000
x-ms-ratelimit-remaining-subscription-writes: 799985
x-ms-correlation-request-id : 00000000-1111-2222-1111-000000000000
x-ms-routing-request-id : WESTUS:20141203T213307Z: 00000000-1111-2222-1111-000000000000
Strict-Transport-Security : max-age=31536000; includeSubDomains
Cache-Control : no-cache
Date : Wed, 03 Dec 2014 21:33:07 GMT
Server : Microsoft-IIS/8.5
X-Powered-By : ASP.NET
The response includes the following XML body.
"value": {
"activityWindows": [
{
"resourceGroupName": "rgName",
"dataFactoryName": "dfName",
"pipelineName": "pipelineName",
"activityName": "activityName",
"linkedServiceName": "LS_Wikipedia",
"activityType": "Copy",
"runAttempts": 1,
"runStart": "2016-01-26T21:41:48.636Z",
"runEnd": "2016-01-26T21:41:48.886Z",
"duration": "00:00:00.0000250",
"percentComplete": 100,
"windowStart": "2016-01-25T01:00:00Z",
"windowEnd": "2016-01-25T02:00:00Z",
"windowState": "Pending",
"windowSubstate": "Validating",
"inputDatasets": ["DA_WikipediaEvents"],
"outputDatasets": [
"dsName"
],
"inputDatasetIds": ["308F5E07-5B0C-4AE9-95EB-9CEF26CFDD21"],
"outputDatasetIds": [
"1e4a0e42-5df3-4a7e-9860-7f75d306ff75"
]
},
{
"resourceGroupName": "rgName",
"dataFactoryName": "dfName",
"pipelineName": "pipelineName",
"activityName": "activityName",
"linkedServiceName": "LS_Wikipedia",
"activityType": "Copy",
"runAttempts": 1,
"runStart": "2016-01-26T21:41:48.636Z",
"runEnd": "2016-01-26T21:41:48.886Z",
"duration": "00:00:00.0000250",
"percentComplete": 100,
"windowStart": "2016-01-25T02:00:00Z",
"windowEnd": "2016-01-25T03:00:00Z",
"windowState": "Pending",
"windowSubstate": "Validating",
"inputDatasets": ["DA_WikipediaEvents"],
"outputDatasets": [
"dsName"
],
"inputDatasetIds": ["308F5E07-5B0C-4AE9-95EB-9CEF26CFDD21"],
"outputDatasetIds": [
"1e4a0e42-5df3-4a7e-9860-7f75d306ff75"
]
}
],
"lastUpdate": "2016-01-26T21:43:45.6837301+00:00",
"aggregates": {}
},
"nextLink": "https://localhost:86/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/rgName/datafactories/dfName/datapipelines/pipelineName/activities/activityName/activitywindows/skip=FG154H?ApiVerion01012015"}
Use the nextLink URL to get the next page of results.
Resume
Resumes a suspended pipeline in data factory.
Request
The Resume Pipeline request may be constructed as follows (HTTPS is recommended):
| HTTP Verb | Request URI | HTTP Version |
|---|---|---|
| POST | https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName}/datapipelines/{PipelineName}/resume?api-version={api-version} | HTTP/1.1 |
URI Parameters
| URI Parameter | Required | Description |
|---|---|---|
| SubscriptionID | Yes | Your Azure subscription ID |
| ResourceGroupName | Yes | A unique name for the resource group that hosts your Azure data factory. |
| DataFactoryName | Yes | Name for the data factory that you want to find your pipeline. |
| PipelineName | Yes | Name of pipeline you want to resume. |
| Api-Version | Yes | Specifies the version of the protocol used to make this request. |
Request Headers
The following table describes the request headers.
| Request Header | Required | Description |
|---|---|---|
| x-ms-client-request-id | Yes | The operation id for this request. |
Request Body
None.
Response
The response includes an HTTP status code, a set of response headers, and a response body.
Status Code
200 (OK) - if the request was completed successfully.
400 (Bad Request) - if the request body fails validation.
404 (NotFound) - if the subscription or resource group does not exist.
Response Headers
The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.
| Response Header | Description |
|---|---|
| x-ms-request-id | A unique identifier for the current operation, service generated. |
| x-ms-ratelimit-remaining-subscription-writes | The remaining limit for current subscription. |
| x-ms-correlation-request-id | Specifies the tracing correlation Id for the request; the resource provider must log this so that end-to-end requests can be correlated across Azure. |
| x-ms-routing-request-id | Location+DateTime+correlation-request-ID |
| Date | A UTC date/time value generated by the service that indicates the time at which the response was initiated. |
Response Body
None.
Sample Request and Response
Example URI:
POST: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/test/datapipelines/pipelinetest/resume?api-version=2015-10-01
The request is sent with the following headers.
x-ms-client-request-id : 00000000-1111-1111-1111-000000000000
After the request has been sent, the following response is returned.
Status Code:
OK
Headers:
Pragma : no-cache
x-ms-request-id : 00000000-1111-1111-1111-000000000000
x-ms-ratelimit-remaining-subscription-writes: 11995
x-ms-correlation-request-id : 00000000-1111-2222-1111-000000000000
x-ms-routing-request-id : WESTUS:20141203T213927Z: 00000000-1111-2222-1111-000000000000
Strict-Transport-Security : max-age=31536000; includeSubDomains
Cache-Control : no-cache
Date : Wed, 03 Dec 2014 21:39:27 GMT
Server : Microsoft-IIS/8.5
X-Powered-By : ASP.NET
See Also
Pipelines and Activities in Azure Data Factory
Set Active Period
Configures the active period for a data pipeline.
Request
The Set Slice Status request may be constructed as follows (HTTPS is recommended):
| HTTP Verb | Request URI | HTTP Version |
|---|---|---|
| POST | https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName}/datapipelines/{PipelineName}/update?start={StartDateTime}&end={EndDateTime}&autoResolve=false&forceRecalc=false&api-version={api-version} | HTTP/1.1 |
URI Parameters
Insert subsection body here.
| URI Parameter | Required | Description |
|---|---|---|
| SubscriptionID | Yes | Your Azure subscription ID. |
| ResourceGroupName | Yes | A unique name for the resource group that hosts your Azure data factory. |
| DataFactoryName | Yes | Name for the data factory that you want to set your pipeline. |
| PipelineName | Yes | Name of table you want to create. |
| StartDateTime | Yes | The datetime when slices start. |
| EndDateTime | Yes | The datetime when slices end. The default datetime is 48 hours after start datetime. |
| autoResolve | No | Indicates that this request uses auto-resolve or not. |
| forceRecalc | No | Indicates that this request uses force recalculate or not. |
| Api-Version | Yes | Specifies the version of the protocol used to make this request. |
Request Headers
The following table describes the request headers.
| Request Header | Required | Description |
|---|---|---|
| x-ms-client-request-id | Yes | The operation id for this request. |
Request Body
None.
Response
The response includes an HTTP status code, a set of response headers, and a response body.
Status Code
200 (OK) - if the request completed successfully.
400 (Bad Request) - if the request body fails validation.
404 (NotFound) - if the subscription or resource group does not exist.
Response Headers
The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.
| Response Header | Description |
|---|---|
| x-ms-request-id | A unique identifier for the current operation, service generated. |
| x-ms-ratelimit-remaining-subscription-writes | The remaining limit for current subscription. |
| x-ms-correlation-request-id | Specifies the tracing correlation Id for the request. The resource provider must log this so that end-to-end requests can be correlated across Azure. |
| x-ms-routing-request-id | Location+DateTime+correlation-request-ID |
| Date | A UTC date/time value generated by the service that indicates the time at which the response was initiated. |
| Location | URI of the object you want to create or get. |
Response Body
None.
Sample Request and Response
Example URI:
POST: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/test/datapipelines/pipelinetest/update?start=2014-12-02T00:00:00.0000000&end=2014-12-02T01:00:00.0000000& autoResolve=false&forceRecalc=false&api-version=2015-10-01
The request is sent with the following headers.
x-ms-client-request-id : 00000000-1111-1111-1111-000000000000
After the request has been sent, the following response is returned.
Status Code:
OK
Headers:
Pragma : no-cache
x-ms-request-id : 00000000-1111-1111-1111-000000000000
x-ms-ratelimit-remaining-subscription-writes: 799985
x-ms-correlation-request-id : 00000000-1111-2222-1111-000000000000
x-ms-routing-request-id : WESTUS:20141203T213307Z: 00000000-1111-2222-1111-000000000000
Strict-Transport-Security : max-age=31536000; includeSubDomains
Cache-Control : no-cache
Date : Wed, 03 Dec 2014 21:33:07 GMT
Server : Microsoft-IIS/8.5
X-Powered-By : ASP.NET
See Also
Pipelines and Activities in Azure Data Factory
Suspend
Pauses an existing running pipeline.
Request
The Suspend Pipeline request may be constructed as follows (HTTPS is recommended):
| HTTP Verb | Request URI | HTTP Version |
|---|---|---|
| POST | https://management.azure.com/subscriptions/{SubscriptionID}/resourcegroups/{ResourceGroupName}/providers/Microsoft.DataFactory/datafactories/{DataFactoryName}/datapipelines/{PipelineName}/pause?api-version={api-version} | HTTP/1.1 |
URI Parameters
| URI Parameter | Required | Description |
|---|---|---|
| SubscriptionID | Yes | Your Azure subscription ID. |
| ResourceGroupName | Yes | A unique name for the resource group that hosts your Azure data factory. |
| DataFactoryName | Yes | Name for the data factory that you want to find your pipeline. |
| PipelineName | Yes | Name of pipeline you want to suspend. |
| Api-Version | Yes | Specifies the version of the protocol used to make this request. |
Request Headers
The following table describes the request headers.
| Request Header | Required | Description |
|---|---|---|
| x-ms-client-request-id | Yes | The operation id for this request. |
Request Body
None.
Response
The response includes an HTTP status code, a set of response headers, and a response body.
200 (OK) - if the request completed successfully.
400 (Bad Request) - if the request body fails validation.
404 (NotFound) - if the subscription or resource group does not exist.
Response Headers
The response for this operation includes the following headers. The response may also include additional standard HTTP headers. All standard headers conform to the HTTP/1.1 protocol specification.
| Response Header | Description |
|---|---|
| x-ms-request-id | A unique identifier for the current operation, service generated. |
| x-ms-ratelimit-remaining-subscription-writes | The remaining limit for current subscription. |
| x-ms-correlation-request-id | Specifies the tracing correlation Id for the request. The resource provider must log this so that end-to-end requests can be correlated across Azure. |
| x-ms-routing-request-id | Location+DateTime+correlation-request-ID |
| Date | A UTC date/time value generated by the service that indicates the time at which the response was initiated. |
Response Body
None.
Sample Request and Response
Example:.
POST: https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/adf/providers/Microsoft.DataFactory/datafactories/test/datapipelines/pipelinetest/pause?api-version=2015-10-01
The request is sent with the following headers.
x-ms-client-request-id : 00000000-1111-1111-1111-000000000000
After the request has been sent, the following response is returned.
Status Code:
OK
Headers:
Pragma : no-cache
x-ms-request-id : 00000000-1111-1111-1111-000000000000
x-ms-ratelimit-remaining-subscription-writes: 11995
x-ms-correlation-request-id : 00000000-1111-2222-1111-000000000000
x-ms-routing-request-id : WESTUS:20141203T213927Z: 00000000-1111-2222-1111-000000000000
Strict-Transport-Security : max-age=31536000; includeSubDomains
Cache-Control : no-cache
Date : Wed, 03 Dec 2014 21:39:27 GMT
Server : Microsoft-IIS/8.5
X-Powered-By : ASP.NET