Microsoft Azure Data Factory management client library for .NET

  • Article

Microsoft Azure Data Factory is a cloud-based data integration service that orchestrates and automates the movement and transformation of data.

This library supports managing Microsoft Azure Data Factory resources.

This library follows the new Azure SDK guidelines, and provides many core capabilities:

- Support MSAL.NET, Azure.Identity is out of box for supporting MSAL.NET.
- Support [OpenTelemetry](https://opentelemetry.io/) for distributed tracing.
- HTTP pipeline with custom policies.
- Better error-handling.
- Support uniform telemetry across all languages.

Getting started

Install the package

Install the Microsoft Azure Data Factory management library for .NET with NuGet:

dotnet add package Azure.ResourceManager.DataFactory

Prerequisites

Authenticate the Client

To create an authenticated client and start interacting with Microsoft Azure resources, see the quickstart guide here.

Key concepts

Key concepts of the Microsoft Azure SDK for .NET can be found here.

Documentation

Documentation is available to help you learn how to use this package:

Examples

Code samples for using the management library for .NET can be found in the following locations

Examples using DataFactoryElement

Assign DataFactoryElement with different literal types

  • int
var policy = new PipelineActivityPolicy
{
   Retry = DataFactoryElement<int>.FromLiteral(1),
};
  • bool
var service = new AmazonS3CompatibleLinkedService
{
    ForcePathStyle = DataFactoryElement<bool>.FromLiteral(true),
};
  • list
var source = new Office365Source()
{
    AllowedGroups = DataFactoryElement<IList<string>>.FromLiteral(new List<string> { "a", "b" }),
};
  • Dictionary
Dictionary<string, string> DictionaryValue = new()
{
    { "key1", "value1" },
    { "key2", "value2" }
};
var activity = new AzureMLExecutePipelineActivity("name")
{
    MLPipelineParameters = DataFactoryElement<IDictionary<string, string>?>.FromLiteral(DictionaryValue),
};
  • BinaryData
var varActivity = new SetVariableActivity("name")
{
    Value = DataFactoryElement<BinaryData>.FromLiteral(BinaryData.FromString("a")),
};

Assign DataFactoryElement from expression

var service = new AmazonRdsForOracleLinkedService(DataFactoryElement<string>.FromExpression("foo/bar-@{pipeline().TriggerTime}"));

Assign DataFactoryElement from masked string

var service = new AmazonS3CompatibleLinkedService()
{
    ServiceUri = DataFactoryElement<string>.FromSecretString("some/secret/path"),
};

Assign DataFactoryElement from KeyVault secret reference

var store = new DataFactoryLinkedServiceReference(DataFactoryLinkedServiceReferenceKind.LinkedServiceReference,
    "referenceName");
var keyVaultReference = new DataFactoryKeyVaultSecret(store, "secretName");
var service = new AmazonS3CompatibleLinkedService()
{
    AccessKeyId = DataFactoryElement<string>.FromKeyVaultSecret(keyVaultReference),
};

Troubleshooting

Next steps

For more information about Microsoft Azure SDK, see this website.

Contributing

For details on contributing to this repository, see the contributing guide.

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (for example, label, comment). Follow the instructions provided by the bot. You'll only need to do this action once across all repositories using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information, see the Code of Conduct FAQ or contact opencode@microsoft.com with any other questions or comments.