Binary format in Azure Data Factory

Binary format is supported for the following connectors: Amazon S3, Azure Blob, Azure Data Lake Storage Gen1, Azure Data Lake Storage Gen2, Azure File Storage, File System, FTP, Google Cloud Storage, HDFS, HTTP, and SFTP.

You can use Binary dataset in Copy activity, GetMetadata activity, or Delete activity. When using Binary dataset, ADF does not parse file content but treat it as-is.

Note

When using Binary dataset in copy activity, you can only copy from Binary dataset to Binary dataset.

Dataset properties

For a full list of sections and properties available for defining datasets, see the Datasets article. This section provides a list of properties supported by the Binary dataset.

Property Description Required
type The type property of the dataset must be set to Binary. Yes
location Location settings of the file(s). Each file-based connector has its own location type and supported properties under location. See details in connector article -> Dataset properties section. Yes
compression Group of properties to configure file compression. Configure this section when you want to do compression/decompression during activity execution. No
type The compression codec used to read/write binary files.
Allowed values are bzip2, gzip, deflate, ZipDeflate. to use when saving the file.
No
level The compression ratio. Apply when dataset is used in Copy activity sink.
Allowed values are Optimal or Fastest.
- Fastest: The compression operation should complete as quickly as possible, even if the resulting file is not optimally compressed.
- Optimal: The compression operation should be optimally compressed, even if the operation takes a longer time to complete. For more information, see Compression Level topic.
No

Below is an example of Binary dataset on Azure Blob Storage:

{
    "name": "BinaryDataset",
    "properties": {
        "type": "Binary",
        "linkedServiceName": {
            "referenceName": "<Azure Blob Storage linked service name>",
            "type": "LinkedServiceReference"
        },
        "typeProperties": {
            "location": {
                "type": "AzureBlobStorageLocation",
                "container": "containername",
                "folderPath": "folder/subfolder",
            },
            "compression": {
                "type": "ZipDeflate"
            }
        }
    }
}

Copy activity properties

For a full list of sections and properties available for defining activities, see the Pipelines article. This section provides a list of properties supported by the Binary source and sink.

Note

When using Binary dataset in copy activity, you can only copy from Binary dataset to Binary dataset.

Binary as source

The following properties are supported in the copy activity *source* section.

Property Description Required
type The type property of the copy activity source must be set to BinarySource. Yes
storeSettings A group of properties on how to read data from a data store. Each file-based connector has its own supported read settings under storeSettings. See details in connector article -> Copy activity properties section. No

Binary as sink

The following properties are supported in the copy activity *sink* section.

Property Description Required
type The type property of the copy activity source must be set to BinarySink. Yes
storeSettings A group of properties on how to write data to a data store. Each file-based connector has its own supported write settings under storeSettings. See details in connector article -> Copy activity properties section. No

Next steps