DEPLOY RESOURCE (U-SQL)

Summary

The RESOURCE statement identifies the data to be used as a resource.

Syntax

Deploy_Resource_Statement :=                                                                             
      'DEPLOY' 'RESOURCE' file_path_URI { ',' file_path_URI}.  

Using a Resource

Files have to be in ADLS or WASB. Files are deployed to vertex and are accessible from any custom code. The file paths of deployed resources have to be unique among all deployed resources in a script and the additional files included when referencing assemblies as they are put into a single directory during compilation and execution. Ensure that all resources have unique file names or in case of identical files use only one copy. Otherwise, you will receive the error message E_CSC_USER_SAMERESOURCEWITHDIFFERENTPATH: Script contains <N> different resources with identical file name.

Size Limits

Single resource file limit is 400MB.
Overall limit for deployes resource files is 3GB.

Examples

Basic Example

// a small file containing text
DEPLOY RESOURCE "/Samples/Data/helloworld.txt";

@departments =
  SELECT * 
  FROM (VALUES
      (31, "Sales"),
      (33, "Engineering"),
      (34, "Clerical"),
      (35, "Marketing")
    ) AS D( DepID, DepName );

@departments =
    SELECT DepID, DepName, System.[IO].File.ReadAllText("helloworld.txt") AS Message
    FROM @departments;

OUTPUT @departments 
TO "/ReferenceGuide/Concepts/Resource/Test.txt"
USING Outputters.Tsv();

RESOURCE with a Processor
HelloWorldProcessor
c# code is placed in the associated Code-Behind .cs file. See usage in next section, below.

using Microsoft.Analytics.Interfaces;

namespace ReferenceGuide
{
    [SqlUserDefinedProcessor]
    public class HelloWorldProcessor : IProcessor
    {
        private string hw;

        public HelloWorldProcessor()
        {
            this.hw = System.IO.File.ReadAllText("helloworld.txt");
        }

        public override IRow Process(IRow input, IUpdatableRow output)
        {
            output.Set<int>("DepID", input.Get<int>("DepID"));
            output.Set<string>("DepName", input.Get<string>("DepName"));
            output.Set<string>("HelloWorld", hw);
            return output.AsReadOnly();
        }
    }
}

Using RESOURCE with a Processor
Using Code-Behind from previous section, above.

// a small file containing text
DEPLOY RESOURCE "/Samples/Data/helloworld.txt";

@departments =
  SELECT * 
  FROM (VALUES
      (31, "Sales"),
      (33, "Engineering"),
      (34, "Clerical"),
      (35, "Marketing")
    ) AS D( DepID, DepName );


@departments =
     PROCESS @departments
     PRODUCE DepID int,
             DepName string,
             HelloWorld string
     USING new ReferenceGuide.HelloWorldProcessor();

OUTPUT @departments 
TO "/ReferenceGuide/Concepts/Resource/Test2.txt"
USING Outputters.Tsv();

Other Examples utilizing DEPLOY RESOURCE:

See Also