SparkContext.AddFile(String, Boolean) Method

Definition

Namespace:

Microsoft.Spark

Assembly:

Microsoft.Spark.dll

Package:

Microsoft.Spark v1.0.0

Add a file to be downloaded with this Spark job on every node.

public void AddFile (string path, bool recursive = false);

member this.AddFile : string * bool -> unit

Public Sub AddFile (path As String, Optional recursive As Boolean = false)

Parameters

path

String

File path can be either a local file, a file in HDFS (or other Hadoop-supported filesystems), or an HTTP, HTTPS or FTP URI.

recursive

Boolean

If true, a directory can be given in path. Currently directories are supported only for Hadoop-supported filesystems.

Remarks

If a file is added during execution, it will not be available until the next TaskSet starts.