SparkConf Class

Reference

Definition

Namespace:: Microsoft.Spark

Assembly:: Microsoft.Spark.dll

Package:: Microsoft.Spark v1.0.0

Important

Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

Configuration for a Spark application. Used to set various Spark parameters as key-value pairs.

public sealed class SparkConf

type SparkConf = class

Public NotInheritable Class SparkConf

Inheritance: Object
SparkConf

Remarks

Note that once a SparkConf object is passed to Spark, it is cloned and can no longer be modified by the user. Spark does not support modifying the configuration at runtime.

Constructors

SparkConf(Boolean)

Constructor.

Methods

Get(String, String)	Get a string parameter value, falling back to a default if not set
GetAll()	Get all parameters as a list of pairs.
GetInt(String, Int32)	Get a int parameter value, falling back to a default if not set
Set(String, String)	Set the value of a string config
SetAppName(String)	Set a name for your application. Shown in the Spark web UI.
SetMaster(String)	The master URL to connect to, such as "local" to run locally with one thread, "local[4]" to run locally with 4 cores, or "spark://master:7077" to run on a Spark standalone cluster.
SetSparkHome(String)	Set the location where Spark is installed on worker nodes.

Applies to