SparkSessionOperations Class

  • Reference

SparkSessionOperations async operations.

You should not instantiate directly this class, but create a Client instance that will create it for you and attach it as attribute.

Inheritance
builtins.object
SparkSessionOperations

Constructor

SparkSessionOperations(client, config, serializer, deserializer)

Parameters

Name Description
client
Required

Client for service requests.
config
Required

Configuration of service client.
serializer
Required

An object model serializer.
deserializer
Required

An object model deserializer.

Variables

Name Description
models

Alias to model classes used in this operation group.

Methods

create

Create new spark session.
create_statement

Create statement within a spark session.
delete

Cancels a running spark session.
delete_statement

Kill a statement within a session.
get

Gets a single spark session.
get_statement

Gets a single statement within a spark session.
list

List all spark sessions which are running under a particular spark pool.
list_statements

Gets a list of statements within a spark session.
reset_timeout

Sends a keep alive call to the current session to reset the session timeout.

create

Create new spark session.

async create(workspace_name: str, spark_pool_name: str, livy_request: ExtendedLivySessionRequest, detailed: bool | None = None, **kwargs) -> ExtendedLivySessionResponse

Parameters

Name Description
workspace_name
Required
str

The name of the workspace to execute operations on.
spark_pool_name
Required
str

Name of the spark pool. "ondemand" targets the ondemand pool.
livy_request
Required
ExtendedLivySessionRequest

Livy compatible batch job request payload.
detailed
bool

Optional query param specifying whether detailed response is returned beyond plain livy.

default value: None

Keyword-Only Parameters

Name Description
cls
callable

A custom type or function that will be passed the direct response

Returns

Type Description
ExtendedLivySessionResponse

ExtendedLivySessionResponse or the result of cls(response)

Exceptions

Type Description
HttpResponseError

create_statement

Create statement within a spark session.

async create_statement(workspace_name: str, spark_pool_name: str, session_id: int, livy_request: LivyStatementRequestBody, **kwargs) -> LivyStatementResponseBody

Parameters

Name Description
workspace_name
Required
str

The name of the workspace to execute operations on.
spark_pool_name
Required
str

Name of the spark pool. "ondemand" targets the ondemand pool.
session_id
Required
int

Identifier for the session.
livy_request
Required
LivyStatementRequestBody

Livy compatible batch job request payload.

Keyword-Only Parameters

Name Description
cls
callable

A custom type or function that will be passed the direct response

Returns

Type Description
LivyStatementResponseBody

LivyStatementResponseBody or the result of cls(response)

Exceptions

Type Description
HttpResponseError

delete

Cancels a running spark session.

async delete(workspace_name: str, spark_pool_name: str, session_id: int, **kwargs) -> None

Parameters

Name Description
workspace_name
Required
str

The name of the workspace to execute operations on.
spark_pool_name
Required
str

Name of the spark pool. "ondemand" targets the ondemand pool.
session_id
Required
int

Identifier for the session.

Keyword-Only Parameters

Name Description
cls
callable

A custom type or function that will be passed the direct response

Returns

Type Description
None

None or the result of cls(response)

Exceptions

Type Description
HttpResponseError

delete_statement

Kill a statement within a session.

async delete_statement(workspace_name: str, spark_pool_name: str, session_id: int, statement_id: int, **kwargs) -> LivyStatementCancellationResponse

Parameters

Name Description
workspace_name
Required
str

The name of the workspace to execute operations on.
spark_pool_name
Required
str

Name of the spark pool. "ondemand" targets the ondemand pool.
session_id
Required
int

Identifier for the session.
statement_id
Required
int

Identifier for the statement.

Keyword-Only Parameters

Name Description
cls
callable

A custom type or function that will be passed the direct response

Returns

Type Description
LivyStatementCancellationResponse

LivyStatementCancellationResponse or the result of cls(response)

Exceptions

Type Description
HttpResponseError

get

Gets a single spark session.

async get(workspace_name: str, spark_pool_name: str, session_id: int, detailed: bool | None = None, **kwargs) -> ExtendedLivySessionResponse

Parameters

Name Description
workspace_name
Required
str

The name of the workspace to execute operations on.
spark_pool_name
Required
str

Name of the spark pool. "ondemand" targets the ondemand pool.
session_id
Required
int

Identifier for the session.
detailed
bool

Optional query param specifying whether detailed response is returned beyond plain livy.

default value: None

Keyword-Only Parameters

Name Description
cls
callable

A custom type or function that will be passed the direct response

Returns

Type Description
ExtendedLivySessionResponse

ExtendedLivySessionResponse or the result of cls(response)

Exceptions

Type Description
HttpResponseError

get_statement

Gets a single statement within a spark session.

async get_statement(workspace_name: str, spark_pool_name: str, session_id: int, statement_id: int, **kwargs) -> LivyStatementResponseBody

Parameters

Name Description
workspace_name
Required
str

The name of the workspace to execute operations on.
spark_pool_name
Required
str

Name of the spark pool. "ondemand" targets the ondemand pool.
session_id
Required
int

Identifier for the session.
statement_id
Required
int

Identifier for the statement.

Keyword-Only Parameters

Name Description
cls
callable

A custom type or function that will be passed the direct response

Returns

Type Description
LivyStatementResponseBody

LivyStatementResponseBody or the result of cls(response)

Exceptions

Type Description
HttpResponseError

list

List all spark sessions which are running under a particular spark pool.

async list(workspace_name: str, spark_pool_name: str, from_parameter: int | None = None, size: int | None = None, detailed: bool | None = None, **kwargs) -> ExtendedLivyListSessionResponse

Parameters

Name Description
workspace_name
Required
str

The name of the workspace to execute operations on.
spark_pool_name
Required
str

Name of the spark pool. "ondemand" targets the ondemand pool.
from_parameter
int

Optional param specifying which index the list should begin from.

default value: None
size
int

Optional param specifying the size of the returned list. By default it is 20 and that is the maximum.

default value: None
detailed
bool

Optional query param specifying whether detailed response is returned beyond plain livy.

default value: None

Keyword-Only Parameters

Name Description
cls
callable

A custom type or function that will be passed the direct response

Returns

Type Description
ExtendedLivyListSessionResponse

ExtendedLivyListSessionResponse or the result of cls(response)

Exceptions

Type Description
HttpResponseError

list_statements

Gets a list of statements within a spark session.

async list_statements(workspace_name: str, spark_pool_name: str, session_id: int, **kwargs) -> LivyStatementsResponseBody

Parameters

Name Description
workspace_name
Required
str

The name of the workspace to execute operations on.
spark_pool_name
Required
str

Name of the spark pool. "ondemand" targets the ondemand pool.
session_id
Required
int

Identifier for the session.

Keyword-Only Parameters

Name Description
cls
callable

A custom type or function that will be passed the direct response

Returns

Type Description
LivyStatementsResponseBody

LivyStatementsResponseBody or the result of cls(response)

Exceptions

Type Description
HttpResponseError

reset_timeout

Sends a keep alive call to the current session to reset the session timeout.

async reset_timeout(workspace_name: str, spark_pool_name: str, session_id: int, **kwargs) -> None

Parameters

Name Description
workspace_name
Required
str

The name of the workspace to execute operations on.
spark_pool_name
Required
str

Name of the spark pool. "ondemand" targets the ondemand pool.
session_id
Required
int

Identifier for the session.

Keyword-Only Parameters

Name Description
cls
callable

A custom type or function that will be passed the direct response

Returns

Type Description
None

None or the result of cls(response)

Exceptions

Type Description
HttpResponseError

Attributes

models 

models = <module 'azure.synapse.models' from 'C:\\hostedtoolcache\\windows\\Python\\3.11.9\\x64\\Lib\\site-packages\\azure\\synapse\\models\\__init__.py'>