utilities Module

Utility functions for distributed scenarios.

Classes

PollForMaster

Functions

get_unique_download_path

Temporary function to make paths unique to node

: return: String for path appended with current rank

get_unique_download_path(download_file: str) -> str

Parameters

Name	Description
download_file Required

horovod_initialized

horovod_initialized(func: Callable[[...], Any]) -> Callable[[...], Any]

Parameters

Name	Description
func Required

is_master_process

Function for determining whether the current process is master.

is_master_process() -> bool

Returns

Type	Description
	Boolean for whether this process is master.

to_dask_dataframes_xyw

to_dask_dataframes_xyw(lazy_tabular_data)

Parameters

Name	Description
lazy_tabular_data Required