Set up the PySpark interactive environment for Visual Studio Code

The following steps show you how to set up the PySpark interactive environment in VS Code.

We use python/pip command to build virtual environment in your Home path. If you want to use another version, you need to change default version of python/pip command manually. More details see update-alternatives.

  1. Install Python and pip.

  2. Install virtualenv by running command below.

    pip install virtualenv
    
  3. For Linux only, install the required packages by running commands bellow if you encounter the error message.

    Python pip version

    sudo apt-get install libkrb5-dev 
    
    sudo apt-get install python-dev
    
  4. Restart VS Code, and then go back to the script editor that's running HDInsight: PySpark Interactive.

Next steps

Demo

  • HDInsight for VS Code: Video

Tools and extensions