Installation

The Earth Economy Devstack is a specific way of organizing files, VS Code workspaces, local development repositories and preconfigured launch configurations that let a user contribute to NatCap TEEMs projects. This document contains three possible installation methods:

  1. Standard: A quick-start guide for users who are already familiar with Python, Git, and using pip to install github-based packages.
  2. Standard Detailed: Like 1 but with more step-by-step guidance.
  3. Worskpace: All the steps to configure Python, Git, QGIS and other related software for someone with little to no experience with Python, Git, VS Code, etc.
  4. Workspace Detailed: Like 3 but with step-by-step installation guidance.

Standard Installation

  1. Install git: https://Git-scm.com/downloads
  2. Install Miniforge3 from https://github.com/conda-forge/miniforge?tab=readme-ov-file#miniforge3 - Be sure to select the correct version for your operating system (Windows, Mac, Linux)
    1. Install just for your User Account (you can do for All Users, but you will need to manually set paths to conda)
      1. If you have an Apple “m1 or m2” chip (a relatively new apple chip, make sure you select the Apple Silicon option). - Install in C:\Users\<YOUR_USERNAME>\miniforge3 (PC) or ~/miniconda3 (Mac)
    2. During installation, select yes for “Add Mambaforge/Miniforge to my PATH environment Variable” - If you get a “Windows Protected your PC”, click more info then Run Anyway.
      1. If you don’t add conda to your path, you can do this manually. On PC, you could use the command
        • SETX PATH "%PATH%;C:\Users\<YOUR_USERNAME>\miniforge3;C:\Users\<YOUR_USERNAME>\miniforge3;"
  3. Create a new mamba environment and install required packages with the following mamba command (you could use the conda command instead, but this is MUCH slower).
    1. mamba install hazelbean cython libgdal-hdf5
  4. Clone all desired repos
    1. To run e.g. seals, you start by cloning the repo at https://github.com/jandrewjohnson/seals_dev. You can clone it to whatever folder you want, but if you want to switch to a Workspace configuration later, we recommend installing it in /Users//Files// which using the SEALS example on my machine would be C:/Users/jajohns/Files/seals/seals_dev.
    2. Other supported repositories include
      1. https://github.com/jandrewjohnson/hazelbean_dev
      2. https://github.com/jandrewjohnson/gtap_invest_dev
      3. https://github.com/jandrewjohnson/global_invest_dev
      4. https://github.com/jandrewjohnson/gtappy_dev
  5. OPTIONAL, but required for seals or recompilation of hazelbean. Install C/C++ compiler.
    1. Windows:
      1. Option 1: You could go to https://visualstudio.microsoft.com/visual-cpp-build-tools/ and select download build tools.
      2. Option 2: Enter the following command in the Terminal: winget install Microsoft.VisualStudio.2022.BuildTools --force --override "--passive --wait --add Microsoft.VisualStudio.Workload.VCTools;includeRecommended" This will launch the build-tools installer (you could do this manually via the MS website if you want, but this ensures you get the right tools).
      3. Option 3: Run the install.bat file in the Earth Economy Devestack repo’s root. This just runs the winget command above.
    2. Mac:
      1. You can use Xcode to compile the cython files. Most users will already have this installed but if not, follow the directions below.
        1. If you don’t have Xcode, you can get it by running xcode-select --install in the Terminal. This command downloads and installs the Xcode Command Line Tools, which includes gcc and clang, the compilers needed to compile C/C++ code on macOS. This is somewhat analogous to the Visual Studio Build Tools on Windows.
  6. Install the clone repositories. Open a command prompt in the directory where you cloned the additional repos. In there, you will see a setup.py and/or a pyproject.toml file. Install this by running the following command (you must be in the repo’s root directory for it to find the setup.py).
    1. pip install -e .
      1. Note, the ‘e’ option makes this an “editable” install, which means the code will live in the repository you cloned, but it will still be importable through normal python methods (a symlink is created in the site-packages directory).

Common problems:

If Windows PowerShell isn’t working with Conda, follow these steps to resolve the issue:

conda init powershell

Workspace Installation

This installation is like above but organizes your repositories so that you can edit/contribute to them via a VS Code Workspace.

Install Git

Install Miniforge

  • Install Miniforge from https://github.com/conda-forge/miniforge?tab=readme-ov-file#miniforge3 - Be sure to select the correct version for your operating system (Windows, Mac, Linux)

    • Install just for your user account (rather than “all users”).
      • If you have an Apple “m1 or m2” chip (a relatively new apple chip, make sure you select the Apple Silicon option).
      • Install in C:\Users\<YOUR_USERNAME>\miniforge3 (PC) or ~/miniconda3 (Mac)
    • During installation, select yes for “Add Mambaforge/Miniforge to my PATH environment Variable”
    • If you get a “Windows Protected your PC”, click more info then Run Anyway.
      • Conda can be very slow at solving complex environments. The previous solution was to install mambaforge, which was much faster. Now, mamba is included in miniforge, so you can use mamba to install packages instead of conda. However, if the conda command is still unmodified, so you still have to explicitly call mamba.
      • If you don’t add mamba to your path, you can do this manually. On PC, you could use the command
        • SETX PATH "%PATH%;C:\Users\<YOUR_USERNAME>\miniforge;C:\Users\<YOUR_USERNAME>\miniforge;"

Create a new Python environment

  • Create a new mamba environment with the following commands (putting your desired environment name in place of <env_name>):
    • mamba create -n <env_name>
  • Activate the environment
    • mamba activate <env_name>
  • Install libraries using the following mamba command:
    • mamba install hazelbean cython libgdal-hdf5
  • Install more with pip using the following command:
    • pip install winshell

Clone the Earth Economy Devstack

  • Clone the Earth Economy devstack into the correct location
    • Create a directory for called “Files” at C:\Users\<YOUR_USERNAME>\Files (PC) or ~/Files (Mac)
      • We add a Files directory to keep the root directory clean and to make it easier to find the Earth Economy Devstack in the file explorer.
  • Open a terminal or command prompt and navigate to the Files directory you created
  • Run the git clone command to clone the Earth Economy Devstack repository
    • git clone https://github.com/jandrewjohnson/earth_economy_devstack
    • This will add a new folder C:\Users\<YOUR_USERNAME>\Files\earth_economy_devstack

Install Visual Studio Code (VS Code)

  • Install from <code.visualstudio.com/download>
    • For PC, I recommend selecting the “User Installer”, 64-bit option for windows.
    • Use the default install options with the exception of the two that start with “Add Open with Code” action…

Open the Earth Economy Devstack in VS Code workspace

  • Navigate to the directory where you cloned the Earth Economy Devstack
    • In the repository, there is a file called earth_economy_devstack.code-workspace. Launch this.
      • If configured correctly, the full path would be C:\Users\<YOUR_USERNAME>\Files\earth_economy_devstack\earth_economy_devstack.code-workspace (PC) or ~/Files/earth_economy_devstack/earth_economy_devstack.code-workspace (Mac)
  • The workspace file adds a bunch of different repositories to the workspace (which you will need to git clone in to where the workspace says they should be)
  • The workspace also sets up a bunch of launch configurations for debugging and running code using the repositories in your workspace

Configure VS CODE

  • Install required extensions
    • Install the Python extension in VS Code
      • Click on the extensions icon in the left sidebar
      • Search for Python and install the one by Microsoft
    • Install Quarto extension in VS Code
    • Install GitGraph extension in VS Code
  • Sign in to VS Code with your GitHub account
    • Click on the account icon in the bottom left corner of the window and follow prompts

Get the other required repositories and put them in the right directory

Install a Compiler

  • Install C/C++ compiler. This is necessary to edit the underlying C/Cython/.pyx files.
    • If you run any part of the code that uses compiled C++ code (mostly SEALS and the optimized raster functions in Hazelbean), then you will need to have a C++ compiler on your system.
      • Windows:
        • Option 1: You could go to https://visualstudio.microsoft.com/visual-cpp-build-tools/ and select download build tools.
        • Option 2: Enter the following command in the Terminal: winget install Microsoft.VisualStudio.2022.BuildTools --force --override "--passive --wait --add Microsoft.VisualStudio.Workload.VCTools;includeRecommended" This will launch the build-tools installer (you could do this manually via the MS website if you want, but this ensures you get the right tools).
      • Mac can use Xcode to compile the cython files. Most users will already have this installed but if not, follow the directions below.
        • Xcode Command Line Tools: macOS users can install these tools by running xcode-select –install in the Terminal. This command downloads and installs the Xcode Command Line Tools, which includes gcc and clang, the compilers needed to compile C/C++ code on macOS. This is somewhat analogous to the Visual Studio Build Tools on Windows.
  • Run the appropriate setup.py to trigger the Cython compilation
    • Most libraries will auto-compile when you first import them (or change the underlying .pyx file)
    • Alternatively, you can manually compile them by running the corresponding compile_cython_files.py in the repo’s
      • Navigate to the scripts directory and use the terminal to run python compile_cython_files.py build_ext --inplace

Workspace Step by Step Installation

Install Git

  • Get the git software
    • Install and run the installer for your operating system at <Git-scm.com/downloads>
  • Git vs. GitHub
    • Github is a website that hosts code and connects a community of coders.
    • Git is a “version control” software tool that records the history of files in a Git repository.
    • Nearly every coder uses Git to push their repository of code to GitHub.
  • Use the default options for everything
    • Unless you REALLY know what you’re doing.
  • Configure your user name via the command prompt with the following commands
    • git config --global user.name "John Doe"
    • git config --global user.email johndoe@example.com

Verify Git Installation

  • Open up the command prompt and type git to test that it’s installed
    • (PC) Search for cmd in the start menu
  • This is the OG way of working on computers
  • All version control tasks can be done via git commands here, but we will be using VS Code instead

Configure Windows Explorer (optional)

  • Make hidden folders visible
  • Make sure file extensions are visible

Create a python environment

  1. Install Miniforge from https://github.com/conda-forge/miniforge?tab=readme-ov-file#miniforge3 - Be sure to select the correct version for your operating system (Windows, Mac, Linux)

    • If you have an Apple “m1 or m2” chip (a relatively new apple chip, make sure you select the Apple Silicon option).
    • Install in C:\Users\<YOUR_USERNAME>\miniforge3 (PC) or ~/miniconda3 (Mac)
    • During installation, select yes for “Add Mambaforge/Miniforge to my PATH environment Variable”
    • If you get a “Windows Protected your PC”, click more info then Run Anyway.

  2. Open the Miniforge Prompt (PC only, search for it in the start menu) or just type “mamba init” in a new terminal/command line

  • You’ll know it worked if you see (base) in front of your path
    • Base is the default “environment” that you will use.
  1. Create a new mamba environment with the following commands (putting your desired environment name in place of <env_name>):

mamba create -n <env_name> -c conda-forge

- When you have lots of projects, most people create multiple environments specific to each project. 
- For now, we’re going to install everything to the base environment
  1. Activate the environment

mamba activate <env_name>

- You’ll know it worked if you see (env_name) in front of your path
- You can deactivate the environment with `conda deactivate`
- ![](images/2024-02-02-10-39-21.png)
- You can list all environments with `conda env list`
  1. Install libraries using the following mamba command:

mamba install -c conda-forge natcap.invest geopandas rasterstats netCDF4 cartopy xlrd markdown qtpy qtawesome plotly descartes pygeoprocessing taskgraph cython rioxarray dask google-cloud-datastore google-cloud-storage aenum anytree statsmodels openpyxl seaborn twine pyqt ipykernel imageio pandoc conda numba intake more-itertools google-api-python-client google-auth google-auth-oauthlib google-auth-httplib2 gdown tqdm sympy gekko

  • If you don’t add mamba to your path, you can do this manually. On PC, you could use the command
    • SETX PATH "%PATH%;C:\Users\<YOUR_USERNAME>\miniforge;C:\Users\<YOUR_USERNAME>\miniforge;"
  • This step may take a long time because you are downloading and installing hundreds of libraries
  • When you’re done, it should look like the image here.
  • Success! You now have a modern scientific computing environment (sometimes called a scientific computing stack) on your computer!

Clone the Earth Economy Devstack

  • Create a directory for the Earth Economy Devstack at C:\Users\<YOUR_USERNAME>\Files (PC) or ~/Files (Mac)
    • We add a Files directory to keep the root directory clean and to make it easier to find the Earth Economy Devstack in the file explorer.
  • Open a terminal or command prompt and navigate to the directory you created
    • By default, your terminal will open in your user directory, so C:\Users\<YOUR_USERNAME>
    • You can navigate to the directory you created with the command cd Files
      • See image below.
  • Run the git clone command to clone the Earth Economy Devstack repository
    • git clone https://github.com/jandrewjohnson/earth_economy_devstack

Install Visual Studio Code (VS Code)

  • Install from <code.visualstudio.com/download>
    • For PC, I recommend selecting the “User Installer”, 64-bit option for windows.
    • Use the default install options with the exception of the two that start with “Add Open with Code” action…
  • You could just open VS Code now, but we’re going to open it up with a specific Worspace Configuration file below

Open the Earth Economy Devstack in VS Code workspace

  • Navigate to the directory where you cloned the Earth Economy Devstack
    • In the repository, there is a file called earth_economy_devstack.code-workspace. Launch this.
      • If configured correctly, the full path would be C:\Users\<YOUR_USERNAME>\Files\earth_economy_devstack\earth_economy_devstack.code-workspace (PC) or ~/Files/earth_economy_devstack/earth_economy_devstack.code-workspace (Mac)
  • The workspace file adds a bunch of different repositories to the workspace (which you will need to git clone in to where the workspace says they should be)
  • The workspace also sets up a bunch of launch configurations for debugging and running code using the respositories in your workspace
    • We’ll describe debugging more in future sections
      • For now, just know that these launch configurations make sure you’re using the repositories that we’ve added to your workspace
  • Also note that the other repositories shown in the VS Code file explorer will be empty until you git clone them (described below)

Configure VS CODE

  • Sign in to VS Code with your GitHub account
    • Click on the account icon in the bottom left corner of the window and follow prompts
  • Install required extensions
    • Install the Python extension
      • Click on the extensions icon in the left sidebar
      • Search for Python and install the one by Microsoft
    • Install Quarto extension in VS Code
      • We use Quarto to create reports and documents on .qmd, .md and .ipynb files
    • Install Git Graph extension in VS Code
      • Once installed, click the Git Graph button on the bottom-left status bar to see a visual representation of your git history

Organize Directories for the devstack

  • The Earth Economy Devstack organizes repositories in a specific way, described here.
    • Recall that our workspace links to e.g. the hazelbean_dev repository, but it currently points to a directory that doesn’t yet exist.
      • In our Files directory, create a directory named hazelbean (not hazelbean_dev as that will be the repository’s name)
      • Below, we will use git to clone the hazelbean_dev into this directory we just created

Clone the required repositories

  • Instead of using the command line, we will use Git via VS Code’s “Command Pallate”
    • The Command Pallate is accessed via
    • It is a search bar that you can use to run commands in VS Code
  • Once you’ve opened the Command Pallate, type “git clone” and it will search for the command
  • Once you select the command, it will prompt you if you want to write in your Repo’s GitHub URL manually, or you can use VS Code to search the different repositories you have access to
  • Searching via GitHub found, for instance, NatCapTEEMs/gep repo
  • Once you select that, it will ask you what local directory you want to clone the repository to.
    • By default it assumes you want to clone it to your user directory as below
    • We instead want to clone it to the hazelbean directory we created earlier, which will put a new folder hazelbean_dev in that directory
    • To do this, navigate to the hazelbean directory and select it
  • After you’ve cloned a repository, you can now access it in the file explorer

Repeat the cloning approach with all the repos you need

  • If you are a member of NatCap TEEMs or the JohnsonPolaskyLab, you should have access to these repositories via our GitHub organization
  • If you’re not a member, you will still be able to clone all of the public repositories (which are all documented in various journal articles)

Optional Cython Installation steps

If you run any part of the code that uses compiled C++ code (mostly SEALS and the optimized raster functions in Hazelbean), then you will need to have a C++ compiler on your system.

Windows Cython Installation

You could go to https://visualstudio.microsoft.com/visual-cpp-build-tools/ and select download build tools.

However, the following command line in the preferred way. Enter the following command in the Terminal:

winget install Microsoft.VisualStudio.2022.BuildTools --force --override "--passive --wait --add Microsoft.VisualStudio.Workload.VCTools;includeRecommended"

This will launch the build-tools installer (you could do this manually via the MS website if you want, but this ensures you get the right tools).

Mac Cython Installation

Xcode Command Line Tools: macOS users can install these tools by running xcode-select –install in the Terminal. This command downloads and installs the Xcode Command Line Tools, which includes gcc and clang, the compilers needed to compile C/C++ code on macOS. This is somewhat analogous to the Visual Studio Build Tools on Windows.