Installation ============ The installation procedure depends on what version you'd like to install. Activate **e3sm_unified** environment ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If you have an account on one of the E3SM supported machines (NERSC, Compy, Acme1, LCRC, Cooley, Rhea), you can access ``e3sm_diags`` by activating ``e3sm_unified``, which is a conda environment that pulls together Python and other E3SM analysis tools such as ``e3sm_diags``, ``mpas-analysis``, ``NCO``, ``cdat`` and ``processflow``. The paths to ``e3sm_unified`` activation scripts are machine dependent: **Compy** :: source /share/apps/E3SM/conda_envs/load_latest_e3sm_unified_compy.sh **NERSC** :: source /global/common/software/e3sm/anaconda_envs/load_latest_e3sm_unified_cori-haswell.sh **LCRC** :: source /lcrc/soft/climate/e3sm-unified/load_latest_e3sm_unified_chrysalis.sh **Cooley** :: source /lus/theta-fs0/projects/ccsm/acme/tools/e3sm-unified/load_latest_e3sm_unified_cooley.sh **acme1** :: source /usr/local/e3sm_unified/envs/load_latest_e3sm_unified_acme1.sh Change ``.sh`` to ``.csh`` for csh shells. Note that ``e3sm_unified``'s development cycle is not in phase with ``e3sm_diags``, therefore the version of ``e3sm_diags`` included may not be the latest. To install latest stable releases, refer to following: .. _conda_environment: Installation in a Conda Environment ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If the E3SM Unified environment doesn't serve your needs, you can alternatively install the latest version in your own custom conda environment. First, activate conda or install it if not available. Details vary on the machine. Compy ~~~~~ :: module load anaconda3/2019.03 source /share/apps/anaconda3/2019.03/etc/profile.d/conda.sh NERSC ~~~~~ :: module load python/3.7-anaconda-2019.10 source /global/common/cori_cle7/software/python/3.7-anaconda-2019.10/etc/profile.d/conda.sh .. _conda_environment_others: Others/Local ~~~~~~~~~~~~ If the system doesn't come with conda pre-installed, follow these instructions: 1. Download Mambaforge Linux :: wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh MacOS x86_64 :: wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-MacOSX-x86_64.sh 2. Install Mambaforge Linux :: bash ./Mambaforge-Linux-x86_64.sh MacOS x86_64 :: bash ./Mambaforge-MacOSX-x86_64.sh When you see: :: by running conda init? [yes|no] [no] >>> yes respond with ``yes`` so ``conda`` and ``mamba`` commands are available on initializing a new bash terminal. 3. If you are working on a machine/network that intercepts SSL communications (such as acme1), you will get an SSL error unless you disable the SSL verification: :: conda config --set ssl_verify false binstar config --set ssl_verify False 4. Once conda and mamba are properly working, you can install the **(a) Latest Stable Release** or create a **(b) Development Environment**. .. _install_latest: (a) Latest Stable Release ------------------------- 1. Follow :ref:`"Others/Local" ` section for installing Conda. 2. Get the yml file to create an environment. :: wget https://raw.githubusercontent.com/E3SM-Project/e3sm_diags/main/conda-env/prod.yml 3. Change ``prefix`` in that file to be your conda prefix. Typically, this will be ``~/miniconda3/envs/e3sm_diags_env``. 4. Remove any cached conda packages. This will ensure that you always get the latest packages :: mamba clean --all 5. Use conda to create a new environment with E3SM Diags (``e3sm_diags``) included. These steps should not be necessary if you installed Mambaforge as suggested above but may be needed if you have previously installed Miniconda3 instead: :: conda install -y -n base mamba conda config --add channels conda-forge conda config --set channel_priority strict Create a new conda environment with ``e3sm_diags`` installed and activate it: :: mamba env create -f conda-env/prod.yml # Tip: Add the flag ``-n `` to customize the name of the environment conda activate e3sm_diags_env .. _dev-env: (b) Development Environment --------------------------- Unlike the latest stable release (i.e., the user environment), the development environment does not include E3SM Diags (``e3sm-diags``). Instead, the developer will ``pip install .`` to build ``e3sm-diags`` with changes (see step 6 below). .. note:: The dev environment includes quality assurance (QA) tools such as code formatters, linters, and ``pre-commit``. **You must use the dev environment for all contributions** because these QA tools are enforced using ``pre-commit`` checks in the continuous integration/continuous deployment build. 1. Follow :ref:`"Others/Local" ` section for installing conda. 2. Clone your fork and keep it in sync with the main repo's ``main`` :: # Go to https://github.com/E3SM-Project/e3sm_diags # Click "Fork" in the upper right hand corner. This will fork the main repo. # Click the green "Code" button # Choose the HTTPS or SSH option. # (To use the SSH option, you need to have a SSH connection to GitHub set up). # Click the clipboard icon to copy the path. # On your command line: git clone git remote -v # You should see your fork listed as `origin` or if you already have a clone of your fork, rebase your fork on the main repo's ``main`` to keep it in sync: :: # Add the main repo as a remote. # You can call it anything but "upstream" is recommended. # We'll use `` here. git remote add https://github.com/E3SM-Project/e3sm_diags.git # Fetch all the branches of that remote into remote-tracking branches git fetch # Make sure that you're on your main branch: git checkout main # Rewrite your main branch so that any of your commits that # aren't already in /main are replayed on top of that branch: git rebase /main # Push your main branch to your GitHub fork: # Note that should be `origin` if you cloned your fork as above. git push -f main Checkout a new branch from ``main``. :: git checkout -b main 3. Remove any cached conda packages. This will ensure that you always get the latest packages. :: mamba clean --all 4. Enter the fork directory. :: cd e3sm_diags 5. Use conda to create a new dev environment (``e3sm_diags`` **is not included in this environment**). - Tip: Add the flag ``-n `` to customize the name of the environment :: mamba env create -f conda-env/dev.yml conda activate e3sm_diags_env_dev 6. Install ``pre-commit``. :: pre-commit install 7. Make the desired changes to E3SM Diags, then rebuild and install with: :: pip install . 8. Check that tests pass: ``./tests/test.sh``. This takes about 4 minutes. 9. Commit changes and make sure ``pre-commit`` checks pass :: git commit -m "..." .. figure:: pre-commit-passing.png :alt: pre-commit Output ``pre-commit`` Output