Unable to install ctsm_pylib environment for mksurfdata_esmf in CTSM5.2.005

[keerzhang1]

Brief summary of bug

The ctsm_pylib environment cannot be installed for CTSM5.2.005.

General bug information

CTSM version you are using: CTSM5.2.005

Does this bug cause significantly incorrect results in the model's science? NO

Configurations affected: mksurfdata_esmf tool

Details of bug

I couldn't install the ctsm_pylib environment when running ./py_env_create at the top level of CTSM5.2.005. However, I was able to set up mksurfdata_esmf using the same setup processes for CTSM ctsm5.2.mksurfdata branch.

The error message in CTSM5.2.005 is " PackagesNotFoundError: The following packages are not available from current channels:

• ./python/conda_env_ctsm_py.txt

Current channels:

• https://conda.anaconda.org/conda-forge
• https://conda.anaconda.org/r

To search for alternate channels that may provide the conda package you're
looking for, navigate to

https://anaconda.org

and use the search bar at the top of the page.

Trouble installing the ctsm_pylib python environment
There must be a problem in the ./python/conda_env_ctsm_py.txt conda specification environment file
Change the file and try again"

[olyson]

I can confirm this didn't work for me either with ctsm5.2.005. I got the same error.

cd /glade/work/oleson/ctsm5.2.005
module load conda
./py_env_create

[ekluzek]

In working with @wwieder it looks like the change needed is:

diff --git a/py_env_create b/py_env_create
index 9a9b8d940..ac705edb3 100755
--- a/py_env_create
+++ b/py_env_create
@@ -122,7 +122,7 @@ if [ "$verbose" == "Yes" ]; then
   verbosity="--verbose"
   loglevel="INFO"
 fi
-cmd="conda install --yes $verbosity --channel conda-forge --name $ctsm_python $condadir/$condafile $option"
+cmd="conda install --yes $verbosity --channel conda-forge --name $ctsm_python --file $condadir/$condafile $option"
 echo "$cmd"
 $cmd
 if [ $? != 0 ]; then

I'm guessing the conda version on Derecho was updated to have just slightly different behavior (now requiring the "--file" option for the filename, rather than just accepting the filename as is).

[ekluzek]

Since, we've had this break before, it shows we need to have testing for this. So we need to think about how to do that. One way would be a github action. But, really we want this for Derecho, so we might need to do a cronjob to run this at some frequency. Or add a test for this to our python testing maybe?

The other thing we might need is a check on the conda version to see if it's compatible with the script...

[olyson]

That change fixes the problem for me.

[ekluzek]

One way to try this out is to put the above difference (#2563 (comment)) in a file (called patch.diff) and then do this at the top level

patch --strip=1 < patch.diff

The result of the difference and the patch file most likely will then be identical

git diff py_env_create > git.diff
diff patch.diff git.diff

So this is using the UNIX patch command to apply the difference to the code. There is also a "git apply" command that works similarly.