Build on a cluster#
You may want to build the model on a cluster. While you can build Euro-Calliope on any cluster that is supported by Snakemake, our default configuration is targeted at, and tested on, ETH's Euler cluster.
To build the model on Euler, use the following command:
snakemake --use-conda --profile config/euler
If you want to run on another cluster, read snakemake's documentation on cluster execution and take
config/euler as a starting point.
Work local, build on remote#
If you are like us, you may want to work locally (to change configuration parameters, add modules etc), but execute remotely on the cluster.
We support this workflow through three Snakemake rules:
It works like the following.
First, start local and make sure the
cluster-sync configuration parameters fit your environment.
snakemake --use-conda --cores 1 send to send the entire repository to your cluster.
On the cluster, execute the workflow with Snakemake (see above).
After the workflow has finished, download results by locally running
snakemake --use-conda --cores 1 receive.
By default, this will download results into
This workflow works iteratively too.
After analysing your cluster results locally, you may want to make changes locally, send these changes to the cluster (
snakemake --use-conda --cores 1 send), rerun on the cluster, and download updated results (
snakemake --use-conda --cores 1 receive).
To remove cluster results on your local machine, run
snakemake --use-conda --cores 1 clean_cluster_results.
Be notified of build successes or fails#
As the execution of this workflow may take a while, you can be notified whenever the execution terminates either successfully or unsuccessfully.
Notifications are sent by email.
To activate notifications, add the email address of the recipient to the configuration key
snakemake --use-conda --config email=<your-email>