3. Submit, monitor and debug calculations¶
The goal of this section is to understand how to create new data in AiiDA. We will launch a Grand Canonical Monte Carlo simulation and check its results. While for now we will do it ‘manually’, workflows (that we will learn later in this tutorial) can automate this procedure considerably.
3.1. Computer setup and configuration¶
For the tutorial, we’ve created a “virtual supercomputer” on the Amazon elastic cloud, where you can submit your calculations.
Please set up the computer as follows:
$ verdi computer setup
At any prompt, type ? to get some help.
—————————————
=> Computer name: aws Creating new computer with name 'aws'
=> Fully-qualified hostname: 34.244.10.104
=> Description: AWS instance for tutorial
=> Enabled: True
=> Transport type: ssh
=> Scheduler type: torque
=> shebang line at the beginning of the submission script: \#!/bin/console
=> AiiDA work directory: /tmp/{username}/aiida~r~un/
=> mpirun command: mpirun -np {tot_num_mpiprocs}
=> Default number of CPUs per machine: 2
=> Text to prepend to each command execution:
# This is a multiline input, press CTRL+D on a
# empty line when you finish
# ——————————————
# End of old input. You can keep adding
# lines, or press CTRL+D to store this value
# ——————————————
=> Text to append to each command execution:
# This is a multiline input, press CTRL+D on a
# empty line when you finish
# ——————————————
# End of old input. You can keep adding
# lines, or press CTRL+D to store this value
# ——————————————
Computer 'aws' successfully stored in DB.
At this point, the computer node has been created in the database, see
$ verdi computer list -a
but it hasn’t yet been configured.
In order to access the computer, download the SSH key
$ wget https://www.dropbox.com/s/.../aiida_tutorial_aiidaaccount?dl=1 -O /home/max/.ssh/aws.pem
and use it to configure the aws
computer:
$ verdi computer configure aws
Configuring computer 'aws' for the AiiDA user 'aiida@localhost'
Computer aws has transport of type ssh
Note: to leave a field unconfigured, leave it empty and press [Enter]
=> username = aiida
=> port = 22
=> look for keys = True
=> key filename = /home/max/.ssh/aws.pem
=> timeout = 60
=> allow agent =
=> proxy command =
=> compress = True
=> gssauth = no
=> gsskex = no
=> gssdelegcreds = no
=> gsshost = 34.244.10.104
=> load system hostkeys = True
=> key policy = AutoAddPolicy
Configuration stored for your user on computer 'aws'.
Finally, let aiida test the computer:
$ verdi computer test aws
3.2. Code setup and configuration¶
Next, we need to let AiiDA know about the computer codes available on our “virtual supercomputer”.
Let’s set up the RASPA2 code as follows:
$ verdi code setup
At any prompt, type ? to get some help.
—————————————
=> Label: raspa
=> Description: Raspa code for molsim course
=> Local: False
=> Default input plugin: raspa
=> Remote computer name: aws
=> Remote absolute path: /home/aiida/.local/bin/simulate
=> Text to prepend to each command execution
FOR INSTANCE, MODULES TO BE LOADED FOR THIS CODE:
# This is a multiline input, press CTRL+D on a
# empty line when you finish
# ——————————————
# End of old input. You can keep adding
# lines, or press CTRL+D to store this value
# ——————————————
=> Text to append to each command execution:
# This is a multiline input, press CTRL+D on a
# empty line when you finish
# ——————————————
# End of old input. You can keep adding
# lines, or press CTRL+D to store this value
# ——————————————
Code 'raspa' successfully stored in DB.
The list of codes should now include your new code raspa@aws
$ verdi computer test aws
3.3. The AiiDA daemon¶
First of all check that the AiiDA daemon is actually running. The AiiDA daemon is a program running all the time in the background, checking if new calculations appear and need to be submitted to the scheduler. The daemon also takes care of all the necessary operations before the calculation submission, and after the calculation has completed on the cluster. Type in the terminal
$ verdi daemon status
If the daemon is running, the output should look like
# Most recent daemon timestamp:0h:00m:26s ago
## Found 1 process running:
* aiida-daemon[aiida-daemon] RUNNING pid 15044, uptime 3 days, 15:38:41
If this is not the case, type in the terminal
$ verdi daemon start
to start the daemon.
3.4. Creating a new calculation¶
To start please download the AiiDA submission
script. To launch a
calculation, you will need to interact with AiiDA mainly in the
verdi shell
. We strongly suggest you to first try the commands in
the shell, and then copy them in a script “test_pw.py” using a text
editor. This will be very useful for later execution of a similar series
of commands.
The best way to run python scripts using AiiDA functionalities is to run them in a terminal by means of the command
$ verdi run <scriptname>
Every calculation sent to a cluster is linked to a code, which describes the executable file to be used. Therefore, first load the suitable code:
from aiida.common.example_helpers import test_and_get_code
code = test_and_get_code(codename, expected_code_type='raspa')
Here test_and_get_code
is an AiiDA function handling all possible
codes, and code
is a class instance provided as codename
(see
the first part of the tutorial for listing all codes installed in your
AiiDA machine). For this example use codename raspa@aws
.
AiiDA calculations are instances of the class Calculation
, more
precisely of one of its subclasses, each corresponding to a code
specific plugin (for example, the Raspa plugin). We create a new
calculation using the new_calc
method of the code
object:
calc = code.new_calc()
This creates and initializes an instance of the RaspaCalculation
class, the subclass associated with the raspa
plugin. Sometimes, you
might find convenient to annotate information assigning a (short) label
or a (long) description, like:
calc.label='Raspa test'
calc.description='My first AiiDA calc with Raspa'
This information will be saved in the database for later query or inspection.
Now you have to specify the number of machines (a.k.a. cluster nodes) you are going to run on and the maximum time allowed for the calculation — this information is passed to the scheduler that handles the queue:
calc.set_resources({'num_machines': 1, 'num_mpiprocs_per_machine':1})
calc.set_max_wallclock_seconds(30*60)