Differences

This shows you the differences between two versions of the page.

--- wiki:uso_sist_en [2022/07/04 13:50]
cnr-guest [Examples of use of some basic commands]
+++ wiki:uso_sist_en [2022/10/07 19:14] (current)
cnr-guest [Available file systems]
@@ Line 23: / Line 23: @@
 To access Ibisco from Windows systems a simple software is PuTTY, freely available at ''https://www.putty.org/''. From Windows 10 onwards it is also possible to use Openssh in a command window (CMD.exe o Powershell.exe). It is pre-installed (if it is not activated, it simply has to be activated in the Optional Features).
-In a few months the access to the cluster will be exclusively via the "user-SSH Key" method.\\
+In a few months the access to the cluster will be exclusively via the "user-SSH Key" method (other secure access methods are being studied).\\
 The current users are invited to generate their key pairs and upload the public key on the server in their home.\\
 The new users, when asking for an account, will follow a lightly different procedure: they will generate the keys pair but will not upload the public key to the server (they will not have yet access): they will send it to the Ibisco admin. The admin will copy it, with the right permissions, in the home of the new user. After that the user will have the ability to enter the system without digiting a server password (but still he/she will have to digit a passphrase, see ahead). \\
@@ Line 73: / Line 73: @@
 In-depth documentation on Lustre is available online, at the link: '' https://www.lustre.org/ ''
+''/ibiscostorage''
+new scratch area shared among UI and computation nodes (available from 07/10/2022), **not** LUSTRE based
 ==== Job preparation ans submission ====
+=== Premise: new job management rules active from 9/10/2022 ===
+To improve the use of resources, the job management rules have been changed.
+ * New usage policies based on // fairshare // mechanisms have been implemented \\
+ * New queues for job submissions have been defined
+    - ** sequential ** queue:
+      * accepts only sequential jobs with a number of tasks not exceeding 1,
+      * who do not use GP-GPUs,
+      * for a total number of jobs running on it not exceeding 128
+      * and maximum execution time limit of 1 week
+    - ** parallel ** queue:
+      * accepts only parallel jobs with task number greater than 1 and less than 1580,
+      * that use no more 64 GP-GPUs
+      * and maximum execution time limit of 1 week
+     - ** gpus ** queue:
+      * only accepts jobs that use no more than 64 GP-GPUs,
+      * with task number less than 1580
+      * and maximum execution time limit of 1 week
+     - ** hparallel ** queue:
+      * accepts only parallel jobs with task number greater than 1580 and less than 3160,
+      * that make use of at least 64 GP-GPUs
+      * and maximum execution time limit of 1 day
+From 9 October the current queue will be disabled and only those defined here will be active, to be explicitly selected. For example, to subdue a job in the ** parallel ** queue, execute \\
+  $ srun -p parallel <MORE OPTIONS> <COMMAND NAME>
+If the job does not comply with the rules of the queue used, it will be terminated.
+=== Use of resources ===
 In the system is installed the resource manager SLURM to manage the cluster resources.
@@ Line 270: / Line 305: @@
 <code>source /nfsexports/SOFTWARE/anaconda3.OK/setupconda.sh
 conda activate bioconda
+<commands execution> [Example: python example.py]
+conda deactivate
+conda deactivate</code>
+=== Pytorch ===
+To use Pytorch sub-environment, the following commands have to be executed.
+<code>source /nfsexports/SOFTWARE/anaconda3.OK/setupconda.sh
+conda activate pytorchenv
 <commands execution> [Example: python example.py]
 conda deactivate
@@ Line 355: / Line 400: @@
    * To use matlab command window, please use '' ssh ibiscohpc-ui.scope.unina.it -l [username] -Y '' when logging into the IBISCO cluster.
    * Setup the matlab environment by using the command '' /nfsexports/SOFTWARE/MATLAB/R2020b/bin/matlab ''. This opens the mathworks command window where you will be able to add the settings file (see ahead).\\
+   * Matlab version R2022a can be accessed using the command '' /nfsexports/SOFTWARE/MATLAB/R2022a/bin/matlab ''.
 == Configuration and execution ==
@@ Line 386: / Line 432: @@
    - the ***Create and Manage Clusters*** window
    - the Matlab Profile commands such as saveProfile  (https://it.mathworks.com/help/parallel-computing/saveprofile.html)
+=== Example of running a parallel matlab script ===
+This is an example of using **parfor** to parallelize the for loop (demonstrated at {{https://it.mathworks.com/help/parallel-computing/decide-when-to-use-parfor.html?s_eid=PSM_15028 | MathWorks}}).
+This example calculates the spectral radius of a matrix and converts a for-loop into a parfor-loop. Open a file named as **test.m** with the following code
+<code>mypool=parpool('SlurmIBISCOHPC', 5) % 5 is the number of workers
+n = 100;
+A = 200;
+a = zeros(n);
+parfor i = 1:n
+    a(i) = max(abs(eig(rand(A))));
+end
+delete(mypool);
+quit
+</code>
+To run this code, the following command executed on the UI can be used:
+<code>
+/nfsexports/SOFTWARE/MATLAB/R2020b/bin/matlab -nodisplay -nosplash -nodesktop -r test
+</code>

ibisco HPC Wiki

User Tools

Site Tools

Differences

Page Tools