Using iPOP-UP core cluster

Join the community forum

iPOP-UP community forum is available at https://discourse.rpbs.univ-paris-diderot.fr/c/ipop-up. Please register in order to stay updated and use the forum to ask for help or new reference files.

Documentation

Please see the complete documentation.

The slides of the training we gave in November 2022 are available here, as well as exemplary sbatch scripts as corrections of the exercices.

The presentation, exercices and corrections of the 2nd session (June 2023) are available here.
The presentation, exercices and corrections of the 3rd session (March 2024) are available here.

Below are the specificities of the iPOP-UP cluster.

Ask for an account and a project

If you had an account on the “old” RPBS cluster, your login and password are unchanged. Otherwise we’ll create a new account for you. In any case you have to ask for a project.

Info
You can have several projects.
You can share a project between several collaborators. All collaborators will have the read and write permissions on the project.

To create an account and/or a project, write an email to ipop-up-account-request@rpbs.univ-paris-diderot.fr with the following information:

• NAME, Surname
• Unit
• Email (institutional adress)
• Projet name (acronym)
• Required disk space (an estimation)
• Data type
• [Facultative] Username of collaborators for this project

We’ll create your account and/or project in the following hours/days. Once your account is ready, you’ll receive an email with your password.

Connect to iPOP-UP cluster

Server : ipop-up.rpbs.univ-paris-diderot.fr

You can connect to iPOP-UP server only via ssh for now. To do so, use your terminal and type the following command replacing username by your iPOP-UP username (last name in lower case usually).

You@YourComputer:~/PathTo/RNAseqProject$ ssh username@ipop-up.rpbs.univ-paris-diderot.fr

You will have to enter your password and then you’ll be connected.

Tip
Once connected, you can change your password using the passwd command.

You@YourComputer:~/PathTo/RNAseqProject$ ssh username@ipop-up.rpbs.univ-paris-diderot.fr
#################################################################
#           _ _____   ____  _____        _    _ _____           #
#          (_)  __ \ / __ \|  __ \      | |  | |  __ \          #
#           _| |__) | |  | | |__) |_____| |  | | |__) |         #
#          | |  ___/| |  | |  ___/______| |  | |  ___/          #
#          | | |    | |__| | |          | |__| | |              #
#          |_|_|     \____/|_|           \____/|_|              #
#                                                               #
#      Hosted by:                                               #
#      Ressource Parisienne en Bioinformatique Structurale      #
#               ---------------------------                     #
#          All connections are monitored and recorded.          #
#   Disconnect IMMEDIATELY if you are not an authorized user!   #
#                                                               #
#################################################################
username@ipop-up.rpbs.univ-paris-diderot.fr's password: 
Last login: Tue Jan 25 15:21:45 2022 from 172.28.18.162
Bienvenue sur le cluster iPOP-UP.

Pour toute question ou demande de support, rejoignez-nous sur le forum de RPBS : https://discourse.rpbs.univ-paris-diderot.fr

Pour changer le compte projet par défaut : sacctmgr update user $USER set defaultaccount=<project-name>
project1 [--------------------]       3 /    1024 GB
project2 [#-------------------]     913 /   10240 GB
Update: 2022-01-21 16:00 - default account in bold - More info: status_bars --help
[username@ipop-up ~]$

Once your project is created you will see it in the list, with the used and available disk space. You can access it at /shared/projects/YourProjectName using the cd command.

[username@ipop-up ~]$ cd /shared/projects/YourProjectName

Tip
In order to navigate easily in your files with your regular file manager, you can mount your project folder as a disk on your local system. Please follow the instructions for Linux or Windows or MacOS. This way, you can modify your files directly using any local text editor.

Set the default account

The first time you use the cluster, it is necessary to define your default account. To do so, run the following command:

sacctmgr update user $USER set defaultaccount=YourProjectName

If you don’t do it, your jobs will quickly be blocked forever in the queue with the AssocGrpCPUMinutesLimit reason.

An alternative in to add the account in your sbatch scripts (see below) using

#SBATCH --partition=ipop-up

Launch jobs

The scheduler is Slurm. You’ll find some examples of scripts in IFB cluster quick start guide.

The only difference is that you have to use the ipop-up partition when starting a job. You can define it using srun :

srun -p ipop-up my_commande 

or sbatch :

sbatch -p ipop-up my_sbatch_script.sh

or directly in your sbatch script adding the line :

#SBATCH --partition=ipop-up

Where to find reference genomes and annotations?

In order to avoid having dozens of copies of the human or mouse genome on the cluster, shared reference files are available in /shared/banks. This folder contains :

• reference genome FASTA files
• aligner (STAR, HISAT2, BWA, …) indexes
• annotation (GTF, GFF) files, …

Before running your analysis, please check if your reference is available in shared/banks. If not, let us know what you need by email or on the community forum and we’ll add it quickly.

Use Jupyter Lab

A Jupyter Lab interface is available at https://jupyterhub.rpbs.univ-paris-diderot.fr to make our work easier!

Use your own Conda environment to run Jupyter notebooks

1. Connect to the cluster with your terminal using SSH.

2. Create your Conda environment (named my_env in the following commands) with all the tools you will need.

3. Add ipykernel in your environment :

   conda activate my_env
   conda install ipykernel
   

4. Add this new environment to your Jupyter Lab :

   conda activate my_env
   python -m ipykernel install --user --name my_env --display-name "my_env"
   conda deactivate
   

5. Connect to Jupyter Lab at https://jupyterhub.rpbs.univ-paris-diderot.fr.

6. Open your notebook, at the top right you can now choose the kernel “my_env” and run your analysis in this environment.

Author : Magali Hennion
Last update : 06/09/2024