******************************************************************************* Routine ******************************************************************************* Variables update =============================================================================== .. .. danger:: .. Cette partie n'est plus d'actualité depuis le passage au fichier trame 3. .. Elle est toujours disponible à titre informatif et peut être utilisée par exemple .. lors de mise à jour de code. 1. **Clone the project** The code is available on GitLab : https://gitlab.com/ifb-elixirfr/covid19/EMERGEN-DB .. code-block:: bash git clone https://gitlab.com/ifb-elixirfr/covid19/EMERGEN-DB.git 2. **Add the updated trame file to the folder** :code:`static-apps/version`. .. danger:: It's very important to respect the name rule. This is because the download button in the application will retrieve the last file using the file name and the version number using a regular expression. You must therefore be particularly careful with this name. The name structure is :code:`YYYY-MM-DD_ACRONYME-PF_semaineXX_emergen_typage_vY.Y.Y.xlsx` whare you replace the Y.Y.Y by the version number. Fot example, when writing this documentation the last trame file had the name: :code:`YYYY-MM-DD_ACRONYME-PF_semaineXX_emergen_typage_v3.0.6.xlsx` 3. **Test** : The documentation for testing the code is available here: :doc:`dev_code_quality` 4. **Repo update** .. code-block:: bash git push origin NOM_BRANCHE Than do a une merge request 5. Connect to the server hosting the application 6. Update the code .. code-block:: bash sudo git pull origin master 7. Image update .. code-block:: bash sudo docker-compose -f docker-compose.prod.yml pull web sudo docker-compose -f docker-compose.prod.yml up -d sudo docker-compose -f docker-compose.prod.yml exec web python manage.py collectstatic --no-input --clear sudo docker system prune -a -f And **ONLY** in case column descrition changes: .. code-block:: bash docker-compose -f docker-compose.prod.yml exec web python manage.py load_description **Extremely rare case** : the connexion to the registry is active, but this connexion may be lost. You can use the following command to reconnecter: .. code-block:: bash sudo docker login registry.gitlab.com 8. Notify users - In the admin page, in the :code:`AUTHENTIFICATION ET AUTORISATION`, click on :code:`Utilisateurs`. Select all the users, then in actions : :code:`Send email`. Users update =============================================================================== **Frequency** : Variable (important at the beginning of the project than decreasing) This part is detailed here : :doc:`user_admin` .. note:: As a reminder, the file must be in CSV, Semicolon separator and Unicode encoding (UTF-8) Description update =============================================================================== **Frequency** : Rare Uniquement lors de l'ajout de nouvelle colonne ou de la modification des descriptions. This part is detailed here : :doc:`user_admin` Annotation update =============================================================================== **Fréquence** : Rare This is currently under developement for an attempt at automatic indexing. This part is detailed here : :doc:`user_admin` Static files =============================================================================== **Fréquence** : Rare L'architecture de l'application impose de mettre à jour de temps en temps les fichiers statiques. Par exemple, lors de l’ajout d’un nouveau fichier trame. Les étapes sont : 1. Connect to your server and go to the application folder 2. Updating the git (as static files pass through Gitlab, the aim is to to have a repo that's always up to date) .. code-block:: bash sudo git pull origin master 3. Static files update .. code-block:: bash sudo docker-compose -f docker-compose.prod.yml exec web python manage.py collectstatic --no-input --clear The documentation =============================================================================== **Fréquence** : Rare The documentation is written in sphinx and all the explanations needed to manage it are here: :doc:`dev_documentation` Documentation is automatically generated when the image is created in the Gitlab CI. Shut down and restart the application =============================================================================== **Fréquence** : Rare .. code-block:: bash sudo docker-compose -f docker-compose.prod.yml down sudo docker-compose -f docker-compose.prod.yml up -d Extremely rare case: .. code-block:: bash sudo systemctl start docker Compilation of the Go scripts =============================================================================== .. code-block:: bash docker-compose -f docker-compose.prod.yml exec web sh cd scripts wget https://go.dev/dl/go1.17.8.linux-amd64.tar.gz rm -rf /usr/local/go && tar -C /usr/local -xzf go1.17.8.linux-amd64.tar.gz export PATH=$PATH:/usr/local/go/bin The "scripts" folder contains 5 folder, each containing a specific Go script. 1) export_sequences_fasta_resultID: export all the results with a header build from the uniqueID. .. code-block:: bash cd export_sequences_fasta_resultID go build export_sequences_fasta_resultID.go 2) export_sequences_fasta_seqID: export all the results with a header build from the sequence id. .. code-block:: bash cd export_sequences_fasta_seqID go export_sequences_fasta_seqID.go 3) export_sequences_tsv: export all the results in a tsv file. .. code-block:: bash cd export_sequences_tsv go export_sequences_tsv.go 4) export_metadata_sequences: export all the results and their sequences. .. code-block:: bash cd export_metadata_sequences go export_metadata_sequences.go 5) export_metadata: export all the results .. code-block:: bash cd export_metadata go export_metadata.go EMERGEN-DB dashboards generation on the IFB cluster =============================================================================== **Moving into the right project** .. code-block:: bash cd /shared/projects/emergen_ifb/emergen-external-reports **Creation/update of the Singularity image** .. code-block:: bash singularity build rtools4emergen.sif docker://registry.gitlab.com/ifb-elixirfr/covid19/rtools4emergen:latest **Start the job with slurm manually** .. code-block:: bash sbatch generate_dashboard_report.sh where :code:`generate_dashboard_report.sh` .. code-block:: bash #!/bin/bash module load singularity cd /shared/projects/emergen_ifb/emergen-external-reports # singularity build rtools4emergen.sif docker://registry.gitlab.com/ifb-elixirfr/covid19/rtools4emergen:latest singularity run rtools4emergen.sif bash -c "Rscript --vanilla generate_dashboard_report.R XXXXXXXXXXXXX" curl -X 'POST' \ 'https://emergen-db.france-bioinformatique.fr/fr/api/external/report/upload/' \ -H 'accept: application/json' \ -H 'Content-Type: multipart/form-data' \ -H 'Authorization: Token XXXXXXXXXXXXX' \ -F 'category=Dashboard' \ -F 'external_report=@dashboard.html' \ -F 'public=false' \ -F 'teamNames=IFB' \ -F 'type_file=HTML' \ -F 'update=true' :code:`XXXXXXXXXXX` is your Token in EMERGEN-DB and :code:`generate_dashboard_report.R` .. code-block:: bash #!/usr/bin/env Rscript args = commandArgs(trailingOnly=TRUE) if (length(args)==0) { stop("At least one argument must be supplied (emergen_token).n", call.=FALSE) } library(Rtools4Emergen) emergenToken=args[1] generate_dashboard_report(emergenToken = emergenToken) The documentation is also available: `Doc Dashboard `_