Application Collector

The HOME application contains the general pages of the EMERGEN-DB application.

Templates

The HOME application contains 3 templates :

  • about.html : about page

  • index.html : home page

  • profile.html : profile page

Key files of the « COLLECTOR » application

management.commands

load_description.py

class apps.collector.management.commands.load_description.Command(stdout=None, stderr=None, no_color=False, force_color=False)
handle(*args, **kwargs)

SUMMARY LINE

Import description column from template

HOW TO USE

docker-compose exec web python manage.py load_description

Add -f docker-compose.prod.yml in Prod

Important

The files are stored: . They must be named according to thesyntax proposed in the following examples:

  • YYYY-MM-DD_ACRONYME-PF_semaineXX_emergen_typage_v1.1.xlsx

  • YYYY-MM-DD_ACRONYME-PF_semaineXX_emergen_typage_v1.2.xlsx

  • YYYY-MM-DD_ACRONYME-PF_semaineXX_emergen_typage_v1.3.xlsx

The last version is automatically retrieved

load_variants.py

class apps.collector.management.commands.load_variants.Command(stdout=None, stderr=None, no_color=False, force_color=False)
add_arguments(parser)

Get arg parser

handle(*args, **options)

SUMMARY LINE

Import variants from template

HOW TO USE

docker-compose exec web python manage.py load_variants

Add -f docker-compose.prod.yml in Prod

You can add -f PATH/FILENAME to specify file use to load description

Important

The files are stored: . They must be named according to thesyntax proposed in the following examples:

  • YYYY-MM-DD_ACRONYME-PF_semaineXX_emergen_typage_v1.1.xlsx

  • YYYY-MM-DD_ACRONYME-PF_semaineXX_emergen_typage_v1.2.xlsx

  • YYYY-MM-DD_ACRONYME-PF_semaineXX_emergen_typage_v1.3.xlsx

The last version is automatically retrieved

admin.py

apps.collector.admin.custom_titled_filter(title)

Function to custom filter title

apps.collector.admin.delete_model_and_files(modeladmin, request, queryset)

Function to delete Excel entries and files

class apps.collector.admin.description_columnsAdmin(model, admin_site)
has_add_permission(request)

Return True if the given request has permission to add an object. Can be overridden by the user in subclasses.

has_delete_permission(request, obj=None)

Return True if the given request has permission to change the given Django model instance, the default implementation doesn’t examine the obj parameter.

Can be overridden by the user in subclasses. In such case it should return True if the given request has permission to delete the obj model instance. If obj is None, this should return True if the given request has permission to delete any object of the given type.

class apps.collector.admin.excel_fileAdmin(model, admin_site)

Create and custom Excel file model admin for admin page

get_groups(obj)

Get user submitter group

has_add_permission(request)

Return True if the given request has permission to add an object. Can be overridden by the user in subclasses.

has_delete_permission(request, obj=None)

Return True if the given request has permission to change the given Django model instance, the default implementation doesn’t examine the obj parameter.

Can be overridden by the user in subclasses. In such case it should return True if the given request has permission to delete the obj model instance. If obj is None, this should return True if the given request has permission to delete any object of the given type.

apps.collector.admin.export_all_to_tsv_admin(modeladmin, request, queryset)

Function to run async export of selected results to Excel in admin

apps.collector.admin.export_zip(modeladmin, request, queryset)

Function to export in zip all Excels

class apps.collector.admin.genomeAdmin(model, admin_site)
has_add_permission(request)

Return True if the given request has permission to add an object. Can be overridden by the user in subclasses.

has_delete_permission(request, obj=None)

Return True if the given request has permission to change the given Django model instance, the default implementation doesn’t examine the obj parameter.

Can be overridden by the user in subclasses. In such case it should return True if the given request has permission to delete the obj model instance. If obj is None, this should return True if the given request has permission to delete any object of the given type.

class apps.collector.admin.multifasta_fileAdmin(model, admin_site)
has_add_permission(request)

Return True if the given request has permission to add an object. Can be overridden by the user in subclasses.

has_delete_permission(request, obj=None)

Return True if the given request has permission to change the given Django model instance, the default implementation doesn’t examine the obj parameter.

Can be overridden by the user in subclasses. In such case it should return True if the given request has permission to delete the obj model instance. If obj is None, this should return True if the given request has permission to delete any object of the given type.

class apps.collector.admin.nextcladeAdmin(model, admin_site)
has_add_permission(request)

Return True if the given request has permission to add an object. Can be overridden by the user in subclasses.

has_delete_permission(request, obj=None)

Return True if the given request has permission to change the given Django model instance, the default implementation doesn’t examine the obj parameter.

Can be overridden by the user in subclasses. In such case it should return True if the given request has permission to delete the obj model instance. If obj is None, this should return True if the given request has permission to delete any object of the given type.

class apps.collector.admin.pangoAdmin(model, admin_site)
has_add_permission(request)

Return True if the given request has permission to add an object. Can be overridden by the user in subclasses.

has_delete_permission(request, obj=None)

Return True if the given request has permission to change the given Django model instance, the default implementation doesn’t examine the obj parameter.

Can be overridden by the user in subclasses. In such case it should return True if the given request has permission to delete the obj model instance. If obj is None, this should return True if the given request has permission to delete any object of the given type.

apps.collector.admin.request_generator(UID_selected)

Function to generate request to export results to excel

class apps.collector.admin.resultAdmin(model, admin_site)

Create and custom result model admin for admin page

has_add_permission(request)

Return True if the given request has permission to add an object. Can be overridden by the user in subclasses.

has_delete_permission(request, obj=None)

Return True if the given request has permission to change the given Django model instance, the default implementation doesn’t examine the obj parameter.

Can be overridden by the user in subclasses. In such case it should return True if the given request has permission to delete the obj model instance. If obj is None, this should return True if the given request has permission to delete any object of the given type.

class apps.collector.admin.variantsAdmin(model, admin_site)

Create and custom result model admin for admin page

changelist_view(request, extra_context=None)

The “change list” admin view for this model.

has_add_permission(request)

Return True if the given request has permission to add an object. Can be overridden by the user in subclasses.

has_delete_permission(request, obj=None)

Return True if the given request has permission to change the given Django model instance, the default implementation doesn’t examine the obj parameter.

Can be overridden by the user in subclasses. In such case it should return True if the given request has permission to delete the obj model instance. If obj is None, this should return True if the given request has permission to delete any object of the given type.

models.py

class apps.collector.models.description_columns(*args, **kwargs)

Description column table

exception DoesNotExist
exception MultipleObjectsReturned
class apps.collector.models.excel_file(*args, **kwargs)

Excel file table

exception DoesNotExist
exception MultipleObjectsReturned
class apps.collector.models.genome(*args, **kwargs)

Genome table

exception DoesNotExist
exception MultipleObjectsReturned
class apps.collector.models.multifasta_file(*args, **kwargs)

Fasta file table

exception DoesNotExist
exception MultipleObjectsReturned
class apps.collector.models.nextclade(*args, **kwargs)

Nextclade table

exception DoesNotExist
exception MultipleObjectsReturned
class apps.collector.models.pango(*args, **kwargs)

Pango table

exception DoesNotExist
exception MultipleObjectsReturned
class apps.collector.models.result(*args, **kwargs)

Result table

exception DoesNotExist
exception MultipleObjectsReturned
clean()

Clean function to check some fields after database checks

apps.collector.models.validate_FINESS(value)

Validator : FINESS géographique du laboratoire préleveur

Paramètres

value (str) – numéro de prélèvement

Lève

ValidationError – Value is not alphanumeric

apps.collector.models.validate_Longeur(value)

Validator : longueur du séquençage

Paramètres

value (string) – Length of sequencing

Lève

ValidationError – Value is not in list of accepted lengths

apps.collector.models.validate_Techno(value)

Validator : Technologie

Paramètres

value (string) – Technology name

Lève

ValidationError – Value is not in list of accepted technologies

apps.collector.models.validate_anneeNaissance(value)

Validator : Année de naissance

Paramètres

value (int) – Year of birth

Lève

ValidationError – Value is not between 1990 and actual year

apps.collector.models.validate_cp(value)

Validator : Code postal

Paramètres

value (int) – postal code

Lève

ValidationError – Value is not numeric or unknown

apps.collector.models.validate_date(value)

Validator : Date

Paramètres

value (date) – Date (format : YYYY-MM-DD)

Lève

ValidationError – Value is empty, not in good format or in the future.

apps.collector.models.validate_date_prelev(value)

Validator : Date de prélèvement

Paramètres

value (date) – Date of sampling (format : YYYY-MM-DD)

Lève

ValidationError – Value is not in good format

apps.collector.models.validate_genre(value)

Validator : genre

Paramètres

value (string) – genre

Lève

ValidationError – Value is not in list of accepted lengths

apps.collector.models.validate_indic_seq(value)

Validator : indication de séquençage

Paramètres

value (int) – 0 if not indicated and 1 if indicated

Lève

ValidationError – Value is not : 0, 1 or “”

apps.collector.models.validate_numPrev(value)

Validator : numéro de prélèvement

Paramètres

value (string) – numéro de prélèvement

Lève

ValidationError – Value is not alphanumeric

class apps.collector.models.variants(*args, **kwargs)

Variants table

exception DoesNotExist
exception MultipleObjectsReturned

test.py

class apps.collector.tests.collectorTestCase(methodName='runTest')

Test for collector app

setUp()

Hook method for setting up the test fixture before exercising it.

tearDown()

Hook method for deconstructing the test fixture after testing it.

test_actions_del()

Test admin action : delete entries

test_actions_export_all_to_tsv_async_admin()

Test admin action : export results

test_actions_zip()

Test admin action : zip

test_admin_custom_titled_filter()

Test admin : change filter title

test_admin_genome_functions()

Test admin : get groups

test_admin_get_groups()

Test admin : get groups

test_ajax_description()

Test ajax return : description column

test_collector_export()

Test post validate to save data

test_collector_export_GISAID()

Test export GISAID files

test_collector_get()

Test get import page

test_collector_post_bad_column()

Test post bad file (with bad column names)

test_collector_post_bad_data()

Test post bad file (with non-conformities)

test_collector_post_bad_data_except()

Test post file with exception

test_collector_post_bad_fasta_except()

Test post fasta file with exception

test_collector_post_bad_file()

Test post bad file (with bad format)

test_collector_post_bad_file_fasta()

Test post bad fasta file (with bad format)

test_collector_post_cancel()

Test cancel with good file

test_collector_post_cancel_wrong()

Test cancel with bad file

test_collector_post_duplicated()

Test post data in same file

test_collector_post_error_prefix()

Test post bad file (with error prefixe)

test_collector_post_good_data()

Test post good file

test_collector_post_good_data_date()

Test post good file with specific date format

test_collector_post_good_data_same_name()

Test post good file with same name

test_collector_post_two_submits()

Test post bad file and corrected file

test_collector_post_validate()

Test post validate to save data

test_collector_post_validate_remove_excel()

Test post validate if excel file is removed

test_collector_post_validate_remove_fasta()

Test post validate if fasta file is removed

test_collector_post_validate_wrong_data()

Test post validate with bad file (with non-conformities)

test_collector_post_validate_wrong_data_bad_template()

Test post validate with bad file (bad template)

test_collector_post_validate_wrong_data_except()

Test post validate with bad file (with exception)

test_download_file()

Test download link : file

test_download_file_bad_user()

Test download link : file (not the owner)

test_download_file_fasta()

Test download link : fasta file

test_download_file_fasta_bad_user()

Test download link : fasta file (not the owner)

test_excel_file()

Test zip file : admin, staff, no group and same group

test_isNum()

Test Validator : is numeric

test_my_uploads_get()

Test get my upload page

test_remove_accents()

Test remove accent

test_request_as_json_desription_column()

Test JSON return : desription columns name

test_request_as_json_uploads_admin()

Test JSON return : uploads

test_request_as_json_uploads_no_group()

Test JSON return : uploads (no group)

test_request_as_json_uploads_same_group()

Test JSON return : uploads (same group as the submitter)

test_str_excel()

Test model : get str from Excel file

test_str_variant()

Test model : get str from variant

test_upload_variant_bad_extension()

Test admin upload variant bad extension

test_upload_variant_bad_file()

Test admin upload variant bad file

test_upload_variants()

Test admin upload variants

test_val2bool()

Test Validator : value to boolean

test_validate_anneeNaissance()

Test Validator : année de naissance

test_validate_cp()

Test Validator : Code postal

test_validate_date()

Test Validator : date

test_validate_date_prelev()

Test Validator : Date du prélèvement

test_validate_indic_seq()

Test Validator : indication de séquençage

test_validate_longeur()

Test Validator : Longueur du séquençage

test_validate_numPrev()

Test Validator : numéro de prélèvement

test_validate_techno()

Test Validator : Technologie de séquençage

test_variants()

Test variant management commands : without param

test_variants_bad_file()

Test variant management commands : bad file format

test_variants_bad_path()

Test variant management commands : bad path file

test_zip()

Test zip file : admin, staff, other group and same group

url.py

The urls.py file contains :

  • Upload/Import page ;

  • My upload page : list of submission by the user and his/her group ;

  • Export page ;

  • Json return results
    • Column descriptions

    • Upload list

  • Download url
    • Download specific file

    • Download zip of files

  • Documentation pages ;

view.py

apps.collector.views.api_report_var_collection_date(self, request, *args, **kwargs)

SUMMARY LINE

Get excel file from report_var_collection_date with API.

Renvoie

Excel file

apps.collector.views.color_conformity_bg(s)

Association background color - status

Paramètres

value (string) – status (Conforme or Non conforme)

Note

  • Green : Conforme

  • Red : Non conforme

apps.collector.views.color_conformity_font(s)

Association Font color - status

Paramètres

value (string) – status (Conforme or Non conforme)

Note

  • Green : Conforme

  • Red : Non conforme

apps.collector.views.creation_result(line, position, excel_file, save)

Creation of result object

Paramètres
  • line – line of Excel file

  • position – Position in the table

  • excel_file – Excel file object

  • save (bool) – True if result is stored in a database, False otherwise

Renvoie

list of error localisations (empty if no error) messages_error_temp: list of error message (empty if no error)

Type renvoyé

loc_error

apps.collector.views.df_var_week_report(query)

Create a dataframe from a query composed by 3 fields - year_week (concatenation of year and week) - Activity (count of labo_sequenceur__name) - Team name (labo_sequenceur__name)

Paramètres

query (sueryset) –

Renvoie

Data frame of team sequencing activity per week

apps.collector.views.download_excel_all(request)

Request and return link to download all group results in Excel file

apps.collector.views.download_fasta_file(request, filename)

Request and return link to download Excel file

apps.collector.views.download_file(request, filename)

Request and return link to download Excel file

apps.collector.views.download_file_all(request, filename)

Request and return link to download a Excel file with all results

apps.collector.views.download_files(request)

Request and return link to download zip of Excel folder

apps.collector.views.download_tar_gz(request, filename)

Request and return link to download tag.gz file

apps.collector.views.export_view(request)

Request group and render export page

apps.collector.views.isNum(data)

Test if Integer

Paramètres

data – Value to be tested

Renvoie

True if integer, False otherwise.

Type renvoyé

bool

apps.collector.views.my_uploads(request)

Request group and render my upload template

apps.collector.views.remove_accents(text)

Remove accents from string. Code available here : https: // www.programcreek.com / python /?CodeExample = remove + accents

Paramètres

text (str) – a string where accents must be replaced

Renvoie

A string without accent

apps.collector.views.remove_greater_than(s)

Remove « > » (greater-than) symbol from string if is in first position

Paramètres

s (string) – fasta header from excel file

Renvoie

String without > in first position

apps.collector.views.report_var_collection_date(request)

Export report by variant and week

apps.collector.views.request_as_json_desription_column(request)

Request and return column descriptions in JSON format

apps.collector.views.request_as_json_uploads(request)

Request and return upload in JSON format

apps.collector.views.send_upload_results(type, mail_submitter, mail_sub_monitor, table_res, count_non_conf=0, link_file=None, count_table=None, PK=None)

Send conformity mail

Paramètres
  • type (string) – success or error

  • mail_submitter (string) – mail of submitter

  • mail_sub_monitor – List of submission monitor mails

  • table_res – Table of results

  • count_non_conf (int) – Number of non conformities

  • link_file (string) – path to Excel file

  • count_table – count of error type

apps.collector.views.upload_file(request)

Import and control Excel file

Renvoie

Render upload template

apps.collector.views.val2bool(value)

Converte val to bool

Paramètres

value – Value to be converted

Renvoie

True if 1, False otherwise.

Type renvoyé

bool