Skip to main content

pangenome

Tags: alignment core-genome pan-genome phylogeny comparative-genomics bactopia-tool

Pangenome analysis with optional core-genome phylogeny.

This Bactopia Tool creates a pangenome from GFF3 annotation files using one of three tools: Panaroo (default), PIRATE, or Roary. It generates core-genome alignments and gene presence/absence matrices, followed by SNP distance calculations. You can supplement your pangenome with completed genomes using the --species or --accessions parameters, which downloads genomes from RefSeq and annotates them with Prokka. A phylogeny based on the core-genome alignment is created by IQ-Tree, with optional recombination masking using ClonalFrameML. Finally, pan-genome wide association studies can be conducted using Scoary.

Usage​

Bactopia CLI:

bactopia --wf pangenome \
  --bactopia /path/to/your/bactopia/results

Nextflow:

nextflow run bactopia/bactopia/workflows/bactopia-tools/pangenome/main.nf \
  --bactopia /path/to/your/bactopia/results

Outputs​

Expected Output Files​

<BACTOPIA_DIR>
└── <SAMPLE_NAME>
    └── pangenome-<TIMESTAMP>
        ├── clonalframeml
        │   ├── core-genome.ML_sequence.fasta.gz
        │   ├── core-genome.em.txt
        │   ├── core-genome.emsim.txt
        │   ├── core-genome.importation_status.txt
        │   ├── core-genome.labelled_tree.newick
        │   ├── core-genome.position_cross_reference.txt.gz
        │   └── logs
        │       ├── nf.command.{begin,err,log,out,run,sh,trace}
        │       └── versions.yml
        ├── core-genome.distance.tsv
        ├── core-genome.masked.aln.gz
        ├── core-genome.masked.distance.tsv
        ├── core-genome.treefile
        ├── iqtree
        │   ├── core-genome.alninfo.gz
        │   ├── core-genome.bionj
        │   ├── core-genome.ckp.gz
        │   ├── core-genome.contree
        │   ├── core-genome.iqtree
        │   ├── core-genome.log
        │   ├── core-genome.mldist
        │   ├── core-genome.splits.nex
        │   ├── core-genome.ufboot
        │   └── logs
        │       ├── nf.command.{begin,err,log,out,run,sh,trace}
        │       └── versions.yml
        ├── iqtree-fast
        │   ├── logs
        │   │   ├── nf.command.{begin,err,log,out,run,sh,trace}
        │   │   └── versions.yml
        │   ├── roary.bionj
        │   ├── roary.ckp.gz
        │   ├── roary.iqtree
        │   ├── roary.log
        │   ├── roary.mldist
        │   ├── roary.model.gz
        │   └── roary.treefile
        ├── nf-reports
        │   ├── pangenome-dag.dot
        │   ├── pangenome-report.html
        │   └── pangenome-timeline.html
        ├── roary
        │   ├── accessory.header.embl
        │   ├── accessory.tab
        │   ├── accessory_binary_genes.fa.gz
        │   ├── accessory_binary_genes.fa.newick
        │   ├── accessory_graph.dot
        │   ├── blast_identity_frequency.Rtab
        │   ├── clustered_proteins
        │   ├── core_accessory.header.embl
        │   ├── core_accessory.tab
        │   ├── core_accessory_graph.dot
        │   ├── core_alignment_header.embl
        │   ├── core_gene_alignment.aln.gz
        │   ├── gene_presence_absence.Rtab
        │   ├── gene_presence_absence.csv
        │   ├── logs
        │   │   ├── nf.command.{begin,err,log,out,run,sh,trace}
        │   │   └── versions.yml
        │   ├── number_of_conserved_genes.Rtab
        │   ├── number_of_genes_in_pan_genome.Rtab
        │   ├── number_of_new_genes.Rtab
        │   ├── number_of_unique_genes.Rtab
        │   ├── pan_genome_reference.fa.gz
        │   └── summary_statistics.txt
        ├── roary.aln.gz
        ├── scoary
        │   ├── Bogus_trait.results.csv
        │   ├── Tetracycline_resistance.results.csv
        │   └── logs
        │       ├── nf.command.{begin,err,log,out,run,sh,trace}
        │       ├── scoary.log
        │       └── versions.yml
        ├── snpdists
        │   └── logs
        │       ├── nf.command.{begin,err,log,out,run,sh,trace}
        │       └── versions.yml
        └── snpdists-masked
            └── logs
                ├── nf.command.{begin,err,log,out,run,sh,trace}
                └── versions.yml

Pangenome Results​

FileDescription
*.alnCore-genome alignment file containing genes present across all input genomes
*.csvGene presence/absence matrix showing which genes are present in each genome
*.tsvSNP distance matrix between all samples

Phylogeny Results​

note

Only created if --skip_phylogeny is not enabled

FileDescription
*.treefileMaximum likelihood phylogenetic tree in Newick format
*.iqtreeIQ-Tree analysis report with model selection and support values
*.logIQ-Tree execution log

Recombination Analysis​

note

Only created if --skip_recombination is not enabled

FileDescription
*.masked.alnCore-genome alignment with recombination regions masked

Association Analysis​

note

Only created if --scoary_traits is specified

FileDescription
scoary/*Scoary association analysis results and plots

Panaroo Results​

note

Only created when Panaroo is selected as the pangenome tool

FileDescription
panaroo/*Panaroo-specific output files including graph and statistics

PIRATE Results​

note

Only created when PIRATE is selected as the pangenome tool

FileDescription
pirate/*PIRATE-specific output files including gene families and clusters

Roary Results​

note

Only created when Roary is selected as the pangenome tool

FileDescription
roary/*Roary-specific output files including gene presence/absence matrices

Audit Trail​

Below are files that can assist you in understanding which parameters and program versions were used.

Logs​

Each process that is executed will have a folder named logs. In this folder are helpful files for you to review if the need ever arises.

ExtensionDescription
.beginAn empty file used to designate the process started
.errContains STDERR outputs from the process
.logContains both STDERR and STDOUT outputs from the process
.outContains STDOUT outputs from the process
.runThe script Nextflow uses to stage/unstage files and queue processes based on given profile
.shThe script executed by bash for the process
.traceThe Nextflow trace report for the process
versions.ymlA YAML formatted file with program versions

Nextflow Reports​

These Nextflow reports provide great a great summary of your run. These can be used to optimize resource usage and estimate expected costs if using cloud platforms.

FilenameDescription
pangenome-dag.dotThe Nextflow DAG visualization
pangenome-report.htmlThe Nextflow Execution Report
pangenome-timeline.htmlThe Nextflow Timeline Report
pangenome-trace.txtThe Nextflow Trace report

Parameters​

Required Parameters​

Define where the pipeline should find input data and save output data.

ParameterTypeDefaultDescription
--bactopiastringThe path to bactopia results to use as inputs

NCBI Genome Download Parameters​

ParameterTypeDefaultDescription
--speciesstringName of the species to download assemblies
--accessionstringAn NCBI Assembly accession to be downloaded
--accessionsstringAn file of NCBI Assembly accessions (one per line) to be downloaded
--formatstringfastaComma separated list of formats to download
--sectionstringrefseqNCBI section to download
--assembly_levelstringcompleteComma separated list of assembly levels to download
--kingdomstringbacteriaComma separated list of formats to download
--limitstringLimit the number of assemblies to download
--keep_downloadsbooleanfalseSave downloaded files into the bactopia-runs folder

Prokka Parameters​

ParameterTypeDefaultDescription
--prokka_proteinsstringFASTA file of trusted proteins to first annotate from
--prokka_prodigal_tfstringTraining file to use for Prodigal
--prokka_compliantbooleanfalseForce Genbank/ENA/DDJB compliance
--prokka_centrestringBactopiaSequencing centre ID
--prokka_coverageinteger80Minimum coverage on query protein
--prokka_evaluestring1e-09Similarity e-value cut-off
--prokka_optsstringExtra Prokka options in quotes.
--prokka_debugbooleanfalseEnable debug mode for Prokka

PIRATE Parameters​

ParameterTypeDefaultDescription
--use_piratebooleanfalseUse PIRATE instead of panaroo in the 'pangenome' subworkflow
--pirate_stepsstring50,60,70,80,90,95,98Percent identity thresholds to use for pangenome construction
--pirate_featuresstringCDSComma-delimited features to use for pangenome construction
--pirate_para_offbooleanfalseSwitch off paralog identification
--pirate_zbooleanfalseRetain all PIRATE intermediate files
--pirate_pan_optstringAdditional arguments to pass to pangenome construction.

Roary Parameters​

ParameterTypeDefaultDescription
--roary_use_prankbooleanfalseUse PRANK instead of MAFFT for core gene
--use_roarybooleanfalseUse Roary instead of PIRATE in the 'pangenome' subworkflow
--roary_iinteger95Minimum percentage identity for blastp
--roary_cdinteger99Percentage of isolates a gene must be in to be core
--roary_ginteger50000Maximum number of clusters
--roary_sbooleanfalseDo not split paralogs
--roary_apbooleanfalseAllow paralogs in core alignment
--roary_ivnumber1.5MCL inflation value

Panaroo Run Parameters​

ParameterTypeDefaultDescription
--panaroo_modestringstrictThe stringency mode at which to run panaroo (choices: strict, moderate, sensitive)
--panaroo_alignmentstringcoreOutput alignments of core genes or all genes (choices: core, pan)
--panaroo_alignerstringmafftAligner to use for core/pan genome alignment (choices: mafft, prank, clustal)
--panaroo_core_thresholdnumber0.95Core-genome sample threshold
--panaroo_thresholdnumber0.98Sequence identity threshold
--panaroo_family_thresholdnumber0.7Protein family sequence identity threshold
--panaroo_len_dif_percentnumber0.98Length difference cutoff
--panaroo_merge_paralogsbooleanfalseDo not split paralogs
--panaroo_optsstringAdditional options to pass to panaroo

SNP-Dists Parameters​

ParameterTypeDefaultDescription
--snpdists_abooleanfalseCount all differences not just [AGTC]
--snpdists_bbooleanfalseKeep top left corner cell
--snpdists_csvbooleanfalseOutput CSV instead of TSV
--snpdists_kbooleanfalseKeep case, don't uppercase all letters

ClonalFrameML Parameters​

ParameterTypeDefaultDescription
--clonalframeml_emsiminteger100Number of simulations to estimate uncertainty in the EM results
--clonalframeml_optsstringExtra ClonalFrameML options in quotes
--skip_recombinationbooleanfalseSkip ClonalFrameML execution in subworkflows

IQ-TREE Parameters​

ParameterTypeDefaultDescription
--iqtree_modelstringHKYSubstitution model name
--iqtree_bbinteger1000Ultrafast bootstrap replicates
--iqtree_alrtinteger1000SH-like approximate likelihood ratio test replicates
--iqtree_asrbooleanfalseAncestral state reconstruction by empirical Bayes
--iqtree_optsstringExtra IQ-TREE options in quotes.
--skip_phylogenybooleanfalseSkip IQ-TREE execution in subworkflows

Scoary Parameters​

ParameterTypeDefaultDescription
--scoary_traitsstringInput trait table (CSV) to test for associations
--scoary_p_value_cutoffnumber0.05For statistical tests, genes with higher p-values will not be reported
--scoary_correctionstringIApply the indicated filtration measure. (choices: I, B, BH, PW, EPW, P)
--scoary_permuteinteger0Perform N number of permutations of the significant results post-analysis
--scoary_start_colinteger15On which column in the gene presence/absence file do individual strain info start
Filtering Parameters

Use these parameters to specify which samples to include or exclude.

ParameterTypeDefaultDescription
--includestringA text file containing sample names (one per line) to include from the analysis
--excludestringA text file containing sample names (one per line) to exclude from the analysis
Optional Parameters

These optional parameters can be useful in certain settings.

ParameterTypeDefaultDescription
--outdirstringbactopiaBase directory to write results to
--skip_compressionbooleanfalseOutput files will not be compressed
--datasetsstringThe path to cache datasets to
--keep_all_filesbooleanfalseKeeps all analysis files created
Max Job Request Parameters

Set the top limit for requested resources for any single job.

ParameterTypeDefaultDescription
--max_retryinteger3Maximum times to retry a process before allowing it to fail.
--max_cpusinteger4Maximum number of CPUs that can be requested for any single job.
--max_memorystring128.GBMaximum amount of memory that can be requested for any single job.
--max_timestring240.hMaximum amount of time that can be requested for any single job.
--max_downloadsinteger3Maximum number of samples to download at a time
Nextflow Configuration Parameters

Parameters to fine-tune your Nextflow setup.

ParameterTypeDefaultDescription
--nfconfigstringA Nextflow compatible config file for custom profiles, loaded last and will overwrite existing variables if set.
--publish_dir_modestringcopyMethod used to save pipeline results to output directory. (choices: symlink, rellink, link, copy, copyNoFollow, move)
--infodirstring${params.outdir}/pipeline_infoDirectory to keep pipeline Nextflow logs and reports.
--forcebooleanfalseNextflow will overwrite existing output files.
--cleanup_workdirbooleanfalseAfter Bactopia is successfully executed, the work directory will be deleted.
Institutional config options

Parameters used to describe centralized config profiles. These should not be edited.

ParameterTypeDefaultDescription
--custom_config_versionstringmasterGit commit id for Institutional configs.
--custom_config_basestringhttps://raw.githubusercontent.com/nf-core/configs/masterBase directory for Institutional configs.
--config_profile_namestringInstitutional config name.
--config_profile_descriptionstringInstitutional config description.
--config_profile_contactstringInstitutional config contact information.
--config_profile_urlstringInstitutional config URL link.
Nextflow Profile Parameters

Parameters to fine-tune your Nextflow setup.

ParameterTypeDefaultDescription
--condadirstringDirectory to Nextflow should use for Conda environments
--registrystringquay.ioRegistry to pull Docker containers from.
--datasets_cachestring<HOME>/.bactopia/datasetsDirectory where downloaded datasets should be stored.
--singularity_cachestringDirectory where remote Singularity images are stored.
--singularity_pull_docker_containerbooleanInstead of directly downloading Singularity images for use with Singularity, force the workflow to pull and convert Docker containers instead.
--force_rebuildbooleanfalseForce overwrite of existing pre-built environments.
--queuestringgeneral,high-memoryComma-separated name of the queue(s) to be used by a job scheduler (e.g. AWS Batch or SLURM)
--cluster_optsstringAdditional options to pass to the executor. (e.g. SLURM: '--account=my_acct_name'
--container_optsstringAdditional options to pass to Apptainer, Docker, or Singularity. (e.g. Singularity: '-D pwd'
--disable_scratchbooleanfalseAll intermediate files created on worker nodes of will be transferred to the head node.
Helpful Parameters

Uncommonly used parameters that might be useful.

ParameterTypeDefaultDescription
--monochrome_logsbooleanDo not use coloured log outputs.
--nfdirbooleanPrint directory Nextflow has pulled Bactopia to
--sleep_timeinteger5The amount of time (seconds) Nextflow will wait after setting up datasets before execution.
--validate_paramsbooleantrueBoolean whether to validate parameters against the schema at runtime
--helpbooleanDisplay help text.
--wfstringbactopiaSpecify which workflow or Bactopia Tool to execute
--list_wfsbooleanList the available workflows and Bactopia Tools to use with '--wf'
--show_hidden_paramsbooleanShow all params when using --help
--help_allbooleanAn alias for --help --show_hidden_params
--versionbooleanDisplay version text.

Composition​

This workflow uses the following subworkflows:

  • clonalframeml - Detect and mask recombination events in bacterial phylogenies.
  • iqtree - Construct maximum likelihood phylogenetic trees from alignments.
  • ncbigenomedownload - Download bacterial genomes from NCBI's RefSeq database.
  • pangenome - Perform pangenome analysis with optional core-genome phylogeny.
  • prokka - Annotate bacterial genomes with functional information.
  • scoary - Pan-genome wide association studies.

Citations​

If you use this in your analysis, please cite the following.

Source​

View source on GitHub