teton

Tags: metagenomics classification kraken2 bracken abundance profiling named-workflow

Taxonomic classification and abundance profiling of metagenomic reads.

This workflow performs metagenomic classification using Kraken2 and Bracken, with optional host read removal using deacon (default), nohuman, or SRA Human Scrubber. It processes metagenomic sequencing reads to estimate bacterial genome sizes and separate bacterial from non-bacterial organisms.

Usage

teton CLI:

teton \
  --input samples.csv \
  --outdir results/

Nextflow:

nextflow run bactopia/bactopia/workflows/teton/main.nf \
  --input samples.csv \
  --outdir results/

Outputs

Expected Output Files

<BACTOPIA_DIR>
├── <SAMPLE_NAME>
│   ├── teton-<TIMESTAMP>
│   │   ├── main
│   │   │   └── gather
│   │   │       ├── <SAMPLE_NAME>-meta.tsv
│   │   │       └── logs
│   │   │           ├── nf.command.{begin,err,log,out,run,sh,trace}
│   │   │           └── versions.yml
│   │   ├── teton-prepare
│   │   │   ├── <SAMPLE_NAME>-sizemeup.txt
│   │   │   ├── <SAMPLE_NAME>.bacteria.tsv
│   │   │   ├── <SAMPLE_NAME>.nonbacteria.tsv
│   │   │   └── logs
│   │   │       ├── nf.command.{begin,err,log,out,run,sh,trace}
│   │   │       └── versions.yml
│   │   └── tools
│   │       └── bracken
│   │           ├── <SAMPLE_NAME>.bracken.abundances.txt
│   │           ├── <SAMPLE_NAME>.bracken.adjusted.abundances.txt
│   │           ├── <SAMPLE_NAME>.bracken.classification.txt
│   │           ├── <SAMPLE_NAME>.bracken.krona.html
│   │           ├── <SAMPLE_NAME>.bracken.report.txt
│   │           ├── <SAMPLE_NAME>.bracken.tsv
│   │           ├── <SAMPLE_NAME>.kraken2.krona.html
│   │           ├── <SAMPLE_NAME>.kraken2.report.txt
│   │           └── logs
│   │               ├── nf.command.{begin,err,log,out,run,sh,trace}
│   │               └── versions.yml
│   └── tools
│       └── scrubber
│           ├── <SAMPLE_NAME>.deacon.json
│           ├── <SAMPLE_NAME>.scrub.report.tsv
│           ├── <SAMPLE_NAME>_R1.scrubbed.fastq.gz
│           ├── <SAMPLE_NAME>_R2.scrubbed.fastq.gz
│           └── logs
│               ├── nf.command.{begin,err,log,out,run,sh,trace}
│               └── versions.yml
├── bactopia-runs
│   └── teton-<TIMESTAMP>
│       ├── merged-results
│       │   ├── bracken-adjusted.tsv
│       │   ├── bracken-species-abundance.tsv
│       │   ├── logs
│       │   │   ├── bracken-adjusted-concat
│       │   │   │   ├── nf.command.{begin,err,log,out,run,sh,trace}
│       │   │   │   └── versions.yml
│       │   │   ├── bracken-species-abundance-concat
│       │   │   │   ├── nf.command.{begin,err,log,out,run,sh,trace}
│       │   │   │   └── versions.yml
│       │   │   ├── meta-concat
│       │   │   │   ├── nf.command.{begin,err,log,out,run,sh,trace}
│       │   │   │   └── versions.yml
│       │   │   ├── scrubber-concat
│       │   │   │   ├── nf.command.{begin,err,log,out,run,sh,trace}
│       │   │   │   └── versions.yml
│       │   │   ├── sizemeup-concat
│       │   │   │   ├── nf.command.{begin,err,log,out,run,sh,trace}
│       │   │   │   └── versions.yml
│       │   │   ├── teton-concat
│       │   │   │   ├── nf.command.{begin,err,log,out,run,sh,trace}
│       │   │   │   └── versions.yml
│       │   │   ├── teton-prepare-concat
│       │   │   │   ├── nf.command.{begin,err,log,out,run,sh,trace}
│       │   │   │   └── versions.yml
│       │   │   └── teton-prepare-nonbacteria-concat
│       │   │       ├── nf.command.{begin,err,log,out,run,sh,trace}
│       │   │       └── versions.yml
│       │   ├── meta.tsv
│       │   ├── scrubber.tsv
│       │   ├── sizemeup.tsv
│       │   ├── teton-prepare-nonbacteria.tsv
│       │   ├── teton-prepare.tsv
│       │   └── teton.tsv
│       └── nf-reports
│           ├── teton-dag.dot
│           ├── teton-report.html
│           └── teton-timeline.html
└── merged-results
    ├── <SAMPLE_NAME>.tsv
    └── logs
        └── <SAMPLE_NAME>-join
            ├── nf.command.{begin,err,log,out,run,sh,trace}
            └── versions.yml

Per-Sample Results

File	Description
bacteria.tsv	Per-sample TSV files containing bacterial organisms and their properties
nonbacteria.tsv	Per-sample TSV files containing non-bacterial organisms
sizemeup.tsv	Per-sample TSV files with genome size estimates

Merged Results

File	Description
merged-bacteria.tsv	Consolidated TSV file of all bacterial organisms across samples
merged-nonbacteria.tsv	Consolidated TSV file of all non-bacterial organisms across samples
merged-sizemeup.tsv	Consolidated TSV file of genome size estimates across samples
report.tsv	Joined TSV file combining scrubber and classification results

Audit Trail

Below are files that can assist you in understanding which parameters and program versions were used.

Logs

Each process that is executed will have a folder named logs. In this folder are helpful files for you to review if the need ever arises.

Extension	Description
.begin	An empty file used to designate the process started
.err	Contains STDERR outputs from the process
.log	Contains both STDERR and STDOUT outputs from the process
.out	Contains STDOUT outputs from the process
.run	The script Nextflow uses to stage/unstage files and queue processes based on given profile
.sh	The script executed by bash for the process
.trace	The Nextflow trace report for the process
versions.yml	A YAML formatted file with program versions

Nextflow Reports

These Nextflow reports provide great a great summary of your run. These can be used to optimize resource usage and estimate expected costs if using cloud platforms.

Filename	Description
teton-dag.dot	The Nextflow DAG visualization
teton-report.html	The Nextflow Execution Report
teton-timeline.html	The Nextflow Timeline Report
teton-trace.txt	The Nextflow Trace report

Parameters

Required Parameters

The following parameters are how you will provide either local or remote samples to be processed by Bactopia.

Parameter	Type	Default	Description
--samples	string		A FOFN (via bactopia prepare) with sample names and paths to FASTQ/FASTAs to process
--r1	string		First set of compressed (gzip) Illumina paired-end FASTQ reads (requires --r2 and --sample)
--r2	string		Second set of compressed (gzip) Illumina paired-end FASTQ reads (requires --r1 and --sample)
--se	string		Compressed (gzip) Illumina single-end FASTQ reads (requires --sample)
--ont	string		Compressed (gzip) Oxford Nanopore FASTQ reads (requires --sample)
--hybrid	boolean	false	Create hybrid assembly using Unicycler. (requires --r1, --r2, --ont and --sample)
--short_polish	boolean	false	Create hybrid assembly from long-read assembly and short read polishing. (requires --r1, --r2, --ont and --sample)
--sample	string		Sample name to use for the input sequences
--accessions	string		A file containing ENA/SRA Experiment accessions or NCBI Assembly accessions to processed
--accession	string		Sample name to use for the input sequences
--assembly	string		A assembled genome in compressed FASTA format. (requires --sample)
--check_samples	boolean	false	Validate the input FOFN provided by --samples

Gather Parameters

Parameter	Type	Default	Description
--skip_fastq_check	boolean		Skip minimum requirement checks for input FASTQs
--min_basepairs	integer	2241820	The minimum amount of basepairs required to continue downstream analyses.
--min_reads	integer	7472	The minimum amount of reads required to continue downstream analyses.
--min_coverage	integer	10	The minimum amount of coverage required to continue downstream analyses.
--min_proportion	number	0.5	The minimum proportion of basepairs for paired-end reads to continue downstream analyses.
--min_genome_size	integer	100000	The minimum estimated genome size allowed for the input sequence to continue downstream analyses.
--max_genome_size	integer	18040666	The maximum estimated genome size allowed for the input sequence to continue downstream analyses.
--attempts	integer	3	Maximum times to attempt downloads
--use_ena	boolean		Download FASTQs from ENA
--no_cache	boolean		Skip caching the assembly summary file from ncbi-genome-download

csvtk concat Parameters

Parameter	Type	Default	Description
--csvtk_concat_opts	string		Extra csvtk concat options in quotes

csvtk join Parameters

Parameter	Type	Default	Description
--csvtk_join_opts	string		Extra csvtk join options in quotes

SRA Human Scrubber Parameters

Parameter	Type	Default	Description
--use_srascrubber	boolean	false	Use SRAHumanScrubber for scrubbing human reads

Nohuman Download Parameters

Parameter	Type	Default	Description
--nohuman_db	string		Path to the nohuman database or directory to download it to
--nohuman_db_version	string		Database version to download (default: latest HPRC release)
--nohuman_save_as_tarball	boolean	false	Save the nohuman database as a tarball
--download_nohuman	boolean	false	Download the nohuman database to the path given by --nohuman_db

Nohuman Run Parameters

Parameter	Type	Default	Description
--nohuman_db	string		Path to the nohuman database directory or tarball
--nohuman_confidence	number	0.0	Kraken2 minimum confidence score for classification (0.0-1.0)
--nohuman_human	boolean	false	Invert output to keep only human reads instead of removing them
--nohuman_save_report	boolean	false	Save the Kraken2 classification report

Deacon Fetch Parameters

Parameter	Type	Default	Description
--deacon_index_name	string	panhuman-1	Name of the pre-built deacon index to fetch
--download_deacon	boolean	false	Download the deacon index to the datasets cache
--use_deacon	boolean	false	Use deacon for host read filtering

Deacon Filter Parameters

Parameter	Type	Default	Description
--deacon_abs_threshold	integer	2	Minimum absolute number of minimizer hits for a match
--deacon_db	string		Path to a pre-existing deacon index (.idx) for host read filtering
--deacon_deplete	boolean	true	Discard matching sequences instead of keeping them
--deacon_opts	string		Additional deacon filter options not covered by other parameters
--deacon_prefix_length	integer	0	Search only the first N nucleotides per sequence (0 for all)
--deacon_rel_threshold	number	0.01	Minimum relative proportion (0.0-1.0) of minimizer hits for a match

Kraken2 and Bracken Parameters

Parameter	Type	Default	Description
--kraken2_db	string		The a single tarball or path to a Kraken2 formatted database
--kraken2_quick_mode	boolean	false	Quick operation (use first hit or hits)
--kraken2_confidence	number	0.0	Confidence score threshold between 0 and 1
--kraken2_minimum_base_quality	integer	0	Minimum base quality used in classification
--kraken2_use_mpa_style	boolean	false	Format report output like Kraken 1's kraken-mpa-report
--kraken2_report_zero_counts	boolean	false	Report counts for ALL taxa, even if counts are zero
--kraken2_report_minimizer_data	boolean	false	Include minimizer and distinct minimizer count information in report
--kraken2_use_names	boolean	false	Print scientific names instead of just taxids
--kraken2_memory_mapping	boolean	false	Avoid loading database into RAM
--kraken2_minimum_hit_groups	integer	2	Minimum number of hit groups needed to make a call
--kraken2_keep_filtered_reads	boolean	false	Keep the classified and unclassified FASTQs produced by Kraken2
--kraken2_keep_raw_output	boolean	false	Keep the STDOUT file produced from Kraken2
--bracken_read_length	integer	0	Read length to get all classifications for (0 = determine at runtime)
--bracken_level	string	S	Level to estimate abundance at
--bracken_threshold	integer	0	Reads required PRIOR to abundance estimation to perform re-estimation
--bracken_max_secondary_percent	number	0.01	The maximum percent abundance for the secondary species, if exceeded, sample will remain unclassified
--bracken_skip_krona	boolean	false	Skip the creation of a Krona report

Dataset Parameters

Define where the pipeline should find input data and save output data.

Parameter	Type	Default	Description
--species	string		Name of species for species-specific dataset to use
--ask_merlin	boolean		Ask Merlin to execute species specific Bactopia tools based on Mash distances
--coverage	integer	100	Reduce samples to a given coverage, requires a genome size
--genome_size	integer	0	Expected genome size (bp) for all samples, required for read error correction and read subsampling
--use_bakta	boolean		Use Bakta for annotation, instead of Prokka

Optional Parameters

These optional parameters can be useful in certain settings.

Parameter	Type	Default	Description
--outdir	string	bactopia	Base directory to write results to
--skip_compression	boolean	false	Output files will not be compressed
--datasets	string		The path to cache datasets to
--keep_all_files	boolean	false	Keeps all analysis files created

Max Job Request Parameters

Set the top limit for requested resources for any single job.

Parameter	Type	Default	Description
--max_retry	integer	3	Maximum times to retry a process before allowing it to fail.
--max_cpus	integer	4	Maximum number of CPUs that can be requested for any single job.
--max_memory	string	128.GB	Maximum amount of memory that can be requested for any single job.
--max_time	string	240.h	Maximum amount of time that can be requested for any single job.
--max_downloads	integer	3	Maximum number of samples to download at a time

Nextflow Configuration Parameters

Parameters to fine-tune your Nextflow setup.

Parameter	Type	Default	Description
--nfconfig	string		A Nextflow compatible config file for custom profiles, loaded last and will overwrite existing variables if set.
--publish_dir_mode	string	copy	Method used to save pipeline results to output directory. (choices: symlink, rellink, link, copy, copyNoFollow, move)
--infodir	string	${params.outdir}/pipeline_info	Directory to keep pipeline Nextflow logs and reports.
--force	boolean	false	Nextflow will overwrite existing output files.
--cleanup_workdir	boolean	false	After Bactopia is successfully executed, the work directory will be deleted.

Institutional config options

Parameters used to describe centralized config profiles. These should not be edited.

Parameter	Type	Default	Description
--custom_config_version	string	master	Git commit id for Institutional configs.
--custom_config_base	string	https://raw.githubusercontent.com/nf-core/configs/master	Base directory for Institutional configs.
--config_profile_name	string		Institutional config name.
--config_profile_description	string		Institutional config description.
--config_profile_contact	string		Institutional config contact information.
--config_profile_url	string		Institutional config URL link.

Nextflow Profile Parameters

Parameters to fine-tune your Nextflow setup.

Parameter	Type	Default	Description
--condadir	string		Directory to Nextflow should use for Conda environments
--registry	string	quay.io	Registry to pull Docker containers from.
--datasets_cache	string	<HOME>/.bactopia/datasets	Directory where downloaded datasets should be stored.
--singularity_cache	string		Directory where remote Singularity images are stored.
--singularity_pull_docker_container	boolean		Instead of directly downloading Singularity images for use with Singularity, force the workflow to pull and convert Docker containers instead.
--force_rebuild	boolean	false	Force overwrite of existing pre-built environments.
--queue	string	general,high-memory	Comma-separated name of the queue(s) to be used by a job scheduler (e.g. AWS Batch or SLURM)
--cluster_opts	string		Additional options to pass to the executor. (e.g. SLURM: '--account=my_acct_name'
--container_opts	string		Additional options to pass to Apptainer, Docker, or Singularity. (e.g. Singularity: '-D pwd'
--disable_scratch	boolean	false	All intermediate files created on worker nodes of will be transferred to the head node.

Helpful Parameters

Uncommonly used parameters that might be useful.

Parameter	Type	Default	Description
--monochrome_logs	boolean		Do not use coloured log outputs.
--nfdir	boolean		Print directory Nextflow has pulled Bactopia to
--sleep_time	integer	5	The amount of time (seconds) Nextflow will wait after setting up datasets before execution.
--validate_params	boolean	true	Boolean whether to validate parameters against the schema at runtime
--help	boolean		Display help text.
--wf	string	bactopia	Specify which workflow or Bactopia Tool to execute
--list_wfs	boolean		List the available workflows and Bactopia Tools to use with '--wf'
--show_hidden_params	boolean		Show all params when using --help
--help_all	boolean		An alias for --help --show_hidden_params
--version	boolean		Display version text.

Composition

This workflow uses the following subworkflows:

• bactopia_gather - Search, validate, gather, and standardize input samples.
• teton - Perform taxonomic classification and estimate bacterial genome sizes.

Citations

If you use this in your analysis, please cite the following.

• Bactopia
  Petit III RA, Read TD Bactopia - a flexible pipeline for complete analysis of bacterial genomes. mSystems 5 (2020)

• Bracken
  Lu J, Breitwieser FP, Thielen P, and Salzberg SL Bracken: estimating species abundance in metagenomics data. PeerJ Computer Science, 3, e104. (2017)

• deacon
  Bede N. deacon: SIMD-accelerated filtering of DNA sequences using minimizer-based comparison. (GitHub)

• Kraken2
  Wood DE, Lu J, Langmead B Improved metagenomic analysis with Kraken 2. Genome Biology, 20(1), 257. (2019)

• SRA Human Scrubber
  Katz KS, Shutov O, Lapoint R, Kimelman M, Brister JR, and O'Sullivan C STAT: a fast, scalable, MinHash-based k-mer tool to assess Sequence Read Archive next-generation sequence submissions. Genome Biology, 22(1), 270 (2021)

Source

View source on GitHub