vRhyme
Binning Virus Genomes from Metagenomes
March 2022
Kristopher Kieft
kieft@wisc.edu
Anantharaman Lab
University of Wisconsin-Madison
Current Version
vRhyme v1.1.0
Citation
If you find vRhyme useful please consider citing our manuscript on Nucleic Acids Research:
Kieft, K., Adams, A., Salamzade, R., Kalan, L., & Anantharaman, K. vRhyme enables binning of viral genomes from metagenomes. Nucleic Acids Research, 2022.
Table of Contents:
- Updates
- Program Description
- Installation
- Requirements
- Program Dependencies
- Python3 Dependencies
- Running vRhyme
- Test examples
- Quick run examples
- Output Explanations
- Useful outputs
- Other outputs
- Interpreting vRhyme bins/vMAGs
- vRhyme Files and Folders
- vRhyme Flag Descriptions
- Flag compatibility
- Commonly used flags
- Other flags
- Contact
Updates for v1.1.0 (March 2022):
- Added coverage outlier removal
- Added circular scaffold (i.e., complete viral genome) removal pre-binning
- Added auxiliary script for identifying circular scaffolds
- Added new dependency: networkx
- Modified --max_edges from a default of 6 to an automatic dynamic range of 3-6
- Modified --penalty_n maximum to be <30% of the number of input samples
- Modified setup and fixed setup bugs
- Replaced MiniBatchKMeans bin refinement with Label Propagation (networkx)
- Resolved bug associated with generating non-connected networks of bins during refinement
Updates for v1.0.0 (December 2021):
- Initial release
- Fixed pip install bugs
Program Description
vRhyme Description
vRhyme is a multi-functional tool for binning virus genomes from metagenomes. vRhyme functions by utilizing coverage variance comparisons and supervised machine learning classification of sequence features to construct viral metagenome-assembled genomes (vMAGs).
IMPORTANT NOTE: vRhyme is built to run on viral sequences/scaffolds. A typical workflow is to predict viruses from a metagenome (e.g., with VIBRANT or VirSorter) and then use those predictions as input to vRhyme. vRhyme can take an entire metagenome as input, but the performance for a whole metagenome has not been fully evaluated. vRhyme is not meant to bin microbes.
Why "vRhyme"? The similarity in sequence features between two scaffolds can be used to identify fragments of the same genome, such as tetranucleotide frequencies, codon usage or GC content. It's almost like rhyming sequences together to create pairs that sound similar, at least metaphorically at the nucleotide level. Coverage variance helps to separate scaffolds that sound the same but are actually different genomes.
vRhyme Features
- Binning vMAGs
- Automatic bin refinement
- Automatic read coverage calculation and refinement
- Bin scoring and redundancy checks (Mmseqs2)
- Supervised machine learning: neural networks and decision trees
- Scaffold dereplication (Mash, Nucmer)
- Read alignment (Bowtie2, BWA)
- ORF prediction (Prodigal)
Installation
GitHub, pip, and conda
git clone https://github.com/AnantharamanLab/vRhyme
cd vRhyme
gunzip vRhyme/models/vRhyme_machine_model_ET.sav.gz ← NOTE: vRhyme is a subdirectory of the parent vRhyme
- optional create a conda environment (see examples below)
- optional activate conda environment if you made one
pip install . ← NOTE: don't forget the dot (pip install [dot])
Installing with pip is optional but suggested. Using pip will collect Python dependencies and add vRhyme to your system PATH. Note that vRhyme.egg-info and build should be created after the pip install. Without pip, vRhyme can still be executed directly from the git clone, just ensure executable permissions (cd vRhyme/; chmod +x vRhyme scripts/*.py aux/*.py). The conda environment is also optional but can be useful for downloading and managing program dependencies.
Example Conda Environments
- with pip install
conda create -c bioconda -n vRhyme python=3 samtools mash mummer mmseqs2 prodigal bowtie2 bwa
-
conda activate vRhyme
-
directly from git clone
conda create -c bioconda -n vRhyme python=3 networkx pandas numpy numba scikit-learn pysam samtools mash mummer mmseqs2 prodigal bowtie2 bwa
conda activate vRhyme
Test the Installation
Test and validate the installed dependencies for vRhyme. In these tests you're looking for Success statements. All Python dependencies must be Success. Please update them if prompted. Both machine learning models must be Success. For program dependencies, only Mmseqs is required to be Success, the others are optional depending on vRhyme usage. If you use any coverage input beside -c then Samtools is required to be Success. If you skip vRhyme -p or -g flags then Prodigal must be Success. If you plan to use vRhyme's dereplication function (see --method) then Mash and Nucmer must be Success. If you input reads (-r/-v/-u) then Bowtie2 and/or BWA (see --aligner) must be Success.
Print the main vRhyme help page:
vRhyme -h
Requirements
Program Dependencies
Please ensure the following programs are installed and in your machine's PATH. Note: most downloads will automatically place these programs in your PATH. See each program's source website for installation guides.
Required
- Python3 (version >= 3.6)
- Mmseqs2
- Samtools
Optional (depends on usage)
- Prodigal
- Mash
- Nucmer
- Bowtie2
- BWA
Python3 Dependencies
There are several Python3 dependencies that must be installed as well. You may already have most of these installed. See each package's source website for installation guides. Versions are important.
- Pandas (version >= 1.0.0)
- Numpy (version >= 1.17.0)
- Scikit-learn (version >= 0.23.0)
- Numba (version >= 0.50.0)
- PySam (version >= 0.15.0)
- NetworkX (version >= 2.0)
Running vRhyme
Test run examples
Test out vRhyme on the provided example dataset. NOTE: If you choose to not install with pip, the vRhyme executable is within the vRhyme/ subdirectory.
cd examples/
minimal coverage table input example
vRhyme -i example_scaffolds.fasta -c example_coverage_values.tsv -t 1
full coverage table input example
vRhyme -i example_scaffolds.fasta -o vRhyme_example_results_coverage-table/ -c example_coverage_values.tsv -p example_scaffolds.prodigal.faa -g example_scaffolds.prodigal.ffn -t 2
Quick run examples
minimum input example with bam files
vRhyme -i fasta -b bam_folder/*.bam
minimum input example with a coverage file
vRhyme -i fasta -c coverage_file.tsv
full BAM input example
vRhyme -i fasta -g genes -p proteins -b bam_folder/*.bam -t threads -o output_folder/
reads input example with dereplication
vRhyme -i fasta -g genes -p proteins -r paired_reads_folder/*.fastq -t threads -o output_folder --method longest
only use dereplicate function
vRhyme -i input_fasta -t threads -o output_folder/ --derep_only --method longest
Output Explanations
Useful outputs
- log_vRhyme_(-i).log: main log file
- vRhyme_best_bins.#.membership.tsv: scaffold membership of best bins
- vRhyme_best_bins.#.summary.tsv: summary stats of best bins
- vRhyme_best_bins_fasta/: folder containing fasta files for best bins
Other outputs
- log_vRhyme_paired_reads.log: log of which reads files were paired together with
-r. Exists with -r.
- (-i).prodigal.faa/(-i).prodigal.ffn: proteins/genes predicted by Prodigal for the length-filtered input sequences. Exists in the absense of
-p/-g.
- (-i).circular.tsv: list of input scaffolds that are circular (i.e., complete viral genomes). The terminal repeat (TR) type can be direct (DTR) or inverted (ITR). vRhyme will check for repeat lengths of at least 20bp and up to 2 mismatches; turn off with --keep_circ. The auxiliary script
aux/flag_circular.py performs the same function and allows for modified parameters.
- vRhyme_machine_distances.tsv: raw distance calculation values used as input for the machine learning model(s)
- vRhyme_coverage_files: folder containing one file per sample with coverage values per scaffold. The row names can be found in vRhyme_names.txt. vRhyme_coverage_values.tsv is the combination of all sample coverages and is equivalent to
-c. Does not exist with -c.
- vRhyme_alternate_bins: folder containing membership and summary files for each alternative binning iteration. vRhyme_bin_scoring.tsv contains the scoring information and rationale for the best iteration chosen as the best bins.
- vRhyme_bam_files: contains vRhyme-generated BAM files. Exists with
-r/-u/-v/-s. Only sorted BAM files will exist unless --keep_bam.
- vRhyme_sam_files: contains vRhyme-generated SAM files. Exists with
-r/-u/-v and --keep_sam.
Hierarchy
- '>' : folder
- '-' : file
- '*' : indicates file/folder may not exist
> main output folder (-o)
- log_vRhyme_(-i).log
- log_vRhyme_paired_reads.log *
- (-i).prodigal.faa *
- (-i).prodigal.ffn *
- (-i).circular.tsv
- vRhyme_best_bins.#.membership.tsv
- vRhyme_best_bins.#.summary.tsv
- vRhyme_machine_distances.tsv
> vRhyme_coverage_files *
- (sample).coverage.tsv
- vRhyme_coverage_values.tsv
- vRhyme_names.txt
> vRhyme_alternate_bins
- #.membership.tsv
- #.summary.tsv
- vRhyme_bin_scoring.tsv
> vRhyme_best_bins_fasta
- vRhyme_bin_#.faa
- vRhyme_bin_#.fasta
- vRhyme_bin_#.ffn
> vRhyme_bam_files *
- (sample).sorted.bam
- (sample).bam *
- (sample).bam.bai *
> vRhyme_sam_files *
- (sample).sam
Interpreting vRhyme bins/vMAGs
The following bullet points are guidelines on interpreting binning results from vRhyme. Please note that this list is not exhaustive. For examples and data, please see analyses done in the vRhyme publication.
- Use protein redundancy as a metric of contamination
- Contamination: a bin containing sequences from multiple viral genomes
- Viruses do not encode universal single copy genes and therefore identifying contaminated vMAGs is difficult. For bacteria and archaea, a common practice is to estimate this with the tool CheckM. A useful tool for viruses is CheckV, though CheckV cannot identify if a vMAG contains sequences from multiple viral genomes, though it may identify if the genome is larger than expected. vRhyme implements a "protein redundancy" check for this purpose based on the concept that few viruses encode redundant proteins (e.g., >50% homology). Foremost, protein redundancy is used by vRhyme to construct bins and select the optimal iteration. However, this metric can be used post-binning to filter out bins that are likely contaminated. The vRhyme publication has a supplemental figure for this topic.
- Bins unlikely to be contaminated will have 0-1 redundant proteins.
- Bins with approximately 2-5 redundant proteins may not be contaminated but there are few such examples.
- Bins with >6 redundant proteins are often contaminated. Notable exceptions include NCLDVs, which can have ~10 redundant proteins in an uncontaminated bin.
- Expected bin sizes
- Bin size: number of sequences (members)
- Most vRhyme bins will be just 2-3 members
- Although some viruses require binning to reconstruct fragmented genomes, there typically aren't many fragments to put back together. Most vRhyme bins will be just 2-3 members and few will exceed 4 members. However, a bin with >4 members should not automatically cause skepticism. In benchmarks, several examples of uncontaminated bins with >4 members were identified. Two examples detailed in the vRhyme publication are a 22-member Herelleviridae vMAG (115kb) and an 11-member NCLDV vMAG (1.6Mb). A megaphage (540kb) artificially split into 51 fragments generated a respective 51-member bin.
- Unbinned sequences are still useful
- Unbinned sequences: any input viral sequence that was not placed into a bin
- Not all viral genomes will require binning and will remain as individual sequences. There is still high utility in unbinned viral sequences as they may accurately represent a viral population/genome. Any unbinned sequence or unused bin (e.g., individual members from bins with contamination) should be combined with the vMAGs for analyses.
- Prophages can be difficult to bin
- Prophage: temperate phage integrated into the hos