MCPcopy Index your code
hub / github.com/AnantharamanLab/vRhyme

github.com/AnantharamanLab/vRhyme @v1.1.0

Chat with this repo
repository ↗ · DeepWiki ↗ · release v1.1.0 ↗ · + Follow
150 symbols 383 edges 32 files 92 documented · 61% updated 3y agov1.1.0 · 2022-07-13★ 7533 open issues
What it actually does AI analysis from the code graph — generated when you open this
loading…
README

vRhyme

Binning Virus Genomes from Metagenomes

March 2022   
Kristopher Kieft  
kieft@wisc.edu  
Anantharaman Lab  
University of Wisconsin-Madison  

Current Version

vRhyme v1.1.0

Citation

If you find vRhyme useful please consider citing our manuscript on Nucleic Acids Research:
Kieft, K., Adams, A., Salamzade, R., Kalan, L., & Anantharaman, K. vRhyme enables binning of viral genomes from metagenomes. Nucleic Acids Research, 2022.


Table of Contents:

  1. Updates
    • v1.1.0
    • v1.0.0
  2. Program Description
  3. Installation
  4. Requirements
    • Program Dependencies
    • Python3 Dependencies
  5. Running vRhyme
    • Test examples
    • Quick run examples
  6. Output Explanations
    • Useful outputs
    • Other outputs
  7. Interpreting vRhyme bins/vMAGs
  8. vRhyme Files and Folders
  9. vRhyme Flag Descriptions
    • Flag compatibility
    • Commonly used flags
    • Other flags
  10. Contact

Updates for v1.1.0 (March 2022):

  • Added coverage outlier removal
  • Added circular scaffold (i.e., complete viral genome) removal pre-binning
  • Added auxiliary script for identifying circular scaffolds
  • Added new dependency: networkx
  • Modified --max_edges from a default of 6 to an automatic dynamic range of 3-6
  • Modified --penalty_n maximum to be <30% of the number of input samples
  • Modified setup and fixed setup bugs
  • Replaced MiniBatchKMeans bin refinement with Label Propagation (networkx)
  • Resolved bug associated with generating non-connected networks of bins during refinement

Updates for v1.0.0 (December 2021):

  • Initial release
  • Fixed pip install bugs

Program Description

vRhyme Description

vRhyme is a multi-functional tool for binning virus genomes from metagenomes. vRhyme functions by utilizing coverage variance comparisons and supervised machine learning classification of sequence features to construct viral metagenome-assembled genomes (vMAGs).

IMPORTANT NOTE: vRhyme is built to run on viral sequences/scaffolds. A typical workflow is to predict viruses from a metagenome (e.g., with VIBRANT or VirSorter) and then use those predictions as input to vRhyme. vRhyme can take an entire metagenome as input, but the performance for a whole metagenome has not been fully evaluated. vRhyme is not meant to bin microbes.

Why "vRhyme"? The similarity in sequence features between two scaffolds can be used to identify fragments of the same genome, such as tetranucleotide frequencies, codon usage or GC content. It's almost like rhyming sequences together to create pairs that sound similar, at least metaphorically at the nucleotide level. Coverage variance helps to separate scaffolds that sound the same but are actually different genomes.

vRhyme Features

  • Binning vMAGs
  • Automatic bin refinement
  • Automatic read coverage calculation and refinement
  • Bin scoring and redundancy checks (Mmseqs2)
  • Supervised machine learning: neural networks and decision trees
  • Scaffold dereplication (Mash, Nucmer)
  • Read alignment (Bowtie2, BWA)
  • ORF prediction (Prodigal)

Installation

GitHub, pip, and conda

  1. git clone https://github.com/AnantharamanLab/vRhyme
  2. cd vRhyme
  3. gunzip vRhyme/models/vRhyme_machine_model_ET.sav.gz ← NOTE: vRhyme is a subdirectory of the parent vRhyme
  4. optional create a conda environment (see examples below)
  5. optional activate conda environment if you made one
  6. pip install . ← NOTE: don't forget the dot (pip install [dot])

Installing with pip is optional but suggested. Using pip will collect Python dependencies and add vRhyme to your system PATH. Note that vRhyme.egg-info and build should be created after the pip install. Without pip, vRhyme can still be executed directly from the git clone, just ensure executable permissions (cd vRhyme/; chmod +x vRhyme scripts/*.py aux/*.py). The conda environment is also optional but can be useful for downloading and managing program dependencies.

Example Conda Environments

  • with pip install
  • conda create -c bioconda -n vRhyme python=3 samtools mash mummer mmseqs2 prodigal bowtie2 bwa
  • conda activate vRhyme

  • directly from git clone

  • conda create -c bioconda -n vRhyme python=3 networkx pandas numpy numba scikit-learn pysam samtools mash mummer mmseqs2 prodigal bowtie2 bwa
  • conda activate vRhyme

Test the Installation

Test and validate the installed dependencies for vRhyme. In these tests you're looking for Success statements. All Python dependencies must be Success. Please update them if prompted. Both machine learning models must be Success. For program dependencies, only Mmseqs is required to be Success, the others are optional depending on vRhyme usage. If you use any coverage input beside -c then Samtools is required to be Success. If you skip vRhyme -p or -g flags then Prodigal must be Success. If you plan to use vRhyme's dereplication function (see --method) then Mash and Nucmer must be Success. If you input reads (-r/-v/-u) then Bowtie2 and/or BWA (see --aligner) must be Success.

  • with pip install
    test_vRhyme.py

  • directly from git clone
    aux/test_vRhyme.py

Print the main vRhyme help page:
vRhyme -h


Requirements

Program Dependencies

Please ensure the following programs are installed and in your machine's PATH. Note: most downloads will automatically place these programs in your PATH. See each program's source website for installation guides.

Required
  1. Python3 (version >= 3.6)
  2. Mmseqs2
  3. Samtools
Optional (depends on usage)
  1. Prodigal
  2. Mash
  3. Nucmer
  4. Bowtie2
  5. BWA

Python3 Dependencies

There are several Python3 dependencies that must be installed as well. You may already have most of these installed. See each package's source website for installation guides. Versions are important.

  1. Pandas (version >= 1.0.0)
  2. Numpy (version >= 1.17.0)
  3. Scikit-learn (version >= 0.23.0)
  4. Numba (version >= 0.50.0)
  5. PySam (version >= 0.15.0)
  6. NetworkX (version >= 2.0)

Running vRhyme

Test run examples

Test out vRhyme on the provided example dataset. NOTE: If you choose to not install with pip, the vRhyme executable is within the vRhyme/ subdirectory.

cd examples/

minimal coverage table input example

vRhyme -i example_scaffolds.fasta -c example_coverage_values.tsv -t 1

full coverage table input example

vRhyme -i example_scaffolds.fasta -o vRhyme_example_results_coverage-table/ -c example_coverage_values.tsv -p example_scaffolds.prodigal.faa -g example_scaffolds.prodigal.ffn -t 2

Quick run examples

minimum input example with bam files

vRhyme -i fasta -b bam_folder/*.bam

minimum input example with a coverage file

vRhyme -i fasta -c coverage_file.tsv

full BAM input example

vRhyme -i fasta -g genes -p proteins -b bam_folder/*.bam -t threads -o output_folder/

reads input example with dereplication

vRhyme -i fasta -g genes -p proteins -r paired_reads_folder/*.fastq -t threads -o output_folder --method longest

only use dereplicate function

vRhyme -i input_fasta -t threads -o output_folder/ --derep_only --method longest


Output Explanations

Useful outputs

  • log_vRhyme_(-i).log: main log file
  • vRhyme_best_bins.#.membership.tsv: scaffold membership of best bins
  • vRhyme_best_bins.#.summary.tsv: summary stats of best bins
  • vRhyme_best_bins_fasta/: folder containing fasta files for best bins

Other outputs

  • log_vRhyme_paired_reads.log: log of which reads files were paired together with -r. Exists with -r.
  • (-i).prodigal.faa/(-i).prodigal.ffn: proteins/genes predicted by Prodigal for the length-filtered input sequences. Exists in the absense of -p/-g.
  • (-i).circular.tsv: list of input scaffolds that are circular (i.e., complete viral genomes). The terminal repeat (TR) type can be direct (DTR) or inverted (ITR). vRhyme will check for repeat lengths of at least 20bp and up to 2 mismatches; turn off with --keep_circ. The auxiliary script aux/flag_circular.py performs the same function and allows for modified parameters.
  • vRhyme_machine_distances.tsv: raw distance calculation values used as input for the machine learning model(s)
  • vRhyme_coverage_files: folder containing one file per sample with coverage values per scaffold. The row names can be found in vRhyme_names.txt. vRhyme_coverage_values.tsv is the combination of all sample coverages and is equivalent to -c. Does not exist with -c.
  • vRhyme_alternate_bins: folder containing membership and summary files for each alternative binning iteration. vRhyme_bin_scoring.tsv contains the scoring information and rationale for the best iteration chosen as the best bins.
  • vRhyme_bam_files: contains vRhyme-generated BAM files. Exists with -r/-u/-v/-s. Only sorted BAM files will exist unless --keep_bam.
  • vRhyme_sam_files: contains vRhyme-generated SAM files. Exists with -r/-u/-v and --keep_sam.

Hierarchy

  • '>' : folder
  • '-' : file
  • '*' : indicates file/folder may not exist
> main output folder (-o)
    - log_vRhyme_(-i).log
    - log_vRhyme_paired_reads.log *
    - (-i).prodigal.faa *
    - (-i).prodigal.ffn *
    - (-i).circular.tsv
    - vRhyme_best_bins.#.membership.tsv
    - vRhyme_best_bins.#.summary.tsv
    - vRhyme_machine_distances.tsv
    > vRhyme_coverage_files *
        - (sample).coverage.tsv
        - vRhyme_coverage_values.tsv
        - vRhyme_names.txt
    > vRhyme_alternate_bins
        - #.membership.tsv
        - #.summary.tsv
        - vRhyme_bin_scoring.tsv
    > vRhyme_best_bins_fasta
        - vRhyme_bin_#.faa
        - vRhyme_bin_#.fasta
        - vRhyme_bin_#.ffn
    > vRhyme_bam_files *
        - (sample).sorted.bam
        - (sample).bam *
        - (sample).bam.bai *
    > vRhyme_sam_files *
        - (sample).sam

Interpreting vRhyme bins/vMAGs

The following bullet points are guidelines on interpreting binning results from vRhyme. Please note that this list is not exhaustive. For examples and data, please see analyses done in the vRhyme publication.

  • Use protein redundancy as a metric of contamination
  • Contamination: a bin containing sequences from multiple viral genomes
  • Viruses do not encode universal single copy genes and therefore identifying contaminated vMAGs is difficult. For bacteria and archaea, a common practice is to estimate this with the tool CheckM. A useful tool for viruses is CheckV, though CheckV cannot identify if a vMAG contains sequences from multiple viral genomes, though it may identify if the genome is larger than expected. vRhyme implements a "protein redundancy" check for this purpose based on the concept that few viruses encode redundant proteins (e.g., >50% homology). Foremost, protein redundancy is used by vRhyme to construct bins and select the optimal iteration. However, this metric can be used post-binning to filter out bins that are likely contaminated. The vRhyme publication has a supplemental figure for this topic.
    • Bins unlikely to be contaminated will have 0-1 redundant proteins.
    • Bins with approximately 2-5 redundant proteins may not be contaminated but there are few such examples.
    • Bins with >6 redundant proteins are often contaminated. Notable exceptions include NCLDVs, which can have ~10 redundant proteins in an uncontaminated bin.
  • Expected bin sizes
  • Bin size: number of sequences (members)
  • Most vRhyme bins will be just 2-3 members
  • Although some viruses require binning to reconstruct fragmented genomes, there typically aren't many fragments to put back together. Most vRhyme bins will be just 2-3 members and few will exceed 4 members. However, a bin with >4 members should not automatically cause skepticism. In benchmarks, several examples of uncontaminated bins with >4 members were identified. Two examples detailed in the vRhyme publication are a 22-member Herelleviridae vMAG (115kb) and an 11-member NCLDV vMAG (1.6Mb). A megaphage (540kb) artificially split into 51 fragments generated a respective 51-member bin.
  • Unbinned sequences are still useful
  • Unbinned sequences: any input viral sequence that was not placed into a bin
  • Not all viral genomes will require binning and will remain as individual sequences. There is still high utility in unbinned viral sequences as they may accurately represent a viral population/genome. Any unbinned sequence or unused bin (e.g., individual members from bins with contamination) should be combined with the vMAGs for analyses.
  • Prophages can be difficult to bin
  • Prophage: temperate phage integrated into the hos

Core symbols most depended-on inside this repo

codon_counter
called by 6
vRhyme/scripts/codon_stuff_threader.py
kmer_nt_counter
called by 4
vRhyme/scripts/kmer_nt_stuff_threader.py
dist_compare
called by 4
vRhyme/scripts/distance_stuff.py
count_check
called by 4
vRhyme/scripts/derep_stuff.py
quick_stats
called by 4
vRhyme/scripts/coverage_stuff_threader.py
denomer_generate
called by 3
vRhyme/scripts/kmer_nt_stuff_threader.py
array_generate
called by 3
vRhyme/scripts/kmer_nt_stuff_threader.py
subtraction_math
called by 3
vRhyme/scripts/distance_stuff.py

Shape

Function 107
Method 35
Class 8

Languages

Python100%

Modules by API surface

vRhyme/aux/flag_circular.py19 symbols
vRhyme/scripts/derep_stuff.py17 symbols
vRhyme/scripts/read_nt_stuff.py14 symbols
vRhyme/scripts/kmer_nt_stuff_threader.py8 symbols
vRhyme/scripts/kmer_nt_stuff.py8 symbols
vRhyme/scripts/extract_stuff.py7 symbols
vRhyme/aux/bin_coverage.py6 symbols
vRhyme/aux/alternative_bins_to_fasta.py6 symbols
vRhyme/scripts/machine_stuff.py5 symbols
vRhyme/scripts/distance_stuff.py5 symbols
vRhyme/scripts/codon_stuff_threader.py5 symbols
vRhyme/scripts/codon_stuff.py5 symbols

For agents

$ claude mcp add vRhyme \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact