• Midi-to-Singing-Voice-Conversion(M2SVC)

Vocal Synthesis Through MIDI and Vocal Transformation Using RVC: Exploration of Innovative Music Processing Methodologies

Contents

• Model Structure
• Prerequisites
• Installation
• Prepare_Datasets
• Usage
• To-do
• References

Structure

Prerequisites

• A Windows/Linux system with a minimum of 16GB RAM.
• A GPU with at least 12GB of VRAM.
• Python = 3.8
• Anaconda installed.
• PyTorch installed.
• CUDA 11.x installed.
• Musescore 3 installed.

Pytorch install command:

pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu117

CUDA 11.7 install

Musescore 3 install

Musescore 3 must be set as an environment variable.

Installation

1. Create an Anaconda environment:

conda create -n m2svc python=3.8

1. Activate the environment:

conda activate m2svc

1. Clone this repository to your local machine:

git clone https://github.com/ORI-Muchim/Midi-to-Singing-Voice-Conversion.git

1. Navigate to the cloned directory:

cd Midi-to-Singing-Voice-Conversion

1. Extract the RVC.zip file and move the folder to this directory.

2. Install the necessary dependencies:

pip install -r requirements.txt

Prepare_Datasets

Place the audio files as follows.

Midi-to-Singing-Voice-Conversion
├────datasets
│    └───kss
│        ├───4_5132.wav
│        ├───4_5133.wav
│        └───4_5134.wav
│
├────inputs
│    ├───cin.mid
│    ├───cin.txt
│    ├───shallow_base.wav
│    ├───shallow.mid
│    └───shallow.txt
│
├────midi2voice
│    ├───__init__.py
│    ├───__main__.py
│    ├───lyrics_tokenizer.py
│    ├───midi2xml.py
│    └───README.md
│
├────RVC
│    └───...
│
├────src
│    └───M2SVC_Flowgraph.png
│
├────final_vocal.wav
├────ko2kana.py
├────main.py
├────Readme.md
├────requirements.txt
├────voice.wav(Raw-Synthesized-Voice)
└────voice.xml

Please put the audio data of Voice Conversation through RVC in ./datasets and txt and mid files to be synthesized with midi2voice in ./inputs folder.

Usage

To start this tool, use the following command, replacing your respective values {language: ko, ja, en, zh} / {gender: female / male(Recommend: female)}:

python main.py {model_name} {text_file} {midi_file} {language} {gender} {bpm}

To-do

• Automatic Pitch Analyzer (UVR -> VOCAL (INST REMOVED) -> RMVPE -> MIDI) / Work In Process!
• Japanese Kanji to Katakana Cleaner / Done!

References

For more information, please refer to the following repositories: - mathigatti/midi2voice - CjangCjengh/vits - RVC-Project/Retrieval-based-Voice-Conversion-WebUI - Kyubyong/g2pK - kdrkdrkdr/ko2kana