简体中文 English

Sub-models

Stopped updating (due to download and upload speed): Vocoder & HiddenUnitBert

Latest repository link : HuggingFace

Notes on exporting onnx sub models: - HuBert: input_names should be ["source"], output_names should be ["embed"], dynamic_axes should be {"source":[0,2],} - hifigan for Diffusion model: input_names should be ["c", "f0"], output_names should be ["audio"], dynamic_axes should be {"c":[0,1], "f0":[0,1],} - hifigan for Tacotron2: input_names should be ["x"], output_names should be ["audio"], dynamic_axes should be {"x":[0,1],}

User Agreement

• Please indicate the project repository when citing this project. This project is temporarily uncompilable (due to the use of an interface library that is not open source)

• When using this project for secondary creation, please indicate the repository of this project or the author's bilibili channel link: https://space.bilibili.com/108592413

By using this project you must agree to these following terms:

• 1、 You must bear all consequences arising from the use of the program at your own risk.
• 2、 You may not sell the program and its affiliated sub-models, and you will be responsible for all consequences resulting from such sale.
• 3、When using the program, you must consciously abide by the local laws and regulations, you must not use MoeSS to engage in illegal activities, if you engage in illegal activities, you will be responsible for all the consequences.
• 4、It is forbidden to use it for any commercial games, low quality games and Galgame production, except free high quality game production and mod production for other games.
• 5、It is forbidden to use the project and its derivatives as well as the released models to make “digital junk” (i.e. most of the game content like artworks are generated by ai).
• 6、Prohibited to use for political-related purposes, the consequences caused by the generation of political-related content shall be borne by you.

Q & A.

Q: Will the project charge in the future?

A: The project is permanently open source and free. If there is a paid version of this software elsewhere, please report it immediately to your shopping site and do not buy it, it is permanently free. If you would like to support Shirakana by making a donation, you can go to https://afdian.net/a/NaruseMioShirakana for more information.

Q: Do you offer a paid model training service?

A: In principle we do not offer this service. Training TTS models is relatively simple and there is no need to spend money on it, just follow the online tutorials step by step. In addition we offer a free Onnx conversion service.

Q: How do we determine if a piece is "digital junk"?

A: 1. Originality. The percentage of your own stuff in the overall project (for AI, creations using models trained entirely independently by you belong to you; work generated using someone else's model belongs to someone else). Aspects covered include, but are not limited to, programming, artwork, audio, designing, etc. For example, a game that uses a template from an engine's market such as Unity is "digital junk".

2. Developer attitude. The attitude of the author is whether he or she is trying to make a profit or simply to satisfy vanity. For example, if the game is promoted with exaggerated adjectives such as "the first" or "the best" to attract attention, but turns out to be very bad or mediocre, and the author clearly has no intention of making the game properly, this type of work is "digital junk".

3. We oppose any commercial use of AI models trained from unlicensed datasets.

Q: Technical support?

A: If it can be established that what you are doing is not “digital junk” and is also legally compliant and not heavily political, I will provide some technical support where I can.

Moe Speech Synthesis

A fully C++ Speech Synthesis UI based on various open source TTS, VC and SVS projects

Supported projects： - DeepLearningExamples - VITS - SoVits - DiffSvc - DiffSinger

The image resource used is derived from： - SummerPockets

目前仅支持Windows

Usage:

1、Download the software package in the release and unzip it.

2. Download the appropriate sub-models or additional modules from the [Vocoder & HiddenUnitBert] repository above and place them in the appropriate folders, the correspondence between the sub-models and the project will be mentioned below.

3. Place the model in the "Mods" folder and select the model from the model selection module at the top left, for the standard model structure please refer to "Supported Projects" below

4, enter the text to be converted in the input box below, click on "Enable Plugin" to execute the text Cleaner, and change the line to the batch conversion clause symbols (SoVits/DiffSvc need to enter the audio path, DiffSinger need to enter the path of the ds or json project file)

5, click on "start synthesis" to start synthesizing the voice,then wait for the progress to complete.When the audio is ready,you can preview the audio in the player at the top right,and you can also save the audio file directly at the top right of the interface.

6、It can also be run from the command line: (version 1.X only)
Shell: & '. \xxx.exe' "ModDir" "InputText." "outputDir" "Symbol"
CMD: "xxx.exe" "ModDir" "InputText." "outputDir" "Symbol"
where ModDir is the "model path\\model name" e.g. "Mods\\Shiroha\\\Shiroha"
InputText is the text to be converted (only spaces, commas and letters are supported)
outputDir is the output file name (not the path, but the file name, no need to add suffixes)
See below for Symbol relevance.
The output file is in the "tmpDir" folder by default.

Model adaption:

• The software standardises the model reading module, models are saved in a sub-folder under the Mods folder. A json file like "**.json" is a model configuration file, which is used to declare the model path and its display name.You need to convert the model to an Onnx model before you can use it. The codes used to convert these models can be found in the repositories on my Github page.

General configuration (required, otherwise the software will not recognise the model).

• Folder: The name of the folder where the model is stored
• Name: the name of the model to be displayed in the UI
• Type: type of the model(see below)
• Rate: the sampling rate (must be exactly the same as the rate you set during training)

Tacotron2：

{
    "Folder" : "Atri",
    "Name" : "atri-Tacotron2",
    "Type" : "Tacotron2",
    "Rate" : 22050,
    "Symbol" : "_-!'(),.:;? ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz",
    "Cleaner" : "JapaneseCleaner",
    "Hifigan": "hifigan"
}
//Symbol：Symbol of the model,If you don't know what it is, you are advised to check the TTS information on the internet.This field must be filled in the Tacotron2 model's configuration file.
//Cleaner：The name of the plugin,can be left blank, but if it is filled in, the corresponding CleanerDll must be placed in the Cleaner folder, if the Dll does not exist or if there is an internal error in the Dll, it will report an error when loading the model
//Hifigan：Hifigan model name, required and must be placed in the "hifigan" folder for hifigan model downloaded from the sub-model repository

Vits：

{
    "Folder" : "SummerPockets",
    "Name" : "SummerPocketsReflectionBlue",
    "Type" : "Vits",
    "Rate" : 22050,
    "Symbol" : "_,.!?-~…AEINOQUabdefghijkmnoprstuvwyzʃʧʦ↓↑ ",
    "Cleaner" : "JapaneseCleaner",
    "Characters" : ["鳴瀬しろは","空門蒼","鷹原うみ","紬ヴェンダース","神山識","水織静久","野村美希","久島鴎","岬鏡子"]
}
//Symbol：Symbol of the model,If you don't know what it is, you are advised to check the TTS information on the internet.This field must be filled in the VITS model's configuration file.
//Cleaner：The name of the plugin,can be left blank, but if it is filled in, the corresponding CleanerDll must be placed in the Cleaner folder, if the Dll does not exist or if there is an internal error in the Dll, it will report an error when loading the model
//Characters：For multi-speaker model this must be filled in as a list of your speakers' names, for single-speaker model it can be left out

Pits：

{
    "Folder" : "SummerPockets",
    "Name" : "SummerPocketsReflectionBlue",
    "Type" : "Pits",
    "Rate" : 22050,
    "Symbol" : "_,.!?-~…AEINOQUabdefghijkmnoprstuvwyzʃʧʦ↓↑ ",
    "Cleaner" : "JapaneseCleaner",
    "Characters" : ["鳴瀬しろは","空門蒼","鷹原うみ","紬ヴェンダース","神山識","水織静久","野村美希","久島鴎","岬鏡子"]
}
//Symbol：Symbol of the model,If you don't know what it is, you are advised to check the TTS information on the internet.This field must be filled in the VITS model's configuration file.
//Cleaner：The name of the plugin,can be left blank, but if it is filled in, the corresponding CleanerDll must be placed in the Cleaner folder, if the Dll does not exist or if there is an internal error in the Dll, it will report an error when loading the model
//Characters：For multi-speaker model this must be filled in as a list of your speakers' names, for single-speaker model it can be left out

SoVits_3.0_32k：

{
    "Folder" : "NyaruTaffySo",
    "Name" : "NyaruTaffy-SoVits",
    "Type" : "SoVits",
    "Rate" : 32000,
    "Hop" : 320,
    "Cleaner" : "",
    "Hubert": "hubert",
    "SoVits3": true,
    "Characters" : ["Taffy","Nyaru"]
}
//Hop：HopLength of the model, if you don't know what it is you are advised to look up the information on the internet. This must be filled in the configuration file of the SoVits model.（The value must be the one you set during training and can be seen in the configuration file you used to train the model）
//Cleaner：The name of the plugin,can be left blank, but if it is filled in, the corresponding CleanerDll must be placed in the Cleaner folder, if the Dll does not exist or if there is an internal error in the Dll, it will report an error when loading the model
//Hubert：Hubert model name, required and must be placed in the "Hubert" folder for Hubert model downloaded from the sub-model repository
//Characters：For multi-speaker model this must be filled in as a list of your speakers' names, for single-speaker model it can be left out

SoVits_3.0_48k：

{
    "Folder" : "NyaruTaffySo",
    "Name" : "NyaruTaffy-SoVits",
    "Type" : "SoVits",
    "Rate" : 48000,
    "Hop" : 320,
    "Cleaner" : "",
    "Hubert": "hubert",
    "SoVits3": true,
    "Characters" : ["Taffy","Nyaru"]
}
//Hop：HopLength of the model, if you don't know what it is you are advised to look up the information on the internet. This must be filled in the configuration file of the SoVits model.（The value must be the one you set during training and can be seen in the configuration file you used to train the model）
//Cleaner：The name of the plugin,can be left blank, but if it is filled in, the corresponding CleanerDll must be placed in the Cleaner folder, if the Dll does not exist or if there is an internal error in the Dll, it will report an error when loading the model
//Hubert：Hubert model name, required and must be placed in the "Hubert" folder for Hubert model downloaded from the sub-model repository
//Characters：For multi-speaker model this must be filled in as a list of your speakers' names, for single-speaker model it can be left out

SoVits_4.0：

{
    "Folder" : "NyaruTaffySo",
    "Name" : "NyaruTaffy-SoVits",
    "Type" : "SoVits",
    "Rate" : 44100,
    "Hop" : 512,
    "Cleaner" : "",
    "Hubert": "hubert4.0",
    "SoVits4": true,
    "Characters" : ["Taffy","Nyaru"]
}
//Hop：HopLength of the model, if you don't know what it is you are advised to look up the information on the internet. This must be filled in the configuration file of the SoVits model.（The value must be the one you set during training and can be seen in the configuration file you used to train the model）
//Cleaner：The name of the plugin,can be left blank, but if it is filled in, the corresponding CleanerDll must be placed in the Cleaner folder, if the Dll does not exist or if there is an internal error in the Dll, it will report an error when loading the model
//Hubert：Hubert model name, required and must be placed in the "Hubert" folder for Hubert models downloaded from the sub-model repository
//Characters：For multi-speaker model this must be filled in as a list of your speakers' names, for single-speaker model it can be left out

DiffSVC：

```jsonc { "Folder" : "DiffShiroha", "Name" : "白羽", "Type" : "DiffSvc", "Rate" : 44100, "Hop" : 512, "MelBins" : 128, "Cleaner" : "", "Hifigan": "nsf_hifigan", "Hubert": "hubert", "Characters" : [], "Pndm" : 100, "V2" : true } //Hop：HopLength of the model, if you don't know what it is you are advised to look up the information on the internet. This must be filled in the configuration file of the SoVits model.（The value must be the one you set during training and can be seen in the configuration file you used to train the model） //Melbins：Melbins of the model, if you don't know what it is you are advised to look up the information on the internet. This must be filled in the configuration file of the Diffsvc model.（The value must be the one you set during training and can be seen in the configuration file you used to train the model） //Cleaner：The name of the plugin,can be left blank, but if it is filled in, the corresponding CleanerDll must be placed in the Cleaner folder, if the Dll d