[!IMPORTANT]
If you're sharing the translated result publicly and no experienced human translator participated in a throughout translating or proofreading, please mark it as machine translation somewhere clear to see.

BallonTranslator

简体中文 | English | Русский | 日本語 | Español | Français | pt-BR | 한국어 | Indonesia | Tiếng Việt

Yet another computer-aided comic/manga translation tool powered by deep learning.

preview

Features

• Fully automated translation
• Support automatic text-detection, recognition, removal, and translation. Overall performance is dependent upon these modules.
• Typesetting is based on the formatting estimation of the original text.
• Works decently with manga and comics.

• Improved manga->English, English->Chinese typesetting (based on the extraction of balloon regions.).

• Image editing

• Support mask editing & inpainting (something like spot healing brush tool in PS)

• Adapted to images with extreme aspect ratio such as webtoons

• Text editing

• Support rich text formatting and text style presets, translated texts can be edited interactively.
• Support search & replace
• Support export/import to/from word documents

Installation

On Windows

Method A (One-Click Local Environment Setup, requires PowerShell): The script installs BallonsTranslator in the directory where you run it:

irm https://raw.githubusercontent.com/dmMaze/BallonsTranslator/dev/scripts/install.ps1 | iex

Or run the following command in the Command Prompt (cmd.exe):

powershell -NoProfile -ExecutionPolicy Bypass -Command "irm https://raw.githubusercontent.com/dmMaze/BallonsTranslator/dev/scripts/install.ps1 | iex"

Method B (Download Pre-configured Package): Download Ballonstranslator_win_minium.zip from GitHub Releases, extract it, and double-click launch_win.bat to launch the application.

These methods do not support Windows 7; Windows 7 users must install Python 3.8 manually and run from source.

If you see errors involving msvcp140.dll, c10.dll, or [WinError 1114], install or update the Microsoft Visual C++ Redistributable x64 (Visual Studio 2015-2022; official download notes).

macOS / Linux

The script installs BallonsTranslator in the directory where you run it:

curl -fLO https://raw.githubusercontent.com/dmMaze/BallonsTranslator/dev/scripts/install.sh && chmod +x install.sh && ./install.sh

If curl is not available, download the script with wget -O ... instead. The app launches automatically after installation; later, use cd BallonsTranslator && ./launch.sh to start it again.

The app checks core dependencies at startup. When you select a module that needs extra libraries, the app will prompt you to install the missing optional dependencies (you can also enable automatic installation in Settings).

Usage

It is recommended to run the program in a terminal in case it crashed and left no information, see the following gif. - The first time you run the application, please select the translator and set the source and target languages by clicking the settings icon. - Open a folder containing images of a comic (manga/manhua/manhwa) that need translation by clicking the folder icon. - Click the Run button and wait for the process to complete.

The font formats such as font size and color are determined by the program automatically in this process, you can predetermine those formats by change corresponding options from "decide by program" to "use global setting" in the config panel->Typesetting. (global settings are those formats shown by the right font format panel when you are not editing any textblock in the scene)

Image Editing

Inpaint Tool

Image Editing Mode, Inpainting Tool

rect tool

Rect Tool

To 'erase' unwanted inpainted results, use the inpainting tool or rect tool with your right button pressed.
The result depends on how accurately the algorithm ("method 1" and "method 2" in the gif) extracts the text mask. It could perform worse on complex text & background.

Text editing

Text Editing Mode

Batch Text Formatting & Auto Layout

OCR & Translate Selected Area

Shortcuts

• A/D or pageUp/Down to turn the page
• Ctrl+Z, Ctrl+Shift+Z to undo/redo most operations. (note the undo stack will be cleared after you turn the page)
• T to text-editting mode (or the "T" button on the bottom toolbar).
• W to activate text block creating mode, then drag the mouse on the canvas with the right button clicked to add a new text block. (see the text editing gif)
• P to image-editting mode.
• In the image editing mode, use the slider on the right bottom to control the original image transparency.
• Disable or enable any automatic modules via titlebar->run, run with all modules disabled will re-letter and re-render all text according to corresponding settings.
• Set parameters of automatic modules in the config panel.
• Ctrl++/Ctrl+- (Also Ctrl+Shift+=) to resize image.
• Ctrl+G/Ctrl+F to search globally/in current page.
• 0-9 to adjust opacity of text layer
• For text editing: bold - Ctrl+B, underline - Ctrl+U, Italics - Ctrl+I
• Set text shadow and transparency in the text style panel -> Effect.
• Alt+Arrow Keys or Alt+WASD (pageDown or pageUp while in text editing mode) to switch between text blocks.

Headless mode (Run without GUI)

python -m ballontranslator --headless --exec_dirs "[DIR_1],[DIR_2]..."

Note the configuration (source language, target language, inpaint model, etc) will load from config/config.json.
If the rendered font size is not right, specify logical DPI manually via --ldpi, typical values are 96 and 72.

Automation modules

This project is heavily dependent upon manga-image-translator, online service and model training is not cheap, please consider to donate the project:
- Ko-fi: https://ko-fi.com/voilelabs - Patreon: https://www.patreon.com/voilelabs - 爱发电: https://afdian.net/@voilelabs

Sugoi translator is created by mingshiba.

Text detection

• Support English and Japanese text detection, training code and more details can be found at comic-text-detector

• Support using text detection from Starriver Cloud (Tuanzi Manga OCR). Username and password need to be filled in, and automatic login will be performed each time the program is launched.

• For detailed instructions, see Tuanzi OCR Instructions: (Chinese & Brazilian Portuguese only)

• YSGDetector models are trained by lhj5426, these models would filter out onomatopoeia in CGs/Manga, download checkpoints from YSGYoloDetector and put into data/models.

OCR

• All mit* models are from manga-image-translator, support English, Japanese and Korean recognition and text color extraction.
• manga_ocr is from kha-white, text recognition for Japanese, with the main focus being Japanese manga.
• PaddleOCRVLManga finetuned on Japanese manga
• Support using OCR from Starriver Cloud (Tuanzi Manga OCR). Username and password need to be filled in, and automatic login will be performed each time the program is launched.
• The current implementation uses OCR on each textblock individually, resulting in slower speed and no significant improvement in accuracy. It is not recommended. If needed, please use the Tuanzi Detector instead.
• When using the Tuanzi Detector for text detection, it is recommended to set OCR to none_ocr to directly read the text, saving time and reducing the number of requests.
• For detailed instructions, see Tuanzi OCR Instructions: (Chinese & Brazilian Portuguese only)
• Added as an "optional" PaddleOCR module. In Debug mode you will see a message stating that it is not there. You can simply install it by following the instructions described there. If you don’t want to install the package yourself, just uncomment (remove the #) the lines with paddlepaddle(gpu) and paddleocr. Bet everything at your own peril andrisk. For me (bropines) and two testers, everything was installed fine, you may have an error. Write about it in issue and tag me.
• Added OneOCR. Local WINDOWS model taken from SnippingTOOL or Win.PHOTOS applications. To use it, you need to place the model and DLL files in the 'data/models/one-ocr' folder. Before running, it is better to throw the files at once. Read how to find and get DLL and model files here: https://github.com/dmMaze/BallonsTranslator/discussions/859#discussioncomment-12876757 . Thanks AuroraWright for the project OneOCR
• OCR setting: Font recognition. Download the Font Recognition Model (YuzuMarker.FontDetection) and place it in the data\models\YuzuMarker.FontDetection directory. The three required files are: data\models\YuzuMarker.FontDetection\font_dataset, data\models\YuzuMarker.FontDetection\name=4x-epoch=18-step=368676.ckpt, and data\font_demo_cache.bin Font names with a recognition confidence rate greater than 60% will be saved in the _detected_font_name field of the JSON file. Currently, no visual display is provided. When exporting LabelPlus txt using the script [scripts/BTjson_to_LPtxt.pyw], you can optionally include font and font size information for importing into other software (such as Photoshop/InDesign) for text embedding.

Inpainting

• AOT is from manga-image-translator.
• All lama* are finetuned using LaMa
• PatchMatch is an algorithm from PyPatchMatch, this program uses a modified version by me.

Translators

• You can find information about Translators modules here.

FAQ & Misc

• If your computer has an Nvidia GPU or Apple silicon, the program will enable hardware acceleration.
• Accelarate performance if you have a NVIDIA's CUDA or AMD's ROCm device as most modules uses PyTorch.
• Fonts are from your system's fonts.
• Thanks to bropines for the Russian localization.
• Added Export to photoshop JSX script by bropines.

To read the instructions, improve the code and just poke around to see how it works, you can go to scripts/export to photoshop -> install_manual.md.