eeh
A tiny Japanese voice that fits in a browser tab. eeh is a Japanese text-to-speech model with two voices and 4.5 million parameters, waveform generator included. It reads kanji, numbers and Tokyo pitch accent, and runs on a CPU or fully inside a browser, with nothing sent anywhere.
There is no neural vocoder. A transformer reads OpenJTalk's phonemes with their pitch-accent tones, predicts how long each one lasts, then predicts pitch, loudness and the settings of a small source-filter synthesizer, which makes the waveform from a pulse train and noise. That is why it is small and fast. It was trained from scratch on 18.6 hours of speech by Irodori-TTS v4.1 Small Yomi.
| Parameters | 4,534,219 |
| Voices | F1 (female, late 20s, calm and clear), M1 (male, early 30s, warm), Tokyo Japanese |
| Output | 24 kHz mono |
| Input | Japanese text (kanji, kana, numbers), read with OpenJTalk |
| Speed | 120x real time on one CPU thread, 280x on four (Apple M4 Max); about 50x in a browser tab. First sound after 9 ms (median) on the CPU, 30â50 ms in a browser |
| Pretrained weights in the model | None |
| License | Apache-2.0 |
Samples
| Text | F1 | M1 |
|---|---|---|
| ăăă«ăĄăŻïŒăăăŻăăăŒăăăœăłăłă§ăćăć°ăăȘéłćٰćæăąăă«ă§ăă | ||
| äŒè°ăŻäžæććæ„ăźććŸäžæćă«ć€ăăŁăăźă§ăăăšććäșćă»ă©ăăăŸăă | ||
| æŹćœă«ăăăȘăăźèȘçæ„ăćżăăăšæăŁăăźïŒ | ||
| ćœŒć„łăŻæ”·ăèŠăăăšăăȘăăŁăăăă°ăăăźéăäœăèšăăă«æłąæăĄéă«ç«ăŁăŠăăă | ||
| æăć „ăăŸăăăăććăšăéŁç”Ąć ăăăĄăă«ăèšć „ăăă ăăŸăăă |
Files
| Path | Contents |
|---|---|
eeh.safetensors |
Weights, float32 (18.2 MB) |
eeh-dict.bin |
The Japanese dictionary the front end reads (36 MB) |
python/ |
Python wheels for macOS (arm64) and Linux (aarch64, x86_64) |
bin/<platform>/eeh |
Command-line program |
lib/<platform>/, include/eeh.h |
C library and header, for any language with a C FFI |
demo/ |
The browser demo: the whole engine compiled to WebAssembly, the weights and the dictionary |
samples/ |
The audio on this page |
eval/report.json, eval/metrics.jsonl |
Evaluation report and the training log |
LICENSE |
Apache-2.0 |
Usage
eeh runs on its own engine, written in Rust: a Japanese front end that reads text exactly as OpenJTalk does (identical on more than 500,000 test texts) and the model. The same engine is behind the Python package, the command-line program, the C library and the browser demo.
hf download ctrltokyo/eeh-1 --local-dir eeh
cd eeh
Python
pip install python/eeh-0.1.0-py3-none-<your platform>.whl
import eeh, soundfile
synth = eeh.Synthesizer("eeh.safetensors", "eeh-dict.bin")
audio = synth.speak("ăăă«ăĄăŻă仿„ăŻăă怩æ°ă§ăăă", voice="F1") # float32 array, 24 kHz
soundfile.write("hello.wav", audio, synth.sample_rate)
speak takes voice (F1 or M1), speed (1.0), pitch_shift (0; 0.058
is about one semitone), pitch_range (1.0) and seed. speak_each(text, callback) hands over each sentence as soon as it is made, for streaming. The
package has no dependencies; it returns a NumPy array when NumPy is installed.
The synthesizer can be shared between threads.
Command line
chmod +x bin/macos-arm64/eeh # the Hub does not keep the executable bit
bin/macos-arm64/eeh speak --weights eeh.safetensors --dictionary eeh-dict.bin \
--voice M1 --text "ăăă«ăĄăŻă仿„ăŻăă怩æ°ă§ăăă" --out hello.wav
C and other languages
lib/<platform>/libeeh_ffi with include/eeh.h: eeh_synthesizer_new takes
the weights and dictionary as bytes, eeh_speak returns the audio and
eeh_speak_each streams it sentence by sentence. Errors come back as status
codes, never as crashes. Go, Node, Java, Ruby and Swift can call it through
their C FFI (all five are tested).
Text of any length works. It is split into sentences at ăïŒïŒ and read one sentence at a time; a quote followed by ăš or ăŁăŠ stays with its sentence, and a sentence too long for one call is cut after ă. A text without a final mark gets ă. The audio never peaks above 0.99.
In a browser
python -m http.server --directory demo 8000
Then open http://localhost:8000. The engine runs in a Web Worker and plays
long text sentence by sentence as it is made. The first visit downloads about
26 MB (gzip): 17 MB of weights, 9 MB of dictionary and 0.4 MB of program, all
kept by the browser afterwards.
How it works
Front end. eeh's own Rust front end reads the text the way OpenJTalk does with the NAIST Japanese Dictionary: it splits it into words, picks each kanji's reading, expands numbers, joins words into accent phrases and assigns their accent types. It turns the result into tokens:
- one token per phoneme (
k,sh,a,Nfor ă,clfor ăŁ); a long vowel is two tokens, one per mora; - devoiced vowels (the u of ă§ă, the i of ăă) are their own tokens
A I U E O; <sp>marks a boundary between accent phrases, and punctuation is kept;- each token carries the Tokyo pitch-accent tone of its mora, low or high, computed from the phrase's accent type and the mora's position.
It is fast: the 8,650 training sentences, 11.8 hours of speech, are read in 0.29 s on one CPU thread (0.03 ms per sentence, about 150,000x real time), 5.5x faster than pyopenjtalk. Its dictionary loads in 3 ms.
Model. A transformer encoder (4 blocks) reads the tokens and tones, a predictor gives each token a length in frames, and two frame-level transformers (2 and 5 blocks) predict pitch, voicing and loudness and then the settings of the synthesizer: a band-limited pulse train at the predicted pitch plus noise, each shaped by a filter predicted every frame (256 samples, 10.7 ms). All blocks use 192 channels, 4 heads and rotary position embeddings.
| Component | Parameters |
|---|---|
| Embeddings (token, tone, voice) | 13,632 |
| Text encoder | 1,588,864 |
| Duration and pause predictor | 123,906 |
| Frame transformer A, pitch/voicing/energy head | 795,971 |
| Frame transformer B | 1,985,984 |
| Source-filter waveform head | 25,862 |
| Total | 4,534,219 |
Training data
The speech is synthetic. Two voices were designed once from Japanese
descriptions with Irodori-TTS v4.1 Small (èœăĄçăă20代ćŸćăźć„łæ§ăæçă§èȘç¶ăȘæ±äșŹăźè©±ăèšèă
and 30代ććăźç·æ§ăæž©ăăèŠȘăăżăăăăèȘç¶ăȘæ±äșŹăźè©±ăèšèă), then every
sentence was read by Irodori-TTS v4.1 Small Yomi
(MIT) cloning those two references. No recording of a real speaker is involved.
On the 300 test sentences in both voices, the teacher's speech has 1.7% kana reading error, 0.70 pitch-accent agreement and a UTMOS of 3.38.
The 8,650 sentences (8,000 train, 350 validation, 300 test, split by sentence) were chosen to cover as many morae and mora pairs as possible; 25% are questions and 10% exclamations.
Every clip was screened before training:
- Reading. kana-whisper transcribed it to katakana, compared with OpenJTalk's reading; more than 10% kana error was rejected.
- Alignment. Phoneme durations come from forced alignment with
facebook/wav2vec2-lv-60-espeak-cv-ft, used only to label the data. Its labels were recalibrated on the teacher's audio, and clips it aligned with low confidence were rejected. - Speaking rate and length.
| Clips | Hours | |
|---|---|---|
| Generated (sentences x voices) | 17,300 | |
| Passed screening | 15,684 | |
| Used for training (F1 7,301, M1 6,783) | 14,084 | 18.6 |
| Validation / test | 614 / 527 | 1.3 / 1.0 |
Training
| Steps | 104,000 |
| Batches | about 9,000 frames (96 s of audio) per step |
| Losses | Multi-resolution spectral losses; adversarial loss with multi-period and multi-resolution discriminators after step 8,000 |
| Hardware | One NVIDIA GB10 (Lenovo ThinkStation PGX) for training; an Apple M4 Max MacBook generated and scored half of the corpus |
| Time | 11.5 hours of training; 5.7 hours to generate the corpus on both machines |
Two changes to the training step made the adversarial phase twice as fast without changing the gradients: the discriminators' gradient at the waveform is computed once instead of twice, and their convolutions run in bfloat16 with autotuned kernels, on batches padded to a few fixed shapes.
Evaluation
Speech from the final checkpoint was scored the same way as the teacher's recordings of the same sentences: kana-whisper transcribes it to katakana (reading error against OpenJTalk's reading), the pitch tracker checks each expected rise and fall of the pitch accent, and UTMOS rates naturalness.
| Sentences | Clips | Reading error, eeh | Reading error, teacher | Pitch-accent agreement, eeh | Pitch-accent agreement, teacher | UTMOS, eeh | UTMOS, teacher |
|---|---|---|---|---|---|---|---|
| Test | 300 | 0.8% | 1.2% | 0.70 | 0.69 | 2.94 | 3.38 |
| Validation | 300 | 1.0% | 1.7% | 0.71 | 0.68 | 2.94 | 3.34 |
| New sentences, not in the dataset | 16 | 0.0% | 0.69 | 2.88 |
eeh reads and accents as well as the teacher it learned from; it sounds less natural (UTMOS about 0.4 lower), which is the cost of the small synthesizer. No listening test has been run.
Limitations
- Japanese only, two fixed voices, no voice cloning.
- Readings come from OpenJTalk's dictionary. Rare kanji readings, names and new words can be read wrong, and the accent of compound phrases is predicted by rules. Whatever OpenJTalk gets wrong, eeh says wrong.
- Latin letters are spelled out one by one (
eehis read ă€ăŒă»ă€ăŒă»ăšă€ă). - The same text is always read the same way.
- The training data is synthetic, so eeh cannot sound more natural than the teacher; the small source-filter synthesizer sounds more buzzy than it.
License
Apache-2.0 for the weights and code.
Citation
@software{nicholson2026eeh,
author = {Nicholson, Alexander},
title = {eeh: a 4.5M-parameter two-voice Japanese text-to-speech model},
year = {2026},
url = {https://huggingface.co/ctrltokyo/eeh-1}
}
Acknowledgements
Inspired by tiny-voca, trained on speech from Irodori-TTS, and reading Japanese with OpenJTalk.