stepfun-tts

StepFun Contextual TTS (default stepaudio-2.5-tts)

Generate Chinese / Japanese speech with StepFun's Contextual TTS — emotion and prosody go through natural-language description, not fixed labels. Default model is stepaudio-2.5-tts (what the bundled script uses): in a 2026-09-16 blind A/B on our own cases, 2.5 won the neutral / jiao / lively-girl pairs 3:2. Pick stepaudio-3-tts when the line is whisper or heavy inline-() prosody — v3 won exactly those two pairs; v3 also raises the instruction cap to 500 chars (2.5 is 200). ⚠️ Never synthesize cloned voices with v3 — v3 speech 接受复刻音色 ID 不报错但静默回退默认女声——根因 = 阶跃复刻链路整体停在 2.5 家族（复刻创建 API 只收 2.5/step-tts-2/step-tts-mini，v3 不在列；v3 合成侧复刻未发布）。两代创建的克隆在 v3 上全丢：step-tts-2 克隆 SIM 0.272、2.5 创建的新克隆 SIM 0.195（锚 0.773），2.5 同 ID 0.743/0.678。

Companion: for transcription with stepaudio-3-asr-max (the sibling model), use the stepfun-asr skill — they share an API key but live on different endpoints with different body shapes.

Why this skill exists — two non-obvious pitfalls that cost hours if you don't know them:

1. stepaudio-3-tts rejects voice_label (the step-tts-2 way) — verified on v3 2026-09-16: HTTP 400 voice_label is not supported for this model. Emotion/prosody goes through instruction (natural-language description, ≤500 chars on v3 — 200 was the 2.5 limit) and inline () parentheses inside the text itself.
2. Censorship behavior is model-version-specific — the 2.5-era trigger list (死 / 消失 / sensitive political terms → censorship_block) did not fire on v3 in a 2026-09-16 single-sample probe; treat censorship as present but re-verify per trigger before building rewrite maps. The 2.5-era options are in references/migration_from_v2.md.

Config and auth

API key lives in $STEPFUN_API_KEY (preferred) or ${CLAUDE_PLUGIN_DATA}/config.json (fallback for cross-session persistence). All bundled scripts try env first, then config.

First-time setup (one-liner):

mkdir -p "${CLAUDE_PLUGIN_DATA}" && cat > "${CLAUDE_PLUGIN_DATA}/config.json" <<EOF
{"api_key": "<paste key here>"}
EOF

If the user hasn't set a key, ask them to paste it (don't guess / don't use a placeholder). StepFun API keys are available at https://platform.stepfun.com/ → API Keys. Use a Normal key, not a Plan key (Plan keys are restricted to text models and silently fail on audio endpoints).

Common tasks — decision tree

User wants...	Script	Key detail
Synthesize 1–500 char Chinese with emotion	scripts/tts_generate.py	Use instruction for mood, () for inline prosody
Synthesize long text (500–1000 char)	scripts/tts_generate.py	1000 char is the hard cap; split at semantic boundaries above that
Batch-generate game/app voice lines	scripts/tts_generate.py --batch <jsonl>	Handle censorship_block fallback individually
A/B compare two TTS models	scripts/ab_compare.sh	Compares duration/size across two directories
Migrate from step-tts-2 / stepaudio-2.5-tts	see references/migration_from_v2.md	voice_label.emotion → instruction rewrite + 2.5-era censorship list

Starting points

• Synthesize a single line: Run python3 scripts/tts_generate.py --text "你好" --out /tmp/hello.mp3 --instruction "温暖的希望感". For fine-grained control read the "Contextual TTS" section below.
• From code: this script is the endpoint's wrapper in llm-registry — llmreg.wrapper_for("stepfun-tts").tts_generate.synthesize(api_key=…, text=…, model=…, extra={…}). extra is merged into the request body as-is; with {"timestamp": True, "return_url": True} the server answers a JSON envelope ({"data": {"url", "subtitles"}}) that comes back under json instead of audio_bytes (verified 2026-09-19). Parameters are not billed — send what you need. Direct /v1/audio/speech calls elsewhere are blocked by the llm-entry-guard hook.
• A full migration from step-tts-2 → Contextual TTS: read references/migration_from_v2.md end-to-end before touching code. It has the INSTRUCTION_MAP, the SKIP_CENSORED list pattern, and the output-directory-strategy for non-destructive A/B (written for 2.5; the migration mechanics are identical on v3).

Contextual TTS — beyond emotion labels

The headline feature of stepaudio-3-tts is that you stop mapping emotions to fixed tags and start describing what you want in natural language. Two layers:

Global context (instruction parameter) — sets the overall tone for the entire utterance. ≤500 chars on v3 (2.5 was 200; a 300-char instruction verified accepted on v3 2026-09-16). Think of it like giving stage direction to a voice actor.

instruction: "克制的悲伤，语气低沉柔弱，像快要消失一样"

Inline context (() parentheses inside input) —句内 directives. Parenthesised content is consumed as directions and is NOT read aloud. Use for precise control of pauses, breath, emphasis, or mid-sentence emotion shifts.

input: "(试探着问)你好吗？(开心地)太好了！(突然沉下来)不过...我快要消失了。"

Examples that worked in practice (verified 2026-04-23 on 2.5; all five re-verified on v3 2026-09-16, including these instruction and inline-prosody cases):

• instruction: "活泼俏皮，像是在撒娇，带点嘴硬" — visibly speeds up delivery vs neutral
• instruction: "耳语声，气声很重，几乎听不清" — produces audible whisper/breath
• input: "你好(停顿一下)我是蕾格(轻声)今天(加重)的天气真不错。" — inline directives all respected

What stepaudio-3-tts will NOT accept — voice_label parameter. Error on v3: voice_label is not supported for this model (2.5 said ...for v2 models). This is the #1 migration gotcha from step-tts-2.

Common error patterns (real errors, real fixes)

Error response	Actual cause	Fix
"voice_label is not supported for this model" (v3) / "...for v2 models" (2.5)	Sent voice_label to a Contextual TTS model	Remove voice_label; put the same intent into instruction as natural language
"The content you provided or machine outputted is blocked." type: censorship_block	Sensitive word (2.5-era: 死 / 消失 / etc.; v3 triggers unverified)	Rewrite the phrase OR fall back to step-tts-2 for that specific line (mixed-model is fine)
Silent audio truncation (input > 1000 chars)	Hard cap exceeded	Split at semantic boundaries; don't truncate mid-sentence

More in references/known_issues.md.

When to read references

• references/api_reference.md — exact request/response JSON for /v1/audio/speech, all fields, error responses. Read when writing raw HTTP calls instead of using the bundled scripts.
• references/migration_from_v2.md — complete playbook for moving a step-tts-2 project to Contextual TTS. Has the emotion→instruction rewrite table, the A/B directory strategy, decision checkpoints, and the 2026-04 speed/quality trade-off data (written against 2.5). Read before any migration work.
• references/known_issues.md — censorship patterns, TTS duration inflation, v2-family parameter naming gotcha, 1000-char hard cap. 2.5-era verification; re-verify on v3 before relying on a specific entry. Read when debugging anomalous output or evaluating whether to adopt.

Design invariants (don't break these)

1. Non-destructive A/B output — when regenerating a corpus with a new model, write to a parallel directory (voice/zh_v3/), never overwrite the production corpus. The migration playbook shows why.
2. Per-line censorship handling — if 2/29 lines get censorship_block, don't fail the batch. Log the skipped IDs, continue. Mixed-model fallback (step-tts-2 for the skipped 2) is normal.
3. Don't duplicate voice_label logic in new code — any new TTS code targeting stepaudio-3-tts should only use instruction + inline (). Do not write a branch that conditionally emits voice_label.

v3-specific facts (verified 2026-09-16)

• Official voices: 60+ via GET /v1/audio/system_voices?model=stepaudio-3-tts — the 2.5 list is fully inherited, plus new voices (e.g. English-named Lisa/Alfie, 上海话 shanghaifemale/shanghaimale).
• Cloned (复刻) voices: do NOT use stepaudio-3-tts — root cause nailed 2026-09-16: StepFun's whole cloning stack is still 2.5-family. The creation API (POST /v1/audio/voices) only accepts stepaudio-2.5-tts / step-tts-2 / step-tts-mini, and v3 /v1/audio/speech silently falls back to a default female voice for ANY cloned ID — SIM 0.272 (step-tts-2 clone) and 0.195 (fresh 2.5-created clone) vs the 0.773 anchor; the same IDs on 2.5 score 0.743/0.678. Synthesize clones with stepaudio-2.5-tts (best SIM) or step-tts-2.
• No WebSocket streaming for v3 (as of 2026-09-16): wss://api.stepfun.com/v1/realtime/audio?model=stepaudio-3-tts is rejected at handshake (404), while 2.5 still streams there. Need char-level subtitle timestamps on v3? Use REST timestamp:true + return_url:true (subtitles arrive in the response JSON data.subtitles[], char-level ms, accumulated absolute axis) — bare stream_format:"audio" cannot carry subtitles (server 400).

Pricing (verified 2026-09-16, volatile)

• stepaudio-3-tts synthesis: 2.5 元 / 万字符 (official model page, 2026-09-16 — cheaper than the 2.5-era ~5.8)
• Zero-shot voice cloning: 9.9 元 / 音色

Re-verify at https://platform.stepfun.com/docs/zh/guides/pricing/details before quoting to stakeholders.