HPRO: Hierarchical Progressive Reward Optimization via
Preference Extraction for Emotional Text-to-Speech

Anonymous submission to Interspeech 2026

Abstract

Recently, LLM-based Text-to-Speech (TTS) has achieved high naturalness. However, Supervised Fine-Tuning often converges to statistically averaged prosody, limiting emotional expressiveness. Preference-driven optimization offers a promising alternative, yet existing approaches suffer from two mismatches: information conflict, where content and emotion in a shared latent space produce conflicting gradients; and scale gap, where sparse sentence-level rewards struggle to guide dense frame-level generation. To address the information conflict, we introduce the HD-Emo codec as a differentiable reward model to extract distinct preference tokens, isolating emotional optimization from semantic sequences. To handle the scale gap, we propose HPRO, a hierarchical progressive reward optimization framework, employing the HD-Emo codec to progressively align frame-, word-, and sentence-level objectives. Experiments demonstrate that HPRO achieves high emotional expressiveness with robust stability.

To demonstrate the effectiveness of our proposed approach, we conducted a comparative study of several popular TTS models.
The evaluated systems include:

• TokenRecon
• HPRO (Ours)
• CosyVoice2
• CosyVoice3
• IndexTTS2
• HD-PPT

Evaluation Metrics. For each generated audio sample, objective scores are reported following the metrics used in our paper:

• wVAD-CCC: Word-level Valence-Arousal-Dominance Concordance Correlation Coefficient.
• EMO-SIM: Emotion Similarity.
• DNSMOS: DNSMOS P.835.

* The scores are shown under each audio sample in the format: [wVAD-CCC, EMO-SIM, DNSMOS].

Sample 1

Sample 2

Sample 3

Sample 4

Sample 5

Sample 6

Sample 7

Sample 8

Sample 9