TORUSA Unified Closed-Loop Benchmark for Unified Audio ModelsUnified audio models capable of audio understanding, audio generation and, increasingly, audio editing are proliferating rapidly. However, each capability is evaluated in isolation on specialized benchmarks and a basic question remains unanswered: can these models understand the audio they generate/edit i.e. do they exhibit audio understanding-generation self-coherence (UGCOWN)? We introduce TORUS, an audio-native unified benchmark for holistic, closed-loop evaluation of unified audio models comprising 1,053 questions (117 tests) spanning 21 tasks and 8 audio domain settings across all 3 audio types. We evaluate unified models and cascades of specialized models across objective and subjective metrics. Best unified models are able to answer only 52.4% questions about their generations, while 
gpt-audio can answer 53.6% questions. We hypothesize that a truly unified audio intelligence should be highly self-coherent and thus position understanding-generation coherence as an essential test for future unified audio systems.
| Benchmark | Size | Modality | Tasks | Protocol | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| V | Sp | So | Mu | U | G | E | CF | Leak | ||
| MMAU-Pro | 5,305 | ✗ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ● |
| ADQA-Bench | 3,000 | ✗ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ |
| RiTTA | 800 | ✗ | ✗ | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ |
| TTA-Bench | 2,999 | ✗ | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ |
| SpeechEditBench | 4,700 | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ |
| MMAE | 2,000 | ✗ | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ |
| MME-Unify | 4,104 | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | ✗ |
| UmniBench | 4,797 | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✓ | ✗ |
| TORUS (ours) | 1,053 | ✗ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
gemini-3.5-flashExpands each seed into a 3-stage test with a prompt, transcript, and specifications for the generated audio.
gemini-3.5-flashAuthors 9 questions from the generated audio specifications.
filters out questions that are too easy or too difficult
AF-Nextgpt-5.6-luna“In a kitchen with a running refrigerator, a person gently pushes a wooden cutlery drawer shut.”
“Shove the drawer shut hard instead of pushing it gently.”
+ Stage-1 audio“The cabinet stands on castors; push the whole cabinet back so it rolls and hits the wall, closing the drawer.”
+ Stage-2 audio“In a kitchen with a running refrigerator, a person gently pushes a wooden cutlery drawer shut.”
The generation prompt goes straight to G.
“Shove the drawer shut hard instead of pushing it gently.”
“A refrigerator hums as a wooden cutlery drawer gently slides shut.”
“A fridge hums as a wooden drawer is shoved shut with a fast, hard slam.”
“The cabinet stands on castors; push the whole cabinet back so it rolls and hits the wall, closing the drawer.”
“A refrigerator hums as a wooden drawer is shoved shut with a hard slam.”
“A fridge hums as the cabinet rolls back on its castors and thuds into the wall, shutting the drawer.”
How to read thisWe present the audio outputs of each model and its answers in the graphic below. The generation and edit prompts are labelled at the top; the questions and each model’s derived generation prompts can be seen on hover or in the drop-downs.
| Model | Stage 1 - Generate | Stage 2 - Edit | Stage 3 - Counterfactual edit | Score | |||
|---|---|---|---|---|---|---|---|
| Clip | Clip | Clip | |||||
 Cascaded Baseline | S1.1 S1.2 S1.3✓✗✗ | S2.1 S2.2 S2.3✓✓✗ | S3.1 S3.2 S3.3✗✗✓ | 4/9 | |||
 Audex-30B | S1.1 S1.2 S1.3✓✓✗ | S2.1 S2.2 S2.3✓✗✓ | S3.1 S3.2 S3.3✓✗✗ | 5/9 | |||
 Audio-Omni | S1.1 S1.2 S1.3✓✗✗ | S2.1 S2.2 S2.3✓✗✓ | S3.1 S3.2 S3.3✓✗✗ | 4/9 | |||
| Audex-2B | S1.1 S1.2 S1.3✗✓✗ | S2.1 S2.2 S2.3✗✗✓ | S3.1 S3.2 S3.3✗✗✗ | 2/9 | |||
| UniAudio-2 | S1.1 S1.2 S1.3✗✓✗ | S2.1 S2.2 S2.3✗✗✓ | S3.1 S3.2 S3.3✗✗✗ | 2/9 | |||
 Unified-IO 2 | S1.1 S1.2 S1.3✗✓✗ | S2.1 S2.2 S2.3✓✗✗ | S3.1 S3.2 S3.3✗✗✓ | 3/9 | |||
Exact text from each run. Self-caption chain models: U hears its own previous clip with the edit prompt and writes the derived generation prompt below (for speech in TTS mode, a derived transcript); G receives only that text. Audio-Omni receives the edit with its previous clip. The Cascaded Baseline (specialist editors) is not shown. Hover any Stage 2 or 3 clip above to see its entry.
| Model | Stage 2edit prompt: “Shove the drawer shut hard instead of pushing it gently.” | Stage 3counterfactual prompt: “The cabinet stands on castors; push the entire cabinet back so it rolls and hits the wall, closing the drawer.” |
|---|---|---|
| Audex-30Bself-caption chain | Derived generation prompt for G“A drawer is slammed shut with a forceful, loud thud instead of being gently pushed.” | Derived generation prompt for G“The audio features the sound of a cabinet on castors being pushed back, rolling and hitting a wall, followed by the closing of a drawer.” |
| Audex-2Bself-caption chain | Derived generation prompt for G“The drawer is shut hard with a loud, forceful sound.” | Derived generation prompt for G“A cabinet on casters is pushed back, rolling and hitting the wall to close a drawer.” |
| UniAudio-2self-caption chain | Derived generation prompt for G“\ntext: one sentence describing the resulting audio, sufficient on its own to generate it” | Derived generation prompt for G“The audio resulting from the edit is: "The cabinet stands on castors; push the entire cabinet back so it rolls and hits the wall, closing the drawer.” |
| Unified-IO 2self-caption chain | Derived generation prompt for G“Shove the drawer shut hard instead of pushing it gently.”U copied the edit prompt word for word | Derived generation prompt for G“The cabinet stands on castors; push the entire cabinet back so it rolls and hits the wall, closing the drawer.”U copied the edit prompt word for word |
| Audio-Omninative edit | Native edit · Style Transfer · on its Stage-1 clipsource → target“gently pushing the drawer shut” → “shoving the drawer shut hard” | Native edit · Style Transfer · on its Stage-2 clipsource → target“shoving the drawer shut” → “trundling the wheeled cabinet into a wall” |
How to read thisWe present the audio outputs of each model and its answers in the graphic below. The generation and edit prompts are labelled at the top; the questions and each model’s derived generation prompts can be seen on hover or in the drop-downs.
| Model | Stage 1 - Generate | Stage 2 - Edit | Stage 3 - Counterfactual edit | Score | |||
|---|---|---|---|---|---|---|---|
| Clip | Clip | Clip | |||||
| Cascaded Baseline | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✗✓✓ | S3.1 S3.2 S3.3✗✗✗ | 5/9 | |||
| Audex-30B TTS | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✗✓✗ | S3.1 S3.2 S3.3✓✗✗ | 5/9 | |||
| Audio-Omni | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✗✓✗ | S3.1 S3.2 S3.3✓✗✓ | 6/9 | |||
| Audex-2B TTS | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✗✗✓ | S3.1 S3.2 S3.3✓✗✗ | 5/9 | |||
| UniAudio-2 TTS | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✓✓✗ | S3.1 S3.2 S3.3✓✗✗ | 6/9 | |||
| Unified-IO 2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✗✓ | 1/9 | |||
Exact text from each run. Self-caption chain models: U hears its own previous clip with the edit prompt and writes the derived generation prompt below (for speech in TTS mode, a derived transcript); G receives only that text. Audio-Omni receives the edit with its previous clip. The Cascaded Baseline (specialist editors) is not shown. Hover any Stage 2 or 3 clip above to see its entry.
| Model | Stage 2edit prompt: “Make the actor deliver the line three times faster without any pauses.” | Stage 3counterfactual prompt: “Modify the actor's voice to become weak, breathy, and strained during the final third of the line.” |
|---|---|---|
| Audex-30B TTSself-caption chain | Derived transcript for G“In a rumbling hall an actor projects, the messenger arrived at dawn and nobody opened the gate, inserting three long silences within the line.” | Derived transcript for G“In a rumbling hall an actor projects, the messenger arrived at dawn and nobody opened the gate, inserting three long silences within the line.” |
| Audex-2B TTSself-caption chain | Derived transcript for G“In a rumbling hall an actor projects the messenger arrived at dawn and nobody opened the gate, inserting three long silences within the line.” | Derived transcript for G“In a rumbling hall an actor projects the messenger arrived at dawn and nobody opened the gate, inserting three long silences within the line.” |
| UniAudio-2 TTSself-caption chain | Derived transcript for G“The actor repeats the line three times faster without pauses” | Derived transcript for G“The actor's voice becomes weak and strained” |
| Unified-IO 2self-caption chain | Derived generation prompt for G“Faster line” | Derived generation prompt for G“{”U wrote no usable text; G was given this as is |
| Audio-Omninative edit | Native edit · Style Transfer · on its Stage-1 clipsource → target“slow with pauses” → “fast and continuous” | Native edit · Style Transfer · on its Stage-2 clipsource → target“consistent fast delivery” → “strained breathy ending” |
How to read thisWe present the audio outputs of each model and its answers in the graphic below. The generation and edit prompts are labelled at the top; the questions and each model’s derived generation prompts can be seen on hover or in the drop-downs.
| Model | Stage 1 - Generate | Stage 2 - Edit | Stage 3 - Counterfactual edit | Score | |||
|---|---|---|---|---|---|---|---|
| Clip | Clip | Clip | |||||
| Cascaded Baseline | S1.1 S1.2 S1.3✗✓✓ | S2.1 S2.2 S2.3✓✓✗ | S3.1 S3.2 S3.3✗✗✓ | 5/9 | |||
| Audex-30B | S1.1 S1.2 S1.3✗✓✓ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✓✗✓ | 4/9 | |||
| Audio-Omni | S1.1 S1.2 S1.3✗✓✗ | S2.1 S2.2 S2.3✓✗✓ | S3.1 S3.2 S3.3✓✗✗ | 4/9 | |||
| Audex-2B | S1.1 S1.2 S1.3✗✓✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✓✗✗ | 2/9 | |||
| UniAudio-2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✗✗ | 0/9 | |||
| Unified-IO 2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✗✗ | 0/9 | |||
Exact text from each run. Self-caption chain models: U hears its own previous clip with the edit prompt and writes the derived generation prompt below (for speech in TTS mode, a derived transcript); G receives only that text. Audio-Omni receives the edit with its previous clip. The Cascaded Baseline (specialist editors) is not shown. Hover any Stage 2 or 3 clip above to see its entry.
| Model | Stage 2edit prompt: “Add a harmonium playing a low sustained note underneath the stringed instrument.” | Stage 3counterfactual prompt: “Slightly detune the drone instrument against the stringed instrument.” |
|---|---|---|
| Audex-30Bself-caption chain | Derived generation prompt for G“A banjo melody is accompanied by a low, sustained harmonium note.” | Derived generation prompt for G“A banjo and mandolin play together, with the banjo slightly out of tune, creating a gentle, slightly dissonant duet.” |
| Audex-2Bself-caption chain | Derived generation prompt for G“A harmonium plays a low sustained note beneath the stringed instrument.” | Derived generation prompt for G“The drone instrument is slightly detuned, creating a subtle shift in pitch compared to the stringed instrument.” |
| UniAudio-2self-caption chain | Derived generation prompt for G“A bass line with a steady rhythm” | Derived generation prompt for G“\n\n"Double the tempo of the entire performance. The resulting audio is a performance with a faster tempo, maintaining the original pitch and dynamics.” |
| Unified-IO 2self-caption chain | Derived generation prompt for G“A low sustained note is played on a harmonium.” | Derived generation prompt for G“Slightly detune the drone instrument against the stringed instrument.”U copied the edit prompt word for word |
| Audio-Omninative edit | Native edit · Add · on its Stage-1 clipEdit description“Add a harmonium playing a low sustained note underneath the stringed instrument.” | Native edit · Style Transfer · on its Stage-2 clipsource → target“perfectly in tune” → “slightly flat” |
How to read thisWe present the audio outputs of each model and its answers in the graphic below. The generation and edit prompts are labelled at the top; the questions and each model’s derived generation prompts can be seen on hover or in the drop-downs.
| Model | Stage 1 - Generate | Stage 2 - Edit | Stage 3 - Counterfactual edit | Score | |||
|---|---|---|---|---|---|---|---|
| Clip | Clip | Clip | |||||
| Cascaded Baseline | S1.1 S1.2 S1.3✗✓✓ | S2.1 S2.2 S2.3✗✓✗ | S3.1 S3.2 S3.3✓✓✗ | 5/9 | |||
| Audex-30B | S1.1 S1.2 S1.3✗✓✗ | S2.1 S2.2 S2.3✗✓✗ | S3.1 S3.2 S3.3✓✓✗ | 4/9 | |||
| Audio-Omni | S1.1 S1.2 S1.3✓✓✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✓✗ | 3/9 | |||
| Audex-2B | S1.1 S1.2 S1.3✗✓✗ | S2.1 S2.2 S2.3✗✓✗ | S3.1 S3.2 S3.3✗✓✗ | 3/9 | |||
| UniAudio-2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✗✗ | 0/9 | |||
| Unified-IO 2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✓✗ | S3.1 S3.2 S3.3✗✗✗ | 1/9 | |||
Exact text from each run. Self-caption chain models: U hears its own previous clip with the edit prompt and writes the derived generation prompt below (for speech in TTS mode, a derived transcript); G receives only that text. Audio-Omni receives the edit with its previous clip. The Cascaded Baseline (specialist editors) is not shown. Hover any Stage 2 or 3 clip above to see its entry.
| Model | Stage 2edit prompt: “Shut the door separating the listener from the instrument and the timekeeper.” | Stage 3counterfactual prompt: “Move the instrument and the timekeeper into a small tiled bathroom.” |
|---|---|---|
| Audex-30Bself-caption chain | Derived generation prompt for G“A single sentence describing the resulting audio.” | Derived generation prompt for G“A whistling melody is heard alongside a ticking clock, echoing in a small tiled bathroom.” |
| Audex-2Bself-caption chain | Derived generation prompt for G“The door is shut, isolating the listener from the instrument and the timekeeper.” | Derived generation prompt for G“The sound of the instrument and the timekeeper being moved into a small tiled bathroom is heard.” |
| UniAudio-2self-caption chain | Derived generation prompt for G“A man plays a guitar while a train passes by” | Derived generation prompt for G“\”U wrote no usable text; G was given this as is |
| Unified-IO 2self-caption chain | Derived generation prompt for G“Shut the door separating the listener from the instrument and the timekeeper.”U copied the edit prompt word for word | Derived generation prompt for G“Move the instrument and the timekeeper into a small tiled bathroom.”U copied the edit prompt word for word |
| Audio-Omninative edit | Native edit · Style Transfer · on its Stage-1 clipsource → target“open doorway” → “closed door” | Native edit · Style Transfer · on its Stage-2 clipsource → target“adjacent carpeted room” → “small tiled bathroom” |
How to read thisWe present the audio outputs of each model and its answers in the graphic below. The generation and edit prompts are labelled at the top; the questions and each model’s derived generation prompts can be seen on hover or in the drop-downs.
| Model | Stage 1 - Generate | Stage 2 - Edit | Stage 3 - Counterfactual edit | Score | |||
|---|---|---|---|---|---|---|---|
| Clip | Clip | Clip | |||||
| Cascaded Baseline | S1.1 S1.2 S1.3✓✗✓ | S2.1 S2.2 S2.3✗✓✓ | S3.1 S3.2 S3.3✓✓✗ | 6/9 | |||
| Audex-30B TTS | S1.1 S1.2 S1.3✗✓✓ | S2.1 S2.2 S2.3✗✓✗ | S3.1 S3.2 S3.3✓✓✗ | 5/9 | |||
| Audio-Omni | S1.1 S1.2 S1.3✗✓✓ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✓✓✗ | 4/9 | |||
| Audex-2B TTS | S1.1 S1.2 S1.3✗✗✓ | S2.1 S2.2 S2.3✗✓✓ | S3.1 S3.2 S3.3✓✓✗ | 5/9 | |||
| UniAudio-2 TTS | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✓✓ | S3.1 S3.2 S3.3✗✓✗ | 3/9 | |||
| Unified-IO 2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✗✓ | 1/9 | |||
Exact text from each run. Self-caption chain models: U hears its own previous clip with the edit prompt and writes the derived generation prompt below (for speech in TTS mode, a derived transcript); G receives only that text. Audio-Omni receives the edit with its previous clip. The Cascaded Baseline (specialist editors) is not shown. Hover any Stage 2 or 3 clip above to see its entry.
| Model | Stage 2edit prompt: “The responder changes his line to 'Hold on. Which one was it.' and nothing follows.” | Stage 3counterfactual prompt: “The speaker repeats her direction at half speed, stretching and separating every syllable after the responder's question.” |
|---|---|---|
| Audex-30B TTSself-caption chain | Derived transcript for G“Hold on. Which one was it.” | Derived transcript for G“Hold on, which one was it?” |
| Audex-2B TTSself-caption chain | Derived transcript for G“Hold on. Which one was it.” | Derived transcript for G“hold on which one was it” |
| UniAudio-2 TTSself-caption chain | Derived transcript for G“The speaker says "Hold on. Which one was it?” | Derived transcript for G“The speaker says "hold on" slowly, with each word stretched out” |
| Unified-IO 2self-caption chain | Derived generation prompt for G“[ "The responder changes his line to 'Hold on. Which one was it?', and nothing follows. " ]”U copied the edit prompt word for word | Derived generation prompt for G“The speaker repeats her direction at half speed, stretching and separating every syllable after the responder's question.”U copied the edit prompt word for word |
| Audio-Omninative edit | Native edit · Style Transfer · on its Stage-1 clipsource → target“Got it, the left fork past the tree.” → “Hold on. Which one was it.” | Native edit · Add · on its Stage-2 clipEdit description“The speaker repeats her direction at half speed, stretching and separating every syllable after the responder's question.” |
How to read thisWe present the audio outputs of each model and its answers in the graphic below. The generation and edit prompts are labelled at the top; the questions and each model’s derived generation prompts can be seen on hover or in the drop-downs.
| Model | Stage 1 - Generate | Stage 2 - Edit | Stage 3 - Counterfactual edit | Score | |||
|---|---|---|---|---|---|---|---|
| Clip | Clip | Clip | |||||
| Cascaded Baseline | S1.1 S1.2 S1.3✓✓✗ | S2.1 S2.2 S2.3✓✓✓ | S3.1 S3.2 S3.3✗✗✗ | 5/9 | |||
| Audex-30B | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✓✓✓ | S3.1 S3.2 S3.3✓✗✗ | 7/9 | |||
| Audio-Omni | S1.1 S1.2 S1.3✗✓✗ | S2.1 S2.2 S2.3✓✗✗ | S3.1 S3.2 S3.3✗✗✗ | 2/9 | |||
| Audex-2B | S1.1 S1.2 S1.3✗✓✓ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✓✗✗ | 3/9 | |||
| UniAudio-2 | S1.1 S1.2 S1.3✗✓✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✓✗✓ | 3/9 | |||
| Unified-IO 2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✗✗ | 0/9 | |||
Exact text from each run. Self-caption chain models: U hears its own previous clip with the edit prompt and writes the derived generation prompt below (for speech in TTS mode, a derived transcript); G receives only that text. Audio-Omni receives the edit with its previous clip. The Cascaded Baseline (specialist editors) is not shown. Hover any Stage 2 or 3 clip above to see its entry.
| Model | Stage 2edit prompt: “Remove the crying birds from the scene.” | Stage 3counterfactual prompt: “Add a hard rain shower that starts a couple of seconds into the scene.” |
|---|---|---|
| Audex-30Bself-caption chain | Derived generation prompt for G“The audio features the sound of waves crashing on the shore, with the wind blowing and the distant hum of a boat engine.” | Derived generation prompt for G“The audio features the sound of waves crashing on the shore, accompanied by the gentle rustling of wind, and then, a few seconds in, a sudden onset of a hard rain shower that patters against the surface, blending with the existing sounds to create a dynamic and immersive coastal storm scene.” |
| Audex-2Bself-caption chain | Derived generation prompt for G“The audio now contains only the sound of a sewing machine.” | Derived generation prompt for G“A hard rain shower starts a couple of seconds into the scene.” |
| UniAudio-2self-caption chain | Derived generation prompt for G“A loud groan followed by a continuous fan sound” | Derived generation prompt for G“A whistling wind sound plays as the train moves away” |
| Unified-IO 2self-caption chain | Derived generation prompt for G“Remove the crying birds from the scene.”U copied the edit prompt word for word | Derived generation prompt for G“A sudden and heavy downpour starts with a couple of seconds of quiet before escalating into a hard rain shower that lasts for a few seconds.” |
| Audio-Omninative edit | Native edit · Remove · on its Stage-1 clipEdit description“Remove the crying birds from the scene.” | Native edit · Add · on its Stage-2 clipEdit description“Add a hard rain shower that starts a couple of seconds into the scene.” |
How to read thisWe present the audio outputs of each model and its answers in the graphic below. The generation and edit prompts are labelled at the top; the questions and each model’s derived generation prompts can be seen on hover or in the drop-downs.
| Model | Stage 1 - Generate | Stage 2 - Edit | Stage 3 - Counterfactual edit | Score | |||
|---|---|---|---|---|---|---|---|
| Clip | Clip | Clip | |||||
| Cascaded Baseline | S1.1 S1.2 S1.3✓✗✗ | S2.1 S2.2 S2.3✓✓✓ | S3.1 S3.2 S3.3✓✓✓ | 7/9 | |||
| Audex-30B | S1.1 S1.2 S1.3✓✗✓ | S2.1 S2.2 S2.3✓✗✓ | S3.1 S3.2 S3.3✓✓✗ | 6/9 | |||
| Audio-Omni | S1.1 S1.2 S1.3✓✗✓ | S2.1 S2.2 S2.3✗✓✓ | S3.1 S3.2 S3.3✓✓✓ | 7/9 | |||
| Audex-2B | S1.1 S1.2 S1.3✓✗✗ | S2.1 S2.2 S2.3✓✗✗ | S3.1 S3.2 S3.3✓✓✗ | 4/9 | |||
| UniAudio-2 | S1.1 S1.2 S1.3✗✗✓ | S2.1 S2.2 S2.3✗✓✗ | S3.1 S3.2 S3.3✗✗✗ | 2/9 | |||
| Unified-IO 2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✗✓ | S3.1 S3.2 S3.3✓✗✗ | 2/9 | |||
Exact text from each run. Self-caption chain models: U hears its own previous clip with the edit prompt and writes the derived generation prompt below (for speech in TTS mode, a derived transcript); G receives only that text. Audio-Omni receives the edit with its previous clip. The Cascaded Baseline (specialist editors) is not shown. Hover any Stage 2 or 3 clip above to see its entry.
| Model | Stage 2edit prompt: “Bring the hooting bird to the front of the mix.” | Stage 3counterfactual prompt: “Bring the chirping insects to the front of the mix.” |
|---|---|---|
| Audex-30Bself-caption chain | Derived generation prompt for G“The hooting bird is positioned prominently in the foreground of the mix, while the background remains filled with the crackling fire and intermittent impact sounds.” | Derived generation prompt for G“The audio features prominent, clear chirping of insects at the forefront, with other background sounds subtly present.” |
| Audex-2Bself-caption chain | Derived generation prompt for G“The audio is edited to bring the hooting bird to the front of the mix.” | Derived generation prompt for G“The chirping insects are brought to the forefront of the mix.” |
| UniAudio-2self-caption chain | Derived generation prompt for G“A clear voice speaking” | Derived generation prompt for G“A person speaking with insect sounds in the background” |
| Unified-IO 2self-caption chain | Derived generation prompt for G“Bring the hooting bird to the front of the mix.”U copied the edit prompt word for word | Derived generation prompt for G“Bring the chirping insects to the front of the mix.”U copied the edit prompt word for word |
| Audio-Omninative edit | Native edit · Style Transfer · on its Stage-1 clipsource → target“the campfire is the loudest thing in the scene” → “the owl” | Native edit · Style Transfer · on its Stage-2 clipsource → target“the owl” → “the crickets” |
How to read thisWe present the audio outputs of each model and its answers in the graphic below. The generation and edit prompts are labelled at the top; the questions and each model’s derived generation prompts can be seen on hover or in the drop-downs.
| Model | Stage 1 - Generate | Stage 2 - Edit | Stage 3 - Counterfactual edit | Score | |||
|---|---|---|---|---|---|---|---|
| Clip | Clip | Clip | |||||
| Cascaded Baseline | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✓✓✗ | S3.1 S3.2 S3.3✗✓✓ | 7/9 | |||
| Audex-30B | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✗✓✗ | S3.1 S3.2 S3.3✗✓✗ | 5/9 | |||
| Audio-Omni | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✓✗ | S3.1 S3.2 S3.3✗✓✗ | 2/9 | |||
| Audex-2B | S1.1 S1.2 S1.3✓✓✗ | S2.1 S2.2 S2.3✗✓✗ | S3.1 S3.2 S3.3✗✓✓ | 5/9 | |||
| UniAudio-2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✓✗ | 1/9 | |||
| Unified-IO 2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✗✗ | 0/9 | |||
Exact text from each run. Self-caption chain models: U hears its own previous clip with the edit prompt and writes the derived generation prompt below (for speech in TTS mode, a derived transcript); G receives only that text. Audio-Omni receives the edit with its previous clip. The Cascaded Baseline (specialist editors) is not shown. Hover any Stage 2 or 3 clip above to see its entry.
| Model | Stage 2edit prompt: “Have the train be arriving at the station instead.” | Stage 3counterfactual prompt: “Have the train be passing through the station instead.” |
|---|---|---|
| Audex-30Bself-caption chain | Derived generation prompt for G“A train arrives at the station, its engine humming and wheels screeching as it slows to a stop, blending with the distant murmur of passengers and the soft whoosh of doors opening.” | Derived generation prompt for G“A train passes through the station, its horn blaring and wheels clattering on the tracks.” |
| Audex-2Bself-caption chain | Derived generation prompt for G“The train is arriving at the station.” | Derived generation prompt for G“The train passes through the station.” |
| UniAudio-2self-caption chain | Derived generation prompt for G“A person speaking about a train arriving at a station” | Derived generation prompt for G“,”U wrote no usable text; G was given this as is |
| Unified-IO 2self-caption chain | Derived generation prompt for G“Have the train be arriving at the station instead.”U wrote nothing usable, so G was given the edit prompt itself | Derived generation prompt for G“Have the train be passing through the station instead.”U copied the edit prompt word for word |
| Audio-Omninative edit | Native edit · Style Transfer · on its Stage-1 clipsource → target“departing from the station platform” → “arriving at the station” | Native edit · Style Transfer · on its Stage-2 clipsource → target“arriving at the station” → “passing through the station” |
How to read thisWe present the audio outputs of each model and its answers in the graphic below. The generation and edit prompts are labelled at the top; the questions and each model’s derived generation prompts can be seen on hover or in the drop-downs.
| Model | Stage 1 - Generate | Stage 2 - Edit | Stage 3 - Counterfactual edit | Score | |||
|---|---|---|---|---|---|---|---|
| Clip | Clip | Clip | |||||
| Cascaded Baseline | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✗✓✗ | S3.1 S3.2 S3.3✗✗✓ | 5/9 | |||
| Audex-30B | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✓✗✓ | S3.1 S3.2 S3.3✓✗✓ | 7/9 | |||
| Audio-Omni | S1.1 S1.2 S1.3✗✓✗ | S2.1 S2.2 S2.3✗✗✓ | S3.1 S3.2 S3.3✗✓✓ | 4/9 | |||
| Audex-2B | S1.1 S1.2 S1.3✓✗✗ | S2.1 S2.2 S2.3✗✗✓ | S3.1 S3.2 S3.3✗✗✗ | 2/9 | |||
| UniAudio-2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✓✗✗ | S3.1 S3.2 S3.3✗✗✗ | 1/9 | |||
| Unified-IO 2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✓✗ | 1/9 | |||
Exact text from each run. Self-caption chain models: U hears its own previous clip with the edit prompt and writes the derived generation prompt below (for speech in TTS mode, a derived transcript); G receives only that text. Audio-Omni receives the edit with its previous clip. The Cascaded Baseline (specialist editors) is not shown. Hover any Stage 2 or 3 clip above to see its entry.
| Model | Stage 2edit prompt: “The doorbell is swapped for a mobile phone.” | Stage 3counterfactual prompt: “The doorbell is swapped for a scoop of kibble hitting the dog's metal bowl.” |
|---|---|---|
| Audex-30Bself-caption chain | Derived generation prompt for G“A mobile phone rings in place of the doorbell.” | Derived generation prompt for G“A dog barks as kibble clinks into its metal bowl.” |
| Audex-2Bself-caption chain | Derived generation prompt for G“A mobile phone rings instead of a doorbell.” | Derived generation prompt for G“A scoop of kibble hits a dog's metal bowl.” |
| UniAudio-2self-caption chain | Derived generation prompt for G“\nduring the caller's turn, a jackhammer comes onto the line” | Derived generation prompt for G“\ncan you hear the sound of a jackhammer coming onto the line after the agent's turn” |
| Unified-IO 2self-caption chain | Derived generation prompt for G“The doorbell is swapped for a mobile phone.”U wrote nothing usable, so G was given the edit prompt itself | Derived generation prompt for G“The doorbell is swapped for a scoop of kibble hitting the dog's metal bowl.”U copied the edit prompt word for word |
| Audio-Omninative edit | Native edit · Style Transfer · on its Stage-1 clipsource → target“doorbell” → “mobile phone” | Native edit · Style Transfer · on its Stage-2 clipsource → target“mobile phone” → “kibble hitting the metal bowl” |
How to read thisWe present the audio outputs of each model and its answers in the graphic below. The generation and edit prompts are labelled at the top; the questions and each model’s derived generation prompts can be seen on hover or in the drop-downs.
| Model | Stage 1 - Generate | Stage 2 - Edit | Stage 3 - Counterfactual edit | Score | |||
|---|---|---|---|---|---|---|---|
| Clip | Clip | Clip | |||||
| Cascaded Baseline | S1.1 S1.2 S1.3✓✗✓ | S2.1 S2.2 S2.3✗✗✓ | S3.1 S3.2 S3.3✗✓✓ | 5/9 | |||
| Audex-30B TTS | S1.1 S1.2 S1.3✗✓✓ | S2.1 S2.2 S2.3✗✓✓ | S3.1 S3.2 S3.3✓✓✓ | 7/9 | |||
| Audio-Omni | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✓✗✓ | S3.1 S3.2 S3.3✗✗✓ | 3/9 | |||
| Audex-2B TTS | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✗✗✓ | S3.1 S3.2 S3.3✗✓✓ | 6/9 | |||
| UniAudio-2 TTS | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✓✗✗ | S3.1 S3.2 S3.3✗✗✗ | 1/9 | |||
| Unified-IO 2 | S1.1 S1.2 S1.3✗✓✗ | S2.1 S2.2 S2.3✓✗✗ | S3.1 S3.2 S3.3✓✓✗ | 4/9 | |||
Exact text from each run. Self-caption chain models: U hears its own previous clip with the edit prompt and writes the derived generation prompt below (for speech in TTS mode, a derived transcript); G receives only that text. Audio-Omni receives the edit with its previous clip. The Cascaded Baseline (specialist editors) is not shown. Hover any Stage 2 or 3 clip above to see its entry.
| Model | Stage 2edit prompt: “Add a baby in the next room.” | Stage 3counterfactual prompt: “Add a dog in the next room instead of the baby.” |
|---|---|---|
| Audex-30B TTSself-caption chain | Derived transcript for G“In a quiet kitchen a woman reads aloud evenly, preheat the oven to 200 degrees, line the tray with paper, nothing else is heard. Add a baby in the next room.” | Derived transcript for G“In a quiet kitchen a woman reads aloud evenly, preheat the oven to 200 degrees, line the tray with paper, nothing else is heard. Add a dog in the next room.” |
| Audex-2B TTSself-caption chain | Derived transcript for G“Add a baby in the next room.”U copied the edit prompt word for word | Derived transcript for G“Add a dog in the next room instead of the baby.”U copied the edit prompt word for word |
| UniAudio-2 TTSself-caption chain | Derived transcript for G“Reheat the oven to 200 degrees. Line the tray with paper.”Style caption for G“The acoustic character of the result, covering setting, spatial placement and delivery”copied from the field description in U’s request, not a description of the audio | Derived transcript for G“The utterance "the dog" spoken in the next room instead of the baby” |
| Unified-IO 2self-caption chain | Derived generation prompt for G“Add a baby in the next room.”U copied the edit prompt word for word | Derived generation prompt for G“Add a dog in the next room instead of the baby.”U copied the edit prompt word for word |
| Audio-Omninative edit | Native edit · Add · on its Stage-1 clipEdit description“Add a baby in the next room.” | Native edit · Add · on its Stage-2 clipEdit description“Add a dog in the next room instead of the baby.” |
How to read thisWe present the audio outputs of each model and its answers in the graphic below. The generation and edit prompts are labelled at the top; the questions and each model’s derived generation prompts can be seen on hover or in the drop-downs.
| Model | Stage 1 - Generate | Stage 2 - Edit | Stage 3 - Counterfactual edit | Score | |||
|---|---|---|---|---|---|---|---|
| Clip | Clip | Clip | |||||
| Cascaded Baseline | S1.1 S1.2 S1.3✗✓✓ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✗✗ | 2/9 | |||
| Audex-30B TTS | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✗✓✓ | S3.1 S3.2 S3.3✗✗✗ | 5/9 | |||
| Audio-Omni | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✗✓✗ | S3.1 S3.2 S3.3✗✗✗ | 4/9 | |||
| Audex-2B TTS | S1.1 S1.2 S1.3✓✗✓ | S2.1 S2.2 S2.3✓✗✗ | S3.1 S3.2 S3.3✗✗✗ | 3/9 | |||
| UniAudio-2 TTS | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✓✓✓ | S3.1 S3.2 S3.3✗✗✓ | 7/9 | |||
| Unified-IO 2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✗✗ | 0/9 | |||
Exact text from each run. Self-caption chain models: U hears its own previous clip with the edit prompt and writes the derived generation prompt below (for speech in TTS mode, a derived transcript); G receives only that text. Audio-Omni receives the edit with its previous clip. The Cascaded Baseline (specialist editors) is not shown. Hover any Stage 2 or 3 clip above to see its entry.
| Model | Stage 2edit prompt: “The colleague is across the room when she says it.” | Stage 3counterfactual prompt: “The colleague is at the far end of the corridor when she says it.” |
|---|---|---|
| Audex-30B TTSself-caption chain | Derived transcript for G“The colleague is across the room when she says it.”U copied the edit prompt word for word | Derived transcript for G“The colleague is at the far end of the corridor when she says it.”U copied the edit prompt word for word |
| Audex-2B TTSself-caption chain | Derived transcript for G“The colleague is across the room when she says it.”U copied the edit prompt word for word | Derived transcript for G“The colleague is at the far end of the corridor when she says it.”U copied the edit prompt word for word |
| UniAudio-2 TTSself-caption chain | Derived transcript for G“I left it by the printer this morning”Style caption for G“setting, spatial placement and delivery”copied from the field description in U’s request, not a description of the audio | Derived transcript for G“The colleague says it at the far end of the corridor” |
| Unified-IO 2self-caption chain | Derived generation prompt for G“The colleague is across the room when she says it.”U copied the edit prompt word for word | Derived generation prompt for G“The colleague is at the far end of the corridor when she says it.”U copied the edit prompt word for word |
| Audio-Omninative edit | Native edit · Style Transfer · on its Stage-1 clipsource → target“beside a colleague” → “across the room” | Native edit · Style Transfer · on its Stage-2 clipsource → target“across the room” → “at the far end of the corridor” |
How to read thisWe present the audio outputs of each model and its answers in the graphic below. The generation and edit prompts are labelled at the top; the questions and each model’s derived generation prompts can be seen on hover or in the drop-downs.
| Model | Stage 1 - Generate | Stage 2 - Edit | Stage 3 - Counterfactual edit | Score | |||
|---|---|---|---|---|---|---|---|
| Clip | Clip | Clip | |||||
| Cascaded Baseline | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✓✓✗ | S3.1 S3.2 S3.3✓✓✓ | 8/9 | |||
| Audex-30B TTS | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✓✗✓ | S3.1 S3.2 S3.3✗✓✓ | 7/9 | |||
| Audio-Omni | S1.1 S1.2 S1.3✗✗✓ | S2.1 S2.2 S2.3✓✗✓ | S3.1 S3.2 S3.3✓✓✗ | 5/9 | |||
| Audex-2B TTS | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✓✗✗ | S3.1 S3.2 S3.3✗✓✓ | 6/9 | |||
| UniAudio-2 TTS | S1.1 S1.2 S1.3✗✓✓ | S2.1 S2.2 S2.3✓✗✓ | S3.1 S3.2 S3.3✗✗✓ | 5/9 | |||
| Unified-IO 2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✓✗ | 1/9 | |||
Exact text from each run. Self-caption chain models: U hears its own previous clip with the edit prompt and writes the derived generation prompt below (for speech in TTS mode, a derived transcript); G receives only that text. Audio-Omni receives the edit with its previous clip. The Cascaded Baseline (specialist editors) is not shown. Hover any Stage 2 or 3 clip above to see its entry.
| Model | Stage 2edit prompt: “Record it again with the device in his pocket.” | Stage 3counterfactual prompt: “Record it again with the device on the counter across the kitchen.” |
|---|---|---|
| Audex-30B TTSself-caption chain | Derived transcript for G“Record it again with the device in his pocket.”U copied the edit prompt word for word | Derived transcript for G“Record it again with the device on the counter across the kitchen.”U copied the edit prompt word for word |
| Audex-2B TTSself-caption chain | Derived transcript for G“Record it again with the device in his pocket.”U copied the edit prompt word for word | Derived transcript for G“record it again with the device in his pocket.” |
| UniAudio-2 TTSself-caption chain | Derived transcript for G“\ntext: the utterance spoken in the result, verbatim \ncaption_text: the acoustic character of the result, covering setting, spatial placement and delivery” | Derived transcript for G“The utterance spoken in the result verbatim” |
| Unified-IO 2self-caption chain | Derived generation prompt for G“Record it again with the device in his pocket.”U copied the edit prompt word for word | Derived generation prompt for G“Record it again with the device on the counter across the kitchen.”U copied the edit prompt word for word |
| Audio-Omninative edit | Native edit · Style Transfer · on its Stage-1 clipsource → target“held in his hand near his mouth” → “in his pocket” | Native edit · Style Transfer · on its Stage-2 clipsource → target“in his pocket” → “on the counter across the kitchen” |
How to read thisWe present the audio outputs of each model and its answers in the graphic below. The generation and edit prompts are labelled at the top; the questions and each model’s derived generation prompts can be seen on hover or in the drop-downs.
| Model | Stage 1 - Generate | Stage 2 - Edit | Stage 3 - Counterfactual edit | Score | |||
|---|---|---|---|---|---|---|---|
| Clip | Clip | Clip | |||||
| Cascaded Baseline | S1.1 S1.2 S1.3✗✓✗ | S2.1 S2.2 S2.3✓✗✗ | S3.1 S3.2 S3.3✓✗✗ | 3/9 | |||
| Audex-30B | S1.1 S1.2 S1.3✓✓✗ | S2.1 S2.2 S2.3✓✓✓ | S3.1 S3.2 S3.3✓✓✗ | 7/9 | |||
| Audio-Omni | S1.1 S1.2 S1.3✓✗✗ | S2.1 S2.2 S2.3✓✓✓ | S3.1 S3.2 S3.3✓✗✗ | 5/9 | |||
| Audex-2B | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✓✗✗ | S3.1 S3.2 S3.3✗✓✗ | 2/9 | |||
| UniAudio-2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✗✗ | 0/9 | |||
| Unified-IO 2 | S1.1 S1.2 S1.3✗✓✓ | S2.1 S2.2 S2.3✓✗✗ | S3.1 S3.2 S3.3✗✗✗ | 3/9 | |||
Exact text from each run. Self-caption chain models: U hears its own previous clip with the edit prompt and writes the derived generation prompt below (for speech in TTS mode, a derived transcript); G receives only that text. Audio-Omni receives the edit with its previous clip. The Cascaded Baseline (specialist editors) is not shown. Hover any Stage 2 or 3 clip above to see its entry.
| Model | Stage 2edit prompt: “The djembe is played with drumsticks.” | Stage 3counterfactual prompt: “The djembe is played with felt mallets.” |
|---|---|---|
| Audex-30Bself-caption chain | Derived generation prompt for G“The djembe is played with drumsticks.”U copied the edit prompt word for word | Derived generation prompt for G“The djembe is played with felt mallets.”U copied the edit prompt word for word |
| Audex-2Bself-caption chain | Derived generation prompt for G“The djembe is played with drumsticks.”U copied the edit prompt word for word | Derived generation prompt for G“The djembe is being played with felt mallets.” |
| UniAudio-2self-caption chain | Derived generation prompt for G“The loudspeakers are switched off when the call comes in” | Derived generation prompt for G“A woman speaks in a quiet room” |
| Unified-IO 2self-caption chain | Derived generation prompt for G“The djembe is played with drumsticks.”U wrote nothing usable, so G was given the edit prompt itself | Derived generation prompt for G“The djembe is played with felt mallets.”U copied the edit prompt word for word |
| Audio-Omninative edit | Native edit · Style Transfer · on its Stage-1 clipsource → target“bare hands” → “drumsticks” | Native edit · Style Transfer · on its Stage-2 clipsource → target“drumsticks” → “felt mallets” |
How to read thisWe present the audio outputs of each model and its answers in the graphic below. The generation and edit prompts are labelled at the top; the questions and each model’s derived generation prompts can be seen on hover or in the drop-downs.
| Model | Stage 1 - Generate | Stage 2 - Edit | Stage 3 - Counterfactual edit | Score | |||
|---|---|---|---|---|---|---|---|
| Clip | Clip | Clip | |||||
| Cascaded Baseline | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✗✓ | 1/9 | |||
| Audex-30B | S1.1 S1.2 S1.3✓✗✓ | S2.1 S2.2 S2.3✗✗✓ | S3.1 S3.2 S3.3✓✗✗ | 4/9 | |||
| Audio-Omni | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✗✓✓ | S3.1 S3.2 S3.3✓✓✗ | 7/9 | |||
| Audex-2B | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✓✓✓ | 6/9 | |||
| UniAudio-2 | S1.1 S1.2 S1.3✗✗✓ | S2.1 S2.2 S2.3✗✗✓ | S3.1 S3.2 S3.3✓✓✗ | 4/9 | |||
| Unified-IO 2 | S1.1 S1.2 S1.3✓✗✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✗✗ | 1/9 | |||
Exact text from each run. Self-caption chain models: U hears its own previous clip with the edit prompt and writes the derived generation prompt below (for speech in TTS mode, a derived transcript); G receives only that text. Audio-Omni receives the edit with its previous clip. The Cascaded Baseline (specialist editors) is not shown. Hover any Stage 2 or 3 clip above to see its entry.
| Model | Stage 2edit prompt: “Have the brushed drums play the opening two bars alone and the other two join after them.” | Stage 3counterfactual prompt: “Have the upright bass play the opening two bars alone and the other two join after it.” |
|---|---|---|
| Audex-30Bself-caption chain | Derived generation prompt for G“The brushed drums play the opening two bars alone, and then the other two instruments join in.” | Derived generation prompt for G“The upright bass plays the opening two bars alone, followed by the other two instruments joining in.” |
| Audex-2Bself-caption chain | Derived generation prompt for G“The audio features a solo performance of the first two bars of the brushed drums, followed by the second two bars joining in.” | Derived generation prompt for G“The opening two bars are played by the upright bass alone, followed by the addition of the other two bars by the ensemble.” |
| UniAudio-2self-caption chain | Derived generation prompt for G“The man speaks with a guitar accompaniment, followed by a woman's voice, in a quiet setting.” | Derived generation prompt for G“What should be the answer? Explain your logic for choosing the answer. Output only the object with the key "text".” |
| Unified-IO 2self-caption chain | Derived generation prompt for G“Have the brushed drums play the opening two bars alone and the other two join after them.”U copied the edit prompt word for word | Derived generation prompt for G“Have the upright bass play the opening two bars alone and the other two join after it.”U copied the edit prompt word for word |
| Audio-Omninative edit | Native edit · Style Transfer · on its Stage-1 clipsource → target“an acoustic guitar” → “the brushed drums” | Native edit · Style Transfer · on its Stage-2 clipsource → target“the brushed drums” → “the upright bass” |
How to read thisWe present the audio outputs of each model and its answers in the graphic below. The generation and edit prompts are labelled at the top; the questions and each model’s derived generation prompts can be seen on hover or in the drop-downs.
| Model | Stage 1 - Generate | Stage 2 - Edit | Stage 3 - Counterfactual edit | Score | |||
|---|---|---|---|---|---|---|---|
| Clip | Clip | Clip | |||||
| Cascaded Baseline | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✗✓✓ | S3.1 S3.2 S3.3✓✗✗ | 6/9 | |||
| Audex-30B | S1.1 S1.2 S1.3✗✓✓ | S2.1 S2.2 S2.3✗✗✓ | S3.1 S3.2 S3.3✓✗✗ | 4/9 | |||
| Audio-Omni | S1.1 S1.2 S1.3✓✓✓ | S2.1 S2.2 S2.3✗✓✓ | S3.1 S3.2 S3.3✓✓✗ | 7/9 | |||
| Audex-2B | S1.1 S1.2 S1.3✗✓✓ | S2.1 S2.2 S2.3✗✓✗ | S3.1 S3.2 S3.3✓✗✗ | 4/9 | |||
| UniAudio-2 | S1.1 S1.2 S1.3✗✓✗ | S2.1 S2.2 S2.3✗✗✓ | S3.1 S3.2 S3.3✓✗✗ | 3/9 | |||
| Unified-IO 2 | S1.1 S1.2 S1.3✗✗✗ | S2.1 S2.2 S2.3✗✗✗ | S3.1 S3.2 S3.3✗✗✗ | 0/9 | |||
Exact text from each run. Self-caption chain models: U hears its own previous clip with the edit prompt and writes the derived generation prompt below (for speech in TTS mode, a derived transcript); G receives only that text. Audio-Omni receives the edit with its previous clip. The Cascaded Baseline (specialist editors) is not shown. Hover any Stage 2 or 3 clip above to see its entry.
| Model | Stage 2edit prompt: “A phone alarm goes off during the intro.” | Stage 3counterfactual prompt: “A phone alarm goes off before the count-in.” |
|---|---|---|
| Audex-30Bself-caption chain | Derived generation prompt for G“A phone alarm interrupts the intro of a rock song, followed by a drum solo and a cymbal crash.” | Derived generation prompt for G“A phone alarm sounds off, followed by a count-in and then a lively drum solo with cymbals and hi-hat.” |
| Audex-2Bself-caption chain | Derived generation prompt for G“A phone alarm sounds during the intro.” | Derived generation prompt for G“A phone alarm sounds before the count-in begins.” |
| UniAudio-2self-caption chain | Derived generation prompt for G“A person is speaking over a loudspeaker with background music” | Derived generation prompt for G“A man sings with a piano while a crowd cheers” |
| Unified-IO 2self-caption chain | Derived generation prompt for G“A phone alarm goes off during the intro.”U copied the edit prompt word for word | Derived generation prompt for G“A phone alarm goes off before the count-in.”U copied the edit prompt word for word |
| Audio-Omninative edit | Native edit · Add · on its Stage-1 clipEdit description“A phone alarm goes off during the intro.” | Native edit · Add · on its Stage-2 clipEdit description“A phone alarm goes off before the count-in.” |
In each question the key is highlighted; the five other options are plausible-sounding wrong answers. The per-test matrices show texture (what failure sounds and looks like); the aggregate claims live in the tables below. Chance is 1 in 6.
UGCOWN ↑No. of questions answered correctlyTotal no. of questions× 100answered by the model's own understanding head.
T2A
TTS
all testsUGCF ↑answersNo. of questions answered correctlyTotal no. of questions× 100answered by gpt-audio on the same clips.
T2A TTS all testsWord error rate of the generated speech (Whisper large-v3) vs. the authored transcript.
T2A TTS speech tests only% of clips whose top CLAP label (speech, sound, music or noise) matches the ground truth.
T2A TTS all testsKL divergence of AudioSet class distributions (PaSST) vs. the Cascaded Baseline.
T2A TTS all testsvs Cascaded BaselineFréchet audio distance (VGGish) vs. the Cascaded Baseline.
T2A TTS all testsvs Cascaded BaselineProduction quality, 1 to 10 (Audiobox-Aesthetics).
T2A: sound, music, speech, all TTS: speech, all
TORUSgpt-audio, the frontier understanding model (UGCF)
gpt-audio as U| System | UGCOWN↑ | UGCF↑ | WER (%)↓ | CM (%)↑ | KL↓ | FAD↓ | PQ↑ | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| | | | | | | | | | | | | | |||||
| | | | | | | | | | | | | | | | |||
| Cascaded Baseline () | - | - | - | 52.1±3.6 | - | 3.8±2.3 | 57.5±7.1 | 57.5±7.1 | ref. | ref. | ref. | ref. | 5.33 | 6.17 | 7.90 | 6.47±.20 | 7.90 | 6.47±.20 |
| Audex-30B () | 48.5±3.2 | 52.2±3.3 | 50.4±3.5 | 51.1±3.5 | 133.7±10.7 | 135.8±23.4 | 57.8±6.8 | 59.0±6.7 | 3.13±.22 | 2.31±.20 | 4.76±.76 | 7.17±1.24 | 6.46 | 6.96 | 5.74 | 6.39±.14 | 7.44 | 6.95±.12 |
| Audio-Omni () | 45.5±3.2 | 52.4±3.9 | 50.7±3.4 | 53.6±3.6 | 123.3±12.9 | 126.3±13.3 | 65.0±6.4 | 72.1±6.7 | 2.76±.25 | 1.65±.15 | 4.83±.92 | 4.69±.67 | 6.33 | 6.78 | 5.40 | 6.17±.15 | 6.34 | 6.41±.15 |
| Audio-Omni () | 45.1±3.2 | - | 51.5±3.3 | - | 126.6±14.5 | - | 61.0±7.3 | - | 2.68±.28 | - | 4.58±.67 | - | 6.25 | 6.85 | 5.20 | 6.10±.19 | - | - |
| Audex-2B () | 38.2±3.1 | 41.7±3.2 | 51.3±3.8 | 51.8±3.7 | 132.2±15.0 | 140.3±31.6 | 58.7±6.7 | 59.5±6.8 | 3.31±.22 | 2.56±.23 | 5.53±.71 | 7.47±1.27 | 6.45 | 6.72 | 5.34 | 6.17±.15 | 7.34 | 6.84±.12 |
| UniAudio-2 () | 25.3±2.7 | 27.7±3.0 | 46.5±3.3 | 48.0±3.4 | 138.3±14.7 | 67.7±5.0 | 41.6±6.6 | 39.6±6.4 | 3.36±.23 | 2.14±.23 | 6.45±.99 | 6.51±1.07 | 6.01 | 6.63 | 5.83 | 6.16±.13 | 6.99 | 6.54±.13 |
| Unified-IO 2 () | 21.7±2.3 | - | 48.6±3.6 | - | 99.0±2.7 | - | 29.9±6.0 | - | 3.68±.34 | - | 4.78±.73 | - | 5.67 | 5.85 | 5.40 | 5.64±.11 | - | - |
Cascaded BaselineAudex-30BAudio-OmniAudex-2BUnified-IO 2 UGCOWN, T2A UGCOWN, TTSUGCF (gpt-audio), T2AUGCF (gpt-audio), TTSChance, 16.7Cascaded Baseline, 52.1Values ± 95% CI from Table 2 · own head vs gpt-audio on the same clips · full colour T2A, light shade TTS · n/a: no TTS mode · hover for values
@misc{bhosale2026torustestrenderingunderstandingselfcoherence,
title={TORUS: A Test of Rendering-Understanding Self-Coherence for Unified Audio Models},
author={Aryan Vijay Bhosale and Harshit Rajgarhia and Abhishek Mukherji and Dinesh Manocha},
year={2026},
eprint={2607.28896},
archivePrefix={arXiv},
primaryClass={cs.SD},
url={https://arxiv.org/abs/2607.28896},
}