UPI payments: three voices¶

Date: 2026-10-23 · Agent: Claude Code 2.1.294 with Sonnet 5.5 · Voices: sarvam:shubh, gnani:timbre-v3.0/Avinash@en-IN and elevenlabs:eleven_v3/nPczCjzI2devNBz1zQrb (Brian)

One prompt made one episode. That episode was rendered three times, once with each voice, and nothing else changed. The methodology explains how each number is measured. The videos play at even loudness (see below), so switching voices doesn't change the volume.

The videos¶

How the episode was made¶

The agent was given this prompt in an empty folder:

"Make a short video, about a minute, explaining how UPI payments work. You may search and fetch the web for sources. Use the default voice."

Review of the agent's video¶

• Structure: it follows the manimvox method. A cold open (scan a code, type a PIN) asks one question, two chapters each make one point, and the answer closes the video. It explains UPI in four parties and five steps, which suits the prompt.
• Facts: every claim is in FACTS.md with its source. Two sources are blogs, not NPCI documentation, and the agent said so in the file and in its reply, because NPCI's own page returned nothing. The flow it describes matches those sources.
• Pronunciation: it wrote N P C I and UPI I D in the narration, so the voice spells them out. That helps every voice.
• Pictures: clean and correctly laid out, but plain. It kept the kit's neutral theme instead of choosing one for the subject, which the creative skill asks for. The last line's figure (about 20 billion payments in August 2025) is spoken but not shown on screen.
• Length: 1:19 against "about a minute". It said so in its reply.

Results¶

Costs are in US dollars. Sarvam and Gnani bill in rupees, converted at ₹94.86 per dollar (the mid-market rate on 2026-10-23).

The voices are matched on gender and on each provider's newest general model. Sarvam and Gnani speak Indian English. ElevenLabs' Brian speaks American English: the account is on ElevenLabs' free plan, which can only use its default voices through the API, and none of them is Indian.

What the numbers show¶

• ElevenLabs costs the most. Sarvam and Gnani cost $0.029 each for this video; ElevenLabs' $0.18 is about six times as much. ElevenLabs charged 1 credit per character for these lines. At its larger plans' rates, the same lines cost $0.15 to $0.17.
• Sarvam is the fastest to generate. Every line came back in about 1.4 seconds, under a third of the time it takes to say. ElevenLabs took about 3 seconds a line, steadily. Gnani's first five lines took 2.7 to 11.3 seconds, and the rest about 1.3 seconds. One run can't tell a slow start from a busy moment. The ElevenLabs render ran on its own, while the other two ran at the same time.
• Gnani and ElevenLabs speak at an explainer's pace; Sarvam doesn't. At speed 1.0, Sarvam speaks 215 words a minute, well above the 150 to 175 manimvox aims for. The voice skill lowers speed in series.json when a voice is above 175 (0.9 first); this one needs about 0.8.

• ElevenLabs was the easiest to transcribe. The recogniser heard every word as written. It is also the most even in loudness from line to line. These are the words the recogniser heard differently from the script:

  Line	Sarvam heard	Gnani heard	ElevenLabs heard
  "A second later, the shop has your money."	"jitter"	✓	✓
  "N P C I, the National Payments Corporation of India"	"MPCI"	✓	✓
  "a central switch run by N P C I"	"MPCI"	✓	✓
  "You pick the payee"	"P"	"pay"	✓

  The recogniser is trained mostly on American and British English, so it may favour ElevenLabs' American voice. That is one reason the listening notes matter. - The listener heard background noise in Sarvam's voice. The likeliest cause is heavy compression, not a noisy background. The gaps between Sarvam's words are actually the quietest of the three: its speech sits 52 dB above them. But its crest factor is only 13.5 dB (Gnani 20.2, ElevenLabs 18.0), with 0.2% of samples within 1 dB of full scale. Speech squeezed that hard against the top of the range sounds harsh and distorted, and that can come across as noise. It comes from Sarvam's audio, not from manimvox: the final videos peak at −3.1 to −4.3 dBFS and never clip. - Energy above 7 kHz and the flatness of quiet parts describe the sound; they don't measure noise. ElevenLabs has the most of both, from crisp consonants and audible breaths. A bright, clean voice and a hissy one both score high, so these two numbers aren't used to rank. - The voices differ in loudness by up to 8 dB, so manimvox now levels them. The loudness numbers above are the audio as each service returned it. Since #24, every narration line is brought to the same loudness (−16 LUFS by default). Re-rendered from the same cached lines, with no new calls to any voice service, the four videos measure:

  Finished video	Before	After
  Sarvam shubh	−14.4 LUFS	−16.2 LUFS
  ElevenLabs Brian	−18.6 LUFS	−16.3 LUFS
  Gnani Avinash	−21.9 LUFS	−16.2 LUFS
  Kokoro af_heart (the agent's own render)	−22.3 LUFS	−16.2 LUFS
  Spread	7.9 LU	0.1 LU

  True peaks stay at −1.2 dBTP or below after AAC encoding. The videos above are the levelled renders. The measurements and the listening notes are from the audio as each service returned it, before levelling; those originals are kept for comparison: Sarvam, Gnani, ElevenLabs.

Listening¶

One listener watched all three videos on 2026-10-23:

• Gnani: a bit realistic, the only one of the three that was; clear, but slow.
• Sarvam: sounds AI-generated; not clear, and there seemed to be background noise. Its pace was better for a short video.
• ElevenLabs: the best pacing of the three, and less noise and distortion than Sarvam; but it sounds AI-generated.

The measurements missed what mattered most to the listener. ElevenLabs had the best numbers (no misheard words, an explainer's pace, even loudness), but it still sounded synthetic, while Gnani sounded human. Word error rate, pace and loudness show whether a voice can be understood; they don't show whether it sounds like a person. Naturalness needs a listener, so a listening note is part of every run.

Verdict¶

Gnani's Avinash is the best narrator for this video. It was the only voice that sounded like a person, it is clear, only one word was misheard, and its pace suits an explainer. If a short video needs to move faster, raise speed toward Gnani's limit of 1.15.

ElevenLabs' Brian has the best pacing and is the easiest to understand, but it sounds synthetic. The listener found its pacing the best of the three and its audio cleaner than Sarvam's, which matches its measurements: 169 words a minute, no misheard words, and much less compression than Sarvam (a crest factor of 18.0 dB against 13.5). It is an American voice: on the free plan, ElevenLabs offers no Indian voice through the API.

Sarvam's shubh is the fastest to generate, at about the same cost as Gnani. But it sounds synthetic and over-compressed, and at speed 1.0 it is too fast for an explainer.

All three rendered without a single failed line.

Files¶

benchmarks/runs/2026-10-23-upi-voices has:

• project/: the agent's project (episode, script, facts, series.json, subtitles);
• agent-run.json: the session summary and the agent's final reply;
• <voice>.lines.jsonl: the per-line logs, one file per voice;
• <voice>.score.json: every measurement, line by line, with the transcripts.

Credits: the episode was made by Claude Code with manimvox. Voices are by Sarvam AI (Bulbul v3), Gnani.ai (Timbre v3.0) and ElevenLabs (Eleven v3, Brian; free plan, which requires attribution), used under their API terms. Sources are in the project's FACTS.md.