DeepL Launches Aure, a Voice Model That Carries Tone and Pace Into Live Translation

Aure is built into DeepL Voice for Online Meetings, DeepL Voice for In-Person Conversations and the DeepL Voice API, and needs no setup, the company says on its product page. It supports eight languages at launch: English, German, Spanish, French, Italian, Japanese, Portuguese and Korean. DeepL says more are coming, and other languages will keep using its earlier voice models.

The change targets one of the main problems with machine-interpreted conversations, which is that the translated voice sounds flat. Until now, DeepL Voice built translated speech from a voice sample taken at the start of a conversation, so it could not follow shifts in mood or pick up sarcasm, according to The Decoder. Aure instead analyses both the translated text and the original audio as the conversation goes on, and DeepL says it adjusts to changes in tone and speed so that a point made slowly, or with excitement, comes across that way in the other language.

During training, DeepL separates voice identity, speech melody and content so that each can be carried into the new language on its own, and it uses reinforcement learning to keep accents out of the output, The Decoder reported. The company says the model matches a speaker's voice without storing a voice profile, filters out background noise, echoes and other speakers, and processes voice data in memory before deleting it when the session ends. Meeting data is not stored permanently or used to train its models, it says.

DeepL claims Aure produces "76% fewer critical translation errors" than translation tools built into meeting platforms. It does not publish the method behind that figure.

The launch follows DeepL's introduction in September of voice preservation, which keeps each speaker's voice recognisable across languages. In May the company cut about 250 jobs as it reorganised into smaller AI-based teams, and it has hired the team behind the audio streaming company Mixhalo to work on real-time speech translation. Other companies are also working on speech systems that pick up meaning beyond the words: Block recently unveiled Dessa, a speech model designed to detect emotion and sarcasm directly from audio.

Aure is available through a free trial, and DeepL has not published separate pricing for it.