Block’s new voice model sounds eerily human

Block's new voice model, Dessa, aims to improve AI-powered conversations by operating as a full-duplex system, enabling simultaneous speaking and listening. This capability, which Block notes is also found in OpenAI's GPT-Live, is intended to facilitate more fluid and natural interactions. The company claims Dessa's emotional responses are "built into" the model, contrasting with other systems like ElevenLabs v4 or Fish Audio Drama 3 that reportedly use voice tags to drive emotion.

The model's design addresses what Block describes as the "unnatural and annoying" qualities of current voice AI, which often converts voice to text before generating speech. Dessa's approach, with emotion reportedly stemming from its conversational data, was demonstrated by examples where the agent conveyed tiredness or emotional distress, suggesting a move towards more nuanced AI interactions. However, it is not clear if this emotion is being steered by some form of tags.

A notable feature is Dessa's ability to run locally on consumer-grade hardware, such as a MacBook M4 Max or better. This on-device processing could reduce operational costs for businesses by lessening reliance on remote servers, potentially expanding the application of voice AI in areas like Block's existing Square ordering system. While Dessa is not yet publicly accessible, Block has issued a call for safety researchers to test the model, particularly for scenarios where a voice agent might mislead users.