Announcing the fastest inference for realtime voice AI agents

Voice interfaces are one of the hallmarks of a truly AI native application. From transcription to speech-to-code to outbound calling to custom podcasts, voice makes applications engaging and productive.  But developers often have to piece together a number of specialized voice services to ship a single voice application. This tends to slow development while adding complexity, latency and cost.

We’re pleased to announce the addition of a greatly expanded set of high performance, low latency voice infrastructure to our cloud. We’ve worked hard to provide voice services that are frontier quality, developer friendly and very low latency.

With these additions, we’ve expanded our voice offering from transcription to a full set of building blocks that can power some or all of an application’s voice pipeline. These services support real-time and batch patterns in developer-friendly serverless and dedicated form factors.

‍Streaming speech-to-text for voice agents

Streaming Whisper

Traditional batch transcription waits for complete audio files. Voice agents need to process speech as it arrives, and intelligently detect when users finish speaking.

We've built the industry's fastest speech-to-text API by combining optimized model inference with intelligent system design — WebSocket streaming to eliminate connection overhead, carefully tuned voice activity detection (VAD), and purpose-built infrastructure for realtime audio processing. The result: Whisper running in real time with minimal quality degradation, completing transcripts up to 35% faster than alternatives.

The key is optimizing for time-to-complete-transcript, not just time-to-first-token. Voice agents need to know precisely when a user stops speaking to begin formulating responses. Our VAD tuning ensures your agent responds at the right moment, not too early (cutting users off) or too late (creating dead air).

Text-to-speech: Serverless open-source models

Together AI is the first cloud to provide serverless open-source text-to-speech models. No more spinning up dedicated instances for sporadic TTS needs — both models are available through REST APIs for batch generation and WebSocket APIs for realtime streaming.

Orpheus TTS: Natural voice quality

Orpheus delivers natural, expressive speech with multiple voice options suitable for customer-facing applications. At 187ms time-to-first-byte, it outpaces premium providers while approaching the speed of lighter models. The result: professional voice quality without sacrificing the responsiveness voice agents require.

Kokoro TTS

When every millisecond counts, Kokoro delivers. With 97ms baseline TTFB, it's built for applications where response speed trumps all else. This predictable performance makes it ideal for high-volume voice agent deployments where cost and latency are critical.

New audio transcriptions

Two new capabilities expand our audio transcriptions API for batch processing workflows:

Voxtral Mini

Voxtral Mini is a higher-accuracy transcription model from Mistral AI, optimized for European languages and challenging audio conditions. Voxtral delivers measurably lower word error rates than standard Whisper — ideal for applications where transcription mistakes create liability or operational overhead.

Speaker Diarization

Automatically identify and label different speakers in recorded audio. Transform raw transcripts into structured conversations showing who said what and when — essential for meeting transcription, call center quality assurance, and multi-party conversation review.

Built for production voice agents

Three architectural decisions make Together AI's audio infrastructure uniquely suited for production voice agents:

Latency: Response times that enable natural conversation

Human conversation flows at a specific pace. Responses that take longer than 500ms feel unnatural. Beyond 2 seconds, users assume the system has failed. Every additional 100ms of latency measurably decreases user satisfaction and task completion rates.

Our infrastructure eliminates unnecessary latency at every layer. WebSocket connections stay alive, avoiding TCP handshake overhead. Models run on the same GPU clusters as your LLMs, eliminating cross-provider networking. Most critically, our optimized serving delivers consistent sub-200ms TTS and millisecond-accurate transcription even during traffic spikes.

Real-world impact: When a customer calls to change their flight, every second of delay increases the chance they'll hang up. The voice agent must capture their request, process it, and begin responding — all within the natural rhythm of human conversation.

Quality: Accurate transcription and natural-sounding speech

Voice agents fail when transcription errors cascade through the conversation. A misheard account number becomes a failed lookup. A garbled product name triggers the wrong workflow. Poor voice quality immediately signals "cheap automation" regardless of underlying intelligence.

That's why we offer multiple quality tiers. Streaming Whisper handles realtime transcription with enough accuracy for natural conversation. When precision matters — legal depositions, medical consultations, financial transactions — Voxtral's superior accuracy justifies its premium pricing. On the output side, Orpheus provides the natural, expressive voices users expect from professional services, while Kokoro offers clear, efficient speech for high-volume informational use cases.

Consider a healthcare scheduling bot: It must accurately capture medication names, understand accented speech, and respond with appropriate empathy. Quality failures at any layer break user trust and force expensive human escalation.

Scale: Consistent performance under production load

Voice infrastructure that performs well in demos but fails under production load creates a trust problem. Users who experience degraded service during peak hours learn to avoid the system entirely.

Our infrastructure maintains performance as load scales. A unique optimization in our WebSocket implementation allows multiplexing multiple conversations through single connections — critical for platforms like contact center software handling hundreds of simultaneous calls. Instead of managing thousands of individual WebSocket connections (with associated memory and networking overhead), you can efficiently route multiple isolated audio streams through shared connections.

This same approach to scale applies across our stack. Geographic distribution ensures low latency regardless of user location. Automatic scaling handles traffic spikes without manual intervention. The result: voice agents that perform identically whether handling 10 or 10,000 concurrent conversations.

Try it now

    
    from together import Together
    import asyncio

    client = Together()

    async def handle_conversation():
        # Listen to user input
        transcription = client.audio.transcriptions.create(
            model="whisper-large-v3",
            file=audio_stream,
            language="en"
        )
        
        # Generate response while monitoring for interruptions
        async with client.audio.speech.create(
            model="canopylabs/orpheus-3b-0.1-ft",
            input="I can help with that-",
            voice="tara"
        ) as tts:
            
            audio_playback = asyncio.create_task(play_audio(tts))
            
            # Simultaneously monitor for user interruptions
            async for chunk in transcription:
                if "wait" in chunk.text.lower() or "actually" in chunk.text.lower():
                    audio_playback.cancel()
                    # Adapt to new user input
                    break
    

Get started:

• Playground: Test audio models before building
• Speech-to-Text Documentation: Complete API reference for transcription
• Text-to-Speech Documentation: Complete API reference for voice generation
• Model library: Complete model specifications

For production deployments:
Contact our sales team for enterprise options and dedicated infrastructure.