Repository navigation
Reference architecture samples best practices
Microsoft has official Foundry Voice Agent Python and JavaScript samples for creation, management, real-time audio, tools, conversations, and telephony.
Microsoft's Voice Agent resource repo contains explicit handoff and subagent examples, including finance and service scenarios. finance sample
Recommendation: Prefer Speech-to-Speech for conversational experiences. Use Cascaded STT → LLM → TTS when additional model, voice, locale, or transcription control is required.
| Architecture | Use When |
|---|---|
| Speech-to-Speech | Low latency and natural, real-time conversation are the priority |
| Cascaded STT → LLM → TTS | You need model/voice flexibility, custom voices, specific locales, or transcription control |
- Native real-time model combines audio understanding and generation.
- Preferred for highly conversational experiences.
- Optimizes for low time-to-first-audio and natural interaction.
- Separates speech recognition, reasoning, and speech synthesis.
- Provides greater flexibility across models and voices.
- Recommended when custom voice, locale coverage, or transcription control outweighs the additional pipeline stages.
Speech-to-Speech = optimize for conversational experience and latency.
Cascade = optimize for control, customization, and model/voice flexibility.
Source: Microsoft Foundry Voice Agent best practices
Foundry voice traces represent complete voice conversations, with turn events, transcripts, voice-pipeline operations, model/tool calls, STT/TTS, VAD, latency, and optional audio references. Conversation API sample
Foundry uses OpenTelemetry conventions for agent tracing.
Foundry Voice Agent traces → Application Insights: Voice conversation traces automatically flow to the Application Insights resource connected to the Foundry project. They can contain turn-level spans for model calls, tools, STT, TTS, VAD, TTFA, and interruptio
It should be possible to export trace from agents and import into HoneyHive.
If customer has such need to build more pro-code voice agent, they can do as below:
- Port prompts, model selection, and tools. Adapt instructions, select a supported model, and reconnect tools.
- Reconnect knowledge and data sources. Restore access to documents and business systems.
- Map topic flows to specialist agents and handoffs. Define each agent’s responsibility, handoff conditions, and context to pass.
- Configure voice and channels. Set the voice, languages, interruption behavior, and application or phone integration.
- Validate and migrate traffic. Test conversation flows and actions, then gradually switch customers to the Foundry agent.