The problem
A wholesale sales call involves more than answering questions. A conversation can require customer verification, inventory lookup, product sharing, order creation and payment follow-up. The engineering task was to connect those steps to a voice interface while retaining a path to a human.
My contribution
I built the real-time phone-sales pipeline and its business integrations, including function calling for inventory, orders and customer verification, WhatsApp product sharing, Razorpay payment links, tracking, and human escalation.
The surrounding application included a dashboard, analytics, CRM and inventory workflows.
Engineering choices
The stack combines Azure OpenAI GPT-Realtime, FastAPI, FreeSWITCH, WebRTC, voice activity detection, Whisper and SQLite. The speech layer supports the conversation; function-calling tools connect the agent to inventory, orders, and customer data.
- Function calling provides an explicit interface for inventory, customer and order operations.
- WhatsApp supports visual product sharing alongside a phone conversation.
- Razorpay links provide a separate payment step rather than collecting payment details in speech.
- Human escalation is part of the workflow when a conversation needs staff involvement.
Proposed next step
Review scenarios involving interrupted calls, unavailable inventory and escalation, with explicit checks before order-changing actions.