Throwback Thursday to October 2021. I walked into Deepgram with two years of webhooks behind me and a comfortable story I had told myself for years: I know APIs, I have built developer tooling, and this is the same job with different endpoints.
It was not the same job. It took about a week to find out.
What I was doing then
At Vonage I lived in request/response. SMS, voice, webhooks, the lot. The day ran on ngrok tunnels, one terminal tab per project, every SMS delivery receipt and incoming call event hitting a local server through a public URL. I kept files of webhook payloads for every Vonage API event. When a callback misbehaved I replayed it with curl. When an integration would not behave I pointed the client at httpbin and watched the request come back as JSON. I built Vue demos on top of that model and never once questioned it.
The model is the comforting part. Every API call ends. You send a request, you get a response or a status code, and the transaction is closed. State lives in your database. Errors arrive as numbers you can look up in a table. The connection is a detail the client library handles, and nobody thinks about it twice.
That is also where I started the TIL habit. The first post went up in September 2019, about wscat, a WebSocket testing tool. I had spent an hour wrestling raw sockets against a WebSocket endpoint before a teammate showed me the tool existed. One command and the connection was open. I wrote it down while the context was warm: 200 words, three code blocks, publish. Then ngrok, then jq, then one a month for years.
The moment it broke
Deepgram is speech-to-text. When I joined, the SDKs were functional but basic: Python and Node.js clients wrapping REST endpoints, with thin WebSocket support on top. The docs assumed developers already understood streaming audio. Most of them did not, and I was one of them.
My first real streaming integration showed me why. The WebSocket did not close. It stayed open for minutes, audio arriving in chunks, transcription results flowing back the other way at the same time. No response to wait for, no 4xx code to catch. When something went wrong the socket just dropped, and you found out because your stream went quiet.
The invisible problem was worse than the new lifecycle. A JSON payload you can read: field names, values, nesting, all visible. An audio file tells you nothing. Its format lives in binary headers, sample rate, bit depth, channel count, encoding, and none of it is visible without a separate tool.
Here is the failure mode I met a hundred times after that week. A developer records audio on their phone, uploads it, gets silence or an error. The phone saved 48kHz AAC in an MP4 container, and the API expects 16kHz 16-bit mono PCM WAV. The developer cannot see the difference because there is nothing to see. Both files look like normal audio in a file manager, and the first error message does not explain any of this.
Most developer problems in voice AI are audio problems, not API problems. I did not know that yet. I was about to spend four years learning it.
How it felt
It was humbling in a way I had not felt since my first dev job. I was the developer advocate, and I could not explain why someone's audio would not transcribe. I would ask for the file, run ffprobe on it, and only then understand the problem.
The imposter feeling was specific. I had built a career on knowing how APIs work, and the ground had moved. The skills still applied at the edges. The core assumptions did not. For the first time in years I was the beginner in the room, and everyone could tell.
I have written about the technical half of this before, in SDKs for Streaming APIs Are Different: the moment I realised everything I knew about SDK design was built on an assumption I had never questioned, that every API call ends. The personal half is simpler. I had stopped being confused for long enough to forget what it felt like. Audio fixed that in a week.
What I learned
Streaming APIs need a different SDK grammar. Connection lifecycle, reconnection, backpressure, graceful shutdown. Those are not edge cases, they are the product. A REST SDK hides the connection. A streaming SDK has to manage it as first-class state, because the connection is the thing being sold.
Invisible state is the hardest problem in the category. Audio format, connection state, timing. You cannot debug what you cannot see, so you debug with tooling, and most developers do not have that tooling until after the first burn. This is why I keep writing about ffprobe and curl -w. The tools that make invisible state visible are worth more than any SDK feature.
Being the beginner again was the best thing that happened to my career. DevRel is an empathy job. You cannot empathise with a developer stuck on a problem you have never been stuck on. I had forgotten what it felt like to not know, and audio reminded me. The questions I answer now are better because I remember being the one asking them.
The TIL habit carried me through the transition. One post a month, one thing I actually learned that week, documented while the context was warm. When the context was an entire new domain, that discipline was the difference between flailing and learning in public. The Deepgram TILs are heavy on audio inspection tools for a reason.
Where it landed
I still carry the lesson at Speechify. Voice agents are pipelines: ASR, LLM, TTS, wired together. Every layer has the same shape as that first streaming socket. The state you cannot see is where the failure lives: wrong audio format, dropped connection, silent gap, model version changing under running clients. That is why I write about Speechify-Version date pinning, latency breakdowns in every response, and validating formats before they hit the wire. Five developer experience wins in voice AI tooling is that lesson written down.
The moment my API knowledge stopped being true was the moment my career got interesting. If you have not had yours yet, it is coming, and you should look forward to it.
FAQ
Why did streaming audio break your REST mental model?
REST assumes every call ends: request, response, close. Streaming audio keeps the connection open for minutes or hours, sends data in both directions at once, and reports failures as dropped sockets instead of status codes. The lifecycle, error handling, and state management patterns that work for HTTP do not transfer.
What is the difference between 48kHz AAC and 16kHz PCM WAV?
They are different encodings and different sample rates. The phone file is compressed audio at 48,000 samples per second in an MP4 container. The API expects uncompressed PCM at 16,000 samples per second, 16-bit, mono, in a WAV container. A file manager shows both as audio files. ffprobe shows the difference in one command.
How do you inspect an audio file's format?
ffprobe shows sample rate, bit depth, channel count, and codec. soxi does the same for WAV files. Run either before sending audio to an API and most format failures become obvious before the API ever sees the file.
What changed in the SDKs because of this?
Connection lifecycle became a first-class concept: connect, disconnect, reconnect with backoff, graceful shutdown. The SDK gained state tracking for every connection state, because send, buffer, and drop behave differently depending on whether the socket is connecting, connected, or closed.
Does the lesson still apply to voice agents?
Yes. A voice agent is ASR, LLM, and TTS in one pipeline, and every stage has state you cannot see. Format errors, dropped connections, and silent failures live in the same invisible layer. Version pinning and latency breakdowns exist to make that layer observable.
Top comments (0)