Prompting Gemini Live and Voice Mode

TL;DR: "Gemini Live" is two different things: a voice-conversation feature in the consumer Gemini app, and a separate developer API for building your own real-time voice apps. Both change how you should prompt, because voice removes formatting, adds real interruption, runs on a latency budget, and caps how much you can hand it in one turn.

Is "Gemini Live" the app feature or the API, and does it matter?

It matters more than almost anything else on this page, because the two products do not share a feature set.

Gemini Live, the name most people mean, is a feature inside the consumer Gemini app. Google's own description: "Gemini Live is a feature that lets you talk with AI from Google using just your voice. Instead of typing out prompts, you can have a natural, back-and-forth AI voice chat directly in the Gemini app" (Google, Gemini Live overview, accessed 3 September 2026). You tap an icon, you talk, and the product decides everything else: which model runs, what it remembers, which of your other Google apps it can touch.

The Live API is a separate, developer-facing product inside the Gemini API, currently in preview. Google's framing: "The Live API enables low-latency, real-time voice and vision interactions with Gemini. It processes continuous streams of audio, images, and text to deliver immediate, human-like spoken responses, creating a natural conversational experience for your users" (Google, Gemini Live API overview, accessed 3 September 2026). Nothing about the consumer app's memory or its access to your Gmail transfers here. You get a stateful WebSocket connection, you choose the model, and you configure every behaviour yourself: how sensitive interruption detection is, whether the model can decide not to respond, how much it "thinks" before speaking.

Confusing the two produces bad advice in both directions: prompting tricks that only work inside the consumer app's fixed feature set get passed off as API behaviour, and API configuration knobs get recommended to people who have no access to them. Everything below is labelled by which one it applies to.

What can the consumer Gemini Live feature actually do?

More than a phone call with a chatbot, and less than the full Gemini app.

You can interrupt it freely and it will follow along; Google's own overview page says you can "feel free to interrupt or change the subject whenever you want". It also carries memory across sessions in a way a plain chat does not: it "can securely remember key details you mention across different sessions, like dietary preferences, important family dates, or your favorite hobbies, so you don't have to keep repeating yourself during future chats" (Google, Gemini Live overview, accessed 3 September 2026). And it reaches into other Google products on your behalf: searching Gmail and Drive, checking Calendar, adding to Tasks or Keep Notes, looking up Maps directions or flight details, or playing a video on YouTube, all from a spoken request.

It also has real, published gaps. Google states directly: "Gemini Live does not have access to Gems or Notebooks. Additionally, Gemini Live does not support Omni or Lyria" (Google, Gemini Live overview, accessed 3 September 2026). If your workflow depends on a saved Gem's custom instructions, voice mode will not carry them; you would want Gemini Gem instruction templates for that mode instead, and switch back to typed chat when you need it.

Two practical consequences follow directly from that feature set. First, because it can act on real accounts with no visible draft to review before it sends anything, a vague spoken request is riskier here than the same vagueness in a typed prompt you can proofread before hitting enter; say the account, the recipient, or the calendar explicitly rather than trusting it to infer the right one. Second, because it remembers details across sessions by design, a one-off correction spoken in the moment ("actually, ignore that, I don't eat shellfish") is worth stating plainly rather than glossing over, since it may be retained the same way a preference would be.

What is the Live API, and why would a developer reach for it instead?

Because it is the version you can actually configure, at the cost of building the client yourself.

The technical shape is concrete: input as raw 16-bit PCM audio at 16kHz, images capped at one frame per second, and text, over a stateful WebSocket; output as raw 16-bit PCM audio at 24kHz (Google, Gemini Live API overview, accessed 3 September 2026). Google's published use cases give a sense of the range this is built for: shopping assistants and support agents, interactive game NPCs, voice interfaces for robotics and smart glasses, patient-support health companions, and real-time spoken translation. None of that is available through the consumer app; you build it against the API directly.

Two model versions are current as of this writing, and the differences matter for prompting, not just for infrastructure:

The thinking-control row is the one that changes how you should write for it. Google states that the newer model "Defaults to minimal to optimize for lowest latency" (Google, Live API capabilities guide, accessed 3 September 2026). A system instruction that demands careful multi-step reasoning before every spoken turn is asking the model to work against its own default, and on 3.1 Flash Live you have to explicitly raise thinkingLevel to get more deliberation, trading it for latency.

Two Google pages disagree with each other on a fact many developers would treat as load-bearing. The product overview advertises "Converse in 70 supported languages." The capabilities guide instead lists "the following 97 languages" with a full BCP-47 code table underneath. Both are Google's own pages, fetched the same day. State which page you are citing rather than picking one number and presenting it as settled.

How is prompting a live voice conversation actually different from prompting text?

Four concrete differences, not a vague sense that "voice is different."

Why can't you lean on formatting?

Because the output has no visual surface. A native audio response is speech, and speech cannot render a bulleted list, a bolded warning, or a markdown table; asking for one produces a spoken description of formatting rather than formatting. Ask for structure the way you would over a phone call instead: "give me three options, and pause after each one," or "say the most important thing first, then explain why." If a written, structured artifact is actually the goal, request it through the transcript channel rather than the spoken one. The Live API supports this directly: setting output_audio_transcription in the session config returns a parallel text transcript of what the model said, alongside the audio, so you get both without asking the audio channel to do a job it cannot do (Google, Live API capabilities guide, accessed 3 September 2026).

config = {
  "response_modalities": ["AUDIO"],
  "output_audio_transcription": {}
}
# The model still speaks. You additionally receive a text transcript
# of what it said, for anything you need in written form afterward.

How should you prompt around being interrupted?

By writing as if the interruption is real, because it is. Google documents the mechanism directly: "When VAD detects an interruption, the ongoing generation is canceled and discarded. Only the information already sent to the client is retained in the session history" (Google, Live API capabilities guide, accessed 3 September 2026). There is no resuming a half-said sentence. That changes how a system instruction should shape a response: front-load the actual answer, then the supporting detail, rather than building up to the point through three sentences of preamble that a user may never let finish. It also changes what "interruptible" costs you: users can interrupt "at any time for responsive interactions", which is the entire design intent, not a failure mode to prompt around.

If you are configuring the API yourself, the turn-boundary timing is also tunable and worth getting right before you touch the wording at all: Google's recommended silenceDurationMs sits at 500 to 800 milliseconds. Set it too low and, in Google's own words, the system "ends speech turns during natural pauses, splitting a single utterance into multiple small audio fragments." Set it past roughly 2,000 milliseconds instead and the conversation measurably feels slower to wait for (Google, Live API capabilities guide, accessed 3 September 2026).

Why does keeping instructions short matter more here than in a text prompt?

Because latency is a stated design default, not an incidental slowdown. The newest Live model ships thinking depth "minimal" specifically to keep responses fast, and every extra clause of reasoning you demand in a system instruction is spent against that same budget. In text prompting, a longer, more carefully qualified instruction is close to free. In a live voice session, it has a latency cost attached that a typed prompt does not, so the practical move is to write shorter, more direct instructions here than you would for the same task in text, and reserve deep, multi-step reasoning for the cases that genuinely need it.

Why can't you just paste in a long reference document?

Because the session has hard limits a typed chat window does not surface the same way. Google states plainly: "Audio-only sessions are limited to 15 minutes, and audio plus video sessions are limited to 2 minutes" by default, and the context window itself caps out at "128k tokens for native audio output models" or 32,000 tokens on other Live models (Google, Live API capabilities guide, accessed 3 September 2026). Video frames sent into a session are additionally capped at one per second. None of this is arbitrary stinginess; it is the cost of running a continuous real-time stream rather than a single request-response call.

The practical fix is the same on both sides of the product line. If you are building against the API, summarize a long document before the session rather than reading it in verbatim, or reach for retrieval so only the relevant passage enters the context window for that turn. If you are a consumer talking to Gemini Live, the product itself is built around this limit and hands you the exit: it lets you "seamlessly flow between talking and typing without missing a beat." Drop back into the typing window for anything that genuinely needs a long pasted block, rather than trying to read it aloud.

How do you write a system instruction for a voice-first persona?

Specify the things text prompting lets you assume and voice does not: turn length, what to do when cut off, and how to signal structure without formatting.

System instruction for a Live API voice session.

You are speaking, not writing. Keep each turn to one or two short
sentences before pausing for the user to respond; do not deliver
long monologues. State the direct answer first, then add detail
only if the user keeps listening. Never use markdown, bullet
points, or headings; if you have a list, say "first... second...
third" instead. If the user starts speaking while you are mid-
answer, stop immediately and address what they just said rather
than finishing your previous thought. Do not apologise for being
interrupted; treat it as a normal part of the conversation.

For a consumer talking to Gemini Live rather than configuring a system instruction, the same discipline still applies to how you phrase the request out loud: ask for one thing per turn, say the account or app you mean by name when the request could touch more than one, and if you actually want a written result rather than a spoken answer, say so directly, since asking for a fully formatted, structured output on the voice channel does not translate into the written result you have in mind.

Spoken to Gemini Live, not typed.

"Check my calendar for tomorrow and tell me my first meeting,
just that one, don't read the whole day."

How do you prompt around language and voice choice?

Directly, in the case of voice, and less directly than a text prompt would suggest for language.

Changing which prebuilt voice speaks is a configuration field on the API, not a prompt: set voiceConfig.prebuiltVoiceConfig.voiceName in the session's speechConfig (Google, Live API capabilities guide, accessed 3 September 2026).

config = {
  "response_modalities": ["AUDIO"],
  "speech_config": {
    "voice_config": {"prebuilt_voice_config": {"voice_name": "Kore"}}
  }
}

Language works differently than it does in a text call. Google's own capabilities guide states that "Native audio output models automatically choose the appropriate language and don't support explicitly setting the language code" (Google, Live API capabilities guide, accessed 3 September 2026). There is no language parameter to set the way a text-generation call would take one; the model infers the language from what it hears, and if you need to restrict it, that instruction has to live in your system prompt rather than in a request parameter. That is a good general lesson for the whole topic covered in more depth in why does the AI answer in English when I asked in another language: a missing parameter is not the same problem as a missing instruction, and the fix differs accordingly.

Where does this connect to prompting Gemini outside voice mode?

Everywhere the input stops being pure speech. The Live API accepts images at up to one frame per second alongside audio, and the consumer app pairs its camera with Nano Banana so you can show Gemini what you are looking at and ask it to reimagine the scene. Once a second modality enters the conversation, the prompting rules shift again, and that is deliberately a separate piece: multi-modal prompting with Gemini, across image, video and audio in one request covers what changes when the input is not text at all. For everyday, typed Gemini prompting, Gemini prompt templates is the wider library this page's voice-specific advice sits alongside.

The core discipline stays the same whichever surface you are on: know exactly which product you are prompting, write for the constraints that product actually has rather than the ones text prompting trained you to expect, and check Google's own current documentation before you promise a client a capability, because a preview product with two model versions and two disagreeing pages of its own is not a stable target to memorize once and reuse.