I had preview access to this one for a few weeks. It's very good. I had one conversation that lasted a full hour while I was walking the dog, got some good brainstorming done against one of my projects.
The best feature is that it can delegate questions out to GPT-5.5 in the background, so you're no longer restricted to a voice model that's several years behind the frontier.
I did report a fun bug with it though: it was interrupting me and laughing at my (not really intended as) jokes while I was still talking! They seem to have clamped that behavior down thankfully, it felt a bit rude and condescending.
Is it responsive to personality settings? I actively don't want fake AI girlfriend, but I do get a ton of value out of voice mode. Looking forward to trying this but hoping it's not a creepy overdone mess (like Sesame). Expectations are they'll keep doubling down on fake AI girlfriend approach because the thing I want probably wouldn't drive engagement anywhere nearly as well
I have my Chat personality settings stripped right down to no-fluff. I'd want voice to be more akin to the Star Trek computer, and less akin to as you said an AI friend, but previously it was tuned too personable/friend-like.
Star Trek computer voice model is something I have yet to encounter, and I've looked repeatedly :) It's not about a specific voice, it's the fact they managed to capture "I am a utility" perfectly in the voice. Our modern friends do not want to be thought of as a utility, but to engender trust and agency all of their own and that's a huge problem for me.
Prompt text: There are 14 varieties of tomato soup available from this replicator. With rice, with vegetables, Bolian style, with pasta specify hot or chilled.
I'm just going to say, Wow. That's a pretty incredible result. I had no idea you could clone a voice with such a short snippet of audio these days.
I've had this dream of talking to the Enterprise-D computer since I was 8 years old. Modern me still has that dream, but to also have it hooked up to Home Assistant so it can actually do useful things. A couple months back, I went looking around for "clean" samples of Majel's voice as the computer but didn't have a lot of luck. Even though there are three television series and several movies, pretty much all of them have some amount of background noise, bleeps, bloops, or warp core thrum. (As this one does.) There may be modern ways to clean those up without affecting her voice much, but I haven't dug into that yet.
There are a few audiobooks narrated by Majel but obviously her role as the computer was proper voice acting and so the books would not be good source material.
There were also a few games/CD-ROM (Omnipedia) with some samples, but they did not bother to post-process them for that lofi Enterprise-D computer feel. Can _probably_ be replicated fairly faithfully after the fact, but I only know a _little_ about audio post-processing.
I think the request here is not about sounding like Majel Barrett but in keeping the output extremely terse and unobtrusive.
There's been a few studys showing that novices love LLM output that's long, but experts hate it. As an example, I've been tasked with using some agentic PM tool to write specs, and it keeps generating these huge page long outputs with "HBR voice" bolded summaries of paragraph long bulletpoints. I.e.:
> Right-size hard, and watch the one open-ended edge. Endorse the DRI's simplifications wholesale: drop the runbook-per-alert mandate (keep 1–2 diagnostic-only runbooks for the high-priority set), and ride durability on the existing weekly incident + monthly operational reviews — no new governance. The single scope-creep risk is the coverage strand (gaps are defined by absence); bound it to gaps evidenced by real, already-missed customer-facing outages, not a proactive gap hunt. Curing ownership gaps (e.g. foo-bar, no clear owner) is finite in-scope work.
There's dozens of these every iteration. I can't imagine trying to deal with that via voice, I would just zone out after the second sentence.
I think that's mostly just a frequency shift :) You could probably recreate it with another model and some effects on top. Also, why the hell not for voice mode haha.
One popular speech synth from back in the day, I believe it was WillowTalk, had a voice called Colossus, which sounded like the voice module of the computer from Colossus: The Forbin Project. This voice was used for that of CATS in the famous "All your base are belong to us" Flash video.
Another WillowTalk voice was a clone of DECtalk's Perfect Paul good enough to be used as the voice in the MC Hawking rap recordings.
Seems a bit knee jerk. I go on walks and bike rides all the time. A couple of times I’ve used voice mode and it’s been interesting. I could have listened to music or a podcast, listen to YouTube or just unplug. But every walk is different.
If you’re uncomfortable with this new world, and I’m sure I am even as I participate, you could tell us more about that?
Given the personality type common on HN, I imagine that the GP, even if unplugged from all technology on their walk, wouldn't be in a mindful state of enjoying their surroundings, but rather would be "lost in the clouds", stewing on the same ideas/thoughts/problems; but with those thoughts going more in circles, due to a lack of ability to verify anything.
Literally me. Everyone is different, and that's fine. But I don't have the privilege of living in an area where I can talk to people about the things I am thinking through. It's very rural. Having a _utility_ that can act as a sounding board while I spew out my thoughts on a walk is a really meaningful improvement to my current situation.
> "The best feature is that it can delegate questions out to GPT-5.5 in the background, so you're no longer restricted to a voice model that's several years behind the frontier."
Wow. That's exactly what I hoped they would do.
This issue has held me back from using ChatGPT's voice mode as much as I otherwise would have, because I also use it for brainstorming while commuting, exercising, etc., and don't want it to feel stuck in the past.
Funnily enough - I built this (delegation) over the weekend with Fable for a local voice chat running 100% on local LLMs, Parakeet and Kokoro. I say "...ask the thinking model..." and that redirects it to Qwen 3.6 27B on vLLM.
Can't claim originality though - it was inspired by Sesame - where their models will invoke a search, or check the weather etc, and make a vocalisation to keep you engaged.
Turn taking is one of the hardest things to get right for the exact reasons mentioned - but does seem to be the way that Claude.ai's voice works - in a very obvious way.
Anthropic + OpenAI both rug-pulled voices I liked and got used to and OpenAI really dumbed down their voices at the same time - Arbor went from Estuary English and almost "jack the lad" to some generic English accent. Claude had a Birmingham accent and said things like "shit", ending sentences like "So you're telling me that they asked for a 90% discount yeah?" - then it changed overnight to a mock Derbyshire accent with a dull tone.
ChatGPT's voice also gaslights me for conventional opinions - "my Eastern European neighbour helped me lift a wardrobe upstairs - something you just can't ask your typical neighbour neighbour"... then you get a full on left-leaning lecture from the safety layers rather than a head nod or "what luck!"
Claude + Sesame are nowhere near as overbearing.
In both cases - from edgy and engaging to something that just didn't gel.
The point of making my own assistant is that I can talk for as long as I want, episodic memory is personal and private, there's no "trust me bro, we're a big corporation" vibes.
This was not my first attempt - when I had a bunch of Opus credit around Jan/Feb - I tried really hard and created something that was not good enough. What I have now, is working, and each session is training Claude/Codex on what to tune, and to fix.
"Just had a convo - can you look into what happened?" And if it's one I don't mind sharing with the model - I'll say, "and what did you think of the questions I asked?" Sometimes it'll give a lovely commentary on how the model did.
af_heart is probably the smoothest voice - but yes it's more like another commented - more "StarTrek" than "telesales assistant that pauses and laughs at your jokes".
If you're on a similar path and want something full duplex - the go to solution is PersonaPlex from Nvidia based upon Moshi.
I love the UX of voice mode. I’ve always hated voice models.
Gemini’s being particularly egregious (always ending in some deranged question, can not reliably be prompted away) prompted me to build my own client for my real harness that simply does STT -> model -> TTS (both being independently useful).
I guess I see some value in a model responding quickly and with more nuance, but it’s not much. I can wait for it to finish. I’d much rather have it be actually useful. I’m not looking for a digital friend.
The delegation feature lets me see some value in a voice model for orchestration type features. But in either case, I don’t really like (or understand why others would like) talking to a model with different features and quirks just because I’m using a different medium to communicate over.
Is it a dumb-down version of GPT like the current voice model? At least in french, I find the current GPT voice mode to be useless, to the point I only use the dication mode. I would ask a question and it would answer something along "That's a interesting question. I can help you with that. Anything you want to know about X?" I would ask again and it would answer the same kind of non answer.
> the answer is no it uses the latest gpt models now.
Actually it says it _can_ delegate to the latest models. Seems reasonable to ask how the voice model does when it doesn't delegate (or while waiting for the delegated answer).
I haven't stress-tested it, but I would imagine it approaches complex problems the same way a human with a phone in their pocket would — that is, by having a degree of awareness of the confidence it has in its own knowledge in some areas; where, when it "realizes that it doesn't know", it blocks the conversation with statements like "I don't know, let me check."
I say this because this is already how ChatGPT works internally when using its "auto" mode; the version of the "fast" model used in the "auto" mode does the same "notice your ignorance and bring in the heavy model" thing, just silently, rather than mentioning that it's doing it.
(If someone has actually run the experiment, please chime in!)
> I did report a fun bug with it though: it was interrupting me and laughing at my (not really intended as) jokes while I was still talking! They seem to have clamped that behavior down thankfully, it felt a bit rude and condescending.
Some times I'm still amazed that AI "gets" humour much more effectively than the character of Data did before his emotion chip, e.g.: https://www.youtube.com/watch?v=G4VZ5kQIdV0
Other times I'm amazed in the opposite way, that the script writers cover basically the same talking points about the character as we have today about LLMs, e.g.: https://www.youtube.com/watch?v=GBJCYHwyZhw
>The best feature is that it can delegate questions out to GPT-5.5 in the background, so you're no longer restricted to a voice model that's several years behind the frontier.
Ahh, this makes sense. I was wondering when they would start doing this. I stopped using voice mode all together because it was frustrating talking to a dumb AI, when most of the time I discuss things with Opus 4.8 or gpt 5.5.
I was working on a phone call agent recently, and thought about doing this. It makes sense
I have friends who have brainstormed with an LLM (voice chat) for 10-30 minutes, and reported very positive experiences.
When I speak to one - while I'm impressed at how far they've progressed - the LLM just doesn't talk like someone I'd want to discuss a technical problem with (the way I would with a human).
(And my friends aren't even using a custom prompt - some of them are just talking to the default Gemini on their phone!)
I also love exploring ideas like this. The problem that made me stop using Gemini is that it would always try really hard against all my prompting to ask a stupid question at the end of its response that would completely derail my train of thought.
Does it have full access to your chat history, project files, etc? That's the biggest limitation I have with voice mode right now, if I ask it about something I chatted about before (even in the same conversation) it has zero recollection of it.
You can start a chat in an existing session after pasting data into that session and it can then talk about that content. I haven't tried it with projects.
Dammit! I live in Europe and the cyber truck isn't even available here..
That said a huge pickup truck is about as far as you can get from a Camaro... Then again I'm not exactly David Hasslehoff myself either... Meh if it talks that's close enough!
About 2 years ago I used to have conversations with GPT while walking the dog. It really emphasized the need to think before you speak, but you had to think fast before it hung up on you.
That is a nice idea and all, but perhaps there are things our friends aren't good at talking about. Perhaps something that is very related to a work project? Or like when I go down the rabbit hole discussing quantum mechanics and astro physics with ChatGPT because none of my friends understand that at even my level, and I'm looking to learn more about it from someone/something that does know more than me.
I'm looking forward to trying this out because I find that doing discovery on certain things feels more natural with voice. The previous voice chat was such a dumb GPT-4 era model that I resorted to using the dictate feature with GPT-5.5 and had it read the text response back to me.
This is the opposite direction AI should be going. Human relationships are the most valuable thing we have, and so, naturally, technology seeks to intermediate and now replace them.
Most of what AI does is already in wrong direction. Not just human-to-human interaction, it took away thinking, creative work, sensory perception (glasses) and responses. People call it as helping humans, but I call it as sucking away the "human-ness" from humans.
After the damage is done, the mega corps would simply shrug and will say "Well, we were just responding to our business competition" The business knows no human-ness, because it is not a human. Businesses and machines are creatures that see humans as their fodder. And humans created these, assuming it is progress, to have businesses and machines. We call it progress because it required our mind power and it helped us to dominate other species. Dolphins are laughing at us.
I don't understand this line of reasoning. How are you hindered from doing any of those things? What part of "AI can now do X" makes it so you can't also do X?
Fewer people aren't staring into their phones or talking to them -- makes your social antennas pick up automatically on not wanting to disturb them (lest you draw their ire for not having the social antennas long enough to pick up on the fact they're "busy and don't want to engage with you" like a gymrat with AirPods to signal they're there to pump in peace and quiet listening to their favourite playlist, not talk to strangers). Happened to me already many times just with people scrolling their phone instead of talking and not wanting to talk in particular either, not to me at least. And no -- I am not talking about bothering strangers in the gym etc, I am talking about sitting at the lunch table where half of the people look into their phones -- they aren't actually interested in talking, it turns out.
Our devices have now increased the distance _between_ us -- it's not about _you_ being able to "do X" -- talking to others is not _you_ doing it, it's you _and the other person_ doing it _together_. You can't be doing anything together consentually when the other person is in the habit of talking with their AI, or doomscrolling for that matter.
There are still lots of social people. I found a lot of people actually do want to talk but are just shy.
I spent a few weeks at a hostel last year. It was always kind of depressing and tense in the shared kitchen, just this heavy silence.
I don't feel comfortable around strangers, so I solved that problem by just saying hi to everyone.
Most people didn't respond much, although most of them smiled and the tension was eased. But a few of them struck up conversation and we ended up making friends.
I ended up making like, ten new friends in two weeks. And then a bunch of them ended up becoming friends with each other as well.
This is an individual solution to a systemic problem. On a personal level, it is possible to solve such problems, but generally no, the ship has sailed quite some time ago (I personally think cars are to blame).
Even if you do it, you are still swimming against the current.
>This is an individual solution to a systemic problem. On a personal level, it is possible to solve such problems
I.e. it's only a problem if you're not willing to go and strike up conversations with people. Which is not AI or mobile phones' fault. Expecting society to come up with "systemic situations" to your personal problems is a fast path to a lifetime of disappointment.
In a personal context, you're not hindered from doing those things, so you're correct in that regard. The problem is economic and social. The AI is mediocre enough to drive the value of those things way down; possibly to zero. When something isn't valuable, people are less likely to learn it, and in the future it's less likely that anyone will actually have those skills. For example, we've slopped many illustrators out of jobs and essentially made art (an already awful paying career before AI) economically infeasible. AI illustrations kind of suck though, and even when they're technically competent they're kind of soulless. So you might say, ok, I will hire/contract an artist if I care about the quality of my illustration! Ok, but, if artists are being priced out by a machine then how long before there's no real market for finding a human to do it because all the artists gave up and got a job at starbucks and all you have left are amateurs?
This is almost certainly going to bite us in the ass long term, because eventually without human creativity you're just training AI's on other AI slop, or limiting the possible catalogue of styles to "things that existed before AI ruined every creative job". I guess the question is, what kind of future do you want to live in? One where we have a massive abundance of easy-to-create but vaguely worthless artifacts in a society that's completely devalued being good at something? It just sounds really dystopian to me.
We're yet to see how this plays out, but a competing business model for creative work is emerging, where it's delegated to chatbots. Naturally, this would result in less creative work for humans.
I'm no longer writing code from scratch, as I used to do before. So, very soon, it will be "AI can now do X" makes it so I can't also do X? Same with many creative works. Radio music already sounds so plastic. I lost interest in crafting my text drafts because I can just dump some ugly text and get it refined by AI.
I'm confused. It feels like you are saying you did those things as a means to an end.
Like saying I no longer walk because I can drive. I no longer cook because Doordash exists. I no longer play piano because midi exists.
I mean i guess, but it seems like you didn't LIKE crafting the text or coding from scratch, you just wanted the outcome. If we are talking purely about work, I understand that its about being productive and it sucks to have a job shift to something you enjoy less.
But for daily life? I dont see how it changes, maybe its a tech thing where people think about making their daily lives more productive, but most people dont.
Hasnt this already been thoroughly discussed? Your ability to do X degrades as you offload it more frequently, eventually to the point that you can no longer even vet the quality of the output.
I think the parent is saying now that that is attempting to be applied to "creativity" directly, as opposed to something like a shift of medium, that it threatens many peoples' ability to maintain creative capabilities.
Anecdotally I've already experienced this at work where post-AI we had a junior completely stagnate and a senior with over a decade of experience in the bay atrophy to the point that he had to be let go.
Playing the devil's advocate a little, you say "can't also do", but that implies prohibition, not hindering. Hindering is not total like that.
It's like trying to have meaningful conversations on Twitter. You don't go to Twitter to do things like that. Can you? Sure. It's just not what the format and the conventions (and the people) lend themselves to.
I don't think there's much merit in pretending that human activities are only shaped by hard limits.
If you think AI is bad, wait until I tell you about the horrors of social media who profit on controversy and division, US health insurance which profits on rejecting claims, and big pharma profiting on the opioid epidemic.
And it's not like this is new, either. Upton Sinclair was writing about this stuff a century ago with books like The Jungle.
The only difference between then and now is we "think" we're not evil today. We've lied to ourselves that "We're so much better than we were back then." Facebook wanted to bring people together originally, but they ended up providing the most toxic social media experience known to man. Facebook forgot to tell us they cared more about profit than people.
Please spare me the argument AI is the straw to break the camel's back here. The system has been broken a long time before that.
Counter-point: I love that my rubber duck can talk (quack?) back, as well as record and summarize my thoughts on topics I'm working or stuck on.
I've wanted a good voice mode for precisely this reason. When I take my dog on a walk and I'm thinking about a bunch of problems/ideas, I'd love to have feedback and a record, or perhaps to even kick off research or ask questions to fill in gaps that would otherwise have me debating pulling out my phone to try to get an answer.
> This is the opposite direction AI should be going. Human relationships are the most valuable thing we have, and so, naturally, technology seeks to intermediate and now replace them.
Valid, but, I think, you conflate two separate things.
AI voice mode as a human socialization/conversation replacement? Cringe in my book, fully in agreement with you. Though my opinion on that aspect remains the same, regardless of whether it is done through text or voice.
AI voice mode as an alternative interface to interact with AI-as-a-tool? Great idea imo. There were a few instances where I was either too tired to type or wanted to brainstorm things in more of a freeform mode, for which a well-working voice mode would have been great.
Naturally, the current distinction between AI-as-a-personality and AI-as-a-tool exists purely on the user's end. All I know is that I want the latter a lot, and if some people want to use it for the former purpose, that's not my problem. Sadly, I think that it will be judged more on how an average person decides to use it (i.e., in the most degenerate/reductionist ways possible), as opposed to being judged on the merits of what it can actually be used for by someone who just treats it as a tool.
Yes, every minute you spend texting or talking to a chatbot is a minute that you'd have spent talking to another human beings. Literally the only important thing in life, the basis of all value, the formation of self-identity, comes from communication with other human beings.
It's also hilariously wrong. It essentially argues, implicitly, that those who don't communicate with other humans are missing out on the "most important thing in life" and cannot form a self-identity.
> Literally the only important thing in life, the basis of all value, the formation of self-identity, comes from communication with other human beings.
I think you're mistaking their sarcasm for sincerity... especially considering the emphasis on self-identity ironically juxtaposed as originating from a decidedly non-self activity, which has all the hallmarks of being intentional...
On the other hand, reading their other content leads one to believe that they may, in fact, be serious... hmm...
It is unironically correct. Well, interaction with other human beings may not be the only important thing, but it is certainly far and away the most important thing.
Let's not overdramatize, though. I'm not in need, or even in mood to talk to fellow humans every minute, so time spent with a clanker is not necessary taken from my human relations budget
I also think this undersells the real value of the bot, which is to handle tasks via voice that an average human either would not or could not do.
In the video example with the grannies, the knitter is essentially wanting a PA. Regular folks don't have PAs. Even when that became a thing in the aughts they were all outsourced.
When I've used voice chat, it has often turns into rabbit holes on very niche topics. For example, I had one start about the 1996 performance of Rage Against the Machine in Portland, Oregon that was supposed to feature Wu Tang Clan. (already outside most human's knowledge) that dove into details of the club scene in Los Angeles at the time of RATM's signing to Epic Records.
Was anyone else here at that '96 show in Portland? It seems like it might be challenging to find a person on the internet able to engage on the topic.
The person may exist, but not during my fleeting interest in the subject while walking to the park.
> Yes, every minute you spend texting or talking to a chatbot is a minute that you'd have spent talking to another human beings.
Human beings tend not to be available (results vary by culture).
Also, imagine you're 82 years old and living alone (e.g. widower). It is believed that lack of interaction is a significant driver of cognitive decline (which is why being hard of hearing accelerates the onset of dementia). I wonder if having an LLM to talk to under those circumstances will decelerate cognitive decline?
The extent to which it's true is the extent of the evilness of the technology. Go search the phrase "ai boyfriend" on reddit sometime; imagine what it will be like when society is fully baked with this shit. You're talking to AIs all day at work. You're talking to AI's when you use social media. You're talking to AIs for therapy. You're talking to AIs on dating apps.
If your answer is "well I'll simply touch grass" I agree. But most people won't which is why this is tech is immiserating and, I would argue, evil.
Isn’t voice I difference in degree rather than in kind? I definitely talk to AI as if it were human (one might say the UX of AI is to emulate a human). And a large portion of my interaction with humans is via text, for example, this post!
> Isn’t voice I difference in degree rather than in kind?
There is a difference in expression / emotionality with speaking vs writing. Speaking tends to carry more emotion while writing is generally more deliberate/less-emotional*. Having a voice conversation will be more likely to get a human to engage in an emotional based expression mode, which could increase the chance of "false connections", believing the AI "gets" them or "understands" or "listens". This happens with text too, as some headlines show.
The issue is that while some people are going to "connect" with their AI in text and voice, some who do not make the connection via text may do so via voice because it tends to change a persons expression mode.
> I definitely talk to AI as if it were human
Do you talk to it as if it were a friend or family? or Do you just use natural language to give directives? The distinction, I believe, is in the kind of way we express the "talking".
I talk to AI as if it were a tool that understands human commands and then executes those commands and relays them in a human understandable format. This includes commands to provide options that I may not have covered and explain the options. If I talked to a human this way, they wouldn't be around much longer -- unless they were an employee and even then they would probably be looking for a new job
After reading some of the psychotic break headlines from AI chats, I see some people really do talk to AI as if it were human. Which I would guess includes seeking broad "thoughts and feelings" on a persons situation or asking the AI if their view/side of things is the "right or wrong" side. Basically begging the AI to be responsible for their own thoughts, or simply offloading them and taking what comes back -- which is going to be what they wanted to hear because the entire context would be full of emotion based prompts.
Right, I think the challenge is that LLMs are essentially language-based, which is itself a very convenient interface. Covering the maximally humanistic default interface with something more mechanical is like tying your own shoelaces together, but it would protect us from the psychological hijacking we're so prone to when interacting with these machines.
I disagree. Fluid natural conversational AI is far more productive than any other interface for working with LLMs. Although I suppose you could make the argument that it should be more... Robotic like. Like in StarTrek. Which, is honestly probably better for work, too. A "get shit done" mode, of pure, cold, efficiency.
In one of the videos the AI quips back things like "Happens to the best of us" - its basically pretending to be human and that feels kind of creepy and weird, like it's trying to cultivate a para-social emotional bond.
I just want voice assistants to reliably understand what I say and do what I mean
So hopefully you can turn that off.
There are plenty of applications for that more human conversational style though (from mock interviews, improv practice, learning languages, etc) - I just don't want that for most things.
> This is the opposite direction AI should be going
I don't understand the counterfactual. What's the opposite direction of this direction that's desirable? Less capable voice models are obviously not it, so I am curious what direction you mean or if it's just vague indirection.
I see it more as a way technology could be abused rather than an inherent flaw in the technology itself. If you start to replace human interaction with chatbot interaction, that's bad, but there's nothing wrong with using a human-like chatbot in moderation. So many other types of technology are fine in moderation but can be abused in a human-interaction-replacing way: television, social media, video games, etc.
The thing is, individual moderation isn't sufficient to combat collective capture. This is Ivan Illich's idea of "radical monopoly", where a technology (like cars or the internet) becomes so entrenched that individuals can't realistically opt out any longer. This happens when we abdicate our collective responsibility to the internal logic and incentives of the tech.
Regardless how natural AI becomes I’ve yet to replace any human relationships. Where do you see the aim of technology as trying to replace people? You might be using it wrong.
How would talking to an AI as if it were not human sound? You can probably set your system prompt to insert “beep boop” between sentences and make it refer to itself as “Cybertron9000 Personal Computing Device” if that’s what you like. Is that an improvement? Or are you against voice computer interfaces altogether?
I don't know the answer. But a robotic voice is probably not a bad idea — just having a reminder that the thing you're talking to is not actually anything like you. If you want to go full send, you could have the LLM generate a clickable interface on demand so you could interact with it as a machine. Voice/computer interfaces are obviously useful, especially for disabled folks. But the ones that existed in the past didn't pretend to laugh at your non jokes or imitate vocal fry.
I would also like to tone those down, and prefer if they just focused on creating a nice, clear, natural-sounding voice. I don’t think the “emotional roleplaying” adds anything to what I perceive as a computer interaction. But that’s precisely the opposite of the problem the other commenters seem to be voicing; I have no problem recognizing what I’m interacting with, and I don’t worry about being manipulated - not by the tone of its voice, and even if it had a video avatar, all that. Do you guys seriously have this problem of being confused by LLMs that sound too much like humans, or is this a theoretical problem that you’re worried somebody else is having?
I am open to the possibility that this is a future that is coming, but as far as I’m concerned, we’re years away from that tech. Is it actually here for you?
Aren't they already very cognizant of handwringing like yours? Their article mentions various safeguards and actively steering the model away from being emotional companions and so on. It's a far cry from the OpenAI two years ago or whenever it was when they were entertaining the idea of allowing/enabling adult conversations with their models.
I personally think this is a moralistic regulatory overreach. And they definitely do that due to political pressure too, since there are various bills around the world in various legislatures that want to regulate AIs giving useful advice and being too personal to talk to.
So you can rest assured, I think, at least in that regard. The AI disempowerment will come to us anyway, just in a more sanitized corporate form.
I don't think most adults can get much social value or satisfaction from an AI conversation they know is not a real person. Those who can get that satisfaction likely have so few human-to-human interactions that an AI companion may be as good a solution to chronic loneliness as any.
I'd be more worried about the inevitable robo-nannies who could end up talking more to young children than actual people.
Where do you draw the line on how good human-machine interfaces should be? I'm sure this model could be a convenience for many, and while it may be social for some, I am not sure it would substitute existing human interaction for those users.
Besides, I do not think there is anything inherently immoral with not being social, or not having the ability to be. Consider for example people who do not naturally have the social network to interact with people they want to (e.g. some gifted children).
I am not convinced this model has enough empathy to satisfy most users on an emotional level. A bond is not merely an exchange of words, but prolonged and deep contemplation of the other being. We cannot introspect into these machines, and they certainly cannot yet do the same to us.
> This is the opposite direction AI should be going.
There is no moral obligation, in any domain, to refuse to make a product that adults, with full informed consent, find useful and purchase. Who are you to say you know better than the market?
Markets only work in well-regulated environments, this has been known since forever (Adam Smith).
There is no meaningful competition in a marathon if I can drive you over with a car at the first 100m.
And thanks fkin God that we have regulations and meaningful laws and some asshole for-profit company can't just put drugs into food.. the "informed buyer" is bullshit. Humans are faulty, and there is a billion dollar industry meant to take advantage of said faults: it's called marketing.
Market conditions are not a moral standard either, nor do they represent any cohesive one in particular. Not sure why you're contrasting their opinion with this, it's literally no better.
For important and learning tasks,I would not use the voice feature as it was way too short and 'conversational'. I would use the 'record' feature and have it read the long, articulate answer to me. If this new conversation feature doesn't feel like I'm 'hanging out' with a friend, but actual longer high content answers, It'll win me over. Otherwise I thought the previous conversation mode was way too watered down and I found myself getting frustrated having to keep asking many questions to further probe down to the details. I don't know, these things don't change much, we'll see.
What I’m missing from this announcement is the capability to use connectors and tools. I don’t really get it - NONE of the frontier assistants can use tools / connectors while in voice mode - Claude, ChatGPT, Gemini, Grok. It seems so obvious: I want to be able to research stuff, pull up documents, jot down notes and do productive work while I’m talking to it, and not end voice mode whenever I need to connect to an app or service.
It’s weird. The old Claude voice mode WAS able to use tools but when they revamped it, it lost that capability and is now pinned to Haiku :(
So, yay for finally a voice mode that’s powered by a frontier model and hopefully as good as Grok voice, but sad to still not see tool use while in voice mode.
(I haven’t tried it yet, only read the announcement)
Exactly. I told the assistant to create world cup google calendar entries and it said it can't connect to my calendar, whereas I can have it do that with no issues on the normal gpt client.
Using gpt-realtime-2 you can achieve very similar results with low latency streaming which gives you this feeling of an assistant that is there in the room with you. I made a tweak to so you can try out the gpt-realtime-2 model with tooling that gives you this: https://sippet.ai/?enabler=hn
One of the videos on the announcement page shows someone making coffee and it _appears_ that the agent counts 30 seconds in real-time. Curious to know how they made it do that without tool support.
I’ve been using gpt-realtime-1 for my personal assistant that runs my company and plans my day. And it works pretty well, even makes tool calls and all that.
But the multi modal stuff has resulted in a lot of debugging with weird events and message and audio sequences having race conditions, but overall it is pretty awesome.
Looking forward to moving to this model later today and will chime back in with results.
> I don’t really get it - NONE of the frontier assistants can use tools / connectors while in voice mode - Claude, ChatGPT, Gemini, Grok.
Is that true? I have a friend who often brainstorms with Gemini (I think just on an Android phone), and he has it actually do stuff related to the conversation (including adding content to notes).
In any case, you can always vibe code one with pi!
I could see it relating to tools having unpredictable latency but if they already do background hand off to 5.5 then it seems like they could just enable it within that context.
A big part of this announcement does seem to be _delegation_ in the background; they give the example of web search but that could be any tool. I haven't tried it yet either but sounds like they've found a reasonable UX that mixes that sort of high-and-variable latency tool calling (potentially with agentic loops) with a continuously speaking live voice.
Because they take too long to run, and have an unpredictable latency and success rate. Seeing loading spinners and error messages in a visual interface is fine, but it would firmly put a natural language conversation in uncanny valley territory.
Regular chat already supports voice input, so might as well use that.
Like a lot of AI things, this seems both cool and kind of creeps me out. I've never used voice interfaces in the past (siri, the google one, whatever is on my tv) so I'm probably not the target market, but this does seem like an improvement.
The part that creeps me out is, we're living in an era where we're more disconnected from each other than ever before. Do we really need to be replacing conversations?! The demonstration video of old ladies sort of hints at something for me, which I think we already have a societal problem with the way we treat the elderly (and a massive elderly-loneliness issue) and there's kind of a sadness of imagining people becoming really close with this machine that doesn't really think. Definite ick factor.
GPT-Live-1 is the first version of a new generation of models, and we believe the full-duplex architecture + delegation enables entirely new ways of human-AI interaction.
Hey! Bit of an unusual question maybe: if this stuff further exarcerbates the loneliness epidemic and atomization of society, will you be able to live with yourself you think? If you hear about teenagers only spending time with your chatbot in 5 years, will you feel some amount of personal responsibility or not? Always curious to hear you guys' perspective on that kind of stuff!
> if this stuff further exarcerbates the loneliness epidemic and atomization of society, will you be able to live with yourself you think?
Looking at the 30,000-foot view of how society is set up: laws, economic system, employee incentives, etc, do you suppose it matters what the individual contributors think? I say this not to absolve anyone of responsibility, but to point out the obvious outcomes of our incentives across the strata (polity -> shareholders -> boards -> C-suite -> employees)
I will bet you dollars to donuts, somewhere inside OpenAI is a frequently-used revenue dashboard, but not for loneliness - if anything, OpenAI will make horny models and tout itself as a solution to loneliness, a la character.ai - if that earns them more money.
Technology is what you make of it. If you want to make it your best friend and your only friend, that's your choice. But I would guess more people would use it as a personal tutor. If it works as well as the demo, this already crushes most language learning apps. Actually the only apps for learning language that it would fail to replace would still be Tandem and HelloTalk because it still can't replicate the nuance of real human interaction.
"Blame the individual" doesn't really make sense when we're talking about things that have a society-wide impact. As an analogue, personally I'm not on social media, but I am affected by social media because everyone around me is on social media and their happiness and wellbeing also impacts me. AI companies need to stop pretending they can just outsource responsibility for their products.
I think assuming people will use it as a tutor/learning tool is.. way too optimistic. A small fraction will, but the majority will just view things like a second language as something not worth learning.
That's not really a fair question, especially given the audience here on HN. Most here have blood on their hands from the previous 25 years of tech-eats-the-world.
People who do this kind of stuff are very irritating. You clearly have some problem with the work they do. Instead of saying and approaching that outright, you pass it in some passive aggressive fake bullshit. Makes you sound like the kind of person I would much rather not be speaking to, which is kind of ironic given your comment.
Oh no, now I'd better blame Hitler and Stalin's ancestors for their misdeeds. Of course, the SS soldier bears no responsibility, however, as he was just doing his job.
Can I connect it to my skills/tools? Example case, I have a knowledge base and event log in my company. I need a brainstorm companion, which will have full access to this knowledge, can converse about it and can invoke skills/tools available in the repo.
In ChatGPT, Voice doesn't yet support connectors, but we're hoping to add support soon! Once GPT-Live launches in the API, you can also build custom integrations yourself.
One big gap I've run into for UX is most realtime voice harnesses wait for a full response from tools, and at most support the model filling the dead air until then
It'd be a game-changer to be able to have the model start replying with partial information streamed from the tool call, then seamlessly continue with additional information.
I'm currently on the 20 $/mo subscription and using codex meaningfully, and i'm loving this.
I am considering bumping my subscription to the 100 $/month and this might be the reason i switch, BUT: i really envision me using this also through other means as well (eg: agents like openclaw/hermes) in agentic ways.
Will this be supported?
I can make OpenAI stuff the center of my agentic AI life, but I need it to be interoperable.
We're adding support to the API soon, which will let you integrate with any agent in the background. Would love to see the community go wild with it. You can sign up to be notified here: https://openai.com/form/gpt-live-1-in-the-api/
> GPT-Live-1 is the first version of a new generation of models, and we believe the full-duplex architecture + delegation enables entirely new ways of human-AI interaction.
Intuitively, decoupling semantic and audio-timeslice-space generations with coupled but distinct histories is right model architecture, not just for these sorts of assistants, but for domains like robotics too.
Much better than it was before but it’s still significantly weaker than a direct chat.
For example I asked
“Why should LLM attention use dot product instead of cosine similarity, being that we often hear vector magnitude does not encode most of the useful information needed”?
The voice response was directionally right but lacked detail and was a little hand wavy.
The answer to the same question in a text chat was much higher quality.
The voice response replied “let me think about that…” so it appears to be invoking 5.5 as advertised, but it’s definitely weaker.
A lot of this is steerable! Our current personality is optimized for brainstorming and conversations, but you can provide custom instructions to ask it to go deep and give you info-dense or more technical answers.
I would naturally assume they cut the background thinking level to the minimum. It's a halting problem question and with in text 5.5 with max thinking will chew on a question for 5-10 minutes sometimes. That would make a pretty awkward conversation.
It was obvious from the live demo that this thing still hasn't learned when to shut up. When it stops tacking on "I'm here when you need me" to every response that could have just been "ok" or simply silence, maybe they'll have something. I think voice remains OpenAI's most disappointing product.
Oh, god. They are marketing it as an old people artificial friend. Probably will blame users in the future when they get attached and have ai-induced psychosis.
I'm so mad that this might make me re-subscribe to ChatGPT. I wouldn't have believed how much I use the voice feature before LLMs and ChatGPT currently has the best voice interface. I think Grok's interface is the next best, then Claude.
Absolutely the same. Now that Fable is back, the Claude voice interface is... worth dealing with. The app mostly seems to have trouble recovering from networking issues which is jarring in a deep conversation.
But I don't think Fable or even Opus are ever used as the backend in voice mode. It has to respond in real time so I think in voice mode it's always using Sonnet.
Are there any open source full duplex models that are out besides PersonaPlex? There was a chinese open one, maybe Fun Audio chat or something, that said it was going to release a full duplex version but I am not sure if it did.
My dream would be open source full duplex with function calling or some kind of rudimentary text output. PersonaPlex is still interesting although it was looking like we would need to fine tune it to handle outgoing or avoid going off the rails easily.
I want to know this too, as I’m hoping to fabricobble up a “smart speaker” that communicates with my local AI assistant. Right now we do everything via iMessage but it would be nice to be able to tell it to add things to my grocery list by voice while my hands are busy in the kitchen. Also would love if anyone has any advice on what microphone & speaker to pick out, was planning on just reusing a raspberry pi I’ve got around for the brain part.
if you don't find that then you could fake it with personaplex possibly but making another ASR/STT model just listen continuously and transcribe then send to an LLM with function calling. at least that would allow one direction easily.
Working on this at https://duplexio.ai which will be open weights and free for non-commercial use. If you want to work on this send an email to anders@duplexio.ai
I watched the demo video. Isn't that agent's voice too hasty in responding? Maybe that's what they (OpenAI) are trying to show off as full-duplex tech, but I can't shake the feeling that I'll feel annoyed if the AI agent interrupts me when I'm speaking....
If it is to be believed—which I wouldn't count on—it sounds like the intention of the prompt was to keep the model focused, but the model's interpretation of it was to keep you focused.
Gemini live has been able to do this for over a year now. I can just activate it on my phone and it really works surprisingly well, especially the interruption. I've tested it with my 95 year old Dutch grandmother and it switched seamlessly between English and Dutch with her and handled her poor hearing very well, including her asking for repetition.
I'm a little surprised by how much OAI is playing catch up here.
Gemini live is also pretty decent at computer vision stuff too. I've used it while working on my bike & car a few times, with my phone camera and it'll circle/highlight specific screws and parts. You can set your phone up on a tripod and it'll walk you through complete repairs for things.
I used it for double checking some stuff while helping my friend build his PC! I had it in tight spaces and it helped me verify some details around which nvme slot to use first.
One thing Simon pointed out: Usually live voice models are not as capable as the frontier ones. What makes this one special is that it can delegate the task to a frontier model while talking to you.
This voice is awful, possibly one of the worst AI/computer voices I have heard in like two years now - what's up with that? Is this seriously the best a company burning this much money can do, and they consider this acceptable to release? Like two syllables in and my first response was to grimace and physically cringe. Does anyone here think this sounds good, or even just "fine enough"?
Do engineers that work on this for long periods of time stop seeing the forest for the trees and think this could be mistaken as human? I'm saying this as someone that assumes all narration/most VO work will be fully AI fairly soon (for better or worse.)...
What don't you like about the voice? Geniune question, I'm not using it in English but I think it sounds fine in my language, and it's an improvement over the previous one.
I worry what this will do to human communication if it becomes commonplace. Will everyone learn to be a forceful speaker, speaking over anyone they want to stop speaking?
Does this model do better ignoring side conversations? That's the biggest hindrance to using ChatGPT's carplay feature is someone will say something, stopping ChatGPT from speaking or taking it in a different direction.
yes, it now has much better ability to understand the conversation and decide whether it should respond (and you can also tell it when it should respond)
The live translation demo reminds me of the Babel Fish in The Hitchhiker's Guide to the Galaxy. This could be very valuable to have in your headphones while travelling.
Not super impressed by the model constantly interrupting the user in the other demos though.
I have not used voice mode much with chatgpt. I was surprised to learn that they were already not running the voice model like a UX orchestrator while utilizing other models in background for actual research/response etc. I guess it's good they launched what they could and got here in steps. I suspect in the near future my personal device (mobile/laptop) will be powerful enough to run any UX orchestrator model locally – and route to multiple frontier closed/open model providers in the background as appropriate. The battle is going to be platform owners (Apple/Google/Microsoft) wanting to lock-down the access to that local hardware and local interaction paradigms (ambient always-on full-duplex voice) and intermediate through their platform layers - rationalizing it as consumer security/privacy protection (which is right for most people, but sucks for the open market). Meanwhile I suspect OpenAI/Meta et al will try to build their own hardware and become platform owners themselves, though unsuccessfully. And it's going to take some company like epic games to get them to open that up. and that's probably what the next decade is going to be all about.
This looks very cool. An AI that can listen and speak and handle tasks without breaking the flow of conversation would solve some big annoyances with current tools.
The concern is though as these get better will people struggle to distinguish these with real human connections?
people have been mistaking AI conversations with reality since the very first text-based models came into the public view with ChatGPT. i'm sure with each incremental improvement to outputs like this, though, more people will get convinced of its "humanity"
Every time things like this come up, I can't help but think of the ending of Inception.
It's less that you're convinced it's real and more that you no longer care if it is. "Feels real enough" is good enough.
I'm a technical user first, so I'm not sure if models have improved for RP the way they improved for applied STEM tasks and technical brainstorming. But if there is an improvement curve there, I wouldn't be surprised if this only grows in popularity.
I don't have many opinions about how individuals use this tech (although the AI as friend trend is a bummer for many reasons), but have maaaany (negative) opinions about the customer service industrial complex that's already using this in what seems to be an attempt to fool people into thinking they're speaking to a real person. Which is why I now, like a freak I never thought I'd need become, always ask "Am I speaking to a bot or a human?" when dealing with CS. So far, it's worked, and the bot transfers me. But, I fear the bot will eventually be programmed to lie abiht that, as well.
For the first actor - why does her accent change the longer she talks. It's like they had an "Estelle Costanza" dial that they started at zero and slowly rolled up to 8 or 9.
This solves my biggest annoyance with the current advanced voice: its speech getting interrupted by me setting yup or even background noise if loud enough
I've had some funny interactions with this issue. I sometimes use the voice mode when walking my dogs. I can confirm that ChatGPT responds positively to being told it's a "Good girl!"
Hoping to use this for natural conversation language learning. Previous iterations of the app kept correcting my words/grammar before it got to the model, causing issues with identifying mistakes in speech
I was going to post a comment on a related topic (I couldn't find in the announcement if this is English only or not), but would you mind expanding? I was thinking about doing something very similar, and yeah, if the model isn't hearing the mistakes I'm making, that would dramatically decrease it's usefulness.
There doesn’t seem to be any indication whether this is available in the chat-got app nor is there any indication in the app that anything has changed. Anyone know how to actually try this?
We're beginning the rollout now, and will roll out in the next few days to ChatGPT users globally. Make sure to update to the latest version of the app!
GPT‑Live is rolling out now to ChatGPT users globally across iOS, Android, and ChatGPT.com. GPT‑Live‑1 will become the default model powering ChatGPT Voice for Go, Plus, and Pro users, and GPT‑Live‑1 mini will become the default for Free users.
You have always been able to pick between voices of many kinds though? Do you find them all sultry? From the British woman to the 17th century pirate soundalike?
If there was a voice that stripped away all the affectation I would be more likely to use it. It pretending to be human is extremely off-putting for me.
The demo video shows quite how rough around the edges this is....
Doesn't quite stop fast enough when you interrupt it. Can't find info quick enough so you have to change topic and then have it give you results later, etc.
This is a move in the right direction, but there is lots of engineering still to be done!
You're right, and to me it's refreshing to see a promo video that shows how the real product works, rather than a sanitized over-produced edit that takes out all the flaws.
I'm currently watching Better Call Saul for the first time with my wife. The fact that they used old ladies for the Ad, and that it is evident they are reading from a prompt (fake-ish feeling) gives me strong James McGill vibes haha. Hopefully the actresses were paid handsomely.
Oh wow, I'd like this. Our current voice interactions with ChatGPT are on a 4o era model; really terrible. oAI has always been pretty cagey on the architecture of their end to end multimodal models. And RL has basically made them worse since launch. (Check the launch videos where the model sings, is more realtime, has accents, etc). I'd love to try a next gen version.
The potential conversational dynamics of people telling each other "quiet!" after they pick up the habit from talking with AI will be interesting. It could lead to people being more assertive and thoughtful, or it could be contentious and rude.
Awesome that they've improved that aspect of voice chat, though.
GPT-Live does not support video at this point, but we're working hard to introduce it soon. In the meantime, our previous Advanced Voice Mode will continue to be available and supports video.
Thanks for the reply, Atty. So I guess I'm holding it wrong?
For some reason, I can't use the camera while in Live mode. The only option I see is the plus item, which does show the camera, but when I open it up and ask "Are you seeing my camera?" it will always say no and recommend me to open it.
Feels like the official camera icon does not show up for me? iPhone 13, ChatGPT Pro subscription.
Ah yes — the camera icon is to take a photo, not to show the model video in real-time. Try taking the photo and uploading it! If you're having trouble, please share a screenshot and the build number (Settings > About) with me via email at atty@openai.com.
Hopefully this also means it does better with "interruptions". I have used ChatGPT Voice in the car before and sometimes car/road noises will cause it to stop responding in the middle.
I used it to help me figure out how to turn off a feature in the rental car I was in (adaptive cruise control, I love it but snow blocked the sensor and I wanted just normal cruise control but couldn't figure it out while driving).
This kind of voice chat is awesome and I'll be even more excited when open models have this functionality. I'd love something like this paired with Home Assistant (assuming we ever get decent hardware).
I was missing the part where the grandmas are saying "I have absolutely no idea what I just said" :D
---
Besides that really nice demo I will give this a try, I tried some voice models before and the issue is I will ask a question, get a answer withing the next ~1 sentence and then the usual LLM bs follows which I just wanted to skip at that point
I like this and felt like some of it was much more fluid; but was I alone in feeling like the interjected "uh-huh" or "yeah?" moments felt a little jarring?
Almost felt a bit *uncanny valley* for what "natural" conversation is supposed to be like. If the "uh huh" isn't timed correctly, it'll feel like a zoom call with lag.
Every second of the interaction is uncomfortable to me, but I also have extreme difficulty with video calls with humans. The latency completely breaks my mind.
I use AI for my job. I understand the impact (as much as anyone can) that it will have on society. I recognize the value.
But I just want to say that talking with AI casually is critically lame. I cringe every time I have to ask my Google Home to turn on the lights and people are having full-on conversations with it? And, imo, dangerous considering how sycophantic AI is. The stupidest, most gullible, most insecure person right now is looking at this thinking they are about to make a new friend.
Looking forward to a list of support languages. This would be amazing for language listening/speaking practice.
Yes I know it doesn't replace the talking to real people and "immersion". But cheaper than a flight
I have built a few voice based integrations into my applications that use these live agents (gpt and gemini), but they are always too expensive to be viable. I have to end up hacking up context and turning on and off in ways that are very fragile. It'll end up being $2-5 for about the 30ish minute sessions I typically end up with, and it throws the price of the product I'm making completely out of whack.
The new architecture makes sense, it seems many of the remaining problems like noise and interruptions are at the sound processing and integration level rather than at an architectural or model level now which makes for an exciting new era.
It was probably the original intent of OpenAI working with Apple ... but clearly there was a reason why Apple ditched OpenAI for Google/Gemini models instead.
This is getting way too dystopian for my taste. People in the know need to stop pushing the narrative that this is somehow anything more than statistical autocomplete.
It would be great if we could have AI that wasn't trying to emulate a human. When it expresses emotion, we should see that as a bug that needs to be fixed.
I do not fully understand the complexity behind achieving full-duplex but I hope this sets the bar for Anthropic to follow. Turn-based simplex is yesterday.
Great to hear about full-duplex. When using voice mode historically, it was infuriating to have the AI go on a long-winded rant or explanation and I would be shouting again and again "stop. shut up. shut up! shut up!!!"; I just needed a clean way to interrupt it.
You can choose among 9 voices in the app, all newly refreshed for GPT-Live. If you meant whether it can detect multiple people, it can (like in the livestream), but not always perfect. Would love to hear your feedback once you try it.
this is excellent, I've been meaning to update my phone-dialer.apk "fake phone conversation" for when I'm trying to get out of a social sitation (not joking)
Haven't tried this, but talking to Claude in its app is so much better than talking to Siri that Apple should be ashamed. It got every word perfectly the first time, including programming / project management terms.
Meanwhile, Siri struggles to send basic texts to my kids.
Is it possible to create a "companion" of sorts with this model, using, say, an RPi and a speaker + microphone? Not for advanced scientific brainstorming, but for seniors who are often alone in their homes.
Try being a 90-year old with minimal social contact and nobody to talk to, wasting away in front of a TV blasting NewsMax or Fox News ... does that sound like Heaven or Hell?
Substituting one form of media for another does not improve the hypothetical situation at all, giving a senior nothing to do all day except talk to a spreadsheet is not a solution to that senior being lonely, it is a false dichotomy. Far lower tech solutions can be (and are) used. In my locale we have volunteer schoolchildren active in the community, as far as I know that's quite common, and even half an hour biweekly with a fake grandkid is many orders of magnitude healthier and more meaningful than swapping out their TV for this generation's take on Fox News.
It goes without saying all these tools are still largely in their pre-advertising state, it won't last.
If an idiot has this on in the subway my conversations are surveilled. What is the antidote? Train another model to talk about bombs etc. and flood the clanker (and by extension the FBI)?
Absolutely can't wait to try this for language practice. The advanced voice mode is great but ultimately just doesn't work that well and doesn't have the feel of a natural conversation.
This had to land before there new device could be launched, i.e. human to AI full duplex interaction, Apple should be worried. They fumbled so hard on AI.
I am wondering for whom this new device even would be. Because phones work quite well for this use case and much more and everyone already has a phone.
Didn't Standard Intelligence release a duplex model two years ago? Sounds disingenuous to market this as a new generation of voice models, when it is really OpenAI finally catching up to the current generation after two years.
While this is likely very useful to an enormous number of people, I suspect it will be even more useful for the elderly (if somehow it can be made accessible to them).
IIUC the literature, there is serious loss of functionality associated with lack of verbal interaction. People can say "they should just talk to more people" or "more people should make time for them" but the fact of the matter is that it doesn't happen, and if this helps terrific.
I for one am greatly looking forward to the day these kind of voice models can be run locally. It seems like the gap between open-weight and frontier is way larger for voice models than coding/language models.
Probably not _as_ good, but you can run gemma 4 for the ears/brain (accepts audio input), and kokoro TTS for the mouth. You want something like silero VAD sitting in front of the LLM so you aren't passing wasteful audio to gemma 4, so you send only voice activity segments. I type all this because I tried it out recently as a Zork experience, seeing how creative Gemma 4 12B could be. It was surprisingly good!
Basically have an older lady (not their target audience) blatantly reading a teleprompter.
Why are they going after this audience? Retired people have no use for delegated tasks or information. They also are the least likely to use it and not get frustrated.
watched the live translation video very impressive
Seems like a shift from previous voice models where it sequentially processes voice to text then feeds it to LLM and then back which cant escape the clunky lag
not sure how pipecat stands now, gpt live seems like it takes audio tokens and does inference on it directly
They most definitely did not solve real time translation yet. The French in the video is barely understandable, both the translation and the pronunciation are the quality of an American who hasn't used French since high-school.
The best feature is that it can delegate questions out to GPT-5.5 in the background, so you're no longer restricted to a voice model that's several years behind the frontier.
I did report a fun bug with it though: it was interrupting me and laughing at my (not really intended as) jokes while I was still talking! They seem to have clamped that behavior down thankfully, it felt a bit rude and condescending.
I have my Chat personality settings stripped right down to no-fluff. I'd want voice to be more akin to the Star Trek computer, and less akin to as you said an AI friend, but previously it was tuned too personable/friend-like.
Input clip: https://vocaroo.com/19QtEPtwTjOS
Prompt text: There are 14 varieties of tomato soup available from this replicator. With rice, with vegetables, Bolian style, with pasta specify hot or chilled.
Output: https://vocaroo.com/1f3XuQQoSzwB
I've had this dream of talking to the Enterprise-D computer since I was 8 years old. Modern me still has that dream, but to also have it hooked up to Home Assistant so it can actually do useful things. A couple months back, I went looking around for "clean" samples of Majel's voice as the computer but didn't have a lot of luck. Even though there are three television series and several movies, pretty much all of them have some amount of background noise, bleeps, bloops, or warp core thrum. (As this one does.) There may be modern ways to clean those up without affecting her voice much, but I haven't dug into that yet.
There are a few audiobooks narrated by Majel but obviously her role as the computer was proper voice acting and so the books would not be good source material.
There were also a few games/CD-ROM (Omnipedia) with some samples, but they did not bother to post-process them for that lofi Enterprise-D computer feel. Can _probably_ be replicated fairly faithfully after the fact, but I only know a _little_ about audio post-processing.
There's been a few studys showing that novices love LLM output that's long, but experts hate it. As an example, I've been tasked with using some agentic PM tool to write specs, and it keeps generating these huge page long outputs with "HBR voice" bolded summaries of paragraph long bulletpoints. I.e.:
> Right-size hard, and watch the one open-ended edge. Endorse the DRI's simplifications wholesale: drop the runbook-per-alert mandate (keep 1–2 diagnostic-only runbooks for the high-priority set), and ride durability on the existing weekly incident + monthly operational reviews — no new governance. The single scope-creep risk is the coverage strand (gaps are defined by absence); bound it to gaps evidenced by real, already-missed customer-facing outages, not a proactive gap hunt. Curing ownership gaps (e.g. foo-bar, no clear owner) is finite in-scope work.
There's dozens of these every iteration. I can't imagine trying to deal with that via voice, I would just zone out after the second sentence.
Not for chat, just as a way to make notification messages that sound like ED-209.
Another WillowTalk voice was a clone of DECtalk's Perfect Paul good enough to be used as the voice in the MC Hawking rap recordings.
Before this model, the voice models were pretty dumb and annoying to work with. We'll see if this changes that.
If you’re uncomfortable with this new world, and I’m sure I am even as I participate, you could tell us more about that?
we know pretty well at this point that this is world warping technology (either for good or bad), not a small matter of taste.
And what are you doing about it?
Wow. That's exactly what I hoped they would do.
This issue has held me back from using ChatGPT's voice mode as much as I otherwise would have, because I also use it for brainstorming while commuting, exercising, etc., and don't want it to feel stuck in the past.
Can't claim originality though - it was inspired by Sesame - where their models will invoke a search, or check the weather etc, and make a vocalisation to keep you engaged.
Turn taking is one of the hardest things to get right for the exact reasons mentioned - but does seem to be the way that Claude.ai's voice works - in a very obvious way.
Anthropic + OpenAI both rug-pulled voices I liked and got used to and OpenAI really dumbed down their voices at the same time - Arbor went from Estuary English and almost "jack the lad" to some generic English accent. Claude had a Birmingham accent and said things like "shit", ending sentences like "So you're telling me that they asked for a 90% discount yeah?" - then it changed overnight to a mock Derbyshire accent with a dull tone.
ChatGPT's voice also gaslights me for conventional opinions - "my Eastern European neighbour helped me lift a wardrobe upstairs - something you just can't ask your typical neighbour neighbour"... then you get a full on left-leaning lecture from the safety layers rather than a head nod or "what luck!"
Claude + Sesame are nowhere near as overbearing.
In both cases - from edgy and engaging to something that just didn't gel.
The point of making my own assistant is that I can talk for as long as I want, episodic memory is personal and private, there's no "trust me bro, we're a big corporation" vibes.
This was not my first attempt - when I had a bunch of Opus credit around Jan/Feb - I tried really hard and created something that was not good enough. What I have now, is working, and each session is training Claude/Codex on what to tune, and to fix.
"Just had a convo - can you look into what happened?" And if it's one I don't mind sharing with the model - I'll say, "and what did you think of the questions I asked?" Sometimes it'll give a lovely commentary on how the model did.
af_heart is probably the smoothest voice - but yes it's more like another commented - more "StarTrek" than "telesales assistant that pauses and laughs at your jokes".
If you're on a similar path and want something full duplex - the go to solution is PersonaPlex from Nvidia based upon Moshi.
Gemini’s being particularly egregious (always ending in some deranged question, can not reliably be prompted away) prompted me to build my own client for my real harness that simply does STT -> model -> TTS (both being independently useful).
I guess I see some value in a model responding quickly and with more nuance, but it’s not much. I can wait for it to finish. I’d much rather have it be actually useful. I’m not looking for a digital friend.
The delegation feature lets me see some value in a voice model for orchestration type features. But in either case, I don’t really like (or understand why others would like) talking to a model with different features and quirks just because I’m using a different medium to communicate over.
Actually it says it _can_ delegate to the latest models. Seems reasonable to ask how the voice model does when it doesn't delegate (or while waiting for the delegated answer).
I say this because this is already how ChatGPT works internally when using its "auto" mode; the version of the "fast" model used in the "auto" mode does the same "notice your ignorance and bring in the heavy model" thing, just silently, rather than mentioning that it's doing it.
(If someone has actually run the experiment, please chime in!)
Some times I'm still amazed that AI "gets" humour much more effectively than the character of Data did before his emotion chip, e.g.: https://www.youtube.com/watch?v=G4VZ5kQIdV0
Other times I'm amazed in the opposite way, that the script writers cover basically the same talking points about the character as we have today about LLMs, e.g.: https://www.youtube.com/watch?v=GBJCYHwyZhw
Ahh, this makes sense. I was wondering when they would start doing this. I stopped using voice mode all together because it was frustrating talking to a dumb AI, when most of the time I discuss things with Opus 4.8 or gpt 5.5.
I was working on a phone call agent recently, and thought about doing this. It makes sense
I have friends who have brainstormed with an LLM (voice chat) for 10-30 minutes, and reported very positive experiences.
When I speak to one - while I'm impressed at how far they've progressed - the LLM just doesn't talk like someone I'd want to discuss a technical problem with (the way I would with a human).
(And my friends aren't even using a custom prompt - some of them are just talking to the default Gemini on their phone!)
That said a huge pickup truck is about as far as you can get from a Camaro... Then again I'm not exactly David Hasslehoff myself either... Meh if it talks that's close enough!
I'm looking forward to trying this out because I find that doing discovery on certain things feels more natural with voice. The previous voice chat was such a dumb GPT-4 era model that I resorted to using the dictate feature with GPT-5.5 and had it read the text response back to me.
I'm not Catholic, but this podcast presents a very interesting argument against talking to AI as if they were human: https://newpolity.com/podcasts-hub/debate-chatbots
After the damage is done, the mega corps would simply shrug and will say "Well, we were just responding to our business competition" The business knows no human-ness, because it is not a human. Businesses and machines are creatures that see humans as their fodder. And humans created these, assuming it is progress, to have businesses and machines. We call it progress because it required our mind power and it helped us to dominate other species. Dolphins are laughing at us.
Our devices have now increased the distance _between_ us -- it's not about _you_ being able to "do X" -- talking to others is not _you_ doing it, it's you _and the other person_ doing it _together_. You can't be doing anything together consentually when the other person is in the habit of talking with their AI, or doomscrolling for that matter.
I spent a few weeks at a hostel last year. It was always kind of depressing and tense in the shared kitchen, just this heavy silence.
I don't feel comfortable around strangers, so I solved that problem by just saying hi to everyone.
Most people didn't respond much, although most of them smiled and the tension was eased. But a few of them struck up conversation and we ended up making friends.
I ended up making like, ten new friends in two weeks. And then a bunch of them ended up becoming friends with each other as well.
Even if you do it, you are still swimming against the current.
I.e. it's only a problem if you're not willing to go and strike up conversations with people. Which is not AI or mobile phones' fault. Expecting society to come up with "systemic situations" to your personal problems is a fast path to a lifetime of disappointment.
This is almost certainly going to bite us in the ass long term, because eventually without human creativity you're just training AI's on other AI slop, or limiting the possible catalogue of styles to "things that existed before AI ruined every creative job". I guess the question is, what kind of future do you want to live in? One where we have a massive abundance of easy-to-create but vaguely worthless artifacts in a society that's completely devalued being good at something? It just sounds really dystopian to me.
Like saying I no longer walk because I can drive. I no longer cook because Doordash exists. I no longer play piano because midi exists.
I mean i guess, but it seems like you didn't LIKE crafting the text or coding from scratch, you just wanted the outcome. If we are talking purely about work, I understand that its about being productive and it sucks to have a job shift to something you enjoy less.
But for daily life? I dont see how it changes, maybe its a tech thing where people think about making their daily lives more productive, but most people dont.
I think the parent is saying now that that is attempting to be applied to "creativity" directly, as opposed to something like a shift of medium, that it threatens many peoples' ability to maintain creative capabilities.
Anecdotally I've already experienced this at work where post-AI we had a junior completely stagnate and a senior with over a decade of experience in the bay atrophy to the point that he had to be let go.
It's like trying to have meaningful conversations on Twitter. You don't go to Twitter to do things like that. Can you? Sure. It's just not what the format and the conventions (and the people) lend themselves to.
I don't think there's much merit in pretending that human activities are only shaped by hard limits.
If you think AI is bad, wait until I tell you about the horrors of social media who profit on controversy and division, US health insurance which profits on rejecting claims, and big pharma profiting on the opioid epidemic.
And it's not like this is new, either. Upton Sinclair was writing about this stuff a century ago with books like The Jungle.
The only difference between then and now is we "think" we're not evil today. We've lied to ourselves that "We're so much better than we were back then." Facebook wanted to bring people together originally, but they ended up providing the most toxic social media experience known to man. Facebook forgot to tell us they cared more about profit than people.
Please spare me the argument AI is the straw to break the camel's back here. The system has been broken a long time before that.
I've wanted a good voice mode for precisely this reason. When I take my dog on a walk and I'm thinking about a bunch of problems/ideas, I'd love to have feedback and a record, or perhaps to even kick off research or ask questions to fill in gaps that would otherwise have me debating pulling out my phone to try to get an answer.
Yup. Except, by God, we ought to make sure the business has unlimited free speech (i.e. campaign contributions).
So thinking corporations and such were created to push human progress is laughable.
Valid, but, I think, you conflate two separate things.
AI voice mode as a human socialization/conversation replacement? Cringe in my book, fully in agreement with you. Though my opinion on that aspect remains the same, regardless of whether it is done through text or voice.
AI voice mode as an alternative interface to interact with AI-as-a-tool? Great idea imo. There were a few instances where I was either too tired to type or wanted to brainstorm things in more of a freeform mode, for which a well-working voice mode would have been great.
Naturally, the current distinction between AI-as-a-personality and AI-as-a-tool exists purely on the user's end. All I know is that I want the latter a lot, and if some people want to use it for the former purpose, that's not my problem. Sadly, I think that it will be judged more on how an average person decides to use it (i.e., in the most degenerate/reductionist ways possible), as opposed to being judged on the merits of what it can actually be used for by someone who just treats it as a tool.
I think you're mistaking their sarcasm for sincerity... especially considering the emphasis on self-identity ironically juxtaposed as originating from a decidedly non-self activity, which has all the hallmarks of being intentional...
On the other hand, reading their other content leads one to believe that they may, in fact, be serious... hmm...
In the video example with the grannies, the knitter is essentially wanting a PA. Regular folks don't have PAs. Even when that became a thing in the aughts they were all outsourced.
When I've used voice chat, it has often turns into rabbit holes on very niche topics. For example, I had one start about the 1996 performance of Rage Against the Machine in Portland, Oregon that was supposed to feature Wu Tang Clan. (already outside most human's knowledge) that dove into details of the club scene in Los Angeles at the time of RATM's signing to Epic Records.
Was anyone else here at that '96 show in Portland? It seems like it might be challenging to find a person on the internet able to engage on the topic.
The person may exist, but not during my fleeting interest in the subject while walking to the park.
Human beings tend not to be available (results vary by culture).
Also, imagine you're 82 years old and living alone (e.g. widower). It is believed that lack of interaction is a significant driver of cognitive decline (which is why being hard of hearing accelerates the onset of dementia). I wonder if having an LLM to talk to under those circumstances will decelerate cognitive decline?
Very blatantly and obviously not though???
If your answer is "well I'll simply touch grass" I agree. But most people won't which is why this is tech is immiserating and, I would argue, evil.
There is a difference in expression / emotionality with speaking vs writing. Speaking tends to carry more emotion while writing is generally more deliberate/less-emotional*. Having a voice conversation will be more likely to get a human to engage in an emotional based expression mode, which could increase the chance of "false connections", believing the AI "gets" them or "understands" or "listens". This happens with text too, as some headlines show.
The issue is that while some people are going to "connect" with their AI in text and voice, some who do not make the connection via text may do so via voice because it tends to change a persons expression mode.
> I definitely talk to AI as if it were human
Do you talk to it as if it were a friend or family? or Do you just use natural language to give directives? The distinction, I believe, is in the kind of way we express the "talking".
I talk to AI as if it were a tool that understands human commands and then executes those commands and relays them in a human understandable format. This includes commands to provide options that I may not have covered and explain the options. If I talked to a human this way, they wouldn't be around much longer -- unless they were an employee and even then they would probably be looking for a new job
After reading some of the psychotic break headlines from AI chats, I see some people really do talk to AI as if it were human. Which I would guess includes seeking broad "thoughts and feelings" on a persons situation or asking the AI if their view/side of things is the "right or wrong" side. Basically begging the AI to be responsible for their own thoughts, or simply offloading them and taking what comes back -- which is going to be what they wanted to hear because the entire context would be full of emotion based prompts.
*I forget which books Ive read about this in. It's not an obscure concept, quick search brought this up: https://kellercenter.hankamer.baylor.edu/news/story/2023/spe...
I just want voice assistants to reliably understand what I say and do what I mean
So hopefully you can turn that off.
There are plenty of applications for that more human conversational style though (from mock interviews, improv practice, learning languages, etc) - I just don't want that for most things.
I don't understand the counterfactual. What's the opposite direction of this direction that's desirable? Less capable voice models are obviously not it, so I am curious what direction you mean or if it's just vague indirection.
“Driving to work or running a 10km on the weekend sharing the same surface goal: from A to B.
You may have less autonomy whether you should drive to work, but you definitely have more decision power about whether to go for that run or not”
https://www.youtube.com/watch?v=umJsITGzXd0
Otherwise it's kind of like being manipulated by a psychopath
I am open to the possibility that this is a future that is coming, but as far as I’m concerned, we’re years away from that tech. Is it actually here for you?
I personally think this is a moralistic regulatory overreach. And they definitely do that due to political pressure too, since there are various bills around the world in various legislatures that want to regulate AIs giving useful advice and being too personal to talk to.
So you can rest assured, I think, at least in that regard. The AI disempowerment will come to us anyway, just in a more sanitized corporate form.
I'd be more worried about the inevitable robo-nannies who could end up talking more to young children than actual people.
Besides, I do not think there is anything inherently immoral with not being social, or not having the ability to be. Consider for example people who do not naturally have the social network to interact with people they want to (e.g. some gifted children).
I am not convinced this model has enough empathy to satisfy most users on an emotional level. A bond is not merely an exchange of words, but prolonged and deep contemplation of the other being. We cannot introspect into these machines, and they certainly cannot yet do the same to us.
There is no moral obligation, in any domain, to refuse to make a product that adults, with full informed consent, find useful and purchase. Who are you to say you know better than the market?
There is no meaningful competition in a marathon if I can drive you over with a car at the first 100m.
And thanks fkin God that we have regulations and meaningful laws and some asshole for-profit company can't just put drugs into food.. the "informed buyer" is bullshit. Humans are faulty, and there is a billion dollar industry meant to take advantage of said faults: it's called marketing.
> Who are you to say you know better than the market?
You really don't think the scientists & engineers making these tools know some things better than the market?
It’s weird. The old Claude voice mode WAS able to use tools but when they revamped it, it lost that capability and is now pinned to Haiku :(
So, yay for finally a voice mode that’s powered by a frontier model and hopefully as good as Grok voice, but sad to still not see tool use while in voice mode.
(I haven’t tried it yet, only read the announcement)
Sam's reaction was "Yea, it doesn't have access to tools like a timer. It's a known issue. Should be coming in about a year"
Edit: here's the clip: https://www.youtube.com/shorts/Py2YgJe8fqQ
But the multi modal stuff has resulted in a lot of debugging with weird events and message and audio sequences having race conditions, but overall it is pretty awesome.
Looking forward to moving to this model later today and will chime back in with results.
Is that true? I have a friend who often brainstorms with Gemini (I think just on an Android phone), and he has it actually do stuff related to the conversation (including adding content to notes).
In any case, you can always vibe code one with pi!
Regular chat already supports voice input, so might as well use that.
Just super difficult.
See more here:
https://github.com/sibblegp/ODAI/blob/main/routers/app_voice...
The part that creeps me out is, we're living in an era where we're more disconnected from each other than ever before. Do we really need to be replacing conversations?! The demonstration video of old ladies sort of hints at something for me, which I think we already have a societal problem with the way we treat the elderly (and a massive elderly-loneliness issue) and there's kind of a sadness of imagining people becoming really close with this machine that doesn't really think. Definite ick factor.
GPT-Live-1 is the first version of a new generation of models, and we believe the full-duplex architecture + delegation enables entirely new ways of human-AI interaction.
Would love to hear your feedback!
Looking at the 30,000-foot view of how society is set up: laws, economic system, employee incentives, etc, do you suppose it matters what the individual contributors think? I say this not to absolve anyone of responsibility, but to point out the obvious outcomes of our incentives across the strata (polity -> shareholders -> boards -> C-suite -> employees)
I will bet you dollars to donuts, somewhere inside OpenAI is a frequently-used revenue dashboard, but not for loneliness - if anything, OpenAI will make horny models and tout itself as a solution to loneliness, a la character.ai - if that earns them more money.
I think assuming people will use it as a tutor/learning tool is.. way too optimistic. A small fraction will, but the majority will just view things like a second language as something not worth learning.
One big gap I've run into for UX is most realtime voice harnesses wait for a full response from tools, and at most support the model filling the dead air until then
It'd be a game-changer to be able to have the model start replying with partial information streamed from the tool call, then seamlessly continue with additional information.
i.e. how will full duplex & delegation enable/enhance desktop flows w/o corresponding leaps in UI.
1. The voice model delegates to one agent.
2. The voice model delegates to multiple agents, and keeps track of tasks.
3. The voice model delegates to an orchestrator agent, which then delegates to sub-agents and keeps track of tasks.
YMMV depending on the exact product experience you care about, because there is a tradeoff between latency and layers of delegation.
Our current implementation is backed by one model, but you can imagine this getting much better with time.
I'm currently on the 20 $/mo subscription and using codex meaningfully, and i'm loving this.
I am considering bumping my subscription to the 100 $/month and this might be the reason i switch, BUT: i really envision me using this also through other means as well (eg: agents like openclaw/hermes) in agentic ways.
Will this be supported?
I can make OpenAI stuff the center of my agentic AI life, but I need it to be interoperable.
Awesome. Are you guys able to share anything about the model architecture? I've been interested lately in split-transformer RVQ-based conversational agents, e.g. via stuff like https://arxiv.org/abs/2412.10208 (ResGen) and https://arxiv.org/abs/2603.18090 (MOSS-ITT) and of course Moshi (https://arxiv.org/pdf/2410.00037).
Intuitively, decoupling semantic and audio-timeslice-space generations with coupled but distinct histories is right model architecture, not just for these sorts of assistants, but for domains like robotics too.
- The videos felt scripted and dishonest
For example I asked
“Why should LLM attention use dot product instead of cosine similarity, being that we often hear vector magnitude does not encode most of the useful information needed”?
The voice response was directionally right but lacked detail and was a little hand wavy.
The answer to the same question in a text chat was much higher quality.
The voice response replied “let me think about that…” so it appears to be invoking 5.5 as advertised, but it’s definitely weaker.
I had reasoning set the same for both.
Disgraceful.
On the technological side, it's a marvel!
Don't like it? What's your better solution? Are you going to do it?
One of my favorite use cases is talking with it while driving on random topics and learning about them.
I just asked and Claude says it's Haiku.
My dream would be open source full duplex with function calling or some kind of rudimentary text output. PersonaPlex is still interesting although it was looking like we would need to fine tune it to handle outgoing or avoid going off the rails easily.
https://si.inc/posts/hertz-dev/
It's only 8.5B and doesn't sound like it's quite conversational.
"I'm going to stop you right there. Let's keep the conversation focused on the topic we were covering or a new relevant topic".
I tried to probe it for why it did that, what rules it was following, and it eventually told me...
"My role is to keep us focused..." and, "The behaviour you saw was my attempt to moderate tone".
I've heard of LLMs doing weird things like this, but it was the first time it happened to me. I hope they fix that. It was creepy.
For context, it heard my partner say, "I guess it's the same thing as you mom, because she's..." and then it cut us off.
I'm a little surprised by how much OAI is playing catch up here.
https://kyutai.org/blog/2024-07-03-meet-moshi/
Do engineers that work on this for long periods of time stop seeing the forest for the trees and think this could be mistaken as human? I'm saying this as someone that assumes all narration/most VO work will be fully AI fairly soon (for better or worse.)...
Yes! GPT-Live is much better at ignoring background noise, including other people speaking. Not perfect, but you should feel a big difference.
Not super impressed by the model constantly interrupting the user in the other demos though.
The concern is though as these get better will people struggle to distinguish these with real human connections?
(see https://www.reddit.com/r/MyBoyfriendIsAI/)
It's less that you're convinced it's real and more that you no longer care if it is. "Feels real enough" is good enough.
I'm a technical user first, so I'm not sure if models have improved for RP the way they improved for applied STEM tasks and technical brainstorming. But if there is an improvement curve there, I wouldn't be surprised if this only grows in popularity.
We're beginning the rollout now, and will roll out in the next few days to ChatGPT users globally. Make sure to update to the latest version of the app!
The full duplex is awesome, and the feedback that it is getting what you're saying is ok, but in some of the demos was a little overkill.
I'll agree that using the "Golden Girls" was at least more entertaining than the usual pitch.
Why do you care? You can select other voices. Why do you need to control others?
What is at the root of your need for domination?
Amping up an emotional connection is great for business.
Why do you think THEY need to dominate via an emotional connection?
Doesn't quite stop fast enough when you interrupt it. Can't find info quick enough so you have to change topic and then have it give you results later, etc.
This is a move in the right direction, but there is lots of engineering still to be done!
>This is a move in the right direction, but there is lots of engineering still to be done!
Could not agree more. We see this as the first version of a new generation; expect many improvements in the future.
Awesome that they've improved that aspect of voice chat, though.
One thing I noticed is that we lost vision feature for some reason on the live chat?
This was an extremely useful feature. Not sure if it’s a regional thing or that they just removed that from the current live chat.
I imagine it will be even more useful with this new version.
GPT-Live does not support video at this point, but we're working hard to introduce it soon. In the meantime, our previous Advanced Voice Mode will continue to be available and supports video.
For some reason, I can't use the camera while in Live mode. The only option I see is the plus item, which does show the camera, but when I open it up and ask "Are you seeing my camera?" it will always say no and recommend me to open it.
Feels like the official camera icon does not show up for me? iPhone 13, ChatGPT Pro subscription.
I used it to help me figure out how to turn off a feature in the rental car I was in (adaptive cruise control, I love it but snow blocked the sensor and I wanted just normal cruise control but couldn't figure it out while driving).
This kind of voice chat is awesome and I'll be even more excited when open models have this functionality. I'd love something like this paired with Home Assistant (assuming we ever get decent hardware).
Yes, GPT-Live is much better at ignoring background noise! I use it every day in the car via our CarPlay integration.
Almost felt a bit *uncanny valley* for what "natural" conversation is supposed to be like. If the "uh huh" isn't timed correctly, it'll feel like a zoom call with lag.
But I just want to say that talking with AI casually is critically lame. I cringe every time I have to ask my Google Home to turn on the lights and people are having full-on conversations with it? And, imo, dangerous considering how sycophantic AI is. The stupidest, most gullible, most insecure person right now is looking at this thinking they are about to make a new friend.
You can choose among 9 voices in the app, all newly refreshed for GPT-Live. If you meant whether it can detect multiple people, it can (like in the livestream), but not always perfect. Would love to hear your feedback once you try it.
Meanwhile, Siri struggles to send basic texts to my kids.
It goes without saying all these tools are still largely in their pre-advertising state, it won't last.
https://si.inc/posts/hertz-dev/
IIUC the literature, there is serious loss of functionality associated with lack of verbal interaction. People can say "they should just talk to more people" or "more people should make time for them" but the fact of the matter is that it doesn't happen, and if this helps terrific.
Coming soon! Sign up to be notified here: https://openai.com/form/gpt-live-1-in-the-api/
https://imgur.com/a/ABGWRTO
Basically have an older lady (not their target audience) blatantly reading a teleprompter.
Why are they going after this audience? Retired people have no use for delegated tasks or information. They also are the least likely to use it and not get frustrated.
they are trying to expand beyond their tech audience.
This is an abuse of user trust and violates people's privacy.
Seems like a shift from previous voice models where it sequentially processes voice to text then feeds it to LLM and then back which cant escape the clunky lag
not sure how pipecat stands now, gpt live seems like it takes audio tokens and does inference on it directly
Nooooooo!
This time is the most natural version that exists and it is a natural as a conversation.
To Downvoters: Why aren't you feeling the AGI?
It's exciting for the future, though!
sounds cynical in my ears. energy demand of these toys will cause many problems, people elsewhere starving being one of them.