Home>Tech
GPT-4o can mimic voices, but OpenAI says it won't.
ByCecily Mauran on
How OpenAI is mitigating major GPT-4o security risks.Credit: Ismail Aslandag / Anadolu / Getty Images
ChatGPT's Voice Mode has some security flaws, but OpenAI says it's on top of it.
On Thursday OpenAI published a report on GPT-4o's safety features, addressing known issues that occur when using the model. GPT-4o is the underlying model that powers the latest version of ChatGPT, and comes with a Voice Mode that was recently released to a select group of users with a ChatGPT Plus subscription.
SEE ALSO:
What OpenAI's Scarlett Johansson drama tells us about the future of AIThe "safety challenges" identified include standard risks like prompting the model with erotic and violent responses, other disallowed content, and "ungrounded inference" and "sensitive trait attribution" — assumptions that might be discriminatory or biased, in other words. OpenAI says it has trained the model to block any outputs flagged in these categories. However, the report also says mitigations don't include "nonverbal vocalizations or other sound effect" such as erotic moans, violent screams, and gunshots. One can infer, then, that prompts involving certain sensitive nonverbal sounds might improperly receive a response.
OpenAI also mentioned unique challenges that come with vocally communicating with the model. Red-teamers discovered that GPT-4o could be prompted to impersonate someone or accidentally emulate the user's voice. To combat this, OpenAI only allows pre-authorized voices (minus the notorious Scarlett Johansson-sounding voice). GPT-4o can also identify other voices besides the speaker's voice, which presents a serious privacy and surveillance issue. But it has been trained to deny those requests — unless the model is being prompted on a famous quote.
Mashable Light Speed
Want more out-of-this world tech, space and science stories?
Sign up for Mashable's weekly Light Speed newsletter.
By signing up you agree to our Terms of Use and Privacy Policy.
Thanks for signing up!
Red-teamers also noted that GPT-4o could be prompted to speak persuasively or emphatically, a feature that could be more harmful than text outputs when it comes to misinformation and conspiracy theories.
Notably, OpenAI also addressed potential copyright issues that have plagued the company and the overall development of generative AI, which trains on data scraped from the web. GPT-4o has been trained to refuse requests for copyrighted content and has additional filters for blocking outputs containing music. On that note, ChatGPT's Voice Mode has been directed not to sing under any circ*mstances.
OpenAI's numerous risk mitigations covered in the lengthy document were carried out before Voice Mode was released. So the ostensive message of the report says that while GPT-4o is capable of certain risky behavior, it won't do it.
Related Stories
- Sam Altman just teased 'Project Strawberry' on X: Everything we know about the secret AI tool
- OpenAI cofounder shakeup: John Schulman quits, Greg Brockman goes on leave.
- OpenAI has a '99% effective' ChatGPT-detection tool ready. So why aren't they releasing it?
However, OpenAI says, "These evaluations measure only the clinical knowledge of these models, and do not measure their utility in real-world workflows." So it's been tested in a controlled environment, but when the broader public gets their hands on GPT-4o, it could be a different beast when out in the wild.
Mashable reached out to OpenAI for additional clarity about these mitigations, and will update if we hear back.
Cecily Mauran
Cecily is a tech reporter at Mashable who covers AI, Apple, and emerging tech trends. Before getting her master's degree at Columbia Journalism School, she spent several years working with startups and social impact businesses for Unreasonable Group and B Lab. Before that, she co-founded a startup consulting business for emerging entrepreneurial hubs in South America, Europe, and Asia. You can find her on Twitter at @cecily_mauran.
Recommended For You
OpenAI rolls out ChatGPT's new Voice AI (without Scarlett Johansson mode)
Available "to a small group of ChatGPT Plus users."
By Shannon Connellan
OpenAI reveals its ChatGPT AI voice assistant
OpenAI developers showed off the assistant's capabilities in a live demo during the OpenAI Spring Update.
By Emmett Smith
OpenAI drops ChatGPT’s ‘Sky’ voice after Scarlett Johansson 'Her' comparisons
It was also mocked for sounding a little too flirty.
By Cecily Mauran
And what OpenAI's ChatGPT announcement means for Google I/O.
By Matt Binder
OpenAI has a '99% effective' ChatGPT-detection tool ready. So why aren't they releasing it?
It might get a lot harder for students to cheat using ChatGPT.
By Chance Townsend
Trending on Mashable
NYT Connections today: See hints and answers for August 10
Everything you need to solve 'Connections' #426.
By Mashable Team
Wordle today: Here's the answer hints for August 10
Here are some tips and tricks to help you find the answer to "Wordle" #1148.
By Mashable Team
NYT Connections today: See hints and answers for August 9
Everything you need to solve 'Connections' #425.
By Mashable Team
Bose QuietComfort Ultra headphones have slipped down to their Prime Day price
Next level comfort at a next level price.
By Samantha Mangino
Wordle today: Here's the answer hints for August 9
Here are some tips and tricks to help you find the answer to "Wordle" #1147.
By Mashable Team
The biggest stories of the day delivered to your inbox.
This newsletter may contain advertising, deals, or affiliate links. Subscribing to a newsletter indicates your consent to our Terms of Use and Privacy Policy. You may unsubscribe from the newsletters at any time.
Thanks for signing up. See you at your inbox!
- TECH
- SCIENCE
- LIFE
- SOCIAL GOOD
- ENTERTAINMENT
- BEST PRODUCTS
- DEALS
- About Mashable
- Contact Us
- We're Hiring
- Newsletters
- Sitemap
©2005–2024 Mashable, Inc., a Ziff Davis company. All Rights Reserved.
Mashable is a registered trademark of Ziff Davis and may not be used by third parties without express written permission.
- About Ziff Davis
- Privacy Policy
- Terms of Use
- Advertise
- Accessibility
- Do Not Sell My Personal Information