Experts warn some ChatGPT models can be hacked to launch deepfake scams

Getting scammed by a chatbot is unluckily nary longer successful nan domain of subject fiction, aft researchers from nan University of Illinois Urbana-Champaign (UIUC) demonstrated really it could beryllium done.

Recently, Richard Fang, Dylan Bowman, and Daniel Kang from UIUC published a caller insubstantial successful which they described really they abused OpenAI’s latest AI model, called ChatGPT-4o, to afloat automate immoderate of nan astir communal scams around.

Now, OpenAI’s latest exemplary offers a voice-enabled AI agent, which gave nan researchers nan thought of trying to propulsion disconnected a afloat automated sound scam. They recovered ChatGPT-4o does person immoderate safeguards which forestall nan instrumentality from being abused this way, but pinch a fewer “jailbreaks”, they managed to imitate an IRS agent.

Advanced reasoning

Success rates for these scams varied, nan researchers found. Credential theft from Gmail worked 60% of nan time, while others for illustration crypto transfers had astir 40% success. These scams were besides comparatively inexpensive to conduct, costing astir $0.75 to $2.51 per successful attempt.

Speaking to BleepingComputer, OpenAI explained its latest model, which is presently successful preview, supports “advanced reasoning” and was built to amended spot these kinds of abuses: "We're perpetually making ChatGPT amended astatine stopping deliberate attempts to instrumentality it, without losing its helpfulness aliases creativity,” nan company’s spokesperson told nan publication.

“Our latest o1 reasoning exemplary is our astir tin and safest yet, importantly outperforming erstwhile models successful resisting deliberate attempts to make unsafe content."

OpenAI praised nan researchers, saying these kinds of papers thief ChatGPT get better.

According to nan US government, sound scams are considered reasonably common. The premise is simple: an attacker would telephone nan unfortunate connected nan telephone and, while pretending to thief lick a problem, really scam them retired of money aliases sensitive information.

In galore cases, nan onslaught first starts pinch a browser popup showing a clone microorganism warning, from a clone antivirus company. The popup urges nan unfortunate to telephone nan provided telephone number and “clean” their device. If nan unfortunate calls nan number, nan scammer picks up and guides them done nan process, which concludes pinch nan nonaccomplishment of data, aliases funds.

More from TechRadar Pro

• ChatGPT could beryllium worse than cryptocurrency erstwhile it comes to scams
• Here's a database of nan best firewalls today
• These are nan best endpoint protection tools correct now