A leading diabetes expert has been forced to reassure his patients they’re on the right medication after they saw an AI-generated video of him describing people who prescribe the drug as “idiots”.
Jonathan Shaw, deputy director of the Baker Heart and Diabetes Institute in Melbourne, was alerted to the video when patients started calling his clinic asking for information about a new treatment they saw him promoting on Facebook.
“I clicked on the link and of course it did look exactly like me, but all the words had been changed,” Professor Shaw said.
“I was immediately thinking about the people who might trust what I say, who are now hearing information that I would never give out.”
In the video, since removed by Facebook’s parent company Meta, a fake Karl Stefanovic is seen introducing a fake Professor Shaw, who goes on to say metformin, a first-line treatment for type 2 diabetes, is “not enough to manage diabetes effectively”.
Professor Shaw then introduces a new treatment that “stabilises blood sugar levels in just 24 hours” and “tackles other symptoms of diabetes”.
The product being promoted in the video is a dietary supplement called Glyco Balance, which contains cinnamon sticks, berberine berries and bitter melon.
It’s unclear whether the company that manufactures the supplement, Vellec Group, is responsible for the video.
When Professor Shaw clicked on the fake ad, he was taken to a transcript of an interview he never did with ABC journalist Leigh Sales.
In it, he’s quoted as saying the supplement was developed at the Baker Institute with government funds.
The transcript also claims that metformin, one of the most commonly prescribed drugs in Australia, is dangerous.
While some people can experience side effects from metformin, the drug’s long-term safety has been well studied and established.
Professor Shaw said he was worried the fake ads would stop people with diabetes from taking their medication.
“My major concern isn’t my personal risk or reputation, my major concern is misinformation,” he said.
“This makes it so much more difficult for ordinary members of the public to identify trusted sources.”
Scams getting more sophisticated
In recent years, improvements in generative AI have led to a significant increase in fake ad scams using the identities of celebrities and other public figures to sell products.
Earlier this year, deepfake videos of Dr Karl Kruszelnicki endorsing blood pressure pills were featured in hundreds of ads published on Facebook and Instagram.
Kathy Sundstrom, manager of outreach and education at national anti-scam charity IDCARE, said scams involving AI-generated videos and audio were becoming “the new normal”.
“In the last year, it’s certainly the worst we’ve ever seen,” Ms Sundstrom said.
“As everybody has embraced ChatGPT and other AI platforms, so have criminals. It makes their work easier … and when using deepfake technology, it makes [the scam] look so much more convincing.”Dr Karl deep fakes being used to sell dodgy medicines on Facebook and Instagram
The ABC attempted to contact Glyco Balance manufacturer Vellec Group via email and a phone number listed on the dietary supplement’s website.
A customer service representative was unable to confirm whether Vellec Group was responsible for the fake advertisements and could not provide any further contact information for the company.
Ms Sundstrom said the promotion of medical misinformation was “very concerning” and that scammers may be using health professionals in fake ads to evade Meta’s efforts to remove endorsement scams featuring celebrities.
“[Cyber] crime is a trillion-dollar global industry … They’re always looking for new methods,” she said.
Creating an illusion of credibility
The use of health professionals in fake ads also creates an illusion of credibility, which is often amplified through a labyrinth of other fraudulent AI-generated content.
In the case of Glyco Balance, the deepfake of Professor Shaw appears to be just one part of a coordinated online scam.Tips to protect yourself against scams.
An online search for the dietary supplement brings up a litany of sponsored sites and YouTube videos featuring what appear to be AI-generated customer reviews and testimonials.
It also surfaces a link to ResearchGate, a repository site where scientists keep track of their publications.
An article titled “Glyco Balance Australia Reviews — Does This Blood Sugar Formula Work? Read This!” was published to the site by Stanford researcher “Dr Richard George”.
But Stanford University confirmed it has no record of a Dr Richard George as faculty or staff.
It’s against ResearchGate’s guidelines to publish under a fake name or to advertise products. The ABC contacted ResearchGate about the article and the researcher profile, but it did not respond to our enquiries.
Similarly, the Glyco Balance website states that the product was developed by doctors including US-based physician Alan Gaby, who specialises in nutrition.
The ABC contacted Dr Gaby, and he confirmed he had no involvement with or relationship to the product.
Diabetes sufferers duped
Northern NSW man Michael, who is only comfortable sharing his first name, first heard about Glyco Balance through Facebook.
The 79-year-old, who was diagnosed with diabetes 10 years ago, decided to click on the link after he began experiencing new symptoms.
“I clicked on the ‘read more’ tag and behind it was a prolonged interview with Dr Norman Swan.
“There were many pages detailing the development of the product in conjunction with Oxford University and AstraZeneca — all well-known names.”
Michael noticed there was a time-limited offer — five bottles for the price of three — and decided to give the supplement a go.
But he almost immediately regretted the purchase.
“I went back and started reading the comments … People were saying, ‘I’ve taken one tablet and I feel better,'” he said.
“Come on … You don’t feel better with one tablet.”
Not long after, Michael requested a refund for the $340 he’d spent.
Tackling unfounded health claims
Glyco Balance supplements are purported to contain “natural extracts and essential vitamins” formulated to “support healthy blood sugar levels and lipid profiles”.
But when it comes to managing type 2 diabetes, there is only weak evidence of a possible benefit for a few dietary supplements, and health authorities stress medical treatment should not be replaced with unproven health products.
False advertisements for Glyco Balance also note that it has been “approved” by Australia’s Therapeutic Goods Administration (TGA).
It is not, however, listed on the Australian Register of Therapeutic Goods, and has not been assessed by the TGA for quality, safety, or efficacy.
A spokesperson for the TGA said the regulator was aware of the matter — and others involving health professionals and celebrity endorsements for health products — and was “assessing the situation”.
“We are concerned about the health and safety of Australians accessing unapproved therapeutic products or information about these products online,” the spokesperson said.
“It is important for consumers to be aware that there are no guarantees about the safety or quality of therapeutic goods that are not approved for supply in Australia, such as those purchased over the internet.”
In Australia, health practitioners and medical researchers are also prohibited from endorsing therapeutic goods or making testimonials about them.
Professor Shaw said fake endorsements scams had the potential to create thorny legal issues for the health professionals at the centre of them.
“We have all sorts of regulations that seem to work very well … Then we suddenly get this happening, where all of that has gone out the window,” he said.
The risk of health professionals being impersonated or having their voice cloned by AI was already preventing some from appearing in the media or engaging online, he added.
Whose job is it to address doctor deepfakes?
Meta removed the fake Professor Shaw video on November 29 — nine days after it was reported by the Baker Institute. The institute said it wasn’t until it lodged a separate intellectual property infringement complaint that the video was removed.
“Meta doesn’t want scams on our platforms, and we are continuing to invest in tools and technology to prevent them,” a company spokesperson said.
During a pilot program between April and May, Meta removed more than 8,000 AI-generated scams that used faces of public figures.
The company has also been testing facial recognition technology to combat scammers using celebrities to push their products.Meta is cracking down on fake celebrity scams
But so far, this technology is limited to 50,000 select public figures who must opt in to the program. In the meantime, more medical professionals are being targeted.
“I know other very eminent specialists in diabetes to whom this has also happened,” Professor Shaw said.
A TGA spokesperson said the regulator worked closely with several digital platforms, including Meta, to deter and address unlawful advertising of therapeutic goods.
In 2023–24, the TGA requested the removal of more than 4,800 unlawful advertisements of therapeutic goods from various digital platforms.
Ms Sundstrom said one of the key challenges for tech companies and regulators was the volume of fraudulent material being produced and speed at which it was developed.
She described the situation as a “whack-a-mole”, with new ads often appearing almost immediately after others are removed.
“When we’re looking at [online content], whether it’s a video or a Facebook post … we need to have that default suspicion and go through it and see if it makes sense,” she said.
“It means that you have to give an extra five minutes before you interact with something.
“It’s tiring, but it’s so important.”
