Generative AI boasts a wide range of extensively documented applications, spanning everything from language processing and image generation to music creation and data augmentation. The issue appears to be emerging in state-led initiatives, rather than just remaining a localized problem.
According to a report from Massachusetts-based threat intelligence firm Recorded Future, one recent marketing campaign is believed to have been aided by ElevenLabs’ business AI voice technology products, which were publicly released.
A report reveals a Russian-backed marketing campaign, codenamed “Operation Undercut,” aimed at eroding European support for Ukraine by disseminating disinformation through the use of artificial intelligence-generated voiceovers in fabricated news videos.
Films targeting European viewers often lampooned Ukrainian officials for alleged corruption and cast doubt on the effectiveness of military aid to Ukraine, exploring a range of other themes in the process. For example, a promotional clip claimed, without exception, that even advanced jamming devices used by American Abrams tanks are powerless against incoming missiles, thus underscoring the futility of providing cutting-edge armor to Ukraine.
According to the report, it is highly likely that the video producers employed voice-generated artificial intelligence in conjunction with ElevenLabs technology to create a more credible appearance for their content. To verify this, Recorded Future’s researchers uploaded the audio clips to ElevenLabs’ technology, a platform that enables anyone to “analyze whether an audio file was generated using ElevenLabs,” and obtained a positive match.
Despite no response from ElevenLabs, Recorded Future did acknowledge the potential utilization of several AI-powered voice technology tools, but failed to name any besides ElevenLabs.
The effectiveness of AI voice technology was unwittingly demonstrated by the creators of a marketing campaign who initially produced films featuring human voiceovers with a distinct Russian accent. In contrast, the AI-generated voiceovers delivered in various European languages – including English, French, German, and Polish – were devoid of any foreign-sounding accents.
Accordingly, AI technology enabled the swift dissemination of misleading video snippets across multiple European languages, including English, German, French, Polish, and Turkish, all courtesy of ElevenLabs’ linguistic capabilities.
Recorded Future linked the exercise to the Social Design Company, a Russian-based group that the US government has identified as a key threat actor. This March, authorities uncovered a sophisticated disinformation operation involving more than 60 websites masquerading as legitimate European information sources and using fake social media accounts to disseminate the fabricated content from these spoofed sites. This elaborate scheme was carried out “on behalf of the Authorities of the Russian Federation,” with the U.S. being one of its primary targets. The State Department has always been mindful of the timing.
According to Recorded Future’s analysis, the marketing campaign had a negligible impact on European public opinion.
This isn’t the first time ElevenLabs’ products have been accused of misuse. The company’s technology was behind a robocall impersonating President Joe Biden, which urged voters not to leave and vote during a crucial election in January 2024, according to a voice fraud detection firm’s findings. ElevenLabs revealed that it has introduced novel security features, including the regular suppression of political leaders’ voices.
ElevenLabs effectively detects and prevents “unauthorized, dangerous, or misleading impersonation”, leveraging a suite of tools that combines both automated and human moderation techniques to ensure seamless implementation.
Founded in 2022, ElevenLabs has demonstrated remarkable exponential growth since its inception. According to reports by TechCrunch, the company’s valuation has recently decreased compared to this time last year, but is poised to reach $3 billion in value shortly. The investors behind this acquisition are none other than Andreessen Horowitz and former GitHub CEO Nat Friedman, a powerhouse duo in the tech world.
