Former Pakistani Prime Minister Imran Khan Addresses Supporters from Jail Via Deepfake Generative AI Voice Clone

Pakistan’s former Prime Minister Imran Khan delivered a campaign speech from prison using a voice clone created with deepfake voice cloning technology, according to a report by The Guardian. Khan’s team utilized synthetic audio tools developed by ElevenLabs, a startup specializing in synthetic audio, to create the voice clone. The synthetic voice was introduced during a virtual rally and overlaid on historical footage.

Synthetic Speaker

Khan provided an outline of the text, which was then expanded by members of his political party into a final script. The team used archival samples of Khan’s past speeches to create a model that could mimic his voice and rhetorical style. The final audio was incorporated into a virtual rally broadcast, with captions informing viewers that the voice was digitally recreated.

Overcoming Media Blackout

Since Khan’s arrest in May, he has been subject to a media blackout. However, through the use of deepfake voice cloning, Khan was able to bypass this censorship and reach his supporters. The virtual rally garnered over 4.5 million views on social platforms, despite attempts at internet censorship. While the reaction to the synthetic voice was mixed, this demonstrates how AI impersonation can be utilized to evade state suppression, albeit with limitations and considerations.

Politicians Embrace AI for Campaigns

Khan’s use of deepfake generative AI voice cloning is part of a growing trend among politicians to employ synthetic media and generative AI in their campaigns. New York City Mayor Eric Adams, for example, utilized a generative AI voice clone that spoke multiple languages in robocalls to voters. Concerns over the use of synthetic media in political campaigns have prompted Google to revise its rules for political ads, requiring generative AI disclaimers ahead of the 2024 elections.

In conclusion, Imran Khan’s use of a deepfake generative AI voice clone to address his supporters from jail highlights the potential of AI impersonation to overcome media blackout and censorship. However, the use of synthetic media in politics also raises concerns and calls for regulations. As technology continues to advance, it is important to strike a balance between the benefits and risks presented by AI in political campaigns.