Telegram issue: 4 million users use AI bots to create deepfake nudes of people every month

Telegram is being used for creating and distributing deepfake nudes of people by millions of users every month, a new report has highlight. The Pavel Durov-owned platform hosts at least 50 bots that are being used by more than 4 million monthly users for creating deepfake nudes of women, in some cases even teenage girls.

According to a study by Wired, two of the 50 Telegram bots it analysed had more than 4,00,000 lakh monthly users each, while 14 other bots had more than 1,00,000 members each. These bots are supported by at least 25 Telegram Channels with more than three million subscribers "where people can subscribe to newsfeed-style updates". These channels alert subscribers about the new features being offered by the bot and the special token that can be purchased to operate them and create images. These Telegram channels, as per the study, also act as places where users of the bots can find links to new bots in case they are removed by the company.

Furthermore, the study says that these bots come with varying capabilities. While some claim that they can 'remove clothes' from photos, others claim to create deepfake images of people in 'various sexual acts'. Then there are other bots that require users to share multiple images to create the desired image and train their AI models in the process. The study also claims that while some bots refer to 'nudity and removing women’s clothes' in their names, others don't give explicit material in their name or landing page.

What's exacerbating the situation is that Telegram's recommendation engine highlights 'similar channels' to the users who land on one such channel creating deepfake nude images. This helps users in bouncing between images and channels and increasing usage. Additionally, Telegram can force bot developers to accept the terms of usage, which can prevent users from uploading images of individuals without consent and, in the process, help curb the issue. However, the study says that 'there appears to be little or no enforcement of these rules'.

"It is really concerning that these tools—which are really ruining lives and creating a very nightmarish scenario primarily for young girls and for women—are still so easy to access and to find on the surface web, on one of the biggest apps in the world," deepfake expert, Henry Ajder, said in a statement to the publication.

What is Telegram doing about it?

While Telegram hasn't commented on the matter, it has removed nearly 75 such bots from its platform.

It is worth noting that this report comes shortly after the Telegram founder was arrested at an airport outside of Paris in France for reportedly enabling criminal activity, including the distribution of child abuse images and drug trafficking images, on the platform.