Deepfake Pornography: An Unsettling Symphony of AI and Human Vulnerability
In the vast, uncharted territories of the digital world, a sinister phenomenon is proliferating at an alarming rate. It's a world where artificial intelligence (AI) and human vulnerability intertwine in a disturbing combination, creating a shadowy realm of non-consensual pornography. This is the world of deepfake pornography, a burgeoning industry that is as lucrative as it is unsettling.
According to a recent assessment, at least 100,000 deepfake porn videos are readily available on the internet, with hundreds, if not thousands, being uploaded daily. This staggering statistic prompts a chilling question: what is driving the creation of such a vast number of fakes? Is it merely for amusement, or is there a more sinister motive at play?
Recent Trends and Developments
An investigation by India Today’s Open-Source Intelligence (OSINT) team reveals that deepfake pornography is rapidly morphing into a thriving business. AI enthusiasts, creators, and experts are extending their expertise, investors are injecting money, and even small financial companies to tech giants like Google, VISA, Mastercard, and PayPal are being misused in this dark trade. Synthetic porn has existed for years, but advances in AI and the increasing availability of technology have made it easier—and more profitable—to create and distribute non-consensual sexually explicit material. The 2023 State of Deepfake report by Home Security Heroes reveals a staggering 550% increase in the number of deepfakes compared to 2019.
What’s the Matter with Fakes?
But why should we be concerned about these fakes? The answer lies in the real-world harm they cause. India has already seen cases of extortion carried out by exploiting deepfake technology. An elderly man in UP’s Ghaziabad, for instance, was tricked into paying Rs 74,000 after receiving a deep fake video of a police officer. The situation could have been even more serious if the perpetrators had decided to create deepfake porn of the victim.
The danger is particularly severe for women. The 2023 State of Deepfake Report estimates that at least 98 percent of all deepfakes is porn and 99 percent of its victims are women. A study by Harvard University refrained from using the term “pornography” for creating, sharing, or threatening to create/share sexually explicit images and videos of a person without their consent. “It is abuse and should be understood as such,” it states.
Based on interviews of victims of deepfake porn last year, the study said 63 percent of participants talked about experiences of “sexual deepfake abuse” and reported that their sexual deepfakes had been monetised online. It also found “sexual deepfake abuse to be particularly harmful because of the fluidity and co-occurrence of online offline experiences of abuse, resulting in endless reverberations of abuse in which every aspect of the victim’s life is permanently disrupted”.
Creating deepfake porn is disturbingly easy. There are largely two types of deepfakes: one featuring faces of humans and another featuring computer-generated hyper-realistic faces of non-existing people. The first category is particularly concerning and is created by superimposing faces of real people on existing pornographic images and videos—a task made simple and easy by AI tools.
During the investigation, platforms hosting deepfake porn of stars like Jennifer Lawrence, Emma Stone, Jennifer Aniston, Aishwarya Rai, Rashmika Mandanna to TV actors and influencers like Aanchal Khurana, Ahsaas Channa, and Sonam Bajwa and Anveshi Jain were encountered. It takes a few minutes and as little as Rs 40 for a user to create a high-quality fake porn video of 15 seconds on platforms like FakeApp and FaceSwap.
The Modus Operandi
These platforms brazenly flaunt their business association and hide behind frivolous declarations such as: the content is “meant solely for entertainment” and “not intended to harm or humiliate anyone”. However, the irony of these disclaimers is not lost on anyone, especially when they host thousands of non-consensual deepfake pornography.
As fake porn content and its consumers surge, deepfake porn sites are rushing to forge collaborations with generative AI service providers and have integrated their interfaces for enhanced interoperability. The promise and potential of making quick bucks have given birth to step-by-step guides, video tutorials, and websites that offer tools and programs, recommendations, and ratings.
Nearly 90 per cent of all deepfake porn is hosted by dedicated platforms that charge for long-duration premium fake content and for creating porn—of whoever a user wants, and take requests for celebrities. To encourage them further, they enable creators to monetize their content.
One such website, Civitai, has a system in place that pays “rewards” to creators of AI models that generate “images of real people'', including ordinary people. It also enables users to post AI images, prompts, model data, and LoRA (low-rank adaptation of large language models) files used in generating the images. Model data designed for adult content is gaining great popularity on the platform, and they are not only targeting celebrities. Common people are equally susceptible.
Access to premium fake porn, like any other content, requires payment. But how can a gateway process payment for sexual content that lacks consent? It seems financial institutes and banks are not paying much attention to this legal question. During the investigation, many such websites accepting payments through services like VISA, Mastercard, and Stripe were found.
Those who have failed to register/partner with these fintech giants have found a way out. While some direct users to third-party sites, others use personal PayPal accounts to manually collect money in the personal accounts of their employees/stakeholders, which potentially violates the platform's terms of use that ban the sale of “sexually oriented digital goods or content delivered through a digital medium.”
Among others, the MakeNude.ai web app – which lets users “view any girl without clothing” in “just a single click” – has an interesting method of circumventing restrictions around the sale of non-consensual pornography. The platform has partnered with Ukraine-based Monobank and Dublin’s BetaTransfer Kassa which operates in “high-risk markets”.
BetaTransfer Kassa admits to serving “clients who have already contacted payment aggregators and received a refusal to accept payments, or aggregators stopped payments altogether after the resource was approved or completely freeze your funds”. To make payment processing easy, MakeNude.ai seems to be exploiting the donation ‘jar’ facility of Monobank, which is often used by people to donate money to Ukraine to support it in the war against Russia.
The Indian Scenario
India currently is on its way to design dedicated legislation to address issues arising out of deepfakes. Though existing general laws requiring such platforms to remove offensive content also apply to deepfake porn. However, persecution of the offender and their conviction is extremely difficult for law enforcement agencies as it is a boundaryless crime and sometimes involves several countries in the process.
A victim can register a police complaint under provisions of Section 66E and Section 66D of the IT Act, 2000. Recently enacted Digital Personal Data Protection Act, 2023 aims to protect the digital personal data of users. Recently Union Government issued an advisory to social media intermediaries to identify misinformation and deepfakes. Comprehensive law promised by Union IT minister Ashwini Vaishnav will be able to address these challenges.
Conclusion
In the end, the unsettling dance of AI and human vulnerability continues in the dark web of deepfake pornography. It's a dance that is as disturbing as it is fascinating, a dance that raises questions about the ethical use of technology, the protection of individual rights, and the responsibility of financial institutions. It's a dance that we must all be aware of, for it is a dance that affects us all.
References
- https://www.indiatoday.in/india/story/deepfake-porn-artificial-intelligence-women-fake-photos-2471855-2023-12-04
- https://www.hindustantimes.com/opinion/the-legal-net-to-trap-peddlers-of-deepfakes-101701520933515.html
- https://indianexpress.com/article/opinion/columns/with-deepfakes-getting-better-and-more-alarming-seeing-is-no-longer-believing/
Related Blogs
Introduction
The Union Minister of Information and Broadcasting Ashwini Vaishnaw addressed the Press Council of India on the occasion of National Press Day regarding emergent concerns in the digital media and technology landscape. Union Minister of Information and Broadcasting Ashwini Vaishnaw has identified four major challenges facing news media in India, including fake news, algorithmic bias, artificial intelligence, and fair compensation. He emphasized the need for greater accountability and fairness from Big Tech to combat misinformation and protect democracy. Vaishnaw argued that platforms do not verify information posted online, leading to the spread of false and misleading information. He called on online platforms and Big Tech to combat misinformation and protect democracy.
Key Concerns Highlighted by Union Minister Ashwini Vaishnaw
- Misinformation: Due to India's unique sensitivities, digital platforms should adopt country-specific responsibilities and metrics. The Minister also questioned the safe harbour principle, which shields platforms from liability for user-generated content.
- Algorithmic Biases: The prioritisation of viral content, which is often divisive, by social media algorithms can have serious implications on societal peace.
- Impact of AI on intellectual Property: The training of AI on pre-existing datasets presents the ethical challenge of robbing original creators of their rights to their intellectual property
- Fair compensation: Traditional news media is increasingly facing financial strain since news consumption is shifting rapidly to social media platforms, creating uneven compensation dynamics.
Cyberpeace Insights
- Misinformation: Marked by routine upheavals and moral panics, Indian society is vulnerable to the severe impacts of fake news, including mob violence, political propaganda, health misinformation and more. Inspired by the EU's Digital Services Act, 2022, and other related legislation that addresses hate speech and misinformation, the Indian Minister has called for revisiting the safe harbour protection under Section 79 of the IT Act, 2000. However, any legislation on misinformation must strike a balance between protecting the fundamental rights to freedom of speech, and privacy while safeguarding citizens from its harmful effects.
- Algorithmic Biases: Social media algorithms are designed to boost user engagement since this increases advertisement revenue. This leads to the creation of filter bubbles- exposure to personalized information online and echo chambers interaction with other users with the same opinions that align with their worldview. These phenomena induce radicalization of views, increase intolerance fuel polarization in public discourse, and trigger the spread of more misinformation. Tackling this requires algorithmic design changes such as disincentivizing sensationalism, content labelling, funding fact-checking networks, etc. to improve transparency.
- Impact of AI on Intellectual Property: AI models are trained on data that may contain copyrighted material. It can lead to a loss of revenue for primary content creators, while tech companies owning AI models may financially benefit disproportionately by re-rendering their original works. Large-scale uptake of AI models will significantly impact fields such as advertising, journalism, entertainment, etc by disrupting their market. Managing this requires a push for Ethical AI regulations and the protection of original content creators.
Conclusion: Charting a Balanced Path
The socio-cultural and economic fabric of the Indian subcontinent is not only distinct from the rest of the world but has cross-cutting internal diversities, too. Its digital landscape stands at a crossroads as rapid global technological advancements present increasing opportunities and challenges. In light of growing incidents of misinformation on social media platforms, it is also crucial that regulators consider framing rules that encourage and mandate content verification mechanisms for online platforms, incentivizing them to adopt advanced AI-driven fact-checking tools and other relevant measures. Additionally, establishing public-private partnerships to monitor misinformation trends is crucial to rapidly debunking viral falsehoods. However ethical concerns and user privacy should be taken into consideration while taking such steps. Addressing misinformation requires a collaborative approach that balances platform accountability, technological innovation, and the protection of democratic values.
Sources
- https://www.indiatoday.in/india/story/news-media-4-challenges-ashwini-vaishnaw-national-press-day-speech-big-tech-fake-news-algorithm-ai-2634737-2024-11-17
- https://ec.europa.eu/commission/presscorner/detail/en/ip_24_881
- https://www.legaldive.com/news/digital-services-act-dsa-eu-misinformation-law-propaganda-compliance-facebook-gdpr/691657/
- https://www.fondationdescartes.org/en/2020/07/filter-bubbles-and-echo-chambers/
- https://www.google.com/searchq=News+Media+Bargaining+Code&oq=News+Media+Bargaining+Code&gs_lcrp=EgZjaHJvbWUyBggAEEUYOTIHCAEQABiABDIHCAIQABiABDIHCAMQABiABDIHCAQQABiABDIHCAUQABiABDIICAYQABgWGB4yCAgHEAAYFhgeMggICBAAGBYYHjIICAkQABgWGB7SAQcyMjVqMGo3qAIIsAIB&sourceid=chrome&ie=UTF-8
SVIMS Director and Vice-Chancellor B. Vengamma lighting a lamp to formally launch the cybercrime awareness programme conducted by the police department for the medical students in Tirupati on Wednesday.
An awareness meet on safe Internet practices was held for the students of Sri Venkateswara University University (SVU) and Sri Venkateswara Institute of Medical Sciences (SVIMS) here on Wednesday.
“Cyber criminals on the prowl can easily track our digital footprint, steal our identity and resort to impersonation,” cyber expert I.L. Narasimha Rao cautioned the college students.
Addressing the students in two sessions, Mr. Narasimha Rao, who is a Senior Manager with CyberPeace Foundation, said seemingly common acts like browsing a website, and liking and commenting on posts on social media platforms could be used by impersonators to recreate an account in our name.
Turning to the youth, Mr. Narasimha Rao said the incognito mode and Virtual Private Network (VPN) used as a protected network connection do not ensure total privacy as third parties could still snoop over the websites being visited by the users. He also cautioned them tactics like ‘phishing’, ‘vishing’ and ‘smishing’ being used by cybercriminals to steal our passwords and gain access to our accounts.
“After cracking the whip on websites and apps that could potentially compromise our security, the Government of India has recently banned 232 more apps,” he noted.
Additional Superintendent of Police (Crime) B.H. Vimala Kumari appealed to cyber victims to call 1930 or the Cyber Mitra’s helpline 9121211100. SVIMS Director B. Vengamma stressed the need for caution with smartphones becoming an indispensable tool for students, be it for online education, seeking information, entertainment or for conducting digital transactions.
Introduction
In the era of the internet where everything is accessible at your fingertip, a disturbing trend is on the rise- over 90% of websites containing child abuse material now have self-generated images, obtained from victims as young as three years old. A shocking revelation, shared by the (IWF) internet watch foundation, The findings of the IWF have caused concern about the increasing exploitation of children below the age of 10. The alarming trend highlights the increasing exploitation of children under the age of 10, who are coerced, blackmailed, tricked, or groomed into participating in explicit acts online. The IWF's data for 2023 reveals a record-breaking 275,655 websites hosting child sexual abuse material, with 92% of them containing such "self-generated" content.
Disturbing Tactics Shift
Disturbing numbers came that, highlight a distressing truth. In 2023, 275,655 websites were discovered to hold child sexual abuse content, reaching a new record and reflecting an alarming 8% increase over the previous year. What's more concerning is that 92% of these websites had photos or videos generated by the website itself. Surprisingly, 107,615 of these websites had content involving children under the age of ten, with 2,500 explicitly featuring youngsters aged three to six.
Profound worries
Deep concern about the rising incidence of images taken by extortion or coercion from elementary school-aged youngsters. This footage is currently being distributed on very graphic and specialised websites devoted to child sexual assault. The process begins in a child's bedroom with the use of a camera and includes the exchange, dissemination, and gathering of explicit content by devoted and determined persons who engage in sexual exploitation. These criminals are ruthless. The materials are being circulated via mail, instant messaging, chat rooms, and social media platforms, (WhatsApp, Telegram, Skype, etc.)
Live Streaming of such material involves real-time broadcast which again is a major concern as the nature of the internet is borderless the access to such material is international, national, and regional, which even makes it difficult to get the predators and convict them. With the growth, it has become easy for predators to generate “self-generated “images or videos.
Financial Exploitation in the Shadows: The Alarming Rise of Sextortion
Looking at the statistics globally there have been studies that show an extremely shocking pattern known as “sextortion”, in which adolescents are targeted for extortion and forced to pay money under the threat of exposing images to their families or relatives and friends or on social media. The offender's goal is to obtain sexual gratification.
The financial variation of sextortion takes a darker turn, with criminals luring kids into making sexual content and then extorting them for money. They threaten to reveal the incriminating content unless their cash demands, which are frequently made in the form of gift cards, mobile payment services, wire transfers, or cryptocurrencies, are satisfied. In this situation, the predators are primarily driven by money gain, but the psychological impact on their victims is as terrible. A shocking case was highlighted where an 18-year-old was landed in jail for blackmailing a young girl, sending indecent images and videos to threaten her via Snapchat. The offender was pleaded guilty.
The Question on Security?
The introduction of end-to-end encryption in platforms like Facebook Messenger has triggered concerns within law enforcement agencies. While enhancing user privacy, critics argue that it may inadvertently facilitate criminal activities, particularly the exploitation of vulnerable individuals. The alignment with other encrypted services is seen as a potential challenge, making it harder to detect and investigate crimes, thus raising questions about finding a balance between privacy and public safety.
One of the major concerns in the online safety of children is the implementation of encryption by asserting that it enhances the security of individuals, particularly children, by safeguarding them from hackers, scammers, and criminals. They underscored their dedication to enforcing safety protocols, such as prohibiting adults from texting teenagers who do not follow them and employing technology to detect and counteract bad conduct.
These distressing revelations highlight the urgent need for comprehensive action to protect our society's most vulnerable citizens i.e., children, youngsters, and adolescents throughout the era of digital progress. As experts and politicians grapple with these troubling trends, the need for action to safeguard kids online becomes increasingly urgent.
Role of Technology in Combating Online Exploitation
With the rise of technology, there has been a rise in online child abuse, technology also serves as a powerful tool to combat it. The advanced algorithms and use of Artificial intelligence tools can be used to disseminate ‘self-generated’ images. Additional tech companies can collaborate and develop some effective solutions to safeguard every child and individual.
Role of law enforcement agencies
Child abuse knows no borders, and addressing the issues requires legal intervention at all levels. National, regional, and international law enforcement agencies investigate online child sexual exploitation and abuse and cooperate in the investigation of these cybercrimes, Various investigating agencies need to have mutual legal assistance and extradition, bilateral, and multilateral conventions to conduct to identify, investigate, and prosecute perpetrators of online child sexual exploitation and abuse. Apart from this cooperation between private and government agencies is important, sharing the database of perpetrators can help the agencies to get them caught.
How do you safeguard your children?
Looking at the present scenario it has become a crucial part of protecting and safeguarding our children online against online child abuse here are some practical steps that can help in safeguarding your loved one.
- Open communication: Establish open communication with your children, make them feel comfortable, and share your experiences with them, make them understand what good internet surfing is and educate them about the possible risks without generating fear.
- Teach Online Safety: educate your children about the importance of privacy and the risks associated with it. Teach them strong privacy habits like not sharing any personal information with a stranger on any social media platform. Teach them to create some unique passwords and to make them aware not to click on any suspicious links or download files from unknown sources.
- Set boundaries: As a parent set rules and guidelines for internet usage, set time limits, and monitor their online activities without infringing their privacy. Monitor their social media platforms and discuss inappropriate behaviour or online harassment. As a parent take an interest in your children's online activities, websites, and apps inform them, and teach them online safety measures.
Conclusion
The predominance of self-generated' photos in online child abuse content necessitates immediate attention and coordinated action from governments, technology corporations, and society as a whole. As we negotiate the complicated environment of the digital age, we must be watchful, modify our techniques, and collaborate to defend the innocence of the most vulnerable among us. To combat online child exploitation, we must all work together to build a safer, more secure online environment for children all around the world.
References
- https://www.the420.in/over-90-of-websites-containing-child-abuse-feature-self-generated-images-warns-iwf/
- https://news.sky.com/story/self-generated-images-found-on-92-of-websites-containing-child-sexual-abuse-with-victims-as-young-as-three-13049628
- https://www.firstpost.com/world/russia-rejects-us-proposal-to-resume-talks-on-nuclear-arms-control-13630672.html
- https://www.news4hackers.com/iwf-warns-that-more-than-90-of-websites-contain-self-generated-child-abuse-images/
