#FactCheck- AI-Generated Video Falsely Claims Iran Attack on Apple, Microsoft in Israel

The World Economic Forum reported that AI-generated misinformation and disinformation are the second most likely threat to present a material crisis on a global scale in 2024 at 53% (Sept. 2023). Artificial intelligence is automating the creation of fake news at a rate disproportionate to its fact-checking. It is spurring an explosion of web content mimicking factual articles that instead disseminate false information about grave themes such as elections, wars and natural disasters. 

According to a report by the Centre for the Study of Democratic Institutions, a Canadian think tank, the most prevalent effect of Generative AI is the ability to flood the information ecosystem with misleading and factually-incorrect content. As reported by Democracy Reporting International during the 2024 elections of the European Union, Google's Gemini, OpenAI’s ChatGPT 3.5 and 4.0, and Microsoft’s AI interface ‘CoPilot’ were inaccurate one-third of the time when engaged for any queries regarding the election data.   Therefore, a need for an innovative regulatory approach like regulatory sandboxes which can address these challenges while encouraging responsible AI innovation is desired.

What Is AI-driven Misinformation?

False or misleading information created, amplified, or spread using artificial intelligence technologies is AI-driven misinformation. Machine learning models are leveraged to automate and scale the creation of false and deceptive content. Some examples are deep fakes, AI-generated news articles, and bots that amplify false narratives on social media.

The biggest challenge is in the detection and management of AI-driven misinformation. It is difficult to distinguish AI-generated content from authentic content, especially as these technologies advance rapidly. 

AI-driven misinformation can influence elections, public health, and social stability by spreading false or misleading information. While public adoption of the technology has undoubtedly been rapid, it is yet to achieve true acceptance and actually fulfill its potential in a positive manner because there is widespread cynicism about the technology - and rightly so. The general public sentiment about AI is laced with concern and doubt regarding the technology’s trustworthiness, mainly due to the absence of a regulatory framework maturing on par with the technological development. 

Regulatory Sandboxes: An Overview

Regulatory sandboxes refer to regulatory tools that allow businesses to test and experiment with innovative products, services or businesses under the supervision of a regulator for a limited period. They engage by creating a controlled environment where regulators allow businesses to test new technologies or business models with relaxed regulations.

Regulatory sandboxes have been in use for many industries and the most recent example is their use in sectors like fintech, such as the UK’s Financial Conduct Authority sandbox. These models have been known to encourage innovation while allowing regulators to understand emerging risks. Lessons from the fintech sector show that the benefits of regulatory sandboxes include facilitating firm financing and market entry and increasing speed-to-market by reducing administrative and transaction costs. For regulators, testing in sandboxes informs policy-making and regulatory processes. Looking at the success in the fintech industry, regulatory sandboxes could be adapted to AI, particularly for overseeing technologies that have the potential to generate or spread misinformation.

The Role of Regulatory Sandboxes in Addressing AI Misinformation

Regulatory sandboxes can be used to test AI tools designed to identify or flag misinformation without the risks associated with immediate, wide-scale implementation. Stakeholders like AI developers, social media platforms, and regulators work in collaboration within the sandbox to refine the detection algorithms and evaluate their effectiveness as content moderation tools.

These sandboxes can help balance the need for innovation in AI and the necessity of protecting the public from harmful misinformation. They allow the creation of a flexible and adaptive framework capable of evolving with technological advancements and fostering transparency between AI developers and regulators. This would lead to more informed policymaking and building public trust in AI applications.

CyberPeace Policy Recommendations 

Regulatory sandboxes offer a mechanism to predict solutions that will help to regulate the misinformation that AI tech creates. Some policy recommendations are as follows: 

1. Create guidelines for a global standard for including regulatory sandboxes that can be adapted locally and are useful in ensuring consistency in tackling AI-driven misinformation.
2. Regulators can propose to offer incentives to companies that participate in sandboxes. This would encourage innovation in developing anti-misinformation tools, which could include tax breaks or grants.
3. Awareness campaigns can help in educating the public about the risks of AI-driven misinformation and the role of regulatory sandboxes can help manage public expectations.
4. Periodic and regular reviews and updates to the sandbox frameworks should be conducted to keep pace with advancements in AI technology and emerging forms of misinformation should be emphasized.

‍

Conclusion and the Challenges for Regulatory Frameworks

Regulatory sandboxes offer a promising pathway to counter the challenges that AI-driven misinformation poses while fostering innovation. By providing a controlled environment for testing new AI tools, these sandboxes can help refine technologies aimed at detecting and mitigating false information. This approach ensures that AI development aligns with societal needs and regulatory standards, fostering greater trust and transparency. With the right support and ongoing adaptations, regulatory sandboxes can become vital in countering the spread of AI-generated misinformation, paving the way for a more secure and informed digital ecosystem.

References

• https://www.thehindu.com/sci-tech/technology/on-the-importance-of-regulatory-sandboxes-in-artificial-intelligence/article68176084.ece
• https://www.oecd.org/en/publications/regulatory-sandboxes-in-artificial-intelligence_8f80a0e6-en.html
• https://www.weforum.org/publications/global-risks-report-2024/
• https://democracy-reporting.org/en/office/global/publications/chatbot-audit#Conclusions

‍

Introduction 

Over the past few years, the virtual space has been an irreplaceable livelihood platform for content creators and influencers, particularly on major social media platforms like YouTube and Instagram. Yet, if this growth in digital entrepreneurship is accompanied by anything, it is a worrying trend, a steep surge in account takeover (ATO) attacks against these actors. In recent years, cybercriminals have stepped up the quantity and level of sophistication of such attacks, hacking into accounts, jeopardising the follower base, and incurring economic and reputational damage. They don’t just take over accounts to cause disruption. Instead, they use these hijacked accounts to run scams like fake livestreams and cryptocurrency fraud, spreading them by pretending to be the original account owner. This type of cybercrime is no longer a nuisance; it now poses a serious threat to the creator economy, digital trust, and the wider social media ecosystem.

Why Are Content Creators Prime Targets? 

Content creators hold a special place on the web. They are prominent users who live for visibility, public confidence, and ongoing interaction with their followers. Their social media footprint tends to extend across several interrelated platforms, e.g., YouTube, Instagram, X (formerly Twitter), with many of these accounts having similar login credentials or being managed from the same email accounts. This interconnectivity of their online presence crosses multiple platforms and benefits workflow, but makes them appealing targets for hackers. One entry point can give access to a whole chain of vulnerabilities. Attackers, once they control an account, can wield its influence and reach to share scams, lead followers to phishing sites, or spread malware, all from the cover of a trusted name.

Popular Tactics Used by Attackers

• Malicious Livestream Takeovers and Rebranding - Cybercriminals hijack high-subscriber channels and rebrand them to mimic official channels. Original videos are hidden or deleted, replaced with scammy streams using deep fake personas to promote crypto schemes.
• Fake Sponsorship Offers - Creators receive emails from supposed sponsors that contain malware-infected attachments or malicious download links, leading to credential theft.
• Malvertising Campaigns - These involve fake ads on social platforms promoting exclusive software like AI tools or unreleased games. Victims download malware that searches for stored login credentials.
• Phishing and Social Engineering on Instagram - Hackers impersonate Meta support teams via DMs and emails. They direct creators to login pages that are cloned versions of Instagram's site. Others pose as fans to request phone numbers and trick victims into revealing password reset codes.
• Timely Exploits and Event Hijacking - During major public or official events, attackers often escalate their activity. Hijacked accounts are used to promote fake giveaways or exclusive live streams, luring users to malicious websites designed to steal personal information or financial data.

Real-World Impact and Case Examples

The reach and potency of account takeover attacks upon content creators are far-reaching and profound. In a report presented in 2024 by Bitdefender, over 9,000 malicious live streams were seen on YouTube during a year, with many having been streamed from hijacked creator accounts and reassigned to advertise scams and fake content. Perhaps the most high-profile incident was a channel with more than 28 million subscribers and 12.4 billion total views, which was totally taken over and utilised for a crypto fraud scheme live streaming. Additionally, Bitdefender research indicated that over 350 scam domains were utilised by cybercriminals, directly connected via hijacked social media accounts, to entice followers into phishing scams and bogus investment opportunities. Many of these pieces of content included AI-created deep fakes impersonating recognisable personalities like Elon Musk and other public figures, providing the illusion of authenticity around fake endorsements (CCN, 2024). Further, attackers have exploited popular digital events such as esports events, such as Counter-Strike 2 (CS2), by hijacking YouTube gaming channels and livestreaming false giveaways or referring viewers to imitated betting sites.

Protective Measures for Creators

• Enable Multi-Factor Authentication (MFA) 

Adds an essential layer of defence. Even if a password is compromised, attackers can't log in without the second factor. Prefer app-based or hardware token authentication.

• Scrutinize Sponsorships 

Verify sender domains and avoid opening suspicious attachments. Use sandbox environments to test files. In case of doubt, verify collaboration opportunities through official company sources or verified contacts.

• Monitor Account Activity 

Keep tabs on login history, new uploads, and connected apps. Configure alerts for suspicious login attempts or spikes in activity to detect breaches early. Configure alerts for suspicious login attempts or spikes in activity to detect breaches early.

• Educate Your Team 

If your account is managed by editors or third parties, train them on common phishing and malware tactics.  Employ regular refresher sessions and send mock phishing tests to reinforce awareness.

• Use Purpose-Built Security Tools 

Specialised security solutions offer features like account monitoring, scam detection, guided recovery, and protection for team members. These tools can also help identify suspicious activity early and support a quick response to potential threats.

Conclusion 

Account takeover attacks are no longer random events, they're systemic risks that compromise the financial well-being and personal safety of creators all over the world. As cybercriminals grow increasingly sophisticated and realistic in their scams, the only solution is a security-first approach. This encompasses a mix of technical controls, platform-level collaboration, education, and investment in creator-centric cybersecurity technologies. In today's fast-paced digital landscape, creators not only need to think about content but also about defending their digital identity. As digital platforms continue to grow, so do the threats targeting creators. However, with the right awareness, tools, and safeguards in place, a secure and thriving digital environment for creators is entirely achievable.

References

• https://www.bitdefender.com/en-au/blog/hotforsecurity/account-takeover-attacks-on-social-media-a-rising-threat-for-content-creators-and-influencers
• https://www.arkoselabs.com/account-takeover/social-media-account-takeover/
• https://www.imperva.com/learn/application-security/account-takeover-ato/
• https://www.security.org/digital-safety/account-takeover-annual-report/
• https://www.niceactimize.com/glossary/account-takeover/

‍

Introduction

Misinformation poses a significant challenge to public health policymaking since it undermines efforts to promote effective health interventions and protect public well-being. The spread of inaccurate information, particularly through online channels such as social media and internet platforms, further complicates the decision-making process for policymakers since it perpetuates public confusion and distrust. This misinformation can lead to resistance against health initiatives, such as vaccination programs, and fuels scepticism towards scientifically-backed health guidelines.

Before the COVID-19 pandemic, misinformation surrounding healthcare largely encompassed the effects of alcohol and tobacco consumption, marijuana use, eating habits, physical exercise etc. However, there has been a marked shift in the years since. One such example is the outcry against palm oil in 2024: it is an ingredient prevalent in numerous food and cosmetic products, and came under the scanner after a number of claims that palmitic acid, which is present in palm oil, is detrimental to our health. However, scientific research by reputable institutions globally established that there is no cause for concern regarding the health risks posed by palmitic acid. Such trends and commentaries tend to create a parallel unscientific discourse that has the potential to not only impact individual choices but also public opinion and as a result, market developments and policy conversations. 

A prevailing narrative during the worst of the Covid-19 pandemic was that the virus had been engineered to control society and boost hospital profits. The extensive misinformation surrounding COVID-19 and its management and care increased vaccine hesitancy amongst people worldwide. It is worth noting that vaccine hesitancy has been a consistent trend historically; the World Health Organisation flagged vaccine hesitancy as one of the main threats to global health, and there have been other instances where a majority of the population refused to get vaccinated anticipating unverified, long-lasting side effects. For example, research from 2016 observed a significant level of public skepticism regarding the development and approval process of the Zika vaccine in Africa. Further studies emphasised the urgent need to disseminate accurate information about the Zika virus on online platforms to help curb the spread of the pandemic. 

In India during the COVID-19 pandemic, despite multiple official advisories, notifications and guidelines issued by the government and ICMR, people continued to remain opposed to vaccination, which resulted in inflated mortality rates within the country. Vaccination hesitancy was also compounded by anti-vaccination celebrities who claimed that vaccines were dangerous and contributed in large part to the conspiracy theories doing the rounds. Similar hesitation was noted in misinformation surrounding the MMR vaccines and their likely role in causing autism was examined.  At the time of the crisis, the Indian government also had to tackle disinformation-induced fraud surrounding the supply of oxygens in hospitals. Many critically-ill patients relied on fake news and unverified sources that falsely portrayed the availability of beds, oxygen cylinders and even home set-ups, only to be cheated out of money. 

The above examples highlight the difficulty health officials face in administering adequate healthcare. The special case of the COVID-19 pandemic also highlighted how current legal frameworks failed to address misinformation and disinformation, which impedes effective policymaking. It also highlights how taking corrective measures against health-related misinformation becomes difficult since such corrective action creates an uncomfortable gap in an individual’s mind, and it is seen that people ignore accurate information that may help bridge the gap. Misinformation, coupled with the infodemic trend, also leads to false memory syndrome, whereby people fail to differentiate between authentic information and fake narratives. Simple efforts to correct misperceptions usually backfire and even strengthen initial beliefs, especially in the context of complex issues like healthcare. Policymakers thus struggle with balancing policy making and making people receptive to said policies in the backdrop of their tendencies to reject/suspect authoritative action. Examples of the same can be observed on both the domestic front and internationally. In the US, for example, the traditional healthcare system rations access to healthcare through a combination of insurance costs and options versus out-of-pocket essential expenses. While this has been a subject of debate for a long time, it hadn’t created a large scale public healthcare crisis because the incentives offered to the medical professionals and public trust in the delivery of essential services helped balance the conversation. In recent times, however, there has been a narrative shift that sensationalises the system as an issue of deliberate “denial of care,” which has led to concerns about harms to patients.

Policy Recommendations

The hindrances posed by misinformation in policymaking are further exacerbated against the backdrop of policymakers relying on social media as a method to measure public sentiment, consensus and opinions. If misinformation about an outbreak is not effectively addressed, it could hinder individuals from adopting necessary protective measures and potentially worsen the spread of the epidemic. To improve healthcare policymaking amidst the challenges posed by health misinformation, policymakers must take a multifaceted approach. This includes convening a broad coalition of central, state, local, territorial, tribal, private, nonprofit, and research partners to assess the impact of misinformation and develop effective preventive measures. Intergovernmental collaborations such as the Ministry of Health and the Ministry of Electronics and Information Technology should be encouraged whereby doctors debunk online medical misinformation, in the backdrop of the increased reliance on online forums for medical advice. Furthermore, increasing investment in research dedicated to understanding misinformation, along with the ongoing modernization of public health communications, is essential. Enhancing the resources and technical support available to state and local public health agencies will also enable them to better address public queries and concerns, as well as counteract misinformation. Additionally, expanding efforts to build long-term resilience against misinformation through comprehensive educational programs is crucial for fostering a well-informed public capable of critically evaluating health information.

From an individual perspective, since almost half a billion people use WhatsApp it has become a platform where false health claims can spread rapidly. This has led to a rise in the use of fake health news. Viral WhatsApp messages containing fake health warnings can be dangerous, hence it is always recommended to check such messages with vigilance. This highlights the growing concern about the potential dangers of misinformation and the need for more accurate information on medical matters.

Conclusion

The proliferation of misinformation in healthcare poses significant challenges to effective policymaking and public health management. The COVID-19 pandemic has underscored the role of misinformation in vaccine hesitancy, fraud, and increased mortality rates. There is an urgent need for robust strategies to counteract false information and build public trust in health interventions; this includes policymakers engaging in comprehensive efforts, including intergovernmental collaboration, enhanced research, and public health communication modernization, to combat misinformation. By fostering a well-informed public through education and vigilance, we can mitigate the impact of misinformation and promote healthier communities.

References

1. van der Meer, T. G. L. A., & Jin, Y. (2019), “Seeking Formula for Misinformation Treatment in Public Health Crises: The Effects of Corrective Information Type and Source” Health Communication, 35(5), 560–575. https://doi.org/10.1080/10410236.2019.1573295 
2. “Health Misinformation”, U.S. Department of Health and Human Services. https://www.hhs.gov/surgeongeneral/priorities/health-misinformation/index.html 
3. Mechanic, David, “The Managed Care Backlash: Perceptions and Rhetoric in Health Care Policy and the Potential for Health Care Reform”, Rutgers University. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2751184/pdf/milq_195.pdf 
4. “Bad actors are weaponising health misinformation in India”, Financial Express, April 2024.

https://www.financialexpress.com/life/entertainment-bad-actors-are-weaponising-health-misinformation-in-india-3448871/ 

5. “Role of doctors in eradicating misinformation in the medical sector.”, Times of India, 1 July 2024. https://timesofindia.indiatimes.com/life-style/health-fitness/health-news/national-doctors-day-role-of-doctors-in-eradicating-misinformation-in-the-healthcare-sector/articleshow/111399098.cms 

‍