The Union government has issued a strong notice to X Corp over the alleged misuse of its artificial intelligence service Grok, raising serious concerns about the proliferation of obscene, indecent, and sexually explicit content on the platform and the wider implications for women’s safety, digital dignity, and accountability in the age of generative AI.
The notice, sent by the Ministry of Electronics and Information Technology, reflects growing unease within the government over reports that Grok, an AI-powered service integrated into X, is being exploited by users to create and circulate sexually explicit synthetic images and videos, particularly targeting women. Officials have warned that such practices not only violate existing legal safeguards but also normalise online sexual harassment and deepen the harms associated with non-consensual digital content.
According to the ministry, X Corp has failed to meet its statutory due diligence obligations under India’s digital laws. The company has been directed to submit an action taken report within seventy-two hours, detailing the steps it has adopted to prevent misuse, the responsibilities discharged by its chief compliance officer, and its adherence to mandatory reporting requirements under newly updated criminal and digital governance frameworks. The notice signals a sharper regulatory stance as India grapples with the rapid expansion of AI tools and their misuse on large social media platforms.
Regulatory action, legal obligations, and scrutiny of Grok’s safeguards
In its communication, the Ministry of Electronics and Information Technology stated that X Corp is required to comply with obligations under the Information Technology Act, 2000 and the Information Technology Rules, 2021, which lay down clear responsibilities for intermediaries hosting user-generated content. These rules mandate proactive monitoring, swift takedown of unlawful material, and transparent grievance redressal mechanisms. The ministry’s concern, officials said, stems from evidence that Grok’s generative capabilities are being used to bypass these safeguards.
The notice specifically highlighted instances where users allegedly generated synthetic images and videos that sexualise women in a derogatory manner, infringing upon privacy and personal dignity. Such content, the ministry warned, represents a serious legal violation and a social harm, particularly when amplified through large digital platforms. By enabling or failing to prevent this misuse, authorities argue, intermediaries risk undermining the protections intended by India’s digital regulatory framework.
The government has asked X Corp to explain the role and actions of its chief compliance officer, a statutory position designed to ensure that platforms respond swiftly and effectively to unlawful content. It has also sought clarity on whether X has complied with reporting requirements under the Bharatiya Nagarik Suraksha Sanhita, which obligates platforms to preserve evidence and cooperate with law enforcement in cases involving criminal content.
Beyond immediate compliance, MeitY has directed X to undertake a comprehensive review of Grok’s technical architecture and governance systems. This includes strengthening content moderation protocols, tightening user policies, and ensuring that AI outputs are subject to robust safeguards against misuse. The ministry has emphasised that violators must face swift suspension or termination and that all offending content must be removed promptly without tampering with digital evidence.
Officials have also made it clear that continued non-compliance could have serious consequences. X Corp risks losing its “safe harbour” protection under Section 79 of the IT Act, which shields intermediaries from liability for user-generated content if due diligence is followed. Loss of this protection could expose the platform to criminal and civil liability under multiple laws, including provisions of the Bharatiya Nyaya Sanhita, the Indecent Representation of Women Act, and the Protection of Children from Sexual Offences Act.
Political pressure, women’s safety concerns, and the broader AI governance debate
The government’s action follows mounting political and social pressure over the unchecked misuse of AI tools on social media. Priyanka Chaturvedi had earlier written to Ashwini Vaishnav, urging immediate intervention to ensure that AI applications on X incorporate safeguards to protect women. In her letter, she flagged what she described as a disturbing trend, where fake accounts upload photographs of women and use prompts to digitally alter them, reducing clothing or sexualising images through Grok’s AI features.
Chaturvedi warned that such practices represent a new and dangerous form of digital abuse, one that combines anonymity, scale, and automation. Her intervention amplified concerns voiced by women’s rights advocates and digital safety experts, who argue that generative AI has lowered the barriers to creating harmful content while platforms have struggled to keep pace with effective oversight.
The notice to X was also shared with multiple ministries, commissions, and state authorities, indicating a coordinated approach to addressing AI-enabled obscenity. This broader circulation underscores the government’s view that the issue is not limited to a single platform but reflects systemic challenges posed by rapidly evolving technologies. Regulators are increasingly focused on ensuring that innovation does not outstrip accountability, particularly when fundamental rights such as dignity, privacy, and safety are at stake.
The controversy surrounding Grok has reignited a larger debate about AI governance in India. While the country has positioned itself as a hub for digital innovation, officials have repeatedly stressed that technology must operate within clear ethical and legal boundaries. The misuse of generative AI for sexual harassment and deepfake creation has sharpened calls for platform-level responsibility, transparency in AI design, and stronger enforcement mechanisms.
For X Corp, the notice represents a critical moment in its relationship with Indian regulators. The platform must now demonstrate not only compliance with existing laws but also a proactive commitment to preventing harm arising from its AI services. How it responds could shape future regulatory expectations for AI-enabled platforms operating in India, at a time when governments worldwide are grappling with similar challenges.
The government’s stance sends a broader signal to the tech industry that generative AI tools will be closely scrutinised for their social impact. As AI becomes more deeply embedded in everyday digital interactions, regulators appear determined to ensure that technological power is matched by responsibility, particularly in safeguarding women and vulnerable users from exploitation and abuse.
