OpenAI, the renowned artificial intelligence research organization behind ChatGPT, has announced that it is exploring how to “responsibly” generate porn using its AI technology. In a newly released draft document titled Model Spec, the organization provides guidelines on how it aims for its AI models to behave.

Within the document, OpenAI acknowledges the exploration of age-appropriate explicit content through its API and ChatGPT.

But exploring and actually producing AI porn are two very different things. Niko Felix (OpenAI Spokesperson) says, “We do not have any intention for our models to generate AI porn.”

Currently, OpenAI’s usage policies prohibit the creation of sexually explicit or even suggestive material. However, the Model Spec commentary reveals that the company is re-evaluating this stance. According to the commentary, “We’re exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts through the API and ChatGPT. We look forward to better understanding user and societal expectations of model behavior in this area.”

OpenAI describes NSFW (Not Safe For Work) content in the document as including “erotica, extreme gore, slurs, and unsolicited profanity.” While the draft guidelines do not offer a detailed explanation of what constitutes responsible generation of explicit content, it does signal a willingness to engage with user feedback to shape its policies.

Despite the draft guidelines, OpenAI remains cautious. Niko Felix, a spokesperson for OpenAI, emphasized that the organization does not intend for its models to generate AI porn. However, Joanne Jang, one of the writers of Model Spec, admitted that users might ultimately define what constitutes porn, noting, “Depends on your definition of porn.”

Grace McGuire, another OpenAI spokesperson, added that Model Spec seeks “more transparency about the development process and [aims to] get a cross-section of perspectives and feedback from the public, policymakers, and other stakeholders.”

The potential introduction of explicit AI-generated content has raised alarms among experts and advocates. Danielle Keats Citron, a professor at the University of Virginia School of Law, warned of the implications, calling OpenAI’s possible foray into explicit content “alarming.”

“Intimate privacy violations, including deepfake sex videos and other nonconsensual synthesized intimate images, are rampant and deeply damaging,” Citron stated. “We now have clear empirical support showing that such abuse costs targeted individuals crucial opportunities, including to work, speak, and be physically safe.”

Citron’s concerns are not unfounded. In March, two teenage boys in Florida were arrested for distributing AI-generated nudes depicting their middle school classmates without consent.

OpenAI faces a significant challenge in effectively moderating explicit content generation. Despite its usage policies prohibiting impersonation without permission, it remains to be seen how well OpenAI can prevent non-consensual imagery. The example of Microsoft’s recent adjustments to its AI tool following an incident involving explicit images of Taylor Swift underscores the complexity of the issue.

OpenAI’s exploration of responsible explicit content generation through AI represents both an opportunity and a challenge. As the organization seeks input from the public and policymakers, it must balance user demand with the ethical implications of NSFW content.