In October 2026, a reporter from the "Rule of Law Daily" learned that the People's Court of Yuyao City, Zhejiang Province concluded a case of using generative artificial intelligence to produce pornographic materials for profit.
Zheng, who was once a core programmer in the field of AI (artificial intelligence) at a major domestic game company, "jailbroken" the model by rewriting the system prompt words, setting up character cards to induce dialogue scenes, and developing pornographic picture generation functions. He "trained" the AI model into a criminal tool specialized in outputting obscene materials, providing technical conditions for users to output obscene content... In the end, Zheng and others were found by the court to be guilty of producing pornographic materials for profit.
AI has become a "yellow" tool
The defendant Zheng was once an employee of a major domestic game manufacturer and had certain influence in the field of AI technology. He also went abroad for further study and was a high-tech talent.
In July 2024, after Zheng came into contact with an overseas AI porn chatbot, he came up with the idea of developing similar software to make illegal profits. Subsequently, Zheng, his colleague Xie, and six people including Yan whom he met online formed a "R&D team".
Zheng and others used technical means to "jailbreak" the basic model, breaking through the security restrictions on pornographic content built into the large model. After users recharge and pay, they can select character cards in the model to generate pornographic chat text, and then further generate obscene pictures by consuming different points.
After a period of debugging, in January 2025, the generative artificial intelligence model developed by Zheng and others was officially launched on the overseas platform and connected to multiple recharge channels to collect fees from users.
In July 2025, the public security organs launched a case for investigation after discovering clues about illegal crimes. In September of the same year, all six defendants including Zheng were arrested and brought to justice.
After verification, during the operation of the AI model involved, a total of more than 370,000 static pictures and more than 6,000 dynamic pictures were generated in response to user needs, and a total user recharge amount of 24,000 US dollars, equivalent to more than 170,000 yuan, was collected. Among the 12,500 static pictures and full dynamic pictures randomly selected from the above pictures, more than 9,000 still pictures and more than 5,000 dynamic pictures were identified as obscene materials.
Identified as "producing obscene materials"
The core dispute in this case is whether the output of obscene content by Zheng and other generative artificial intelligence service providers through technical means to break through the ethical and safety restrictions of model algorithms is considered "producing obscene materials."
Zhu Zhanglin, the judge in charge of the case and first-level judge of the Criminal Tribunal of Yuyao City Court, said that "making" obscene materials in the sense of criminal law refers to the act of expressing specific thoughts, concepts or emotions in obscene materials through conception, design, arrangement and other methods.
"In the traditional context, "production" is often reflected in direct actions such as manual creation and editing. However, after entering the era of artificial intelligence, the production method of obscene materials is changing from "artificial creation" to "AI generation", and criminal law evaluation should adapt to the technical reality. Generative artificial intelligence models usually have built-in content security filtering mechanisms to intercept bad and illegal information, and are not themselves criminal tools. In the case, the defendant Zheng and others continued to write and modify system prompt words, and continued to test and break through the ethical and safety restrictions of the AI model through various technical means, making it violate the original safety design and trigger user instructions to generate obscene content. The essence of this series of actions is that the defendant actively "trained" the AI model from a general tool to a criminal tool specialized in outputting obscene materials.
During the trial, the defendant argued that users had the ability to select roles and enter specific instructions. In this regard, the court held that although users had the ability to select roles and enter specific instructions, their actions were limited within the technical framework pre-established by the defendant. The defendant had substantial control over the content output of the AI model, and there was a direct causal relationship under criminal law between his behavior and the generation of obscene content. Therefore, generative artificial intelligence service providers that use technical means to break through the ethical and safety restrictions of model algorithms and output obscene content should be recognized as "producing obscene materials."
After trial, the Yuyao City Court held that the defendant Zheng and six other people ganged up to use AI technology to produce obscene materials for the purpose of making profits. The circumstances were serious and their actions constituted the crime of producing obscene materials for profit. Taking into account the facts of the crime, the circumstances, the attitude of confession and repentance, and the degree of harm to society, the defendant Zheng and other six people were sentenced to fixed-term imprisonment ranging from three years and one month to three years and ten months, and were also fined. After the first-instance judgment, none of the parties filed an appeal. At present, the judgment has taken effect.
"Technology neutrality" is not a shield
"'Technology neutrality' is not a shield for illegal crimes. Just because the content is output by AI, it cannot deny the developer's status as a "producer."" Zhu Zhanglin said that this case is a typical case in which AI technology is abused to commit crimes, exposing the criminal risks hidden in the rapid development of generative artificial intelligence. Developers actively tamper with the model security mechanism, build a technical environment that facilitates the production of obscene content, and have substantial control over the generation of illegal content. Even if content output is triggered by user input instructions, the developer is still a producer of obscene materials in the sense of criminal law and should bear corresponding criminal liability.
Zhu Zhanglin pointed out that for such cases, a comprehensive evaluation of the scope of transmission, proliferation risk, and actual harm should be combined to distinguish between batch generation and on-demand generation, open communication and closed interaction, so as to achieve appropriate responsibilities and punishments. It is necessary not only to severely punish acts that are generated in large quantities and spread widely, but also to take into account the technical characteristics of one-click generation and batch output of AI, and to reasonably limit the falsely high number caused by technical characteristics, so that sentencing can accurately adapt to the crime pattern of the AI era.
"While pursuing commercial interests, generative artificial intelligence service providers must strictly abide by the legal bottom line and technological ethics, and proactively fulfill content security management obligations. Any attempt to use technical means to circumvent supervision and break through legal red lines will be severely punished by the law." said Chen Jing, president of the Second Criminal Tribunal of the Ningbo Intermediate People's Court.
Our reporter Wang Chun
Correspondents of this newspaper Zheng Shanshan Zhong Fa
