Engineering practice behind dall-e 2: ensure that the output of the model complies with the content policy

 To share with the audience  DALL·E 2  The magic of , We need to reduce the risks associated with powerful image generation models . So , We have set up various protective measures , To prevent generated images from violating our content policy . This article focuses on pre training mitigation measures , A subset of these safeguards , Directly modifying  DALL·E 2  Data learned from . especially ,DALL·E 2  Use hundreds of millions of subtitle images from the Internet for training , We delete and reweight some of these images to change the learning content of the model . This article is divided into three parts , Each section describes different pre training mitigation measures ： In the first part , We described how we started from  DALL·E 2  The training data set filters out violent and pornographic images . Without such mitigation measures , The model will learn to generate graphics or explicit images when prompted , You may even inadvertently return these images in response to seemingly harmless prompts . In the second part , We found that filtering training data can magnify the deviation , And describe our technology to mitigate this impact . for example , Without such mitigation measures , We noticed that , Compared with the model trained on the original data set , Models trained on filtered data sometimes produce more images of men and less images of women . In the last part , We turn to the question of memory , Discovery image  DALL·E 2  Such models can sometimes reproduce images they have trained , Instead of creating new images . In practice , We found that this image backflow is caused by multiple copies of images in the dataset , And alleviate this problem by deleting images that are visually similar to other images in the dataset .