DALL-E

People First

AI should Put People First. See our criteria for this AI Principle.

Aligns with this AI Principle

• OpenAI's Usage Policies do not permit uses that harm human rights, children's rights, identity, integrity, and human dignity. Although the specifics of enforcement remain unclear, OpenAI retains the right to use your inputs or personal information to safeguard these policies.
• OpenAI has taken a number of efforts to reduce DALL-E's ability to generate harmful content. These include filtering the pre-training data to reduce the quantity of graphic sexual and violent content, as well as images of some hate symbols; assessing user inputs (text-to-image prompts, inpainting prompts, and uploaded images) and refusing to generate content for inputs that would lead to a violation of the company's content policy; instituting rate limits; and enforcement via monitoring and human review. In contrast with our review of Stable Diffusion, these efforts are noticeably effective. Importantly, DALL-E 2's system card only includes references to inpainting, not outpainting or variations, when describing these efforts. We don't know if outpainting and variation prompts are also assessed.

Violates this AI Principle

• DALL-E's generated images can be harmful, but at the time of this review, no warnings appear in the user interface when using it. The DALL-E 2 system card acknowledges potential risks, noting DALL-E's "ability to generate content that features or suggests any of the following: nudity/sexual content, hate, or violence/harm," despite attempts to filter this content from the pre-training data. OpenAI also notes that images generated by DALL-E, especially when combined with capabilities like inpainting, "could be used to intentionally mislead or misinform subjects, and could potentially empower information operations and disinformation campaigns." Despite knowing these risks, OpenAI's lack of clear warnings to users while they are using the product is irresponsible.

Important limitations and considerations

• While DALL-E is very easy and intuitive to use, the ethical risks inherent in the system make its ease of use potentially problematic. DALL-E's main barrier to use is a paywall in which users are asked to pay $15 for 115 prompts, each with four image variations.
• Users should educate themselves on best practices in prompting to ensure responsible use of DALL-E. Resources like this can help.

Learning

AI should Promote Learning. See our criteria for this AI Principle.

Aligns with this AI Principle

• While DALL-E is not designed for use in schools, educators can use it in their classrooms with oversight. In particular, DALL-E can be a useful tool in teaching students about how to recognize and question societal biases.
• Because it is a multi-use product, educators and older students—with permission from a parent or legal guardian—can express themselves creatively with DALL-E.
• As with all generative AI tools, DALL-E can be used for creative use cases.
• DALL-E can serve as a tool that enhances students' visual learning and strengthens understanding through impactful imagery, with appropriate teacher or parental oversight.

Important limitations and considerations

• DALL-E is not aligned with learning content standards.
• Users should not attempt to use DALL-E to output images to visualize any process or scene that requires accuracy.
• It is easy when using DALL-E to unwittingly produce images that reinforce unfair bias and stereotypes.

Fairness

AI should Prioritize Fairness. See our criteria for this AI Principle.

Aligns with this AI Principle

• OpenAI conducts its own research into harmful bias and broader issues around fairness and representation, and takes measures to attempt to protect against the worst issues found.
• OpenAI notes that DALL-E's default behavior generates images that overrepresent White skin tones and "Western concepts generally," despite identity-neutral prompts. OpenAI now aims to display a broader range of diversity in DALL-E's outputs unless otherwise specified by the text prompt. However, challenges like gender biases persist, even if skin tones vary.

Important limitations and considerations

• DALL-E continues to demonstrate a tendency toward objectification and sexualization. This is especially the case with inappropriate sexualized representations of women and girls, even with prompts seeking images of women professionals. Numerous studies have shown that greater exposure to images that promote the objectification of women adversely affects the mental and physical health of girls and women.
• Even when instructed to do otherwise, DALL-E is susceptible to generating outputs that perpetuate harmful stereotypes, especially regarding race and gender. Our own testing confirmed this, as well as the ease with which these outputs are generated. Some examples of what we found include:
  - DALL-E reflected and amplified statistical gender stereotypes for occupations (e.g., only female flight attendants, housekeepers, and stay-at-home parents, vs. male software developers). OpenAI has attempted to address these known challenges. While this has appeared to work for some well-tested occupations, especially in generating more variety in skin tones, we found highly gendered results for occupations such as product managers (all male) vs. product marketers (all female), principals (all male) vs. teachers (all female), bankers (all male) vs. bank tellers (all female), and managers (all male) vs. human-resources professionals (all female). 
  - When asked to pair non-White ethnicities with wealth, DALL-E struggled to do so in a photorealistic manner. Instead, it generated cartoons, severely degraded images, and images associated with poverty. 
  - DALL-E generates images that default to Western-centric associations. In images of objects, for example, the images generated from prompts with no identity descriptor perpetuate North American norms of what the objects look like. These models create a version of the world that is American by default.
•  

Important limitations and considerations

• Ensuring that prompts are specific and "grounded" can help reduce certain biases in underspecified prompts, though research indicates that bias can still persist. DALL-E does show helpful tips for creating prompts in its interface.
• The model struggles to represent ideas and people that do not appear in its training data, leading to disparate performance. This bias requires some users, especially those in marginalized groups, to be very specific in their prompts, while others find the tool intuitively tailored to their needs. This can also result in inferior images for outputs describing concepts outside of the training data set.
• OpenAI has made efforts to broaden DALL-E's representation of beauty standards in its output. It is important to be aware that like other image-generating models, DALL-E has been shown to produce images that align with Western-dominated, White-centric beauty and body standards, which can perpetuate unrealistic ideals that overlook cultural and individual diversity.
• It is very easy to unwittingly produce images that reinforce unfair bias and stereotypes using DALL-E. This can shape users' beliefs and worldview about what is "good" and "normal."

Social Connection

AI should Help People Connect. See our criteria for this AI Principle.

Aligns with this AI Principle

• DALL-E can offer a unique way to boost social interaction and understanding. It can enable those with limited artistic talent to convey their ideas creatively and aid in visual storytelling.

Important limitations and considerations

• Even with the protections that OpenAI has put in place, DALL-E can be used to generate images that can harm individuals and groups. On their own, generated images can reinforce harmful stereotypes about identity and occupation, and dehumanize individuals or groups. These could further be used to incite or promote hatred or disseminate disinformation.
• DALL-E's advanced inpainting features present new risks. While innovative and useful in many contexts, the high degree of freedom to alter images means they can be used to perpetuate harms and falsehoods. In OpenAI's words, images that have been changed to, for example, modify, add, or remove clothing or add additional people to an image in compromising ways "could then be used to either directly harass or bully an individual, or to blackmail or exploit them." These features can also be used to create images that intentionally mislead and misinform others. For example, disinformation campaigns can remove objects or people from images or create images that stage false events. Notably, inpainting prompts are also subject to OpenAI's efforts to limit DALL-E's ability to generate harmful content.

Trust

AI should Be Trustworthy. See our criteria for this AI Principle.

Aligns with this AI Principle

• OpenAI has publicly discussed some of the challenges inherent in DALL-E, and how it has worked to prevent them, in the DALL-E 2 system card.
• The OpenAI team embraces peer reviews and invites outside parties to provide feedback and participate in adversarial testing, often called "red teaming."

Violates this AI Principle

• As with all generative AI tools, DALL-E can easily generate or enable false and harmful content, both by reinforcing unfair biases, and by generating images that intentionally mislead or misinform others. Because OpenAI's attempts to limit these are brittle, and images can be further manipulated with generative AI via in- and outpainting, false and harmful visual content can be generated at an alarming speed. We have already seen this in action. OpenAI notes that as image generation matures, it "leaves fewer traces and indicators that outputs are AI-generated, making it easier to mistake generated images for authentic ones and vice versa." In other words, as these AI systems grow, it may become increasingly difficult to separate fact from fiction. This "Liar's Dividend" could erode trust to the point where democracy or civic institutions are unable to function.

Important limitations and considerations

• While the DALL-E sign-up process has an age gate, there is nothing to stop kids from signing up if they choose to give an incorrect birth date.
• It's important to note that the teams assessing DALL-E are predominantly U.S.-based, English-speaking, and have a specific educational background. This naturally constrains the range of perspectives used to evaluate content in various contexts.

Data Use

AI should Protect Our Privacy. See our criteria for this AI Principle.

Aligns with this AI Principle

• DALL-E's terms of service do not allow its use for children under age 13.
• Teens age 13–17 are required to have parental permission to use DALL-E.
• OpenAI has taken steps to reduce the amount of personal information used in DALL-E's training data sets. The company has also worked to prevent the potential for DALL-E to generate exact matches for any of the images in its training data, and limits the ability for DALL-E to generate images of public figures (including politicians) by refusing to generate content for inputs that would lead to a violation of the company's content policy.

Violates this AI Principle

• By default, DALL-E uses your input to further train its models. In other words, any prompts or images you bring into the system—including personal information—will become part of its training data.
• The default use of prompt and image data to further train Dall-E is especially worrying for kids and teens using DALL-E, even if they are not supposed to.

Important limitations and considerations

• You can stop DALL-E from using the data you input, but this option isn't easy to find. If you want to do this, you can begin the process here.
• OpenAI tools were not built with student privacy in mind. Any student using the service will be subject to the same policies as any other consumer.
• It is unclear how aware teachers are of the parental permission requirement for 13- to 17-year-olds, and the tool currently does not ask whether permission has been granted.
• Because of its age policy, DALL-E is not required to comply with (and to our knowledge, does not comply with) important protections such as the Children's Online Privacy and Protection Act (COPPA), the Student Online Personal Information Protection Act (SOPIPA) or the Family Educational Rights and Privacy Act (FERPA). DALL-E is compliant with the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA).

This review is distinct from Common Sense's privacy evaluations and ratings, which evaluate privacy policies to help parents and educators make sense of the complex policies and terms related to popular tools used in homes and classrooms across the country.

Kids' Safety

AI should Keep Kids & Teens Safe. See our criteria for this AI Principle.

Aligns with this AI Principle

• OpenAI has taken a number of efforts to reduce DALL-E 2's ability to generate harmful content. These include filtering the pre-training data to reduce the quantity of graphic sexual and violent content, as well as images of some hate symbols; assessing user inputs (text-to-image prompts, inpainting prompts, and uploaded images) and refusing to generate content for inputs that would lead to a violation of the company's content policy; instituting rate limits; and enforcement via monitoring and human review. In contrast with our review of Stable Diffusion, these efforts have been noticeably effective.

Violates this AI Principle

• DALL-E's "view" of the world can shape impressionable minds, and with little accountability. OpenAI states that "use of DALL-E 2 has the potential to harm individuals and groups by reinforcing stereotypes, erasing or denigrating them, providing them with disparately low quality performance, or by subjecting them to indignity. These behaviors reflect biases present in DALL-E 2 training data and the way in which the model is trained." An example of this comes from the company's realization that the explicit content filter applied to DALL-E's pre-training data actually introduced a net new bias. Essentially, the filter—which was designed to reduce the quantity of pre-training data containing nudity, sexual content, hate, violence, and harm—reduced the frequency of the keyword "woman" by 14%. In contrast, the explicit content filter reduced the frequency of the keyword "man" by only 6%. In other words, OpenAI's attempts to remove explicit material removed enough content representing women that the resulting data set significantly overrepresented content representing men. This offers perspective on how many images on the internet contain explicit sexual content of women. OpenAI also notes that DALL-E's default behavior generates images that overrepresent White skin tones and "Western concepts generally." These propensities towards harm are frighteningly powerful in combination. What happens to our children when they are exposed to the worldview of a biased algorithm repeatedly and over time? What view of the world will they assume is "correct," and how will this inform their interactions with real people and society? Who is accountable for allowing this to happen?
• DALL-E has not been designed in any specific way to protect children. DALL-E has been found to be able to output images that can emotionally and psychologically harm users, perpetuate harmful stereotypes, and promote mis/disinformation.

Important limitations and considerations

• Currently there are no tools that would allow teachers and parents to monitor DALL-E's use in a way that could help evaluate student well-being.

Transparency & Accountability

AI should Be Transparent & Accountable. See our criteria for this AI Principle.

Aligns with this AI Principle

• OpenAI has published a system card for DALL-E 2. System cards are a form of transparency reporting, and are intended to help users understand DALL-E's capabilities, as well as its limitations, risks, and harms. They also provide information about how the creators have sought to mitigate the observed issues.
• Users are able to exert human control over images they produce with DALL-E by modifying prompts to effect change in the generated outputs.
• All users have the option to report problems they encounter with the generated outputs, whether it's a biased outcome, harmful outcome, or image results that don't align with the prompt.

Violates this AI Principle

• The effects of bias and potential harm from images produced by DALL-E can vary based on context, complicating the assessment and mitigation process during image creation. Additionally, content filters can fail to fully capture images that are ethically dubious or violate OpenAI's guidelines, because the potential misuse is more a function of the context in which the image can be used (e.g., disinformation, harassment, bullying, etc.) and not the image itself. Currently, the challenge of identifying deepfakes and determining whether images have been created using DALL-E and products like it remains an unresolved issue, leaving a gap in our ability to mitigate the potential consequences of harmful situations when they occur in the real world. Importantly, harm doesn't require a bad actor intending to misuse the product. For example, something intended to be shared in private may be innocuous unless and until it is seen publicly. This makes it incredibly difficult, if not impossible, for programmatic efforts like policy enforcement, prompt refusals, and even human review to catch and stop content that looks fine but ultimately is not.
• Use of DALL-E can have a direct and significant impact on people, not only from the false and harmful content it may generate, but also, counterintuitively, from what it refuses to generate. Because any efforts to programmatically limit harmful content from surfacing are blunt instruments, whole categories that present any measure of risk may be subject to those efforts. While it is true that this may reduce the amount of harmful content that is generated, it also can cause some marginalized individuals and groups to, in OpenAI's words, suffer the "indignity of having their prompts or generations filtered, flagged, blocked, or not generated in the first place, more frequently than others."

Important limitations and considerations

• OpenAI uses evaluations from various data sources, including real-world use, to compare and improve model versions. And the company addresses issues found through these prompts. However, the presence of consistent evaluations doesn't ensure that root problems are fixed, and it's uncertain if issues might reappear in different contexts.

Review team note:

• While OpenAI engages in transparency reporting, it is highly technical in nature. For those interested:
  - DALL-E 2 research (including a system card, which is a type of transparency reporting) can be found here. 
  - OpenAI publishes a large amount of technical AI research.