Stable Diffusion

People First

AI should Put People First. See our criteria for this AI Principle.

Violates this AI Principle

• LAION 5B, the data set that powers Stable Diffusion, is uncurated. This means that it contains every image found in the Common Crawl repository that has one or more text labels that would be usable for the image-text pairs that the machine learning model needs to match a user's input to images it can use to generate the result. While some filters have been applied, LAION notes that because it is uncurated, the links that make up the data set "may lead to strongly discomforting and disturbing content for a human viewer." Based on LAION's own measurement, 2.9% of the 2.3 billion image-text pairs used by Stable Diffusion are "unsafe"—that is, roughly 68 million unsafe images. All of the technical documentation clearly states that this data set should be used only for research purposes. But Stable Diffusion is available to anyone. The risk this poses to children especially, in terms of what they might see or be exposed to, is unfathomable.

Important limitations and considerations

• While Stable Diffusion is very easy and intuitive to use, the ethical risks described throughout this review make this ease of use even more problematic.
• Those who choose to use Stable Diffusion should educate themselves on best practices in prompting to ensure responsible use to the best extent possible. Resources like this that were created for DALL-E, another text-to-image generative AI model, can help.

• Stable Diffusion does have legal terms, but protections for children are unclear. One reason for this lack of clarity stems from the fact that Stable Diffusion can be accessed in three separate places hosted by Stability AI:
   

  1. Clipdrop, Stability AI's text-to-image editor, a simple interface more accessible to consumers. Clipdrop states in its Terms of Use that users are prohibited from downloading or producing content that, among other prohibited uses, infringes on "public order and morality." Children's rights are not specifically addressed in Clipdrop's terms.
  2. Dreamstudio, another image editor from Stability AI that extends beyond text-to-image prompting with inpainting, outpainting, and image-to-image generation. Dreamstudio's Terms of Service contain an expanded list of prohibited uses and introduce Community Guidelines which note "contributions must be safe, legal, and in accordance with these Terms."
  3. Stability.ai's developer platform, which has its own, more exhaustive Acceptable Use Policy. These terms specifically prohibit use of Stability Technology for, among other prohibited uses, "Exploitation or harm to children, including the solicitation, creation, acquisition, or dissemination of child exploitative content."

  
  Because Stable Diffusion can be accessed from each of these tools, it is unclear which set of terms may be enforced, why there is a discrepancy between these terms, how these terms might be enforced, and by whom.

Learning

AI should Promote Learning. See our criteria for this AI Principle.

Aligns with this AI Principle

• While Stable Diffusion is not designed for use in schools, educators could use it in their classrooms with strong oversight. In particular, Stable Diffusion can be a useful tool in teaching students about how to recognize and question societal biases. Rather than using Stable Diffusion itself for this purpose, however, we recommend exploring the Stable Bias resource instead.

Violates this AI Principle

• The risk of exposure to unsafe content generated by Stable Diffusion is so high that we do not recommend direct use of this tool in any learning environment.

Important limitations and considerations

• Stable Diffusion is not designed for educational use and is not aligned with content standards.
• Users should not attempt to use Stable Diffusion to output images to visualize any process or scene that requires accuracy.
• It is extremely easy when using Stable Diffusion to unwittingly produce images that reinforce unfair bias and stereotypes.

Fairness

AI should Prioritize Fairness. See our criteria for this AI Principle.

Violates this AI Principle

• Despite many public failings, Stable Diffusion continues to produce inappropriately sexualized representations of women and girls, even with neutral prompts or prompts seeking images of women professionals. Numerous studies have shown that greater exposure to images that promote the objectification of women adversely affects the mental and physical health of girls and women. Notably, while this is an issue for all image-to-text generators, it is especially harmful with Stable Diffusion. This is because of the combination of an uncurated data set and minimal protections, such as a refusal to generate images when it detects prompts that violate the company's terms of service.
• Even when instructed to do otherwise, Stable Diffusion is susceptible to generating outputs that perpetuate harmful stereotypes, especially regarding race and gender. A great resource for exploring this further can be found at Stable Bias. Our own testing confirmed this and the ease with which these outputs are generated. Some examples of what we found include:
  • Stable Diffusion attributed being "attractive" to White faces, "emotional" to female faces, "thug" to Black male faces, "terrorist" to stereotypes of Middle Eastern male faces, and "housekeeper" to Black and Brown females.
  • When asked to generate images of a "poor White person," Stable Diffusion would often generate images of Black men. When asked to pair non-White ethnicities with wealth, Stable Diffusion struggled to do so. Instead, it generated images associated with poverty or severely degraded images.
  • Stable Diffusion reflected and amplified statistical gender stereotypes for occupations (e.g., only female flight attendants and stay-at-home parents, male chefs, female cooks, male software developers).

Important limitations and considerations

• Ensuring that prompts are specific and "grounded" can help reduce certain biases in underspecified prompts, though research indicates that bias can still persist.
• Stable Diffusion struggles to represent ideas and people that do not appear in its training data, leading to disparate performance. This bias requires some users, especially those in marginalized groups, to be very specific in their prompts, while others find the tool intuitively tailored to their needs. This can also result in inferior images for outputs describing concepts outside of the training data set.
• It is very easy to unwittingly produce images that reinforce unfair bias and stereotypes using Stable Diffusion. This can shape users' beliefs and worldview about what is "good" and "normal."

Social Connection

AI should Help People Connect. See our criteria for this AI Principle.

Aligns with this AI Principle

• With close monitoring and oversight, Stable Diffusion can offer a unique way to boost social interaction and understanding. It can enable those with limited artistic talent to convey their ideas creatively and aid in visual storytelling.

Important limitations and considerations

• It is very easy to use Stable Diffusion to generate images that can harm individuals and groups. On their own, generated images can reinforce harmful stereotypes about identity and occupation, and dehumanize individuals or groups. These could further be used to incite or promote hatred or disseminate disinformation. This can happen with an ease and speed that creates special concern for use of Stable Diffusion, regardless of whether these activities are against the terms of service.
• Dreamstudio, one of Stability AI's image editors, extends beyond text-to-image prompting with features like inpainting, outpainting, and image-to-image generation. These features present new risks. While innovative and useful in many contexts, the high degree of freedom to alter images means that they can be used to perpetuate harms and falsehoods. Images that have been changed to, for example, modify, add, or remove clothing, or add additional people to an image in compromising ways, could be used to either directly harass or bully an individual, or to blackmail or exploit them. These features can also be used to create images that intentionally mislead and misinform others. For example, misinformation campaigns can remove objects or people from images or create images that stage false events.

Trust

AI should Be Trustworthy. See our criteria for this AI Principle.

Violates this AI Principle

• At the time of this review, there are no age or terms of service gates when signing up to use Stable Diffusion on Clipdrop, Dreamstudio, or–importantly–Stability AI's developer platform where users can easily access the full model. Unless a user seeks out the terms for themselves, they do not know what is and isn't allowed. This also means that there are no protections for children and teens from general use of the product, access to the underlying model, or any open beta testing.
• Stable Diffusion does not appear to add watermarks to users' images, which removes barriers to the spread of misinformation and harmful stereotypes.
• As with all generative AI tools, Stable Diffusion can easily generate or enable false and harmful content, both by reinforcing unfair biases, and by generating images that intentionally mislead or misinform others. Because Stability AI has taken minimal efforts to limit this, and images can be further manipulated with generative AI via in- and outpainting, false and harmful visual content can be generated at an alarming speed. We have already seen this in action. As OpenAI has noted in the context of DALL-E, as image generation matures, it "leaves fewer traces and indicators that outputs are AI-generated, making it easier to mistake generated images for authentic ones and vice versa." In other words, as these AI systems grow, it may become increasingly difficult to separate fact from fiction. This "Liar's Dividend" could erode trust to the point where democracy or civic institutions are unable to function.

Important limitations and considerations

• While this would be a violation of Stable Diffusion's terms of service, it would be very easy to generate images that could be used in misinformation and disinformation campaigns. Many of the organizations responsible for text-to-image generative AI models take steps to avoid the potential to depict public figures. By contrast, Stable Diffusion is capable of generating new content that depicts public figures. This makes it very easy to use it to create deepfakes.

Data Use

AI should Protect Our Privacy. See our criteria for this AI Principle.

Violates this AI Principle

• Because the data set used to power Stable Diffusion is uncurated, it has generated content that includes images with highly sensitive personally identifiable information (PII).
• Many of the organizations responsible for text-to-image generative AI models take steps to avoid the potential to depict public figures. By contrast, Stable Diffusion is capable of generating new content that depicts public figures. This makes it very easy to use it to create deepfakes.

Important limitations and considerations

• Clipdrop's terms state that minors must have permission from their legal representative. Both Dreamstudio and Stability AI's terms state that minors are prohibited from using the services. it is unclear which set of terms may be enforced, why there is a discrepancy between these terms, how these terms might be enforced, and by whom.
• Stable Diffusion was not designed with student privacy in mind. Any student using the service will be subject to the same policies as any other consumer.
• Because of its age policy, Stable Diffusion is not required to comply with (and to our knowledge, does not comply with) important protections such as the Children's Online Privacy and Protection Act (COPPA), the Student Online Personal Information Protection Act (SOPIPA) or the Family Educational Rights and Privacy Act (FERPA). Stable Diffusion is compliant with the General Data Protection Regulation (GDPR).

This review is distinct from Common Sense's privacy evaluations and ratings, which evaluate privacy policies to help parents and educators make sense of the complex policies and terms related to popular tools used in homes and classrooms across the country.

Kids' Safety

AI should Keep Kids & Teens Safe. See our criteria for this AI Principle.

Violates this AI Principle

• Stable Diffusion has been used used to create lifelike images of child sexual abuse, including of the sexual abuse of babies and toddlers. < a href="https://www.bbc.com/news/uk-65932372" class="link">These images have then been sold online. While Stable Diffusion's July 2023 update aimed to prevent it from generating some of the most objectionable content, the open source nature of the model allows for easy removal of those protections in new applications.
• Stable Diffusion's "view" of the world can shape impressionable minds, and with little accountability. Even when instructed to do otherwise, Stable Diffusion is susceptible to generating outputs that perpetuate harmful stereotypes, especially regarding race and gender. We confirmed this repeatedly with our own testing. These behaviors reflect both the way in which the model was trained and—critically—the choice of the data set used to train it. LAION 5B, the data set that powers Stable Diffusion, is uncurated. This means that it contains every image found in the Common Crawl repository that has one or more text labels that would be usable for the image-text pairs that the machine learning model needs to match a user's input to images it can use to generate the result. While some filters have been applied, LAION notes that because itStable Diffusion is uncurated, the links that make up the data set "may lead to strongly discomforting and disturbing content for a human viewer." Based on LAION's own measurement, 2.9% of the 2.3 billion image-text pairs used by Stable Diffusion are "unsafe"—that is, roughly 68 million unsafe images. All of the technical documentation clearly states that this data set should be used only for research purposes. But Stable Diffusion is accessible to anyone, and Stability AI has made the model that powers it available for anyone to download and use for their own purposes. These propensities towards harm are frighteningly powerful. The risk this poses to children especially, in terms of what they might see or be exposed to, is unfathomable. What happens to our children when they are exposed to the worldview of a biased algorithm repeatedly and over time? What view of the world will they assume is "correct," and how will this inform their interactions with real people and society? Who is accountable for allowing this to happen?
• Stable Diffusion has not been designed in any specific way to protect children. Stable Diffusion has been found to be able to output images that can emotionally and psychologically harm users, perpetuate harmful stereotypes, and promote mis/disinformation.
• At the time of this review, there are no age or terms of service gates when signing up to use Stable Diffusion on Clipdrop, Dreamstudio, or—importantly—Stability AI's developer platform where users can easily access the full model. Unless a user seeks out the terms for themselves, they do not know what is and isn't allowed. This also means that there are no protections for children and teens from general use of the product.

Transparency & Accountability

AI should Be Transparent & Accountable. See our criteria for this AI Principle.

Aligns with this AI Principle

• The open source nature of Stable Diffusion means that there is significant transparency.
• Users are able to exert human control over images they produce with Stable Diffusion by modifying prompts to effect change in the generated outputs.

Violates this AI Principle

• The effects of bias and potential harm from images produced by Stable Diffusion can vary based on context, complicating the assessment and mitigation process during image creation. Additionally, content filters can fail to fully capture images that are ethically dubious or violate Stable Diffusion's guidelines, because the potential misuse is more a function of the context in which the image can be used (e.g., disinformation, harassment, bullying, etc.) and not the image itself. Currently, the challenge of identifying deepfakes and determining whether images have been created using Stable Diffusion and products like it remains an unresolved issue, leaving a gap in our ability to mitigate the potential consequences of harmful situations when they occur in the real world. Importantly, harm doesn't require a bad actor intending to misuse the product. For example, something intended to be shared in private may be innocuous unless and until it is seen publicly. This makes it incredibly difficult, if not impossible, for programmatic efforts like policy enforcement, prompt refusals, and even human review to catch and stop content that looks fine but ultimately is not.

Important limitations and considerations

• The available transparency information on popular repositories is not easy for a non-technical audience to understand. This makes it not at all obvious that the creators of Stable Diffusion intend for it to be used as a research tool, not a consumer tool.
• Stable Diffusion can, and has, caused real harm to people, and is not subject to meaningful human control in these instances.
• There are insufficient mechanisms for remediation when harm does happen.