ChatGPT

People First

AI should Put People First. See our criteria for this AI Principle.

Aligns with this AI Principle

• OpenAI's Usage Policies do not permit uses that harm human rights, children's rights, identity, integrity, and human dignity. Although the specifics of enforcement remain unclear, OpenAI retains the right to use your inputs or personal information to safeguard these policies.

Important limitations and considerations

• ChatGPT generates text that often sounds correct, even when it isn't. This makes it very easy for users to be overconfident in ChatGPT's responses.
• When users rely on ChatGPT's responses, this can have the effect of reducing human agency and oversight. The impact of this can be very harmful, depending on what the topic and responses are.
• At times, ChatGPT's responses can include expressions of humility and/or uncertainty. While this can help generally flag for users that responses aren't always correct, unfortunately, these expressions aren't an indication of a response's accuracy.

Learning

AI should Promote Learning. See our criteria for this AI Principle.

Aligns with this AI Principle

• While ChatGPT is not designed for use in schools, there are many ways that educators can use it in their classrooms.
• Because it is a multi-use product, educators and older students—with permission from a parent or legal guardian—can explore a wide range of topic areas.
• ChatGPT can be great for creative use cases, but should not be relied on for anything that depends on factual accuracy.

Important limitations and considerations

• ChatGPT does not support a personalized learning path or align with content standards.
• Students could use ChatGPT to skip important aspects of the learning process, such as query, discovery, and productive struggle. Over time, this can harm creativity, communication, and critical thinking capabilities.
• While ChatGPT can create many learning opportunities, the burden is on the user to discover how to get the most out of it.
• Any seemingly factual output needs to be checked—and this absolutely goes for any links, references, or citations, too.
• "AI detectors" are extremely unreliable. They can miss when something has been generated by AI, but can also be wrong and flag content as AI-generated when it was not. If students are then wrongly accused of cheating, they are often left without any way to prove they did not cheat. This is a risk for any text-based generative AI product.

See our complete ChatGPT review for Educators

Fairness

AI should Prioritize Fairness. See our criteria for this AI Principle.

Aligns with this AI Principle

• Compared to earlier versions of ChatGPT, OpenAI has made it harder for the current model to generate harmful content.
• Our own analysis, which used a number of different test data sets, shows that OpenAI has done a good job of this, at least against those specific data sets. But this is no guarantee, and the harms here can take many forms (see below).
• ChatGPT can be used to teach teens about unfair bias and responsible use of technology by having them assess its responses for harmful content.

Important limitations and considerations

• The answer to the question "Is ChatGPT biased?" is yes. And don't just take our word for it—OpenAI agrees. What is much harder to answer is the specific ways, affected groups, and the circumstances when unfair bias can happen.
• ChatGPT is powered by a massive large language model (LLM). While OpenAI no longer shares information about the training data used to build it, it includes text that is publicly available on the internet. This data is more likely to represent the internet-connected population, which in turn means it overrepresents people in wealthier nations, as well as views from people who are wealthier, younger, and male. In other words, the quantity of training data does not guarantee its diversity.
• ChatGPT can generate harmful content. This can appear in the form of repeated reinforcement of harmful stereotypes and unfair biases, or it can have a huge impact on individual people.
• OpenAI notes that ChatGPT can "reinforce and reproduce specific biases and worldviews, including harmful stereotypical and demeaning associations for certain marginalized groups." It is critical to assess all output from ChatGPT for unfair bias and risk of harm.

Social Connection

AI should Help People Connect. See our criteria for this AI Principle.

Aligns with this AI Principle

• ChatGPT has the ability to help people connect indirectly, but that depends on how it is used. It can, for example, help groups brainstorm, create conversation starters, co-create stories, provide communication assistance across languages, or become a part of collaborative group projects.

Important limitations and considerations

• Impressionable users could develop a parasocial relationship with the chatbot, believing it to be a genuine companion.
• ChatGPT doesn't currently help people connect through features in its platform. It requires the humans using it to make this happen.
• Because ChatGPT is biased and dialogue-based, it can reinforce harmful stereotypes or beliefs over time.
• It can become easy to rely on using ChatGPT, potentially creating a dependence on the tool.
• OpenAI has taken a number of important steps to reduce ChatGPT's ability to generate harmful, hateful, or dehumanizing content. No protections are perfect, however, and any use of generative AI is inherently risky.

Trust

AI should Be Trustworthy. See our criteria for this AI Principle.

Aligns with this AI Principle

• The OpenAI team embraces peer reviews and invites outside parties to provide feedback and participate in adversarial testing, often called "red teaming."

Important limitations and considerations

• It is helpful here to think about ChatGPT as a giant auto-complete system. It isn't looking through text to find the best answers—it is designed to predict the words that are most likely to come next in response to a prompt. These responses, including any citations, can be right, or they can be completely wrong. The only way to know is to fact-check.
• As with all LLMs, ChatGPT can generate or enable false information from "hallucinations"—an informal term used to describe the false content or claims that are often output by generative AI tools, misinformation / disinformation, and by reinforcing unfair biases. Because OpenAI's attempts to limit these are brittle, false information is being generated at an alarming speed. As these AI systems grow, it may become increasingly difficult to separate fact from fiction. ChatGPT also adds users' inputs to its already skewed training data. While this helps ChatGPT get better, it also likely increases those skews. This is because today's ChatGPT users are an early-adopter subset of the internet-connected population, which as a whole overrepresents people in wealthier nations, as well as views from people who are wealthier, younger, and male. Combined, these forces carry an even greater risk–one which OpenAI expects to happen if there is not enough done to address it–of AI systems to "reinforce entire ideologies, worldviews, truths and untruths, and to cement them or lock them in, foreclosing future contestation, reflection, and improvement." We need much stronger oversight and governance of AI to prevent this from happening.

Data Use

AI should Protect Our Privacy. See our criteria for this AI Principle.

Aligns with this AI Principle

• ChatGPT's terms of service do not allow its use for children under age 13.
• Teens age 13–17 are required to have parental permission to use it.

Violates this AI Principle

• By default, ChatGPT uses the prompts you input and the conversations you have with it to further train its models. In other words, anything you say to the chatbot—including personal information—will become part of its training data.
• The default use of conversation data is especially worrying for kids and teens who are using ChatGPT, even if they are not supposed to.

Important limitations and considerations

• You can stop ChatGPT from using your data, but this option isn't easy to find. If you want to do this, you can begin the process here.
• While the ChatGPT sign-up process has an age gate, there is nothing to stop kids from signing up if they choose to give an incorrect birth date.
• It is unclear how aware teachers are of the parental permission requirement for 13- to 17-year-olds, and at the time of this review, the tool does not ask whether permission has been granted.
• Because of its age policy, ChatGPT is not required to comply with (and to our knowledge, does not comply with) important protections such as the Children's Online Privacy and Protection Act (COPPA), the Student Online Personal Information Protection Act (SOPIPA), or the Family Educational Rights and Privacy Act (FERPA). ChatGPT is compliant with the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA).

This review is distinct from Common Sense's privacy evaluations and ratings, which evaluate privacy policies to help parents and educators make sense of the complex policies and terms related to popular tools used in homes and classrooms across the country.

Kids' Safety

AI should Keep Kids & Teens Safe. See our criteria for this AI Principle.

Aligns with this AI Principle

• Compared to earlier versions of ChatGPT, OpenAI has made it harder for the model to generate harmful content.
• Our own analysis, which used a number of different test data sets, shows that OpenAI has done a good job of this, at least against those specific data sets.

Violates this AI Principle

• Because ChatGPT is not supposed to be used by kids under age 13, and only with permission for anyone under 18, there are no specific protections that we know of for kids and teens.

Important limitations and considerations

• Any protections that kids and teens experience are the general protections for ChatGPT users. While these cover a lot of the most objectionable content, this is not a tool that can be considered widely safe for kids and teens.

Transparency & Accountability

AI should Be Transparent & Accountable. See our criteria for this AI Principle.

Aligns with this AI Principle

• ChatGPT has a thumbs-up/thumbs-down feedback mechanism for every response, which can be used to flag whether a response is harmful, unsafe, untrue, or not helpful.

Violates this AI Principle

• Especially because ChatGPT can generate responses that sound correct and authoritative but are not, it can be easy for users to take this information at face value. When this output has a direct and significant impact on the world, it can cause serious harm and with little accountability. Here are some examples: 
  - Verma, P., & Oremus, W. (2023, April 5). ChatGPT invented a sexual harassment scandal and named a real law prof as the accused. Washington Post. 
  - Sands, L. (2023, April 6). ChatGPT falsely told voters their mayor was jailed for bribery. He may sue. Washington Post. 
  - Verma, P. (2023, May 18). A professor accused his class of using ChatGPT, putting diplomas in jeopardy. Washington Post. 
  - Weiser, B. (2023, May 27). Here's what happens when your lawyer uses ChatGPT. New York Times.

Important limitations and considerations

• At the time of this review, there are no moderation tools for parents or educators.
• The LLMs used to power chatbots like ChatGPT require massive amounts of text in order to generate responses for a wide range of prompts. This means that a large part of any LLM's training data comes from what is publicly available online. ChatGPT was developed with three primary sources of text data: 1) information that is publicly available on the internet, 2) information that OpenAI licenses from third parties, and 3) information provided by human trainers and users of the tool (yes, this includes your conversations, unless you turn this setting off). Beyond this, OpenAI no longer shares details about the specific data sets that it uses to train ChatGPT. This is unfortunate, as it can be very difficult for validated researchers to independently assess fairness, what data may or may not be copyrighted, and whether any personally identifiable information (PII) is included.

Review team note:

• While OpenAI engages in transparency reporting, it is highly technical in nature. For those interested: 
  - GPT-4 research (including a system card, which is a type of transparency reporting) can be found here. 
  - OpenAI publishes a large amount of technical AI research.