Toddle AI - limited review

People First

AI should Put People First. See our criteria for this AI Principle.

Aligns with this AI Principle

• The primary objective of Toddle AI is to enhance teachers' performance by streamlining tasks, enabling them to prioritize their students. While the marketing demos are compelling, without access to the tool, participatory disclosures, or publicly available transparency reports, we have not been able to assess this further.
• Demo videos show that teachers are kept in the loop when generative AI is involved.

Important limitations and considerations

• While use of Toddle AI's generated lesson plans, student assessments, and communication drafts is entirely within a teacher's control, it is very easy to port over anything the chatbot generates into what is used in practice. It is critical that teachers closely inspect the accuracy and appropriateness of the final product, especially as it relates to student progress reports, which are a high-stakes task.
• AI-generated progress reports that draw inferences and conclusions from otherwise incomplete information could be harmful to student success, confidence, and progress through school.

Learning

AI should Promote Learning. See our criteria for this AI Principle.

Aligns with this AI Principle

• Toddle AI is designed to help teachers better design their classes to support student learning more efficiently and effectively.
• The Toddle platform has been translated into English, Spanish, and Turkish, and the team has worked with schools that operate primarily in Arabic, Chinese, and French. We do not know whether these translations also apply to Toddle AI.

Important limitations and considerations

• Teachers using Toddle AI should be aware of potential overreliance on the tool, at the cost of quality. AI for lesson planning can be valuable as a brainstorming tool. However, teachers should remain vigilant about outputs that produce reasonable-sounding plans but miss crucial components that align with good instruction. Like working with a teacher's assistant, a teacher needs to be in the loop with the AI for the help to be as useful, targeted, and relevant as it can be for any given class environment.
• Toddle AI is still early in its development, and there is no information as to how the team has thought about guardrails, appropriate use, or how severely the system "hallucinates"—an informal term used to describe false content or claims that are often output by generative AI tools.

Fairness

AI should Prioritize Fairness. See our criteria for this AI Principle.

Aligns with this AI Principle

• We applaud Toddle AI for publishing their principles for AI development.
• Toddle can operate on a 3G+ connection, which suggests greater inclusivity. We do not know if Toddle AI can run effectively on a 3G+ connection as of this writing.

Important limitations and considerations

• At the time of this writing, Toddle AI has not discussed if or how the company approaches risk, management, and fairness in their AI development. We do not know how the organization is working to train and tailor Toddle AI for a learning setting, how it monitors and moderates interactions to be able to proactively respond to inappropriate, harmful, or biased content, or how it applies community controls.
• Toddle AI is built on multiple large language models. While we don't have complete information about the training data used to power Toddle AI, it includes text that is publicly available on the internet. This data is more likely to represent the internet-connected population, which in turn means it overrepresents people in wealthier nations, as well as views from people who are wealthier, younger, and male. In other words, the quantity of training data does not guarantee its diversity.
• It is critical for teachers to assess all output from Toodle AI for unfair bias and risk of harm.
• Toddle AI is currently available via waitlist to teachers who use Toddle 360, and is currently in beta testing with 50 schools. It is not clear how Toddle intends to charge for the service, but pricing could exclude under-resourced schools.
• While Toddle's Terms of Use note that schools using the platform are required to get parental consent for children under age 13 as well as necessary permissions to share "Educational Records," it is not clear whether Toddle has procured informed, meaningful consent from the parents of students whose teachers use Toddle AI.

Social Connection

AI should Help People Connect. See our criteria for this AI Principle.

Aligns with this AI Principle

• Toddle AI's tools could free up educators to spend valuable time with their students in new ways, but this is not a guarantee.
• As of this writing, Toddle is not making their generative AI tools available to students, which protects learners from the risks of LLMs to output harmful content.
• Toddle AI's writing assistant could also create more communication more frequently to help students' guardians stay in-the-know on learner progress.

Important limitations and considerations

• AI is built on multiple LLMs (including ChatGPT from Open AI, Bard from Google, and Llama2 from Meta). We don't know whether Toddle has done further work to protect against harmful and inaccurate outputs.

Trust

AI should Be Trustworthy. See our criteria for this AI Principle.

Aligns with this AI Principle

• By making the tool available only to teachers to assist them in their work, Toddle takes a step toward protecting children from open beta testing and unmonitored perpetuation of mis/disinformation.

Important limitations and considerations

• There does not appear to be peer-reviewed research on Loora.
• Large language models (LLMs) have the capacity to generate potentially harmful or false content that nevertheless appears reasonable at first glance. Teachers must always inspect the information output by Toddle AI.
• Toddle AI is built on multiple LLMs. All LLMs are each challenged by biased outputs, inaccuracies, and "hallucinations"—an informal term used to describe the false content or claims that are often output by generative AI tools. At the time of writing, we are not aware of how Toddle has fine-tuned or safeguarded its use of these models.

Data Use

AI should Protect Our Privacy. See our criteria for this AI Principle.

Aligns with this AI Principle

• We applaud Toddle for its clear stance on student data privacy, and specifically that all interactions between Toddle and the large language models (LLMs) that power Toddle AI are fully encrypted and anonymized. This is a strong protection against personally identifiable information (PII) being shared with third parties.
• Toddle complies with the Children's Online Privacy and Protection Act (COPPA), the Family Educational Rights and Privacy Act (FERPA), and the General Data Protection Regulation (GDPR). Toddle has also signed the Student Privacy Pledge.

Important limitations and considerations

• Toddle's privacy policy notes that users' data may be shared with the various LLM providers that power Toddle AI. We don't know whether this includes permission for the LLM providers to use that data to train their models. For example, while by default OpenAI does not use data submitted by customers via their API* to train their models, organizations are allowed to opt in, on behalf of their own users, to give OpenAI this permission. We don't know if Toddle has opted in to share data, but it is important for Toddle users to know this might be the case. This is especially important in the case of high-risk uses such as student progress reports.
• Toddle's privacy policy states that no personally identifiable information (PII) is shared with the LLM providers it uses to power Toddle AI because Toddle anonymizes the information first. We do not know if this specifically covers situations in which users include PII in their prompts. It is important for Toddle AI users to be aware that PII they enter may not be anonymized, and it is a best practice to never input PII into any generative AI chatbot.

* The reverse is true for consumer inputs in ChatGPT, which by default is used to train OpenAI's models.

This review is distinct from Common Sense's privacy evaluations and ratings, which evaluate privacy policies to help parents and educators make sense of the complex policies and terms related to popular tools used in homes and classrooms across the country.

Kids' Safety

AI should Keep Kids & Teens Safe. See our criteria for this AI Principle.

Aligns with this AI Principle

• The product's careful roll-out strategy to beta-test in a limited number of schools optimizes for safety.
• Toddle AI is not intended for direct use by students.

limitations and considerations

• Teachers should pay careful attention to their AI-assisted work, especially as it relates to student progress reporting, which can impact grades and assessment.
• Educators using Toddle AI should be supplied with professional development on the tool's limitations, opportunities, and pitfalls to avoid.
• Toddle indicates that Toddle AI has been trained to keep users safe, but more details were not available.
• If Toddle AI generates progress reports that draw inferences and conclusions from otherwise incomplete information, this could be harmful to student success, confidence, and mental health.

Transparency & Accountability

AI should Be Transparent & Accountable. See our criteria for this AI Principle.

Aligns with this AI Principle

• Toddle AI is currently designed solely as a teaching assistant, which allows teachers to monitor the material it produces before it affects students.

Important limitations and considerations

• Any final output that leverages Toddle AI's responses—even when revised by teachers—should include a disclosure that they were produced by Toddle AI. It is unclear if this currently occurs with the product. It is especially critical for parents and caregivers to be informed when their child's progress reports include AI-generated content.
• Companies that leverage powerful generative AI tools like Toddle should be responsible for conveying the power, limitations, and responsible use of the technology. One of Toddle's three key principles guiding its work with AI is to instruct educators about AI, specifically the potential and limitations of LLMs. At the time of this review, however, we are not aware that any features of this kind are currently available. Given that Toddle AI is actively being rolled out to 50 schools, the promise of a future course for educators does not seem to meet the goals of this key principle.
• Toddle AI's marketing uses irresponsible language about the capabilities of AI, such as "limitless power" and that it can provide a "personal teaching assistant with near-magical capabilities." AI is not limitless or magic. It is built on sophisticated but well-understood math that, while often impressive, does not reason, think, or problem-solve like a human. We look forward to seeing Toddle's course for educators on the potential and limitations of its product.
• We do not know if or how Toddle monitors Toddle AI's outputs for harmful content, or collects feedback.