AI tutoring systems: Carnegie Learning vs Khanmigo compared

No — AI tutoring systems are most effective as supplements to teacher-led instruction, not replacements. The evidence for AI tutoring effectiveness is strongest in the blended learning model: AI handles individualised practice and immediate feedback (tasks that don't require human judgment and are time-consuming at classroom scale), while teachers focus on conceptual discussion, collaborative work, social-emotional support, and the higher-order skills that benefit from human facilitation. AI tutoring systems cannot provide the human connection, motivation, and mentorship that research consistently shows is critical for learning, especially for younger students and those with social-emotional barriers to learning. The appropriate framing is AI as a teaching assistant that extends individual attention capacity — not AI as a teacher replacement.

Both platforms have accessibility features but approach differentiation differently. Carnegie Learning's mastery model naturally accommodates different learning paces — students who need more practice to reach mastery get more practice, without the social pressure of falling behind classmates. The structured hint system provides scaffolded support without revealing answers, which benefits students who need more structured guidance. Khanmigo's conversational model can adapt explanation style to the student's expressed needs ("can you explain this more simply?") and is accessible through text, which works well for students who process information better through reading than listening. Neither platform provides full IEP implementation or specialised learning disability support at the level of a trained special education teacher — they should be implemented as part of a broader differentiated instruction approach, not as a standalone accommodation.

Both Carnegie Learning and Khan Academy/Khanmigo operate under COPPA (Children's Online Privacy Protection Act) for US schools, FERPA for student education records, and GDPR/UK GDPR for EU/UK institutions. Key considerations for institutional deployment: ensure a Data Processing Agreement (DPA) is in place with the platform before deployment; verify that student interaction data is not used to train AI models without explicit consent (both platforms have committed to not using student data for AI training); review data retention policies (how long is interaction data kept?); and ensure the platform is included in the institution's data inventory for breach notification purposes. For schools in the EU, confirm the platform's data transfer mechanisms (standard contractual clauses, adequacy decisions) for any data processed outside the EU. Both platforms have legal and compliance resources for education institutions and can support DPA execution for institutional deployments.

Measuring AI tutoring effectiveness requires both platform-native metrics and external validation. Platform-native metrics (mastery progression, time-on-task, hint usage rates, error pattern trends) are available in both platforms' reporting dashboards. External validation uses pre/post assessments on standardised tests or teacher-created assessments — measuring learning gains in the AI-tutored group against a comparison group or against previous cohort performance. For Carnegie Learning, the platform's mastery metrics have been validated as predictors of standardised test performance, so mastery progression is a reliable leading indicator. For Khanmigo, external assessment is more important given the shorter evidence base. Key metrics to monitor: mastery rate (are students mastering more skills per unit time?), student engagement (time-on-task, voluntary usage beyond assigned work), and teacher-assessed understanding (are students demonstrating conceptual understanding in class, not just passing AI-assessed practice?).

Yes — the AI tutoring market has grown significantly. Duolingo Max (language learning with AI conversation practice and explanation), Synthesis (mathematics problem-solving and strategy, originally developed for SpaceX employee children), IXL Learning (broad K-12 adaptive practice across subjects with strong state standards alignment), Brilliant (STEM learning for older students and adults), and Socratic by Google (homework help via photo of question) all serve specific niches well. For higher education, Kira Talent (graduate admissions AI tutoring), Coursera's AI coach, and Duolingo for institutions extend the category. Enterprise learning and professional development has its own emerging AI tutoring applications through platforms like Docebo, 360Learning, and custom LLM tutoring implementations on corporate LMS platforms. The right tool depends strongly on age group, subject matter, and whether procedural mastery or conceptual exploration is the primary learning goal.

Khanmigo is designed with K-12 safety requirements in mind, including content guardrails, age-appropriate interaction design, and restrictions on off-topic discussions. However, the conversational model introduces variables that are harder to control than Carnegie Learning's structured interaction model — students can ask questions outside the curriculum context, and while Khanmigo redirects off-topic conversations, younger children may be confused by a system that sometimes doesn't answer their questions. For K-3 specifically, Carnegie Learning and similar structured adaptive platforms are generally more appropriate — the structured interaction model is more predictable, less confusing for young children, and more easily supervised by teachers. Khanmigo is better positioned for grades 4+ where students have sufficient reading comprehension to benefit from dialogue-based tutoring and enough self-direction to use a conversational interface productively.

Both platforms require internet connectivity and a device (tablet, Chromebook, or computer), which creates equity concerns for students with limited home technology access. Research on AI tutoring consistently shows that students from lower socioeconomic backgrounds benefit as much as or more than higher-SES peers when they have consistent access — the personalised, non-judgmental interaction of AI tutoring is particularly beneficial for students who experience social anxiety about asking questions in class. The access gap, not the platform design, is the primary equity challenge. Schools implementing AI tutoring for equity goals should ensure device and connectivity provision is addressed alongside platform deployment — after-school computer access programmes, Chromebook lending programmes, and school Wi-Fi hotspot lending are commonly deployed alongside AI tutoring programmes to close the access gap. Both Carnegie Learning and Khan Academy have equity-focused pricing and grant programmes available for Title I and under-resourced schools.

Carnegie Learning has the strongest standardised test score evidence in the AI tutoring category: multiple peer-reviewed studies and RAND Corporation evaluations of large school district deployments show statistically significant improvements in state mathematics assessment scores for MATHia users versus comparison groups — effect sizes ranging from 0.1 to 0.4 sigma depending on implementation fidelity and student population. Khanmigo's evidence base is newer and focused more on learning engagement and student satisfaction metrics than standardised test performance as of 2025–26. Khan Academy's core platform (non-AI) has strong SAT score improvement evidence from its partnership with College Board. For institutions where standardised test score improvement is a primary programme goal, Carnegie Learning's evidence base is stronger. For institutions prioritising student engagement, conceptual understanding, and multi-subject support, Khanmigo's broader evidence base (including teacher satisfaction and engagement metrics) is compelling despite the shorter research timeline.