Researchers Explore AI’s Ability to Assess Teacher Awareness in the Classroom

Research Initiative Explores AI in Enhancing Teaching Practices

Understanding students’ problem-solving approaches and making informed instructional decisions is fundamental to effective teaching. However, the skill of “professional noticing”—the ability to assess and respond to students’ thought processes—remains difficult to evaluate, often lacking timely feedback for educators.

A collaborative research team is investigating whether artificial intelligence (AI) can provide quicker and more useful feedback to enhance pedagogical practices and foster student learning.

Funded by the National Science Foundation, the project seeks to determine if commercially available generative AI tools can accurately evaluate professional noticing skills in mathematics instruction and offer constructive guidance more efficiently than traditional human evaluators.

Dr. Molly Fisher, a professor at Rowan University’s College of Education, is working alongside colleagues from the University of Kentucky College of Education to utilize AI tools such as Claude and ChatGPT, among others, to assess professional noticing in 150 math teachers—both in-service and preservice.

All data collected will be anonymized, and no student video footage will be included in the AI assessments. The two-year grant, worth $489,316, will support this initiative until its completion in July 2028.

The Importance of Professional Noticing

Professional noticing involves the ability to focus on, interpret, and make educational decisions based on students’ cognitive processes. Recent studies have underscored its significance as a vital aspect of teacher expertise, particularly within mathematics education.

The research team will leverage data from prior studies to train AI models capable of identifying instances of effective professional noticing. The ultimate aim is to create a tool that enables educators to assess their noticing capabilities, enhance their instructional decisions, and better support their students’ learning, according to Fisher.

Effective professional noticing can illuminate a student’s mathematical development, including early arithmetic strategies and confidence-building, noted Dr. Jonathan N. Thomas, a principal investigator and chair of the Department of Teaching and Learning at the University of Kentucky.

Challenges in Measuring Professional Noticing

Fisher has dedicated years to studying professional noticing in both individual and group settings and has identified challenges in reliably assessing a teacher’s abilities. Existing methods often involve lengthy observations and post-lesson discussions that fail to provide timely feedback.

“The traditional approach includes observing a teacher, noting their responses, and debriefing afterward. This method offers insight but lacks the immediacy,” Fisher explained.

This study aims to investigate how effectively generative AI tools can measure teachers’ observations of children’s mathematical thinking and generate supportive feedback for their professional growth.

Fisher emphasized the decision to use commercially available AI tools to evaluate their practical application in educational settings, aiming for beneficial insights rather than isolated laboratory conditions.

The research team will create comprehensive scoring materials, including rubrics and decision trees, to assist AI tools in evaluating professional noticing. These resources will be crafted into standardized packages for broader educator use.

Evaluating AI’s Performance

The investigation will recruit 100 in-service and 50 preservice elementary teachers who will engage in a structured framework of attending, interpreting, and deciding while completing various assessments. Each teacher will undergo three evaluation protocols, including two based on classroom video and one focused on an in-class writing task.

Expert raters will provide scores using established protocols, which will then be compared against the scores generated by AI tools, refined through iterative testing.

Initial data collection is slated to begin in spring 2027, aimed at assessing the reliability and validity of the AI tools against expert criteria. Meanwhile, feedback reports are in development to highlight strengths and areas for improvement, targeting a response time of 24 hours for teachers upon completion of assessments.

Perspectives on AI in Education

Some educators, like Court Shuller, a middle school English teacher from New Jersey, express cautious optimism regarding the AI initiative, questioning whether such tools will genuinely aid teachers in enhancing their observational skills.

“For AI to be effective, we would need to capture specific interactions in real-time, followed by a structured analysis,” Shuller noted, adding that traditional feedback often takes a couple of days, raising questions about the feasibility of rapid AI-generated feedback.

Fisher acknowledges potential resistance among educators concerning AI evaluations, highlighting the need for human elements such as emotional intelligence and body language interpretation. However, she believes that professional noticing can nonetheless be quantitatively assessed.

Riya Nathrani, an instructional technology coach, views the potential of AI as a means to swiftly identify learning patterns, helping teachers respond more effectively to student misconceptions.

“AI could quickly indicate where discrepancies lie in student reasoning, providing a critical point for follow-ups in the learning process,” Nathrani commented, underscoring the importance of merging AI insights with human interaction in education.