AI-based Technologies: HowTo-4-Teaching

SERIES OF TWO SEMINARS
This seminar will start from introducing the Technology Acceptance Model (TAM) to evaluate how generative AI (GenAI) is perceived and used by students and teachers. Through a series of physics problems, we will test and discuss the ability of GenAI systems for physics teaching and learning. The seminar also reviews institutional guidelines (comparing Imperial College and Padova) for ethical AI use. It concludes with survey data showing that while students use AI for coding and brainstorming, they are highly uncomfortable with AI being used for automated marking or replacing personalized feedback from lecturers.

SERIES OF TWO SEMINARS
This seminar provides a comprehensive overview of how Machine Learning (ML) and Natural Language Processing (NLP) are used to analyze both quantitative and qualitative data in physics education. After briefly tracing the history of text analysis from traditional “bag-of-words” classifiers to the current era of Large Language Models (LLMs), the seminar will focus on recent developments and uses in Physics Education Research. Specifically, it will present the use of LLMs to automate the coding of student lab notebooks, comparing the performance of different models against human inter-rater reliability.

Imperial College London has invested in a variety of projects to explore the use of generative artificial intelligence (GAI) and machine learning (ML) in higher education The main project which I will discuss aims to test the effectiveness of Large Language Models (LLMs) in assisting large-scale qualitative research and consequently whether workflows incorporating LLMs can provide useful and timely feedback to instructors on the impact of their teaching. The focus of the work is on student use of scientific argumentation in lab reports. We have constructed a codebook for scientific argumentation to identify elements of argumentation, the relationships between those elements, and their veracity. Through building a training dataset of over 250 lab reports from students in their first- and second-year lab courses, we aim to analyse the whole set of 1899 lab reports from two cohorts of students by utilizing fine-tuning of open-source LLMs to automate the coding process (Fussell et al., 2025). The specific purpose here is to test how changes to teaching between the two cohorts affected students’ scientific argumentation skills, and, therefore, help us to evaluate the impact of the changes.

In addition to this, I will provide an overview of the projects that are currently being undertaken across Imperial related to AI and education. These institutionally funded projects include: (1) the use of GAI in providing feedback to students through the in-house problem sheet web-interface Lambda Feedback; (2) the use of LLMs to analyse and provide in-the-moment feedback to students on the structural elements of their lab reports; (3) using ML to identify at-risk students through analysis of interaction data from Lambda feedback; and (4) collecting data on students’ views about how instructors use GAI in their teaching.