Building Generative AI Applications: Architectures, Evaluation, and Experiments

Speaker: Professor John C. Mitchell
The Mary and Gordon Crary Family Professor
Professor of Computer Science
Professor of Education
Stanford University

Title: Building Generative AI Applications: Architectures, Evaluation, and Experiments

Date: Wednesday, 7 May 2025

Time: 2:30pm - 4:00pm

Venue: Room 3598 (via lift 27/28), HKUST

Abstract:

As machine learning models continue to improve, architectures for building applications that leverage foundation models are also developing rapidly. In addition to various prompting methods, retrieval-augmented generation (RAG) is well developed and agentic architectures are gaining traction. For any of these methods to be effective, systematic evaluation is also essential: it is often relatively easy to produce a working tool simply by writing a prompt. However, building a reliable, effective and trustworthy tool requires continued effort based on clear evaluation metrics. This talk will consider three approaches – prompting, RAG and agents – and corresponding evaluation methods, illustrated through a set of experiments related to educational support tools and other domains.

Biography:

John C. Mitchell is the Mary and Gordon Crary Family Professor, Professor of Computer Science, and by courtesy Professor of Education at Stanford University. He served previously as Stanford Vice Provost for Teaching and Learning and as Chair of the Computer Science Department. Mitchell’s research focuses on programming languages, computer security and privacy, blockchain, machine learning, and technology for education. His current research projects are focused on collaborative learning, education technology and trustworthy generative AI. He is currently Faculty Director of the Hasso Plattner Institute of Design, which is also known as the Stanford d.school.