Monitoring Food Waste in Restaurants Using Computer Vision and Data Visualization

MPhil Thesis Defence


Title: "Monitoring Food Waste in Restaurants Using Computer Vision and Data 
Visualization"

By

Mr. Ayush GUPTA


Abstract

Food waste is a significant problem in the modern world, as about one-third of 
the food produced for human consumption gets wasted annually. A significant and 
reducible portion of food waste comes from restaurants due to unfinished food. 
This thesis aims to identify and solve the challenges in monitoring food waste 
in an on-campus restaurant to help restaurant managers make data-driven 
decisions toward its reduction. This research took over a year, during which, 
from 10 cameras, we collected a total of around 22000 hours (55 TB) of videos. 
Due to the enormous magnitude of video data, manual analysis was infeasible. 
Thus, deep learning approaches were needed to process the video data and 
extract food waste information computationally. Also, a data visualization 
dashboard was needed to help summarize the information and quickly discover 
valuable insights.

We used computer vision and data visualization tools to build deep learning 
models for extracting food tray images from camera videos, classifying dishes 
and quantities of food waste from those images, and creating dashboards for 
data analysis. We designed an active learning approach to create models for 
dish classification efficiently and built a visual analytics (VA) system to 
monitor model performance over several months. The best dish classification 
model achieved 91.2% accuracy for eight categories of dishes.

During our research, we conducted several surveys and interviews with end-users 
and domain experts to gather requirements and test the usefulness of deep 
learning models, the VA system, and dashboards. The data analysts were 
satisfied with the design of the final dashboard, and the model developers were 
satisfied with the models for tray image extraction, dish classification, VA 
system, and prototype for quantity classification. However, quantity 
classification needs deeper exploration. The approaches from this research can 
be extended to many restaurants worldwide to reduce food waste.


Date:  			Wednesday, 17 August 2022

Time:			9:00am - 11:00am

Zoom Meeting:
https://hkust.zoom.us/j/99912327156?pwd=cXFkbnNqSDd5SEsyRDh1SnhmZHlLUT09

Committee Members:	Prof. Huamin Qu (Supervisor)
 			Prof. Tim Cheng (Chairperson)
 			Dr. Shuai Wang


**** ALL are Welcome ****