Building Bodies, Building Symbols: Language acquisition from unsupervised perception and text

The Hong Kong University of Science & Technology
Department of Computer Science and Engineering
Human Language Technology Center

        Joint Seminar
-----------------------------------------------------------------

Speaker:	Professor Amitabha MUKERJEE
		Department of Computer Science and Engineering
		Indian Institute of Technology, Kanpur
		India

Title:		"Building Bodies, Building Symbols: Language
		 acquisition from unsupervised perception and text"

Date:		Friday, 11 January 2008

Time:		4:00pm - 5:00pm

Venue:		Lecture Theatre F
		(Leung Yat Sing Lecture Theater, near lift 25/26)
		HKUST
Abstract:

In the end, both perceptual and linguistic inputs need to be understood in
terms of an agent's own experience, as opposed to other symbols.  In this
work, we consider the very earliest stages of an agent's experience, and
consider how it may be using the stream of visual data to cluster the
environment into chunks corresponding to objects, relative spatial pose,
and behaviours. Later, when socially exposed to descriptions of the same
scene in language (English), we show how these clusters can be mapped to
linguistic tokens that may correspond to nouns, prepositions, and verbs.
This provides a grounded interpretation for a base set of symbols which
can be expanded with subsequent inputs to create a grounded lexicon. We
present results on simple 2D video as well as complex 3D scenes.


*************************
Biography:

Details are available at http://www.cse.iitk.ac.in/~amit/