IMPROVISING HIP HOP LYRICS VIA TRANSDUCTION GRAMMAR INDUCTION

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "IMPROVISING HIP HOP LYRICS VIA TRANSDUCTION GRAMMAR INDUCTION"

By

Mr. Venkata Sai Karteek ADDANKI


Abstract

Among the many genres of language that have been studied in computational 
linguistics and spoken language processing, there has been a dearth of work on 
lyrics in music, despite the major impact that this form of language has across 
almost all human cultures. In this thesis, we propose theoretically motivated 
symbolic and distributed models for improvising lyrics in music and we choose 
the genre of hip hop lyrics as our domain. Through our work, we model the 
issues in song lyric improvisation using modern statistical language 
technologies and attempt to bridge the gap between language and music in 
natural language processing (NLP).

Firstly, we describe a novel hidden Markov Model (HMM) based rhyme scheme 
detection module which identifies the rhyming scheme within a given stanza in a 
completely unsupervised fashion without using any linguistic or phonetic 
features. We use this rhyme scheme detection module to select the training data 
for our improvisation models so as to generate fluent and rhyming output and 
demonstrate that using the rhyme scheme detection module improves the model 
performance considerably.

Secondly, we improvise hip hop lyrics by generating responses to challenges 
similar to a freestyle rap battle. We model the problem of improvisation as a 
machine translation problem where the challenge needs to be “translated” into a 
response and train a bottom-up token based inversion transduction grammar model 
to perform the transduction. We also propose a search heuristic in our decoding 
algorithm and disfluency handling strategies to improve our model output. We 
contrast our model with an off-the-shelf phrase based SMT (PBSMT) model and 
show that our model generates significantly better responses that are more 
fluent and rhyme better with the challenges.

We also propose a novel model that improvises rhyming and fluent responses for 
a hip hop lyric challenge by combining both bottom-up token based rule 
induction and top-down rule segmentation strategies to learn a stochastic 
transduction grammar. We demonstrate that the combined token based and rule 
segmentation induction method performs better than the bottom-up token based 
inversion transduction grammar model. We also show good model performance on 
Maghrebi French hip hop lyrics demonstrating the language independence of our 
models.

Another improvisation algorithm using TRAAM, a fully bilingual generalization 
of Pollack’s (1990) monolingual Recursive Auto-Associative Memory neural 
network model, in which each distributed vector represents a bilingual 
constituent is also presented. TRAAM models capture cross-lingual 
generalizations via soft bilingual categories and hence have attractive 
properties which can be used for the tasks such as bilingual grammar induction 
and statistical machine translation approaches. Using a novel pattern 
completion decoding algorithm, we use a trained TRAAM model to improvise hip 
hop lyrics.

Lastly, we discuss the challenges in evaluating the performance on the 
improvisation task of evaluating hip hop lyrics as a first step toward 
designing robust evaluation strategies for improvisation tasks, a relatively 
neglected area to date. We discuss our observations regarding inter-evaluator 
agreement on judging improvisation quality as a means to better understand the 
high degree of subjectivity at play in improvisation tasks, thereby enabling 
the design of more discriminative evaluation strategies to drive future model 
development.


Date:  			Thursday, 26 November 2015

Time:			11:00am - 1:00pm

Venue:			Room 4475
 			Lifts 25/26

Chairman:		Prof. Huai Liang Chang (MATH)

Committee Members:	Prof. Dekai Wu (Supervisor)
 			Prof. Andrew Horner
 			Prof. Xiaojuan Ma
 			Prof. Pascale Fung (ECE)
 			Prof. David Johnston (CityU)


**** ALL are Welcome ****