Optimizing Worker Performance in Crowdsourcing Platforms

PhD Thesis Proposal Defence


Title: "Optimizing Worker Performance in Crowdsourcing Platforms"

by

Miss Ting WU


Abstract:

Recently, the popularity of crowdsourcing has brought a new opportunity for 
engaging human intelligence into the process of data analysis. Crowdsourcing 
provides a fundamental mechanism for enabling online workers to participate 
tasks that are either too difficult to be solved solely by computers or too 
expensive to employ experts to perform. Though human is intelligent, meanwhile, 
human is erroneous and greedy, which causes the quality of crowdsourcing 
results quite questionable. In this thesis, we discuss three novel approaches 
to optimize the worker performance in Crowdsourcing platforms. They are 
Diversity-Based Worker Selection, Pay-As-You-Go Scheme and Panel Training.

In the field of social science, four elements are required to form a wise crowd 
- Diversity of opinion, Independence, Decentralization and Aggregation. 
Diversity-Based Worker Selection addresses the algorithmic optimizations 
towards the ``diversity of opinion'' of crowdsourcing marketplaces. We propose 
Similarity-driven Model(S-Model) and Task-driven Model(T-Model) for two basic 
paradigms of worker selection. Pay-As-You-Go-Scheme is a new crowdsourcing 
paradigm for Object Identification tasks. In this paradigm, requester 
interactively evaluate each detected object from the crowd, and a worker is 
paid unit of reward for each detected object if it is verified by the 
requester. Such a paradigm not only resolves the difficulty for requester to 
evaluate the performance of the worker, but also avoids same objects being 
detected by many workers and ending up being meaningless workload. Panel 
Training focus on one of the most common and natural practice of crowdsourcing 
- collecting ratings of items. We design a sample-driven rubric to train 
workers, so they would standardized understanding of the rating criteria.


Date:			Wednesday, 10 August 2016

Time:                  	9:00am - 11:00am

Venue:                  Room 5508
                         (lifts 25/26)

Committee Members:	Prof. Lei Chen (Supervisor)
 			Dr. Pan Hui (Supervisor)
  			Dr. Yangqiu Song (Chairperson)
 			Prof. Huamin Qu
  			Dr. Raymond Wong


**** ALL are Welcome ****