Scott Sievert
University of Wisconsin–Madison

Daniel Ross
University of Wisconsin–Madison

Lalit Jain
University of Michigan, Ann Arbor

Kevin Jamieson
University of California, Berkeley

Rob Nowak
University of Wisconsin–Madison

Robert Mankoff
The New Yorker

Abstract

Obtaining useful crowdsourcing results often requires more responses than can be easily collected. Reducing the number of responses required can be done by adapting to previous responses with \textquotedbl{}adaptive\textquotedbl{} sampling algorithms, but these algorithms present a fundamental challenge when paired with crowdsourcing. At UW–Madison, we have built a powerful crowdsourcing data collection tool called NEXT (http://nextml.org) that can be used with arbitrary adaptive algorithms. Each week, our system is used by The New Yorker to run their Cartoon Caption contest (http://www.newyorker.com/cartoons/vote). In this paper, we will explain what NEXT is and it's applications, architecture and experimentalist use.

Keywords

crowdsourcing, adaptive sampling, system

Bibtex entry

Full text PDF

Copyright © 2017 Scott Sievert et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.