Welcome to My Boss is a Robot.
We created this blog to chronicle our attempt to answer a question: can unskilled, crowdsourced labor be used to create a product that require skills, experience and insight?
We’re both journalists, so we’ve chosen a product we’re familiar with: the news story. We want to create a high-quality piece that could run in a reputable news magazine or newspaper. And we’re going to assign this job to the workers on Amazon’s Mechanical Turk, an outsourcing website. We want the workers on MTurk to do the reporting, writing, editing and fact-checking — all the parts of the editorial process.
We’ll elaborate on the process and why we think it’s interesting in future posts, but here are two of our motivations:
It’s happening already (almost). Publishers have already automated some parts of the editorial process and out-sourced others. It’s probably only a matter of time before a publisher quietly conducts this experiment, if they haven’t already. We think it’s better to explore the pros and cons of this approach in public.
It pushes the boundaries of crowdsourcing. The jobs on Mechanical Turk tend to be quick and easy. Workers are often tasked to find email addresses or translate short sentences. Tasks that involve planning and creativity don’t seem to lend themselves to the platform. But maybe that’s because no one has figured out a way of breaking these more complex processes into a series of straightforward tasks. If so, many more processes — product design? medical diagnosis? — might be crowdsourced. This idea is explored in a recent news story that one of us wrote for New Scientist magazine (registration required, but the piece is free to access).
Before we begin, two obvious questions:
Are we confident that our experiment will work? Absolutely not. In fact, we’d be surprised if it did. But we think it will be interesting to try. Equally importantly, we hope that chronicling our progress here will trigger debate about the future of journalism and of crowdsourcing. We’d love to hear your thoughts on our plans and, later, our progress.
Where does the robot boss come in to it? The crowdsourced workers will be coordinated by an algorithm. If the process works, we will hand the algorithm a research paper and, perhaps a couple of days later, it will spit out a polished news article. Bear with us on this one — we’ll clarify the details soon.
Finally, the credits. This blog is the work of MacGregor Campbell and Jim Giles. We’re San Francisco-based science and technology journalists. But most of the hard work of designing and running the experiment is being done by our collaborators: Niki Kittur and colleagues at Carnegie Mellon University. See the about us page for more.
[...] are also likely to be plenty of pitfalls along the way, and as such, Giles is looking for feedback on the My Boss Is a Robot blog or via Twitter so that he can document reactions and understand what challenges the scheme poses. [...]
Posted by Content Farms 2.0: Can Robots Help Write the News?: Tech News and Analysis « on February 8th, 2011.
[...] are also likely to be plenty of pitfalls along the way, and as such, Giles is looking for feedback on the My Boss Is a Robot blog or via Twitter so that he can document reactions and understand what challenges the scheme poses. [...]
Posted by Content Farms 2.0: Can Robots Help Write the News? | Boomeroo Web Resources on February 8th, 2011.
[...] are also likely to be plenty of pitfalls along the way, and as such, Giles is looking for feedback on the My Boss Is a Robot blog or via Twitter so that he can document reactions and understand what challenges the scheme poses. [...]
Posted by Fastest growing news network » Content Farms 2.0: Can Robots Help Write the News? on February 8th, 2011.
Who will be responsible for fact-checking the finished product and issuing retractions/corrections?
Posted by Lisa Duggan on February 8th, 2011.
Hi Lisa, thanks for your comment. The fact-checking could become part of the process — i.e. the workers on Mechanical Turk could do it. Not sure if we’ll include that in the first round of tests though. We’ll have to see how we get on with the reporting and writing parts of the process. This is an experiment and we don’t plan on using the system to publish articles (although someone else might want to), so retractions and corrections aren’t something we’ve thought about yet.
Posted by Jim Giles on February 8th, 2011.
[...] Boss is a Robot: scientific experiment outsources the editorial process to Amazon Mechanical Turk; [...]
Posted by Robot News « Wir sprechen Online. on February 8th, 2011.
[...] a 500 word piece of original science journalism. There are a million reasons this might not work, admit Giles and Campbell, but the exercise is meant to generate insight and discussion, whether or not it [...]
Posted by Can This Journalist Be Replaced by Software and Mechanical Turk? : NerdvanaShow.com on February 9th, 2011.
[...] are also likely to be plenty of pitfalls along the way, and as such, Giles is looking for feedback on the My Boss Is a Robot blog or via Twitter so that he can document reactions and understand what challenges the scheme poses. [...]
Posted by Content Farms 2.0: Can Robots Help Write the News? | Social Networking, Social Media, News, Web, Technology, Tech, Information, on February 10th, 2011.
A compiler is incomplete until it has compiled itself; a journalizing robot is incomplete until it reports on itself. When can we expect the article?
Posted by Vincent Tomaino on February 24th, 2011.
We’d love to try that. The first iteration of the software is being designed to report on academic papers, but the obvious next step is to extend it so that it can tackle a broader range of assignments. It’s really hard to tell if we’ll get there, but, if we do, it’d fun to set it loose on our own project.
Posted by Jim Giles on February 24th, 2011.