The workers of Mechanical Turk, Amazon’s on-demand micro-task platform, say they have encountered mutilated bodies, graphic videos of botched surgeries, and what appeared to be child pornography. They say they have been asked to transcribe Social Security numbers and other personal data. Sometimes their temporary bosses, “requesters” in Amazon’s parlance, allegedly ask their anonymised employees to send along their underwear, take photos of their feet, or to draw pictures of their genitals. They say they have been paid to recount traumatic instances in their lives—a cancer diagnosis, severe depression, or the death of a loved one—often for less than a dollar.
These are some of 1,100 responses to two surveys Gizmodo recently posed to gig workers on Mechanical Turk asking for their experiences using the platform. Unlike the tribulations of Amazon’s warehouse workers, which have been well documented, the experiences of gig workers on Mechanical Turk are far less visible. Telling the stories of this disparate, virtually anonymous workforce can be difficult, which is why Gizmodo turned to the platform itself.
The first survey, run on Mechanical Turk in December, asked workers (aka Turkers) specifically for their strangest and worst experiences on the platform. The second survey, which we ran in early January, asked more generally about their experience working on Mechanical Turk, positive and/or negative. In total, Gizmodo spent a little over $US500 ($740) on the pair of surveys.
We provided our contact information in our surveys for any Turker to reach out to us outside the platform, but none did. And Mechanical Turk’s terms of service understandably prohibited us from asking for their personal information. As such, it was impossible for us to follow up on individual claims. However, the ample commonalities between the stories, and the breadth of experiences, paint a rare, if incomplete, picture of life inside this low-wage online labour ecosystem.
At its core, Mechanical Turk (often called MTurk) is a marketplace for tedious “micro-tasks” that computers have difficulty completing, like labelling images or training artificial intelligence used to moderate online content. Because the platform provides access to large numbers of workers at low cost, Mechanical Turk is often appealing for academics and nonprofit research organisations operating on a budget or major tech companies that turn to the platform to train their moderation tools.
The premise is simple: A requester posts a task, known as a HIT (Human Intelligence Task), to MTurk and sets a price that can be as low as one cent per micro-task. Workers can then choose to complete the tasks that they feel are worth their time and labour. Both workers and requesters are technically customers of Amazon, which takes a cut of every job posted to the platform.
The responses to Gizmodo’s surveys paint a high-contrast picture of the reality of living in the gig economy. On one hand, Turkers describe a platform on which workers seem to have no recourse for underpayment, technical errors, or general abuse at the hands of temporary bosses, which they say Amazon frequently ignores. However, a large number of workers said they are grateful for the work and flexibility that Mechanical Turk affords them in a precarious economy.
According to the web analytics service Similar Web, Mechanical Turk’s workers’ portal had 536,832 unique visitors in December, providing a glimpse of how many people potentially find work through the platform. And as you might imagine, these people turn to Mechanical Turk for a variety of reasons, our surveys found.
While the vast majority of respondents did not offer a reason for why they started using the platform, 10 per cent mentioned that they use Mechanical Turk to supplement their income from another full- or part-time job. Five per cent mentioned that they started with Mechanical Turk after having been laid-off from another job. And four per cent said they have a medical condition that prevents them from full-time employment.
“I love working on mturk…”, one respondent wrote, “…im disabled so mturk gave me back my self confidence, and i don’t have to depend on my daughter for every little thing now, I was able to get my grand kids christmas presents.” Another worker said, “I am unable to hold a traditional job due to my health, but because of extenuating circumstances, I do not qualify for disability. So, when I discovered Mechanical Turk, it was a Godsend. Mechanical Turk allows me to be able to work within my limitations, which is really helping to build my confidence back up. I know I’m not making a livable wage doing this, but that’s OK. It’s a heck of a lot better than nothing.”
For those looking to pick up some side money, the platform seems to largely be a success. “My experience overall has been pretty positive,” one respondent said. “I work full time as a teacher, so I do surveys for extra money, but this won’t make or break me. I’m tech savvy and not desperate, so I don’t do batch work for pennies.” Another respondent, who described their MTurk experience as “mostly positive” said they view the work as a side-gig that “pays for my food, gas, and pocket money every week, letting me save more money from my regular full time job.”
In one survey, 38 per cent of respondents felt extremely positive about the platform. “I love Amazon Mechanical Turk. I have been working on Mturk for about 5 years now,” a respondent wrote. “My positive experience is getting to work from home in my pajamas and complete HITS whenever I want to. I am able to make a decent income in a week now than when I first started.”
Many Turkers who said they rely on the platform for full-time work, however, expressed a sense of hopelessness and demoralisation, and underpayment was a common complaint among respondents to our survey. “I’m disabled and have difficulty finding or maintaining other work, so for the most part this is all that I have to survive on, and it’s not enough,” one respondent said. They added that, during the holidays, they felt “lucky to get $US5 ($7) a day” and had to “lean heavily on friends to pay the rent.” They called the platform “exploitative” saying that “platforms like this, they exist to twist people who have no other recourse … It’s depressing, and makes it hard to feel like it’s worth getting up in the morning.”
Worse than underpayment is no payment whatsoever. Hundreds of respondents reported at least one instance of not getting paid for their labour. Based on responses to our surveys, this usually occurs one of three ways: unfair rejections, negligence, or technical issues.
Unfair rejections, which 14 per cent of respondents reported, occur when the requester rejects a worker’s labour without cause. When a HIT is completed by a Turker, a requester has the opportunity to approve or reject what the worker submitted; as Mechanical Turk’s Documentation succinctly states, “when you approve an assignment, the Worker gets paid; when you reject an assignment, the Worker does not get paid.” Because a requester can “approve results individually or all at once,” as Amazon explains it, Turkers say it’s possible for requesters to reject a job after it’s mostly completed and pay workers nothing while still having access to the data from the partially completed job. Amazon, meanwhile, gets paid a fee the moment a job goes live on the platform.
Several respondents mentioned having completed hundreds of HITs for a requestor, investing hours of their time, only to have the requestor mass-reject all of their work. “Workers have no right to appeal rejections, and requesters are able to reject whatever they want,” one respondent claimed. “This negatively affected my approval rate, and getting work on this platform depends on a high approval rate.” After one worker had 200 of their HITS rejected for no reason, they said, “It tanked my approval rating. … It’s inexcusable to hurt workers like that.”
Sometimes, workers said, they don’t get paid due to requester negligence. Because many academic researchers prefer to run surveys using platforms like Qualtrics or Survey Monkey, Amazon supports HITs linking out to external websites. However, in order for respondents to get paid, the platform needs to be able to match survey responses on external sites to the MTurk workers, which allows them to get credit for completing the work. This is typically accomplished by supplying the worker with a survey completion code that they can enter into Mechanical Turk once they’ve finished the survey.
The thing is, 20 per cent of respondents reported incidents where, after spending hours on surveys, they were not paid due to the fact that there was no code supplied at the end of the survey. As one respondent told us, “The longer the survey the worse it is. they also get all the information they wanted without paying and i waste my time and have my work stolen from me. Amazon should not let this happen to us.”
Many workers allege that both mass rejections and this type of requestor negligence are types of scams that allow requesters to capitalise on free data without even paying the worker the potentially meagre wage that they had originally offered. Amazon could close the loophole that allows non-payment scams to operate simply by not allowing rejected work to be used by the requester. But so far, at least, it has left the loophole wide open.
“These are people’s lives and it’s a shame for a company so profitable to treat us like this,” one Turker said in response to one of our surveys.
Amazon did not respond to Gizmodo’s multiple requests for comment.
Turkers can “Report a HIT” when they believe a requester has violated the rules. But workers repeatedly alleged that Amazon is often slow and unwilling to respond when they complain that they are being cheated out of their wages.
“The thing I hate most about this platform by far is Amazon is completely hands off about bad requester behaviour,” one respondent said. “There is no accountability on the platform itself, Amazon is happy to take [its] slice of the pie and stay hands off when requesters scam honest workers on here.” Another respondent said that “I have no way to get help from Amazon, they don’t care and the requestor is always right. This is my only source of income and I [am] terrified every day. I am not making enough to live”
Even when the platform functions as advertised, respondents said, workers are often subjected to a variety of horrors, likely due to the fact that they are often tasked with training artificial intelligence that’s used to moderate offensive content. In our second survey, 11 per cent of respondents reported that their worst experience on the platform involved having seen an extremely graphic image or video. For instance, two respondents claimed to have seen images of beheadings, 10 respondents reported witnessing depictions of animal abuse, 22 said they witnessed images of grotesque car crashes, and nine claimed to have seen child pornography. “The worst thing I’ve seen on Mechanical Turk is probably having people rate medical surgery videos for lower than minimum wage,” one respondent claimed.
“I have had to look at the faces of people who shot their faces off trying to commit suicide, and they survived. They were unrecognisable as even human. … I will never forget those faces. It was horrifying and made me sick,” said another respondent. “I have also had HITS with images of what looked like child porn, I hope the kids were of age. I have no idea.”
Mechanical Turk’s Acceptable Use Policy, of course, strictly forbids the transmission of child pornography or anything that depicts non-consensual sex acts. Adult content is permitted as long as the title includes: “(WARNING: This HIT may contain adult content. Worker discretion is advised.)“ However, this worker said, “There was NO warning that I would have to look at images like this, and it paid A QUARTER.”
Though requesters rely on the workers of MTurk to train content-moderation tools, Amazon seems to be having difficulty moderating its own platform. Dozens of respondents reported instances that appear to be outright scams, privacy violations, and other violations of Mechanical Turk’s Acceptable Use Policy. While the policy clearly states that a user “may not use, or encourage others to use, MTurk for any illegal, harmful, fraudulent, infringing, or objectionable activities,” respondents detailed several examples of alleged activity that would, in aggregate, violate each of the 18 principles Amazon cites in its own Acceptable Use Policy.
For instance, while the policy forbids requesters from “collecting personally identifiable information,” “attempting to derive any personally identifiable information about Workers,” or “posting HITs that contain personal information of third parties,” dozens of people reported having been asked to submit their own personal information or transcribe other people’s intimate data. According to one respondent, “I saw a HIT which revealed the personal identification information for customers of a car dealership” including “SSN, home address, and credit card numbers.”
Amazon makes it possible for Turkers to contact the company to report alleged violations of its rules, and it was not clear from the responses to our surveys whether workers did so in each of the cases they detailed to Gizmodo.
While definitions of “harmful” and “objectionable” are subjective, dozens of respondents wrote about bizarre experiences wherein they were asked for personal information for questionable purposes. For example, four respondents told Gizmodo that they had been asked to draw pictures of their genitals for reasons unknown, and more than a dozen respondents claimed to have been asked for photographs of their feet for an undisclosed purpose. As one worker tells it, they were once working for a requester who wanted “many images of my feet in different positions with socks and without socks.” Since the requestor allegedly never paid, never responded to the workers’ inquiries, but still received the images before ultimately rejecting the job, they concluded that the HIT was probably a scam. “But I always wonder what happened to all those pictures of feet that he collected.”
While categorising the graphic videos or images can be traumatic, many respondents said that completing academic surveys can be even worse. According to a 2016 Pew Research Centre report, more than 800 studies—ranging from medical research to social science—were published using data on Mechanical Turk in 2015 alone. In fact, outgoing web traffic from Mechanical Turk to external survey providers like Qualtrics indicate that in the last six months Mechanical Turk has seen tasks from dozens of universities including NYU, Columbia, Cornell, Stanford, and Yale.
Yet as benign as academic research may seem, 12 per cent of respondents claimed that the worst or strangest experience on Mechanical Turk was due to what can only be described as uncomfortable personal data requests wherein the worker reported feeling emotionally traumatised by an academic survey.
A worker claimed that after having to fill in a suicide-related survey, they had “intensely negative feelings come up,” even though it had been 10 years since they last felt depressed, while a different Turker said they were “brought to tears” recalling the cancer diagnosis of a loved one.
“Someone paid me like 50 cents ($0.74) to recall the most painful memory of my life,” recalled another respondent, “it fucked me up for the entire day.”
If you or someone you know is contemplating suicide, please call Lifeline, Australia’s 24-hour crisis support hotline, on 13 11 14.