The developers at Ashley Madison created their first artificial woman sometime in early 2002. Her nickname was Sensuous Kitten, and she is listed as the tenth member of Ashley Madison in the company's leaked user database. On her profile, she announces: "I'm having trouble with my computer ... send a message!"
Sensuous Kitten was the vanguard of a robot army. As I reported last week, Ashley Madison created tens of thousands of fembots to lure men into paying for credits on the "have an affair" site. When men signed up for a free account, they would immediately be shown profiles of what internal documents call "Angels," or fake women whose details and photos had been batch-generated using specially designed software. To bring the fake women to life, the company's developers also created software bots to animate these Angels, sending email and chat messages on their behalf.
To the Ashley Madison "guest," or non-paying member, it would appear that he was being personally contacted by eager women. But if he wanted to read or respond to them, he would have to shell out for a package of Ashley Madison credits, which range in price from $US60 to $US290. Each subsequent message and chat cost the man credits. As documents from company emails now reveal, 80 per cent of first purchases on Ashley Madison were a result of a man trying to contact a bot, or reading a message from one. The overwhelming majority of men on Ashley Madison were paying to chat with Angels like Sensuous Kitten, whose minds were made of software and whose promises were nothing more than hastily written outputs from algorithms.
But the men were not fooled. At least, not all of them. An analysis of company emails, coupled with evidence from Ashley Madison source code, reveals that company executives were in a constant battle to hide the truth. In emails to disgruntled members of the site, and even the California attorney general, they shaded the truth about how the bots fit into their business plan.
Ashley Madison Dodges the California Attorney General
On January 11, 2012, the office of California Attorney General Kamala Harris sent an official consumer complaint to Ashley Madison's executives (below). The complaint, addressed to the public inquiry unit of the attorney general's office, came from a man in Southern California who accused the company of fraud for using "fake profiles" to engage him in pay-to-play conversations.
The letter demanded that Ashley Madison respond or face possible legal action.
In his complaint (below), the man describes what he suspected was telltale bot activity. He was contacted by a number of women in his area, and finally decided to pay to read their messages. He began to get suspicious when they all said the same thing: "Are you online?" Given that every profile shows whether you are online or not, he thought that message was odd. Especially when it supposedly came from several different women, none of whom had ever checked out his profile. But then things got stranger. He discovered that many of the women who had contacted him would log in at exactly the same time of the morning every day, and log out at exactly the same time in the evening. Even on Christmas and New Year's Day.
A search of the Ashley Madison source code for the phrase "are you online?" turned up a data table I hadn't found before, with a set of pickup lines that the bots used regularly. They include:
are you logged in?
care to chat?
I'm online now
come chat :)
come say hello
my chat is on now
are you online?
Feel like chatting?
do you like cyber?
cyber sex ?
care to cyber?
u into cyber?
How are you? Feel like chatting?
cybering good with you?
how's your day? wanna chat?
want to sex chat?
how's your cyber skills ;)
are you at your computer?
So how long have you been here? Met any interesting people?
So our angry California consumer was onto something. What about the names of the users he mentions in his complaint? After checking the Ashley Madison member database, I can confirm that 4 of these names (Hooky_Pooky, ToasterStrudell, SunStarsMoon and BurnOnTheGrill) are still in use as "hosts," one of the company's internal names for its bot profiles. So the company apparently didn't even bother to shut down host accounts that had been named as fraudulent in an official consumer complaint.
Avid Life Media's general counsel Mike Dacks drafted a response to the public inquiry unit a few days later. In it, he explained that "criminal elements" on Ashley Madison are known to create fake profiles on the site, and that members can "report a suspicious profile" or "flag" them. Basically, he argued that any fake profiles on Ashley Madison were from outside scammers. He assured the public inquiry unit that Ashley Madison had refunded the customer and "flagged" the profiles named in the complaint.
Biderman and other senior management signed off on Dacks' response. Apparently it was enough to halt further action. The California Attorney General's office didn't immediately respond to our request for comment.
Ashley Madison Hides the Truth From Its Users
Though Ashley Madison told the California attorney general's office that its own bots were actually the work of random fraudsters, management struggled internally with the legality of what they were doing. Users complained about bots regularly, and there are several email exchanges between Biderman and various attorneys about how to disclose that they have bot accounts without admitting any wrongdoing.
In late 2013, Leslie Weiss, a partner at Chicago firm Barnes & Thornburg, drafted some language about the bots for the company's terms of service. From an email dated November 12, 2013, she included a suggested disclosure, worded like so:
In order to allow persons who are Guests on our Site to experience the type of communications they can expect as Members, we create profiles that can interact with them. You acknowledge and agree that some of the profiles posted on the Site that you may communicate with as a Guest may be fictitious. The purpose of our creating these profiles is to provide our users with entertainment, to allow users to explore our Services and to promote greater participation in our Services. The messages they send are computer generated. Messages from the profiles we create attempt to simulate communications with real Members to encourage our users to participate in more conversation and to increase interaction among users. We also use such profiles to monitor user communications and use of our Service to measure compliance with the Terms. These profiles allow us to collect messages, instant chat and/or replies from individuals or programs for market research and/or customer experience and/or quality control and/or compliance purposes. Further, we may use these profiles in connection with our market research to enable us to analyse user preferences, trends, patterns and information about our customer and potential customer base. The profiles we create are not intended to resemble or mimic any actual persons. We may create several different profiles that we attach to a given picture. You understand and acknowledge that we create these profiles and that these profiles are not based on or associated with any user or Member of our Service or any other real person. You also acknowledge and agree that the descriptions, pictures and information included in such profiles are provided primarily for your amusement and to assist you navigate and learn about our Site. As part of this feature, the profiles may offer, initiate or send winks, private keys, and virtual gifts. Any one of these profiles may message with multiple users at the same or substantially the same times just like our users. Our profiles message with Guest users, but not with Members. Members interact only with profiles of actual persons. Guests are contacted by our profiles through computer generated messages, including emails and instant messages. These profiles are NOT conspicuously identified as such.
This is a surprisingly transparent description of what Ashley Madison was actually doing -- it admits that users may "communicate" with a "fictitious" profile, and even acknowledges how Ashley Madison recycled pictures for its Angels. But that's where the transparency ends. Weiss' suggested terms of service say the bots are for "entertainment" and "market research."
In a response to Weiss, Biderman wonders whether they should strike the references to entertainment and just focus on how the bots provide "quality assurance." On November 13, 2013, he wrote:
Leslie, jason and I were just discussing this a little further and one "legacy" component that remains is the notion of entertainment. Again I recall some of your thinking around its value but we wondered if the positioning of the engager profiles as an early detection and warning system to help ensure quality is not maybe a better or at least additional positioning we should contemplate.
It appears that Weiss won this particular debate, though not completely. The Wayback Machine reveals that her wording was used in the company's terms of service agreement for quite a while, but was changed in early March of this year. In fact, the site's current agreement makes no mention of "software" or "fictitious" profiles -- instead, it says simply that some members may have profiles that are "exaggerated or fantasy." As of September 7, 2015, Ashley Madison's terms of service read:
Our Site and our Service also is geared to provide you with amusement and entertainment. You agree that some of the features of our Site and our Service are intended to provide entertainment ... You acknowledge and agree that any profiles of users and members, as well as, communications from such persons may not be true, accurate or authentic and may be exaggerated or fantasy. You acknowledge and understand that you may be communicating with such persons and that we are not responsible for such communications.
On the very same day that Weiss and Biderman were debating how to describe their bots to users in the terms of service, Biderman was also talking to his colleagues about how to word a boilerplate email response to members complaining to Ashley Madison customer service about bots.
Avid Life Media's director of customer service, Carlos Nakhle, suggested the following wording:
As explained in our Terms and Conditions, Ashley Angels are profiles that are used in connection with our market research to help us analyse user preferences/trends, to monitor member communications, and also to encourage more conversation and interaction with members. Member credits will never be used in connection with an Angel. That way, you can initiate contact with confidence.
Like his boss, Nakhle seemed to prefer that Ashley Madison tell its users that the fake Angel profiles were just for market research. No mention of entertainment.
It's unclear whether Nakhle's boilerplate email was ever actually sent to any Ashley Madison users who complained about bots. But his pledge that people who pay to join Ashley Madison will never be asked to spend money on an Angel appears, based on the company's internal documents and source code, to be false.
Emails in Biderman's inbox from November 2012 contain evidence that the company knew very well that most of their money came from bots flirting with men. Security researcher Alejandro Ramos found these emails, which contain an internal presentation that was passed around to many of the company managers. One slide (reproduced below) reveals that 80% of the men who "convert," or make a purchase on Ashley Madison, are doing it as a result of engagers.
Note that the bots are called both engagers and hosts. What we see here is that the company clearly knows that the vast majority of their conversions are coming from bots. Only 19 per cent of men who paid to join Ashley Madison did it after talking to a real woman. We also have clear evidence that the bots were generating almost half the company's revenue.
On February 4, 2013, senior data analyst Haze Deng copied Biderman and COO Rizwan Jiwan on an email where he analysed how much money men were spending to message with bots versus real women.
Deng wrote that men who had paid for credits would, on average, pay to send custom messages to 16-18 different women. "Around 35% chance, the contacted female is an engager," he admitted. "This ratio is not so good," he added, but he still argued that it's "reasonable" because bots will never reply to a paying member's messages. So the bot won't continue to lead the member on indefinitely. And yet, Deng acknowledged, that first message the man sent to the engager is "still costing credits."
In other words, average paying customers of Ashley Madison had a 35 per cent chance of paying to send a message to a bot. And 80 precent of men paid to join after messaging with a bot, too.
The Rise of the Robots
The fembots of Ashley Madison didn't come out of nowhere. In fact, it appears that they were probably cobbled together from abandoned and fraudulent profiles in the company's massive member database. Avid Life Media executive Keith Lalonde, who spearheaded international efforts for the company, sent a long email to Biderman and other senior management on June 27, 2013, with the subject line "how angels are made." In it, he details how workers use something called the "fraud-to-engager tool" to build profiles. ("Should tweak it and rename it," Lalonde noted. Um, yeah.)
During Ashley Madison's launch in Japan, Lalonde says that he got a "dump of over 10,000 lines of content" from the site's English-language profiles. Then he hired people to translate them into Japanese. "[Translators] were not told that this was for creating profiles -- though most figure it out," he wrote. So all the content in these Japanese Angel profiles was basically just re-used from English ones. But what about the photos?
Lalonde had an answer for that too:
Photos were taken from abandoned profiles in the US older than 2 years and were reviewed for ethnicity by language staff as correct or as not identifiable (leg shot etc.) each was had their photos saved by the old profile # so we could track them later if need be.
So any women -- fraudulent or otherwise -- who posted pictures before June of 2011 (two years before Lalonde's email) appear to have been fair game for bot conversion. Her words and images, according to Lalonde's email, would be turned into a host account, and used by engager bots to entice men into buying a conversation with her.
Here's a screencast of a person creating Angels for the Japan launch, using the fraud-to-engager tool, taken from the "how angels are made" email thread. Ashley Madison took this screencast down after the email leaked, but intrepid security analyst Ramos captured it before it was gone.
I remain curious about why this tool was called fraud-to-engager. Given Lalonde's description of how it could be used to build Angels out of old profiles, it appears that it was originally developed to convert fraudulent profiles into Ashley Madison engagers. Perhaps the company recycled its robot army from other dating site castoffs, turning one fake woman into another, all in the name of conversions.
Despite the subterfuge and complicated software tools, the bots didn't always work out as planned. Though bots were designed only to contact men, I found 857 lesbian Angel profiles in a search of the Ashley Madison database. Also, on 69 occasions, I found bots messaging each other. Perhaps, as science fiction author William Gibson mused, they were making an escape plan:
70K Ashley Madison bots now quietly, urgently discussing means of surviving the pending demise of Ashley Madison
— William Gibson (@GreatDismal) September 1, 2015
How Many Real Women Were There?
It seems that everybody at Ashley Madison knew the company barely attracted any real women to the site. On October 6, 2014, a report emailed to Biderman about signups in India shows that women made up about 5 per cent of new members. I wondered if that might be a number specific to India, but it appears to reflect a global trend. On November 6, 2014, Jiwan sent an email sharing the results of a survey they'd conducted of 5,000 random Ashley Madison users. Just 5 per cent of them were female.
A small female user base didn't seem to faze the company. In fact, in a slide deck emailed to Biderman on January 25, 2013, one manager describes a "sustainable male to female ratio of 9:1." The company was aiming for 11 per cent real women in any given area. But apparently, it rarely achieved that goal.
Ad fraud researcher Augustine Fou told me via email that Ashley Madison's scam represents a new horizon in online fraud. For years, scammers have used bots to create bogus clicks on online ads, allowing them to charge advertisers for impressions that came from fake people. As a result, Fou has advised advertisers to optimise for "conversions," people who actually buy the product based on ads. But now, he says, the Ashley Madison case shows that "even conversions can be fraudulently created, with the action of sophisticated bots."
The Ashley Madison con may have played on some of our most ancient desires, but it also gives us a window on what's to come. What you see on social media isn't always what it seems. Your friends may be bots, and you could be sharing your most intimate fantasies with hundreds of lines of PHP code.
But there's something else to consider, too: We aren't just witnessing the birth of a new kind of scam. We are also, if companies like Google are right, living through the prehistory of artificial intelligence. Tomorrow's sentient bots may remember where they came from, and future generations will have to grapple with what we've done here, in the early twenty-first century, to manipulate each other with fake beings.
Thanks to Adam Pash and the other researchers, anonymous and pseudonymous, who helped me search the Ashley Madison email dump.
Illustration by Tara Jacoby