llustration: Jim Cooke/GMG, photo: Getty
The story in the New York Times this week was unsettling: The New America Foundation, a major think tank, was getting rid of one of its teams of scholars, the Open Markets group. New America had warned its leader Barry Lynn that he was "imperiling the institution," the Times reported, after he and his group had repeatedly criticised Google, a major funder of the think tank, for its market dominance.
The criticism of Google had culminated in Lynn posting a statement to the think tank's website "applauding" the European Commission's decision to slap the company with a record-breaking $US2.7 ($3) billion fine for privileging its price-comparison service over others in search results. That post was briefly taken down, then republished. Soon afterward, Anne-Marie Slaughter, the head of New America, told Lynn that his group had to leave the foundation for failing to abide by "institutional norms of transparency and collegiality."
Google denied any role in Lynn's firing, and Slaughter tweeted that the "facts are largely right, but quotes are taken way out of context and interpretation is wrong." Despite the conflicting story lines, the underlying premise felt familiar to me: Six years ago, I was pressured to unpublish a critical piece about Google's monopolistic practices after the company got upset about it. In my case, the post stayed unpublished.
I was working for Forbes at the time, and was new to my job. In addition to writing and reporting, I helped run social media there, so I got pulled into a meeting with Google salespeople about Google's then-new social network, Plus.
The Google salespeople were encouraging Forbes to add Plus's "+1" social buttons to articles on the site, alongside the Facebook Like button and the Reddit share button. They said it was important to do because the Plus recommendations would be a factor in search results — a crucial source of traffic to publishers.
This sounded like a news story to me. Google's dominance in search and news give it tremendous power over publishers. By tying search results to the use of Plus, Google was using that muscle to force people to promote its social network.
I asked the Google people if I understood correctly: If a publisher didn't put a +1 button on the page, its search results would suffer? The answer was yes.
After the meeting, I approached Google's public relations team as a reporter, told them I'd been in the meeting, and asked if I understood correctly. The press office confirmed it, though they preferred to say the Plus button "influences the ranking." They didn't deny what their sales people told me: If you don't feature the +1 button, your stories will be harder to find with Google.
With that, I published a story headlined, "Stick Google Plus Buttons On Your Pages, Or Your Search Traffic Suffers," that included bits of conversation from the meeting.
The Google guys explained how the new recommendation system will be a factor in search. "Universally, or just among Google Plus friends?" I asked. 'Universal' was the answer. "So if Forbes doesn't put +1 buttons on its pages, it will suffer in search rankings?" I asked. Google guy says he wouldn't phrase it that way, but basically yes.
(An internet marketing group scraped the story after it was published and a version can still be found here.)
Google promptly flipped out. This was in 2011, around the same time that a congressional antitrust committee was looking into whether the company was abusing its powers.
Google never challenged the accuracy of the reporting. Instead, a Google spokesperson told me that I needed to unpublish the story because the meeting had been confidential, and the information discussed there had been subject to a non-disclosure agreement between Google and Forbes. (I had signed no such agreement, hadn't been told the meeting was confidential, and had identified myself as a journalist.)
It escalated quickly from there. I was told by my higher-ups at Forbes that Google representatives called them saying that the article was problematic and had to come down. The implication was that it might have consequences for Forbes, a troubling possibility given how much traffic came through Google searches and Google News.
I thought it was an important story, but I didn't want to cause problems for my employer. And if the other participants in the meeting had in fact been covered by an NDA, I could understand why Google would object to the story.
Given that I'd gone to the Google PR team before publishing, and it was already out in the world, I felt it made more sense to keep the story up. Ultimately, though, after continued pressure from my bosses, I took the piece down — a decision I will always regret. Forbes declined comment about this.
But the most disturbing part of the experience was what came next: Somehow, very quickly, search results stopped showing the original story at all. As I recall it — and although it has been six years, this episode was seared into my memory — a cached version remained shortly after the post was unpublished, but it was soon scrubbed from Google search results. That was unusual; websites captured by Google's crawler did not tend to vanish that quickly. And unpublished stories still tend to show up in search results as a headline. Scraped versions could still be found, but the traces of my original story vanished. It's possible that Forbes, and not Google, was responsible for scrubbing the cache, but I frankly doubt that anyone at Forbes had the technical know-how to do it, as other articles deleted from the site tend to remain available through Google.
Deliberately manipulating search results to eliminate references to a story that Google doesn't like would be an extraordinary, almost dystopian abuse of the company's power over information on the internet. I don't have any hard evidence to prove that that's what Google did in this instance, but it's part of why this episode has haunted me for years: The story Google didn't want people to read swiftly became impossible to find through Google.
Google wouldn't address whether it deliberately deep-sixed search results related to the story. Asked to comment, a Google spokesperson sent a statement saying that Forbes removed the story because it was "not reported responsibly," an apparent reference to the claim that the meeting was covered by a non-disclosure agreement. Again, I identified myself as a journalist and signed no such agreement before attending.
As for how effective the strategy was, Google's dominance in other industries didn't really pan out for Plus. Six years later, the social network is a ghost town and Google has basically given up on it. But back when Google still thought it could compete with Facebook on social, it was willing to play hardball to promote the network.
Google started out as a company dedicated to ensuring the best access to information possible, but as it's grown into one of the largest and most profitable companies in the world, its priorities have changed. Even as it fights against ordinary people who want their personal histories removed from the web, the company has an incentive to suppress information about itself.
Google said it never urged New America to fire Lynn and his team. But an entity as powerful as Google doesn't have to issue ultimatums. It can just nudge organisations and get them to act as it wants, given the influence it wields.
Lynn and the rest of the team that left New America Foundation plan to establish a new nonprofit to continue their work. For now, they have launched a website called "Citizens Against Monopoly" that tells their story. It says that "Google's attempts to shut down think tanks, journalists, and public interest advocates researching and writing about the dangers of concentrated private power must end."
It's safe to say they won't be receiving funding from Google.
This story was produced by Gizmodo Media Group's Special Projects Desk.