Where does the fun stop and the societal harm begin? Artificial intelligence companies behind image and text generators seem to be grappling with that question in real-time, as their products gain traction across the internet. Simultaneously, the same companies are also running up against some very standard internet issues — like users angling to take advantage of free trial offers.
Popular AI image generator, Midjourney, halted its free trial access this week, as first reported by the Washington Post. The announcement came in a Tuesday morning Discord post from Midjourney CEO, David Holz citing “extraordinary demand and trial abuse.”
According to a follow-up post, Holz and co tried to address the issues of abuse with safety patches. However, the attempted changes weren’t enough to fix the problems. “We tried turning trials back on again with new safeties for abuse but they didn’t seem to be sufficient..we are turning it back off again,” the CEO posted Wednesday afternoon. The company’s paid service remains available and while the free trials remain paused for now, they could come back at any time.
Though the Washington Post report initially insinuated that the “abuse” in question was related to a recent slew of viral deepfakes, the company denies that account. “There were some pretty big misunderstandings in the [WaPo] article,” Holz wrote to Gizmodo in an email. “We stopped trials because of massive amounts of people making throwaway accounts to get free images…This happened at the same time as a temporary gpu shortage. The two things came together and it was bringing down the service for paid users,” he added.
The “new safeties” referenced in the Wednesday Discord post? According to Holz, this, too, was about trying to limit users to one free trial each. “We’re still trying to figure out how to bring free trials back, we tried to require an active email but that wasn’t enough so we’re back to the drawing board,” he said to Gizmodo.
Thanks to the recent generative AI boom, the use of image generators to create deepfakes are having a moment. Social media feeds have become clogged with not real, but pretty convincing, images of things like a dripped out, swaggy AF Pope Francis wearing a Balenciaga puffer jacket or signing the hood of a Ferrari-esque sports car.
I think Balenciaga pope might be the first real mass-level AI misinformation case. Kinda cool to watch tbh. https://t.co/hPoJeXwA1z
— Ryan Broderick (@broderick) March 26, 2023
Various fakes of Donald Trump being arrested in various positions circulated widely after news broke of the former president’s expected indictment.
Making pictures of Trump getting arrested while waiting for Trump’s arrest. pic.twitter.com/4D2QQfUpLZ
— Eliot Higgins (@EliotHiggins) March 20, 2023
Elsewhere, you may have come across such images as Russian President Vladimir Putin facing a Hague tribunal, or maybe John F. Kennedy aiming a rifle out of a window. All of these images were created using Midjourney, one of the most popular, publicly available AI’s.
Yet the images aren’t reason enough to restrict or shut down Midjourney access, Holz implied. Instead, the company claims its current prohibition on free trials is a form of profit protection. It wants users to move from the free trial to a paid subscription, not another free trial.
Holz further implied that the most recent suite of problematic, political images are the product of Midjourney v5 — which has only been made available to paying users, anyway. When they were open, free trial sign-ups limited users to the 5-month old v4, Holz said.
Nonetheless, the CEO still said that Midjourney is working on improved moderation. “I think we’re still trying to figure out what the right moderation policies are. We are taking feedback from experts and the community and trying to be really thoughtful. We already have some new systems coming which should ship soon,” he wrote.
The company’s so-far limited efforts to halt the rise of entertaining, yet potentially dangerous and deceptive, false images demonstrate the murky issues of AI-regulation we’ve waded into. How will generative AI software developers balance the pressure for access, “free speech,” and making a product people actually want to use with social responsibility and the risk for glaring misuse? And will any U.S. agencies step in?
For now, the answers remain unclear, to everyone — tech CEOs included.
Compared with some of its peers like OpenAI’s DALL-E, which has policies barring users from creating any sexual or violent content and including real political figures, Midjourney’s standards are much more permissive. Yet compared with Stable Diffusion, where users can download an open-source software and effectively do whatever they like, Midjourney has more guardrails.
You can’t, for instance, use Midjourney to generate images of China’s president Xi Jinping, though other world leaders remain fair game. Holz explained this decision with a Discord post last year. “Political satire in [China] is pretty not-ok,” the CEO wrote, per the Washington Post. “The ability for people in China to use this tech is more important than your ability to generate satire,” he added.
And the company recently added the term “arrested” to its list of banned prompt phrases, according to The Verge, likely in response to those aforementioned Trump images.
But still, navigating the vast and nebulous centre of what should and shouldn’t be allowed on the internet is clearly proving difficult for Holz. On Wednesday, during a live Discord session, the exec told an audience of thousands that he was having a hard time zeroing in on content rules — particularly surrounding the use of real peoples’ images, according to WaPo. “There’s an argument to go full Disney or go full Wild West, and everything in the middle is kind of painful,” the CEO reportedly said. “We’re kind of in the middle right now, and I don’t know how to feel about it.”
The moratorium on free Midjourney access doesn’t solve the issue. And, again, the CEO claims the decision to stop free trials isn’t the result of deepfake dissemination at all. But maybe, perhaps, it might slow the problem down by limiting the number of people who can produce scary-real, fake pictures of nearly whatever they want.