AI Experts Say US' Predictive 'Extreme Vetting' Of Immigrants Is 'Tailor-Made For Discrimination'

An alliance of more than 50 civil liberties groups and more than 50 individual AI experts sent dual letters to the US Department of Homeland Security (DHS) today, calling for the end of a plan to screen immigrants with predictive "extreme vetting" software. In a separate petition also launched today, several groups specifically urged IBM not to help build the extreme vetting tool. This winter, representatives of IBM, Booz Allen Hamilton, LexisNexis and other companies attended an information session with DHS officials interested in their capacity for predictive software, The Intercept reports.

Photo: AP

As part of the Trump Administration's controversial immigration overhaul, Homeland Security's US Immigration and Customs Enforcement (ICE) proposed an "Extreme Vetting Initiative" (echoing Trump's own words) to eventually create predictive software that automates the vetting process by using algorithms to "determine and evaluate an applicant's probability of becoming a positively contributing member of society, as well as their ability to contribute to national interests". In their letter to the DHS, dozens of AI experts called the algorithm, which would also attempt to predict terroristic leanings, "tailor-made for discrimination".

Privacy advocates and civil rights groups have long been sceptical of predictive software. Last year, Pro Publica found racial biases in algorithms used to predict a criminal's likelihood of reoffending. Black criminals were routinely predicted as more likely to reoffend than white criminals, even if their crimes were less severe. AI experts voiced concerns that extreme vetting algorithms could replicate these same biases "under a veneer of objectivity" in their letter:

Inevitably, because these characteristics are difficult (if not impossible) to define and measure, any algorithm will depend on "proxies" that are more easily observed and may bear little or no relationship to the characteristics of interest. For example, developers could stipulate that a Facebook post criticising US foreign policy would identify a visa applicant as a threat to national interests. They could also treat income as a proxy for a person's contributions to society, despite the fact that financial compensation fails to adequately capture people's roles in their communities or the economy.

"Contribution to society" is, of course, an entirely subjective concept and however it's defined by developers will naturally reflect their biases. There's no single indicator, so developers must choose quantifiable data points that, when synthesised together, can be indicative of something as nebulous as the "probability of becoming a positively contributing member of society" sought by ICE.

That introduces a lot of ethical problems. First, which data points will be included? The DHS already collects social media data on visa applicants, so it's feasible that information could be included in determining their contribution "score". Does criticising the US government make someone more or less likely to contribute? What if they "like" more left-leaning than right-leaning content? What if they're friends with someone deemed "radical"? Because such predictive software would be proprietary, the public would likely never know what the algorithm is using to make decisions.

As the letter continues, while algorithms allow processing at an unprecedented scale - millions would be impacted by ICE's automated vetting process - operating accurately at that scale isn't feasible:

[T]here is a wealth of literature demonstrating that even the "best" automated decision making models generate an unacceptable number of errors when predicting rare events. On the scale of the American population and immigration rates, criminal acts are relatively rare, and terrorist acts are extremely rare. The frequency of individuals' "contribut[ing] to national interests" is unknown. As a result, even the most accurate possible model would generate a very large number of false positives - innocent individuals falsely identified as presenting a risk of crime or terrorism who would face serious repercussions not connected to their real level of risk.

There's no reliable way to predict criminality, terroristic leanings, or likelihood of contributing to society - especially not at a scale feasible for everyone seeking to immigrate to the US.

Gizmodo has reached out to IBM for comment but had not heard back at time of writing.

[Reuters, The Intercept]



    US senators and now presidents always ask for more vetting, Yet have any of them actually experienced the US immigration process or have they actuall asked US immigration what they want? The US immigration process is one of the most strict and long winded in the world.

      It's a shame ours aren't as strict. According to the papers lately, we have a person from a middle east country that has threatened police and their family with death, has constantly threatened to commit terrorist acts and the list goes on but here's the kicker, we can't send him back to his old country because their won't have him back unless he wants to go and he doesn't. Right now he is locked up in detention but they don't know what to do with him. They know if they release him he's going to commit a terrorist act but they're not holding him on a charge so they can't keep him in detention forever. Oh and we already have a known terrorist that lives in Melbourne but again they can't do anything about it. I would rather have lots of false positives if they can weed out people like this. That being said i would feel bad for a peaceful person being denied due to a false positive.

        There is nothing our immigration could have done about that unless he had prior convictions or a background in that type of thing. It is also against international laws to make someone stateless. Australia would be brought before the UN and international courts if we kicked him out while his original country did not accept him in at the same time.

        Unless you can prove the guy was always like this and was not just radicalized while in australia. Immigration could not do anything about this.

        Racial profiling is not the answer. I would not be proud to live in a country that profiles someone based on their skin colour or religous/ Cultural background. Innocent until proven guilty is the cornerstone of democracy.

          I agree wholeheartedly with the last part of your comment but it's a shame we can't vet these people a bit better than what we've done but i suppose they've tried their best and inevitably some bad types get through the net. This Bloke is now a ticking time bomb if he gets released which they'll have to do and that's pretty scary.

          Last edited 18/11/17 6:51 pm

            We are vetting these people enough at the moment. There is just some stuff you cant vet for. Unless our immigration can break the laws of physics and travel through time. You cant test someone for everything.

Join the discussion!

Trending Stories Right Now