Google Search Beats The Crap Out Of Siri In 1600 Question Test

Pitting an Google search against Siri using a monster 1600-question test shows how useful Siri really is: not at all. Android answered correctly 86 per cent of the time. Siri achieved just 68 per cent accuracy. At that point, it's not much better than a crystal ball.

We knew that Siri isn't very good. But this intense test shows just how ridiculous the gimmicky voice assistant could be.

The fact is that even Apple co-founder Steve Wozniak has been saying this since the day Apple introduced the iPhone 4S with Siri. It just sucks. Siri as an independent product, before Apple acquired it, the Woz told us at the Gizmodo Gallery:

It was really accurate, but now it's full of marketing-driven answers that are not correct.

How bad is it now? Here are some good examples from the test, which was conducted by Piper Jaffray analyst Gene Munster, a character who is well-known for his pro-Apple view of the tech world:

When did the movie Cinderella come out? Responded with a movie theatre search on Yelp.

What spices are in Lasagna? Responded with a Yelp search with lasagna on the menu.

I want to go to Lake Superior? Responded with directions to the company Lake Superior X-Ray.

Clearly, Woz is right: Apple's version of Siri is tainted because it's marketing driven, giving preference to commercial sites like Yelp or companies over actual, useful results.

Of course, you can argue that Siri is labelled as beta by Apple. But, to Woz's point, how did it end up being worse than it was as a standalone app available at the App Store? The one Apple bought when Steve Jobs was still running the company?

Which brings me back to a earlier point. Jobs' authorised biography says that he was at diminished capacity when Siri was being tested, too weak to come into the office. He only tried the current form of Siri at his last board meeting. He briefly played with it and, understandably given the moment, didn't show much interest. That was it. It's hard to believe that he would have let software with 68% accuracy to ever be installed in a shipping product.

A new version of Siri is coming in the new iOS 6. It looks a lot more useful, but I just hope that Apple ditches the commercialism in favour of giving the answers you actually need. [Fortune]


Comments

    A search engine vs a voice recognition assistant.
    Plus the google questions were all TYPED in. Of course it would understand 100% of the questions... So not really a level playing field to start off with is it?? Which pretty much means its not a valid comparison right from the start.
    Sounds more like the survey itself is gimmicky.

      Yeah I agree. From what I read elsewhere in a noisy area it recognised the question itself correctly 83% of the time and 89% in a quiet room. Starting at an 83% question recognition makes it hard to compete with Google which had 100% recognition due to the question being typed in.

      Thats not to say Apple's results are great. They aren't. However the test is incredibly biased because it is factoring in not only the ability to answer the question, but in the case of Siri the voice recognition technology also. It would have still done worse, but perhaps not as bad as it did.

      Really they should have used Google Voice when doing this. The results seem like they are geared towards sensationalism more than providing any real comparison.

      Its is completely valid, given that Siri is supposed to be a viable replacement for typing stuff in. If the only thing it shows is that it is not a valid replacement for typing queries into a search engine, then its job done.

    WRONG! This comparison is based on Google's new 4.1 software! The Google questions were SPOKEN.

    The test description says clearly that Google question were typed!

      Not the test description, just the artcle author's description of it. even the source article doesn't have any detail of the original test, it may be that he lacks knowlede of the Android optipns available and just assumed it was a typed test.
      It doesn't really make much sense that they wouldn't do a voice test on Google, since that's a feature that's been around forever...
      And makes a lot of mistake in voice recognition. The search function on Android is usually amazingly good, but the voice recognition function isn't perfect, at least in my experience.

        The article from which GIz is referencing says:
        "Piper Jaffray published the results Thursday in a note to clients:
        Google understands 100% of the questions (not surprisingly, since they are keyed in)"

        "Keyed in" sounds very much like "typed in" to me.

        At any rate, it's like comparing how well Google search vs siri is able to say, remind me to buy milk when I leave work, or send a message to my sister, wake me at 6am. Clearly the "study" isn't comparing apples with -er- apples!

          you might want to look at google now and see what it can do

    It makes no difference if you are buying a phone just for siri you are not getting the full picture, Google does not have to be better they just have to be close and this shows they are. I personally don't use apple due to itunes and the plane home screens but I do love the way it has music, however in a year from now so will android and then you will just be picking a device on looks nothing else.

    I'm sure Google search is crap at setting location based reminders, taking a spoken note for me or reading my text messages while I am driving.

    I can't believe somebody actually expected Siri to know "What spices are in Lasagna?"

      Why? That is exactly the kinds of things they show in their TV ads. I would expect it to at least find a Lasagna recipe if I asked it that.

        I have never seen any ad where somebody asks Siri such a vague question. There is no correct answer for that question, because there is no unified lasagna recipe.

        Also, Siri isn't linked to a recipe database. It is linked to specific functions on the phone, the yelp database and wolfram alpha. It works within those constraints.

        If you ask a food related question of course it refers to it's list of food outlets.

        If you say "Search for a lasagna recipe" it will probably do a Google search, and let you pick your own recipe.

          "I can’t believe somebody actually expected Siri to know “What spices are in Lasagna?”"

          Maybe you should watch more of their recent commercials. There's a Samuel L Jackson one where he uses Siri to ask almost the exact same question only for Risotto. This is exactly the use they're advertising and it can't do that.

            At no point does Samuel L Jackson ask for a recipe.
            He fcancels an appointment, finds a business, sets a reminder and does a unit conversion.

    absolute link bait article

    people who conduct tests to post about on the internet care about asking their phone the height of the eiffel tower, people in the real world send message or look up restaurants and cinemas.

    The picture shows the voice search function of android 4.1 much like siri.

    This article stinks. Like Google isn't "marketing driven"! LOL.That's their whole whole raison d'être! To sell advertisements!

Join the discussion!

Trending Stories Right Now