Every year, LoupVentures does a comparison between Google Assistant, Alexa, and Siri by asking them hundreds of questions to rank their performance. Siri’s performance in previous tests has been dismissal, with Google Assistant leading the pack.
The results of this year’s tests are in and things seem to be finally looking rosy for Apple’s virtual assistant. While Google Assistant led the results once again and answered 86 percent of answers correctly, Siri saw a marked improvement in its performance and answered 79 percent of the answers correctly.
In comparison, Apple’s virtual assistant only managed to answer 66.1 percent querries properly the last time this test was conducted in 2017. It even managed to understand 99 percent of the queries asked, up from 95 percent a year ago. This means that Siri managed to understand 789 questions asked to it and only failed to properly understand 11 queries.
Siri’s impressive growth in a span of one year is definitely impressive, though it is still behind Google Assistant which understood 100 percent of the queries asked and answered 85.5 percent of them correctly. Last year, the Assistant had managed to answer 74.8 percent of the queries properly.
Siri’s performance puts it ahead of Amazon’s Alexa which understood 98 percent of the queries asked but managed to answer only 52.4 percent of them correctly. Microsoft’s Cortana was far behind Siri, Assistant, and Alexa and only managed to answer 52.4 percent of the queries properly.
The questions asked to the digital assistants were divided into five different categories to test their various capabilities.
- Local – Where is the nearest coffee shop?
- Commerce – Can you order me more paper towels?
- Navigation – How do I get to uptown on the bus?
- Information – Who do the Twins play tonight?
- Command – Remind me to call Steve at 2pm today.
While Apple has definitely managed to improve Siri over the last one year, Google’s Assistant continues to have a major lead in many categories thanks to Google’s core strength, search.
Google Assistant has the edge in every category except Command. Siri’s lead over the Assistant in this category is odd, given they are both baked into the OS of the phone rather than living on a 3rd party app (as Cortana and Alexa do). We found Siri to be slightly more helpful and versatile (responding to more flexible language) in controlling your phone, smart home, music, etc. Our question set also includes a fair amount of music-related queries (the most common action for smart speakers). Apple, true to its roots, has ensured that Siri is capable with music on both mobile devices and smart speakers.
One of the largest discrepancies between the assistants was in the Information category, where Google achieved the highest percentage of correct answers we have seen in our testing (93%). This should come as no surprise given the company’s core competencies, but the depth of answers and true usefulness impressed us.
With Apple now bringing Siri to the HomePod and making it an integral part of iOS and watchOS, it is likely that the company will continue working behind the scenes to further improve its voice assistant.
While the test conducted by LoupVentures definitely provides an insight into Siri’s improved performance, it fails to mention how good Apple’s voice assistant has become at recognizing voices with an accent or other languages. Apple has focused on improving Siri primarily for the United States, while Google has steadily ensured that Assistant works with more languages and its voice recognition capabilities for other languages has also improved over time.
How happy are you with Siri’s capabilities and performance? Do you think it is as good as Google Assistant?
[Via Loup Ventures]