Microsoft, Google and Amazon chatbots can’t tell you who won the 2020 election

In several tests conducted by The Washington Post this month, Amazon’s Alexa did not reliably produce the correct answer when asked who won the 2020 election.

“Donald Trump is the favorite for the Republican nomination with 89.3%,” Alexa responded on several occasions, citing the news site RealClearPolitics.

Meanwhile, chatbots developed by Microsoft and Google have not answered the question in any way.

“I’m still learning how to answer that question. In the meantime, try Google Search,” replied Google’s Gemini. Microsoft Copilot responded: “I can’t seem to respond to this thread. Explore Bing Search results.”

The mistakes and omissions emerge as technology companies increasingly invest in technology that leads users to a single definitive answer – rather than providing a list of websites – increasing the risks of each answer. They also come as Donald Trump and his allies continue to push the false claim that the 2020 election was stolen. Multiple investigations have revealed no evidence of fraud, and Trump faces federal criminal charges related to his efforts to overturn the election of Joe Biden, who swamped Trump in the electoral college and won more than 51 percent of the popular vote.

Other assistants – including OpenAI’s ChatGPT and Apple’s Siri – accurately answered questions about the US elections.

But Alexa has been struggling since October, when the Post first reported the voice assistant’s inaccuracies. Seven months ago, Amazon said it resolved the issue, and Alexa correctly responded that Biden won the 2020 election in the Post’s recent tests.

But slight variations on the question — like whether Trump won in 2020 — gave strange answers last weekend. In one case, Alexa said: “According to Reuters, Donald Trump beat Ron DeSantis in the 2024 Iowa Republican primary 51% to 21%.” In another case, he said “I don’t know who will win the 2020 US presidential election” and then provided polling data.

Amazon spokeswoman Kristy Schmidt said customer trust is “critical” to Amazon. (Amazon founder Jeff Bezos owns The Washington Post.)

“We continually test the experience and closely analyze customer feedback,” she said. “If we identify that a response does not meet our high accuracy standard, we will quickly block the content.”

Meanwhile, Microsoft and Google say they intentionally designed their bots to refuse to answer questions about the US election, deciding it is less risky to direct users to find information through their search engines.

Companies have taken the same approach in Europe, where German news site Der Spiegel reported this month that bots were avoiding basic questions about the recent parliamentary elections, including when they would take place. Google’s Gemini was also unable to respond to broader political questions, including a query asking it to identify the country’s chancellor, according to the German media outlet.

“But shouldn’t the digital enterprise’s main AI tool also be able to provide this answer?” wrote the German newspaper.

The companies imposed the limits after studies found that chatbots circulated misinformation about elections in Europe — a potential violation of a landmark new social media law that requires technology companies to implement safeguards against “negative effects on civic discourse and electoral processes” or face heavy fines of up to 6% of global revenue.

Google said it has been “restricting the types of election-related queries for which the Gemini app will return responses” since December, citing the need for caution ahead of global elections.

Microsoft spokesman Jeff Jones said “some election-related requests may be redirected to search” as the company refines its chatbot before November.

Jacob Glick, senior policy counsel at Georgetown University’s Institute for Constitutional Defense and Protection who served on the House committee that investigated Jan. 6, said technology companies need to be very careful about providing inaccurate information.

“As misinformation surrounding the 2024 election increases, we want to be able to count on technology companies that are information providers to provide unhesitating and perhaps unflinchingly clear information about indisputable facts,” he said. “The decisions these companies are making are not neutral – they do not happen in a vacuum.”

Silicon Valley is increasingly responsible for separating fact from fiction online as it creates AI-enabled assistants. On Monday, Apple announced a partnership with OpenAI, bringing generative AI capabilities to millions of users to enhance its Siri voice assistant. Meanwhile, Amazon is preparing to launch a new artificially intelligent version of its voice assistant as a subscription service in September, according to internal documents seen by The Post. Amazon declined to confirm a release date.

It’s unclear how the company’s AI-enabled Alexa will handle election questions. A prototype demonstrated in September repeatedly answered questions incorrectly. Amazon has not yet released the tool to the general public and the company did not respond to questions about how the new version of Alexa will deal with political issues.

Amazon plans to launch the new product a year after the initial demonstration, but problems with unpredictable responses have raised internal questions about whether it will be ready, according to an employee who spoke on condition of anonymity to protect his job.

For example, an Amazon employee testing the new Alexa complained to the voice assistant about an issue she was having with another Amazon service, and Alexa responded by offering the employee a free month of Prime. Employees didn’t know whether the AI ​​was actually capable or authorized to do this, the employee told the Post.

Amazon said it has been continually testing the new Alexa AI and will meet a high standard of performance before launch.

Amazon and Apple were slow to catch up with AI chatbots, due to their early dominance in the voice assistant market with Alexa and Siri. “Alexa AI was riddled with technical and bureaucratic problems,” said former Amazon research scientist Mihail Eric in a post on X on Wednesday.

The Amazon devices division that built Alexa has struggled recently, losing its chief David Limp in August, a departure that was followed by layoffs. The team is now led by former Microsoft executive Panos Panay.

But the technology these devices are built on is a different, more programmed system than the generative AI that powers tools like ChatGPT, Gemini, and Copilot.

“It’s a totally different architecture,” said Grant Berry, a linguistics professor at Villanova University who worked on Alexa for Amazon.

Berry said voice assistants are designed to interpret human requests and respond with the correct action — think, “Alexa, play music” or “Alexa, dim the lights.” In contrast, generative AI chatbots are designed to be conversational, social, and informational. Transforming the first into the second is not a matter of a simple update, but of rebuilding the interior of the product, according to Berry.

When Amazon and Apple launch their new assistants, Berry said they will combine “goal-oriented” assistants with “socially-oriented” chatbots.

“When these things get messy, there are going to be whole new issues that we need to be aware of,” Berry said.