Safaricom's Zuri on trial: Man in court over use of AI for customer care

Investigator Chris Raven is suspected of murdering his wife in the film Mercy, released this year. He ends up being tried by Judge Maddox, an Artificial Intelligence (AI) system he created and strongly supported for handling crime.

The 90-minute thriller, which features Chris Pratt as the accused person and Rebecca Ferguson as the AI, portrays a futuristic criminal justice system. In it, the accused must reduce the “guilt score” assigned by the AI, set at 92 per cent, using text records, surveillance evidence and camera footage, to avoid instant execution by the so called  “mercy chair.”

Maddox is depicted as a cold, unflinching system that knows all the laws and applies evidence fed into it. In effect, a person is deemed guilty  before a full trial unfolds.

The real culprit turns out to be Raven’s police partner, Jaq, who committed the murder after discovering a flaw in Maddox’s logic.

The system is also aware of this flaw, but still frames Raven as an example to others, reinforcing the idea that no one was above AI’s-driven justice. In the final moments, Raven manages to escape by hacking into Maddox. The movie depicts the dilemmas that arise when artificial intelligence is used in place of human judgement.

Closer home, Zuri, an AI-powered virtual assistant chatbot created by Safaricom to handle a wide range of customer issues, is at the centre of a landmark case.

The telecommunications company automated the customer care system, removing human agents as the first point of contact. The chatbot is designed to handle services such as M-Pesa reversals, account services and subscriptions.

However, Victor Odhiambo has challenged Zuri’s effectiveness, arguing that it cannot adequately resolve all customer concerns. His case against the AI system is now before the High Court.

Customers’ data

Zuri is accessed through dialling 100, 234 as well as via Safaricom’s WhatsApp platforms.

Onyango, however, argues that machine learning based on millions of customers’ data and their behaviour is not sufficient to make clear judgments on the unique queries sent by users. He further contends that where Zuri fails, or a customer is dissatisfied, the telecom giant does not provide an automatic human fallback.

“Consumers dialing the principal customer care numbers are typically routed to ‘Zuri’, an automated system, and do not consistently have access to a clear, guaranteed, or reasonably accessible pathway to a human agent capable of exercising discretionary intervention in matters affecting their rights,” Odhiambo’s case, exclusively seen by The Standard on Saturday, reads in part.

Safaricom, on August 13, 2025, announced that it had expanded Zuri’s reach to WhatsApp.

The system, according to the telco,  serves at least 20 million subscribers.

However, Odhiambo argues that customers raising urgent complaints, such as cases of money fraud, are often unable to access human intervention or immediate discretion in resolving their issues.

Instead, he portrays Zuri as a monotonous gatekeeper whose responses, much like Judge Maddox’s, are generic and largely based on patterns derived from its training data.

Odhiambo argues that this amounts to algorithmic bias, as Zuri is unable to distinguish between genuine complaints and routine queries.

He maintains that the system’s shortcomings begin the moment a user dials for assistance.

According to him, between July 2025 and February 2026, there was a consistent and widespread pattern of concern that the AI system repeatedly fails to process queries, instead looping users through a “sorry, repeat, I did not get that clearly” response.

“Consumers requesting urgent intervention in financial emergencies, including mistaken transfers, fraud incidents and account compromise, are unable to access a human agent capable of discretionary, time-sensitive intervention.

Consumers who require reversal of mistaken transfers or fraud mitigation report prolonged or permanent financial loss linked to the inability to reach human escalation pathways,” the petition reads.

“A consumer who has been defrauded, or who has sent money to a wrong number, or whose account has been wrongly restricted, must be able to reach a human being capable of reasoned, discretionary intervention. The elimination of that human element from decisions affecting fundamental economic rights is not a service design inconvenience. It is a constitutional failure,” Odhiambo’s case, filed by his lawyers Mulanya and Maondo Advocates, continues to read.

Lack clear logic

He states that the M-Pesa Fintech 2.0 operations process at least 100 million transactions daily and about 37.15 billion annually.

Odhiambo argues that automated decisions lack clear logic, despite directly affecting how customers access funds and interact with the telecom company.

“For consumers adversely affected by such automated determinations, there is no clear and accessible disclosure of the logic involved, the significance and envisaged consequences of the decision, or guaranteed access to timely human intervention and reconsideration, as contemplated under Section 35 of the Data Protection Act, 2019, and the procedural fairness guarantees under Article 47 of the Constitution,” he argues.

He laments that the closest the AI system comes to empathy is issuing an apology and acknowledging user frustration.

Kenya currently has no specific law regulating artificial intelligence. The closest legislative effort is a Bill sponsored by Nominated Senator  Karen Nyamu. She is of the view that the government should not have allowed, or should not allow, the deployment of AI systems without proper safeguards in place.

Safaricom identifies its customers using biometrics, identification numbers and photographs provided during SIM card registration. It also profiles users through voice authentication.

The company additionally operates a credit-scoring system that determines how much a customer can borrow in emergencies under its overdraft service, Fuliza.

The petitioner has raised concerns over these systems as well. He claims that in instances where money is lost through SIM swap fraud or deception, user information is allegedly shared with third parties without consent.

He further argues that the presence of human oversight would make it far more difficult for hackers to manipulate or trick individuals into surrendering sensitive access credentials.

He adds that vulnerable groups, including rural consumers, the elderly, persons with disabilities and those with low digital literacy, are disadvantaged due to the lack of an effective non-digital fallback system.

Creditworthiness

On Fuliza, he argues that the criteria used to determine creditworthiness is not transparent.

He also maintains that there is no clear mechanism for customers to challenge credit limits they believe are unfairly set. According to him, the system denies users a fair hearing in decisions affecting their financial access. He argues that the system has entrenched discrimination by allocating different credit limits to users without providing clear explanations for the differences.

At the same time, Odhiambo warns of potential AI-enabled abuse of dominance, where more compliant customers receive better services based on their data profiles, while those who interact less with the system are disadvantaged.

“Article 27(1) guarantees equal protection and benefit of the law. Where algorithmic credit scoring incorporates proxies for protected characteristics, the resulting differential allocation constitutes indirect discrimination, prohibited whether or not intentional,” Odhiambo says.

He maintains that artificial intelligence should not serve as the primary point of contact between service providers and users.

He further argues that it undermines human dignity to have machines collect personal data, analyse behaviour and render judgments without human involvement.

He says it is cold to rely on algorithms for commercial optimisation.

“To replicate a person as an algorithmic construct, to simulate their behaviour and preferences for commercial purposes and to act upon those simulations without the individual’s knowledge or any mechanism for challenge, is to treat human identity as raw material for technological exploitation. This is inconsistent with the constitutional recognition of inherent dignity as an attribute of personhood that cannot be algorithmically commodified,” he says.

Odhiambo has sued the Attorney-General, Safaricom, the Communications Authority of Kenya (CA), the Competition Authority of Kenya (CAK) and the Kenya Consumers Protection Advisory Committee.