When AI Makes the Decisions, Who Is Responsible?

We have a habit of talking about AI as though it decides things on its own. The algorithm rejected the application, the system flagged a risk, the technology preferred another candidate. But an AI system doesn't arrive in an organisation with its own idea of fairness, success or what makes someone suitable for a job. People choose to introduce it, decide what it should measure, determine which information it can access and set the rules for how its recommendations will be used.

Imagine a company that introduces AI to help identify employees with promotion potential. The system weighs performance ratings, qualifications, completed projects and other available data, then produces a ranked list. The results look consistent, because everyone has been assessed against the same criteria. But who decided those were the right criteria? What about the employee who spent the year mentoring colleagues rather than leading visible projects, the person who took parental leave, or the team whose manager consistently rates people who work late more highly?

An AI system can process every record correctly and still produce a misleading assessment, because the problem may lie in what the organisation chose to count rather than in the calculation itself. That's why checking whether the technology is functioning as designed is only the start of accountability. A system can work exactly as intended and still produce an outcome that should never have happened.

Suppose you're a manager choosing between two employees for promotion. You know both of them well, you've seen how they handle difficult situations and support colleagues, and you have a preferred candidate. Then the AI system produces its assessment, and it strongly favours the other person. Would you challenge it?

Perhaps. Now suppose senior leadership introduced the system specifically to improve consistency and reduce subjective judgement, and any manager who disagrees with its recommendation has to submit a written justification. Suddenly, going along with the system feels like the safer option. You may still believe your own assessment was right, but now the burden is on you to defend it.

This is one way automation bias develops: people give too much weight to an automated recommendation, particularly when the system seems authoritative or questioning it feels difficult. The technology never has to issue an instruction, because the organisation's processes turn its recommendation into one. Over time, people stop asking whether the AI is right and start asking whether they have enough evidence to disagree with it, which is a very different question. A decision doesn't become genuinely human simply because a person clicks "Approve".

"Human in the loop" is one of those phrases that sounds reassuring. The AI does the analysis, a person reviews the result and the person makes the final call. But meaningful oversight takes more than placing someone somewhere in the process. Picture an employee reviewing hundreds of AI-generated recommendations a day, each with a summary and a suggested outcome. They have limited time, they may not be able to see the underlying information, they may not understand the system's limitations and they may have no authority to question the criteria. Technically a human is involved, but it's hard to say what they're really reviewing. If the only realistic choices are to approve the recommendation or fall further behind with a growing queue, oversight becomes a formality.

For human review to mean something, the reviewer needs to understand what they're assessing, have enough information and time to recognise a problem, and hold genuine authority to change the outcome. Organisations sometimes confuse having a human involved with having an effective safeguard, and they're not the same thing. A signature at the bottom of a document doesn't prove anyone understood what they signed, and an approval button doesn't transfer understanding from a machine to a person.

This distinction now has legal weight in the UK, as I'll come to shortly. Whether a decision counts as "solely automated" depends on whether there was meaningful human involvement, which means a reviewer who simply rubber-stamps the system's output may not change the legal position at all.

People make poor decisions all the time. Recruiters overlook strong candidates, managers misjudge performance and organisations introduce policies that turn out to be unfair. None of that started with AI, so it's fair to ask why automated decisions deserve more concern than the mistakes humans already make. Part of the answer is scale. A recruiter who misunderstands a qualification and rejects a suitable applicant affects one person. An AI screening system that consistently misreads the same qualification could affect hundreds or thousands of applicants before anyone notices. The system doesn't get tired or forget to apply its criteria. It simply repeats the same process, and if that process contains a flaw, its consistency makes the problem worse rather than better.

Not all mistakes carry the same consequences either. An AI assistant that puts a meeting in the wrong slot creates an inconvenience. A system that wrongly marks someone as unsuitable for promotion could affect their income, confidence and future prospects, and one that influences dismissals, financial decisions or access to essential services could do more serious harm still. So the useful question is whether an AI system is accurate enough for the particular decision it's being used to support. The more serious the consequences, the less acceptable it is to treat an AI recommendation as something nobody needs to question.

Go back to the rejected applicant. They ask which requirement they failed to meet and the recruiter can't say. They ask for the assessment to be reviewed, and the reviewer sees the same score and the same recommendation. Nobody can tell whether the system misunderstood the application, relied on unsuitable criteria or produced a perfectly reasonable result, so the applicant is left trying to challenge something they can't see. This is where accountability stops being a corporate principle and becomes a practical question. Can someone find out what happened, correct inaccurate information and challenge a significant outcome, and will anyone with real authority reconsider it?

These questions are reflected in UK law. Section 80 of the Data (Use and Access) Act 2025, which came into force on 5 February 2026, replaced the UK GDPR's rules on automated decision-making.^[1][2] Where a significant decision about someone is based solely on automated processing, organisations must have safeguards that allow the person to receive information about the decision, make representations, obtain human intervention and contest the outcome. A decision counts as solely automated if there's no meaningful human involvement in reaching it, and stricter limits apply where special category data, such as health information, is involved. The precise rights and exceptions depend on the circumstances, and the Information Commissioner's Office publishes guidance for organisations on how they apply.^[3]

The underlying principle is broader than any single law, though. Nobody should need to understand machine learning to ask why an important decision about their life was made, or be told that the answer is unavailable simply because software was involved.

Now suppose an employer discovers that its AI recruitment system has been unfairly screening out applicants. A technology supplier developed the product, an implementation team configured it, recruiters used it and managers accepted the results. The supplier says the employer chose how to deploy it. The employer says it relied on the supplier's assurances. The recruiter says they followed the approved process, and the manager says they never saw the rejected applications. Every one of those explanations may contain some truth, and none of them helps the people who were affected.

This is one of the real difficulties with AI systems that span several organisations and teams: responsibility becomes fragmented. Legal obligations may differ for the developer, the supplier, the employer and the individuals involved, depending on the system and where it's used. The EU AI Act, for example, treats AI used for recruitment and certain other employment decisions as high-risk, and sets separate obligations for providers, who build the systems, and deployers, who use them. Deployers must, among other things, assign human oversight to people with the necessary competence and authority.^[6] Following amendments agreed in 2026, those high-risk obligations are now due to apply from December 2027.

Complying with a set of rules isn't the same as having a culture of accountability, however. An employer should understand what a system is being used for, what its limitations are and how people can raise concerns. A supplier should be clear about what its product can reliably do. And someone needs the authority to investigate and fix problems when they arise. Otherwise, everyone ends up responsible for one part of the process while nobody feels responsible for the outcome, and the person affected shouldn't have to work out an organisation's internal division of responsibilities before their concern is taken seriously.

So far I've mainly discussed AI making recommendations. Increasingly, though, AI systems are being designed to take action, carrying out the next step rather than simply suggesting it. This is where my own work with HR systems makes me particularly cautious, because in an HR system a change to one record rarely stays in one place. An update to someone's job, hours or working pattern can flow through to pay, absence entitlements, pension contributions, reporting and notifications to other teams.

Consider an AI agent working with employee data that spots what looks like an inconsistency in someone's record. Instead of flagging it for review, it updates the record. That change triggers another workflow, a notification is sent and a different team acts on the updated information. Several steps later, the employee discovers that something important about their employment has changed, and that the agent's original interpretation was wrong. Nobody intended the outcome, and each step may have looked reasonable on its own, but by then the mistake has travelled through several systems.

That's a different kind of challenge from reviewing a generated report. When AI can act, organisations need to decide in advance what authority it should have. Can it update records, contact employees or customers directly, initiate payments, or approve and reject requests? What happens when one action triggers several others, and how does someone stop or reverse a process that's already under way?

An agent that organises internal meeting notes doesn't need the same controls as one that changes employee records or initiates payments, but the principle is the same in both cases. The system's authority should be deliberate, limited and understood, there should be a record of what it did, and there should be a practical way to intervene. Once AI moves from recommending decisions to carrying them out, accountability has to be designed in from the start rather than added afterwards.

There's another person in this story who deserves more attention: the employee who notices a problem. It might be a recruiter who suspects the screening system is excluding suitable candidates, a manager who believes an AI-generated performance assessment is unfair, or an administrator who spots a change that could have serious consequences for someone else. What happens when they speak up? Are they listened to, or told that the system has been tested and approved by people who know more about it? Does questioning it count as slowing down a process that's supposed to be faster, or as resistance to change?

An organisation can have impressive AI policies and still create a culture where nobody feels comfortable saying the system is wrong, and that's a problem technology can't solve. Employees need to know they can challenge an automated recommendation without being seen as difficult, managers need to be willing to investigate, and senior leaders need to recognise that an awkward question can prevent a much bigger problem later. The person who stops a faulty automated process may have contributed more than the person who approved a hundred recommendations without a second look. Sometimes the most responsible use of AI is knowing when not to follow it.