AI Agent Liability: Who Pays When an AI Agent Goes Wrong?

iTechTimes banner: Who pays when an AI agent goes wrong, showing DSCI six-step Prevent First, Halt Last safeguards

India’s Data Security Council has put out a practical guide on who carries the blame when autonomous AI agents cause harm, and it puts the burden on whoever holds the controls.

Key takeaways

  • The Data Security Council of India (DSCI), a not-for-profit body set up by Nasscom, has released “AI Liability: Industry Guidance for Agentic AI” as a live discussion paper.
  • It proposes no new law. It applies existing doctrines such as duty of care, product liability, causation and comparative negligence to AI agents.
  • Liability follows control: each party answers for the decisions it could make and the tools it had to enforce them.
  • It sets out a six-step “Prevent First, Halt Last” approach, with a kill switch only as the last resort.
  • It lands as TRAI consults on AI rules for telecom (comments due 27 November) and MeitY prepares an AI regulation consultation paper.

Who is liable when an AI agent causes harm in India? Under DSCI’s new framework, there is no single answer. Responsibility sits with whichever party controlled the decision that went wrong and could have prevented or limited the harm, whether that is the company deploying the agent, the integrator, the tool vendor, the cloud provider or the model maker.

What exactly has DSCI released?

DSCI published the paper at the end of October 2026 and opened it for industry consultation. It is called a “live” discussion paper because DSCI plans to update it as companies test it against real deployments.

The paper starts from a simple position: Indian law already covers most of what can go wrong with agentic AI. What is missing is engineering that can prove controls actually worked. DSCI CEO Vinayak Godse has described the aim as moving AI governance from principle to practice.

The legal ideas it leans on are familiar ones: causation, reasonable foreseeability, apparent authority, principal and agent law, duty of care, comparative negligence and product liability.

How does the framework decide who is responsible?

The core tool is an “authority map”. For any harmful action, it asks who controlled that decision and whether they could have stopped or limited the damage. The paper groups the players and assigns each a zone of responsibility.

PartyWhat it is responsible for
Deployers (end-user organisations)Choosing the use case, setting permissions, operational risk, deciding when human approval is needed
Product companies and startupsProduct claims, default settings, model versions
Service providers and integratorsSystem design, approval mechanisms, failure handling
Tool and framework providersOrchestration, memory, built-in safety features
Cloud and infrastructure providersIsolation, identity management, data retention, logs
Model providers (assessed separately)Model capabilities, safety measures, disclosing limits, notice of major behaviour changes

A model provider’s share of liability may shrink when others control how the model is deployed. On the other hand, vendors who withhold the controls customers need to enforce their own policies may face liability themselves.

What does “liability follows control” mean in practice?

DSCI describes three layers: governance intent (what the policy says), decision space (whether the system’s design even allows a given action) and instruments of execution (permission scopes, logs, approval screens and circuit breakers). Liability tends to land where this chain breaks.

The paper’s example is easy to grasp. A refund limit written only into an AI agent’s prompt is weaker than the same limit hard-coded into the payment tool the agent uses. If the agent overspends, the company that relied only on the prompt is in a weaker position.

It also warns that controls which exist on paper but are undocumented or switched off can be worse than having no control at all, because they show the risk was known and left open.

Hosting choices matter too. Running an open-weight model on your own infrastructure gives more control but shifts more auditing and monitoring duties onto you. Using a closed model through a managed API may allow recourse under contract, subject to the contract’s terms and liability caps.

What is the “Prevent First, Halt Last” approach?

DSCI recommends a graded set of safeguards, in this order:

  1. Do not give the agent risky capabilities it does not need.
  2. Restrict its permissions.
  3. Require human confirmation only for material, hard-to-reverse actions.
  4. Monitor for drift in behaviour.
  5. Scale back gracefully when something looks wrong.
  6. Use a scoped circuit breaker, or kill switch with clear ownership, as the last resort.

The reason the kill switch comes last is practical: stopping an agent in the middle of a task can leave a payment or process half done, which creates its own damage.

The paper also flags operational problems that make liability harder to settle, including changes in model behaviour, human approval fatigue, agents delegating work to other agents, and the difficulty of reproducing how an AI system reached a particular action. It lists 12 harm areas, among them privacy breaches, security threats, discrimination, inaccurate outputs, IP violations, deepfakes, loss of control and AI-enabled cybercrime.

How does this fit with India’s wider AI rules?

DSCI’s paper is industry guidance, not regulation. But it arrives during a busy stretch for AI policy in India.

InitiativeBodyStatus
AI Liability: Industry Guidance for Agentic AIDSCIDiscussion paper, open for feedback
Responsible AI in telecom networks and servicesTRAIDraft guidelines, comments due 27 November
DEPA for AI Training (consent-based data sharing)MeitYProposed in October
AI regulation consultation paperMeitYBeing prepared, no date announced

TRAI’s draft would require telecom operators to adopt board-approved AI policies, set up governance and safety committees and report serious AI incidents. Together, these signal that India is leaning towards sector-specific and practical rules rather than one sweeping AI law, at least for now.

What should Indian businesses using AI agents do?

If your company is piloting or running AI agents, for customer support, refunds, coding or internal workflows, the paper is a useful checklist even before any rule becomes binding.

  • Map authority. For each agent, write down who decides its use case, permissions and approval rules.
  • Hard-code limits in tools, not prompts. Spending caps, data access and irreversible actions should be enforced in the systems the agent calls.
  • Keep logs you can replay. If something goes wrong, you will need to show what the agent did and why.
  • Fix contracts now. DSCI advises adding liability clauses to procurement contracts with model, tool and integration vendors.
  • Assess before you deploy, and review regularly. Treat model updates as changes that need fresh testing.
  • Send feedback. DSCI has invited CIOs, CISOs, risk leaders, startups, legal bodies and think tanks to test the guide and report gaps.

FAQs

Is DSCI’s AI liability framework legally binding?

No. It is an industry discussion paper. It explains how existing Indian legal doctrines are likely to apply to AI agents, but it does not create new legal obligations.

What is agentic AI?

Agentic AI refers to AI systems that can take actions on their own, such as sending emails, processing refunds or running code, rather than only answering questions. Because they act, the question of who is responsible when they make mistakes becomes more pressing.

Does the model maker always carry the blame?

No. Model providers remain responsible for their models’ capabilities, safety measures and disclosures. But their share may shrink when the deploying company or integrator controls how the model is used.

Is there a deadline to comment on the DSCI paper?

No deadline has been stated. DSCI calls it a live document and plans further guidance on authority mapping, AI in physical and biological systems, and agents run by customers and partners.

Who is DSCI?

The Data Security Council of India is a not-for-profit industry body on data protection and cybersecurity set up by Nasscom. It often publishes guidance that feeds into government policy discussions.

What is the first step for a small startup using AI agents?

Start by listing every action your agent can take and removing any it does not need. Then put hard limits on money, data and irreversible actions inside your own systems, and keep logs.

Sources

Related on iTechTimes

Leave a Reply

Your email address will not be published. Required fields are marked *