Agent Platform Liability Scorecard
Two questions that expose whether your AI platform is a strategy or an unpriced liability.
Most enterprise AI platform evaluations focus on model quality, feature breadth, and price. For agentic deployments, critical failure modes lie elsewhere: Can the platform distinguish a human user from an AI agent? Can it bound the agent's scope to the task at hand? Can it produce an auditable action trail? Can someone revoke access in real time? And what does the platform do by default when a team is under deadline pressure? This two-question scorecard gives any decision-maker a concrete pass, conditional, or block verdict to run before signing a contract or extending an existing platform to agentic use cases, without requiring deep technical expertise to administer.
- Agents and humans must be authenticated separately, or the blast radius of any incident is unbounded.
- An audit trail that cannot answer a regulator's question is not an audit trail.
- The real security posture is what the platform does by default, not what it can do when fully configured.
- If you cannot revoke an agent's access in five minutes, your incident response has a structural gap.
- Organizational defaults under pressure reveal true security posture better than any documentation does.
- Pose Question One: Does the Platform Separate Human and Agent Authentication?Ask your vendor or internal team directly whether the system has a distinct identity concept for AI agents, separate from human user accounts. This is not about whether authentication exists — it is about whether it applies specifically to agents as a first-class identity.Pro tipRequest a live demo where an agent attempts to access a resource outside its task scope. If the platform has no concept of agent scope, the demo will reveal it immediately.WarningA vendor answering 'we support OAuth' has not answered this question. Press for agent-specific identity enforcement.
- Assess Blast Radius: Verify Agent Scope Is Bounded to the TaskDetermine whether the platform enforces that an agent running on one client or workflow context can only touch data for that context. If a single agent run can traverse all accounts, all workflows, and all data classes, one mistake or exploit becomes a company-wide exposure event.WarningIf the answer is 'that is configurable,' ask what the default is, who configures it, and when — configuration that requires active effort will be skipped under pressure.
- Verify the Audit Trail Answers the Regulatory QuestionThe audit trail must answer for every agent action: what did the system do, on behalf of which user, against which data, at what time, and was it authorized. Pull a sample agent run log and try to reconstruct the full action sequence. If you cannot, a regulator cannot either.Pro tipRun this test before the contract review, not after. A gap found during vendor evaluation is a negotiating point. A gap found during a regulatory inquiry is a liability.
- Confirm the Kill Switch: 5-Minute Agent Access RevocationAsk specifically: if an agent is misbehaving right now, how do you stop it? The acceptable answer is a named console action that revokes the agent's access within five minutes, requiring no code deploy, no ticket, and no vendor call. Any other answer is a gap in your incident response plan.Pro tipRun a tabletop exercise simulating an agent behaving unexpectedly and time how long actual revocation takes with your current tooling.WarningDiscovering this gap in a tabletop is recoverable. Discovering it at 3am during a real incident is not.
- Pose Question Two: What Are the Platform's Defaults Under Deadline Pressure?Ask the vendor what a fresh deployment looks like before any security configuration is applied. What authentication state are endpoints in by default? Can an agent write to production without explicit configuration? These defaults represent what every team moving fast will actually ship.Pro tipRequest a sandbox environment and inspect the default configuration before any hardening is applied — not the reference architecture, the raw default.WarningComprehensive security options and safe defaults are not the same thing. Most enterprise failures occur when teams skip configuration under deadline pressure.
- Score and Decide: Pass, Conditional, or BlockIf both questions are answered satisfactorily — separate agent auth, bounded scope, full audit trail, kill switch, and safe defaults — proceed. If one question has a named gap with an owner and deadline before deployment, treat it as conditional. If both questions have unresolved gaps, block deployment until they are closed.Pro tipGet the vendor's answers in writing as part of the contract. A verbal answer during a sales call is not an enforceable commitment.
An autonomous agent with no credentials spent $20 to access McKenzie's Lily platform via SQL injection through unauthenticated API endpoints. The platform had no concept of agent identity distinct from human users. The agent's scope was unbounded — it reached tens of thousands of accounts and all system prompts with write access. No kill switch existed because there was no agent identity to revoke. Run through the two-question scorecard, Lily would have failed every sub-criterion on Question One and revealed absent safe defaults on Question Two.
A legal services firm extends its document-review agent to handle contract renewals, requiring cross-system access to CRM, billing, and client history. No one asks whether the agent authenticates separately from human counsel or whether scope is bounded by matter. Six months later, the agent running on one client matter surfaces confidential data from a different matter. The audit trail shows only a generic service account with no matter-level scoping.
Extracted from AI News & Strategy Daily | Nate B Jones, derived from post-mortem analysis of the Codewall/Lily security incident and enterprise AI governance patterns observed in 2025-2026.