Name: Rethinking how we evaluate security agents for real-world use
Start: 2026-09-22T13:15:00-0700
End: 2026-09-22T14:00:00-0700

Rethinking how we evaluate security agents for real-world use

Tuesday September 22, 2026 1:15pm - 2:00pm PDT

Security agents are gaining momentum across industry, but the way we evaluate them remains rooted in narrow, outcome-only benchmarks. These evaluations tell us whether an agent produced a correct answer, but not “how” it arrived there or whether that behavior will remain stable once deployed.

In practice, enterprise security is not a sequence of isolated tasks. It is a connected, end-to-end workflow that follows a find → confirm exploit → patch → validate loop. Agents that perform well on task-specific benchmarks often fail in these multi-stage settings due to contextual loss and brittle transitions across steps.

This talk introduces a practical framework for evaluating security agents by mapping agentic capabilities (planning, reasoning, memory, perception, tool use) to security functions (reconnaissance, exploit confirmation, root-cause analysis, patching, validation) across the full lifecycle. We also share insights from our large-scale survey of existing agentic systems, highlighting which capabilities consistently drive success at each stage. Finally, we present a lightweight, unified end-to-end scoring perspective that teams can use to assess an agent’s readiness for real operational environments.

Speakers

Mudita Khurana

Staff Security Engineer, Airbnb

Mudita Khurana is a Tech Lead at Airbnb, where she builds scalable security tooling and automation across the software development lifecycle. Previously at Meta, she drove key initiatives in product security, including bug bounty strategy, privacy-focused reviews, and automated vulnerability... Read More →

Tuesday September 22, 2026 1:15pm - 2:00pm PDT

Testing

Audience Advanced

OWASP 25th Anniversary Virtual Conference (September)

Mudita Khurana

Attendees (12)

Get help with the event

OWASP 25th Anniversary Virtual Conference (September)

Mudita Khurana

Attendees (12)

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Get help with the event