More about q52 security
Overview
Automation in the SOC is too often pursued for its own sake: faster closures, fewer touches, and a dashboard that rewards volume over judgment. This design starts from the opposite premise. The goal isn’t to remove analysts from the loop. It’s to protect their attention for the work that deserves it. The mundane is machine work: sorting telemetry from signal, confirming that an audit event was expected, gathering the same enrichment for the thousandth time, and drafting the summary nobody enjoys writing. Each of those is delegated to a narrow, single-purpose agent that must show its evidence and say what it doesn’t know. The consequential work stays with people: weighing ambiguity, deciding whether to isolate a host that a clinic or a factory floor depends on, and choosing when to pick up the phone. The agents advise, but they never contain, and every path where confidence is low, data is missing, or the stakes are high ends at a human with the full case already assembled. When the system works as intended, the measure of success isn’t how many cases the machine closed. It’s how much more time a person has to think carefully about the few that
matter. At q52.ai, automation is there to give analyst judgment more room.
Solution Brief
Platform: Google Security Operations (SIEM + SOAR) with Gemini
An orchestrator playbook runs five single-purpose agents in sequence, routing between them with deterministic branches. Each agent is a nested playbook block that
(1) assembles context from case data,
(2) calls Gemini with a tightly specified prompt,
(3) sanitizes and regex-parses the response,
(4) validates it with fail-safe defaults, and
(5) writes typed values to namespaced case custom fields.
3.1 Triage Agent
| Question answered | What kind of item is this? |
| Inputs | Case title, description, environment name, original alert JSON |
| Output fields | classification (alert, signal, telemetry, audit), confidence (0-1), severity_recommendation (Information, Low, Medium, High, Critical), recommended_path (investigate, enrich_only, observe, close_candidate), evidence_fields_evaluated, reasoning_summary |
| Case fields written | triage_classification, triage_confidence, triage_severity, triage_route, triage_requires_human_review, triage_evidence_fields_evaluated, triage_version |
| Prompt hard rules | (1) If the security product reports BLOCK/DROP/DENY, classify as signal or close-candidate, never alert. (2) Known authorized-scanner callback domains in the payload mean signal or close-candidate. (3) Only after (1) and (2), evaluate signature, severity label and volume. (4) Vulnerability or exposure findings without exploitation evidence are signals. (5) Do not claim exploitation unless the alert states it. |
| Playbook validation | Empty classification: default to signal, confidence 0, human review true, route to investigator. Confidence above 1 clamps to 1; below 0 clamps to 0. Confidence below 0.50 sets human review. |
| Playbook actions | Sets case stage, tags environment type, maps advisory severity to case and alert priority, writes a completion comment. |
3.2 Investigator Agent (two-pass, evidence-seeking)
| Question answered | What happened, who or what is involved, and how sure are we? |
| Pass 1 prompt | “Context engine”: frame a hypothesis, list existing evidence and gaps, choose enrichment categories. Forbidden: attacker intent without evidence, containment advice, final verdict. |
| Pass 1 output | investigation_summary, confidence_in_story, existing_evidence, evidence_gaps, recommended_enrichment (identity, asset, network, authentication, vulnerability, threat_intel, timeline, none), investigation_risk_level |
| Confidence banding | Strong at 0.70 or above, weak at or below 0.50, normal otherwise; stored as case context. |
| Tool discovery | Reads the tenant’s integration inventory and only calls products that exist for that environment (EDR, directory/IdP, firewall analyzer, vulnerability scanner, threat intel). Environments without tool coverage get an explicit “no tool enrichment” comment. |
| Enrichment branches | Asset and identity; network and domain; timeline (similar alerts plus similar cases by rule and by entity); threat intel (IOC reputation and sandbox); vulnerability (scanner data plus a Gemini-generated UDM query against ingested scan logs). Each runs only if Pass 1 requested it. |
| Narrators | One Gemini call per domain. Each answers 5-6 fixed questions in one paragraph and is told it is not determining a verdict. Output is sanitized and stored in an enrichment_* field. |
| Pass 2 prompt | “Reasoning engine”: reassess using only the supplied evidence; missing data means unknown, not safe. |
| Pass 2 output | investigation_disposition (true_positive, false_positive, benign, inconclusive), investigation_confidence, investigation_evidence_summary, investigation_behavioral_signal_assessment, investigation_asset_identity_risk_context, investigation_enrichment_consensus (agree, conflict, insufficient), investigation_key_uncertainties, investigation_recommended_action (no_action, monitor_only, containment_recommended, containment_required, escalate_for_human_review) |
| Failure handling | If Pass 2 returns a blank action, write “Summary Data Blank” and let downstream placeholder logic force escalation. |
3.3 Response Advisor Agent
| Question answered | What should the customer or operator do next? |
| Prompt A: capability assessment | From the integration inventory only, determine capability_endpoint_isolation, capability_network_blocking, capability_identity_control, capability_email_control, plus capability_confidence and notes. Shared case-management integrations do not count. Ambiguous means unavailable. |
| Prompt B: recommendation | Inputs: alert, capabilities, triage fields, investigation fields. |
| Output | response_decision (escalate, contain_recommended, contain_required, monitor_only, no_action), response_containment_eligible, response_approval_required, response_containment_targets (type:value list or none), response_immediate_next_steps, response_recommendation_summary, response_rationale, response_risk_notes, response_version |
| Guardrails (in prompt) | Containment only if disposition is true_positive AND confidence is at least 0.7 AND behavior supports it. Targets must be explicitly named in the evidence, otherwise “none”. Weak confidence or material uncertainty: escalate or monitor-only, approval required. If containment could cause significant service impact, never “required,” at most “recommended.” Do not contradict the investigation without new evidence. |
| Re-entry | A stage check detects duplicate assessment runs and closes the duplicate alert. |
3.4 Disposition Validator (control gate)
| Question answered | Given everything, what is the case’s final handling? |
| Mostly deterministic | Verifies triage, investigation and response fields all exist. If not: abort, comment, escalate. |
| Decision rules | Escalate when response_decision = escalate AND human review is flagged AND investigation confidence meets threshold AND the recommended action says escalate. Contain branches key off contain_required and contain_recommended. Monitor-only keys off low confidence with false_positive or benign. No match: escalate for human review. |
| Priority table | Fixed mapping of (disposition x triage severity x classification) to case and alert priority. |
| Gemini’s only job | Closure note (3-4 sentences) and closure_reason from a fixed enum (Human error, Legit action, Misconfigured system, Rule under construction) for clean reporting. |
| Containment outcome | Posts “no response actions have been taken,” raises an incident, assigns to a human. |
3.5 Notification Dispatcher
| Question answered | Who needs to know, how, and what do we tell them? |
| Summary prompt | Outputs case_summary, text_format (character-limited SMS), phone_format (word-limited TTS script), method (text, voice, email, ticket, messenger). Rules bias method selection by detection source and environment type. |
| Team picker prompt | Plain-language role definitions in the prompt; model returns one or more workgroup labels from a closed list based on the entities and actions involved. |
| SBAR email prompt | Situation, Background, Assessment, Recommendation; includes containment advice, unknowns and next steps. |
| Gemini case summary | Native “Create Gemini Case Summary” action attached to the case record. |
| Human gate | Analyst prompt: create ticket, close case, or stop automation. SLA clock pauses once customer communication is raised. |
| Known gap | Voice is recommended but handled manually; automated TTS is not implemented. |
3.6 Non-incident handlers
- Audit handler: confirm the activity was expected or authorized; if so, close as legitimate and move the case to an improvement stage; otherwise send it on to response assessment.
- Telemetry handler: record why the item is telemetry, post to a baseline log channel, lower priority, close. Also receives investigator “benign” outcomes.
Case state contract
triage_* | Triage | Investigator, Response Advisor, Disposition, Notifier |
enrichment_* | Investigator narrators | Investigator Pass 2 |
investigation_* | Investigator Pass 2 | Response Advisor, Disposition, Notifier |
capability_* | Response Advisor (assessment) | Response Advisor (recommendation) |
response_* | Response Advisor | Disposition, Notifier |
isEscalated, EscalationMethod | Notifier | SLA and assignment logic |
Standalone-run safety: every agent that reads upstream fields first checks for them and, if absent, writes labeled placeholders (for example triage unknown, investigation inconclusive at 0.30, response DEFER) so it can run ad hoc without the full chain.
5. Routing matrix
| Not the first alert on the case | Close as duplicate of the case under investigation |
| Triage = audit | Audit handler; if expected, close; else Response Advisor |
| Triage = telemetry | Telemetry handler |
| Triage = alert or signal | Investigator |
| Investigation = true_positive | Response Advisor |
| Investigation = benign | Telemetry handler |
| Investigation = false_positive | Disposition Validator |
| Investigation = inconclusive | Disposition Validator (human review) |
| Response = monitor_only or no_action | Close as not malicious with AI-written note |
| Response = escalate, contain_recommended, contain_required | Disposition Validator, then Notification Dispatcher |
| Disposition = escalate | Notification Dispatcher, human assignment |
| Disposition = contain | Notification Dispatcher, containment NOT executed |
| Required agent output missing | Abort, escalate to human |
Evolution of the design
The playbook inventory shows these distinct generations. The ordering is inferred from their complexity; confirm the dates against your own history before publishing a timeline.
| 0 | Alert explainer: plain-language description of an alert for Tier 1/Tier 2 | No decision support |
| 1 | Quick verdict: two 1-100 scores (malicious, misconfiguration), regex-extracted, threshold branches; manual SBAR and auto-close variants | Scores carry no evidence; one prompt did everything |
| 2 | Gemini-generated UDM query, executed in the SIEM, results analyzed by a second call | Useful pattern, but ad hoc and single-purpose |
| 3 | Five-agent chain with typed contracts, conditional enrichment, hard guardrails, deterministic control gate | Current design |
Prompt-engineering patterns
- Role line + non-goals. “You are a SOC [role]. You are NOT determining a verdict / executing containment.”
- Output contract. Exact labels, closed value sets, one value per line, no markdown/JSON/code blocks.
- Evidence discipline. “Use only the data provided. Missing data is unknown, not safe. Placeholders in brackets are missing.”
- Ordered evaluation. Force checks in sequence before any judgment (action outcome, then scanner infrastructure, then signature).
- Absolute disqualifiers. State which conditions cannot be overridden by severity labels or signature names.
- Definitions inline. Define every enum value in the prompt.
- Narrators vs deciders. Narrators answer fixed questions; only one prompt may produce a verdict.
- Capability grounding. Pass in what the tenant can actually do and require ignoring everything else.
- Similar history as awareness only. Never as equivalence.
- Closure-reason enums. Free text for humans, enums for reporting.
SOAR plumbing patterns
- HTML-wrapped responses: sanitize, strip paragraph tags, regex each labeled line into its own variable.
- Parallel extraction containers for the 5-10 fields per agent.
- Fail-safe branch after every parse: empty classification, blank action, out-of-range confidence.
- Case stage as state machine: Triage, Investigation, Assessment/Research, Incident/Escalation, Improvement.
- First-alert gate at the orchestrator and a re-entry gate in the advisor.
- Per-agent trigger playbooks to run, test and retire agents independently of the master flow.
- Notification log channels separated by class (incident, signals and leads, audit and telemetry).
Risks and recommendations
- LLM confidence is uncalibrated; the thresholds (0.50 human review, 0.70 containment) are routing heuristics. Calibrate against analyst outcomes per detection source.
- Several guardrails are encoded in prompts only. Mirror the critical ones (blocked events, containment eligibility) as deterministic playbook checks.
- Keep a regression set of past cases and re-run it after each prompt change.
- Track: analyst override rate, auto-close precision, false-escalation rate, median time to triage. [INSERT METRIC]
- Voice notification and human-approved containment execution are the main open items.
Mermaid Diagram
%% q52.ai - Agentic SOC - generalized playbook flow
%% Paste into any Mermaid renderer (mermaid.live, GitHub, Notion, Confluence, WordPress plugin).
%% Legend: rounded = agent (LLM reasoning), rectangle = deterministic playbook logic,
%% diamond = deterministic router, cylinder = shared case state, hexagon = human gate.
flowchart TD
classDef agent fill:#e8f0fe,stroke:#1a73e8,stroke-width:2px,color:#0b2a5b
classDef det fill:#f1f3f4,stroke:#5f6368,color:#202124
classDef router fill:#fef7e0,stroke:#f9ab00,color:#4a3500
classDef human fill:#fce8e6,stroke:#d93025,stroke-width:2px,color:#5c0a05
classDef state fill:#e6f4ea,stroke:#188038,color:#0d3b1a
classDef terminal fill:#ffffff,stroke:#202124,stroke-dasharray:4 3,color:#202124
START(["Detection fires<br/>alert ingested into SOAR case"]):::terminal
DEDUP{"First alert<br/>for this case?"}:::router
DUP(["Close as duplicate<br/>already under investigation"]):::terminal
subgraph STATE["Shared case state - typed custom fields are the agent contract"]
direction LR
S1[("triage_*")]:::state
S2[("investigation_*")]:::state
S3[("response_*")]:::state
S4[("capability_*")]:::state
end
subgraph TRIAGE["Stage 1 - Triage Agent"]
T1("LLM: classify<br/>alert / signal / telemetry / audit<br/>+ confidence + severity + path"):::agent
T2["Parse output<br/>sanitize + regex-extract fields"]:::det
T3{"Output valid?<br/>confidence 0-1?"}:::router
T4["Fail-safe defaults<br/>class=signal, conf=0<br/>human_review=true"]:::det
T5{"Confidence<br/>below 0.50?"}:::router
T6["Flag human review"]:::det
T7["Map fields to case state<br/>set advisory severity"]:::det
T1 --> T2 --> T3
T3 -- "no" --> T4 --> T7
T3 -- "yes" --> T5
T5 -- "yes" --> T6 --> T7
T5 -- "no" --> T7
end
ROUTE{"Classification<br/>router"}:::router
subgraph NONSEC["Non-incident handlers"]
AUD("Audit handler<br/>is this expected / authorized?"):::agent
AUDQ{"Expected?"}:::router
TEL["Telemetry handler<br/>retain for baseline<br/>log + close"]:::det
end
subgraph INVEST["Stage 2 - Investigator Agent, two-pass"]
I1("Pass 1: LLM frames hypothesis<br/>evidence gaps + recommended enrichments"):::agent
I2{"Pass-1 confidence<br/>strong / normal / weak"}:::router
I3{"Any enrichment<br/>recommended?"}:::router
I0["Discover available integrations<br/>per-tenant inventory"]:::det
subgraph FAN["Conditional enrichment fan-out - only what Pass 1 asked for"]
direction LR
E1["Asset + identity<br/>EDR / directory / IdP"]:::det
E2["Network + domain<br/>WHOIS / reputation / SIEM context"]:::det
E3["Timeline<br/>similar alerts + similar cases"]:::det
E4["Threat intel<br/>IOC reputation"]:::det
E5["Vulnerability<br/>scanner data + generated query"]:::det
end
N1("Per-domain narrator LLMs<br/>answer fixed questions<br/>never issue a verdict"):::agent
I4("Pass 2: reasoning LLM<br/>typed verdict from evidence only<br/>disposition + confidence + uncertainties"):::agent
I5["Validate + write investigation_*"]:::det
I1 --> I2 --> I3
I3 -- "none" --> I4
I3 -- "some" --> I0 --> FAN --> N1 --> I4
I4 --> I5
end
VERD{"Investigation<br/>disposition"}:::router
subgraph RESP["Stage 3 - Response Advisor Agent, advisory only"]
R1("LLM: capability assessment<br/>from integration inventory<br/>isolate / block / identity / email"):::agent
R2("LLM: response recommendation<br/>under hard guardrails"):::agent
R3{"Decision"}:::router
R1 --> R2 --> R3
end
subgraph DISP["Stage 4 - Disposition Validator, final control gate"]
D1{"All upstream outputs<br/>present?"}:::router
D2["Deterministic rules<br/>triage x investigation x response"]:::det
D3{"Outcome"}:::router
D4("LLM: closure note<br/>+ reason from fixed enum"):::agent
D5["Auto-close case"]:::det
D6["Escalate to human analyst<br/>raise priority"]:::det
D7["Containment recommended / required<br/>NOT EXECUTED - human approves"]:::det
D8["Abort and escalate<br/>missing agent output"]:::det
D1 -- "no" --> D8
D1 -- "yes" --> D2 --> D3
D3 -- "monitor / false positive / benign" --> D4 --> D5
D3 -- "escalate / unknown" --> D6
D3 -- "contain" --> D7
end
subgraph NOTIFY["Stage 5 - Notification Dispatcher"]
N2("LLM: summaries<br/>email SBAR, SMS, voice script"):::agent
N3("LLM: pick responsible team(s)<br/>from role definitions"):::agent
N4("LLM: pick channel(s)<br/>text / voice / email / ticket / chat"):::agent
N5{{"Analyst confirms<br/>ticket / close / stop automation"}}:::human
N6["Post to notification log<br/>pause SLA clock"]:::det
N7["Create ticket<br/>notify customer / team"]:::det
N2 --> N3 --> N4 --> N5
N5 -- "ticket" --> N7
N5 -- "close" --> D5
N5 -- "stop" --> N6
N7 --> N6
end
HUMAN(["Human analyst owns the case"]):::human
CLOSED(["Case closed with audit trail"]):::terminal
START --> DEDUP
DEDUP -- "no" --> DUP
DEDUP -- "yes" --> T1
T7 --> ROUTE
ROUTE -- "audit" --> AUD --> AUDQ
AUDQ -- "yes" --> D5
AUDQ -- "no" --> R1
ROUTE -- "telemetry" --> TEL
ROUTE -- "alert / signal" --> I1
I5 --> VERD
VERD -- "true_positive" --> R1
VERD -- "benign" --> TEL
VERD -- "false_positive / inconclusive" --> D1
R3 -- "monitor_only / no_action" --> D5
R3 -- "escalate / contain_*" --> D1
D6 --> N2
D7 --> N2
D8 --> N2
N6 --> HUMAN
D5 --> CLOSED
T7 -.-> S1
I5 -.-> S2
R3 -.-> S3
R1 -.-> S4
S1 -.-> I1
S1 -.-> R2
S2 -.-> R2
S2 -.-> D2
S3 -.-> D2
S4 -.-> R2