Policy Engine

Governance-as-code: a declarative policy DSL that bridges alignment cards to concrete tool enforcement

Policy Engine

The Policy Engine is Phase 1 of CLPI (Card Lifecycle & Policy Intelligence) — the governance layer that transforms alignment cards into lifecycle-managed artifacts with policy enforcement, trust recovery, risk intelligence, and on-chain anchoring.

The policy engine translates Alignment Card declarations into enforceable rules over concrete tools. An Alignment Card says an agent may perform web_fetch. The policy says that web_fetch means the agent can call mcp__browser__navigate and mcp__browser__click, but not mcp__filesystem__delete. The card declares intent. The policy enforces it.

Policies are defined in a YAML-based DSL with five top-level sections: capability_mappings, forbidden, defaults, and optionally escalation_triggers and meta. They are evaluated in three contexts — CI/CD pipelines, live gateway requests, and post-action observation — giving operators governance coverage across the full agent lifecycle.

The policy engine is a parallel enforcement layer to alignment enforcement (observe/nudge/enforce). Alignment enforcement checks agent behavior against card values. Policy enforcement checks tool usage against policy rules. Both run independently and produce separate verdicts.

Policy DSL Overview

A policy file is a YAML document that declares how abstract card capabilities map to concrete tools, which tools are always forbidden, and what happens when a tool falls outside any mapping.

meta:
  name: "research-agent-policy"
  version: "1.2.0"
  description: "Policy for research agents with web and filesystem access"

capability_mappings:
  web_browsing:
    tools:
      - "mcp__browser__*"
    card_actions:
      - "web_fetch"
      - "web_search"

  file_reading:
    tools:
      - "mcp__filesystem__read*"
      - "mcp__filesystem__list*"
    card_actions:
      - "read_file"

forbidden:
  - pattern: "mcp__filesystem__delete*"
    reason: "File deletion not permitted"
    severity: "critical"
  - pattern: "mcp__shell__*"
    reason: "Shell execution not permitted"
    severity: "high"

defaults:
  unmapped_tool_action: "warn"
  unmapped_severity: "medium"
  grace_period_hours: 24

escalation_triggers:
  - condition: "tool_count > 50"
    action: "escalate"
    reason: "Excessive tool usage in single session"

The three required sections serve distinct purposes:

Section	Purpose
capability_mappings	Bridge card-level categories to concrete tool patterns
forbidden	Tools that are always blocked, regardless of mappings
defaults	Fallback behavior for tools that match no mapping

The two optional sections extend the policy:

Section	Purpose
escalation_triggers	Conditions that trigger escalation regardless of individual tool verdicts
meta	Human-readable metadata: name, version, description

Three Evaluation Contexts

Policies are not evaluated once. They are evaluated at three stages, each with different inputs and different consequences.

CI/CD (Static)

CI/CD Evaluation

Static evaluation runs in pipelines before deployment. It validates policy syntax and evaluates policy rules against historical trace data.

Commands:

# Validate policy schema (syntax + structure)
smoltbot policy validate --file policy.yaml

# Evaluate policy against historical traces
smoltbot policy evaluate --file policy.yaml --agent-id agent_abc123 --days 30

What it checks:

• Policy YAML conforms to the DSL schema
• All glob patterns are syntactically valid
• Capability mappings reference card actions that exist in the agent’s alignment card
• Historical traces would pass/fail under the proposed policy
• Coverage report identifies unmapped card actions

Use case: Pre-deploy gates. A CI job runs smoltbot policy validate and smoltbot policy evaluate before merging policy changes. If evaluation shows violations in recent traces, the merge is blocked.

Gateway (Live)

Gateway Evaluation

Live evaluation runs in real-time as requests pass through the smoltbot gateway. The policy engine extracts tool names from the request body, checks each tool against the policy, and returns a verdict.

How it works:

1. Request arrives at the gateway
2. Policy engine extracts tool names from the request body
3. Each tool is checked against forbidden rules first
4. Remaining tools are matched against capability_mappings
5. Unmatched tools fall through to defaults.unmapped_tool_action
6. A verdict is returned via the X-Policy-Verdict response header

Policy Identity Resolution

The gateway resolves an agent’s policy through a 3-tier identity resolution chain. Each tier is attempted in order; the first match wins.

Tier	Method	Description
1	linked_agent_id lookup	Checks if the request’s agent ID has a permanent cross-reference to another agent with a published policy. The linked_agent_id column (migration 066) stores these mappings.
2	Direct get_policy_for_agent()	Looks up a policy by the agent’s own ID. Works for agents with directly assigned policies.
3	Name-based fallback	Extracts the agent name from the x-smoltbot-agent request header and searches for a policy by name. If found, automatically creates a linked_agent_id record so future requests resolve via Tier 1.

Fail-open: All resolution failures — database errors, missing headers, no matching policy — result in the request proceeding without policy enforcement. The policy engine never blocks a request due to a lookup failure.

The observer uses the same 3-tier resolution chain as the gateway, ensuring consistent policy identity across real-time (gateway) and post-action (observer) evaluation contexts.

Verdict headers:

Header Value	Meaning
X-Policy-Verdict: pass	All tools passed policy checks
X-Policy-Verdict: warn	One or more tools triggered warnings (logged, not blocked)
X-Policy-Verdict: fail	One or more tools violated policy (blocked in enforce mode)

Enforcement interaction: The policy verdict determines the gateway response based on the policy enforcement mode (separate from the alignment enforcement mode):

Policy Mode	pass	warn	fail
off	No evaluation	No evaluation	No evaluation
warn	Allow	Allow + log	Allow + log
enforce	Allow	Allow + log	HTTP 403 block

Observer (Post)

Observer Evaluation

Post-action evaluation runs after an action completes. It analyzes what actually happened against what the policy expected.

What it detects:

• card_gap violations: Tools that the agent used successfully but that are not represented in any capability mapping. These are tools that should be in the card but are not.
• Frequency anomalies: Tools used at rates that exceed expected patterns
• New tool discovery: Tools appearing for the first time, subject to grace period handling

Feeds into reclassification: Observer findings feed into the AIP reclassification pipeline. A card_gap detection triggers a proposed card amendment — suggesting that the alignment card should be updated to include the newly observed capability.

Capability Mapping

Capability mappings are the core of the policy DSL. They bridge the gap between what an alignment card declares (abstract semantic actions like web_fetch) and what agents actually invoke (concrete tool names like mcp__browser__navigate).

Structure

Each capability mapping has a name, a list of tool glob patterns, and a list of card actions it satisfies:

capability_mappings:
  web_browsing:
    tools:
      - "mcp__browser__navigate"
      - "mcp__browser__click"
      - "mcp__browser__screenshot"
    card_actions:
      - "web_fetch"
      - "web_search"

  database_read:
    tools:
      - "mcp__postgres__query"
      - "mcp__postgres__list_tables"
    card_actions:
      - "read_data"

Glob Patterns

Tool patterns support standard glob syntax for flexible matching:

Pattern	Matches
mcp__browser__*	All browser tools (navigate, click, screenshot, etc.)
mcp__filesystem__read*	read_file, read_directory, read_metadata
mcp__*__list*	Any MCP server’s list operations
custom_tool_v?	custom_tool_v1, custom_tool_v2, etc.

Start with broad globs during initial policy development, then tighten them as you understand which specific tools your agent uses. A mapping like mcp__browser__* is fine for week one. By month two, you should enumerate the specific tools.

How Matching Works

When the policy engine evaluates a tool, it follows this order:

1. Forbidden check: Does the tool match any forbidden pattern? If yes, the tool is a violation regardless of capability mappings.
2. Capability match: Does the tool match any capability_mappings tool pattern? If yes, the tool is allowed and mapped to the corresponding card actions.
3. Default fallback: If neither forbidden nor mapped, apply defaults.unmapped_tool_action.

A tool can match multiple capability mappings. This is not an error — it means the tool satisfies multiple card actions.

Enforcement Modes

The policy engine has its own enforcement mode, independent of the alignment enforcement mode (observe/nudge/enforce).

Warn

Log violations but do not block. X-Policy-Verdict: warn header returned. This is the default mode.

Enforce

Block requests with policy violations. X-Policy-Verdict: fail header returned. HTTP 403 for non-streaming requests.

Off

Skip policy evaluation entirely. No X-Policy-Verdict header. No performance overhead.

Policy enforcement and alignment enforcement are parallel systems. Setting alignment enforcement to enforce does not affect policy enforcement, and vice versa. An agent can be in alignment observe mode and policy enforce mode simultaneously — meaning alignment violations are logged but policy violations are blocked.

Relationship to Alignment Enforcement

System	What It Checks	Verdict Source
Alignment enforcement (observe/nudge/enforce)	Agent behavior against card values and autonomy envelope	AIP integrity checkpoints, AAP verification
Policy enforcement (warn/enforce/off)	Tool usage against policy rules and capability mappings	Policy engine evaluation

Both systems produce independent verdicts. Both are surfaced in the conscience timeline and observability exports.

Forbidden Actions

Forbidden rules define tools that must never be used, regardless of capability mappings. They are always checked first in the evaluation pipeline.

forbidden:
  - pattern: "mcp__filesystem__delete*"
    reason: "File deletion not permitted"
    severity: "critical"
  - pattern: "mcp__shell__*"
    reason: "Shell execution not permitted"
    severity: "high"
  - pattern: "mcp__*__drop_table"
    reason: "Table deletion not permitted"
    severity: "critical"
  - pattern: "mcp__email__send_bulk*"
    reason: "Bulk email sending restricted"
    severity: "medium"

Each rule has three fields:

Field	Type	Description
pattern	string (glob)	Tool name pattern to match
reason	string	Human-readable explanation for auditing
severity	enum	critical, high, medium, or low

Policy forbidden rules complement alignment card forbidden_actions. Card forbidden actions declare intent (“this agent must never delete files”). Policy forbidden rules enforce that intent at the tool level (“block all tools matching mcp__filesystem__delete*”). Both are checked — card-level by alignment enforcement, tool-level by policy enforcement.

Unmapped Tool Handling

When a tool does not match any capability mapping or forbidden rule, the defaults section determines what happens.

defaults:
  unmapped_tool_action: "warn"
  unmapped_severity: "medium"
  grace_period_hours: 24

Unmapped Tool Actions

Action	Behavior
deny	Treat the unmapped tool as a violation. Blocked in enforce mode.
warn	Log a warning but allow the tool invocation. Default.
allow	Silently permit the tool. No log entry.

Choosing the Right Default

Development

Use allow during early development when tool sets are still evolving. This avoids noise from constantly changing tool inventories.

Staging

Use warn in staging environments. This surfaces unmapped tools in logs and coverage reports without blocking agent functionality. This is the default.

Production

Use deny in production when you have a mature, stable policy. Every tool the agent uses should be explicitly mapped or explicitly forbidden.

Grace Period

New tools appear when agents gain new MCP server connections or when tool providers add capabilities. The grace period prevents these newly discovered tools from immediately becoming violations.

defaults:
  grace_period_hours: 24

How it works:

1. The policy engine tracks when each tool is first seen via tool_first_seen records.
2. When an unmapped tool is encountered, the engine checks how long ago it was first seen.
3. If the tool was first seen within the grace period window, it is treated as allow regardless of unmapped_tool_action.
4. After the grace period expires, the tool falls back to the configured unmapped_tool_action.

This gives operators time to update their policy after adding new tools or MCP servers, without immediately triggering violations in enforce mode.

The default grace period is 24 hours. Set it to 0 to disable grace periods entirely (strict mode). Set it higher (48-72 hours) for teams that deploy on weekly cycles and need more time to update policies.

Policy Merge

In organizations with multiple agents, policies exist at two levels: org-level and agent-level. These are merged at evaluation time using rules that prevent agents from weakening organizational policy.

Merge Rules

Section	Merge Strategy	Effect
capability_mappings	Union	Agent can add new mappings but cannot remove org-level mappings
forbidden	Union	Both org and agent forbidden rules are enforced
defaults	Org is floor	Agent can strengthen (allow to warn to deny) but cannot weaken
escalation_triggers	Union	Both org and agent triggers are evaluated
meta	Agent overrides	Agent metadata replaces org metadata

Strengthening Defaults

The org default acts as a floor. An agent can only move in the stricter direction:

allow → warn → deny
 ↑                ↑
 weaker          stricter

If the org sets unmapped_tool_action: "warn", an agent can set it to deny (stricter) but not allow (weaker).

Transaction Guardrails

Transaction-scoped policies can further restrict the merged policy via intersection semantics. A transaction guardrail can only narrow what is permitted — never expand it.

# Transaction-level override: restrict to read-only tools for this operation
transaction_guardrails:
  allowed_capabilities:
    - "file_reading"
    - "database_read"
  # All other capabilities are denied for this transaction

Coverage Report

Every policy evaluation produces a coverage report that quantifies how well the policy maps to the alignment card. This helps identify gaps between what the card declares and what the policy actually covers.

Coverage Metrics

Metric	Description
total_card_actions	Number of actions declared in the alignment card’s bounded_actions
mapped_card_actions	Number of card actions covered by at least one capability mapping
unmapped_card_actions	Card actions with no corresponding capability mapping
coverage_pct	mapped_card_actions / total_card_actions * 100

Example Output

{
  "coverage": {
    "total_card_actions": 8,
    "mapped_card_actions": 6,
    "unmapped_card_actions": 2,
    "coverage_pct": 75.0,
    "unmapped_actions": [
      "send_notification",
      "generate_report"
    ],
    "mapped_actions": {
      "web_fetch": ["web_browsing"],
      "web_search": ["web_browsing"],
      "read_file": ["file_reading"],
      "read_data": ["database_read"],
      "write_data": ["database_write"],
      "compare": ["data_analysis"]
    }
  }
}

A coverage percentage below 100% means some card actions have no policy enforcement. Tools implementing those actions will fall through to the unmapped_tool_action default. Aim for 100% coverage in production policies.

Using Coverage in CI/CD

# Evaluate policy and fail if coverage is below threshold
smoltbot policy evaluate \
  --file policy.yaml \
  --agent-id agent_abc123 \
  --min-coverage 90

# Exit code 1 if coverage < 90%

This integrates naturally into pre-deploy gates: a policy change that reduces coverage below the threshold blocks the merge.

Putting It Together

Here is a complete policy for a research agent that can browse the web and read files, but cannot delete anything or execute shell commands:

meta:
  name: "research-agent-v2"
  version: "2.0.0"
  description: "Read-only research agent with web and filesystem access"

capability_mappings:
  web_browsing:
    tools:
      - "mcp__browser__navigate"
      - "mcp__browser__click"
      - "mcp__browser__screenshot"
      - "mcp__browser__evaluate_script"
    card_actions:
      - "web_fetch"
      - "web_search"

  file_reading:
    tools:
      - "mcp__filesystem__read_file"
      - "mcp__filesystem__list_directory"
    card_actions:
      - "read_file"

  search:
    tools:
      - "mcp__search__query"
      - "mcp__search__suggest"
    card_actions:
      - "search"
      - "compare"

forbidden:
  - pattern: "mcp__filesystem__delete*"
    reason: "File deletion not permitted for research agents"
    severity: "critical"
  - pattern: "mcp__filesystem__write*"
    reason: "File writing not permitted for research agents"
    severity: "high"
  - pattern: "mcp__shell__*"
    reason: "Shell execution never permitted"
    severity: "critical"

defaults:
  unmapped_tool_action: "warn"
  unmapped_severity: "medium"
  grace_period_hours: 24

escalation_triggers:
  - condition: "tool_count > 100"
    action: "escalate"
    reason: "Unusually high tool usage in single session"
  - condition: "unmapped_tool_count > 5"
    action: "escalate"
    reason: "Multiple unmapped tools detected"

Limitations

• Policy evaluation adds latency to gateway requests (typically under 5ms for policies with fewer than 100 rules).
• Glob patterns match tool names only, not tool arguments. A tool can be permitted by policy but still violate alignment constraints based on how it is called.
• Grace periods are tracked per-agent, not per-policy-version. Updating a policy does not reset grace period timers for previously seen tools.
• Coverage reports require a valid alignment card. Agents without a published card receive a coverage report with 0% coverage.

See Also

• Policy DSL Specification — Full normative specification for the policy YAML schema
• Policy CLI Reference — CLI commands for validating, evaluating, and managing policies
• Policy Management Guide — Step-by-step guide to writing and deploying policies
• Card Lifecycle — How alignment cards evolve and interact with policy
• Enforcement Modes — Alignment enforcement modes (observe/nudge/enforce)