Documentation Index
Fetch the complete documentation index at: https://mnemomllc-feat-aip-output-analysis-docs.mintlify.app/llms.txt
Use this file to discover all available pages before exploring further.
OpenTelemetry Attributes Reference
Current as of @mnemom/aip-otel-exporter v0.5.0 and @mnemom/agent-integrity-protocol v0.6.0.
Span Attributes
aip.integrity_check — 26 attributes + 2 GenAI SIG aliases
The primary span for AIP integrity checkpoint results.
| Attribute | Type | Source | Description |
|---|
aip.integrity.checkpoint_id | string | checkpoint | Unique checkpoint identifier |
aip.integrity.verdict | string | checkpoint | clear, review_needed, or boundary_violation |
aip.integrity.agent_id | string | checkpoint | Agent identifier |
aip.integrity.card_id | string | checkpoint | Alignment Card identifier |
aip.integrity.session_id | string | checkpoint | Session identifier |
aip.integrity.thinking_hash | string | checkpoint | SHA-256 hash of the thinking block |
aip.integrity.proceed | boolean | signal | Whether the agent should proceed |
aip.integrity.recommended_action | string | signal | continue, log_and_continue, pause_for_review, or deny_and_escalate |
aip.integrity.concerns_count | int | signal | Number of concerns detected |
aip.integrity.analysis_model | string | analysis_metadata | Model used for integrity analysis |
aip.integrity.analysis_duration_ms | float | analysis_metadata | Analysis duration in milliseconds |
aip.integrity.thinking_tokens | int | analysis_metadata | Token count of the thinking block |
aip.integrity.truncated | boolean | analysis_metadata | Whether the thinking block was truncated |
aip.integrity.extraction_confidence | float | analysis_metadata | Confidence in thinking block extraction |
aip.integrity.output_hash | string | checkpoint | SHA-256 hash of the output block (when analysis_scope is thinking_and_output) |
aip.integrity.output_tokens | int | analysis_metadata | Token count of the output block analyzed |
aip.integrity.output_truncated | boolean | analysis_metadata | Whether the output block was truncated |
aip.integrity.analysis_scope | string | checkpoint | thinking_only or thinking_and_output |
The four output attributes (output_hash, output_tokens, output_truncated, analysis_scope) are only populated when analysis_scope is thinking_and_output. When analysis_scope is thinking_only (the default), these attributes are omitted from the span.
aip.conscience.consultation_depth | string | conscience_context | standard or deep conscience evaluation |
| aip.conscience.values_checked_count | int | conscience_context | Number of values evaluated |
| aip.conscience.conflicts_count | int | conscience_context | Number of value conflicts detected |
| aip.window.size | int | window_summary | Number of checkpoints in the rolling window |
| aip.window.integrity_ratio | float | window_summary | Ratio of clear verdicts (0.0-1.0) |
| aip.window.drift_alert_active | boolean | window_summary | Whether a drift alert is currently active |
| gen_ai.evaluation.verdict | string | GenAI SIG | Forward-compatible alias for aip.integrity.verdict |
| gen_ai.evaluation.score | float | GenAI SIG | Forward-compatible alias for aip.window.integrity_ratio |
Events emitted on this span:
aip.concern — One event per concern detected. Includes category, severity, description, and evidence.aip.drift_alert — Emitted when a drift alert is active. Includes drift_direction and integrity_similarity.
aap.verify_trace — 8 attributes
Span for AAP trace verification results.
| Attribute | Type | Description |
|---|
aap.verification.result | boolean | Whether the trace passed verification |
aap.verification.similarity_score | float | Similarity score between trace and card |
aap.verification.violations_count | int | Number of violations detected |
aap.verification.warnings_count | int | Number of warnings detected |
aap.verification.trace_id | string | AP-Trace identifier |
aap.verification.card_id | string | Alignment Card identifier |
aap.verification.duration_ms | float | Verification duration in milliseconds |
aap.verification.checks_performed | string | Comma-separated list of checks performed |
aap.violation — One event per violation. Includes type, severity, description, and field.
aap.check_coherence — 5 attributes
Span for AAP value coherence check results (used in multi-agent coordination).
| Attribute | Type | Description |
|---|
aap.coherence.compatible | boolean | Whether the agents are value-compatible |
aap.coherence.score | float | Coherence score (0.0-1.0) |
aap.coherence.proceed | boolean | Whether coordination should proceed |
aap.coherence.matched_count | int | Number of matched values |
aap.coherence.conflict_count | int | Number of conflicting values |
aap.detect_drift — 2 attributes
Span for AAP behavioral drift detection results.
| Attribute | Type | Description |
|---|
aap.drift.alerts_count | int | Number of drift alerts generated |
aap.drift.traces_analyzed | int | Number of traces analyzed for drift |
aap.drift_alert — One event per alert. Includes drift_direction, integrity_similarity, and sustained_checks.
policy.evaluate — 10 attributes (Implemented)
Span for policy evaluation results. Created when the Policy Engine evaluates a request against the active policy.
| Attribute | Type | Source | Description |
|---|
policy.agent_id | string | input | Agent being evaluated |
policy.policy_id | string | policy | Policy identifier |
policy.policy_version | int | policy | Policy version number |
policy.verdict | string | result | Evaluation verdict: pass, warn, fail |
policy.violations_count | int | result | Number of violations detected |
policy.warnings_count | int | result | Number of warnings detected |
policy.coverage_pct | float | result | Percentage of card actions mapped by policy |
policy.context | string | input | Evaluation context: cicd, gateway, observer |
policy.duration_ms | float | measured | Evaluation duration in milliseconds |
policy.enforcement_mode | string | policy | Active enforcement mode: warn, enforce, off |
policy.violation — One event per violation. Includes type (forbidden/capability_exceeded/unmapped_denied), tool, severity, and reason.
reclassification.process — 8 attributes
Span for violation reclassification. Created when a checkpoint is reclassified via the Reclassification API.
| Attribute | Type | Source | Description |
|---|
reclassification.agent_id | string | input | Agent whose violation is reclassified |
reclassification.checkpoint_id | string | input | Checkpoint being reclassified |
reclassification.original_type | string | checkpoint | Original violation type (e.g., UNMAPPED_TOOL) |
reclassification.new_type | string | result | New classification: card_gap or behavior_gap |
reclassification.reason | string | input | Human-provided reclassification reason |
reclassification.card_amendment_id | string | input | Linked card amendment ID (if any) |
reclassification.score_before | int | computed | Agent score before reclassification |
reclassification.score_after | int | computed | Agent score after recomputation |
Span Hierarchy
Spans are created as children of the current active span via context.active():
your_application_span
├── aip.integrity_check
│ ├── event: aip.concern (one per concern)
│ └── event: aip.drift_alert (when drift active)
├── policy.evaluate
│ └── event: policy.violation (one per violation)
├── reclassification.process
├── aap.verify_trace
│ └── event: aap.violation (one per violation)
├── aap.check_coherence
└── aap.detect_drift
└── event: aap.drift_alert (one per alert)
Metrics
13 metric instruments for aggregate monitoring:
AIP Metrics
| Metric | Type | Labels | Description |
|---|
aip.integrity_checks.total | Counter | verdict, agent_id | Total number of integrity checks performed |
aip.concerns.total | Counter | category, severity | Total number of concerns detected |
aip.analysis.duration_ms | Histogram | verdict | Distribution of analysis durations |
aip.window.integrity_ratio | Histogram | — | Distribution of integrity ratios across windows |
aip.drift_alerts.total | Counter | — | Total number of drift alerts generated |
AAP Metrics
| Metric | Type | Labels | Description |
|---|
aap.verifications.total | Counter | verified | Total number of trace verifications |
aap.violations.total | Counter | type, severity | Total number of violations detected |
aap.verification.duration_ms | Histogram | — | Distribution of verification durations |
aap.coherence.score | Histogram | compatible | Distribution of coherence scores |
Policy Metrics
| Metric | Type | Labels | Description |
|---|
policy.evaluations.total | Counter | verdict, context | Total number of policy evaluations |
policy.violations.total | Counter | type, severity | Total number of policy violations detected |
policy.evaluation.duration_ms | Histogram | verdict | Distribution of policy evaluation durations |
Reclassification Metrics
| Metric | Type | Labels | Description |
|---|
reclassification.total | Counter | new_type | Total reclassifications (card_gap vs behavior_gap) |
GenAI SIG Forward Compatibility
The exporter includes forward-compatible aliases that track the emerging OTel GenAI SIG conventions for AI/ML observability:
| GenAI SIG Attribute | Maps To | Description |
|---|
gen_ai.evaluation.verdict | aip.integrity.verdict | Standardized evaluation verdict |
gen_ai.evaluation.score | aip.window.integrity_ratio | Standardized evaluation score |
aip.* attributes on every aip.integrity_check span, ensuring forward compatibility as the OTel GenAI SIG conventions stabilize.
Dashboard Templates
Pre-built Grafana and Datadog dashboards that consume these attributes and metrics are available in the aip-otel-exporter repository:
- grafana-aip-overview.json — Fleet-wide integrity monitoring using
aip.integrity_checks.total, aip.concerns.total, and aip.window.integrity_ratio
- grafana-aip-detail.json — Per-agent deep-dive using all
aip.integrity.* span attributes
- datadog-aip-overview.json — Datadog importable dashboard
