Cosmix Self-Aware Layer — Observability Contract

The substrate exposes its state. Every daemon is queryable, every change is observable, every spec is reachable over the protocol it describes. An agent joining the mesh can build a complete world model without out-of-band documentation.

This chapter is the first of three substrate-layer SPECs (07 self-aware, 08 self-repair, 09 self-improve). Together they specify how Cosmix satisfies the mandate's three criteria — legibility, modifiability, and reconstructibility by agents. This SPEC covers legibility: the contract every cosmix component MUST meet so that agents (and humans, and other components) can see what the system is doing and what state it is in.

The constitution (Ch 00) governs autonomy. This SPEC governs observability. They are orthogonal: a system can be observable without being autonomous, and the constitution's invariants (audit trailers, circuit breaker, forbidden targets) depend on the observability primitives this SPEC defines.

---

1. Purpose and Non-Goals

1.1 What "agent-legible" means

A cosmix component is agent-legible when an agent that has never seen its source code can, using only ABP, answer:

1. What does this component do? — HELP, INFO
2. What state does it have? — <svc>.props.list, <svc>.props.describe
3. What is its state right now? — <svc>.props.get, world.<svc> topic
4. When does its state change? — <svc>.props.changed events
5. What contract does it implement? — SPEC command, spec.get

The five questions cover capability, schema, snapshot, dynamics, and contract. A component answering all five at the conformance levels in §9 is fully agent-legible.

1.2 Non-goals

This SPEC explicitly does not require:

• Rust-function-level reflection. Function signatures, call graphs, and type definitions remain build-time artifacts. Agents reason about ABP surfaces, not source.
• Debugger integration. Stepping, breakpoints, stack inspection — out of scope. The substrate is observable in production, not introspectable like a REPL.
• OS-level tracing. strace / eBPF / perf data are not part of the contract. Components MAY surface aggregated metrics via props.get, but raw OS traces are not required.
• Universal property mutability. This SPEC defines how state is read. Mutation happens through service-specific commands today and through SPEC 09 (self-improve) actions tomorrow. The two layers compose; this SPEC does not presume them.

1.3 Why this layer matters

Modifiability (mandate criterion 2) and reconstructibility (criterion 3) both depend on legibility. An agent cannot safely modify what it cannot observe; a system cannot reconstruct what it cannot inventory. The constitution's circuit breaker (Article IV.5) detects defective autonomous output by observing reverts in git log — that detection requires the audit trail (IV.4) is itself observable, which this SPEC formalises and extends.

---

2. The Uniform Property Surface

Moved to 2026-06-05-07b-property-surface.md (2026-06-05). The property read model — what a property is, the path grammar, props.get, and props.describe — is the reusable, code-backed foundation that this chapter's conformance + event-emission contract builds on, and that SPEC 12 (mutation, audit, collections) extends. Cross-refs to "SPEC 07 §2.x" resolve in 07b.

3. Mandatory Event Emission

A daemon at conformance level L2 or higher MUST emit a <svc>.props.changed event on the topic broker (Ch 03 §3.11) whenever an observable property changes.

3.1 Event shape

---
amp: 1
type: event
from: <svc>
command: <svc>.props.changed
topic: <svc>.props.changed
---
{
  "path": "lifecycle.health",
  "old": "ok",
  "new": "degraded",
  "cause": "01HXP7K3V8YYZ...",
  "ts": "2026-04-25T18:42:11.244Z"
}

Required body fields: path, old, new, ts. Optional: cause — the ABP message id that triggered the change, used for cross-service causal chains (constitution IV.4 trailers serve the analogous role for autonomous-commit causality).

3.2 What counts as an observable change

A change is observable when both hold:

• The path is enumerated by props.list.
• The path's describe does not declare transient: true.

Daemons MAY mark high-cardinality paths (counters, sizes that change rapidly under steady-state load) as transient: true to opt out of per-mutation events. Subscribers wanting current values use props.get directly.

3.3 Coalescing

Multiple changes within a single ABP request handler MAY be coalesced into one event per path, carrying the final value. Bulk operations (e.g., mailboxes.import) SHOULD emit a single summary event with path: "mailboxes" rather than one event per imported mailbox.

3.4 Lost-event recovery

props.changed is best-effort delivery. A subscriber that detects a gap (e.g., its locally-cached state contradicts a world.<svc> snapshot it later receives) MUST re-fetch via props.get rather than synthesising deltas. The retained snapshot in §4 is the authoritative recovery source.

3.5 Activity events (sister taxonomy)

Moved to 2026-06-05-07a-activity-events.md (2026-06-05). props.changed (§3.1–3.4 above) covers state transitions. Activity events are the orthogonal sister taxonomy for discrete actions (tool invocations, message dispatches, scheduled tasks, proposals applied) — they are not property state and are currently unbuilt. Extracting them keeps this chapter's implemented observability contract clean. Cross-references to "SPEC 07 §3.5.x" now resolve in 07a.

4. The world.* Topic Family

A daemon at conformance level L3 MUST publish a retained topic named world.<svc> whose payload is the most recent props.get result. The broker manages retention per Ch 03 §10.3.1: the snapshot lives until the producer disconnects + orphan timeout (default 60s).

4.1 The meta-subscriber pattern

A process subscribing to the wildcard world.* receives one snapshot per registered L3 service immediately on subscription, plus updated snapshots as producers republish. This is the live-world-model construction primitive: the subscriber maintains in-memory state covering every L3 daemon without polling, without service-by-service handshakes.

on world.* do
  $svc = $event.headers["topic"]
  state[$svc] = json_parse($event.body)
end

emit "topic.subscribe" args={"name": "world.*"}
-- state[] now fills as snapshots arrive from each L3 daemon

4.2 Update cadence

A daemon SHOULD republish world.<svc> after any non-trivial state change. "Non-trivial" is daemon-defined; default heuristic: republish on every event that would also be emitted to props.changed, capped at one republish per second (coalescing rapid changes). The broker does not enforce republish cadence; consumers requiring strict freshness use world.* for the cheap path and fall back to props.get for correctness.

4.3 Staleness handling

Per Ch 03 §10.3.1, the broker marks a snapshot stale on producer disconnect. Subscribers receive topic_stale: true on the first delivery after staleness is detected. A meta-subscriber SHOULD treat stale snapshots as "last known state" and either wait for republish or query the daemon directly via props.get if reachable.

---

5. Spec Distribution and Capability Discovery

Moved to 2026-06-05-07c-spec-distribution.md (2026-06-05). A distinct subsystem from the observability contract: fetching the specs over ABP (spec.get, world.specs.*, the SPEC universal, agent bootstrap). Cross-refs to "SPEC 07 §5.x" resolve in 07c.

6. Encoding Decisions (Open)

The decisions below are flagged for resolution before this SPEC moves to stable. Implementations MAY pick reasonable defaults; the SPEC will pin once ≥ 2 daemons have shipped real implementations and operational experience surfaces a preference.

6.1 props.get payload encoding

Options: (a) JSON object in body. (b) Embedded ABP child message tree.

Current recommendation: JSON in body. Matches existing args: patterns, universally parseable, no novel encoding required. Embedded ABP only if/when a property's value is itself an ABP message (rare).

6.2 props.changed delta vs snapshot

Options: (a) Delta (path, old, new). (b) Full snapshot per event. (c) Both, selected by subscriber preference.

Current recommendation: Delta. Subscribers wanting full state already have world.<svc> and props.get. Delta keeps event volume tractable and makes coalescing semantics simple.

6.3 Property-path syntax

Options: dotted (config.bind), JSON Pointer (/config/bind), Mix-style ($cfg.bind).

Current recommendation: Dotted. Matches Mix dot-access (Ch 04) and ABP command naming (Ch 02 §1). JSON Pointer is unfamiliar; Mix-style implies variable semantics that don't apply here.

6.4 Schema language for describe

Options: JSON-Schema subset, custom minimal schema, no formal schema (prose only).

Current recommendation: Custom minimal schema (the field set in §2.4). Full JSON-Schema is heavy; prose-only is unparseable. The minimal set covers the cases agents actually need without committing to JSON-Schema's full surface.

Resolved (2026-05-11): SPEC 12 §4.3 pins the PropertySchema field set (name, type, default, secret, validators, help, since/until) as the v0.1 commitment for namespaces registered via SPEC 12. Flat-path describe responses continue to use the §2.4 envelope.

---

7. Operational Concerns

7.1 Cardinality and rate

props.changed events traverse the topic broker. A daemon emitting one event per microsecond will saturate any subscriber. Discipline:

• Mark high-frequency paths transient: true and rely on props.get for current value.
• Coalesce per §3.3.
• Per-path event rate SHOULD NOT exceed 10 Hz under steady-state load. Bursts are acceptable; sustained higher rates indicate the path belongs in a metrics stream, not the property surface.

The broker MAY apply backpressure on a producer whose props.changed emissions saturate the broker. Backpressure shape (drop, queue, slow-publish) is a broker-implementation choice, not a SPEC requirement.

7.2 Privacy and sensitive values

A property whose describe declares sensitive: true:

• props.get returns the value redacted by default (e.g., "***" for strings, null for numbers).
• Full value access requires an explicit reveal: true arg AND the request originating from a peer in the WG /24 trust domain (constitution Article III.2). Out-of-mesh requests always see redacted.
• props.changed events for sensitive paths emit only path and cause — no old or new values.

This SPEC does not define the trust check; it defines the contract daemons MUST honour. Broker-level enforcement of reveal: true against the mesh trust domain is a noded responsibility, specified separately when the shape stabilises.

7.3 Schema evolution

Property surfaces evolve. Discipline:

• Additive (new property, new optional field in describe): no version bump required; meta-subscribers ignore unknown paths gracefully.
• Renaming (path moved): emit both old and new for one release cycle; describe the old path as deprecated: true; remove in the cycle after.
• Breaking (type change, semantic shift): bump the daemon's INFO.version major; meta-subscribers comparing version strings can refuse to model versions they don't understand.

7.4 Root-cause discipline

When a property change is caused by an upstream failure or another daemon's event, the cause field SHOULD reference the ABP id of the originating event, not the local handler's correlation id. This makes world.* subscribers able to reconstruct chains of causation across services without daemon-specific log scraping.

This is a soft requirement; daemons SHOULD do this and meta-subscribers SHOULD NOT depend on it. SPEC 08 (self-repair) §8 formalises the pattern for repair-escalation chains, where causation tracing is mandatory.

---

8. Migration

8.1 Order

8.2 Compatibility

This SPEC is purely additive. Existing service-specific commands (mailbox.list, noded.list, syncd.share.add, etc.) remain. props.* commands augment the surface; they do not replace existing commands. A daemon implementing only the existing surface remains conformant with chapters 01–06 but is L0 against this chapter.

8.3 Conformance staging

A daemon SHOULD ship at L1 first (props.get + props.list + props.describe), then upgrade to L2 (props.watch + props.changed) as event-emission paths are identified, then L3 (world.<svc> retained topic) once steady-state behaviour is understood. Skipping levels is permitted but discouraged — L3 without L2 means consumers depend on snapshot replay rather than live deltas, which amplifies cardinality concerns from §7.1.

Exception — new daemons designed against this SPEC: a daemon greenfielded after this SPEC stabilises SHOULD ship at L3 from day one rather than staging through L1/L2. Staging exists for retrofit, not for new construction. §8.4 walks the worked example.

8.4 Worked example: MDS-class daemons at L3 from day one

cosmix-mds (per-set SQLite metadata + content-addressable blob store) is the first substrate component whose architecture maps cleanly onto the property surface. The maild-on-MDS migration (roadmap Phase 8) is the first opportunity to ship a fresh L3 daemon rather than retrofit one. This subsection records the worked shape so future MDS-shaped daemons (any daemon with multi-set state + immutable content + a change log) inherit the same property layout.

8.4.1 Property tree shape

{
  "config": {
    "root": "/var/lib/cosmix/maild",
    "gc_quiescence_secs": 60,
    "verify_schedule": "weekly"
  },
  "lifecycle": {
    "started_at": "2026-05-10T08:14:22Z",
    "uptime_s": 38291,
    "health": "ok",
    "props_level": "L3",
    "activity_topics": ["maild.message.delivered", "agent.tool.invoked"]
  },
  "sets": {
    "count": 3,
    "list": ["018f-...", "019a-...", "01a0-..."]
  },
  "blob_index": {
    "blobs_total": 14823,
    "bytes_on_disk": 1073741824,
    "dedup_ratio": 1.42,
    "last_gc_at": "2026-05-09T03:00:00Z",
    "last_gc_blobs_freed": 217
  },
  "verify": {
    "last_full_at": "2026-05-08T02:00:00Z",
    "last_full_mismatches": 0,
    "blobs_verified_24h": 12340,
    "ledger_rows": 14823
  }
}

Per-set detail lives at sets.<uuid>.* and is fetched on demand — listing every blob count under every set inline would explode world.<svc> payload size:

{
  "sets.018f-...": {
    "containers": { "count": 12, "list": ["INBOX", "Sent", "Drafts", ...] },
    "items": { "count": 8421, "bytes": 421887488 },
    "last_change_seq": 42017
  }
}

8.4.2 What gets transient: true

• blob_index.bytes_on_disk — recomputed lazily; per-mutation events would saturate the broker on heavy put_blob traffic.
• blob_index.blobs_total — same.
• sets.<uuid>.last_change_seq — updates on every add_item / store_flags. Consumers wanting deltas use changes_since, not props.changed.

These paths still appear in props.list and props.get; they simply don't emit per-mutation events. Their values become consistent on the next world.<svc> republish (capped at 1 Hz per §4.2).

8.4.3 What emits activity events

Activity events (per §3.5) cover discrete actions whose observability would otherwise require log scraping:

• maild.message.delivered — per-message dispatch outcome. Required details: set_uuid, container, size_bytes, dkim_pass.
• maild.gc.completed — per-GC-pass summary. Mirrors the MDS mds.gc.completed event already emitted post-commit (per cosmix-mds's EventSink); the daemon-level activity event adds scoping context (which account triggered, if any).
• maild.verify.completed — per-verify-sweep summary. Required details: scope (full|since|container), mismatches, duration_ms.

These are activity (§3.5), not property changes — running a verify sweep does not transition any persistent property; the result of the sweep updates verify.last_full_at, which IS a props.changed event, but the event of running it is the activity event.

8.4.4 Why L3 from day one

A maild greenfielded against this SPEC has no retrofit cost:

• Property paths are designed alongside the schema, not bolted on.
• world.maild is published from the start, so the Codex-suggested "substrate doctor" (a future SPEC 07 conformance dashboard) gets a full snapshot the moment maild registers.
• The activity-event topics are declared at startup via lifecycle.activity_topics, so subscribers can discover them through props.describe rather than reading source.

Staging through L1 → L2 → L3 is a retrofit pattern. New daemons whose state is structured (MDS-shaped, JMAP-shaped, indexd-shaped) skip straight to L3 and amortise the design cost during construction instead of paying it twice.

---

9. Conformance

The SPEC defines four conformance levels:

A daemon declares its level via a property at path lifecycle.props_level:

---
to: indexd
command: indexd.props.get
args: {"path": "lifecycle.props_level"}
---

Response body: {"value": "L2"}.

Meta-subscribers building world models SHOULD prefer L3 daemons, fall back to L2 with explicit props.get polling on subscription, and degrade to L1 with periodic poll. L0 daemons cannot participate in the world model and are inventory-only via noded.list.

---

10. Open Questions and Future Work

• Mutation contract. This SPEC defines reads. The wire-level mutation contract — <svc>.props.set, <svc>.props.delete, <svc>.props.validate, optimistic concurrency, hooks, capability gating, audit — is supplied by SPEC 12 (Property Substrate, amends: [07]). SPEC 09 (self-improve) composes on top of SPEC 12 by determining the trust gradient — which agents may invoke which mutations, prompt vs automatic — without respecifying the wire. Conformance levels L4 (mutable) and L5 (auditable) defined in SPEC 12 §13 extend the L0..L3 ladder in §9 above.
• Activity event schema for cross-runtime families. §3.5.2 allows cross-runtime families like agent.tool.invoked where multiple daemons publish to the same topic. The required details schema for the agent.* family is sketched in §3.5.3 but should be pinned in a short addendum (or in the cosmix-lib-tools extraction plan, _doc/2026-05-03-cosmix-agent-runtime-unification-plan.md) once the first two co-publishers (cosmix-mcp + cosmix-agentd) are wired and operational experience surfaces missing fields.
• Activity-event retention. Properties live forever (until the daemon evicts them); activity events are inherently transient. Whether the topic broker SHOULD offer a bounded replay buffer for agent.tool.invoked (so a fresh meta-subscriber can see the last N invocations) is open. Default: no replay; subscribers that need history scrape the activity log a daemon SHOULD persist alongside its data store. Promote to MUST only if the no-replay default proves unworkable for substrate-doctor-class consumers.
• Cross-mesh world.*. A meta-subscriber on node A receiving world.* from node B currently requires explicit cross-node topic subscription (Ch 01 §10). A federated world.<node>.<svc> namespace would simplify multi-node observability; deferred until a second mesh node is in production use.
• Schema registry. props.describe returns per-path schema. A service-wide schema document (consolidating all describes) might be useful for static analysis. Deferred — premature until describe coverage exists.
• Privacy enforcement at the broker. §7.2 describes the contract. Where enforcement happens (every daemon? broker-level interception?) is open.
• Promotion to stable. This SPEC moves from draft to stable when: (a) ≥ 3 daemons are L1-conformant, (b) ≥ 1 daemon is L3-conformant, (c) §6 encoding decisions are pinned, (d) at least one meta-subscriber is in production use.

---

Document created: 2026-04-25. Drafted in collaboration between Mark Constable and Claude Opus 4.7 as the first of three substrate-layer SPECs (self-aware / self-repair / self-improve).

§3.5 (activity events) and §8.4 (MDS-class L3-from-day-one worked example) added 2026-05-03 from a Codex review of the cosmix-mds Phase 7 work and the agentd ↔ mcp unification plan (_doc/2026-05-03-cosmix-agent-runtime-unification-plan.md). Activity events fill the gap where props.changed covered state transitions but not discrete actions — the audit shape the unification plan needed. The MDS worked example formalises the "new daemons skip retrofit staging" exception so Phase 8 (maild on MDS) ships L3 from day one.

§3.5.7 (agent session identity) and §3.5.2 verb-collision note added 2026-05-03 to lift the unification plan's <runtime>:<instance_uuid> session-actor shape from an implementation detail into a substrate invariant binding on every future agent runtime, and to plant the discipline (without yet building the registry) for verb-family ownership across cross-runtime topics.