cog-ha-matter (ADR-116 P4): witness file persistence + chain-level verify
Closes the witness audit-bundle surface. The hash-chain primitive
+ JSONL serializer from earlier iters only handled one event at a
time; this lands the file-stream surface that operations actually
need:
* `WitnessChain::write_jsonl(&mut impl Write) -> io::Result<()>`
— streams every event as one line + `\n`, empty chain writes
zero bytes
* `WitnessChain::read_jsonl(impl BufRead) -> Result<WitnessChain,
WitnessReadError>` — parses event-by-event AND runs chain-level
`verify()` on the loaded chain, catching reordered or replayed
prefixes that per-event hashing alone misses
Critical security property: `read_jsonl` calls `WitnessChain::verify`
on the loaded chain BEFORE returning Ok. A forged bundle assembled
from two valid chains pasted together would slip past the
per-event hash check (each event's `this_hash` is internally
consistent) but the cross-event `prev_hash` linkage detects the
seam. Test `read_jsonl_chain_verify_catches_reordered_events`
locks this — swap two events in a 2-event bundle, see Verify error.
Error surface (new `WitnessReadError` enum):
* `Io { line_no, msg }` — read failure mid-stream
* `Parse { line_no, source }` — per-event from_jsonl_line failure
* `Verify { source }` — chain-level verify failure
`line_no` is 1-indexed so an auditor sees the same number their
text editor shows. Blank lines tolerated for hand-edited bundles.
7 new tests:
* empty chain writes zero bytes
* write→read round-trips a 3-event chain
* exactly N newlines for N events; trailing newline present
* blank lines / leading newline tolerated
* parse error surfaces with correct line_no
* reordered events caught by chain-level verify
* no-trailing-newline still loads the final event
51/51 cog tests green (44 → 51).
Co-Authored-By: claude-flow <ruv@ruv.net>
This commit is contained in:
ruv
parent
a3478ea3b5
commit
1f5b7b48c9
|
|
@ -95,7 +95,7 @@ Ranked by build cost × user impact:
|
|||
| **P1** | Research dossier ([`docs/research/ADR-116-ha-matter-cog-research.md`](../research/ADR-116-ha-matter-cog-research.md)) | ✅ **done** — 8 sections, 30+ citations, v1 scope ranked |
|
||||
| **P2** | Cog crate scaffold (`v2/crates/cog-ha-matter/`) — Cargo.toml + `src/{lib,main,manifest}.rs`, workspace member, CLI args, `--print-manifest` flag, 2 manifest unit tests | ✅ **done** — `cargo check` + `cargo test` green |
|
||||
| **P3** | Wrap existing ADR-115 MQTT publisher as cog entry point | ✅ **wiring done** — `main.rs` boots ADR-115's `publisher::spawn` via `runtime::spawn_publisher` thin wrapper, holds a long-lived `broadcast::Sender<VitalsSnapshot>`, awaits Ctrl-C. Live-handle test green without a broker. Next (P3.5): subscribe to sensing-server `/v1/snapshot` WS and republish into the channel. |
|
||||
| **P4** | Seed-native enhancements (embedded broker, mDNS, witness) | in progress — (a) mDNS service-record builder shipped. (b) Witness hash-chain primitive shipped (append-only SHA-256, `verify()` catches tampering). (c) **Witness JSONL persistence shipped** — `WitnessEvent::{to,from}_jsonl_line` round-trips with alphabetical field order for byte-stable archival hashes; parser re-verifies stored `this_hash` against canonical bytes so tampered bundles fire `HashMismatch` before loading. (d) Responder (mdns-sd) + embedded rumqttd + Ed25519 signing layer still pending. |
|
||||
| **P4** | Seed-native enhancements (embedded broker, mDNS, witness) | in progress — (a) mDNS service-record builder. (b) Witness hash-chain primitive. (c) Witness JSONL line serializer. (d) **Witness file persistence shipped** — `WitnessChain::{write_jsonl, read_jsonl}` accept any `Write`/`BufRead`, tolerate blank lines, surface `line_no` on parse error, run chain-level `verify()` on load to catch reordered/replayed events. 7 new tests including reorder-detection. (e) Responder (mdns-sd) + embedded rumqttd + Ed25519 signing layer still pending. |
|
||||
| **P5** | RuVector-backed threshold learning (SONA adaptation) | pending |
|
||||
| **P6** | Multi-Seed federation (cross-Seed dedup + witness) | pending |
|
||||
| **P7** | Matter Bridge mode (depends on matter-rs / esp-matter readiness) | pending |
|
||||
|
|
|
|||
|
|
@ -30,6 +30,8 @@
|
|||
//! when the chain spans days and the auditor wants O(log n)
|
||||
//! inclusion proofs.
|
||||
|
||||
use std::io::{self, BufRead, Write};
|
||||
|
||||
use sha2::{Digest, Sha256};
|
||||
|
||||
/// 32-byte hash output. Lifted into a newtype so a future migration
|
||||
|
|
@ -198,6 +200,49 @@ impl WitnessChain {
|
|||
&self.events
|
||||
}
|
||||
|
||||
/// Stream every event to a JSONL sink. Each event becomes one
|
||||
/// line terminated by `\n`. Empty chains write zero bytes.
|
||||
///
|
||||
/// The caller owns the writer — `File`, `BufWriter`, an
|
||||
/// in-memory `Vec<u8>` for tests — so this method never
|
||||
/// allocates beyond per-event line buffers.
|
||||
pub fn write_jsonl<W: Write>(&self, w: &mut W) -> io::Result<()> {
|
||||
for ev in &self.events {
|
||||
w.write_all(ev.to_jsonl_line().as_bytes())?;
|
||||
w.write_all(b"\n")?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Read a JSONL audit bundle into a fresh `WitnessChain`. Each
|
||||
/// non-empty line is parsed via `WitnessEvent::from_jsonl_line`
|
||||
/// (which re-verifies the stored hash), then the loaded chain
|
||||
/// is end-to-end verified via [`WitnessChain::verify`] to catch
|
||||
/// out-of-order events or replayed prefixes.
|
||||
///
|
||||
/// Bundle errors surface with their `line_no` (1-indexed) so an
|
||||
/// auditor can point at the bad record.
|
||||
pub fn read_jsonl<R: BufRead>(r: R) -> Result<WitnessChain, WitnessReadError> {
|
||||
let mut chain = WitnessChain::new();
|
||||
for (i, line_res) in r.lines().enumerate() {
|
||||
let line_no = i + 1;
|
||||
let line = line_res.map_err(|e| WitnessReadError::Io {
|
||||
line_no,
|
||||
msg: e.to_string(),
|
||||
})?;
|
||||
if line.trim().is_empty() {
|
||||
continue; // tolerate blank lines / trailing \n
|
||||
}
|
||||
let ev = WitnessEvent::from_jsonl_line(&line)
|
||||
.map_err(|source| WitnessReadError::Parse { line_no, source })?;
|
||||
chain.events.push(ev);
|
||||
}
|
||||
chain
|
||||
.verify()
|
||||
.map_err(|source| WitnessReadError::Verify { source })?;
|
||||
Ok(chain)
|
||||
}
|
||||
|
||||
/// Verify every event's `this_hash` matches the canonical bytes,
|
||||
/// every `prev_hash` matches the predecessor's `this_hash`, and
|
||||
/// `seq` is gap-free starting at 0.
|
||||
|
|
@ -239,6 +284,23 @@ pub enum WitnessVerifyError {
|
|||
HashMismatch { at: usize },
|
||||
}
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum WitnessReadError {
|
||||
#[error("io error at line {line_no}: {msg}")]
|
||||
Io { line_no: usize, msg: String },
|
||||
#[error("parse error at line {line_no}: {source}")]
|
||||
Parse {
|
||||
line_no: usize,
|
||||
#[source]
|
||||
source: WitnessParseError,
|
||||
},
|
||||
#[error("chain-level verify failed: {source}")]
|
||||
Verify {
|
||||
#[source]
|
||||
source: WitnessVerifyError,
|
||||
},
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
|
||||
pub enum WitnessParseError {
|
||||
#[error("invalid JSON: {0}")]
|
||||
|
|
@ -612,4 +674,122 @@ mod tests {
|
|||
let err = WitnessHash::from_hex("ab").unwrap_err();
|
||||
assert!(matches!(err, WitnessParseError::HashLength { found: 2 }));
|
||||
}
|
||||
|
||||
// ---- file persistence (write_jsonl / read_jsonl) ----
|
||||
|
||||
#[test]
|
||||
fn write_jsonl_empty_chain_writes_zero_bytes() {
|
||||
let c = WitnessChain::new();
|
||||
let mut buf = Vec::new();
|
||||
c.write_jsonl(&mut buf).unwrap();
|
||||
assert_eq!(buf, b"");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn write_then_read_round_trips_multi_event_chain() {
|
||||
let mut written = WitnessChain::new();
|
||||
written.append("a", b"first", 100);
|
||||
written.append("b", b"second", 101);
|
||||
written.append("c", br#"{"x":1}"#, 102);
|
||||
|
||||
let mut buf = Vec::new();
|
||||
written.write_jsonl(&mut buf).unwrap();
|
||||
|
||||
let read_back = WitnessChain::read_jsonl(buf.as_slice()).unwrap();
|
||||
assert_eq!(read_back.len(), 3);
|
||||
assert_eq!(read_back.events(), written.events());
|
||||
assert_eq!(read_back.tip(), written.tip());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn write_jsonl_separates_events_with_newline() {
|
||||
let mut c = WitnessChain::new();
|
||||
c.append("a", b"1", 100);
|
||||
c.append("b", b"2", 101);
|
||||
let mut buf = Vec::new();
|
||||
c.write_jsonl(&mut buf).unwrap();
|
||||
let s = std::str::from_utf8(&buf).unwrap();
|
||||
// Exactly N newlines for N events.
|
||||
assert_eq!(s.matches('\n').count(), 2);
|
||||
assert!(s.ends_with('\n'));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn read_jsonl_tolerates_blank_lines() {
|
||||
let mut c = WitnessChain::new();
|
||||
c.append("a", b"1", 100);
|
||||
c.append("b", b"2", 101);
|
||||
let mut buf = Vec::new();
|
||||
c.write_jsonl(&mut buf).unwrap();
|
||||
// Inject blanks — sometimes happens when files are edited.
|
||||
let with_blanks = format!(
|
||||
"\n{}\n\n",
|
||||
std::str::from_utf8(&buf).unwrap().trim_end()
|
||||
);
|
||||
let read = WitnessChain::read_jsonl(with_blanks.as_bytes()).unwrap();
|
||||
assert_eq!(read.len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn read_jsonl_surfaces_line_no_on_parse_error() {
|
||||
// Two good events, then one with a flipped payload byte.
|
||||
let mut c = WitnessChain::new();
|
||||
c.append("a", b"1", 100);
|
||||
c.append("b", b"2", 101);
|
||||
let mut buf = Vec::new();
|
||||
c.write_jsonl(&mut buf).unwrap();
|
||||
let mut text = String::from_utf8(buf).unwrap();
|
||||
let forged = c.events()[0].to_jsonl_line().replacen(
|
||||
"payload_hex\":\"31",
|
||||
"payload_hex\":\"32",
|
||||
1,
|
||||
);
|
||||
text.push_str(&forged);
|
||||
text.push('\n');
|
||||
|
||||
let err = WitnessChain::read_jsonl(text.as_bytes()).unwrap_err();
|
||||
match err {
|
||||
WitnessReadError::Parse { line_no, .. } => assert_eq!(line_no, 3),
|
||||
other => panic!("expected Parse error at line 3, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn read_jsonl_chain_verify_catches_reordered_events() {
|
||||
// Build a chain, then write it out with the events swapped.
|
||||
// Each individual event still verifies its own hash (because
|
||||
// its prev_hash is internally consistent with what *it*
|
||||
// claimed), but the cross-event chain check fires.
|
||||
let mut original = WitnessChain::new();
|
||||
original.append("a", b"1", 100);
|
||||
original.append("b", b"2", 101);
|
||||
let mut buf = Vec::new();
|
||||
original.write_jsonl(&mut buf).unwrap();
|
||||
let lines: Vec<&[u8]> = buf.split(|&b| b == b'\n').filter(|s| !s.is_empty()).collect();
|
||||
// Reverse order, send through reader.
|
||||
let mut reversed: Vec<u8> = Vec::new();
|
||||
reversed.extend_from_slice(lines[1]);
|
||||
reversed.push(b'\n');
|
||||
reversed.extend_from_slice(lines[0]);
|
||||
reversed.push(b'\n');
|
||||
let err = WitnessChain::read_jsonl(reversed.as_slice()).unwrap_err();
|
||||
assert!(matches!(err, WitnessReadError::Verify { .. }));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn read_jsonl_no_trailing_newline_still_works() {
|
||||
// BufRead's lines() handles the no-final-newline case; lock
|
||||
// the behavior so a future swap to a different reader can't
|
||||
// silently truncate the last event.
|
||||
let mut c = WitnessChain::new();
|
||||
c.append("only", b"x", 100);
|
||||
let mut buf = Vec::new();
|
||||
c.write_jsonl(&mut buf).unwrap();
|
||||
// Strip the trailing \n.
|
||||
if buf.last() == Some(&b'\n') {
|
||||
buf.pop();
|
||||
}
|
||||
let read = WitnessChain::read_jsonl(buf.as_slice()).unwrap();
|
||||
assert_eq!(read.len(), 1);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in New Issue