0.0.002

ddat interop with dat js
cable interop with cabal js
ddat's frontend = dat-thang
cable fe = cable-thang
dillo support based on cel's dillo-dat
bootstrap node
xplatform support

Signed-off-by: divo <418@cryspace.space>

docs/primitives.md

@@ -0,0 +1,107 @@
+# datgpt primitives
+
+This is the working map of the old `dat://` implementation. Names follow the
+project: SLEEP owns feeds, cesspool owns Hyperswarm-style discovery, and dat
+joins the feeds into an archive.
+
+The standalone [libcess package](libcess.md) owns Cesspool discovery and UDP
+transport. Both the Dat (`ddat`) and Cable (`cable`) applications link it.
+
+## SLEEP storage
+
+- Feed header, tree nodes, signatures, bitfield pages, keys, and data files.
+- Flat-tree addressing, spans, roots, leaf and parent hashing.
+- Feed open, length, indexed read, append, flush, and close.
+- Public-key-only in-memory read-only feeds accept verified remote blocks and
+  retain their proofs without pretending the local node has signing keys. A
+  `.remote` cache reopens sparse blocks only after verifying each stored proof.
+- Data/tree bit access, index regeneration, scan, encode, and decode.
+- Key generation, public/secret persistence, hashes, signatures, root signing.
+- Merkle proof nodes, proof encode/decode, hash folding, and root signature check.
+
+## Archive
+
+- Paired metadata and content feeds.
+- Create, open, close, public keys, writable keypairs.
+- Separate secret-key save and writable reopen.
+- Header pointing from metadata to content.
+- File, directory, symlink, and deletion history entries.
+- Entry enumeration, reverse path lookup, and content block lookup.
+- Recursive filesystem import for regular files, directories, and symlinks.
+- Files are chunked across content blocks and can be exported recursively.
+- Snapshot sync compares content, mode, target, and millisecond mtime before
+  appending; paths removed locally receive tombstones, while unchanged paths
+  do not extend the feed.
+- `dat status`, `dat files`, `dat verify`, `dat resolve`, historical `cat`,
+  `checkout`, and versioned URL commands are available from the CLI.
+
+## Peer protocol
+
+- The root `NETWORK_PROTOCOL` selects legacy V6 framing (`hypercore`) for the
+  old Dat target; `cesspool` selects V7 framing for the in-tree network mode.
+- Protobuf feed-open messages, handshake id/live/userData, request/range/data
+  messages, extension payloads, and standard DATA node/signature fields.
+- Have/want ranges, block requests, cancellation.
+- Legacy WANT messages with omitted length run to the peer's current feed end;
+  responses are bounded by the local feed length and representable bitfield.
+- Data plus Merkle proof, and named extensions.
+- Per-channel replication state, remote bitfield, and request queue.
+- `dat_session` joins one archive key to its discovery topic, carries discovered
+  peer addresses, opens the archive feeds, gates the wire handshake on an
+  externally authenticated transport, exchanges HAVE ranges, requests missing
+  blocks, and verifies signed proofs before caching received blocks.
+- A peer callback exposes newly discovered or rediscovered peers so a transport
+  adapter can dial them and call `dat_session_peer_secure` after authentication.
+- `dat_session_poll` runs an optional transport poll callback alongside the
+  Cesspool UDP poll, refreshes topic membership, and reconciles discovered peers.
+- `dat_session_tick` is the nonblocking form of that poll cycle.
+- `dat_session_receive` accepts arbitrary byte-stream chunk boundaries, buffers
+  partial frames, decrypts the V6 stream at byte offsets, and dispatches multiple
+  coalesced frames from one read.
+- A read-only replica keeps received proofs and can serve them onward. The
+  caller still supplies the authenticated byte transport and poll loop.
+
+## cesspool
+
+- UDP socket transport, bootstrap nodes, listen, poll, and destroy.
+- Legacy lookup, announce, unannounce, ping, and holepunch vocabulary.
+- Topic join/leave and periodic refresh.
+- Peer table, failure/last-seen state, topic table, discovery callbacks.
+- Legacy protobuf requests/responses and compact public/local peer parsing.
+- XOR-distance peer selection and dht-rpc transaction tracking.
+
+## uTP
+
+- Header encode/decode and connection identifiers.
+- SYN/connect/accept, state acknowledgements, data, FIN, and reset states.
+- Send segmentation and timer boundary.
+- Receive reordering, cumulative and selective acknowledgements, congestion
+  window, retransmission timers, and in-flight packet retirement. DATA packets
+  are capped at the 1200-byte payload size, and malformed control packets are
+  rejected before entering the reorder buffer.
+
+## Name resolution
+
+- `dat-dns` resolution through HTTPS `/.well-known/dat`, plus the local
+  `~/.dat/dat-dns` name map and TTL-aware cache. The TXT packet decoder is a
+  compatibility helper and is not used for runtime name resolution.
+- Canonical `dat://<key>` output; the domain remains a name for the resolved
+  archive key rather than a separate archive identity.
+
+## Later layer
+
+- S7-facing records use one tagged, named-field form:
+  `(dat:wire-message (channel . N) (type . N) (payload . #u8(...)))`.
+  `dat:record-ref` and `dat:record-set` provide the shared field access layer;
+  protocol payload bytes remain bytevectors.
+- `dat-wire->sexp` and `dat-sexp->wire` adapt the existing C frame codec. This
+  keeps the C codec authoritative for peer bytes while Scheme is the common
+  application representation; a Scheme record is never serialized directly
+  onto the network.
+- Initial S7 bindings expose archive create, open, close, entry count, and the
+  wire-record adapter. The Scheme adapters and JSON reader are part of the default install.
+- Hyperdrive's JS `Index` and `Stat` records should use the same named-field
+  convention as their read/write adapters are added; they must retain their
+  original protocol field numbers and binary fields.
+- The local CLI sits above the archive API; peer transport polling and the
+  authenticated stream layer remain separate callers of `dat_session`.