divorce/dat-thang
2
1 Fork 0
Code Issues Pull Requests Actions Packages Releases Activity

0.0.002

Browse Source 
ddat interop with dat js
cable interop with cabal js
ddat's frontend = dat-thang
cable fe = cable-thang
dillo support based on cel's dillo-dat
bootstrap node
xplatform support

Signed-off-by: divo <418@cryspace.space>
This commit is contained in:
divorce committed 2026-10-19 00:17:02 -04:00
1 parent f0f91540be
commit b735900f2e
323 files changed
+134347 -343
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/beaker-networking.md
+109
View File
@@ -0,0 +1,109 @@
# Beaker networking notes for `dat-gpt`
These notes use the archived `beakerbrowser/beaker` 1.1.0 source and the
included old `dillo-dat` plugin as references. Beaker was archived in 2022.
Its later Hyperdrive network stack is useful for subsystem boundaries, but it
is not proof of compatibility with the old Dat protocol or a replacement for
the Dillo plugin's `hyperdrive` + `hyperdiscovery` path.
## What Beaker separates
1. **Name resolution.** URLs are parsed first. A 64-character key is used
directly; a domain requires a resolver. Beaker has a `dat-dns` based
resolver and local name/key maps. Its v1.1 HyperDNS implementation leaves
the remote DNS logic disabled, and its `dat://` handler calls a separate
`dat-dns` resolver. Name resolution therefore belongs above transport and
should return the archive key before opening feeds.
2. **Drive sessions and persistence.** Beaker's later stack talks to a
long-lived Hyperspace process through `hyperdrive-daemon-client`. Sessions
are keyed by archive key plus version and read/write mode. The daemon owns
the storage and network handles; Beaker expires idle sessions and reconnects
them if the daemon restarts.
3. **Announce versus lookup.** Beaker explicitly configures a drive with
`announce: true, lookup: true` when it should be hosted and discover peers.
Merely opening a drive and resolving its key are separate operations.
4. **Archive reads.** The newer `hyper://` handler resolves the host to a key,
opens/loads the drive (which starts network lookup), checks out an optional
version, reads its manifest, resolves the path, and streams the selected
file from the drive. It supports `GET`/`HEAD`, byte ranges, redirects for
directory URLs without a trailing slash, manifest MIME overrides, and
content sniffing. A drive with version zero becomes a clear “no peers found
yet” response, so lookup-in-progress is distinct from a missing path.
The separate legacy `dat://` handler resolves through `dat-dns`, runs the
legacy conversion/download tool into temporary storage, and then serves
that filesystem through the same path resolver. That conversion route is
materially different from the native Hyperdrive read path.
5. **Operational visibility.** Beaker has separate debug categories for the
application, Dat, DHT discovery, DNS discovery, and the Hypercore protocol.
That separation makes it possible to tell whether a failure is lookup,
connection, replication, or file resolution.
## What applies to this C implementation
- Preserve the existing `NETWORK_PROTOCOL` switch and `cesspool` spelling.
The default Cesspool build uses the V7 message format. `hypercore` selects
the old Dillo dependency's V6 channels, nonce-bearing feed handshake, and
XSalsa20 stream encryption. The `CSP0` UDP discovery packet format is still
separate from the old JS `hyperdiscovery` format.
- Keep hostname resolution, discovery, authenticated peer sessions, feed
replication, and browser path handling as separate layers. The Dillo DPI
helper already consumes the URL/path and local archive layers; its cache
lookup is not yet a remote archive loader.
- The archive session joins a key's topic, consumes topic-tagged Cesspool
announcements, carries peer feed channels, exchanges HAVE/WANT state,
requests blocks, and verifies/forwards signed proofs. The UDP discovery
packet format still needs old JS `hyperdiscovery` interoperability work;
Noise authentication and socket-to-session transport integration are
separate networking layers.
- Beaker's daemon restart/session handling suggests a lifecycle boundary for
the C version: one archive session owns the discovery membership, peer set,
replication state, pending block requests, and close/retry behavior. Keep
this below both the CLI and Dillo DPI callers so they can share it.
- `dat-dns` names should be resolved as a mapping to a key, not treated as a
second archive identity. Cache the mapping separately from archive content;
keep the resolved key visible for debugging and canonical `dat://<key>`
links.
- `dat_dns_resolve` follows the old `dat-dns` URL-to-key contract. The bundled
Dillo plugin disables DNS-over-HTTPS and resolves through HTTPS
`/.well-known/dat`; the C resolver follows that same path after checking its
local alias map. It does not call the operating system's DNS TXT resolver.
The record contains a `dat://<64-hex-key>` line and optional `TTL=<seconds>`.
This lookup returns a key only; archive discovery and transfer remain
separate Cesspool/session operations.
## Suggested implementation order
1. Define an archive network-session API with explicit `open`, `announce`,
`lookup`, `peer-up`, `peer-down`, `request-block`, `block-arrived`, and
`close` transitions. Keep transport details behind the existing `cesspool`
boundary.
2. Make one peer connection carry a real authenticated protocol handshake and
identify the metadata and content feeds. Do not start with browser DNS or
directory listing behavior.
3. Connect remote metadata block requests to feed verification and archive
reads. A cold `dat://<key>/index.html` should be able to wait for and serve
the requested bytes, while earlier verified blocks remain cached.
4. Add retry, peer removal, and session cleanup. Expose peer and missing-block
state through a diagnostic command before adding a dashboard.
5. Add name resolution and any Beaker-style convenience behaviors only after
key-addressed remote reads work.
## Source references
- Beaker 1.1.0 `app/package.json`: `dat-dns`, `discovery-swarm`, `hyperspace`,
`hyperdrive-daemon-client`, and `pauls-dat-api2` dependencies.
- `app/bg/hyper/daemon.js`: daemon setup/reconnect, per-drive sessions, peer
addresses, and session cleanup.
- `app/bg/hyper/drives.js`: key/domain resolution, network configuration,
drive loading, manifest reads, and version checkouts.
- `app/bg/hyper/dns.js`: local name/key map; remote DNS implementation is
commented out in this archived revision.
- `app/bg/protocols/dat.js`: legacy `dat://` URL host resolution, local
legacy download/conversion to temporary storage, path resolution, MIME
selection, and response handling.
- `app/bg/protocols/hyper.js`: native Hyperdrive network open, historic
checkout, path resolution, no-peer state, range reads, and browser response
behavior.
- Beaker's repository README identifies version 1.1.0 as the last release and
the repository as archived; its debug namespaces include `dat`,
`bittorrent-dht`, `dns-discovery`, and `hypercore-protocol`.
docs/cable-interop.md
+154
View File
@@ -0,0 +1,154 @@
# Cable and Cabal interoperability notes
This document describes the **test-only Cabal/Hypercore interoperability
harness** under `tests/interop/cabal/`. It is kept to compare historical
behavior and is excluded from production builds and runtime paths.
The production `cable` and `cable-thang` clients are C/S7 applications:
Cesspool handles discovery and transport, Sleep handles persistent local
feeds, and S7 represents room events as S-expressions. Cabal CLI and
Hypercore are test targets, not production dependencies.
## Which client to run
The working historical Cabal terminal client is `cabal-cli`. To join a room,
run:
```sh
cabal-cli cabal://<room-key>
```
The `cable` executable also supports local Sleep feed and Cesspool bootstrap
commands. The native Cable chat client is `cable-thang`, using a terminal-style
SDL2 layout inspired by Cabal's TUI. Build with `make ENABLE_SDL2=1`; create an
address in the terminal with `build/linux/cable --new`, then launch the GUI with
`build/linux/cable-thang <cable://address>`. Use `/join NAME`, `/nick NAME`,
`Ctrl+N`, and `Ctrl+P` in the client.
## Cabal behaviors being ported
The Cabal CLI's `/whoami` (also `/key`) displays the local user key, while
`/nick` changes that user's display name. Those are different things: the
user's public key is the stable identity within that room; the nickname is a
readable label published as an `about` record in that user's room feed.
The CLI's usage and commands are listed in the
[cabal-cli README](https://github.com/cabal-club/cabal-cli#commands). The
[cabal-core API](https://github.com/cabal-club/cabal-core#cabalgetlocalkeycb)
documents `getLocalKey()` as the local user's key and documents messages as
records published to user feeds. Cabal creates its local writer feed inside
each room's storage, so a user's key is room-scoped, stable for that room, and
can differ in another room. A room key and its optional `?admin=<key>`
moderation key are separate from each user's feed key. Cabal text records use
`chat/text` with channel and text fields.
## Dat-era Cabal reference
Cabal Core 5.0.1 is a concrete older reference candidate: it was published about
seven years ago, and a 2019 Cabal walkthrough describes the chat as running
over Dat (the walkthrough was later updated for Cabal Core 5.0). The bundled
Dillo Dat tree supplies a nearby protocol reference with Hypercore 7.7.1,
hypercore-protocol 6.12.0, Hyperdrive 9.16.0, and hyperdiscovery 7.1.1. Use
those versions to guide backwards-adaptation; do not treat current Cabal Core
16 or its Hyper:// networking as the target.
The repeatable JS behavior tests remain useful for room identity, nicknames,
channels, and record shapes. To run them against the historical package, point
CABAL_JS_MODULE and CABAL_CLIENT_MODULE at a Cabal Core 5.0.x install, then run
make test-cabal-interop.
## Current C groundwork
- `cable address` accepts room keys with `cabal://`, `cable://`, or
`hyper://`, including a `?admin=<64 hex>` parameter, and prints canonical
`cable://` room addresses while retaining the admin key separately.
- `cable whoami <room-address>` and dat-thang expose a persistent local user
key as a `cable://<64 hex>` user address. Each room gets its own 64-byte
Ed25519 secret at `~/.cable/rooms/<room-key>/user-feed.key`, with owner-only
permissions. The key is stable in that room across restarts and distinct
from the room and admin keys.
- `hyper://` parsing only aliases the key representation; production Cable
does not implement the newer Hypercore stack.
- The library defines a bounded Cable text-message envelope and libsodium
sealed-box encrypt/decrypt helpers.
- dat-thang's left panel is a peer list again. Double-clicking a discovered
peer opens a direct-message pane; messages use the C-only sealed Cesspool
record and can be exchanged between Cable-enabled dat-thang clients.
- `cable-thang` is a native C/S7 room client. Cesspool handles discovery and
authenticated transport, Sleep persists local feeds, and S7 builds signed
room-event S-expressions. The room UI supports room creation, named
channels, nickname changes, and channel cycling. This path is Cable's own
protocol and does not claim Cabal wire interoperability.
### Hypercore feed storage check
The test-only Cable compatibility writer occupies the `room/0` slot used by Cabal Core 5's
`multifeed`, exposes the expected `localname`, and shares the Hypercore tree,
signature, bitfield, key, and data files. Run
`tests/interop/dat/cable_hypercore_compat.ps1` on Windows with WSL and Node installed. It
writes Cabal-shaped nick and chat JSON blocks from C, opens and verifies them
with the bundled Hypercore 7.7.1, appends a JavaScript block, then confirms C
reads that appended block. This verifies the standalone C feed format against
the legacy Hypercore fixture. It is separate from the native Sleep feeds used
by cable-thang.
## Side-by-side regression harness
`tests/interop/cabal/cabal_cable_side_by_side.js` creates a Cabal JS room, publishes a nick
and `chat/text` entry, and verifies that a second Cabal JS instance receives
both through Cabal's live replication stream. It then gives Cable that exact
room address, including `?admin=`, checks address normalization, and checks
that Cable's local user address persists in the room and changes for a second
room. This covers address normalization and the CLI's local identity utility;
it does not exercise cable-thang's network session.
`tests/interop/cabal/cabal_cli_features.js` uses cabal-cli's own `cabal-client` API to create
a room, execute `/nick` and `/join`, traverse joined channels using the same
channel ordering as the CLI's `Ctrl+N` handler, and exchange chat/text entries
between two independently opened Cabal clients over live Hypercore replication.
The C SDL live harness is `tests/interop/cabal/cable_js_c_live.js`; it launches one Cabal
Core JavaScript peer and one headless SDL/C client, then checks peer discovery
and chat in both directions through the Cabal Hypercore feeds. Run it with
`make test-cable-live ENABLE_SDL2=1` after building SDL2 support and installing
Node.js and Cabal Core. The test uses a local Hyperswarm DHT bootstrap node and
separate client UDP ports.
Run it once the historical JS `cabal-core` package and Node are available. The
test invokes Cabal JS directly; the UI command for people is `cabal-cli`:
```sh
make test-cabal-interop CABAL_JS_MODULE=/path/to/cabal-cli/node_modules/cabal-core
make test-cabal-interop CABAL_CLIENT_MODULE=/path/to/cabal-cli/node_modules/cabal-client
```
On Windows with Cable built in WSL, run the same harness with Windows Node and
the installed Cabal package, routing each Cable CLI call through WSL:
```powershell
$env:CABAL_JS_MODULE = "$env:APPDATA\npm\node_modules\cabal-cli\node_modules\cabal-core"
$env:CABLE_BIN = "$PWD\build\cable"
$env:CABLE_RUNNER = 'wsl'
node tests/interop/cabal/cabal_cable_side_by_side.js
```
The harness maps its temporary Cable home directory into WSL for this mode.
This is useful when the WSL distribution has no Node installation; no second
Node runtime is needed.
The normal `make test` runs this harness when it can resolve `cabal-core`; it
prints `SKIP` when the configured Node runtime cannot resolve the optional JS
package. A standalone `make test-cabal-interop` reports the resolution error
directly.
## Compatibility boundary
The upstream Cabal components documented here use Hypercore feed replication
and their own Cabal database/index conventions. Cable's current GUI messages
use a Cesspool transport record and do not enter Cabal feeds, so they are not
yet readable by Cabal JS clients. `/nick` in Cabal appends a `{ type: "about",
content: { name: nick } }` record to the user's room feed; the public user-feed
key remains its unique identity. Cable has not yet implemented that nickname
record in the Cabal message schema.
The next compatibility layer needs the Hypercore transport/session selected
for the deployed Cabal version, feed discovery, Cabal's multiwriter user-feed
model, and its JSON message/index schema. Dat archive wire identifiers and
`dat://` handling remain independent.
docs/cable-live-streaming.md
+60
View File
@@ -0,0 +1,60 @@
# Cable live media
Cable live media travels beside Cable room chat and Dat archives. A live
session is not an archive, does not append to a user's permanent chat feed, and does not
advance a Dat version. It is sent as volatile Cable records over authenticated
Cesspool/uTP peer sessions. The current audio path fans packets out from the
broadcaster to authenticated room peers; multi-hop relay is not yet wired.
## Audio, first slice
`cable-thang`'s `BROADCAST` button opens the default SDL microphone as 48 kHz,
mono, signed 16-bit PCM. FFmpeg's AAC-LC encoder converts 1,024-sample frames
to 96 kbps AAC and wraps each access unit in an ADTS header. The `cable-live`
packet header carries a random session id, sequence number, GUI timestamp,
media kind, and bounded payload. The C viewer validates packets, decodes AAC,
places PCM in a half-second maximum SDL playback queue, and plots those decoded
samples in the waveform panel. `Ctrl+Shift+A` toggles the local broadcast.
Received room audio begins playing as soon as its start packet is seen.
Cable audio and video are experimental and disabled by default. A normal SDL
build has no FFmpeg dependency. Enable both together with
`make ENABLE_SDL2=1 ENABLE_CABLE_MEDIA=1`; the makefile requires compatible
FFmpeg 5.0+ development files (`libavcodec`, `libavformat`, `libavutil`,
`libswresample`, and `libswscale`). To build the SDL clients without compiling
either media feature, use `make ENABLE_SDL2=1 ENABLE_CABLE_MEDIA=0`.
On Debian or Ubuntu, install the development headers and enable the feature:
```sh
sudo apt install libavcodec-dev libavformat-dev libavutil-dev \
libswresample-dev libswscale-dev
make ENABLE_SDL2=1 ENABLE_CABLE_MEDIA=1
```
With media enabled, the Cable video pane can listen to a local OBS MPEG-TS
output. Click `OBS INPUT` or press `Ctrl+Shift+V`; Cable listens on
`udp://127.0.0.1:5000`, decodes the video on a
worker thread, scales the local preview to at most 640x360, and displays the
latest frame. New frames are reduced to 160x90 JPEG and sent in 4,000-byte
fragments over authenticated Cesspool/uTP sessions; receivers reassemble and
decode them in the video pane. Transmission is capped at five frames per
second. In OBS, add a Custom FFmpeg Output with container `mpegts`, video
encoder `libx264`, and output URL `udp://127.0.0.1:5000?pkt_size=1316`. OBS
input provides local preview and direct broadcast to authenticated peers.
Camera capture without OBS and multi-hop forwarding are not implemented.
## Storage boundary
No media packets are written to `.dat`, test-only Cabal JSON chat feeds, or the
versioned archive metadata/content feeds. Closing the room discards the
decoder and playback queue. This avoids turning a live broadcast into
permanent archive history. Hypervision's
historic approach chunked WebM media and appended the chunks to a dedicated
Hypercore feed. That is a useful precedent for chunking and peer redistribution,
but an append-only archive is not the right retention policy for this live
session path.
The packet kinds cover audio and video start/frame/stop. Direct camera capture,
multi-hop relay, remote stream selection, and adaptive bitrate are follow-on
work.
docs/cable-thang.1
+124
View File
@@ -0,0 +1,124 @@
.TH CABLE-THANG 1 "October 2026" "cable-thang" "User Commands"
.SH NAME
cable-thang \- SDL2 Cable room chat client
.SH SYNOPSIS
.B cable-thang
.RB [ cable-address ]
.RB [ --offline ]
.br
.B cable-thang --help
.SH DESCRIPTION
.B cable-thang
is the C and SDL2 room chat client. Create a new room address in the terminal
with
.BR cable --new ,
then launch the client with the printed address as its positional argument:
.B cable-thang cable://address .
Launching
.B cable-thang
without an address opens its SDL room window; choose CREATE ROOM to create and
enter a new room, or paste a
.B cable://
or
.B cabal://
room address there and press Enter to join it.
Cesspool handles discovery and authenticated peer transport, Sleep stores
persistent local feeds, and S7 constructs room records as S-expressions. Its
fixed-cell, terminal-style layout
shows joined channels, room messages, local nickname, room address, and
authenticated peers. It joins a room by address.
.PP
Click PLAY AUDIO to start sending live room audio; the button changes to STOP
AUDIO while active. Ctrl+Shift+A toggles the same control. The first media
path captures the default microphone and sends 96 kbps AAC-LC. Incoming audio
is decoded and played in the live waveform panel. Media packets are relayed
through Cable peers without being appended to chat history or Dat archives.
See the cable-live-streaming document for the packet and storage boundary.
.PP
Click PLAY VIDEO to start sending the local OBS MPEG-TS input at
udp://127.0.0.1:5000; the button changes to STOP VIDEO while active. Ctrl+Shift+V
toggles the same control. Configure OBS Custom
FFmpeg Output with MPEG-TS, libx264, and that UDP URL. This previews the local
source and sends 160x90 JPEG frames to authenticated room peers over Cesspool.
.PP
Room creation stores the room signing key and local user's persistent key in
.IR ~/.cable/rooms/<room-key>/ .
The room address is the same 32-byte key rendered as
.BR cable:// .
Joined channels and the local nickname are stored in the room's profile.
.PP
Without a configured Cesspool rendezvous, the room broadcasts discovery on
the local network automatically and shows LAN DISCOVERY. Messages are kept
in the local chat until a peer connects. Local network broadcasts do not cross
routers. For discovery over the Internet, a reachable Cesspool rendezvous node
is required; configure one with
.B cable bootstrap add host:port
or set
.BR CABLE_BOOTSTRAP .
If the local UDP network setup itself fails, the SDL2 client still opens and
reports the error in its status line. Use
.B --offline
to deliberately skip network startup.
.PP
The production client is C and S7. Dat archives and Cable rooms both use
Cesspool, with separate topics and record formats. Cabal CLI and Hypercore
interoperability helpers remain under the test tree and are not launched or
packaged by the production client.
.SH COMMANDS
.TP
.B /join \fIname\fR
Join or create a channel and announce the channel to connected Cable peers.
.TP
.B /nick \fIname\fR
Change the local nickname and announce it to connected Cable peers.
.TP
.B /channels
Show the number of joined channels in the status line.
.TP
.B /help
Show the available commands and key bindings in the status line.
.SH KEY BINDINGS
.TP
.B Ctrl+N
Select the next joined channel.
.TP
.B Ctrl+P
Select the previous joined channel.
.TP
.B Ctrl+C
Copy the room address to the clipboard.
.TP
.B Enter
Send a message to connected room peers, or execute a slash command.
.TP
.B Escape
Close the client.
.SH ENVIRONMENT
.TP
.B DDAT_BOOTSTRAP
Comma- or semicolon-separated Cesspool rendezvous endpoints shared by Dat and
Cable clients.
.TP
.B CABLE_BOOTSTRAP
Comma- or semicolon-separated Cesspool rendezvous endpoints in host:port form.
.TP
.B DDAT_PORT
Local UDP listening port for Dat archive Cesspool discovery.
.TP
.B cable bootstrap add host:port
Persist a Cesspool rendezvous endpoint in
.IR ~/.config/dat-gpt/bootstrap .
Use cable bootstrap list to inspect the shared Dat/Cable list.
.TP
.B cable bootstrap serve host port
Run a Cesspool rendezvous node in the foreground. It must be reachable over
UDP at the advertised IPv4 address and port.
.SH EXAMPLES
.nf
cable --new
cable-thang cabal://0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
cable-thang cable://0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef --offline
.fi
.SH SEE ALSO
.BR cabal-cli (1),
.BR cable (1)
docs/cable.1
+114
View File
@@ -0,0 +1,114 @@
.TH CABLE 1 "October 2026" "cable" "User Commands"
.SH NAME
cable \- Cabal/Cable address, feed, and bootstrap utility
.SH SYNOPSIS
.B cable --new
.br
.B cable address
.I address
.br
.B cable whoami
.I room-address
.br
.B cable bootstrap add
.I host:port
.br
.B cable bootstrap list
.br
.B cable bootstrap serve
.I public-ipv4-host udp-port
.br
.B cable nick
.I room-address name
.br
.B cable publish
.I room-address channel text
.br
.B cable feed
.I room-address
.br
.B cable --version
.SH DESCRIPTION
.B cable
provides address, local feed, and bootstrap utilities. Its SDL2 chat client is
.BR cable-thang (1).
The CLI normalizes legacy
.B cabal://
and
.B cable://
room-address forms to canonical
.B cable://
addresses, preserving an optional
.B ?admin=\fI64-hex-key\fR
room moderation key. Raw 64-digit hexadecimal keys are accepted.
.PP
The room UI uses embedded S7 and S-expressions for its C-facing data model.
Cesspool handles peer discovery and transport, Sleep stores persistent local
feeds, and S7 represents room records as S-expressions. Hypercore and Cabal
CLI interoperability tools are test-only and are not production dependencies.
.PP
The historical JavaScript terminal client remains available as
.BR cabal-cli .
.SH COMMANDS
.TP
.B --new
Create a new Cabal room, save its Ed25519 room signing key with mode 0600
under
.IR ~/.cable/rooms/<room-key>/room-secret.key ,
and print its reusable
.B cable://
address. Pass that address to
.BR cable-thang (1)
on this or another client.
.TP
.BI address " address"
Validate a Cabal/Cable/Hyper room address and print the key as
.B cable://
plus 64 lowercase hexadecimal digits, preserving any admin key.
.TP
.BI whoami " room-address"
Print the persistent local C-side user's
.B cable://
address in the selected room, creating that room's local keypair on first use.
.TP
.BI bootstrap " add host:port"
Persist a Cesspool rendezvous endpoint in
.IR ~/.config/dat-gpt/bootstrap .
.TP
.B bootstrap list
List configured Cesspool rendezvous endpoints.
.TP
.BI bootstrap " serve public-ipv4-host udp-port"
Run the legacy Cesspool rendezvous server in the foreground. For a persistent
node with a stable identity, service unit, and operator guidance, use
.BR cesspool-bootstrap (1).
The UDP port still must be reachable through the host firewall and router.
Clients can configure its endpoint with
.B cable bootstrap add host:port
or CABLE_BOOTSTRAP.
.TP
.BI nick " room-address name"
Append a local S-expression `about` record to the room's Sleep feed. This feed
is experimental and is not a Cabal Hypercore feed used by
.BR cable-thang .
.TP
.BI publish " room-address channel text"
Append a local C-side feed chat record.
.TP
.BI feed " room-address"
Print local C-side feed records.
.TP
.B --version
Print the Cable version.
.SH EXAMPLES
.nf
cable --new
cable address cabal://0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
cable-thang cable://0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
cable bootstrap add bootstrap.example.net:49737
cable bootstrap list
cable-thang cable://0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
.fi
.SH SEE ALSO
.BR cabal-cli (1),
.BR cable-thang (1)
docs/cesspool-bootstrap.1
+82
View File
@@ -0,0 +1,82 @@
.TH CESSPOOL-BOOTSTRAP 1 "October 2026" "dat-gpt" "User Commands"
.SH NAME
cesspool-bootstrap \- dedicated Cesspool rendezvous node
.SH SYNOPSIS
.B cesspool-bootstrap
.RI [ options ]
.br
.B cesspool-bootstrap --help
.SH DESCRIPTION
.B cesspool-bootstrap
runs a Cesspool discovery node in the foreground. It answers discovery
lookups and tracks peer announcements; it does not relay chat, archive blocks,
or other peer payloads. Leave it running on a machine that should provide
discovery to clients on other networks.
.PP
At startup it reports its bound UDP port and prints the Windows Firewall rule
for that port. Use
.B --advertise HOST:PORT
to print the externally reachable endpoint for client setup. It does not
query a third-party service to discover the host's public address.
.PP
The node creates and reuses a persistent Cesspool identity at
.IR state-dir/identity.key .
Keep the state directory across restarts and protect it from other users.
Losing it creates a new node identity, but does not change archive or room
addresses.
.SH OPTIONS
.TP
.BI --bind " IPv4"
Local IPv4 interface to bind. Default: 0.0.0.0.
.TP
.BI --port " UDP-PORT"
UDP listen port. Default: 3282.
.TP
.BI --advertise " HOST:PORT"
Print the externally reachable endpoint in client setup commands. The
endpoint must resolve to this node and accept inbound UDP on the selected port.
.TP
.BI --state-dir " DIR"
Directory for the persistent node identity. Default on Windows:
.IR %LOCALAPPDATA%/dat-gpt/cesspool-bootstrap-state ;
on GNU/Linux:
.IR $XDG_STATE_HOME/dat-gpt/cesspool-bootstrap ,
or
.IR $HOME/.local/state/dat-gpt/cesspool-bootstrap
when XDG_STATE_HOME is unset.
The directory is created if needed.
.TP
.B --help
Show usage.
.SH RUNNING A NODE
Start the executable directly:
.PP
.nf
cesspool-bootstrap --bind 0.0.0.0 --port 3282
.fi
.PP
Keep that process running. Allow inbound UDP on port 3282 in the host
firewall. If the machine is behind a router or cloud firewall, forward the
same UDP port to it there. The program prints a Windows Firewall command;
other firewalls need an equivalent UDP allow rule. A public hostname can be
used with
.B --advertise
when one is available. Clients set the printed endpoint in Settings, or run
.B dat bootstrap set HOST:PORT
or
.BR "cable bootstrap set HOST:PORT" .
.PP
The periodic status line's peer count is the number of distinct Cesspool
identities with at least one current announcement. The topic count is the
number of distinct announced discovery keys. One identity can announce many
topics; a Cesspool identity is a network key, not necessarily a human user.
.SH EXAMPLE
.nf
cesspool-bootstrap --bind 0.0.0.0 --port 3282 \\
--state-dir cesspool-bootstrap-state \\
--advertise bootstrap.example.net:3282
.fi
.SH SEE ALSO
.BR dat (1),
.BR cable (1),
.BR cable-thang (1)
docs/dat-paper.pdf
BIN
View File
Binary file not shown.
docs/dat-thang.1
+188
View File
@@ -0,0 +1,188 @@
.TH DAT-THANG 1 "October 2026" "dat-thang v0.0.002" "User Commands"
.SH NAME
dat-thang \- SDL2 local archive and peer discovery client
.SH SYNOPSIS
.B dat-thang
.RI [ --archive DIRECTORY ]
.RI [ --site HOSTNAME ]
.RI [ --offline ]
.br
.B dat-thang --version
.SH DESCRIPTION
dat-thang is the SDL2 graphical client for browsing a local filesystem directory,
explicitly loading or creating Dat archives, and controlling the client's online
discovery state. The top-left switch applies to the whole client. OFF leaves all
loaded archive topics, unannounces when possible, closes the network socket, and
clears peer state. ON connects the client and announces archives in its loaded
archive list. Selecting an archive or browsing a directory does not toggle the
client's network state or create an archive.
.PP
Version v0.0.002 is a prototype in pre-alpha.
.PP
The pane to the right of the Merkle graph navigates the local filesystem.
Click a directory to enter it; navigation alone never creates or shares an
archive. The bottom bar displays the selected filename and directory. Use
F9 or IMPORT DAT to load an existing archive from the selected directory, or
CREATE NEW DAT to deliberately create one. Loaded archives are remembered in
the archive switcher and shared whenever the client is online. Secret signing keys are stored under
.B $HOME/.dat/secret_keys/
using Dat Desktop's sharded discovery-key paths, with one raw secret file per
feed. See the installed key-management documentation.
.PP
Click the local SITE field while an archive is loaded to assign that archive a
dat-dns hostname. Type the hostname and press Return to save a local alias; this
client resolves it immediately. The confirmation dialog copies the corresponding
dat-dns value to the clipboard. Public Dillo-compatible resolution uses the
hostname's HTTPS /.well-known/dat response, whose first line is dat:// followed
by the 64-digit archive key and whose optional second line is TTL=<seconds>.
Escape cancels hostname editing. The site field can also display the
hostname supplied with --site. Archive displays the selected archive key in
hexadecimal or compact Base32768 form; click the format control to switch and
COPY to copy its dat:// address. Version shows the active
selected metadata-feed version and latest version; click the arrows to check out
another recorded version. The matching Merkle view shows the feed's block count
and a tree view of its active version.
The peer pane distinguishes discovered peer announcements from authenticated
connected peers with a marker on each buddy-list row. Double-click a peer to
open Cable-thang's encrypted direct-message pane. Copy the desired dat:// link
from the local ARCHIVE field, paste it into chat with Ctrl+V, and press Enter
to send it. Click a received dat:// message to copy it, then click the
right-hand ARCHIVE field and press Ctrl+V followed by Enter to begin importing
into the currently selected local directory. The
incoming archive appears in the lower transfer table while peers are found and
metadata downloads. The current Cesspool message record is Cable-specific and
is not Cabal feed interoperability.
.PP
By default dat-thang listens on UDP 3282 and cable-thang listens on UDP 3000,
the stable ports used by Cesspool LAN discovery. If a preferred port is busy,
the client chooses another known Cesspool port. Set DDAT_PORT to pin the local
port. With no configured rendezvous, discovery is limited to the local network;
across networks, both clients need a reachable Cesspool node.
The YOU panel shows the selected room's local Cable feed address. Like Cabal,
the key is scoped to that room's local feed and remains stable across restarts;
room and optional admin keys are separate. The CLI prints the full address with
.BR "cable whoami <room-address>" .
The panel also shows connected peers, discovered peers, and the total number of
loaded local directories available for sharing.
The lower table lists loaded archives as well as active transfers. Its progress
column is a bar: local archives are complete, while transfer bars count received
blocks against the peer's advertised feed lengths. Received blocks are checked
against SLEEP proofs and saved in per-feed .remote caches for later sessions.
For each local archive, TITLE and DESCRIPTION come from its root `dat.scm`
S-expression. Missing fields fall back to the directory name and archive key.
.SH OPTIONS
.TP
.B --version, -v
Print \fBdat-thang v0.0.002 (prototype, pre-alpha)\fR and exit.
.TP
.B --archive DIRECTORY
Start in DIRECTORY. If it contains an existing .dat archive, load that archive;
otherwise browse the directory without creating or sharing an archive. Defaults
to the current directory.
.TP
.B --site HOSTNAME
Display HOSTNAME in the site field.
.TP
.B --offline
Start with peer discovery disabled, even when the directory already has an
archive.
.TP
.B --smoke
Render one frame and save build/linux/dat-thang-smoke.bmp on GNU/Linux or
build/windows/dat-thang-smoke.bmp on Windows. Intended for headless
render checks.
.SH CONTROLS
.TP
.B F1
Show this manual page in the client. Use Up/Down or the mouse wheel to scroll;
press Escape to close it.
.TP
.B F9
Load an existing Dat archive from the selected directory. This does not create
an archive or start discovery; use ON to share it.
.TP
.B CREATE NEW DAT
Create a Dat archive for the selected local directory and import its current
files. Use ON to start discovery after creation.
.TP
.B IMPORT DAT
With no remote address entered, load an existing archive in the selected local
directory. To import a remote archive, enter a dat:// URL in the
right-hand SITE or ARCHIVE field, select an empty destination directory, then
click IMPORT DAT. Verified metadata and content blocks are cached below that
directory's .dat folder.
.TP
.B Escape
Toggle the client menu.
.TP
.B Mouse
Click OFF to disconnect all peers and go offline. Click ON to bring the client
online and announce loaded archives. With no loaded archives it still connects
to Cesspool without announcing a topic. Archive selection changes the local view.
Click entries in the right-hand pane to navigate
directories or select a filename. Click ARCHIVES above the pane to switch
between known local archives. The directory pane has its own scrollbar and
mouse-wheel scrolling. The bottom archive/transfer table includes known shared
archives and active transfers; its horizontal scrollbar can be dragged, and
the mouse wheel scrolls rows (Shift+wheel scrolls columns). Enter a remote dat:// address in the mirrored
SITE or ARCHIVE field before selecting IMPORT DAT. Drag transfer-table header dividers to resize
columns; use Shift+wheel over the table to scroll horizontally. The bottom
track indicates horizontal position. Right-click opens load, create, and
copy-address actions. COPY includes the selected archive version in its dat://
link.
.SH NETWORK
The client reads endpoints saved by \fBdat bootstrap add host:port\fR or
\fBdat bootstrap set host:port\fR from
$HOME/.config/dat-gpt/bootstrap and also accepts DDAT_BOOTSTRAP as a comma- or
semicolon-separated list. The Settings menu also edits this endpoint and
restarts discovery immediately. Run \fBcesspool-bootstrap\fR on the rendezvous
host with \fB--advertise HOST:PORT\fR to print the address clients should use.
It does not query an outside service for a public IP. Allow the chosen UDP
port through the host firewall and router.
This is Cesspool rendezvous for these C clients; it is not a BitTorrent DHT
node and cannot be substituted for the historical JavaScript Dat bootstrap
servers. OFF leaves every archive topic and closes the Cesspool socket. ON
starts archive discovery; the indicator turns green only after a Cesspool
response or connected peer.
.SH ENVIRONMENT
.TP
.B DDAT_BOOTSTRAP
Comma- or semicolon-separated Cesspool rendezvous endpoints, for example
127.0.0.1:3282.
Saved endpoints from dat bootstrap add are also loaded automatically.
.TP
.B DDAT_PORT
UDP listening port. If unset, Cesspool selects an available default port.
.TP
.B DAT_THANG_MANPAGE
Optional path to the dat-thang manual source displayed by F1.
.SH MENU
Escape opens the vertical Settings, Return, Exit menu. Settings contains
persistent controls for the default copied-address format, sonar audio, and
connecting on startup. Use Up/Down and Enter, or click an item. Return resumes
the client; Exit closes it. Escape from Settings returns to the menu.
.SH AUDIO
Click the speaker icon to play and send a two-second low square-wave cue.
Clicking the dolphin plays and sends a one-second arpeggiated chiptune cue.
The dolphin cue goes to connected authenticated peers. The speaker cue refreshes
discovery for the active archive and is sent to discovered peers as their secure
sessions become ready. Peers play received cues locally; the speaker response
causes the peer to return one dolphin chirp, which plays at the sender and marks
the peer found. The waveform viewer draws samples captured from the actual SDL
audio output.
.SH BUILD DATE
The header shows the program build date in Discordian format (ddate). The
calendar conversion is compiled directly into dat-thang; no external ddate
program or runtime lookup is needed on Windows or GNU/Linux.
.SH DILLO
To install the Dillo DPI helper for the current Dillo source, build with make, run make install using the
same PREFIX as Dillo, add proto.dat=dat/dat.dpi to ~/.dillo/dpidrc, and restart
Dillo. The helper first reads its local archive cache at
~/.dillo/dat/archives/XX/<key-tail>. On a cache miss, it can use the configured
Cesspool bootstrap to find a peer, authenticate the SLEEP session, and fetch
the archive metadata and content feeds into that cache. Start the bootstrap
before the clients so the first archive announcement is recorded. The Linux
side-by-side runbook uses the default GNU/Linux output path, build/linux, to
keep its objects separate from Windows artifacts in a shared checkout.
.SH SEE ALSO
ddat(1), dat(1)
docs/dat.1
+70 -2
View File
@@ -25,14 +25,82 @@
.Oc
.Sh DESCRIPTION
[WIP] dis dat mf thang
The C dat command creates and maintains local Dat archives. Archive metadata
is append-only, so each metadata entry advances the archive version and older
file contents remain addressable by their earlier version number.
.Pp
The status subcommand reports the archive key, current version, metadata entry
count, and latest live path counts. The files subcommand lists the paths at
the current version, or at a selected historical version.
.Pp
Canonical archive addresses use the historical 64-character hexadecimal key.
Base32768 is available for compact test addresses with Cm dat url --compact;
the parser accepts both forms. Secret key filenames and dat-dns records
continue to use hexadecimal keys.
.Pp
The library provides an archive session, Cesspool discovery, and a Hypercore
feed codec. The `hypercore` build selects the legacy V6 wire session. Discovery
and transport establishment are separate layers.
.Sh COMMANDS
.Bl -tag -width "checkout"
.It Cm create , init
Create an archive, write its title, description, and dat:// address as a
`dat.scm` S-expression, and print the address. New archives do not get a
`dat.json` sidecar. `--title` and `--description` set the presentation fields.
.It Cm share , sync
Import changed local paths into the archive.
.It Cm status
Show the archive key and metadata/version counts.
.It Cm files Op version
List the live files and directories at a version.
.It Cm log
Show appended metadata entries, including history and tombstones.
.It Cm cat Ar version path
Write a historical file's bytes to standard output.
.It Cm checkout Ar version destination
Export a historical snapshot to a directory.
.It Cm url Op version Op path
Print an archive URL, optionally pinned to a version and path.
.It Cm resolve Ar hostname
Resolve a dat-dns hostname through the legacy HTTPS /.well-known/dat response
and print its canonical dat:// key and TTL. This does not use the operating
system DNS TXT resolver.
.It Cm dns set Ar hostname archive-dir|dat://key
Create or replace this machine's local dat-dns name mapping. Names are stored
in Pa ~/.dat/dat-dns and used by resolve and the dat-thang SITE field. For
public Dillo-compatible resolution, publish an HTTPS /.well-known/dat file
whose first line is dat:// followed by the archive's 64-digit key and whose
optional second line is TTL=<seconds>.
.It Cm dns list
List local dat-dns name mappings.
.It Cm dns remove Ar hostname
Remove a local dat-dns name mapping.
.It Cm verify Op dir
Verify metadata and content feeds with the archive's public keys.
.It Cm bootstrap Op udp-port
Run a Cesspool rendezvous node on all local interfaces. The node prints the
DDAT_BOOTSTRAP setting participating C clients should use. The UDP port must
be reachable from those clients through the host firewall and router. Peer
records expire after ten minutes without an announce or lookup. This is the
Cesspool protocol used by dat-thang and cable-thang, not the historical
JavaScript Dat BitTorrent DHT bootstrap protocol.
.It Cm bootstrap add Ar host:port
Save a Cesspool rendezvous endpoint for dat-thang and cable-thang in the
per-user configuration file. It is used in addition to DDAT_BOOTSTRAP.
.It Cm bootstrap set Ar host:port
Replace the active rendezvous list with one Cesspool endpoint. The desktop
client exposes the same setting under Settings, so this command is only needed
for terminal-only setup.
.El
.Sh OPTIONS
.It Fl d
Debug
.Sh FILES
.Bl -tag -width -indent
.It Pa ~/.dat/secret_keys/
its there
Contains private feed secrets as raw Ed25519 secret bytes, indexed by
discovery key. See the key format and Dat Desktop compatibility notes in
the installed key-management.md documentation.
.Sh AUTHORS
"Dat" was first written about by Mathias Buus Madsen & Maxwell Ogden (c) 2015
.Nm
docs/ddat-gui.md
+126
View File
@@ -0,0 +1,126 @@
# `dat-thang` v0.0.002
This release is a prototype in pre-alpha.
Build with `make ENABLE_SDL2=1` (requires SDL2 and SDL2_ttf development files); the Dat GUI is `build/linux/dat-thang` and the
separate Cable room chat client is `build/linux/cable-thang`. `make
ENABLE_SDL2=1 install` installs both clients and their manuals. Run
`build/linux/dat-thang --archive DIRECTORY --site HOSTNAME` to begin in a local
directory. The initial directory defaults to `.`.
The GNU/Linux default build path is `build/linux`; the native Windows target
uses `build/windows`. Set `BUILD=...` only when you intentionally need another
isolated build tree.
Both interfaces are painted with SDL2 and intentionally resemble terminal
clients: fixed-width labels, high-contrast panes, keyboard shortcuts, and
terminal-style status text. They do not use Notcurses. For the existing
JavaScript Cabal terminal client, run `cabal-cli <room-address>`; `cable` is
only the C address and identity utility.
Both clients load the bundled `assets/fonts/MorePerfectDOSVGA.ttf`, verify its
family name is `More Perfect DOS VGA`, and render hard edged glyphs at 14 px
with SDL2_ttf. Installed clients load that same font from
`$(PREFIX)/share/datgpt/fonts/`; there is no alternate font setting.
The dat-thang speaker graphic is the original 84 by 84 pixel crop from the
provided mockup, stored as `assets/datthang-speaker.bmp`. Green pixels from the
mockup background are transparent in the GUI. Installed clients load it from
`$(PREFIX)/share/datgpt/icons/`.
The top-left OFF/ON switch controls the client network. OFF leaves every joined
topic, unannounces when possible, closes the Cesspool socket, and clears peer
state. ON connects and announces only archives already loaded in the archive
list; navigating the filesystem never creates or loads an archive. The status
indicator turns green after a Cesspool response or connected peer. F9 loads an
existing archive or imports the entered remote link. `--offline` starts with
network discovery disabled.
Run `build/linux/cesspool-bootstrap --port 3282 --advertise HOST:3282` on the
rendezvous host. It prints the endpoint you supplied and a Windows firewall
rule; it does not query an outside IP lookup service. Forward UDP 3282 through
the edge router or cloud firewall. In each GUI, open Escape > Settings
> Cesspool Node and enter that reachable `host:port`; saving it restarts
discovery. Terminal users can use `dat bootstrap set host:port` or `cable
bootstrap set host:port`. `CESSPOOL_BOOTSTRAP` and `DDAT_BOOTSTRAP` accept comma- or
semicolon-separated endpoints. `DDAT_PORT` selects each client's local UDP
port. With no explicit `DDAT_PORT`, dat-thang listens on UDP 3282 and
cable-thang on UDP 3000 so same-LAN discovery can reach both clients on their
stable ports. If the preferred port is busy, a client chooses another known
Cesspool port; set `DDAT_PORT` to pin a specific local port. The node keeps short-lived peer announcements and returns endpoints
announcing the same topic. This is Cesspool rendezvous for the C clients, not
the historical JavaScript Dat/Bittorrent DHT.
The pane to the right of the Merkle graph is a local filesystem browser. Click
a directory to navigate into and select it as the local directory. Click a file
to select its name. The bottom bar reports that filename and selected local
directory. `CREATE NEW DAT` opens or creates that directory's archive and begins
discovery when online. With no remote address entered, `IMPORT DAT` opens and
syncs the archive already present in the selected directory. To share an archive,
copy its address with the `COPY` control beside the local ARCHIVE field.
Double-click a peer in the buddy list to open private chat, paste the address
with Ctrl+V, and press Enter to send it. The recipient clicks the received
`dat://` message to copy its address, selects an empty destination directory,
clicks the right-hand ARCHIVE field, presses Ctrl+V, then Enter. Enter starts the remote import; an incoming
archive row appears in the transfer table as peer discovery and metadata download
begin. The mirrored SITE, ARCHIVE, and VERSION fields show its hostname, key,
path, and selected version. Received metadata and content blocks are
proof-verified and cached under that directory's `.dat` folder. The client
learns the content-feed key from the authenticated, verified metadata header.
The status beside the remote fields reports address, discovery, and feed state.
Escape opens the vertical Settings, Return, Exit menu. Settings persists the
default address format, sonar audio, and online-at-startup preferences; use
Up/Down and Enter or click a row.
The table columns can be resized by dragging their header dividers;
the wheel scrolls horizontally over the transfer table. Its bottom track shows
the current horizontal position. The ARCHIVES control above the filesystem pane
switches among remembered shared directories; the list persists in
`$HOME/.dat/dat-thang-shares`. Right-click a transfer for pause/resume, cancel,
reveal destination, or copy address. Right-click elsewhere for load/share,
create, and copy-address actions.
For each locally shared archive, the table's TITLE and DESCRIPTION come from
the archive root's `dat.scm` S-expression, for example
`((title "dat-thang") (description "dat:// implemented over c & scheme"))`.
The client reads this file as Scheme data; missing fields fall back to the
directory name and archive address.
The client passes discovered Cesspool peers to the archive session, opens uTP,
authenticates each connection with a signed nonce exchange bound to the peer
key, then passes session frames through the feed handshake and replication
code. The peer pane distinguishes discovered announcements from authenticated
connections. The lower table lists authenticated archive sessions and their
replication state. Upload/download readouts count session frames, and the top
diamonds pulse on payload activity. The center hexagon is red offline, amber
online without a connected peer, and green when connected. Awkward Moments,
Share Guy, and Rank remain planned placeholders. Received blocks are verified
against their SLEEP proofs and stored in per-feed `.remote` caches, then
reloaded when the next session starts.
The original dolphin-in-a-porthole and radar/speaker art from the mockup are
embedded in the client. The dolphin sits at the upper right and sends a
one-second arpeggiated chiptune sonar cue to authenticated connected peers. The
speaker sits at the lower right of the YOU panel and plays a two-second low
square-wave cue; while online on an archive topic, it refreshes discovery and
sends the cue to discovered peers as their authenticated sessions come up. A
receiving peer plays the speaker cue and returns one dolphin chirp; the sender
plays that echo and marks the peer found. The waveform panel plots samples
captured from SDL's audio output ring, so it shows the actual waveform being
played. Cues travel as a small Dat-thang UI record over the authenticated
Cesspool/uTP connection and never enter archive feeds.
F1 opens the client manual from the source tree or installed manpage. Up/Down
and the mouse wheel scroll it; Escape closes it. Escape toggles the menu when
the manual is closed. The source can be selected with `DAT_THANG_MANPAGE`.
The site field shows the explicit hostname supplied at launch or a local
`dat dns set` alias associated with the selected archive. Beaker's legacy
`dat:` handler resolves hostnames to keys before opening an archive, and its
`dat-dns` supports DNS-over-HTTPS TXT and `/.well-known/dat`; the bundled
Dillo plugin disables the first option and uses HTTPS `/.well-known/dat`.
The C resolver follows that Dillo behavior after checking local aliases; it
does not call the operating system DNS TXT resolver. The archive field defaults
to the historical 64-digit hexadecimal address; its format control switches to
compact Base32768, and COPY produces the selected format.
The build date uses the Discordian `ddate` calendar.
The Merkle panel is a shape preview driven by metadata entry count, not hashes
or proofs.
docs/dillo.md
+201
View File
@@ -0,0 +1,201 @@
# Dillo integration
Use the Dillo source maintained at <https://git.dillo-browser.org/dillo>.
The project also lists <https://codeberg.org/dillo/dillo> as an up-to-date
mirror; that mirror was used for the Linux run documented below.
The latest published release is 3.2.0; the current source tree identifies its
development version as 3.3.0. The old `dillo.org` host is not controlled by the
Dillo developers.
`make` builds `$(BUILD)/dat.dpi` (normally `build/linux/dat.dpi`), a native C Dillo
Plugin Interface (DPI) helper.
It uses Dillo's external DPIP protocol and does not link against Dillo internals
or depend on a particular browser ABI. The helper authenticates each accepted
DPIP connection with the `~/.dillo/dpid_comm_keys` value and handles `open_url`
and `DpiBye`. It follows the historical plugin lifecycle: `dpid` passes its
listening socket on stdin, the helper accepts a request connection, and closes
that connection after the response body. DPIP responses do not use HTTP status
lines or `Content-Length`; closing the request socket marks the end of the
page. Current Dillo
continues to load `.dpi` plugins and associate them with protocols in
`~/.dillo/dpidrc`.
Build and install the DPI with the same prefix used by Dillo. For the usual
`/usr/local` installation:
```sh
make
sudo make install
```
If Dillo uses a different plugin directory, set `PREFIX` and `DPIDIR` on both
commands. The default plugin path is `/usr/local/lib/dillo/dpi/dat/dat.dpi`.
The per-user `~/.dillo/dpidrc` must contain both the DPI root and this mapping
(Dillo does not merge `dpi_dir` from the system config into the per-user file):
```ini
dpi_dir=/usr/local/lib/dillo/dpi
proto.dat=dat/dat.dpi
```
Then restart Dillo (which restarts `dpid`). The plugin first checks Dillo's
local archive cache. On a cache miss it can join the archive's Cesspool topic,
authenticate a peer session, fetch the metadata and content feeds, and then
serve the requested path. Both Dillo's DPI and the seeding Dat-thang client
must use the same Cesspool bootstrap node. Configure that endpoint for the
environment that launches each program:
```sh
build/linux/dat bootstrap set 127.0.0.1:3282
```
The command stores the node in `~/.config/dat-gpt/bootstrap`. For a public
test, use the reachable `HOST:PORT` of the Cesspool node instead of the local
loopback address. The Cygwin Dillo and native Windows Dat-thang configurations
are separate when their `HOME` directories differ, so configure each using its
own `dat` executable/environment. The helper does not use Hyperswarm or
Hypercore; its remote feed transfer uses this project's Cesspool and SLEEP
session protocol.
Use the full key shown by Dat-thang, then enter `dat://<key>/` in Dillo. The
helper caches fetched data at `~/.dillo/dat/archives/XX/<remaining-key>` and
serves historical versions, `index.html`, `.html` fallback, explicit `//`
directory listings, parent-directory navigation, stat queries, and the MIME
rules used by the old `dillo-dat` plugin. Directory listings retain its
up/directory/file icons and sort directories first, then names in descending
order. The internal `dpi:/dat/` page lists archives opened by this DPI process,
shows title, description, and version from `dat.scm` (read as an S-expression
without evaluating archive contents), falls back to `CNAME`, and provides the
unswarm/reload and quit controls. Status messages identify name resolution,
archive retrieval, and file serving. If a fetch cannot complete, it returns a
text error page that distinguishes an unconfigured or unreachable node, no
peer announcing the archive key, and a peer transfer timeout.
Version pins use the metadata-entry count shown by `dat status`:
`dat://<key>+2/` reads the snapshot at version 2, while `dat://<key>/` follows
the latest version. Root navigation looks for `index.html`. `make test` checks
the archive behaviors and starts the helper with a real TCP listening socket,
then verifies response bytes and EOF framing over an accepted connection.
## Linux bootstrap, Dat-thang, and Dillo smoke run
Run these steps from the repository root after installing the build
dependencies in `INSTALL`. GNU/Linux objects and executables default to
`build/linux`, separate from the Windows build in `build/windows`. The bundled `site/`
archive already has an address and three recorded versions; this flow leaves it
unchanged.
Build and install the GUI plus the C Dillo helper:
```sh
./configure
make ENABLE_SDL2=1
sudo make ENABLE_SDL2=1 install
```
Merge these entries into `~/.dillo/dpidrc`, preserving any other DPI entries,
then restart Dillo so `dpid` reloads the mapping:
```ini
dpi_dir=/usr/local/lib/dillo/dpi
proto.dat=dat/dat.dpi
```
Use three terminals. Terminal 1 runs a local discovery node for this same-host
smoke run. Start it before either client so it receives the initial announce:
```sh
build/linux/cesspool-bootstrap --bind 127.0.0.1 --port 3282 \
--state-dir "$HOME/.local/state/dat-gpt/cesspool-bootstrap" \
--advertise 127.0.0.1:3282
```
Terminal 2 configures the client and starts the SDL client with the bundled
archive explicitly loaded. Leave its window open and online:
```sh
build/linux/dat bootstrap set 127.0.0.1:3282
DDAT_BOOTSTRAP=127.0.0.1:3282 DDAT_PORT=3000 \
build/linux/dat-thang --archive "$PWD/site"
```
Terminal 3 starts Dillo with an empty per-run cache, so the page must arrive
from the archive being announced by Dat-thang. The temporary home gives Dillo
its own fresh cache and DPID keys without disturbing the normal profile:
keep this shell open for the two `dilloc` version checks below.
```sh
dillo_home=$(mktemp -d)
mkdir -p "$dillo_home/.dillo"
cat > "$dillo_home/.dillo/dpidrc" <<'EOF'
dpi_dir=/usr/local/lib/dillo/dpi
proto.dat=dat/dat.dpi
EOF
archive=$(build/linux/dat url / "$PWD/site")
HOME="$dillo_home" DDAT_BOOTSTRAP=127.0.0.1:3282 dillo "$archive/" &
sleep 5
HOME="$dillo_home" dilloc title
HOME="$dillo_home" dilloc dump | grep -F 'The site is online'
```
The latest `site/` snapshot is version 3 and displays “Site is online.” Open
the version-1 pin in the existing Dillo window to verify historical navigation;
it predates `dat.scm` and `index.html` and shows the earlier archive listing:
```sh
historical=$(build/linux/dat url 1 / "$PWD/site")
HOME="$dillo_home" dilloc open "$historical/"
sleep 2
HOME="$dillo_home" dilloc url
HOME="$dillo_home" dilloc dump | grep -F 'dat-logo.png'
```
Return to the setup page at version 3:
```sh
latest=$(build/linux/dat url 3 / "$PWD/site")
HOME="$dillo_home" dilloc open "$latest/"
sleep 2
HOME="$dillo_home" dilloc url
HOME="$dillo_home" dilloc title
HOME="$dillo_home" dilloc dump | grep -F 'The site is online'
```
For another archive, replace `site` with its loaded local archive directory,
and choose a version from `build/linux/dat log DIRECTORY`. The same Cesspool node
must be reachable by both the Dat-thang GUI and Dillo helper. The bootstrap
process reports announced peer and topic counts. A successful run shows the
GUI online, a cold Dillo cache populated with `.dat/metadata.remote` and
`.dat/content.remote`, and the Dillo title `Site is online`. With this archive,
Dillo loads both `+1` (historical listing) and `+3` (the current setup page).
## Windows with Cygwin Dillo
Build the helper from the repository root in Cygwin Bash. The DPI uses the
normal Cygwin `gcc`, `make`, `libcurl-devel`, and `libsodium-devel` packages.
The helper links the bundled S7 reader so `dat.scm` metadata is parsed using the
project's Scheme representation; it never evaluates archive-supplied forms.
```sh
make build/linux/dat.dpi
install -d /usr/local/lib/dillo/dpi/dat
install -m 755 build/linux/dat.dpi /usr/local/lib/dillo/dpi/dat/dat.dpi.exe
```
Put both entries in `~/.dillo/dpidrc` (for this installation that is
`C:\cygwin64\home\Home\.dillo\dpidrc`):
```ini
dpi_dir=/usr/local/lib/dillo/dpi
proto.dat=dat/dat.dpi.exe
```
Restart Dillo so `dpid` reads the configuration. The `site/` folder in this
repository is the archive source; Dillo's cache is separate at
`~/.dillo/dat/archives/XX/<remaining-key>`. A seeded archive is browseable only
after its feed data is present in that cache or a live Cesspool fetch completes.
The setup page source is `site/index.html`. `tests/test_dpi.sh` checks the
historical and latest snapshots, S-expression metadata, listing, stat, and
dashboard output. The Windows end-to-end flow, including `+N` navigation across
multiple archives, has been confirmed by the user. The Linux cold-cache flow
above was also run against current Dillo source and the Cesspool bootstrap.
docs/key-management.md
+85
View File
@@ -0,0 +1,85 @@
# Archive addresses, keys, and write access
## ELI5
A Dat address is like a mailbox number that is also a read capability: anyone
who knows it can request the archive. The address is not the pen. The private
metadata signing key is the pen that lets a writer append new archive history.
Dat Desktop's `dat-node` exposes this distinction as `archive.writable`; its
file importer requires a writable archive. A reader can download and serve the
archive without being able to change its history.
The historical address
`dat://59d8099399510e23a3c693fb5887b7bba1586f4117f7ef1c5872622ada0c1181`
contains the public metadata key. It does not contain or reveal the private
signing key. Losing that secret does not make the archive unreadable; it means
you cannot write as its existing owner. Making a new keypair creates a different
archive address. Dat's original archive model is single-writer: a reader does
not gain write permission by downloading a copy.
## How Dat Desktop stores the secret
Dat Desktop's `dat-secret-storage` indexes each Hypercore feed's secret by its
**discovery key**, not by the `dat://` metadata public key. The discovery key
is BLAKE2b of `hypercore`, keyed by that feed's public key. The Unix path is
sharded:
```text
$HOME/.dat/secret_keys/<first-two-discovery-key-hex>/<remaining-discovery-key-hex>
```
The file is random-access storage for the feed's raw 64-byte Ed25519 secret.
The C client reads and writes that same file path and byte format. Dat Desktop
also uses a separate local owner marker to avoid competing local writers; the
C client does not currently maintain that marker. See the
[archived dat-secret-storage implementation](https://github.com/dat-ecosystem-archive/dat-secret-storage/blob/master/index.js)
and [dat-storage documentation](https://www.skypack.dev/view/dat-storage).
## What this C implementation does
The C implementation now uses the same per-feed discovery-key paths and raw
64-byte secret format. Both the metadata feed and content feed have their own
files:
```text
$HOME/.dat/secret_keys/<first-two-metadata-discovery-hex>/<remaining-metadata-discovery-hex>
$HOME/.dat/secret_keys/<first-two-content-discovery-hex>/<remaining-content-discovery-hex>
```
The public halves remain in `<archive>/.dat/metadata.key` and
`<archive>/.dat/content.key`. The C client validates each secret against its
matching public key before using it. This discovery-key layout and raw secret
file format are the sole supported secret store.
When the C client opens an archive, its public metadata remains readable
without local signing keys. Writing requires the matching metadata and content
keypairs in the C secret directory, plus normal operating-system write access
to the local archive directory. A local filesystem being writable does not
grant Dat write authority; the matching signing secrets are still required.
Conversely, having the secrets does not override read-only filesystem
permissions.
## Finding the historical key
The historical metadata secret's Dat Desktop path is derived from the address
above. Compute its `hypercore` discovery key, then use:
```text
$HOME/.dat/secret_keys/<first-two-metadata-discovery-hex>/<remaining-metadata-discovery-hex>
```
The matching content-feed secret is stored by its own content discovery key.
The C implementation searches both paths directly; it does not need a
conversion step for Dat Desktop's raw secret files. Preserve the original
`~/.dat/secret_keys` tree while testing access.
`dat keys export <archive-directory>` prints the two keypairs as hexadecimal;
treat that output as secret. `dat keys import <metadata-keypair-hex>
<content-keypair-hex> <archive-directory>` checks that both public halves match
the archive before saving their secret bytes in the shared format. Importing newly generated
keypairs cannot restore write access to an existing address.
For context, the archived [dat-node documentation](https://awesome.datproject.org/dat-node)
describes `archive.writable` and notes that importing files requires a writable
archive. The historical Dat model is single-writer; see the
[Dat security FAQ](https://dat-ecosystem.github.io/docs/docs/security-faq).
docs/libcess.md
+62
View File
@@ -0,0 +1,62 @@
# libcess
`libcess` is the Cesspool base used by both p2p applications:
```text
┌───────────┐
│ libcess │
│ UDP, peer │
│ discovery│
└─────┬─────┘
┌───────┴───────┐
┌────▼────┐ ┌────▼────┐
│ ddat/dat│ │ cable │
│ SLEEP │ │ Cabal │
│ archives│ │ rooms │
└─────────┘ └─────────┘
```
The package owns the UDP socket transport, Cesspool messages, bootstrap and
rendezvous client/server logic, peer/topic state, and its Ed25519 bootstrap
identity. It has no SLEEP, Dat, Cable, or JSON dependency. Dat and Cable link
the same `libcess.a` so a configured `NETWORK_PROTOCOL` selects one discovery
network for both applications. The protocol token is data in the root
`NETWORK_PROTOCOL` file for the full project; a standalone libcess build reads
`libcess/NETWORK_PROTOCOL`.
## Standalone build
From the project root:
```sh
make -C libcess
make -C libcess install PREFIX="$HOME/.local"
```
This builds `libcess/build/libcess.a` and `libcess/build/cesspool-bootstrap`.
Only a C99 compiler, GNU make, and libsodium development files are required.
The bootstrap executable links only libcess, libsodium, and the platform UDP
socket library; it does not need libcurl or the Dat application library.
Apps including `cesspool.h` and `cess_net.h` link `libcess.a` after their own
static archive, then link libsodium. The installed headers are under
`include/libcess/`.
## Bootstrap node
`cesspool-bootstrap` is the bootstrap service shipped with the library. It
tracks topic announcements and returns peer endpoints; archive and chat data
still travels between peers. The process must stay running and its chosen UDP
port must be reachable from clients. `--advertise HOST:PORT` prints the
externally reachable address; the tool does not call an outside IP lookup
service. Configure that address in `ddat` or `cable` settings on each client.
The default identity location used by the integrated application is
`dat-gpt/cesspool-bootstrap-state/identity.key` under the platform's state
directory. The raw 96-byte public-plus-secret Ed25519 keypair matches the
existing bootstrap identity file layout, so extracting the library keeps a
running node's identity stable.
The old `dat_peer_t`, `datnet_t`, and `datnet_*` spellings remain as thin
source-compatibility aliases in the main application headers. New Cesspool
consumers should use `cess_peer_t`, `cessnet_t`, and `cessnet_*`.
docs/network-protocol.md
+41
View File
@@ -0,0 +1,41 @@
# Network protocol identity
`NETWORK_PROTOCOL` in the repository root is the single editable protocol
token. `cesspool_discovery_key` uses it as the Cesspool topic namespace.
Hypercore V6 has no fixed protocol-name field: `userData` is optional
application data, and the handshake `id` is protocol-local.
The default file contains `cesspool`. Production uses the Cesspool discovery
and transport path. `hypercore` is reserved for compatibility experiments
against the historical Dat/Dillo implementation; it is not the production
network stack and does not refer to Cabal's newer Hypercore stack. The V6
session opens channel 0 with a clear 24-byte nonce, then encrypts subsequent
stream bytes with XSalsa20; channel 1 carries the content feed.
Dat-thang keeps one client network key in `~/.dat/dat-thang.network-key`.
Loading or switching a local or remote archive must not replace that key: the
same peer identity announces every archive the client shares. Cable keeps its
Cabal-compatible room identity separately. Archive feed keys still determine
the archive topic; they are not the client's network identity.
The discovery UDP socket tries 3282 first, then 3000, 3002, 3004, 2001, 2003,
and 2005, matching Dillo's `hyperdiscovery` port list. Callers can request a
specific port.
The compatibility builds use the packed channel/type header and protobuf payload schema
from `extra/schema.proto`. The `cesspool` token selects V7 channel numbering;
`hypercore` selects V6 channel numbering. This message codec is independent
of the transport and Cesspool's UDP discovery packets.
The Cesspool V7 handshake includes the selected token in `userData`. The V6
Hypercore handshake omits `userData`, because the legacy implementation treats
it as application-owned data rather than network identity.
## Scheme application view
When S7 is enabled, protocol frames can be viewed as a tagged Scheme record
with named fields: `(dat:wire-message (channel . N) (type . N) (payload .
#u8(...)))`. `dat-wire->sexp` and `dat-sexp->wire` call the existing C frame
codec. The record is an application representation only; the frame header,
payload bytes, channel numbering, encryption, and protocol identity continue
to come from the selected C network implementation. Scheme code is installed
under `share/datgpt/scheme` as part of the default install.
docs/primitives.md
+107
View File
@@ -0,0 +1,107 @@
# datgpt primitives
This is the working map of the old `dat://` implementation. Names follow the
project: SLEEP owns feeds, cesspool owns Hyperswarm-style discovery, and dat
joins the feeds into an archive.
The standalone [libcess package](libcess.md) owns Cesspool discovery and UDP
transport. Both the Dat (`ddat`) and Cable (`cable`) applications link it.
## SLEEP storage
- Feed header, tree nodes, signatures, bitfield pages, keys, and data files.
- Flat-tree addressing, spans, roots, leaf and parent hashing.
- Feed open, length, indexed read, append, flush, and close.
- Public-key-only in-memory read-only feeds accept verified remote blocks and
retain their proofs without pretending the local node has signing keys. A
`.remote` cache reopens sparse blocks only after verifying each stored proof.
- Data/tree bit access, index regeneration, scan, encode, and decode.
- Key generation, public/secret persistence, hashes, signatures, root signing.
- Merkle proof nodes, proof encode/decode, hash folding, and root signature check.
## Archive
- Paired metadata and content feeds.
- Create, open, close, public keys, writable keypairs.
- Separate secret-key save and writable reopen.
- Header pointing from metadata to content.
- File, directory, symlink, and deletion history entries.
- Entry enumeration, reverse path lookup, and content block lookup.
- Recursive filesystem import for regular files, directories, and symlinks.
- Files are chunked across content blocks and can be exported recursively.
- Snapshot sync compares content, mode, target, and millisecond mtime before
appending; paths removed locally receive tombstones, while unchanged paths
do not extend the feed.
- `dat status`, `dat files`, `dat verify`, `dat resolve`, historical `cat`,
`checkout`, and versioned URL commands are available from the CLI.
## Peer protocol
- The root `NETWORK_PROTOCOL` selects legacy V6 framing (`hypercore`) for the
old Dat target; `cesspool` selects V7 framing for the in-tree network mode.
- Protobuf feed-open messages, handshake id/live/userData, request/range/data
messages, extension payloads, and standard DATA node/signature fields.
- Have/want ranges, block requests, cancellation.
- Legacy WANT messages with omitted length run to the peer's current feed end;
responses are bounded by the local feed length and representable bitfield.
- Data plus Merkle proof, and named extensions.
- Per-channel replication state, remote bitfield, and request queue.
- `dat_session` joins one archive key to its discovery topic, carries discovered
peer addresses, opens the archive feeds, gates the wire handshake on an
externally authenticated transport, exchanges HAVE ranges, requests missing
blocks, and verifies signed proofs before caching received blocks.
- A peer callback exposes newly discovered or rediscovered peers so a transport
adapter can dial them and call `dat_session_peer_secure` after authentication.
- `dat_session_poll` runs an optional transport poll callback alongside the
Cesspool UDP poll, refreshes topic membership, and reconciles discovered peers.
- `dat_session_tick` is the nonblocking form of that poll cycle.
- `dat_session_receive` accepts arbitrary byte-stream chunk boundaries, buffers
partial frames, decrypts the V6 stream at byte offsets, and dispatches multiple
coalesced frames from one read.
- A read-only replica keeps received proofs and can serve them onward. The
caller still supplies the authenticated byte transport and poll loop.
## cesspool
- UDP socket transport, bootstrap nodes, listen, poll, and destroy.
- Legacy lookup, announce, unannounce, ping, and holepunch vocabulary.
- Topic join/leave and periodic refresh.
- Peer table, failure/last-seen state, topic table, discovery callbacks.
- Legacy protobuf requests/responses and compact public/local peer parsing.
- XOR-distance peer selection and dht-rpc transaction tracking.
## uTP
- Header encode/decode and connection identifiers.
- SYN/connect/accept, state acknowledgements, data, FIN, and reset states.
- Send segmentation and timer boundary.
- Receive reordering, cumulative and selective acknowledgements, congestion
window, retransmission timers, and in-flight packet retirement. DATA packets
are capped at the 1200-byte payload size, and malformed control packets are
rejected before entering the reorder buffer.
## Name resolution
- `dat-dns` resolution through HTTPS `/.well-known/dat`, plus the local
`~/.dat/dat-dns` name map and TTL-aware cache. The TXT packet decoder is a
compatibility helper and is not used for runtime name resolution.
- Canonical `dat://<key>` output; the domain remains a name for the resolved
archive key rather than a separate archive identity.
## Later layer
- S7-facing records use one tagged, named-field form:
`(dat:wire-message (channel . N) (type . N) (payload . #u8(...)))`.
`dat:record-ref` and `dat:record-set` provide the shared field access layer;
protocol payload bytes remain bytevectors.
- `dat-wire->sexp` and `dat-sexp->wire` adapt the existing C frame codec. This
keeps the C codec authoritative for peer bytes while Scheme is the common
application representation; a Scheme record is never serialized directly
onto the network.
- Initial S7 bindings expose archive create, open, close, entry count, and the
wire-record adapter. The Scheme adapters and JSON reader are part of the default install.
- Hyperdrive's JS `Index` and `Stat` records should use the same named-field
convention as their read/write adapters are added; they must retain their
original protocol field numbers and binary fields.
- The local CLI sits above the archive API; peer transport polling and the
authenticated stream layer remain separate callers of `dat_session`.
docs/scheme-interop.md
+80
View File
@@ -0,0 +1,80 @@
# Scheme records at protocol boundaries
## Representation contract
The old Dat/Cabal implementations use JavaScript objects and JSON because they
run on Node. This implementation uses S7 Scheme values and tagged
S-expressions as its native application representation. Keep records,
configuration, and the C-to-client boundary in that form; translate to JSON
only at a compatibility edge whose existing peer format requires JSON. Dat
wire bytes and Cabal's canonical Hypercore JSON remain their established
protocol formats.
S7 is the application view for structured records that would otherwise be
assembled as JavaScript objects or JSON-shaped maps. New archive descriptions
are stored as `dat.scm` association lists. Existing wire codecs, feed files,
and peer protocol bytes retain their established formats; `dat.json` is not
written for newly created archives.
## Record form
Records are tagged lists with named association fields:
```scheme
(dat:wire-message
(channel . 1)
(type . 3)
(payload . #u8(...)))
```
`dat:record-ref` reads a named field and accepts an optional fallback.
`dat:record-set` returns a record with one field replaced or added. Names are
symbols; strings stay strings; protocol integers stay exact integers; binary
data stays an S7 byte-vector. Absence is represented by no field, not by
conflating an absent value with `#f` or an empty byte-vector.
## Wire boundary
`dat-wire->sexp` and `dat-sexp->wire` adapt the C `dat_wire` frame codec. The
frame record keeps the channel, type, and opaque payload bytes named and
inspectable. The active C codec remains authoritative for varints, framing,
message payloads, channel assignments, protocol identity, and encryption.
S-expressions are not sent to Dat peers and do not alter feed blocks or
signatures. Cable's local UI and room-event boundary uses S7 and
S-expressions inside the native C application; it does not start a Node
adapter.
Protocol-specific adapters can add named records such as `dat:feed-open` or
`dat:hyperdrive-stat` above this frame boundary. Each adapter should map
explicitly to the existing C type or legacy field numbers, preserve unknown
fields when forwarding is required, and keep byte strings as byte-vectors.
This allows Cesspool and test-only legacy Hypercore views to share Scheme-level record
operations while retaining their separate codecs and wire identities.
## Build and use
S7 is bundled and required by the default build. The build produces
`ddat-scheme` and links the runtime into the application library. With no
arguments it starts the REPL; `-e` evaluates an expression, `-l` loads a file,
`-I` adds a module directory, and `--repl` enters the REPL after scripts run. Dat's host
runtime registers the C bindings, assembles the module search path, and loads
`dat-init.scm`, `dat-record.scm`, `dat-wire.scm`, and `dat-json.scm` before
user code. `dat:json-parse` accepts strict JSON and returns tagged Scheme
objects/arrays while preserving JSON strings, booleans, numbers, and null.
`dat:json-stringify` maps those same tagged Scheme values back to JSON at
compatibility boundaries. Scheme/S-expression records remain the application
representation; JSON is emitted only where an existing peer format requires
it. The test-only Cabal harness owns Cabal's canonical JSON Hypercore encoding.
The runtime searches `DAT_SCHEME_PATH` (a platform-separated list),
`DAT_SCHEME_DIR`, the source-tree `scheme/` directory, and the configured
installed directory. Application modules map feature names such as
`dat:archive` to `dat/archive.scm`, load through `(dat:require 'dat:archive)`,
and finish with `(provide 'dat:archive)`. `dat:with-archive` closes an archive
even when its body exits through a Scheme condition. Archive values are S7
foreign objects: explicit close is idempotent, and garbage collection closes
an archive that the caller abandons.
This is the embedding boundary S7 expects: the host owns startup, load paths,
native types, resource lifetime, CLI policy, and application modules. The C
wire/feed implementations still own protocol bytes and on-disk formats.
docs/sleep.pdf
BIN
View File
Binary file not shown.
docs/source-distribution.md
+38
View File
@@ -0,0 +1,38 @@
# Source and release layout
The repository has one source tree for both platforms. Generated files have
separate destinations:
- GNU/Linux compiler outputs: `build/linux/`
- Windows compiler outputs: `build/windows/`
- Windows portable runtime staging: `dist/windows-x64/`
- Installers and platform packages: `dist/installers/` and `dist/linux-*/`
- Source archive: `dist/source/`
`build/` is disposable compiler output and is ignored by Git. External Dillo
checkouts live in `test-deps/`; machine keys and test-run output live in
`local-data/`. Both are local-only and ignored. Reusable tests and archive
fixtures live under `tests/`, so a clean operation cannot remove them. Do not
zip the working directory directly. To make a source-only archive that retains
the historical fixtures and interoperability tests while excluding machine
data and generated outputs, run from PowerShell at the repository root:
```powershell
.\package-source.ps1
```
The script writes `dist\source\dat-gpt-v0.0.002-source.zip` (the version comes
from `VERSION`). It includes project sources, documentation, assets, the
historical code under `extra/`, `third_party/`, the complete `tests/` tree, and
the root historical `.dat` verifier fixture. It excludes installed
`node_modules/`, `build/`, `dist/`,
`local-data/`, `test-deps/`, `.git/`, per-machine Dillo keys, and generated
configure files. It also skips local identity keys, cookie jars, SSH private
keys, and `.env` files by name. It does not compile or bundle executables.
Keep persistent node identity files in per-user application data outside the
checkout. The source ZIP excludes local runtime state.
For target packages, use `make package` on GNU/Linux and
`build-windows-installer.ps1` on Windows. The Windows `build-windows.ps1` also
creates a portable ZIP without requiring Inno Setup.
docs/windows-build.md
+96
View File
@@ -0,0 +1,96 @@
# Windows build
The native Windows build uses MSYS2's UCRT64 toolchain and creates a runnable
folder of PE executables plus their runtime DLLs and data files. It does not
reuse the GNU/Linux `build/` directory or generated `config.mk`.
Install MSYS2, then install the UCRT64 packages needed by the project:
```sh
pacman -S --needed \
mingw-w64-ucrt-x86_64-gcc \
mingw-w64-ucrt-x86_64-make \
mingw-w64-ucrt-x86_64-pkgconf \
mingw-w64-ucrt-x86_64-libsodium \
mingw-w64-ucrt-x86_64-curl \
mingw-w64-ucrt-x86_64-SDL2 \
mingw-w64-ucrt-x86_64-SDL2_ttf
```
From the repository root in PowerShell, run:
```powershell
.\build-windows.ps1
```
Compiler outputs, including tests and intermediate files, stay in
`build\windows`. Distribution output is staged separately: `dist\windows-x64`
contains only the eight user-facing programs, their runtime DLLs, assets,
Scheme files, and manuals. `dist\installers` contains a portable ZIP and, when
Inno Setup is installed, the setup executable. Do not distribute the contents
of `build\windows` directly. The build date's Discordian calendar conversion
is compiled into the GUI; no external `ddate` installation is needed.
To create just the portable ZIP, run `build-windows.ps1`. To create the install
wizard as well, run `build-windows-installer.ps1`.
`cable-thang.exe` is a native C/S7 client using Cesspool and Sleep. Node.js
and Cabal JavaScript modules are needed only for optional interoperability
tests. Cable audio and video support are experimental and disabled by default,
so the regular Windows build does not require FFmpeg. To enable both, install
FFmpeg and build with:
```powershell
pacman -S --needed mingw-w64-ucrt-x86_64-ffmpeg
.\build-windows.ps1 -EnableCableMedia
```
The UCRT64 FFmpeg libraries provide Cable's AAC-LC encoder/decoder, OBS MPEG-TS
ingest, and low-resolution JPEG video exchange; their runtime DLLs are copied
beside the Windows executables. In `cable-thang`, `OBS INPUT` (or
Ctrl+Shift+V) listens on local UDP port 5000; OBS can publish MPEG-TS there
using its Custom FFmpeg Output.
The GNU/Linux workflow is `./configure`, `make`, and `make install`; its default
compiler output is `build/linux`. To create a staged GNU/Linux tarball, use
`make package`. Release artifacts go under `dist/` on both platforms. To create
a source-only archive, run `package-source.ps1`; its contents and exclusions
are documented in `docs\source-distribution.md`.
That configuration checks for libcurl because the Dillo-compatible
`dat-dns` lookup uses HTTPS `/.well-known/dat`; it does not call the operating
system DNS TXT resolver.
## Install wizard
Install [Inno Setup 6](https://jrsoftware.org/isinfo.php) after installing the
MSYS2 build tools. From PowerShell at the repository root, run:
```powershell
.\build-windows-installer.ps1
```
The script rebuilds the Windows applications, bundles their DLLs and runtime
assets, and creates `dist\installers\dat-thang-setup-0.0.002-x64.exe`.
The application identifies itself as `dat-thang v0.0.002 (prototype,
pre-alpha)`; the executable file version is `0.0.2.0`.
The wizard offers a per-user or all-users install, a destination folder,
optional desktop shortcut, optional PATH entry for the terminal commands, and
an optional launch at the end. Uninstall removes a PATH entry only when this
installer added it. The app and installer use the same generic green hexagon
icon. The Start menu includes dat-thang and a command prompt with the installed
tools on PATH.
`cable-thang.exe` is included for opening a room address from the command
prompt and connects through Cesspool.
`cesspool-bootstrap.exe` runs a dedicated Cesspool discovery node in the
foreground. It is for an operator deploying a reachable public rendezvous
host, not something chat users need to run. The node needs a stable state
directory and inbound UDP on the selected port; see `docs\cesspool-bootstrap.1`
for process-supervisor and firewall setup. The Windows installer includes the
executable and the operator manpage in its `docs` folder.
Keep the node identity outside the checkout. In PowerShell, run the development
executable with `--state-dir "$env:LOCALAPPDATA\dat-gpt\cesspool-bootstrap-state"`.
That keeps the private identity key in per-user application data rather than in
the source tree or disposable compiler output.