Add Android client, packaging, Makefile entrypoint and handover docs

Why: the set has to be installable and publishable, not just buildable, and the next person needs the reasoning, not only the code. What: - android/: Kotlin client with no third-party dependencies. A DocumentsProvider publishes the mirrored databases to every file picker, so KeePassDX opens them as ordinary documents; closing a file it wrote triggers a sync. A foreground service holds the event stream open. Mirror.decide transcribes the Go merge table, so both platforms resolve a conflict identically. - Makefile as the single entrypoint, with file targets so only what changed is rebuilt: host and cross builds, both .deb packages, the macOS DMG with its /Applications symlink, the Windows zip, the APK under its F-Droid publishing name, and manual-only android-publish. - versions.mk holds one version per platform plus the Android versionCode; `make bump` and `make changelog` move them. Debian changelog is tracked. - systemd units (hardened system unit for the server, user unit for the agent), macOS bundle and launch agent, Windows Task Scheduler job. - docs/HANDOVER.md on architecture and every decision that shaped it, docs/PROTOCOL.md for the API, README for the quick start. Proved: make check and make test-race both pass; built both .deb packages and inspected their contents, built the Windows zip, cross-compiled all five desktop targets, and exercised make bump. The Android module is written but not compiled: no JDK or Android SDK in this environment, which HANDOVER.md states plainly.
2026-10-29 16:26:33 +01:00
parent 4d0d53ff04
commit 0e83acb9b1
40 changed files with 2563 additions and 24 deletions
@@ -0,0 +1,318 @@
+# KeepOrg: developer handover
+
+KeepOrg makes a KeePass database collaborative without changing KeePass. A
+server holds many KDBX files and arbitrates who wrote last; clients present
+each assigned database as an ordinary `.kdbx` file that any KeePass
+application can open, and synchronize it on every save.
+
+The server never opens a database. It moves encrypted bytes and metadata
+about them. Master passwords and key files stay on the clients, so a
+compromised server leaks the ciphertext and the group structure, nothing else.
+
+## Repository map
+
+```
+cmd/keeporgd/          server entry point: flags, config file, bootstrap
+cmd/keeporg/           desktop client: login, run, sync, status, administration
+internal/store/        domain model, JSON metadata store, ACL resolution
+internal/blob/         versioned KDBX payload storage on disk
+internal/server/       HTTP surface: auth, OIDC, sync, admin, SSE hub, pages
+internal/client/       shared client engine: API wrapper, merge policy, mirror
+internal/secret/       password verifiers, tokens, identifiers
+internal/totp/         RFC 6238 second factor
+android/               Android client: DocumentsProvider, sync service, UI
+packaging/             deb, macOS bundle and DMG, Windows zip, systemd units
+scripts/               package builders, version bump, changelog
+versions.mk            the one place every platform's version lives
+Makefile               the only entry point for building and publishing
+```
+
+## The three programs
+
+### Server (`keeporgd`)
+
+One process, one data directory, no database engine:
+
+- `state.json` holds users, groups, memberships, databases, grants, invites,
+  sessions and pending device logins.
+- `blobs/<database-id>/<version>.kdbx` holds the payloads, keeping the last
+  20 versions of each database.
+
+`internal/store.Store` is the only writer of `state.json`. Every mutation
+runs through `Update(func(*State) error)`, which applies the change to a
+private copy, writes it with `write` + `fsync` + `rename`, and only then
+publishes it. A failed mutation changes nothing, and a crash leaves either
+the old document or the new one.
+
+Reads go through `Read(func(*State))` under a read lock. Access decisions are
+pure functions over `*State` in `internal/store/acl.go`, which is why they
+are cheap enough to call per request and easy to test exhaustively.
+
+### Desktop client (`keeporg`)
+
+`keeporg run` keeps a mirror directory (default `~/KeepOrg`) in step with the
+server:
+
+- a rescan every two seconds notices that a KeePass application saved a file;
+- the server's event stream delivers other people's pushes immediately;
+- `.keeporg-state.json` in the mirror records what this device last agreed
+  with the server for each database.
+
+All of the synchronization policy is the pure `decide()` table in
+`internal/client/decide.go`. Everything else in the mirror applies the action
+it returns. To change behaviour, change that table and its test.
+
+### Android client
+
+`KeepOrgProvider` is a `DocumentsProvider`, so the mirrored files appear in
+every file picker under "KeepOrg" and KeePassDX or Keepass2Android opens them
+as ordinary documents. When such an application closes a file it had open for
+writing, the provider triggers a synchronization pass; that is how a save
+reaches the server. `SyncService` is a foreground service that holds the
+event stream open so colleagues' changes arrive without opening the app.
+
+`Mirror.decide` is a transcription of the Go decision table. **Keep the two in
+step**: a change to one belongs in the other in the same commit.
+
+## Access control
+
+Groups form one tree. Two rules decide everything:
+
+1. **A grant on a group reaches every member below it.** A grant on
+   `Company` applies to a member of `Company/Engineering/Backend`, because
+   the member's effective group set is their own groups plus all ancestors.
+2. **An administrator of a group administers everything below it.** A
+   `Company` admin administers `Backend`; a `Sales` admin does not.
+
+A database is owned by one group. Administering that group means `admin` on
+the database: renaming it, changing its merge strategy, granting other groups
+access and deleting it. Other groups reach it only through explicit grants,
+with `read`, `write` or `admin`. A user's permission on a database is the
+strongest grant that reaches them. Server administrators administer
+everything.
+
+Nothing is browsable. A user who has no permission on a database cannot see
+that it exists: the blob endpoint answers `404`, not `403`.
+
+## Synchronization protocol
+
+Full endpoint reference: [PROTOCOL.md](PROTOCOL.md). The part that matters:
+
+```
+GET /api/v1/databases                 what is assigned, with version and digest
+GET /api/v1/databases/{id}/blob       bytes, ETag: "<version>"
+PUT /api/v1/databases/{id}/blob       If-Match: "<version this edit was based on>"
+GET /api/v1/events                    SSE stream of db-updated notifications
+```
+
+A push always states the version it was based on. If that is still current,
+the server stores version+1, publishes `db-updated` to everyone who can read
+the database, and prunes old versions. If it is not current, the stored merge
+strategy decides, and the rejection carries both the current version and the
+strategy so the client can resolve without another round trip.
+
+Pushing bytes the server already holds is a no-op that answers
+`{"unchanged": true}`: a save that changed nothing does not burn a version
+and does not wake other clients.
+
+### Merge strategies
+
+Set per database, changed by an administrator of the owning group
+(`keeporg db -id <id> -strategy <name>`).
+
+| Strategy | Server behaviour on a stale push | Client behaviour |
+| --- | --- | --- |
+| `server-wins` (default) | rejects with `409 conflict` | keeps the local save as `<name>.conflict-<timestamp>.kdbx`, then writes the server copy |
+| `client-wins` | accepts a retry carrying `If-Match: *` | retries with force; the server copy is overwritten and kept in history |
+| `manual` | rejects, and refuses force | writes the server copy as `<name>.server-v<n>.kdbx` beside the file, touches nothing, reports a conflict once |
+
+A losing save is never discarded: it is always written somewhere the user can
+find it. A read-only user who edits their copy gets the same treatment, with
+the server copy restored.
+
+**There is no entry-level merge.** The server cannot perform one, because it
+cannot decrypt a database. A true three-way merge belongs in a client that
+already holds the master password; the strategy field is the hook for it, and
+`manual` is the honest behaviour until then. KeePassXC's own "Merge from
+database" on the two files is the current answer.
+
+## Authentication
+
+Three ways in, one kind of credential out: an opaque bearer token stored in
+`state.json` with an expiry (90 days by default).
+
+- **Password** (`POST /api/v1/auth/login`), with the built-in TOTP second
+  factor when the account has enrolled one. The server answers
+  `totp_required` so a client can ask for the code without guessing.
+- **Invite link.** A group administrator creates one; the invitee opens it in
+  a browser, picks a username and password, and lands in the invite's group
+  with the invite's role.
+- **Browser-approved device login**, which is also how OIDC works. The client
+  gets a short user code, the user approves it in a browser with a password
+  or with single sign-on, and the client polls until a token appears. The
+  same flow serves a headless desktop and an Android phone.
+
+OIDC is implemented against the discovery document with the standard library
+only. The server is a confidential client: it exchanges the authorization
+code over TLS with the token endpoint and reads the identity from the
+userinfo endpoint, so there is no local JWT verification to get wrong. An
+identity links to an existing account by `preferred_username` or `email`, or
+provisions one when `oidc.auto_create` is set.
+
+## Decisions worth knowing
+
+**No third-party Go dependencies, no cgo.** The whole server and desktop
+client build with `CGO_ENABLED=0` from the standard library: SSE instead of a
+WebSocket library, `crypto/pbkdf2` (new in Go 1.24) instead of an Argon2
+package, 60 lines of HMAC for TOTP, `html/template` for the three pages.
+Cross-compiling to five platforms is then one `go build` each, and there is
+no dependency tree to audit for a program that guards passwords.
+
+**A JSON document instead of SQLite.** The metadata is hundreds of records,
+never thousands: users, groups, grants. One writer with atomic rename gives
+durability and consistency without cgo or a migration story, and an operator
+can read and back up the whole server state with `cat`. The cost is a single
+node and a full rewrite per mutation. If this ever needs several servers or
+tens of thousands of users, replace `internal/store` — the rest of the code
+only touches it through `Read`, `Update` and the ACL functions.
+
+**A mirror directory instead of FUSE.** The requirement is that a KeePass
+application sees a regular `.kdbx` file. A real file in a real directory is
+that, on Linux, macOS, Windows and BSD alike, with no kernel module, no
+WinFsp or macFUSE dependency, no cgo and nothing to break on an OS upgrade.
+Local writes are atomic (`write` + `rename`), so an application never reads
+half a database. The price is that synchronization happens on a save rather
+than on a `write()` syscall, which is exactly the granularity KDBX has
+anyway.
+
+**SSE instead of WebSocket.** The channel only carries server-to-client
+notifications. SSE is a `GET` that survives proxies, reconnects by itself and
+needs no library on any of the three clients. A subscriber that falls behind
+is dropped rather than waited for; it reconnects and reconciles from scratch,
+because every notification only ever triggers a full reconcile.
+
+**Versions, not just a digest.** Keeping the last 20 blobs makes a rejected
+push recoverable, makes "who broke the database" answerable, and costs a few
+megabytes.
+
+## Operating it
+
+```sh
+make deb                      # keeporg and keeporgd packages for this arch
+sudo dpkg -i dist/keeporgd_*.deb
+sudo nano /etc/keeporg/server.json
+sudo systemctl enable --now keeporgd
+sudo journalctl -u keeporgd   # the first administrator's password is logged once
+```
+
+On first start with an empty state the server creates the `admin` account and
+an `everyone` group, and logs the generated password once. Set
+`KEEPORG_ADMIN_PASSWORD` to choose it instead.
+
+`/etc/keeporg/server.json`:
+
+```json
+{
+  "addr": ":8765",
+  "data_dir": "/var/lib/keeporg",
+  "external_url": "https://keeporg.example.com",
+  "session_ttl_hours": 2160,
+  "max_db_size_mb": 32,
+  "oidc": {
+    "issuer": "https://id.example.com/realms/main",
+    "client_id": "keeporg",
+    "client_secret": "...",
+    "scopes": "openid profile email",
+    "auto_create": false,
+    "default_group_id": ""
+  }
+}
+```
+
+`external_url` must be what a browser sees: invite links, device approval
+pages and the OIDC redirect URI are built from it. Terminate TLS in front of
+the server, or set `tls_cert` and `tls_key`.
+
+Back up `/var/lib/keeporg`. It is the whole server: `state.json` plus the
+blobs. There is nothing else to dump.
+
+### Setting up a team
+
+```sh
+keeporg login -server https://keeporg.example.com -user admin
+keeporg group -create engineering -parent <everyone-id>
+keeporg db -create "Engineering" -group <engineering-id>
+keeporg invite -group <engineering-id> -role member -uses 5 -hours 72
+keeporg grant -group <readers-id> -db <db-id> -perm read
+keeporg db -id <db-id> -strategy manual
+```
+
+A new database has no bytes until someone saves one. Create the KDBX in your
+KeePass application under the exact name the server shows, inside the mirror
+directory; the client adopts it and pushes it as version 1.
+
+## Versioning and publishing
+
+`versions.mk` is the only place a version lives, one per platform plus the
+Android `versionCode`. Binaries are stamped at build time through
+`-ldflags -X main.Version=...`.
+
+```sh
+make bump PLATFORM=android PART=minor   # also increments versionCode
+make changelog MSG="what changed"       # prepends to the Debian changelog
+```
+
+Application identifiers follow `cc.tiredsysadmin.<os>.<app>`:
+`cc.tiredsysadmin.android.keeporg`, `.macos.keeporg`, `.linux.keeporg`,
+`.windows.keeporg`, `.bsd.keeporg`, and `.linux.keeporgd` for the server
+package.
+
+Desktop and server artifacts stay local (`dist/`). Android publishing is
+manual, never automatic:
+
+```sh
+make android          # unsigned APK, named cc.tiredsysadmin.android.keeporg_<code>.apk
+make android-publish  # scp to the private F-Droid unsigned and metadata directories
+```
+
+Then sign and index on the repository host (`fdroid publish && fdroid update`).
+
+## Testing
+
+```sh
+make check      # gofmt, go vet, go test ./...
+make test-race  # the same suite under the race detector
+```
+
+What the suite actually covers:
+
+- `internal/totp` against the RFC 6238 vectors.
+- `internal/store` for grant inheritance, sibling isolation, admin
+  inheritance, cycle safety, update rollback and expiry.
+- `internal/server` for every push verdict as a table, and an end-to-end HTTP
+  pass over login, stale push, forced push, idempotent push, read-only
+  grants, invite redemption and reuse, TOTP enrolment, device login and the
+  event stream.
+- `internal/client` for the decision table, and for the mirror driven against
+  a real in-process server: adopting a local file, following a remote push,
+  server-wins, client-wins, manual, read-only restore, server-side rename,
+  and an update arriving over the event stream.
+
+## Known limits and the next things to do
+
+- **One server node.** The JSON store is single-writer by design. Horizontal
+  scale means replacing `internal/store`.
+- **No entry-level merge.** See the strategies section. The pluggable place
+  is the client, which holds the key.
+- **The Android module has not been compiled in this environment** (no JDK or
+  Android SDK was available). The Gradle wrapper is deliberately not
+  committed; run `gradle wrapper --gradle-version 8.9` once, then
+  `make android`. Expect to fix small things on the first build.
+- **No key-file or Yubikey handling**, and none needed: those live in the
+  KeePass application.
+- **Sessions are not listable or revocable individually** beyond
+  `keeporg logout` on the device itself. Adding `GET`/`DELETE
+  /api/v1/auth/sessions` is a small, well-bounded job.
+- **Android does not restart its watcher at boot.** Add a
+  `RECEIVE_BOOT_COMPLETED` receiver if that matters more than the permission
+  it costs.
@@ -0,0 +1,91 @@
+# KeepOrg HTTP API
+
+Every `/api/` endpoint takes and returns JSON and needs
+`Authorization: Bearer <token>`, except the login, device and invite
+endpoints. Errors carry `{"code": "...", "error": "..."}`; `code` is stable
+and meant to be matched on, `error` is for humans.
+
+Three endpoints serve HTML, because a browser is the right place for them:
+`/device`, `/invite/{token}` and `/oidc/callback`.
+
+## Authentication
+
+| Method | Path | Body | Answer |
+| --- | --- | --- | --- |
+| POST | `/api/v1/auth/login` | `{username, password, totp, label}` | `{token, expires_at, user}`; `401 totp_required` when a second factor is enrolled and missing, `401 totp_invalid`, `401 bad_credentials` |
+| POST | `/api/v1/auth/logout` | – | revokes the calling token |
+| GET | `/api/v1/auth/whoami` | – | the account and its group memberships |
+| POST | `/api/v1/auth/totp/enroll` | – | `{secret, uri}`; not active until confirmed |
+| POST | `/api/v1/auth/totp/confirm` | `{code}` | `{totp_enabled: true}` |
+| POST | `/api/v1/auth/device/start` | `{label}` | `{device_code, user_code, verification_url, interval, expires_in}` |
+| POST | `/api/v1/auth/device/poll` | `{device_code}` | `202 {status: "pending"}`, then `200 {token, ...}` once; `410 expired` |
+
+A device code is single use: the first successful poll consumes it.
+
+## Synchronization
+
+| Method | Path | Notes |
+| --- | --- | --- |
+| GET | `/api/v1/databases` | `{databases: [{id, name, perm, merge_strategy, version, sha256, size, updated_at, updated_by}]}` |
+| GET | `/api/v1/databases/{id}/blob` | bytes; `ETag: "<version>"`, `X-KeepOrg-SHA256`; `204` when the database has no content yet; `?version=<n>` reads a stored older version |
+| PUT | `/api/v1/databases/{id}/blob` | body is the KDBX file. `If-Match` is required |
+| GET | `/api/v1/events` | SSE. Accepts `?access_token=` because an event stream cannot always set headers |
+
+### PUT semantics
+
+`If-Match` carries the version the edit was based on, or `*` to force.
+
+| Condition | Status | Body |
+| --- | --- | --- |
+| `If-Match` missing | `428 precondition_required` | – |
+| version matches, bytes differ | `200` | `{version, sha256, unchanged: false}` |
+| version matches, bytes identical | `200` | `{version, sha256, unchanged: true}` |
+| version stale | `409 conflict` | `{current_version, merge_strategy}`, plus `ETag` and `X-KeepOrg-Strategy` |
+| `If-Match: *`, strategy is `client-wins` | `200` | the server copy is replaced |
+| `If-Match: *`, any other strategy | `409 force_not_allowed` | – |
+| read-only access | `403 forbidden` | – |
+| no access | `404 not_found` | a database you cannot read does not exist to you |
+| larger than `max_db_size_mb` | `413 too_large` | – |
+
+`X-KeepOrg-SHA256` on a `PUT` is optional; when present it must match the
+body or the request is rejected with `400 checksum_mismatch`.
+
+### Events
+
+```
+event: db-updated
+data: {"type":"db-updated","database_id":"...","name":"Team","version":7,"sha256":"...","by":"alice"}
+
+event: db-list-changed
+data: {"type":"db-list-changed","database_id":"..."}
+```
+
+`db-updated` follows an accepted push; `db-list-changed` follows a grant,
+rename, strategy change or deletion. A comment line (`: keepalive`) arrives
+every 25 seconds. Reconnect on any error and reconcile fully; events are
+notifications, never state.
+
+## Administration
+
+Permission needed is in the last column. "Group admin" means administering
+the group named or any group above it.
+
+| Method | Path | Body / query | Needs |
+| --- | --- | --- | --- |
+| GET | `/api/v1/groups` | – | any account; lists what you can see |
+| POST | `/api/v1/groups` | `{name, parent_id}` | group admin of the parent; server admin for a root group |
+| DELETE | `/api/v1/groups/{id}` | – | group admin; refuses while it has children, members or databases |
+| POST | `/api/v1/groups/{id}/members` | `{username or user_id, role}` | group admin |
+| DELETE | `/api/v1/groups/{id}/members/{user}` | – | group admin |
+| POST | `/api/v1/databases` | `{name, owner_group_id, merge_strategy}` | group admin of the owner |
+| PATCH | `/api/v1/databases/{id}` | `{name?, merge_strategy?}` | database admin |
+| DELETE | `/api/v1/databases/{id}` | – | database admin; deletes every stored version |
+| POST | `/api/v1/grants` | `{group_id, database_id, perm}` | database admin |
+| DELETE | `/api/v1/grants` | `?group_id=&database_id=` | database admin |
+| POST | `/api/v1/invites` | `{group_id, role, ttl_hours, max_uses}` | group admin |
+| GET | `/api/v1/invites` | – | your own invites, and those of groups you administer |
+| GET | `/api/v1/users` | – | yourself, plus members of groups you administer |
+| GET | `/healthz` | – | unauthenticated liveness |
+
+`merge_strategy` is one of `server-wins`, `client-wins`, `manual`. `perm` is
+one of `read`, `write`, `admin`. `role` is `member` or `admin`.