keeporg/docs/HANDOVER.md

# KeepOrg: developer handover

KeepOrg makes a KeePass database collaborative without changing KeePass. A
server holds many KDBX files and arbitrates who wrote last; clients present
each assigned database as an ordinary `.kdbx` file that any KeePass
application can open, and synchronize it on every save.

The server never opens a database. It moves encrypted bytes and metadata
about them. Master passwords and key files stay on the clients, so a
compromised server leaks the ciphertext and the group structure, nothing else.

## Repository map

```
cmd/keeporgd/          server entry point: flags, config file, bootstrap
cmd/keeporg/           desktop client: login, run, sync, status, administration
internal/store/        domain model, JSON metadata store, ACL resolution
internal/blob/         versioned KDBX payload storage on disk
internal/server/       HTTP surface: auth, OIDC, sync, admin, SSE hub, pages
internal/client/       shared client engine: API wrapper, merge policy, mirror
internal/secret/       password verifiers, tokens, identifiers
internal/totp/         RFC 6238 second factor
android/               Android client: DocumentsProvider, sync service, UI
packaging/             deb, macOS bundle and DMG, Windows zip, systemd units
scripts/               package builders, version bump, changelog
versions.mk            the one place every platform's version lives
Makefile               the only entry point for building and publishing
```

## The three programs

### Server (`keeporgd`)

One process, one data directory, no database engine:

- `state.json` holds users, groups, memberships, databases, grants, invites,
  sessions and pending device logins.
- `blobs/<database-id>/<version>.kdbx` holds the payloads, keeping the last
  20 versions of each database.

`internal/store.Store` is the only writer of `state.json`. Every mutation
runs through `Update(func(*State) error)`, which applies the change to a
private copy, writes it with `write` + `fsync` + `rename`, and only then
publishes it. A failed mutation changes nothing, and a crash leaves either
the old document or the new one.

Reads go through `Read(func(*State))` under a read lock. Access decisions are
pure functions over `*State` in `internal/store/acl.go`, which is why they
are cheap enough to call per request and easy to test exhaustively.

### Desktop client (`keeporg`)

`keeporg run` keeps a mirror directory (default `~/KeepOrg`) in step with the
server:

- a rescan every two seconds notices that a KeePass application saved a file;
- the server's event stream delivers other people's pushes immediately;
- `.keeporg-state.json` in the mirror records what this device last agreed
  with the server for each database.

All of the synchronization policy is the pure `decide()` table in
`internal/client/decide.go`. Everything else in the mirror applies the action
it returns. To change behaviour, change that table and its test.

### Android client

`KeepOrgProvider` is a `DocumentsProvider`, so the mirrored files appear in
every file picker under "KeepOrg" and KeePassDX or Keepass2Android opens them
as ordinary documents. When such an application closes a file it had open for
writing, the provider triggers a synchronization pass; that is how a save
reaches the server. `SyncService` is a foreground service that holds the
event stream open so colleagues' changes arrive without opening the app.

`Mirror.decide` is a transcription of the Go decision table. **Keep the two in
step**: a change to one belongs in the other in the same commit.

## Access control

Groups form one tree. Two rules decide everything:

1. **A grant on a group reaches every member below it.** A grant on
   `Company` applies to a member of `Company/Engineering/Backend`, because
   the member's effective group set is their own groups plus all ancestors.
2. **An administrator of a group administers everything below it.** A
   `Company` admin administers `Backend`; a `Sales` admin does not.

A database is owned by one group. Administering that group means `admin` on
the database: renaming it, changing its merge strategy, granting other groups
access and deleting it. Other groups reach it only through explicit grants,
with `read`, `write` or `admin`. A user's permission on a database is the
strongest grant that reaches them. Server administrators administer
everything.

Nothing is browsable. A user who has no permission on a database cannot see
that it exists: the blob endpoint answers `404`, not `403`.

## Synchronization protocol

Full endpoint reference: [PROTOCOL.md](PROTOCOL.md). The part that matters:

```
GET /api/v1/databases                 what is assigned, with version and digest
GET /api/v1/databases/{id}/blob       bytes, ETag: "<version>"
PUT /api/v1/databases/{id}/blob       If-Match: "<version this edit was based on>"
GET /api/v1/events                    SSE stream of db-updated notifications
```

A push always states the version it was based on. If that is still current,
the server stores version+1, publishes `db-updated` to everyone who can read
the database, and prunes old versions. If it is not current, the stored merge
strategy decides, and the rejection carries both the current version and the
strategy so the client can resolve without another round trip.

Pushing bytes the server already holds is a no-op that answers
`{"unchanged": true}`: a save that changed nothing does not burn a version
and does not wake other clients.

### Merge strategies

Set per database, changed by an administrator of the owning group
(`keeporg db -id <id> -strategy <name>`).

| Strategy | Server behaviour on a stale push | Client behaviour |
| --- | --- | --- |
| `server-wins` (default) | rejects with `409 conflict` | keeps the local save as `<name>.conflict-<timestamp>.kdbx`, then writes the server copy |
| `client-wins` | accepts a retry carrying `If-Match: *` | retries with force; the server copy is overwritten and kept in history |
| `manual` | rejects, and refuses force | writes the server copy as `<name>.server-v<n>.kdbx` beside the file, touches nothing, reports a conflict once |

A losing save is never discarded: it is always written somewhere the user can
find it. A read-only user who edits their copy gets the same treatment, with
the server copy restored.

**There is no entry-level merge.** The server cannot perform one, because it
cannot decrypt a database. A true three-way merge belongs in a client that
already holds the master password; the strategy field is the hook for it, and
`manual` is the honest behaviour until then. KeePassXC's own "Merge from
database" on the two files is the current answer.

## Authentication

Three ways in, one kind of credential out: an opaque bearer token stored in
`state.json` with an expiry (90 days by default).

- **Password** (`POST /api/v1/auth/login`), with the built-in TOTP second
  factor when the account has enrolled one. The server answers
  `totp_required` so a client can ask for the code without guessing.
- **Invite link.** A group administrator creates one; the invitee opens it in
  a browser, picks a username and password, and lands in the invite's group
  with the invite's role.
- **Browser-approved device login**, which is also how OIDC works. The client
  gets a short user code, the user approves it in a browser with a password
  or with single sign-on, and the client polls until a token appears. The
  same flow serves a headless desktop and an Android phone.

OIDC is implemented against the discovery document with the standard library
only. The server is a confidential client: it exchanges the authorization
code over TLS with the token endpoint and reads the identity from the
userinfo endpoint, so there is no local JWT verification to get wrong. An
identity links to an existing account by `preferred_username` or `email`, or
provisions one when `oidc.auto_create` is set.

## Decisions worth knowing

**No third-party Go dependencies, no cgo.** The whole server and desktop
client build with `CGO_ENABLED=0` from the standard library: SSE instead of a
WebSocket library, `crypto/pbkdf2` (new in Go 1.24) instead of an Argon2
package, 60 lines of HMAC for TOTP, `html/template` for the three pages.
Cross-compiling to five platforms is then one `go build` each, and there is
no dependency tree to audit for a program that guards passwords.

**A JSON document instead of SQLite.** The metadata is hundreds of records,
never thousands: users, groups, grants. One writer with atomic rename gives
durability and consistency without cgo or a migration story, and an operator
can read and back up the whole server state with `cat`. The cost is a single
node and a full rewrite per mutation. If this ever needs several servers or
tens of thousands of users, replace `internal/store` — the rest of the code
only touches it through `Read`, `Update` and the ACL functions.

**A mirror directory instead of FUSE.** The requirement is that a KeePass
application sees a regular `.kdbx` file. A real file in a real directory is
that, on Linux, macOS, Windows and BSD alike, with no kernel module, no
WinFsp or macFUSE dependency, no cgo and nothing to break on an OS upgrade.
Local writes are atomic (`write` + `rename`), so an application never reads
half a database. The price is that synchronization happens on a save rather
than on a `write()` syscall, which is exactly the granularity KDBX has
anyway.

**SSE instead of WebSocket.** The channel only carries server-to-client
notifications. SSE is a `GET` that survives proxies, reconnects by itself and
needs no library on any of the three clients. A subscriber that falls behind
is dropped rather than waited for; it reconnects and reconciles from scratch,
because every notification only ever triggers a full reconcile.

**Versions, not just a digest.** Keeping the last 20 blobs makes a rejected
push recoverable, makes "who broke the database" answerable, and costs a few
megabytes.

## Operating it

```sh
make deb                      # keeporg and keeporgd packages for this arch
sudo dpkg -i dist/keeporgd_*.deb
sudo nano /etc/keeporg/server.json
sudo systemctl enable --now keeporgd
sudo journalctl -u keeporgd   # the first administrator's password is logged once
```

On first start with an empty state the server creates the `admin` account and
an `everyone` group, and logs the generated password once. Set
`KEEPORG_ADMIN_PASSWORD` to choose it instead.

`/etc/keeporg/server.json`:

```json
{
  "addr": ":8765",
  "data_dir": "/var/lib/keeporg",
  "external_url": "https://keeporg.example.com",
  "session_ttl_hours": 2160,
  "max_db_size_mb": 32,
  "oidc": {
    "issuer": "https://id.example.com/realms/main",
    "client_id": "keeporg",
    "client_secret": "...",
    "scopes": "openid profile email",
    "auto_create": false,
    "default_group_id": ""
  }
}
```

`external_url` must be what a browser sees: invite links, device approval
pages and the OIDC redirect URI are built from it. Terminate TLS in front of
the server, or set `tls_cert` and `tls_key`.

Back up `/var/lib/keeporg`. It is the whole server: `state.json` plus the
blobs. There is nothing else to dump.

### Setting up a team

```sh
keeporg login -server https://keeporg.example.com -user admin
keeporg group -create engineering -parent <everyone-id>
keeporg db -create "Engineering" -group <engineering-id>
keeporg invite -group <engineering-id> -role member -uses 5 -hours 72
keeporg grant -group <readers-id> -db <db-id> -perm read
keeporg db -id <db-id> -strategy manual
```

A new database has no bytes until someone saves one. Create the KDBX in your
KeePass application under the exact name the server shows, inside the mirror
directory; the client adopts it and pushes it as version 1.

## Versioning and publishing

`versions.mk` is the only place a version lives, one per platform plus the
Android `versionCode`. Binaries are stamped at build time through
`-ldflags -X main.Version=...`.

```sh
make bump PLATFORM=android PART=minor   # also increments versionCode
make changelog MSG="what changed"       # prepends to the Debian changelog
```

Application identifiers follow `cc.tiredsysadmin.<os>.<app>`:
`cc.tiredsysadmin.android.keeporg`, `.macos.keeporg`, `.linux.keeporg`,
`.windows.keeporg`, `.bsd.keeporg`, and `.linux.keeporgd` for the server
package.

Desktop and server artifacts stay local (`dist/`). Android publishing is
manual, never automatic:

```sh
make android          # unsigned APK, named cc.tiredsysadmin.android.keeporg_<code>.apk
make android-publish  # scp to the private F-Droid unsigned and metadata directories
```

Then sign and index on the repository host (`fdroid publish && fdroid update`).

## Testing

```sh
make check      # gofmt, go vet, go test ./...
make test-race  # the same suite under the race detector
```

What the suite actually covers:

- `internal/totp` against the RFC 6238 vectors.
- `internal/store` for grant inheritance, sibling isolation, admin
  inheritance, cycle safety, update rollback and expiry.
- `internal/server` for every push verdict as a table, and an end-to-end HTTP
  pass over login, stale push, forced push, idempotent push, read-only
  grants, invite redemption and reuse, TOTP enrolment, device login and the
  event stream.
- `internal/client` for the decision table, and for the mirror driven against
  a real in-process server: adopting a local file, following a remote push,
  server-wins, client-wins, manual, read-only restore, server-side rename,
  and an update arriving over the event stream.

## Known limits and the next things to do

- **One server node.** The JSON store is single-writer by design. Horizontal
  scale means replacing `internal/store`.
- **No entry-level merge.** See the strategies section. The pluggable place
  is the client, which holds the key.
- **The Android module has not been compiled in this environment** (no JDK or
  Android SDK was available). The Gradle wrapper is deliberately not
  committed; run `gradle wrapper --gradle-version 8.9` once, then
  `make android`. Expect to fix small things on the first build.
- **No key-file or Yubikey handling**, and none needed: those live in the
  KeePass application.
- **Sessions are not listable or revocable individually** beyond
  `keeporg logout` on the device itself. Adding `GET`/`DELETE
  /api/v1/auth/sessions` is a small, well-bounded job.
- **Android does not restart its watcher at boot.** Add a
  `RECEIVE_BOOT_COMPLETED` receiver if that matters more than the permission
  it costs.