Merge concurrent saves on the server, with vaults created from the web

Why: the point of a sync server is that two people can work at once. Picking
a winner was all this one could do while it could not read a vault.

What, end to end:
- A vault is created at /web or with `keeporg vault -create`: the server
  builds an empty KDBX 4 file, encrypts it with the password from the form,
  and stores it as version 1. An existing database can adopt a stored key
  with `keeporg db -id <id> -store-password`, which is verified against the
  stored file before it is accepted.
- A stale push to a `merge` vault is no longer refused. The server opens the
  stored version and the pushed one in memory, folds them together, and
  stores the re-encrypted result. Clients push as usual and pull what comes
  back. Any failure answers 409 merge_failed and the client keeps its copy
  aside, so a failed merge still loses nothing.
- At rest: vault contents stay KDBX-encrypted as always. A merge vault's
  password is kept AES-256-GCM-wrapped under a master key that lives outside
  the data directory, bound to the vault id, never returned by the API. A
  stolen data directory is ciphertext; a stolen data directory plus the
  master key is not. The server runs without a master key and simply refuses
  to create or merge vaults.

This moves the server inside the trust boundary for vaults it can merge,
which is inherent to merging and is stated plainly in the README, the
handover notes and the web form itself. Key-less strategies are unchanged and
remain the default for a database created without a password.

Proved: two clients editing the same version are merged rather than refused,
the newer edit of one entry wins, a merge failure leaves the stored vault
untouched, `merge` is refused without a stored key, the wrapped key never
appears in state.json or the API, and the web interface rejects a form
without its CSRF token, withholds its cookie over plain http, and only offers
groups the user administers. Checked by hand against the real thing: a vault
created in a browser, edited by keepassxc-cli on two sides from the same
version, merged by the server, and both entries read back out with
keepassxc-cli. make check, make test-race, android lint and both .deb builds
are clean.
This commit is contained in:
2026-10-29 17:34:47 +01:00
parent 4d2f0b047b
commit a9308a3e89
33 changed files with 2168 additions and 88 deletions
+20 -8
View File
@@ -4,19 +4,29 @@ A synchronization adapter for KeePass. Your databases stay ordinary `.kdbx`
files that KeePassXC, KeePass2 or KeePassDX opens directly; a server keeps files that KeePassXC, KeePass2 or KeePassDX opens directly; a server keeps
them in step between everyone who has access. them in step between everyone who has access.
The server never opens a database and cannot read one. Master passwords stay When two people save at once, the server can **merge** both edits: it opens
on the clients. the two copies in memory, folds them together by KeePass's own rules and
stores the result. Nothing is lost, and no one has to reconcile two files by
hand.
Merging means decrypting, so a merge vault's password is kept on the server,
encrypted under a master key held outside the data directory. That makes the
server part of your trust boundary. Vaults on any other conflict strategy
keep their key off the server entirely, exactly as before.
- **Server** (`keeporgd`, Linux): hosts many KDBX databases, controls access - **Server** (`keeporgd`, Linux): hosts many KDBX databases, controls access
with a tree of user groups, accepts registrations by invite link or OIDC, with a tree of user groups, accepts registrations by invite link or OIDC,
and pushes updates to connected clients. merges concurrent edits, and pushes updates to connected clients. Its web
page at `/web` is for creating a vault and setting its password.
- **Desktop client** (`keeporg`, Linux, macOS, Windows, BSD): mirrors every - **Desktop client** (`keeporg`, Linux, macOS, Windows, BSD): mirrors every
assigned database into one directory as a real file, pushes each save and assigned database into one directory as a real file, pushes each save and
applies what others push. applies what others push.
- **Android client**: the same, published to every file picker as a storage - **Android client**: the same, published to every file picker as a storage
provider. provider.
No third-party dependencies, no cgo, no FUSE. No third-party dependencies, no cgo, no FUSE. The KDBX 4 codec, including
ChaCha20, BLAKE2b and Argon2, is part of the repository and is tested against
both the relevant RFCs and a real KeePassXC.
## Quick start ## Quick start
@@ -30,13 +40,15 @@ KEEPORG_ADMIN_PASSWORD=dev-password-1 make run-server
# in another shell # in another shell
build/linux-arm64/keeporg login -server http://127.0.0.1:8765 -user admin build/linux-arm64/keeporg login -server http://127.0.0.1:8765 -user admin
build/linux-arm64/keeporg groups build/linux-arm64/keeporg groups
build/linux-arm64/keeporg db -create "Team Vault" -group <group-id> build/linux-arm64/keeporg vault -create "Team Vault" -group <group-id>
build/linux-arm64/keeporg run build/linux-arm64/keeporg run
``` ```
Then save a new KDBX database as `Team Vault.kdbx` in `~/KeepOrg` from your `Team Vault.kdbx` appears in `~/KeepOrg`; open it in KeePassXC with the vault
KeePass application. It becomes version 1 on the server, and every later save password you just set. Every save is pushed, and a save that races with a
is pushed automatically. colleague's is merged rather than refused.
Or open `http://127.0.0.1:8765/web` and create the vault from the browser.
## Packaging ## Packaging
@@ -41,7 +41,14 @@ data class DeviceStart(
val expiresIn: Int, val expiresIn: Int,
) )
data class PushResult(val version: Long, val sha256: String, val unchanged: Boolean) data class PushResult(
val version: Long,
val sha256: String,
val unchanged: Boolean,
/** Set when the server folded this push into another edit. */
val merged: Boolean = false,
val mergeSummary: String = "",
)
/** /**
* The KeepOrg HTTP API over the platform's own HTTP client. The server * The KeepOrg HTTP API over the platform's own HTTP client. The server
@@ -131,7 +138,13 @@ class Api(baseUrl: String, var token: String?) {
val status = c.responseCode val status = c.responseCode
if (status != 200) throw errorOf(c, status) if (status != 200) throw errorOf(c, status)
val o = JSONObject(c.inputStream.use { it.readAllBytesCompat() }.decodeToString()) val o = JSONObject(c.inputStream.use { it.readAllBytesCompat() }.decodeToString())
return PushResult(o.getLong("version"), o.optString("sha256"), o.optBoolean("unchanged")) return PushResult(
o.getLong("version"),
o.optString("sha256"),
o.optBoolean("unchanged"),
o.optBoolean("merged"),
o.optString("merge_summary"),
)
} finally { } finally {
c.disconnect() c.disconnect()
} }
@@ -20,7 +20,7 @@ import java.util.Locale
*/ */
class Mirror(private val context: Context, private val api: Api, private val prefs: Prefs) { class Mirror(private val context: Context, private val api: Api, private val prefs: Prefs) {
enum class Action { NOTHING, PULL, PUSH, FORCE_PUSH, SERVER_WINS, RESTORE, MANUAL } enum class Action { NOTHING, PULL, PUSH, PUSH_FOR_MERGE, FORCE_PUSH, SERVER_WINS, RESTORE, MANUAL }
/** The mirror directory, private to the app. */ /** The mirror directory, private to the app. */
val dir: File = File(context.filesDir, "databases").apply { mkdirs() } val dir: File = File(context.filesDir, "databases").apply { mkdirs() }
@@ -79,7 +79,7 @@ class Mirror(private val context: Context, private val api: Api, private val pre
Action.PULL -> pull(db, st, file) Action.PULL -> pull(db, st, file)
Action.PUSH -> push(db, st, file, local, force = false) Action.PUSH, Action.PUSH_FOR_MERGE -> push(db, st, file, local, force = false)
Action.FORCE_PUSH -> push(db, st, file, local, force = true) Action.FORCE_PUSH -> push(db, st, file, local, force = true)
@@ -118,6 +118,16 @@ class Mirror(private val context: Context, private val api: Api, private val pre
val res = api.push(db.id, st.version, data, force) val res = api.push(db.id, st.version, data, force)
st.version = res.version st.version = res.version
st.sha256 = res.sha256 st.sha256 = res.sha256
if (res.merged) {
// The result is neither copy: take what the server holds.
pull(db, st, file)
prefs.putStatus(db.id, "merged: ${res.mergeSummary}")
return
}
if (res.sha256 != digest(data)) {
pull(db, st, file)
return
}
prefs.putStatus(db.id, if (res.unchanged) "in sync" else "pushed version ${res.version}") prefs.putStatus(db.id, if (res.unchanged) "in sync" else "pushed version ${res.version}")
} catch (e: ApiException) { } catch (e: ApiException) {
when { when {
@@ -210,6 +220,9 @@ class Mirror(private val context: Context, private val api: Api, private val pre
!localChanged && remoteChanged -> Action.PULL !localChanged && remoteChanged -> Action.PULL
localChanged && remoteChanged -> when { localChanged && remoteChanged -> when {
!canWrite -> Action.RESTORE !canWrite -> Action.RESTORE
// The server holds this vault's key and merges both
// copies, so send ours and take what comes back.
strategy == "merge" -> Action.PUSH_FOR_MERGE
strategy == "client-wins" -> Action.FORCE_PUSH strategy == "client-wins" -> Action.FORCE_PUSH
strategy == "manual" -> if (conflictSeen) Action.NOTHING else Action.MANUAL strategy == "manual" -> if (conflictSeen) Action.NOTHING else Action.MANUAL
else -> Action.SERVER_WINS else -> Action.SERVER_WINS
@@ -16,12 +16,15 @@ Description: |-
Keepass2Android opens them as ordinary files. Keepass2Android opens them as ordinary files.
Each save is pushed to the server, and changes other people push are applied Each save is pushed to the server, and changes other people push are applied
here. When two people save at once, the database's merge strategy decides here. When two people save at once, a merge vault has both edits folded
the outcome; the losing copy is always kept beside the file so nothing is together by the server; on any other setting the database's conflict
strategy decides, and the losing copy is kept beside the file so nothing is
lost silently. lost silently.
The app never opens a database and never sees a master password. It moves The app itself never opens a database and never sees a master password. It
encrypted bytes and nothing else. moves encrypted bytes and nothing else. Note that merging requires the
server to hold that vault's key, which is a choice made per vault when it
is created.
Sign in with a password and the server's built-in time-based second factor, Sign in with a password and the server's built-in time-based second factor,
or approve the device in a browser, which is also how single sign-on works. or approve the device in a browser, which is also how single sign-on works.
@@ -38,7 +41,15 @@ Builds:
- yes - yes
output: android/app/build/outputs/apk/release/app-release-unsigned.apk output: android/app/build/outputs/apk/release/app-release-unsigned.apk
- versionName: 0.2.0
versionCode: 2
commit: v0.2.0-android
subdir: android/app
gradle:
- yes
output: android/app/build/outputs/apk/release/app-release-unsigned.apk
AutoUpdateMode: None AutoUpdateMode: None
UpdateCheckMode: Tags UpdateCheckMode: Tags
CurrentVersion: 0.1.0 CurrentVersion: 0.2.0
CurrentVersionCode: 1 CurrentVersionCode: 2
+80 -2
View File
@@ -375,16 +375,37 @@ func cmdDB(ctx context.Context, args []string) error {
path := configFlag(fs) path := configFlag(fs)
create := fs.String("create", "", "create a database with this name") create := fs.String("create", "", "create a database with this name")
group := fs.String("group", "", "owning group id, required with -create") group := fs.String("group", "", "owning group id, required with -create")
strategy := fs.String("strategy", "", "merge strategy: server-wins, client-wins or manual") strategy := fs.String("strategy", "",
"merge strategy: merge, server-wins, client-wins or manual")
id := fs.String("id", "", "database id, for -strategy, -rename or -delete") id := fs.String("id", "", "database id, for -strategy, -rename or -delete")
rename := fs.String("rename", "", "new name for -id") rename := fs.String("rename", "", "new name for -id")
del := fs.Bool("delete", false, "delete the database given by -id") del := fs.Bool("delete", false, "delete the database given by -id")
storePassword := fs.Bool("store-password", false,
"give the server this vault's password so it can merge the vault")
forgetPassword := fs.Bool("forget-password", false,
"remove the vault password the server keeps")
_ = fs.Parse(args) _ = fs.Parse(args)
_, api, err := session(*path) _, api, err := session(*path)
if err != nil { if err != nil {
return err return err
} }
switch { switch {
case *id != "" && *storePassword:
password, perr := readPassword("Vault password: ")
if perr != nil {
return perr
}
if err := api.StoreVaultKey(ctx, *id, password); err != nil {
return err
}
fmt.Println("stored; this database can now use -strategy merge")
return nil
case *id != "" && *forgetPassword:
if err := api.ForgetVaultKey(ctx, *id); err != nil {
return err
}
fmt.Println("the server no longer holds this vault's password")
return nil
case *create != "": case *create != "":
if *group == "" { if *group == "" {
return errors.New("-group is required with -create") return errors.New("-group is required with -create")
@@ -414,11 +435,48 @@ func cmdDB(ctx context.Context, args []string) error {
} }
fmt.Println("deleted on the server; local files were left in place") fmt.Println("deleted on the server; local files were left in place")
default: default:
return errors.New("use -create <name> -group <id>, or -id <id> with -strategy, -rename or -delete") return errors.New("use -create <name> -group <id>, or -id <id> with -strategy, " +
"-rename, -store-password, -forget-password or -delete")
} }
return nil return nil
} }
func cmdVault(ctx context.Context, args []string) error {
fs := flag.NewFlagSet("vault", flag.ExitOnError)
path := configFlag(fs)
create := fs.String("create", "", "name of the vault to create")
group := fs.String("group", "", "owning group id")
strategy := fs.String("strategy", "merge",
"merge, server-wins, client-wins or manual")
_ = fs.Parse(args)
cfg, api, err := session(*path)
if err != nil {
return err
}
if *create == "" || *group == "" {
return errors.New("use -create <name> -group <id>")
}
if *strategy == "merge" {
fmt.Println("A merge vault is stored with its key on the server, which is what lets")
fmt.Println("the server fold two people's edits together. The server can then read")
fmt.Println("this vault. Use -strategy server-wins to keep the key off the server.")
fmt.Println()
}
password, err := readNewPassword("Vault password: ")
if err != nil {
return err
}
db, err := api.CreateVault(ctx, *create, *group, *strategy, password)
if err != nil {
return err
}
fmt.Printf("created vault %s (%s), strategy %s\n", db.Name, db.ID, db.MergeStrategy)
fmt.Printf("it will appear in %s as %s.kdbx once the agent syncs; open it with the\n",
cfg.MirrorDir, db.Name)
fmt.Println("password you just set.")
return nil
}
func cmdGrant(ctx context.Context, args []string) error { func cmdGrant(ctx context.Context, args []string) error {
fs := flag.NewFlagSet("grant", flag.ExitOnError) fs := flag.NewFlagSet("grant", flag.ExitOnError)
path := configFlag(fs) path := configFlag(fs)
@@ -534,6 +592,26 @@ func readPassword(label string) (string, error) {
return strings.TrimSpace(line), err return strings.TrimSpace(line), err
} }
// readNewPassword asks twice, so a vault is never created with a password
// its owner cannot reproduce.
func readNewPassword(label string) (string, error) {
first, err := readPassword(label)
if err != nil {
return "", err
}
if len(first) < 12 {
return "", errors.New("a vault password must be at least 12 characters")
}
again, err := readPassword("Repeat: ")
if err != nil {
return "", err
}
if first != again {
return "", errors.New("the two passwords do not match")
}
return first, nil
}
// hideInput turns terminal echo off where a terminal and stty are available. // hideInput turns terminal echo off where a terminal and stty are available.
// Elsewhere the password is typed visibly, which is stated by the prompt. // Elsewhere the password is typed visibly, which is stated by the prompt.
func hideInput() (restore func(), hidden bool) { func hideInput() (restore func(), hidden bool) {
+1
View File
@@ -38,6 +38,7 @@ func main() {
{"groups", "list groups you can see", cmdGroups}, {"groups", "list groups you can see", cmdGroups},
{"group", "create or delete a group", cmdGroup}, {"group", "create or delete a group", cmdGroup},
{"member", "add or remove a group member", cmdMember}, {"member", "add or remove a group member", cmdMember},
{"vault", "create a new KDBX vault on the server and set its password", cmdVault},
{"db", "create, rename, re-strategy or delete a database", cmdDB}, {"db", "create, rename, re-strategy or delete a database", cmdDB},
{"grant", "give or take a group's access to a database", cmdGrant}, {"grant", "give or take a group's access to a database", cmdGrant},
{"invite", "create or list invite links", cmdInvite}, {"invite", "create or list invite links", cmdInvite},
+21
View File
@@ -18,6 +18,7 @@ import (
"syscall" "syscall"
"time" "time"
"cc.tiredsysadmin.keeporg/internal/kdbx"
"cc.tiredsysadmin.keeporg/internal/secret" "cc.tiredsysadmin.keeporg/internal/secret"
"cc.tiredsysadmin.keeporg/internal/server" "cc.tiredsysadmin.keeporg/internal/server"
"cc.tiredsysadmin.keeporg/internal/store" "cc.tiredsysadmin.keeporg/internal/store"
@@ -36,13 +37,31 @@ type fileConfig struct {
MaxDBSizeMB int64 `json:"max_db_size_mb"` MaxDBSizeMB int64 `json:"max_db_size_mb"`
TLSCert string `json:"tls_cert"` TLSCert string `json:"tls_cert"`
TLSKey string `json:"tls_key"` TLSKey string `json:"tls_key"`
MasterKeyFile string `json:"master_key_file"`
VaultKDF vaultKDF `json:"vault_kdf"`
OIDC server.OIDCConfig `json:"oidc"` OIDC server.OIDCConfig `json:"oidc"`
} }
// vaultKDF is the work factor for vaults this server creates, in the plain
// units an operator wants to edit.
type vaultKDF struct {
MemoryMB uint32 `json:"memory_mb"`
Iterations uint32 `json:"iterations"`
Lanes uint8 `json:"lanes"`
}
func (f fileConfig) toConfig() server.Config { func (f fileConfig) toConfig() server.Config {
c := server.Config{ c := server.Config{
Addr: f.Addr, DataDir: f.DataDir, ExternalURL: f.ExternalURL, Addr: f.Addr, DataDir: f.DataDir, ExternalURL: f.ExternalURL,
TLSCert: f.TLSCert, TLSKey: f.TLSKey, OIDC: f.OIDC, TLSCert: f.TLSCert, TLSKey: f.TLSKey, OIDC: f.OIDC,
MasterKeyFile: f.MasterKeyFile,
}
if f.VaultKDF.MemoryMB > 0 {
c.VaultKDF = kdbx.KDFParams{
MemoryKiB: f.VaultKDF.MemoryMB * 1024,
Iterations: f.VaultKDF.Iterations,
Lanes: f.VaultKDF.Lanes,
}
} }
if f.SessionTTLHours > 0 { if f.SessionTTLHours > 0 {
c.SessionTTL = time.Duration(f.SessionTTLHours) * time.Hour c.SessionTTL = time.Duration(f.SessionTTLHours) * time.Hour
@@ -193,6 +212,8 @@ func writeExampleConfig(path string) error {
ExternalURL: "https://keeporg.example.com", ExternalURL: "https://keeporg.example.com",
SessionTTLHours: 24 * 90, SessionTTLHours: 24 * 90,
MaxDBSizeMB: 32, MaxDBSizeMB: 32,
MasterKeyFile: "/etc/keeporg/master.key",
VaultKDF: vaultKDF{MemoryMB: 32, Iterations: 2, Lanes: 2},
OIDC: server.OIDCConfig{ OIDC: server.OIDCConfig{
Issuer: "https://id.example.com/realms/main", ClientID: "keeporg", Issuer: "https://id.example.com/realms/main", ClientID: "keeporg",
ClientSecret: "change-me", Scopes: "openid profile email", AutoCreate: false, ClientSecret: "change-me", Scopes: "openid profile email", AutoCreate: false,
+205 -22
View File
@@ -5,9 +5,21 @@ server holds many KDBX files and arbitrates who wrote last; clients present
each assigned database as an ordinary `.kdbx` file that any KeePass each assigned database as an ordinary `.kdbx` file that any KeePass
application can open, and synchronize it on every save. application can open, and synchronize it on every save.
The server never opens a database. It moves encrypted bytes and metadata There are two kinds of database here, and the difference is the whole
about them. Master passwords and key files stay on the clients, so a security model:
compromised server leaks the ciphertext and the group structure, nothing else.
- **A key-less database** is one the server only ever moves. It cannot be
opened here, so a compromised server leaks ciphertext and the group
structure and nothing else. Conflicts are settled by picking a winner.
- **A merge vault** is one the server can open, because it keeps that
vault's password. That is the price of folding two people's edits
together: merging means decrypting. Whoever controls the server and its
master key can read every merge vault on it.
Both are first-class. A deployment that wants no server-side key uses
`server-wins`, `client-wins` or `manual` and loses nothing but the merge. A
deployment that wants merging accepts that the server is now part of its
trust boundary, and should treat the host accordingly.
## Repository map ## Repository map
@@ -16,7 +28,8 @@ cmd/keeporgd/ server entry point: flags, config file, bootstrap
cmd/keeporg/ desktop client: login, run, sync, status, administration cmd/keeporg/ desktop client: login, run, sync, status, administration
internal/store/ domain model, JSON metadata store, ACL resolution internal/store/ domain model, JSON metadata store, ACL resolution
internal/blob/ versioned KDBX payload storage on disk internal/blob/ versioned KDBX payload storage on disk
internal/server/ HTTP surface: auth, OIDC, sync, admin, SSE hub, pages internal/kdbx/ KDBX 4 codec and the merge engine, with its own crypto
internal/server/ HTTP surface: auth, OIDC, sync, admin, SSE hub, web, pages
internal/client/ shared client engine: API wrapper, merge policy, mirror internal/client/ shared client engine: API wrapper, merge policy, mirror
internal/secret/ password verifiers, tokens, identifiers internal/secret/ password verifiers, tokens, identifiers
internal/totp/ RFC 6238 second factor internal/totp/ RFC 6238 second factor
@@ -34,9 +47,12 @@ Makefile the only entry point for building and publishing
One process, one data directory, no database engine: One process, one data directory, no database engine:
- `state.json` holds users, groups, memberships, databases, grants, invites, - `state.json` holds users, groups, memberships, databases, grants, invites,
sessions and pending device logins. sessions and pending device logins. For a merge vault it also holds that
vault's password, encrypted under the server's master key.
- `blobs/<database-id>/<version>.kdbx` holds the payloads, keeping the last - `blobs/<database-id>/<version>.kdbx` holds the payloads, keeping the last
20 versions of each database. 20 versions of each database.
- The master key itself lives **outside** the data directory, so a copy of
the data directory alone yields ciphertext.
`internal/store.Store` is the only writer of `state.json`. Every mutation `internal/store.Store` is the only writer of `state.json`. Every mutation
runs through `Update(func(*State) error)`, which applies the change to a runs through `Update(func(*State) error)`, which applies the change to a
@@ -74,6 +90,33 @@ event stream open so colleagues' changes arrive without opening the app.
`Mirror.decide` is a transcription of the Go decision table. **Keep the two in `Mirror.decide` is a transcription of the Go decision table. **Keep the two in
step**: a change to one belongs in the other in the same commit. step**: a change to one belongs in the other in the same commit.
### The KDBX codec (`internal/kdbx`)
A real KeePass 2 implementation, because merging cannot be done on
guesswork. It reads and writes KDBX 4.0 and 4.1 with AES-256-CBC or ChaCha20
payloads, AES-KDF or Argon2d/Argon2id derivation, and the ChaCha20 inner
stream that protects individual values. ChaCha20, BLAKE2b and Argon2 are
implemented in the package: the standard library has none of them, and this
repository carries no third-party dependencies. Each is checked against its
RFC's test vectors.
Two decisions in there are worth keeping:
**The XML payload is a generic node tree, not Go structs.** A merge server
that dropped elements it did not model would silently destroy data written by
a newer KeePass. Everything unrecognised round-trips untouched, and the merge
logic reaches only into the paths it understands: UUID, Times, String,
History, Binary, DeletedObjects, Meta.
**Anything unsupported is reported, never guessed.** A KDBX 3 file, a Twofish
payload or a Salsa20 inner stream comes back as `*kdbx.ErrUnsupported`, and
the server then falls back to a non-merging strategy rather than mangling the
file.
The format is verified against the real thing: the test suite has
keepassxc-cli read, decrypt and re-save vaults this code writes, and has this
code read what KeePassXC saved.
## Access control ## Access control
Groups form one tree. Two rules decide everything: Groups form one tree. Two rules decide everything:
@@ -122,19 +165,91 @@ Set per database, changed by an administrator of the owning group
| Strategy | Server behaviour on a stale push | Client behaviour | | Strategy | Server behaviour on a stale push | Client behaviour |
| --- | --- | --- | | --- | --- | --- |
| `server-wins` (default) | rejects with `409 conflict` | keeps the local save as `<name>.conflict-<timestamp>.kdbx`, then writes the server copy | | `merge` | opens both copies in memory, folds them together, stores the result as a new version | pushes as usual, then pulls the merged result |
| `server-wins` | rejects with `409 conflict` | keeps the local save as `<name>.conflict-<timestamp>.kdbx`, then writes the server copy |
| `client-wins` | accepts a retry carrying `If-Match: *` | retries with force; the server copy is overwritten and kept in history | | `client-wins` | accepts a retry carrying `If-Match: *` | retries with force; the server copy is overwritten and kept in history |
| `manual` | rejects, and refuses force | writes the server copy as `<name>.server-v<n>.kdbx` beside the file, touches nothing, reports a conflict once | | `manual` | rejects, and refuses force | writes the server copy as `<name>.server-v<n>.kdbx` beside the file, touches nothing, reports a conflict once |
`merge` is the default for a vault created through the web interface;
`server-wins` is the default for a database created without a password,
because the server has no key for it.
A losing save is never discarded: it is always written somewhere the user can A losing save is never discarded: it is always written somewhere the user can
find it. A read-only user who edits their copy gets the same treatment, with find it. A read-only user who edits their copy gets the same treatment, with
the server copy restored. the server copy restored.
**There is no entry-level merge.** The server cannot perform one, because it ### What merging actually does
cannot decrypt a database. A true three-way merge belongs in a client that
already holds the master password; the strategy field is the hook for it, and `kdbx.Merge` follows KeePass's own synchronization rules. Objects are matched
`manual` is the honest behaviour until then. KeePassXC's own "Merge from by UUID across the two copies:
database" on the two files is the current answer.
- An object only one side has is added, unless the other side deleted it
more recently than it was last modified.
- An object both sides changed is resolved by `Times/LastModificationTime`.
The newer version wins and **the loser is filed in the entry's history**,
so a merge never destroys an edit, it demotes one.
- A deletion applies only when it is newer than the other side's edit.
Otherwise the edit resurrects the object and the stale deletion record is
dropped.
- Groups keep their nesting; an object moves when the other side moved it
more recently, by `Times/LocationChanged`.
- Database settings follow `Meta/SettingsChanged`; custom data and icons are
unioned by key.
- Attachments are remapped from the incoming pool into the target's. An entry
that kept its old index would silently gain the wrong file.
The ordering of objects inside a group is not merged, and neither does
KeePass reconcile it.
Where it runs: `internal/server/merge.go` decrypts the stored version and the
pushed one, merges, re-encrypts, and stores the result as a new version. The
plaintext exists only as two `kdbx.Database` values in memory for the
duration of one request; nothing is ever written to disk in the clear. If any
step fails — most often because a client changed the vault's own password, so
the stored key no longer opens it — the push is answered with
`409 merge_failed` and the client falls back to keeping its copy aside. A
failed merge never loses an edit.
**The cost.** A merge runs the vault's key derivation three times: the stored
version, the pushed version, and the re-encryption, each with a different
salt. At the default 32 MiB / 2 iterations that is roughly 150 ms and 100 MB
of transient memory per merging push. `vault_kdf` in the configuration is
the dial; a transformed-key cache would remove most of it and has not been
written.
## Key custody and encryption at rest
Three different things are encrypted here, and they are worth keeping apart:
1. **The vault contents.** Always encrypted, by KDBX itself: AES-256 under a
key derived with Argon2d from the vault password. This is true on the
server's disk, on every client's disk and on the wire. The server stores
exactly the file a KeePass application would.
2. **The vault password**, for merge vaults only. Encrypted with AES-256-GCM
under the server's master key, with the vault's own id as additional
authenticated data, so a wrapped secret moved to another vault's record
fails to open instead of unlocking the wrong database. It is never
returned by the API, never logged, and never leaves the process.
3. **Account passwords.** Not stored at all; only PBKDF2-SHA256 verifiers.
The master key comes from `KEEPORG_MASTER_KEY` (base64) or from
`master_key_file`, which defaults to `/etc/keeporg/master.key` and is created
with mode 0600 on first start. Without it the server still runs and still
serves every key-less strategy; only vault creation and merging are refused,
with a line in the log saying so.
What this buys and what it does not:
- A stolen copy of `/var/lib/keeporg` is useless on its own. The vaults are
ciphertext and the wrapped keys do not open without the master key.
- A stolen copy of the data directory **and** the master key opens every
merge vault. Keep them on different media, or feed the key in through
`KEEPORG_MASTER_KEY` from a secret manager or a systemd credential so it
never touches the disk at all.
- Backups must cover both, separately. A backup of the data directory with no
master key cannot be used to merge after a restore; the vaults still open
for their users, and the keys can be stored again with
`keeporg db -id <id> -store-password`.
## Authentication ## Authentication
@@ -159,12 +274,34 @@ userinfo endpoint, so there is no local JWT verification to get wrong. An
identity links to an existing account by `preferred_username` or `email`, or identity links to an existing account by `preferred_username` or `email`, or
provisions one when `oidc.auto_create` is set. provisions one when `oidc.auto_create` is set.
## The web interface
The server has one browsable page set, at `/web`, and it exists for the one
job a command line does badly: creating a vault and setting its password.
Sign in, see which vaults you can reach, create one. It never shows a vault's
contents — the server opens a vault only to merge it — and there is no
entry browsing, by design.
Browser sessions are the same session tokens the API uses, carried in a
cookie that is `HttpOnly`, `SameSite=Strict`, `Path=/`, and `Secure` whenever
`external_url` is https. Every form carries a per-session CSRF token that is
checked on each POST. The group selector only offers groups the signed-in
user administers, and a user who administers none is told so instead of being
shown a form that would fail.
Creating a vault is one transaction: the server builds an empty KDBX 4
database, encrypts it with the password from the form, stores it as version
1, and — for a merge vault — wraps the password under the master key. If the
state update fails, the blob is removed again, so a half-created vault cannot
be left behind.
## Decisions worth knowing ## Decisions worth knowing
**No third-party Go dependencies, no cgo.** The whole server and desktop **No third-party Go dependencies, no cgo.** The whole server and desktop
client build with `CGO_ENABLED=0` from the standard library: SSE instead of a client build with `CGO_ENABLED=0` from the standard library: SSE instead of a
WebSocket library, `crypto/pbkdf2` (new in Go 1.24) instead of an Argon2 WebSocket library, `crypto/pbkdf2` (new in Go 1.24) for account passwords,
package, 60 lines of HMAC for TOTP, `html/template` for the three pages. hand-written ChaCha20, BLAKE2b and Argon2 for the KDBX format, 60 lines of
HMAC for TOTP, `html/template` for the pages.
Cross-compiling to five platforms is then one `go build` each, and there is Cross-compiling to five platforms is then one `go build` each, and there is
no dependency tree to audit for a program that guards passwords. no dependency tree to audit for a program that guards passwords.
@@ -218,6 +355,8 @@ an `everyone` group, and logs the generated password once. Set
"external_url": "https://keeporg.example.com", "external_url": "https://keeporg.example.com",
"session_ttl_hours": 2160, "session_ttl_hours": 2160,
"max_db_size_mb": 32, "max_db_size_mb": 32,
"master_key_file": "/etc/keeporg/master.key",
"vault_kdf": { "memory_mb": 32, "iterations": 2, "lanes": 2 },
"oidc": { "oidc": {
"issuer": "https://id.example.com/realms/main", "issuer": "https://id.example.com/realms/main",
"client_id": "keeporg", "client_id": "keeporg",
@@ -233,23 +372,37 @@ an `everyone` group, and logs the generated password once. Set
pages and the OIDC redirect URI are built from it. Terminate TLS in front of pages and the OIDC redirect URI are built from it. Terminate TLS in front of
the server, or set `tls_cert` and `tls_key`. the server, or set `tls_cert` and `tls_key`.
Back up `/var/lib/keeporg`. It is the whole server: `state.json` plus the Back up `/var/lib/keeporg` **and** `/etc/keeporg/master.key`, separately.
blobs. There is nothing else to dump. The data directory is the whole server: `state.json` plus the blobs. The
master key is what turns the wrapped vault keys in it back into vault
passwords; without it a restored server can still serve, but can no longer
merge.
### Setting up a team ### Setting up a team
```sh ```sh
keeporg login -server https://keeporg.example.com -user admin keeporg login -server https://keeporg.example.com -user admin
keeporg group -create engineering -parent <everyone-id> keeporg group -create engineering -parent <everyone-id>
keeporg db -create "Engineering" -group <engineering-id> keeporg vault -create "Engineering" -group <engineering-id> # prompts for the vault password
keeporg invite -group <engineering-id> -role member -uses 5 -hours 72 keeporg invite -group <engineering-id> -role member -uses 5 -hours 72
keeporg grant -group <readers-id> -db <db-id> -perm read keeporg grant -group <readers-id> -db <db-id> -perm read
keeporg db -id <db-id> -strategy manual
``` ```
A new database has no bytes until someone saves one. Create the KDBX in your Or create the vault at `https://keeporg.example.com/web`, which is the same
KeePass application under the exact name the server shows, inside the mirror operation with a form around it.
directory; the client adopts it and pushes it as version 1.
`keeporg vault` creates the KDBX file itself, so the vault has content from
the start: it appears in every member's mirror directory and opens with the
password that was set. `keeporg db -create` is the other path — it registers
a name with no bytes, and the first client to save a KDBX of that name in its
mirror directory adopts it as version 1. A database created that way has no
key on the server, so it cannot merge until one is stored:
```sh
keeporg db -id <db-id> -store-password # checked against the stored file
keeporg db -id <db-id> -strategy merge
keeporg db -id <db-id> -forget-password # the server can no longer read it
```
## Versioning and publishing ## Versioning and publishing
@@ -294,6 +447,21 @@ What the suite actually covers:
pass over login, stale push, forced push, idempotent push, read-only pass over login, stale push, forced push, idempotent push, read-only
grants, invite redemption and reuse, TOTP enrolment, device login and the grants, invite redemption and reuse, TOTP enrolment, device login and the
event stream. event stream.
- `internal/kdbx` for ChaCha20 (RFC 8439), BLAKE2b (RFC 7693) and all three
Argon2 variants (RFC 9106) against their published vectors; round trips
that preserve unknown elements; rejection of a wrong password, a flipped
payload byte and a flipped header byte; and twelve merge cases covering
both conflict directions with history, deletion, resurrection, nesting,
relocation, attachment remapping, idempotence and order independence.
Three tests drive keepassxc-cli: it reads what this code writes, this code
reads what it writes back, and it decrypts an entry that reached a vault
only through a merge.
- `internal/server` additionally for vault creation, two clients editing the
same version being merged rather than refused, the newer edit of one entry
winning, a merge failure falling back to a conflict with the stored vault
untouched, the refusal to switch to `merge` without a stored key, and the
web interface: cookie hardening, CSRF rejection, group scoping and that a
vault created from the form opens with its password.
- `internal/client` for the decision table, and for the mirror driven against - `internal/client` for the decision table, and for the mirror driven against
a real in-process server: adopting a local file, following a remote push, a real in-process server: adopting a local file, following a remote push,
server-wins, client-wins, manual, read-only restore, server-side rename, server-wins, client-wins, manual, read-only restore, server-side rename,
@@ -308,8 +476,23 @@ what an arm64 Linux build host needs.
- **One server node.** The JSON store is single-writer by design. Horizontal - **One server node.** The JSON store is single-writer by design. Horizontal
scale means replacing `internal/store`. scale means replacing `internal/store`.
- **No entry-level merge.** See the strategies section. The pluggable place - **Merging requires the server to hold the vault key.** That is inherent, not
is the client, which holds the key. an implementation gap: the alternative is a client-side merge, which would
mean the pushing client fetching the server copy, merging locally and
pushing the result. That is a worthwhile future option and would let a
deployment keep the key off the server entirely.
- **KDBX 3 files cannot be merged.** They are reported as unsupported, not
mangled; re-save such a file as KDBX 4 in KeePassXC, or keep it on a
key-less strategy. Reading KDBX 3 is a contained addition: a different
payload layout and the Salsa20 inner stream.
- **A merge vault must keep one password.** If a client changes the vault's
own password, the server's stored key stops working and pushes come back as
`merge_failed` until an administrator stores the new one. The server does
not detect the change in advance.
- **Key files and hardware keys are not supported for merge vaults**, because
the server would have to hold those too. They work fine on any key-less
strategy, where the server never opens the file.
- **No transformed-key cache.** Each merge pays the Argon2 cost three times.
- **The Android app has never run on a device.** It compiles clean, lint - **The Android app has never run on a device.** It compiles clean, lint
reports no issues and the DEX carries every class the manifest names, but reports no issues and the DEX carries every class the manifest names, but
no emulator or phone was available here. The first real run is where to no emulator or phone was available here. The first real run is where to
+40 -5
View File
@@ -5,8 +5,10 @@ Every `/api/` endpoint takes and returns JSON and needs
endpoints. Errors carry `{"code": "...", "error": "..."}`; `code` is stable endpoints. Errors carry `{"code": "...", "error": "..."}`; `code` is stable
and meant to be matched on, `error` is for humans. and meant to be matched on, `error` is for humans.
Three endpoints serve HTML, because a browser is the right place for them: HTML is served at `/web` (sign in, list vaults, create one) and at `/device`,
`/device`, `/invite/{token}` and `/oidc/callback`. `/invite/{token}` and `/oidc/callback`. The web pages use the same session
tokens in a cookie that is `HttpOnly`, `SameSite=Strict` and `Secure` behind
https, and every form carries a per-session CSRF token.
## Authentication ## Authentication
@@ -40,7 +42,9 @@ A device code is single use: the first successful poll consumes it.
| `If-Match` missing | `428 precondition_required` | – | | `If-Match` missing | `428 precondition_required` | – |
| version matches, bytes differ | `200` | `{version, sha256, unchanged: false}` | | version matches, bytes differ | `200` | `{version, sha256, unchanged: false}` |
| version matches, bytes identical | `200` | `{version, sha256, unchanged: true}` | | version matches, bytes identical | `200` | `{version, sha256, unchanged: true}` |
| version stale | `409 conflict` | `{current_version, merge_strategy}`, plus `ETag` and `X-KeepOrg-Strategy` | | version stale, strategy is `merge` | `200` | `{version, sha256, merged: true, merge_summary, based_on}` — the two copies were folded together and the result stored as a new version |
| version stale, strategy is `merge`, but merging failed | `409 merge_failed` | `{error, current_version, merge_strategy}`; the stored vault is untouched |
| version stale, any other strategy | `409 conflict` | `{current_version, merge_strategy}`, plus `ETag` and `X-KeepOrg-Strategy` |
| `If-Match: *`, strategy is `client-wins` | `200` | the server copy is replaced | | `If-Match: *`, strategy is `client-wins` | `200` | the server copy is replaced |
| `If-Match: *`, any other strategy | `409 force_not_allowed` | – | | `If-Match: *`, any other strategy | `409 force_not_allowed` | – |
| read-only access | `403 forbidden` | – | | read-only access | `403 forbidden` | – |
@@ -50,6 +54,10 @@ A device code is single use: the first successful poll consumes it.
`X-KeepOrg-SHA256` on a `PUT` is optional; when present it must match the `X-KeepOrg-SHA256` on a `PUT` is optional; when present it must match the
body or the request is rejected with `400 checksum_mismatch`. body or the request is rejected with `400 checksum_mismatch`.
After any accepted push, compare the `sha256` in the answer with what you
sent. If they differ the server stored something else — a merge — and the
local copy must be replaced by a fresh `GET`.
### Events ### Events
``` ```
@@ -65,6 +73,31 @@ rename, strategy change or deletion. A comment line (`: keepalive`) arrives
every 25 seconds. Reconnect on any error and reconcile fully; events are every 25 seconds. Reconnect on any error and reconcile fully; events are
notifications, never state. notifications, never state.
## Vaults
A vault is a database the server created and can open. Creating one is the
only way the server comes to hold a KDBX key, and the only strategy that
needs one is `merge`.
| Method | Path | Body | Needs |
| --- | --- | --- | --- |
| POST | `/api/v1/vaults` | `{name, owner_group_id, merge_strategy, password}` | group admin of the owner |
| POST | `/api/v1/databases/{id}/key` | `{password}` or `{forget: true}` | database admin |
`POST /api/v1/vaults` builds an empty KDBX 4 database, encrypts it with
`password`, stores it as version 1 and answers with the database view. The
password must be at least 12 characters. For `merge_strategy: "merge"`
(the default here) the password is kept, encrypted under the server's master
key; for every other strategy it is used once and forgotten. A server with no
master key refuses with `400`.
`POST /api/v1/databases/{id}/key` stores the password of a database that
already has content, after checking that it opens the stored version, which
is how an uploaded database becomes mergeable. `{"forget": true}` removes it;
that is refused while the strategy is still `merge`.
The wrapped key is never returned by any endpoint.
## Administration ## Administration
Permission needed is in the last column. "Group admin" means administering Permission needed is in the last column. "Group admin" means administering
@@ -87,5 +120,7 @@ the group named or any group above it.
| GET | `/api/v1/users` | – | yourself, plus members of groups you administer | | GET | `/api/v1/users` | – | yourself, plus members of groups you administer |
| GET | `/healthz` | – | unauthenticated liveness | | GET | `/healthz` | – | unauthenticated liveness |
`merge_strategy` is one of `server-wins`, `client-wins`, `manual`. `perm` is `merge_strategy` is one of `merge`, `server-wins`, `client-wins`, `manual`.
one of `read`, `write`, `admin`. `role` is `member` or `admin`. Switching to `merge` requires a stored key and is otherwise refused with
`400`. `perm` is one of `read`, `write`, `admin`. `role` is `member` or
`admin`.
+28
View File
@@ -155,3 +155,31 @@ func (a *API) ConfirmTOTP(ctx context.Context, code string) error {
return a.do(ctx, http.MethodPost, "/api/v1/auth/totp/confirm", return a.do(ctx, http.MethodPost, "/api/v1/auth/totp/confirm",
map[string]string{"code": code}, nil) map[string]string{"code": code}, nil)
} }
// CreateVault asks the server to create a new, empty KDBX vault encrypted
// with this password. For the merge strategy the server keeps the password,
// wrapped under its master key, because merging means opening the file.
func (a *API) CreateVault(ctx context.Context, name, groupID, strategy, password string) (*Database, error) {
var out Database
return &out, a.do(ctx, http.MethodPost, "/api/v1/vaults", map[string]string{
"name": name, "owner_group_id": groupID,
"merge_strategy": strategy, "password": password,
}, &out)
}
// StoreVaultKey gives the server the password of an existing database, so it
// can merge it. The server verifies the password against the stored file
// before accepting it.
func (a *API) StoreVaultKey(ctx context.Context, dbID, password string) error {
return a.do(ctx, http.MethodPost,
"/api/v1/databases/"+url.PathEscape(dbID)+"/key",
map[string]any{"password": password}, nil)
}
// ForgetVaultKey removes the stored password, which also ends any ability to
// merge that database here.
func (a *API) ForgetVaultKey(ctx context.Context, dbID string) error {
return a.do(ctx, http.MethodPost,
"/api/v1/databases/"+url.PathEscape(dbID)+"/key",
map[string]any{"forget": true}, nil)
}
+4
View File
@@ -107,6 +107,10 @@ type PushResult struct {
Version int64 `json:"version"` Version int64 `json:"version"`
SHA256 string `json:"sha256"` SHA256 string `json:"sha256"`
Unchanged bool `json:"unchanged"` Unchanged bool `json:"unchanged"`
// Merged is set when the server folded this push into another edit; the
// stored version is then neither copy and must be pulled.
Merged bool `json:"merged"`
MergeSummary string `json:"merge_summary"`
} }
// Login exchanges a password and optional second factor for a session. // Login exchanges a password and optional second factor for a session.
+9
View File
@@ -18,6 +18,9 @@ const (
// actManual writes the server copy beside the local file and waits for a // actManual writes the server copy beside the local file and waits for a
// human. // human.
actManual actManual
// actPushForMerge sends the local file to a server that will fold the
// two copies together and hand back the result.
actPushForMerge
) )
func (a action) String() string { func (a action) String() string {
@@ -34,6 +37,8 @@ func (a action) String() string {
return "restore" return "restore"
case actManual: case actManual:
return "manual-conflict" return "manual-conflict"
case actPushForMerge:
return "push-for-merge"
} }
return "nothing" return "nothing"
} }
@@ -82,6 +87,10 @@ func decide(s situation) action {
switch { switch {
case !s.CanWrite: case !s.CanWrite:
return actRestore return actRestore
case s.Strategy == "merge":
// The server holds this vault's key and merges the two copies,
// so the right move is to send ours and take what comes back.
return actPushForMerge
case s.Strategy == "client-wins": case s.Strategy == "client-wins":
return actForcePush return actForcePush
case s.Strategy == "manual": case s.Strategy == "manual":
+6
View File
@@ -31,6 +31,12 @@ func TestDecideTable(t *testing.T) {
{"both changed, client wins", with(func(s *situation) { {"both changed, client wins", with(func(s *situation) {
s.LocalSHA, s.RemoteSHA, s.RemoteVersion, s.Strategy = "b", "c", 3, "client-wins" s.LocalSHA, s.RemoteSHA, s.RemoteVersion, s.Strategy = "b", "c", 3, "client-wins"
}), actForcePush}, }), actForcePush},
{"both changed, merge", with(func(s *situation) {
s.LocalSHA, s.RemoteSHA, s.RemoteVersion, s.Strategy = "b", "c", 3, "merge"
}), actPushForMerge},
{"local save on a merge vault with no remote change", with(func(s *situation) {
s.LocalSHA, s.Strategy = "b", "merge"
}), actPush},
{"both changed, manual", with(func(s *situation) { {"both changed, manual", with(func(s *situation) {
s.LocalSHA, s.RemoteSHA, s.RemoteVersion, s.Strategy = "b", "c", 3, "manual" s.LocalSHA, s.RemoteSHA, s.RemoteVersion, s.Strategy = "b", "c", 3, "manual"
}), actManual}, }), actManual},
+22 -1
View File
@@ -234,7 +234,7 @@ func (m *Mirror) reconcile(ctx context.Context, db Database) error {
case actPull: case actPull:
return m.pull(ctx, db, st, path) return m.pull(ctx, db, st, path)
case actPush: case actPush, actPushForMerge:
return m.push(ctx, db, st, path, local, false) return m.push(ctx, db, st, path, local, false)
case actForcePush: case actForcePush:
@@ -297,6 +297,21 @@ func (m *Mirror) push(ctx context.Context, db Database, st *fileState, path stri
switch { switch {
case err == nil: case err == nil:
st.Version, st.SHA256 = res.Version, res.SHA256 st.Version, st.SHA256 = res.Version, res.SHA256
if res.Merged {
// The server folded our save into someone else's. What it holds
// now is neither copy, so take it.
m.log.Printf("%s: server merged version %d (%s)", db.Name, res.Version, res.MergeSummary)
if err := m.pull(ctx, db, st, path); err != nil {
return err
}
m.setStatus(db, "merged", res.MergeSummary)
return nil
}
if res.SHA256 != digest(data) {
// Whatever the server stored differs from what we sent, so the
// local file is no longer what the vault holds.
return m.pull(ctx, db, st, path)
}
if !res.Unchanged { if !res.Unchanged {
m.log.Printf("%s: pushed version %d", db.Name, res.Version) m.log.Printf("%s: pushed version %d", db.Name, res.Version)
} }
@@ -315,6 +330,12 @@ func (m *Mirror) push(ctx context.Context, db Database, st *fileState, path stri
if strategy == "client-wins" && !force { if strategy == "client-wins" && !force {
return m.push(ctx, db, st, path, data, true) return m.push(ctx, db, st, path, data, true)
} }
if e.Code == "merge_failed" {
// The server could not merge, most often because this vault's
// own password was changed. Keep our copy and take the server's,
// which is exactly what server-wins does.
m.log.Printf("%s: the server could not merge (%s)", db.Name, e.Message)
}
copyPath, cerr := m.keepConflictCopy(path, data) copyPath, cerr := m.keepConflictCopy(path, data)
if cerr != nil { if cerr != nil {
return cerr return cerr
+108 -1
View File
@@ -11,6 +11,7 @@ import (
"testing" "testing"
"time" "time"
"cc.tiredsysadmin.keeporg/internal/kdbx"
"cc.tiredsysadmin.keeporg/internal/secret" "cc.tiredsysadmin.keeporg/internal/secret"
"cc.tiredsysadmin.keeporg/internal/server" "cc.tiredsysadmin.keeporg/internal/server"
"cc.tiredsysadmin.keeporg/internal/store" "cc.tiredsysadmin.keeporg/internal/store"
@@ -31,7 +32,11 @@ const password = "admin-password-1"
// "everyone" group, and a mirror logged in as the administrator. // "everyone" group, and a mirror logged in as the administrator.
func newStack(t *testing.T) *stack { func newStack(t *testing.T) *stack {
t.Helper() t.Helper()
srv, err := server.New(server.Config{DataDir: t.TempDir()}, log.New(io.Discard, "", 0)) srv, err := server.New(server.Config{
DataDir: t.TempDir(),
MasterKeyFile: filepath.Join(t.TempDir(), "master.key"),
VaultKDF: kdbx.KDFParams{MemoryKiB: 64, Iterations: 1, Lanes: 1},
}, log.New(io.Discard, "", 0))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -360,3 +365,105 @@ func TestRunAppliesAPushedUpdateWithoutWaitingForThePoll(t *testing.T) {
} }
} }
} }
// A merge vault end to end: this client and another one both edit version 1,
// and the mirror ends up holding both edits.
func TestMirrorEndsUpWithBothEditsOnAMergeVault(t *testing.T) {
s := newStack(t)
ctx := context.Background()
vault, err := s.admin.CreateVault(ctx, "Merge Vault", "everyone", "merge", vaultTestPassword)
if err != nil {
t.Fatal(err)
}
s.sync()
file := filepath.Join(s.mir.Dir(), "Merge Vault.kdbx")
base, err := os.ReadFile(file)
if err != nil {
t.Fatalf("the vault was not mirrored: %v", err)
}
if _, err := kdbx.Open(base, vaultTestPassword); err != nil {
t.Fatalf("the mirrored file is not a readable vault: %v", err)
}
// Another client pushes its own edit of version 1.
theirs := addEntry(t, base, "From Them", "their-password")
if _, err := s.admin.Push(ctx, vault.ID, 1, theirs, false); err != nil {
t.Fatal(err)
}
// This client saves its edit of version 1, as KeePassXC would.
mine := addEntry(t, base, "From Me", "my-password")
if err := os.WriteFile(file, mine, 0o600); err != nil {
t.Fatal(err)
}
s.sync()
merged, err := os.ReadFile(file)
if err != nil {
t.Fatal(err)
}
got := vaultTitles(t, merged)
if got["From Me"] != "my-password" || got["From Them"] != "their-password" {
t.Fatalf("the mirrored vault holds %v, want both edits", got)
}
// And the mirror is settled: another pass must change nothing.
before, _ := os.ReadFile(file)
s.sync()
after, _ := os.ReadFile(file)
if string(before) != string(after) {
t.Fatal("the mirror kept rewriting the file after a merge")
}
var seen bool
for _, st := range s.mir.Status() {
if st.DatabaseID != vault.ID {
continue
}
seen = true
if st.State != "merged" {
t.Fatalf("vault status = %q (%s)", st.State, st.Message)
}
}
if !seen {
t.Fatal("the vault is missing from the status report")
}
}
const vaultTestPassword = "vault-password-12"
func addEntry(t *testing.T, data []byte, title, password string) []byte {
t.Helper()
db, err := kdbx.Open(data, vaultTestPassword)
if err != nil {
t.Fatal(err)
}
db.RootGroup().Nodes = append(db.RootGroup().Nodes,
kdbx.NewEntry(title, "user", password, "", ""))
out, err := db.Marshal(vaultTestPassword)
if err != nil {
t.Fatal(err)
}
return out
}
func vaultTitles(t *testing.T, data []byte) map[string]string {
t.Helper()
db, err := kdbx.Open(data, vaultTestPassword)
if err != nil {
t.Fatal(err)
}
out := map[string]string{}
for _, e := range db.RootGroup().FindAll("Entry") {
title, pw := "", ""
for _, str := range e.FindAll("String") {
switch str.Value("Key") {
case "Title":
title = str.Value("Value")
case "Password":
pw = str.Value("Value")
}
}
out[title] = pw
}
return out
}
+110
View File
@@ -0,0 +1,110 @@
package secret
import (
"crypto/aes"
"crypto/cipher"
"encoding/base64"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
)
// Keyring wraps the per-vault secrets the server must keep in order to merge
// a vault: it can decrypt a vault, so it holds that vault's key.
//
// The wrapped secrets live in the server's state document; the key that
// unwraps them does not. It comes from the environment or from a file
// outside the data directory, so a copy of the data directory alone yields
// ciphertext. Whoever holds both holds the vaults.
type Keyring struct {
aead cipher.AEAD
}
// MasterKeyEnv names the environment variable holding a base64 master key,
// which takes precedence over the key file.
const MasterKeyEnv = "KEEPORG_MASTER_KEY"
// LoadKeyring reads the master key from the environment or from path,
// creating one at path if neither exists.
func LoadKeyring(path string) (*Keyring, error) {
if v := strings.TrimSpace(os.Getenv(MasterKeyEnv)); v != "" {
key, err := base64.StdEncoding.DecodeString(v)
if err != nil {
return nil, fmt.Errorf("secret: %s is not valid base64: %w", MasterKeyEnv, err)
}
return NewKeyring(key)
}
if path == "" {
return nil, errors.New("secret: no master key: set " + MasterKeyEnv +
" or configure master_key_file")
}
b, err := os.ReadFile(path)
switch {
case err == nil:
key, derr := base64.StdEncoding.DecodeString(strings.TrimSpace(string(b)))
if derr != nil {
return nil, fmt.Errorf("secret: master key file %s is not valid base64: %w", path, derr)
}
return NewKeyring(key)
case !os.IsNotExist(err):
return nil, err
}
// First run: mint one. It is written with owner-only permissions and is
// the one file an operator must back up separately from the data.
key := randBytes(32)
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
return nil, err
}
enc := base64.StdEncoding.EncodeToString(key)
if err := os.WriteFile(path, []byte(enc+"\n"), 0o600); err != nil {
return nil, fmt.Errorf("secret: write master key %s: %w", path, err)
}
return NewKeyring(key)
}
// NewKeyring builds a keyring from a 32-byte key.
func NewKeyring(key []byte) (*Keyring, error) {
if len(key) != 32 {
return nil, fmt.Errorf("secret: master key must be 32 bytes, got %d", len(key))
}
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
aead, err := cipher.NewGCM(block)
if err != nil {
return nil, err
}
return &Keyring{aead: aead}, nil
}
// Wrap encrypts a vault secret. The vault's own id is authenticated with it,
// so a wrapped secret moved to another vault's record fails to open rather
// than unlocking the wrong database.
func (k *Keyring) Wrap(vaultID, plaintext string) (string, error) {
if vaultID == "" {
return "", errors.New("secret: wrapping needs a vault id")
}
nonce := randBytes(k.aead.NonceSize())
out := k.aead.Seal(nonce, nonce, []byte(plaintext), []byte(vaultID))
return base64.StdEncoding.EncodeToString(out), nil
}
// Unwrap decrypts a vault secret wrapped for this vault.
func (k *Keyring) Unwrap(vaultID, wrapped string) (string, error) {
raw, err := base64.StdEncoding.DecodeString(wrapped)
if err != nil {
return "", fmt.Errorf("secret: wrapped value is not base64: %w", err)
}
n := k.aead.NonceSize()
if len(raw) < n {
return "", errors.New("secret: wrapped value is too short")
}
plain, err := k.aead.Open(nil, raw[:n], raw[n:], []byte(vaultID))
if err != nil {
return "", errors.New("secret: wrapped value does not open with this master key")
}
return string(plain), nil
}
+92
View File
@@ -0,0 +1,92 @@
package secret
import (
"encoding/base64"
"os"
"path/filepath"
"testing"
)
func TestKeyringWrapRoundTrip(t *testing.T) {
k, err := NewKeyring(randBytes(32))
if err != nil {
t.Fatal(err)
}
wrapped, err := k.Wrap("vault-1", "the-vault-password")
if err != nil {
t.Fatal(err)
}
if wrapped == "the-vault-password" {
t.Fatal("the secret was stored in the clear")
}
got, err := k.Unwrap("vault-1", wrapped)
if err != nil || got != "the-vault-password" {
t.Fatalf("unwrap = %q, %v", got, err)
}
}
func TestWrappedSecretIsBoundToItsVault(t *testing.T) {
k, _ := NewKeyring(randBytes(32))
wrapped, _ := k.Wrap("vault-1", "secret")
if _, err := k.Unwrap("vault-2", wrapped); err == nil {
t.Fatal("a secret wrapped for one vault opened under another id")
}
}
func TestAnotherMasterKeyCannotUnwrap(t *testing.T) {
a, _ := NewKeyring(randBytes(32))
b, _ := NewKeyring(randBytes(32))
wrapped, _ := a.Wrap("v", "secret")
if _, err := b.Unwrap("v", wrapped); err == nil {
t.Fatal("a different master key opened the secret")
}
}
func TestLoadKeyringCreatesThenReusesTheKeyFile(t *testing.T) {
t.Setenv(MasterKeyEnv, "")
path := filepath.Join(t.TempDir(), "keys", "master.key")
first, err := LoadKeyring(path)
if err != nil {
t.Fatal(err)
}
fi, err := os.Stat(path)
if err != nil {
t.Fatal(err)
}
if fi.Mode().Perm() != 0o600 {
t.Fatalf("master key file mode = %v", fi.Mode().Perm())
}
wrapped, _ := first.Wrap("v", "secret")
second, err := LoadKeyring(path)
if err != nil {
t.Fatal(err)
}
if got, err := second.Unwrap("v", wrapped); err != nil || got != "secret" {
t.Fatalf("reloaded keyring: %q, %v", got, err)
}
}
func TestLoadKeyringPrefersTheEnvironment(t *testing.T) {
key := randBytes(32)
t.Setenv(MasterKeyEnv, base64.StdEncoding.EncodeToString(key))
path := filepath.Join(t.TempDir(), "master.key")
k, err := LoadKeyring(path)
if err != nil {
t.Fatal(err)
}
if _, err := os.Stat(path); !os.IsNotExist(err) {
t.Fatal("a key file was written although the environment supplied the key")
}
direct, _ := NewKeyring(key)
wrapped, _ := direct.Wrap("v", "secret")
if got, _ := k.Unwrap("v", wrapped); got != "secret" {
t.Fatal("the environment key was not the one used")
}
}
func TestLoadKeyringRejectsAShortKey(t *testing.T) {
t.Setenv(MasterKeyEnv, base64.StdEncoding.EncodeToString([]byte("too short")))
if _, err := LoadKeyring(""); err == nil {
t.Fatal("a short master key was accepted")
}
}
+4
View File
@@ -96,6 +96,10 @@ func (s *Server) handlePatchDatabase(w http.ResponseWriter, r *http.Request, u *
if !req.MergeStrategy.Valid() { if !req.MergeStrategy.Valid() {
return errors.New("unknown merge strategy") return errors.New("unknown merge strategy")
} }
if req.MergeStrategy.NeedsKey() && db.WrappedKey == "" {
return errors.New("merging needs the vault password: store it with " +
"`keeporg db -id <id> -store-password` first")
}
db.MergeStrategy = *req.MergeStrategy db.MergeStrategy = *req.MergeStrategy
} }
out = toView(db, store.PermAdmin) out = toView(db, store.PermAdmin)
+22 -7
View File
@@ -9,6 +9,7 @@ import (
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"net/url" "net/url"
"os"
"path/filepath" "path/filepath"
"strings" "strings"
"testing" "testing"
@@ -20,9 +21,10 @@ import (
) )
type harness struct { type harness struct {
t *testing.T t *testing.T
srv *Server srv *Server
http *httptest.Server http *httptest.Server
dataDir string
} }
const adminPassword = "admin-password-1" const adminPassword = "admin-password-1"
@@ -30,8 +32,12 @@ const adminPassword = "admin-password-1"
func newHarness(t *testing.T) *harness { func newHarness(t *testing.T) *harness {
t.Helper() t.Helper()
dir := t.TempDir() dir := t.TempDir()
srv, err := New(Config{DataDir: dir, ExternalURL: "https://keeporg.test"}, srv, err := New(Config{
log.New(io.Discard, "", 0)) DataDir: dir,
ExternalURL: "http://keeporg.test",
MasterKeyFile: filepath.Join(t.TempDir(), "master.key"),
VaultKDF: testKDF,
}, log.New(io.Discard, "", 0))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -50,8 +56,17 @@ func newHarness(t *testing.T) *harness {
} }
ts := httptest.NewServer(srv.Handler()) ts := httptest.NewServer(srv.Handler())
t.Cleanup(ts.Close) t.Cleanup(ts.Close)
_ = filepath.Join(dir, "state.json") return &harness{t: t, srv: srv, http: ts, dataDir: dir}
return &harness{t: t, srv: srv, http: ts} }
// stateBytes is the server's whole metadata document as stored on disk.
func (h *harness) stateBytes() []byte {
h.t.Helper()
b, err := os.ReadFile(filepath.Join(h.dataDir, "state.json"))
if err != nil {
h.t.Fatal(err)
}
return b
} }
// call sends a JSON request and decodes the JSON response. // call sends a JSON request and decodes the JSON response.
+132
View File
@@ -0,0 +1,132 @@
package server
import (
"errors"
"fmt"
"net/http"
"time"
"cc.tiredsysadmin.keeporg/internal/blob"
"cc.tiredsysadmin.keeporg/internal/kdbx"
"cc.tiredsysadmin.keeporg/internal/store"
)
// mergePush resolves a stale push by decrypting both copies in memory,
// folding them together and storing the result as a new version. Nothing is
// written to disk in the clear: the plaintext exists only as these two
// kdbx.Database values, and what is stored is an encrypted KDBX file again.
//
// Any failure answers with a conflict instead, so the client falls back to
// keeping its copy aside rather than losing an edit.
func (s *Server) mergePush(w http.ResponseWriter, u *store.User, id string, basedOn int64, incoming []byte) {
var (
wrapped string
current int64
name string
)
s.st.Read(func(st *store.State) {
if d := st.Databases[id]; d != nil {
wrapped, current, name = d.WrappedKey, d.Version, d.Name
}
})
result, merged, err := s.mergeInMemory(id, wrapped, current, incoming)
if err != nil {
s.log.Printf("merge %s (%s): %v", name, id, err)
w.Header().Set("ETag", etag(current))
w.Header().Set("X-KeepOrg-Strategy", string(store.StrategyMerge))
writeJSON(w, http.StatusConflict, map[string]any{
"code": "merge_failed", "error": err.Error(),
"current_version": current, "merge_strategy": store.StrategyMerge,
})
return
}
sum := blob.SHA256(merged)
var (
next int64
watchers []string
stale bool
)
if err := s.st.Update(func(st *store.State) error {
d := st.Databases[id]
if d == nil {
return errors.New("no such database")
}
if d.Version != current {
// Someone else landed a version while this merge was running.
stale = true
return nil
}
next = d.Version + 1
if err := s.blobs.Put(id, next, merged); err != nil {
return err
}
d.Version, d.SHA256, d.Size = next, sum, int64(len(merged))
d.UpdatedAt, d.UpdatedBy = time.Now(), u.Username
watchers = st.Watchers(id)
return nil
}); err != nil {
fail(w, http.StatusInternalServerError, "internal", err.Error())
return
}
if stale {
// Ask the client to come back with the newer base; its next attempt
// merges against what just landed.
w.Header().Set("ETag", etag(current))
w.Header().Set("X-KeepOrg-Strategy", string(store.StrategyMerge))
writeJSON(w, http.StatusConflict, map[string]any{
"code": "conflict", "error": "another merge landed first, retry",
"current_version": current, "merge_strategy": store.StrategyMerge,
})
return
}
_ = s.blobs.Prune(id)
s.log.Printf("merged %s into version %d for %s: %s", name, next, u.Username, result)
s.hub.Publish(watchers, Event{Type: EventUpdated, DatabaseID: id, Name: name,
Version: next, SHA256: sum, By: u.Username})
w.Header().Set("ETag", etag(next))
writeJSON(w, http.StatusOK, map[string]any{
"version": next, "sha256": sum, "unchanged": false,
"merged": true, "merge_summary": result.String(), "based_on": basedOn,
})
}
// mergeInMemory does the cryptography and the merge. It is separated so the
// handler holds no plaintext longer than it must.
func (s *Server) mergeInMemory(id, wrapped string, current int64, incoming []byte) (
kdbx.MergeResult, []byte, error) {
var empty kdbx.MergeResult
password, err := s.vaultPassword(id, wrapped)
if err != nil {
return empty, nil, err
}
if current == 0 {
return empty, nil, errors.New("this vault has no stored version to merge with")
}
stored, err := s.blobs.Get(id, current)
if err != nil {
return empty, nil, fmt.Errorf("reading version %d: %w", current, err)
}
target, err := kdbx.Open(stored, password)
if err != nil {
return empty, nil, fmt.Errorf("opening the stored vault: %w", err)
}
source, err := kdbx.Open(incoming, password)
if err != nil {
// Most often this means the vault's own password was changed by a
// client; the server's stored key is then out of date.
return empty, nil, fmt.Errorf("opening the pushed vault: %w", err)
}
result, err := kdbx.Merge(target, source)
if err != nil {
return empty, nil, err
}
out, err := target.Marshal(password)
if err != nil {
return empty, nil, fmt.Errorf("re-encrypting the merged vault: %w", err)
}
return result, out, nil
}
+291
View File
@@ -0,0 +1,291 @@
package server
import (
"bytes"
"io"
"net/http"
"testing"
"time"
"cc.tiredsysadmin.keeporg/internal/kdbx"
)
// testKDF keeps the vault work factor negligible in tests; the merge logic
// does not care what the key cost was.
var testKDF = kdbx.KDFParams{MemoryKiB: 64, Iterations: 1, Lanes: 1}
const vaultPassword = "vault-password-12"
// makeVault creates a merge vault through the API and returns its id.
func (h *harness) makeVault(token, name string) string {
h.t.Helper()
status, body := h.call("POST", "/api/v1/vaults", token, map[string]string{
"name": name, "owner_group_id": "everyone",
"merge_strategy": "merge", "password": vaultPassword,
})
if status != http.StatusCreated {
h.t.Fatalf("create vault: %d %v", status, body)
}
if body["merge_strategy"] != "merge" || body["version"].(float64) != 1 {
h.t.Fatalf("new vault = %v", body)
}
return body["id"].(string)
}
// pull downloads the current bytes of a database.
func (h *harness) pull(token, dbID string) []byte {
h.t.Helper()
req, _ := http.NewRequest("GET", h.http.URL+"/api/v1/databases/"+dbID+"/blob", nil)
req.Header.Set("Authorization", "Bearer "+token)
resp, err := h.http.Client().Do(req)
if err != nil {
h.t.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
h.t.Fatalf("pull: %d", resp.StatusCode)
}
data, _ := io.ReadAll(resp.Body)
return data
}
// edit opens a vault, appends an entry and re-encrypts it, which is what a
// KeePass application does on save.
func edit(t *testing.T, data []byte, title, password string) []byte {
t.Helper()
db, err := kdbx.Open(data, vaultPassword)
if err != nil {
t.Fatal(err)
}
db.RootGroup().Nodes = append(db.RootGroup().Nodes,
kdbx.NewEntry(title, "user", password, "", ""))
out, err := db.Marshal(vaultPassword)
if err != nil {
t.Fatal(err)
}
return out
}
// entryTitles lists the live entry titles of a vault.
func entryTitles(t *testing.T, data []byte) map[string]string {
t.Helper()
db, err := kdbx.Open(data, vaultPassword)
if err != nil {
t.Fatal(err)
}
out := map[string]string{}
var visit func(g *kdbx.Node)
visit = func(g *kdbx.Node) {
for _, e := range g.FindAll("Entry") {
title, pw := "", ""
for _, s := range e.FindAll("String") {
switch s.Value("Key") {
case "Title":
title = s.Value("Value")
case "Password":
pw = s.Value("Value")
}
}
out[title] = pw
}
for _, sub := range g.FindAll("Group") {
visit(sub)
}
}
visit(db.RootGroup())
return out
}
func TestVaultCreatedByTheServerIsAUsableKDBX(t *testing.T) {
h := newHarness(t)
tok, _, _ := h.login("admin", adminPassword, "")
id := h.makeVault(tok, "Team Vault")
data := h.pull(tok, id)
db, err := kdbx.Open(data, vaultPassword)
if err != nil {
t.Fatalf("the vault the server created does not open: %v", err)
}
if db.Name() != "Team Vault" {
t.Fatalf("vault name = %q", db.Name())
}
if _, err := kdbx.Open(data, "wrong-password-here"); err != kdbx.ErrWrongPassword {
t.Fatalf("a wrong password opened the vault: %v", err)
}
// The vault password must not be readable from the server's state file.
if bytes.Contains(h.stateBytes(), []byte(vaultPassword)) {
t.Fatal("the vault password is stored in the clear in state.json")
}
}
func TestConcurrentEditsAreMergedNotRejected(t *testing.T) {
h := newHarness(t)
tok, _, _ := h.login("admin", adminPassword, "")
id := h.makeVault(tok, "Team Vault")
base := h.pull(tok, id)
// Two people open version 1 and each add an entry.
fromA := edit(t, base, "From A", "a-password")
fromB := edit(t, base, "From B", "b-password")
status, body, _ := h.push(tok, id, `"1"`, fromA)
if status != http.StatusOK || body["version"].(float64) != 2 {
t.Fatalf("first push: %d %v", status, body)
}
// The second push is based on version 1, which is now stale. A
// server-wins vault would refuse it; this one merges.
status, body, hdr := h.push(tok, id, `"1"`, fromB)
if status != http.StatusOK {
t.Fatalf("second push: %d %v", status, body)
}
if body["merged"] != true || body["version"].(float64) != 3 {
t.Fatalf("merge response = %v", body)
}
if hdr.Get("ETag") != `"3"` {
t.Fatalf("etag = %q", hdr.Get("ETag"))
}
titles := entryTitles(t, h.pull(tok, id))
if titles["From A"] != "a-password" || titles["From B"] != "b-password" {
t.Fatalf("merged vault holds %v, want both edits", titles)
}
}
func TestMergeKeepsTheNewerEditOfTheSameEntry(t *testing.T) {
h := newHarness(t)
tok, _, _ := h.login("admin", adminPassword, "")
id := h.makeVault(tok, "Team Vault")
// Start from a vault that already holds the entry both people edit.
base := edit(t, h.pull(tok, id), "Router", "original")
if status, body, _ := h.push(tok, id, `"1"`, base); status != http.StatusOK {
t.Fatalf("seed push: %d %v", status, body)
}
base = h.pull(tok, id)
changeRouter := func(data []byte, password string) []byte {
db, err := kdbx.Open(data, vaultPassword)
if err != nil {
t.Fatal(err)
}
for _, e := range db.RootGroup().FindAll("Entry") {
for _, s := range e.FindAll("String") {
if s.Value("Key") == "Password" {
s.Find("Value").Text = password
}
}
e.Find("Times").Set("LastModificationTime",
kdbx.EncodeTime(time.Now().Add(clockSkew(password))))
}
out, err := db.Marshal(vaultPassword)
if err != nil {
t.Fatal(err)
}
return out
}
first := changeRouter(base, "edited-first")
second := changeRouter(base, "edited-second")
if status, _, _ := h.push(tok, id, `"2"`, first); status != http.StatusOK {
t.Fatalf("first edit push: %d", status)
}
status, body, _ := h.push(tok, id, `"2"`, second)
if status != http.StatusOK || body["merged"] != true {
t.Fatalf("second edit push: %d %v", status, body)
}
titles := entryTitles(t, h.pull(tok, id))
if titles["Router"] != "edited-second" {
t.Fatalf("Router = %q, want the later edit to win", titles["Router"])
}
}
func TestMergeFailureFallsBackToAConflictInsteadOfLosingData(t *testing.T) {
h := newHarness(t)
tok, _, _ := h.login("admin", adminPassword, "")
id := h.makeVault(tok, "Team Vault")
base := h.pull(tok, id)
if status, _, _ := h.push(tok, id, `"1"`, edit(t, base, "From A", "p")); status != http.StatusOK {
t.Fatal("seed push failed")
}
// A client that changed the vault's own password pushes a file the
// server's stored key no longer opens.
db, err := kdbx.Open(base, vaultPassword)
if err != nil {
t.Fatal(err)
}
reencrypted, err := db.Marshal("a-completely-different-password")
if err != nil {
t.Fatal(err)
}
status, body, _ := h.push(tok, id, `"1"`, reencrypted)
if status != http.StatusConflict || body["code"] != "merge_failed" {
t.Fatalf("push = %d %v, want a merge_failed conflict", status, body)
}
// The stored vault is untouched.
if titles := entryTitles(t, h.pull(tok, id)); titles["From A"] != "p" {
t.Fatalf("the stored vault changed: %v", titles)
}
}
func TestMergeStrategyRefusedWithoutAStoredKey(t *testing.T) {
h := newHarness(t)
tok, _, _ := h.login("admin", adminPassword, "")
// A plain database, created without a password, cannot be switched to
// merge until a key is stored for it.
db := h.makeDB(tok, "No Key")
if status, body := h.call("PATCH", "/api/v1/databases/"+db, tok,
map[string]string{"merge_strategy": "merge"}); status != http.StatusBadRequest {
t.Fatalf("patch to merge = %d %v, want a refusal", status, body)
}
}
func TestAnExistingDatabaseCanAdoptAStoredKey(t *testing.T) {
h := newHarness(t)
tok, _, _ := h.login("admin", adminPassword, "")
id := h.makeDB(tok, "Adopted")
// Someone uploads a real KDBX file as the first version.
first, err := kdbx.New("Adopted", testKDF).Marshal(vaultPassword)
if err != nil {
t.Fatal(err)
}
if status, _, _ := h.push(tok, id, `"0"`, first); status != http.StatusOK {
t.Fatal("initial upload failed")
}
// A wrong password must not be accepted as the vault key.
if status, body := h.call("POST", "/api/v1/databases/"+id+"/key", tok,
map[string]string{"password": "not-the-vault-password"}); status != http.StatusBadRequest {
t.Fatalf("wrong key accepted: %d %v", status, body)
}
if status, body := h.call("POST", "/api/v1/databases/"+id+"/key", tok,
map[string]string{"password": vaultPassword}); status != http.StatusOK {
t.Fatalf("storing the key: %d %v", status, body)
}
if status, body := h.call("PATCH", "/api/v1/databases/"+id, tok,
map[string]string{"merge_strategy": "merge"}); status != http.StatusOK {
t.Fatalf("switching to merge: %d %v", status, body)
}
// And now it merges.
base := h.pull(tok, id)
if status, _, _ := h.push(tok, id, `"1"`, edit(t, base, "From A", "a")); status != http.StatusOK {
t.Fatal("push A failed")
}
status, body, _ := h.push(tok, id, `"1"`, edit(t, base, "From B", "b"))
if status != http.StatusOK || body["merged"] != true {
t.Fatalf("merge after adoption: %d %v", status, body)
}
}
// clockSkew gives the two concurrent edits different timestamps, the way two
// real saves a minute apart would have.
func clockSkew(password string) time.Duration {
if password == "edited-second" {
return time.Minute
}
return 0
}
+114 -1
View File
@@ -19,7 +19,9 @@ var pages = template.Must(template.New("layout").Parse(`
*{box-sizing:border-box} *{box-sizing:border-box}
body{margin:0;min-height:100vh;display:grid;place-items:center;background:var(--bg);color:var(--fg); body{margin:0;min-height:100vh;display:grid;place-items:center;background:var(--bg);color:var(--fg);
font:16px/1.5 ui-sans-serif,system-ui,-apple-system,"Segoe UI",sans-serif;padding:24px} font:16px/1.5 ui-sans-serif,system-ui,-apple-system,"Segoe UI",sans-serif;padding:24px}
body.wide{place-items:start center}
main{width:100%;max-width:26rem;background:var(--card);border:1px solid var(--line);border-radius:12px;padding:28px} main{width:100%;max-width:26rem;background:var(--card);border:1px solid var(--line);border-radius:12px;padding:28px}
body.wide main{max-width:54rem}
h1{font-size:1.25rem;margin:0 0 .25rem} h1{font-size:1.25rem;margin:0 0 .25rem}
p.sub{margin:0 0 1.25rem;opacity:.7;font-size:.9rem} p.sub{margin:0 0 1.25rem;opacity:.7;font-size:.9rem}
label{display:block;font-size:.8rem;text-transform:uppercase;letter-spacing:.04em;opacity:.7;margin:.9rem 0 .25rem} label{display:block;font-size:.8rem;text-transform:uppercase;letter-spacing:.04em;opacity:.7;margin:.9rem 0 .25rem}
@@ -30,7 +32,28 @@ background:var(--accent);border:0;border-radius:8px;cursor:pointer}
.err{margin:0 0 1rem;padding:.6rem .7rem;border-radius:8px;background:#f8d7da;color:#842029;font-size:.9rem} .err{margin:0 0 1rem;padding:.6rem .7rem;border-radius:8px;background:#f8d7da;color:#842029;font-size:.9rem}
.alt{display:block;margin-top:1rem;text-align:center;font-size:.9rem;color:var(--accent)} .alt{display:block;margin-top:1rem;text-align:center;font-size:.9rem;color:var(--accent)}
code{font-size:1.1rem;letter-spacing:.08em} code{font-size:1.1rem;letter-spacing:.08em}
</style></head><body><main>{{template "body" .}}</main></body></html>{{end}} header{display:flex;align-items:baseline;justify-content:space-between;gap:1rem;flex-wrap:wrap}
header form{margin:0}
header button{width:auto;margin:0;padding:.35rem .7rem;font-size:.85rem;font-weight:500;
background:transparent;color:var(--accent);border:1px solid var(--line)}
table{width:100%;border-collapse:collapse;margin:.5rem 0 1.5rem;font-size:.9rem}
th{text-align:left;font-size:.75rem;text-transform:uppercase;letter-spacing:.04em;opacity:.6;
padding:.4rem .5rem;border-bottom:1px solid var(--line)}
td{padding:.5rem;border-bottom:1px solid var(--line);vertical-align:top}
td.name{font-weight:600}
.tag{display:inline-block;padding:.1rem .4rem;border:1px solid var(--line);border-radius:5px;
font-size:.75rem;opacity:.85}
.ok{margin:0 0 1rem;padding:.6rem .7rem;border-radius:8px;background:#d1e7dd;color:#0f5132;font-size:.9rem}
.note{margin:1rem 0 0;padding:.7rem .8rem;border-left:3px solid var(--accent);background:rgba(0,0,0,.03);
font-size:.85rem}
@media(prefers-color-scheme:dark){.note{background:rgba(255,255,255,.04)}
.ok{background:#15372a;color:#9ad8b8}.err{background:#3b1f23;color:#f2b8c0}}
select{width:100%;padding:.6rem .7rem;font:inherit;color:var(--fg);background:var(--card);
border:1px solid var(--line);border-radius:8px}
fieldset{border:1px solid var(--line);border-radius:10px;padding:1rem 1.2rem 1.4rem;margin:0}
legend{padding:0 .4rem;font-size:.8rem;text-transform:uppercase;letter-spacing:.04em;opacity:.7}
.empty{opacity:.7;font-size:.9rem;margin:.5rem 0 1.5rem}
</style></head><body{{if .Wide}} class="wide"{{end}}><main>{{template "body" .}}</main></body></html>{{end}}
`)) `))
var deviceTmpl = template.Must(template.Must(pages.Clone()).Parse(` var deviceTmpl = template.Must(template.Must(pages.Clone()).Parse(`
@@ -60,6 +83,82 @@ var inviteTmpl = template.Must(template.Must(pages.Clone()).Parse(`
</form> </form>
{{end}}`)) {{end}}`))
var signInTmpl = template.Must(template.Must(pages.Clone()).Parse(`
{{define "body"}}<h1>KeepOrg</h1>
<p class="sub">Sign in to create and manage your vaults.</p>
{{if .Error}}<p class="err">{{.Error}}</p>{{end}}
<form method="post" action="/web/signin">
<label for="username">Username</label><input id="username" name="username" autocomplete="username" required autofocus>
<label for="password">Password</label><input id="password" name="password" type="password" autocomplete="current-password" required>
<label for="totp">One-time code <small>(if enabled)</small></label><input id="totp" name="totp" inputmode="numeric" autocomplete="one-time-code">
<button type="submit">Sign in</button>
</form>
{{if .OIDC}}<a class="alt" href="/oidc/login">Sign in with single sign-on &rarr;</a>{{end}}
{{end}}`))
var vaultsTmpl = template.Must(template.Must(pages.Clone()).Parse(`
{{define "body"}}
<header>
<div><h1>Your vaults</h1><p class="sub">{{.User}}</p></div>
<form method="post" action="/web/signout">
<input type="hidden" name="csrf" value="{{.CSRF}}">
<button type="submit">Sign out</button>
</form>
</header>
{{if .Error}}<p class="err">{{.Error}}</p>{{end}}
{{if .Notice}}<p class="ok">{{.Notice}}</p>{{end}}
{{if .Vaults}}
<table>
<thead><tr><th>Vault</th><th>Access</th><th>On conflict</th><th>Version</th><th>Last saved</th></tr></thead>
<tbody>
{{range .Vaults}}<tr>
<td class="name">{{.Name}}.kdbx</td>
<td>{{.Perm}}</td>
<td><span class="tag">{{.Strategy}}</span></td>
<td>{{.Version}}</td>
<td>{{.Updated}}{{if .By}}<br><small>by {{.By}}</small>{{end}}</td>
</tr>{{end}}
</tbody></table>
{{else}}
<p class="empty">No vaults yet. Create one below, then open it from the KeepOrg client
on your computer or phone.</p>
{{end}}
{{if .Groups}}
<fieldset><legend>Create a vault</legend>
<form method="post" action="/web/vaults">
<input type="hidden" name="csrf" value="{{.CSRF}}">
<label for="name">Name</label>
<input id="name" name="name" required placeholder="Team Vault">
<label for="group">Owned by</label>
<select id="group" name="group" required>
{{range .Groups}}<option value="{{.ID}}">{{.Name}}</option>{{end}}
</select>
<label for="strategy">When two people save at once</label>
<select id="strategy" name="strategy">
<option value="merge">Merge both edits (the server holds the key)</option>
<option value="server-wins">Keep the server copy</option>
<option value="client-wins">Keep the newest save</option>
<option value="manual">Leave both copies for a human</option>
</select>
<label for="password">Vault password</label>
<input id="password" name="password" type="password" autocomplete="new-password" minlength="12" required>
<label for="password2">Repeat vault password</label>
<input id="password2" name="password2" type="password" autocomplete="new-password" minlength="12" required>
<button type="submit">Create vault</button>
</form>
<p class="note">This password encrypts the vault itself and is what everyone with
access types into KeePass. It is not your account password, and it cannot be
recovered: write it down before you continue.<br><br>
Merging requires the server to keep this password, so a merge vault can be read
by whoever controls the server. Any other setting keeps the key off the server.</p>
</fieldset>
{{else}}
<p class="empty">You do not administer a group yet, so there is nowhere to put a
vault. Ask an administrator to add you to one.</p>
{{end}}
{{end}}`))
var doneTmpl = template.Must(template.Must(pages.Clone()).Parse(` var doneTmpl = template.Must(template.Must(pages.Clone()).Parse(`
{{define "body"}}<h1>{{.Title}}</h1><p class="sub">{{.Message}}</p>{{end}}`)) {{define "body"}}<h1>{{.Title}}</h1><p class="sub">{{.Message}}</p>{{end}}`))
@@ -73,6 +172,20 @@ func (s *Server) renderInvite(w http.ResponseWriter, token, errMsg string) {
render(w, inviteTmpl, map[string]any{"Title": "Create account", "Token": token, "Error": errMsg}) render(w, inviteTmpl, map[string]any{"Title": "Create account", "Token": token, "Error": errMsg})
} }
func (s *Server) renderSignIn(w http.ResponseWriter, errMsg string) {
render(w, signInTmpl, map[string]any{
"Title": "Sign in", "Error": errMsg, "OIDC": s.oidc != nil,
})
}
func (s *Server) renderVaults(w http.ResponseWriter, sess *webSession, errMsg, notice string) {
vaults, groups := s.vaultRows(sess.user)
render(w, vaultsTmpl, map[string]any{
"Title": "Vaults", "Wide": true, "User": sess.user.Username, "CSRF": sess.csrf,
"Vaults": vaults, "Groups": groups, "Error": errMsg, "Notice": notice,
})
}
func (s *Server) renderDone(w http.ResponseWriter, title, msg string) { func (s *Server) renderDone(w http.ResponseWriter, title, msg string) {
render(w, doneTmpl, map[string]any{"Title": title, "Message": msg}) render(w, doneTmpl, map[string]any{"Title": title, "Message": msg})
} }
+45 -7
View File
@@ -15,6 +15,7 @@ import (
"time" "time"
"cc.tiredsysadmin.keeporg/internal/blob" "cc.tiredsysadmin.keeporg/internal/blob"
"cc.tiredsysadmin.keeporg/internal/kdbx"
"cc.tiredsysadmin.keeporg/internal/secret" "cc.tiredsysadmin.keeporg/internal/secret"
"cc.tiredsysadmin.keeporg/internal/store" "cc.tiredsysadmin.keeporg/internal/store"
) )
@@ -29,6 +30,14 @@ type Config struct {
TLSCert string `json:"tls_cert,omitempty"` TLSCert string `json:"tls_cert,omitempty"`
TLSKey string `json:"tls_key,omitempty"` TLSKey string `json:"tls_key,omitempty"`
OIDC OIDCConfig `json:"oidc"` OIDC OIDCConfig `json:"oidc"`
// MasterKeyFile holds the key that wraps the vault keys the server keeps
// in order to merge. It belongs outside DataDir: a copy of the data
// directory alone must not be enough to open a vault.
MasterKeyFile string `json:"master_key_file"`
// VaultKDF is the work factor of vaults this server creates. The server
// pays it three times per merge, and so does every client on every open.
VaultKDF kdbx.KDFParams `json:"vault_kdf"`
} }
// Defaults fills in every value a deployment may leave unset. // Defaults fills in every value a deployment may leave unset.
@@ -49,16 +58,23 @@ func (c *Config) Defaults() {
if c.MaxDBSize == 0 { if c.MaxDBSize == 0 {
c.MaxDBSize = 32 << 20 c.MaxDBSize = 32 << 20
} }
if c.MasterKeyFile == "" {
c.MasterKeyFile = "/etc/keeporg/master.key"
}
if c.VaultKDF.MemoryKiB == 0 {
c.VaultKDF = kdbx.DefaultKDF
}
} }
type Server struct { type Server struct {
cfg Config cfg Config
st *store.Store st *store.Store
blobs *blob.Store blobs *blob.Store
hub *Hub hub *Hub
oidc *oidcProvider oidc *oidcProvider
mux *http.ServeMux keyring *secret.Keyring
log *log.Logger mux *http.ServeMux
log *log.Logger
} }
// New opens the server's data directory and wires up its routes. // New opens the server's data directory and wires up its routes.
@@ -73,6 +89,13 @@ func New(cfg Config, lg *log.Logger) (*Server, error) {
return nil, err return nil, err
} }
s := &Server{cfg: cfg, st: st, blobs: blobs, hub: NewHub(), mux: http.NewServeMux(), log: lg} s := &Server{cfg: cfg, st: st, blobs: blobs, hub: NewHub(), mux: http.NewServeMux(), log: lg}
// Without a master key the server still serves every strategy that does
// not need to read a vault; only merging and vault creation are refused.
if kr, kerr := secret.LoadKeyring(cfg.MasterKeyFile); kerr != nil {
lg.Printf("no master key, so this server cannot create or merge vaults: %v", kerr)
} else {
s.keyring = kr
}
if cfg.OIDC.Enabled() { if cfg.OIDC.Enabled() {
s.oidc = newOIDCProvider(cfg.OIDC, cfg.ExternalURL+"/oidc/callback") s.oidc = newOIDCProvider(cfg.OIDC, cfg.ExternalURL+"/oidc/callback")
} }
@@ -120,6 +143,16 @@ func (s *Server) routes() {
m.HandleFunc("POST /api/v1/auth/device/start", s.handleDeviceStart) m.HandleFunc("POST /api/v1/auth/device/start", s.handleDeviceStart)
m.HandleFunc("POST /api/v1/auth/device/poll", s.handleDevicePoll) m.HandleFunc("POST /api/v1/auth/device/poll", s.handleDevicePoll)
// The web interface: sign in, see your vaults, create one.
m.HandleFunc("GET /{$}", func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/web", http.StatusFound)
})
m.HandleFunc("GET /web", s.handleWebHome)
m.HandleFunc("POST /web/signin", s.handleWebSignIn)
m.HandleFunc("POST /web/signout", s.handleWebSignOut)
m.HandleFunc("GET /web/vaults", s.handleWebVaults)
m.HandleFunc("POST /web/vaults", s.handleWebCreateVault)
// Browser-facing pages: device approval, invite registration, OIDC. // Browser-facing pages: device approval, invite registration, OIDC.
m.HandleFunc("GET /device", s.handleDevicePage) m.HandleFunc("GET /device", s.handleDevicePage)
m.HandleFunc("POST /device", s.handleDeviceApprove) m.HandleFunc("POST /device", s.handleDeviceApprove)
@@ -134,6 +167,10 @@ func (s *Server) routes() {
m.HandleFunc("PUT /api/v1/databases/{id}/blob", s.authed(s.handlePutBlob)) m.HandleFunc("PUT /api/v1/databases/{id}/blob", s.authed(s.handlePutBlob))
m.HandleFunc("GET /api/v1/events", s.authed(s.handleEvents)) m.HandleFunc("GET /api/v1/events", s.authed(s.handleEvents))
// Vaults: a database the server created, and can merge.
m.HandleFunc("POST /api/v1/vaults", s.authed(s.handleCreateVault))
m.HandleFunc("POST /api/v1/databases/{id}/key", s.authed(s.handleSetVaultKey))
// Administration. // Administration.
m.HandleFunc("POST /api/v1/databases", s.authed(s.handleCreateDatabase)) m.HandleFunc("POST /api/v1/databases", s.authed(s.handleCreateDatabase))
m.HandleFunc("PATCH /api/v1/databases/{id}", s.authed(s.handlePatchDatabase)) m.HandleFunc("PATCH /api/v1/databases/{id}", s.authed(s.handlePatchDatabase))
@@ -237,6 +274,7 @@ func (s *Server) newSession(st *store.State, userID, label string) *store.Sessio
Token: secret.Token(), Token: secret.Token(),
UserID: userID, UserID: userID,
Label: label, Label: label,
CSRF: secret.Token(),
CreatedAt: time.Now(), CreatedAt: time.Now(),
ExpiresAt: time.Now().Add(s.cfg.SessionTTL), ExpiresAt: time.Now().Add(s.cfg.SessionTTL),
} }
+9 -1
View File
@@ -117,7 +117,10 @@ func (s *Server) handlePutBlob(w http.ResponseWriter, r *http.Request, u *store.
return return
} }
v := decidePush(current, r.Header.Get("If-Match"), strategy) v := decidePush(current, r.Header.Get("If-Match"), strategy)
if !v.Accept { // A merge vault answers a stale push by folding the two copies together
// instead of refusing it, so the body is read either way.
merging := !v.Accept && v.Code == "conflict" && strategy == store.StrategyMerge
if !v.Accept && !merging {
w.Header().Set("ETag", etag(current)) w.Header().Set("ETag", etag(current))
w.Header().Set("X-KeepOrg-Strategy", string(strategy)) w.Header().Set("X-KeepOrg-Strategy", string(strategy))
writeJSON(w, v.Status, map[string]any{ writeJSON(w, v.Status, map[string]any{
@@ -142,6 +145,11 @@ func (s *Server) handlePutBlob(w http.ResponseWriter, r *http.Request, u *store.
return return
} }
if merging {
s.mergePush(w, u, id, current, data)
return
}
var ( var (
next int64 next int64
unchanged bool unchanged bool
+193
View File
@@ -0,0 +1,193 @@
package server
import (
"errors"
"fmt"
"net/http"
"strings"
"time"
"cc.tiredsysadmin.keeporg/internal/blob"
"cc.tiredsysadmin.keeporg/internal/kdbx"
"cc.tiredsysadmin.keeporg/internal/secret"
"cc.tiredsysadmin.keeporg/internal/store"
)
// MinVaultPasswordLen is the shortest vault password the server will create
// a vault with. It guards an organization's whole password database.
const MinVaultPasswordLen = 12
var errNoKeyring = errors.New(
"this server has no master key, so it cannot hold vault keys: set " +
secret.MasterKeyEnv + " or master_key_file and restart")
// createVault makes a new, empty KDBX database, encrypts it with the given
// vault password and stores it as version 1. When the strategy needs the
// key, the password is wrapped under the server's master key and kept; for
// every other strategy it is used once and forgotten.
func (s *Server) createVault(u *store.User, name, groupID string, strategy store.Strategy,
password string) (dbView, error) {
name = strings.TrimSpace(name)
switch {
case name == "":
return dbView{}, errors.New("the vault needs a name")
case len(password) < MinVaultPasswordLen:
return dbView{}, fmt.Errorf("the vault password must be at least %d characters",
MinVaultPasswordLen)
case !strategy.Valid():
return dbView{}, errors.New("unknown merge strategy")
case strategy.NeedsKey() && s.keyring == nil:
return dbView{}, errNoKeyring
}
// Build the empty database before touching any state, so a failure here
// leaves nothing behind.
db := kdbx.New(name, s.cfg.VaultKDF)
data, err := db.Marshal(password)
if err != nil {
return dbView{}, fmt.Errorf("creating the vault file: %w", err)
}
id := secret.ID()
wrapped := ""
if strategy.NeedsKey() {
if wrapped, err = s.keyring.Wrap(id, password); err != nil {
return dbView{}, err
}
}
if err := s.blobs.Put(id, 1, data); err != nil {
return dbView{}, err
}
var out dbView
err = s.st.Update(func(st *store.State) error {
if st.Groups[groupID] == nil {
return errors.New("unknown owner group")
}
if !st.IsGroupAdmin(u.ID, groupID) {
return errForbidden
}
rec := &store.Database{
ID: id, Name: name, OwnerGroupID: groupID, MergeStrategy: strategy,
WrappedKey: wrapped, Version: 1, SHA256: blob.SHA256(data),
Size: int64(len(data)), UpdatedAt: time.Now(), UpdatedBy: u.Username,
}
st.Databases[id] = rec
out = toView(rec, store.PermAdmin)
return nil
})
if err != nil {
if rerr := s.blobs.Forget(id); rerr != nil {
s.log.Printf("rolling back vault %s: %v", id, rerr)
}
return dbView{}, err
}
s.notifyList(id)
s.log.Printf("%s created vault %q (%s), strategy %s", u.Username, name, id, strategy)
return out, nil
}
// handleCreateVault is the API behind the web form: it creates the database
// record and its first, empty KDBX version in one step.
func (s *Server) handleCreateVault(w http.ResponseWriter, r *http.Request, u *store.User) {
var req struct {
Name string `json:"name"`
OwnerGroupID string `json:"owner_group_id"`
MergeStrategy store.Strategy `json:"merge_strategy"`
Password string `json:"password"`
}
if !readJSON(w, r, &req) {
return
}
if req.MergeStrategy == "" {
req.MergeStrategy = store.StrategyMerge
}
out, err := s.createVault(u, req.Name, req.OwnerGroupID, req.MergeStrategy, req.Password)
if err != nil {
deny(w, err)
return
}
writeJSON(w, http.StatusCreated, out)
}
// handleSetVaultKey stores, replaces or removes the key the server keeps for
// a database, which is what lets an existing database start being merged.
// The password is checked against the stored file before it is accepted.
func (s *Server) handleSetVaultKey(w http.ResponseWriter, r *http.Request, u *store.User) {
id := r.PathValue("id")
var req struct {
Password string `json:"password"`
Forget bool `json:"forget"`
}
if !readJSON(w, r, &req) {
return
}
var (
version int64
perm store.Perm
)
s.st.Read(func(st *store.State) {
perm = st.DatabasePerm(u.ID, id)
if d := st.Databases[id]; d != nil {
version = d.Version
}
})
if perm != store.PermAdmin {
deny(w, errForbidden)
return
}
if !req.Forget {
if s.keyring == nil {
deny(w, errNoKeyring)
return
}
if version == 0 {
deny(w, errors.New("this database has no content yet, so the password cannot be checked"))
return
}
data, err := s.blobs.Get(id, version)
if err != nil {
deny(w, err)
return
}
if _, err := kdbx.Open(data, req.Password); err != nil {
deny(w, fmt.Errorf("that password does not open the stored vault: %w", err))
return
}
}
wrapped := ""
if !req.Forget {
var err error
if wrapped, err = s.keyring.Wrap(id, req.Password); err != nil {
deny(w, err)
return
}
}
err := s.st.Update(func(st *store.State) error {
d := st.Databases[id]
if d == nil {
return errors.New("no such database")
}
if req.Forget && d.MergeStrategy == store.StrategyMerge {
return errors.New("switch the merge strategy away from merge before removing the key")
}
d.WrappedKey = wrapped
return nil
})
if err != nil {
deny(w, err)
return
}
writeJSON(w, http.StatusOK, map[string]bool{"key_stored": wrapped != ""})
}
// vaultPassword unwraps the stored key for a database.
func (s *Server) vaultPassword(id, wrapped string) (string, error) {
if wrapped == "" {
return "", errors.New("no vault key is stored for this database")
}
if s.keyring == nil {
return "", errNoKeyring
}
return s.keyring.Unwrap(id, wrapped)
}
+212
View File
@@ -0,0 +1,212 @@
package server
import (
"errors"
"net/http"
"sort"
"strings"
"time"
"cc.tiredsysadmin.keeporg/internal/secret"
"cc.tiredsysadmin.keeporg/internal/store"
)
// The web interface exists for one job the command line cannot do well:
// creating a vault and setting its password in a browser. It is not a
// password manager and never shows a vault's contents; the server does not
// open a vault except to merge one.
const webCookie = "keeporg_web"
// webSession is a signed-in browser.
type webSession struct {
user *store.User
csrf string
}
// webUser resolves the session cookie, or nil when the browser is not
// signed in.
func (s *Server) webUser(r *http.Request) *webSession {
c, err := r.Cookie(webCookie)
if err != nil || c.Value == "" {
return nil
}
var out *webSession
s.st.Read(func(st *store.State) {
sess := st.Sessions[c.Value]
if sess == nil || time.Now().After(sess.ExpiresAt) {
return
}
if u := st.Users[sess.UserID]; u != nil && !u.Disabled {
cp := *u
out = &webSession{user: &cp, csrf: sess.CSRF}
}
})
return out
}
// checkCSRF verifies the form token against the session's.
func (s *Server) checkCSRF(r *http.Request, sess *webSession) bool {
return sess.csrf != "" && secret.Equal(sess.csrf, r.FormValue("csrf"))
}
func (s *Server) setWebCookie(w http.ResponseWriter, token string, ttl time.Duration) {
http.SetCookie(w, &http.Cookie{
Name: webCookie,
Value: token,
Path: "/",
HttpOnly: true,
Secure: strings.HasPrefix(s.cfg.ExternalURL, "https://"),
SameSite: http.SameSiteStrictMode,
MaxAge: int(ttl.Seconds()),
})
}
func (s *Server) handleWebHome(w http.ResponseWriter, r *http.Request) {
if s.webUser(r) != nil {
http.Redirect(w, r, "/web/vaults", http.StatusFound)
return
}
s.renderSignIn(w, "")
}
func (s *Server) handleWebSignIn(w http.ResponseWriter, r *http.Request) {
if err := r.ParseForm(); err != nil {
fail(w, http.StatusBadRequest, "bad_request", err.Error())
return
}
var token string
err := s.st.Update(func(st *store.State) error {
u, err := checkPassword(st, r.FormValue("username"), r.FormValue("password"),
r.FormValue("totp"))
if err != nil {
return err
}
token = s.newSession(st, u.ID, "web").Token
return nil
})
switch {
case errors.Is(err, errTOTPRequired):
s.renderSignIn(w, "This account needs its one-time code.")
case errors.Is(err, errTOTPInvalid):
s.renderSignIn(w, "That one-time code was rejected.")
case errors.Is(err, errBadCredentials):
s.renderSignIn(w, "Unknown user or wrong password.")
case err != nil:
s.renderSignIn(w, err.Error())
default:
s.setWebCookie(w, token, s.cfg.SessionTTL)
http.Redirect(w, r, "/web/vaults", http.StatusFound)
}
}
func (s *Server) handleWebSignOut(w http.ResponseWriter, r *http.Request) {
sess := s.webUser(r)
if sess == nil {
http.Redirect(w, r, "/web", http.StatusFound)
return
}
if err := r.ParseForm(); err != nil || !s.checkCSRF(r, sess) {
fail(w, http.StatusForbidden, "csrf", "stale form, reload the page")
return
}
if c, err := r.Cookie(webCookie); err == nil {
_ = s.st.Update(func(st *store.State) error {
delete(st.Sessions, c.Value)
return nil
})
}
s.setWebCookie(w, "", -time.Second)
http.Redirect(w, r, "/web", http.StatusFound)
}
// webVault is one row of the vault list.
type webVault struct {
ID string
Name string
Perm string
Strategy string
Version int64
Updated string
By string
HasKey bool
}
// webGroup is one option in the owning-group selector.
type webGroup struct {
ID string
Name string
}
func (s *Server) handleWebVaults(w http.ResponseWriter, r *http.Request) {
sess := s.webUser(r)
if sess == nil {
s.renderSignIn(w, "")
return
}
s.renderVaults(w, sess, "", "")
}
func (s *Server) handleWebCreateVault(w http.ResponseWriter, r *http.Request) {
sess := s.webUser(r)
if sess == nil {
s.renderSignIn(w, "")
return
}
if err := r.ParseForm(); err != nil || !s.checkCSRF(r, sess) {
fail(w, http.StatusForbidden, "csrf", "stale form, reload the page")
return
}
password := r.FormValue("password")
if password != r.FormValue("password2") {
s.renderVaults(w, sess, "The two vault passwords do not match.", "")
return
}
strategy := store.Strategy(r.FormValue("strategy"))
if strategy == "" {
strategy = store.StrategyMerge
}
view, err := s.createVault(sess.user, r.FormValue("name"), r.FormValue("group"),
strategy, password)
if err != nil {
s.renderVaults(w, sess, humanError(err), "")
return
}
s.renderVaults(w, sess, "",
"Created "+view.Name+". Open "+view.Name+".kdbx from your KeepOrg client "+
"with the password you just set.")
}
// humanError keeps internal wording out of the browser.
func humanError(err error) string {
if errors.Is(err, errForbidden) {
return "You do not administer that group."
}
return err.Error()
}
// vaultRows collects what the signed-in user may see, and the groups they
// may create a vault in.
func (s *Server) vaultRows(u *store.User) ([]webVault, []webGroup) {
vaults := []webVault{}
groups := []webGroup{}
s.st.Read(func(st *store.State) {
for _, a := range st.Assignments(u.ID) {
updated := "never"
if a.DB.Version > 0 {
updated = a.DB.UpdatedAt.UTC().Format("2006-01-02 15:04 UTC")
}
vaults = append(vaults, webVault{
ID: a.DB.ID, Name: a.DB.Name, Perm: string(a.Perm),
Strategy: string(a.DB.MergeStrategy), Version: a.DB.Version,
Updated: updated, By: a.DB.UpdatedBy, HasKey: a.DB.WrappedKey != "",
})
}
for id, g := range st.Groups {
if st.IsGroupAdmin(u.ID, id) {
groups = append(groups, webGroup{ID: id, Name: g.Name})
}
}
})
sort.Slice(groups, func(i, j int) bool { return groups[i].Name < groups[j].Name })
return vaults, groups
}
+290
View File
@@ -0,0 +1,290 @@
package server
import (
"io"
"net/http"
"net/http/cookiejar"
"net/url"
"regexp"
"strings"
"testing"
"cc.tiredsysadmin.keeporg/internal/kdbx"
)
// browser is a cookie-keeping client, so the web tests behave like a real
// browser including the session cookie and the CSRF token in each form.
type browser struct {
t *testing.T
h *harness
http *http.Client
}
func newBrowser(t *testing.T, h *harness) *browser {
jar, err := cookiejar.New(nil)
if err != nil {
t.Fatal(err)
}
c := *h.http.Client()
c.Jar = jar
return &browser{t: t, h: h, http: &c}
}
func (b *browser) get(path string) (int, string) {
b.t.Helper()
resp, err := b.http.Get(b.h.http.URL + path)
if err != nil {
b.t.Fatal(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
return resp.StatusCode, string(body)
}
func (b *browser) post(path string, form url.Values) (int, string) {
b.t.Helper()
resp, err := b.http.PostForm(b.h.http.URL+path, form)
if err != nil {
b.t.Fatal(err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
return resp.StatusCode, string(body)
}
func (b *browser) cookie() string {
u, _ := url.Parse(b.h.http.URL)
for _, c := range b.http.Jar.Cookies(u) {
if c.Name == webCookie {
return c.Value
}
}
return ""
}
var csrfPattern = regexp.MustCompile(`name="csrf" value="([^"]+)"`)
func (b *browser) csrf(page string) string {
b.t.Helper()
m := csrfPattern.FindStringSubmatch(page)
if m == nil {
b.t.Fatalf("no CSRF token in the page:\n%s", page)
}
return m[1]
}
// signIn takes the browser through the sign-in form.
func (b *browser) signIn() string {
b.t.Helper()
status, page := b.post("/web/signin", url.Values{
"username": {"admin"}, "password": {adminPassword}})
if status != http.StatusOK || !strings.Contains(page, "Your vaults") {
b.t.Fatalf("sign in: %d\n%s", status, page)
}
return b.csrf(page)
}
func TestWebRootOffersSignIn(t *testing.T) {
b := newBrowser(t, newHarness(t))
status, page := b.get("/")
if status != http.StatusOK || !strings.Contains(page, "Sign in") {
t.Fatalf("root = %d\n%s", status, page)
}
if b.cookie() != "" {
t.Fatal("a session cookie was set before signing in")
}
}
func TestWebSignInRejectsWrongPassword(t *testing.T) {
b := newBrowser(t, newHarness(t))
status, page := b.post("/web/signin", url.Values{
"username": {"admin"}, "password": {"nope"}})
if status != http.StatusOK || !strings.Contains(page, "wrong password") {
t.Fatalf("status %d\n%s", status, page)
}
if b.cookie() != "" {
t.Fatal("a session cookie was set after a failed sign-in")
}
}
func TestWebSessionCookieIsHardened(t *testing.T) {
h := newHarness(t)
b := newBrowser(t, h)
resp, err := b.http.PostForm(h.http.URL+"/web/signin", url.Values{
"username": {"admin"}, "password": {adminPassword}})
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
// The harness serves over http, so Secure is correctly off; the rest
// must be set.
var found bool
for _, c := range resp.Request.Response.Cookies() {
if c.Name != webCookie {
continue
}
found = true
if !c.HttpOnly || c.SameSite != http.SameSiteStrictMode || c.Path != "/" {
t.Fatalf("cookie = %+v", c)
}
}
if !found {
t.Fatal("no session cookie was issued")
}
}
// A server reached over https must mark its cookie Secure, which is also
// why the harness above is configured with an http external URL.
func TestWebCookieIsSecureBehindHTTPS(t *testing.T) {
h := newHarness(t)
h.srv.cfg.ExternalURL = "https://keeporg.example.com"
resp, err := h.http.Client().PostForm(h.http.URL+"/web/signin", url.Values{
"username": {"admin"}, "password": {adminPassword}})
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
for _, c := range resp.Request.Response.Cookies() {
if c.Name == webCookie && !c.Secure {
t.Fatal("the cookie is not marked Secure although the server is https")
}
}
}
func TestWebCreatesAVaultThatOpensWithItsPassword(t *testing.T) {
h := newHarness(t)
b := newBrowser(t, h)
csrf := b.signIn()
status, page := b.post("/web/vaults", url.Values{
"csrf": {csrf}, "name": {"Web Vault"}, "group": {"everyone"},
"strategy": {"merge"},
"password": {vaultPassword}, "password2": {vaultPassword},
})
if status != http.StatusOK {
t.Fatalf("create = %d\n%s", status, page)
}
if !strings.Contains(page, "Created Web Vault") {
t.Fatalf("no confirmation:\n%s", page)
}
if !strings.Contains(page, "Web Vault.kdbx") {
t.Fatal("the new vault is not listed")
}
// The vault the form created is a real KDBX file with that password.
tok, _, _ := h.login("admin", adminPassword, "")
var id string
_, list := h.call("GET", "/api/v1/databases", tok, nil)
for _, row := range list["databases"].([]any) {
m := row.(map[string]any)
if m["name"] == "Web Vault" {
id = m["id"].(string)
if m["merge_strategy"] != "merge" {
t.Fatalf("strategy = %v", m["merge_strategy"])
}
}
}
if id == "" {
t.Fatal("the vault is missing from the API listing")
}
db, err := kdbx.Open(h.pull(tok, id), vaultPassword)
if err != nil {
t.Fatalf("the vault created from the web does not open: %v", err)
}
if db.Name() != "Web Vault" {
t.Fatalf("vault name = %q", db.Name())
}
// The API must never hand the wrapped key back out.
if strings.Contains(strings.ToLower(string(h.stateBytes())), vaultPassword) {
t.Fatal("the vault password is in the state file in the clear")
}
for _, row := range list["databases"].([]any) {
if _, leaked := row.(map[string]any)["wrapped_key"]; leaked {
t.Fatal("the API returned the wrapped vault key")
}
}
}
func TestWebRefusesAFormWithoutItsToken(t *testing.T) {
b := newBrowser(t, newHarness(t))
b.signIn()
status, _ := b.post("/web/vaults", url.Values{
"name": {"No Token"}, "group": {"everyone"},
"password": {vaultPassword}, "password2": {vaultPassword},
})
if status != http.StatusForbidden {
t.Fatalf("a form without a CSRF token returned %d", status)
}
status, _ = b.post("/web/vaults", url.Values{
"csrf": {"not-the-token"}, "name": {"Wrong Token"}, "group": {"everyone"},
"password": {vaultPassword}, "password2": {vaultPassword},
})
if status != http.StatusForbidden {
t.Fatalf("a form with the wrong CSRF token returned %d", status)
}
}
func TestWebRejectsMismatchedAndShortVaultPasswords(t *testing.T) {
b := newBrowser(t, newHarness(t))
csrf := b.signIn()
_, page := b.post("/web/vaults", url.Values{
"csrf": {csrf}, "name": {"V"}, "group": {"everyone"},
"password": {vaultPassword}, "password2": {"something-else"},
})
if !strings.Contains(page, "do not match") {
t.Fatalf("mismatch not reported:\n%s", page)
}
_, page = b.post("/web/vaults", url.Values{
"csrf": {csrf}, "name": {"V"}, "group": {"everyone"},
"password": {"short"}, "password2": {"short"},
})
if !strings.Contains(page, "at least 12") {
t.Fatalf("a short vault password was not reported:\n%s", page)
}
}
func TestWebOnlyOffersGroupsYouAdminister(t *testing.T) {
h := newHarness(t)
admin, _, _ := h.login("admin", adminPassword, "")
// A member who administers nothing sees no create form at all.
status, inv := h.call("POST", "/api/v1/invites", admin,
map[string]any{"group_id": "everyone", "role": "member", "ttl_hours": 1, "max_uses": 1})
if status != http.StatusCreated {
t.Fatalf("invite: %d %v", status, inv)
}
form := url.Values{"username": {"member"}, "password": {"member-password"},
"password2": {"member-password"}}
resp, err := h.http.Client().PostForm(h.http.URL+"/invite/"+inv["token"].(string), form)
if err != nil {
t.Fatal(err)
}
resp.Body.Close()
b := newBrowser(t, h)
status, page := b.post("/web/signin", url.Values{
"username": {"member"}, "password": {"member-password"}})
if status != http.StatusOK {
t.Fatalf("member sign in: %d", status)
}
if strings.Contains(page, "Create vault") {
t.Fatal("a member who administers no group was offered the create form")
}
if !strings.Contains(page, "do not administer a group") {
t.Fatalf("no explanation shown:\n%s", page)
}
}
func TestWebSignOutEndsTheSession(t *testing.T) {
b := newBrowser(t, newHarness(t))
csrf := b.signIn()
if b.cookie() == "" {
t.Fatal("no cookie after signing in")
}
if status, _ := b.post("/web/signout", url.Values{"csrf": {csrf}}); status != http.StatusOK {
t.Fatalf("sign out = %d", status)
}
status, page := b.get("/web/vaults")
if status != http.StatusOK || !strings.Contains(page, "Sign in") {
t.Fatalf("after sign out: %d\n%s", status, page)
}
}
+28 -13
View File
@@ -59,13 +59,22 @@ const (
// StrategyManual rejects the stale push and leaves both copies in place // StrategyManual rejects the stale push and leaves both copies in place
// for a human to reconcile. // for a human to reconcile.
StrategyManual Strategy = "manual" StrategyManual Strategy = "manual"
// StrategyMerge decrypts both copies in memory and folds them together,
// so neither edit is lost. It is the only strategy that requires the
// server to hold the vault's key.
StrategyMerge Strategy = "merge"
) )
// Valid reports whether s is a strategy that may be stored. // Valid reports whether s is a strategy that may be stored.
func (s Strategy) Valid() bool { func (s Strategy) Valid() bool {
return s == StrategyServerWins || s == StrategyClientWins || s == StrategyManual return s == StrategyServerWins || s == StrategyClientWins ||
s == StrategyManual || s == StrategyMerge
} }
// NeedsKey reports whether this strategy can only run when the server holds
// the vault's key.
func (s Strategy) NeedsKey() bool { return s == StrategyMerge }
type User struct { type User struct {
ID string `json:"id"` ID string `json:"id"`
Username string `json:"username"` Username string `json:"username"`
@@ -94,15 +103,19 @@ type Membership struct {
} }
type Database struct { type Database struct {
ID string `json:"id"` ID string `json:"id"`
Name string `json:"name"` Name string `json:"name"`
OwnerGroupID string `json:"owner_group_id"` OwnerGroupID string `json:"owner_group_id"`
MergeStrategy Strategy `json:"merge_strategy"` MergeStrategy Strategy `json:"merge_strategy"`
Version int64 `json:"version"` // WrappedKey is the vault's own password, encrypted under the server's
SHA256 string `json:"sha256"` // master key. It is present only for vaults that are merged here, and it
Size int64 `json:"size"` // is never returned by the API.
UpdatedAt time.Time `json:"updated_at"` WrappedKey string `json:"wrapped_key,omitempty"`
UpdatedBy string `json:"updated_by,omitempty"` Version int64 `json:"version"`
SHA256 string `json:"sha256"`
Size int64 `json:"size"`
UpdatedAt time.Time `json:"updated_at"`
UpdatedBy string `json:"updated_by,omitempty"`
} }
type Grant struct { type Grant struct {
@@ -128,9 +141,11 @@ func (i *Invite) Usable(now time.Time) bool {
} }
type Session struct { type Session struct {
Token string `json:"token"` Token string `json:"token"`
UserID string `json:"user_id"` UserID string `json:"user_id"`
Label string `json:"label,omitempty"` Label string `json:"label,omitempty"`
// CSRF guards the browser forms of a session held in a cookie.
CSRF string `json:"csrf,omitempty"`
CreatedAt time.Time `json:"created_at"` CreatedAt time.Time `json:"created_at"`
ExpiresAt time.Time `json:"expires_at"` ExpiresAt time.Time `json:"expires_at"`
} }
+16
View File
@@ -1,3 +1,19 @@
keeporg (0.2.0) unstable; urgency=medium
* Server-side merging. A vault the server can open has its two concurrent
saves folded together in memory, by KeePass's own synchronization rules,
instead of one being refused. The losing version of an edited entry is
kept in that entry's history.
* New vault creation: a web interface at /web, and `keeporg vault -create`,
both of which build the KDBX file and set its password.
* Vault keys are stored encrypted under a master key in
/etc/keeporg/master.key, created on first start. Back it up separately
from /var/lib/keeporg.
* The KDBX 4 codec, its merge engine, and the ChaCha20, BLAKE2b and Argon2
it needs are part of the package; they carry no new dependencies.
-- tiredsysadmin <rcmd@garage.rcmd.space> Thu, 29 Oct 2026 17:28:45 +0100
keeporg (0.1.0) unstable; urgency=medium keeporg (0.1.0) unstable; urgency=medium
* First release. * First release.
+8 -2
View File
@@ -11,8 +11,14 @@ Description: KeePass database synchronization server
writes between clients. It hosts many databases, controls access through a writes between clients. It hosts many databases, controls access through a
tree of user groups, and accepts registrations by invite link or OIDC. tree of user groups, and accepts registrations by invite link or OIDC.
. .
It never opens a database and cannot read one: master passwords stay on the When two clients save at once it can merge both edits in memory, following
clients. KeePass's own synchronization rules, so neither is lost.
.
Merging requires the server to hold the vault's password, which it keeps
encrypted under a master key in /etc/keeporg/master.key, created on first
start. Back that file up separately from /var/lib/keeporg: whoever holds
both can read every merge vault. Databases on any other conflict strategy
keep their key off the server entirely.
. .
The service runs as the keeporg user with its data in /var/lib/keeporg and The service runs as the keeporg user with its data in /var/lib/keeporg and
its configuration in /etc/keeporg/server.json. Terminate TLS in front of it its configuration in /etc/keeporg/server.json. Terminate TLS in front of it
+2
View File
@@ -63,6 +63,8 @@ if [ "$1" = configure ]; then
systemctl daemon-reload || true systemctl daemon-reload || true
echo "keeporgd: edit /etc/keeporg/server.json, then" \ echo "keeporgd: edit /etc/keeporg/server.json, then" \
"systemctl enable --now keeporgd" >&2 "systemctl enable --now keeporgd" >&2
echo "keeporgd: a master key will be created at /etc/keeporg/master.key;" \
"back it up separately from /var/lib/keeporg" >&2
fi fi
fi fi
exit 0 exit 0
+2 -1
View File
@@ -18,7 +18,8 @@ email=${DEBEMAIL:-rcmd@garage.rcmd.space}
tmp=$(mktemp) tmp=$(mktemp)
{ {
printf 'keeporg (%s) unstable; urgency=medium\n\n' "$version" printf 'keeporg (%s) unstable; urgency=medium\n\n' "$version"
printf ' * %s\n\n' "$message" printf '%s\n' "$message" | fold -s -w 72 | sed -e '1s/^/ * /' -e '2,$s/^/ /'
printf '\n'
printf ' -- %s <%s> %s\n\n' "$author" "$email" "$(date -R)" printf ' -- %s <%s> %s\n\n' "$author" "$email" "$(date -R)"
cat "$file" cat "$file"
} > "$tmp" } > "$tmp"
+7 -7
View File
@@ -1,13 +1,13 @@
# Single source of truth for all platform versions (semver). # Single source of truth for all platform versions (semver).
# Bump with: make bump PLATFORM=<server|linux|macos|windows|bsd|android> PART=<major|minor|patch> # Bump with: make bump PLATFORM=<server|linux|macos|windows|bsd|android> PART=<major|minor|patch>
VERSION_SERVER := 0.1.0 VERSION_SERVER := 0.2.0
VERSION_LINUX := 0.1.0 VERSION_LINUX := 0.2.0
VERSION_MACOS := 0.1.0 VERSION_MACOS := 0.2.0
VERSION_WINDOWS := 0.1.0 VERSION_WINDOWS := 0.2.0
VERSION_BSD := 0.1.0 VERSION_BSD := 0.2.0
VERSION_ANDROID := 0.1.0 VERSION_ANDROID := 0.2.0
# Monotonic Android versionCode, never decreases. # Monotonic Android versionCode, never decreases.
ANDROID_VERSION_CODE := 1 ANDROID_VERSION_CODE := 2
# Reverse-DNS application identifiers. # Reverse-DNS application identifiers.
APPID_ANDROID := cc.tiredsysadmin.android.keeporg APPID_ANDROID := cc.tiredsysadmin.android.keeporg