# Sizzletracker (Android) — Developer Handover This document is written for a developer who is **new to Android** and needs to understand, build, and extend this codebase. Read it top to bottom once; after that it works as a reference. --- ## 1. What this app is A retro, grid-based **music tracker** for Android. The visual language is a monospace font + pixel blocks laid out on a strict grid (demoscene tracker style). It is playable with **four equally-capable input methods**: touchscreen, physical keyboard, gamepad, and MIDI (USB or Bluetooth). The UI has four tabs: | Tab | Package | What it does | |-----|---------|--------------| | **Tracker** | `ui/tracker` | Pattern grid (4 tracks) + arrangement piano-roll (8 lanes) | | **Mix** | `ui/mixer` | 4-channel mixer: instrument + 4 FX, MIDI ch, volume, mute/solo | | **Toolbox** | `ui/toolbox` | 16 slots of instruments/effects with a generated parameter editor | | **Setup** | `ui/settings` | Project save/load, themes, gamepad & MIDI bindings | --- ## 2. How to build & run Requirements: **Android Studio** (Koala / 2024.1 or newer) which bundles a JDK 17 and the Android SDK. 1. Open the project root in Android Studio. It will detect the Gradle build. 2. Let it download the SDK for `compileSdk 34`. 3. Pick a device/emulator running **Android 8.0 (API 26)** or newer and press **Run**. > The Gradle wrapper (`gradle/wrapper/gradle-wrapper.jar`) **is committed** > (Gradle's recommended practice), so `./gradlew assembleDebug` works on a fresh > clone with no extra setup. A `staging` build type (`./gradlew assembleStaging`) > gives release-like performance while still using the debug signing key. Key versions live in **one file**: `gradle/libs.versions.toml` (the "version catalog"). Change a number there and the whole project follows. --- ## 3. The big picture (architecture) ``` ┌──────────── Input sources ────────────┐ Touch ───▶│ (Compose gestures, call ViewModel) │ Keyboard ─▶ KeyboardInput ─┐ │ Gamepad ──▶ GamepadInput ──┤─▶ InputRouter (flow) ──┼──▶ AppViewModel MIDI ─────▶ MidiInput ─────┘ │ │ └───────────────────────────────────────┘ │ reads/writes ▼ Project (the song) ▲ reads on audio thread │ AudioEngine ─────────┘ ──▶ Oboe / AudioTrack ──▶ speakers (own thread) ``` Three ideas hold it together: 1. **One neutral action type.** Every non-touch device is translated into an `InputAction` (see `input/InputAction.kt`) and pushed through `InputRouter`. The ViewModel reacts to those actions in exactly one place (`AppViewModel.onAction`). Touch gestures call the same ViewModel methods directly. That is *why* all four inputs are equally capable — features are implemented once. 2. **One song object.** `model/Project.kt` holds the entire song in memory. The UI edits it; the audio engine reads it; the file format serializes it. 3. **Two threads.** The UI runs on the main thread (Jetpack Compose). The sound runs on a dedicated high-priority thread inside `AudioEngine`. They share the `Project` object. Because the audio thread only ever *reads* simple `Int`/ `Float` fields, we accept the occasional benign race instead of locking (locks would risk audio glitches). See §7. --- ## 4. Package/file map ``` space.rcmd.android.sizzle ├── SizzleApp.kt Application = tiny DI container (Project, InputRouter, AudioEngine) ├── MainActivity.kt Hosts Compose UI; forwards HW key/motion events; starts MIDI + service │ ├── model/ Pure data, no Android imports (unit-test friendly) │ ├── Music.kt Pitch, TimeSignature, Scale (incl. scale-step logic) │ ├── Pattern.kt Cell, CellColumn, Pattern (4 tracks x N lines) │ ├── Arrangement.kt Arrangement (8 lanes; canvas denominated in bars, up to 256), LoopRegion A/B │ ├── Mixer.kt Mixer, MixerChannel (4 channels; volume/mute/solo/limiter) │ ├── Toolbox.kt ToolboxType (all instruments+effects), ParamSpec, ToolboxSlot, SamplerPads │ ├── AmbiencePresets.kt Factory reverb spaces seeded into the preset library │ └── Project.kt The whole song │ ├── input/ The unified input layer │ ├── InputAction.kt The neutral action vocabulary │ ├── InputActions.kt Bindable-action catalogue (for the rebind/learn UI) │ ├── InputRouter.kt Flow-based message bus (+ learn mode) │ ├── BindingStore.kt Persists gamepad/MIDI bindings │ ├── KeyboardInput.kt KeyEvent -> InputAction (+ a 2-octave typing piano) │ ├── GamepadInput.kt KeyEvent/MotionEvent -> InputAction (rebindable) │ └── MidiInput.kt MIDI bytes -> InputAction (USB & Bluetooth via MidiManager) │ ├── audio/ The real-time sound engine (own thread) │ ├── Transport.kt TransportState snapshot published to the UI │ ├── SynthVoice.kt One NES-style voice: pulse/triangle/noise + ADSR │ ├── SampleVoice.kt Pitched + sliced playback of a decoded sample │ ├── SampleStore.kt Process-wide decoded-PCM cache + WAV encode/decode │ ├── SampleRecorder.kt Ad-hoc mic/USB capture into a sample │ ├── SoundFontLoader.kt Extract PCM from an .sf2 / .xi into the SampleStore │ ├── Effects.kt Insert DSP: TapeDelay, Filter, Bitcrusher, GraphicEq (+ AudioEffect API) │ ├── AmbienceReverb.kt FDN reverb processor │ ├── MasterRecorder.kt Renders the master bus (with FX tail) to a WAV │ ├── NativeAudioBridge.kt JNI bridge to the Oboe callback (calls fillBlock) │ └── AudioEngine.kt Oboe/AudioTrack + sample-accurate sequencer + voice/FX mixing │ ├── cpp/native_audio.cpp Low-latency Oboe stream; RT callback pulls audio via JNI │ ├── io/ Human-readable text persistence │ ├── PresetIo.kt Instrument/effect preset <-> KEY=VALUE text │ ├── PresetLibrary.kt On-disk preset library (+ .zip sample bundles) │ ├── ProjectIo.kt Whole song <-> internal .sng text (full fidelity) │ ├── ProjectStore.kt Autosave / load of the current project │ ├── SongLibrary.kt Named project save/load browser backing │ ├── SngFormat.kt Desktop-compatible .sng interchange read/write │ ├── SettingsStore.kt App settings (theme, recording dir, keyboard) via DataStore │ ├── ThemeIo.kt RetroPalette <-> .szt text │ └── ThemeLibrary.kt On-disk theme library (import/export/delete) │ ├── playback/ │ ├── EnginePlayer.kt SimpleBasePlayer wrapping the engine for MediaSession │ └── PlaybackService.kt MediaSessionService: background playback + media notification │ └── ui/ ├── AppViewModel.kt Screen state + THE single InputAction handler ├── App.kt Tab shell ├── Responsive.kt Portrait/landscape layout helpers (nav rail, two-column) ├── StackedKeyboard.kt On-screen audition / punch-in piano ├── theme/Theme.kt RetroPalette + monospace typography + SizzleTheme ├── components/ Widgets (RetroButton/Dropdown…), GlyphCache, Piano ├── tracker/ TrackerScreen, PatternGrid, ArrangementRoll ├── mixer/MixerScreen.kt ├── toolbox/ ToolboxScreen, ParamControl, Sampler/SoundFont/Delay/Lfo editors └── settings/SettingsScreen.kt ``` --- ## 5. Key concepts you must understand ### 5.1 Ticks, beats, bars A tracker **line** (row) is one tick. The number of lines per beat is set by the **time signature** (`model/Music.kt`): - 3/4 → 3 lines per beat, lengths 12/24/48 - 4/4 → 4 lines per beat, lengths 16/32/64 - 5/4 → 5 lines per beat, lengths 20/40/80 `PatternGrid` tints every beat row and every bar row differently using `line % linesPerBeat` and `line % linesPerBar`. ### 5.2 The "revision" redraw trick (and the strong-skipping rule) The model classes are **plain** (not Compose-observable) so the audio thread can read them cheaply. After any edit the ViewModel does `revision++` (`AppViewModel.touched()`, exposed as `bumpForToolbar()` for direct model edits). **The rule that makes this actually work:** the Compose compiler we use (2.0.20) has **strong skipping** enabled by default. It will *skip* a composable whose parameters are referentially unchanged — even "unstable" ones like `vm` — so a `revision++` in a *parent* is NOT enough; a child that reads plain model data would be skipped and only refresh on a tab switch (which recreates the subtree). So: **every composable that reads plain, mutable model data must itself read `vm.revision`** (e.g. `val rev = vm.revision`). Reading a snapshot state directly subscribes that composable's restart scope, so `revision++` invalidates it regardless of skipping. Where a composable has no `vm` (e.g. `SlotTile`), pass `rev = vm.revision` as a parameter instead — a changed `Int` param defeats skipping too. Observable state (cursor position, `transport`, `palette`) already works without this because it is real Compose state. If you edit the model and the screen doesn't update: (a) did you call a `touched()`/`bumpForToolbar()` path, and (b) does the leaf that renders that data read `vm.revision`? This was the cause of an early "UI only redraws on tab switch" bug. ### 5.3 The note-entry gesture Tap a cell → cursor moves there (`focusCell`). Drag up/down → `editFocused(dir)`: - On the NOTE column, `Scale.step()` moves to the next in-scale pitch. - On VELOCITY/CHANNEL it steps the number. Every step fires a haptic tick. The same `editFocused` runs from keyboard PageUp/ Down, gamepad right-stick, and MIDI CC — one implementation, four inputs. ### 5.4 Toolbox devices are data, not classes Instead of a class per instrument/effect, every device is one `ToolboxType` enum entry that lists its `ParamSpec`s. A `ToolboxSlot` stores the chosen type + a `name -> value` string map. The parameter editor UI and the preset text format are both **generated** from that list. See §6 to add a new device. --- ## 6. How to extend ### Add a new instrument or effect 1. Add an entry to `ToolboxType` (`model/Toolbox.kt`) with its `ParamSpec` list. 2. Teach the engine how to sound it: - **Instrument**: handle its type where the engine picks a voice per channel (see how `NES_SYNTH`, `SAMPLER`, and `SOUNDFONT` are dispatched in `AudioEngine`). - **Signal effect**: add a processor in `audio/Effects.kt` implementing `AudioEffect` and register it in `AudioEffect.create(...)`; the per-channel insert chain picks it up automatically. - **MIDI effect** (arp/transpose/LFO-style): hook into the sequencer instead of the signal path (see `advanceArps` / `applyLfos` in `AudioEngine`). The Toolbox picker, parameter editor, and preset save/load all work automatically from the `ParamSpec` list. A device with many parameters can ship a bespoke editor (see `ui/toolbox/DelayEditor.kt`, `SamplerEditor.kt`). ### Add a new input gesture/control 1. Add a case to `InputAction`. 2. Emit it from each source (`KeyboardInput`, `GamepadInput`, `MidiInput`) and/or a Compose gesture. 3. Handle it once in `AppViewModel.onAction`. ### Add a new color theme Add a `RetroPalette` to `RetroPalette.ALL` in `ui/theme/Theme.kt`. It appears in the Settings theme picker immediately. --- ## 7. Threading & performance notes - The audio render loop is in `AudioEngine.renderLoop()` on a `MAX_PRIORITY` thread. It **must never allocate or block**. Keep it that way. - Timing is derived from *counting audio samples* (`samplesPerLine`), not from timers, so playback never drifts. - UI ⇄ audio sharing is lock-free by design (audio reads plain fields). This can produce a one-frame-stale read during a simultaneous edit — harmless for a tracker. If you ever add compound state that must be read atomically, publish it as an immutable snapshot (like `TransportState`) rather than adding locks. - Block size is `BLOCK_FRAMES = 192` (~4 ms @ 48 kHz). Lower = tighter latency, higher = safer against underruns. - **UI rendering**: the pattern grid (`PatternGrid`) and arrangement roll (`ArrangementRoll`) are each drawn as ONE `Canvas` with a cached `rememberTextMeasurer`, not a tree of per-cell composables. This is what keeps the grid smooth while the playhead moves — a single draw pass instead of recomposing hundreds of `Text` nodes each frame. If you add grid features, draw them in the Canvas rather than adding child composables. - **Arrangement model**: each of the 8 lanes is tied to its own block (lane i ⟷ block i; `Project.patterns` holds 8 blocks). Toggling a beat cell on lane i enables block i at that beat; the sequencer plays it when the playhead crosses. Tap a lane number (left column) to edit that block in the tracker above. --- ## 8. File formats ### Preset (`PresetIo`) ``` SIZZLE-PRESET 1 TYPE=NES_SYNTH NAME=Lead wave=Pulse25 attack=0.01 ... ``` Copy/paste from the Toolbox parameter editor uses this exact text. ### Project `.sng` (`ProjectIo`) Line-oriented, sectioned (`[PATTERN ...]`, `[ARRANGEMENT ...]`, `[MIXER]`, `[TOOLBOX]`). Only non-empty cells are written. It is fully round-trip safe. --- ## 9. Current status — implemented vs. TODO This is an honest map so you know where the edges are. The **architecture and all four tabs are in place and interactive**, and the feature set below is implemented and working; the short "Remaining / optional" list at the end is polish and interchange niceties, not functional gaps. ### Implemented and working - Full 4-tab UI, retro monospace theme (+ theme switching), tab navigation. - Tracker pattern grid: beat/bar coloring, cursor, tap-to-focus, drag-to-edit with haptics, scale-aware note entry, transport toolbar, length/scale/root. - Arrangement roll: 8 lanes, place/clear blocks, playhead follow, loop toolbar with A/B enable + repeat counts. - Mixer: 4 strips, instrument/FX routing to toolbox, MIDI ch, volume, mute/solo. - Toolbox: 16 slots, device picker, **auto-generated** parameter editor, preset copy/paste (text). - Audio engine: dedicated thread, sample-accurate sequencer, NES-style synth (pulse/tri/noise + ADSR), per-track mute/solo/volume, live note audition. - **Arrangement-driven playback** (milestone 1 ✓): the sequencer walks a pre-expanded beat playlist (with A/B loop repeats) across all 8 lanes; each lane keeps its own within-pattern cursor so multi-beat blocks advance through the pattern. Falls back to looping the active pattern when the arrangement is empty. Voices are a lane×track matrix so lanes layer polyphonically. - **Per-channel effect DSP** (milestone 3 ✓): each mixer channel runs an insert chain built from its FX slots — Tape Delay, Digital Reverb, Filter (LP/HP/BP), Bitcrusher, and 10-band EQ are real processors in `audio/Effects.kt`. Effect parameters update live per audio block; add/remove rebuilds the chain. - **Sampler instrument** ✓: WAV import (SAF file picker) + ad-hoc mic/USB recorder (`audio/SampleRecorder.kt`), decoded to a mono cache (`audio/SampleStore.kt`), played pitched + sliced by `audio/SampleVoice.kt`. Its editor (`ui/toolbox/SamplerEditor.kt`) shows a waveform with two draggable slice markers, start-octave + volume, and a two-octave audition keyboard. Each (lane, track) has both a synth and a sample voice; the channel instrument type selects which one a note triggers. - Input: keyboard (nav + typing-piano + transport), gamepad (buttons + sticks + hat), MIDI in (note + CC, USB & BT), all via the unified router. - **MIDI-learn & gamepad rebind** ✓: the Settings tab lists every bindable action with its current binding and a Learn button; arming it sets the router's `learnMode`, and the next unbound CC / gamepad button is written into `MidiInput.ccBindings` / `GamepadInput.bindings` (the handlers are now app singletons shared with Settings). Bindings are clearable per action. - **MIDI effects (Transposer / Arpeggiator / LFO)** ✓: Transposer shifts notes by its semitone amount; the Arpeggiator octave-cycles a captured note (Up/Down/UpDown/Random, 1–4 octaves) at a tempo-synced division (`AudioEngine.advanceArps`); the LFO (`applyLfos`) modulates a chosen target parameter of another device around a captured centre (Sine/Tri/Saw/Square/S&H), with a target picker in its editor (`ui/toolbox/LfoEditor.kt`). - **USB / audio device selection** ✓: Settings → Audio Devices lists the system's output and input devices; choosing one routes the engine's `AudioTrack` and the recorder's `AudioRecord` via `setPreferredDevice` (API 28+), so a USB DAC / USB audio interface can be used for playback and recording. - **Full MediaSession** ✓: `playback/PlaybackService` is now a media3 `MediaSessionService` hosting a session that wraps the engine via `EnginePlayer` (`SimpleBasePlayer`). This gives lock-screen controls, Bluetooth/headset media buttons, and the media notification for free, mirroring the on-screen transport. - Persistence: preset text I/O, a full-fidelity internal project text format (`ProjectIo`, keeps mixer/FX/instruments), AND **desktop-compatible `.sng`** read/write (`io/SngFormat.kt`) matching the reference sizzletracker line format (`version`/`bpm`/`sig`, `block`…`endblock` with `roll`/`track`/steps) — covered by passing JVM unit tests in `app/src/test`. - Background playback foreground service with Play/Pause/Stop notification. - **Oboe / AAudio native output** ✓ (default): `src/main/cpp/native_audio.cpp` opens a low-latency Oboe stream whose real-time callback pulls audio via JNI (`audio/NativeAudioBridge.kt`) from the SAME `AudioEngine.fillBlock` the AudioTrack loop uses — so both backends produce identical sound, only the delivery path differs. Oboe is the default engine; it falls back to the AudioTrack loop only if it fails to initialise (or the native lib is absent), surfacing a popup with the reason (there is no user toggle). Built via **NDK r27** (`ndkVersion` in `app/build.gradle.kts`) + CMake + the `com.google.oboe:oboe:1.10.0` prefab AAR. All packaged `.so` files are **16 KB page-aligned** (Android 15 / Google Play requirement): NDK r27 aligns ELF load segments to 16 KB and ships a 16 KB-aligned `libc++_shared.so`, Oboe 1.10.0's lib is aligned, our `CMakeLists.txt` also passes `-Wl,-z,max-page-size=16384`, and AGP stores each `.so` at a 16 KB-aligned offset in the APK. (Verify with `llvm-readelf -l .so` → LOAD `Align 0x4000`.) - **SF2 / XI loader instrument** ✓: `audio/SoundFontLoader.kt` extracts PCM from an `.sf2` / `.xi` into the `SampleStore`, played back through the same `SampleVoice` path as the Sampler; edited via `ui/toolbox/SoundFontEditor.kt` (root note, volume, ADSR). - **Master recording** ✓: `audio/MasterRecorder.kt` renders the stereo master bus (plus a configurable FX tail) to a WAV in a user-chosen folder (SAF). - **Stereo output & recording** ✓: `fillBlock` renders interleaved stereo and the recorder captures two channels (voices/inserts are mono, so both channels carry the same sum). - **Responsive landscape layout** ✓ (`ui/Responsive.kt`): nav rail, side-by-side screens, two-column mixer/editors, wider on-screen keyboards for tablets/handhelds. - **Arrangement niceties** ✓: 256-bar canvas with viewport-culled drawing, per-lane mutes, and centre-follow scrolling. - **Per-bus soft-clip limiter** ✓, sampler QoL (fire-on-tap, clear, normalize), theme import/export, and preset `.zip` bundle import/export. - **Sample-pack builder** ✓: `tools/samplepack-builder.html`, a standalone offline page that assembles importable Sampler `.zip` bundles. ### Remaining / optional - **Byte-exact desktop `.sng` compatibility.** `io/SngFormat.kt` reads/writes the reference line format and round-trips musically; a field-for-field byte match with the CLI project layout is the remaining interchange nicety. - **Full C++ audio port.** The DSP currently runs in Kotlin behind the Oboe callback (via JNI). Porting the hot path to C++ would remove JVM/GC from the real-time thread — a performance hardening step, not a functional gap. - **Route the Oboe backend through the chosen output device** (Oboe `setDeviceId`), matching the AudioTrack path's device selection. --- ## 10. Testing suggestions - Pure `model/` classes (`Scale.step`, `TimeSignature`, `Pattern.resize`, `ProjectIo` round-trip, `PresetIo` round-trip) are plain Kotlin — cover them with JVM unit tests first; they need no device. - For the engine, render a few blocks with a known pattern and assert the voice triggers on the expected sample offsets.