sizzletracker-android/docs/DEVELOPER_HANDOVER.md

Sizzletracker (Android) — Developer Handover

This document is written for a developer who is new to Android and needs to understand, build, and extend this codebase. Read it top to bottom once; after that it works as a reference.

---

1. What this app is

A retro, grid-based music tracker for Android. The visual language is a monospace font + pixel blocks laid out on a strict grid (demoscene tracker style). It is playable with four equally-capable input methods: touchscreen, physical keyboard, gamepad, and MIDI (USB or Bluetooth).

The UI has four tabs:

Tab	Package	What it does
Tracker	ui/tracker	Pattern grid (4 tracks) + arrangement piano-roll (8 lanes)
Mix	ui/mixer	4-channel mixer: instrument + 4 FX, MIDI ch, volume, mute/solo
Toolbox	ui/toolbox	16 slots of instruments/effects with a generated parameter editor
Setup	ui/settings	Project save/load, themes, gamepad & MIDI bindings

---

2. How to build & run

Requirements: Android Studio (Koala / 2024.1 or newer) which bundles a JDK 17 and the Android SDK.

1. Open the project root in Android Studio. It will detect the Gradle build.
2. Let it download the SDK for compileSdk 34.
3. Pick a device/emulator running Android 8.0 (API 26) or newer and press Run.

The Gradle wrapper (gradle/wrapper/gradle-wrapper.jar) is committed (Gradle's recommended practice), so ./gradlew assembleDebug works on a fresh clone with no extra setup. A staging build type (./gradlew assembleStaging) gives release-like performance while still using the debug signing key.

Key versions live in one file: gradle/libs.versions.toml (the "version catalog"). Change a number there and the whole project follows.

---

3. The big picture (architecture)

            ┌──────────── Input sources ────────────┐
  Touch ───▶│ (Compose gestures, call ViewModel)     │
  Keyboard ─▶ KeyboardInput ─┐                        │
  Gamepad ──▶ GamepadInput ──┤─▶ InputRouter (flow) ──┼──▶ AppViewModel
  MIDI ─────▶ MidiInput ─────┘                        │        │
            └───────────────────────────────────────┘        │ reads/writes
                                                              ▼
                                                          Project  (the song)
                                                              ▲
                                     reads on audio thread    │
                                          AudioEngine ─────────┘  ──▶ Oboe / AudioTrack ──▶ speakers
                                          (own thread)

Three ideas hold it together:

1. One neutral action type. Every non-touch device is translated into an InputAction (see input/InputAction.kt) and pushed through InputRouter. The ViewModel reacts to those actions in exactly one place (AppViewModel.onAction). Touch gestures call the same ViewModel methods directly. That is why all four inputs are equally capable — features are implemented once.

2. One song object. model/Project.kt holds the entire song in memory. The UI edits it; the audio engine reads it; the file format serializes it.

3. Two threads. The UI runs on the main thread (Jetpack Compose). The sound runs on a dedicated high-priority thread inside AudioEngine. They share the Project object. Because the audio thread only ever reads simple Int/ Float fields, we accept the occasional benign race instead of locking (locks would risk audio glitches). See §7.

---

4. Package/file map

space.rcmd.android.sizzle
├── SizzleApp.kt            Application = tiny DI container (Project, InputRouter, AudioEngine)
├── MainActivity.kt         Hosts Compose UI; forwards HW key/motion events; starts MIDI + service
│
├── model/                  Pure data, no Android imports (unit-test friendly)
│   ├── Music.kt            Pitch, TimeSignature, Scale (incl. scale-step logic)
│   ├── Pattern.kt          Cell, CellColumn, Pattern (4 tracks x N lines)
│   ├── Arrangement.kt      Arrangement (8 lanes; canvas denominated in bars, up to 256), LoopRegion A/B
│   ├── Mixer.kt            Mixer, MixerChannel (4 channels; volume/mute/solo/limiter)
│   ├── Toolbox.kt          ToolboxType (all instruments+effects), ParamSpec, ToolboxSlot, SamplerPads
│   ├── AmbiencePresets.kt  Factory reverb spaces seeded into the preset library
│   └── Project.kt          The whole song
│
├── input/                  The unified input layer
│   ├── InputAction.kt      The neutral action vocabulary
│   ├── InputActions.kt     Bindable-action catalogue (for the rebind/learn UI)
│   ├── InputRouter.kt      Flow-based message bus (+ learn mode)
│   ├── BindingStore.kt     Persists gamepad/MIDI bindings
│   ├── KeyboardInput.kt    KeyEvent  -> InputAction (+ a 2-octave typing piano)
│   ├── GamepadInput.kt     KeyEvent/MotionEvent -> InputAction (rebindable)
│   └── MidiInput.kt        MIDI bytes -> InputAction (USB & Bluetooth via MidiManager)
│
├── audio/                  The real-time sound engine (own thread)
│   ├── Transport.kt        TransportState snapshot published to the UI
│   ├── SynthVoice.kt       One NES-style voice: pulse/triangle/noise + ADSR
│   ├── SampleVoice.kt      Pitched + sliced playback of a decoded sample
│   ├── SampleStore.kt      Process-wide decoded-PCM cache + WAV encode/decode
│   ├── SampleRecorder.kt   Ad-hoc mic/USB capture into a sample
│   ├── SoundFontLoader.kt  Extract PCM from an .sf2 / .xi into the SampleStore
│   ├── Effects.kt          Insert DSP: TapeDelay, Filter, Bitcrusher, GraphicEq (+ AudioEffect API)
│   ├── AmbienceReverb.kt   FDN reverb processor
│   ├── MasterRecorder.kt   Renders the master bus (with FX tail) to a WAV
│   ├── NativeAudioBridge.kt  JNI bridge to the Oboe callback (calls fillBlock)
│   └── AudioEngine.kt      Oboe/AudioTrack + sample-accurate sequencer + voice/FX mixing
│
├── cpp/native_audio.cpp    Low-latency Oboe stream; RT callback pulls audio via JNI
│
├── io/                     Human-readable text persistence
│   ├── PresetIo.kt         Instrument/effect preset  <->  KEY=VALUE text
│   ├── PresetLibrary.kt    On-disk preset library (+ .zip sample bundles)
│   ├── ProjectIo.kt        Whole song  <->  internal .sng text (full fidelity)
│   ├── ProjectStore.kt     Autosave / load of the current project
│   ├── SongLibrary.kt      Named project save/load browser backing
│   ├── SngFormat.kt        Desktop-compatible .sng interchange read/write
│   ├── SettingsStore.kt    App settings (theme, recording dir, keyboard) via DataStore
│   ├── ThemeIo.kt          RetroPalette  <->  .szt text
│   └── ThemeLibrary.kt     On-disk theme library (import/export/delete)
│
├── playback/
│   ├── EnginePlayer.kt     SimpleBasePlayer wrapping the engine for MediaSession
│   └── PlaybackService.kt  MediaSessionService: background playback + media notification
│
└── ui/
    ├── AppViewModel.kt     Screen state + THE single InputAction handler
    ├── App.kt              Tab shell
    ├── Responsive.kt       Portrait/landscape layout helpers (nav rail, two-column)
    ├── StackedKeyboard.kt  On-screen audition / punch-in piano
    ├── theme/Theme.kt      RetroPalette + monospace typography + SizzleTheme
    ├── components/         Widgets (RetroButton/Dropdown…), GlyphCache, Piano
    ├── tracker/            TrackerScreen, PatternGrid, ArrangementRoll
    ├── mixer/MixerScreen.kt
    ├── toolbox/            ToolboxScreen, ParamControl, Sampler/SoundFont/Delay/Lfo editors
    └── settings/SettingsScreen.kt

---

5. Key concepts you must understand

5.1 Ticks, beats, bars

A tracker line (row) is one tick. The number of lines per beat is set by the time signature (model/Music.kt):

• 3/4 → 3 lines per beat, lengths 12/24/48
• 4/4 → 4 lines per beat, lengths 16/32/64
• 5/4 → 5 lines per beat, lengths 20/40/80

PatternGrid tints every beat row and every bar row differently using line % linesPerBeat and line % linesPerBar.

5.2 The "revision" redraw trick (and the strong-skipping rule)

The model classes are plain (not Compose-observable) so the audio thread can read them cheaply. After any edit the ViewModel does revision++ (AppViewModel.touched(), exposed as bumpForToolbar() for direct model edits).

The rule that makes this actually work: the Compose compiler we use (2.0.20) has strong skipping enabled by default. It will skip a composable whose parameters are referentially unchanged — even "unstable" ones like vm — so a revision++ in a parent is NOT enough; a child that reads plain model data would be skipped and only refresh on a tab switch (which recreates the subtree).

So: every composable that reads plain, mutable model data must itself read vm.revision (e.g. val rev = vm.revision). Reading a snapshot state directly subscribes that composable's restart scope, so revision++ invalidates it regardless of skipping. Where a composable has no vm (e.g. SlotTile), pass rev = vm.revision as a parameter instead — a changed Int param defeats skipping too. Observable state (cursor position, transport, palette) already works without this because it is real Compose state.

If you edit the model and the screen doesn't update: (a) did you call a touched()/bumpForToolbar() path, and (b) does the leaf that renders that data read vm.revision? This was the cause of an early "UI only redraws on tab switch" bug.

5.3 The note-entry gesture

Tap a cell → cursor moves there (focusCell). Drag up/down → editFocused(dir):

• On the NOTE column, Scale.step() moves to the next in-scale pitch.
• On VELOCITY/CHANNEL it steps the number. Every step fires a haptic tick. The same editFocused runs from keyboard PageUp/ Down, gamepad right-stick, and MIDI CC — one implementation, four inputs.

5.4 Toolbox devices are data, not classes

Instead of a class per instrument/effect, every device is one ToolboxType enum entry that lists its ParamSpecs. A ToolboxSlot stores the chosen type + a name -> value string map. The parameter editor UI and the preset text format are both generated from that list. See §6 to add a new device.

---

6. How to extend

Add a new instrument or effect

1. Add an entry to ToolboxType (model/Toolbox.kt) with its ParamSpec list.
2. Teach the engine how to sound it:
   • Instrument: handle its type where the engine picks a voice per channel (see how NES_SYNTH, SAMPLER, and SOUNDFONT are dispatched in AudioEngine).
   • Signal effect: add a processor in audio/Effects.kt implementing AudioEffect and register it in AudioEffect.create(...); the per-channel insert chain picks it up automatically.
   • MIDI effect (arp/transpose/LFO-style): hook into the sequencer instead of the signal path (see advanceArps / applyLfos in AudioEngine). The Toolbox picker, parameter editor, and preset save/load all work automatically from the ParamSpec list. A device with many parameters can ship a bespoke editor (see ui/toolbox/DelayEditor.kt, SamplerEditor.kt).

Add a new input gesture/control

1. Add a case to InputAction.
2. Emit it from each source (KeyboardInput, GamepadInput, MidiInput) and/or a Compose gesture.
3. Handle it once in AppViewModel.onAction.

Add a new color theme

Add a RetroPalette to RetroPalette.ALL in ui/theme/Theme.kt. It appears in the Settings theme picker immediately.

---

7. Threading & performance notes

• The audio render loop is in AudioEngine.renderLoop() on a MAX_PRIORITY thread. It must never allocate or block. Keep it that way.
• Timing is derived from counting audio samples (samplesPerLine), not from timers, so playback never drifts.
• UI ⇄ audio sharing is lock-free by design (audio reads plain fields). This can produce a one-frame-stale read during a simultaneous edit — harmless for a tracker. If you ever add compound state that must be read atomically, publish it as an immutable snapshot (like TransportState) rather than adding locks.
• Block size is BLOCK_FRAMES = 192 (~4 ms @ 48 kHz). Lower = tighter latency, higher = safer against underruns.
• UI rendering: the pattern grid (PatternGrid) and arrangement roll (ArrangementRoll) are each drawn as ONE Canvas with a cached rememberTextMeasurer, not a tree of per-cell composables. This is what keeps the grid smooth while the playhead moves — a single draw pass instead of recomposing hundreds of Text nodes each frame. If you add grid features, draw them in the Canvas rather than adding child composables.
• Arrangement model: each of the 8 lanes is tied to its own block (lane i ⟷ block i; Project.patterns holds 8 blocks). Toggling a beat cell on lane i enables block i at that beat; the sequencer plays it when the playhead crosses. Tap a lane number (left column) to edit that block in the tracker above.

---

8. File formats

Preset (PresetIo)

SIZZLE-PRESET 1
TYPE=NES_SYNTH
NAME=Lead
wave=Pulse25
attack=0.01
...

Copy/paste from the Toolbox parameter editor uses this exact text.

Project .sng (ProjectIo)

Line-oriented, sectioned ([PATTERN ...], [ARRANGEMENT ...], [MIXER], [TOOLBOX]). Only non-empty cells are written. It is fully round-trip safe.

---

9. Current status — implemented vs. TODO

This is an honest map so you know where the edges are. The architecture and all four tabs are in place and interactive, and the feature set below is implemented and working; the short "Remaining / optional" list at the end is polish and interchange niceties, not functional gaps.

Implemented and working

• Full 4-tab UI, retro monospace theme (+ theme switching), tab navigation.
• Tracker pattern grid: beat/bar coloring, cursor, tap-to-focus, drag-to-edit with haptics, scale-aware note entry, transport toolbar, length/scale/root.
• Arrangement roll: 8 lanes, place/clear blocks, playhead follow, loop toolbar with A/B enable + repeat counts.
• Mixer: 4 strips, instrument/FX routing to toolbox, MIDI ch, volume, mute/solo.
• Toolbox: 16 slots, device picker, auto-generated parameter editor, preset copy/paste (text).
• Audio engine: dedicated thread, sample-accurate sequencer, NES-style synth (pulse/tri/noise + ADSR), per-track mute/solo/volume, live note audition.
• Arrangement-driven playback (milestone 1 ✓): the sequencer walks a pre-expanded beat playlist (with A/B loop repeats) across all 8 lanes; each lane keeps its own within-pattern cursor so multi-beat blocks advance through the pattern. Falls back to looping the active pattern when the arrangement is empty. Voices are a lane×track matrix so lanes layer polyphonically.
• Per-channel effect DSP (milestone 3 ✓): each mixer channel runs an insert chain built from its FX slots — Tape Delay, Digital Reverb, Filter (LP/HP/BP), Bitcrusher, and 10-band EQ are real processors in audio/Effects.kt. Effect parameters update live per audio block; add/remove rebuilds the chain.
• Sampler instrument ✓: WAV import (SAF file picker) + ad-hoc mic/USB recorder (audio/SampleRecorder.kt), decoded to a mono cache (audio/SampleStore.kt), played pitched + sliced by audio/SampleVoice.kt. Its editor (ui/toolbox/SamplerEditor.kt) shows a waveform with two draggable slice markers, start-octave + volume, and a two-octave audition keyboard. Each (lane, track) has both a synth and a sample voice; the channel instrument type selects which one a note triggers.
• Input: keyboard (nav + typing-piano + transport), gamepad (buttons + sticks + hat), MIDI in (note + CC, USB & BT), all via the unified router.
• MIDI-learn & gamepad rebind ✓: the Settings tab lists every bindable action with its current binding and a Learn button; arming it sets the router's learnMode, and the next unbound CC / gamepad button is written into MidiInput.ccBindings / GamepadInput.bindings (the handlers are now app singletons shared with Settings). Bindings are clearable per action.
• MIDI effects (Transposer / Arpeggiator / LFO) ✓: Transposer shifts notes by its semitone amount; the Arpeggiator octave-cycles a captured note (Up/Down/UpDown/Random, 1–4 octaves) at a tempo-synced division (AudioEngine.advanceArps); the LFO (applyLfos) modulates a chosen target parameter of another device around a captured centre (Sine/Tri/Saw/Square/S&H), with a target picker in its editor (ui/toolbox/LfoEditor.kt).
• USB / audio device selection ✓: Settings → Audio Devices lists the system's output and input devices; choosing one routes the engine's AudioTrack and the recorder's AudioRecord via setPreferredDevice (API 28+), so a USB DAC / USB audio interface can be used for playback and recording.
• Full MediaSession ✓: playback/PlaybackService is now a media3 MediaSessionService hosting a session that wraps the engine via EnginePlayer (SimpleBasePlayer). This gives lock-screen controls, Bluetooth/headset media buttons, and the media notification for free, mirroring the on-screen transport.
• Persistence: preset text I/O, a full-fidelity internal project text format (ProjectIo, keeps mixer/FX/instruments), AND desktop-compatible .sng read/write (io/SngFormat.kt) matching the reference sizzletracker line format (version/bpm/sig, block…endblock with roll/track/steps) — covered by passing JVM unit tests in app/src/test.
• Background playback foreground service with Play/Pause/Stop notification.
• Oboe / AAudio native output ✓ (default): src/main/cpp/native_audio.cpp opens a low-latency Oboe stream whose real-time callback pulls audio via JNI (audio/NativeAudioBridge.kt) from the SAME AudioEngine.fillBlock the AudioTrack loop uses — so both backends produce identical sound, only the delivery path differs. Oboe is the default engine; it falls back to the AudioTrack loop only if it fails to initialise (or the native lib is absent), surfacing a popup with the reason (there is no user toggle). Built via NDK r27 (ndkVersion in app/build.gradle.kts) + CMake + the com.google.oboe:oboe:1.10.0 prefab AAR. All packaged .so files are 16 KB page-aligned (Android 15 / Google Play requirement): NDK r27 aligns ELF load segments to 16 KB and ships a 16 KB-aligned libc++_shared.so, Oboe 1.10.0's lib is aligned, our CMakeLists.txt also passes -Wl,-z,max-page-size=16384, and AGP stores each .so at a 16 KB-aligned offset in the APK. (Verify with llvm-readelf -l <lib>.so → LOAD Align 0x4000.)
• SF2 / XI loader instrument ✓: audio/SoundFontLoader.kt extracts PCM from an .sf2 / .xi into the SampleStore, played back through the same SampleVoice path as the Sampler; edited via ui/toolbox/SoundFontEditor.kt (root note, volume, ADSR).
• Master recording ✓: audio/MasterRecorder.kt renders the stereo master bus (plus a configurable FX tail) to a WAV in a user-chosen folder (SAF).
• Stereo output & recording ✓: fillBlock renders interleaved stereo and the recorder captures two channels (voices/inserts are mono, so both channels carry the same sum).
• Responsive landscape layout ✓ (ui/Responsive.kt): nav rail, side-by-side screens, two-column mixer/editors, wider on-screen keyboards for tablets/handhelds.
• Arrangement niceties ✓: 256-bar canvas with viewport-culled drawing, per-lane mutes, and centre-follow scrolling.
• Per-bus soft-clip limiter ✓, sampler QoL (fire-on-tap, clear, normalize), theme import/export, and preset .zip bundle import/export.
• Sample-pack builder ✓: tools/samplepack-builder.html, a standalone offline page that assembles importable Sampler .zip bundles.

Remaining / optional

• Byte-exact desktop .sng compatibility. io/SngFormat.kt reads/writes the reference line format and round-trips musically; a field-for-field byte match with the CLI project layout is the remaining interchange nicety.
• Full C++ audio port. The DSP currently runs in Kotlin behind the Oboe callback (via JNI). Porting the hot path to C++ would remove JVM/GC from the real-time thread — a performance hardening step, not a functional gap.
• Route the Oboe backend through the chosen output device (Oboe setDeviceId), matching the AudioTrack path's device selection.

---

10. Testing suggestions

• Pure model/ classes (Scale.step, TimeSignature, Pattern.resize, ProjectIo round-trip, PresetIo round-trip) are plain Kotlin — cover them with JVM unit tests first; they need no device.
• For the engine, render a few blocks with a known pattern and assert the voice triggers on the expected sample offsets.