diskOS installer: initial public beta

Flashes the diskOS custom UI onto the FiiO Snowsky Disc over Ingenic mask-ROM USB,
building the image from your own stock firmware. Runs from source via install.sh.
This commit is contained in:
b0hemia
2026-08-26 15:26:14 +10:00
commit e0bc4785e9
109 changed files with 12625 additions and 0 deletions
+52
View File
@@ -0,0 +1,52 @@
# Vendored native tools - portability requirements
The installer bundles native binaries under `vendor/<os>-<arch>/` and calls them
via subprocess. For a distributable build these MUST be portable across the user's
machines - not just copies of the build host's dynamically-linked binaries.
| tool | why bundled | portability requirement |
|---|---|---|
| `usbboot` | Ingenic mask-ROM USB loader (the flasher) | Build **static** against libusb compiled `--disable-udev` (drops libudev + libcap; libusb falls back to sysfs enumeration). `usbboot` has **no direct udev symbols** - they were only transitive via libusb - so this yields a binary needing only libc. Prefer full-static (musl) or, at minimum, `$ORIGIN/lib` rpath + a bundled `libusb-1.0.so` in `vendor/<tag>/lib/`. Do **not** ship arbitrary host glibc `.so`s or rely on `LD_LIBRARY_PATH` as the primary strategy. |
| `mksquashfs`, `unsquashfs` | build/extract the rootfs image | Ship **static** squashfs-tools (with lzo support - the stock image uses `-comp lzo`). |
| `my_write5_dram.bin` | the DRAM NAND writer run on-device | Architecture-neutral blob (runs on the device, not the host). Copy as-is. |
| `disc_spl_lpddr3.bin` | X2000 SPL | Same - device blob, copy as-is. |
## Status
- `vendor/linux-x86_64/` holds **fully static** `usbboot`, `mksquashfs`, and
`unsquashfs`, produced by `build/build-usbboot-static.sh` and
`build/build-squashfs-static.sh` (each verifies the binary is `statically linked`;
the squashfs build additionally runs an LZO round-trip self-test). They are static
against glibc (built with `gcc -static`), so no runtime `.so` is needed; note the
static glibc/liblzo2/libusb licensing in `../NOTICE.md` and the corresponding source
in `../corresponding-source/`. `my_write5_dram.bin` and `disc_spl_lpddr3.bin` are
device blobs, copied as-is.
- `vendor/macos-*/` is produced on a Mac by **`vendor/setup-macos.sh`** (compiles
`usbboot` from `src/usbboot/usbboot.c` against Homebrew libusb, gathers
lzo-capable squashfs-tools, copies the device blobs, and runs `dylibbundler` so
every tool is self-contained under `vendor/macos-<arch>/lib/` - no Homebrew at
runtime). `usbboot.c` is a single libusb-only file (no udev / no Linux-isms), so
it builds on macOS as-is. Then `build/build-macos.sh` packages the app.
## macOS: Intel + Apple Silicon coverage
`usbboot.c` and pyusb are portable; only the *build* is per-arch. Options:
1. **Native per-arch (most robust):** run `build/build-macos.sh` on an Intel Mac
and on an Apple Silicon Mac → two binaries. Recommended.
2. **Single file via Rosetta:** build **x86_64** only (on an Intel Mac, or with an
x86_64 Homebrew under `arch -x86_64`). The x86_64 app runs natively on Intel and
under Rosetta 2 on M-series (macOS offers to install Rosetta on first run).
One file covers both, at a small speed cost - fine for a flasher.
3. **Universal2 (advanced):** build both arches, `lipo` the vendor tools into fat
binaries, and set `target_arch='universal2'` in the spec with a universal2
Python. One native file for both, most work.
## Building portable usbboot (sketch)
1. Get the Ingenic `usbboot` source (the X2000 burn tool).
2. Build libusb static: `./configure --disable-udev --enable-static --disable-shared`
→ `libusb-1.0.a`.
3. Link `usbboot` against the static libusb; confirm with `ldd` that libudev/libcap
are gone and only libc (or nothing, if fully static) remains.
4. Verify it still enters mask-ROM and flashes on a real unit before shipping.
+40
View File
@@ -0,0 +1,40 @@
#!/bin/bash
# build-macos.sh - build the standalone diskos-installer app on macOS.
# RUN ON A MAC. Produces build/dist/diskos-installer for this Mac's arch.
#
# Intel + Apple Silicon coverage (pick one):
# * Simplest single file: build x86_64 (below, on an Intel Mac OR via an
# x86_64 Homebrew under Rosetta) - it runs natively on Intel and under
# Rosetta 2 on M-series. Needs Rosetta on M-series (macOS offers to install it).
# * Best native: build on each arch -> two binaries (diskos-installer-arm64 /
# -x86_64). Run this script on an Intel Mac and an Apple Silicon Mac.
# * Universal2: build both arches, lipo the vendor tools + use PyInstaller
# target_arch=universal2 (advanced; see build/README-vendor.md).
set -euo pipefail
cd "$(dirname "$0")/.." # installer/
[ "$(uname)" = "Darwin" ] || { echo "run this on macOS." >&2; exit 2; }
if [ "${1:-}" != "--skip-setup" ]; then
echo ">> populating vendor/ for this Mac"
bash vendor/setup-macos.sh
fi
VENV=build/venv
[ -d "$VENV" ] || python3 -m venv "$VENV"
"$VENV/bin/pip" install --quiet --upgrade pip pyinstaller pyusb pycryptodome
"$VENV/bin/pyinstaller" --clean --noconfirm \
--distpath build/dist --workpath build/work \
build/diskos-installer.spec
APP=build/dist/diskos-installer
# ad-hoc sign so it runs locally without a Developer ID (enthusiast tool).
# For distribution you'd sign + notarize with a real identity.
codesign --force --deep --sign - "$APP" 2>/dev/null || \
echo " (codesign ad-hoc skipped/failed - 'xattr -dr com.apple.quarantine $APP' if Gatekeeper blocks it)"
echo
echo "Built: $APP ($(uname -m))"
echo "Smoke test: $APP doctor"
echo "If Gatekeeper blocks it: xattr -dr com.apple.quarantine $APP"
+45
View File
@@ -0,0 +1,45 @@
#!/bin/bash
# build-squashfs-static.sh - build portable, fully-static mksquashfs + unsquashfs
# for Linux, with the compressors the installer actually uses.
#
# The installer repacks the rootfs with `-comp lzo -b 131072` and unpacks the
# stock (lzo) rootfs, so LZO is mandatory; gzip + xz are included for reading
# other firmware. Built static (liblzo2.a/libz.a/liblzma.a) so there are no
# liblzo2/libz/liblzma/glibc-version runtime deps on the user's machine.
#
# Output: installer/vendor/linux-x86_64/{mksquashfs,unsquashfs} (static, stripped).
# A cosmetic getpwuid NSS warning at link time is expected (owner-name display in
# the summary only; packing uses numeric uid/gid) and does not affect output.
set -euo pipefail
cd "$(dirname "$0")/.." # installer/
OUTDIR="$(pwd)/vendor/linux-x86_64"
WORK=$(mktemp -d); trap 'rm -rf "$WORK"' EXIT
cd "$WORK"
echo "==> fetching squashfs-tools source"
apt-get source squashfs-tools >/dev/null 2>&1 || {
echo "ERROR: 'apt-get source squashfs-tools' failed - enable deb-src or fetch squashfs-tools 4.x." >&2
exit 3; }
cd squashfs-tools-*/squashfs-tools
echo "==> building static (gzip+lzo+xz)"
make clean >/dev/null 2>&1 || true
make GZIP_SUPPORT=1 LZO_SUPPORT=1 XZ_SUPPORT=1 \
EXTRA_CFLAGS="-static" LDFLAGS="-static" -j"$(nproc)" mksquashfs unsquashfs >/dev/null
strip mksquashfs unsquashfs
for b in mksquashfs unsquashfs; do
file "$b" | grep -q "statically linked" || { echo "ERROR: $b not static" >&2; exit 5; }
done
# prove the installer's exact LZO path round-trips before installing
t="$WORK/smoke"; mkdir -p "$t/src"; echo diskos > "$t/src/f"
./mksquashfs "$t/src" "$t/o.sqsh" -comp lzo -b 131072 -noappend >/dev/null 2>&1
./unsquashfs -d "$t/u" "$t/o.sqsh" >/dev/null 2>&1
diff -r "$t/src" "$t/u" >/dev/null || { echo "ERROR: LZO round-trip failed" >&2; exit 6; }
echo "==> LZO round-trip OK"
for b in mksquashfs unsquashfs; do
install -m 0755 "$b" "$OUTDIR/$b"
echo "==> installed $OUTDIR/$b ($(sha256sum "$OUTDIR/$b" | cut -d' ' -f1))"
done
+48
View File
@@ -0,0 +1,48 @@
#!/bin/bash
# build-usbboot-static.sh - build a portable, fully-static `usbboot` for Linux.
#
# The distro's libusb-1.0.a is built WITH udev, so linking against it drags in
# libudev + libcap dynamically - which defeats portability (missing/old libudev
# on other distros). This rebuilds libusb with --disable-udev --enable-static so
# usbboot links fully static: no libusb.so, no libudev, no glibc-version pin.
# udev is only needed for hotplug events; usbboot does a one-shot vid/pid open,
# which uses libusb's sysfs backend and works without udev.
#
# Output: installer/vendor/linux-x86_64/usbboot (statically linked, stripped).
# Run on an x86_64 Linux host with build-essential + apt source access.
set -euo pipefail
cd "$(dirname "$0")/.." # installer/
ROOT=$(pwd)
SRC="$ROOT/src/usbboot/usbboot.c"
OUT="$ROOT/vendor/linux-x86_64/usbboot"
[ -f "$SRC" ] || { echo "ERROR: $SRC missing" >&2; exit 2; }
WORK=$(mktemp -d)
trap 'rm -rf "$WORK"' EXIT
cd "$WORK"
echo "==> fetching libusb source"
apt-get source libusb-1.0 >/dev/null 2>&1 || {
echo "ERROR: 'apt-get source libusb-1.0' failed - enable deb-src or fetch libusb 1.0.x manually." >&2
exit 3; }
cd libusb-1.0-*/
echo "==> configuring libusb (static, no udev)"
./configure --disable-udev --enable-static --disable-shared \
--disable-examples-build --disable-tests-build CFLAGS="-O2" >/dev/null
echo "==> building libusb"
make -j"$(nproc)" >/dev/null
LIBA=$(find "$PWD" -name libusb-1.0.a | head -1)
[ -f "$LIBA" ] || { echo "ERROR: libusb-1.0.a not produced" >&2; exit 4; }
INC=$(dirname "$LIBA")/../.. ; INC="$PWD/libusb"
echo "==> linking usbboot (fully static)"
gcc -O2 -std=c99 -I"$INC" -static -o usbboot "$SRC" "$LIBA" -lpthread
strip usbboot
file usbboot | grep -q "statically linked" || { echo "ERROR: not static" >&2; exit 5; }
install -m 0755 usbboot "$OUT"
echo "==> installed: $OUT"
echo " sha256: $(sha256sum "$OUT" | cut -d' ' -f1)"
echo " $(file -b "$OUT")"
echo "NOTE: confirm a mask-ROM GET_CPU_INFO handshake on a real device before shipping."
Executable
+35
View File
@@ -0,0 +1,35 @@
#!/bin/bash
# build.sh - produce the standalone `diskos-installer` executable for THIS host
# (Linux or macOS). PyInstaller can't cross-build, so run this on each target OS.
#
# Prereqs: python3. Everything else is created in a local venv; nothing is
# installed system-wide.
#
# Before building, populate vendor/<os>-<arch>/ with the native tools for this
# host (see build/README-vendor.md): usbboot, mksquashfs, unsquashfs,
# my_write5_dram.bin, disc_spl_lpddr3.bin (+ lib/ for any bundled .so/.dylib).
set -euo pipefail
cd "$(dirname "$0")/.." # installer/
VENV=build/venv
[ -d "$VENV" ] || python3 -m venv "$VENV"
"$VENV/bin/pip" install --quiet --upgrade pip pyinstaller pyusb pycryptodome
# sanity: the vendor dir for this host must exist
tag=$("$VENV/bin/python" - <<'PY'
import platform
s=platform.system().lower(); o={"darwin":"macos","linux":"linux"}.get(s,s)
m=platform.machine().lower()
a={"x86_64":"x86_64","amd64":"x86_64","arm64":"arm64","aarch64":"arm64"}.get(m,m)
print(f"{o}-{a}")
PY
)
[ -d "vendor/$tag" ] || { echo "ERROR: vendor/$tag missing - add native tools for this host first." >&2; exit 2; }
"$VENV/bin/pyinstaller" --clean --noconfirm \
--distpath build/dist --workpath build/work \
build/diskos-installer.spec
echo
echo "Built: build/dist/diskos-installer ($(du -h build/dist/diskos-installer | cut -f1))"
echo "Smoke test: build/dist/diskos-installer doctor"
+70
View File
@@ -0,0 +1,70 @@
# PyInstaller spec - one self-contained diskos-installer executable.
# Bundles the native flashing tools (vendor/<os>-<arch>/) and the payload
# (mq_ui + on-device init scripts) as data, plus pyusb. The result needs no
# system Python and no system packages: "remove the tool" = delete this file.
import os
ROOT = os.path.abspath(os.path.join(os.getcwd())) # run from installer/
# host tag for the vendor dir to bundle (build on each target OS)
import platform
_sys = platform.system().lower()
_os = {"darwin": "macos", "linux": "linux"}.get(_sys, _sys)
_arch = {"x86_64": "x86_64", "amd64": "x86_64", "arm64": "arm64",
"aarch64": "arm64"}.get(platform.machine().lower(), platform.machine().lower())
TAG = f"{_os}-{_arch}"
# files under vendor/ that must NOT be bundled into the app (dev-only backups, notes)
def _skip_vendor(fn):
return fn.endswith(".bak") or fn.endswith(".dynamic.bak") or fn in ("README.md",)
datas = []
vend = os.path.join(ROOT, "vendor", TAG)
if os.path.isdir(vend):
for fn in os.listdir(vend):
if _skip_vendor(fn) or os.path.isdir(os.path.join(vend, fn)):
continue
datas.append((os.path.join(vend, fn), f"vendor/{TAG}"))
libdir = os.path.join(vend, "lib")
if os.path.isdir(libdir):
for fn in os.listdir(libdir):
datas.append((os.path.join(libdir, fn), f"vendor/{TAG}/lib"))
payload = os.path.join(ROOT, "payload")
if os.path.isdir(payload):
for fn in os.listdir(payload):
datas.append((os.path.join(payload, fn), "payload"))
# FAIL the build (don't ship an unusable artifact) if any required native tool or
# payload for this host is missing.
_required_native = ["usbboot", "mksquashfs", "unsquashfs",
"my_write5_dram.bin", "disc_spl_lpddr3.bin"]
_required_payload = ["mq_ui", "S97diskos_install", "S99usbserial", "diskos-debug.sh", "dropbearmulti"]
_missing = [n for n in _required_native if not os.path.exists(os.path.join(vend, n))]
_missing += [f"payload/{n}" for n in _required_payload if not os.path.exists(os.path.join(payload, n))]
if _missing:
raise SystemExit(f"REFUSING to build: missing bundled files for {TAG}: {_missing}. "
f"Populate vendor/{TAG}/ and payload/ first.")
# pycryptodome loads C submodules dynamically; collect them all so the frozen
# app can AES-decrypt the FiiO OTA chunks.
from PyInstaller.utils.hooks import collect_submodules
hidden = ["usb", "usb.core", "usb.backend.libusb1"] + collect_submodules("Crypto")
a = Analysis(
[os.path.join(ROOT, "diskos_installer", "__main__.py")],
pathex=[ROOT],
binaries=[],
datas=datas,
hiddenimports=hidden,
hookspath=[],
runtime_hooks=[],
excludes=[],
noarchive=False,
)
pyz = PYZ(a.pure)
# onefile: pass binaries + datas straight into EXE with no COLLECT step.
exe = EXE(
pyz, a.scripts, a.binaries, a.datas, [],
name="diskos-installer",
debug=False, bootloader_ignore_signals=False, strip=False, upx=False,
console=True,
)
+114
View File
@@ -0,0 +1,114 @@
#!/bin/bash
# stage-release.sh - assemble the release tarball(s) into build/release/ and generate SHA256SUMS.
#
# The release ships as SOURCE (run with your own Python via ./install.sh) plus the native flash
# tools for the target platform - NOT a bundled onefile binary. Each tarball is self-contained:
# extract it, run ./install.sh once, then ./diskos-installer.
#
# Usage:
# build/stage-release.sh [TAG ...] # default TAG: the host's (e.g. linux-x86_64)
# Build the native tools for each TAG first (Linux: build/build-*-static.sh; macOS: build-macos.sh).
set -euo pipefail
cd "$(dirname "$0")/.." # installer/
ROOT=$(pwd)
REL="$ROOT/build/release"
mkdir -p "$REL"
# default to the host platform tag if none given
if [ "$#" -eq 0 ]; then
_os=$(uname -s | tr '[:upper:]' '[:lower:]'); case "$_os" in darwin) _os=macos;; esac
_arch=$(uname -m); case "$_arch" in amd64) _arch=x86_64;; aarch64|arm64) _arch=arm64;; esac
set -- "${_os}-${_arch}"
fi
# files/dirs that must NEVER go in a public tarball (secrets, FiiO-derived images, build scratch,
# internal docs). Mirrors .gitignore; kept here so staging works even outside a git checkout.
EXCLUer=(
--exclude='./.git' --exclude='./.venv' --exclude='./build/venv' --exclude='./build/work'
--exclude='./build/dist' --exclude='./build/release' --exclude='*/__pycache__/*'
--exclude='__pycache__' --exclude='*.pyc' --exclude='*.pyo' --exclude='*.bak'
--exclude='./signing' --exclude='*.pem' --exclude='*.key'
--exclude='./diskos_dev*.bin' --exclude='./diskos_public*.bin' --exclude='*_recovery.bin'
--exclude='*.squashfs' --exclude='*.sqfs' --exclude='./_retired_unlicensed'
--exclude='./flash/qual_lpddr3.sh' --exclude='./flash/scan_check.sh'
--exclude='./flash/my_write5_scan_dram.bin' --exclude='./flash/usbboot'
--exclude='./mq_ui' --exclude='./S97diskos_install' --exclude='./S99usbserial'
--exclude='./INSTALL.md' --exclude='./mkdiskos.sh' --exclude='./flash/flash_diskos.sh'
--exclude='./flash/extract_stock_rootfs.sh'
--exclude='./PUBLICATION_PLAN.md' --exclude='./BETA_CHECKLIST.md' --exclude='./RELEASE.md'
--exclude='./RELEASE_READINESS.md' --exclude='./UPDATE2_DRAFT.md' --exclude='./UPDATE_ARCHITECTURE.md'
--exclude='./INSTALL_v209_legacy.md' --exclude='./SHA256SUMS' --exclude='*.log' --exclude='*.tmp'
--exclude='.DS_Store'
)
# native tools + payload that MUST be present for a TAG (fail rather than ship an unusable tarball)
REQ_VENDOR=(usbboot mksquashfs unsquashfs my_write5_dram.bin disc_spl_lpddr3.bin)
REQ_PAYLOAD=(mq_ui S97diskos_install S99usbserial diskos-debug.sh dropbearmulti)
made=()
for TAG in "$@"; do
echo "== staging $TAG =="
miss=()
for f in "${REQ_VENDOR[@]}"; do [ -f "vendor/$TAG/$f" ] || miss+=("vendor/$TAG/$f"); done
for f in "${REQ_PAYLOAD[@]}"; do [ -f "payload/$f" ] || miss+=("payload/$f"); done
if [ "${#miss[@]}" -gt 0 ]; then
echo " SKIP $TAG - missing: ${miss[*]}" >&2
echo " (build the native tools for $TAG first)" >&2
continue
fi
# Stage into a temp dir named diskos-installer/, keeping ONLY this TAG's vendor dir, then tar it.
stage=$(mktemp -d)
dest="$stage/diskos-installer"
mkdir -p "$dest"
tar -c "${EXCLUer[@]}" --exclude='./vendor/*' -C "$ROOT" . | tar -x -C "$dest"
mkdir -p "$dest/vendor/$TAG"
cp -a "vendor/$TAG/." "$dest/vendor/$TAG/"
rm -f "$dest/vendor/$TAG/"*.bak 2>/dev/null || true
# macOS build helper that build/build-macos.sh calls - ship it so that path isn't broken
[ -f vendor/setup-macos.sh ] && cp -a vendor/setup-macos.sh "$dest/vendor/setup-macos.sh"
out="$REL/diskos-installer-$TAG.tar.gz"
tar -czf "$out" -C "$stage" diskos-installer
rm -rf "$stage"
echo " -> $out"
made+=("diskos-installer-$TAG.tar.gz")
done
[ "${#made[@]}" -gt 0 ] || { echo "nothing staged." >&2; exit 1; }
echo "== generating SHA256SUMS =="
( cd "$REL" && sha256sum "${made[@]}" > SHA256SUMS ) && cat "$REL/SHA256SUMS"
NOTES="$REL/RELEASE_NOTES.md"
[ -f "$NOTES" ] || cat > "$NOTES" <<'EOF'
# diskOS installer - release
Installer for diskOS on the FiiO Snowsky Disc. Runs from source with your own Python (GUI + CLI). It
builds a diskOS image *from your own official FiiO firmware zip* (FiiO's rootfs is never
redistributed), flashes over mask-ROM with a bad-block-aware verify-every-block writer, and keeps a
checksum-verified stock image so **restore/remove is one action**.
## What's in the tarball
Source + the native flash tools for the platform. **No bundled Python** - you run it with your own.
## Setup
```
tar -xzf diskos-installer-<platform>.tar.gz
cd diskos-installer
./install.sh # builds a local .venv, installs pyusb + pycryptodome
./diskos-installer doctor
```
Needs Python 3.8+. The GUI additionally needs Tk (`apt install python3-tk`); device detection needs
`libusb-1.0` (`apt install libusb-1.0-0`). `install.sh` checks for both and tells you what to install.
## Install / restore / remove
See the bundled `README.md`. Requires putting the device in mask-ROM (power off, hold Vol-Down, plug
USB). ~60-90 min; normally recoverable via mask-ROM, but not guaranteed.
## Honest status
Enthusiast flasher. Linux tested end-to-end on real hardware (V2.09 + V2.28). macOS build validation
in progress. Not affiliated with FiiO.
EOF
echo " -> $NOTES"
echo "Done. Attach $REL/* to the GitHub Release."