Publish a signed F-Droid build from the release so F-Droid can reproduce it and ship it with our key

This commit is contained in:
Jon Staab 2026-09-24 16:09:11 -07:00
parent a03b9b73b6
commit 0a606e7a8a
11 changed files with 305 additions and 62 deletions

View file

@ -67,17 +67,6 @@ jobs:
with:
node-version-file: .nvmrc
- name: Set up Java
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: 21
- name: Set up Android SDK
uses: android-actions/setup-android@v3
with:
packages: platform-tools
- name: Install dependencies
run: |
corepack enable
@ -87,4 +76,5 @@ jobs:
- name: Release
env:
GITEA_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITEA_PACKAGE_TOKEN: ${{ secrets.PACKAGE_TOKEN }}
run: pnpm release:ci --yes

View file

@ -213,7 +213,8 @@ step that fails stops the run and prints the command to pick up from there.
| `apk` | local | `assembleRelease` signed with the distribution key, renamed to the path in `zapstore.yaml` |
| `play` | local | `bundleRelease` signed with the upload key, uploaded to a Play track as a draft (or `PLAY_STATUS`) |
| `ios` | local | `cap build ios` to an archive and IPA, uploaded with `altool`, then attached to the App Store version with its release notes once processed |
| `fdroid` | ci | reruns F-Droid's own preparation and build against the tag in a throwaway worktree |
| `fdroid` | ci | builds the tag with F-Droid's recipe in their buildserver image and uploads the unsigned apk to the `flotilla-fdroid` package |
| `fdroid-sign` | local | waits for that apk, signs it with the distribution key, and uploads it beside it for F-Droid to verify its own build against |
| `desktop` | both | `package:desktop:*` for this OS: signed and notarized macOS from a Mac, Linux and Windows from Linux |
| `gitea` | both | creates a draft release from the changelog, attaches what this run built, and publishes it once every platform is there |
| `zapstore` | local | `zsp publish --skip-preview --quiet zapstore.yaml` |
@ -227,12 +228,12 @@ Release notes come from the `CHANGELOG.md` section matching `package.json`'s ver
store shows the same text. The APK and zapstore share one artifact, whose path lives in
`zapstore.yaml`.
F-Droid builds from the tag on its own servers, so the `fdroid` step uploads nothing. It runs
[their preparation and build](fdroid/README.md) against the tag in a throwaway git worktree, and if
that build breaks, the workflow stops before attaching the Linux and Windows packages, which keeps
the release a draft. Preparation patches source
with exact-match replacements, so it breaks quietly when the files it rewrites change. The step is
slow because it installs and builds from scratch.
F-Droid builds from the tag on its own servers and ships our apk instead of its own when the two
match, so it keeps our signature ([reproducible builds](fdroid/README.md)). The `fdroid` step makes
that apk the way F-Droid will, and if the build breaks, the workflow stops before attaching the
Linux and Windows packages, which keeps the release a draft. Preparation patches source with
exact-match replacements, so it breaks quietly when the files it rewrites change. The step is slow
because it installs and builds from scratch.
### Credentials
@ -241,7 +242,7 @@ along with how to get them.
| variable | what it is |
| --- | --- |
| `GITEA_TOKEN` | gitea access token with `write:repository`, from Settings → Applications |
| `GITEA_TOKEN` | gitea access token with `write:repository` and `write:package`, from Settings → Applications |
| `ANDROID_KEYSTORE_PATH`, `ANDROID_KEYSTORE_PASSWORD`, `ANDROID_KEYSTORE_ALIAS` | the key APKs outside the app stores are signed with; it can never change without breaking updates |
| `PLAY_KEYSTORE_PATH`, `PLAY_KEYSTORE_PASSWORD`, `PLAY_KEYSTORE_ALIAS` | the Play upload key |
| `PLAY_SERVICE_ACCOUNT` | path to a service account json with the Release manager role, from Play Console → Setup → API access |

View file

@ -43,11 +43,23 @@ and use its configured Gradle runner for `assembleFdroidRelease`.
[`metadata/social.flotilla.fdroid.yml`](metadata/social.flotilla.fdroid.yml) is the recipe to submit
to `fdroiddata`. The listing's text, icon, feature graphic and per-version changelogs come from
`fastlane/metadata/android/en-US/` at the tag F-Droid builds. Preparation installs dependencies before F-Droid's source scan, so the recipe
scan-ignores `node_modules`, which holds FLOSS build tools such as esbuild and sharp. The build
server's JDK is older than the 21 Capacitor needs, so the recipe installs it from Debian trixie,
along with Node from nodejs.org at a pinned checksum. None of that has been through `fdroid build`
yet.
`fastlane/metadata/android/en-US/` at the tag F-Droid builds. Preparation installs dependencies
before F-Droid's source scan, so the recipe scan-ignores `node_modules`, which holds FLOSS build
tools such as esbuild and sharp. The build server's JDK is older than the 21 Capacitor needs, so
the recipe installs it from Debian trixie, along with Node from nodejs.org at a pinned checksum.
## Reproducible builds
F-Droid rebuilds each tag, downloads the apk at the recipe's `Binaries` url, and ships that apk
instead of its own when copying its signature onto F-Droid's build verifies. So the F-Droid app
carries the distribution key, the one `AllowedAPKSigningKeys` names, and can never switch to
F-Droid's key.
The release workflow's `fdroid` step runs `scripts/fdroid/reproduce.sh` in F-Droid's
`buildserver-trixie` image, the way fdroiddata's own CI builds a recipe, against the recipe in
`metadata/` pointed at the tag. It uploads the unsigned apk to the `flotilla-fdroid` generic
package on gitea, and `pnpm release:local fdroid-sign` signs it and uploads the result beside it.
The recipe in `metadata/` is the one both builds read, so a change to it goes to `fdroiddata` too.
## Updates
@ -56,5 +68,4 @@ Stable releases use bare version tags such as `1.9.1`, and the recipe checks tag
`android/app/build.gradle` at that tag.
The initial `fdroiddata` submission must still review scanner exceptions for FLOSS
build tools, optional OpenRouter use for a possible `NonFreeNet` declaration, and
the final F-Droid signing certificate for Android App Links.
build tools and optional OpenRouter use for a possible `NonFreeNet` declaration.

View file

@ -2,6 +2,7 @@ Categories:
- Internet
License: MIT
AuthorName: Jon Staab
AuthorWebSite: https://flotilla.social
WebSite: https://app.flotilla.social
SourceCode: https://gitea.coracle.social/coracle/flotilla
IssueTracker: https://gitea.coracle.social/coracle/flotilla/issues
@ -11,6 +12,7 @@ AutoName: Flotilla
RepoType: git
Repo: https://gitea.coracle.social/coracle/flotilla.git
Binaries: https://gitea.coracle.social/api/packages/coracle/generic/flotilla-fdroid/%v/flotilla-fdroid-%v.apk
Builds:
- versionName: 1.11.1
@ -35,6 +37,8 @@ Builds:
- node_modules
build: ../../scripts/fdroid/build.sh
AllowedAPKSigningKeys: 6daf683e1ca83a4cd88573e9739e2aa944c85d56154e344230557cffed4ad78c
MaintainerNotes: |-
scripts/fdroid/prepare.sh removes Firebase, Google Services, Plausible and the ACINQ
native secp256k1 library, then installs dependencies, so it runs as prebuild ahead of
@ -42,6 +46,10 @@ MaintainerNotes: |-
Capacitor needs JDK 21, installed from trixie, and Node comes from nodejs.org at a
pinned checksum to match the lts/jod in .nvmrc.
Builds are reproducible: upstream's release workflow runs this recipe in the
buildserver-trixie image (scripts/fdroid/reproduce.sh) and publishes that apk, signed
with the distribution key, at Binaries.
AutoUpdateMode: Version
UpdateCheckMode: Tags ^[0-9]+\.[0-9]+\.[0-9]+$
UpdateCheckData: android/app/build.gradle|(?m)^\s*versionCode\s+(\d+)\s*$|.|(?m)^\s*versionName\s+"([^"]+)"\s*$

58
scripts/fdroid/reproduce.sh Executable file
View file

@ -0,0 +1,58 @@
#!/usr/bin/env bash
# Builds the tag the way fdroiddata's "fdroid build" CI job does, in the same buildserver image, so
# the apk is the one F-Droid will rebuild and compare against. Runs as root inside
# registry.gitlab.com/fdroid/fdroidserver:buildserver-trixie, with the recipe at /work/recipe.yml,
# and leaves the unsigned apk at /work/unsigned.apk.
set -euo pipefail
source /etc/profile.d/bsenv.sh
export ANDROID_HOME=/opt/android-sdk
apt-get update
apt-get -y dist-upgrade
sdkmanager "platform-tools" "build-tools;31.0.0"
rm -rf "$fdroidserver"
mkdir "$fdroidserver"
curl --silent https://gitlab.com/fdroid/fdroidserver/-/archive/master/fdroidserver-master.tar.gz |
tar -xz --directory="$fdroidserver" --strip-components=1
git -C "$home_vagrant/gradlew-fdroid" pull
apt-get install -y sudo openjdk-21-jdk-headless
update-alternatives --set java /usr/lib/jvm/java-21-openjdk-amd64/bin/java
# The recipe with its one build pointed at this tag, and without the published apk it would
# otherwise try to compare against, since this is the build that becomes it
mkdir -p "$home_vagrant/metadata" "$home_vagrant/build" "$home_vagrant/tmp" "$home_vagrant/unsigned"
python3 - <<'EOF'
import os
import yaml
recipe = yaml.safe_load(open("/work/recipe.yml"))
recipe.pop("Binaries", None)
recipe.pop("AllowedAPKSigningKeys", None)
build = recipe["Builds"][-1]
build.update(
versionName=os.environ["VERSION"],
versionCode=int(os.environ["VERSION_CODE"]),
commit=os.environ["COMMIT"],
)
recipe["Builds"] = [build]
recipe["CurrentVersion"] = os.environ["VERSION"]
recipe["CurrentVersionCode"] = int(os.environ["VERSION_CODE"])
with open(os.path.join(os.environ["home_vagrant"], "metadata/social.flotilla.fdroid.yml"), "w") as file:
yaml.safe_dump(recipe, file, sort_keys=False)
EOF
chown -R vagrant "$home_vagrant"
cd "$home_vagrant"
sudo --preserve-env --user vagrant \
env PATH="$fdroidserver:$PATH" \
env PYTHONPATH="$fdroidserver:$fdroidserver/examples" \
env PYTHONUNBUFFERED=true \
env HOME="$home_vagrant" \
fdroid build --verbose --test --refresh-scanner --on-server --no-tarball \
"social.flotilla.fdroid:$VERSION_CODE"
cp "$home_vagrant/tmp/social.flotilla.fdroid_$VERSION_CODE.apk" /work/unsigned.apk

View file

@ -0,0 +1,12 @@
import {name, repository, version} from "./context.mjs"
import {giteaPackage} from "./gitea.mjs"
// F-Droid rebuilds each tag and ships this apk, signed with our key, only if its own build matches
export const fdroidPackage = token =>
giteaPackage({repository, token, name: `${name}-fdroid`, version})
export const unsignedApk = `${name}-fdroid-${version}-unsigned.apk`
export const signedApk = `${name}-fdroid-${version}.apk`
// CI publishes packages with its own token, since the job's gitea token can't
export const packageToken = process.env.GITEA_PACKAGE_TOKEN || process.env.GITEA_TOKEN

View file

@ -61,3 +61,48 @@ export const gitea = ({repository, token}) => {
},
}
}
// Gitea's generic package registry, for files that shouldn't sit on the release itself
export const giteaPackage = ({repository, token, name, version}) => {
const [, owner] = repository.pathname.split("/")
const url = file =>
`${repository.origin}/api/packages/${owner}/generic/${name}/${version}/${encodeURIComponent(file)}`
const request = async (method, file, body) => {
const response = await fetch(url(file), {
method,
headers: {Authorization: `token ${token}`},
body,
})
if (response.status === 404 && method !== "PUT") {
return undefined
}
if (!response.ok) {
throw new Error(
`${method} ${url(file)} responded ${response.status}: ${await response.text()}`,
)
}
return response
}
return {
url,
download: async file => {
const response = await request("GET", file)
return response && new Uint8Array(await response.arrayBuffer())
},
// A package file can't be overwritten, so a rebuild of the same version replaces it
upload: async (file, data) => {
await request("DELETE", file)
await request("PUT", file, data)
return url(file)
},
}
}

View file

@ -3,10 +3,11 @@
import {release} from "./lib/pipeline.mjs"
import apk from "./steps/apk.mjs"
import desktop from "./steps/desktop.mjs"
import fdroidSign from "./steps/fdroid-sign.mjs"
import gitea from "./steps/gitea.mjs"
import ios from "./steps/ios.mjs"
import play from "./steps/play.mjs"
import web from "./steps/web.mjs"
import zapstore from "./steps/zapstore.mjs"
await release("pnpm release:local", [web, apk, play, ios, desktop, gitea, zapstore])
await release("pnpm release:local", [web, apk, play, ios, desktop, fdroidSign, gitea, zapstore])

View file

@ -0,0 +1,93 @@
import {existsSync, readFileSync, readdirSync} from "node:fs"
import {mkdtemp, readFile, rm, writeFile} from "node:fs/promises"
import {tmpdir} from "node:os"
import {join} from "node:path"
import {HOUR, MINUTE, ago, ms, now, sleep} from "@welshman/lib"
import {keystoreEnv} from "../lib/android.mjs"
import {missingEnv, root} from "../lib/context.mjs"
import {fdroidPackage, packageToken, signedApk, unsignedApk} from "../lib/fdroid.mjs"
import {output, run} from "../lib/shell.mjs"
const sdk =
process.env.ANDROID_HOME ??
readFileSync(join(root, "android/local.properties"), "utf-8").match(/^sdk\.dir=(.+)$/m)?.[1]
const apksigner = () => {
const tools = join(sdk, "build-tools")
const [latest] = readdirSync(tools).sort((a, b) => b.localeCompare(a, "en", {numeric: true}))
return join(tools, latest, "apksigner")
}
export default {
name: "fdroid-sign",
title: "Sign the release workflow's F-Droid build with the distribution key",
missing: () => [
...missingEnv("ANDROID_KEYSTORE_PATH", "ANDROID_KEYSTORE_PASSWORD", "ANDROID_KEYSTORE_ALIAS"),
...(packageToken ? [] : ["GITEA_TOKEN"]),
...(sdk && existsSync(join(sdk, "build-tools")) ? [] : ["the Android SDK's build-tools"]),
],
setup: [
"Signs with the same ANDROID_KEYSTORE_* key as the apk step. GITEA_TOKEN needs the",
"write:package scope. apksigner comes from the Android SDK in ANDROID_HOME, or the sdk.dir",
"Android Studio writes to android/local.properties.",
],
run: async () => {
const api = fdroidPackage(packageToken)
const started = now()
let unsigned = await api.download(unsignedApk)
while (!unsigned) {
if (started < ago(2 * HOUR)) {
throw new Error(
`The release workflow hasn't uploaded ${api.url(unsignedApk)} after two hours`,
)
}
console.log(`Waiting for the release workflow to upload ${unsignedApk}`)
await sleep(ms(MINUTE))
unsigned = await api.download(unsignedApk)
}
const work = await mkdtemp(join(tmpdir(), "flotilla-fdroid-sign-"))
try {
const signing = keystoreEnv("ANDROID")
await writeFile(join(work, unsignedApk), unsigned)
// F-Droid copies this signature onto its own build, so nothing but the signature may change
await run(
apksigner(),
[
"sign",
"--ks",
signing.ANDROID_KEYSTORE_PATH,
"--ks-key-alias",
signing.ANDROID_KEYSTORE_ALIAS,
"--ks-pass",
"env:ANDROID_KEYSTORE_PASSWORD",
"--key-pass",
"env:ANDROID_KEYSTORE_ALIAS_PASSWORD",
"--alignment-preserved",
"--out",
join(work, signedApk),
join(work, unsignedApk),
],
{env: {...process.env, ...signing}},
)
const recipe = readFileSync(join(root, "fdroid/metadata/social.flotilla.fdroid.yml"), "utf-8")
const allowed = recipe.match(/^AllowedAPKSigningKeys: (\w+)$/m)[1]
const certificates = output(apksigner(), ["verify", "--print-certs", join(work, signedApk)])
if (!certificates.includes(`certificate SHA-256 digest: ${allowed}`)) {
throw new Error(`${signedApk} isn't signed with the key the recipe allows, ${allowed}`)
}
console.log(await api.upload(signedApk, await readFile(join(work, signedApk))))
} finally {
await rm(work, {recursive: true, force: true})
}
},
}

View file

@ -1,53 +1,65 @@
import {existsSync} from "node:fs"
import {mkdtemp, rm} from "node:fs/promises"
import {cp, mkdtemp, readFile, rm} from "node:fs/promises"
import {tmpdir} from "node:os"
import {join} from "node:path"
import {git, root, version} from "../lib/context.mjs"
import {run} from "../lib/shell.mjs"
import {git, root, version, versionCode} from "../lib/context.mjs"
import {fdroidPackage, packageToken, unsignedApk} from "../lib/fdroid.mjs"
import {installed, run} from "../lib/shell.mjs"
const docker = process.env.DOCKER || "docker"
export default {
name: "fdroid",
title: "Rebuild the tag the way F-Droid will",
missing: () =>
git("cat-file", "-e", `${version}:scripts/fdroid/prepare.sh`) === undefined
title: "Build the tag the way F-Droid will, and upload the unsigned apk",
missing: () => [
...(git("cat-file", "-e", `${version}:scripts/fdroid/reproduce.sh`) === undefined
? [`F-Droid support in the ${version} tag`]
: [],
: []),
...(installed(docker) ? [] : [docker]),
...(packageToken ? [] : ["GITEA_PACKAGE_TOKEN or GITEA_TOKEN"]),
],
setup: [
`The ${version} tag is older than scripts/fdroid/, so there is nothing for F-Droid to build`,
"from it. Name the steps you do want, or release a tag that has it.",
`The ${version} tag is older than scripts/fdroid/reproduce.sh, so it can't be built the way`,
"F-Droid builds it. The build runs in F-Droid's buildserver image, which needs docker. The",
"token needs the write:package scope.",
],
run: async () => {
const parent = await mkdtemp(join(tmpdir(), "flotilla-fdroid-"))
const checkout = join(parent, "flotilla")
const work = await mkdtemp(join(tmpdir(), "flotilla-fdroid-"))
const container = `flotilla-fdroid-${process.pid}`
// Preparation rewrites source and dependencies in place, so it only runs against a checkout
// that can be thrown away
// Copied in and out rather than mounted, like the desktop packages, for runners that share
// the host's docker socket
try {
await run("git", ["worktree", "add", "--detach", checkout, version], {cwd: root})
await run("./scripts/fdroid/prepare.sh", [], {cwd: checkout})
await run("./scripts/fdroid/build.sh", [], {cwd: checkout})
await cp(join(root, "scripts/fdroid/reproduce.sh"), join(work, "reproduce.sh"))
await cp(join(root, "fdroid/metadata/social.flotilla.fdroid.yml"), join(work, "recipe.yml"))
await run(docker, [
"create",
"--name",
container,
"--platform",
"linux/amd64",
"--env",
`VERSION=${version}`,
"--env",
`VERSION_CODE=${versionCode}`,
"--env",
`COMMIT=${git("rev-parse", `${version}^{commit}`)}`,
"registry.gitlab.com/fdroid/fdroidserver:buildserver-trixie",
"bash",
"/work/reproduce.sh",
])
await run(docker, ["cp", `${work}/.`, `${container}:/work`])
await run(docker, ["start", "--attach", container])
await run(docker, ["cp", `${container}:/work/unsigned.apk`, join(work, unsignedApk)])
// F-Droid signs its own builds, so keep the distribution key out of gradle's environment
const env = {...process.env}
delete env.ANDROID_KEYSTORE_PATH
await run("./gradlew", ["--no-daemon", "assembleFdroidRelease"], {
cwd: join(checkout, "android"),
env,
})
const built = join(
checkout,
"android/app/build/outputs/apk/fdroid/release/app-fdroid-release-unsigned.apk",
const url = await fdroidPackage(packageToken).upload(
unsignedApk,
await readFile(join(work, unsignedApk)),
)
if (!existsSync(built)) {
throw new Error(`the F-Droid build produced no apk at ${built}`)
}
console.log(url)
} finally {
await rm(parent, {recursive: true, force: true})
await run("git", ["worktree", "prune"], {cwd: root})
await run(docker, ["rm", "--force", container], {stdio: "ignore"}).catch(() => {})
await rm(work, {recursive: true, force: true})
}
},
}

View file

@ -12,5 +12,17 @@
"8C:EE:37:F9:8A:08:02:A7:BB:55:2B:64:E5:A5:93:D8:58:73:14:26:66:71:DD:B0:4F:AB:9D:D5:4C:DF:FB:F7"
]
}
},
{
"relation": [
"delegate_permission/common.handle_all_urls"
],
"target": {
"namespace": "android_app",
"package_name": "social.flotilla.fdroid",
"sha256_cert_fingerprints": [
"6D:AF:68:3E:1C:A8:3A:4C:D8:85:73:E9:73:9E:2A:A9:44:C8:5D:56:15:4E:34:42:30:55:7C:FF:ED:4A:D7:8C"
]
}
}
]