Our apps download AI models from Hugging Face. Downloading a model to your device is redistribution, which means the model’s licence travels with it and we have to show you the right one. We got that wrong twice, in opposite directions, and the second time is the interesting one.
Round one: right by accident, then wrong on purpose
The catalogue declared every model Apache-2.0. For most of them that was correct. For Google’s Gemma it was not — Gemma generations 1 to 3 ship under Google’s own Gemma Terms of Use, which carry use-based restrictions that travel with the weights and apply to what the model produces.
The wrong value got there the way wrong values usually do: the field above it said Apache-2.0, and so did the one above that. Nobody decided it. It was copied.
We fixed the declaration and, more importantly, changed the test. Instead of asserting that each model has a non-blank licence — which is exactly the check that lets a plausible wrong string through — the test now pins the licence per model ID in a map. Adding a model to the catalogue without deciding what it ships under is a red build.
Round two: the fix was the bug
Months later the catalogue was rewritten for a new model format. The Gemma entry came back as Apache-2.0, the pinning test caught it, and the reviewer did the responsible-looking thing: restored “Gemma Terms of Use” and filed a to-do to vendor the verbatim terms into the app so they would be readable offline like every other licence.
That reviewer had no network access. They reasoned by analogy from Gemma 1–3, which was sound reasoning and produced a wrong answer.
Because the model in question is Gemma 4, and Gemma 4 is published under the plain Apache
License 2.0. The Gemma Terms’ own appendix lists Gemma 1, 2, 3, 3n and the variants — and
excludes Gemma 4, pointing instead at a separate licence page that turns out to be Apache-2.0
verbatim. Both Hugging Face cards for the exact revision we download say license: apache-2.0.
So the first fix was correct, the “regression” was not a regression, and the careful correction introduced the error.
What actually stopped it
Not diligence. Both engineers were being diligent; one of them was diligent in the wrong direction with no way to check.
What stopped it was that the value could not be changed without the test forcing the question. The pinning map is a single place where a licence string has to be typed deliberately, next to a comment recording where the value came from. When the third session arrived — this one, with a network — the test failure pointed straight at the one line that needed a source, and the source was two clicks away.
The comment in that map now reads, in part: change this line only with the licence page in front of you. It names the appendix, the licence link and the date it was checked.
The general shape
A guard that asks “is this field filled in?” catches an empty field. It does not catch a confidently wrong field, and confidently wrong is the failure mode that actually ships.
The guards worth writing are the ones that make a claim unrepeatable without evidence:
- Licences pinned per model, not asserted non-blank.
- The open-source screen checked against the packaged APK — we unzip the artefact and fail the build if it ships a native library no licence entry attributes, or attributes one it does not ship. Both directions, because over-claiming on a compliance screen is its own lie.
- Counts interpolated from the list they describe, never typed. We once told users an app shipped 59 native libraries when it shipped 38.
That last check is the one that found ML Kit’s two binaries sitting unattributed in a shipping app the first time we ran it against a real package. No human review had noticed in months.