sitr.us

I have a project, git-format-staged, that I build with Nix. It includes NPM dependencies, and it is convenient to have Dependabot keep those up-to-date for me. Dependabot creates pull requests that update package-lock.json when it sees updates with security fixes and such. But my Nix configuration includes a hash of the project NPM dependencies - that hash must be updated when package-lock.json changes. Unfortunately Dependabot does not know how to do that. So I came up with a workflow to help that bot out.

The hash is in test/test.nix:

The product of that repo is actually a zero-dependency Python program. I'm just using Node and NPM to run a test framework (for perfectly-valid reasons). I have implemented test runs as derivations which means they run in Nix' sandboxed build environment. To get reproducibility that means network requests are not allowed unless I specify the hash of what's going to be downloaded up front. The hash here is a recursive hash of a directory of downloaded NPM packages that can be installed later by running npm install --cache.

(When I'm working on Rust projects I use Crane which is able to infer dependency hashes from Cargo.lock so I don't need to update a hash in a Nix expression when dependencies change. I haven't found a tool that does that for NPM, so for now at least I have this hash to keep up to date.)

Updating the Hash

So what I want is an automated process that updates that hash when package-lock.json changes. That means I need to be able to:

1. compute the new dependencies hash
2. update test/test.nix with the new hash

There are existing solutions out there for doing this kind of thing. For example the nixpkgs repo has maintainers/scripts/update.nix. I did some looking to see if there is something out there that would work for me. But then I decided it would be easier to write my own solution.

fetchNpmDeps is a fetcher in nixpkgs that is specialized for fetching NPM dependencies - given a source directory with a package-lock.json file it fetches exactly what you need. Most Nix fetchers come with a corresponding "prefetch" tool that tells you the hash of the fetched content. fetchNpmDeps is paired with prefetch-npm-deps (defined in the same file) for that purpose. I can use prefetch-npm-deps for step 1, and a little sed for step 2. I ended up with this package definition in my flake.nix:

Now I can run this command to update the hash automatically:

The script ends up being quite simple, partly because I only have one hash in test/test.nix so it is easy to target the sed script. I could keep things cleaner by separating the hashes out into a separate file:

In that case the update script would look like this:

Automation to Help Dependabot

Every one of the Dependabot PRs in my repo was failing required checks because the PRs updated package-lock.json, but did not update that hash. So the next step was to set up a Github workflow to run the hash-updating script after every change from Dependabot.

It would be great if I could configure Dependabot to run a custom shell command along with its updates. But as far as I can tell that is not an option. Instead I added a workflow that runs on pushes to Dependabot's PR branches. Those all have names of the form dependabot/npm_and_yarn/*. After some research I used this workflow, .github/workflows/dependabot-post.yml:

This workflow enables a special permission to allow it to push commits back to the repo. That can be dangerous if you have third-party code running because that code would have access to modify your repository. Notably NPM packages can run arbitrary code when they are installed. But that is not an issue here: prefetch-npm-deps does not "install" dependencies so it does not run package install scripts. Instead it pre-populates a local cache that can be "installed" later.

Normally Dependabot will refuse to automatically update one of its PRs after someone else has pushed commits to it. The workflow includes [dependabot skip] in its commit messages to signal to Dependabot that it is OK to throw those commits out when it recreates or rebases its PRs. When that happens the workflow runs again, and re-applies the correct hash.

Note that I don't have to worry about getting into an infinite loop because once the correct hash is set any subsequent runs will be noops, and so will not trigger more branch push events.

So now I'm happily automated, and I've been merging a bunch of Dependabot PRs that I let pile up.