Reproducible Jupyter Notebook with Nix
So you have a Jupyter Notebook with pyproject.toml
and poetry.lock
files,
and you want to productionise it? You’ll just need a Linux or macOS machine.
This is the magic sauce:
{
pkgs ?
import
(
fetchTarball {
name = "22.05";
url = "https://github.com/NixOS/nixpkgs/archive/ce6aa13369b667ac2542593170993504932eb836.tar.gz";
sha256 = "0d643wp3l77hv2pmg2fi7vyxn4rwy0iyr8djcw1h5x72315ck9ik";
}
)
{},
}: let
poetryEnvironment = pkgs.poetry2nix.mkPoetryEnv {
projectDir = builtins.path {
path = ./.;
name = "my_project_name";
};
};
in
poetryEnvironment.env.overrideAttrs (
oldAttrs: {
buildInputs = [
pkgs.cacert
];
}
)
The top half of that
pins the
most recent nixpkgs release, to make
sure we always build from the same sources. mkPoetryEnv
creates a Nix
derivation based on your poetry.lock
file. And the certificate authority certs
package is needed to download Python packages using HTTPS.
And this is the magic spoon:
nix-shell --pure --run 'jupyter nbconvert --debug --execute --inplace --to=notebook my.ipynb'
.
It runs the notebook from start to finish within the Nix shell defined above. It
is very verbose to make it easier to detect any issues, so you might want to
remove --debug
.
Based on a real-life project which includes automated tests for both Nix and Poetry-based workflows.