Reproducible Jupyter Notebook with Nix
So you have a Jupyter Notebook with pyproject.toml
and poetry.lock
files,
and you want to productionise it? You’ll just need a Linux or macOS machine.
This is the magic sauce:
{
pkgs ?
import
(
fetchTarball {
name = "22.05";
url = "https://github.com/NixOS/nixpkgs/archive/ce6aa13369b667ac2542593170993504932eb836.tar.gz";
sha256 = "0d643wp3l77hv2pmg2fi7vyxn4rwy0iyr8djcw1h5x72315ck9ik";
}
)
{},
}: let
poetryEnvironment = pkgs.poetry2nix.mkPoetryEnv {
projectDir = builtins.path {
path = ./.;
name = "my_project_name";
};
};
in
poetryEnvironment.env.overrideAttrs (
oldAttrs: {
buildInputs = [
pkgs.cacert
];
}
)
The top half of that
pins the
most recent nixpkgs release, to make
sure we always build from the same sources. mkPoetryEnv
creates a Nix
derivation based on your poetry.lock
file. And the certificate authority certs
package is needed to download Python packages using HTTPS.
And this is the magic spoon:
nix-shell --pure --run 'jupyter nbconvert --debug --execute --inplace --to=notebook my.ipynb'
.
It runs the notebook from start to finish within the Nix shell defined above. It
is verbose, to make it easier to detect any issues, so you might want to remove
--debug
.
Based on a real-life project which includes automated tests for both Nix and Poetry-based workflows.