Bytes in, verified files in the project's _build out. Manifests are checked against
the hash the batch names; files against the hash the manifest names,
before anything reaches the build directory. Install is atomic per
dependency: staged under _build/<t>/lib/.precompiled/<dep>.<os pid>-<n>
(one dir per install), renamed into place with Mix's "already compiled"
marker.
Summary
Functions
GET url's body, capped at :max_bytes. A file:// URL is read from
the local disk instead: an absolute, canonical path with no symlink on
it (else {:error, "unsafe file URL"}), and a missing file is
{:error, "not found: <path>"}.
GET a metadata document (the batch), sending etag as If-None-Match
when there is one: the body and its ETag, or :not_modified. A failure
to reach the source (transport error, timeout, the run deadline, a 5xx,
offline) is {:error, {:unreachable, reason}}; anything the source
actually answered (a 4xx, an oversized or refused redirect) is
{:error, reason}.
Verifies and installs one dependency's build into build_lib.
Everything (files, Mix's marker, provenance, the include/src links) is
put together in staging first; replacing the old dep dir by a rename is
the last step, so any failure leaves _build as it was.
Whether dep's build dir holds exactly entry's files, judged from the
disk alone, never from the provenance file: the entry passes the same
checks as an install; ebin (and priv, when the entry ships one) are
plain directories holding only regular files, exactly the entry's paths;
each has the entry's mode and size, and its bytes hash to the entry's
sha256; and the .app owns exactly the beams. A priv the entry does not
ship must be absent, or Mix's symlink to deps/<dep>/priv, which is not
followed: Mix makes and remakes that link itself from the sources
deps.get verified, so it holds nothing an install placed. A real priv
dir, even an empty one, is refused. Other top-level entries (.mix, a
consolidated dir, Mix's include link) are not looked at: Mix
writes them, and none of them is code the build's .app loads.
installed_as?/3 for a directory named anything (<app>-<vsn> in a
store, say): dir is the entry's own directory and dep the app whose
.app it must hold. A priv the entry does not ship must be absent:
even an empty one is refused, since :code.priv_dir/1 points there.
Only with mix_priv_link: true (a Mix project's _build) may it be a
symlink to .../<dep>/priv, Mix's link to the dep's sources.
The manifest the batch names by expected_sha, under path: "manifests"
for a format-1 document, "v2/manifests" for a format-2 one. Either comes
back as format 2 (see Precompiled.Manifest).
Mix's "already compiled" marker (Mix.Dep.ElixirSCM) in the format the
running Mix reads: Elixir 1.20 added the lock (format 2); 1.17 to 1.19
read format 1 and would treat format 2 as stale and recompile the dep.
The provenance file under a dependency's build dir, written by install/6
when it is given :provenance.
Fetches entry's files into staging and verifies them: the entry's
paths and modes, each file's size and sha256, opts[:verify_app] (a
check on the staged .app, fn path -> :ok | {:fail, why} end), and
the .app owning exactly the staged beams. Nothing else: no Mix marker,
no provenance, no links, no rename, so it serves an install outside a
Mix project too. On a failure staging is removed.
Removes the temp files (*.tmp-*) that runs killed mid-write left in
the cache's files and manifests dirs, when last modified before
cutoff (Unix seconds). Another run sharing the home may be writing its
own right now, so the cutoff is well before this run started. Only
regular files directly in a plain cache dir are removed, and a failure
is ignored: it costs disk, never a build.
Rewrites an installed dep's provenance: a kept build that a newer
manifest still lists. Only into a .mix that is a plain directory.
Functions
GET url's body, capped at :max_bytes. A file:// URL is read from
the local disk instead: an absolute, canonical path with no symlink on
it (else {:error, "unsafe file URL"}), and a missing file is
{:error, "not found: <path>"}.
@spec get_conditional(map(), String.t(), String.t() | nil) :: {:ok, binary(), String.t() | nil} | :not_modified | {:error, String.t() | {:unreachable, String.t()}}
GET a metadata document (the batch), sending etag as If-None-Match
when there is one: the body and its ETag, or :not_modified. A failure
to reach the source (transport error, timeout, the run deadline, a 5xx,
offline) is {:error, {:unreachable, reason}}; anything the source
actually answered (a 4xx, an oversized or refused redirect) is
{:error, reason}.
A file:// URL (a bundle on local storage) is read from the disk, never
over HTTP, and never answers :not_modified: see get_body/2.
@spec install(map(), atom(), map(), tuple(), Path.t(), keyword()) :: {:ok, %{ files: non_neg_integer(), bytes: non_neg_integer(), cached: non_neg_integer() }} | {:fail, String.t()}
Verifies and installs one dependency's build into build_lib.
Everything (files, Mix's marker, provenance, the include/src links) is
put together in staging first; replacing the old dep dir by a rename is
the last step, so any failure leaves _build as it was.
Options: :deps_path (required), :provenance, :verify_app (a check
on the staged .app), and :link_sources, a test seam that replaces
the linker ((dep, entry, staging, dest, deps_path -> :ok)).
Whether dep's build dir holds exactly entry's files, judged from the
disk alone, never from the provenance file: the entry passes the same
checks as an install; ebin (and priv, when the entry ships one) are
plain directories holding only regular files, exactly the entry's paths;
each has the entry's mode and size, and its bytes hash to the entry's
sha256; and the .app owns exactly the beams. A priv the entry does not
ship must be absent, or Mix's symlink to deps/<dep>/priv, which is not
followed: Mix makes and remakes that link itself from the sources
deps.get verified, so it holds nothing an install placed. A real priv
dir, even an empty one, is refused. Other top-level entries (.mix, a
consolidated dir, Mix's include link) are not looked at: Mix
writes them, and none of them is code the build's .app loads.
installed_as?/3 for a directory named anything (<app>-<vsn> in a
store, say): dir is the entry's own directory and dep the app whose
.app it must hold. A priv the entry does not ship must be absent:
even an empty one is refused, since :code.priv_dir/1 points there.
Only with mix_priv_link: true (a Mix project's _build) may it be a
symlink to .../<dep>/priv, Mix's link to the dep's sources.
@spec manifest(map(), String.t(), String.t(), String.t()) :: {:ok, map()} | {:miss, String.t()} | {:fail, String.t()} | {:unreachable, String.t()}
The manifest the batch names by expected_sha, under path: "manifests"
for a format-1 document, "v2/manifests" for a format-2 one. Either comes
back as format 2 (see Precompiled.Manifest).
Looked up, in order: the cache (<home>/manifests/<sha>.json, re-hashed
on every read); where by-sha paths exist, <origin>/<path>/by-sha/<sha>.json
for each content origin and then the source; and last the source's
<path>/<name>.json, whose answer decides the result. Whatever bytes come
back are used only if they hash to expected_sha.
{:unreachable, why}: the manifest is not cached and the network could
not deliver it (offline, a transport error, a 5xx, the run deadline), as
opposed to a source that answered and refused.
Mix's "already compiled" marker (Mix.Dep.ElixirSCM) in the format the
running Mix reads: Elixir 1.20 added the lock (format 2); 1.17 to 1.19
read format 1 and would treat format 2 as stale and recompile the dep.
The provenance file under a dependency's build dir, written by install/6
when it is given :provenance.
@spec stage_verified(map(), atom(), map(), Path.t(), keyword()) :: {:ok, %{ files: non_neg_integer(), bytes: non_neg_integer(), cached: non_neg_integer() }} | {:fail, String.t()}
Fetches entry's files into staging and verifies them: the entry's
paths and modes, each file's size and sha256, opts[:verify_app] (a
check on the staged .app, fn path -> :ok | {:fail, why} end), and
the .app owning exactly the staged beams. Nothing else: no Mix marker,
no provenance, no links, no rename, so it serves an install outside a
Mix project too. On a failure staging is removed.
Removes the temp files (*.tmp-*) that runs killed mid-write left in
the cache's files and manifests dirs, when last modified before
cutoff (Unix seconds). Another run sharing the home may be writing its
own right now, so the cutoff is well before this run started. Only
regular files directly in a plain cache dir are removed, and a failure
is ignored: it costs disk, never a build.
Rewrites an installed dep's provenance: a kept build that a newer
manifest still lists. Only into a .mix that is a plain directory.