VextaProject/ Vexta
Vexta Vexta-CPU-Miner Vexta-Android-Wallet
Watch
1
0
Fork
You've already forked Vexta
 0
Code Issues Pull requests Projects Packages Wiki Activity Actions

Initial Vexta Core v0.1.0 release

Browse source
This commit is contained in:
Capri 2026-07-15 14:07:28 +00:00
commit 25c454329d
2481 changed files with 689289 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

813
contrib/guix/INSTALL.md Normal file
View file

@ -0,0 +1,813 @@
# Guix Installation and Setup
This only needs to be done once per machine. If you have already completed the
installation and setup, please proceed to [perform a build](./README.md).
Otherwise, you may choose from one of the following options to install Guix:
1. Using the official **shell installer script** [⤓ skip to section][install-script]
- Maintained by Guix developers
- Easiest (automatically performs *most* setup)
- Works on nearly all Linux distributions
- Only installs latest release
- Binary installation only, requires high level of trust
- Note: The script needs to be run as root, so it should be inspected before it's run
2. Using the official **binary tarball** [⤓ skip to section][install-bin-tarball]
- Maintained by Guix developers
- Normal difficulty (full manual setup required)
- Works on nearly all Linux distributions
- Installs any release
- Binary installation only, requires high level of trust
3. Using fanquake's **Docker image** [↗︎ external instructions][install-fanquake-docker]
- Maintained by fanquake
- Easy (automatically performs *some* setup)
- Works wherever Docker images work
- Installs any release
- Binary installation only, requires high level of trust
4. Using a **distribution-maintained package** [⤓ skip to section][install-distro-pkg]
- Maintained by distribution's Guix package maintainer
- Normal difficulty (manual setup required)
- Works only on distributions with Guix packaged, see: https://repology.org/project/guix/versions
- Installs a release decided on by package maintainer
- Source or binary installation depending on the distribution
5. Building **from source** [⤓ skip to section][install-source]
- Maintained by you
- Hard, but rewarding
- Can be made to work on most Linux distributions
- Installs any commit (more granular)
- Source installation, requires lower level of trust
## Options 1 and 2: Using the official shell installer script or binary tarball
The installation instructions for both the official shell installer script and
the binary tarballs can be found in the GNU Guix Manual's [Binary Installation
section](https://guix.gnu.org/manual/en/html_node/Binary-Installation.html).
Note that running through the binary tarball installation steps is largely
equivalent to manually performing what the shell installer script does.
Note that at the time of writing (July 5th, 2021), the shell installer script
automatically creates an `/etc/profile.d` entry which the binary tarball
installation instructions do not ask you to create. However, you will likely
need this entry for better desktop integration. Please see [this
section](#add-an-etcprofiled-entry) for instructions on how to add a
`/etc/profile.d/guix.sh` entry.
Regardless of which installation option you chose, the changes to
`/etc/profile.d` will not take effect until the next shell or desktop session,
so you should log out and log back in.
## Option 3: Using fanquake's Docker image
Please refer to fanquake's instructions
[here](https://github.com/fanquake/core-review/tree/master/guix).
Note that the `Dockerfile` is largely equivalent to running through the binary
tarball installation steps.
## Option 4: Using a distribution-maintained package
Note that this section is based on the distro packaging situation at the time of
writing (July 2021). Guix is expected to be more widely packaged over time. For
an up-to-date view on Guix's package status/version across distros, please see:
https://repology.org/project/guix/versions
### Debian 11 (Bullseye)/Ubuntu 21.04 (Hirsute Hippo)
Guix v1.2.0 is available as a distribution package starting in [Debian
11](https://packages.debian.org/bullseye/guix) and [Ubuntu
21.04](https://packages.ubuntu.com/hirsute/guix).
Note that if you intend on using Guix without using any substitutes (more
details [here][security-model]), v1.2.0 has a known problems when building
GnuTLS from source. Solutions and workarounds are documented
[here](#gnutls-test-suite-fail-status-request-revoked).
To install:
```sh
sudo apt install guix
```
For up-to-date information on Debian and Ubuntu's release history:
- [Debian release history](https://www.debian.org/releases/)
- [Ubuntu release history](https://ubuntu.com/about/release-cycle)
### Arch Linux
Guix is available in the AUR as
[`guix`](https://aur.archlinux.org/packages/guix/), please follow the
installation instructions in the Arch Linux Wiki ([live
link](https://wiki.archlinux.org/index.php/Guix#AUR_Package_Installation),
[2021/03/30
permalink](https://wiki.archlinux.org/index.php?title=Guix&oldid=637559#AUR_Package_Installation))
to install Guix.
At the time of writing (2021/03/30), the `check` phase will fail if the path to
guix's build directory is longer than 36 characters due to an anachronistic
character limit on the shebang line. Since the `check` phase happens after the
`build` phase, which may take quite a long time, it is recommended that users
either:
1. Skip the `check` phase
- For `makepkg`: `makepkg --nocheck ...`
- For `yay`: `yay --mflags="--nocheck" ...`
- For `paru`: `paru --nocheck ...`
2. Or, check their build directory's length beforehand
- For those building with `makepkg`: `pwd | wc -c`
## Option 5: Building from source
Building Guix from source is a rather involved process but a rewarding one for
those looking to minimize trust and maximize customizability (e.g. building a
particular commit of Guix). Previous experience with using autotools-style build
systems to build packages from source will be helpful. *hic sunt dracones.*
I strongly urge you to at least skim through the entire section once before you
start issuing commands, as it will save you a lot of unncessary pain and
anguish.
### Installing common build tools
There are a few basic build tools that are required for most things we'll build,
so let's install them now:
Text transformation/i18n:
- `autopoint` (sometimes packaged in `gettext`)
- `help2man`
- `po4a`
- `texinfo`
Build system tools:
- `g++` w/ C++11 support
- `libtool`
- `autoconf`
- `automake`
- `pkg-config` (sometimes packaged as `pkgconf`)
- `make`
- `cmake`
Miscellaneous:
- `git`
- `gnupg`
- `python3`
### Building and Installing Guix's dependencies
In order to build Guix itself from source, we need to first make sure that the
necessary dependencies are installed and discoverable. The most up-to-date list
of Guix's dependencies is kept in the ["Requirements"
section](https://guix.gnu.org/manual/en/html_node/Requirements.html) of the Guix
Reference Manual.
Depending on your distribution, most or all of these dependencies may already be
packaged and installable without manually building and installing.
For reference, the graphic below outlines Guix v1.3.0's dependency graph:
![boostrap map](https://user-images.githubusercontent.com/6399679/125064185-a9a59880-e0b0-11eb-82c1-9b8e5dc9950d.png)
#### Guile
##### Choosing a Guile version and sticking to it
One of the first things you need to decide is which Guile version you want to
use: Guile v2.2 or Guile v3.0. Unlike the python2 to python3 transition, Guile
v2.2 and Guile v3.0 are largely compatible, as evidenced by the fact that most
Guile packages and even [Guix
itself](https://guix.gnu.org/en/blog/2020/guile-3-and-guix/) support running on
both.
What is important here is that you **choose one**, and you **remain consistent**
with your choice throughout **all Guile-related packages**, no matter if they
are installed via the distribution's package manager or installed from source.
This is because the files for Guile packages are installed to directories which
are separated based on the Guile version.
###### Example: Checking that Ubuntu's `guile-git` is compatible with your chosen Guile version
On Ubuntu Focal:
```sh
$ apt show guile-git
Package: guile-git
...
Depends: guile-2.2, guile-bytestructures, libgit2-dev
...
```
As you can see, the package `guile-git` depends on `guile-2.2`, meaning that it
was likely built for Guile v2.2. This means that if you decided to use Guile
v3.0 on Ubuntu Focal, you would need to build guile-git from source instead of
using the distribution package.
On Ubuntu Hirsute:
```sh
$ apt show guile-git
Package: guile-git
...
Depends: guile-3.0 | guile-2.2, guile-bytestructures (>= 1.0.7-3~), libgit2-dev (>= 1.0)
...
```
In this case, `guile-git` depends on either `guile-3.0` or `guile-2.2`, meaning
that it would work no matter what Guile version you decided to use.
###### Corner case: Multiple versions of Guile on one system
It is recommended to only install one version of Guile, so that build systems do
not get confused about which Guile to use.
However, if you insist on having both Guile v2.2 and Guile v3.0 installed on
your system, then you need to **consistently** specify one of
`GUILE_EFFECTIVE_VERSION=3.0` or `GUILE_EFFECTIVE_VERSION=2.2` to all
`./configure` invocations for Guix and its dependencies.
##### Installing Guile
Guile is most likely already packaged for your distribution, so after you have
[chosen a Guile version](#choosing-a-guile-version-and-sticking-to-it), install
it via your distribution's package manager.
If your distribution splits packages into `-dev`-suffixed and
non-`-dev`-suffixed sub-packages (as is the case for Debian-derived
distributions), please make sure to install both. For example, to install Guile
v2.2 on Debian/Ubuntu:
```sh
apt install guile-2.2 guile-2.2-dev
```
#### Mixing distribution packages and source-built packages
At the time of writing, most distributions have _some_ of Guix's dependencies
packaged, but not all. This means that you may want to install the distribution
package for some dependencies, and manually build-from-source for others.
Distribution packages usually install to `/usr`, which is different from the
default `./configure` prefix of source-built packages: `/usr/local`.
This means that if you mix-and-match distribution packages and source-built
packages and do not specify exactly `--prefix=/usr` to `./configure` for
source-built packages, you will need to augment the `GUILE_LOAD_PATH` and
`GUILE_LOAD_COMPILED_PATH` environment variables so that Guile will look
under the right prefix and find your source-built packages.
For example, if you are using Guile v2.2, and have Guile packages in the
`/usr/local` prefix, either add the following lines to your `.profile` or
`.bash_profile` so that the environment variable is properly set for all future
shell logins, or paste the lines into a POSIX-style shell to temporarily modify
the environment variables of your current shell session.
```sh
# Help Guile v2.2.x find packages in /usr/local
export GUILE_LOAD_PATH="/usr/local/share/guile/site/2.2${GUILE_LOAD_PATH:+:}$GUILE_LOAD_PATH"
export GUILE_LOAD_COMPILED_PATH="/usr/local/lib/guile/2.2/site-ccache${GUILE_LOAD_COMPILED_PATH:+:}$GUILE_COMPILED_LOAD_PATH"
```
Note that these environment variables are used to check for packages during
`./configure`, so they should be set as soon as possible should you want to use
a prefix other than `/usr`.
<!-- ##### Example: Consistently using Guile 3.0 on Ubuntu -->
<!-- For example, on Ubuntu, if you choose to use Guile 3.0 and install the -->
<!-- `guile-3.0` package, you want to make sure that if you also want to install the -->
<!-- `guile-git` package with `apt` that said `guile-git` package was built for Guile -->
<!-- v3.0. This can be checked by invoking the following: -->
<!-- ``` -->
<!-- apt update -->
<!-- apt show guile-git -->
<!-- ``` -->
#### Building and installing source-built packages
***IMPORTANT**: A few dependencies have non-obvious quirks/erratas which are documented in the
sub-sections immediately below. Please read these sections before proceeding to
build and install these packages.*
Although you should always refer to the README or INSTALL files for the most
accurate information, most of these dependencies use autoconf-style build
systems (check if there's a `configure.ac` file), and will likely do the right
thing with the following:
Clone the repository and check out the latest release:
```sh
git clone <git-repo-of-dependency>/<dependency>.git
cd <dependency>
git tag -l # check for the latest release
git checkout <latest-release>
```
For autoconf-based build systems (if `./autogen.sh` or `configure.ac` exists at
the root of the repository):
```sh
./autogen.sh || autoreconf -vfi
./configure --prefix=<prefix>
make
sudo make install
```
For CMake-based build systems (if `CMakeLists.txt` exists at the root of the
repository):
```sh
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=<prefix>
sudo cmake --build . --target install
```
If you choose not to specify exactly `--prefix=/usr` to `./configure`, please
make sure you've carefully read the [previous section] on mixing distribution
packages and source-built packages.
##### Binding packages require `-dev`-suffixed packages
Relevant for:
- Everyone
When building bindings, the `-dev`-suffixed version of the original package
needs to be installed. For example, building `Guile-zlib` on Debian-derived
distributions requires that `zlib1g-dev` is installed.
When using bindings, the `-dev`-suffixed version of the original package still
needs to be installed. This is particularly problematic when distribution
packages are mispackaged like `guile-sqlite3` is in Ubuntu Focal such that
installing `guile-sqlite3` does not automatically install `libsqlite3-dev` as a
dependency.
Below is a list of relevant Guile bindings and their corresponding `-dev`
packages in Debian at the time of writing.
| Guile binding package | -dev Debian package |
|-----------------------|---------------------|
| guile-gcrypt | libgcrypt-dev |
| guile-git | libgit2-dev |
| guile-lzlib | liblz-dev |
| guile-ssh | libssh-dev |
| guile-sqlite3 | libsqlite3-dev |
| guile-zlib | zlib1g-dev |
##### `guile-git` actually depends on `libgit2 >= 1.1`
Relevant for:
- Those building `guile-git` from source against `libgit2 < 1.1`
- Those installing `guile-git` from their distribution where `guile-git` is
built against `libgit2 < 1.1`
As of v0.4.0, `guile-git` claims to only require `libgit2 >= 0.28.0`, however,
it actually requires `libgit2 >= 1.1`, otherwise, it will be confused by a
reference of `origin/keyring`: instead of interpreting the reference as "the
'keyring' branch of the 'origin' remote", the reference is interpreted as "the
branch literally named 'origin/keyring'"
This is especially notable because Ubuntu Focal packages `libgit2 v0.28.4`, and
`guile-git` is built against it.
Should you be in this situation, you need to build both `libgit2 v1.1.x` and
`guile-git` from source.
Source: http://logs.guix.gnu.org/guix/2020-11-12.log#232527
##### `{scheme,guile}-bytestructures` v1.0.8 and v1.0.9 are broken for Guile v2.2
Relevant for:
- Those building `{scheme,guile}-bytestructures` from source against Guile v2.2
Commit
[707eea3](https://github.com/TaylanUB/scheme-bytestructures/commit/707eea3a85e1e375e86702229ebf73d496377669)
introduced a regression for Guile v2.2 and was first included in v1.0.8, this
was later corrected in commit
[ec9a721](https://github.com/TaylanUB/scheme-bytestructures/commit/ec9a721957c17bcda13148f8faa5f06934431ff7)
and included in v1.1.0.
TL;DR If you decided to use Guile v2.2, do not use `{scheme,guile}-bytestructures` v1.0.8 or v1.0.9.
### Building and Installing Guix itself
Start by cloning Guix:
```
git clone https://git.savannah.gnu.org/git/guix.git
cd guix
```
You will likely want to build the latest release, however, if the latest release
when you're reading this is still 1.2.0 then you may want to use 95aca29 instead
to avoid a problem in the GnuTLS test suite.
```
git branch -a -l 'origin/version-*' # check for the latest release
git checkout <latest-release>
```
Bootstrap the build system:
```
./bootstrap
```
Configure with the recommended `--localstatedir` flag:
```
./configure --localstatedir=/var
```
Note: If you intend to hack on Guix in the future, you will need to supply the
same `--localstatedir=` flag for all future Guix `./configure` invocations. See
the last paragraph of this
[section](https://guix.gnu.org/manual/en/html_node/Requirements.html) for more
details.
Build Guix (this will take a while):
```
make -j$(nproc)
```
Install Guix:
```
sudo make install
```
### Post-"build from source" Setup
#### Creating and starting a `guix-daemon-original` service with a fixed `argv[0]`
At this point, guix will be installed to `${bindir}`, which is likely
`/usr/local/bin` if you did not override directory variables at
`./configure`-time. More information on standard Automake directory variables
can be found
[here](https://www.gnu.org/software/automake/manual/html_node/Standard-Directory-Variables.html).
However, the Guix init scripts and service configurations for Upstart, systemd,
SysV, and OpenRC are installed (in `${libdir}`) to launch
`${localstatedir}/guix/profiles/per-user/root/current-guix/bin/guix-daemon`,
which does not yet exist, and will only exist after [`root` performs their first
`guix pull`](#guix-pull-as-root).
We need to create a `-original` version of these init scripts that's pointed to
the binaries we just built and `make install`'ed in `${bindir}` (normally,
`/usr/local/bin`).
Example for `systemd`, run as `root`:
```sh
# Create guix-daemon-original.service by modifying guix-daemon.service
libdir=# set according to your PREFIX (default is /usr/local/lib)
bindir="$(dirname $(command -v guix-daemon))"
sed -E -e "s|/\S*/guix/profiles/per-user/root/current-guix/bin/guix-daemon|${bindir}/guix-daemon|" "${libdir}"/systemd/system/guix-daemon.service > /etc/systemd/system/guix-daemon-original.service
chmod 664 /etc/systemd/system/guix-daemon-original.service
# Make systemd recognize the new service
systemctl daemon-reload
# Make sure that the non-working guix-daemon.service is stopped and disabled
systemctl stop guix-daemon
systemctl disable guix-daemon
# Make sure that the working guix-daemon-original.service is started and enabled
systemctl enable guix-daemon-original
systemctl start guix-daemon-original
```
#### Creating `guix-daemon` users / groups
Please see the [relevant
section](https://guix.gnu.org/manual/en/html_node/Build-Environment-Setup.html)
in the Guix Reference Manual for more details.
## Optional setup
At this point, you are set up to [use Guix to build DigiByte
Core](./README.md#usage). However, if you want to polish your setup a bit and
make it "what Guix intended", then read the next few subsections.
### Add an `/etc/profile.d` entry
This section definitely does not apply to you if you installed Guix using:
1. The shell installer script
2. fanquake's Docker image
3. Debian's `guix` package
#### Background
Although Guix knows how to update itself and its packages, it does so in a
non-invasive way (it does not modify `/usr/local/bin/guix`).
Instead, it does the following:
- After a `guix pull`, it updates
`/var/guix/profiles/per-user/$USER/current-guix`, and creates a symlink
targeting this directory at `$HOME/.config/guix/current`
- After a `guix install`, it updates
`/var/guix/profiles/per-user/$USER/guix-profile`, and creates a symlink
targeting this directory at `$HOME/.guix-profile`
Therefore, in order for these operations to affect your shell/desktop sessions
(and for the principle of least astonishment to hold), their corresponding
directories have to be added to well-known environment variables like `$PATH`,
`$INFOPATH`, `$XDG_DATA_DIRS`, etc.
In other words, if `$HOME/.config/guix/current/bin` does not exist in your
`$PATH`, a `guix pull` will have no effect on what `guix` you are using. Same
goes for `$HOME/.guix-profile/bin`, `guix install`, and installed packages.
Helpfully, after a `guix pull` or `guix install`, a message will be printed like
so:
```
hint: Consider setting the necessary environment variables by running:
GUIX_PROFILE="$HOME/.guix-profile"
. "$GUIX_PROFILE/etc/profile"
Alternately, see `guix package --search-paths -p "$HOME/.guix-profile"'.
```
However, this is somewhat tedious to do for both `guix pull` and `guix install`
for each user on the system that wants to properly use `guix`. I recommend that
you instead add an entry to `/etc/profile.d` instead. This is done by default
when installing the Debian package later than 1.2.0-4 and when using the shell
script installer.
#### Instructions
Create `/etc/profile.d/guix.sh` with the following content:
```sh
# _GUIX_PROFILE: `guix pull` profile
_GUIX_PROFILE="$HOME/.config/guix/current"
if [ -L $_GUIX_PROFILE ]; then
export PATH="$_GUIX_PROFILE/bin${PATH:+:}$PATH"
# Export INFOPATH so that the updated info pages can be found
# and read by both /usr/bin/info and/or $GUIX_PROFILE/bin/info
# When INFOPATH is unset, add a trailing colon so that Emacs
# searches 'Info-default-directory-list'.
export INFOPATH="$_GUIX_PROFILE/share/info:$INFOPATH"
fi
# GUIX_PROFILE: User's default profile
GUIX_PROFILE="$HOME/.guix-profile"
[ -L $GUIX_PROFILE ] || return
GUIX_LOCPATH="$GUIX_PROFILE/lib/locale"
export GUIX_PROFILE GUIX_LOCPATH
[ -f "$GUIX_PROFILE/etc/profile" ] && . "$GUIX_PROFILE/etc/profile"
# set XDG_DATA_DIRS to include Guix installations
export XDG_DATA_DIRS="$GUIX_PROFILE/share:${XDG_DATA_DIRS:-/usr/local/share/:/usr/share/}"
```
Please note that this will not take effect until the next shell or desktop
session (log out and log back in).
### `guix pull` as root
Before you do this, you need to read the section on [choosing your security
model][security-model] and adjust `guix` and `guix-daemon` flags according to
your choice, as invoking `guix pull` may pull substitutes from substitute
servers (which you may not want).
As mentioned in a previous section, Guix expects
`${localstatedir}/guix/profiles/per-user/root/current-guix` to be populated with
`root`'s Guix profile, `guix pull`-ed and built by some former version of Guix.
However, this is not the case when we build from source. Therefore, we need to
perform a `guix pull` as `root`:
```sh
sudo --login guix pull --branch=version-<latest-release-version>
# or
sudo --login guix pull --commit=<particular-commit>
```
`guix pull` is quite a long process (especially if you're using
`--no-substitute`). If you encounter build problems, please refer to the
[troubleshooting section](#troubleshooting).
Note that running a bare `guix pull` with no commit or branch specified will
pull the latest commit on Guix's master branch, which is likely fine, but not
recommended.
If you installed Guix from source, you may get an error like the following:
```sh
error: while creating symlink '/root/.config/guix/current' No such file or directory
```
To resolve this, simply:
```
sudo mkdir -p /root/.config/guix
```
Then try the `guix pull` command again.
After the `guix pull` finishes successfully,
`${localstatedir}/guix/profiles/per-user/root/current-guix` should be populated.
#### Using the newly-pulled `guix` by restarting the daemon
Depending on how you installed Guix, you should now make sure that your init
scripts and service configurations point to the newly-pulled `guix-daemon`.
##### If you built Guix from source
If you followed the instructions for [fixing argv\[0\]][fix-argv0], you can now
do the following:
```sh
systemctl stop guix-daemon-original
systemctl disable guix-daemon-original
systemctl enable guix-daemon
systemctl start guix-daemon
```
##### If you installed Guix via the Debian/Ubuntu distribution packages
You will need to create a `guix-daemon-latest` service which points to the new
`guix` rather than a pinned one.
```sh
# Create guix-daemon-latest.service by modifying guix-daemon.service
sed -E -e "s|/usr/bin/guix-daemon|/var/guix/profiles/per-user/root/current-guix/bin/guix-daemon|" /etc/systemd/system/guix-daemon.service > /lib/systemd/system/guix-daemon-latest.service
chmod 664 /lib/systemd/system/guix-daemon-latest.service
# Make systemd recognize the new service
systemctl daemon-reload
# Make sure that the old guix-daemon.service is stopped and disabled
systemctl stop guix-daemon
systemctl disable guix-daemon
# Make sure that the new guix-daemon-latest.service is started and enabled
systemctl enable guix-daemon-latest
systemctl start guix-daemon-latest
```
##### If you installed Guix via lantw44's Arch Linux AUR package
At the time of writing (July 5th, 2021) the systemd unit for "updated Guix" is
`guix-daemon-latest.service`, therefore, you should do the following:
```sh
systemctl stop guix-daemon
systemctl disable guix-daemon
systemctl enable guix-daemon-latest
systemctl start guix-daemon-latest
```
##### Otherwise...
Simply do:
```sh
systemctl restart guix-daemon
```
### Checking everything
If you followed all the steps above to make your Guix setup "prim and proper,"
you can check that you did everything properly by running through this
checklist.
1. `/etc/profile.d/guix.sh` should exist and be sourced at each shell login
2. `guix describe` should not print `guix describe: error: failed to determine
origin`, but rather something like:
```
Generation 38 Feb 22 2021 16:39:31 (current)
guix f350df4
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: version-1.2.0
commit: f350df405fbcd5b9e27e6b6aa500da7f101f41e7
```
3. `guix-daemon` should be running from `${localstatedir}/guix/profiles/per-user/root/current-guix`
# Troubleshooting
## Derivation failed to build
When you see a build failure like below:
```
building /gnu/store/...-foo-3.6.12.drv...
/ 'check' phasenote: keeping build directory `/tmp/guix-build-foo-3.6.12.drv-0'
builder for `/gnu/store/...-foo-3.6.12.drv' failed with exit code 1
build of /gnu/store/...-foo-3.6.12.drv failed
View build log at '/var/log/guix/drvs/../...-foo-3.6.12.drv.bz2'.
cannot build derivation `/gnu/store/...-qux-7.69.1.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/...-bar-3.16.5.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/...-baz-2.0.5.drv': 1 dependencies couldn't be built
guix time-machine: error: build of `/gnu/store/...-baz-2.0.5.drv' failed
```
It means that `guix` failed to build a package named `foo`, which was a
dependency of `qux`, `bar`, and `baz`. Importantly, note that the last "failed"
line is not necessarily the root cause, the first "failed" line is.
Most of the time, the build failure is due to a spurious test failure or the
package's build system/test suite breaking when running multi-threaded. To
rebuild _just_ this derivation in a single-threaded fashion (please don't forget
to add other `guix` flags like `--no-substitutes` as appropriate):
```sh
$ guix build --cores=1 /gnu/store/...-foo-3.6.12.drv
```
If the single-threaded rebuild did not succeed, you may need to dig deeper.
You may view `foo`'s build logs in `less` like so (please replace paths with the
path you see in the build failure output):
```sh
$ bzcat /var/log/guix/drvs/../...-foo-3.6.12.drv.bz2 | less
```
`foo`'s build directory is also preserved and available at
`/tmp/guix-build-foo-3.6.12.drv-0`. However, if you fail to build `foo` multiple
times, it may be `/tmp/...drv-1` or `/tmp/...drv-2`. Always consult the build
failure output for the most accurate, up-to-date information.
### python(-minimal): [Errno 84] Invalid or incomplete multibyte or wide character
This error occurs when your `$TMPDIR` (default: /tmp) exists on a filesystem
which rejects characters not present in the UTF-8 character code set. An example
is ZFS with the utf8only=on option set.
More information: https://bugs.python.org/issue37584
### GnuTLS: test-suite FAIL: status-request-revoked
*The derivation is likely identified by: `/gnu/store/vhphki5sg9xkdhh2pbc8gi6vhpfzryf0-gnutls-3.6.12.drv`*
This unfortunate error is most common for non-substitute builders who installed
Guix v1.2.0. The problem stems from the fact that one of GnuTLS's tests uses a
hardcoded certificate which expired on 2020-10-24.
What's more unfortunate is that this GnuTLS derivation is somewhat special in
Guix's dependency graph and is not affected by the package transformation flags
like `--without-tests=`.
The easiest solution for those encountering this problem is to install a newer
version of Guix. However, there are ways to work around this issue:
#### Workaround 1: Using substitutes for this single derivation
If you've authorized the official Guix build farm's key (more info
[here](./README.md#step-1-authorize-the-signing-keys)), then you can use
substitutes just for this single derivation by invoking the following:
```sh
guix build --substitute-urls="https://ci.guix.gnu.org" /gnu/store/vhphki5sg9xkdhh2pbc8gi6vhpfzryf0-gnutls-3.6.12.drv
```
See [this section](./README.md#removing-authorized-keys) for instructions on how
to remove authorized keys if you don't want to keep the build farm's key
authorized.
#### Workaround 2: Temporarily setting the system clock back
This workaround was described [here](https://issues.guix.gnu.org/44559#5).
Basically:
1. Turn off networking
2. Turn off NTP
3. Set system time to 2020-10-01
4. guix build --no-substitutes /gnu/store/vhphki5sg9xkdhh2pbc8gi6vhpfzryf0-gnutls-3.6.12.drv
5. Set system time back to accurate current time
6. Turn NTP back on
7. Turn networking back on
### coreutils: FAIL: tests/tail-2/inotify-dir-recreate
The inotify-dir-create test fails on "remote" filesystems such as overlayfs
(Docker's default filesystem) due to the filesystem being mistakenly recognized
as non-remote.
A relatively easy workaround to this is to make sure that a somewhat traditional
filesystem is mounted at `/tmp` (where `guix-daemon` performs its builds). For
Docker users, this might mean [using a volume][docker/volumes], [binding
mounting][docker/bind-mnt] from host, or (for those with enough RAM and swap)
[mounting a tmpfs][docker/tmpfs] using the `--tmpfs` flag.
Please see the following links for more details:
- An upstream coreutils bug has been filed: [debbugs#47940](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47940)
- A Guix bug detailing the underlying problem has been filed: [guix-issues#47935](https://issues.guix.gnu.org/47935)
- A commit to skip this test in Guix has been merged into the core-updates branch:
[savannah/guix@6ba1058](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=6ba1058df0c4ce5611c2367531ae5c3cdc729ab4)
[install-script]: #options-1-and-2-using-the-official-shell-installer-script-or-binary-tarball
[install-bin-tarball]: #options-1-and-2-using-the-official-shell-installer-script-or-binary-tarball
[install-fanquake-docker]: #option-3-using-fanquakes-docker-image
[install-distro-pkg]: #option-4-using-a-distribution-maintained-package
[install-source]: #option-5-building-from-source
[fix-argv0]: #creating-and-starting-a-guix-daemon-original-service-with-a-fixed-argv0
[security-model]: ./README.md#choosing-your-security-model
[docker/volumes]: https://docs.docker.com/storage/volumes/
[docker/bind-mnt]: https://docs.docker.com/storage/bind-mounts/
[docker/tmpfs]: https://docs.docker.com/storage/tmpfs/

485
contrib/guix/README.md Normal file
View file

@ -0,0 +1,485 @@
# Bootstrappable DigiByte Core Builds
This directory contains the files necessary to perform bootstrappable DigiByte
Core builds.
[Bootstrappability][b17e] furthers our binary security guarantees by allowing us
to _audit and reproduce_ our toolchain instead of blindly _trusting_ binary
downloads.
We achieve bootstrappability by using Guix as a functional package manager.
# Requirements
Conservatively, a x86_64 machine with:
- 16GB of free disk space on the partition that /gnu/store will reside in
- 8GB of free disk space **per platform triple** you're planning on building
(see the `HOSTS` [environment variable description][env-vars-list])
# Installation and Setup
If you don't have Guix installed and set up, please follow the instructions in
[INSTALL.md](./INSTALL.md)
# Usage
If you haven't considered your security model yet, please read [the relevant
section](#choosing-your-security-model) before proceeding to perform a build.
## Making the Xcode SDK available for macOS cross-compilation
In order to perform a build for macOS (which is included in the default set of
platform triples to build), you'll need to extract the macOS SDK tarball using
tools found in the [`macdeploy` directory](../macdeploy/README.md).
You can then either point to the SDK using the `SDK_PATH` environment variable:
```sh
# Extract the SDK tarball to /path/to/parent/dir/of/extracted/SDK/Xcode-<foo>-<bar>-extracted-SDK-with-libcxx-headers
tar -C /path/to/parent/dir/of/extracted/SDK -xaf /path/to/Xcode-<foo>-<bar>-extracted-SDK-with-libcxx-headers.tar.gz
# Indicate where to locate the SDK tarball
export SDK_PATH=/path/to/parent/dir/of/extracted/SDK
```
or extract it into `depends/SDKs`:
```sh
mkdir -p depends/SDKs
tar -C depends/SDKs -xaf /path/to/SDK/tarball
```
## Building
*The author highly recommends at least reading over the [common usage patterns
and examples](#common-guix-build-invocation-patterns-and-examples) section below
before starting a build. For a full list of customization options, see the
[recognized environment variables][env-vars-list] section.*
To build DigiByte Core reproducibly with all default options, invoke the
following from the top of a clean repository:
```sh
./contrib/guix/guix-build
```
## Codesigning build outputs
The `guix-codesign` command attaches codesignatures (produced by codesigners) to
existing non-codesigned outputs. Please see the [release process
documentation](/doc/release-process.md) for more context.
It respects many of the same environment variable flags as `guix-build`, with 2
crucial differences:
1. Since only Windows and macOS build outputs require codesigning, the `HOSTS`
environment variable will have a sane default value of `x86_64-w64-mingw32
x86_64-apple-darwin18` instead of all the platforms.
2. The `guix-codesign` command ***requires*** a `DETACHED_SIGS_REPO` flag.
* _**DETACHED_SIGS_REPO**_
Set the directory where detached codesignatures can be found for the current
DigiByte Core version being built.
_REQUIRED environment variable_
An invocation with all default options would look like:
```
env DETACHED_SIGS_REPO=<path/to/digibyte-detached-sigs> ./contrib/guix-codesign
```
## Cleaning intermediate work directories
By default, `guix-build` leaves all intermediate files or "work directories"
(e.g. `depends/work`, `guix-build-*/distsrc-*`) intact at the end of a build so
that they are available to the user (to aid in debugging, etc.). However, these
directories usually take up a large amount of disk space. Therefore, a
`guix-clean` convenience script is provided which cleans the current `git`
worktree to save disk space:
```
./contrib/guix/guix-clean
```
## Attesting to build outputs
Much like how Gitian build outputs are attested to in a `gitian.sigs`
repository, Guix build outputs are attested to in the [`guix.sigs`
repository](https://github.com/digibyte-core/guix.sigs).
After you've cloned the `guix.sigs` repository, to attest to the current
worktree's commit/tag:
```
env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=<gpg-key-name> ./contrib/guix/guix-attest
```
See `./contrib/guix/guix-attest --help` for more information on the various ways
`guix-attest` can be invoked.
## Verifying build output attestations
After at least one other signer has uploaded their signatures to the `guix.sigs`
repository:
```
git -C <path/to/guix.sigs> pull
env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify
```
## Common `guix-build` invocation patterns and examples
### Keeping caches and SDKs outside of the worktree
If you perform a lot of builds and have a bunch of worktrees, you may find it
more efficient to keep the depends tree's download cache, build cache, and SDKs
outside of the worktrees to avoid duplicate downloads and unnecessary builds. To
help with this situation, the `guix-build` script honours the `SOURCES_PATH`,
`BASE_CACHE`, and `SDK_PATH` environment variables and will pass them on to the
depends tree so that you can do something like:
```sh
env SOURCES_PATH="$HOME/depends-SOURCES_PATH" BASE_CACHE="$HOME/depends-BASE_CACHE" SDK_PATH="$HOME/macOS-SDKs" ./contrib/guix/guix-build
```
Note that the paths that these environment variables point to **must be
directories**, and **NOT symlinks to directories**.
See the [recognized environment variables][env-vars-list] section for more
details.
### Building a subset of platform triples
Sometimes you only want to build a subset of the supported platform triples, in
which case you can override the default list by setting the space-separated
`HOSTS` environment variable:
```sh
env HOSTS='x86_64-w64-mingw32 x86_64-apple-darwin18' ./contrib/guix/guix-build
```
See the [recognized environment variables][env-vars-list] section for more
details.
### Controlling the number of threads used by `guix` build commands
Depending on your system's RAM capacity, you may want to decrease the number of
threads used to decrease RAM usage or vice versa.
By default, the scripts under `./contrib/guix` will invoke all `guix` build
commands with `--cores="$JOBS"`. Note that `$JOBS` defaults to `$(nproc)` if not
specified. However, astute manual readers will also notice that `guix` build
commands also accept a `--max-jobs=` flag (which defaults to 1 if unspecified).
Here is the difference between `--cores=` and `--max-jobs=`:
> Note: When I say "derivation," think "package"
`--cores=`
- controls the number of CPU cores to build each derivation. This is the value
passed to `make`'s `--jobs=` flag.
`--max-jobs=`
- controls how many derivations can be built in parallel
- defaults to 1
Therefore, the default is for `guix` build commands to build one derivation at a
time, utilizing `$JOBS` threads.
Specifying the `$JOBS` environment variable will only modify `--cores=`, but you
can also modify the value for `--max-jobs=` by specifying
`$ADDITIONAL_GUIX_COMMON_FLAGS`. For example, if you have a LOT of memory, you
may want to set:
```sh
export ADDITIONAL_GUIX_COMMON_FLAGS='--max-jobs=8'
```
Which allows for a maximum of 8 derivations to be built at the same time, each
utilizing `$JOBS` threads.
Or, if you'd like to avoid spurious build failures caused by issues with
parallelism within a single package, but would still like to build multiple
packages when the dependency graph allows for it, you may want to try:
```sh
export JOBS=1 ADDITIONAL_GUIX_COMMON_FLAGS='--max-jobs=8'
```
See the [recognized environment variables][env-vars-list] section for more
details.
## Recognized environment variables
* _**HOSTS**_
Override the space-separated list of platform triples for which to perform a
bootstrappable build.
_(defaults to "x86\_64-linux-gnu arm-linux-gnueabihf aarch64-linux-gnu
riscv64-linux-gnu powerpc64-linux-gnu powerpc64le-linux-gnu
x86\_64-w64-mingw32 x86\_64-apple-darwin arm64-apple-darwin")_
* _**SOURCES_PATH**_
Set the depends tree download cache for sources. This is passed through to the
depends tree. Setting this to the same directory across multiple builds of the
depends tree can eliminate unnecessary redownloading of package sources.
The path that this environment variable points to **must be a directory**, and
**NOT a symlink to a directory**.
* _**BASE_CACHE**_
Set the depends tree cache for built packages. This is passed through to the
depends tree. Setting this to the same directory across multiple builds of the
depends tree can eliminate unnecessary building of packages.
The path that this environment variable points to **must be a directory**, and
**NOT a symlink to a directory**.
* _**SDK_PATH**_
Set the path where _extracted_ SDKs can be found. This is passed through to
the depends tree. Note that this is should be set to the _parent_ directory of
the actual SDK (e.g. `SDK_PATH=$HOME/Downloads/macOS-SDKs` instead of
`$HOME/Downloads/macOS-SDKs/Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers`).
The path that this environment variable points to **must be a directory**, and
**NOT a symlink to a directory**.
* _**JOBS**_
Override the number of jobs to run simultaneously, you might want to do so on
a memory-limited machine. This may be passed to:
- `guix` build commands as in `guix environment --cores="$JOBS"`
- `make` as in `make --jobs="$JOBS"`
- `xargs` as in `xargs -P"$JOBS"`
See [here](#controlling-the-number-of-threads-used-by-guix-build-commands) for
more details.
_(defaults to the value of `nproc` outside the container)_
* _**SOURCE_DATE_EPOCH**_
Override the reference UNIX timestamp used for bit-for-bit reproducibility,
the variable name conforms to [standard][r12e/source-date-epoch].
_(defaults to the output of `$(git log --format=%at -1)`)_
* _**V**_
If non-empty, will pass `V=1` to all `make` invocations, making `make` output
verbose.
Note that any given value is ignored. The variable is only checked for
emptiness. More concretely, this means that `V=` (setting `V` to the empty
string) is interpreted the same way as not setting `V` at all, and that `V=0`
has the same effect as `V=1`.
* _**SUBSTITUTE_URLS**_
A whitespace-delimited list of URLs from which to download pre-built packages.
A URL is only used if its signing key is authorized (refer to the [substitute
servers section](#option-1-building-with-substitutes) for more details).
* _**ADDITIONAL_GUIX_COMMON_FLAGS**_
Additional flags to be passed to all `guix` commands.
* _**ADDITIONAL_GUIX_TIMEMACHINE_FLAGS**_
Additional flags to be passed to `guix time-machine`.
* _**ADDITIONAL_GUIX_ENVIRONMENT_FLAGS**_
Additional flags to be passed to the invocation of `guix environment` inside
`guix time-machine`.
# Choosing your security model
No matter how you installed Guix, you need to decide on your security model for
building packages with Guix.
Guix allows us to achieve better binary security by using our CPU time to build
everything from scratch. However, it doesn't sacrifice user choice in pursuit of
this: users can decide whether or not to use **substitutes** (pre-built
packages).
## Option 1: Building with substitutes
### Step 1: Authorize the signing keys
Depending on the installation procedure you followed, you may have already
authorized the Guix build farm key. In particular, the official shell installer
script asks you if you want the key installed, and the debian distribution
package authorized the key during installation.
You can check the current list of authorized keys at `/etc/guix/acl`.
At the time of writing, a `/etc/guix/acl` with just the Guix build farm key
authorized looks something like:
```lisp
(acl
(entry
(public-key
(ecc
(curve Ed25519)
(q #8D156F295D24B0D9A86FA5741A840FF2D24F60F7B6C4134814AD55625971B394#)
)
)
(tag
(guix import)
)
)
)
```
If you've determined that the official Guix build farm key hasn't been
authorized, and you would like to authorize it, run the following as root:
```
guix archive --authorize < /var/guix/profiles/per-user/root/current-guix/share/guix/ci.guix.gnu.org.pub
```
If
`/var/guix/profiles/per-user/root/current-guix/share/guix/ci.guix.gnu.org.pub`
doesn't exist, try:
```sh
guix archive --authorize < <PREFIX>/share/guix/ci.guix.gnu.org.pub
```
Where `<PREFIX>` is likely:
- `/usr` if you installed from a distribution package
- `/usr/local` if you installed Guix from source and didn't supply any
prefix-modifying flags to Guix's `./configure`
For dongcarl's substitute server at https://guix.carldong.io, run as root:
```sh
wget -qO- 'https://guix.carldong.io/signing-key.pub' | guix archive --authorize
```
#### Removing authorized keys
To remove previously authorized keys, simply edit `/etc/guix/acl` and remove the
`(entry (public-key ...))` entry.
### Step 2: Specify the substitute servers
Once its key is authorized, the official Guix build farm at
https://ci.guix.gnu.org is automatically used unless the `--no-substitutes` flag
is supplied. This default list of substitute servers is overridable both on a
`guix-daemon` level and when you invoke `guix` commands. See examples below for
the various ways of adding dongcarl's substitute server after having [authorized
his signing key](#authorize-the-signing-keys).
Change the **default list** of substitute servers by starting `guix-daemon` with
the `--substitute-urls` option (you will likely need to edit your init script):
```sh
guix-daemon <cmd> --substitute-urls='https://guix.carldong.io https://ci.guix.gnu.org'
```
Override the default list of substitute servers by passing the
`--substitute-urls` option for invocations of `guix` commands:
```sh
guix <cmd> --substitute-urls='https://guix.carldong.io https://ci.guix.gnu.org'
```
For scripts under `./contrib/guix`, set the `SUBSTITUTE_URLS` environment
variable:
```sh
export SUBSTITUTE_URLS='https://guix.carldong.io https://ci.guix.gnu.org'
```
## Option 2: Disabling substitutes on an ad-hoc basis
If you prefer not to use any substitutes, make sure to supply `--no-substitutes`
like in the following snippet. The first build will take a while, but the
resulting packages will be cached for future builds.
For direct invocations of `guix`:
```sh
guix <cmd> --no-substitutes
```
For the scripts under `./contrib/guix/`:
```sh
export ADDITIONAL_GUIX_COMMON_FLAGS='--no-substitutes'
```
## Option 3: Disabling substitutes by default
`guix-daemon` accepts a `--no-substitutes` flag, which will make sure that,
unless otherwise overridden by a command line invocation, no substitutes will be
used.
If you start `guix-daemon` using an init script, you can edit said script to
supply this flag.
# Purging/Uninstalling Guix
In the extraordinarily rare case where you messed up your Guix installation in
an irreversible way, you may want to completely purge Guix from your system and
start over.
1. Uninstall Guix itself according to the way you installed it. (e.g. `sudo apt
purge guix` for Ubuntu packaging, `sudo make uninstall` for
built-from-source).
2. Remove all build users and groups
You may check for relevant users and groups using:
```
getent passwd | grep guix
getent group | grep guix
```
Then, you may remove users and groups using:
```
sudo userdel <user>
sudo groupdel <group>
```
3. Remove all possible Guix-related directories
- `/var/guix/`
- `/var/log/guix/`
- `/gnu/`
- `/etc/guix/`
- `/home/*/.config/guix/`
- `/home/*/.cache/guix/`
- `/home/*/.guix-profile/`
- `/root/.config/guix/`
- `/root/.cache/guix/`
- `/root/.guix-profile/`
[b17e]: http://bootstrappable.org/
[r12e/source-date-epoch]: https://reproducible-builds.org/docs/source-date-epoch/
[guix/install.sh]: https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
[guix/bin-install]: https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html
[guix/env-setup]: https://www.gnu.org/software/guix/manual/en/html_node/Build-Environment-Setup.html
[guix/substitutes]: https://www.gnu.org/software/guix/manual/en/html_node/Substitutes.html
[guix/substitute-server-auth]: https://www.gnu.org/software/guix/manual/en/html_node/Substitute-Server-Authorization.html
[guix/time-machine]: https://guix.gnu.org/manual/en/html_node/Invoking-guix-time_002dmachine.html
[debian/guix-bullseye]: https://packages.debian.org/bullseye/guix
[ubuntu/guix-hirsute]: https://packages.ubuntu.com/hirsute/guix
[fanquake/guix-docker]: https://github.com/fanquake/core-review/tree/master/guix
[env-vars-list]: #recognized-environment-variables

258
contrib/guix/guix-attest Executable file
View file

@ -0,0 +1,258 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
# Source the common prelude, which:
# 1. Checks if we're at the top directory of the DigiByte Core repository
# 2. Defines a few common functions and variables
#
# shellcheck source=libexec/prelude.bash
source "$(dirname "${BASH_SOURCE[0]}")/libexec/prelude.bash"
###################
## Sanity Checks ##
###################
################
# Required non-builtin commands should be invokable
################
check_tools cat env basename mkdir diff sort
if [ -z "$NO_SIGN" ]; then
check_tools gpg
fi
################
# Required env vars should be non-empty
################
cmd_usage() {
cat <<EOF
Synopsis:
env GUIX_SIGS_REPO=<path/to/guix.sigs> \\
SIGNER=GPG_KEY_NAME[=SIGNER_NAME] \\
[ NO_SIGN=1 ]
./contrib/guix/guix-attest
Example w/o overriding signing name:
env GUIX_SIGS_REPO=/home/achow101/guix.sigs \\
SIGNER=achow101 \\
./contrib/guix/guix-attest
Example overriding signing name:
env GUIX_SIGS_REPO=/home/dongcarl/guix.sigs \\
SIGNER=0x96AB007F1A7ED999=dongcarl \\
./contrib/guix/guix-attest
Example w/o signing, just creating SHA256SUMS:
env GUIX_SIGS_REPO=/home/achow101/guix.sigs \\
SIGNER=achow101 \\
NO_SIGN=1 \\
./contrib/guix/guix-attest
EOF
}
if [ -z "$GUIX_SIGS_REPO" ] || [ -z "$SIGNER" ]; then
cmd_usage
exit 1
fi
################
# GUIX_SIGS_REPO should exist as a directory
################
if [ ! -d "$GUIX_SIGS_REPO" ]; then
cat << EOF
ERR: The specified GUIX_SIGS_REPO is not an existent directory:
'$GUIX_SIGS_REPO'
Hint: Please clone the guix.sigs repository and point to it with the
GUIX_SIGS_REPO environment variable.
EOF
cmd_usage
exit 1
fi
################
# The key specified in SIGNER should be usable
################
IFS='=' read -r gpg_key_name signer_name <<< "$SIGNER"
if [ -z "${signer_name}" ]; then
signer_name="$gpg_key_name"
fi
if [ -z "$NO_SIGN" ] && ! gpg --dry-run --list-secret-keys "${gpg_key_name}" >/dev/null 2>&1; then
echo "ERR: GPG can't seem to find any key named '${gpg_key_name}'"
exit 1
fi
################
# We should be able to find at least one output
################
echo "Looking for build output SHA256SUMS fragments in ${OUTDIR_BASE}"
shopt -s nullglob
sha256sum_fragments=( "$OUTDIR_BASE"/*/SHA256SUMS.part ) # This expands to an array of directories...
shopt -u nullglob
noncodesigned_fragments=()
codesigned_fragments=()
if (( ${#sha256sum_fragments[@]} )); then
echo "Found build output SHA256SUMS fragments:"
for outdir in "${sha256sum_fragments[@]}"; do
echo " '$outdir'"
case "$outdir" in
"$OUTDIR_BASE"/*-codesigned/SHA256SUMS.part)
codesigned_fragments+=("$outdir")
;;
*)
noncodesigned_fragments+=("$outdir")
;;
esac
done
echo
else
echo "ERR: Could not find any build output SHA256SUMS fragments in ${OUTDIR_BASE}"
exit 1
fi
##############
## Attest ##
##############
# Usage: out_name $outdir
#
# HOST: The output directory being attested
#
out_name() {
basename "$(dirname "$1")"
}
shasum_already_exists() {
cat <<EOF
--
ERR: An ${1} file already exists for '${VERSION}' and attests
differently. You likely previously attested to a partial build (e.g. one
where you specified the HOST environment variable).
See the diff above for more context.
Hint: You may wish to remove the existing attestations and their signatures by
invoking:
rm '${PWD}/${1}'{,.asc}
Then try running this script again.
EOF
}
# Given a document with unix line endings (just <LF>) in stdin, make all lines
# end in <CR><LF> and make sure there's no trailing <LF> at the end of the file.
#
# This is necessary as cleartext signatures are calculated on text after their
# line endings are canonicalized.
#
# For more information:
# 1. https://security.stackexchange.com/a/104261
# 2. https://datatracker.ietf.org/doc/html/rfc4880#section-7.1
#
rfc4880_normalize_document() {
sed 's/$/\r/' | head -c -2
}
echo "Attesting to build outputs for version: '${VERSION}'"
echo ""
outsigdir="$GUIX_SIGS_REPO/$VERSION/$signer_name"
mkdir -p "$outsigdir"
(
cd "$outsigdir"
temp_noncodesigned="$(mktemp)"
trap 'rm -rf -- "$temp_noncodesigned"' EXIT
if (( ${#noncodesigned_fragments[@]} )); then
cat "${noncodesigned_fragments[@]}" \
| sort -u \
| sort -k2 \
| rfc4880_normalize_document \
> "$temp_noncodesigned"
if [ -e noncodesigned.SHA256SUMS ]; then
# The SHA256SUMS already exists, make sure it's exactly what we
# expect, error out if not
if diff -u noncodesigned.SHA256SUMS "$temp_noncodesigned"; then
echo "A noncodesigned.SHA256SUMS file already exists for '${VERSION}' and is up-to-date."
else
shasum_already_exists noncodesigned.SHA256SUMS
exit 1
fi
else
mv "$temp_noncodesigned" noncodesigned.SHA256SUMS
fi
else
echo "ERR: No noncodesigned outputs found for '${VERSION}', exiting..."
exit 1
fi
temp_codesigned="$(mktemp)"
trap 'rm -rf -- "$temp_codesigned"' EXIT
if (( ${#codesigned_fragments[@]} )); then
# Note: all.SHA256SUMS attests to all of $sha256sum_fragments, but is
# not needed if there are no $codesigned_fragments
cat "${sha256sum_fragments[@]}" \
| sort -u \
| sort -k2 \
| sed 's/$/\r/' \
| rfc4880_normalize_document \
> "$temp_codesigned"
if [ -e codesigned.SHA256SUMS ]; then
# The SHA256SUMS already exists, make sure it's exactly what we
# expect, error out if not
if diff -u all.SHA256SUMS "$temp_codesigned"; then
echo "An all.SHA256SUMS file already exists for '${VERSION}' and is up-to-date."
else
shasum_already_exists all.SHA256SUMS
exit 1
fi
else
mv "$temp_codesigned" codesigned.SHA256SUMS
fi
else
# It is fine to have the codesigned outputs be missing (perhaps the
# detached codesigs have not been published yet), just print a log
# message instead of erroring out
echo "INFO: No codesigned outputs found for '${VERSION}', skipping..."
fi
if [ -z "$NO_SIGN" ]; then
echo "Signing SHA256SUMS to produce SHA256SUMS.asc"
for i in *.SHA256SUMS; do
if [ ! -e "$i".asc ]; then
gpg --detach-sign \
--digest-algo sha256 \
--local-user "$gpg_key_name" \
--armor \
--output "$i".asc "$i"
else
echo "Signature already there"
fi
done
else
echo "Not signing SHA256SUMS as \$NO_SIGN is not empty"
fi
echo ""
)

477
contrib/guix/guix-build Executable file
View file

@ -0,0 +1,477 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
# Source the common prelude, which:
# 1. Checks if we're at the top directory of the DigiByte Core repository
# 2. Defines a few common functions and variables
#
# shellcheck source=libexec/prelude.bash
source "$(dirname "${BASH_SOURCE[0]}")/libexec/prelude.bash"
###################
## SANITY CHECKS ##
###################
################
# Required non-builtin commands should be invocable
################
check_tools cat mkdir make getent curl git guix
################
# GUIX_BUILD_OPTIONS should be empty
################
#
# GUIX_BUILD_OPTIONS is an environment variable recognized by guix commands that
# can perform builds. This seems like what we want instead of
# ADDITIONAL_GUIX_COMMON_FLAGS, but the value of GUIX_BUILD_OPTIONS is actually
# _appended_ to normal command-line options. Meaning that they will take
# precedence over the command-specific ADDITIONAL_GUIX_<CMD>_FLAGS.
#
# This seems like a poor user experience. Thus we check for GUIX_BUILD_OPTIONS's
# existence here and direct users of this script to use our (more flexible)
# custom environment variables.
if [ -n "$GUIX_BUILD_OPTIONS" ]; then
cat << EOF
Error: Environment variable GUIX_BUILD_OPTIONS is not empty:
'$GUIX_BUILD_OPTIONS'
Unfortunately this script is incompatible with GUIX_BUILD_OPTIONS, please unset
GUIX_BUILD_OPTIONS and use ADDITIONAL_GUIX_COMMON_FLAGS to set build options
across guix commands or ADDITIONAL_GUIX_<CMD>_FLAGS to set build options for a
specific guix command.
See contrib/guix/README.md for more details.
EOF
exit 1
fi
################
# The git worktree should not be dirty
################
if ! git diff-index --quiet HEAD -- && [ -z "$FORCE_DIRTY_WORKTREE" ]; then
cat << EOF
ERR: The current git worktree is dirty, which may lead to broken builds.
Aborting...
Hint: To make your git worktree clean, You may want to:
1. Commit your changes,
2. Stash your changes, or
3. Set the 'FORCE_DIRTY_WORKTREE' environment variable if you insist on
using a dirty worktree
EOF
exit 1
fi
mkdir -p "$VERSION_BASE"
################
# Build directories should not exist
################
# Default to building for all supported HOSTs (overridable by environment)
export HOSTS="${HOSTS:-x86_64-linux-gnu arm-linux-gnueabihf aarch64-linux-gnu riscv64-linux-gnu powerpc64-linux-gnu powerpc64le-linux-gnu
x86_64-w64-mingw32
x86_64-apple-darwin arm64-apple-darwin}"
# Usage: distsrc_for_host HOST
#
# HOST: The current platform triple we're building for
#
distsrc_for_host() {
echo "${DISTSRC_BASE}/distsrc-${VERSION}-${1}"
}
# Accumulate a list of build directories that already exist...
hosts_distsrc_exists=""
for host in $HOSTS; do
if [ -e "$(distsrc_for_host "$host")" ]; then
hosts_distsrc_exists+=" ${host}"
fi
done
if [ -n "$hosts_distsrc_exists" ]; then
# ...so that we can print them out nicely in an error message
cat << EOF
ERR: Build directories for this commit already exist for the following platform
triples you're attempting to build, probably because of previous builds.
Please remove, or otherwise deal with them prior to starting another build.
Aborting...
Hint: To blow everything away, you may want to use:
$ ./contrib/guix/guix-clean
Specifically, this will remove all files without an entry in the index,
excluding the SDK directory, the depends download cache, the depends built
packages cache, the garbage collector roots for Guix environments, and the
output directory.
EOF
for host in $hosts_distsrc_exists; do
echo " ${host} '$(distsrc_for_host "$host")'"
done
exit 1
else
mkdir -p "$DISTSRC_BASE"
fi
################
# When building for darwin, the macOS SDK should exists
################
for host in $HOSTS; do
case "$host" in
*darwin*)
OSX_SDK="$(make -C "${PWD}/depends" --no-print-directory HOST="$host" print-OSX_SDK | sed 's@^[^=]\+=@@g')"
if [ -e "$OSX_SDK" ]; then
echo "Found macOS SDK at '${OSX_SDK}', using..."
else
echo "macOS SDK does not exist at '${OSX_SDK}', please place the extracted, untarred SDK there to perform darwin builds, exiting..."
exit 1
fi
;;
esac
done
################
# VERSION_BASE should have enough space
################
avail_KiB="$(df -Pk "$VERSION_BASE" | sed 1d | tr -s ' ' | cut -d' ' -f4)"
total_required_KiB=0
for host in $HOSTS; do
case "$host" in
*darwin*) required_KiB=440000 ;;
*mingw*) required_KiB=7600000 ;;
*) required_KiB=6400000 ;;
esac
total_required_KiB=$((total_required_KiB+required_KiB))
done
if (( total_required_KiB > avail_KiB )); then
total_required_GiB=$((total_required_KiB / 1048576))
avail_GiB=$((avail_KiB / 1048576))
echo "Performing a DigiByte Core Guix build for the selected HOSTS requires ${total_required_GiB} GiB, however, only ${avail_GiB} GiB is available. Please free up some disk space before performing the build."
exit 1
fi
################
# Check that we can connect to the guix-daemon
################
cat << EOF
Checking that we can connect to the guix-daemon...
Hint: If this hangs, you may want to try turning your guix-daemon off and on
again.
EOF
if ! guix gc --list-failures > /dev/null; then
cat << EOF
ERR: Failed to connect to the guix-daemon, please ensure that one is running and
reachable.
EOF
exit 1
fi
# Developer note: we could use `guix repl` for this check and run:
#
# (import (guix store)) (close-connection (open-connection))
#
# However, the internal API is likely to change more than the CLI invocation
################
# Services database must have basic entries
################
if ! getent services http https ftp; then
cat << EOF
ERR: Your system's C library can not find service database entries for at least
one of the following services: http, https, ftp.
Hint: Most likely, /etc/services does not exist yet (common for docker images
and minimal distros), or you don't have permissions to access it.
If /etc/services does not exist yet, you may want to install the
appropriate package for your distro which provides it.
On Debian/Ubuntu: netbase
On Arch Linux: iana-etc
For more information, see: getent(1), services(5)
EOF
fi
#########
# SETUP #
#########
# Determine the maximum number of jobs to run simultaneously (overridable by
# environment)
JOBS="${JOBS:-$(nproc)}"
# Usage: host_to_commonname HOST
#
# HOST: The current platform triple we're building for
#
host_to_commonname() {
case "$1" in
*darwin*) echo osx ;;
*mingw*) echo win ;;
*linux*) echo linux ;;
*) exit 1 ;;
esac
}
# Determine the reference time used for determinism (overridable by environment)
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(git log --format=%at -1)}"
# Execute "$@" in a pinned, possibly older version of Guix, for reproducibility
# across time.
time-machine() {
# shellcheck disable=SC2086
guix time-machine --url=https://git.savannah.gnu.org/git/guix.git \
--commit=aa34d4d28dfe25ba47d5800d05000fb7221788c0 \
--cores="$JOBS" \
--keep-failed \
--fallback \
${SUBSTITUTE_URLS:+--substitute-urls="$SUBSTITUTE_URLS"} \
${ADDITIONAL_GUIX_COMMON_FLAGS} ${ADDITIONAL_GUIX_TIMEMACHINE_FLAGS} \
-- "$@"
}
# Precious directories are those which should not be cleaned between successive
# guix builds
depends_precious_dir_names='SOURCES_PATH BASE_CACHE SDK_PATH'
precious_dir_names="${depends_precious_dir_names} OUTDIR_BASE PROFILES_BASE"
# Usage: contains IFS-SEPARATED-LIST ITEM
contains() {
for i in ${1}; do
if [ "$i" = "${2}" ]; then
return 0 # Found!
fi
done
return 1
}
# If the user explicitly specified a precious directory, create it so we
# can map it into the container
for precious_dir_name in $precious_dir_names; do
precious_dir_path="${!precious_dir_name}"
if [ -n "$precious_dir_path" ]; then
if [ ! -e "$precious_dir_path" ]; then
mkdir -p "$precious_dir_path"
elif [ -L "$precious_dir_path" ]; then
echo "ERR: ${precious_dir_name} cannot be a symbolic link"
exit 1
elif [ ! -d "$precious_dir_path" ]; then
echo "ERR: ${precious_dir_name} must be a directory"
exit 1
fi
fi
done
mkdir -p "$VAR_BASE"
# Record the _effective_ values of precious directories such that guix-clean can
# avoid clobbering them if appropriate.
#
# shellcheck disable=SC2046,SC2086
{
# Get depends precious dir definitions from depends
make -C "${PWD}/depends" \
--no-print-directory \
-- $(printf "print-%s\n" $depends_precious_dir_names)
# Get remaining precious dir definitions from the environment
for precious_dir_name in $precious_dir_names; do
precious_dir_path="${!precious_dir_name}"
if ! contains "$depends_precious_dir_names" "$precious_dir_name"; then
echo "${precious_dir_name}=${precious_dir_path}"
fi
done
} > "${VAR_BASE}/precious_dirs"
# Make sure an output directory exists for our builds
OUTDIR_BASE="${OUTDIR_BASE:-${VERSION_BASE}/output}"
mkdir -p "$OUTDIR_BASE"
# Download the depends sources now as we won't have internet access in the build
# container
for host in $HOSTS; do
make -C "${PWD}/depends" -j"$JOBS" download-"$(host_to_commonname "$host")" ${V:+V=1} ${SOURCES_PATH:+SOURCES_PATH="$SOURCES_PATH"}
done
# Usage: outdir_for_host HOST SUFFIX
#
# HOST: The current platform triple we're building for
#
outdir_for_host() {
echo "${OUTDIR_BASE}/${1}${2:+-${2}}"
}
# Usage: profiledir_for_host HOST SUFFIX
#
# HOST: The current platform triple we're building for
#
profiledir_for_host() {
echo "${PROFILES_BASE}/${1}${2:+-${2}}"
}
#########
# BUILD #
#########
# Function to be called when building for host ${1} and the user interrupts the
# build
int_trap() {
cat << EOF
** INT received while building ${1}, you may want to clean up the relevant
work directories (e.g. distsrc-*) before rebuilding
Hint: To blow everything away, you may want to use:
$ ./contrib/guix/guix-clean
Specifically, this will remove all files without an entry in the index,
excluding the SDK directory, the depends download cache, the depends built
packages cache, the garbage collector roots for Guix environments, and the
output directory.
EOF
}
# Deterministically build DigiByte Core
# shellcheck disable=SC2153
for host in $HOSTS; do
# Display proper warning when the user interrupts the build
trap 'int_trap ${host}' INT
(
# Required for 'contrib/guix/manifest.scm' to output the right manifest
# for the particular $HOST we're building for
export HOST="$host"
# shellcheck disable=SC2030
cat << EOF
INFO: Building ${VERSION:?not set} for platform triple ${HOST:?not set}:
...using reference timestamp: ${SOURCE_DATE_EPOCH:?not set}
...running at most ${JOBS:?not set} jobs
...from worktree directory: '${PWD}'
...bind-mounted in container to: '/digibyte'
...in build directory: '$(distsrc_for_host "$HOST")'
...bind-mounted in container to: '$(DISTSRC_BASE=/distsrc-base && distsrc_for_host "$HOST")'
...outputting in: '$(outdir_for_host "$HOST")'
...bind-mounted in container to: '$(OUTDIR_BASE=/outdir-base && outdir_for_host "$HOST")'
EOF
# Run the build script 'contrib/guix/libexec/build.sh' in the build
# container specified by 'contrib/guix/manifest.scm'.
#
# Explanation of `guix environment` flags:
#
# --container run command within an isolated container
#
# Running in an isolated container minimizes build-time differences
# between machines and improves reproducibility
#
# --pure unset existing environment variables
#
# Same rationale as --container
#
# --no-cwd do not share current working directory with an
# isolated container
#
# When --container is specified, the default behavior is to share
# the current working directory with the isolated container at the
# same exact path (e.g. mapping '/home/satoshi/digibyte/' to
# '/home/satoshi/digibyte/'). This means that the $PWD inside the
# container becomes a source of irreproducibility. --no-cwd disables
# this behaviour.
#
# --share=SPEC for containers, share writable host file system
# according to SPEC
#
# --share="$PWD"=/digibyte
#
# maps our current working directory to /digibyte
# inside the isolated container, which we later cd
# into.
#
# While we don't want to map our current working directory to the
# same exact path (as this introduces irreproducibility), we do want
# it to be at a _fixed_ path _somewhere_ inside the isolated
# container so that we have something to build. '/digibyte' was
# chosen arbitrarily.
#
# ${SOURCES_PATH:+--share="$SOURCES_PATH"}
#
# make the downloaded depends sources path available
# inside the isolated container
#
# The isolated container has no network access as it's in a
# different network namespace from the main machine, so we have to
# make the downloaded depends sources available to it. The sources
# should have been downloaded prior to this invocation.
#
# --keep-failed keep build tree of failed builds
#
# When builds of the Guix environment itself (not DigiByte Core)
# fail, it is useful for the build tree to be kept for debugging
# purposes.
#
# ${SUBSTITUTE_URLS:+--substitute-urls="$SUBSTITUTE_URLS"}
#
# fetch substitute from SUBSTITUTE_URLS if they are
# authorized
#
# Depending on the user's security model, it may be desirable to use
# substitutes (pre-built packages) from servers that the user trusts.
# Please read the README.md in the same directory as this file for
# more information.
#
# shellcheck disable=SC2086,SC2031
time-machine environment --manifest="${PWD}/contrib/guix/manifest.scm" \
--container \
--pure \
--no-cwd \
--share="$PWD"=/digibyte \
--share="$DISTSRC_BASE"=/distsrc-base \
--share="$OUTDIR_BASE"=/outdir-base \
--expose="$(git rev-parse --git-common-dir)" \
${SOURCES_PATH:+--share="$SOURCES_PATH"} \
${BASE_CACHE:+--share="$BASE_CACHE"} \
${SDK_PATH:+--share="$SDK_PATH"} \
--cores="$JOBS" \
--keep-failed \
--fallback \
--link-profile \
--root="$(profiledir_for_host "${HOST}")" \
${SUBSTITUTE_URLS:+--substitute-urls="$SUBSTITUTE_URLS"} \
${ADDITIONAL_GUIX_COMMON_FLAGS} ${ADDITIONAL_GUIX_ENVIRONMENT_FLAGS} \
-- env HOST="$host" \
DISTNAME="$DISTNAME" \
JOBS="$JOBS" \
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:?unable to determine value}" \
${V:+V=1} \
${SOURCES_PATH:+SOURCES_PATH="$SOURCES_PATH"} \
${BASE_CACHE:+BASE_CACHE="$BASE_CACHE"} \
${SDK_PATH:+SDK_PATH="$SDK_PATH"} \
DISTSRC="$(DISTSRC_BASE=/distsrc-base && distsrc_for_host "$HOST")" \
OUTDIR="$(OUTDIR_BASE=/outdir-base && outdir_for_host "$HOST")" \
DIST_ARCHIVE_BASE=/outdir-base/dist-archive \
bash -c "cd /digibyte && bash contrib/guix/libexec/build.sh"
)
done

83
contrib/guix/guix-clean Executable file
View file

@ -0,0 +1,83 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
# Source the common prelude, which:
# 1. Checks if we're at the top directory of the DigiByte Core repository
# 2. Defines a few common functions and variables
#
# shellcheck source=libexec/prelude.bash
source "$(dirname "${BASH_SOURCE[0]}")/libexec/prelude.bash"
###################
## Sanity Checks ##
###################
################
# Required non-builtin commands should be invokable
################
check_tools cat mkdir make git guix
#############
## Clean ##
#############
# Usage: under_dir MAYBE_PARENT MAYBE_CHILD
#
# If MAYBE_CHILD is a subdirectory of MAYBE_PARENT, print the relative path
# from MAYBE_PARENT to MAYBE_CHILD. Otherwise, return 1 as the error code.
#
# NOTE: This does not perform any symlink-resolving or path canonicalization.
#
under_dir() {
local path_residue
path_residue="${2##${1}}"
if [ -z "$path_residue" ] || [ "$path_residue" = "$2" ]; then
return 1
else
echo "$path_residue"
fi
}
# Usage: dir_under_git_root MAYBE_CHILD
#
# If MAYBE_CHILD is under the current git repository and exists, print the
# relative path from the git repository's top-level directory to MAYBE_CHILD,
# otherwise, exit with an error code.
#
dir_under_git_root() {
local rv
rv="$(under_dir "$(git_root)" "$1")"
[ -n "$rv" ] && echo "$rv"
}
shopt -s nullglob
found_precious_dirs_files=( "${version_base_prefix}"*/"${var_base_basename}/precious_dirs" ) # This expands to an array of directories...
shopt -u nullglob
exclude_flags=()
for precious_dirs_file in "${found_precious_dirs_files[@]}"; do
# Make sure the precious directories (e.g. SOURCES_PATH, BASE_CACHE, SDK_PATH)
# are excluded from git-clean
echo "Found precious_dirs file: '${precious_dirs_file}'"
# Exclude the precious_dirs file itself
if dirs_file_exclude_fragment=$(dir_under_git_root "$(dirname "$precious_dirs_file")"); then
exclude_flags+=( --exclude="${dirs_file_exclude_fragment}/precious_dirs" )
fi
# Read each 'name=dir' pair from the precious_dirs file
while IFS='=' read -r name dir; do
# Add an exclusion flag if the precious directory is under the git root.
if under=$(dir_under_git_root "$dir"); then
echo "Avoiding ${name}: ${under}"
exclude_flags+=( --exclude="$under" )
fi
done < "$precious_dirs_file"
done
git clean -xdff "${exclude_flags[@]}"

392
contrib/guix/guix-codesign Executable file
View file

@ -0,0 +1,392 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
# Source the common prelude, which:
# 1. Checks if we're at the top directory of the DigiByte Core repository
# 2. Defines a few common functions and variables
#
# shellcheck source=libexec/prelude.bash
source "$(dirname "${BASH_SOURCE[0]}")/libexec/prelude.bash"
###################
## SANITY CHECKS ##
###################
################
# Required non-builtin commands should be invocable
################
check_tools cat mkdir git guix
################
# Required env vars should be non-empty
################
cmd_usage() {
cat <<EOF
Synopsis:
env DETACHED_SIGS_REPO=<path/to/digibyte-detached-sigs> \\
./contrib/guix/guix-codesign
EOF
}
if [ -z "$DETACHED_SIGS_REPO" ]; then
cmd_usage
exit 1
fi
################
# GUIX_BUILD_OPTIONS should be empty
################
#
# GUIX_BUILD_OPTIONS is an environment variable recognized by guix commands that
# can perform builds. This seems like what we want instead of
# ADDITIONAL_GUIX_COMMON_FLAGS, but the value of GUIX_BUILD_OPTIONS is actually
# _appended_ to normal command-line options. Meaning that they will take
# precedence over the command-specific ADDITIONAL_GUIX_<CMD>_FLAGS.
#
# This seems like a poor user experience. Thus we check for GUIX_BUILD_OPTIONS's
# existence here and direct users of this script to use our (more flexible)
# custom environment variables.
if [ -n "$GUIX_BUILD_OPTIONS" ]; then
cat << EOF
Error: Environment variable GUIX_BUILD_OPTIONS is not empty:
'$GUIX_BUILD_OPTIONS'
Unfortunately this script is incompatible with GUIX_BUILD_OPTIONS, please unset
GUIX_BUILD_OPTIONS and use ADDITIONAL_GUIX_COMMON_FLAGS to set build options
across guix commands or ADDITIONAL_GUIX_<CMD>_FLAGS to set build options for a
specific guix command.
See contrib/guix/README.md for more details.
EOF
exit 1
fi
################
# The codesignature git worktree should not be dirty
################
if ! git -C "$DETACHED_SIGS_REPO" diff-index --quiet HEAD -- && [ -z "$FORCE_DIRTY_WORKTREE" ]; then
cat << EOF
ERR: The DETACHED CODESIGNATURE git worktree is dirty, which may lead to broken builds.
Aborting...
Hint: To make your git worktree clean, You may want to:
1. Commit your changes,
2. Stash your changes, or
3. Set the 'FORCE_DIRTY_WORKTREE' environment variable if you insist on
using a dirty worktree
EOF
exit 1
fi
################
# Build directories should not exist
################
# Default to building for all supported HOSTs (overridable by environment)
export HOSTS="${HOSTS:-x86_64-w64-mingw32 x86_64-apple-darwin arm64-apple-darwin}"
# Usage: distsrc_for_host HOST
#
# HOST: The current platform triple we're building for
#
distsrc_for_host() {
echo "${DISTSRC_BASE}/distsrc-${VERSION}-${1}-codesigned"
}
# Accumulate a list of build directories that already exist...
hosts_distsrc_exists=""
for host in $HOSTS; do
if [ -e "$(distsrc_for_host "$host")" ]; then
hosts_distsrc_exists+=" ${host}"
fi
done
if [ -n "$hosts_distsrc_exists" ]; then
# ...so that we can print them out nicely in an error message
cat << EOF
ERR: Build directories for this commit already exist for the following platform
triples you're attempting to build, probably because of previous builds.
Please remove, or otherwise deal with them prior to starting another build.
Aborting...
Hint: To blow everything away, you may want to use:
$ ./contrib/guix/guix-clean
Specifically, this will remove all files without an entry in the index,
excluding the SDK directory, the depends download cache, the depends built
packages cache, the garbage collector roots for Guix environments, and the
output directory.
EOF
for host in $hosts_distsrc_exists; do
echo " ${host} '$(distsrc_for_host "$host")'"
done
exit 1
else
mkdir -p "$DISTSRC_BASE"
fi
################
# Unsigned tarballs SHOULD exist
################
# Usage: outdir_for_host HOST SUFFIX
#
# HOST: The current platform triple we're building for
#
outdir_for_host() {
echo "${OUTDIR_BASE}/${1}${2:+-${2}}"
}
unsigned_tarball_for_host() {
case "$1" in
*mingw*)
echo "$(outdir_for_host "$1")/${DISTNAME}-win-unsigned.tar.gz"
;;
*darwin*)
echo "$(outdir_for_host "$1")/${DISTNAME}-osx-unsigned.tar.gz"
;;
*)
exit 1
;;
esac
}
# Accumulate a list of build directories that already exist...
hosts_unsigned_tarball_missing=""
for host in $HOSTS; do
if [ ! -e "$(unsigned_tarball_for_host "$host")" ]; then
hosts_unsigned_tarball_missing+=" ${host}"
fi
done
if [ -n "$hosts_unsigned_tarball_missing" ]; then
# ...so that we can print them out nicely in an error message
cat << EOF
ERR: Unsigned tarballs do not exist
...
EOF
for host in $hosts_unsigned_tarball_missing; do
echo " ${host} '$(unsigned_tarball_for_host "$host")'"
done
exit 1
fi
################
# Check that we can connect to the guix-daemon
################
cat << EOF
Checking that we can connect to the guix-daemon...
Hint: If this hangs, you may want to try turning your guix-daemon off and on
again.
EOF
if ! guix gc --list-failures > /dev/null; then
cat << EOF
ERR: Failed to connect to the guix-daemon, please ensure that one is running and
reachable.
EOF
exit 1
fi
# Developer note: we could use `guix repl` for this check and run:
#
# (import (guix store)) (close-connection (open-connection))
#
# However, the internal API is likely to change more than the CLI invocation
#########
# SETUP #
#########
# Determine the maximum number of jobs to run simultaneously (overridable by
# environment)
JOBS="${JOBS:-$(nproc)}"
# Determine the reference time used for determinism (overridable by environment)
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(git log --format=%at -1)}"
# Execute "$@" in a pinned, possibly older version of Guix, for reproducibility
# across time.
time-machine() {
# shellcheck disable=SC2086
guix time-machine --url=https://git.savannah.gnu.org/git/guix.git \
--commit=aa34d4d28dfe25ba47d5800d05000fb7221788c0 \
--cores="$JOBS" \
--keep-failed \
--fallback \
${SUBSTITUTE_URLS:+--substitute-urls="$SUBSTITUTE_URLS"} \
${ADDITIONAL_GUIX_COMMON_FLAGS} ${ADDITIONAL_GUIX_TIMEMACHINE_FLAGS} \
-- "$@"
}
# Make sure an output directory exists for our builds
OUTDIR_BASE="${OUTDIR_BASE:-${VERSION_BASE}/output}"
mkdir -p "$OUTDIR_BASE"
# Usage: profiledir_for_host HOST SUFFIX
#
# HOST: The current platform triple we're building for
#
profiledir_for_host() {
echo "${PROFILES_BASE}/${1}${2:+-${2}}"
}
#########
# BUILD #
#########
# Function to be called when codesigning for host ${1} and the user interrupts
# the codesign
int_trap() {
cat << EOF
** INT received while codesigning ${1}, you may want to clean up the relevant
work directories (e.g. distsrc-*) before recodesigning
Hint: To blow everything away, you may want to use:
$ ./contrib/guix/guix-clean
Specifically, this will remove all files without an entry in the index,
excluding the SDK directory, the depends download cache, the depends built
packages cache, the garbage collector roots for Guix environments, and the
output directory.
EOF
}
# Deterministically build DigiByte Core
# shellcheck disable=SC2153
for host in $HOSTS; do
# Display proper warning when the user interrupts the build
trap 'int_trap ${host}' INT
(
# Required for 'contrib/guix/manifest.scm' to output the right manifest
# for the particular $HOST we're building for
export HOST="$host"
# shellcheck disable=SC2030
cat << EOF
INFO: Codesigning ${VERSION:?not set} for platform triple ${HOST:?not set}:
...using reference timestamp: ${SOURCE_DATE_EPOCH:?not set}
...from worktree directory: '${PWD}'
...bind-mounted in container to: '/digibyte'
...in build directory: '$(distsrc_for_host "$HOST")'
...bind-mounted in container to: '$(DISTSRC_BASE=/distsrc-base && distsrc_for_host "$HOST")'
...outputting in: '$(outdir_for_host "$HOST" codesigned)'
...bind-mounted in container to: '$(OUTDIR_BASE=/outdir-base && outdir_for_host "$HOST" codesigned)'
...using detached signatures in: '${DETACHED_SIGS_REPO:?not set}'
...bind-mounted in container to: '/detached-sigs'
EOF
# Run the build script 'contrib/guix/libexec/build.sh' in the build
# container specified by 'contrib/guix/manifest.scm'.
#
# Explanation of `guix environment` flags:
#
# --container run command within an isolated container
#
# Running in an isolated container minimizes build-time differences
# between machines and improves reproducibility
#
# --pure unset existing environment variables
#
# Same rationale as --container
#
# --no-cwd do not share current working directory with an
# isolated container
#
# When --container is specified, the default behavior is to share
# the current working directory with the isolated container at the
# same exact path (e.g. mapping '/home/satoshi/digibyte/' to
# '/home/satoshi/digibyte/'). This means that the $PWD inside the
# container becomes a source of irreproducibility. --no-cwd disables
# this behaviour.
#
# --share=SPEC for containers, share writable host file system
# according to SPEC
#
# --share="$PWD"=/digibyte
#
# maps our current working directory to /digibyte
# inside the isolated container, which we later cd
# into.
#
# While we don't want to map our current working directory to the
# same exact path (as this introduces irreproducibility), we do want
# it to be at a _fixed_ path _somewhere_ inside the isolated
# container so that we have something to build. '/digibyte' was
# chosen arbitrarily.
#
# ${SOURCES_PATH:+--share="$SOURCES_PATH"}
#
# make the downloaded depends sources path available
# inside the isolated container
#
# The isolated container has no network access as it's in a
# different network namespace from the main machine, so we have to
# make the downloaded depends sources available to it. The sources
# should have been downloaded prior to this invocation.
#
# ${SUBSTITUTE_URLS:+--substitute-urls="$SUBSTITUTE_URLS"}
#
# fetch substitute from SUBSTITUTE_URLS if they are
# authorized
#
# Depending on the user's security model, it may be desirable to use
# substitutes (pre-built packages) from servers that the user trusts.
# Please read the README.md in the same directory as this file for
# more information.
#
# shellcheck disable=SC2086,SC2031
time-machine environment --manifest="${PWD}/contrib/guix/manifest.scm" \
--container \
--pure \
--no-cwd \
--share="$PWD"=/digibyte \
--share="$DISTSRC_BASE"=/distsrc-base \
--share="$OUTDIR_BASE"=/outdir-base \
--share="$DETACHED_SIGS_REPO"=/detached-sigs \
--expose="$(git rev-parse --git-common-dir)" \
--expose="$(git -C "$DETACHED_SIGS_REPO" rev-parse --git-common-dir)" \
${SOURCES_PATH:+--share="$SOURCES_PATH"} \
--cores="$JOBS" \
--keep-failed \
--fallback \
--link-profile \
--root="$(profiledir_for_host "${HOST}" codesigned)" \
${SUBSTITUTE_URLS:+--substitute-urls="$SUBSTITUTE_URLS"} \
${ADDITIONAL_GUIX_COMMON_FLAGS} ${ADDITIONAL_GUIX_ENVIRONMENT_FLAGS} \
-- env HOST="$host" \
DISTNAME="$DISTNAME" \
JOBS="$JOBS" \
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:?unable to determine value}" \
${V:+V=1} \
${SOURCES_PATH:+SOURCES_PATH="$SOURCES_PATH"} \
DISTSRC="$(DISTSRC_BASE=/distsrc-base && distsrc_for_host "$HOST")" \
OUTDIR="$(OUTDIR_BASE=/outdir-base && outdir_for_host "$HOST" codesigned)" \
DIST_ARCHIVE_BASE=/outdir-base/dist-archive \
DETACHED_SIGS_REPO=/detached-sigs \
UNSIGNED_TARBALL="$(OUTDIR_BASE=/outdir-base && unsigned_tarball_for_host "$HOST")" \
bash -c "cd /digibyte && bash contrib/guix/libexec/codesign.sh"
)
done

142
contrib/guix/guix-verify Executable file
View file

@ -0,0 +1,142 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
# Source the common prelude, which:
# 1. Checks if we're at the top directory of the DigiByte Core repository
# 2. Defines a few common functions and variables
#
# shellcheck source=libexec/prelude.bash
source "$(dirname "${BASH_SOURCE[0]}")/libexec/prelude.bash"
###################
## Sanity Checks ##
###################
################
# Required non-builtin commands should be invokable
################
check_tools cat diff gpg
################
# Required env vars should be non-empty
################
cmd_usage() {
cat <<EOF
Synopsis:
env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify
EOF
}
if [ -z "$GUIX_SIGS_REPO" ]; then
cmd_usage
exit 1
fi
################
# GUIX_SIGS_REPO should exist as a directory
################
if [ ! -d "$GUIX_SIGS_REPO" ]; then
cat << EOF
ERR: The specified GUIX_SIGS_REPO is not an existent directory:
'$GUIX_SIGS_REPO'
Hint: Please clone the guix.sigs repository and point to it with the
GUIX_SIGS_REPO environment variable.
EOF
cmd_usage
exit 1
fi
##############
## Verify ##
##############
OUTSIGDIR_BASE="${GUIX_SIGS_REPO}/${VERSION}"
echo "Looking for signature directories in '${OUTSIGDIR_BASE}'"
echo ""
# Usage: verify compare_manifest current_manifest
verify() {
local compare_manifest="$1"
local current_manifest="$2"
if ! gpg --quiet --batch --verify "$current_manifest".asc "$current_manifest" 1>&2; then
echo "ERR: Failed to verify GPG signature in '${current_manifest}'"
echo ""
echo "Hint: Either the signature is invalid or the public key is missing"
echo ""
elif ! diff --report-identical "$compare_manifest" "$current_manifest" 1>&2; then
echo "ERR: The SHA256SUMS attestation in these two directories differ:"
echo " '${compare_manifest}'"
echo " '${current_manifest}'"
echo ""
else
echo "Verified: '${current_manifest}'"
echo ""
fi
}
shopt -s nullglob
all_noncodesigned=( "$OUTSIGDIR_BASE"/*/noncodesigned.SHA256SUMS )
shopt -u nullglob
echo "--------------------"
echo ""
if (( ${#all_noncodesigned[@]} )); then
compare_noncodesigned="${all_noncodesigned[0]}"
for current_manifest in "${all_noncodesigned[@]}"; do
verify "$compare_noncodesigned" "$current_manifest"
done
echo "DONE: Checking output signatures for noncodesigned.SHA256SUMS"
echo ""
else
echo "WARN: No signature directories with noncodesigned.SHA256SUMS found"
echo ""
fi
shopt -s nullglob
all_all=( "$OUTSIGDIR_BASE"/*/all.SHA256SUMS )
shopt -u nullglob
echo "--------------------"
echo ""
if (( ${#all_all[@]} )); then
compare_all="${all_all[0]}"
for current_manifest in "${all_all[@]}"; do
verify "$compare_all" "$current_manifest"
done
# Sanity check: there should be no entries that exist in
# noncodesigned.SHA256SUMS that doesn't exist in all.SHA256SUMS
if [[ "$(comm -23 <(sort "$compare_noncodesigned") <(sort "$compare_all") | wc -c)" -ne 0 ]]; then
echo "ERR: There are unique lines in noncodesigned.SHA256SUMS which"
echo " do not exist in all.SHA256SUMS, something went very wrong."
exit 1
fi
echo "DONE: Checking output signatures for all.SHA256SUMS"
echo ""
else
echo "WARN: No signature directories with all.SHA256SUMS found"
echo ""
fi
echo "===================="
echo ""
if (( ${#all_noncodesigned[@]} + ${#all_all[@]} == 0 )); then
echo "ERR: Unable to perform any verifications as no signature directories"
echo " were found"
echo ""
exit 1
fi

449
contrib/guix/libexec/build.sh Executable file
View file

@ -0,0 +1,449 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
export TZ=UTC
# Although Guix _does_ set umask when building its own packages (in our case,
# this is all packages in manifest.scm), it does not set it for `guix
# environment`. It does make sense for at least `guix environment --container`
# to set umask, so if that change gets merged upstream and we bump the
# time-machine to a commit which includes the aforementioned change, we can
# remove this line.
#
# This line should be placed before any commands which creates files.
umask 0022
if [ -n "$V" ]; then
# Print both unexpanded (-v) and expanded (-x) forms of commands as they are
# read from this file.
set -vx
# Set VERBOSE for CMake-based builds
export VERBOSE="$V"
fi
# Check that required environment variables are set
cat << EOF
Required environment variables as seen inside the container:
DIST_ARCHIVE_BASE: ${DIST_ARCHIVE_BASE:?not set}
DISTNAME: ${DISTNAME:?not set}
HOST: ${HOST:?not set}
SOURCE_DATE_EPOCH: ${SOURCE_DATE_EPOCH:?not set}
JOBS: ${JOBS:?not set}
DISTSRC: ${DISTSRC:?not set}
OUTDIR: ${OUTDIR:?not set}
EOF
ACTUAL_OUTDIR="${OUTDIR}"
OUTDIR="${DISTSRC}/output"
#####################
# Environment Setup #
#####################
# The depends folder also serves as a base-prefix for depends packages for
# $HOSTs after successfully building.
BASEPREFIX="${PWD}/depends"
# Given a package name and an output name, return the path of that output in our
# current guix environment
store_path() {
grep --extended-regexp "/[^-]{32}-${1}-[^-]+${2:+-${2}}" "${GUIX_ENVIRONMENT}/manifest" \
| head --lines=1 \
| sed --expression='s|^[[:space:]]*"||' \
--expression='s|"[[:space:]]*$||'
}
# Set environment variables to point the NATIVE toolchain to the right
# includes/libs
NATIVE_GCC="$(store_path gcc-toolchain)"
NATIVE_GCC_STATIC="$(store_path gcc-toolchain static)"
unset LIBRARY_PATH
unset CPATH
unset C_INCLUDE_PATH
unset CPLUS_INCLUDE_PATH
unset OBJC_INCLUDE_PATH
unset OBJCPLUS_INCLUDE_PATH
export LIBRARY_PATH="${NATIVE_GCC}/lib:${NATIVE_GCC}/lib64:${NATIVE_GCC_STATIC}/lib:${NATIVE_GCC_STATIC}/lib64"
export C_INCLUDE_PATH="${NATIVE_GCC}/include"
export CPLUS_INCLUDE_PATH="${NATIVE_GCC}/include/c++:${NATIVE_GCC}/include"
export OBJC_INCLUDE_PATH="${NATIVE_GCC}/include"
export OBJCPLUS_INCLUDE_PATH="${NATIVE_GCC}/include/c++:${NATIVE_GCC}/include"
prepend_to_search_env_var() {
export "${1}=${2}${!1:+:}${!1}"
}
# Set environment variables to point the CROSS toolchain to the right
# includes/libs for $HOST
case "$HOST" in
*mingw*)
# Determine output paths to use in CROSS_* environment variables
CROSS_GLIBC="$(store_path "mingw-w64-x86_64-winpthreads")"
CROSS_GCC="$(store_path "gcc-cross-${HOST}")"
CROSS_GCC_LIB_STORE="$(store_path "gcc-cross-${HOST}" lib)"
CROSS_GCC_LIBS=( "${CROSS_GCC_LIB_STORE}/lib/gcc/${HOST}"/* ) # This expands to an array of directories...
CROSS_GCC_LIB="${CROSS_GCC_LIBS[0]}" # ...we just want the first one (there should only be one)
# The search path ordering is generally:
# 1. gcc-related search paths
# 2. libc-related search paths
# 2. kernel-header-related search paths (not applicable to mingw-w64 hosts)
export CROSS_C_INCLUDE_PATH="${CROSS_GCC_LIB}/include:${CROSS_GCC_LIB}/include-fixed:${CROSS_GLIBC}/include"
export CROSS_CPLUS_INCLUDE_PATH="${CROSS_GCC}/include/c++:${CROSS_GCC}/include/c++/${HOST}:${CROSS_GCC}/include/c++/backward:${CROSS_C_INCLUDE_PATH}"
export CROSS_LIBRARY_PATH="${CROSS_GCC_LIB_STORE}/lib:${CROSS_GCC}/${HOST}/lib:${CROSS_GCC_LIB}:${CROSS_GLIBC}/lib"
;;
*darwin*)
# The CROSS toolchain for darwin uses the SDK and ignores environment variables.
# See depends/hosts/darwin.mk for more details.
;;
*linux*)
CROSS_GLIBC="$(store_path "glibc-cross-${HOST}")"
CROSS_GLIBC_STATIC="$(store_path "glibc-cross-${HOST}" static)"
CROSS_KERNEL="$(store_path "linux-libre-headers-cross-${HOST}")"
CROSS_GCC="$(store_path "gcc-cross-${HOST}")"
CROSS_GCC_LIB_STORE="$(store_path "gcc-cross-${HOST}" lib)"
CROSS_GCC_LIBS=( "${CROSS_GCC_LIB_STORE}/lib/gcc/${HOST}"/* ) # This expands to an array of directories...
CROSS_GCC_LIB="${CROSS_GCC_LIBS[0]}" # ...we just want the first one (there should only be one)
export CROSS_C_INCLUDE_PATH="${CROSS_GCC_LIB}/include:${CROSS_GCC_LIB}/include-fixed:${CROSS_GLIBC}/include:${CROSS_KERNEL}/include"
export CROSS_CPLUS_INCLUDE_PATH="${CROSS_GCC}/include/c++:${CROSS_GCC}/include/c++/${HOST}:${CROSS_GCC}/include/c++/backward:${CROSS_C_INCLUDE_PATH}"
export CROSS_LIBRARY_PATH="${CROSS_GCC_LIB_STORE}/lib:${CROSS_GCC}/${HOST}/lib:${CROSS_GCC_LIB}:${CROSS_GLIBC}/lib:${CROSS_GLIBC_STATIC}/lib"
;;
*)
exit 1 ;;
esac
# Sanity check CROSS_*_PATH directories
IFS=':' read -ra PATHS <<< "${CROSS_C_INCLUDE_PATH}:${CROSS_CPLUS_INCLUDE_PATH}:${CROSS_LIBRARY_PATH}"
for p in "${PATHS[@]}"; do
if [ -n "$p" ] && [ ! -d "$p" ]; then
echo "'$p' doesn't exist or isn't a directory... Aborting..."
exit 1
fi
done
# Disable Guix ld auto-rpath behavior
case "$HOST" in
*darwin*)
# The auto-rpath behavior is necessary for darwin builds as some native
# tools built by depends refer to and depend on Guix-built native
# libraries
#
# After the native packages in depends are built, the ld wrapper should
# no longer affect our build, as clang would instead reach for
# x86_64-apple-darwin18-ld from cctools
;;
*) export GUIX_LD_WRAPPER_DISABLE_RPATH=yes ;;
esac
# Make /usr/bin if it doesn't exist
[ -e /usr/bin ] || mkdir -p /usr/bin
# Symlink file and env to a conventional path
[ -e /usr/bin/file ] || ln -s --no-dereference "$(command -v file)" /usr/bin/file
[ -e /usr/bin/env ] || ln -s --no-dereference "$(command -v env)" /usr/bin/env
# Determine the correct value for -Wl,--dynamic-linker for the current $HOST
case "$HOST" in
*linux*)
glibc_dynamic_linker=$(
case "$HOST" in
i686-linux-gnu) echo /lib/ld-linux.so.2 ;;
x86_64-linux-gnu) echo /lib64/ld-linux-x86-64.so.2 ;;
arm-linux-gnueabihf) echo /lib/ld-linux-armhf.so.3 ;;
aarch64-linux-gnu) echo /lib/ld-linux-aarch64.so.1 ;;
riscv64-linux-gnu) echo /lib/ld-linux-riscv64-lp64d.so.1 ;;
powerpc64-linux-gnu) echo /lib64/ld64.so.1;;
powerpc64le-linux-gnu) echo /lib64/ld64.so.2;;
*) exit 1 ;;
esac
)
;;
esac
# Environment variables for determinism
export TAR_OPTIONS="--owner=0 --group=0 --numeric-owner --mtime='@${SOURCE_DATE_EPOCH}' --sort=name"
export TZ="UTC"
case "$HOST" in
*darwin*)
# cctools AR, unlike GNU binutils AR, does not have a deterministic mode
# or a configure flag to enable determinism by default, it only
# understands if this env-var is set or not. See:
#
# https://github.com/tpoechtrager/cctools-port/blob/55562e4073dea0fbfd0b20e0bf69ffe6390c7f97/cctools/ar/archive.c#L334
export ZERO_AR_DATE=yes
;;
esac
####################
# Depends Building #
####################
# Build the depends tree, overriding variables that assume multilib gcc
make -C depends --jobs="$JOBS" HOST="$HOST" \
${V:+V=1} \
${SOURCES_PATH+SOURCES_PATH="$SOURCES_PATH"} \
${BASE_CACHE+BASE_CACHE="$BASE_CACHE"} \
${SDK_PATH+SDK_PATH="$SDK_PATH"} \
i686_linux_CC=i686-linux-gnu-gcc \
i686_linux_CXX=i686-linux-gnu-g++ \
i686_linux_AR=i686-linux-gnu-ar \
i686_linux_RANLIB=i686-linux-gnu-ranlib \
i686_linux_NM=i686-linux-gnu-nm \
i686_linux_STRIP=i686-linux-gnu-strip \
x86_64_linux_CC=x86_64-linux-gnu-gcc \
x86_64_linux_CXX=x86_64-linux-gnu-g++ \
x86_64_linux_AR=x86_64-linux-gnu-ar \
x86_64_linux_RANLIB=x86_64-linux-gnu-ranlib \
x86_64_linux_NM=x86_64-linux-gnu-nm \
x86_64_linux_STRIP=x86_64-linux-gnu-strip \
qt_config_opts_i686_linux='-platform linux-g++ -xplatform digibyte-linux-g++' \
qt_config_opts_x86_64_linux='-platform linux-g++ -xplatform digibyte-linux-g++' \
FORCE_USE_SYSTEM_CLANG=1
###########################
# Source Tarball Building #
###########################
GIT_ARCHIVE="${DIST_ARCHIVE_BASE}/${DISTNAME}.tar.gz"
# Create the source tarball if not already there
if [ ! -e "$GIT_ARCHIVE" ]; then
mkdir -p "$(dirname "$GIT_ARCHIVE")"
touch "${DIST_ARCHIVE_BASE}"/SKIPATTEST.TAG
git archive --prefix="${DISTNAME}/" --output="$GIT_ARCHIVE" HEAD
fi
mkdir -p "$OUTDIR"
###########################
# Binary Tarball Building #
###########################
# CONFIGFLAGS
CONFIGFLAGS="--enable-reduce-exports --disable-bench --disable-gui-tests --disable-fuzz-binary"
case "$HOST" in
*linux*) CONFIGFLAGS+=" --disable-threadlocal" ;;
esac
# CFLAGS
HOST_CFLAGS="-O2 -g"
case "$HOST" in
*linux*) HOST_CFLAGS+=" -ffile-prefix-map=${PWD}=." ;;
*mingw*) HOST_CFLAGS+=" -fno-ident" ;;
*darwin*) unset HOST_CFLAGS ;;
esac
# CXXFLAGS
HOST_CXXFLAGS="$HOST_CFLAGS"
case "$HOST" in
arm-linux-gnueabihf) HOST_CXXFLAGS="${HOST_CXXFLAGS} -Wno-psabi" ;;
esac
# LDFLAGS
case "$HOST" in
*linux*) HOST_LDFLAGS="-Wl,--as-needed -Wl,--dynamic-linker=$glibc_dynamic_linker -static-libstdc++ -Wl,-O2" ;;
*mingw*) HOST_LDFLAGS="-Wl,--no-insert-timestamp" ;;
esac
# Using --no-tls-get-addr-optimize retains compatibility with glibc 2.17, by
# avoiding a PowerPC64 optimisation available in glibc 2.22 and later.
# https://sourceware.org/binutils/docs-2.35/ld/PowerPC64-ELF64.html
case "$HOST" in
*powerpc64*) HOST_LDFLAGS="${HOST_LDFLAGS} -Wl,--no-tls-get-addr-optimize" ;;
esac
case "$HOST" in
powerpc64-linux-*|riscv64-linux-*) HOST_LDFLAGS="${HOST_LDFLAGS} -Wl,-z,noexecstack" ;;
esac
# Make $HOST-specific native binaries from depends available in $PATH
export PATH="${BASEPREFIX}/${HOST}/native/bin:${PATH}"
mkdir -p "$DISTSRC"
(
cd "$DISTSRC"
# Extract the source tarball
tar --strip-components=1 -xf "${GIT_ARCHIVE}"
./autogen.sh
# Configure this DISTSRC for $HOST
# shellcheck disable=SC2086
env CONFIG_SITE="${BASEPREFIX}/${HOST}/share/config.site" \
./configure --prefix=/ \
--disable-ccache \
--disable-maintainer-mode \
--disable-dependency-tracking \
${CONFIGFLAGS} \
${HOST_CFLAGS:+CFLAGS="${HOST_CFLAGS}"} \
${HOST_CXXFLAGS:+CXXFLAGS="${HOST_CXXFLAGS}"} \
${HOST_LDFLAGS:+LDFLAGS="${HOST_LDFLAGS}"}
sed -i.old 's/-lstdc++ //g' config.status libtool src/univalue/config.status src/univalue/libtool
# Build DigiByte Core
make --jobs="$JOBS" ${V:+V=1}
# Check that symbol/security checks tools are sane.
make test-security-check ${V:+V=1}
# Perform basic security checks on a series of executables.
make -C src --jobs=1 check-security ${V:+V=1}
# Check that executables only contain allowed version symbols.
make -C src --jobs=1 check-symbols ${V:+V=1}
mkdir -p "$OUTDIR"
# Make the os-specific installers
case "$HOST" in
*mingw*)
make deploy ${V:+V=1} DIGIBYTE_WIN_INSTALLER="${OUTDIR}/${DISTNAME}-win64-setup-unsigned.exe"
;;
esac
# Setup the directory where our DigiByte Core build for HOST will be
# installed. This directory will also later serve as the input for our
# binary tarballs.
INSTALLPATH="${PWD}/installed/${DISTNAME}"
mkdir -p "${INSTALLPATH}"
# Install built DigiByte Core to $INSTALLPATH
case "$HOST" in
*darwin*)
make install-strip DESTDIR="${INSTALLPATH}" ${V:+V=1}
;;
*)
make install DESTDIR="${INSTALLPATH}" ${V:+V=1}
;;
esac
case "$HOST" in
*darwin*)
make osx_volname ${V:+V=1}
make deploydir ${V:+V=1}
mkdir -p "unsigned-app-${HOST}"
cp --target-directory="unsigned-app-${HOST}" \
osx_volname \
contrib/macdeploy/detached-sig-create.sh
mv --target-directory="unsigned-app-${HOST}" dist
(
cd "unsigned-app-${HOST}"
find . -print0 \
| sort --zero-terminated \
| tar --create --no-recursion --mode='u+rw,go+r-w,a+X' --null --files-from=- \
| gzip -9n > "${OUTDIR}/${DISTNAME}-osx-unsigned.tar.gz" \
|| ( rm -f "${OUTDIR}/${DISTNAME}-osx-unsigned.tar.gz" && exit 1 )
)
make deploy ${V:+V=1} OSX_DMG="${OUTDIR}/${DISTNAME}-osx-unsigned.dmg"
;;
esac
(
cd installed
case "$HOST" in
*mingw*)
mv --target-directory="$DISTNAME"/lib/ "$DISTNAME"/bin/*.dll
;;
esac
# Prune libtool and object archives
find . -name "lib*.la" -delete
find . -name "lib*.a" -delete
# Prune pkg-config files
rm -rf "${DISTNAME}/lib/pkgconfig"
case "$HOST" in
*darwin*) ;;
*)
# Split binaries and libraries from their debug symbols
{
find "${DISTNAME}/bin" -type f -executable -print0
find "${DISTNAME}/lib" -type f -print0
} | xargs -0 -n1 -P"$JOBS" -I{} "${DISTSRC}/contrib/devtools/split-debug.sh" {} {} {}.dbg
;;
esac
case "$HOST" in
*mingw*)
cp "${DISTSRC}/doc/README_windows.txt" "${DISTNAME}/readme.txt"
;;
*linux*)
cp "${DISTSRC}/README.md" "${DISTNAME}/"
;;
esac
# Finally, deterministically produce {non-,}debug binary tarballs ready
# for release
case "$HOST" in
*mingw*)
find "${DISTNAME}" -not -name "*.dbg" -print0 \
| xargs -0r touch --no-dereference --date="@${SOURCE_DATE_EPOCH}"
find "${DISTNAME}" -not -name "*.dbg" \
| sort \
| zip -X@ "${OUTDIR}/${DISTNAME}-${HOST//x86_64-w64-mingw32/win64}.zip" \
|| ( rm -f "${OUTDIR}/${DISTNAME}-${HOST//x86_64-w64-mingw32/win64}.zip" && exit 1 )
find "${DISTNAME}" -name "*.dbg" -print0 \
| xargs -0r touch --no-dereference --date="@${SOURCE_DATE_EPOCH}"
find "${DISTNAME}" -name "*.dbg" \
| sort \
| zip -X@ "${OUTDIR}/${DISTNAME}-${HOST//x86_64-w64-mingw32/win64}-debug.zip" \
|| ( rm -f "${OUTDIR}/${DISTNAME}-${HOST//x86_64-w64-mingw32/win64}-debug.zip" && exit 1 )
;;
*linux*)
find "${DISTNAME}" -not -name "*.dbg" -print0 \
| sort --zero-terminated \
| tar --create --no-recursion --mode='u+rw,go+r-w,a+X' --null --files-from=- \
| gzip -9n > "${OUTDIR}/${DISTNAME}-${HOST}.tar.gz" \
|| ( rm -f "${OUTDIR}/${DISTNAME}-${HOST}.tar.gz" && exit 1 )
find "${DISTNAME}" -name "*.dbg" -print0 \
| sort --zero-terminated \
| tar --create --no-recursion --mode='u+rw,go+r-w,a+X' --null --files-from=- \
| gzip -9n > "${OUTDIR}/${DISTNAME}-${HOST}-debug.tar.gz" \
|| ( rm -f "${OUTDIR}/${DISTNAME}-${HOST}-debug.tar.gz" && exit 1 )
;;
*darwin*)
find "${DISTNAME}" -print0 \
| sort --zero-terminated \
| tar --create --no-recursion --mode='u+rw,go+r-w,a+X' --null --files-from=- \
| gzip -9n > "${OUTDIR}/${DISTNAME}-${HOST//x86_64-apple-darwin18/osx64}.tar.gz" \
|| ( rm -f "${OUTDIR}/${DISTNAME}-${HOST//x86_64-apple-darwin18/osx64}.tar.gz" && exit 1 )
;;
esac
) # $DISTSRC/installed
case "$HOST" in
*mingw*)
cp -rf --target-directory=. contrib/windeploy
(
cd ./windeploy
mkdir -p unsigned
cp --target-directory=unsigned/ "${OUTDIR}/${DISTNAME}-win64-setup-unsigned.exe"
find . -print0 \
| sort --zero-terminated \
| tar --create --no-recursion --mode='u+rw,go+r-w,a+X' --null --files-from=- \
| gzip -9n > "${OUTDIR}/${DISTNAME}-win-unsigned.tar.gz" \
|| ( rm -f "${OUTDIR}/${DISTNAME}-win-unsigned.tar.gz" && exit 1 )
)
;;
esac
) # $DISTSRC
rm -rf "$ACTUAL_OUTDIR"
mv --no-target-directory "$OUTDIR" "$ACTUAL_OUTDIR" \
|| ( rm -rf "$ACTUAL_OUTDIR" && exit 1 )
(
cd /outdir-base
{
echo "$GIT_ARCHIVE"
find "$ACTUAL_OUTDIR" -type f
} | xargs realpath --relative-base="$PWD" \
| xargs sha256sum \
| sort -k2 \
| sponge "$ACTUAL_OUTDIR"/SHA256SUMS.part
)

110
contrib/guix/libexec/codesign.sh Executable file
View file

@ -0,0 +1,110 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
export TZ=UTC
# Although Guix _does_ set umask when building its own packages (in our case,
# this is all packages in manifest.scm), it does not set it for `guix
# environment`. It does make sense for at least `guix environment --container`
# to set umask, so if that change gets merged upstream and we bump the
# time-machine to a commit which includes the aforementioned change, we can
# remove this line.
#
# This line should be placed before any commands which creates files.
umask 0022
if [ -n "$V" ]; then
# Print both unexpanded (-v) and expanded (-x) forms of commands as they are
# read from this file.
set -vx
# Set VERBOSE for CMake-based builds
export VERBOSE="$V"
fi
# Check that required environment variables are set
cat << EOF
Required environment variables as seen inside the container:
UNSIGNED_TARBALL: ${UNSIGNED_TARBALL:?not set}
DETACHED_SIGS_REPO: ${DETACHED_SIGS_REPO:?not set}
DIST_ARCHIVE_BASE: ${DIST_ARCHIVE_BASE:?not set}
DISTNAME: ${DISTNAME:?not set}
HOST: ${HOST:?not set}
SOURCE_DATE_EPOCH: ${SOURCE_DATE_EPOCH:?not set}
DISTSRC: ${DISTSRC:?not set}
OUTDIR: ${OUTDIR:?not set}
EOF
ACTUAL_OUTDIR="${OUTDIR}"
OUTDIR="${DISTSRC}/output"
git_head_version() {
local recent_tag
if recent_tag="$(git -C "$1" describe --exact-match HEAD 2> /dev/null)"; then
echo "${recent_tag#v}"
else
git -C "$1" rev-parse --short=12 HEAD
fi
}
CODESIGNATURE_GIT_ARCHIVE="${DIST_ARCHIVE_BASE}/${DISTNAME}-codesignatures-$(git_head_version "$DETACHED_SIGS_REPO").tar.gz"
# Create the codesignature tarball if not already there
if [ ! -e "$CODESIGNATURE_GIT_ARCHIVE" ]; then
mkdir -p "$(dirname "$CODESIGNATURE_GIT_ARCHIVE")"
git -C "$DETACHED_SIGS_REPO" archive --output="$CODESIGNATURE_GIT_ARCHIVE" HEAD
fi
mkdir -p "$OUTDIR"
mkdir -p "$DISTSRC"
(
cd "$DISTSRC"
tar -xf "$UNSIGNED_TARBALL"
mkdir -p codesignatures
tar -C codesignatures -xf "$CODESIGNATURE_GIT_ARCHIVE"
case "$HOST" in
*mingw*)
find "$PWD" -name "*-unsigned.exe" | while read -r infile; do
infile_base="$(basename "$infile")"
# Codesigned *-unsigned.exe and output to OUTDIR
osslsigncode attach-signature \
-in "$infile" \
-out "${OUTDIR}/${infile_base/-unsigned}" \
-sigin codesignatures/win/"$infile_base".pem
done
;;
*darwin*)
# Apply detached codesignatures to dist/ (in-place)
signapple apply dist/DigiByte-Qt.app codesignatures/osx/dist
# Make a DMG from dist/
xorrisofs -D -l -V "$(< osx_volname)" -no-pad -r -dir-mode 0755 \
-o "${OUTDIR}/${DISTNAME}-${HOST}.dmg" \
dist \
-- -volume_date all_file_dates ="$SOURCE_DATE_EPOCH"
;;
*)
exit 1
;;
esac
) # $DISTSRC
rm -rf "$ACTUAL_OUTDIR"
mv --no-target-directory "$OUTDIR" "$ACTUAL_OUTDIR" \
|| ( rm -rf "$ACTUAL_OUTDIR" && exit 1 )
(
cd /outdir-base
{
echo "$UNSIGNED_TARBALL"
echo "$CODESIGNATURE_GIT_ARCHIVE"
find "$ACTUAL_OUTDIR" -type f
} | xargs realpath --relative-base="$PWD" \
| xargs sha256sum \
| sort -k2 \
| sponge "$ACTUAL_OUTDIR"/SHA256SUMS.part
)

66
contrib/guix/libexec/prelude.bash Normal file
View file

@ -0,0 +1,66 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
# shellcheck source=../../shell/realpath.bash
source contrib/shell/realpath.bash
# shellcheck source=../../shell/git-utils.bash
source contrib/shell/git-utils.bash
################
# Required non-builtin commands should be invocable
################
check_tools() {
for cmd in "$@"; do
if ! command -v "$cmd" > /dev/null 2>&1; then
echo "ERR: This script requires that '$cmd' is installed and available in your \$PATH"
exit 1
fi
done
}
check_tools cat env readlink dirname basename git
################
# We should be at the top directory of the repository
################
same_dir() {
local resolved1 resolved2
resolved1="$(bash_realpath "${1}")"
resolved2="$(bash_realpath "${2}")"
[ "$resolved1" = "$resolved2" ]
}
if ! same_dir "${PWD}" "$(git_root)"; then
cat << EOF
ERR: This script must be invoked from the top level of the git repository
Hint: This may look something like:
env FOO=BAR ./contrib/guix/guix-<blah>
EOF
exit 1
fi
################
# Set common variables
################
VERSION="${VERSION:-$(git_head_version)}"
DISTNAME="${DISTNAME:-digibyte-${VERSION}}"
version_base_prefix="${PWD}/guix-build-"
VERSION_BASE="${version_base_prefix}${VERSION}" # TOP
DISTSRC_BASE="${DISTSRC_BASE:-${VERSION_BASE}}"
OUTDIR_BASE="${OUTDIR_BASE:-${VERSION_BASE}/output}"
var_base_basename="var"
VAR_BASE="${VAR_BASE:-${VERSION_BASE}/${var_base_basename}}"
profiles_base_basename="profiles"
PROFILES_BASE="${PROFILES_BASE:-${VAR_BASE}/${profiles_base_basename}}"

660
contrib/guix/manifest.scm Normal file
View file

@ -0,0 +1,660 @@
(use-modules (gnu)
(gnu packages)
(gnu packages autotools)
(gnu packages base)
(gnu packages bash)
(gnu packages bison)
(gnu packages certs)
(gnu packages cdrom)
(gnu packages check)
(gnu packages cmake)
(gnu packages commencement)
(gnu packages compression)
(gnu packages cross-base)
(gnu packages curl)
(gnu packages file)
(gnu packages gawk)
(gnu packages gcc)
(gnu packages gnome)
(gnu packages image)
(gnu packages imagemagick)
(gnu packages installers)
(gnu packages linux)
(gnu packages llvm)
(gnu packages mingw)
(gnu packages moreutils)
(gnu packages perl)
(gnu packages pkg-config)
(gnu packages python)
(gnu packages python-web)
(gnu packages shells)
(gnu packages tls)
(gnu packages version-control)
(guix build-system font)
(guix build-system gnu)
(guix build-system python)
(guix build-system trivial)
(guix download)
(guix gexp)
(guix git-download)
((guix licenses) #:prefix license:)
(guix packages)
(guix profiles)
(guix utils))
(define-syntax-rule (search-our-patches file-name ...)
"Return the list of absolute file names corresponding to each
FILE-NAME found in ./patches relative to the current file."
(parameterize
((%patch-path (list (string-append (dirname (current-filename)) "/patches"))))
(list (search-patch file-name) ...)))
(define (make-ssp-fixed-gcc xgcc)
"Given a XGCC package, return a modified package that uses the SSP function
from glibc instead of from libssp.so. Our `symbol-check' script will complain if
we link against libssp.so, and thus will ensure that this works properly.
Taken from:
http://www.linuxfromscratch.org/hlfs/view/development/chapter05/gcc-pass1.html"
(package
(inherit xgcc)
(arguments
(substitute-keyword-arguments (package-arguments xgcc)
((#:make-flags flags)
`(cons "gcc_cv_libc_provides_ssp=yes" ,flags))))))
(define (make-gcc-rpath-link xgcc)
"Given a XGCC package, return a modified package that replace each instance of
-rpath in the default system spec that's inserted by Guix with -rpath-link"
(package
(inherit xgcc)
(arguments
(substitute-keyword-arguments (package-arguments xgcc)
((#:phases phases)
`(modify-phases ,phases
(add-after 'pre-configure 'replace-rpath-with-rpath-link
(lambda _
(substitute* (cons "gcc/config/rs6000/sysv4.h"
(find-files "gcc/config"
"^gnu-user.*\\.h$"))
(("-rpath=") "-rpath-link="))
#t))))))))
(define (make-binutils-with-mingw-w64-disable-flags xbinutils)
(package-with-extra-patches xbinutils
(search-our-patches "binutils-mingw-w64-disable-flags.patch")))
(define (make-cross-toolchain target
base-gcc-for-libc
base-kernel-headers
base-libc
base-gcc)
"Create a cross-compilation toolchain package for TARGET"
(let* ((xbinutils (cross-binutils target))
;; 1. Build a cross-compiling gcc without targeting any libc, derived
;; from BASE-GCC-FOR-LIBC
(xgcc-sans-libc (cross-gcc target
#:xgcc base-gcc-for-libc
#:xbinutils xbinutils))
;; 2. Build cross-compiled kernel headers with XGCC-SANS-LIBC, derived
;; from BASE-KERNEL-HEADERS
(xkernel (cross-kernel-headers target
base-kernel-headers
xgcc-sans-libc
xbinutils))
;; 3. Build a cross-compiled libc with XGCC-SANS-LIBC and XKERNEL,
;; derived from BASE-LIBC
(xlibc (cross-libc target
base-libc
xgcc-sans-libc
xbinutils
xkernel))
;; 4. Build a cross-compiling gcc targeting XLIBC, derived from
;; BASE-GCC
(xgcc (cross-gcc target
#:xgcc base-gcc
#:xbinutils xbinutils
#:libc xlibc)))
;; Define a meta-package that propagates the resulting XBINUTILS, XLIBC, and
;; XGCC
(package
(name (string-append target "-toolchain"))
(version (package-version xgcc))
(source #f)
(build-system trivial-build-system)
(arguments '(#:builder (begin (mkdir %output) #t)))
(propagated-inputs
`(("binutils" ,xbinutils)
("libc" ,xlibc)
("libc:static" ,xlibc "static")
("gcc" ,xgcc)
("gcc-lib" ,xgcc "lib")))
(synopsis (string-append "Complete GCC tool chain for " target))
(description (string-append "This package provides a complete GCC tool
chain for " target " development."))
(home-page (package-home-page xgcc))
(license (package-license xgcc)))))
(define base-gcc
(package-with-extra-patches gcc-8
(search-our-patches "gcc-8-sort-libtool-find-output.patch")))
;; Building glibc with stack smashing protector first landed in glibc 2.25, use
;; this function to disable for older glibcs
;;
;; From glibc 2.25 changelog:
;;
;; * Most of glibc can now be built with the stack smashing protector enabled.
;; It is recommended to build glibc with --enable-stack-protector=strong.
;; Implemented by Nick Alcock (Oracle).
(define (make-glibc-without-ssp xglibc)
(package-with-extra-configure-variable
(package-with-extra-configure-variable
xglibc "libc_cv_ssp" "no")
"libc_cv_ssp_strong" "no"))
(define* (make-digibyte-cross-toolchain target
#:key
(base-gcc-for-libc gcc-7)
(base-kernel-headers linux-libre-headers-4.9)
(base-libc (make-glibc-without-ssp glibc-2.24))
(base-gcc (make-gcc-rpath-link base-gcc)))
"Convenience wrapper around MAKE-CROSS-TOOLCHAIN with default values
desirable for building DigiByte Core release binaries."
(make-cross-toolchain target
base-gcc-for-libc
base-kernel-headers
base-libc
base-gcc))
(define (make-gcc-with-pthreads gcc)
(package-with-extra-configure-variable gcc "--enable-threads" "posix"))
(define (make-mingw-pthreads-cross-toolchain target)
"Create a cross-compilation toolchain package for TARGET"
(let* ((xbinutils (make-binutils-with-mingw-w64-disable-flags (cross-binutils target)))
(pthreads-xlibc mingw-w64-x86_64-winpthreads)
(pthreads-xgcc (make-gcc-with-pthreads
(cross-gcc target
#:xgcc (make-ssp-fixed-gcc base-gcc)
#:xbinutils xbinutils
#:libc pthreads-xlibc))))
;; Define a meta-package that propagates the resulting XBINUTILS, XLIBC, and
;; XGCC
(package
(name (string-append target "-posix-toolchain"))
(version (package-version pthreads-xgcc))
(source #f)
(build-system trivial-build-system)
(arguments '(#:builder (begin (mkdir %output) #t)))
(propagated-inputs
`(("binutils" ,xbinutils)
("libc" ,pthreads-xlibc)
("gcc" ,pthreads-xgcc)
("gcc-lib" ,pthreads-xgcc "lib")))
(synopsis (string-append "Complete GCC tool chain for " target))
(description (string-append "This package provides a complete GCC tool
chain for " target " development."))
(home-page (package-home-page pthreads-xgcc))
(license (package-license pthreads-xgcc)))))
(define (make-nsis-with-sde-support base-nsis)
(package-with-extra-patches base-nsis
(search-our-patches "nsis-SConstruct-sde-support.patch")))
(define-public font-tuffy
(package
(name "font-tuffy")
(version "20120614")
(source
(origin
(method url-fetch)
(uri (string-append "http://tulrich.com/fonts/tuffy-" version ".tar.gz"))
(file-name (string-append name "-" version ".tar.gz"))
(sha256
(base32
"02vf72bgrp30vrbfhxjw82s115z27dwfgnmmzfb0n9wfhxxfpyf6"))))
(build-system font-build-system)
(home-page "http://tulrich.com/fonts/")
(synopsis "The Tuffy Truetype Font Family")
(description
"Thatcher Ulrich's first outline font design. He started with the goal of producing a neutral, readable sans-serif text font. There are lots of \"expressive\" fonts out there, but he wanted to start with something very plain and clean, something he might want to actually use. ")
(license license:public-domain)))
(define-public lief
(package
(name "python-lief")
(version "0.11.5")
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/lief-project/LIEF.git")
(commit version)))
(file-name (git-file-name name version))
(sha256
(base32
"0qahjfg1n0x76ps2mbyljvws1l3qhkqvmxqbahps4qgywl2hbdkj"))))
(build-system python-build-system)
(native-inputs
`(("cmake" ,cmake)))
(home-page "https://github.com/lief-project/LIEF")
(synopsis "Library to Instrument Executable Formats")
(description "Python library to to provide a cross platform library which can
parse, modify and abstract ELF, PE and MachO formats.")
(license license:asl2.0)))
(define osslsigncode
(package
(name "osslsigncode")
(version "2.0")
(source (origin
(method url-fetch)
(uri (string-append "https://github.com/mtrojnar/"
name "/archive/" version ".tar.gz"))
(sha256
(base32
"0byri6xny770wwb2nciq44j5071122l14bvv65axdd70nfjf0q2s"))))
(build-system gnu-build-system)
(native-inputs
`(("pkg-config" ,pkg-config)
("autoconf" ,autoconf)
("automake" ,automake)
("libtool" ,libtool)))
(inputs
`(("openssl" ,openssl)))
(arguments
`(#:configure-flags
`("--without-gsf"
"--without-curl"
"--disable-dependency-tracking")))
(home-page "https://github.com/mtrojnar/osslsigncode")
(synopsis "Authenticode signing and timestamping tool")
(description "osslsigncode is a small tool that implements part of the
functionality of the Microsoft tool signtool.exe - more exactly the Authenticode
signing and timestamping. But osslsigncode is based on OpenSSL and cURL, and
thus should be able to compile on most platforms where these exist.")
(license license:gpl3+))) ; license is with openssl exception
(define-public python-asn1crypto
(package
(name "python-asn1crypto")
(version "1.4.0")
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/wbond/asn1crypto")
(commit version)))
(file-name (git-file-name name version))
(sha256
(base32
"19abibn6jw20mzi1ln4n9jjvpdka8ygm4m439hplyrdfqbvgm01r"))))
(build-system python-build-system)
(arguments
'(#:phases
(modify-phases %standard-phases
(replace 'check
(lambda _
(invoke "python" "run.py" "tests"))))))
(home-page "https://github.com/wbond/asn1crypto")
(synopsis "ASN.1 parser and serializer in Python")
(description "asn1crypto is an ASN.1 parser and serializer with definitions
for private keys, public keys, certificates, CRL, OCSP, CMS, PKCS#3, PKCS#7,
PKCS#8, PKCS#12, PKCS#5, X.509 and TSP.")
(license license:expat)))
(define-public python-elfesteem
(let ((commit "87bbd79ab7e361004c98cc8601d4e5f029fd8bd5"))
(package
(name "python-elfesteem")
(version (git-version "0.1" "1" commit))
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/LRGH/elfesteem")
(commit commit)))
(file-name (git-file-name name commit))
(sha256
(base32
"1nyvjisvyxyxnd0023xjf5846xd03lwawp5pfzr8vrky7wwm5maz"))))
(build-system python-build-system)
;; There are no tests, but attempting to run python setup.py test leads to
;; PYTHONPATH problems, just disable the test
(arguments '(#:tests? #f))
(home-page "https://github.com/LRGH/elfesteem")
(synopsis "ELF/PE/Mach-O parsing library")
(description "elfesteem parses ELF, PE and Mach-O files.")
(license license:lgpl2.1))))
(define-public python-oscrypto
(package
(name "python-oscrypto")
(version "1.2.1")
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/wbond/oscrypto")
(commit version)))
(file-name (git-file-name name version))
(sha256
(base32
"1d4d8s4z340qhvb3g5m5v3436y3a71yc26wk4749q64m09kxqc3l"))
(patches (search-our-patches "oscrypto-hard-code-openssl.patch"))))
(build-system python-build-system)
(native-search-paths
(list (search-path-specification
(variable "SSL_CERT_FILE")
(file-type 'regular)
(separator #f) ;single entry
(files '("etc/ssl/certs/ca-certificates.crt")))))
(propagated-inputs
`(("python-asn1crypto" ,python-asn1crypto)
("openssl" ,openssl)))
(arguments
`(#:phases
(modify-phases %standard-phases
(add-after 'unpack 'hard-code-path-to-libscrypt
(lambda* (#:key inputs #:allow-other-keys)
(let ((openssl (assoc-ref inputs "openssl")))
(substitute* "oscrypto/__init__.py"
(("@GUIX_OSCRYPTO_USE_OPENSSL@")
(string-append openssl "/lib/libcrypto.so" "," openssl "/lib/libssl.so")))
#t)))
(add-after 'unpack 'disable-broken-tests
(lambda _
;; This test is broken as there is no keyboard interrupt.
(substitute* "tests/test_trust_list.py"
(("^(.*)class TrustListTests" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
(substitute* "tests/test_tls.py"
(("^(.*)class TLSTests" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
#t))
(replace 'check
(lambda _
(invoke "python" "run.py" "tests")
#t)))))
(home-page "https://github.com/wbond/oscrypto")
(synopsis "Compiler-free Python crypto library backed by the OS")
(description "oscrypto is a compilation-free, always up-to-date encryption library for Python.")
(license license:expat)))
(define-public python-oscryptotests
(package (inherit python-oscrypto)
(name "python-oscryptotests")
(arguments
`(#:tests? #f
#:phases
(modify-phases %standard-phases
(add-after 'unpack 'hard-code-path-to-libscrypt
(lambda* (#:key inputs #:allow-other-keys)
(chdir "tests")
#t)))))))
(define-public python-certvalidator
(let ((commit "e5bdb4bfcaa09fa0af355eb8867d00dfeecba08c"))
(package
(name "python-certvalidator")
(version (git-version "0.1" "1" commit))
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/achow101/certvalidator")
(commit commit)))
(file-name (git-file-name name commit))
(sha256
(base32
"18pvxkvpkfkzgvfylv0kx65pmxfcv1hpsg03cip93krfvrrl4c75"))))
(build-system python-build-system)
(propagated-inputs
`(("python-asn1crypto" ,python-asn1crypto)
("python-oscrypto" ,python-oscrypto)
("python-oscryptotests", python-oscryptotests))) ;; certvalidator tests import oscryptotests
(arguments
`(#:phases
(modify-phases %standard-phases
(add-after 'unpack 'disable-broken-tests
(lambda _
(substitute* "tests/test_certificate_validator.py"
(("^(.*)class CertificateValidatorTests" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
(substitute* "tests/test_crl_client.py"
(("^(.*)def test_fetch_crl" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
(substitute* "tests/test_ocsp_client.py"
(("^(.*)def test_fetch_ocsp" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
(substitute* "tests/test_registry.py"
(("^(.*)def test_build_paths" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
(substitute* "tests/test_validate.py"
(("^(.*)def test_revocation_mode_hard" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
(substitute* "tests/test_validate.py"
(("^(.*)def test_revocation_mode_soft" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
#t))
(replace 'check
(lambda _
(invoke "python" "run.py" "tests")
#t)))))
(home-page "https://github.com/wbond/certvalidator")
(synopsis "Python library for validating X.509 certificates and paths")
(description "certvalidator is a Python library for validating X.509
certificates or paths. Supports various options, including: validation at a
specific moment in time, whitelisting and revocation checks.")
(license license:expat))))
(define-public python-requests-2.25.1
(package (inherit python-requests)
(version "2.25.1")
(source (origin
(method url-fetch)
(uri (pypi-uri "requests" version))
(sha256
(base32
"015qflyqsgsz09gnar69s6ga74ivq5kch69s4qxz3904m7a3v5r7"))))))
(define-public python-altgraph
(package
(name "python-altgraph")
(version "0.17")
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/ronaldoussoren/altgraph")
(commit (string-append "v" version))))
(file-name (git-file-name name version))
(sha256
(base32
"09sm4srvvkw458pn48ga9q7ykr4xlz7q8gh1h9w7nxpf001qgpwb"))))
(build-system python-build-system)
(home-page "https://github.com/ronaldoussoren/altgraph")
(synopsis "Python graph (network) package")
(description "altgraph is a fork of graphlib: a graph (network) package for
constructing graphs, BFS and DFS traversals, topological sort, shortest paths,
etc. with graphviz output.")
(license license:expat)))
(define-public python-macholib
(package
(name "python-macholib")
(version "1.14")
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/ronaldoussoren/macholib")
(commit (string-append "v" version))))
(file-name (git-file-name name version))
(sha256
(base32
"0aislnnfsza9wl4f0vp45ivzlc0pzhp9d4r08700slrypn5flg42"))))
(build-system python-build-system)
(propagated-inputs
`(("python-altgraph" ,python-altgraph)))
(arguments
'(#:phases
(modify-phases %standard-phases
(add-after 'unpack 'disable-broken-tests
(lambda _
;; This test is broken as there is no keyboard interrupt.
(substitute* "macholib_tests/test_command_line.py"
(("^(.*)class TestCmdLine" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
(substitute* "macholib_tests/test_dyld.py"
(("^(.*)def test_\\S+_find" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line))
(("^(.*)def testBasic" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line))
)
#t)))))
(home-page "https://github.com/ronaldoussoren/macholib")
(synopsis "Python library for analyzing and editing Mach-O headers")
(description "macholib is a Macho-O header analyzer and editor. It's
typically used as a dependency analysis tool, and also to rewrite dylib
references in Mach-O headers to be @executable_path relative. Though this tool
targets a platform specific file format, it is pure python code that is platform
and endian independent.")
(license license:expat)))
(define-public python-signapple
(let ((commit "b084cbbf44d5330448ffce0c7d118f75781b64bd"))
(package
(name "python-signapple")
(version (git-version "0.1" "1" commit))
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/achow101/signapple")
(commit commit)))
(file-name (git-file-name name commit))
(sha256
(base32
"0k7inccl2mzac3wq4asbr0kl8s4cghm8982z54kfascqg45shv01"))))
(build-system python-build-system)
(propagated-inputs
`(("python-asn1crypto" ,python-asn1crypto)
("python-oscrypto" ,python-oscrypto)
("python-certvalidator" ,python-certvalidator)
("python-elfesteem" ,python-elfesteem)
("python-requests" ,python-requests-2.25.1)
("python-macholib" ,python-macholib)
("libcrypto" ,openssl)))
;; There are no tests, but attempting to run python setup.py test leads to
;; problems, just disable the test
(arguments '(#:tests? #f))
(home-page "https://github.com/achow101/signapple")
(synopsis "Mach-O binary signature tool")
(description "signapple is a Python tool for creating, verifying, and
inspecting signatures in Mach-O binaries.")
(license license:expat))))
(define-public glibc-2.24
(package
(inherit glibc)
(version "2.24")
(source (origin
(method git-fetch)
(uri (git-reference
(url "https://sourceware.org/git/glibc.git")
(commit "0d7f1ed30969886c8dde62fbf7d2c79967d4bace")))
(file-name (git-file-name "glibc" "0d7f1ed30969886c8dde62fbf7d2c79967d4bace"))
(sha256
(base32
"0g5hryia5v1k0qx97qffgwzrz4lr4jw3s5kj04yllhswsxyjbic3"))
(patches (search-our-patches "glibc-ldd-x86_64.patch"
"glibc-versioned-locpath.patch"
"glibc-2.24-elfm-loadaddr-dynamic-rewrite.patch"
"glibc-2.24-no-build-time-cxx-header-run.patch"))))))
(define glibc-2.27/digibyte-patched
(package-with-extra-patches glibc-2.27
(search-our-patches "glibc-2.27-riscv64-Use-__has_include__-to-include-asm-syscalls.h.patch")))
(packages->manifest
(append
(list ;; The Basics
bash
which
coreutils
util-linux
;; File(system) inspection
file
grep
diffutils
findutils
;; File transformation
patch
gawk
sed
moreutils
;; Compression and archiving
tar
bzip2
gzip
xz
;; Build tools
gnu-make
libtool
autoconf-2.71
automake
pkg-config
bison
;; Scripting
perl
python-3
;; Git
git
;; Tests
lief
;; Native gcc 7 toolchain
gcc-toolchain-7
(list gcc-toolchain-7 "static"))
(let ((target (getenv "HOST")))
(cond ((string-suffix? "-mingw32" target)
;; Windows
(list zip
(make-mingw-pthreads-cross-toolchain "x86_64-w64-mingw32")
(make-nsis-with-sde-support nsis-x86_64)
osslsigncode))
((string-contains target "-linux-")
(list (cond ((string-contains target "riscv64-")
(make-digibyte-cross-toolchain target
#:base-libc glibc-2.27/digibyte-patched
#:base-kernel-headers linux-libre-headers-4.19))
(else
(make-digibyte-cross-toolchain target)))))
((string-contains target "darwin")
(list clang-toolchain-10 binutils imagemagick libtiff librsvg font-tuffy cmake xorriso python-signapple))
(else '())))))