The hushd build should be reproducible. This issue will be used to keep track of various changes needed to make that happen.
More info at https://reproducible-builds.org/
When gcc gets updated to a new version, the hushd code always needs to be changed to add some new header(s) for it to continue to successfully build.
This commit is an example of gcc13 fix
https://git.hush.is/hush/hush3/commit/121ec4b9d43b8a122141499496fbea5409d425d4
This commit is an example of gcc12 fix
https://git.hush.is/hush/hush3/commit/dd9be59e4f18c2364c9befd24440eea77617dfe0
I can't find the fix for gcc11 (or was it gcc10?)
@jahway603 I think that I haven't been clear in exactly what it means to be "reproducible". Your example of new versions of compilers requiring us to change Hush source code is definitely annoying, but that is orthogonal to reproducibility.
Reproducibility means the turning source code into a binary is 100% deterministic. Very common things that prevent a binary from being deterministic/reproducible are including the current time or date, the username or hostname of the bulid server, current timezone, the PATH when built, the current status of the git repo (you will see something like -dirty in the version when building manpages) and various other things. It is not related to needing to change source code because some external dependency changed.
Making code reproducible usually means changing the build system but can also require changing the code itself.
And to clarify more, reproducibility is per-architecture. We are not trying to make a Windows binary that is exactly the same as a Linux binary. We are trying to make it so that if two developers compile a Linux binary on their Linux servers, those binaries are byte-for-byte 100% exactly the same. A way to check this is that both binaries should have exactly the same SHA256.
@jahway603 I think that I haven't been clear in exactly what it means to be "reproducible". Your example of new versions of compilers requiring us to change Hush source code is definitely annoying, but that is orthogonal to reproducibility.
Reproducibility means the turning source code into a binary is 100% deterministic. Very common things that prevent a binary from being deterministic/reproducible are including the current time or date, the username or hostname of the bulid server, current timezone, the PATH when built, the current status of the git repo (you will see something like -dirty in the version when building manpages) and various other things. It is not related to needing to change source code because some external dependency changed.
Making code reproducible usually means changing the build system but can also require changing the code itself.
And to clarify more, reproducibility is per-architecture. We are not trying to make a Windows binary that is exactly the same as a Linux binary. We are trying to make it so that if two developers compile a Linux binary on their Linux servers, those binaries are byte-for-byte 100% exactly the same. A way to check this is that both binaries should have exactly the same SHA256.
BTC used to use a system called "Gitian" to do reproducible builds, which I helped port to Hush in The Olden Times for the old hush.git repo: https://git.hush.is/hush/hush-gitian
There are likely very good reasons why BTC Core decided to migrate away from Gitian and it's probably best for us to go with Guix vs Gitian, but if somebody wants to make a reproducible build by porting our old Gitian stuff to the new hush3.git repo, that would be better than nothing.
The easiest way to get reproducible builds in Hush will be to port over how Bitcoin Core does things with Guix (which is something similar to Nix) : https://github.com/bitcoin/bitcoin/blob/master/contrib/guix/README.md
The BTC PR that added Guix support also has some interesting comments : https://github.com/bitcoin/bitcoin/pull/15277
BTC used to use a system called "Gitian" to do reproducible builds, which I helped port to Hush in The Olden Times for the old hush.git repo: https://git.hush.is/hush/hush-gitian
There are likely very good reasons why BTC Core decided to migrate away from Gitian and it's probably best for us to go with Guix vs Gitian, but if somebody wants to make a reproducible build by porting our old Gitian stuff to the new hush3.git repo, that would be better than nothing.
@jahway603 installing guix from source looks brutal but thankfully there is an AUR package and a Debian 11 or 12 package and Ubuntu package starting at 21.04
@jahway603 installing guix from source looks brutal but thankfully there is an AUR package and a Debian 11 or 12 package and Ubuntu package starting at 21.04
jahway603
The hushd build should be reproducible. This issue will be used to keep track of various changes needed to make that happen.
More info at https://reproducible-builds.org/
As a next step, we could make a list of issues that make our build non-reproducible
@jahway603 are you aware of exactly what issues make hushd build currently non-reproducible ? How hard of a task is this?
The build always breaks when a new version of gcc is released. I can look into this further soon.
@jahway603 how does it break in regards to being reproducible ?
When gcc gets updated to a new version, the hushd code always needs to be changed to add some new header(s) for it to continue to successfully build.
This commit is an example of gcc13 fix
121ec4b9d4This commit is an example of gcc12 fix
dd9be59e4fI can't find the fix for gcc11 (or was it gcc10?)
@jahway603 I think that I haven't been clear in exactly what it means to be "reproducible". Your example of new versions of compilers requiring us to change Hush source code is definitely annoying, but that is orthogonal to reproducibility.
Reproducibility means the turning source code into a binary is 100% deterministic. Very common things that prevent a binary from being deterministic/reproducible are including the current time or date, the username or hostname of the bulid server, current timezone, the PATH when built, the current status of the git repo (you will see something like -dirty in the version when building manpages) and various other things. It is not related to needing to change source code because some external dependency changed.
Making code reproducible usually means changing the build system but can also require changing the code itself.
And to clarify more, reproducibility is per-architecture. We are not trying to make a Windows binary that is exactly the same as a Linux binary. We are trying to make it so that if two developers compile a Linux binary on their Linux servers, those binaries are byte-for-byte 100% exactly the same. A way to check this is that both binaries should have exactly the same SHA256.
The easiest way to get reproducible builds in Hush will be to port over how Bitcoin Core does things with Guix (which is something similar to Nix) : https://github.com/bitcoin/bitcoin/blob/master/contrib/guix/README.md
The BTC PR that added Guix support also has some interesting comments : https://github.com/bitcoin/bitcoin/pull/15277
BTC used to use a system called "Gitian" to do reproducible builds, which I helped port to Hush in The Olden Times for the old hush.git repo: https://git.hush.is/hush/hush-gitian
There are likely very good reasons why BTC Core decided to migrate away from Gitian and it's probably best for us to go with Guix vs Gitian, but if somebody wants to make a reproducible build by porting our old Gitian stuff to the new hush3.git repo, that would be better than nothing.
@jahway603 installing guix from source looks brutal but thankfully there is an AUR package and a Debian 11 or 12 package and Ubuntu package starting at 21.04