Windows cross-compiling broken by RandomX v1.2.1 #367

In our 3.9.4 release we used an older version of Randomx df6e15e1303034831dc14128d654c157ddc0dce3 (which was from their master branch, not a release) which worked well for us.

When we upgraded to RandomX v1.2.1 (102f8acf90a7649ada410de5499a7ec62e49e1da) which was used in the 3.10.0 release we ran into compile failures when trying to cross-compile for windows. Our temporary fix to get release binaries made was to downgrade to the old version of RandomX on the dev-old-randomx branch and use that branch to produce Windows binaries.

It's a bit confusing but Linux and Mac 3.10.0 use the RandomX 1.2.1 release while Windows uses the older code. This means that some performance improvements in 1.2.1 are not available on Windows and so mining DragonX and other RandomX HSC's will be slightly slower than using Linux or Mac.

This is the error:

/usr/bin/x86_64-w64-mingw32-ld: /hush3/src/RandomX/src/dataset.cpp:147: undefined reference to `randomx::JitCompilerX86::enableExecution()'
/usr/bin/x8664-w64-mingw32-ld: RandomX/build/librandomx.a(dataset.cpp.o): in function `void randomx::deallocCache<randomx::AlignedAllocator<64ull> >(randomxcache*)':
/hush3/src/RandomX/src/dataset.cpp:65: undefined reference to `randomx::JitCompilerX86::~JitCompilerX86()'
/usr/bin/x8664-w64-mingw32-ld: RandomX/build/librandomx.a(dataset.cpp.o): in function `void randomx::deallocCache<randomx::LargePageAllocator>(randomxcache*)':
/hush3/src/RandomX/src/dataset.cpp:65: undefined reference to `randomx::JitCompilerX86::~JitCompilerX86()'
collect2: error: ld returned 1 exit status
make: * [Makefile:2272: hushd.exe] Error 1

Submitted issue to their git here: https://github.com/tevador/RandomX/issues/288

They suggest -DARCH_ID=x86_64 to fix the problem

Unfortunately this did not work, but gave me a clue. Old CMakeLists.txt has this:

if(ARCH_ID STREQUAL "x86_64" OR ARCH_ID STREQUAL "x86-64" OR ARCH_ID STREQUAL "amd64")

while new has this

if((CMAKE_SIZEOF_VOID_P EQUAL 8) AND (ARCH_ID STREQUAL "x86_64" OR ARCH_ID STREQUAL "x86-64" OR ARCH_ID STREQUAL "amd64"))

Removing CMAKE_SIZEOF_VOID_P appears to work so src/jit_compiler_x86.cpp is included in build. It is some additional check for 64-bit that's apparently failing.

System is 64-bit so I'm not sure what's up, but I'll test the bins it created now.

@fekt are you using DARCH=native and DARCH_ID=x86_64 or just DARCH_ID=x86_64 ?

I tried with DARCH_ID=x86_64 only on a fresh clone and it did not work. I then tried with both and also DARCH=x86_64 and none of those options worked either.

The build is set to x86_64-w64-mingw32 for MXE. Bin seems to work fine with mining and shows as 64-bit so I'm not sure what's up with that check.

@fekt if you found something that works, it sounds good to me. I looked at the history of that line of code and it was added in 3f69ad7b79 and seems to be related to fixing a bug in 32bit systems. It looks like in fixing 32bit they broke cross-compiling, so you may want to report it to them as a bug and what you did to fix it.

I fixed in 032f9b62da and let them know in issue that removing CMAKE_SIZEOF_VOID_P worked.

Should we update all of the Windows bins with this version and update download links everywhere? There's a decent amount of Windows downloads already and will need to tell people to update again too if they want RandomX improvements.

@fekt I am not sure what we should do right now, seems like a team decision.

If we update 3.10.0 windows bins to use RandomX v1.2.1 it will later on be very confusing as to which version of RandomX people are using. They will all identify as 3.10.0 . We are also unsure if we will need another full node release soon to deal with getblocktemplate issues we are seeing.

What seems more reasonable is to make a new SD/SDX release, 1.4.1, that contains binaries from the dev branch for Windows. Then we will know that if people on Windows are using SD/SDX 1.4.0 they have the old randomx and if 1.4.1 the new randomx.

RandomX v1.2.1 speed improvements versus the old version are roughly in the 1% area, while the speed improvements in the way we use randomx code is about 10% faster vs the previous release, to put things in perspective. 1.2.1 is technically faster but most of the speedup was in our code changes, not randomx code changes.

I think adding to a new release makes most sense and probably doesn't need to be done right now. It really only affects DGRX users.

1.2.1 is technically faster but most of the speedup was in our code changes, not randomx code changes.

There was at least one DRGX Windows user reporting greater performance with new release and old RandomX so they are seeing improvements just with the code changes.

@fekt

No matter what, DRGX miners are going to see a large speedup if they use the new 3.10.0/1.4.0 release versus the old release. When we fixed the mining coredumps/bugs it lead to a pretty large 10% hashrate increase across all OS's.

Now, for Windows, they still get that ~10% speedup but lose out on the much smaller ~1% speedup of using RandomX 1.2.1 versus the older version we used to use. That ~1% speedup also will depend on the exact CPU and RAM they use and most people won't really notice it.

This change will be in the upcoming 3.10.1 hushd release so I think this issue can be closed