Increase nMinDiskSpace to make wallet.dat corruption less likely #84

Closed
opened 3 years ago by duke · 5 comments
duke commented 3 years ago
Owner

Recently I saw a new failure mode that we very much want to avoid users experiencing: A node ran out of disk space and corrupted a very large (620MB) wallet.dat in such a way that even -salvagewallet cannot recover it. The -salvagewallet argument fails even before printing anything to debug.log, even with -debug=1 .

./hushd -salvagewallet -zapwallettxes=2 -debug=1

cannot recover the wallet.dat and doesn't even print anything useful to debug.log before exiting. The wallet may be recoverable via external tools like wack: https://git.hush.is/duke/wack (which is why I wrote that, for similar problems long ago) but most users will not be able to use CLI tools to fix things like this.

We inherited a minimum free space limit of 50MB from BTC, but that is not sufficient for us:

  • BTC transactions are much smaller, a few hundred bytes, while Hush ztx's are about 10KB
  • A different process can be actively using lots of space very fast, and between checks for the minimum free disk space, hushd can totally run out of space, which is what happened here
  • GUI wallets take up more disk space because they have their own log files that use disk (A GUI wallet wasn't being used here, but most users do use GUI wallets)

I propose increasing the minimum free disk space to either 500MB (10x larger) or 1GB (20x larger) to make this problem much less likely to occur.

@onryo @odinzu @jahway603 what are your thoughts?

Recently I saw a new failure mode that we very much want to avoid users experiencing: A node ran out of disk space and corrupted a very large (620MB) wallet.dat in such a way that even `-salvagewallet` cannot recover it. The `-salvagewallet` argument fails even before printing anything to debug.log, even with `-debug=1` . ``` ./hushd -salvagewallet -zapwallettxes=2 -debug=1 ``` cannot recover the wallet.dat and doesn't even print anything useful to debug.log before exiting. The wallet may be recoverable via external tools like wack: https://git.hush.is/duke/wack (which is why I wrote that, for similar problems long ago) but most users will not be able to use CLI tools to fix things like this. We inherited a minimum free space limit of 50MB from BTC, but that is not sufficient for us: * BTC transactions are much smaller, a few hundred bytes, while Hush ztx's are about 10KB * A different process can be actively using lots of space very fast, and between checks for the minimum free disk space, hushd can totally run out of space, which is what happened here * GUI wallets take up more disk space because they have their own log files that use disk (A GUI wallet wasn't being used here, but most users do use GUI wallets) I propose increasing the minimum free disk space to either 500MB (10x larger) or 1GB (20x larger) to make this problem much less likely to occur. @onryo @odinzu @jahway603 what are your thoughts?
duke changed title from Increase nMinDiskSpace to prevent wallet.dat corruption to Increase nMinDiskSpace to make wallet.dat corruption less likely 3 years ago
Collaborator

A different process can be actively using lots of space very fast, and between checks for the minimum free disk space, hushd can totally run out of space, which is what happened here

For most users it won't be a problem, storage is cheap and I never heard someone run out of disk space, it can be easily increased to 1GB.

This line? https://git.hush.is/hush/hush3/src/branch/master/src/main.cpp#L3732

> A different process can be actively using lots of space very fast, and between checks for the minimum free disk space, hushd can totally run out of space, which is what happened here For most users it won't be a problem, storage is cheap and I never heard someone run out of disk space, it can be easily increased to 1GB. This line? https://git.hush.is/hush/hush3/src/branch/master/src/main.cpp#L3732
Collaborator

I think this is a good change and I think either 500MB or 1GB is fine for space.

I think this is a good change and I think either 500MB or 1GB is fine for space.
Poster
Owner

@onryo yes, that is the function which uses the the nMinDiskSpace variable which is defined in main.h, it is called from 5 different places. The optional argument it gets is how much extra space above nMinDiskSpace it should require, as a safety buffer for when it's not known exactly how much space will be used

src/main.h
171-/** Best header we've seen so far (used for getheaders queries' starting points). */
172-extern CBlockIndex *pindexBestHeader;
173-
174:/** Minimum disk space required - used in CheckDiskSpace() */
175-static const uint64_t nMinDiskSpace = 52428800;
@onryo yes, that is the function which uses the the `nMinDiskSpace` variable which is defined in `main.h`, it is called from 5 different places. The optional argument it gets is how much extra space above `nMinDiskSpace` it should require, as a safety buffer for when it's not known exactly how much space will be used ``` src/main.h 171-/** Best header we've seen so far (used for getheaders queries' starting points). */ 172-extern CBlockIndex *pindexBestHeader; 173- 174:/** Minimum disk space required - used in CheckDiskSpace() */ 175-static const uint64_t nMinDiskSpace = 52428800; ```
Poster
Owner

nMinDiskSpace has been increased to 1GB on dev branch

nMinDiskSpace has been increased to 1GB on `dev` branch
Poster
Owner

this was done a while ago

this was done a while ago
duke closed this issue 2 years ago
Sign in to join this conversation.
No Milestone
No project
No Assignees
3 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Loading…
There is no content yet.