1 changed files with 323 additions and 2 deletions
@ -1,2 +1,323 @@ |
|||||
# mastering_dpow |
# Mastering Komodo Delayed Proof-of-Work |
||||
Mastering Delayed Proof Of Work |
|
||||
|
## by Duke Leto |
||||
|
|
||||
|
This is a short technical document aimed at developers who are interested to learn |
||||
|
more about DPoW and/or want to add it to their a cryptocoin to protect |
||||
|
against consensus (51%) attacks. These attacks are becoming very common |
||||
|
against coins from large to small. |
||||
|
|
||||
|
This book assumes basic knowledge of the Bitcoin protocol, Bitcoin source code, |
||||
|
C++ and basic compiler knowledge. No prior knowledge of Komodo is assumed, this |
||||
|
work contains a basic introduction of Komodo features. |
||||
|
|
||||
|
# High Level Overview of KMD DPoW |
||||
|
|
||||
|
DPoW sends blockhash data from your coin, to potentially another coin like HUSH, which is then sent to Bitcoin, |
||||
|
providing a small coin with a small hashrate the security of BTC hashrate. This |
||||
|
data is embedded in OP\_RETURN metadata which Notary Nodes continuosly |
||||
|
send. Notary nodes run full nodes of coins they inject data into. |
||||
|
|
||||
|
Delayed-Proof-of-Work is an abstract algorithm that is not tied to any particular |
||||
|
blockchain. Anybody can use it and implement the service. |
||||
|
|
||||
|
|
||||
|
## Global Notary Node Network |
||||
|
|
||||
|
In more detail, there exist a network of 64 notary nodes spread around the |
||||
|
globe. They are not in any one physical location, which makes the system |
||||
|
resilient against a datacenter outage or natural disaster in one geographical |
||||
|
location. The algorithm NNs use require 13 notary nodes to come together and |
||||
|
agree on what the latest Merkle root and other metadata that will be embedded |
||||
|
into the Komodo and Bitcoin blockchains inside the OP\_RETURN data of a |
||||
|
multisig transaction. This means that to take down the notarization process, to |
||||
|
stop the data flow, one must attack and knock offline 52 different servers in |
||||
|
diverse locations, simultaneously. In this case, notarization data would stop |
||||
|
flowing, but no "fake" or "incorrect" notarization data can be created, because |
||||
|
the source code of Komodo has the public keys of notary nodes embedded inside. |
||||
|
|
||||
|
Only notary nodes have the corresponding private keys to make transactions that |
||||
|
the network will trust as valid notarizations. So DDoS attacks can only stop |
||||
|
notarization data from flowing temporarily, they can not be used to knock |
||||
|
actual notaries off-line and then impersonate or man-in-middle the process. |
||||
|
|
||||
|
## Adding or changing DPoW is a Hard Fork |
||||
|
|
||||
|
To be clear, DPoW are consensus-level changes to a cryptocoin source code, |
||||
|
they require a coordinated effort of all exchanges/mining pools/users upgrading, |
||||
|
since it is a hard fork. Take this into consideration when planning upgrade |
||||
|
timelines. |
||||
|
|
||||
|
## Yearly Hard Forks |
||||
|
|
||||
|
Every year Komodo does a hard fork, which coincides with electing new notaries, |
||||
|
which usually happens in early May. Electing new notaries means changing the |
||||
|
KMD source code to list these new pubkeys and removing old ones. This means |
||||
|
that any coin using DPoW will *also need to hardfork at a coordinated time*, |
||||
|
i.e. coins using DPoW will also fork in May each year, after new KMD notaries |
||||
|
have been elected and the new notarization season begins. |
||||
|
|
||||
|
Coins can turn this yearly hard-fork into an opportunity to make any other |
||||
|
non-emergency changes that would be consensus-level hard-forks. It allows |
||||
|
a yearly upgrade window for each coin, where a hardfork is going to happen |
||||
|
anyway. For instance, the size of the maximum OP\_RETURN might be changed, |
||||
|
or a different difficulty algorithm used (test it thoroughly first!). |
||||
|
|
||||
|
Additionally, there is a preferred time of year where it makes the most sense |
||||
|
to integrate DPoW, essentially directly after the yearly KMD notary election |
||||
|
and hard-fork. That prevents a situation where adding DPoW integration in March |
||||
|
requires a hard-fork and then the KMD notary election requires another |
||||
|
hard-fork just a short-time later. Exchanges are not excited to update full |
||||
|
nodes and some exchanges attempt to charge for full node updates (which the |
||||
|
author finds scammy), so timing the integration of DPoW services can be very |
||||
|
strategic. |
||||
|
|
||||
|
# Double Spend Attack Prevention |
||||
|
|
||||
|
It is the authors recommendation that exchanges pause any transactions for |
||||
|
which data has not yet been notarized to the appropriate chain. For a game |
||||
|
allowing in-game purchases with digital currencies, it might allow transactions |
||||
|
up to $10 if the data has been notarized to Komodo, or no limit on transactions |
||||
|
that have been notarized to Bitcoin. This allows small-value transactions to |
||||
|
proceed very quickly (Komodo has a 1 minute blocktime) while ensuring |
||||
|
protection against double-spend attacks for large value transactions. This |
||||
|
protection comes at the cost of waiting longer, as Bitcoin has 10 minute |
||||
|
blocktime. |
||||
|
|
||||
|
It is important to understand that one half of the process is notarization |
||||
|
data, which must go from COIN -> KMD -> BTC, and the other half of Double Spend |
||||
|
Protection is actually looking at this notarization data and deciding if it's |
||||
|
safe to proceed with some action. Komodo is working on an API to see if a txid |
||||
|
of a certain dpow-enabled coin has been notarized to KMD and BTC, so that |
||||
|
exchanges and other parties wanting to protect against double spends can make |
||||
|
informed decisions, in much shorter time periods than waiting until 120 confirmations. |
||||
|
|
||||
|
The API for when notarizations have made it to the KMD chain is at |
||||
|
https://komodostats.com/api/notary/summary.json . |
||||
|
|
||||
|
# DPOWconfs protect against double spends with no code changes |
||||
|
|
||||
|
A recent innovation called "DPoW confs" allows |
||||
|
exchanges to seemlessly protect themselves against double spend attacks, with |
||||
|
no code changes at all! Exchanges can simple continue to look at the |
||||
|
`confirmations` key of whichever RPCs they use. DPoW confs is enabled by |
||||
|
default, and when enabled, the confirmation count of a transaction will never |
||||
|
go above 1, until it is notarized. |
||||
|
|
||||
|
To summarize |
||||
|
|
||||
|
* confirmations=0 is unconfirmed (same as before) |
||||
|
* confirmations=1 is confirmed, but not notarized |
||||
|
* confirmations>1 means confirmed + notarized |
||||
|
|
||||
|
Since exchanges currently require anything from 10 to 100 or more |
||||
|
confirmations, their current code already enforces that transactions are |
||||
|
notarized. |
||||
|
|
||||
|
The old/original concept of `confirmations` is now stored in the |
||||
|
`rawconfirmations` key of any RPC that returns `confirmations`. |
||||
|
|
||||
|
DPoW confs are defaulted to on in the latest versions of HUSH, exchanges |
||||
|
do not need to change any code, any HUSH daemon setting or CLI flags. They |
||||
|
are protected by default. |
||||
|
|
||||
|
The KMD official docs have more details about [dpowconfs](https://docs.komodoplatform.com/komodo/dPOW-conf.html). |
||||
|
|
||||
|
# Cost of DPoW |
||||
|
|
||||
|
The raw cost of 64 global notary nodes making transactions roughly once per |
||||
|
minute (the Komodo block interval) and continually on other coins has a cost. |
||||
|
|
||||
|
Various costs related to DPoW can be broken down into these categories: |
||||
|
|
||||
|
* Transaction fees denominated in HUSH (paid by notaries making transactions) |
||||
|
* Transaction fees in the coin being notarized (paid by notaries) |
||||
|
* Yearly cost paid to HUSH for DPoW services |
||||
|
* One-time integration costs to add DPoW to external coins |
||||
|
|
||||
|
Contact Hush [Telegram](https://t.me/hushcoin) for more info about pricing. |
||||
|
|
||||
|
Also note, that if you decide to migrate your existing coin to a [Hush Smart |
||||
|
Chain](https://github.com/MyHush/hush-smart-chains), there are no integrations needed or integration development costs. Just |
||||
|
the cost for notarization transactions. This can be accomplished by doing an |
||||
|
airdrop from your current chain to a new Hush Smart Cain, where people use their |
||||
|
private keys to unlock the funds they owned on the original chain. This can be |
||||
|
done by the project itself or Komodo can help with this process. |
||||
|
|
||||
|
# Integrating DPoW Into Your Coin |
||||
|
|
||||
|
Let us assume Alice is the lead developer of a cryptocoin based on the Bitcoin |
||||
|
source code. For example, since Litecoin and Zcash are Bitcoin forks, any LTC |
||||
|
or ZEC forks count as Bitcoin forks, too. The version of Bitcoin internals that |
||||
|
the coin forked is important, this determines which header file that will be |
||||
|
immediately compatible or at least a close starting point for development work. |
||||
|
|
||||
|
For example, if your coin is based on Bitcoin 0.15.x, there already exists a |
||||
|
header file that matches that version of Bitcoin internals exactly. The GAME |
||||
|
coin was the first external coin to start using dPoW and [this commit](https://github.com/gamecredits-project/GameCredits/commit/e65fe302111408c02d2bf7e286205d4273fa0fed) |
||||
|
shows how they integrated. This was first created by the author of Komodo, |
||||
|
jl777, and then the author of this document ported that header file to Bitcoin |
||||
|
0.11.x internals, which Zcash and all Zcash forks have as their internals. So |
||||
|
if your coin is a Zcash fork, you should use the |
||||
|
[komodo\_validation011.h](https://github.com/MyHush/hush/blob/e529ad8d35d4dabf66fa10982e056149414f179b/src/komodo_validation011.h) |
||||
|
header file. If your coin has SegWit support, or is 0.14-0.16 internals, the |
||||
|
[komodo\_validation015.h](https://github.com/jl777/chips3/blob/dev/src/komodo_validation015.h) |
||||
|
file is most likely the best starting point to integrate. |
||||
|
|
||||
|
For coins with older internals, such as BTC 0.10.x and earlier, use the BTC |
||||
|
0.11.x as a starting point, so less changes are required. |
||||
|
|
||||
|
If your coin has made various internals changes and selectively added BIPs |
||||
|
or other internals changes, you will most likely need to make various changes |
||||
|
specific to your coin. |
||||
|
|
||||
|
## Calculating Notarization Lag |
||||
|
|
||||
|
We will use Hush, a Zcash fork, as an example to estimate notarization lag, |
||||
|
i.e. the time it takes for notarization data to make it from one chain to |
||||
|
another. To estimate the "worst case" time it takes notarization to get to |
||||
|
Bitcoin, we can simple add up all the blocktimes involved, so we will be adding |
||||
|
blocktimes of |
||||
|
|
||||
|
HUSH + BTC = 75s + 600s |
||||
|
|
||||
|
which is 675s or about 11 minutes for blockhash data from Hush to be notarized all |
||||
|
the way to Bitcoin. This is the time it would take such that an exchange could |
||||
|
ask the Bitcoin blockchain if a given txid has been notarized. |
||||
|
|
||||
|
Note that the above block-times are "worst case" under normal network |
||||
|
conditions, i.e. a transaction is made a millisecond after a block, so it has |
||||
|
to wait the entire block interval on HUSH. And then is unlucky enough to have |
||||
|
to wait an entire KMD block interval and an entire Bitcoin block interval. |
||||
|
|
||||
|
It's possible that a bug or attack or large difficulty change makes one block |
||||
|
much longer than the average block interval time, which would increase the |
||||
|
time it takes for notarization data to make it to BTC. This is why a specific |
||||
|
time cannot be used to decide it's "safe", very often 11 minutes would be enough |
||||
|
time for Hush block hash data to get to the Bitcoin chain, but there could |
||||
|
be times when it is not. |
||||
|
|
||||
|
On average, users will only wait half of each coins block time, which |
||||
|
means 11/2 = 5.5 minutes for Hush blockhash data to be written to Bitcoin. |
||||
|
|
||||
|
## Is My PoW algorithm compatible with DPoW? |
||||
|
|
||||
|
There are many PoW algorithms to choose from these days and thankfully |
||||
|
the DPoW algorithm is blissfully ignorant of all these differences. It |
||||
|
does not care if something is SHA256 versus Equihash, all it sees is a |
||||
|
256 bit block hash, i.e. a 256 bit integer. It's just a number. Any |
||||
|
coin using a 256 bit blockhash is compatible, which includes most |
||||
|
known coins. |
||||
|
|
||||
|
|
||||
|
|
||||
|
## Integrating a Zcash fork |
||||
|
|
||||
|
For the specific case of adding DPoW to a Zcash fork, the 0.11.x header file, |
||||
|
first implemented for Hush, should be used. Zcash forks are characterized by |
||||
|
being pre-Segwit and having some differences in how UTXOs are stored in LevelDB |
||||
|
compared to post-Segwit coins. |
||||
|
|
||||
|
## Integrating a Litecoin fork |
||||
|
|
||||
|
For Litecoin forks, the version of BTC internals inside is what makes the biggest |
||||
|
difference. Being pre-Segwit means using the 0.11.x header file as a starting point, |
||||
|
and post-Segwit can use 0.15 header file. |
||||
|
|
||||
|
## Integrating a CryptoNote/Monero fork |
||||
|
|
||||
|
There is recent work in this area by BLUR, see this repo for details: https://github.com/blur-network/dpow-blur |
||||
|
|
||||
|
## Integrating a non-Bitcoin-derived coin |
||||
|
|
||||
|
It will be most likely be challenging to integrate a coin not derived from |
||||
|
Bitcoin source code but we are eager to see if it's possible. The currently |
||||
|
available 0.11 and 0.15 header files can be used if the project is in C++ but |
||||
|
if the coin is written in another language then those header files must be |
||||
|
ported to that language first. |
||||
|
|
||||
|
If your coin reimplements Bitcoin protocol from scratch, then it may still be |
||||
|
compatible. Please join #developer on Komodo Discord to ask more questions. |
||||
|
|
||||
|
### Integration Procedure |
||||
|
|
||||
|
This is a high-level procedure of how to integrate an arbitrary coin. It will |
||||
|
not describe every single step but it will be applicable to most situations. |
||||
|
|
||||
|
* Create a new branch for dpow work |
||||
|
* Copy the correct header file (komodo\_validation0XX.h) to the `src/` directory of your coin |
||||
|
* Copy the [komodo\_rpcblockchain.h](https://github.com/MyHush/hush3/blob/dev/src/komodo_rpcblockchain.h) to your `src/` directory. This file will not need |
||||
|
many, if any, changes |
||||
|
|
||||
|
* If you are already using BTC internals 0.11/0.15, skip the next step |
||||
|
* Make any API changes required to get header files compiling on your |
||||
|
specific version of BTC internals |
||||
|
* Make sure your code dynamically generates addresses (0.11 does, 0.15 does |
||||
|
not yet) |
||||
|
* Do a fresh sync of 2 new nodes on the branch with dpow code |
||||
|
* One node with have txindex=1 |
||||
|
* One node will not have txindex |
||||
|
* If both nodes can sync from scratch with no bad errors, then you are most |
||||
|
likely done with integration! |
||||
|
* Contact Komodo for the next round of instructions, which is Notary Nodes |
||||
|
testing your code |
||||
|
|
||||
|
* List of integrated coins and their header files |
||||
|
* [Hush](https://github.com/MyHush/hush/blob/dev/src/komodo_validation011.h) BTC 0.11.2 |
||||
|
* [EMC2](https://github.com/emc2foundation/einsteinium/blob/master/src/komodo_validation013.h) BTC 0.13 |
||||
|
* [KREDS](https://github.com/leto/kreds-core/blob/dpow/src/komodo_validation015.h) BTC 0.15 |
||||
|
* [GAME](https://github.com/gamecredits-project/GameCredits/blob/master/src/komodo_validation015.h) BTC 0.15 |
||||
|
* [CHIPS](https://github.com//jl777/chips3/blob/master/src/komodo_validation015.h) BTC 0.15 |
||||
|
* [SUQA](https://github.com/leto/SUQA-0.17.1/blob/dpow/src/komodo_validation017.h) BTC 0.17.1 |
||||
|
* [GIN](https://github.com/GIN-coin/gincoin-core/blob/master/src/komodo_validation012.h) BTC 0.12 |
||||
|
|
||||
|
### When do notarizations actually start? |
||||
|
|
||||
|
Here are the steps for notarizations to begin: |
||||
|
|
||||
|
* Consensus changes to full node |
||||
|
* RPC changes to full node |
||||
|
* Regtests for dpow (if applicable) |
||||
|
* Payment of Hush DPoW yearly service fee |
||||
|
* Notary nodes asked to install/configure new daemon |
||||
|
* Notaries work on any changes to iguana that are needed |
||||
|
* Notaries begin to install full node + sync |
||||
|
* Notaries update iguana changes for new full node |
||||
|
* When 13 notaries have synced+configured the full node, notarization rounds begin |
||||
|
|
||||
|
When all notary nodes have installed, configured and synced the new full node, |
||||
|
notarizations will happen with greater frequency. |
||||
|
|
||||
|
### Iguana Changes |
||||
|
|
||||
|
Iguana is the low-level software which actually does the complex work |
||||
|
of notarizing data across chains. Some glue is needed when new |
||||
|
coins are added to notarization process: |
||||
|
|
||||
|
* Update dpowassets, example: https://github.com/jl777/komodo/pull/887 |
||||
|
* 7776 file, example: https://github.com/jl777/SuperNET/pull/890 |
||||
|
* Other miscellaneous changes |
||||
|
|
||||
|
For instance, Hush was the first coin with 2 bytes of base58 prefix to |
||||
|
be notarized by KMD. This caused issues because the internals of the |
||||
|
coin assumes a 1 byte prefix in various places. Thankfully one of |
||||
|
the amazing notary node operators stepped in and wrote a good chunk |
||||
|
of C++ code to enable 2 byte prefixes, which makes many Zcash forks |
||||
|
compatible! |
||||
|
|
||||
|
### Notary Node duties when adding a coin |
||||
|
|
||||
|
This is a non-exhaustive list of things that notary node operator will do |
||||
|
to enable the new coin on production: |
||||
|
|
||||
|
* Verify notary addresses match those generated/hardcoded in code |
||||
|
* Install full node software |
||||
|
* Compile on correct branch |
||||
|
* Fully sync full node with txindex=1 |
||||
|
* Import their address to the new coin via a WIF generated from pubkey |
||||
|
* Run full node with -pubkey option |
||||
|
* Receive coins funds to be used in notarization UTXOs |
||||
|
* Do a round of testing with enough NNs to do a dpow round |
||||
|
to verify code more |
||||
|
* Set up automated procedures specific to each coin, such as UTXO splitting/etc |
||||
|
|
||||
|
Loading…
Reference in new issue