Categories
Feature Story

TurtleCoin­® v2 Overview

In the early days, at least two forevers ago, we blew the dust off a legacy Bytecoin codebase and launched a coin using just enough special sauce to call TurtleCoin our own, all on a whim. As a new network, there were the usual struggles, like when ASIC for CryptoNote emerged, or when we found a pretty serious RPC issue in Simplewallet.

As our team and their contributions grew, we eventually found ourselves in a new codebase, rock solid, but still carrying the scars of our past.

Fast forward to now, we are 3 million blocks wiser, our team is many lines of code more experienced, and as we have chiseled our way through, we made a list of a some areas where we can do better. Taking a step back, the last few years have shown us that we are reaching the limit of our current software, and the time is now for us to think radically different and put something new on the table. It is time to build something of our own.

With great respect and thanks to the CryptoNote family, Forknote, Bytecoin, and those who enabled us to get where we are now, we humbly begin our own path and embark on a new journey. What we produce next will be from scratch, written by us, exactly how we have always wanted it.

TurtleCoin v2 is the future, and we are excited to share it with you!

– RockSteady

The Goal

We aim to redesign TurtleCoin from the ground up to avoid the mistakes of the past while putting an eye to the future state of the network. This is accomplished by replacing traditional Proof of Work (PoW) based mechanisms with a round-based hybrid Proof of Stake (PoS)/PoW model. We’re going to be talking about some of the fundamentals required in this model so buckle your seat belt and be ready for a ride. Be warned, there’s a bit of math below.

Proof of Work

The traditional approach to cryptocurrency networks is to deploy a PoW model such that miners perform endless repetitive hashing of block data searching for a hash that meets a set difficulty. The miner is rewarded for burning fossil fuels via a specially constructed transaction (the “coinbase transaction”) that allows the miner to create new funds as a result of mining that block.

Proof of Stake

PoS deviates from the PoW consensus model in that PoW is typically no longer performed, instead passing the torch to block producers (the “producers”) whose work is then validated by the block validators (the “validators”). These parties are typically elected via users staking, or otherwise backing, their preferred candidates for those roles. Selection of producers and validators typically takes place via an election mechanism as part of the consensus rules.

Our Vision, Round Based Hybrid Delegated PoS w/ PoW

We are planning on combining the best of PoW with the stability and speed of PoS to create one kick ass network. We’ll use DPoS for managing the distributed ledger and PoW as part of the transaction construction and validation process.

Operational Overview

When a user wishes to send a transaction, they generate the transaction in the “normal” way. However, there is a nonce field included in the transaction that requires for the user to perform PoW for that singular transaction to meet a preset difficulty for transactions. The resulting transaction and the nonce is then forwarded to a node for broadcast to the network.

Every transaction will require a base amount of work (PoW) before it can be submitted to the network in addition to a fee-per-byte network transaction fee. However, if a user wishes to discount their transaction fee for a particular transaction on the network they may submit a PoW for the transaction that meets yet a higher tier of difficulty. While the discount tiers have yet to be defined, the maximum discount for the transaction fees will not exceed 50% of the fee-per-byte fee. There are no free rides.

Fusion transactions will no longer exist on the network. As a wise man once said, “ass, cash, or hash, no one rides for free”. Everyone pays their fair share for transacting on the network regardless of the reason for that transaction.

Once a transaction is received by a node, the node performs basic checks (i.e.. transaction construction, valid nonce, PoW meets required difficulty, verifies signatures, etc.). Once that node completes the basic verification, it forwards the transaction to the rest of the network including the producers. Producers hold the transactions in the global memory pool after performing their own validations of the transaction ensuring that it is indeed a valid transaction. At this point, users can presume some level of assurance that the transaction will be committed to a block solely because the producers have accepted the transaction into the pool.

Once enough PoW transactions have accumulated in the pool to satisfy the network difficulty, the then current producer creates a block that ratifies the selection of transactions necessary to meet (or exceed) the network difficulty requirement. When doing so, the producer creates a block reward equivalent, 1 for 1 matching the aggregate transaction fees included in the block.

This means that emission is directly tied to the actual use of the network as a transactional platform. We will no longer be emitting currency on a pre-defined schedule as a result of this change. The more the network is used, the more currency that will become available.

The block reward is distributed proportionately between the block producer, the validators, and a subset of the users who staked that producer. Upon completion of the block the block is signed by the producer and transmitted to the network.

The validators, upon receiving the block, validate (at minimum) that:

  • The producer was permitted to produce the block (their turn)
  • Enough PoW transactions have been included to meet network difficulty
  • Transaction hashes included in the block have been included such that the hashes are arranged in descending order
  • The block is properly signed by the producer
  • The transactions in the block meet all of the consensus rules of the network (no double spends, etc.); and
  • No unnecessary data is included in the block
  • That the producer claimed and paid out the proper block rewards.

A minimum of 51% of the validators must then sign the block and submit their signatures to the network thus ratifying the block itself.

After the block is ratified, the network difficulty is adjusted (if necessary) to account for things such as the number of transactions included, the PoW of transactions included, and any other metrics required to keep the network operating correctly.

No Empty Blocks Means Storage Saving Is Built In

One of the largest benefits in this scenario is that we no longer produce empty blocks and thus save tons of storage space. This makes the chain faster to sync, easier to store, and overall much easier to manage.

Tut, Tut: The Reward Lottery

Gone are the times when a single miner/pool received the entirety of a block’s reward. In this new model, the block reward is distributed to multiple recipients via the payments to the producer, a pseudo random validator, and a pseudo random subset of users who have staked the given producer and validator.

The Reward, Divided

We’re still playing with the numbers but there’s a pretty good chance that the block rewards will be divided as follows:

  • 20% to the producer of the block
  • 20% to the one of the validators of the block
  • 30% to the users who staked the producer
    • Maximum of 10 randomly selected stakers
  • 30% to the users who staked the validator
    • Maximum of 10 randomly selected stakers

Becoming a Block Producer / Validator

A user (“Alice”) wishing to operate a block producer or validator “announces” their intent for candidacy by staking their node(s) with a minimum stake of a yet to be determined amount of TRTL. Upon successfully staking their node, Alice’s node is now a candidate for either a producer or validator spot in a round. It is important to note that Alice simply staking her node does not automatically make Alice a block producer or a validator.

To be a candidate for either role within the network, Alice must be “endorsed” by other users of the network via users staking their funds in support of Alice. Once Alice has at least 1 TRTL staking her, she is now a candidate in the running for a producer or validator spot.

The details of how these stakes are sent, recorded, recalled, etc. will be discussed in further articles and will undoubtedly involve a lot of math. Don’t worry, we’ll warn you ahead of time so that you can bring a protractor and a compass.

Electing Producers & Validators

Producers & Validators are elected for spans of blocks such that each producer is entrusted to create ten (10) new blocks in the round for which they are elected. How they are elected into these rounds takes place as follows.

In the event that a producer or validator fails to perform the work that was entrusted to them, their “trust” rating will fall for each block that they fail to producer and/or submit a validation result. If their trust rating falls below the lower threshold (TBD), the stake they put forth to submit their node as a candidate will be locked indefinitely (aka. burned).

First, we take the hashes of every block in the now closed round \{B_1, B_2, B_3, \dots\} and calculate the Merkle root for those hashes to establish M as the election seed for the next round.

We then take M and convert that to a scalar p:

p = H_s(M)

Then compute the public key P of p:

P = pG

We then tally T the individual bytes of S to determine if the result is odd or even.

T = \{S_1 + S_2 + \dots\} \mod 2

  • If T is odd (T \equiv 1) we collect public keys from the candidates list that are T \geq P.
  • If T is even (T \equiv 0), we collect public keys from the candidates list that are T \leq P.

Once the collection of public keys in the new defined range have been collected. We review the number of votes (stakes) received for each public key and construct a listing of candidates with their tally of active votes. The list is then sorted by vote count in ascending order and the top and bottom candidates in the results are discarded. The total votes submitted for the remaining candidates are then tallied to arrive at a total votes casted count (V).

The resulting candidates are then assigned ranges for which the selection algorithm must land for them to become a producer/validator.

If the remaining candidates consist of:

Candidate 1: 11,583 votes
Candidate 2: 25,749 votes
Candidate 3: 99,000 votes
Candidate 4: 127,342 votes

The the ranges for each node are as follows:

Candidate 1: 0 - 11583
Candidate 2: 11,584 - 25,749
Candidate 3: 25,750 - 99,000
Candidate 4: 99,001 - 127,342

We then perform the selection of the first producer via e_1 = p \mod V. The value of e_1 is compared against the node ranges thereby establishing that node as a producer.

On the next iteration of the election process, the previously selected node (e_n) is removed from the collection and V is recomputed before the selection algorithm runs again. Thereby guaranteeing that we do not elect the same producer/validator into more than one round slot.

The process is repeated until the required number of producers (more detail later) have been selected for the next round. At which point, the same process is performed for the candidates on the opposite side of P as T had dictated for producers until we have selected enough validators for the next round.

In the event that we are unable to select the necessary number of producers or validators from the pool of candidates as required for the round (i.e.. there are less candidates than there are required slots) the number of required producers/validators will be decreased until there are enough candidates to fulfill the request; however, at no point shall the number of producers and/or stakers for a round be an even number of spots.

There will always be a minimum of three (3) producers and/or validators required in a round. The number of required producers and/or validators for any given round will adjust dynamically (details to come later).

Pseudo Random Selection of Validator & Staker Selection for Block Rewards

When a block is produced, the producer must include relevants outputs to not only pay themselves but to also pay a deterministicially selected validator as well as stakers that staked the producer and that validator.

To select who receives the reward(s) we perform the same selection algorithm as we do for producers and validators with the exception that the prior block hash is used in place of the Merkle root for the previous round (M).

In the case of selecting which validator gets rewarded, the process is ran exactly once (1) for the set of validators for that round.

For determining the stakers to reward, the process is completed exactly ten (10) times for the set of stakers staking the winning producer/validator and the number of stakes the staker has submitted for the given candidate determine their “vote count”.

If, any time during the loop used in selecting stakers, the number of available stakers is exhausted (there are less than 10 stakers that backed that producer/validator), the process resumes as if all stakers are included in pool again and the iteration(s) continue until the necessary ten (10) stakers are identified for reward.

After completing the above, the producer then constructs a transaction for inclusion in the block that rewards the parties in the percentages necessary such that there are twenty-two (22) outputs in the miner transaction that is included in the block.

  • 1 for the producer
  • 1 for the selected validator
  • 1 each for the 10 stakers of producer
  • 1 each for the 10 stakers of validator

The entire process is designed with a sense of pseudo-randomness that can be recreated by every node on the network using the defined process. In essence, the whole reward structure is a bit of an election with an upper and lower house as well as a lottery for those that backed the elected producer/validator.

The Issues in a Nutshell

Relaunch

Problem: The current blockchain is very, very, very big, and growing every day whether transactions are included in blocks or not. There is also an insane amount of bloat in the chain including transactional spam (fusions anyone?), CantiPixels, and gobs of other useless data that is like walking around with cement shoes with two boat anchors tied chained to your ankles.

Solution: Relaunch the chain. Out with the old and in with the new. To facilitate this, we’re going to need to let the coins on the old network burn in a provable way to prevent errant inflation or other nefarious incidents. In addition, the old chain must be spun down in such a way that the chain officially ends at a certain block. Users need a way to transfer their funds from the old chain to the new chain and that means we’re going to need a…

Coin Swap

Problem: We have to efficiently move funds from the old chain to the new chain.

Solution: A coin swap window of opportunity. The coin swap window to the new chain will be 1,000,000 blocks (as measured by the old chain) during which time users will submit transactions on the old chain to a predetermined wallet address that will serve as a burn address whereby no one user can pick up the funds out of that address (this will be a multisig N:N wallet to make sure no one party controls it). An outside watcher server will watch for transactions on the old chain going to that wallet as instructed by the swap “service”, and release funds on the new chain to the address/keys provided by the user. The swap itself will be funded via a pre-mine on the new chain equivalent to the estimated circulating supply at the time of of the end of the old chain.

After the swap window completes, the remainder of the funds left of the premine will be redistributed via normal block rewards as transaction fees on the new chain.

Note: The redistribution of any unclaimed (whether through inaction or as “lost” wallets) via block rewards will be executed over a number of rounds on the new chain to attempt to ensure a somewhat even distribution of those funds to producers, validators, and stakers. Details on the exact schedule of distribution will come later as it will be highly dependent on how much TRTL will have to be dealt with.

In short, the swap mechanism will be a website that helps you transition from old coins to new coins. The technical mechanics of which will be discussed in later articles.

Wait… the chain is massive… we can’t just carry over everything from old to new we have to think about…

Swapping 107,198,508+ Outputs…

Problem: Swapping over 100M outputs to the new chain is… let’s put it simply, incredibly stupid. If we were to do that, yes, the new chain would not have the history of all of those outputs, but considering that we don’t know which outputs are still spendable, there is an unknown number of outputs that we have to create on the new chain as part of the swap process. That number is somewhere between 0 and 107,198,508 (as of the time of this writing). That’s just insane not only to think about but to also execute on.

Solution: During the relaunch of the network and required coin swap, we will be performing the equivalent of a 100,000:1 TRTL (or 10,000,000:100 atomic units) swap while maintaining two decimal places. Swaps of less than 1,000 TRTL will simply become dust and will not be swappable. Don’t worry though, the swap “service” will gladly take your unoptimized outputs and give you brand spanking new optimized outputs.

This swap ratio will eliminate the need to transfer more than ~81% (as of the time of this writing) of the outputs on the old chain thereby avoiding the pollution of the new chain with much of the problems that exist with the current chain.

A Note on Coin Swap Transaction Fees

The swap service will be set up such that as long as 100% of the funds in a transaction sent to the swap service are actually sent to the swap service (no change outputs) then all network transaction fees you pay to send those coins to the swap service will be included in the amount that you receive on the new chain. Furthermore, the swap service will have the ability to bypass the new chain transaction fee mechanics so that you receive the full value of your swapped funds at the ratio given (truncated to two decimals).

A Sustainable, Usage-Based Emission

Right now our blocks come out every 30 seconds, empty or not, and this keeps our emission at a steady pace toward the maximum of 1 Trillion TRTL some time over 100+ years from now. On the new network, there are no empty blocks, and block times won’t mean anything, so we cannot rely on that timing to release new coins into the supply.
Given that the converted supply is quite low (10 Million, two digits after decimal), and block time is now irregular, we have come up with a new way of keeping new coins entering into the system.
On TRTL v2, the network will match all fees 1:1 with additional coins, making our emission tied to actual usage on chain. 10 Million would be a soft cap, and given enough usage to provide a demand for more coins, more coins will automatically become available via the fees produced by transaction volume.

Now What?

From here, we’re working at solidifying protocol level communications, working on designing the next generation p2p network itself, working on basic structures (transactions, blocks, etc.) and defining some of the inner workings of the actual consensus mechanisms that we’ll be introducing. We’ll provide additional details as they become available. Until then, keep chatting and don’t forget to let us know what you think at http://chat.turtlecoin.lol

Categories
Feature Story

TurtleCoin v1.0.0 Upgrade

image

v1.0.0

Master Build Status

Build Status

Special Notes

Upgrade to this release is required

Network Upgrade at block 3,000,000

This release includes a mandatory network upgrade at block 3,000,000 that implements a change to the Proof of Work (PoW) algorithm for the network to Chukwa v2 (1 Thread, 1MB memory, 4 iterations). Pools, nodes, and services are required to upgrade to this new PoW to continue mining blocks at block 3,000,000 as blocks mined with the previous version of Chukwa will be rejected.

Daemon API Changes

The daemon API has changed significantly in this version. The vast majority of the API calls have been removed, migrated, or otherwise adjusted such that all services using such API require that you perform updates to your packages. The API calls have moved to a most RESTful approach and the endpoint routes have changed. Please see https://meta.turtlecoin.dev/proposals/daemon-api/ for information on what the new API looks like and update your packages accordingly.

Note: Please be mindful when updating your packages that the property names in the returned responses including their name, case, structure, and/or data type may have changed.

Client / Wallet Upgrades are Mandatory

The API changes indicated in the daemon require that you use the latest version of the core wallet software. If you use a wallet other than wallet-api or zedwallet++ that wallet software must be updated to support the new API. As such using older wallet software with public node(s) that have upgraded to this release will fail until your wallet software is upgraded.

End of Life (EOL) and End of Support (EOS) Notice

Please note that v0.28.3 was the last release to contain turtle-service. This release no longer contains the turtle-service binaries or source code and thus is no longer supported. You must convert to wallet-api to continue working with the network as it is the only service to be upgraded in conjunction with numerous daemon API changes. If you have not converted your wallet(s) and or services over to Proton, zedwallet, or wallet-api you are encouraged to do so now. Should you experience any issues working with wallet-api, please make sure that you open an issue so that the issue may be resolved as quickly as possible.

Legacy Wallet Support Removed

Support for legacy wallets (ie. those created with simplewallet and/or turtle-service) has been removed from the project. We highly suggest that you upgrade your wallet(s) using a prior version of the core suite via wallet-api or zedwallet++ or recreate your wallets from the recovery mnemonic or keys using zedwallet or wallet-api.

As always, make sure you have backed up your keys as that is the only foolproof way to restore your wallet in the event of an issue.

Release Notes

General Updates

  • Bump version to 1.0.0
  • Removed turtle-service
  • Removed NodeRpcProxy
  • Removed support for legacy wallet versions
  • Removed WalletUpgrader
  • Removed support for upgrading legacy wallets to new wallet format
  • Removed WalletGreen
  • Implemented consensus change at block 3,000,000 via major block v7 to Chukwa v2
  • Replaced all usage of nlohmann::json with rapidjson
  • Updated coverage of TX_EXTRA field parsing
  • Numerous CI/CD process changes in support of newer GH runners

TurtleCoind Updates

  • Required P2P minimum version bump to v11
  • Upgraded API endpoints to RESTful style calls (see. https://meta.turtlecoin.dev/proposals/daemon-api/).
  • Resolved a bug in the retrieval of headers such that the iterator was decrementing below 0 and thus trying to look for blocks that will not exist
  • Change the /indexes endpoint to be inclusive of the end height supplied
  • Added explorer property to the /info API call
  • Adjusted TX_EXTRA field ordering in block templates such that they are ordered by tag type (ascending)
  • Catch and handle segfaults when block is not found in some explorer methods

zedwallet++ Updates

  • Updated user expoded file path opening with error messages
  • Added --reset CLI flag
  • Added rewind to height CLI flag
  • Updated WalletBackend to rewind on subwallet import

wallet-api Updates

  • Updated WalletBackend to rewind on subwallet import
  • Added rewind endpoint (see documentation for more information)
  • Prevent the import of wallets that already exist in the wallet container
  • Update Regex matchers in API endpoints

miner Updates

  • Updated support for new daemon API
  • Updated support for Chukwa v2

cryptotest Updates

  • Inclusion of Chukwa v2 test hashes and verification

Known Issues

  • N/A

How To Sync Quickly

Visit the checkpoints how to use site for simple to follow directions on using checkpoints.

How To Compile

Please see the How To Compile section of the project README for instructions on how to compile this release on your system.

Changelog

See the TurtleCoin Release page for the full change history.

Thanks

Cryptonote Developers, Bytecoin Developers, Forknote Project, TurtleCoin Community

Categories
Feature Story

The TurtleCoin Journey: 2 Years and Counting…

Earlier this month, TurtleCoin celebrated its two year anniversary of starting the project. The community and project have come a long way in the last 24 months and we are all proud of what we, as a community, have accomplished during that time. That’s not to say that we didn’t have a few set backs, learning opportunities, and challenges to overcome that tried our patience at times and ultimately drove the community to make changes to how the network and core suite operates. From multiple PoW algorithm changes, difficulty calculation changes, and a move to static mixins, each change was evaluated and adopted by the community with excitement and hopes for the future health of the network.

We’ve heard the pleas of the community over the last two years and by far the majority of the requests revolved around the folowing:

  • What is the community doing about the chain storage size?
  • When can the community expect more stable daemon/node operation?
  • Is RockSteady Satoshi? (TBD)

These pleas have not fallen on deaf ears and the result of those requests include a culmination of upcoming changes to the network and software with the next release of the TurtleCoin core suite. We are very excited about these changes and believe they are important for everyone to understand.

zedwallet (Legacy Edition)

The application formerly known as simplewallet

Zedwallet Legacy: End of Life

We announced in the v0.20.0 release notes that the legacy version of zedwallet contained in that release would be the last and final release containing legacy zedwallet. This statement still holds true and the code for that old version of zedwallet has been removed from the project. The next release of the core suite replaces the binary named zedwallet with zedwallet++ which is a new, faster, stronger version of the CLI wallet software. The zedwallet binary released going forward will be powered by zedwallet++.

Zedwallet Legacy: End of Support

We also announced in the latest release, that support would end for legacy zedwallet December 31, 2019. Starting January 1, 2020 if you ask for help with legacy zedwallet issues, we will kindly insist (with a stick in hand) that you upgrade your wallet using the latest release of zedwalletwallet-upgrader, or wallet-api. Again, the old code has been removed, lost, buried, and forgotten about at this point.

But, Why?

Don’t be that guy, don’t say you didn’t know.

Simple: portability, maintainability, and scalability. Legacy zedwallet (also turtle-service) based on WalletGreen creates wallets using a very unfriendly internal wallet schema that makes it very difficult to work with in different languages, platforms, and etc.

The new hotness, WalletBackend, creates a friendly, easy to use, fully documented, and portable wallet schema that is a JSON object when decrypted. WalletBackend lowers the barrier of entry for other developers to work with data created by the core tools and thus makes it far easier to develop other cool portable projects like TonChan and many others.

Deterministic Subwallets

It’s pretty much like a choose your own adventure book

We’ve updated the code in WalletBackend to default to the deterministic creation of subwallet addresses and support the restoration of those addresses in a deterministic way. This change is designed to help services in being able to quickly restore subwallet in the event that they lose any subwallet keys.

Instead of generating new random spend keys for each new wallet created in a single container, new wallets in the same container are now derived from the primary wallet address. The details of the how we derive subsequent private spend keys for subwallets is detailed in Issue #831.

Making subwallet private spend key generation deterministic allows for the restoration of any subwallet by just maintaining its index (position) in the wallet container. No more need to store all those extra keys! This is a major improvement for the teams in the community that handle subwallet addresses and users who use subwallet for a variety of purposes.

TurtleCoind Threading

Threading solves everything

RPC

Zpalmtree has worked tirelessly to implement process threading into the core daemon for the RPC interface. This helps keep the RPC interface responsive during heavy load and greatly reduces the impact to the network when the transaction pool gets hit with an influx of transactions. This change will help minimize the effects of heavy load on the network and improve the overall stability of node operations.

Transaction Input Validation

Zpalmtree also added support for threading of the input validation routines for new transactions entering the network. This allows inputs to be validated in parallel and thus drops the transaction validation time considerably. In doing so, transactions enter the network, are propagated across the network, and are ultimately included in blocks faster.

Transaction Pool Changes

How the transaction pool is managed by the core software has been changed recently to help decrease the effects of transaction spam, chain bloating, and other less than helpful actions on the network. We quickly noticed some less than friendly transaction activity taking advantage of our 30 second block times, low network transaction fees, and our good nature as turtles to bloat the chain with low-cost and free transactions that consisted of only breaking apart their funds into shells and recombining them into whole TRTL over and over again.

Transaction Selection Process

We are not the sort to create our own sorting algorithm to sort our sortables

We’ve improved upon the transaction pool ordering and selection criteria for when a miner creates a new block. In short, we’ll always de-preference transactions that are fusions and low total amount transactions as the odds are considerably better that such transactions are “spam”. As always, miners will prefer transactions with higher fees, size, and other metrics to move transactions through the network faster.

Fusion Transaction Limits

We’ve employed additional changes to the transaction pool where each node’s copy of the transaction pool can no longer accept any more than 20 fusion transactions at a time. All other fusion transactions will be rejected until there is room in the transaction pool for more.

This may result in an extended number of fusion rounds for highly disorganized (non-optimized) wallets. If you haven’t fused your wallet in a while, you might want to do so soon.

The line for the Coinstar starts at the back…

Network Upgrade at Block 2,200,000

We have a few consensus changes coming at block 2,200,000 to help improve the stability of the network. These changes are designed to be as transparent as possible but some will reach out and hit you in the face.

Some of these changes are the result of previous discussions in a meta issue and a thorough review of where the network and software can improve the overall experience of the community while maintaining the goals of the project.

Events unfolding over the last few months solidified in our minds the fact that some people just want to see the world burn and as a community, we must take precautionary steps to ensure the longevity of the project.

Output Creation Limits

Starting at the next network upgrade, the network will no longer accept transactions that create more than 90 outputs per transaction. Why apply a maximum you may ask?

+3,500 new 0.01 TRTL outputs created… why do we need those?

There is no legitimate use case in which a transaction need ever create 3,500+ outputs. All this type of transaction traffic manages to do is bloat the blockchain at the expense of every community member. It also ultimately causes the reverse effect and bloats things even further by stuffing the transaction pool and chain full of fusion transactions. Such transaction cycles serve absolutely no purpose and degrade the experience of all users.

Oh yeah… that’s why we need those: fusion spam.

Fee Per Byte

The Backstory

While setting the minimum transaction fee to 0.10 TRTL at the project’s launch sounded like a good idea at the time, such a minimum has serious long term implications. Most importantly, it costs just 0.10 TRTL to send data (transactions) on the network and that data is stored indefinitely and processed by thousands of nodes across the globe thereby consuming a growing amount of resources.

Using such resources is not free and unfortunately, as the chain grows, the resources required to operate nodes grows. The more resources required to operate a node, the less people will be able and willing to commit the necessary resources. Simply speaking, 0.10 TRTL per transaction is not maintainable for the long haul operation of the network.

Okay, Great, Now What?

That math stuff you hoped you’d never need, well… I hate to tell you…

Keeping this in mind, with the network upgrade at block 2,200,000, the network will require a minimum fee for each transaction that is based on a calculation of 0.01953125 TRTL per byte of data of the transaction. This equates to 5.00 TRTL for every 256 bytes of data sent across the network.

Increasing the minimum network fee for transactions means that each transaction is required to pay for its existence and the consumption of network resources proportionate to its size in bytes.

This means that the if you send a transaction using a lot of inputs, or a lot of outputs, contains extra data in the transaction, payment IDs, or etc. (increasing the size of the transaction), you’ll be required to pay your fair share to use the network to transmit and store that data.

As the fees for each transaction are included in the miner block reward, miners will be rewarded with these higher fees for processing transactions. The vast majority of nodes on the network are operated by miners (and pools) so the increase in fees will help to offset the costs of operating those resources to run the network.

So… What Now?

What does this mean for the average user? You’ll see an automatic calculation of the minimum network fee when sending transactions. Depending on how optimized your wallet is, how much you’re sending, and what data you include in a transaction, you’ll see the minimum network fee climb from 0.10 TRTL to about 10.00 TRTL at a minimum.

Fusion transactions will remain free so we highly recommend that you keep your wallet optimized. As a miner, you’ll also benefit from raising your minimum payout with mining pools to higher thresholds as you’ll indirectly pay less in fees for the transactions that pay you for your mining efforts.

If your wallet is mostly composed of shells, you’ll want to optimize your wallet before this new fee structure kicks in. If you mine only on the biggest pool or make most of your TRTL through tips, you’ll definitely want to optimize your wallet immediately.

What’s The Catch?

Sending small transactions, when we needed more mixable outputs for fungibility purposes, helped provide the necessary data to make everything possible. This could be done at a relatively low, low, low cost of 0.10 TRTL; however, with this change, sending 1.00 TRTL to someone will now incur a cost of 10.00 TRTL. We’re sure you can do the math there and understand the sending small amounts (ie. tips) less than the network fee won’t make sense going forward.

TL;DR

Changes to the core software, including moving to fee-per-byte network fees, better transaction selection algorithms, and limits on fusion transactions will bring additional stability to the network, the operation of nodes, and help to address the concerns of the community.

Our work doesn’t stop with these changes and we’ll continue to work on enhancing the network, tools, and experience for all of the TurtleCoin community members.

You can do your part by spreading the word regarding these changes, showing your support, and voicing your opinion(s) in the chat.

As always, Pull Requests are your best avenue to shape the TurtleCoin experience.

Categories
Weekly Update

This Week In TurtleCoin® (September 24, 2019)

Developer Updates

This is a place where anybody in our community can submit a post about the TRTL project they’re working on. It’s a great way to attract helpers for your project and show people what to keep an eye out for. We encourage you to show works in progress as well as finished products, as we’re happy to see them all and it shows that we’re an active community. To submit your post, click this link

He said it was a “test” but we know he is really REAL_BITMAIN_NO1

violetminer

If you read last weeks update, you’ll see I was pretty close to making a violetminer release. This release went out a few days ago, and in fact, i made another release just yesterday.

If you didn’t read last weeks update, the major feature is NVIDIA GPU support.

Please download it and give it a go, and let me know what you think. There are binaries for Windows, Linux, Mac OSX, and ARM.

I wrote a guide on how to use it here: https://docs.turtlecoin.lol/guides/mining/violetminer-guide

>tfw z writes a guide

The update I made just yesterday, adds SSL pool support. I kinda forgot about adding this earlier, the HashVault admin helpfully reminded me 🙂

If your pool offers SSL ports, you can now connect to them with violetminer – Just download v0.1.1 and set ssl to “true” in your config file.

I also added some better validation of your config file, so if you accidentally mess up your NVIDIA settings, you should get an informative error instead of a crash.

I have a few more quality of life features to add, then I’ll probably start trying to add AMD support.

https://github.com/turtlecoin/violetminer

– Zpalm

TurtleCoin-Utils

It’s been a fun week in the land of TurtleCoin-Utils this week. Forgetting how software you put together, refactored, etc works can drive any developer mad. It started with a simple thought, “I’m going to build out the documentation to make this easy for everyone to use.” Well, all good things start with good intentions, right?

Can it really be that simple?

I’ve added a number of features, most notably of which allows for packing up the utils into a single JS bundle that is super easy to import into web project. It packages up the necessary cryptography (both WASM and native JS) and auto-loads the fastest code. Talk about easy!

In short, I’ve gone through and refactored quite a bit of the code throughout the package including implementing the exposed libraries as classes. In addition, I’ve labored to add JSDoc documentation to the package that allows for building easy to read and follow developer documentation. This makes it a lot easier to work with the TurtleCoin-Utils package. The full documentation has also been published at https://utils.turtlecoin.dev/

https://www.npmjs.com/package/turtlecoin-utils

– IBurnMyCD

Sometimes, if you try hard enough, you get what you ask for.

TurtleCoin® Core Update

This happened a couple of weeks ago, but I don’t think anyone ever mentioned it. We recently merged a pull request from CapEtn which adds ZSTD compression to the daemon, and enables it by default.

ZSTD is much more performant than the LZ4 compression we had in place previously, and also has a minimal effect on daemon speed from our testing. If you want to take advantage of the database compression, you’ll need to compile the latest development code, or wait for a new release. Then, if you have an existing database, you’ll need to delete the DB folder, or resync your daemon.

You should find that your database folder has decreased to about 59GB from somewhere around 76GB. A quite nice saving!

You can of course disable this option if you want your daemon to be as snappy as possible.

– Core Development Team

https://github.com/turtlecoin/turtlecoin/pull/886

TurtleCoin® Website – TurtleCoin.lol

yo dawg heard you like menus –

Thanks to The Judge#9063 from Discord who rightfully pointed out the misuse of the Windows 10 logo on turtlecoin.lol. While fixing this squeeze, it came to my attention that our own dang website does not only break the Microsoft guidelines but also our own. yikes! Luckily, it was not much of a hassle to fix it.

Meanwhile, I have started the development of a brand new version of turtlecoin.lol. Powered by nuxt.js, allowing serverless deployment it will follow a more streamlined design deployed already by our block explorer.

– Fexra

TurtlePay® Blockchain Cache

Quite a bit of performance based changes have taken place in the cache API over the last few weeks. I’ve added redis support to help with the speedy delivery of wallet sync data, started adding different data structures, looked at altering the database schema to squeeze more performance out and refactored quite a bit of code to make it easier to read.

My most recent scheme involves a few changes to the cache system (transparent to users) that will allow me to serve data via the CDN edges that will make wallet syncing much, much, much faster. There’s a few things to work out to get that going but I have it framed up in my mind and am ready to get going on it to see how well it performs.

https://docs.turtlepay.io/blockapi/

– IBurnMyCd

TurtleCoin® Web Wallet

This week updated the styling of the web wallet according to TurtleCoin brand guidelines and completed validation on wallet creation. andrew | trtl.rocks in the Discord chat was so kind to show me how to properly use Vuex store. Recently, I have decided to give nuxt.js a try, and decided to use this framework also for this web wallet, which currently only uses vue.js. nuxt.js extends vue.js by offering various presets to build progressive web apps that are SEO friendly and can be hosted in serverless environments.

Fexra

These… you can actually help us with

Good First Issues

Good First Issues are tickets that are marked as ‘easy wins’ for new developers. If you want to be a TurtleCoin® Developer, these are great tasks to start with!

  • Use matches property in ApiDispatcher regex #862
    Some calls in the ApiDispatcher use a regex, for example, getTransactionDetails. They then extract the query parameters. We could instead extract hashStr using the matches property on the req object, by adding a capture group to the hash regex. 
    https://github.com/turtlecoin/turtlecoin/issues/862
  • Remove no longer relevant asserts #811
    Since pretty much everyone runs the daemon in release mode, instead of debug mode, we’ve ended up where we have a number of asserts which constantly trigger, due to altered/moved/rewritten sections of code.
    https://github.com/turtlecoin/turtlecoin/issues/811
  • Daemon+WalletBackend timestamp adjustments #704
    The current /getwalletsyncdata rounds a timestamp to midnight. Depending on what time of the day you start a fresh wallet, you may have no blocks to grab (we need to roll back a bit more than we currently do with the timestamp adjustment), or too many (since it’s rounding to midnight which is quite far away).
    https://github.com/turtlecoin/turtlecoin/issues/704

Pay With TRTL

In Discord we have a channel called #Merchandise where people can post things you can buy with TRTL. To view items for sale, check the pinned posts in that channel. These are a few of the items from this week.

All items in our shop:

  • ‘Small NEOS Voyager Overshoes’ by Dustin Thewind | turtleland.fun#1350
    ID: #124437
  • ‘Xbox 360 120GB with 10 Games (+1 PS3 Game)’ by Dustin Thewind | turtleland.fun#1350
    ID: #196185
  • ‘Alan Wake Collector’s edition (Xbox 360)’ by Dustin Thewind | turtleland.fun#1350
    ID: #196847
  • ‘Diablo 3 + ROS Collectors edition’ by Dustin Thewind | turtleland.fun#1350
    ID: #362655
  • ‘Lot of Zombie Books (Walking Dead Mostly)’ by Dustin Thewind | turtleland.fun#1350
    ID: #654412
  • ‘Lot of 15 PC games (Most of them are redeemed on steam and will not be usable online)’ by Dustin Thewind | turtleland.fun#1350
    ID: #654681
  • ‘Lot of Xbox One Games (12 Games)’ by Dustin Thewind | turtleland.fun#1350
    ID: #702770
  • ‘eBook’ by DroppingThePacketsHard²#4751
    ID: #726088
  • ‘SC2 Collector Editions (Main Game + 2 Expansions)’ by Dustin Thewind | turtleland.fun#1350
    ID: #750847
  • ‘Gigabyte X570 AORUS MASTE’ by Elkim#7747
    ID: #753245
  • ‘Lot of BluRay discs (Movies, Series)’ by Dustin Thewind | turtleland.fun#1350
    ID: #858719
  • ‘ASUS X570 STRIX GAMING-F’ by Elkim#7747
    ID: #862191
  • ‘Final Fantasy XIII2 Collector’s Edition (PS3)’ by Dustin Thewind | turtleland.fun#1350
    ID: #867107
  • ‘Lot of 4 Nintendo Gamecub Games.’ by Dustin Thewind | turtleland.fun#1350
    ID: #957775
  • ‘ASUS X370 ROG CROSSHAIR VI EXTREME’ by Elkim#7747
    ID: #010771
  • ‘Sega Dreamcast with 3 original games’ by Dustin Thewind | turtleland.fun#1350
    ID: #081097
  • ‘Wacom Bamboo Tab MTE-450’ by Dustin Thewind | turtleland.fun#1350
    ID: #001659
  • ‘Halo Reach Collector Edition for Xbox 360’ by Dustin Thewind | turtleland.fun#1350
    ID: #032088
  • ‘Logitech MX Master 910-004337 5 Buttons 1 x Wheel USB Bluetooth Wireless 1600 dp’ by Dustin Thewind | turtleland.fun#1350
    ID: #027270

    Provided by fipsi#0789 and DroppingThePacketsHard²#4751
Categories
Feature Story

The Quest for Decentralized Proof-of-Work

In this article, we will look at the some of the goals of the TurtleCoin project, the concept of centralization, where the project stands, and what the project is doing to remain true to itself by maintaining its commitment to the community.

TurtleCoin Core Goals

Fun, Fast, and Easy

For those of you that have been to the TurtleCoin website, the goals below may seem repetitive. For those of you that do not know, TurtleCoin was born with a few goals in mind:

  1. Fast Transactions
  2. Privacy
  3. Easy to Use
  4. Easy to Mine
  5. Community
  6. Support

The community is reminded of these goals every time they interact with the network, participate in discussions, and help spread TurtleCoin around the globe. These core values drive everything that is done within the community from core development, documentation generation, learning opportunities, support, and community project management. We take these goals very seriously and will do whatever we can to make sure that TurtleCoin remains true to the initial vision.

Decentralization of Mining Resources

What is Decentralization?

Decentralization is, to put it plainly, the process by which planning, and decision making are shifted away from a central authority or group. The process can take many forms but in our case in the context of a Proof of Work (PoW), it means that no central authority or group controls most of the resources needed to complete the PoW.

Why Care About Decentralization?

When most of the necessary resources to satisfy the PoW requirements end up centralized, those that control those resources can, when motivated to do so, take control of the network away from the community. Such control can manifest itself in ways such as:

  • 51% attack where history can be rewritten which then typically results in the double-spending of currency that is made possible by off chain account balance tracking performed in traditional databases
  • Selfish Mining where miners remain ahead of the public chain and release just enough blocks to stay ahead of everyone else thereby reaping the majority of block rewards

Each of the above attack vectors result in a network that is no longer “Fun, Fast, and Easy”. These attacks are, in the eyes of many, fraud and in the very least theft. No one wants to use a network where their funds are constantly at risk.

In addition to the above sampling of on-chain attacks, centralization of mining resources also poses additional problems when those resources are controlled within, produced by, or otherwise restricted by one or two entities.

Where Does PoW Centralization Come From?

The most common forms of PoW centralization come from technologies designed to make PoW calculations more efficient, including:

ASICs and FPGAs are, to be fair, technologies that help secure PoW networks by increasing the overall hashrate of the network while drastically reducing the resources (electricity, physical space, management, monitoring, etc.) to do so. By increasing the efficiency of mining, others are encouraged to participate in the mining process. The more distributed hashrate a PoW network has, the harder it is to attack via the methods described above.

Supply Chain Centralization

Unfortunately, both ASICs and FPGAs are largely the product of a handful of special interest groups and organizations that create high barriers to entry and generally avoid transparency and accountability. This presents a problem in that a project that embraces these technologies today relies on just a few manufacturers to create the specialized ASICs and FPGAs needed to secure their blockchain.

While this may not sound like that big of a deal, all we have to do is think back to the countless instances in history where one group controlled the supply of a product or service.

Such centralization of manufacturing also presents a problem in that the production of ASICs and FPGAs for mining now falls under a limited number of jurisdictions. Governments could, and have, changed the legality of producing, owning, or operating such hardware on a moment’s notice. The fact that a single entity could control the hardware necessary for operating the network is in direct opposition of the goals of decentralization.

To prevent centralization of the manufacturing of the hardware there must be a multitude of manufacturers spread all over the globe. Only then is it possible to reduce the risk of a single group impacting the supply chain of ASICs and FPGAs.

Note: CPUs suffer from some of the same issues mentioned above; however, CPUs are general purpose integrated circuits that are commercially available in large quantities at affordable prices and their manufacturers are not solely focused on cryptocurrency mining activities.

TurtleCoin’s Commitment to Decentralization

The TurtleCoin community and core development team remains committed to the stance that TurtleCoin must be easy for all to mine, fair, open, and most importantly decentralized. We’ve posted numerous articles, videos, and GitHub threads reaffirming this commitment since the inception of the project. Notable examples of such include:

While others may deviate and split from their goals of decentralization, the TurtleCoin community remains fully committed to the vision that a decentralized PoW remains our best chance at long-term sustainability.

Upcoming Proof-of-Work Algorithm Change

The upgrade to CN Turtle at block 1,200,000 was a success; however, as mentioned in the Proof-of-Work Algorithm Change, we’ve had another algorithm change on the burner well before that upgrade. We always knew that CN Turtle would be a temporary step meant only to give us a bit of breathing room to test, what we hope to be, a PoW algorithm that will prevail in our quest for decentralization for longer than prior algorithms have allowed.

We have mentioned codename Chukwa in a few different places over the last few months. If you have not been following discussions in Discord or taken at look at the GitHub Chukwa Hashing Results thread, Chukwa is actually Argon2.

What is Argon2?

Argon2 is the memory hard winner of the 2015 Password Hashing Competition (PHC). Argon2 comes in three different versions; each with their own design goals.

  • Argon2d is designed to maximize resistance against GPU cracking attacks and accesses memory in a data dependent order. This means that the input data itself defines how the memory is accessed; however, it is susceptible to side-channel attacks
  • Argon2i is designed to minimize side-channel attacks and accesses memory in a data independent order
  • Argon2id is a hybrid between the two where Argon2i is used for the first pass over the memory and Argon2d is used for each pass after that

The Argon2 IETF RFC draft recommends the use of Argon2id.

Why Argon2?

Argon2 was selected for the following reasons (in no particular order):

  • Winner of the PHC that follows the same kind of processes as the NIST’s AES & SHA-3 competitions
  • Memory hard algorithm
  • Source code is GPL-3.0 compatible
  • Easily integrated into the core code, pools, etc
  • Only one known cryptocurrency project (Aquachain) uses Argon2id

Argon2 is also relatively unique in that it allows for a high-level of customisation in how the hashes are calculated including parameters such as:

  • The number of threads to use (parallelism)
  • Arbitrary resultant hash length
  • Memory requirements (memory hardness)
  • Number of iterations (time cost)
  • The use of salts

Argon2id Parameters

The various input parameters allow us to tune the implementation of Argon2 such that it makes sense for TurtleCoin.

Memory Requirement
  • Be large enough to fit an entire block so that all of the data can be shuffled
  • Not exceed common L2 CPU cache sizes to allow for the largest range of CPUs to participate in mining
Iterations
  • Provide a higher base hashrate than previous algorithms to increase the mining efficiency
  • Take advantage of the multiple memory passes used in Argon2id (>2 iterations)
Parallelism (Threads)
  • Make the use of as many physical and logical cores as possible by using 1 thread per hashing operation
Benchmark Testing Results

We solicited single-core performance benchmarks from the community in the Chukwa Hashing Results issue thread on the main repository. The summary of those results are below.

Note: For brevity, we have truncated the summary table below to the algorithms we have used before and Argon2 parameters that meet the requirements above.

Chukwa Parameters

Given the above requirements and the testing results provided by the community we were left with 5 clear options. One option stands out and sits nicely in the middle among the group of options. It provides a healthy 2.5x increase in hashrate and meets the requirements above.

We’ve selected the following Argon2id parameters for the next PoW:

  • Threads: 1
  • Iterations: 3
  • Memory: 512kb

Note: You can play with different parameters with a simple Argon2 hash generator at argon2.online

TurtleCoin’s Argon2 Implementation

Like other PoW algorithm changes, there is quite a bit of work to be done to ensure that this network upgrade is a success at block 1,800,000. We have adapted the Argon2 reference implementation for our use.

Caveats

Miner Package Availability

At this time, we have found very few miner packages that support Argon2id. While the native CPU miner provided in the core project will happily support the algorithm for solo mining, we understand that the network and community requires the availability of mining pools. To help facilitate pooled mining, we are currently working on building pool support into the native CPU miner provided by the project.

GPU Mining Support

We have not currently been able to find any miner packages that support Argon2id GPU mining. While we are confident that the community at large will work towards having an Argon2id GPU miner available at some point, it is unlikely that a GPU miner will be available at the time of the upgrade. As a result, we fully expect a substanial drop in the network hashrate as a result of this upgrade. We are planning a difficulty reset to account for this drop at the time of the upgrade. This has a byproduct of making TurtleCoin a CPU only coin for the foreseeable future.

Completed

Core

We have completed the necessary changes in the core project to implement Argon2id at block 1,800,000 that will activate with block major version 6. The changes can be found on the codename_chuckwa branch of the TurtleCoin repository. This code has not been pulled into the development tree or master as of the time of this writing.

Support Packages

The necessary changes have also been applied to the development branch of the turtlecoin-multi-hashing Node.js module that pools use to validate miner shares.

Pool Changes

The necessary changes to the turtle-pool software have been completed on the chukwa branch.

In Progress

Testnet

We will also be launching multiple testnets to test the algorithm change including a difficulty reset to adjust for the anticipanted loss of GPU mining hashrate.

How You Can Help

We need help from the community to test this algorithm change to try to ensure that everything goes as smoothly as possible. The more people that are involved, the easier it is to spot issues and correct for them before the upgrade.

Community Reminder

As always, be mindful of TurtleCoin core releases. Watch or star the main TurtleCoin repo to help stay abreast of changes and updates. Join Discord and read the #announcements on a regular basis. Or, sign up for the @news role by typing *news in the chat and be alerted whenever a new announcement is posted in Discord.

Make sure you’re ready for the network upgrade as early as possible. As with any network upgrade, prior versions of the software will no longer be compatible with the rest of the network after upgrade completion.

Remember that you too can participate in discussions regarding the direction of the project via Discord and the TurtleCoin Meta Issues. Join the discussing regarding the PoW change via the Chukwa: The Argon2 PoW Algorithm discussion on GitHub.

Correction: The article above was corrected to reflect the intent of the writer as the original copy referred to the security of PoW networks based solely on the network hashrate and not the distribution of such. Thank you to @Taegus for pointing this out.
Categories
Feature Story

The Quest for Lightning Block Propagation

Here we see a Turtle having a bright idea.

TurtleCoin has 30 second block time which makes transactions on the network faster than other projects; however, it poses some challenges in making sure that new blocks propagate across the entire network as quickly as possible to prevent the creation of unnecessary network forks. While increasing the number of nodes on the network helps to achieve this goal, reducing the size of the blocks that are transmitted between nodes allows us to propagate a block so fast it’s almost as if they were transmitted by lightning.

Network Propagation

Block Propagation at its finest

Understanding what information is propagated throughout the network is a requirement for making sense of what we’re trying to accomplish when we talk about speeding up propagation.

Each node that is connected to the network performs vital functions such as, but not limited to:

  • Verifies transactions (signatures, no double spending, structures, etc.)
  • Verifies blocks (makes sure that they meet the difficulty requirements, are valid in the chain, contain valid transactions, etc.)
  • Maintains a transaction pool (transactions that have been submitted to the network but not included in a block)
  • Relays transactions & blocks to peer nodes (those the node is connected to)

Each time that a new transaction, in its entirety, is submitted to a node for processing by a user, that transaction is added to the node’s transaction pool and then relayed to all its connected peers who relay it to their peers until everyone on the network has those transactions.

Mining a Block

We know you crave finding those blocks

Once a transaction is in the transaction pool, it’s hash is then included in the block template that is passed to miners via pools or directly from the node if solo mining.

When a miner (or pool) finds a block the block, with all its glory, it is submitted to the node, it is validated, added to the chain, and then the full block containing every transaction including its inputs, outputs, signatures, etc. is relayed to all the node’s peers. Those peers relay it to their peers after verifying it and adding it to their copy of the blockchain, and so on until everyone has the new block.

Block Propagation Payloads

The full blocks that are sent between nodes have the sizes as reported on the block explorers. Some blocks are small at only a few hundred bytes while others with many transactions or transactions with lots of inputs/outputs can climb to well over 100KB. 100KB does not seem like a lot of data, but when you need 100KB to propagate between >1,000 nodes, it takes considerable time. We need the elapsed time from block creation to full network propagation to be as little as possible to make sure that the nodes that are mined against always have the latest information about the chain.

But… Why?

You may be asking yourself, “If transactions are propagated around the network for the transaction pool and a node already has them, why are we sending them again, in their entirety, with every block that is propagated?”

Things don’t have to make sense to work, right?

To put it bluntly, we don’t need to send everything all the time if a node already has most of the information needed to verify a new block. Sending all of it again is just duplicating things it already has.

Lite Blocks to the Rescue

Block Propagation is like a game of cooperative Tetris

Enter the concept of Lite Blocks. Lite Blocks are a special kind of block created just for synchronization between nodes. A Lite Block is just like a full block except it only contains the transaction hashes not the full contents of the transactions (inputs, outputs, signatures, etc.).

When a node receives a Lite Block, it looks through the list of transaction hashes the block contains, checks its local transaction pool, and plucks the needed transactions from the pool. If it encounters a transaction that is not in its transaction pool (for any reason), it reaches out to its peers and requests just that transaction. Once the node has all the transactions the block contains, the block is verified, and added to the node’s local copy of the blockchain.

The node then starts the relaying process to propagate the new block to its peers. In going through the list of connected peers, it identifies which of its peers support Lite Blocks and which do not support Lite Blocks. It relays the appropriate block type (Full or Lite) to its peers as needed.

If you’re keen for additional reading on the underlying concept of Lite Blocks check out BTC’s BIP-0152 as well as Monero’s Fluffy Blocks.

TurtleCoin’s Implementation

Adding Lite Block functionality results in significantly smaller blocks propagating across the network. In testing, we’ve observed Lite Blocks as little as 1% of the full block size. Transferring 1KB versus 100KB is a momentous reduction in the amount of data transferred between nodes. This reduction helps to improve block propagation times in a massive way. TurtleCoin nodes will now prefer to send Lite Blocks whenever possible to reduce the size of the information exchanged between nodes. The node will fall back to Full Blocks only when necessary.

@rashedmyt has completed the work required to add Lite Blocks into the core TurtleCoin code. He’s built it in such a way that it does not require a fork of any kind as it relies on the P2P version that we use in a couple of different places. Our plans are to activate Lite Block capability with an upgrade to P2P Version 4. This P2P update will make it into the next release of the core TurtleCoin software.

The addition of Lite Blocks was listed as a TurtleCoin bounty and @rashedmyt will be picking up a 2.05M TRTL bounty for the efforts he’s put forth in getting Lite Blocks implemented in the core code. When you see him in Discord give him a high-five and a round of applause.

Categories
Feature Story

Proof-of-Work Algorithm Change

The core development team has observed the fact that the network hash rate has climbed substantially over the last few weeks. Luckily, the hashrate has remained rather decentralized and balanced over the large number of pools that the community operates. However, the significant increase in hashrate has made it more difficult for the project to meet one of its core goals. To remain mineable for everyone. After careful consideration, we believe it’s once again time to perform a soft fork to a new Proof-of-Work (PoW) algorithm.

Such a change is not taken lightly, and we’ve given considerable thought to where the project should go in this regard. We’ve seen a few networks using CryptoNight Soft Shell variants with success. Although considerable work has been done to support Soft Shell pools and the like we’re not quite ready to move over to Soft Shell just yet.

You’re probably asking yourself, “So what’s the plan?”

Supported PoW Algorithms

If you’ve read through the TurtleCoin fork guide, forked TurtleCoin yourself, or have read through the code, the next few statements are going to come to no surprise to you.

We like to collect PoW algorithms like they are pogs. The more you have, the more fun you’re having with your friends.

Keeping this in mind, TurtleCoin currently has built-in support for no less than twelve (12) different PoW algorithms. Yes, you read that right, twelve.

They are, in no particular order:

  • CryptoNight v0
  • CryptoNight v1
  • CryptoNight v2
  • CryptoNight Lite v0
  • CryptoNight Lite v1
  • CryptoNight Lite v2
  • CryptoNight Turtle v0
  • CryptoNight Turtle v1
  • CryptoNight Turtle v2
  • CryptoNight Soft Shell v0
  • CryptoNight Soft Shell v1
  • CryptoNight Soft Shell v2

Woah, Wait, What’s CryptoNight Turtle?

To put it mildly, CryptoNight Turtle is a tweak to the standard CryptoNight family that significantly increases the speed at which hashes can be computed. This is accomplished in two ways.

Iteration Change

We’re dropping the iteration count in fourth as well. From 524,288 iterations to 131,072 iterations. This will result in the base hashrate increasing for every miner.

Scratch Pad Change

We’re dropping the scratch pad size in fouth. From 1MB (CN Lite) to 256KB. Why? because we want to make everything go faster and when we’re dropping the iteration count, we need to make sure that the scratchpad is used effectively to prevent some interesting attack vectors.

Faster? Doesn’t That Mean an Even Higher Network Hash Rate?

It sure does! There’s a reason for what appears to be chaos and we’re just getting to the good part.

TurtleCoin Major Block v5

As you’re likely thinking, “This doesn’t sound better, no, no, these guys are crazy…”

Block Major version 5 is planned to activate CryptoNight Turtle v2. That’s right, variant 2. Those of you following other projects using v2 likely know that there is a performance penalty incurred by variant 2 hashing. We’ve personally seen a performance hit of anywhere from 12% to 45% on some hardware. We think that’s unacceptable; however, we do like the premise behind variant 2.

To balance the performance impact of variant 2 out, we’re going to hash faster with CryptoNight Turtle. The two effectively cancel each other out and we gain the benefit of soft forking away from potential ASIC/FPGA/etc. miners. As a matter of fact, you’ll see a hashrate increase on your miner(s) because of the move to CryptoNight Turtle.

The move to variant 2 will help strengthen TurtleCoin’s ASIC/FPGA resistance. Future PoW changes will also be designed to further strengthen the project’s ASIC/FPGA resistance to try to keep TurtleCoin mineable for everyone.

Doing Your Part

We’re confident that we have the core code ready for the soft fork to CN Turtle tentatively scheduled for block 1,200,000. The main pool software has been updated to support it as well as the underlying Node.js Hashing Library and the Node.js Cryptonote Library have been updated.

There’s still work to be done and any community assistance is, as always, appreciated.

Test Network

We’ll need help from the community to test this algorithm change to try to ensure that everything goes as smoothly as possible. The more people that are involved, the easier it is to spot issues.

Miner Support

We’re working on adding support to common miner projects to support this change including:

Miner software needs support added before we can fully test the update pool deployments. If you want to give us a hand in getting that support completed, join us on discord at http://chat.turtlecoin.lol in #dev_general.

Alternative Pools

The pool software changes need ported to the other pools that the community service operators use on a daily basis. If you are familiar with one of the following pools, your assistance is appreciated in this regard.

Upgrades

As always, be mindful of TurtleCoin core releases. Watch or star the main TurtleCoin repo to help stay abreast of changes and updates. Join Discord and read the #announcements on a regular basis. Or, sign up for the @news role by typing *news in the chat and be alerted whenever a new announcement is posted in Discord.

Make sure you’re ready for the soft fork as early as possible.

As with any soft fork, if and when the fork occurs, prior versions of the software will no longer be compatible with the rest of the network after soft fork completion.

What’s Next?

We’re already putting thought into the next PoW algorithm change after v5. Those discussions will be a different article, so as always, stay tuned.

Categories
Feature Story

That’s It, Old Nodes Have Got To Go!

Ever wonder when your node is syncing why it jumps back and forth between 2 days and 2 months behind? Yeah, we’ve had enough of that too.

It’s Time We Clean Up the Town!

We take network upgrades very seriously here at TurtleCoin. We want the network to be as safe, easy, and secure as possible for our Turtle brethren. We do our best to test all changes and make sure that our fellow Turtle are given plenty of time to upgrade their software before a change takes effect. Unfortunately… not everyone upgrades when necessary.

Running Legacy Software

We attempt to provide everyone ample notice before upgrades occur, what the upgrades will do, and how they benefit the network. However, not every Turtle visits the TurtleCoin discord, checks the GitHub repo, or watches the various forms of social media we have.

There are uncountable nodes running around on the network playing by antiquated network rules disrupting the network and causing utter chaos at times. When Turtles run old software it causes a variety of issues, including but not limited to the following:

  • Slower syncronization as the out of date nodes attempt to push blocks that don’t match the current network rules
    • Your node has to waste time checking their block and finding out it doesn’t fit the mold before the connection is dropped. Then your node has to find a new peer to hopefully get the correct block(s) from
  • The daemon must store old alternative chains in memory in the event that they one day resolve (highly unlikely)
    • This results in the blockchain wasting space on your system
  • Your log is filled with messages about unexpected hashes, failed hashes, etc.
  • Many other issues we don’t know about yet

The Solution

TurtleCoin v0.8.3 implements a new version checking mechanism that starts the process of checking the version of peer software when trying to exchange blockchain data. When we connect to a peer, or a peer connects to us, the version of the peer is compared against a minimum version that is supported. If the peer is not at or above that minimum version, the connection is dropped. This helps us to ensure seamless network upgrades in the future that aren’t cluttered by those that do not keep their software updated.

Note: Version 0.8.3 does not perform this purge of old nodes, it merely sets the stage for the event in future release(s).

We’ve also added additional log messages that warn you when you may be out of date (as you’re seeing nodes with higher versions than the one you’re running). If you see such a message, as always head over to http://latest.turtlecoin.lol to make sure you have the latest version of the software.

Future Purges

The development team aims to utilize this feature for soft forks that are the result of a mining algorithm change, difficulty algorithm change, or any other event that requires that the project to implement a soft fork.

Such a strategy helps us maintain the security of the network by alterting our fellow Turtles of upgrades and keeping legacy nodes off the network.

Thanks, No, Seriously, Thanks

We appreciate everyone that runs a node on the network, public, private, or some mix in between. It helps the stability of the network and shows everyone that we are here to stay. However, if you are one of the people running an ancient copy of TurtleCoin, please upgrade to v0.8.3 and all future releases within the posted ugprade window in the version release notes. You’ll help the community out considerably.

Otherwise,

Categories
Feature Story

Running A Public Node For Fun & Profit: Part Deux

Make it Rain Yo!

Hopefully you’ve read Part 1 of this series of articles and already have your public node up and running. If not, we highly suggest you head over there and give it a read before continuing with this article.

Note: To make sure that you’ve read this document before proceeding, you won’t be able to copy and paste any of the commands.

What Next?

Running a public node that the community can use can be challenging at times. 100% uptime is great but that’s not always possible in some environments. Running a public node takes work. Despite our best efforts right now, we can’t just Set it and Forget it.

We can; however, use some cool tools developed by the community to help make it easier to maintain a public node. We’ll take a look at how to do this below.

TurtleCoind High Availability Daemon Wrapper

This project is designed to wrap the TurtleCoind daemon on a system and monitor it for hangups, locks, fork, or other events that cause the daemon to stop responding to requests in an accurate manner.

The wrapper does its best to keep TurtleCoind running in tip top shape and always responding to requests from your users. Follow the directions below to set it up on your system. We’ve included instructions below designed to work on a your favorite flavour of *nix variants.

Node.js Setup

You’re going to need a newer version of Node to get things started. Use of any of the older v0.x releases hasn’t been supported for a while so don’t even bother trying to use those. If you are not sure what version of Node.js is installed on your system (if at all), simple run the below command:

Hopefully you’ll see something relatively modern come back like version 8:

If not, head on over to Installing Node.js via package manager for instructions on how to install the latest version for your distro.

Once you’ve completed that step and have verified you are running an up-to-date copy, we’ll continue on below.

Setting up PM2

Setting it up from here is pretty easy going forward. We’ll start by installing PM2: Advanced, production process manager for Node.js to help keep our node script running 24/7/365 and persist across system reboots.

To do so, we’ll run the following commands:

If everything completes successfully, we’ll have the PM2 process up and running and ready to run our script.

Image result for saran wrap fail gif

Setting up the Wrapper

The turtlecoind-ha package is registered with NPM, so installation is pretty easy. We’ll create a new directory for our process to live in and install it there.

After a few moments, the package will download and install with all the needed dependencies. As long as the installation finishes with the line(s) like those below you are good to proceed.

Next, we’re going to create a new file in our folder named turtlenode.js and dump the code below into that file. We’ve linked to the raw code here.

Most of the code above is designed for spitting us understand what’s going on with our node at any given time. For education purposes, we’re going to leave that intact.

We want to pay attention to the section above that initializes the node parameters.

You’ll need to change those values to match the TurtleCoin address you’re using to collect your node convenience charges as well as the amount of that fee. Remember, as stated in Part 1 of this series, the fee amount is expressed in shells or “atomic units” not TRTL.

Once we’ve made the necessary changes, we’ll save that file and we’re almost ready to rock and roll.

Prepping the Node

For maximum ease of setup you’ll need to copy the latest version of TurtleCoind for your system into the directory we created above. You can compile your own, or download the latest release binaries from the TurtleCoin GitHub repository.

We also advise that you download the latest checkpoints from the Offical Repo to help your node sync up as quickly as possible. The following command will make this quick and easy for you.

Testing the Node

Now that we have all of that set up and ready to rock and roll, we’ll test our node to make sure everything is set up correctly.

Run the following command to find out if everything will start up correctly.

If all goes well, you’ll see output like this:

Depending on if you’ve ever synced a daemon on this machine – it could take a while. Don’t fret if that’s the case. We can continue with even an unsyncronized daemon.

As long as we see TurtleCoind has started followed by TurtleCoind is attempting to synchronize with the network… we know that we’ve set up our environment correctly. Press CTRL-C to stop the node from running. We’re ready to hand off control to PM2.

Launching the Node

Handing off control to PM2 is pretty simple. To do so enter the following command(s):

Checking the Node

To verify that the node is running, we can issue the following command:

PM2 will spit back a list of processes maintained by the software such as:

To view our node logs, issue the command:

Once your node is synced, you’ll start to see log entries such as:

Now sit back, relax, and let people know about your public node.

Categories
Feature Story

Introducing CryptoNight Soft Shell

Objective

To provide the next generation PoW algorithm for TurtleCoin that accomplishes the following goals:

  • Must be a PoW that is different than any other PoW algorithm deployed by another blockchain.
  • Provides additional ASIC/FPGA resistance
  • Encourages solo-mining for decentralization of hashrate
  • Uses a variable amount of of resources (CPU & Memory) that changes dynamically as the chain progresses.
  • Helps to stablize hashrate over time

Problem

Typical implementations of the CryptoNight slow hash routines require static resource amounts to complete the hash calculations.

The figure below is provided for illustrative purposes.

Developers typically jump between any of the resource levels presented by the PoW algorithms or create slight variations of those algorithms to define new resource requirement levels for generating PoW hashes. What fun is that though?

NERVA

As @angrywasp explains in the interview CN Adaptive, Nerva, and the Quest For Fair Mining they approach this issue by changing the PoW every block with CN-Adaptive. This provides a resource requirement that changes over time.

The figure below is provided for illustrative purposes.

As you can see, the line is not straight like normal CryptoNight variants. This change is provided by altering the number of iterations the slow hash algorithm performs in relation to the chain height. They do this in the following way (see permalink). This variation is enough that it makes it harder to implement specialized hardware to mine this PoW algorithm.

Let’s break this down with some sample math. Don’t worry, I’ll keep this simple.

We’ll use an example height of 10,000 to make things easy.

  1. We take height and add 1 to it giving us 10,001.
  2. We then perform a modulo operation on the values such that 10001 % 1024 and get the result 785.
    Note: A modulo operation is a division operation that returns just the remainder.
  3. The result of #3 is fed to a simple addition operation that adds the value to either 0x40000 (262144) or 0x80000 (524288) arriving at the result 262929.
  4. That value, 262929 is then fed to the slow hash routine as the number of iterations to perform.

NERVA creates a Sawtooth that climbs a hill that at it’s peak increases the iterations performed by 1,023. Zooming in, it might look something like this:

Sawtooth Wave

The work NERVA is doing is interesting and got us thinking… if they can change the iterations… what happens when we go crazy?

CryptoNight Soft Shell

I want more!

Being a Turtle has it’s advantages at times, we’re slow, steady, reliable, and like to do things our own way. As such, our buddy @iburnmycd has created a variation of the PoW that is complex enough to meet the requirements above and provides an interesting resource utilization curve.

Instead of a flat (or nearly flat) resource utilization curve CryptoNight Soft Shell has a curve like this:

How do we do this? To put it simply, we vary both the iterations and the scratchpad memory used in such a way that we move from what we’ll call CryptoNight Ultra Fast all the way up past CryptoNightv1 and then cycle back around again to CryptoNight Ultra Fast. One full cycle is completed every 4,096 blocks (and is highly configurable).

We still use height as the foundation of how the math is performed as it is something that we can all easily agree upon. The resulting code looks something like this (subject to change and protected under GPLv3).

Changing just one of the initial defitions drastically changes the results.

If we change the CN_SOFT_SHELL_WINDOW to 1024 our cycle looks something like this:

If we change the CN_SOFT_SHELL_MULTIPLIER to 1 our cycle looks something like this:

Or change the CN_SOFT_SHELL_MULTIPLIER to 16 our cycle looks something like this:

Or we can have a little bump here and there by changing CN_SOFT_SHELL_MULTIPLIER to 16 and CN_SOFT_SHELL_WINDOW to 512

The options are practically endless in how the PoW algorithm can be altered.

Hashrate Variance

A by-product of CryptoNight Soft Shell is that a miner’s hashrate moves according to where in the cycle the blockchain is as the algorithm itself changes over time. This means that as the blockchain reaches the peak of the given cycle, a miner’s hashrate slows. Then, as it starts to return back to the lower valley of the curve, a miner’s hashrate speeds up very fast.

Difficulty Variance

One problem with a Sawtooth Wave is that if the peak of the wave is too far off from the bottom it becomes possible that a mining difficulty adjustment is required otherwise the first few blocks of any given cycle may come out very fast or in some situations very slow. Unlike a Sawtooth Wave, CryptoNight Soft Shell climbs and falls in a safe way. Each step is clearly defined throughout the entire cycle. This helps protect network difficulty from sudden changes as a result of the mining resource utilization rising and/or dropping with each block.

Questions? Comments?

Given the choice, which PoW algorithm looks like the most fun to you?

Hit us up on Discord at http://chat.turtlecoin.lol in the #dev_general channel to learn more.