Categories
Feature Story

The TurtleCoin Journey: 2 Years and Counting…

Earlier this month, TurtleCoin celebrated its two year anniversary of starting the project. The community and project have come a long way in the last 24 months and we are all proud of what we, as a community, have accomplished during that time. That’s not to say that we didn’t have a few set backs, learning opportunities, and challenges to overcome that tried our patience at times and ultimately drove the community to make changes to how the network and core suite operates. From multiple PoW algorithm changes, difficulty calculation changes, and a move to static mixins, each change was evaluated and adopted by the community with excitement and hopes for the future health of the network.

We’ve heard the pleas of the community over the last two years and by far the majority of the requests revolved around the folowing:

  • What is the community doing about the chain storage size?
  • When can the community expect more stable daemon/node operation?
  • Is RockSteady Satoshi? (TBD)

These pleas have not fallen on deaf ears and the result of those requests include a culmination of upcoming changes to the network and software with the next release of the TurtleCoin core suite. We are very excited about these changes and believe they are important for everyone to understand.

zedwallet (Legacy Edition)

The application formerly known as simplewallet

Zedwallet Legacy: End of Life

We announced in the v0.20.0 release notes that the legacy version of zedwallet contained in that release would be the last and final release containing legacy zedwallet. This statement still holds true and the code for that old version of zedwallet has been removed from the project. The next release of the core suite replaces the binary named zedwallet with zedwallet++ which is a new, faster, stronger version of the CLI wallet software. The zedwallet binary released going forward will be powered by zedwallet++.

Zedwallet Legacy: End of Support

We also announced in the latest release, that support would end for legacy zedwallet December 31, 2019. Starting January 1, 2020 if you ask for help with legacy zedwallet issues, we will kindly insist (with a stick in hand) that you upgrade your wallet using the latest release of zedwalletwallet-upgrader, or wallet-api. Again, the old code has been removed, lost, buried, and forgotten about at this point.

But, Why?

Don’t be that guy, don’t say you didn’t know.

Simple: portability, maintainability, and scalability. Legacy zedwallet (also turtle-service) based on WalletGreen creates wallets using a very unfriendly internal wallet schema that makes it very difficult to work with in different languages, platforms, and etc.

The new hotness, WalletBackend, creates a friendly, easy to use, fully documented, and portable wallet schema that is a JSON object when decrypted. WalletBackend lowers the barrier of entry for other developers to work with data created by the core tools and thus makes it far easier to develop other cool portable projects like TonChan and many others.

Deterministic Subwallets

It’s pretty much like a choose your own adventure book

We’ve updated the code in WalletBackend to default to the deterministic creation of subwallet addresses and support the restoration of those addresses in a deterministic way. This change is designed to help services in being able to quickly restore subwallet in the event that they lose any subwallet keys.

Instead of generating new random spend keys for each new wallet created in a single container, new wallets in the same container are now derived from the primary wallet address. The details of the how we derive subsequent private spend keys for subwallets is detailed in Issue #831.

Making subwallet private spend key generation deterministic allows for the restoration of any subwallet by just maintaining its index (position) in the wallet container. No more need to store all those extra keys! This is a major improvement for the teams in the community that handle subwallet addresses and users who use subwallet for a variety of purposes.

TurtleCoind Threading

Threading solves everything

RPC

Zpalmtree has worked tirelessly to implement process threading into the core daemon for the RPC interface. This helps keep the RPC interface responsive during heavy load and greatly reduces the impact to the network when the transaction pool gets hit with an influx of transactions. This change will help minimize the effects of heavy load on the network and improve the overall stability of node operations.

Transaction Input Validation

Zpalmtree also added support for threading of the input validation routines for new transactions entering the network. This allows inputs to be validated in parallel and thus drops the transaction validation time considerably. In doing so, transactions enter the network, are propagated across the network, and are ultimately included in blocks faster.

Transaction Pool Changes

How the transaction pool is managed by the core software has been changed recently to help decrease the effects of transaction spam, chain bloating, and other less than helpful actions on the network. We quickly noticed some less than friendly transaction activity taking advantage of our 30 second block times, low network transaction fees, and our good nature as turtles to bloat the chain with low-cost and free transactions that consisted of only breaking apart their funds into shells and recombining them into whole TRTL over and over again.

Transaction Selection Process

We are not the sort to create our own sorting algorithm to sort our sortables

We’ve improved upon the transaction pool ordering and selection criteria for when a miner creates a new block. In short, we’ll always de-preference transactions that are fusions and low total amount transactions as the odds are considerably better that such transactions are “spam”. As always, miners will prefer transactions with higher fees, size, and other metrics to move transactions through the network faster.

Fusion Transaction Limits

We’ve employed additional changes to the transaction pool where each node’s copy of the transaction pool can no longer accept any more than 20 fusion transactions at a time. All other fusion transactions will be rejected until there is room in the transaction pool for more.

This may result in an extended number of fusion rounds for highly disorganized (non-optimized) wallets. If you haven’t fused your wallet in a while, you might want to do so soon.

The line for the Coinstar starts at the back…

Network Upgrade at Block 2,200,000

We have a few consensus changes coming at block 2,200,000 to help improve the stability of the network. These changes are designed to be as transparent as possible but some will reach out and hit you in the face.

Some of these changes are the result of previous discussions in a meta issue and a thorough review of where the network and software can improve the overall experience of the community while maintaining the goals of the project.

Events unfolding over the last few months solidified in our minds the fact that some people just want to see the world burn and as a community, we must take precautionary steps to ensure the longevity of the project.

Output Creation Limits

Starting at the next network upgrade, the network will no longer accept transactions that create more than 90 outputs per transaction. Why apply a maximum you may ask?

+3,500 new 0.01 TRTL outputs created… why do we need those?

There is no legitimate use case in which a transaction need ever create 3,500+ outputs. All this type of transaction traffic manages to do is bloat the blockchain at the expense of every community member. It also ultimately causes the reverse effect and bloats things even further by stuffing the transaction pool and chain full of fusion transactions. Such transaction cycles serve absolutely no purpose and degrade the experience of all users.

Oh yeah… that’s why we need those: fusion spam.

Fee Per Byte

The Backstory

While setting the minimum transaction fee to 0.10 TRTL at the project’s launch sounded like a good idea at the time, such a minimum has serious long term implications. Most importantly, it costs just 0.10 TRTL to send data (transactions) on the network and that data is stored indefinitely and processed by thousands of nodes across the globe thereby consuming a growing amount of resources.

Using such resources is not free and unfortunately, as the chain grows, the resources required to operate nodes grows. The more resources required to operate a node, the less people will be able and willing to commit the necessary resources. Simply speaking, 0.10 TRTL per transaction is not maintainable for the long haul operation of the network.

Okay, Great, Now What?

That math stuff you hoped you’d never need, well… I hate to tell you…

Keeping this in mind, with the network upgrade at block 2,200,000, the network will require a minimum fee for each transaction that is based on a calculation of 0.01953125 TRTL per byte of data of the transaction. This equates to 5.00 TRTL for every 256 bytes of data sent across the network.

Increasing the minimum network fee for transactions means that each transaction is required to pay for its existence and the consumption of network resources proportionate to its size in bytes.

This means that the if you send a transaction using a lot of inputs, or a lot of outputs, contains extra data in the transaction, payment IDs, or etc. (increasing the size of the transaction), you’ll be required to pay your fair share to use the network to transmit and store that data.

As the fees for each transaction are included in the miner block reward, miners will be rewarded with these higher fees for processing transactions. The vast majority of nodes on the network are operated by miners (and pools) so the increase in fees will help to offset the costs of operating those resources to run the network.

So… What Now?

What does this mean for the average user? You’ll see an automatic calculation of the minimum network fee when sending transactions. Depending on how optimized your wallet is, how much you’re sending, and what data you include in a transaction, you’ll see the minimum network fee climb from 0.10 TRTL to about 10.00 TRTL at a minimum.

Fusion transactions will remain free so we highly recommend that you keep your wallet optimized. As a miner, you’ll also benefit from raising your minimum payout with mining pools to higher thresholds as you’ll indirectly pay less in fees for the transactions that pay you for your mining efforts.

If your wallet is mostly composed of shells, you’ll want to optimize your wallet before this new fee structure kicks in. If you mine only on the biggest pool or make most of your TRTL through tips, you’ll definitely want to optimize your wallet immediately.

What’s The Catch?

Sending small transactions, when we needed more mixable outputs for fungibility purposes, helped provide the necessary data to make everything possible. This could be done at a relatively low, low, low cost of 0.10 TRTL; however, with this change, sending 1.00 TRTL to someone will now incur a cost of 10.00 TRTL. We’re sure you can do the math there and understand the sending small amounts (ie. tips) less than the network fee won’t make sense going forward.

TL;DR

Changes to the core software, including moving to fee-per-byte network fees, better transaction selection algorithms, and limits on fusion transactions will bring additional stability to the network, the operation of nodes, and help to address the concerns of the community.

Our work doesn’t stop with these changes and we’ll continue to work on enhancing the network, tools, and experience for all of the TurtleCoin community members.

You can do your part by spreading the word regarding these changes, showing your support, and voicing your opinion(s) in the chat.

As always, Pull Requests are your best avenue to shape the TurtleCoin experience.

Categories
Weekly Update

This Week In TurtleCoin® (September 24, 2019)

Developer Updates

This is a place where anybody in our community can submit a post about the TRTL project they’re working on. It’s a great way to attract helpers for your project and show people what to keep an eye out for. We encourage you to show works in progress as well as finished products, as we’re happy to see them all and it shows that we’re an active community. To submit your post, click this link

He said it was a “test” but we know he is really REAL_BITMAIN_NO1

violetminer

If you read last weeks update, you’ll see I was pretty close to making a violetminer release. This release went out a few days ago, and in fact, i made another release just yesterday.

If you didn’t read last weeks update, the major feature is NVIDIA GPU support.

Please download it and give it a go, and let me know what you think. There are binaries for Windows, Linux, Mac OSX, and ARM.

I wrote a guide on how to use it here: https://docs.turtlecoin.lol/guides/mining/violetminer-guide

>tfw z writes a guide

The update I made just yesterday, adds SSL pool support. I kinda forgot about adding this earlier, the HashVault admin helpfully reminded me 🙂

If your pool offers SSL ports, you can now connect to them with violetminer – Just download v0.1.1 and set ssl to “true” in your config file.

I also added some better validation of your config file, so if you accidentally mess up your NVIDIA settings, you should get an informative error instead of a crash.

I have a few more quality of life features to add, then I’ll probably start trying to add AMD support.

https://github.com/turtlecoin/violetminer

– Zpalm

TurtleCoin-Utils

It’s been a fun week in the land of TurtleCoin-Utils this week. Forgetting how software you put together, refactored, etc works can drive any developer mad. It started with a simple thought, “I’m going to build out the documentation to make this easy for everyone to use.” Well, all good things start with good intentions, right?

Can it really be that simple?

I’ve added a number of features, most notably of which allows for packing up the utils into a single JS bundle that is super easy to import into web project. It packages up the necessary cryptography (both WASM and native JS) and auto-loads the fastest code. Talk about easy!

In short, I’ve gone through and refactored quite a bit of the code throughout the package including implementing the exposed libraries as classes. In addition, I’ve labored to add JSDoc documentation to the package that allows for building easy to read and follow developer documentation. This makes it a lot easier to work with the TurtleCoin-Utils package. The full documentation has also been published at https://utils.turtlecoin.dev/

https://www.npmjs.com/package/turtlecoin-utils

– IBurnMyCD

Sometimes, if you try hard enough, you get what you ask for.

TurtleCoin® Core Update

This happened a couple of weeks ago, but I don’t think anyone ever mentioned it. We recently merged a pull request from CapEtn which adds ZSTD compression to the daemon, and enables it by default.

ZSTD is much more performant than the LZ4 compression we had in place previously, and also has a minimal effect on daemon speed from our testing. If you want to take advantage of the database compression, you’ll need to compile the latest development code, or wait for a new release. Then, if you have an existing database, you’ll need to delete the DB folder, or resync your daemon.

You should find that your database folder has decreased to about 59GB from somewhere around 76GB. A quite nice saving!

You can of course disable this option if you want your daemon to be as snappy as possible.

– Core Development Team

https://github.com/turtlecoin/turtlecoin/pull/886

TurtleCoin® Website – TurtleCoin.lol

yo dawg heard you like menus –

Thanks to The Judge#9063 from Discord who rightfully pointed out the misuse of the Windows 10 logo on turtlecoin.lol. While fixing this squeeze, it came to my attention that our own dang website does not only break the Microsoft guidelines but also our own. yikes! Luckily, it was not much of a hassle to fix it.

Meanwhile, I have started the development of a brand new version of turtlecoin.lol. Powered by nuxt.js, allowing serverless deployment it will follow a more streamlined design deployed already by our block explorer.

– Fexra

TurtlePay® Blockchain Cache

Quite a bit of performance based changes have taken place in the cache API over the last few weeks. I’ve added redis support to help with the speedy delivery of wallet sync data, started adding different data structures, looked at altering the database schema to squeeze more performance out and refactored quite a bit of code to make it easier to read.

My most recent scheme involves a few changes to the cache system (transparent to users) that will allow me to serve data via the CDN edges that will make wallet syncing much, much, much faster. There’s a few things to work out to get that going but I have it framed up in my mind and am ready to get going on it to see how well it performs.

https://docs.turtlepay.io/blockapi/

– IBurnMyCd

TurtleCoin® Web Wallet

This week updated the styling of the web wallet according to TurtleCoin brand guidelines and completed validation on wallet creation. andrew | trtl.rocks in the Discord chat was so kind to show me how to properly use Vuex store. Recently, I have decided to give nuxt.js a try, and decided to use this framework also for this web wallet, which currently only uses vue.js. nuxt.js extends vue.js by offering various presets to build progressive web apps that are SEO friendly and can be hosted in serverless environments.

Fexra

These… you can actually help us with

Good First Issues

Good First Issues are tickets that are marked as ‘easy wins’ for new developers. If you want to be a TurtleCoin® Developer, these are great tasks to start with!

  • Use matches property in ApiDispatcher regex #862
    Some calls in the ApiDispatcher use a regex, for example, getTransactionDetails. They then extract the query parameters. We could instead extract hashStr using the matches property on the req object, by adding a capture group to the hash regex. 
    https://github.com/turtlecoin/turtlecoin/issues/862
  • Remove no longer relevant asserts #811
    Since pretty much everyone runs the daemon in release mode, instead of debug mode, we’ve ended up where we have a number of asserts which constantly trigger, due to altered/moved/rewritten sections of code.
    https://github.com/turtlecoin/turtlecoin/issues/811
  • Daemon+WalletBackend timestamp adjustments #704
    The current /getwalletsyncdata rounds a timestamp to midnight. Depending on what time of the day you start a fresh wallet, you may have no blocks to grab (we need to roll back a bit more than we currently do with the timestamp adjustment), or too many (since it’s rounding to midnight which is quite far away).
    https://github.com/turtlecoin/turtlecoin/issues/704

Pay With TRTL

In Discord we have a channel called #Merchandise where people can post things you can buy with TRTL. To view items for sale, check the pinned posts in that channel. These are a few of the items from this week.

All items in our shop:

  • ‘Small NEOS Voyager Overshoes’ by Dustin Thewind | turtleland.fun#1350
    ID: #124437
  • ‘Xbox 360 120GB with 10 Games (+1 PS3 Game)’ by Dustin Thewind | turtleland.fun#1350
    ID: #196185
  • ‘Alan Wake Collector’s edition (Xbox 360)’ by Dustin Thewind | turtleland.fun#1350
    ID: #196847
  • ‘Diablo 3 + ROS Collectors edition’ by Dustin Thewind | turtleland.fun#1350
    ID: #362655
  • ‘Lot of Zombie Books (Walking Dead Mostly)’ by Dustin Thewind | turtleland.fun#1350
    ID: #654412
  • ‘Lot of 15 PC games (Most of them are redeemed on steam and will not be usable online)’ by Dustin Thewind | turtleland.fun#1350
    ID: #654681
  • ‘Lot of Xbox One Games (12 Games)’ by Dustin Thewind | turtleland.fun#1350
    ID: #702770
  • ‘eBook’ by DroppingThePacketsHard²#4751
    ID: #726088
  • ‘SC2 Collector Editions (Main Game + 2 Expansions)’ by Dustin Thewind | turtleland.fun#1350
    ID: #750847
  • ‘Gigabyte X570 AORUS MASTE’ by Elkim#7747
    ID: #753245
  • ‘Lot of BluRay discs (Movies, Series)’ by Dustin Thewind | turtleland.fun#1350
    ID: #858719
  • ‘ASUS X570 STRIX GAMING-F’ by Elkim#7747
    ID: #862191
  • ‘Final Fantasy XIII2 Collector’s Edition (PS3)’ by Dustin Thewind | turtleland.fun#1350
    ID: #867107
  • ‘Lot of 4 Nintendo Gamecub Games.’ by Dustin Thewind | turtleland.fun#1350
    ID: #957775
  • ‘ASUS X370 ROG CROSSHAIR VI EXTREME’ by Elkim#7747
    ID: #010771
  • ‘Sega Dreamcast with 3 original games’ by Dustin Thewind | turtleland.fun#1350
    ID: #081097
  • ‘Wacom Bamboo Tab MTE-450’ by Dustin Thewind | turtleland.fun#1350
    ID: #001659
  • ‘Halo Reach Collector Edition for Xbox 360’ by Dustin Thewind | turtleland.fun#1350
    ID: #032088
  • ‘Logitech MX Master 910-004337 5 Buttons 1 x Wheel USB Bluetooth Wireless 1600 dp’ by Dustin Thewind | turtleland.fun#1350
    ID: #027270

    Provided by fipsi#0789 and DroppingThePacketsHard²#4751
Categories
Feature Story

The Quest for Decentralized Proof-of-Work

In this article, we will look at the some of the goals of the TurtleCoin project, the concept of centralization, where the project stands, and what the project is doing to remain true to itself by maintaining its commitment to the community.

TurtleCoin Core Goals

Fun, Fast, and Easy

For those of you that have been to the TurtleCoin website, the goals below may seem repetitive. For those of you that do not know, TurtleCoin was born with a few goals in mind:

  1. Fast Transactions
  2. Privacy
  3. Easy to Use
  4. Easy to Mine
  5. Community
  6. Support

The community is reminded of these goals every time they interact with the network, participate in discussions, and help spread TurtleCoin around the globe. These core values drive everything that is done within the community from core development, documentation generation, learning opportunities, support, and community project management. We take these goals very seriously and will do whatever we can to make sure that TurtleCoin remains true to the initial vision.

Decentralization of Mining Resources

What is Decentralization?

Decentralization is, to put it plainly, the process by which planning, and decision making are shifted away from a central authority or group. The process can take many forms but in our case in the context of a Proof of Work (PoW), it means that no central authority or group controls most of the resources needed to complete the PoW.

Why Care About Decentralization?

When most of the necessary resources to satisfy the PoW requirements end up centralized, those that control those resources can, when motivated to do so, take control of the network away from the community. Such control can manifest itself in ways such as:

  • 51% attack where history can be rewritten which then typically results in the double-spending of currency that is made possible by off chain account balance tracking performed in traditional databases
  • Selfish Mining where miners remain ahead of the public chain and release just enough blocks to stay ahead of everyone else thereby reaping the majority of block rewards

Each of the above attack vectors result in a network that is no longer “Fun, Fast, and Easy”. These attacks are, in the eyes of many, fraud and in the very least theft. No one wants to use a network where their funds are constantly at risk.

In addition to the above sampling of on-chain attacks, centralization of mining resources also poses additional problems when those resources are controlled within, produced by, or otherwise restricted by one or two entities.

Where Does PoW Centralization Come From?

The most common forms of PoW centralization come from technologies designed to make PoW calculations more efficient, including:

ASICs and FPGAs are, to be fair, technologies that help secure PoW networks by increasing the overall hashrate of the network while drastically reducing the resources (electricity, physical space, management, monitoring, etc.) to do so. By increasing the efficiency of mining, others are encouraged to participate in the mining process. The more distributed hashrate a PoW network has, the harder it is to attack via the methods described above.

Supply Chain Centralization

Unfortunately, both ASICs and FPGAs are largely the product of a handful of special interest groups and organizations that create high barriers to entry and generally avoid transparency and accountability. This presents a problem in that a project that embraces these technologies today relies on just a few manufacturers to create the specialized ASICs and FPGAs needed to secure their blockchain.

While this may not sound like that big of a deal, all we have to do is think back to the countless instances in history where one group controlled the supply of a product or service.

Such centralization of manufacturing also presents a problem in that the production of ASICs and FPGAs for mining now falls under a limited number of jurisdictions. Governments could, and have, changed the legality of producing, owning, or operating such hardware on a moment’s notice. The fact that a single entity could control the hardware necessary for operating the network is in direct opposition of the goals of decentralization.

To prevent centralization of the manufacturing of the hardware there must be a multitude of manufacturers spread all over the globe. Only then is it possible to reduce the risk of a single group impacting the supply chain of ASICs and FPGAs.

Note: CPUs suffer from some of the same issues mentioned above; however, CPUs are general purpose integrated circuits that are commercially available in large quantities at affordable prices and their manufacturers are not solely focused on cryptocurrency mining activities.

TurtleCoin’s Commitment to Decentralization

The TurtleCoin community and core development team remains committed to the stance that TurtleCoin must be easy for all to mine, fair, open, and most importantly decentralized. We’ve posted numerous articles, videos, and GitHub threads reaffirming this commitment since the inception of the project. Notable examples of such include:

While others may deviate and split from their goals of decentralization, the TurtleCoin community remains fully committed to the vision that a decentralized PoW remains our best chance at long-term sustainability.

Upcoming Proof-of-Work Algorithm Change

The upgrade to CN Turtle at block 1,200,000 was a success; however, as mentioned in the Proof-of-Work Algorithm Change, we’ve had another algorithm change on the burner well before that upgrade. We always knew that CN Turtle would be a temporary step meant only to give us a bit of breathing room to test, what we hope to be, a PoW algorithm that will prevail in our quest for decentralization for longer than prior algorithms have allowed.

We have mentioned codename Chukwa in a few different places over the last few months. If you have not been following discussions in Discord or taken at look at the GitHub Chukwa Hashing Results thread, Chukwa is actually Argon2.

What is Argon2?

Argon2 is the memory hard winner of the 2015 Password Hashing Competition (PHC). Argon2 comes in three different versions; each with their own design goals.

  • Argon2d is designed to maximize resistance against GPU cracking attacks and accesses memory in a data dependent order. This means that the input data itself defines how the memory is accessed; however, it is susceptible to side-channel attacks
  • Argon2i is designed to minimize side-channel attacks and accesses memory in a data independent order
  • Argon2id is a hybrid between the two where Argon2i is used for the first pass over the memory and Argon2d is used for each pass after that

The Argon2 IETF RFC draft recommends the use of Argon2id.

Why Argon2?

Argon2 was selected for the following reasons (in no particular order):

  • Winner of the PHC that follows the same kind of processes as the NIST’s AES & SHA-3 competitions
  • Memory hard algorithm
  • Source code is GPL-3.0 compatible
  • Easily integrated into the core code, pools, etc
  • Only one known cryptocurrency project (Aquachain) uses Argon2id

Argon2 is also relatively unique in that it allows for a high-level of customisation in how the hashes are calculated including parameters such as:

  • The number of threads to use (parallelism)
  • Arbitrary resultant hash length
  • Memory requirements (memory hardness)
  • Number of iterations (time cost)
  • The use of salts

Argon2id Parameters

The various input parameters allow us to tune the implementation of Argon2 such that it makes sense for TurtleCoin.

Memory Requirement
  • Be large enough to fit an entire block so that all of the data can be shuffled
  • Not exceed common L2 CPU cache sizes to allow for the largest range of CPUs to participate in mining
Iterations
  • Provide a higher base hashrate than previous algorithms to increase the mining efficiency
  • Take advantage of the multiple memory passes used in Argon2id (>2 iterations)
Parallelism (Threads)
  • Make the use of as many physical and logical cores as possible by using 1 thread per hashing operation
Benchmark Testing Results

We solicited single-core performance benchmarks from the community in the Chukwa Hashing Results issue thread on the main repository. The summary of those results are below.

Note: For brevity, we have truncated the summary table below to the algorithms we have used before and Argon2 parameters that meet the requirements above.

Chukwa Parameters

Given the above requirements and the testing results provided by the community we were left with 5 clear options. One option stands out and sits nicely in the middle among the group of options. It provides a healthy 2.5x increase in hashrate and meets the requirements above.

We’ve selected the following Argon2id parameters for the next PoW:

  • Threads: 1
  • Iterations: 3
  • Memory: 512kb

Note: You can play with different parameters with a simple Argon2 hash generator at argon2.online

TurtleCoin’s Argon2 Implementation

Like other PoW algorithm changes, there is quite a bit of work to be done to ensure that this network upgrade is a success at block 1,800,000. We have adapted the Argon2 reference implementation for our use.

Caveats

Miner Package Availability

At this time, we have found very few miner packages that support Argon2id. While the native CPU miner provided in the core project will happily support the algorithm for solo mining, we understand that the network and community requires the availability of mining pools. To help facilitate pooled mining, we are currently working on building pool support into the native CPU miner provided by the project.

GPU Mining Support

We have not currently been able to find any miner packages that support Argon2id GPU mining. While we are confident that the community at large will work towards having an Argon2id GPU miner available at some point, it is unlikely that a GPU miner will be available at the time of the upgrade. As a result, we fully expect a substanial drop in the network hashrate as a result of this upgrade. We are planning a difficulty reset to account for this drop at the time of the upgrade. This has a byproduct of making TurtleCoin a CPU only coin for the foreseeable future.

Completed

Core

We have completed the necessary changes in the core project to implement Argon2id at block 1,800,000 that will activate with block major version 6. The changes can be found on the codename_chuckwa branch of the TurtleCoin repository. This code has not been pulled into the development tree or master as of the time of this writing.

Support Packages

The necessary changes have also been applied to the development branch of the turtlecoin-multi-hashing Node.js module that pools use to validate miner shares.

Pool Changes

The necessary changes to the turtle-pool software have been completed on the chukwa branch.

In Progress

Testnet

We will also be launching multiple testnets to test the algorithm change including a difficulty reset to adjust for the anticipanted loss of GPU mining hashrate.

How You Can Help

We need help from the community to test this algorithm change to try to ensure that everything goes as smoothly as possible. The more people that are involved, the easier it is to spot issues and correct for them before the upgrade.

Community Reminder

As always, be mindful of TurtleCoin core releases. Watch or star the main TurtleCoin repo to help stay abreast of changes and updates. Join Discord and read the #announcements on a regular basis. Or, sign up for the @news role by typing *news in the chat and be alerted whenever a new announcement is posted in Discord.

Make sure you’re ready for the network upgrade as early as possible. As with any network upgrade, prior versions of the software will no longer be compatible with the rest of the network after upgrade completion.

Remember that you too can participate in discussions regarding the direction of the project via Discord and the TurtleCoin Meta Issues. Join the discussing regarding the PoW change via the Chukwa: The Argon2 PoW Algorithm discussion on GitHub.

Correction: The article above was corrected to reflect the intent of the writer as the original copy referred to the security of PoW networks based solely on the network hashrate and not the distribution of such. Thank you to @Taegus for pointing this out.
Categories
Feature Story

The Quest for Lightning Block Propagation

Here we see a Turtle having a bright idea.

TurtleCoin has 30 second block time which makes transactions on the network faster than other projects; however, it poses some challenges in making sure that new blocks propagate across the entire network as quickly as possible to prevent the creation of unnecessary network forks. While increasing the number of nodes on the network helps to achieve this goal, reducing the size of the blocks that are transmitted between nodes allows us to propagate a block so fast it’s almost as if they were transmitted by lightning.

Network Propagation

Block Propagation at its finest

Understanding what information is propagated throughout the network is a requirement for making sense of what we’re trying to accomplish when we talk about speeding up propagation.

Each node that is connected to the network performs vital functions such as, but not limited to:

  • Verifies transactions (signatures, no double spending, structures, etc.)
  • Verifies blocks (makes sure that they meet the difficulty requirements, are valid in the chain, contain valid transactions, etc.)
  • Maintains a transaction pool (transactions that have been submitted to the network but not included in a block)
  • Relays transactions & blocks to peer nodes (those the node is connected to)

Each time that a new transaction, in its entirety, is submitted to a node for processing by a user, that transaction is added to the node’s transaction pool and then relayed to all its connected peers who relay it to their peers until everyone on the network has those transactions.

Mining a Block

We know you crave finding those blocks

Once a transaction is in the transaction pool, it’s hash is then included in the block template that is passed to miners via pools or directly from the node if solo mining.

When a miner (or pool) finds a block the block, with all its glory, it is submitted to the node, it is validated, added to the chain, and then the full block containing every transaction including its inputs, outputs, signatures, etc. is relayed to all the node’s peers. Those peers relay it to their peers after verifying it and adding it to their copy of the blockchain, and so on until everyone has the new block.

Block Propagation Payloads

The full blocks that are sent between nodes have the sizes as reported on the block explorers. Some blocks are small at only a few hundred bytes while others with many transactions or transactions with lots of inputs/outputs can climb to well over 100KB. 100KB does not seem like a lot of data, but when you need 100KB to propagate between >1,000 nodes, it takes considerable time. We need the elapsed time from block creation to full network propagation to be as little as possible to make sure that the nodes that are mined against always have the latest information about the chain.

But… Why?

You may be asking yourself, “If transactions are propagated around the network for the transaction pool and a node already has them, why are we sending them again, in their entirety, with every block that is propagated?”

Things don’t have to make sense to work, right?

To put it bluntly, we don’t need to send everything all the time if a node already has most of the information needed to verify a new block. Sending all of it again is just duplicating things it already has.

Lite Blocks to the Rescue

Block Propagation is like a game of cooperative Tetris

Enter the concept of Lite Blocks. Lite Blocks are a special kind of block created just for synchronization between nodes. A Lite Block is just like a full block except it only contains the transaction hashes not the full contents of the transactions (inputs, outputs, signatures, etc.).

When a node receives a Lite Block, it looks through the list of transaction hashes the block contains, checks its local transaction pool, and plucks the needed transactions from the pool. If it encounters a transaction that is not in its transaction pool (for any reason), it reaches out to its peers and requests just that transaction. Once the node has all the transactions the block contains, the block is verified, and added to the node’s local copy of the blockchain.

The node then starts the relaying process to propagate the new block to its peers. In going through the list of connected peers, it identifies which of its peers support Lite Blocks and which do not support Lite Blocks. It relays the appropriate block type (Full or Lite) to its peers as needed.

If you’re keen for additional reading on the underlying concept of Lite Blocks check out BTC’s BIP-0152 as well as Monero’s Fluffy Blocks.

TurtleCoin’s Implementation

Adding Lite Block functionality results in significantly smaller blocks propagating across the network. In testing, we’ve observed Lite Blocks as little as 1% of the full block size. Transferring 1KB versus 100KB is a momentous reduction in the amount of data transferred between nodes. This reduction helps to improve block propagation times in a massive way. TurtleCoin nodes will now prefer to send Lite Blocks whenever possible to reduce the size of the information exchanged between nodes. The node will fall back to Full Blocks only when necessary.

@rashedmyt has completed the work required to add Lite Blocks into the core TurtleCoin code. He’s built it in such a way that it does not require a fork of any kind as it relies on the P2P version that we use in a couple of different places. Our plans are to activate Lite Block capability with an upgrade to P2P Version 4. This P2P update will make it into the next release of the core TurtleCoin software.

The addition of Lite Blocks was listed as a TurtleCoin bounty and @rashedmyt will be picking up a 2.05M TRTL bounty for the efforts he’s put forth in getting Lite Blocks implemented in the core code. When you see him in Discord give him a high-five and a round of applause.

Categories
Feature Story

Proof-of-Work Algorithm Change

The core development team has observed the fact that the network hash rate has climbed substantially over the last few weeks. Luckily, the hashrate has remained rather decentralized and balanced over the large number of pools that the community operates. However, the significant increase in hashrate has made it more difficult for the project to meet one of its core goals. To remain mineable for everyone. After careful consideration, we believe it’s once again time to perform a soft fork to a new Proof-of-Work (PoW) algorithm.

Such a change is not taken lightly, and we’ve given considerable thought to where the project should go in this regard. We’ve seen a few networks using CryptoNight Soft Shell variants with success. Although considerable work has been done to support Soft Shell pools and the like we’re not quite ready to move over to Soft Shell just yet.

You’re probably asking yourself, “So what’s the plan?”

Supported PoW Algorithms

If you’ve read through the TurtleCoin fork guide, forked TurtleCoin yourself, or have read through the code, the next few statements are going to come to no surprise to you.

We like to collect PoW algorithms like they are pogs. The more you have, the more fun you’re having with your friends.

Keeping this in mind, TurtleCoin currently has built-in support for no less than twelve (12) different PoW algorithms. Yes, you read that right, twelve.

They are, in no particular order:

  • CryptoNight v0
  • CryptoNight v1
  • CryptoNight v2
  • CryptoNight Lite v0
  • CryptoNight Lite v1
  • CryptoNight Lite v2
  • CryptoNight Turtle v0
  • CryptoNight Turtle v1
  • CryptoNight Turtle v2
  • CryptoNight Soft Shell v0
  • CryptoNight Soft Shell v1
  • CryptoNight Soft Shell v2

Woah, Wait, What’s CryptoNight Turtle?

To put it mildly, CryptoNight Turtle is a tweak to the standard CryptoNight family that significantly increases the speed at which hashes can be computed. This is accomplished in two ways.

Iteration Change

We’re dropping the iteration count in fourth as well. From 524,288 iterations to 131,072 iterations. This will result in the base hashrate increasing for every miner.

Scratch Pad Change

We’re dropping the scratch pad size in fouth. From 1MB (CN Lite) to 256KB. Why? because we want to make everything go faster and when we’re dropping the iteration count, we need to make sure that the scratchpad is used effectively to prevent some interesting attack vectors.

Faster? Doesn’t That Mean an Even Higher Network Hash Rate?

It sure does! There’s a reason for what appears to be chaos and we’re just getting to the good part.

TurtleCoin Major Block v5

As you’re likely thinking, “This doesn’t sound better, no, no, these guys are crazy…”

Block Major version 5 is planned to activate CryptoNight Turtle v2. That’s right, variant 2. Those of you following other projects using v2 likely know that there is a performance penalty incurred by variant 2 hashing. We’ve personally seen a performance hit of anywhere from 12% to 45% on some hardware. We think that’s unacceptable; however, we do like the premise behind variant 2.

To balance the performance impact of variant 2 out, we’re going to hash faster with CryptoNight Turtle. The two effectively cancel each other out and we gain the benefit of soft forking away from potential ASIC/FPGA/etc. miners. As a matter of fact, you’ll see a hashrate increase on your miner(s) because of the move to CryptoNight Turtle.

The move to variant 2 will help strengthen TurtleCoin’s ASIC/FPGA resistance. Future PoW changes will also be designed to further strengthen the project’s ASIC/FPGA resistance to try to keep TurtleCoin mineable for everyone.

Doing Your Part

We’re confident that we have the core code ready for the soft fork to CN Turtle tentatively scheduled for block 1,200,000. The main pool software has been updated to support it as well as the underlying Node.js Hashing Library and the Node.js Cryptonote Library have been updated.

There’s still work to be done and any community assistance is, as always, appreciated.

Test Network

We’ll need help from the community to test this algorithm change to try to ensure that everything goes as smoothly as possible. The more people that are involved, the easier it is to spot issues.

Miner Support

We’re working on adding support to common miner projects to support this change including:

Miner software needs support added before we can fully test the update pool deployments. If you want to give us a hand in getting that support completed, join us on discord at http://chat.turtlecoin.lol in #dev_general.

Alternative Pools

The pool software changes need ported to the other pools that the community service operators use on a daily basis. If you are familiar with one of the following pools, your assistance is appreciated in this regard.

Upgrades

As always, be mindful of TurtleCoin core releases. Watch or star the main TurtleCoin repo to help stay abreast of changes and updates. Join Discord and read the #announcements on a regular basis. Or, sign up for the @news role by typing *news in the chat and be alerted whenever a new announcement is posted in Discord.

Make sure you’re ready for the soft fork as early as possible.

As with any soft fork, if and when the fork occurs, prior versions of the software will no longer be compatible with the rest of the network after soft fork completion.

What’s Next?

We’re already putting thought into the next PoW algorithm change after v5. Those discussions will be a different article, so as always, stay tuned.

Categories
Feature Story

That’s It, Old Nodes Have Got To Go!

Ever wonder when your node is syncing why it jumps back and forth between 2 days and 2 months behind? Yeah, we’ve had enough of that too.

It’s Time We Clean Up the Town!

We take network upgrades very seriously here at TurtleCoin. We want the network to be as safe, easy, and secure as possible for our Turtle brethren. We do our best to test all changes and make sure that our fellow Turtle are given plenty of time to upgrade their software before a change takes effect. Unfortunately… not everyone upgrades when necessary.

Running Legacy Software

We attempt to provide everyone ample notice before upgrades occur, what the upgrades will do, and how they benefit the network. However, not every Turtle visits the TurtleCoin discord, checks the GitHub repo, or watches the various forms of social media we have.

There are uncountable nodes running around on the network playing by antiquated network rules disrupting the network and causing utter chaos at times. When Turtles run old software it causes a variety of issues, including but not limited to the following:

  • Slower syncronization as the out of date nodes attempt to push blocks that don’t match the current network rules
    • Your node has to waste time checking their block and finding out it doesn’t fit the mold before the connection is dropped. Then your node has to find a new peer to hopefully get the correct block(s) from
  • The daemon must store old alternative chains in memory in the event that they one day resolve (highly unlikely)
    • This results in the blockchain wasting space on your system
  • Your log is filled with messages about unexpected hashes, failed hashes, etc.
  • Many other issues we don’t know about yet

The Solution

TurtleCoin v0.8.3 implements a new version checking mechanism that starts the process of checking the version of peer software when trying to exchange blockchain data. When we connect to a peer, or a peer connects to us, the version of the peer is compared against a minimum version that is supported. If the peer is not at or above that minimum version, the connection is dropped. This helps us to ensure seamless network upgrades in the future that aren’t cluttered by those that do not keep their software updated.

Note: Version 0.8.3 does not perform this purge of old nodes, it merely sets the stage for the event in future release(s).

We’ve also added additional log messages that warn you when you may be out of date (as you’re seeing nodes with higher versions than the one you’re running). If you see such a message, as always head over to http://latest.turtlecoin.lol to make sure you have the latest version of the software.

Future Purges

The development team aims to utilize this feature for soft forks that are the result of a mining algorithm change, difficulty algorithm change, or any other event that requires that the project to implement a soft fork.

Such a strategy helps us maintain the security of the network by alterting our fellow Turtles of upgrades and keeping legacy nodes off the network.

Thanks, No, Seriously, Thanks

We appreciate everyone that runs a node on the network, public, private, or some mix in between. It helps the stability of the network and shows everyone that we are here to stay. However, if you are one of the people running an ancient copy of TurtleCoin, please upgrade to v0.8.3 and all future releases within the posted ugprade window in the version release notes. You’ll help the community out considerably.

Otherwise,

Categories
Feature Story

Running A Public Node For Fun & Profit: Part Deux

Make it Rain Yo!

Hopefully you’ve read Part 1 of this series of articles and already have your public node up and running. If not, we highly suggest you head over there and give it a read before continuing with this article.

Note: To make sure that you’ve read this document before proceeding, you won’t be able to copy and paste any of the commands.

What Next?

Running a public node that the community can use can be challenging at times. 100% uptime is great but that’s not always possible in some environments. Running a public node takes work. Despite our best efforts right now, we can’t just Set it and Forget it.

We can; however, use some cool tools developed by the community to help make it easier to maintain a public node. We’ll take a look at how to do this below.

TurtleCoind High Availability Daemon Wrapper

This project is designed to wrap the TurtleCoind daemon on a system and monitor it for hangups, locks, fork, or other events that cause the daemon to stop responding to requests in an accurate manner.

The wrapper does its best to keep TurtleCoind running in tip top shape and always responding to requests from your users. Follow the directions below to set it up on your system. We’ve included instructions below designed to work on a your favorite flavour of *nix variants.

Node.js Setup

You’re going to need a newer version of Node to get things started. Use of any of the older v0.x releases hasn’t been supported for a while so don’t even bother trying to use those. If you are not sure what version of Node.js is installed on your system (if at all), simple run the below command:

Hopefully you’ll see something relatively modern come back like version 8:

If not, head on over to Installing Node.js via package manager for instructions on how to install the latest version for your distro.

Once you’ve completed that step and have verified you are running an up-to-date copy, we’ll continue on below.

Setting up PM2

Setting it up from here is pretty easy going forward. We’ll start by installing PM2: Advanced, production process manager for Node.js to help keep our node script running 24/7/365 and persist across system reboots.

To do so, we’ll run the following commands:

If everything completes successfully, we’ll have the PM2 process up and running and ready to run our script.

Image result for saran wrap fail gif

Setting up the Wrapper

The turtlecoind-ha package is registered with NPM, so installation is pretty easy. We’ll create a new directory for our process to live in and install it there.

After a few moments, the package will download and install with all the needed dependencies. As long as the installation finishes with the line(s) like those below you are good to proceed.

Next, we’re going to create a new file in our folder named turtlenode.js and dump the code below into that file. We’ve linked to the raw code here.

Most of the code above is designed for spitting us understand what’s going on with our node at any given time. For education purposes, we’re going to leave that intact.

We want to pay attention to the section above that initializes the node parameters.

You’ll need to change those values to match the TurtleCoin address you’re using to collect your node convenience charges as well as the amount of that fee. Remember, as stated in Part 1 of this series, the fee amount is expressed in shells or “atomic units” not TRTL.

Once we’ve made the necessary changes, we’ll save that file and we’re almost ready to rock and roll.

Prepping the Node

For maximum ease of setup you’ll need to copy the latest version of TurtleCoind for your system into the directory we created above. You can compile your own, or download the latest release binaries from the TurtleCoin GitHub repository.

We also advise that you download the latest checkpoints from the Offical Repo to help your node sync up as quickly as possible. The following command will make this quick and easy for you.

Testing the Node

Now that we have all of that set up and ready to rock and roll, we’ll test our node to make sure everything is set up correctly.

Run the following command to find out if everything will start up correctly.

If all goes well, you’ll see output like this:

Depending on if you’ve ever synced a daemon on this machine – it could take a while. Don’t fret if that’s the case. We can continue with even an unsyncronized daemon.

As long as we see TurtleCoind has started followed by TurtleCoind is attempting to synchronize with the network… we know that we’ve set up our environment correctly. Press CTRL-C to stop the node from running. We’re ready to hand off control to PM2.

Launching the Node

Handing off control to PM2 is pretty simple. To do so enter the following command(s):

Checking the Node

To verify that the node is running, we can issue the following command:

PM2 will spit back a list of processes maintained by the software such as:

To view our node logs, issue the command:

Once your node is synced, you’ll start to see log entries such as:

Now sit back, relax, and let people know about your public node.

Categories
Feature Story

Introducing CryptoNight Soft Shell

Objective

To provide the next generation PoW algorithm for TurtleCoin that accomplishes the following goals:

  • Must be a PoW that is different than any other PoW algorithm deployed by another blockchain.
  • Provides additional ASIC/FPGA resistance
  • Encourages solo-mining for decentralization of hashrate
  • Uses a variable amount of of resources (CPU & Memory) that changes dynamically as the chain progresses.
  • Helps to stablize hashrate over time

Problem

Typical implementations of the CryptoNight slow hash routines require static resource amounts to complete the hash calculations.

The figure below is provided for illustrative purposes.

Developers typically jump between any of the resource levels presented by the PoW algorithms or create slight variations of those algorithms to define new resource requirement levels for generating PoW hashes. What fun is that though?

NERVA

As @angrywasp explains in the interview CN Adaptive, Nerva, and the Quest For Fair Mining they approach this issue by changing the PoW every block with CN-Adaptive. This provides a resource requirement that changes over time.

The figure below is provided for illustrative purposes.

As you can see, the line is not straight like normal CryptoNight variants. This change is provided by altering the number of iterations the slow hash algorithm performs in relation to the chain height. They do this in the following way (see permalink). This variation is enough that it makes it harder to implement specialized hardware to mine this PoW algorithm.

Let’s break this down with some sample math. Don’t worry, I’ll keep this simple.

We’ll use an example height of 10,000 to make things easy.

  1. We take height and add 1 to it giving us 10,001.
  2. We then perform a modulo operation on the values such that 10001 % 1024 and get the result 785.
    Note: A modulo operation is a division operation that returns just the remainder.
  3. The result of #3 is fed to a simple addition operation that adds the value to either 0x40000 (262144) or 0x80000 (524288) arriving at the result 262929.
  4. That value, 262929 is then fed to the slow hash routine as the number of iterations to perform.

NERVA creates a Sawtooth that climbs a hill that at it’s peak increases the iterations performed by 1,023. Zooming in, it might look something like this:

Sawtooth Wave

The work NERVA is doing is interesting and got us thinking… if they can change the iterations… what happens when we go crazy?

CryptoNight Soft Shell

I want more!

Being a Turtle has it’s advantages at times, we’re slow, steady, reliable, and like to do things our own way. As such, our buddy @iburnmycd has created a variation of the PoW that is complex enough to meet the requirements above and provides an interesting resource utilization curve.

Instead of a flat (or nearly flat) resource utilization curve CryptoNight Soft Shell has a curve like this:

How do we do this? To put it simply, we vary both the iterations and the scratchpad memory used in such a way that we move from what we’ll call CryptoNight Ultra Fast all the way up past CryptoNightv1 and then cycle back around again to CryptoNight Ultra Fast. One full cycle is completed every 4,096 blocks (and is highly configurable).

We still use height as the foundation of how the math is performed as it is something that we can all easily agree upon. The resulting code looks something like this (subject to change and protected under GPLv3).

Changing just one of the initial defitions drastically changes the results.

If we change the CN_SOFT_SHELL_WINDOW to 1024 our cycle looks something like this:

If we change the CN_SOFT_SHELL_MULTIPLIER to 1 our cycle looks something like this:

Or change the CN_SOFT_SHELL_MULTIPLIER to 16 our cycle looks something like this:

Or we can have a little bump here and there by changing CN_SOFT_SHELL_MULTIPLIER to 16 and CN_SOFT_SHELL_WINDOW to 512

The options are practically endless in how the PoW algorithm can be altered.

Hashrate Variance

A by-product of CryptoNight Soft Shell is that a miner’s hashrate moves according to where in the cycle the blockchain is as the algorithm itself changes over time. This means that as the blockchain reaches the peak of the given cycle, a miner’s hashrate slows. Then, as it starts to return back to the lower valley of the curve, a miner’s hashrate speeds up very fast.

Difficulty Variance

One problem with a Sawtooth Wave is that if the peak of the wave is too far off from the bottom it becomes possible that a mining difficulty adjustment is required otherwise the first few blocks of any given cycle may come out very fast or in some situations very slow. Unlike a Sawtooth Wave, CryptoNight Soft Shell climbs and falls in a safe way. Each step is clearly defined throughout the entire cycle. This helps protect network difficulty from sudden changes as a result of the mining resource utilization rising and/or dropping with each block.

Questions? Comments?

Given the choice, which PoW algorithm looks like the most fun to you?

Hit us up on Discord at http://chat.turtlecoin.lol in the #dev_general channel to learn more.