Storage DeFi, Züs Weekly Debrief April 26th

Tiago Souza

April 26, 2023

Happy Wednesday! The highly anticipated cloud cover AMA has been rescheduled for next week. Why the delay? The Apps are preparing for their big release. Stay tuned for more updates and do not miss out on this exciting event. Also, read below on what is Storage DeFi?

Mainnet update

We are progressing well with our critical items on MPT and 2-phase commit and expect to have positive results in 2 weeks, when we can give a more definitive timeline on Mainnet.

Our apps have been delayed as we uncovered some deployment issues with the recent merge of three big PR’s of chain fee, blocks management and final configuration changes. We expect to start releasing the apps next week during the AMA.

Storage DeFi? What is it?

Storage DeFi is an innovative type of DeFi that operates on cloud storage. It enables individuals to purchase and sell storage space within a decentralized economy where everyone can participate and earn. Similar to traditional DeFi, transactions are processed on the blockchain using smart contracts, eliminating the need for intermediaries. However, what sets Storage DeFi apart is that it fosters a complete ecosystem centered around a tangible asset – cloud storage.

The Storage DeFi economy consists of service providers who operate servers on the network, clients who pay for storage, and stakers who stake tokens to receive a steady income. All participants benefit from the real-world asset of storage, which grows as more individuals store their videos, pictures, documents, and host applications and websites.

By leveraging blockchain technology, Storage DeFi offers a secure and transparent way for individuals to participate in a new kind of economy. It provides a decentralized solution that benefits both storage providers and users, while also allowing individuals to earn rewards by staking tokens. With Storage DeFi, the potential for growth and innovation in the cloud storage industry is limitless.

Blockchain Update:

Chain Stuck Issue

Last week, the team continued investigating the network stuck issue. However, the `value not present` error is still present on loading the last partition. Even though the fixes mentioned in the last updates were related, they did not fix the root cause. As a response, the team did the following tests in the last week for debugging/fixing the issue:

Since there have not been other places where the ‘value not present’ causes the state hash mismatch issue, we still assume the issue was caused by the partition package. Therefore, the team has refactored the partitions package to store the last partition in the partition’s controller. Previously, they only stored the partition’s name, size, and the number of partitions in the controller. So, each time when we add/remove a partition, we will increase/decrease the partition number, and insert the partition with an index of `partitions_num – 1` to MPT. The partitions saving/deleting and updating of partitions num are two different MPT actions. By refactoring, the team bound the updating of the last partition and the number of partitions together, thus we should not see the ‘value not present’ error on loading the last partition.

After the fix we have not seen the ‘value not present’ error. However, there is a similar error when removing the item location from the previous partition (previous one of last partition). When the last partition is empty, we will move the items from the previous location to the partition’s controller as the last partition. The item’s location will be removed from MPT, and we got ‘value not present’ when removing the locations. The same pattern as before, some miners got this error, some did not. We have added more detailed logs to check the MPT state root when deleting the item, as well as the logs in MPT in all the places where ‘value not present’ error could happen.

Another assumption is that we only got this error when running `syncAndRetry`. It is used for syncing missing nodes and retrying the work. We got the ‘value not present’ error all in a similar pattern, that the block state computing failed due to ‘node not found’ error, then we will sync the missing nodes captured, and retry the block state computing. The `value not present` error happens when doing the retry. So, the team will test to turn off the syncAndRetry and see if we still see this error. This has not yet been tested, as the team waits for clues from the previous assumption.

Sharders Chaos Testing:

During the investigation, the blockchain team found some sharders that could not get up after a while during the chaos test. The main issue is that sharders got missing nodes error on loading LFB from local storage, which should not happen if they all worked well. The team will also do further investigation after mainnet. For now, they will do a workaround to sync missing nodes from remote so that sharders can get up during the chaos tests.

Other Issues:

Beyond the chain stuck investigation, our teams also merged 16 PRs in 0chain, 10 PRs in gosdk. See core PRs below:

Finished the config update for mainnet preparation. Removed unused configs and adjusted the settings.

Extended the TTL of aggregation for event db.

Made extra fixes for the txn fee feature and changed back to get balance from eventdb.

Added total_storage_income, total_read_income and total_slashed_stake to blobbers, blobber_snapshots and blobber_aggregates.

Returned true errors for /getBlobber endpoint.

Changed node register timeout to 10s.

Cherry-picked fixes from the network stuck debug branch to fix the ‘item not found’ error on loading blobber allocations.

Added balance checking before and after executing a transaction to ensure there’s no tokens burned and minted unexpectedly. Added state context cache for clients to avoid duplicate MPT reading when checking the clients balances.