Decentralization

a year ago

3 min read

The Augur V2 client is a serverless, fast syncing, completely decentralized application. There are:

  • No hosts
  • No applications to run
  • No massive sync times
  • No admin keys

And it runs entirely contained in any modern browser.

No Hosts

The task of providing the actual Augur client is a bit more involved and difficult than in traditional application development. The Augur core developers will never run any live mainnet service, so spinning something up on AWS isn’t feasible. That leaves limited options for making the client accessible.

Relying on altruistic third parties to host the client is a brittle solution and potentially a way for people to fall victim to scams. Providing scripts or executables to self-host is simply a UX nightmare. With that in mind, IPFS for client distribution will be supported. Using IPFS means that the absence of no individual party can shut down client distribution and alterations of the code from what is openly available on GitHub can be provably detected.

No Applications to Run

Augur needs a lot of data to provide an experience approaching what could be accomplished by a comparable centralized application. Traders expect to see a list of relevant markets and be able to perform full text searches and narrow results by complex criteria. Reporters expect to scroll potentially hundreds of disputes scanning for a worthy opportunity without having to wait for data to load.

In Augur V1 the solution was for users to download an executable to their machine which would sync Ethereum logs relevant to Augur and build a database which could be queried to get such information. While functional, this syncing process could take as long as a whole day, and more generally, asking users to download an application to use a web page in the year 2020 is unacceptable.

In Augur V2 there is a similar process, to pull logs relevant to the protocol, however it now takes place entirely within the browser. Raw logs are pulled, stored, and indexed for efficient queries in IndexedDB using the Dexie library.

Of course there is still an ever increasing amount of data to sync and store, so some additional new techniques are being deployed to handle speed and growth.

No Massive Sync Times

There are 4 architectural design choices that enable the Augur V2 client to provide relatively fast load times and limit the long term impact of scaling with growth:

Generous Utilization of On-Chain Storage

The Augur V2 contracts tend to store more user relevant data than many other similar contracts. For example profit and loss data as well as market metadata is stored on chain. While this does mean transactions become slightly more expensive it enables applications like the Augur V2 client to make less network requests and immediately pull relevant data.

Logs Emit Current State

One common problem when aggregating Ethereum log data is that they must be processed serially in order to produce mass data points, such as the current balances accounts have of a token. This makes processing slow and potentially error prone depending on the nature of the aggregator.

Augur V2 provides current state in its logs. Along any index only the most recent log is technically necessary to provide a picture of present and avoid the requirement of serial processing.

Pruning

Even if pulling logs and processing them is speedy this doesn’t address the infinite growth problem. To address this, Augur employs a pruning strategy which deletes data correlated with any market which has expired after a window of time (presently configured to be 2 months).

Warp Sync

Pruning addresses the issue of how to keep infinite growth in the long term from being untenable, but it doesn’t actually address how to make initially syncing a potentially very large corpus of data fast. Pulling logs for a moderately busy 2 months of Augur usage could end up taking 100s of requests, each sending a huge amount of data.

In order to address this, Augur V2 includes a new feature called Warp Sync. Warp Sync at its core is just pulling a large file containing all currently non-pruned raw logs from IPFS and generating a database from that. This is a simple enough concept, but suffers from what turns out to be a very difficult problem: How does a client know what hash to use with IPFS to get the most recent warp sync file?

To solve that problem the Augur V2 protocol itself has a built-in recurring market which asks the question: “What will the next Augur Warp Sync Hash be?”. Clients can, with a single request, get the last resolved such market and find the hash immediately. Hash generation for new warp sync files is automated by the client as well and whoever reports this hash is compensated by the protocol.

No Admin Keys

A common practice in the Ethereum application world is to deploy contracts that have some sort of administrative or “owner” based control. In such a practice there is some individual (or group of individuals) that have the power to exercise special functions on a contract that, among other things, allow them to drain funds, freeze functionality, or even alter the underlying code being executed on the state of the contract.

Augur V2, from the time it is deployed, will have NO admins, owners, or keys with elevated authority over the protocol.