Github Icon-icon-clyde-white-rgb

Building the Blockchain Time Machine - Mission and Roadmap

Token Flow

Token Flow

Our roadmap to build the Blockchain Time Machine - providing three layers of data from raw to semantic from the genesis block to the block that hasn't yet been mined.

Our Mission

Simply stated, our mission is:

Give blockchain ecosystem participants the analytical tools, contextualized data and insights that they need to deeply understand and navigate the maze of on-chain activities.

A few words and phrases in our mission are worth focusing on:

Our users are broadly defined as blockchain ecosystem participants. This includes both existing participants and new entrants. As well as continuing to serve the needs of existing participants like security engineers, protocol developers, token issuers, analysts, traders and market makers we see our tools and products as being potential enablers for new entrants to DeFi in particular since they provide robust, high quality data for analysis and decision making.

We specify analytical tools to reflect the continued importance of EthTx in our focus. It is one of the most used transaction decoders in the ecosystem, particularly for understanding exploits and complex transactions. We continue to develop the open source and free-to-use Community Edition and we will be launching the Pro version in 2022.

Our core product is the data itself. Whilst we build some GUIs, dashboard and portals for customers and specific communities to show the power of the data, we are focused on providing data to enable others to build the visualizations and analytics that they, and their communities, need.

The on-chain activities wording reflects that our products are focused on the wealth of complex low level on-chain data rather than higher level price, volume and market data or centralized exchange data.

A note on architecture

Early on we took the decision to not pick specific protocols, or segments to focus on at the complete exclusion of others. Our natural focus has been DeFi due to the huge amount of liquidity deployed, but our tools go far beyond just DeFi.

Our space moves so quickly that we have no idea what will be the next funnel for liquidity to flow through. When we started building Token Flow the huge growth in NFTs, the Metaverse and Play2Earn (and now GameFi and MetaFi) was not obvious.

This is why the Ethereum Data Warehouse includes all of Ethereum since genesis – every call, event, transaction, block and state. This means that we have data on everything and anything that happened – be it DeFi, NFTs, or just a simple transfer.

Alongside our database of all that happened on-chain, we also have a separate database of semantics of every smart contract on Ethereum that we are able to gather or approximate.

This is where the power of Token Flow comes from: we can combine the whole history of a blockchain in a level of detail that cannot even be obtained from a full archive node (more on this below) with a library of every smart contract’s semantics to decipher the activity in those transactions and smart contract interactions into business context.

Providing data at three levels

We think of data in three levels of sophistication and can return data to consumers at each level:

  • RAW data – this is unprocessed, undecoded low level data but with much more than you can get from another data service or even running your own archive node. Our RAW data includes events, calls etc but also the link between calls and events, state diffs, storage diffs, storage reads, reverted actions and more;
  • DECODED data – this is the RAW data enriched with smart contract level naming and typing so that it is easier for users to understand without having to go to the smart contract themselves;
  • SEMANTIC data – this is our ultimate goal and becomes specific to the protocol or use case since it provides semantically decoded and contextualized data.

Building the blockchain time machine

One of the most frequent questions we get is whether we provide live data or processed data with a lag. The answer is both, and more …

  • Past– for our data stored in EDW we enforce a lag to the tip of the chain (approximately 15 blocks) to avoid reorgs and other complications, and to add semantic decoding and enrichment;
  • Present – since we run our own blockchain infrastructure we are able to provide RAW data to the latest block via our Blockchain Processor API which we use ourselves to feed our data warehouse;
  • Future – we are able to simulate what would happen if a transaction were to be mined by our own nodes given the state at the end of the preceding block and our understanding of the smart contract semantics. This Transaction Simulator is a hugely powerful tool for predicting the effect of transactions and identifying potential threats.

Combining these three levels of data and three time horizons will allow Token Flow users to pick their own à la carte combination of products:

  • A user could take just RAW live data because they want access to storage diffs, state diffs or reverts that they can’t get themselves from their own node or API provider and they don’t need this data to be decoded or semantically enriched;
  • Or they could take SEMANTIC or DECODED data for a more detailed history combined with RAW data for the last 15 blocks so that they have complete time coverage;
  • Similarly a user could just use the Transaction Simulator and embed it into their dApp or wallet UI to give potential threat warnings.

Going multichain

Our core products are currently focused on Ethereum mainnet, but have been architected to handle any EVM compliant chain, and are flexible enough that with some tinkering they can be applied to most non-EVM chains.

As a next step we will be expanding to EVM-based scaling solutions such as Polygon, Avalanche and Fantom. We have already done some preliminary planning work for this expansion but we want the community’s feedback on where our next focus should be (see below).

We are already working on Layer 2 rollups e.g. StarkNet (more coming on this soon) and will expand to both Optimistic and other Zero Knowledge rollup L2s.

Non-EVM layer 1s are also on our longer-term roadmap.

Supercharging EthTx

As mentioned above, we have big plans for EthTx.

As well as continuing to maintain and develop the Community Edition, we will launch the Pro edition this year. Since the Pro edition is fed by the Data Warehouse it means that it will be faster, can provide much more detailed information for each trace and storage transitions, and can batch decode multiple transactions via API.

Our multi-chain strategy will also cover EthTx and we will add other EVM and L2 support or versions. We are already working on StarkTx, a decoder for StarkNet L2.

Token Flow as a Platform

Since the early days of building EthTx our focus has been on developing tools for the community, and engaging with the community.

We will continue to develop and maintain EthTx Community Edition as a free-to-use tool. We will soon publish contribution guidelines to give clearer guidance to communities and developers who want to add to EthTx, and we welcome suggestions on features that you would like to us to add (see below).

We see the Ethereum Data Warehouse in particular as a product for the community, or protocol specific communities, to build their own analytics, visualizations and tools on top of as an alternative to some of the other providers and data sources currently available.

We will share more about “Token Flow as a Platform” and how we will support and incentivize builders in the near future, but if you want to discuss particular tools and analytics for specific protocols or communities we are planning to add such use cases in the upcoming Ethereum Data Warehouse Beta program.

Summarizing out Roadmap

Taking all of the above together, the key features of our medium-term roadmap are:

1. Complete the launch of the Blockchain Time Machine for Ethereum Mainnet:

  • Data Warehouse with data at three levels – RAW, DECODED, SEMANTIC;
  • Blockchain Processor API for live data;
  • Transaction Simulator API.

2. Take the Data Warehouse multichain:

  • Replicate (1) on other EVMs and L2s, using feedback from the community to help prioritize where you want us to focus next;
  • StarkNet tools including StarkTx Transaction Decoder are already being worked on.

3. Supercharge EthTx:

  • Continue to improve EthTx Community Edition;
  • Launch EthTx Pro, powered by the Data Warehouse to allow much more functionality and performance;
  • Take EthTx multichain (StarkTx is already being built).

4. Improve support and incentives to involve the community in EthTx, semantics and building on Token Flow as a Platform.

Join us in illuminating the dark forest

We have already announced the Beta program for the Ethereum Data Warehouse which will initially focus on RAW and then later also DECODED data for Ethereum mainnet. Sign up will open soon.

We are in discussions with a few partners to become Alpha testers for the live Blockchain Processor API and Transaction Simulator and then will move into Beta later in 2022. If you have a use case that would be a good test for these products we want to hear from you so that we can potentially include you in the Alpha or Beta.

We are always keen to talk to builders, protocols and communities about their use cases and what could be built – you can join the discussion in our Discord server and if you have ideas for features or new chains to focus for either EthTx or other Token Flow products, let us know. 

If you want to get involved more directly our team is growing quickly and we are always looking for talented team members – you can find our vacancies and a general application form on our careers page.

More articles