Why it’s still hard to access data from Blockchain?

Blockchain evidently means — a digital ledger of transactions stored in a chain of blocks across decentralized networks. Each Block contains headers and a list of transactions that are mined. The transactions are verified by a consensus mechanism and a copy is stored in the blocks, making it an immutable “Append-only” Database.

At the time of writing, there exist more than 10,000 applications that are built on top of blockchains. This shows the relevance of blockchain technology and its wider acceptance. My focus here is to elaborate on the most recurring aspect that undeniably all the 10,000+ applications need or would need, which is accessing data from the blockchains.

Access to Blockchain Data

Though blockchains are public and everyone has the right to access data, extracting useful information from it remains a challenge. The blocks are highly encrypted and stockpiled across several systems, making it a strenuous job to retrieve, decode and make use of it. Since its inception, a lot of articles were published explaining why extracting meaningful data from blockchains is a tough task. Let’s examine and see whether things have changed now.

Data is the propelling fuel for digital economy, only if it’s in a functioning form. To illustrate — “crude oil cannot run the car, instead the petrol extracted from the crude oil does”

If I were to be a developer building a resilient dApp, I would ask myself these questions:

1. Is data rich enough to serve my needs?

2. Is the data real-time and reliable?

3. How easy is it to integrate the data provider?

4. How flexible is the data source to accommodate the change-requests?

5. Specific to DeFi, how fast a new protocol can be indexed from the time I request?

6. Information not just data: It’s rather fair to say, we need information from blockchain and not just the data. Given, blockchain stores data in an encoded and non-linear format, we would want something that makes sense.

The only way I can accomplish my priorities is to set up an in-house Indexer which would cost me a fortune. Or the other option would be to make use of either decentralized or centralized third party providers in the market.

Are these efficient? — Let’s take a step back in understanding both

Decentralized Indexers

Decentralized indexers may be real-time, reliable, and flexible. Still, they lack in many aspects.

The drawbacks:

• Mostly support single chains. Ex. The Graph — supports Ethereum only
• They require users to build smart contracts
• Can only provide data in raw format.
• Requires Protocols to build APIs for their users
• Off-chain data such as price, assets are not included
• Most of them provide data in API format only. Notifications, WebSockets, and data streams not supported.

Centralized Indexers

Centralized databases provide ease of use, rich data, and support multichain, but still have a few major flaws.

The drawbacks:

• They mostly have generic Rest APIs for all use-cases. Building “One t-shirt size that fits all” is a no go in a constantly evolving ecosystem. Each user has different use-cases that can only be fulfilled with superset data. I will have to reach out to providers for every new field, and providers have to fit that in the roadmap.
• No support for protocols such as Binance smart chain, Polkadot, etc.
• Centralized providers have high stability risk that comes with a single point of failure.
• As data increases, the latency also increases

To strike a balance, I would need a service that provides flexibility and stability like the decentralized network, and richness, ease of use like centralized providers along with the following,

• Decoded and unmarshalled data.
• Extendibility: eg: Tracking new protocols in minutes.
• Support for all major protocols
• Websocket support for data updates on the fly and accurate price changes.
• Data in many avenues such as notifications, WebSockets, data streams, analytic dashboards
• A global network of servers for low latency service.
• SLAs for protocol support requests.
• 24/7 support helpdesk.

Conclusion

The DeFi space is badly in need of a data solution that solves the problems discussed above and can also handle the load that will result from the widespread adoption of DeFi.

The lack of such a solution is what inspired me to start Unmarshal, where we aim to build an ecosystem that is customer-centric, developer-friendly, and highly scalable.

Do you like what we are solving? or have any queries? Say hi at info@unmarshal.io, I will be happy to talk more about it.