David Langellotti, Growth Lead at Subsquid, explained at Polkadot Weekend how to solve the problem of indexation in Web3 development with Subsquid. The project offers a data indexing solution for networks on Substrate and large NFT projects on Polkadot. We prepared a short summary of the talk. Enjoy! ## About the speaker I am in Web3 space since 2017 (since before it even was called like that). I worked in almost all verticals of the crypto industry. Furthermore, I am BD (business developer) who is going to talk about dev tools. And there is a great sense because BD exists to solve problems, especially developers’ problems. And I feel really fortunate to work with Subsquid now because we are now working at all these verticals. But with us these verticals really started to work. ## Why do we live in a total Web2? Maybe that's terrible that most of you are developers because I'm not. But the good news is we will talk about the developing tool. We hear about Web3 all the time. But for many people, it is just a buzzword that no one understands fully. Let’s just define it very simply as a decentralized Web. Are we in Web3 now? No, I don't think so. The things we are doing online still are happening in Web2, aren't they? Even at this event, we are talking through Telegram and other socials. ![alt_text](https://cyberacademy-space.fra1.digitaloceanspaces.com/uploads/post/images/69/lightbox_001.jpeg "image_tooltip") ## How can we get to the next level of the Web? The next level of the Web is the decentralized Internet connecting to our wallets. I can offer only the part of the answer and this is Subsquid. It is the most performant and resource-efficient indexer, API framework, and backend-as-a-service platform. **Data indexing** — speeding up access to data. And today we are talking about Web3 data — data from blockchains. We need data indexing for many reasons: * It is better for UX — many things now just don’t work properly in Web3 (instant cashing, gaming), we need fast loading and responsiveness, versatile backends for the data-rich UI, scaling to handle thousands of online users and high TPS. * It is better for the dev UX — let devs focus on the dApp business logic, not data issues like storage, cashing, serialization and fast queries. In Web2 devs do not spend so much time figuring out how to get access to the data as they have ready tools for that. Web3 toolkit is not so big. ![alt_text](https://cyberacademy-space.fra1.digitaloceanspaces.com/uploads/post/images/69/lightbox_002.jpeg "image_tooltip") ## How data indexing currently works? Before there were tools for the data indexing in Web3, we basically had a blockchain and developers had to figure out the way to get data despite changes in Polkadot js library, updates to the chain, missing type definition, runtime upgrades and slow gRPC. You have access only to a single gRPC, and it is very slow. So the key issue is that indexing is slow. Any significant change in API requires full scan of the blockchain, which can take hours and days. gRPC node becomes a bottleneck for high-tps use cases. ![alt_text](https://cyberacademy-space.fra1.digitaloceanspaces.com/uploads/post/images/69/lightbox_003.jpeg "image_tooltip") ## How Subsquid can solve the problem? Subsquid can index the whole Kusama network in 15 min by analyzing 50K blocks per second. The service can index other chains 20-100x faster than the competitors. We split up the indexer. We have an archive and a squid. We are able to solve this problem with speed processors caring our batch requests to the archive and the archive sending back filtered data. We created a subgraph that has the synch with the entire blockchain. Also, we created a squid that doesn't have to synch with the blockchain at all because they have all the data in the archive. It releases a lot of time for development. ![alt_text](https://cyberacademy-space.fra1.digitaloceanspaces.com/uploads/post/images/69/lightbox_004.jpeg "image_tooltip") ## API framework How to build a squid: to clone the template on our Github, fill in the schema, iterate and deploy. It has a ton of dev-friendly features. * Type safety * Built-in autocompletion * Bespoke SQLqueries * Custom codec (bypass polkadot.js, because may devs have problems with it) * Supports external libraries (subgraphs with monolithic architecture are not able to do this) * External API calls * Aliases (clean endpoint URLs) * GraphQL subscriptions ![alt_text](https://cyberacademy-space.fra1.digitaloceanspaces.com/uploads/post/images/69/lightbox_005.jpeg "image_tooltip") ## Back-end-as-a-service We offer a back end as a service called Aquarium where we can host your squids, your APIs for free. We are looking at this as a shitcoin free service in the future and with freemium model. Right now it powers 200+ projects, dApps and wallets on DotSama. Subsquid currently supports all substrate chains, EVM and EVM+, WASM. ![alt_text](https://cyberacademy-space.fra1.digitaloceanspaces.com/uploads/post/images/69/lightbox_006.jpeg "image_tooltip") ## Use cases Crowdloans need a lot of data, and we helped to a bunch of the crowdloans projects which needed to index their essential data. Our service can help to easily calculate the rewards for participants for example. Subwallet are our partners and our API framework helped them to avoid a lot of development. Talisman Wallet is using squids as Web3 cash. Also we work with the biggest NFT metaverse on DotSama — RMRK / Skybreach. We help them to arrange the land sale, and also we power the biggest NFT marketplace in Polkadot — Raresama. They about to launch EVM-compatible smart contract platform. Interlay — doing bridge between bitcoin and DeFi. ![alt_text](https://cyberacademy-space.fra1.digitaloceanspaces.com/uploads/post/images/69/lightbox_007.jpeg "image_tooltip") ## The next steps We have got some really good EVM and are interested in building more interesting services. Soon there will be some announcements about the support of some EVM-chains including Polygon and Ethereum, and also we are looking for adding some parachains. ## Resources: ● Docs: [https://docs.subsquid.io](https://docs.subsquid.io) ● Project repository (give us a ⭐, would you?): [https://github.com/subsquid/squid](https://github.com/subsquid/squid) ● Template project: [https://github.com/subsquid/squid-template](https://github.com/subsquid/squid-template) ● Browse the Aquarium [https://app.subsquid.io/aquarium](https://app.subsquid.io/aquarium) ● Public registry of Squid Archives: [https://github.com/subsquid/archive-registry](https://github.com/subsquid/archive-registry) ● Tech community: [SquidDevs](https://t.me/HydraDevs) Telegram group

David Langellotti

Speaker's reports

Speaker's events

Polkadot Weekend 👾 Georgia

Articles

See also

Sergey Prilutskiy

Kirill Pimenov

Denis Soldatov

Andrei Navoichyk

Dmitry Zaharov

Makar Cherniaev

Lana Ives

Denis Pisarev

Sergei Lonshakov

Pavel Tarasov