Comment on page
Ep1 - Get started
In this tutorial, we'll be exploring how to index and query data on Starknet using Checkpoint. We'll cover the basics of setting up a Checkpoint project to index data on Starknet, including defining a Checkpoint configuration, a GraphQL entity schema, and data writers. We'll also cover how to start the Checkpoint indexer and query the indexed data using the generated GraphQL API.
By the end of this tutorial, you'll have a good understanding of how to use Checkpoint to index and query data on Starknet
💡 We’ll be using https://github.com/snapshot-labs/checkpoint-template as an exemple. Feel free to follow along with the repository.
Step 1: Installing Checkpoint
To get started with Checkpoint, you'll need to install the module using either npm or yarn. Open your terminal and navigate to your project directory, then run the following command:
npm install @snapshot-labs/checkpoint
Or, if you prefer using yarn
yarn add @snapshot-labs/checkpoint
Step 2: Creating the Project Structure
Next, you'll need to create a project structure for your Checkpoint application. In this tutorial, we'll be creating the following structure:
project/
├── src/
│ ├── config.json
│ ├── index.ts
│ ├── schema.gql
│ └── writers.ts
└── package.json
The
src directory will contain all the source files for your application, while package.json will be used to manage your application's dependencies.Step 3: Creating the Checkpoint Configuration
Checkpoint uses a simple process to index data. It traces the blockchain block by block and at each of these blocks, it checks if the smart contract we want to track has emitted events and if so, do these events correspond to those we want?
To do this, we need to create a configuration file for Checkpoint. In the
src directory, create a file named config.json and define the following configuration:{
"network_node_url": "<https://starknet-goerli.infura.io/v3/46a5dd9727bf48d4a132672d3f376146>",
"sources": [
{
"contract": "0x04d10712e72b971262f5df09506bbdbdd7f729724030fa909e8c8e7ac2fd0012",
"start": 185778,
"deploy_fn": "handleDeploy",
"events": [
{
"name": "new_post",
"fn": "handleNewPost"
}
]
}
]
}
The
network_node_url property specifies the URL of the StarkNet node we want to connect to. The sources property is an array of objects that define the smart contract addresses and their respective events we want to index. In this example, we're tracking a list of posts and authors, and listening to the new_post event emitted by the smart contract deployed at 0x04d10712e72b971262f5df09506bbdbdd7f729724030fa909e8c8e7ac2fd0012. The start property specifies the block number from which Checkpoint starts scanning. The deploy_fn and fn properties are the names of the data writer functions to be invoked when the contract deployment and new_post events are encountered respectively.Step 4: Defining GraphQL Entity Schemas
Checkpoint requires a set of defined GraphQL Schema Objects. These schema objects will be used to create the database tables for indexing records and also generate GraphQL queries for accessing the indexed data.
In the
src directory, create a file named schema.gql and define the schema for the Post entity we'll be tracking:type Post {
id: String!
author: String!
created_at_block: Int!
}
Checkpoint will use the above entity (Post) to generate a MySQL database table named
posts with columns matching the defined fields. It will also generate a list of GraphQL queries to enable querying indexed data.Step 5: Creating Data Writers
Data writers are typescript functions that get invoked by Checkpoint when it discovers a block containing a relevant event. A data writer is responsible for writing records to the database. These records will eventually be exposed via Checkpoint's GraphQL endpoint.
In the
src directory, create a file named writers.ts and define the data writer functions for the handleDeploy and handleNewPost events:import { hexStrArrToStr, toAddress } from './utils';
import type { CheckpointWriter } from '@snapshot-labs/checkpoint';
import { Post } from '../.checkpoint/models';
export async function handleDeploy() {
// Run logic as at the time Contract was deployed.
}
// This decodes the new_post events data and stores successfully
// decoded information in the `posts` table.
//
// See here for the original logic used to create post transactions:
// https://gist.github.com/perfectmak/417a4dab69243c517654195edf100ef9#file-index-ts
export async function handleNewPost({ block, tx, event, mysql }: Parameters<CheckpointWriter>[0]) {
if (!event) return;
const author = toAddress(event.data[0]);
let content = '';
let tag = '';
const contentLength = BigInt(event.data[1]);
const tagLength = BigInt(event.data[2 + Number(contentLength)]);
const timestamp = block!.timestamp;
const blockNumber = block!.block_number;
// parse content bytes
try {
content = hexStrArrToStr(event.data, 2, contentLength);
} catch (e) {
console.error(`failed to decode content on block [${blockNumber}]: ${e}`);
return;
}
// parse tag bytes
try {
tag = hexStrArrToStr(event.data, 3 + Number(contentLength), tagLength);
} catch (e) {
console.error(`failed to decode tag on block [${blockNumber}]: ${e}`);
return;
}
// Create new Post from generated models
const post = new Post(`${author}/${tx.transaction_hash}`);
post.author = author;
post.content = content;
post.tag = tag;
post.tx_hash = tx.transaction_hash!,
post.created_at = timestamp;
post.created_at_block = blockNumber;
// Save Posts into your db
await post.save();
}
Step 7: Mounting the GraphQL Endpoint
Checkpoint exposes a GraphQL endpoint to enable querying the indexed data. In the
index.ts file, add the following code to mount the GraphQL endpoint on a port:import 'dotenv/config';
import express from 'express';
import cors from 'cors';
import path from 'path';
import fs from 'fs';
import Checkpoint, { LogLevel } from '@snapshot-labs/checkpoint';
import config from './config.json';
import * as writers from './writers';
import checkpointBlocks from './checkpoints.json';
const dir = __dirname.endsWith('dist/src') ? '../' : '';
const schemaFile = path.join(__dirname, `${dir}../src/schema.gql`);
const schema = fs.readFileSync(schemaFile, 'utf8');
const checkpointOptions = {
logLevel: LogLevel.Info
// prettifyLogs: true, // uncomment in local dev
};
// Initialize checkpoint
// @ts-ignore
const checkpoint = new Checkpoint(config, writers, schema, checkpointOptions);
// resets the entities already created in the database
// ensures data is always fresh on each re-run
checkpoint
.reset()
.then(() => checkpoint.seedCheckpoints(checkpointBlocks))
.then(() => {
// start the indexer
checkpoint.start();
});
const app = express();
app.use(express.json({ limit: '4mb' }));
app.use(express.urlencoded({ limit: '4mb', extended: false }));
app.use(cors({ maxAge: 86400 }));
// mount Checkpoint's GraphQL API on path /
app.use('/', checkpoint.graphql);
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => console.log(`Listening at <http://localhost>:${PORT}`));
We're creating a new Express application and mounting the Checkpoint GraphQL endpoint on the
/graphql path. We're then starting Checkpoint with the required parameters and the server on the specified port.Step 8: Testing the Checkpoint Application
Now that we've set up our Checkpoint application, we can test it by running the following command in the terminal:
yarn devThis will start the Checkpoint indexer and mount the GraphQL endpoint. You can now query the indexed data using the generated GraphQL API.
Checkpoint exposes two types of queries:
- 1.
entities: This query enables you to fetch information about the entities being indexed. - 2.
records: This query enables you to fetch the indexed data.
For example, to fetch all the posts in the database, you can run the following query onhttp://localhost:3000:
query {
posts {
id
author
content
tag
created_at_block
created_at
tx_hash
}
}
Step 9: Customizing Checkpoint
Checkpoint provides several options for customizing its behavior. In the
config.json file, you can set the following options:network_node_url: The URL of the StarkNet node to connect to.sources: An array of objects representing the contracts and their events to track.start_block: The block number to start indexing from..- In the
writers.tsfile, you can customize the data writer functions to suit your indexing needs. - In the
schema.gqlfile, you can define additional entities to track and expose via the GraphQL API.
Conclusion
That’s it! You should have Checkpoint running, indexing your contracts data and serving this indexed data via graphql.