Building a Stateful VIDA - Part 1

Stateful VIDAs are robust, consistent, and reliable applications that maintain and validate persistent state across execution instances. They are essential for critical use cases such as financial applications, token systems, voting mechanisms, or any application where data integrity and consistency are paramount.

Unlike stateless VIDAs, stateful VIDAs require:

  • Persistent storage for maintaining state between transactions

  • State validation to ensure data consistency across network participants

  • Consensus mechanisms for distributed state agreement

  • Recovery systems for handling errors and network partitions

What Makes a Production Stateful VIDA Different?

While Lite Stateful VIDAs demonstrate core concepts with simplified implementations, production Stateful VIDAs include:

Feature
Lite Stateful VIDA
Production Stateful VIDA

Storage

HashMap (memory)

MerkleTree (persistent database)

State Verification

None

Cryptographic root hashes

Consensus

Single instance

Multi-peer validation

Recovery

Restart from scratch

Checkpoint-based recovery

Production Ready

❌

✅

Architecture Overview

This tutorial builds a token transfer system that demonstrates all key concepts of stateful VIDAs:

  • Database Layer: RocksDB with MerkleTree for cryptographic state verification

  • Transaction Processing: Handles token transfers with balance validation

  • Peer Consensus: Validates state consistency across network nodes

  • API Layer: HTTP endpoints for inter-node communication

  • Error Recovery: Robust handling of consensus failures and data inconsistencies

Prerequisites

  • Completed "Building a Lite Stateful VIDA" tutorial

  • Basic knowledge of REST APIs

Steps to Build a Stateful VIDA

1. Project Setup and Dependencies

Create a new Maven project with the required dependencies for PWR SDK, database storage, and HTTP server.

mkdir pwr-stateful-vida && cd pwr-stateful-vida
npm init -y
npm install @pwrjs/core express

2. Database Layer Implementation

Create a singleton service for managing persistent state using MerkleTree for cryptographic verification.

// databaseService.js
import { MerkleTree } from "@pwrjs/core/services"

class DatabaseService {
    static #tree = null;
    static #initialized = false;
    static #LAST_CHECKED_BLOCK_KEY = Buffer.from('lastCheckedBlock');
    static #BLOCK_ROOT_PREFIX = 'blockRootHash_';

    static async initialize() {
        if (this.#initialized) {
            throw new Error('DatabaseService already initialized');
        }
        
        try {
            this.#tree = new MerkleTree('database');
            await this.#tree.ensureInitialized();
            this.#initialized = true;
            
            process.on('SIGINT', () => this.#cleanup());
            process.on('SIGTERM', () => this.#cleanup());
            process.on('exit', () => this.#cleanup());
            process.on('uncaughtException', () => this.#cleanup());
            process.on('unhandledRejection', () => this.#cleanup());
            
        } catch (error) {
            throw new Error(`Failed to initialize DatabaseService: ${error.message}`);
        }
    }

    static #getTree() {
        if (!this.#initialized || !this.#tree) {
            throw new Error('DatabaseService not initialized. Call initialize() first.');
        }
        return this.#tree;
    }

    static async #cleanup() {
        if (this.#tree && !this.#tree.closed) {
            try {
                await this.#tree.close();
            } catch (error) {
                console.error('Error closing MerkleTree:', error);
            }
        }
    }

    static async getRootHash() {
        const tree = this.#getTree();
        return await tree.getRootHash();
    }

    static async flush() {
        const tree = this.#getTree();
        return await tree.flushToDisk();
    }

    static async revertUnsavedChanges() {
        const tree = this.#getTree();
        return await tree.revertUnsavedChanges();
    }

    static async getBalance(address) {
        if (!address || address.length === 0) {
            throw new Error('Address must not be empty');
        }

        const tree = this.#getTree();
        const data = await tree.getData(Buffer.from(address));
        
        if (!data || data.length === 0) {
            return 0n;
        }
        
        return this.#bytesToBigInt(data);
    }

    static async setBalance(address, balance) {
        if (!address || address.length === 0) {
            throw new Error('Address must not be empty');
        }

        const tree = this.#getTree();
        const balanceBytes = this.#bigIntToBytes(balance);
        return await tree.addOrUpdateData(Buffer.from(address), balanceBytes);
    }

    static async transfer(sender, receiver, amount) {
        if (!sender || sender.length === 0) {
            throw new Error('Sender address must not be empty');
        }
        if (!receiver || receiver.length === 0) {
            throw new Error('Receiver address must not be empty');
        }

        const senderBalance = await this.getBalance(sender);
        
        if (senderBalance < amount) {
            return false;
        }

        const newSenderBalance = senderBalance - amount;
        const receiverBalance = await this.getBalance(receiver);
        const newReceiverBalance = receiverBalance + amount;

        await this.setBalance(sender, newSenderBalance);
        await this.setBalance(receiver, newReceiverBalance);

        return true;
    }

    static async getLastCheckedBlock() {
        const tree = this.#getTree();
        const data = await tree.getData(this.#LAST_CHECKED_BLOCK_KEY);
        
        if (!data || data.length < 8) {
            return 0;
        }

        const buffer = Buffer.from(data.slice(0, 8));
        return Number(buffer.readBigUInt64BE(0));
    }

    static async setLastCheckedBlock(blockNumber) {
        const tree = this.#getTree();
        const buffer = Buffer.allocUnsafe(8);
        buffer.writeBigUInt64BE(BigInt(blockNumber), 0);
        return await tree.addOrUpdateData(this.#LAST_CHECKED_BLOCK_KEY, buffer);
    }

    static async setBlockRootHash(blockNumber, rootHash) {
        if (!rootHash || rootHash.length === 0) {
            throw new Error('Root hash must not be empty');
        }

        const tree = this.#getTree();
        const key = Buffer.from(`${this.#BLOCK_ROOT_PREFIX}${blockNumber}`);
        return await tree.addOrUpdateData(key, Buffer.from(rootHash));
    }

    static async getBlockRootHash(blockNumber) {
        const tree = this.#getTree();
        const key = Buffer.from(`${this.#BLOCK_ROOT_PREFIX}${blockNumber}`);
        return await tree.getData(key);
    }

    static async close() {
        if (this.#tree && !this.#tree.closed) {
            await this.#tree.close();
        }
        this.#initialized = false;
        this.#tree = null;
    }

    static #bytesToBigInt(bytes) {
        if (bytes.length === 0) return 0n;
        let result = 0n;
        for (let i = 0; i < bytes.length; i++) {
            result = (result << 8n) | BigInt(bytes[i]);
        }
        return result;
    }

    static #bigIntToBytes(bigint) {
        if (bigint === 0n) return Buffer.from([0]);
        
        const bytes = [];
        let value = bigint;
        while (value > 0n) {
            bytes.unshift(Number(value & 0xFFn));
            value = value >> 8n;
        }
        return Buffer.from(bytes);
    }
}

export default DatabaseService;

Key Features:

  • MerkleTree Integration: Provides cryptographic state verification through root hashes

  • Atomic Transfers: Ensures balance consistency during token transfers

  • Block State Management: Tracks validated states for specific block heights

  • Error Recovery: Can revert changes when consensus validation fails

3. HTTP API Implementation

Create HTTP endpoints for peer communication and state queries.

// api/get.js
import DatabaseService from '../databaseService.js';

export class GET {
    static run(app) {
        app.get('/rootHash', async (req, res) => {
            try {
                const response = await this.handleRootHash(req.query);
                
                if (response === '') {
                    return res.status(500).send('');
                }
                
                if (response.startsWith('Block root hash not found') || 
                    response === 'Invalid block number') {
                    return res.status(400).send(response);
                }
                
                return res.send(response);
                
            } catch (error) {
                return res.status(500).send('');
            }
        });
    }

    static async handleRootHash(params) {
        const blockNumberStr = params.blockNumber;
        if (!blockNumberStr) {
            throw new Error('Missing blockNumber parameter');
        }
        
        const blockNumber = parseInt(blockNumberStr, 10);
        if (isNaN(blockNumber)) {
            throw new Error('Invalid block number format');
        }
        
        const lastCheckedBlock = await DatabaseService.getLastCheckedBlock();
        
        if (blockNumber === lastCheckedBlock) {
            const rootHash = await DatabaseService.getRootHash();
            if (rootHash) {
                return rootHash.toString('hex');
            } else {
                return '';
            }
        }
        else if (blockNumber < lastCheckedBlock && blockNumber > 1) {
            const blockRootHash = await DatabaseService.getBlockRootHash(blockNumber);
            
            if (blockRootHash !== null) {
                return blockRootHash.toString('hex');
            } else {
                return `Block root hash not found for block number: ${blockNumber}`;
            }
        } else {
            return 'Invalid block number';
        }
    }
}

export default { GET };

Summary of Part 1

In Part 1, we covered the foundational components of a stateful VIDA:

  1. Project Setup: Configured dependencies for PWR SDK, HTTP server, and database integration

  2. Database Layer: Implemented a MerkleTree-backed service for cryptographic state verification and persistent storage

  3. HTTP API: Created endpoints for peer communication and root hash queries

These components provide the infrastructure needed for maintaining and validating persistent state across network participants. In Part 2, we'll implement the transaction processing logic, main application orchestration, and demonstrate how to run the complete stateful VIDA system.

The database layer ensures data integrity through cryptographic verification, while the API layer enables peer consensus validation. Together, they form the foundation for building robust, production-ready stateful applications on the PWR Chain.

PreviousBuilding a Lite Stateful VIDANextBuilding a Stateful VIDA - Part 2

Last updated

Was this helpful?