GitHub - wmaslonek/guardian-db: GuardianDB: The Rust Implementation of OrbitDB. A peer-to-peer database for the Decentralized Web.

GuardianDB: The Rust Implementation of OrbitDB. A peer-to-peer database for the Decentralized Web.

GuardianDB is a complete rewrite of OrbitDB in Rust, designed to overcome the limitations of previous implementations while providing superior performance, safety, and functionality.

What is GuardianDB?

GuardianDB is a high-performance, decentralized peer-to-peer database built with native Rust and IPFS. It provides distributed data storage without centralized servers, using CRDTs (Conflict-free Replicated Data Types) for automatic conflict resolution and eventual consistency. Think of GuardianDB as having a "MongoDB" or "CouchDB", but completely decentralized, running on a native IPFS network where each participant maintains their own data while seamlessly sharing updates.

Performance & Technical Specifications

Native Performance Metrics

Discovery Speed: 2-5x faster than basic implementations with intelligent caching
Storage Backend: Sled-based high-performance embedded database
Network Stack: Native libp2p with Gossipsub, DHT, and mDNS
Throughput: 600+ discoveries/second in parallel operations
Cache Efficiency: >85% hit ratio for frequently accessed data
Startup Time: <2 seconds for full node initialization
Memory Usage: Optimized for both resource-constrained and server environments

Technical Architecture

Backend: Native Iroh IPFS node (v0.92.0) - no external dependencies
Networking: libp2p with multi-transport support (TCP, QUIC, WebSocket)
Discovery: Multi-method peer finding (Kademlia DHT + mDNS + Bootstrap)
PubSub: Production Gossipsub with flood protection and peer scoring
Storage: Pluggable datastore with Sled default (RocksDB-style performance)
Async Runtime: Full Tokio integration with structured concurrency
Observability: Complete OpenTelemetry tracing with distributed context

Architecture

GuardianDB v0.10.x
├── Core Database Interface
│   ├── GuardianDB (guardian.rs)        # Main database API
│   └── BaseGuardianDB (base_guardian.rs) # Core implementation
│
├── Store Implementations
│   ├── EventLogStore                   # Immutable append-only logs
│   ├── KeyValueStore                   # Distributed key-value pairs  
│   ├── DocumentStore                   # JSON documents with queries
│   └── BaseStore                       # Common store functionality
│
├── Native IPFS Core API
│   ├── IpfsClient (client.rs)          # Unified IPFS client interface
│   ├── Backends/
│   │   ├── IrohBackend                 # Native Rust IPFS (iroh 0.92.0)
│   │   ├── IrohPubsub                  # Gossipsub integration
│   │   └── Backend Selection           # Multi-backend support
│   ├── Config (config.rs)             # IPFS configuration 
│   └── Compatibility (compat.rs)       # Legacy API support
│
├── Advanced PubSub System
│   ├── DirectChannel                  # Peer-to-peer messaging
│   │   ├── SwarmManager              # libp2p Swarm integration
│   │   ├── GossipsubInterface        # Production-ready Gossipsub
│   │   └── Multi-method Discovery    # DHT + mDNS + Bootstrap
│   ├── EventBus                      # Type-safe event system
│   └── Raw PubSub                    # Low-level messaging
│
├── Access Control Layer
│   ├── GuardianDBAccessController    # Custom AC with signatures
│   ├── IPFSAccessController          # IPFS-based access control
│   └── SimpleAccessController        # Open access (development)
│
├── Data Layer
│   ├── Cache (Sled-based)            # High-performance local storage
│   ├── Keystore                      # Ed25519 cryptographic keys  
│   ├── DataStore                     # Pluggable storage interface
│   └── Message Marshaling           # Efficient serialization
│
├── IPFS Log System
│   ├── Log Entries                   # CRDT log entries
│   ├── Identity Management           # Peer identity & verification
│   ├── Lamport Clocks               # Distributed time ordering
│   └── Replication                   # Automatic data sync
│
└── Observability & Monitoring
    ├── Tracing Integration           # Distributed tracing (completed)
    ├── Performance Metrics           # Real-time performance data
    └── Health Checks                 # System health monitoring

Why GuardianDB is Superior?

Compared to Original OrbitDB (JavaScript):

Memory Safety: Zero memory vulnerabilities thanks to Rust's ownership system
Superior Performance: Elimination of V8 overhead and garbage collector
Type Safety: Compile-time error prevention vs JavaScript runtime errors
Native Binary: Standalone executables without Node.js runtime dependencies
True Concurrency: Fearless concurrency with Tokio async runtime

Compared to OrbitDB (GO):

Zero-Cost Abstractions: High-level abstractions without performance overhead
Ownership System: Deterministic memory management vs Go's garbage collector
LLVM Optimization: Compilation to highly optimized machine code
Safe Concurrency: Thread-safe by design vs potential goroutine data races
Predictable Performance: No GC pauses, consistent latency

Core Rust Advantages

Type Safety: Compile-time guarantees prevent runtime errors
High Performance: Native Rust IPFS (Iroh) + zero-cost abstractions
Memory Safety: Ownership system prevents leaks and use-after-free
True P2P: Multi-method peer discovery with DHT, mDNS, and Bootstrap
Native IPFS: Embedded Iroh node with advanced caching and optimization
Advanced PubSub: Production-ready Gossipsub with flood protection
Observability: Complete tracing integration for monitoring and debugging
Zero Dependencies: Standalone binaries with embedded networking stack

Current Implementation Status

Networking & Discovery:

✔️ Native Iroh IPFS backend with embedded node

✔️ Multi-backend architecture (Iroh + future extensibility)

✔️ Advanced peer discovery (DHT + mDNS + Bootstrap + Relay)

✔️ Production Gossipsub with anti-spam and rate limiting

✔️ SwarmManager with connection pooling and optimization

✔️ Real-time performance metrics and health monitoring

Data Stores & Operations:

✔️ EventLogStore (immutable append-only logs)

✔️ KeyValueStore (distributed key-value pairs)

✔️ DocumentStore (JSON documents with queries)

✔️ BaseStore (common functionality and CRDT operations)

✔️ Multi-level caching with Sled backend

✔️ Cryptographic access control with Ed25519 signatures

Developer Experience:

✔️ 20+ comprehensive examples and demos

✔️ Complete tracing integration for observability

✔️ Type-safe event system with tokio channels

✔️ Extensive test suite (78 passing tests)

✔️ Clear error handling and documentation

Store Types

Event Log Store

use guardian_db::{GuardianDB, NewGuardianDBOptions};
use guardian_db::ipfs_core_api::IpfsClient;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create development IPFS client (uses Iroh backend)
    let ipfs = IpfsClient::development().await?;

    // Create GuardianDB instance
    let options = NewGuardianDBOptions {
        directory: Some("./guardian_data".to_string()),
        ..Default::default()
    };
    let db = GuardianDB::new(ipfs, Some(options)).await?;

    // Create an event log
    let log = db.log("my-event-log", None).await?;

    // Add events to the log
    log.add("Hello, GuardianDB!".as_bytes().to_vec()).await?;
    log.add("This is a decentralized database".as_bytes().to_vec()).await?;
    log.add("Built with Rust and IPFS".as_bytes().to_vec()).await?;

    // Get all entries
    let entries = log.all().await?;
    println!("Total entries: {}", entries.len());

    // Iterate over entries
    for (i, entry) in entries.iter().enumerate() {
        println!("Entry {}: {:?}", i + 1, String::from_utf8_lossy(entry.payload()));
    }

    Ok(())
}

Key-Value Store

use guardian_db::{GuardianDB, NewGuardianDBOptions, CreateDBOptions};
use guardian_db::ipfs_core_api::IpfsClient;

#[tokio::main] 
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize IPFS client and database
    let ipfs = IpfsClient::development().await?;
    let db = GuardianDB::new(ipfs, None).await?;

    // Create a key-value store
    let mut kv = db.key_value("my-kv-store", None).await?;

    // CRUD operations
    kv.put("app_name", "GuardianDB".as_bytes().to_vec()).await?;
    kv.put("version", "0.9.13".as_bytes().to_vec()).await?;
    kv.put("language", "Rust".as_bytes().to_vec()).await?;

    // Get values
    if let Some(name_bytes) = kv.get("app_name").await? {
        let name = String::from_utf8(name_bytes)?;
        println!("App: {}", name);
    }

    if let Some(version_bytes) = kv.get("version").await? {
        let version = String::from_utf8(version_bytes)?;
        println!("Version: {}", version);
    }

    // List all keys
    let all_keys = kv.all().await?;
    println!("All keys: {:?}", all_keys);

    // Delete a key
    kv.del("version").await?;
    println!("After deletion: {:?}", kv.all().await?);

    Ok(())
}

Document Store

use guardian_db::{GuardianDB, NewGuardianDBOptions};
use guardian_db::ipfs_core_api::IpfsClient;
use serde_json::json;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize database
    let ipfs = IpfsClient::development().await?;
    let db = GuardianDB::new(ipfs, None).await?;

    // Create a document store
    let mut docs = db.docs("my-document-store", None).await?;

    // Add JSON documents
    let project_doc = json!({
        "_id": "guardian-db",
        "name": "GuardianDB", 
        "type": "database",
        "version": "0.9.13",
        "language": "Rust",
        "features": ["decentralized", "peer-to-peer", "CRDT", "IPFS"]
    });

    let network_doc = json!({
        "_id": "libp2p-network",
        "name": "LibP2P Network",
        "type": "networking",
        "version": "0.56.0", 
        "protocols": ["gossipsub", "kad-dht", "mdns"]
    });

    // Store documents
    docs.put(project_doc).await?;
    docs.put(network_doc).await?;

    // Query documents by type
    let database_docs = docs.query(|doc| {
        Ok(doc.get("type").and_then(|v| v.as_str()) == Some("database"))
    })?;

    println!("Found {} database documents", database_docs.len());
    
    // Get specific document
    let guardian_docs = docs.get("guardian-db", None).await?;
    println!("GuardianDB doc: {:?}", guardian_docs);

    Ok(())
}

Event System & PubSub

use guardian_db::events::EventBus;
use guardian_db::pubsub::direct_channel::{DirectChannel, create_libp2p_swarm_interface};
use serde::{Serialize, Deserialize};
use tracing::info;

#[derive(Debug, Clone, Serialize, Deserialize)]
struct DatabaseEvent {
    action: String,
    store_name: String,
    timestamp: u64,
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize tracing
    tracing_subscriber::fmt().init();

    // Create type-safe event bus
    let event_bus = EventBus::new();
    
    // Create event emitter and subscriber
    let emitter = event_bus.emitter::<DatabaseEvent>().await?;
    let mut receiver = event_bus.subscribe::<DatabaseEvent>().await?;

    // Spawn event listener task
    let listener_handle = tokio::spawn(async move {
        while let Ok(event) = receiver.recv().await {
            info!("Event received: {} on {}", event.action, event.store_name);
        }
    });

    // Emit some events
    emitter.emit(DatabaseEvent {
        action: "store_created".to_string(),
        store_name: "my-documents".to_string(),
        timestamp: chrono::Utc::now().timestamp() as u64,
    }).await?;

    emitter.emit(DatabaseEvent {
        action: "data_added".to_string(), 
        store_name: "my-documents".to_string(),
        timestamp: chrono::Utc::now().timestamp() as u64,
    }).await?;

    // Create peer-to-peer communication channel
    let span = tracing::info_span!("direct_channel");
    let interface = create_libp2p_swarm_interface(span).await?;
    
    // The interface provides peer-to-peer messaging capabilities
    info!("P2P interface initialized with Gossipsub");

    Ok(())
}

Native IPFS Core API

use guardian_db::ipfs_core_api::{IpfsClient, ClientConfig};
use std::io::Cursor;
use tokio::io::AsyncReadExt;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Quick development setup (uses Iroh backend)
    let client = IpfsClient::development().await?;
    println!("✓ IPFS client initialized with native Iroh backend");

    // Advanced configuration with multiple backends
    let config = ClientConfig {
        backend_type: "iroh".to_string(),           // Native Rust IPFS
        enable_pubsub: true,                        // Gossipsub support
        enable_swarm: true,                         // libp2p networking
        data_store_path: Some("./ipfs_data".into()),
        listening_addrs: vec![
            "/ip4/127.0.0.1/tcp/0".to_string(),     // Local IPv4
            "/ip6/::1/tcp/0".to_string(),           // Local IPv6
        ],
        bootstrap_peers: vec![
            // Connect to IPFS bootstrap nodes
            "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN".to_string(),
        ],
        enable_mdns: true,                          // Local peer discovery
        enable_kad: true,                           // DHT support
        enable_relay: false,                        // Relay functionality
        ..Default::default()
    };

    let ipfs = IpfsClient::new(config).await?;
    println!("✓ Advanced IPFS client configured");

    // Add data to IPFS network
    let data = "Hello from GuardianDB! This is stored on IPFS.";
    let cursor = Cursor::new(data.as_bytes().to_vec());
    let add_response = ipfs.add(cursor).await?;

    println!("Added to IPFS: {}", add_response.hash);
    println!("Size: {} bytes", add_response.size);

    // Retrieve data from IPFS
    let mut stream = ipfs.cat(&add_response.hash).await?;
    let mut buffer = Vec::new();
    stream.read_to_end(&mut buffer).await?;

    let retrieved_text = String::from_utf8(buffer)?;
    println!("Retrieved: {}", retrieved_text);

    // Pin the content (keep it available)
    ipfs.pin_add(&add_response.hash, None).await?;
    println!("Content pinned to local node");

    // Get node info
    let node_id = ipfs.id().await?;
    println!("Node ID: {}", node_id.id);
    println!("Addresses: {:?}", node_id.addresses);

    Ok(())
}

Configuration

Basic Configuration

use guardian_db::{GuardianDB, NewGuardianDBOptions};
use guardian_db::ipfs_core_api::IpfsClient;
use tracing::{info, Level};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize logging/tracing
    tracing_subscriber::fmt()
        .with_max_level(Level::INFO)
        .init();

    // Configure IPFS client with native Iroh backend
    let ipfs = IpfsClient::development().await?;
    info!("IPFS client initialized");
    
    // Configure GuardianDB
    let options = NewGuardianDBOptions {
        directory: Some("./guardian_data".to_string()),
        cache_size: Some(1000),                     // Cache 1000 entries
        ..Default::default()
    };
    
    let db = GuardianDB::new(ipfs, Some(options)).await?;
    info!("GuardianDB initialized successfully");

    // Database is ready for use
    let stores_info = format!(
        "Available store types: EventLog, KeyValue, Document"
    );
    info!("{}", stores_info);
    
    Ok(())
}

Advanced Configuration

use guardian_db::{
    GuardianDB, NewGuardianDBOptions, CreateDBOptions,
    ipfs_core_api::{IpfsClient, ClientConfig},
};
use tracing::{info, Level};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize comprehensive logging
    tracing_subscriber::fmt()
        .with_max_level(Level::DEBUG)
        .init();

    // Production IPFS configuration with Iroh backend
    let ipfs_config = ClientConfig {
        backend_type: "iroh".to_string(),           // Native Rust IPFS
        enable_pubsub: true,                        // Gossipsub messaging
        enable_swarm: true,                         // P2P networking
        enable_mdns: true,                          // Local discovery
        enable_kad: true,                           // DHT routing
        enable_relay: true,                         // NAT traversal
        data_store_path: Some("./production_ipfs".into()),
        
        // Network configuration
        listening_addrs: vec![
            "/ip4/0.0.0.0/tcp/4001".to_string(),    // Public IPv4
            "/ip6/::/tcp/4001".to_string(),         // Public IPv6
            "/ip4/127.0.0.1/tcp/5001".to_string(),  // Local API
        ],
        
        // Bootstrap to public IPFS network
        bootstrap_peers: vec![
            "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN".to_string(),
            "/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa".to_string(),
        ],
        
        // Performance tuning
        connection_idle_timeout: Some(30),          // 30 second timeout
        max_connections: Some(100),                 // Max peer connections
        
        ..Default::default()
    };

    let ipfs = IpfsClient::new(ipfs_config).await?;
    info!("Production IPFS client initialized");

    // Advanced GuardianDB configuration
    let db_options = NewGuardianDBOptions {
        directory: Some("./production_guardian".to_string()),
        cache_size: Some(10000),                    // Large cache for production
        ..Default::default()
    };

    let db = GuardianDB::new(ipfs, Some(db_options)).await?;
    info!("GuardianDB configured for production use");

    // Create stores with custom options
    let log_options = CreateDBOptions {
        create: Some(true),
        store_type: Some("eventlog".to_string()),
        ..Default::default()
    };

    let event_log = db.log("production-events", Some(log_options)).await?;
    info!("Production event log created");

    Ok(())
}

Available Examples (20 Demos)

Run the comprehensive example suite to explore GuardianDB features:

# Core functionality
cargo run --example ipfs_core_api_demo
cargo run --example event_bus_usage
cargo run --example constructor_functionality_demo

# Store implementations  
cargo run --example fsstore_persistence_demo
cargo run --example simple_fsstore_test

# Advanced networking & discovery
cargo run --example advanced_discovery_demo
cargo run --example direct_channel_transport_demo
cargo run --example dht_discovery_demo
cargo run --example simple_dht_demo
cargo run --example iroh_gossipsub_integration_demo

# Performance & optimization
cargo run --example performance_optimization_demo
cargo run --example networking_metrics_demo
cargo run --example hybrid_backend_demo

# Production features
cargo run --example production_integration_demo
cargo run --example access_control_verification_demo
cargo run --example identity_verification_demo

# Advanced examples
cargo run --example p2p_endpoint_demo
cargo run --example key_synchronization_demo
cargo run --example iroh_node_functionality_test

# With debug logging
RUST_LOG=debug cargo run --example advanced_discovery_demo

Development

Prerequisites

Rust 1.89+ (edition 2024)
Git

Build

git clone https://github.com/wmaslonek/guardian-db.git
cd guardian-db
cargo build

Tests

# Run all unit tests (78 passing tests)
cargo test --lib

# Run all tests including integration tests  
cargo test -- --include-ignored

# Run specific test modules
cargo test events::tests                   # Event system tests
cargo test ipfs_core_api::client::tests    # IPFS client tests
cargo test stores::base_store::tests       # Store implementation tests
cargo test access_controller::tests        # Access control tests

# Run with detailed output
RUST_LOG=debug cargo test -- --nocapture

# Test specific functionality
cargo test test_event_log_store
cargo test test_document_store_operations
cargo test test_key_value_operations

# Performance tests
cargo test --release performance_tests

Build Features

# Build with all features (default)
cargo build

# Build optimized release version
cargo build --release

# Check code quality and formatting
cargo clippy                   # Comprehensive linting
cargo fmt                      # Code formatting
cargo check                    # Fast compilation check

# Build documentation
cargo doc --open               # Generate and open docs

# Development tools
cargo watch -x check           # Auto-rebuild on changes  
cargo audit                    # Security audit

Community & Support

GuardianDB is an open-source project welcoming contributions and discussions from developers interested in decentralized systems, IPFS, and Rust programming.

If you are excited about the project, don't hesitate to join our Discord! We try to be as welcoming as possible to everybody from any background. You can ask your questions and share what you built with the community!

Get Involved:

Issues: Report bugs and request features on GitHub Issues
Discussions: Technical discussions and Q&A on GitHub Discussions
Documentation: Contribute to docs and examples
Code: Submit PRs for bug fixes and new features

We welcome developers from all backgrounds and experience levels!

Status

GuardianDB is currently in active development, and there will be breaking changes. While any resulting issues are likely to be easy to fix, there are no guarantees at this stage.

Contributing

See CONTRIBUTING.md for contribution instructions.

License

GuardianDB is distributed under the terms of the MIT license. See LICENSE-MIT for details. Opening a pull request is assumed to signal agreement with these licensing terms.

Acknowledgments

OrbitDB - Original JavaScript implementation and design inspiration
go-orbit-db - Go implementation reference and architecture insights
ipfs-log-rs - CRDT log implementation foundation
Iroh - Native Rust IPFS implementation (v0.92.0)
libp2p - Peer-to-peer networking protocols (v0.56.0)

This project incorporates and builds upon code from ipfs-log-rs, licensed under the MIT License © EQLabs. Significant enhancements and optimizations have been added for production use in decentralized applications.

Useful Links

Original OrbitDB - JavaScript implementation
OrbitDB Golang - Go implementation
Iroh Documentation - Native IPFS for Rust
libp2p Rust - P2P networking

Project Resources

Example Code - 20+ comprehensive demos
Architecture Docs - Design and implementation details

GuardianDB - A secure, performant, and fully decentralized peer-to-peer database for the modern Web.

Name		Name	Last commit message	Last commit date
Latest commit History 508 Commits
.github/workflows		.github/workflows
benches		benches
docs		docs
examples		examples
scripts		scripts
src		src
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
CODE-OF-CONDUCT.md		CODE-OF-CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
Cargo.toml		Cargo.toml
LICENSE-APACHE		LICENSE-APACHE
LICENSE-MIT		LICENSE-MIT
README.md		README.md
codecov.yml		codecov.yml
tarpaulin.toml		tarpaulin.toml

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Licenses found

Uh oh!

Repository files navigation

What is GuardianDB?

Performance & Technical Specifications

Native Performance Metrics

Technical Architecture

Why GuardianDB is Superior?

Core Rust Advantages

Current Implementation Status

Store Types

Configuration

Development

Prerequisites

Build

Community & Support

Status

Contributing

License

Acknowledgments

Useful Links

Project Resources

About

Licenses found

Uh oh!

Releases

Packages

Contributors 2

Languages

License

Licenses found

wmaslonek/guardian-db

Folders and files

Latest commit

History

Repository files navigation

What is GuardianDB?

Performance & Technical Specifications

Native Performance Metrics

Technical Architecture

Why GuardianDB is Superior?

Core Rust Advantages

Current Implementation Status

Store Types

Configuration

Development

Prerequisites

Build

Community & Support

Status

Contributing

License

Acknowledgments

Useful Links

Project Resources

About

Topics

Resources

License

Licenses found

Code of conduct

Contributing

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Contributors 2

Languages

Packages