ByzzBench is a benchmark suite designed to evaluate the performance of testing algorithms in detecting bugs in Byzantine Fault Tolerance (BFT) protocols. It is designed to be modular and extensible, allowing for easy integration of new protocols and scenarios.
ByzzBench is designed for a standardized implementation of BFT protocols and their execution in a controlled testing environment, controlling the nondeterminism in concurrency, network and process faults in the protocol execution, enabling the functionality to enforce particular execution scenarios and thereby facilitating the implementation of testing algorithm for BFT protocols.
This is a Gradle monorepo that contains the following modules:
simulator: The core benchmarking suite, written in Java/Spring Boot. It currently also includes the protocol implementations.webui: A web interface for the benchmarking suite, written in React/NextJS.
For the benchmarking suite to work, you need to have the following installed on your system:
- Java 21
For the user interface to work, you need to have the following installed on your system:
- Node.js
- pnpm
Installing everything on macOS using HomeBrew can be done with the following commands:
brew install openjdk@17 node pnpm
Installing JDK:
-
Through Eclipse Adoptium:
- Download the version you need (JDK-21)
- When installing, select "Set or override JAVA_HOME variable"
-
Through Windows Package Manager - "winget":
// Eclipse Temurin from Eclipse Adoptium
winget install EclipseAdoptium.Temurin.21.JDK
// from Microsoft Build
winget install Microsoft.OpenJDK.21
Note
You might need to set a PATH and JAVA_HOME!
To verify the installation execute the following in cmd (should match the installed version):
java -version
Note
If it displays a different version, then the PATH or JAVA_HOME could be set incorrectly (pointing to another version)!
Installing Node.js:
- Through installer: https://nodejs.org/en/download/prebuilt-installer
- Through package managers: https://nodejs.org/en/download/package-manager
To install pnpm, you can use npm package manager (installed alongside Node.js):
npm install -g pnpm
or Corepack:
corepack enable pnpm
or PowerShell:
Invoke-WebRequest https://get.pnpm.io/install.ps1 -UseBasicParsing | Invoke-Expression
Warning
Windows Defender may block this option!
Using npm or Corepack is the recommended way!
Reference: https://pnpm.io/installation
For other operating systems, please refer to the respective installation instructions.
To build and run the benchmarking suite, run the following command:
./gradlew bootRun
To configure the simulator, you can modify the
application.properties file in the simulator module.
It has two main subsections:
explorationStrategy: The explorationStrategy configuration: which explorationStrategy to use, and its parameters such as the probability of dropping messages.scenario: The scenario configuration: which scenario to run, and its parameters such as conditions to stop the simulation.
The web UI is a simple React application (using NextJS/TypeScript) that allows you to interact with the simulator. It is a work in progress, but provides useful insights into the behavior of the protocols.
To build the web interface, run the following command while the simulator is running:
(cd webui && pnpm install && pnpm run kubb:generate && pnpm run build)
Note
The simulator must be running for kubb:generate to succeed.
The above command will generate the necessary TypeScript bindings for the simulator and build the web interface. You only need to run it once.
Then, to start the web server, run the following command:
(cd webui && pnpm run start)
The UI should then be available at http://localhost:3000