Build, test, and compare workflow systems on realistic data

WfCommons is an open-source ecosystem of workflow execution instances, synthetic workflow generators, and benchmark specifications. It helps the community study scheduling, performance, resilience, and emerging AI-driven workflow automation on modern distributed and HPC platforms.

   
Ecosystem components Documentation
Real instances Workflow executions curated in a common JSON format (WfFormat).
Synthetic realism Generate realistic workflows from real traces.
Benchmarks Produce executable specs for repeatable experiments and fair comparisons.

WfFormat

The WfCommons JSON Format

WfCommons uses a common format for representing workflow execution instances and generated synthetic workflows instances, so that workflow simulators and simulation frameworks (that provide support for WfFormat) can use such instances interchangeably.

This common format uses a JSON specification available in the WfFormat GitHub repository. The schema GitHub repository provides detailed explanation of WfFormat (including required fields), and a validator script for verifying the compatibility of instances.

Below, is a visualization of the schema version 1.3: