Build, test, and compare workflow systems on realistic data

WfCommons is an open-source ecosystem of workflow execution instances, synthetic workflow generators, and benchmark specifications. It helps the community study scheduling, performance, resilience, and emerging AI-driven workflow automation on modern distributed and HPC platforms.

   
Real instances Workflow executions curated in a common JSON format (WfFormat).
Synthetic realism Generate realistic workflows from real traces.
Benchmarks Produce executable specs for repeatable experiments and fair comparisons.

WfSim

Workflow Simulation

3

workflow simulators

An alternative to conducting scientific workflow research via real-world experiments is to use simulation, i.e., implement a software artifact that models the functional and performance behaviors of software and hardware stacks of interest. Simulation is used in many computer science domains and can address the limitations of real-world experiments, e.g. they are confined to application and platform configurations available at hand, and thus cover only a small subset of the relevant scenarios that may be encountered in practice.

WfSim fosters the use of simulation for the development, evaluation, and verification of scheduling and resource provisioning algorithms (e.g., multi-objective function optimization, etc.), evaluation of current and emerging computing platforms (e.g., clouds, IoT, extreme scale, etc.), among others.

In this page, we keep a list of open source workflow management systems simulators and simulation frameworks that provide support for WfFormat, which is used for generating workflow execution traces (WfTraces), and synthetic workflows obtained with WfGen.

We are constantly seeking for additional simulators to be added to the list below. If you want to also have your simulator listed in this page, please contact us. Note that each simulator should be compatible with WfFormat.