Build, test, and compare workflow systems on realistic data

WfCommons is an open-source ecosystem of workflow execution instances, synthetic workflow generators, and benchmark specifications. It helps the community study scheduling, performance, resilience, and emerging AI-driven workflow automation on modern distributed and HPC platforms.

   
Ecosystem components Documentation
Real instances Workflow executions curated in a common JSON format (WfFormat).
Synthetic realism Generate realistic workflows from real traces.
Benchmarks Produce executable specs for repeatable experiments and fair comparisons.

Publications

Research Papers, Journal Articles, and Technical Reports

Citing WfCommons

When citing WfCommons, please use the following paper. You should also actually read that paper, as it provides a recent and general overview on the framework. 

T. Coleman, H. Casanova, L. Pottier, M. Kaushik, E. Deelman, and R. Ferreira da Silva, "WfCommons: A Framework for Enabling Scientific Workflow Research and Development," Future Generation Computer Systems, vol. 128, pp. 16-27, 2022. DOI: 10.1016/j.future.2021.09.043
@article{coleman2021fgcs,
    title = {WfCommons: A Framework for Enabling Scientific Workflow Research and Development},
    author = {Coleman, Tain\~a and Casanova, Henri and Pottier, Lo\"ic and Kaushik, Manav and Deelman, Ewa and Ferreira da Silva, Rafael},
    journal = {Future Generation Computer Systems},
    volume = {128},
    number = {},
    pages = {16--27},
    doi = {10.1016/j.future.2021.09.043},
    year = {2022},
}

All Publications

  1. T. Coleman, H. Casanova, L. Pottier, M. Kaushik, E. Deelman, and R. Ferreira da Silva, WfCommons: A Framework for Enabling Scientific Workflow Research and Development , Future Generation Computer Systems, vol. 128, pp. 16-27, 2022.
  2. T. Coleman, H. Casanova, and R. Ferreira da Silva, WfChef: Automated Generation of Accurate Scientific Workflow Generators , in 17th IEEE eScience, 2021.
  3. R. Ferreira da Silva, L. Pottier, T. Coleman, E. Deelman, and H. Casanova, WorkflowHub: Community Framework for Enabling Scientific Workflow Research and Development, in 2020 IEEE/ACM Workflows in Support of Large-Scale Science (WORKS), p. 49-56, 2020.