Pegasus/HTCondor Distributed Workflow Deployment on FABRIC ( public ) ( PegasusAI )

This Jupyter notebook automates the deployment of a distributed Pegasus/HTCondor infrastructure on the FABRIC testbed. It provisions a central submit node and multiple worker nodes across different FABRIC sites, connects them via FABNetv4, and configures all nodes for distributed scientific workflow execution.

  • Submit Node: Runs the HTCondor scheduler and Pegasus Workflow Management System.
  • Worker Nodes: Distributed across sites, configured as HTCondor execute nodes.
  • Automated Steps:
    • Slice creation and node provisioning
    • FABNetv4 networking setup
    • Automated installation of HTCondor and Pegasus
    • SSH key exchange for passwordless access
    • Custom /etc/hosts configuration for hostname resolution
    • Role-based configuration for submit and worker nodes

This notebook is intended for researchers and educators who want to run scalable, distributed workflows on FABRIC using Pegasus and HTCondor.


4    3 (0)    1
Jan. 18, 2026, 9:41 p.m.


Versions

2026-01-18 Jan. 18, 2026, 9:43 p.m. urn:fabric:contents:renci:5a5978b5-c92b-41b1-aae2-a47e39b81a6f 3 download

Authors