Design and Modeling of a New File Transfer Architecture to Reduce Undetected Errors Evaluated in the FABRIC Testbed ( public )
The Multi-Level Error Detection (MLED) framework is a recursive architecture designed to reduce the probability of undetected errors (UEP) during large-scale file transfers utilizing the in-network resources. It organizes communication in layers, each implementing its own error detection policy, and can be configured to replicate error detection mechanisms of existing file transfer frameworks. Through a mathematical formulation and experimental validation on the FABRIC testbed, MLED demonstrates a reduction in undetected errors by intercepting them at intermediate layers. This enables early recovery and minimizes the need for full file retransmission under non-zero error rates. This enhanced error detection capability is crucial for scientific applications where ensuring data integrity is essential for reliable analysis and outcomes.
This artifact contains a comprehensive collection of Jupyter notebooks designed to facilitate the entire workflow for conducting and analyzing MLED experiments on the FABRIC testbed. This artifact includes the following components:
-
Setting Up a FABRIC Slice: Two Notebooks (
MLED_setup_no_facility_port.ipynb
andMLED_setup_facility_port.ipynb
) to assist with reserving and configuring slices on the FABRIC testbed, both with and without the use of facility ports. These notebooks guide the setup of the network topology, including the configuration of virtual machines and optional integration of a remote data source via a facility port for greater experimental flexibility. They ensure proper communication between nodes and provide a fully functional environment for MLED deployment. -
Running Experiments: The
MLED_run_experiments.ipynb
notebook is included to automate and control the execution of experiments across multiple configurations. It allows users to deploy MLED processes on designated nodes and conduct file transfer experiments under varying conditions and error rates. -
Analyzing Experimental Results: The
MLED_result_analysis.ipynb
notebook is tailored for processing and visualizing the results of the conducted experiments. It provides tools to calculate performance metrics such as goodput, memory usage, and undetected error probability (UEP), generating graphical representations for easier analysis and comparison between configurations.
These notebooks collectively ensure a reproducible, structured approach for configuring, executing, and analyzing MLED experiments on the FABRIC testbed. Users with an active FABRIC account can replicate the results presented in the research paper by following the provided instructions.
The artifact also contains a folder sigmetrics_paper_output
which contains our experimental results that we included in our paper titled 'Design and Modeling of a New File Transfer Architecture to Reduce Undetected Errors Evaluated in the FABRIC Testbed'. If you want to generate the plots based on our data, then a FABRIC account is not required for analyzing the results using the sigmetrics_paper_output/MLED_result_analysis.ipynb
notebook.
March 22, 2025, 5:03 p.m.
Versions
2025-03-22 | March 22, 2025, 4:44 p.m. | urn:fabric:contents:renci:e0ec4d6c-f297-4827-92fa-f8097eda67d7 | 6 | download |
Authors
- Abraham I Matta , Boston University (matta@bu.edu)
- Arash SARABI , Arizona State University (sarabi.arash@asu.edu)
- Prateek Jain , Boston University (jainp@bu.edu)
- Violet Syrotiuk , Arizona State University (syrotiuk@asu.edu)