# Testing Plan: Baseline VM Utilization Capture

> This testing plan is filled in for the default experiment included with the Weave Template.
> Use it as a reference when writing test procedures for your own experiment.

---

## Smoke Test (5-10 minutes)

**Goal:** Verify the basic deployment pipeline works.

1. Run with minimal configuration:
   - Slice name: `test-smoke`
   - Duration: `10` seconds
   - Data dir: `data`

2. Verify:
   - [x] Slice reaches StableOK state
   - [x] SSH access works to node1
   - [x] sysstat and jq install without errors (boot config)
   - [x] vmstat.json has ~10 records in valid JSON format
   - [x] memory.json has ~10 records in valid JSON format
   - [x] network.json has ~10 records in valid JSON format
   - [x] `### PROGRESS:` markers appear in Build Log for each step
   - [x] Slice is deleted automatically after experiment completes
   - [x] No orphaned slices remain (check with `fabric_list_slices`)

---

## Integration Test (15-30 minutes)

**Goal:** Verify the complete flow with default parameters.

1. Run with default configuration:
   - Slice name: `test-integration`
   - Duration: `60` seconds (default)
   - Data dir: `data`

2. Verify:
   - [x] All three data files have exactly 60 records each
   - [x] JSON parses correctly (no malformed records, valid timestamps)
   - [x] metadata.json records correct slice name, site, IP, timing
   - [x] summary.json contains CPU, memory, and network sections
   - [x] CPU idle percentage is >80% (idle VM)
   - [x] Memory used percentage is <50%
   - [x] Network throughput is <100 KB/s (idle VM, management traffic only)
   - [x] Progress reporting updates throughout the run (Steps 1/3, 2/3, 3/3)

3. Check notebooks:
   - [x] `notebooks/results.ipynb` renders correctly with collected data
   - [x] CPU, memory, and network charts generate without errors
   - [x] Charts saved as PNG in `data/`

4. Check resource cleanup:
   - [x] Slice deleted after experiment
   - [x] No leftover processes

---

## Full Run

**Goal:** Execute with default parameters and verify end-to-end correctness.

1. Deploy with default configuration:
   - Default slice name, duration 60s

2. Monitoring:
   - [x] Check Build Log for progress updates
   - [x] Use `notebooks/progress.ipynb` for live status

3. Post-completion verification:
   - [x] All 3 data files present and complete (60 records each)
   - [x] summary.json computed with expected metrics
   - [x] Visualizations generated in `notebooks/results.ipynb`
   - [x] No orphaned slices (check with `fabric_list_slices`)

---

## Failure Recovery Test

**Goal:** Verify graceful handling of common failures.

### Site Unavailable
- [x] The `auto` site selection handles this automatically
- [x] If all sites are full, the error message is clear

### Node Failure (SSH timeout)
- [x] `wait_ssh()` retries automatically with timeout
- [x] If SSH never connects, experiment reports the error and exits

### Measurement Tool Failure
- [x] If sysstat fails to install, boot config reports the error
- [x] If capture script fails, experiment prints a WARNING and continues

### Token Expiry
- [x] For a 60-second experiment, token expiry is unlikely
- [x] If it occurs, the error message advises refreshing the token in Configure view

---

## Validation Checklist

After all tests pass, confirm:

- [x] All `### PROGRESS:` markers display correctly in WebUI Build Log
- [x] Slice is deleted automatically after experiment completes
- [x] Data directory contains: vmstat.json, memory.json, network.json, metadata.json, summary.json
- [x] All JSON files parse correctly
- [x] No hardcoded site names (uses `auto` selection)
- [x] Scripts are idempotent (safe to re-run)
- [x] Notebooks produce correct charts from collected data
- [x] `weave.json` has meaningful defaults for all args
- [x] README accurately describes the experiment
