Spacecraft Telemetry Anomaly-Detection Pipeline
Full Track · single-student · lecture-by-lecture validated
Seyed Vahid Ghayoomie · 11142478
Prof. Elisabetta Di Nitto · Dr. Simone Reale
| 🤖 Tool | Claude (Anthropic) — AI coding agent with persistent context, tool-use, and project-local domain skills |
| 🔭 Scope | Full SE lifecycle covered with AI collaboration: requirements → architecture → V&V → CI/CD → HPC deployment |
| ⚙ Role | Pair-programming, doc drafting, diagram generation, test scaffolding — student retained all design decisions and full accountability |
| 📐 Skills | Project-local AI skill definitions in docs/skills/ — domain-aware, course-specific prompting bound to real artifacts |
| 📊 Scale | ~168 h · 24 version tags · 1 student · 1 AI · 13 lectures validated end-to-end |
Student is responsible for all content, design decisions, and results (SE4HPC code of conduct §7).
.sif · SLURM on CINECA G100 (job 20768390, 468/1883 anomalies)| #1 C++ vs Python | Chose C++17 for NFR-03 throughput + POSIX threading + HPC container fit; Python retained for notebook analysis only |
| #2 AI skill design | Two-layer skill system: global SE4HPC course skill + project-local docs/skills/; CLAUDE.md as always-loaded session constitution |
| #3 Lecture mapping | Spiral review: brief + slides read in tandem; se4hpc-checklist.md per-lecture status + 21-GAP backlog driving version planning |
| #4 Doc / code sync | Change-propagation matrix in CLAUDE.md; mandatory pre-commit bookkeeping checklist; per-session command log (98+ entries) |
| #5 Static dashboard | One-time ~60 s real-broker capture → frozen JSON on gh-pages; dashboard replays via fetch() timed loop — no live dependency |
| #6 Three-env DevOps | MK Docker → GitHub Actions → CINECA G100; build-once .sif (NFR-17); 4 handoff scripts; GAP-14-1 anomalies=0 root-caused & fixed v0.9.1 |
| Correctness | NFR-01 rule semantics under deterministic inputs |
| Performance | NFR-03 stable throughput on target CSV workloads |
| Reliability | NFR-05 / NFR-11 defensive boundary validation |
| Scalability | NFR-04 documented distribution strategy (≥3) |
| Portability | NFR-13 / NFR-17 local · VM · SLURM · same .sif |
| Maintainability | NFR-07 / NFR-12 modular, evolvable components |
| Operability | NFR-15 observable logs & deterministic outputs |
config/alpha_mission/ (20 rules, all 4 types) from SimoneReale/astralog-control. Output formats match PDF §3.ingestion → rule_engine → anomaly_detector → output_writer.RuntimeExecutionAdapter isolating local vs SLURM.RuntimeExecutionAdapter hides the runtime.src/cpp/include/astralog/*.hpp realise
each interface; the full-track component diagram shows provided/required ports.// C&C view realised as four C++ components
class IngestionValidator { // FR-02, NFR-05
std::optional<Record>
validate(const TelemetrySample&) const;
};
class RuleEngine { // FR-04/06
std::vector<Violation>
evaluate(const Record&) const;
};
class AnomalyClassifier { // FR-07/08
AnomalyEvent classify(
const Record&,
const std::vector<Violation>&) const;
};
// OutputWriter → valid_data.csv · alarms.log
// · pipeline_summary.jsonMQTTIngestionAdapter as the project's sole broker client (parallelized across both canonical brokers via run_multi_broker_ingestion, GAP-03-2/DD-06) — the JS dashboard never touches the broker, it reads the adapter's output files. Both paths feed the same pipeline via the abstract MQTTTransport interface.MQTTIngestionAdapter is the
sole wss://broker:8884 (MQTT v5) consumer, run in parallel across both canonical
brokers — the static dashboard never opens a broker connection, it polls the adapter's result
files instead (GAP-03-2/DD-06, "trust the C++ verdict").run_multi_broker_ingestion
as a P/T net: T1 ‖ T2 concurrent threads; T3 join guard [P3 ∧ P4] enforces ordering before
write_merged_summary(). Verified: deadlock-free, safe (1-bounded),
live. Maps 1-to-1 to the std::thread + join() implementation.cppcheck static-analysis CI stage + boundary/negative
fixtures; the pipeline must never crash on malformed input (NFR-05/11). v0.8.1:
every rejected MQTT packet is now triaged by classify_corruption()
(syntax → schema → type gates) into MalformedJson/SchemaError/
TypeError and logged to corruption_log.json for operator
diagnosis (GAP-08-1 partially closed).| 1/9 smoke_test ✅ 0.12s | End-to-end pipeline: CSV in → validated records → anomaly JSON out |
| 2/9 fr_matrix_test ✅ 0.04s | FR-02/04/07 assertions on deterministic boundary fixtures |
| 3/9 config_loader_test ✅ 0.09s | YAML mission-config loading + all 4 rule-type deserialization |
| 4/9 rule_engine_full_test ✅ 0.18s | All 4 rule types (Simple/Stateful/Delta/Compound) against live config |
| 5/9 mqtt_test ✅ 0.31s | MockMQTTTransport: connect/disconnect success & failure paths (19 sub-tests) |
| 6/9 metamorphic_test ✅ 0.07s | MR1 alarm uniqueness · MR2 sensor independence · MR3 stateful reset |
| 7/9 fuzz_test ✅ 0.89s | 1100 random inputs, seed 0xDEADBEEF; invariants I1–I4 never violated |
| 8/9 output_writer_int… ✅ 0.21s | PDF §3 format, all 3 CorruptionKind events, empty-output edge cases |
| 9/9 slurm_char_test ★ characterization | 2000 G100 SLURM records replayed (job 20768390); golden-file match: 468 anomalies, 1415 nominal (FR-01/04/05/06/08, FR-14) |
metamorphic_test.cpp (v0.9.4, closes GAP-09-1) implements MR1 (alarm uniqueness), MR2 (sensor independence), MR3 (stateful persistence via consecutive-violation counting) using Pipeline({rules}) + hand-built RuleConfig fixtures.// MR3: Stateful rule fires on exactly the N-th
// consecutive violation (consec_violations=3),
// not before — and resets on a nominal sample.
Pipeline p({stateful_rule}); // threshold = 3
assert(!p.process(spike).has_value()); // #1 ✗
assert(!p.process(spike).has_value()); // #2 ✗
assert( p.process(spike).has_value()); // #3 ✓
assert(!p.process(nominal).has_value());// reset| Component replication | k=2 on weakest stages — dual-broker MQTT already does this at ingestion; maps to SLURM multi-task (T6) |
| Checkpoint / restart | Write checkpoint.json every N records; SLURM resubmit resumes from last checkpoint — eliminates full-job loss |
| Active-passive failover | Watchdog monitors heartbeat file; secondary pipeline takes over if primary silent > threshold |
| Full pipeline × voter | Two full instances run in parallel; write_merged_summary() acts as voter — total $A≈0.995$ |
CLAUDE.md change-propagation matrix + file-registry.md keep artifacts in sync.PahoTransport live capture (720 msgs, 229 anomalies)anomalies=0 root cause) + G100 re-run (job 20768390, 468/1883)kPipelineVersion constant + 6 V&V gaps closed (8/8 CTest, lcov, fuzz, metamorphic, perf, signing)ci-baseline.yml builds, lints, tests, runs the batch
pipeline, validates JSON, and uploads evidence; slurm-deploy.yml ships the signed .sif.- name: Configure C++ # cmake -S src/cpp -B build
- name: Build C++ # cmake --build build
- name: Static analysis (cppcheck)
- name: Run C++ tests # ctest --output-on-failure
- name: Run C++ batch pipeline # --batch 200 --out results
- name: Validate pipeline_summary.json schema
- name: Upload CI evidence artifacts # ci-evidenceslurmctld · slurmd · slurmdbd.sbatch · srun · squeue; FIFO → priority → backfill.RuleEngine --parallel · run_multi_broker_ingestion · output fan-out.--parallel: std::async per independent rule (Simple + StepDiff, 13/20) — scales with --cpus-per-task.std::thread per broker (fixed 2) · output fan-out: 4 concurrent std::async writers.#SBATCH --job-name=astralog
#SBATCH --account=<budget> # G100 association
#SBATCH --partition=g100_usr_prod
#SBATCH --nodes=1 --ntasks-per-node=1
#SBATCH --cpus-per-task=4 --mem=8G --time=00:30:00
module load apptainer
srun apptainer exec --bind $HOME/astralog:/data \
astralog.sif astralog_demo --batch 200 --out /data/resultsastralog.def builds the image from source; the same
signed .sif runs in CI and on G100 (NFR-16/17).Bootstrap: docker
From: ubuntu:22.04
%post
apt-get install -y --no-install-recommends cmake g++ make
cmake -S /opt/astralog/src/cpp -B /opt/astralog/build
cmake --build /opt/astralog/build -j 2
%runscript
exec /opt/astralog/build/astralog_demo "$@"fr_matrix_test.cpp +
smoke_test.cpp; T3 now has a dedicated
output_writer_integration_test.cpp (v0.9.4, closes GAP-Ex5-1) with 5 named
test cases covering the PDF §3 format spec, all 3 CorruptionKind events, and
empty-output edge cases; T5 ran on G100 (job 20768390); T6/T7 remain stretch
goals.| Thread | Boundary | Status |
|---|---|---|
| T1 | Ingestion + Rule | ✅ done |
| T2 | Rule + Classifier | ✅ done |
| T3 | Classifier + Output | ✅ done (v0.9.4) |
| T4 | End-to-end pipeline | ✅ done |
| T-MQTT | MQTT ingestion → pipeline | ✅ done |
| T5 | SLURM + config-driven pipeline | ✅ done (v1.0.1) |
| T6–T7 | SLURM multi-task/aggregate | ⚠️ stretch |
docs/skills/) loaded per context (SLURM, MQTT, V&V) to keep sessions focused.docs/se4hpc-checklist.md + GAP backlog.CLAUDE.md; mandatory pre-commit bookkeeping checklist (07-log, file-registry, CHANGELOG, version cells) enforced before every commit gate..sif on server + nginx for live feed.scripts/slurm/01–04 automate the fetch/stage/submit/retrieve cycle. Same CI-built .sif transferred without on-cluster rebuild (NFR-17 mobility-of-compute).| Challenge | Resolution |
|---|---|
| Language choice | C++ pipeline / Python analysis |
| AI context management | Layered skill architecture |
| Lecture coverage | Bidirectional traceability + GAP table |
| Version drift | Propagation matrix + commit gate |
| Static demo | One-time live capture → frozen replay |
| Multi-env DevOps | 4-script automation + NFR-17 .sif |
Every lecture is traced through Phase 1/2 requirements & design, the C++ implementation,
the test suite, the CI/CD pipeline, the dashboard, and this deck — recorded in
docs/se4hpc-checklist.md. 9/9 CTest tests wired (8 fast pass always; 9th characterization test replays 2000 G100 records with golden file, [SKIP] in CI). G100 evidence: job
20768390 · samples=2000 · anomalies=468. Petri net verified concurrency.
Two-tier Apptainer signing in CI.
Presentation → vahidgh.github.io/ghayoomie-se4hpc/ · Dashboard → /ui/ · Checklist → docs/se4hpc-checklist.md