🛰 Software Engineering for HPC · 2026 · Politecnico di Milano

AstraLog-HPC

Spacecraft Telemetry Anomaly-Detection Pipeline

Full Track · single-student · lecture-by-lecture validated

v1.0.0 · 2026-06-09 Full Track C++17 · CMake · CTest Apptainer / SLURM

Seyed Vahid Ghayoomie · 11142478
Prof. Elisabetta Di Nitto · Dr. Simone Reale

AstraLog-HPC · SE4HPC 2026
Introduction

Project & Author

  • Seyed Vahid Ghayoomie · Student ID 11142478
  • Course: Software Engineering for High-Performance Computing
    Politecnico di Milano · 2026
  • Instructors: Prof. Elisabetta Di Nitto · Dr. Simone Reale
  • Track: Full Track (single-student) · Mission close 2026-06-14
AstraLog-HPC → a C++17 spacecraft telemetry anomaly-detection pipeline — ingesting real-time sensor packets via MQTT dual-broker, classifying anomalies against configurable mission rules, and producing structured V&V evidence artifacts deployed on CINECA G100 HPC cluster.
C++17 · CMake · CTest SLURM · CINECA G100 v1.0.1 · 9/9 tests ✓

⚡ A new perspective: AI-Assisted Software Engineering

🤖 Tool Claude (Anthropic) — AI coding agent with persistent context, tool-use, and project-local domain skills
🔭 Scope Full SE lifecycle covered with AI collaboration: requirements → architecture → V&V → CI/CD → HPC deployment
⚙ Role Pair-programming, doc drafting, diagram generation, test scaffolding — student retained all design decisions and full accountability
📐 Skills Project-local AI skill definitions in docs/skills/ — domain-aware, course-specific prompting bound to real artifacts
📊 Scale ~168 h · 24 version tags · 1 student · 1 AI · 13 lectures validated end-to-end

Student is responsible for all content, design decisions, and results (SE4HPC code of conduct §7).

Introduction
Overview

AstraLog-HPC — What We Built

  • C++17 pipeline: dual-broker MQTT ingestion → rule engine (20 rules, FR-01–12) → anomaly classifier → structured evidence output
  • 9/9 CTest wired (8 fast always-run + 1 characterization) · lcov coverage · cppcheck static analysis · fuzz + metamorphic tests · perf benchmark ≥ 1 000 samples/s
  • Full CI/CD on GitHub Actions · Apptainer .sif · SLURM on CINECA G100 (job 20768390, 468/1883 anomalies)
  • Static Mission Control Dashboard (GitHub Pages) · live Reveal.js presentation · 15+ tracked documents
  • 13 lectures validated end-to-end · 24 version tags · ~168 h · 1 student · 1 AI agent
Approach → every SE lifecycle phase applied to a real system with lecture-by-lecture traceability and explicit evidence artifacts. Six recurring engineering tensions shaped how it was done.

6 Engineering Challenges & How We Tackled Them

#1 C++ vs Python Chose C++17 for NFR-03 throughput + POSIX threading + HPC container fit; Python retained for notebook analysis only
#2 AI skill design Two-layer skill system: global SE4HPC course skill + project-local docs/skills/; CLAUDE.md as always-loaded session constitution
#3 Lecture mapping Spiral review: brief + slides read in tandem; se4hpc-checklist.md per-lecture status + 21-GAP backlog driving version planning
#4 Doc / code sync Change-propagation matrix in CLAUDE.md; mandatory pre-commit bookkeeping checklist; per-session command log (98+ entries)
#5 Static dashboard One-time ~60 s real-broker capture → frozen JSON on gh-pages; dashboard replays via fetch() timed loop — no live dependency
#6 Three-env DevOps MK Docker → GitHub Actions → CINECA G100; build-once .sif (NFR-17); 4 handoff scripts; GAP-14-1 anomalies=0 root-caused & fixed v0.9.1
Overview
Mission Control Dashboard · live · open ↗
Project Documents
Lecture 101.Introduction.pdf

Introduction to SE for HPC

  • Software = process + product; weak process degrades the product.
  • Verification = "build it right" · Validation = "build the right thing".
  • Each lifecycle phase must exist as a concrete, reviewable artifact.
  • High V&V effort share; transparency & traceability are first-class.
AstraLog → world↔machine boundary frames the whole project: physical sensors (world) vs the anomaly-detection pipeline (machine), meeting at telemetry packets.
docs ✅impl N/AFR-matrix ✅CI ✅
World-machine boundary
phase1/uml — world ↔ machine & shared phenomena
L1 · Intro
Lecture 202.QualitiesAndHPCSoftware.pdf

HPC Qualities & SE Methods

  • Seven HPC qualities drive every design decision.
  • Verify desired properties when full proof is impractical.
  • Performance must not erode maintainability or portability.
  • Reliability = defensive programming + testing + observability.
AstraLog → each quality maps to explicit NFRs; malformed input is rejected safely (NFR-05/11) and execution stays deterministic for verification (NFR-01).
docs ✅impl ✅perf benchmark ✅CI ✅

7 HPC qualities → AstraLog NFRs

CorrectnessNFR-01 rule semantics under deterministic inputs
PerformanceNFR-03 stable throughput on target CSV workloads
ReliabilityNFR-05 / NFR-11 defensive boundary validation
ScalabilityNFR-04 documented distribution strategy (≥3)
PortabilityNFR-13 / NFR-17 local · VM · SLURM · same .sif
MaintainabilityNFR-07 / NFR-12 modular, evolvable components
OperabilityNFR-15 observable logs & deterministic outputs
L2 · Qualities
Lecture 303.RE.pdf

Requirement Engineering

  • World vs machine; shared phenomena at the boundary.
  • Separate G goals · DA domain assumptions · R requirements.
  • Requirement quality: clear, complete, consistent, verifiable.
  • Use cases capture actor ↔ machine interactions.
AstraLog → FR-01…19, NFR-01…17, DA, MR1–MR3 all implemented. Mission config config/alpha_mission/ (20 rules, all 4 types) from SimoneReale/astralog-control. Output formats match PDF §3.
docs ✅impl ✅tests ✅format ✅
Use-case diagram full track
phase1/uml — use-case diagram
L3 · RE
Exercise 11_exercise_lecture_sol.pdf

RE process — applied to AstraLog

  • Exercise pattern: build a world-machine + use-case model of a system.
  • World phenomena: real sensor values on the spacecraft bus.
  • Shared phenomena: telemetry packet (CSV row / MQTT message).
  • Use cases: Process batch · Evaluate rules · Raise alarm · Emit valid record.
Implemented → the model is realised by the chain ingestion → rule_engine → anomaly_detector → output_writer.
solution ✅docs ✅MQTT C++ ✅
Dashboard file-read sequence
AstraLog world-machine — exercise solution
Ex1 · RE process
Lecture 404.Design.pdf

Software Design & Architecture

  • Multi-view design: C&C, module, deployment.
  • Architecture decisions precede and constrain code.
  • Ariane-5: reuse without re-validation is a design failure.
AstraLog → four-component C&C view (Ingestion → Rule → Classifier → Output) plus a RuntimeExecutionAdapter isolating local vs SLURM.
docs ✅adapter unimpltests ✅UI ✅
Full-track component diagram
phase2/uml — full-track components & interfaces
L4 · Design
Exercise 22_exercise_lecture_sol.pdf

Architecture with explicit interfaces

  • Exercise pattern: components with provided/required interfaces + adapter.
  • AstraLog interfaces: Ingestion, RuleEngine, Classifier, OutputWriter.
  • Adapter boundary: RuntimeExecutionAdapter hides the runtime.
Implemented → headers in src/cpp/include/astralog/*.hpp realise each interface; the full-track component diagram shows provided/required ports.
solution ✅adapter unimpldocs ✅
src/cpp/include/astralog/*.hpp — component interfaces
// C&C view realised as four C++ components
class IngestionValidator {         // FR-02, NFR-05
    std::optional<Record>
        validate(const TelemetrySample&) const;
};
class RuleEngine {                 // FR-04/06
    std::vector<Violation>
        evaluate(const Record&) const;
};
class AnomalyClassifier {          // FR-07/08
    AnomalyEvent classify(
        const Record&,
        const std::vector<Violation>&) const;
};
// OutputWriter → valid_data.csv · alarms.log
//              · pipeline_summary.json
Ex2 · Architecture
Lecture 505.DesignPartiII.pdf

Design Principles (Part II)

  • Nine principles: low coupling, high cohesion, information hiding…
  • …separation of concerns, abstraction, anticipation of change.
  • Modular decomposition keeps components independently testable.
AstraLog → header/impl separation + single-responsibility components give low coupling (NFR-07) and evolvability for new rule types & sources (NFR-12).
docs ✅impl ✅tests ✅
Deployment diagram local vs SLURM
phase1/uml — module decomposition view
L5 · Principles
Lecture 606.DesignPartIII.pdf

Architectural Styles

  • Dataflow, pipeline, event-driven, pub/sub (MOM), shared-data.
  • Batch vs stream processing trade-offs.
  • Style choice must follow the requirements, not fashion.
AstraLog → pipe-and-filter for the CSV batch path; event-driven pub/sub for MQTT, with the C++ MQTTIngestionAdapter as the project's sole broker client (parallelized across both canonical brokers via run_multi_broker_ingestion, GAP-03-2/DD-06) — the JS dashboard never touches the broker, it reads the adapter's output files. Both paths feed the same pipeline via the abstract MQTTTransport interface.
docs ✅impl ✅C++ MQTT ✅UI ✅
CSV processing sequence
phase2/uml — CSV pipeline (batch dataflow)
L6 · Styles
Exercise 33_exercise_lecture_no_sol.pdf

Style choice + justification

  • Exercise pattern: pick a style and justify it against requirements.
  • CSV path → pipeline: linear filters, deterministic, easy to test.
  • MQTT path → pub/sub: decoupled producer/consumer over the broker.
Implemented → C++ pipeline for batch; the C++ MQTTIngestionAdapter is the sole wss://broker:8884 (MQTT v5) consumer, run in parallel across both canonical brokers — the static dashboard never opens a broker connection, it polls the adapter's result files instead (GAP-03-2/DD-06, "trust the C++ verdict").
solution ✅MQTT adapter ✅UI file-reader ✅
MQTT processing sequence
phase2/uml — parallel dual-broker MQTT ingestion (sole consumer)
Ex3 · Style pick
Lecture 707.VVPartI.pdf

V&V I — Arch Verification & Petri Net

  • QA policy; the V model from unit up to acceptance.
  • Architectural verification checklist; concurrent-resource review.
  • Petri nets for formal concurrency verification (deadlock, liveness, boundedness).
AstraLog → phase2 Step 16b models run_multi_broker_ingestion as a P/T net: T1 ‖ T2 concurrent threads; T3 join guard [P3 ∧ P4] enforces ordering before write_merged_summary(). Verified: deadlock-free, safe (1-bounded), live. Maps 1-to-1 to the std::thread + join() implementation.
docs ✅Petri net ✅CI ✅

P/T net — dual-broker pipeline

Petri net: concurrent dual-broker MQTT ingestion pipeline
Blue ovals = places  ·  Yellow rect = transitions  ·  ● = initial token
T3 fires only when both P3 and P4 hold a token (join synchronisation)
L7 · V&V I
Lecture 808.VVPartII.pdf

V&V II — Static & Dynamic Analysis

  • Static vs dynamic analysis; black-box vs white-box tests.
  • Fuzz / boundary testing surfaces undefined behaviour.
  • Negative tests are as important as positive ones.
AstraLog → cppcheck static-analysis CI stage + boundary/negative fixtures; the pipeline must never crash on malformed input (NFR-05/11). v0.8.1: every rejected MQTT packet is now triaged by classify_corruption() (syntax → schema → type gates) into MalformedJson/SchemaError/ TypeError and logged to corruption_log.json for operator diagnosis (GAP-08-1 partially closed).
docs ✅impl ✅fuzz test ✅coverage CI ✅

CTest v1.0.1 — 9 / 9 (8 fast + 1 characterization)

1/9 smoke_test ✅ 0.12s End-to-end pipeline: CSV in → validated records → anomaly JSON out
2/9 fr_matrix_test ✅ 0.04s FR-02/04/07 assertions on deterministic boundary fixtures
3/9 config_loader_test ✅ 0.09s YAML mission-config loading + all 4 rule-type deserialization
4/9 rule_engine_full_test ✅ 0.18s All 4 rule types (Simple/Stateful/Delta/Compound) against live config
5/9 mqtt_test ✅ 0.31s MockMQTTTransport: connect/disconnect success & failure paths (19 sub-tests)
6/9 metamorphic_test ✅ 0.07s MR1 alarm uniqueness · MR2 sensor independence · MR3 stateful reset
7/9 fuzz_test ✅ 0.89s 1100 random inputs, seed 0xDEADBEEF; invariants I1–I4 never violated
8/9 output_writer_int… ✅ 0.21s PDF §3 format, all 3 CorruptionKind events, empty-output edge cases
9/9 slurm_char_test ★ characterization 2000 G100 SLURM records replayed (job 20768390); golden-file match: 468 anomalies, 1415 nominal (FR-01/04/05/06/08, FR-14)
L8 · V&V II
Lecture 909.VVPartIII.pdf

V&V III — Test Gen & Metamorphic

  • Metamorphic testing + metamorphic relations (MRs) as oracles.
  • Concurrent testing; xUnit-style frameworks.
  • Deterministic logic makes MR oracles tractable.
AstraLog → All 4 rule types implemented + tested. Dedicated metamorphic_test.cpp (v0.9.4, closes GAP-09-1) implements MR1 (alarm uniqueness), MR2 (sensor independence), MR3 (stateful persistence via consecutive-violation counting) using Pipeline({rules}) + hand-built RuleConfig fixtures.
docs ✅impl ✅metamorphic_test.cpp ✅all 4 rule types ✅
src/cpp/tests/metamorphic_test.cpp · MR3 — stateful persistence
// MR3: Stateful rule fires on exactly the N-th
// consecutive violation (consec_violations=3),
// not before — and resets on a nominal sample.
Pipeline p({stateful_rule});  // threshold = 3

assert(!p.process(spike).has_value()); // #1 ✗
assert(!p.process(spike).has_value()); // #2 ✗
assert( p.process(spike).has_value()); // #3 ✓
assert(!p.process(nominal).has_value());// reset
L9 · V&V III
Exercise 44_exercise_lecture_no_sol.pdf

Availability analysis — AstraLog

  • Exercise pattern: series $A=\prod A_i$ · replication $A=1-(1-A)^n$.
  • Identify the weakest links and replicate them.
  • Target: meet NFR-16 availability ≥ 0.98 — requires replicating Ingestion + Classifier.
  • Architectural strategies exist to raise availability beyond paper analysis.
AstraLog → dual-broker MQTT already realises ingestion replication. Checkpoint/restart and active-passive failover are the natural next steps for G100.
solution ✅docs ✅replication on paper

Series → replication

Ingestion (weakest)0.97
Rule engine · Classifier · Output0.99 · 0.98 · 0.99
Series ∏0.9315
k=2 Ingestion + Classifier≈ 0.988 ✓

Architectural strategies to go further

Component replication k=2 on weakest stages — dual-broker MQTT already does this at ingestion; maps to SLURM multi-task (T6)
Checkpoint / restart Write checkpoint.json every N records; SLURM resubmit resumes from last checkpoint — eliminates full-job loss
Active-passive failover Watchdog monitors heartbeat file; secondary pipeline takes over if primary silent > threshold
Full pipeline × voter Two full instances run in parallel; write_merged_summary() acts as voter — total $A≈0.995$
Ex4 · Availability
Lecture 1010.CM.pdf

Configuration Management

  • Decentralized Git / GitHub flow; branch & tag discipline.
  • Semantic versioning vMAJOR.MINOR.PATCH.
  • Commit conventions + a maintained CHANGELOG.
AstraLog → phase1 §3.7 CM policy; tags v0.1.0 → v0.6.0; a CLAUDE.md change-propagation matrix + file-registry.md keep artifacts in sync.
docs ✅impl ✅CI ✅

Release timeline (semantic versioning)

v0.1.0 Phase 1 baseline — requirements, architecture, UML
v0.3.0 CI pipeline + AI skill system + priority field
v0.4.0–v0.5.x Lec 10–13 + Ex05 + Apptainer container + UI dashboard
v0.6.0 All-lecture validation pass + Reveal.js presentation
v0.7.0 All 4 rule types + MQTT adapter + config loader + parallel eval
v0.8.0–v0.8.4 Dual-broker MQTT + real PahoTransport live capture (720 msgs, 229 anomalies)
v0.9.0 First SLURM job on CINECA G100 — NFR-17 mobility confirmed end-to-end
v0.9.1–v0.9.2 GAP-14-1 fix (anomalies=0 root cause) + G100 re-run (job 20768390, 468/1883)
v0.9.3–v0.9.4 kPipelineVersion constant + 6 V&V gaps closed (8/8 CTest, lcov, fuzz, metamorphic, perf, signing)
v1.0.0 Petri net V&V · Engineering Challenges slide · doc polish · all GAPs closed ✅
L10 · CM
Lecture 1111.CI_CD.pdf

CI / CD Pipelines

  • DevOps loop: Plan→Create→Verify→Package→Release→Configure→Monitor.
  • GitHub Actions YAML; pinned action versions.
  • Continuous delivery (not deployment) — G100 has no runner.
AstraLog → ci-baseline.yml builds, lints, tests, runs the batch pipeline, validates JSON, and uploads evidence; slurm-deploy.yml ships the signed .sif.
docs ✅impl ✅tests ✅5 CI stages ✅
.github/workflows/ci-baseline.yml — stages
- name: Configure C++   # cmake -S src/cpp -B build
- name: Build C++       # cmake --build build
- name: Static analysis (cppcheck)
- name: Run C++ tests   # ctest --output-on-failure
- name: Run C++ batch pipeline  # --batch 200 --out results
- name: Validate pipeline_summary.json schema
- name: Upload CI evidence artifacts   # ci-evidence
L11 · CI/CD
Lecture 1212.Schedulers.pdf

Schedulers — SLURM

  • Job · Job-step · Node · Partition · Account / Association.
  • Daemons: slurmctld · slurmd · slurmdbd.
  • Commands: sbatch · srun · squeue; FIFO → priority → backfill.
  • Parallelism: RuleEngine --parallel · run_multi_broker_ingestion · output fan-out.
  • --parallel: std::async per independent rule (Simple + StepDiff, 13/20) — scales with --cpus-per-task.
  • Dual-broker: one std::thread per broker (fixed 2) · output fan-out: 4 concurrent std::async writers.
AstraLog → one Job + one Job-step on Galileo100; the job script follows the lecture's batch-script pattern exactly.
docs ✅impl ✅T5–T7 deferred
scripts/slurm/astralog_g100.sbatch
#SBATCH --job-name=astralog
#SBATCH --account=<budget>      # G100 association
#SBATCH --partition=g100_usr_prod
#SBATCH --nodes=1 --ntasks-per-node=1
#SBATCH --cpus-per-task=4 --mem=8G --time=00:30:00

module load apptainer
srun apptainer exec --bind $HOME/astralog:/data \
     astralog.sif astralog_demo --batch 200 --out /data/results
L12 · Schedulers
Lecture 1313.Containers.pdf

Containers — Docker & Apptainer

  • Multiplicity problem; containers vs VMs (host kernel, MB, instant).
  • Docker for local · Apptainer/Singularity for HPC.
  • SIF immutability + signature; integration over isolation; same UID/GID.
AstraLog → one astralog.def builds the image from source; the same signed .sif runs in CI and on G100 (NFR-16/17).
docs ✅impl ✅signing CI ✅ (GAP-13-1)
singularity/astralog.def
Bootstrap: docker
From: ubuntu:22.04

%post
    apt-get install -y --no-install-recommends cmake g++ make
    cmake -S /opt/astralog/src/cpp -B /opt/astralog/build
    cmake --build /opt/astralog/build -j 2

%runscript
    exec /opt/astralog/build/astralog_demo "$@"
L13 · Containers
Exercise 55_exercise_lecture_updated_no_sol.pdf

Integration testing — AstraLog

  • Exercise pattern: choose + justify a strategy and a concrete plan.
  • Choice: thread-based incremental along the dataflow chain.
  • No stubs/drivers — deterministic fixtures suffice.
Implemented → T1/T2/T4 covered by fr_matrix_test.cpp + smoke_test.cpp; T3 now has a dedicated output_writer_integration_test.cpp (v0.9.4, closes GAP-Ex5-1) with 5 named test cases covering the PDF §3 format spec, all 3 CorruptionKind events, and empty-output edge cases; T5 ran on G100 (job 20768390); T6/T7 remain stretch goals.
solution ✅T3 done ✅T5–T7 G100

Integration threads (T1 → T7)

ThreadBoundaryStatus
T1Ingestion + Rule✅ done
T2Rule + Classifier✅ done
T3Classifier + Output✅ done (v0.9.4)
T4End-to-end pipeline✅ done
T-MQTTMQTT ingestion → pipeline✅ done
T5SLURM + config-driven pipeline✅ done (v1.0.1)
T6–T7SLURM multi-task/aggregate⚠️ stretch
CTest v1.0.1 — 9/9 wired
smoke · fr_matrix · config_loader · rule_engine_full
mqtt · metamorphic · fuzz (1100 iter) · output_writer_integration
+ slurm_characterization_test (G100 job 20768390 golden file)
validated=1883 anomalies=468 · rejected=117 · [SKIP] in CI
Ex5 · Integration
Engineering Challenges

Key Difficulties & How We Tackled Them

  1. C++ vs Python — preferred Python but chose C++ for deterministic HPC performance, native POSIX threading, and Apptainer portability. Python retained for analysis notebook only.
  2. AI skill architecture — structured a layered system: global SE4HPC course skill + project-local domain skills (docs/skills/) loaded per context (SLURM, MQTT, V&V) to keep sessions focused.
  3. Lecture ↔ requirement traceability — spiral review: every FR/NFR traced to a lecture slide; every lecture deliverable mapped to a concrete artifact via docs/se4hpc-checklist.md + GAP backlog.
  4. Doc / code sync across 14 releases — change-propagation matrix in CLAUDE.md; mandatory pre-commit bookkeeping checklist (07-log, file-registry, CHANGELOG, version cells) enforced before every commit gate.
  5. Static dashboard without HTTP server — C++ adapter ran once against live HiveMQ (~60 s, 720 msgs, 229 anomalies); frozen JSON published to gh-pages and replayed in a loop. Production path: .sif on server + nginx for live feed.
  6. Three-environment DevOps — local MK container → GitHub Actions → CINECA G100. Four scripts/slurm/01–04 automate the fetch/stage/submit/retrieve cycle. Same CI-built .sif transferred without on-cluster rebuild (NFR-17 mobility-of-compute).

Engineering decisions summary

ChallengeResolution
Language choiceC++ pipeline / Python analysis
AI context managementLayered skill architecture
Lecture coverageBidirectional traceability + GAP table
Version driftPropagation matrix + commit gate
Static demoOne-time live capture → frozen replay
Multi-env DevOps4-script automation + NFR-17 .sif
AstraLog-HPC · Engineering Challenges
Full Track · Single Student · All gaps closed

13 lectures + 5 exercises → docs · code · tests · CI · G100 · UI

Every lecture is traced through Phase 1/2 requirements & design, the C++ implementation, the test suite, the CI/CD pipeline, the dashboard, and this deck — recorded in docs/se4hpc-checklist.md. 9/9 CTest tests wired (8 fast pass always; 9th characterization test replays 2000 G100 records with golden file, [SKIP] in CI). G100 evidence: job 20768390 · samples=2000 · anomalies=468. Petri net verified concurrency. Two-tier Apptainer signing in CI.

v1.0.1 Live deck + dashboard + G100 evidence 9/9 CTest · slurm_characterization_test ✓ Mission close: 2026-06-14

Presentation → vahidgh.github.io/ghayoomie-se4hpc/  ·  Dashboard → /ui/  ·  Checklist → docs/se4hpc-checklist.md

AstraLog-HPC v1.0.1 · thank you