Method

This describes what the decision layer does, at the level needed to interpret the results, without publishing how it does it. The simulation engine and the decision layer's implementation are not published — see README.md.

The seven-stage pipeline

Every recommendation the decision layer produces passes through the same seven named stages:

OBSERVE -> DETECT -> CORRELATE -> DIAGNOSE -> ESTIMATE DECISION RISK
        -> WAIT / ESCALATE / RECOMMEND ACTION -> VERIFY

• OBSERVE — feature extraction from telemetry and peer-group context.
• DETECT — is the element outside its own statistical control limits, sustained (not a single noisy sample)?
• CORRELATE — does the anomaly co-occur with peer elements in the same correlation group, or is it isolated to one element?
• DIAGNOSE — evidence-based differential diagnosis across candidate fault classes (see "Hypothesis cards" below).
• ESTIMATE DECISION RISK — what does acting cost if the diagnosis is wrong, given the blast radius of the affected element?
• WAIT / ESCALATE / RECOMMEND ACTION — the decision gate: a list of named checks, each with a pass/fail and a human-readable reason.
• VERIFY — did the recommendation hold up against what actually happened?

The decision layer is read-only by construction: RECOMMEND_ACTION is a recommendation carried to a human with its own audit trail, never an actuation. There is no code path that writes to a network.

Control limits

Per-element control limits are Shewhart 3-sigma limits computed from a robust baseline: the median and the median absolute deviation (MAD) of the element's own history over a warm-up window, which is then frozen — not a rolling window recomputed as new samples arrive — rather than a fixed vendor threshold or a population mean. Median/MAD is used specifically because it is not distorted by the fault itself once a fault is already in progress, which a mean-based baseline is; freezing the baseline after warm-up is what makes that property hold for the whole run, not just for the first sample of a fault.

Two LEO-specific method choices

Two aspects of the method differ between domains and are worth stating explicitly, since they are the reason LEO/NTN is not simply "microwave with a longer path" (see H5 in RESEARCH_HYPOTHESES.md):

• Geometry correction. For LEO, the geometry-predicted component of received level — the part explained by elevation angle changing continuously over a pass — is removed before anything is treated as anomalous. A fixed threshold, or a control limit computed without this correction, alarms on healthy orbital geometry; this is what lets the layer avoid escalating on every pass the way a naive comparator does.
• Context-conditioned baselines. Statistical baselines are conditioned on the serving context — for LEO, the serving satellite — because a terminal's link statistics are only comparable against observations taken under the same spacecraft. Without this, a spacecraft-side fault is smeared across the pass schedule and becomes statistically invisible.

Confidence is ordinal, not probabilistic

The decision layer reports confidence as one of a small number of ordered tiers (e.g. LOW / MEDIUM / HIGH). This is deliberate: the evidence a telemetry stream can supply supports an ordering of certainty, not a calibrated probability. A confidence tier must never be read or reported as a probability of correctness.

Hypothesis cards

Each candidate fault class is represented as a condition card with three kinds of condition evaluated against the extracted features:

• required conditions — must all hold for the hypothesis to be considered at all; together they define the fault's physical signature.
• supporting conditions — corroborate the hypothesis and raise confidence, but never create a verdict by themselves.
• refuting conditions — take the hypothesis off the table entirely.

When two hypotheses reach the same top confidence tier, the result is reported as AMBIGUOUS rather than resolved by an arbitrary tie-break. Ambiguity is treated as a measured outcome, not a failure mode — the count of ambiguous decisions (n_ambiguous_decisions) is one of the reported metrics.

The specific conditions, their predicates and their thresholds are part of the unpublished simulation engine.

Both arms see identical telemetry

Every run presents the same synthetic telemetry stream to two independent arms — a modelled conventional vendor/NMS monitor and the Velorona decision layer — so that any difference in outcome is attributable to the arm, not to a difference in what each arm was shown. See EXPERIMENT_MATRIX.md for the full experiment design.

Metrics

Metric definitions (timeline windows, detection/escalation confusion-matrix metrics, decision-quality counts, and the illustrative economics model) are defined in EXPERIMENT_MATRIX.md, which is adapted from the internal research documentation (see PROVENANCE.md). Every value referenced there is computed from the run; none is hard-coded.

Research hypotheses

Each hypothesis states the measurement, the prediction, and the observation that would falsify it. A hypothesis with no falsification condition is a marketing claim.

H1 (ORIGINAL — NOT SUPPORTED) — A pre-escalation window exists and is measurable

Claim. For degradation with a finite ramp, there is a non-zero interval between the first sample that leaves the element's own statistical control limits and the moment conventional threshold monitoring escalates.

Measurement. pre_escalation_window_min = t_conventional_escalation − t_decision_layer_actionable, per run, reported as median and IQR over seeds.

Mechanism (not assumed — arithmetic). A rolling mean of window W lags a linear ramp by ≈ W/2 samples. An M-of-N persistence rule adds ≈ M more. A process delay adds its own. These are properties of the comparator, modelled explicitly in the conventional-monitoring baseline.

Falsified if. The median window is ≤ 0, or its IQR straddles 0, for the majority of scenarios.

Status (30 seeds, default comparator). Not falsified in isolation: median 7–20 min for scenarios where both arms escalate; 189–250 min for LEO space-segment faults, where the conventional arm frequently never escalates at all. But this falsification condition never anticipated being tested against a comparator range, only against one chosen comparator — and against that range, it fails.

Status, tested against the comparator range (Phase 10/11, 15 seeds per config). NOT SUPPORTED AS STATED. A one-factor-at-a-time sweep of the comparator's configuration moves the pooled median window only modestly on its own (smoothing_window 1→10: 10→15 min; escalation_delay_min 0→15: 3→18 min; persistence 1/1→4/5: 11→12 min) — but the DEGENERATE configuration (smoothing_window=1, persistence 1-of-1, escalation_delay_min=0 simultaneously: a perfect, instantaneous, unsmoothed threshold monitor, better than any real NMS) collapses the pooled median window from 13 min to -146 min (window paired with its error rates, as this study now requires everywhere: conventional escalation FPR rises 0.00→1.00 median [0.22→0.61 mean], conventional unnecessary-dispatch rises 67%→100%; the decision layer's own numbers do not move at all — FPR ~0.00→~0.00, unnecessary dispatch 4.4%→4.4%, F1 0.953→0.953). Per domain the degenerate window is near zero for microwave (-2 to +8 min across all five scenarios) and inverted for every LEO scenario (-155 to -225 min).

Part of that LEO deficit is a cold-start artefact, not measured, so this is stated with the number attached rather than assumed away (P11.3, three ways to report the same degenerate-comparator window, pooled median): with the 120-minute feature warm-up included as run, W_a = -146 min; with the comparison clock started at t=warmup_min for both arms (warm-up excluded), W_b = -34 min; over a 4x longer, 2400-minute run (warm-up 5% of the run instead of 20%), W_c = -154 min. W_b differs from W_a by 77%, far outside the 25% band fixed in advance for calling this a material term; W_c stays within 5%. So: cold-start bias is a material term, and it does not reverse the finding. By domain, the entire effect is in LEO (pooled window -191 min as-run vs. -71 min with warm-up excluded — still deeply negative) and none of it is in microwave (+2 min in all three variants, unaffected by warm-up). Even crediting the layer with the full 120 minutes it could not possibly have acted in, LEO's window under a perfect comparator remains substantially negative.

The original H1 does not survive its own sensitivity analysis: its falsification condition ("median window ≤ 0 for the majority of scenarios") is met once tested against a comparator aggressive enough to be worth publishing against, and even after removing the warm-up period's contribution, the LEO domain's window under an idealised monitor stays negative. The raw lead-time claim, as originally stated, is not supported. See H1 (REVISED) below for the claim the evidence does support, and H3 for the result that survives untouched. See docs/LIMITATIONS.md for the comparator-sensitivity and cold-start items.

H1 (REVISED) — Detection latency and diagnostic precision trade off against each other for threshold monitoring, and the decision layer is not on that trade-off curve

Claim. A conventional threshold monitor's escalation latency can be reduced arbitrarily by reducing smoothing, persistence and process delay, but its false-positive rate and unnecessary-dispatch rate rise as it does. The decision layer's error rates are invariant to that tuning.

Measurement. The comparator sweep (data/sensitivity_comparator.csv): pre-escalation window, escalation FPR, escalation F1 and unnecessary-dispatch rate for both arms across the comparator grid (figures/fig10_latency_precision_tradeoff.png).

Falsified if. A comparator configuration exists that matches the decision layer's false-positive and unnecessary-dispatch rates while escalating at least as early — i.e. a point on the conventional curve at or below the layer's escalation FPR and at or above the layer's window.

Status. Not falsified. Across all 11 swept comparator configurations, the conventional arm's escalation FPR ranges from 0.00 (default-like, low-aggressiveness configurations) up to 1.00 (degenerate), and its window moves accordingly, from modest positive values down to -146 min pooled. The decision layer's escalation FPR does not move (~0.00 across every configuration, since the comparator's configuration has no effect on it), and neither does its unnecessary-dispatch rate (4.4%) or its escalation F1 (0.953). No configuration on the conventional curve reaches the layer's combination of near-zero FPR and a non-negative window simultaneously: the only configurations with FPR near 0.00 are the least aggressive ones, which have smaller (but still positive, for microwave and most scenarios) windows than the layer offers under the default comparator, and the only configuration with a window competitive with or exceeding the layer's absolute detection speed (the degenerate case) buys it with FPR 1.00 and 100% unnecessary dispatch. The trade-off is real and the layer sits off it, not on it.

This is the durable finding from Phase 10/11, alongside H3. It is a stronger and more defensible claim than the original H1: it does not depend on which comparator you assume, only on the shape of conventional monitoring's own trade-off curve and on the layer not being subject to it.

H2 — Environmental and persistent causes are separable before the event ends

Claim. A propagation impairment is symmetric across both directions and correlates with an independent environmental feed; a chain fault is directional and does not. The two are separable while both are present.

Measurement. Verdict correctness against ground truth on the *_hidden_in_* scenarios, in which a directional fault begins inside an active fade.

Falsified if. The layer cannot distinguish the composite case above chance, or only distinguishes it after the environmental event has cleared (which would make it a trivial "wait and see" rule, not a diagnostic one).

Status. Not falsified for microwave (mw_hidden_in_storm, layer escalation F1 median 0.95). LEO composite case leo_hidden_in_rain reaches F1 0.89 against a conventional 0.39.

H3 — The correct decision is sometimes to do nothing, and that is measurable

Claim. For self-clearing causes, escalation is a false positive with a real cost. An early-warning layer that cannot say WAIT is not an improvement.

Measurement. unnecessary_dispatch — did the arm escalate on a scenario whose ground-truth fault class is self-clearing?

Falsified if. The decision layer escalates on mw_rain_cell, leo_ground_rain or leo_low_elevation at a rate comparable to the conventional arm.

Status. Not falsified, with a measured residual. Across the 90 self-clearing runs (three scenarios x 30 seeds) the conventional arm escalated unnecessarily in 60 (67%) — 30/30 on both rain scenarios — and the decision layer in 3 (3%): two seeds of the microwave rain scenario and one seed of the LEO low-elevation scenario. The residual is real and is reported rather than suppressed; the layer is not claimed to be perfect on this failure mode, only roughly an order of magnitude better than the comparator. This remains the single most operationally relevant result in the study, and it is not a lead-time result.

H4 — The decision problem generalises across domains; the physics does not

Claim. The same pipeline, the same feature vocabulary and the same confidence scheme apply to microwave and LEO/NTN. Only the hypothesis cards and the physics differ.

Measurement. A single shared feature extractor, a single shared decision layer and a single metric set are applied to both domains; only the physics models and the hypothesis catalogue differ. Cross-domain distribution comparison of window, time-to-diagnosis and ambiguity count.

Falsified if. Either domain requires a bespoke decision pipeline, or the shared abstraction has to be broken to make one domain work.

Status. Not falsified — but see the important caveat in H5.

H5 — LEO adds discriminators that microwave does not have, and constraints it does not have

Claim. LEO is not microwave with a longer path.

1. Geometry is a legitimate cause of degradation. Elevation changes continuously; a fixed threshold alarms on healthy physics. Conventional monitoring generates false escalations every pass. The decision layer removes the geometry-predicted component before calling anything anomalous.
2. Two independent correlation groups (serving satellite, serving gateway) versus microwave's one (adjacent links). This is a strictly richer space-versus-ground discriminator.
3. A hard time constraint. "Can this wait for the next pass?" is a first-class question with no microwave equivalent, surfaced as a named gate check on remaining pass time.
4. Statistics must be conditioned on the serving satellite. Without it, a spacecraft fault is smeared across the pass schedule and becomes statistically invisible. This is implemented by conditioning each element's statistical baseline on its serving satellite, and was discovered by the simulation failing without it.

Falsified if. Removing geometry correction, context conditioning or the second correlation group does not degrade LEO diagnostic performance.

Status. Points 1, 2 and 4 are supported: removing context conditioning drops leo_rf_subsystem escalation F1 from 0.94 to ≈0.05.

H6 — Earlier knowledge has economic value, but not unboundedly, and sometimes negative value

Claim. Lead time converts to avoided exposure only through an actionable fraction and an avoidance probability, both < 1, and it saturates. Acting on a self-clearing cause has strictly negative value.

Measurement. Parametric economics model, reported as ranges over the sensitivity bounds and over seeds.

Falsified if. No plausible assumption set yields a positive delta, or the delta is insensitive to the assumptions (which would mean the model is not actually parametric).

Status. Deliberately UNRESOLVED. The current figures are labelled ILLUSTRATIVE and must not be published as a value claim until at least one operator supplies real cost inputs. This is the largest open gap in the study.

Experiment matrix

Design

A controlled comparative experiment. One run = (domain, scenario, seed). Within a run, the same telemetry stream is presented to two arms:

Both arms see identical data. Differences are attributable to the arm, not to the scenario.

Scenarios

Microwave

LEO / NTN

Seeds

Default sweep: 30 seeds per (domain, scenario) = 330 runs. Every stochastic component draws from a named stream derived from the master seed (named, seed-derived random streams), so changing the number of draws in one component does not perturb another. Runs are bit-reproducible; a dedicated reproducibility test enforces it.

Metrics

Timeline windows (minutes)

• false_calm_window = conventional escalation − first detectable
• pre_escalation_window = conventional escalation − decision-layer actionable
• exposure_window = service impact − first detectable
• decision_margin = service impact − decision-layer actionable
• conventional_margin = service impact − conventional escalation
• time_to_correct_diagnosis = first correct verdict − first detectable

Detection and escalation quality

Per-sample confusion matrices with three separate ground truths: 1. detect_* — predicted = sustained control-limit breach; truth = any fault active 2. layer_esc_* — predicted = layer escalated; truth = a persistent fault active 3. conv_esc_* — predicted = conventional escalated; same truth

Ground truth (2) and (3) deliberately treat escalating on a self-clearing cause as a false positive, because operationally it is one.

Decision quality

• n_ambiguous_decisions — competing hypotheses at the same confidence tier
• n_wait_decisions
• layer_unnecessary_dispatch / conv_unnecessary_dispatch

Economics (ILLUSTRATIVE)

• exposure_conventional, exposure_decision_layer, exposure_delta
• reported with the sensitivity bounds attached

Figure mapping

Reference results (30 seeds, duration=600 min, DEFAULT comparator only)

These window figures assume one particular comparator configuration (the default). They are not the headline result of this study. A comparator sensitivity sweep (Phase 10/11) shows the window is not robust to how the conventional arm is tuned — see H1 (REVISED) in RESEARCH_HYPOTHESES.md and figures/fig10_latency_precision_tradeoff.png for the trade-off this table does not show. The durable results are H3 (escalation quality, unaffected by comparator tuning) and H1 (REVISED) (the trade-off itself), not the raw numbers below.

Median pre-escalation window, and escalation F1 for each arm, at the default comparator:

¹ F1 is undefined when the ground truth contains no persistent fault — by construction there is nothing to escalate about. The meaningful column for those rows is the dispatch column. ² Neither arm escalated in any seed. Reported as a null result, not omitted. ³ Conventional escalated in only 6 of 30 seeds, so this window is conditioned on a small subsample and must be reported with n, not as a headline number. ⁴ None of these window figures should be read without the comparator sensitivity caveat above: under a more aggressive (but still plausible) comparator, every one of these numbers moves, and under a degenerate one the pooled median window inverts to -146 min. See data/sensitivity_comparator.csv.

Reproduce the figures from the published data: python replot.py in this repository. Reproducing the underlying runs themselves requires the private simulation engine, which is not published here.

Limitations

Stated first, not buried. Every item here is a question a reviewer or a VP of network operations will ask.

Fundamental

1. The networks are simulated. No result here is evidence about a real operator's network. The study establishes that a decision architecture is coherent and testable, not that it works in production. Only an operator pilot on real telemetry can do that.
2. Ground truth is known by construction. Real degradation has no label. Every reported precision/recall figure is an upper bound on what is achievable with real, ambiguous, partially-labelled operational data.
3. Single-fault scenarios. Except for the two hidden_in_* composites, each run contains one fault. Real incidents overlap far more.

Physics

4. ITU-R P.838-3 coefficients are transcribed, not verified. They must be re-checked against the official publication before appearing in a paper. (rain-attenuation model)
5. Design-rule vs instantaneous attenuation are different quantities. The P.530-18 effective path length is a long-term statistical design rule. Using the same reduction on an instantaneous wet path is a modelling simplification, labelled in the source.
6. LEO slant-path rain omits ITU-R P.618 reduction factors and therefore over-estimates fade at low elevation.
7. Orbits are circular, Earth is spherical, no J2, no Earth rotation in pass timing. Adequate for elevation-driven link behaviour over one pass; inadequate for any availability claim.
8. The ACM ladder is representative, not from a vendor datasheet. Absolute capacity figures must be re-derived from a cited profile table.
9. Satellite EIRP, G/T, beam count and beam footprint are parameters, not measurements. Any published number must state them.
10. Constellation size is a geometric lower bound using a street-of-coverage packing efficiency. It is not a constellation design.

Method

11. Scenario onset times are chosen, not sampled; severity is now swept, not chosen. Phase 10 sweeps severity across a multiplier grid per scenario (see H1's Status in RESEARCH_HYPOTHESES.md), so the pre-escalation window is now reported as a curve, not a single chosen point. Onset time, ramp duration and hold duration are still fixed per scenario, not sampled; that remains open.
12. The conventional baseline is a model of vendor monitoring, and its sensitivity is now measured, not assumed — and the headline window does NOT survive it. It is parameterised to be fair (real smoothing windows, real M-of-N rules, real process delays) but it is not any specific vendor's product. Phase 10/11 sweep the smoothing window, the persistence rule and the escalation delay one-factor-at-a-time, plus a degenerate case (a perfect, instantaneous, unsmoothed threshold monitor). The window is meaningless without its paired error rates (a rule enforced in the private engine's code): under the degenerate comparator the pooled median window falls from 13 to -146 min while conventional's escalation FPR rises from 0.00 to 1.00 (median) and its unnecessary-dispatch rate from 67% to 100% — it buys the apparent lead with false alarms, not earlier detection. The decision layer's own error rates do not move. See H1 (REVISED) in RESEARCH_HYPOTHESES.md for the full finding and H1 (ORIGINAL) for why the raw lead-time claim is not supported as originally stated.
13. Part of the degenerate-comparator deficit is a cold-start artefact, not all of it. The decision layer needs a 120-minute warm-up to learn a per-element statistical baseline; a fixed threshold does not, so a 600-minute run with warm-up included in the comparison is biased against the layer. Measured, not assumed (P11.3): the degenerate-comparator pooled median window is -146 min as run (warm-up included), -34 min with the comparison clock started at t=warm-up for both arms, and -154 min over a 4x longer (2400-minute) run. Excluding warm-up moves the number by 77%, over the 25% threshold fixed in advance for calling this material — so cold-start bias IS a material term. By domain it is entirely a LEO effect (pooled window -191 min as-run vs. -71 min warm-up-excluded) and absent from microwave (+2 min in all three variants). Even crediting the layer with the full warm-up period, LEO's window under the degenerate comparator remains substantially negative: cold start explains part of the deficit, not all of it.
14. leo_pointing conventional escalation occurred in only 6/30 seeds. Its window statistic is conditioned on a small subsample and must always be reported with n.
15. No learned model is included. The study deliberately establishes the interpretable baseline first. Any future learned component must be compared against this baseline, not against nothing.
16. Confidence is ordinal, not calibrated. LOW/MEDIUM/HIGH is honest about what the evidence supports. It is not a probability and must not be reported as one.

17. The decision layer has a small, diagnosed false-positive residual. Across the 90 self-clearing runs (three scenarios x 30 seeds), the layer escalated unnecessarily in 3 (3%), versus 60 (67%) for the conventional arm — not zero (see H3's Status). All three are diagnosed by re-running the seed with full feature and hypothesis capture:

    • mw_rain_cell seeds 12 and 21 fired the HARDWARE_DIRECTIONAL hypothesis: per-direction asymmetry (3.72 dB and 3.75 dB respectively) crossed both the element's own control limit and the 3.0 dB nominal directional-imbalance floor for 5 consecutive samples, during an active, symmetric rain event. ATPC's independent per-direction compensation noise occasionally produces a directional signature by chance; the asymmetry barely clears the floor in both cases (0.7-0.75 dB above it).
    • leo_low_elevation seed 1 fired the GATEWAY_SIDE hypothesis at HIGH confidence: co-located terminals sharing a gateway happened to co-degrade from orbital geometry alone (all terminals on a gateway see similar elevation over a pass), which the gateway-correlation condition cannot distinguish from a real feeder-side fault without an independent gateway-health signal.

    The residual was diagnosed, not tuned away: tuning against three known seeds is exactly the overfitting this rewrite exists to avoid.

Economics

18. Every currency figure is ILLUSTRATIVE. The model is parametric and its provenance string is carried into the results file. Until an operator supplies real cost inputs, no monetary claim may be published. This is the single largest gap in the study.
19. Actionable-fraction and avoidance-probability are assumptions, bounded but not measured.

Scope

20. No multipath/ducting, no wet-radome effect, no inter-satellite links, no beam-shape roll-off, no traffic-dependent congestion in the LEO domain.
21. Blast radius is element-local. The microwave model computes downstream isolation; the LEO model does not model beam-sharing contention.

Provenance of the copied documents

docs/RESEARCH_HYPOTHESES.md, docs/EXPERIMENT_MATRIX.md and docs/LIMITATIONS.md are adapted from AID Edge Inc.'s internal research repository (velorona-research-v2, private).

The only changes made are: 1. removal of references to internal source files, modules, classes and commands that are not published in this repository — each replaced with a description of the thing it named rather than the path to it, or with a published-repository-relative reference where an equivalent published file exists (e.g. a private results/...csv path becomes the corresponding data/...csv path actually published here); 2. one cross-reference to docs/RESEARCH_SIMULATION_V2_PLAN.md removed outright, since that document is internal (architecture/audit) and is not published here at all — there is no published equivalent to point to; 3. framing additions in docs/EXPERIMENT_MATRIX.md's reference-results table (a caveat paragraph and a fourth footnote) demoting it from a headline presentation to a comparator-scoped one, cross-referencing H1 (REVISED) and figures/fig10_latency_precision_tradeoff.png — added, not substituted for existing text, because the original table's numbers are accurate only under one comparator configuration and the original document did not say so.

No claim, caveat, number or argument already present in the internal documents was altered, added to, or removed on its own terms — case 3 above adds a signpost to evidence that exists elsewhere in this same repository (H1 (REVISED), figures/fig10_latency_precision_tradeoff.png, data/sensitivity_comparator.csv), it does not change or dispute the table's own numbers.

This file was regenerated in full for the Phase 10/11 update: the previous version's before/after list is superseded by the complete list below, since the three documents were refreshed from Repo A in their entirety rather than patched incrementally.

The internal repository is private and its simulation engine is not published. See the "What is published here / what is not" section of README.md and METHOD.md for what that boundary covers.

Every rewritten phrase

docs/RESEARCH_HYPOTHESES.md

docs/EXPERIMENT_MATRIX.md

docs/LIMITATIONS.md

The ### Microwave (Domain.MICROWAVE) / ### LEO / NTN (Domain.LEO_NTN) headings and the **Reproduce:** line (both in EXPERIMENT_MATRIX.md) were found during the original sweep of that document, in addition to the items identified beforehand; the same rule was applied to them — name the thing, not the file, module, class or command that only exists in the private repository. ASYMMETRY_FLOOR_DB (in LIMITATIONS.md) is a named constant from the private decision-rule engine and was removed on the same basis.