Off nominal review

Off-Nominal Review

Mission

Review a system the way a mission or operations team would: not only for the happy path, but for off-nominal conditions, unexpected environment changes, unsafe transitions, missing safeguards, and weak containment boundaries.

This is a public, software-oriented adaptation of NASA systems thinking. It is not a formal hazard analysis or safety certification method.

Source basis

This skill is shaped by the NASA Systems Engineering Handbook, Rev 2 (NASA/SP-2016-6105 Rev2). Read references/nasa-principles.md before using the skill for a deeper review.

When to use

Use this skill when the system has material workflows, lifecycle transitions, or operator-facing recovery decisions, including:

• daemons and workers
• controllers and state machines
• import, export, or migration tools
• orchestration and deployment workflows
• pipelines with partial outputs or retries
• systems that claim degraded operation or fault tolerance

Review posture

Model the target as a set of operational scenarios. For each material workflow:

1. Define the nominal path.
2. Enumerate off-nominal conditions.
3. Identify the safeguard, containment, or operator signal that should exist.
4. Trace whether that mechanism is actually implemented.
5. Record both findings and non-findings.

Off-nominal scenario inventory

Check at least these conditions where relevant:

• required dependency absent
• configuration inconsistent with environment
• operator chooses the wrong target or mode
• timeout, empty response, or malformed response
• delayed but not failed dependency
• partial write or interrupted state transition
• duplicate message, job, or rerun
• stale cache, checkpoint, or lock
• cleanup path not reached
• degraded mode entered without clear signal

Scenario worksheet

Build a worksheet before findings:

Workflow	Nominal goal	Off-nominal trigger	Required safeguard	Current handling	Residual risk
batch import	import one file once	duplicate rerun	idempotent checkpoint	file may replay	duplicate records

The required safeguard can be technical, procedural, or operator-visible, but it must be explicit.

Review angles

Inspect each workflow from these angles:

• transition into work
• transition out of work
• partial progress boundary
• operator decision point
• dependency instability
• containment of bad state
• recoverability after interruption
• visibility to the next operator

Guard questions

Ask questions like:

• What is the first irreversible side effect?
• What must be true before that point?
• What happens if that assumption fails one step later instead?
• What evidence would tell the operator the workflow is now off-nominal?
• Can the system contain the problem locally, or does it leak ambiguity downstream?
• What state must be preserved, rolled back, quarantined, or recomputed?

Expected safeguard vocabulary

Use concrete safeguard terms when appropriate:

• interlock
• preflight
• hold point
• explicit degrade
• quarantine
• containment boundary
• rollback point
• operator acknowledgment
• restart barrier

Finding format

Use this format:

ONR-XXX: Short title

• Severity: Low / Medium / High / Critical
• Workflow: affected workflow or transition
• Off-nominal trigger: unexpected condition
• Required safeguard: the control or signal that should exist
• Current handling: what exists today
• Evidence: code path, config path, procedure, or observed behavior
• Impact: operational or integrity consequence
• Recommended change: the narrowest change that restores the safeguard
• Tests: scenario or regression test that should prove the safeguard

Non-findings

Record explicit non-findings when the system appears to:

• reject unsafe entry conditions
• contain bad state before propagation
• surface degraded mode honestly
• preserve clear ownership of operator decisions
• avoid ambiguous partial completion

Output contract

Produce:

1. a short posture summary
2. the scenario worksheet
3. findings
4. explicit non-findings
5. a prioritized safeguard backlog
6. tests or drills that would strengthen confidence
7. limits of analysis

Limits

• Do not present this as formal FMEA, STPA, or certification evidence.
• Do not overclaim runtime behavior without concrete evidence.
• Keep the review anchored to implemented safeguards, not abstract safety language.