ISO's nonconformity clause and NIST's monitoring guidance turn a failing check into a response someone has to record, while a check that could not run has shown nothing and belongs in neither column.
Continuous monitoring, by NIST's own account, happens at intervals. SP 800-137, the guide to information security continuous monitoring, defines the practice as "maintaining ongoing awareness of information security, vulnerabilities, and threats to support organizational risk management decisions". Footnote 2 then explains what "continuous" and "ongoing" mean: controls and risks are "assessed and analyzed at a frequency sufficient to support risk-based security decisions". It ends with this sentence:
"Data collection, no matter how frequent, is performed at discrete intervals."
An automated check that runs every night is a series of dated observations. So is a quarterly access review. Both are monitoring. The auditor's question for either is the same: who picked the interval, and on what grounds?
The ISO side asks the same question. ISO/IEC 27001 is one of the management-system standards built on the Harmonized Structure, which ISO's Directives describe as "identical clause numbers, clause titles, text and common terms and core definitions". Its clause 9.1 reads:
"The organization shall determine:- what needs to be monitored and measured;- the methods for monitoring, measurement, analysis and evaluation, as applicable, to ensure valid results;- when the monitoring and measuring shall be performed;- when the results from monitoring and measurement shall be analysed and evaluated.Documented information shall be available as evidence of the results."
ISO/IEC 27001 adds its own wording on top of this text. The standard is paywalled, so I quote only the common core. The core is enough for this argument. Frequency is a decision the organization makes and owns. No clause hands you a number.
NIST's control catalog says the same thing in its own format. SP 800-53 Rev. 5, control CA-7, requires "[Assignment: organization-defined frequencies] for monitoring and [Assignment: organization-defined frequencies] for assessment of control effectiveness". The discussion adds: "Different types of controls may require different monitoring frequencies." SP 800-137 §2.2 goes further. Frequencies "are not static, and they are not uniform across all metrics."
In practice, write the interval down next to each check, along with the reason for it. "Daily, because one admin can switch branch protection off in a few seconds" is a reason an auditor can test. A bare "we monitor continuously" gives them nothing to test.
The Harmonized Structure defines monitoring in clause 3.20 as "determining the status of a system, a process (3.8) or an activity". A passing run tells you the status at the moment it ran, and nothing more.
That makes a single pass weak evidence and a series of passes strong evidence. Take a check that passed yesterday, with no record of the runs before it. It shows that the setting was on yesterday. Take the same check with a year of dated runs on the interval you chose, each one kept. That series shows the control operating across the whole period. 9.1 closes on this point: "Documented information shall be available as evidence of the results." Keep the results, including the passes, because the passes make up the series.
A failing check has found a nonconformity, which the Harmonized Structure defines in clause 3.16 as "non-fulfilment of a requirement". Clause 10.2 then says what follows. When a nonconformity occurs, the organization shall:
"a) react to the nonconformity, and as applicable:- take action to control and correct it;- deal with the consequences;b) evaluate the need for action to eliminate the cause(s) of the nonconformity, in order that it does not recur or occur elsewhere, by:- reviewing the nonconformity;- determining the causes of the nonconformity;- determining if similar nonconformities exist, or can potentially occur;c) implement any action needed;d) review the effectiveness of any corrective action taken;e) make changes to the XXX management system, if necessary."
It ends with the record:
"Documented information shall be available as evidence of:- the nature of the nonconformities and any subsequent actions taken;- the results of any corrective action."
Read against a red check, this list is a worklist. Someone contains the problem; someone asks why the setting drifted, and whether the same drift exists in other repositories or accounts; someone fixes it and later confirms the fix held. All of it gets written down.
NIST spells out a response that 10.2 does not name. SP 800-137 §3.5 says: "Response to findings at all tiers may include risk mitigation, risk acceptance, risk avoidance/rejection, or risk sharing/transfer, in accordance with organizational risk tolerance." A fail does not have to end in a fix. It can end in a decision to accept the risk: the legacy repository that cannot take branch protection, or the service account that has to skip MFA. That decision is legitimate. It still has to be made by someone with the authority to make it, and it has to be recorded with its reason. CA-7(f) asks for "response actions to address results of the analysis". Whichever response you choose, it has to exist on paper.
A check can do the detecting and nothing after it. SP 800-137 §2.3 says plainly that "it is not possible to fully automate all of an organization's information security program functions", and that tools still require human "interpretation of findings". The exception decision is one of the functions that stays with a person. Any workflow that turns a failing check green without an owner and a reason has deleted exactly the record 10.2 asks for.
Scheduled checks have a third outcome. The check could not reach the vendor's API, the credential was revoked, or the endpoint timed out. The run finished, but it showed nothing about the control.
Counting that run as a pass hides a gap: for however long the errors last, you have no observation at all. Counting it as a fail invents a nonconformity. Someone opens a 10.2 record for a setting that may be perfectly fine, and the corrective-action history fills with noise that an auditor will ask about. Neither is honest. An error is a failure of the monitoring, and it needs its own owner and its own response.
NIST already treats the failure of a monitoring mechanism as its own event, in a neighboring context. SP 800-53 control AU-5 covers audit logging, not compliance checks, but the logic carries over. It requires organizations to "Alert [Assignment: organization-defined personnel or roles] within [Assignment: organization-defined time period] in the event of an audit logging process failure". The broken log pipeline gets its own alert and its own named recipient, apart from whatever the logs would have shown. The CA-7 discussion makes a related point about diagnosis: "a root-cause analysis may be needed to determine the specific control that has failed." Before you open a record against a control, confirm it was the control that failed.
| Outcome | What it shows | State of the evidence | Record that must follow |
|---|---|---|---|
| Pass | Status at the run time | Current until the next scheduled run | The dated result, kept as part of the series |
| Fail | A nonconformity at the run time | Failed | Nature, containment, cause, action or accepted risk with its reason, and the check that the action held (HS 10.2) |
| Error | Nothing about the control | Unchanged from the last real verdict | An alert to whoever owns the monitoring, and the gap noted if it lasts |
Check your setup against the last column. If a fail can close without a named decision, or an error can go unnoticed for a month, your results will look fine right up to the audit.
devguard released integration checks in September 2026, built around the table above.
A check runs on a schedule you attach to it. Each run is kept with its result, so a piece of evidence has a history rather than a single upload date.
A pass marks the linked evidence done and records the run as its latest review. A fail marks the evidence failed, records what was wrong, and leaves the review dates where they were; the evidence owner gets a notification when the check starts failing. An error leaves the evidence untouched, because a rejected credential or an unreachable API proves nothing about the control. After three errors in a row, the check is flagged unhealthy and the evidence owner is emailed. A test run shows the full result without writing any evidence.
Two of these choices have a cost, and devguard accepts both on purpose.
The first is that an error leaves the evidence alone. While a check keeps erroring, the evidence keeps the status from its last real verdict, so for a few runs evidence marked done can sit on top of an API that has stopped answering. Flipping it to failed would put a false nonconformity into the record; the unhealthy flag is how the gap reaches the owner instead.
The second is that the fail notification goes out once. It fires on the change into failing, not on every run while the check stays red, because an hourly check would otherwise send one email per hour for the same problem. The failed status stays visible the whole time, but nobody is reminded by email.
The check does not decide the response. It records that the evidence failed and tells the owner, and nothing is remediated or waived automatically. The 10.2 work stays with a person: the cause, the fix or the accepted risk, and the reason for it.
Before you automate a check, write down three things next to it: the interval and why you chose it, who owns a fail, and who hears about it when the check itself breaks.
devguard runs integration checks on the schedule you set and records pass, fail, and error as separate outcomes on your evidence; the changelog has the details.
Vadim leads engineering, the architecture, automation, and integrations that let compliance live. He's set on making a serious, audit-grade system genuinely easy to run.
Start free, bring your frameworks, and keep the evidence attached as the work ships — no setup friction, no audit-week fire drill.
Start free, bring your frameworks, and keep the evidence attached as the work ships — no setup friction, no audit-week fire drill.