A security team runs a controlled attack simulation. The endpoint platform detects suspicious behaviour, the SIEM generates an alert, the SOC opens an incident, and the dashboard shows that the controls are responding. On paper, everything looks good. But the test continues, and the attacker is still able to steal a session, move laterally, reach a privileged system and access the target data. The controls saw the activity, but they did not change the outcome.
That is why the argument in Security Products Should Be Tested Against the Outcome, Not the Alert matters.
Security teams often measure effectiveness through detection. Did the EDR alert? Did the SIEM correlate the event? Did the SOC create a case? These are useful measurements, but they can also create false confidence if the attacker can still achieve the objective.
The more important question is what happened after the alert.
Did the endpoint isolate the device? Did the identity platform revoke the session? Did network controls stop lateral movement? Did another security layer prevent the attacker from reaching the business asset that actually mattered?
This is also why Good Security Architecture Assumes Every Control Will Eventually Fail is an important principle. Security architecture should not depend on one control being perfect. One layer may detect the attack, another may contain it, and another may limit the damage.
That should change how security testing is performed.
Do not stop the exercise when the dashboard turns red. Continue the attack path and see how far it can actually go.
An alert tells you the product noticed something.
The outcome tells you whether the security architecture worked.