Decision Theory: Expected Utility and Decision Trees

Decision Theory: Expected Utility and Decision Trees#

Probability distributions tell you what might happen. Decision theory tells you what to do about it. This chapter bridges the gap between quantified uncertainty and actionable choices, using security investment as the running example.

Most security teams already make decisions under uncertainty — they just do it informally. Decision theory does not replace judgment; it provides a framework for making the judgment explicit, repeatable, and auditable. When you can show exactly which assumptions drove a recommendation, you can also show exactly what would need to change for a different recommendation to win.

This notebook covers:

  1. Decision trees for structuring choices under uncertainty

  2. Expected value (risk-neutral) decisions

  3. Expected utility (risk-averse) decisions

  4. Monte Carlo decision analysis with full loss distributions

  5. Expected Value of Perfect Information (EVPI)

  6. EVSI: what imperfect information is actually worth

  7. Opportunity loss and regret framing

  8. Greedy control selection under budget constraints

Setup#

We use the Monte Carlo engine for loss simulation, VOI functions for EVPI and control selection, and synth for reproducible random generation.

import numpy as np
import matplotlib.pyplot as plt

from decision_security.synth import make_rng
from decision_security.montecarlo import (
    simulate_aggregate_losses,
    make_lognormal_severity,
    risk_bands,
)
from decision_security.voi import evpi, select_controls_by_roi

rng = make_rng(42)

plt.rcParams.update({
    "font.family": "serif",
    "font.size": 10,
    "axes.labelsize": 11,
    "axes.titlesize": 12,
    "xtick.labelsize": 9,
    "ytick.labelsize": 9,
    "legend.fontsize": 9,
    "figure.dpi": 150,
    "axes.spines.top": False,
    "axes.spines.right": False,
})

PRIMARY = "#1A1A1A"
ACCENT = "#E74C3C"
DARK_BG = "#34495E"
LIGHT_GRAY = "#95A5A6"
MED_GRAY = "#7F8C8D"
VERY_LIGHT = "#BDC3C7"

Decision Tree: Deploy EDR Now vs. Wait#

A decision model separates what you control from what you don’t. Decision nodes (squares in a tree diagram) represent choices — deploy EDR now or next quarter, fund a red-team exercise or spend on phishing training. Chance nodes (circles) represent uncertain events — whether an attacker targets you this year, whether a vulnerability is exploitable.

The discipline of drawing the tree matters as much as computing the answer. Each branch requires you to name the alternatives (not just the one you already prefer), assign probabilities to chance events (not just “likely” or “unlikely”), and attach outcomes to every terminal node. This structure alone eliminates a surprising number of bad decisions — the ones that fail not because the math was wrong but because an option or a risk was never considered.

A security team faces a classic decision: deploy an EDR solution now (incurring a certain cost) or wait and accept the risk of a breach.

                          ┌── ○ Breach (p=0.2) ──── Loss: $800K
             ┌─ Wait ─────┤
             │            └── ○ No breach (p=0.8) ── Loss: $0
  □ Decision ┤
             │            ┌── ○ Breach (p=0.2) ──── Loss: $200K (EDR cost only)
             └─ Deploy ───┤                       (EDR contains the breach)
                          └── ○ No breach (p=0.8) ── Loss: $200K (EDR cost)

Assumptions for the simple model:

  • Breach probability: 0.2 over the decision horizon

  • Unmitigated breach cost: $800K

  • EDR annual cost: $200K

  • If EDR is deployed and breach occurs, EDR fully contains it (loss = EDR cost only)

Note the numbers are set so the risk-neutral and risk-averse answers disagree — the whole point of the next two sections.

Expected Value Decision#

The simplest rule is expected value (EV): multiply each outcome by its probability, sum, and pick the option with the highest total (or lowest total cost). A risk-neutral decision-maker — one who only cares about the long-run average — would always follow this rule.

For the EDR example, if the breach probability is 20%, the EV of waiting is 20% of $800K = $160K, while deploying costs a certain $200K. Waiting is $40K cheaper in expectation, so a purely risk-neutral decision-maker would Wait and accept the gamble.

But expected value treats every dollar identically, whether it is the first dollar or the dollar that triggers insolvency. A $160K expected loss that occasionally produces an $800K hit may be worse than a guaranteed $200K spend, even though the EV says otherwise.

breach_prob = 0.2
breach_cost = 800_000
edr_cost = 200_000

ev_wait = breach_prob * breach_cost + (1 - breach_prob) * 0
ev_deploy = edr_cost  # EDR contains any breach, so the cost is certain

print(f"EV(Wait)   = ${ev_wait:,.0f}")
print(f"EV(Deploy) = ${ev_deploy:,.0f}")
print()
if ev_deploy < ev_wait:
    print(f"Risk-neutral choice: Deploy EDR (saves ${ev_wait - ev_deploy:,.0f} in expectation)")
else:
    print(f"Risk-neutral choice: Wait (saves ${ev_deploy - ev_wait:,.0f} in expectation)")

EV(Wait)   = $160,000
EV(Deploy) = $200,000

Risk-neutral choice: Wait (saves $40,000 in expectation)

Expected Utility: Risk Aversion Changes the Answer#

Most organizations are risk-averse — an $800K loss hurts about eight times as much as a $200K loss under the utility function below. Expected utility replaces dollar outcomes with a utility function \(u(\cdot)\).

A concave utility function — \(u(w) = \sqrt{w}\) or \(u(w) = \ln(w)\) for wealth \(w\), or, for a loss \(x\) as here, the exponential form below — captures risk aversion: losing $10M hurts more than gaining $10M helps. The decision rule becomes: choose the option that maximizes \(\mathbb{E}[u(X)]\).

We model this with an exponential utility function:

\[u(x) = -\exp\left(\frac{x}{\rho}\right)\]

where \(\rho\) is the risk tolerance parameter. Smaller \(\rho\) means more risk-averse. We use \(\rho = 500{,}000\) here.

Consider two controls with equal expected-value reduction of $500K. Control A reduces many small incidents; Control B reduces a single catastrophic scenario. Under expected value they are equivalent, but under a concave utility function the catastrophic-loss reduction is worth more, which matches how most executives actually think.

rho = 500_000  # risk tolerance

def utility(x):
    """Exponential utility of a cost x: u(x) = -exp(x / rho), so higher cost gives lower utility."""
    return -np.exp(x / rho)

eu_wait = breach_prob * utility(breach_cost) + (1 - breach_prob) * utility(0)
eu_deploy = utility(edr_cost)

print(f"EU(Wait)   = {eu_wait:.6f}")
print(f"EU(Deploy) = {eu_deploy:.6f}")
ce_wait = rho * np.log(-eu_wait)  # certain cost with the same utility as the Wait gamble
print(f"Certainty equivalent of Wait: ${ce_wait:,.0f}  vs  ${edr_cost:,.0f} certain for Deploy")
print()
if eu_deploy > eu_wait:
    print(f"Risk-averse choice: Deploy EDR (the certain ${edr_cost:,.0f} beats the gamble)")
else:
    print("Risk-averse choice: Wait")

EU(Wait)   = -1.790606
EU(Deploy) = -1.491825
Certainty equivalent of Wait: $291,277  vs  $200,000 certain for Deploy

Risk-averse choice: Deploy EDR (the certain $200,000 beats the gamble)

x = np.linspace(0, 1_200_000, 500)
u = utility(x)

fig, ax = plt.subplots(figsize=(8, 4))
ax.plot(x / 1e6, u, color=DARK_BG, linewidth=2)
ax.axvline(breach_cost / 1e6, color=ACCENT, linestyle="--", alpha=0.6, label="Breach cost ($800K)")
ax.axvline(edr_cost / 1e6, color=DARK_BG, linestyle="--", alpha=0.6, label="EDR cost ($200K)")
ax.set_xlabel("Loss ($M)")
ax.set_ylabel("Utility  u(x)")
ax.set_title("Exponential Utility Curve (rho = $500K)")
ax.text(0.30, 0.06, "x is a loss: higher loss, lower utility.",
        transform=ax.transAxes, fontsize=8, style="italic", color=MED_GRAY)
ax.legend()
plt.tight_layout()
plt.show()
../_images/3149c74a4a393f32200919d4ee549d0c832c6509cd757feab99bc6fdb60d8d91.png

The concavity of the utility curve is what drives risk aversion: the marginal disutility of each additional dollar of loss increases. Here that is decisive. Risk aversion does not just tip a close call; it reverses the risk-neutral answer.

The gap between the EV-optimal choice and the EU-optimal choice widens as the stakes grow. For small decisions the two frameworks agree; for decisions near the organization’s pain threshold, risk aversion dominates. This is precisely the regime where security investments tend to live.

Monte Carlo Decision Analysis#

When outcomes are continuous distributions rather than a handful of scenarios, closed-form EV calculations become unwieldy. Monte Carlo simulation handles this naturally: draw thousands of samples from your loss and cost distributions, compute utility for each, and compare the resulting distributions of outcomes across options.

We simulate the pure loss distributions for both scenarios. The control cost enters once in the decision comparison, not inside the simulation. This keeps the loss distributions clean and the economics simple: if mean risk reduction > annual control cost, the control pays for itself.

Two options may have the same average outcome, but one has a long right tail representing catastrophic losses. A risk-averse organization should care about that tail, and simulation makes it visible. The histograms below overlay the two options; overlaying their CDFs would show the probability of exceeding any given loss threshold.

Note: This is deliberately the simplest possible investment calculation. A real analysis would consider: net present value (NPV) with discount rates, recurring vs. one-time costs (licensing, staffing, maintenance), implementation timelines and partial-year exposure, correlation between controls (deploying EDR may reduce the value of a separate SIEM investment), and opportunity cost of the budget. The goal is to show that even a basic simulation beats a point estimate.

severity = make_lognormal_severity(meanlog=12.2, sdlog=1.0)  # ~$200K median
n_periods = 10_000

# Independent seeds: we compare two distributions, not paired scenarios. A shared seed
# would not pair rows anyway, because the Poisson counts diverge once lam differs.
losses_no_edr   = simulate_aggregate_losses(n_periods, lam=1.2, severity_sampler=severity, rng=make_rng(42))
losses_with_edr = simulate_aggregate_losses(n_periods, lam=0.6, severity_sampler=severity, rng=make_rng(43))

print('--- No EDR (pure risk) ---')
bands_no = risk_bands(losses_no_edr)
for k, v in bands_no.items():
    print(f'  {k}: ${v:,.0f}')

print()
print('--- With EDR (pure risk) ---')
bands_edr = risk_bands(losses_with_edr)
for k, v in bands_edr.items():
    print(f'  {k}: ${v:,.0f}')

print()
mean_reduction = losses_no_edr.mean() - losses_with_edr.mean()
net_value = mean_reduction - edr_cost
net_se = np.sqrt(losses_no_edr.var() / n_periods + losses_with_edr.var() / n_periods)
print(f'Mean risk reduction:  ${mean_reduction:,.0f}')
print(f'EDR annual cost:      ${edr_cost:,.0f}')
print(f'Net annual value:     ${net_value:,.0f}  (Monte Carlo noise: ±${2 * net_se:,.0f}, 2 SE)')
print()
if abs(net_value) < 2 * net_se:
    print('Net value is within noise of zero: EDR roughly breaks even.')
elif net_value > 0:
    print('EDR pays for itself on a risk-neutral basis.')
else:
    print('EDR does not pay for itself on a risk-neutral basis.')
tree_ev = 'Deploy' if ev_deploy < ev_wait else 'Wait'
tree_eu = 'Deploy' if eu_deploy > eu_wait else 'Wait'
print(f'Simple tree: risk-neutral says {tree_ev}, expected utility says {tree_eu}.')

--- No EDR (pure risk) ---
  p50: $194,829
  p90: $1,033,951
  p95: $1,482,220

--- With EDR (pure risk) ---
  p50: $0
  p90: $588,039
  p95: $947,120

Mean risk reduction:  $199,322
EDR annual cost:      $200,000
Net annual value:     $-678  (Monte Carlo noise: ±$14,519, 2 SE)

Net value is within noise of zero: EDR roughly breaks even.
Simple tree: risk-neutral says Wait, expected utility says Deploy.

fig, ax = plt.subplots(figsize=(8, 4))
bins = np.linspace(0, np.percentile(losses_no_edr, 99), 60)
ax.hist(losses_no_edr, bins=bins, alpha=0.5, density=True, label='No EDR (lam=1.2)', color=ACCENT)
ax.hist(losses_with_edr, bins=bins, alpha=0.5, density=True, label='With EDR (lam=0.6)', color=DARK_BG)
ax.set_xlabel('Annual Aggregate Loss ($)')
ax.set_ylabel('Density')
ax.set_title('Pure Loss Distributions: EDR Shifts the Entire Curve Left')
ax.legend()
plt.tight_layout()
plt.show()
../_images/363d4b9bb9059e9f5c5bc843ce0cf1598c1c3a01b9569fe4dadc233e9ac9c97f.png

EVPI: Expected Value of Perfect Information#

Before spending $500K on a threat-intelligence platform to refine your estimates, ask: what is the most that perfect information could be worth? The Expected Value of Perfect Information (EVPI) answers this. It is the difference between the expected payoff if you could know the true state of the world before deciding and the expected payoff of the best decision under current uncertainty.

We build a loss matrix with shape (S, D) — S scenarios, D decisions — and pass it to evpi().

EVPI is especially useful as a triage tool. Computing EVPI for each parameter (strictly, the expected value of partial perfect information, EVPPI) tells you which uncertainties actually matter for the decision and which are “don’t care” variables. If the decision is the same whether breach probability is 3% or 12%, there is no point commissioning a study to narrow that range. Focus data-collection effort where EVPI is highest because that is where better information changes what you do, not just what you believe.

# EVPI needs both decisions evaluated on the SAME scenarios, so EDR is applied as a
# containment transformation on one set of simulated years, not as a separate simulation.
rng_evpi = make_rng(7)
loss_state = simulate_aggregate_losses(n_periods, lam=1.2, severity_sampler=severity, rng=rng_evpi)

# Unlike the 50% frequency cut above (roughly break-even), EDR here contains 60% of each
# year's loss, so its expected saving exceeds its cost and the verdict flips to Deploy.
edr_effectiveness = 0.6
loss_matrix = np.column_stack([
    loss_state,                                        # Wait
    edr_cost + (1 - edr_effectiveness) * loss_state,   # Deploy: EDR cost + residual loss
])

evpi_val = evpi(loss_matrix)
assert evpi_val >= 0, "EVPI must be non-negative -- a negative value means a bug."

print(f"E[loss | Wait]   = ${loss_matrix[:, 0].mean():,.0f}")
print(f"E[loss | Deploy] = ${loss_matrix[:, 1].mean():,.0f}")
best = "Deploy" if loss_matrix[:, 1].mean() < loss_matrix[:, 0].mean() else "Wait"
print(f"Best decision under current uncertainty: {best}")
print(f"EVPI: ${evpi_val:,.0f}")

E[loss | Wait]   = $394,090
E[loss | Deploy] = $357,636
Best decision under current uncertainty: Deploy
EVPI: $93,034

fig, ax = plt.subplots(figsize=(8, 4))
bins = np.linspace(0, np.percentile(loss_matrix, 99), 60)
ax.hist(loss_matrix[:, 0], bins=bins, alpha=0.5, color=DARK_BG, label="Wait (full loss)")
ax.hist(loss_matrix[:, 1], bins=bins, alpha=0.5, color=ACCENT, label="Deploy EDR (contain + $200K)")
ax.set_xlabel("Total Loss ($)")
ax.set_ylabel("Count")
ax.set_title("EVPI: two decisions evaluated on the same scenarios")

# EVPI is a difference in expected loss, not a threshold, so annotate instead of drawing a line.
ax.text(0.97, 0.95,
        f"EVPI = ${evpi_val:,.0f}\n(value of knowing the\nscenario before deciding)",
        transform=ax.transAxes, ha="right", va="top", fontsize=9,
        bbox=dict(boxstyle="round", facecolor="white", edgecolor=LIGHT_GRAY))
ax.legend(loc="center right")
plt.tight_layout()
plt.show()
../_images/f85d202054d24a96a48ef005f65568f242bec7b94a851d5d0d8bd5c8b6165367.png

EVPI tells you the ceiling on what better information is worth. If EVPI is $15K, spending $50K on a threat intelligence platform to resolve this uncertainty is overpaying because no amount of data collection could improve your decision by more than $15K. If EVPI is $200K, even an expensive assessment could pay for itself.

Tip: A quick EVPI sanity check: if the best and worst plausible values of a parameter lead to the same decision (and, with two options, the loss moves monotonically with the parameter), that parameter has zero information value for this decision. You can stop debating it and move on.

In practice: EVPI assumes you could get perfect information, which never happens. The realistic metric is EVSI (Expected Value of Sample Information) — what a specific, imperfect source of data is actually worth. That requires modeling the information source’s accuracy.

EVSI: What Is Imperfect Information Actually Worth?#

Real information sources are noisy: a penetration test reveals some vulnerabilities but not all, a threat-intelligence feed is directionally correct but not perfect. EVSI (Expected Value of Sample Information) extends the framework to handle imperfect evidence.

The calculation follows Raiffa’s (1968) structure:

  1. For each possible signal the information source might produce, compute the posterior probability of each state using Bayes’ theorem.

  2. Determine the optimal action given that posterior.

  3. Weight by the probability of receiving each signal.

  4. The difference between this expected value and the expected value of the best unconditional action is the EVSI.

Scenario: Before deciding on EDR deployment, the CISO can commission a $50K threat assessment. The assessment has 80% accuracy — it correctly predicts whether a breach will occur 80% of the time (i.e., 80% true positive rate, 80% true negative rate).

\[\text{EVSI} = \mathbb{E}_{\text{signal}}\bigl[V^*(\text{posterior})\bigr] - V^*(\text{prior})\]

If EVSI < $50K, the assessment costs more than the decision value it provides. If EVSI > $50K, it pays for itself.

EVSI is especially valuable for comparing information sources with different costs and accuracies. A $10K vulnerability scan with 60% accuracy might have higher EVSI-per-dollar than a $100K red-team engagement with 90% accuracy, depending on the decision context. The key insight from Raiffa: the value of information depends not just on its accuracy but on how much the decision changes in response to the information. If you would deploy EDR regardless of what the assessment finds, the assessment has zero decision value no matter how accurate it is.

tpr = 0.80  # P(says "breach" | breach)
tnr = 0.80  # P(says "safe" | no breach)

p_says_breach = tpr * breach_prob + (1 - tnr) * (1 - breach_prob)
p_says_safe = 1 - p_says_breach

# Posteriors via Bayes
p_breach_given_says_breach = (tpr * breach_prob) / p_says_breach
p_breach_given_says_safe = ((1 - tpr) * breach_prob) / p_says_safe

ev_deploy = edr_cost
ev_wait_if_breach_signal = p_breach_given_says_breach * breach_cost
best_if_breach_signal = min(ev_deploy, ev_wait_if_breach_signal)

ev_wait_if_safe_signal = p_breach_given_says_safe * breach_cost
best_if_safe_signal = min(ev_deploy, ev_wait_if_safe_signal)

ev_with_assessment = p_says_breach * best_if_breach_signal + p_says_safe * best_if_safe_signal
ev_without = min(ev_deploy, breach_prob * breach_cost)

evsi = ev_without - ev_with_assessment

# EVPI for the same 2-state tree: the ceiling EVSI cannot exceed
evpi_tree = min(ev_deploy, breach_prob * breach_cost) - (
    breach_prob * min(edr_cost, breach_cost) + (1 - breach_prob) * min(edr_cost, 0)
)

print(f"P(assessment says 'breach') = {p_says_breach:.2f}")
print(f"P(assessment says 'safe')   = {p_says_safe:.2f}")
print()
print(f"If 'breach' signal: P(actual breach) = {p_breach_given_says_breach:.1%}")
print(f"  -> Best action: {'Deploy' if ev_deploy < ev_wait_if_breach_signal else 'Wait'} (EV = ${best_if_breach_signal:,.0f})")
print(f"If 'safe' signal:   P(actual breach) = {p_breach_given_says_safe:.1%}")
print(f"  -> Best action: {'Deploy' if ev_deploy < ev_wait_if_safe_signal else 'Wait'} (EV = ${best_if_safe_signal:,.0f})")
print()
print(f"EV without assessment: ${ev_without:,.0f}")
print(f"EV with assessment:    ${ev_with_assessment:,.0f}")
print(f"EVSI:                  ${evsi:,.0f}")
print(f"EVPI (perfect oracle): ${evpi_tree:,.0f}")
print(f"EVSI / EVPI:           {evsi / evpi_tree:.0%}")
print()
assessment_cost = 50_000
if evsi > assessment_cost:
    print(f"Assessment cost ${assessment_cost:,.0f} < EVSI: worth commissioning.")
else:
    print(f"Assessment cost ${assessment_cost:,.0f} >= EVSI: not worth commissioning.")

P(assessment says 'breach') = 0.32
P(assessment says 'safe')   = 0.68

If 'breach' signal: P(actual breach) = 50.0%
  -> Best action: Deploy (EV = $200,000)
If 'safe' signal:   P(actual breach) = 5.9%
  -> Best action: Wait (EV = $47,059)

EV without assessment: $160,000
EV with assessment:    $96,000
EVSI:                  $64,000
EVPI (perfect oracle): $120,000
EVSI / EVPI:           53%

Assessment cost $50,000 < EVSI: worth commissioning.

Opportunity Loss: Reframing as Regret#

Savage (1951) introduced, and Raiffa popularised, an alternative framing that security audiences often find more intuitive: instead of computing what each action is worth, compute what you give up by choosing the wrong action in each possible state. This is the opportunity loss or regret for each state-action pair.

For any decision, the opportunity loss of an action in a given state is the difference between the cost of that action and the cost of the best possible action for that state. The Expected Opportunity Loss (EOL) weights these regrets by state probabilities.

A remarkable property: the EOL of the optimal action always equals the EVPI. This provides both a sanity check on calculations and an intuitive interpretation. EVPI is the expected regret of having to decide under uncertainty. In security terms, it answers: “on average, how much do we leave on the table because we don’t know which scenario will materialize?”

Presenting both — the EV comparison and the opportunity-loss table — gives decision-makers two complementary framings of the same problem; they always rank the options the same way.

import pandas as pd

# Losses: lower is better
payoff_matrix = pd.DataFrame(
    {
        "Deploy EDR": [edr_cost, edr_cost],
        "Wait":       [breach_cost, 0],
    },
    index=[f"Breach (p={breach_prob})", f"No Breach (p={1 - breach_prob:.1f})"],
)

best_per_state = payoff_matrix.min(axis=1)
opp_loss = payoff_matrix.subtract(best_per_state, axis=0)

print("=== Cost Matrix ===")
print(payoff_matrix.to_string())
print()
print("=== Opportunity Loss ===")
print(opp_loss.to_string())
print()

probs = np.array([breach_prob, 1 - breach_prob])
eol = opp_loss.T @ probs
print("=== Expected Opportunity Loss ===")
for action, val in eol.items():
    print(f"  {action}: ${val:,.0f}")

print(f"\nEVPI = EOL of the best action = ${eol.min():,.0f}")
print("(A within-model identity: the Monte Carlo EVPI above is a different model, so it differs.)")

=== Cost Matrix ===
                   Deploy EDR    Wait
Breach (p=0.2)         200000  800000
No Breach (p=0.8)      200000       0

=== Opportunity Loss ===
                   Deploy EDR    Wait
Breach (p=0.2)              0  600000
No Breach (p=0.8)      200000       0

=== Expected Opportunity Loss ===
  Deploy EDR: $160,000
  Wait: $120,000

EVPI = EOL of the best action = $120,000
(A within-model identity: the Monte Carlo EVPI above is a different model, so it differs.)

Control Selection Under Budget Constraints#

Given a menu of candidate controls — EDR, WAF, MFA, SIEM, pen testing — each with an estimated cost and risk-reduction effect, a greedy ROI approach ranks them by risk reduction per dollar and fills the budget from the top. This is fast, intuitive, and often close to optimal for modest portfolios.

For example, in the control set below MFA costs $30K and reduces expected loss by $150K — an ROI of $5.00 per dollar, the highest — so the greedy rule funds it first. A WAF and a pen test come next (each about $1.60 of risk reduction per dollar), and with budget still left the rule reaches for the next-best ratio. EDR, despite reducing risk in absolute terms, has a weak ratio ($0.60 per dollar), and at $200K it is the one squeezed out under the $300K cap — the greedy rule skips it and instead funds the cheaper SIEM, whose $0.33-per-dollar ratio is actually the lowest in the menu. That a weaker-but-affordable control can edge out a stronger one that will not fit is exactly where greedy knapsack departs from the true optimum.

Note: The greedy approach assumes control effects are independent: deploying MFA does not change the risk-reduction value of EDR. In practice, controls interact and overlapping detective controls may have diminishing marginal returns, while complementary controls (prevention + detection) may multiply effectiveness. Optimization methods exist that can handle some of these interactions.

control_names = ["EDR", "WAF", "MFA", "SIEM", "Pen Test"]
deltas = [120_000, 80_000, 150_000, 60_000, 40_000]   # annual risk reduction
costs  = [200_000, 50_000, 30_000, 180_000, 25_000]    # annual cost
budget = 300_000

chosen_idx, total_cost, total_delta = select_controls_by_roi(deltas, costs, budget)

print(f"Budget: ${budget:,}")
print(f"\nSelected controls (by ROI rank):")
for i in chosen_idx:
    roi = deltas[i] / costs[i]
    print(f"  {control_names[i]:>10s}  cost=${costs[i]:>8,}  delta=${deltas[i]:>8,}  ROI={roi:.2f}")

print(f"\nTotal spend: ${total_cost:,.0f} / ${budget:,}")
print(f"Total risk reduction: ${total_delta:,.0f}")
print(f"Unspent budget: ${budget - total_cost:,.0f}")

Budget: $300,000

Selected controls (by ROI rank):
         MFA  cost=$  30,000  delta=$ 150,000  ROI=5.00
         WAF  cost=$  50,000  delta=$  80,000  ROI=1.60
    Pen Test  cost=$  25,000  delta=$  40,000  ROI=1.60
        SIEM  cost=$ 180,000  delta=$  60,000  ROI=0.33

Total spend: $285,000 / $300,000
Total risk reduction: $330,000
Unspent budget: $15,000

roi = np.array(deltas) / np.array(costs)
order = np.argsort(roi)[::-1]
bar_colors = [ACCENT if i in chosen_idx else VERY_LIGHT for i in order]

fig, ax = plt.subplots(figsize=(8, 4))
ax.barh([control_names[i] for i in order],
        [roi[i] for i in order],
        color=bar_colors, edgecolor='white')
ax.set_xlabel('ROI (Risk Reduction / Cost)')
ax.set_title(f'Control ROI Ranking (selected in red, budget = ${budget:,.0f})')
plt.tight_layout()
plt.show()
../_images/d34e659be4a6d174874be5635787c31c6801ddfc9d96d8f0c967f927b1c8cb56.png

Pitfalls#

Decision-theoretic analysis is only as good as its inputs and structure. Recurring traps in security:

Means hide variance. Reporting that the “expected annual loss is $2M” obscures the difference between a near-certain $2M loss and a 1% chance of $200M. Always present distributional summaries (percentiles, density plots), not just point estimates. The EV analysis above says “Wait” is cheaper by $40K, but the no-EDR distribution has a much heavier right tail. A single bad year wipes out many years of savings.

Omitting costs. A control that “reduces risk by 40%” looks great until you account for license fees, staffing, maintenance, and the productivity drag of false positives. Include total cost of ownership in every branch of the tree.

Double-counting. Reputation damage and revenue loss often overlap — the revenue drops because of the reputation hit. Counting both at full value inflates the benefit of mitigation and distorts the analysis.

Ignoring the “do nothing” baseline. Every option should be compared against the status quo, not against zero. If current controls already reduce expected loss by 60%, the marginal value of an additional control is measured against that baseline.

Sanity-check your tooling. Some quantities have hard sign constraints: EVPI and EVSI are always \(\geq 0\) (information cannot make a well-posed decision worse), and EVSI \(\leq\) EVPI (imperfect information is worth no more than a perfect oracle).

What We Left Out#

This notebook intentionally simplifies to teach the core mechanics. Production-grade decision analysis would also address:

  • Time value of money. A dollar of risk reduction next year is worth less than a dollar today. Discounting future costs and benefits via NPV is standard in finance but often skipped in security analyses.

  • Correlated risks. A single breach event may trigger regulatory fines, litigation, customer churn, and operational downtime simultaneously. Modeling these as independent overstates diversification.

  • Real options. Some decisions are irreversible (decommissioning a legacy system) while others are deferrable (waiting for next-gen tooling). Real-options thinking values the flexibility to wait, which pure EV analysis ignores.

  • Portfolio effects. Controls interact as a system. The marginal value of a detective control depends on what preventive controls are already in place. Later chapters address portfolio-level modeling.

Even the simplest quantitative framing — distributions + expected utility + EVPI — outperforms gut feel.