Chapter 23 — Cryptography and trust services: evaluation | Technologist Cryptography and trust services

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 23 — Cryptography and trust services: evaluation

Lesson map

Lesson 23 published. Map, example, EX067.

flowchart TD
  N0["Learning outcomes"]
  N1["23.1 Did it work"]
  N2["23.2 Measures"]
  N3["23.3 Honest reporting"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4
  N4 --> EX["Worked example"]
  EX --> Q["Exercise EX067"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["23.1 Did it work"]
  N2["23.2 Measures"]
  N3["23.3 Honest reporting"]
  N4["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> Q["EX067"]

[Learning outcomes]
    |
    v
[23.1 Did it work]
    |
    v
[23.2 Measures]
    |
    v
[23.3 Honest reporting]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX067]

Learning outcomes

You will (1) evaluate whether trust-service changes worked; (2) choose before-and-after measures; (3) report results honestly.

23.1 Did it work

After six months of changes, Owen asks Rosa to evaluate them. She returns to the goals set in the risk scenarios: fewer certificate outages, no weak TLS on public services, safer password storage and verified webhooks. Evaluation compares outcomes with those goals rather than listing completed tasks. Finishing a migration is an activity; reducing outages is an outcome. Rosa writes this distinction at the top of her report so readers know what she measured. She also lists the original goals verbatim, so nobody can quietly redefine success after the fact.

23.2 Measures

Rosa picks measures available before and after: certificate-related outage hours fell from nine in the previous half-year to zero; public endpoints offering TLS 1.0 fell from three to zero; the share of citizen accounts with memory-hard hashes rose to 71 percent through rehash-on-login and wrapping; and webhook verification failures are now logged and reviewed. She notes confounders: a quieter period for changes may have helped outage numbers, and six months is short for rare events. Each figure links to the query that produced it, so the committee can rerun it next half-year.

23.3 Honest reporting

Rosa reports what improved, what did not and what is unknown. The legacy-gis appliance remains a gap; one supplier still needs a retest; and outage figures need another year to be convincing. She includes the data sources and how numbers were computed so others can check them. Owen tells her that honest limitations make the report more credible to the risk committee, which has seen too many reports claiming complete success. She closes with three recommendations ranked by risk reduction, each with an owner and a proposed date, so the evaluation leads directly to next steps.

Worked example

Rosa evaluates six months of work, reporting outage hours from nine to zero and TLS 1.0 endpoints from three to zero, with confounders and remaining gaps.

Synthetic case. Figures are invented.

Errors and limits

Before-and-after comparisons can mislead when other factors changed. Activity counts are not outcomes.

Exercises

EX067 — Activity vs outcome. Give an example of an activity and an outcome from the chapter. Rubric: 3 = activity: finishing migration; outcome: fewer outages;; 2 = both;; 1 = one;; 0 = none.
EX068 — Outage figures. How did certificate outage hours change? Rubric: 3 = from nine to zero over the half-year;; 2 = numbers;; 1 = vague;; 0 = none.
EX069 — Confounder. Name one confounder Rosa noted. Rubric: 3 = quieter change period or short six-month window;; 2 = one;; 1 = vague;; 0 = none.

Synthesis

Evaluation compares outcomes with goals, notes confounders and reports gaps honestly.

Continue studying — chapter 23 of 30

← Chapter 22 (draft) · Course chapter index · Chapter 24 (draft) →