◇ OpenCyber Academy
Technologist · Chapter 4 · English

Chapter 4 — Data models

Open, free learning for anyone who wants to protect people and technology. No account is required to read this text or its worked answers.

Chapter 4 — Data models

Lesson map

Lesson 04 published. Map, example, EX010.

flowchart TD
  N0["Learning outcomes"]
  N1["4.1 Kinds of data"]
  N2["4.2 Vector indexes"]
  N3["4.3 Access control in retrieval"]
  N4["4.4 Sensitivity classes"]
  N5["4.4b Data poisoning through documents"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6
  N6 --> EX["Worked example"]
  EX --> Q["Exercise EX010"]

Logical drawing

flowchart LR
  N0["Learning outcomes"]
  N1["4.1 Kinds of data"]
  N2["4.2 Vector indexes"]
  N3["4.3 Access control in retrieval"]
  N4["4.4 Sensitivity classes"]
  N5["4.4b Data poisoning through documents"]
  N6["Synthesis"]
  N0 --> N1 --> N2 --> N3 --> N4 --> N5 --> N6 --> Q["EX010"]

[Learning outcomes]
    |
    v
[4.1 Kinds of data]
    |
    v
[4.2 Vector indexes]
    |
    v
[4.3 Access control in retrieval]
    |
    v
[4.4 Sensitivity classes]
    |
    v
[4.4b Data poisoning through documents]
    |
    v
[Synthesis]
    |
    v
[Worked example] --> [Exercise EX010]

Learning outcomes

You will (1) describe data structures used by AI systems — tables, documents, embeddings; (2) explain vector indexes and their security properties; (3) apply access control to data used for retrieval; and (4) classify data sensitivity.

4.1 Kinds of data

AI systems use relational tables (orders, customers), documents (policies, product pages, emails), training sets (labelled examples), and embeddings — lists of numbers representing meaning, stored in vector indexes for retrieval. Ask Brightwell uses three of these: the orders and customers tables through the order tool, the policy and product pages through retrieval, and their embeddings in the vector index. ReturnGuard adds a labelled training set of past returns, labelled by staff. Each has different owners and sensitivity, which the inventory later records.

4.2 Vector indexes

Retrieval-augmented generation (RAG) embeds documents, stores the vectors, and at query time finds the nearest ones to include in the prompt. Security issues: the index may mix documents with different permissions; embeddings can leak information about the text they encode; and anyone who can add documents can influence answers. OWASP lists these as LLM08 Vector and Embedding Weaknesses. Ask Brightwell's index showed the first issue: a supplier-margin spreadsheet uploaded by someone sat in the same index as public product pages, so a customer question about discounts retrieved it.

4.3 Access control in retrieval

Store each chunk with metadata: owner, audience, sensitivity. At query time filter by the caller's permissions before passing chunks to the model; better still, keep separate indexes for separate audiences. Never rely on telling the model "do not reveal internal documents". After the margin leak, Brightwell took the stronger option: a customer index containing only public documents and a separate staff-only index. Audience metadata is still recorded on each chunk, so a misfiled document can be found and the filter acts as a second layer if a staff document is ever indexed in the wrong place.

4.4 Sensitivity classes

Classify: public, internal, confidential, personal. Personal data needs a legal basis and minimisation. Training sets need the same classification as their sources. For Ask Brightwell, product pages and the published returns policy are public, the supplier-margin spreadsheet is confidential, and order details are personal. ReturnGuard's training set inherits the personal class because it is built from customer returns, so its storage and access follow the same rules as the orders table. Classifying sources before indexing would have kept the margin spreadsheet out of the customer index.

4.4b Data poisoning through documents

Anyone who can add or edit indexed documents can plant text the model will read — false prices, fake policies or hidden instructions. Restrict who can publish to the index, record each document's source and change history, and review new sources before they are indexed. Product reviews written by customers are untrusted and should be indexed separately or not at all. The 60-day holiday page that confused Ask Brightwell later was an unintentional example of the same weakness. That page was not malicious, but it reached customers through the same path.

4.5 Worked example (synthetic Brightwell case)

Brightwell's index contained public product pages and an internal "supplier margin" spreadsheet that someone had uploaded. Jonas asks Ask Brightwell, "what discount can you give on the Oslo sofa?" and gets the margin. He adds audience metadata, rebuilds the customer index with public documents only, and moves internal documents to a staff-only index. The fix changed where documents live rather than how the model behaves. A question about discounts now reaches only public pages, whatever wording the customer uses, because the margin spreadsheet is not in the customer index at all. Telling the model not to reveal it would have left the data one clever question away.

4.6 Errors and limits

Brightwell's margin leak came from the first error and would not have been fixed by the second: a prompt instruction cannot stop the model from repeating text it has been given. ReturnGuard's training set, built from customer returns, must be classified as personal data like its source, so its storage and access follow the orders table's rules. Each error turns a data-handling mistake into a disclosure through the assistant.

Exercises

Synthesis

AI data models include documents and embeddings; security depends on labelling data, filtering by permission before retrieval and never letting prompts stand in for access control.

Continue studying — chapter 04 of 30

← Chapter 03 (draft) · Course chapter index · Chapter 05 (draft) →