docs: Field Trial Log + Decision KPIs + MEMORY update

- Created FIELD_TRIAL_LOG.md — observation protocol for real customer cases
  - Log entry template with 10 fields
  - Two example entries (accepted and rejected decisions)
  - Questions the log answers: adoption, accuracy, calibration, rejection analysis
  - Future KPIs: Decision Adoption Rate, Decision Accuracy, Time to Decision

- Updated MEMORY.md with Control Intelligence & Decision Model section
  - Decision Pipeline v1.0 summary
  - Decision Object v1.0 (STRUCTURE FROZEN)
  - Status: VALIDATED FOR FIELD TRIALS
  - Three target customer cases
  - Future KPIs
  - Key principle: No more modeling until first real customer case

- Added Decision Adoption Rate and Decision Accuracy as future product KPIs
  - Not implemented yet — start collecting data now, calculate later
  - Decision Adoption = accepted / total recommendations
  - Decision Accuracy = correct / total decisions

Rationale: Stop modeling, start observing. First real customer case
will teach more than the last twenty documents combined.
This commit is contained in:
Bernt
2026-07-02 12:19:10 +00:00
parent c8819109db
commit 1c4158c373
2 changed files with 159 additions and 0 deletions
+46
View File
@@ -881,3 +881,49 @@ Mätvärden:
- Leverabler: Engineering Readiness Score (viktad), Policy Enforcement, STOP Register (med risk), Release Gate, Meta-Rules - Leverabler: Engineering Readiness Score (viktad), Policy Enforcement, STOP Register (med risk), Release Gate, Meta-Rules
- Exit Criteria: Varje STOP är eliminerad, har dispens, eller ersatts av kontrollerad process - Exit Criteria: Varje STOP är eliminerad, har dispens, eller ersatts av kontrollerad process
- EOS-regel: "EOS får aldrig blockera något utan att kunna förklara exakt varför och vilken minsta åtgärd som krävs för att häva blockeringen." - EOS-regel: "EOS får aldrig blockera något utan att kunna förklara exakt varför och vilken minsta åtgärd som krävs för att häva blockeringen."
---
## 11. CONTROL INTELLIGENCE & DECISION MODEL (2026-07-02)
**LandveX produces Control Intelligence, not AI analysis.**
AI is implementation. Control Intelligence is the product.
### Decision Pipeline v1.0
```
Reality → Observation → Evidence → Finding → Decision → Action → Business Impact → Learning
```
**Seven steps with input/transformation/output/owner.**
### Decision Object v1.0 (STRUCTURE FROZEN)
- 7 fields: Decision, Why, Evidence, Confidence, Consequence, Action, Business Impact
- Changes require v1.1 + migration note + revalidation
- Implementation/algorithms NOT frozen
### Status: VALIDATED FOR FIELD TRIALS
- 8 scenarios reviewed (6 PASS, 2 OBSERVATION, 0 FAIL)
- Decision Invariance Test: PASS
- Evidence Variation Test: PASS
- Explainability Invariance Test: PASS
- Ready for real customer testing, not production truth
### Three Target Customer Cases
1. Municipality — "Inspect or wait?" (Maintenance prioritization)
2. Property Owner — "Repair now or plan later?" (Cost vs risk)
3. Contractor/Operations — "Which action first?" (Operational planning)
### Future KPIs (collect now, calculate later)
- **Decision Adoption Rate** = Accepted recommendations / Total recommendations
- **Decision Accuracy** = Correct decisions / Total decisions
- **Time to Decision** = Minutes from login to decision
### Key Principle
**No more modeling until first real customer case has run through entire pipeline.**
### Files
- `docs/design/DECISION_MODEL_v1.0.md` — Decision Model
- `docs/design/DECISION_PIPELINE_v1.0.md` — Decision Pipeline
- `docs/design/DECISION_MODEL_REVIEW.md` — Manual Review Results
- `docs/design/FIELD_TRIAL_LOG.md` — Observation Protocol
+113
View File
@@ -0,0 +1,113 @@
# FIELD TRIAL LOG
**Observation Protocol for Decision Model Validation**
| | |
|---|---|
| **Version** | 1.0 |
| **Status** | ACTIVE |
| **Purpose** | Collect raw data from real customer cases |
---
## Log Entry Template
| Field | Example | Notes |
|-------|---------|-------|
| **Date** | 2026-07-15 | |
| **Customer Type** | Municipality / Property Owner / Contractor | |
| **Decision** | Inspect Road A12 | What the system recommended |
| **Decision Accepted** | Yes / No | Did the user accept? |
| **Action Executed** | Yes / No / Pending | Was the action carried out? |
| **Outcome** | Crack confirmed | What actually happened |
| **Time to Decision** | 3 minutes | How long from login to decision |
| **User Feedback** | "We trusted the recommendation." | Verbatim or summary |
| **Confidence** | 87% | System confidence at decision time |
| **Notes** | Free text | Anything else observed |
---
## Example Entries
### Entry 1
| Field | Value |
|-------|-------|
| Date | 2026-07-15 |
| Customer Type | Municipality |
| Decision | Inspect Road A12 |
| Decision Accepted | Yes |
| Action Executed | Yes |
| Outcome | Crack confirmed, repaired within 14 days |
| Time to Decision | 3 minutes |
| User Feedback | "We trusted the recommendation." |
| Confidence | 87% |
| Notes | User clicked through Explainability chain |
### Entry 2
| Field | Value |
|-------|-------|
| Date | 2026-07-16 |
| Customer Type | Property Owner |
| Decision | Repair facade immediately |
| Decision Accepted | No |
| Action Executed | No |
| Outcome | Deferred, user chose to wait |
| Time to Decision | 5 minutes |
| User Feedback | "Too expensive right now." |
| Confidence | 92% |
| Notes | Confidence was high but user rejected due to cost |
---
## Questions This Log Answers
After collecting entries:
- Were recommendations accepted? (Decision Adoption Rate)
- Were actions executed? (Execution Rate)
- Did outcomes match predictions? (Decision Accuracy)
- Was confidence reasonable? (Calibration)
- Which recommendations were ignored? (Rejection Analysis)
- How long did decisions take? (Time to Decision)
---
## Future KPIs
### Decision Adoption Rate
```
Recommendations accepted
────────────────────────
Total recommendations
```
Example: 72 of 100 = **72%**
### Decision Accuracy
```
Decisions that proved correct
─────────────────────────────
Total decisions
```
Example: 65 of 72 executed = **90%**
**Note:** These require months of data. Start collecting now, calculate later.
---
## Status
**ACTIVE — Awaiting first customer case**
---
## ändringshistoria
| Version | Datum | Beskrivning |
|---------|-------|-------------|
| 1.0 | 2026-07-02 | Initial field trial log template |