docs: Field Trial Log + Decision KPIs + MEMORY update
- Created FIELD_TRIAL_LOG.md — observation protocol for real customer cases - Log entry template with 10 fields - Two example entries (accepted and rejected decisions) - Questions the log answers: adoption, accuracy, calibration, rejection analysis - Future KPIs: Decision Adoption Rate, Decision Accuracy, Time to Decision - Updated MEMORY.md with Control Intelligence & Decision Model section - Decision Pipeline v1.0 summary - Decision Object v1.0 (STRUCTURE FROZEN) - Status: VALIDATED FOR FIELD TRIALS - Three target customer cases - Future KPIs - Key principle: No more modeling until first real customer case - Added Decision Adoption Rate and Decision Accuracy as future product KPIs - Not implemented yet — start collecting data now, calculate later - Decision Adoption = accepted / total recommendations - Decision Accuracy = correct / total decisions Rationale: Stop modeling, start observing. First real customer case will teach more than the last twenty documents combined.
This commit is contained in:
@@ -881,3 +881,49 @@ Mätvärden:
|
|||||||
- Leverabler: Engineering Readiness Score (viktad), Policy Enforcement, STOP Register (med risk), Release Gate, Meta-Rules
|
- Leverabler: Engineering Readiness Score (viktad), Policy Enforcement, STOP Register (med risk), Release Gate, Meta-Rules
|
||||||
- Exit Criteria: Varje STOP är eliminerad, har dispens, eller ersatts av kontrollerad process
|
- Exit Criteria: Varje STOP är eliminerad, har dispens, eller ersatts av kontrollerad process
|
||||||
- EOS-regel: "EOS får aldrig blockera något utan att kunna förklara exakt varför och vilken minsta åtgärd som krävs för att häva blockeringen."
|
- EOS-regel: "EOS får aldrig blockera något utan att kunna förklara exakt varför och vilken minsta åtgärd som krävs för att häva blockeringen."
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. CONTROL INTELLIGENCE & DECISION MODEL (2026-07-02)
|
||||||
|
|
||||||
|
**LandveX produces Control Intelligence, not AI analysis.**
|
||||||
|
|
||||||
|
AI is implementation. Control Intelligence is the product.
|
||||||
|
|
||||||
|
### Decision Pipeline v1.0
|
||||||
|
```
|
||||||
|
Reality → Observation → Evidence → Finding → Decision → Action → Business Impact → Learning
|
||||||
|
```
|
||||||
|
|
||||||
|
**Seven steps with input/transformation/output/owner.**
|
||||||
|
|
||||||
|
### Decision Object v1.0 (STRUCTURE FROZEN)
|
||||||
|
- 7 fields: Decision, Why, Evidence, Confidence, Consequence, Action, Business Impact
|
||||||
|
- Changes require v1.1 + migration note + revalidation
|
||||||
|
- Implementation/algorithms NOT frozen
|
||||||
|
|
||||||
|
### Status: VALIDATED FOR FIELD TRIALS
|
||||||
|
- 8 scenarios reviewed (6 PASS, 2 OBSERVATION, 0 FAIL)
|
||||||
|
- Decision Invariance Test: PASS
|
||||||
|
- Evidence Variation Test: PASS
|
||||||
|
- Explainability Invariance Test: PASS
|
||||||
|
- Ready for real customer testing, not production truth
|
||||||
|
|
||||||
|
### Three Target Customer Cases
|
||||||
|
1. Municipality — "Inspect or wait?" (Maintenance prioritization)
|
||||||
|
2. Property Owner — "Repair now or plan later?" (Cost vs risk)
|
||||||
|
3. Contractor/Operations — "Which action first?" (Operational planning)
|
||||||
|
|
||||||
|
### Future KPIs (collect now, calculate later)
|
||||||
|
- **Decision Adoption Rate** = Accepted recommendations / Total recommendations
|
||||||
|
- **Decision Accuracy** = Correct decisions / Total decisions
|
||||||
|
- **Time to Decision** = Minutes from login to decision
|
||||||
|
|
||||||
|
### Key Principle
|
||||||
|
**No more modeling until first real customer case has run through entire pipeline.**
|
||||||
|
|
||||||
|
### Files
|
||||||
|
- `docs/design/DECISION_MODEL_v1.0.md` — Decision Model
|
||||||
|
- `docs/design/DECISION_PIPELINE_v1.0.md` — Decision Pipeline
|
||||||
|
- `docs/design/DECISION_MODEL_REVIEW.md` — Manual Review Results
|
||||||
|
- `docs/design/FIELD_TRIAL_LOG.md` — Observation Protocol
|
||||||
|
|||||||
@@ -0,0 +1,113 @@
|
|||||||
|
# FIELD TRIAL LOG
|
||||||
|
|
||||||
|
**Observation Protocol for Decision Model Validation**
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| **Version** | 1.0 |
|
||||||
|
| **Status** | ACTIVE |
|
||||||
|
| **Purpose** | Collect raw data from real customer cases |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Log Entry Template
|
||||||
|
|
||||||
|
| Field | Example | Notes |
|
||||||
|
|-------|---------|-------|
|
||||||
|
| **Date** | 2026-07-15 | |
|
||||||
|
| **Customer Type** | Municipality / Property Owner / Contractor | |
|
||||||
|
| **Decision** | Inspect Road A12 | What the system recommended |
|
||||||
|
| **Decision Accepted** | Yes / No | Did the user accept? |
|
||||||
|
| **Action Executed** | Yes / No / Pending | Was the action carried out? |
|
||||||
|
| **Outcome** | Crack confirmed | What actually happened |
|
||||||
|
| **Time to Decision** | 3 minutes | How long from login to decision |
|
||||||
|
| **User Feedback** | "We trusted the recommendation." | Verbatim or summary |
|
||||||
|
| **Confidence** | 87% | System confidence at decision time |
|
||||||
|
| **Notes** | Free text | Anything else observed |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Example Entries
|
||||||
|
|
||||||
|
### Entry 1
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
|-------|-------|
|
||||||
|
| Date | 2026-07-15 |
|
||||||
|
| Customer Type | Municipality |
|
||||||
|
| Decision | Inspect Road A12 |
|
||||||
|
| Decision Accepted | Yes |
|
||||||
|
| Action Executed | Yes |
|
||||||
|
| Outcome | Crack confirmed, repaired within 14 days |
|
||||||
|
| Time to Decision | 3 minutes |
|
||||||
|
| User Feedback | "We trusted the recommendation." |
|
||||||
|
| Confidence | 87% |
|
||||||
|
| Notes | User clicked through Explainability chain |
|
||||||
|
|
||||||
|
### Entry 2
|
||||||
|
|
||||||
|
| Field | Value |
|
||||||
|
|-------|-------|
|
||||||
|
| Date | 2026-07-16 |
|
||||||
|
| Customer Type | Property Owner |
|
||||||
|
| Decision | Repair facade immediately |
|
||||||
|
| Decision Accepted | No |
|
||||||
|
| Action Executed | No |
|
||||||
|
| Outcome | Deferred, user chose to wait |
|
||||||
|
| Time to Decision | 5 minutes |
|
||||||
|
| User Feedback | "Too expensive right now." |
|
||||||
|
| Confidence | 92% |
|
||||||
|
| Notes | Confidence was high but user rejected due to cost |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Questions This Log Answers
|
||||||
|
|
||||||
|
After collecting entries:
|
||||||
|
|
||||||
|
- Were recommendations accepted? (Decision Adoption Rate)
|
||||||
|
- Were actions executed? (Execution Rate)
|
||||||
|
- Did outcomes match predictions? (Decision Accuracy)
|
||||||
|
- Was confidence reasonable? (Calibration)
|
||||||
|
- Which recommendations were ignored? (Rejection Analysis)
|
||||||
|
- How long did decisions take? (Time to Decision)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Future KPIs
|
||||||
|
|
||||||
|
### Decision Adoption Rate
|
||||||
|
|
||||||
|
```
|
||||||
|
Recommendations accepted
|
||||||
|
────────────────────────
|
||||||
|
Total recommendations
|
||||||
|
```
|
||||||
|
|
||||||
|
Example: 72 of 100 = **72%**
|
||||||
|
|
||||||
|
### Decision Accuracy
|
||||||
|
|
||||||
|
```
|
||||||
|
Decisions that proved correct
|
||||||
|
─────────────────────────────
|
||||||
|
Total decisions
|
||||||
|
```
|
||||||
|
|
||||||
|
Example: 65 of 72 executed = **90%**
|
||||||
|
|
||||||
|
**Note:** These require months of data. Start collecting now, calculate later.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
**ACTIVE — Awaiting first customer case**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ändringshistoria
|
||||||
|
|
||||||
|
| Version | Datum | Beskrivning |
|
||||||
|
|---------|-------|-------------|
|
||||||
|
| 1.0 | 2026-07-02 | Initial field trial log template |
|
||||||
Reference in New Issue
Block a user