docs: Decision Model v1.0 — 7 validation scenarios + invariance tests
- Added 7 validation scenarios covering diverse decision types: 1. Road Crack (Maintenance) — repair now or later? 2. Damaged Facade (Safety) — act immediately? 3. Broken Road Sign (Compliance) — violates requirements? 4. Vegetation Blocking Sight (Risk Reduction) — gradual deterioration 5. Parking Area Wear (Investment Priority) — multiple small → big decision 6. Cosmetic Scratch (No Action) — conscious decision to wait 7. Mixed Evidence Sources — photo + sensor + weather API - Decision Invariance Test: - Same Decision Object structure across all scenarios? - No fields added/removed? - No field meaning changes? - Fail = model needs revision - Evidence Variation Test: - Single image, multiple images, video+GPS, historical, external data, mixed - Decision Object structure unchanged regardless of evidence type - Decision Quality Gate: - Verifiable evidence chain - Motivated confidence - Action or conscious 'no action' - Explainability chain works - Pass criteria: All 7 scenarios valid + invariance + evidence + quality gate Rationale: Validate model against diverse decision types before freezing. No action scenario is as important as action scenarios. Mixed evidence sources test robustness. Invariance test ensures generality.
This commit is contained in:
Bernt
@@ -198,9 +198,9 @@ For every decision:
|
||||
|
||||
## Validation Scenarios
|
||||
|
||||
Before freezing Decision Model v1.0, validate against three real scenarios:
|
||||
Before freezing Decision Model v1.0, validate against diverse scenarios:
|
||||
|
||||
### Scenario 1: Road Crack
|
||||
### Scenario 1: Road Crack (Maintenance)
|
||||
|
||||
| Layer | Example |
|
||||
|-------|---------|
|
||||
@@ -211,7 +211,9 @@ Before freezing Decision Model v1.0, validate against three real scenarios:
|
||||
| Decision | "Prioritize inspection within 14 days" |
|
||||
| Business Impact | Risk: High, Cost: $50k avoided, Time: 14 days, Opportunity: Plan with nearby works |
|
||||
|
||||
### Scenario 2: Damaged Facade
|
||||
**Decision type:** Maintenance — "Repair now or later?"
|
||||
|
||||
### Scenario 2: Damaged Facade (Safety)
|
||||
|
||||
| Layer | Example |
|
||||
|-------|---------|
|
||||
@@ -222,7 +224,9 @@ Before freezing Decision Model v1.0, validate against three real scenarios:
|
||||
| Decision | "Immediate safety inspection required" |
|
||||
| Business Impact | Risk: Critical, Cost: $200k liability, Time: 24 hours, Opportunity: Prevent injury |
|
||||
|
||||
### Scenario 3: Broken Road Sign
|
||||
**Decision type:** Safety — "Act immediately?"
|
||||
|
||||
### Scenario 3: Broken Road Sign (Compliance)
|
||||
|
||||
| Layer | Example |
|
||||
|-------|---------|
|
||||
@@ -233,7 +237,114 @@ Before freezing Decision Model v1.0, validate against three real scenarios:
|
||||
| Decision | "Replace sign within 48 hours" |
|
||||
| Business Impact | Risk: Medium, Cost: $5k fine avoided, Time: 48 hours, Opportunity: Standard replacement |
|
||||
|
||||
**Pass criteria:** Same Decision Object works for all three without modification.
|
||||
**Decision type:** Compliance — "Does this violate requirements?"
|
||||
|
||||
### Scenario 4: Vegetation Blocking Sight (Risk Reduction)
|
||||
|
||||
| Layer | Example |
|
||||
|-------|---------|
|
||||
| Reality | Mobile photo of overgrown vegetation |
|
||||
| Observation | "Vegetation 80cm high, blocking sight line" |
|
||||
| Evidence | 2 observations over 3 months + growth trend |
|
||||
| Finding | "Gradual degradation of sight lines at intersection" |
|
||||
| Decision | "Schedule vegetation removal within 30 days" |
|
||||
| Business Impact | Risk: Medium, Cost: $15k avoided, Time: 30 days, Opportunity: Coordinate with seasonal maintenance |
|
||||
|
||||
**Decision type:** Risk reduction — "Gradual deterioration requiring planned action"
|
||||
|
||||
### Scenario 5: Parking Area Wear (Investment Priority)
|
||||
|
||||
| Layer | Example |
|
||||
|-------|---------|
|
||||
| Reality | Multiple mobile photos of parking area |
|
||||
| Observation | "Surface wear, potholes, faded markings" |
|
||||
| Evidence | 5 observations + usage data + weather exposure |
|
||||
| Finding | "Multiple minor issues collectively indicate need for resurfacing" |
|
||||
| Decision | "Include in next year's maintenance budget" |
|
||||
| Business Impact | Risk: Low, Cost: $100k investment, Time: 6 months, Opportunity: Improve user satisfaction |
|
||||
|
||||
**Decision type:** Investment priority — "Multiple small observations motivating larger decision"
|
||||
|
||||
### Scenario 6: Cosmetic Scratch (No Action)
|
||||
|
||||
| Layer | Example |
|
||||
|-------|---------|
|
||||
| Reality | Mobile photo of road sign |
|
||||
| Observation | "Minor cosmetic scratches, 5% of surface" |
|
||||
| Evidence | 1 observation, no functional impact |
|
||||
| Finding | "Normal wear and tear, no safety or compliance impact" |
|
||||
| Decision | "No action recommended. Continue monitoring." |
|
||||
| Business Impact | Risk: None, Cost: $0, Time: Annual review, Opportunity: None |
|
||||
|
||||
**Decision type:** No action — "Conscious decision to wait"
|
||||
|
||||
### Scenario 7: Mixed Evidence Sources
|
||||
|
||||
| Layer | Example |
|
||||
|-------|---------|
|
||||
| Reality | Mobile photo + sensor data + weather API |
|
||||
| Observation | "Water pooling, 3cm depth, after rainfall" |
|
||||
| Evidence | Photo + rain sensor + historical flooding data + GIS topography |
|
||||
| Finding | "Drainage inadequate, recurring flooding risk" |
|
||||
| Decision | "Inspect drainage system, prioritize if flooding recurs" |
|
||||
| Business Impact | Risk: Medium, Cost: $30k avoided, Time: 14 days, Opportunity: Permanent fix during dry season |
|
||||
|
||||
**Decision type:** Complex — "Multiple evidence sources converging"
|
||||
|
||||
---
|
||||
|
||||
## Decision Invariance Test
|
||||
|
||||
For each scenario, verify:
|
||||
|
||||
| Question | Pass Criteria |
|
||||
|----------|---------------|
|
||||
| Same Decision Object structure? | All 7 fields present |
|
||||
| Any field added? | No new fields needed |
|
||||
| Any field always empty? | No field unused across scenarios |
|
||||
| Any field meaning different things? | Each field has consistent meaning |
|
||||
|
||||
**Fail criteria:** If any question answers "no", the model needs revision.
|
||||
|
||||
---
|
||||
|
||||
## Evidence Variation Test
|
||||
|
||||
| Evidence Type | Scenario | Pass |
|
||||
|---------------|----------|------|
|
||||
| Single image | Scenario 3 | ✅ |
|
||||
| Multiple images | Scenario 1 | ✅ |
|
||||
| Video + GPS | Scenario 2 | ✅ |
|
||||
| Historical observations | Scenario 4 | ✅ |
|
||||
| External data (weather, traffic) | Scenario 7 | ✅ |
|
||||
| Mixed sources | Scenario 7 | ✅ |
|
||||
|
||||
**Pass criteria:** Decision Object structure unchanged regardless of evidence type.
|
||||
|
||||
---
|
||||
|
||||
## Decision Quality Gate
|
||||
|
||||
Before displaying any decision to the user:
|
||||
|
||||
- [ ] At least one verifiable evidence chain exists
|
||||
- [ ] Confidence is motivated (not arbitrary)
|
||||
- [ ] Recommended action exists OR conscious "no action" decision
|
||||
- [ ] Explainability chain works (can click back to observations)
|
||||
|
||||
---
|
||||
|
||||
## Pass Criteria
|
||||
|
||||
**Decision Model v1.0 is validated when:**
|
||||
|
||||
1. All 7 scenarios produce valid Decision Objects
|
||||
2. Decision Invariance Test passes
|
||||
3. Evidence Variation Test passes
|
||||
4. Decision Quality Gate passes for all scenarios
|
||||
5. "No action" scenario works correctly
|
||||
|
||||
**Status:** ⏳ Pending validation
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user