Developer documentation

The condition cards run versioned JSON model files through one scorer. You can call the scorer from an EHR through the CDS Hooks service, launch the clinician view as a SMART on FHIR app, or run the scorer and the model files in your own code. Synthetic test patients with expected outputs let you check any implementation.

How the pieces fit

Model files

longcovid.json 1.2.0
chronicfatigue.json 1.0.0
fibromyalgia.json 1.0.0
endometriosis.json 1.0.0
lyme.json 1.0.0
lupus.json 1.0.0
migraine.json 1.0.0
depression.json 1.0.0
anxiety.json 1.0.0

JSON: features with their codes, windows, and weights, a calibration map, and risk bands.

Scorer

score_fhir.py, the Python reference
scorer.js, its JavaScript port

In: a model file, a FHIR R4 Bundle, and a scoring date. Out: status, probability, band, and the dated findings behind the score.

Where it runs

CDS Hooks service
SMART on FHIR app
Your code

The service and the apps import the same scorer.js.

Test vectors: 18 synthetic FHIR R4 patients with the reference scorer's output for each model. The JavaScript scorer reproduces every one to 1e-9.

Try the scorer

This runs /shared/scorer.js in your browser on a synthetic test patient and compares the result with the reference scorer's expected output. It reads only files from this site.

CDS Hooks service

A stateless CDS Hooks 2.0 service for the patient-view hook. It scores the prefetched chart with the Long COVID 1.2.0, Chronic fatigue and ME/CFS 1.0.0, Fibromyalgia 1.0.0, Endometriosis 1.0.0, Lyme 1.0.0, Lupus 1.0.0, Migraine 1.0.0, Depression 1.0.0, and Anxiety 1.0.0 model files and answers with one card for each condition in the High band (indicator warning) or the Elevated band (info), plus one info card naming every condition it could not score. Typical and not-scored results give no card, so a chart with nothing to flag gets {"cards": []}.

The service runs at https://undiagnosednomore.org:

Method	Path	Answer
GET	`/cds-services`	The discovery document
POST	`/cds-services/invisible-illness`	`{"cards": [...]}`
OPTIONS	both	204 with the CORS headers (`Access-Control-Allow-Origin: *`), so browser clients such as the CDS Hooks sandbox can call the service

1. Discovery

The service id is invisible-illness. The prefetch templates ask for the Patient and one search each for Condition, Encounter, Observation, MedicationRequest, and Procedure.

bash

curl -s https://undiagnosednomore.org/cds-services

Response

{
  "services": [
    {
      "hook": "patient-view",
      "id": "invisible-illness",
      "title": "Undiagnosed No More: invisible illness check",
      "description": "Scores the chart with the Long COVID, chronic fatigue (ME/CFS), fibromyalgia, endometriosis, Lyme disease, lupus, migraine, depression, and anxiety models and returns a card when a condition is in the High or Elevated band, plus one information card naming any condition it could not score. Research decision support; not a diagnosis. Synthetic-data prototype: the service does not authenticate callers.",
      "prefetch": {
        "patient": "Patient/{{context.patientId}}",
        "conditions": "Condition?patient={{context.patientId}}",
        "encounters": "Encounter?patient={{context.patientId}}",
        "observations": "Observation?patient={{context.patientId}}",
        "medications": "MedicationRequest?patient={{context.patientId}}",
        "procedures": "Procedure?patient={{context.patientId}}"
      },
      "usageRequirements": "Read access to the patient's Condition, Encounter, Observation, MedicationRequest, and Procedure resources."
    }
  ]
}

2. Request

Field	Required	What the service does with it
`hook`	yes	Must be `patient-view`; any other hook gets 400.
`hookInstance`	yes	A nonempty string (the examples use a UUID).
`context.patientId`	yes	A FHIR id. The prefetched Patient must have this id, and every prefetched resource whose `subject` or `patient` names a patient must name this one; otherwise 400.
`context.userId`	no	A string if present; not used for scoring.
`prefetch`	no	The keys from discovery. `patient` holds the Patient resource; the other keys hold searchset Bundles.
`fhirServer`, `fhirAuthorization`	no	Used only to read prefetch keys the client did not send and the `next` pages of prefetched searches, and only when `fhirServer` is on the deployment's allow-list. The token is sent as a bearer token. The default list holds only the synthetic SMART Health IT sandbox, `https://launch.smarthealthit.org/v/r4/`.
`extension["org.undiagnosednomore.scoring-date"]`	no	The scoring date, `YYYY-MM-DD`; without it the service scores as of today (UTC). The examples set June 1, 2024, the date of the test vectors.

The example requests are test patients from the test vectors with the prefetch the service asks for: maria.json, elena.json, and maria-missing-observations.json. To send one:

bash

curl -s https://undiagnosednomore.org/developers/examples/maria.json |
  curl -s -X POST https://undiagnosednomore.org/cds-services/invisible-illness \
    -H "Content-Type: application/json" --data-binary @-

The whole request, maria.json (754 lines)

maria.json

{
 "hookInstance": "00000000-0000-4000-8000-000000000522",
 "hook": "patient-view",
 "context": {
  "userId": "Practitioner/example",
  "patientId": "maria"
 },
 "extension": {
  "org.undiagnosednomore.scoring-date": "2024-06-01"
 },
 "prefetch": {
  "patient": {
   "resourceType": "Patient",
   "id": "maria",
   "gender": "female",
   "birthDate": "1980-03-02",
   "name": [
    {
     "family": "Synthetic",
     "given": [
      "Maria"
     ]
    }
   ]
  },
  "conditions": {
   "resourceType": "Bundle",
   "type": "searchset",
   "total": 9,
   "entry": [
    {
     "fullUrl": "urn:uuid:c4029410-e7b9-5f61-a3d3-333bc5b2fad0",
     "resource": {
      "resourceType": "Condition",
      "id": "maria-cond-4",
      "clinicalStatus": {
       "coding": [
        {
         "code": "active"
        }
       ]
      },
      "code": {
       "coding": [
        {
         "system": "http://hl7.org/fhir/sid/icd-10-cm",
         "code": "J45.909",
         "display": "Asthma, unspecified, uncomplicated"
        }
       ],
       "text": "Asthma, unspecified, uncomplicated"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "onsetDateTime": "2023-06-15",
      "recordedDate": "2023-06-15"
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:e40c2059-f000-56ae-9423-801c5b37874d",
     "resource": {
      "resourceType": "Condition",
      "id": "maria-cond-5",
      "clinicalStatus": {
       "coding": [
        {
         "code": "active"
        }
       ]
      },
      "code": {
       "coding": [
        {
         "system": "http://hl7.org/fhir/sid/icd-10-cm",
         "code": "U07.1",
         "display": "COVID-19"
        }
       ],
       "text": "COVID-19"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "onsetDateTime": "2024-01-08",
      "recordedDate": "2024-01-08"
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:25ed6b87-55a4-56bd-a916-8984df6328aa",
     "resource": {
      "resourceType": "Condition",
      "id": "maria-cond-10",
      "clinicalStatus": {
       "coding": [
        {
         "code": "active"
        }
       ]
      },
      "code": {
       "coding": [
        {
         "system": "http://hl7.org/fhir/sid/icd-10-cm",
         "code": "R53.83",
         "display": "Other fatigue"
        }
       ],
       "text": "Other fatigue"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "onsetDateTime": "2024-02-20",
      "recordedDate": "2024-02-20"
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:c321d403-2315-5aeb-82ac-2166125a8550",
     "resource": {
      "resourceType": "Condition",
      "id": "maria-cond-11",
      "clinicalStatus": {
       "coding": [
        {
         "code": "active"
        }
       ]
      },
      "code": {
       "coding": [
        {
         "system": "http://hl7.org/fhir/sid/icd-10-cm",
         "code": "R00.2",
         "display": "Palpitations"
        }
       ],
       "text": "Palpitations"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "onsetDateTime": "2024-02-20",
      "recordedDate": "2024-02-20"
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:15db45b3-5856-52b6-a29a-d0961a7c0540",
     "resource": {
      "resourceType": "Condition",
      "id": "maria-cond-12",
      "clinicalStatus": {
       "coding": [
        {
         "code": "active"
        }
       ]
      },
      "code": {
       "coding": [
        {
         "system": "http://hl7.org/fhir/sid/icd-10-cm",
         "code": "U07.1",
         "display": "COVID-19"
        }
       ],
       "text": "COVID-19"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "onsetDateTime": "2024-02-20",
      "recordedDate": "2024-02-20"
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:29ad0527-8368-584c-8648-0c78a2237840",
     "resource": {
      "resourceType": "Condition",
      "id": "maria-cond-14",
      "clinicalStatus": {
       "coding": [
        {
         "code": "active"
        }
       ]
      },
      "code": {
       "coding": [
        {
         "system": "http://hl7.org/fhir/sid/icd-10-cm",
         "code": "Z86.16",
         "display": "Personal history of COVID-19"
        }
       ],
       "text": "Personal history of COVID-19"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "onsetDateTime": "2024-03-12",
      "recordedDate": "2024-03-12"
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:0cd08300-2efa-5b4c-b09e-7caec0a526fd",
     "resource": {
      "resourceType": "Condition",
      "id": "maria-cond-16",
      "clinicalStatus": {
       "coding": [
        {
         "code": "active"
        }
       ]
      },
      "code": {
       "coding": [
        {
         "system": "http://hl7.org/fhir/sid/icd-10-cm",
         "code": "R41.840",
         "display": "Attention and concentration deficit"
        }
       ],
       "text": "Attention and concentration deficit"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "onsetDateTime": "2024-03-05",
      "recordedDate": "2024-03-05"
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:22962e0f-227f-5cfd-804d-0351e1f7771a",
     "resource": {
      "resourceType": "Condition",
      "id": "maria-cond-18",
      "clinicalStatus": {
       "coding": [
        {
         "code": "active"
        }
       ]
      },
      "code": {
       "coding": [
        {
         "system": "http://hl7.org/fhir/sid/icd-10-cm",
         "code": "R06.02",
         "display": "Shortness of breath"
        }
       ],
       "text": "Shortness of breath"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "onsetDateTime": "2024-03-25",
      "recordedDate": "2024-03-25"
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:fc529004-fe7f-52c2-bfd9-578bcb712127",
     "resource": {
      "resourceType": "Condition",
      "id": "maria-cond-19",
      "clinicalStatus": {
       "coding": [
        {
         "code": "active"
        }
       ]
      },
      "code": {
       "coding": [
        {
         "system": "http://hl7.org/fhir/sid/icd-10-cm",
         "code": "R07.89",
         "display": "Other chest pain"
        }
       ],
       "text": "Other chest pain"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "onsetDateTime": "2024-03-25",
      "recordedDate": "2024-03-25"
     },
     "search": {
      "mode": "match"
     }
    }
   ]
  },
  "encounters": {
   "resourceType": "Bundle",
   "type": "searchset",
   "total": 9,
   "entry": [
    {
     "fullUrl": "urn:uuid:adb2a6dd-a1b8-5b3a-bf44-9dc9cf54bdd6",
     "resource": {
      "resourceType": "Encounter",
      "id": "maria-enc-1",
      "status": "finished",
      "class": {
       "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
       "code": "AMB"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "period": {
       "start": "2023-02-10"
      }
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:84e09ad6-f8df-5fbf-ab83-95eafd021ba4",
     "resource": {
      "resourceType": "Encounter",
      "id": "maria-enc-2",
      "status": "finished",
      "class": {
       "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
       "code": "AMB"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "period": {
       "start": "2023-06-15"
      }
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:ba0e60d3-b9ee-5a83-add0-d94e660c5e4b",
     "resource": {
      "resourceType": "Encounter",
      "id": "maria-enc-3",
      "status": "finished",
      "class": {
       "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
       "code": "AMB"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "period": {
       "start": "2023-10-20"
      }
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:285fdafa-f1ce-554c-8c61-3a1152318c80",
     "resource": {
      "resourceType": "Encounter",
      "id": "maria-enc-6",
      "status": "finished",
      "class": {
       "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
       "code": "AMB"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "period": {
       "start": "2024-01-08"
      }
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:d9853640-8f56-5b22-b957-e8bf8dbc8ccb",
     "resource": {
      "resourceType": "Encounter",
      "id": "maria-enc-7",
      "status": "finished",
      "class": {
       "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
       "code": "EMER"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "period": {
       "start": "2024-01-10"
      }
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:8bfce810-ceab-538a-bd94-01696606b10f",
     "resource": {
      "resourceType": "Encounter",
      "id": "maria-enc-9",
      "status": "finished",
      "class": {
       "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
       "code": "AMB"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "period": {
       "start": "2024-02-20"
      }
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:ec711b65-6042-54d2-8964-6d3d0ae76a7a",
     "resource": {
      "resourceType": "Encounter",
      "id": "maria-enc-13",
      "status": "finished",
      "class": {
       "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
       "code": "AMB"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "period": {
       "start": "2024-03-12"
      }
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:3fb48051-8bac-5463-b0e6-88779109a284",
     "resource": {
      "resourceType": "Encounter",
      "id": "maria-enc-15",
      "status": "finished",
      "class": {
       "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
       "code": "AMB"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "period": {
       "start": "2024-03-05"
      }
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:62bda78f-1085-5311-aab4-853d7b51944c",
     "resource": {
      "resourceType": "Encounter",
      "id": "maria-enc-17",
      "status": "finished",
      "class": {
       "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
       "code": "AMB"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "period": {
       "start": "2024-03-25"
      }
     },
     "search": {
      "mode": "match"
     }
    }
   ]
  },
  "observations": {
   "resourceType": "Bundle",
   "type": "searchset",
   "total": 3,
   "entry": [
    {
     "fullUrl": "urn:uuid:b7535a4c-9bc0-52b5-9932-ef0f1da743a6",
     "resource": {
      "resourceType": "Observation",
      "id": "maria-obs-8",
      "status": "final",
      "category": [
       {
        "coding": [
         {
          "code": "laboratory"
         }
        ]
       }
      ],
      "code": {
       "coding": [
        {
         "system": "http://loinc.org",
         "code": "48065-7",
         "display": "Fibrin D-dimer FEU"
        }
       ],
       "text": "Fibrin D-dimer FEU"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "effectiveDateTime": "2024-01-10",
      "valueQuantity": {
       "value": 0.4,
       "unit": "ug/mL"
      }
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:e40530c0-c65f-5138-9eed-0ab0a22a4c3f",
     "resource": {
      "resourceType": "Observation",
      "id": "maria-obs-20",
      "status": "final",
      "category": [
       {
        "coding": [
         {
          "code": "laboratory"
         }
        ]
       }
      ],
      "code": {
       "coding": [
        {
         "system": "http://loinc.org",
         "code": "1988-5",
         "display": "C reactive protein"
        }
       ],
       "text": "C reactive protein"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "effectiveDateTime": "2024-02-20",
      "valueQuantity": {
       "value": 4.1,
       "unit": "mg/L"
      }
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:6d04ac0f-ca63-5bb3-9945-5800badffb33",
     "resource": {
      "resourceType": "Observation",
      "id": "maria-obs-25",
      "status": "final",
      "category": [
       {
        "coding": [
         {
          "code": "laboratory"
         }
        ]
       }
      ],
      "code": {
       "coding": [
        {
         "system": "http://loinc.org",
         "code": "3016-3",
         "display": "Thyrotropin"
        }
       ],
       "text": "Thyrotropin"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "effectiveDateTime": "2024-02-20",
      "valueQuantity": {
       "value": 1.8,
       "unit": "mIU/L"
      }
     },
     "search": {
      "mode": "match"
     }
    }
   ]
  },
  "medications": {
   "resourceType": "Bundle",
   "type": "searchset",
   "total": 1,
   "entry": [
    {
     "fullUrl": "urn:uuid:9a7a01a2-33c5-5a99-b389-798e665b3dda",
     "resource": {
      "resourceType": "MedicationRequest",
      "id": "maria-med-24",
      "status": "active",
      "intent": "order",
      "medicationCodeableConcept": {
       "coding": [
        {
         "system": "http://www.nlm.nih.gov/research/umls/rxnorm",
         "code": "8787",
         "display": "propranolol"
        }
       ],
       "text": "propranolol"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "authoredOn": "2024-03-12"
     },
     "search": {
      "mode": "match"
     }
    }
   ]
  },
  "procedures": {
   "resourceType": "Bundle",
   "type": "searchset",
   "total": 3,
   "entry": [
    {
     "fullUrl": "urn:uuid:008ee8e8-a403-5a66-b5f0-801fd0c98751",
     "resource": {
      "resourceType": "Procedure",
      "id": "maria-proc-21",
      "status": "completed",
      "code": {
       "coding": [
        {
         "system": "http://www.ama-assn.org/go/cpt",
         "code": "71275",
         "display": "CT angiography, chest"
        }
       ],
       "text": "CT angiography, chest"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "performedDateTime": "2024-03-27"
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:22e76223-e4aa-57a0-83d8-ee14d1be4a1f",
     "resource": {
      "resourceType": "Procedure",
      "id": "maria-proc-22",
      "status": "completed",
      "code": {
       "coding": [
        {
         "system": "http://www.ama-assn.org/go/cpt",
         "code": "93306",
         "display": "Echocardiography, transthoracic, complete"
        }
       ],
       "text": "Echocardiography, transthoracic, complete"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "performedDateTime": "2024-03-01"
     },
     "search": {
      "mode": "match"
     }
    },
    {
     "fullUrl": "urn:uuid:c3e73679-33f9-560c-bbbc-48d1e2ea32aa",
     "resource": {
      "resourceType": "Procedure",
      "id": "maria-proc-23",
      "status": "completed",
      "code": {
       "coding": [
        {
         "system": "http://www.ama-assn.org/go/cpt",
         "code": "93224",
         "display": "External electrocardiographic recording up to 48 hours"
        }
       ],
       "text": "External electrocardiographic recording up to 48 hours"
      },
      "subject": {
       "reference": "Patient/maria"
      },
      "performedDateTime": "2024-03-10"
     },
     "search": {
      "mode": "match"
     }
    }
   ]
  }
 }
}

3. Response

Card field	Value
`summary`	At most 139 characters: the condition, the band, the multiple of the average, and "Review the dated findings."
`indicator`	`warning` for High; `info` for Elevated and for the could-not-score card. High cards come first.
`detail`	Markdown: the chart pattern, the probability and band, the top three findings with their dates, the next step, and the model id and version.
`source`	`{"label": "Undiagnosed No More (research decision support)", "url": "https://undiagnosednomore.org/developers/"}`
`links`	One link of `type` `smart` to `https://undiagnosednomore.org/smart/`.

The service's answers to the three example requests, generated by running its code on them:

maria.json. Maria's expected results are Long COVID: High; Chronic fatigue and ME/CFS: Typical; Fibromyalgia: Typical; Endometriosis: Typical; Lyme: Typical; Lupus: Typical; Migraine: Typical; Depression: Typical; Anxiety: Typical. The service returns one card.

Response to maria.json

{
  "cards": [
    {
      "summary": "Long COVID: High chart pattern, 6.1× the post-COVID average. Review the dated findings.",
      "indicator": "warning",
      "detail": "**Chart pattern similar to adults later coded with Long COVID.**\n\nEstimated probability, as of day 90, of a Long COVID code 91 to 455 days after COVID-19: 6.2% (6.1× the average for adults 90 days after COVID-19). Band: High (a descriptive score category, not a validated action threshold).\n\nTop findings in the chart:\n\n1. 3+ symptom types coded in days 31-90 (2024-02-20, 2024-03-05, 2024-03-25)\n2. Personal history of COVID-19 coded (days 31-90) (2024-03-12)\n3. COVID-19 still coded at a follow-up visit (days 31-90) (2024-02-20)\n\nNext step: Review the dated findings and ask the patient to complete the RECOVER symptom list, the DePaul post-exertional malaise screen, and the PCFS functional status grade. If the review supports post-COVID illness, consider a post-COVID clinic referral or clinical trial screening.\n\nResearch decision support. Surfaces chart patterns for clinician review. Not a diagnosis. Model unm-longcovid 1.2.0.",
      "source": {
        "label": "Undiagnosed No More (research decision support)",
        "url": "https://undiagnosednomore.org/developers/"
      },
      "links": [
        {
          "label": "Open Undiagnosed No More (SMART app)",
          "url": "https://undiagnosednomore.org/smart/",
          "type": "smart"
        }
      ]
    }
  ]
}

elena.json. Elena's expected results are Long COVID: not scored; Chronic fatigue and ME/CFS: High; Fibromyalgia: Typical; Endometriosis: Typical; Lyme: Typical; Lupus: Typical; Migraine: Elevated; Depression: Elevated; Anxiety: Elevated. The service returns four cards.

Response to elena.json

{
  "cards": [
    {
      "summary": "Chronic fatigue or ME/CFS: High chart pattern, 3.3× the average. Review the dated findings.",
      "indicator": "warning",
      "detail": "**Chart pattern similar to patients who report a chronic fatigue or ME/CFS diagnosis that was never coded.**\n\nProbability that the patient reports a chronic fatigue diagnosis: 17% (3.3× the average for adults with a visit in the past year). Band: High (a descriptive score category, not a validated action threshold).\n\nTop findings in the chart:\n\n1. Five or more symptom types (6 dates, 2020-05-12 to 2023-01-17)\n2. POTS medication (2022-09-28)\n3. Tilt-table test (2022-09-14)\n\nNext step: Ask the patient about post-exertional malaise (DePaul screen) and review the IOM 2015 ME/CFS criteria: reduced activity for more than 6 months with fatigue, post-exertional malaise, unrefreshing sleep, and cognitive impairment or orthostatic intolerance.\n\nResearch decision support. Surfaces chart patterns for clinician review. Not a diagnosis. Model unm-chronicfatigue 1.0.0.",
      "source": {
        "label": "Undiagnosed No More (research decision support)",
        "url": "https://undiagnosednomore.org/developers/"
      },
      "links": [
        {
          "label": "Open Undiagnosed No More (SMART app)",
          "url": "https://undiagnosednomore.org/smart/",
          "type": "smart"
        }
      ]
    },
    {
      "summary": "Migraine: Elevated chart pattern, 1.7× the average. Review the dated findings.",
      "indicator": "info",
      "detail": "**Chart pattern similar to patients who report a migraine diagnosis that was never coded.**\n\nProbability that the patient reports a migraine diagnosis: 23% (1.7× the average for adults with a visit in the past year). Band: Elevated (a descriptive score category, not a validated action threshold).\n\nTop findings in the chart:\n\n1. Female (patient demographics)\n2. Stimulant or wakefulness drug (2023-05-09)\n3. POTS medication (2022-09-28)\n\nNext step: Review the headaches against the ICHD-3 criteria for migraine (frequency, duration, pain features, and nausea or sensitivity to light and sound); if they fit, discuss acute and preventive treatment.\n\nResearch decision support. Surfaces chart patterns for clinician review. Not a diagnosis. Model unm-migraine 1.0.0.",
      "source": {
        "label": "Undiagnosed No More (research decision support)",
        "url": "https://undiagnosednomore.org/developers/"
      },
      "links": [
        {
          "label": "Open Undiagnosed No More (SMART app)",
          "url": "https://undiagnosednomore.org/smart/",
          "type": "smart"
        }
      ]
    },
    {
      "summary": "Depression: Elevated chart pattern, 1.6× the average. Review the dated findings.",
      "indicator": "info",
      "detail": "**Chart pattern similar to patients who report a depression diagnosis that was never coded.**\n\nProbability that the patient reports a depression diagnosis: 33% (1.6× the average for adults with a visit in the past year). Band: Elevated (a descriptive score category, not a validated action threshold).\n\nTop findings in the chart:\n\n1. Stimulant or wakefulness drug (2023-05-09)\n2. Age 18-39 (patient demographics)\n3. Female (patient demographics)\n\nNext step: Screen with the PHQ-9 (USPSTF 2023); if it is 10 or more, assess for major depression and ask about thoughts of self-harm, following your clinic's safety protocol and sharing the 988 Suicide and Crisis Lifeline.\n\nResearch decision support. Surfaces chart patterns for clinician review. Not a diagnosis. Model unm-depression 1.0.0.",
      "source": {
        "label": "Undiagnosed No More (research decision support)",
        "url": "https://undiagnosednomore.org/developers/"
      },
      "links": [
        {
          "label": "Open Undiagnosed No More (SMART app)",
          "url": "https://undiagnosednomore.org/smart/",
          "type": "smart"
        }
      ]
    },
    {
      "summary": "Anxiety: Elevated chart pattern, 1.8× the average. Review the dated findings.",
      "indicator": "info",
      "detail": "**Chart pattern similar to patients who report an anxiety diagnosis that was never coded.**\n\nProbability that the patient reports an anxiety diagnosis: 26% (1.8× the average for adults with a visit in the past year). Band: Elevated (a descriptive score category, not a validated action threshold).\n\nTop findings in the chart:\n\n1. Stimulant or wakefulness drug (2023-05-09)\n2. Age 18-39 (patient demographics)\n3. Female (patient demographics)\n\nNext step: Screen with the GAD-7 (USPSTF 2023 recommends anxiety screening for adults under 65); if it is 10 or more, assess for an anxiety disorder.\n\nResearch decision support. Surfaces chart patterns for clinician review. Not a diagnosis. Model unm-anxiety 1.0.0.",
      "source": {
        "label": "Undiagnosed No More (research decision support)",
        "url": "https://undiagnosednomore.org/developers/"
      },
      "links": [
        {
          "label": "Open Undiagnosed No More (SMART app)",
          "url": "https://undiagnosednomore.org/smart/",
          "type": "smart"
        }
      ]
    }
  ]
}

maria-missing-observations.json. Maria's request without the observations prefetch key and with no fhirServer, so the service cannot read the Observation search. The Long COVID, Chronic fatigue and ME/CFS, Fibromyalgia, Endometriosis, Lyme, Lupus, Migraine, Depression, and Anxiety models read Observation, so they get the could-not-score card instead of a result.

Response to maria-missing-observations.json

{
  "cards": [
    {
      "summary": "Could not score Long COVID, Chronic fatigue or ME/CFS, Fibromyalgia, Endometriosis, Lyme disease, Lupus, Migraine, Depression, Anxiety: Ob…",
      "indicator": "info",
      "detail": "**Could not score this chart for Long COVID, Chronic fatigue or ME/CFS, Fibromyalgia, Endometriosis, Lyme disease, Lupus, Migraine, Depression, Anxiety.** These conditions have no result, which is not the same as a negative result.\n\nThe EHR did not return the patient's complete Observation records (a failed, forbidden, or incomplete search). The models treat missing records as unknown, not as absent.\n\nReview the chart directly or open the SMART app.\n\nResearch decision support. Not a diagnosis. Models: unm-longcovid 1.2.0, unm-chronicfatigue 1.0.0, unm-fibromyalgia 1.0.0, unm-endometriosis 1.0.0, unm-lyme 1.0.0, unm-lupus 1.0.0, unm-migraine 1.0.0, unm-depression 1.0.0, unm-anxiety 1.0.0.",
      "source": {
        "label": "Undiagnosed No More (research decision support)",
        "url": "https://undiagnosednomore.org/developers/"
      },
      "links": [
        {
          "label": "Open Undiagnosed No More (SMART app)",
          "url": "https://undiagnosednomore.org/smart/",
          "type": "smart"
        }
      ]
    }
  ]
}

Errors and incomplete data

400 with {"error": "..."} for a request that fails the checks above, or a body that is not JSON.
412 when the Patient resource is not in the prefetch and cannot be read from an allowed fhirServer.
502 when the FHIR server returns data for another patient.
A search that failed, was forbidden, or was cut short is never read as an empty record. A searchset holding an OperationOutcome with an error, a next link the service may not follow, a total above the entries returned, or more than 10 pages makes that resource type unavailable, and the models that read it get the could-not-score card shown above.

Authentication and logging

The service does not authenticate callers, so it is for synthetic data such as the examples here. A deployment with real patient data must verify the CDS client's signed JWT before handling the request (CDS Hooks 2.0, "Trusting CDS Clients") and limit the allow-list to that client's FHIR base URL. The service code keeps no copy of a request and does not log request bodies.

Send an example from this page

The button posts the chosen example to the service on this site and compares the answer with the response shown above.

From the CDS Hooks sandbox

Open sandbox.cds-hooks.org, open its settings, and add the discovery endpoint https://undiagnosednomore.org/cds-services.
Open the Patient View tab. The sandbox prefetches the selected patient's records and calls the service.

SMART on FHIR app

The clinician view runs as a SMART on FHIR EHR-launch app with the five condition cards and the timeline of the clinician demo. The EHR opens the launch URL with iss and launch. The app checks the issuer, authorizes as a public client with the authorization code flow and PKCE (fhirclient 3.0.0, pkceMode: "required"), reads the chart over FHIR R4, and scores it as of the current date in the clinician's time zone.

SMART Health IT sandbox

Launch URL: https://undiagnosednomore.org/smart/
Redirect URI: https://undiagnosednomore.org/smart/callback.html
Issuer accepted: The SMART Health IT R4 launcher: https://launch.smarthealthit.org/v/r4/fhir, or /v/r4/sim/<id>/fhir on the same host
Client ID: undiagnosednomore-lupus
Scope: launch openid fhirUser patient/*.read
Test patients: Opening the launch URL without launch parameters goes to the launcher with the 18 test patients preselected. model/test_vectors/upload_sandbox.py puts their charts on https://r4.smarthealthit.org, with every date moved from June 1, 2024 to the upload date (COVID-19 diagnoses before July 2, 2021 keep their dates), so each chart has the same history relative to the day it is scored. Rerunning it replaces the same resources.

Launch in the SMART Health IT sandbox

bash: check, then upload (default: every patient, dated as of today)

python3 model/test_vectors/upload_sandbox.py --dry-run
python3 model/test_vectors/upload_sandbox.py maria elena

Epic

Status: Registered; sandbox access pending.
Launch URL: https://undiagnosednomore.org/epic/
Redirect URI: https://undiagnosednomore.org/epic/callback.html
Issuer accepted: https://fhir.epic.com/interconnect-fhir-oauth/api/FHIR/R4 only
Client ID: e8dcbefa-45d6-42da-9159-3ac03e22c24f (non-production)
Scope: launch openid fhirUser. Epic grants resource access from the Incoming APIs on the client record, listed in web/web/epic/README.md.
Framing: Allowed from https://fhir.epic.com and https://open.epic.com

athenaOne

Status: Registered; sandbox access pending.
Launch URL: https://undiagnosednomore.org/athena/launch.html
Redirect URI: https://undiagnosednomore.org/athena/
Issuer accepted: https://api.preview.platform.athenahealth.com/fhir/r4 (Preview) or https://api.platform.athenahealth.com/fhir/r4 (Production)
Client ID: 0oa147h04m1Ny6nsB298
Scope: launch openid patient/Patient.r and patient/<type>.rs for Condition, Observation, DiagnosticReport, Specimen, Encounter, MedicationRequest, MedicationDispense, ServiceRequest, Procedure, QuestionnaireResponse, DocumentReference, Immunization, AllergyIntolerance, CarePlan, Goal, and FamilyMemberHistory
ID token: Signature checked against the issuer's published keys; audience must be the client ID; nonce checked
Framing: Allowed from https://preview.athenahealth.com

What the app reads

The Patient, then one search per resource type with _count=100, following next links on the same server: Condition, Observation, DiagnosticReport, Specimen, Encounter, MedicationRequest, MedicationDispense, MedicationAdministration, MedicationStatement, ServiceRequest, Procedure, QuestionnaireResponse, DocumentReference, ImagingStudy, Immunization, AllergyIntolerance, CarePlan, Goal, and FamilyMemberHistory. A search that is denied or not supported marks that type unavailable, and a chart model that reads it shows "Could not score" instead of a result. Resources labeled NOAUTH in meta.security are left out. The model files read six of these types (FHIR inputs).

Scorer library

Two implementations of the same rules, with no dependencies: model/reference/score_fhir.py, the reference that produced the expected outputs (Python standard library only), and /shared/scorer.js, an ES module port for the browser and Node. Both take a model file, a FHIR R4 Bundle of one patient's resources, and a scoring date.

JavaScript

import { scoreBundle } from "/shared/scorer.js";

const model = await (await fetch("/models/longcovid.json")).json();
const bundle = await (await fetch("/developers/vectors/bundles/maria.json")).json();
const result = scoreBundle(model, bundle, "2024-06-01");
// { status, reason, band, probability, times_average, drivers }

The date is a YYYY-MM-DD string; without it the scorer uses today's date in UTC. When a FHIR search failed or was forbidden, pass its resource type so the scorer does not treat the missing records as absent:

JavaScript

const result = scoreBundle(model, bundle, "2024-06-01", { unavailable: ["Observation"] });
// {
//   status: "could_not_score",
//   reason: "could not score: Observation unavailable",
//   index_date: "2024-01-08",
//   unavailable: ["Observation"]
// }

Python

bash

python3 model/reference/score_fhir.py model/longcovid.json \
  model/test_vectors/bundles/maria.json --as-of 2024-06-01
# a failed search: --unavailable Observation,Procedure

In Python the same call is score_bundle(model, bundle, as_of, unavailable=[...]), with as_of a datetime.date.

The result

The result is one JSON object. not_scored results carry the reason the card shows (for example "fewer than 90 days since the COVID-19 index"); could_not_score results name the unavailable resource types and must never be shown as a Typical result.

Field	Meaning	Maria, Long COVID 1.2.0
`status`	`"scored"`, `"not_scored"`, or `"could_not_score"`	`"scored"`
`reason`	Why it did not score, as a sentence the app shows; absent when scored	absent
`index_date`	Day 0 (YYYY-MM-DD): the index event, or the scoring date for a scoring-date model; absent when there is no index event	2024-01-08
`score`	`intercept` + Σ weight × value	6.560688
`probability`	The calibration map applied to the score	0.062489 (6.25%)
`band`	The first risk band whose `max_probability` is at or above the probability	High
`times_average`	probability / `reference.cohort_base_rate`	6.0575
`features`	Every feature id with its value, 0 or 1	69 features, 26 equal to 1
`drivers`	Present features with a nonzero weight, largest contribution first, each with the dated resources that set it; features that differ only in threshold are one entry	21 drivers
`unavailable`	The resource types that could not be read; only when the status is `could_not_score`	absent

drivers[0] for Maria, longcovid.json

{
  "id": "sym_any_w3_count_ge3",
  "label": "3+ symptom types coded in days 31-90",
  "group": "Symptoms after COVID",
  "contribution": 1.888304,
  "evidence": [
    {
      "resourceType": "Condition",
      "id": "maria-cond-10",
      "date": "2024-02-20"
    },
    {
      "resourceType": "Condition",
      "id": "maria-cond-18",
      "date": "2024-03-25"
    },
    {
      "resourceType": "Condition",
      "id": "maria-cond-19",
      "date": "2024-03-25"
    },
    {
      "resourceType": "Condition",
      "id": "maria-cond-11",
      "date": "2024-02-20"
    },
    {
      "resourceType": "Condition",
      "id": "maria-cond-16",
      "date": "2024-03-05"
    }
  ]
}

Model registry

Every model file in model/ except the placeholder longcovid.example.json, read from the files themselves. /models/index.json lists the same entries for programs: id, version, file, size, and SHA-256. Model files are served from /models/ with Access-Control-Allow-Origin: *.

Long COVID `unm-longcovid` 1.2.0, research

Predicts: Future Long COVID diagnosis code. First ICD-10-CM U09.9 code 91-455 days after the first documented COVID-19 diagnosis.
Population: Adults with documented COVID-19 and no Long COVID code by day 90.
Day 0: The first Condition coded ICD-10-CM U07.1 on or after July 2, 2021. Scored from day 90 to day 455.
Already coded: Not scored when the chart has ICD-10-CM U09.9; SNOMED CT 1119303003, 1119304009, 1325161000000102.
Features: 69, of which 57 carry a weight
Runs in: CDS Hooks service, clinician app (SMART on FHIR and demo), and patient app
Files: longcovid.json (65 KB, SHA-256 520e1363d9de3c3e4cc120243933be5a266820f32f6a4ab5ad36037765f18fbe); model card; expected outputs for 18 test patients

Chronic fatigue and ME/CFS `unm-chronicfatigue` 1.0.0, research

Predicts: Self-reported chronic fatigue diagnosis. Participant reports having been told by a clinician that they have chronic fatigue (All of Us Personal and Family Health History survey).
Population: Adults with an EHR visit in the past year and no chronic fatigue or ME/CFS code (ICD-10-CM G93.3, R53.82).
Day 0: The scoring date. Windows look back from it: H (days -1825 to -1) and R (days -365 to -1).
Already coded: Not scored when the chart has ICD-10-CM G93.3, R53.82.
Features: 73, all with a weight
Runs in: CDS Hooks service, clinician app (SMART on FHIR and demo), and patient app
Files: chronicfatigue.json (147 KB, SHA-256 b5b91b40a2cf5200691dc8346e1a57fb76e5e4152c3ba4e4e37065d18fb027bb); model card; expected outputs for 18 test patients

Fibromyalgia `unm-fibromyalgia` 1.0.0, research

Predicts: Self-reported fibromyalgia diagnosis. Participant reports having been told by a clinician that they have fibromyalgia (All of Us Personal and Family Health History survey).
Population: Adults with an EHR visit in the past year and no fibromyalgia code (ICD-10-CM M79.7).
Day 0: The scoring date. Windows look back from it: H (days -1825 to -1) and R (days -365 to -1).
Already coded: Not scored when the chart has ICD-10-CM M79.7.
Features: 80, all with a weight
Runs in: CDS Hooks service, clinician app (SMART on FHIR and demo), and patient app
Files: fibromyalgia.json (180 KB, SHA-256 8338f095f8c104970bf0d39083773f40d2f470db00f6d8d67a4ca40afe9ab0c7); model card; expected outputs for 18 test patients

Endometriosis `unm-endometriosis` 1.0.0, research

Predicts: Self-reported endometriosis diagnosis. Participant reports having been told by a clinician that they have endometriosis (All of Us Personal and Family Health History survey).
Population: Women with an EHR visit in the past year and no endometriosis code (ICD-10-CM N80).
Day 0: The scoring date. Windows look back from it: H (days -1825 to -1) and R (days -365 to -1).
Already coded: Not scored when the chart has ICD-10-CM N80.
Features: 90, all with a weight
Runs in: CDS Hooks service, clinician app (SMART on FHIR and demo), and patient app
Files: endometriosis.json (200 KB, SHA-256 7c314ccc2e778fc8765b12342455e22eb1aeb5682695785655e0b1dbd5874121); model card; expected outputs for 18 test patients

Lyme `unm-lyme` 1.0.0, research

Predicts: Self-reported Lyme disease diagnosis. Participant reports having been told by a clinician that they have Lyme disease (All of Us Personal and Family Health History survey).
Population: Adults with an EHR visit in the past year and no Lyme disease code (ICD-10-CM A69.2).
Day 0: The scoring date. Windows look back from it: H (days -1825 to -1) and R (days -365 to -1).
Already coded: Not scored when the chart has ICD-10-CM A69.2.
Features: 86, all with a weight
Runs in: CDS Hooks service, clinician app (SMART on FHIR and demo), and patient app
Files: lyme.json (198 KB, SHA-256 5eb97d14ee4d3357cc5ae49df9aaed4dc6f9649ba2510590ca8cc18aa529317d); model card; expected outputs for 18 test patients

Lupus `unm-lupus` 1.0.0, research

Predicts: Self-reported lupus diagnosis. Participant reports having been told by a clinician that they have lupus (All of Us Personal and Family Health History survey).
Population: Adults with an EHR visit in the past year and no systemic lupus erythematosus code (ICD-10-CM M32).
Day 0: The scoring date. Windows look back from it: H (days -1825 to -1) and R (days -365 to -1).
Already coded: Not scored when the chart has ICD-10-CM M32.
Features: 79, all with a weight
Runs in: CDS Hooks service, clinician app (SMART on FHIR and demo), and patient app
Files: lupus.json (192 KB, SHA-256 d4baa235c78ae1458b0337330388aa45ff54443ef77eebde189175f80f04cfcb); model card; expected outputs for 18 test patients

Migraine `unm-migraine` 1.0.0, research

Predicts: Self-reported migraine diagnosis. Participant reports having been told by a clinician that they have migraine (All of Us Personal and Family Health History survey).
Population: Adults with an EHR visit in the past year and no migraine code (ICD-10-CM G43).
Day 0: The scoring date. Windows look back from it: H (days -1825 to -1) and R (days -365 to -1).
Already coded: Not scored when the chart has ICD-10-CM G43.
Features: 88, all with a weight
Runs in: CDS Hooks service, clinician app (SMART on FHIR and demo), and patient app
Files: migraine.json (201 KB, SHA-256 e41f867ae35e3a62d869cb2934051118b1351d7f9c52e2d6b057cc36f254a916); model card; expected outputs for 18 test patients

Depression `unm-depression` 1.0.0, research

Predicts: Self-reported depression diagnosis. Participant reports having been told by a clinician that they have depression (All of Us Personal and Family Health History survey).
Population: Adults with an EHR visit in the past year and no depression code (ICD-10-CM F32, F33).
Day 0: The scoring date. Windows look back from it: H (days -1825 to -1) and R (days -365 to -1).
Already coded: Not scored when the chart has ICD-10-CM F32, F33.
Features: 86, all with a weight
Runs in: CDS Hooks service, clinician app (SMART on FHIR and demo), and patient app
Files: depression.json (185 KB, SHA-256 f7af1331be4222229d0f6f100a2b9e662a0dd7f877eb4927375b45f086576fc4); model card; expected outputs for 18 test patients

Anxiety `unm-anxiety` 1.0.0, research

Predicts: Self-reported anxiety diagnosis. Participant reports having been told by a clinician that they have anxiety (All of Us Personal and Family Health History survey).
Population: Adults with an EHR visit in the past year and no anxiety code (ICD-10-CM F41).
Day 0: The scoring date. Windows look back from it: H (days -1825 to -1) and R (days -365 to -1).
Already coded: Not scored when the chart has ICD-10-CM F41.
Features: 84, all with a weight
Runs in: CDS Hooks service, clinician app (SMART on FHIR and demo), and patient app
Files: anxiety.json (193 KB, SHA-256 92e145027341d0b2045c079c69416ddec29a92d56848d762060f02de6665eeb0); model card; expected outputs for 18 test patients

Model contract

A model file is a linear score over named, dated chart findings, with a calibration map and risk bands, so every point of the score traces to a chart resource. Its top-level fields:

Field	Holds
`model_id`, `version`, `status`	Identity: a stable id, a semantic version, and `research` or `placeholder`
`target`	What the model predicts (`label`, `definition`) and for whom (`population`); for an index-event model, `landmark_days` and `horizon_days`
`index_event`	Day 0: the codes of the index event, or `"type": "scoring_date"`
`exclusion`	Codes that mean the target diagnosis is already recorded, so the chart is not scored
`eligibility`	Optional: the FHIR gender a model is limited to (`sex`) and the reason shown otherwise; the endometriosis model covers women
`history_requirement`	Scoring-date models: the encounters a chart needs in a window to be scored
`windows`	Named day ranges relative to day 0
`features`	Each with an `id`, `label`, `group`, `kind`, codes or classes, windows, and `weight`
`intercept`, `calibration`, `risk_bands`	The score, its piecewise-linear map to a probability, and the bands by upper probability
`reference`	`cohort_base_rate`, the denominator of "times the average"
`provenance`, `statement`	Data, method, and validation; the sentence shown with every result

Scoring

score       = intercept + Σ weight_j × value_j        (each value is 0 or 1)
probability = calibration(score)                      (piecewise-linear, clamped at the ends)
band        = first risk band with max_probability ≥ probability

Features come in nine kinds: any, new_onset, encounter_any, encounter_count_ge, encounter_rate_ratio_ge, derived_count_ge, cluster_count_ge, age_band, and sex. The interface spec defines each, with the date rules and the order of the eligibility checks.

The contract is at version 0.2. Its full text is the interface spec.

FHIR inputs

The scorer reads FHIR R4 Patient, Condition, Encounter, Observation, MedicationRequest, and Procedure resources. A coding matches only when its system is exactly one of these URIs:

Code system	`system`	Where and how it matches
ICD-10-CM	`http://hl7.org/fhir/sid/icd-10-cm`	`Condition.code`, by prefix, with dots removed and letters uppercased
SNOMED CT	`http://snomed.info/sct`	`Condition.code`, exact; used for already-recorded diagnoses
LOINC	`http://loinc.org`	`Observation.code`, exact
CPT	`http://www.ama-assn.org/go/cpt`	`Procedure.code`, exact
ICD-10-PCS	`http://www.cms.gov/Medicare/Coding/ICD10`	`Procedure.code`, exact
RxNorm	`http://www.nlm.nih.gov/research/umls/rxnorm`	`MedicationRequest.medicationCodeableConcept`, by RxCUI, or by an ingredient name in its `text` or a coding `display` (case-insensitive). A `medicationReference` is not followed.

Encounters count by class.code: AMB, EMER, and IMP (HL7 v3 ActCode); virtual (VR) and home (HH) visits count as AMB.
Patient: birthDate gives the age on day 0 (days / 365.25; under 18 is not scored), and gender the sex feature.
Dates come from the first valid field of onsetDateTime, onsetPeriod.start, recordedDate, period.start, effectiveDateTime, effectivePeriod.start, issued, authoredOn, performedDateTime, performedPeriod.start, and occurrenceDateTime, as calendar dates (time and zone ignored). YYYY and YYYY-MM are read as the first day they allow, and as July 1 and the 15th on birthDate.
Ignored: records dated after the scoring date; Conditions whose verificationStatus is refuted or entered-in-error; any record whose status is entered-in-error. An undated record falls in no window, but an undated target diagnosis still counts as already recorded.
Missing records count as absent (value 0), as in training. A search that could not be read is different: pass it as unavailable (spec, section 4c).

Features per resource in each model, with the number that carry a weight in parentheses:

Resource or field	Code system	Long COVID	Chronic fatigue and ME/CFS	Fibromyalgia	Endometriosis	Lyme	Lupus	Migraine	Depression	Anxiety
Condition	ICD-10-CM	39 (27)	33 (33)	36 (36)	43 (43)	40 (40)	35 (35)	38 (38)	39 (39)	36 (36)
Condition (ICD-10-CM clusters)	n/a	2 (2)	2 (2)	2 (2)	2 (2)	2 (2)	2 (2)	2 (2)	2 (2)	2 (2)
Encounter.class	n/a	9 (9)	4 (4)	4 (4)	4 (4)	4 (4)	4 (4)	4 (4)	4 (4)	4 (4)
MedicationRequest	RxNorm, RxNorm ingredient name (fallback)	2 (2)	9 (9)	12 (12)	13 (13)	14 (14)	12 (12)	16 (16)	13 (13)	14 (14)
Observation	LOINC	8 (8)	11 (11)	13 (13)	12 (12)	13 (13)	16 (16)	13 (13)	13 (13)	13 (13)
Patient.birthDate (age on day 0)	n/a	5 (5)	5 (5)	5 (5)	5 (5)	5 (5)	5 (5)	5 (5)	5 (5)	5 (5)
Patient.gender	n/a	1 (1)	1 (1)	1 (1)	none	1 (1)	1 (1)	1 (1)	1 (1)	1 (1)
Procedure	CPT	2 (2)	8 (8)	7 (7)	11 (11)	7 (7)	4 (4)	9 (9)	9 (9)	9 (9)
other features (count within a group)	n/a	1 (1)	none	none	none	none	none	none	none	none

The clinician app also imports the patient app's answers, a QuestionnaireResponse for https://undiagnosednomore.org/fhir/Questionnaire/unm-patient-symptom-check (versions 1.0.0, 1.1.0, 1.2.0, and 1.3.0). The answers are shown next to the chart score and never added to it.

Test vectors

Eighteen synthetic patients (no real data), each a FHIR R4 Bundle, and the reference scorer's output for each of the nine models above, scored as of June 1, 2024: 162 expected outputs.

Download all vectors (zip, 186 KB)

Expected output for each patient and model (18 patients, 9 models, with links to each file)

Patient	Scenario	Long COVID 1.2.0	Chronic fatigue and ME/CFS 1.0.0	Fibromyalgia 1.0.0	Endometriosis 1.0.0	Lyme 1.0.0	Lupus 1.0.0	Migraine 1.0.0	Depression 1.0.0	Anxiety 1.0.0	Bundle
Maria	should score high	High, 6.25%	Typical, 5.42%	Typical, 2.73%	Typical, 8.74%	Typical, 2.42%	Typical, 0.30%	Typical, 14.7%	Typical, 14.7%	Typical, 10.2%	maria.json
James	should score low	Typical, 0.43%	Typical, 1.51%	Typical, 0.93%	not scored: the model covers women (FHIR gender female)	Typical, 1.22%	Typical, 0.12%	Typical, 9.66%	Typical, 18.2%	Typical, 13.1%	james.json
Rosa	already coded: not scored	not scored: a Long COVID diagnosis is already recorded	Typical, 2.04%	Typical, 1.60%	Typical, 6.15%	Typical, 1.18%	Typical, 0.18%	Typical, 14.3%	Typical, 20.1%	Typical, 14.9%	rosa.json
Lee	no COVID: not scored	not scored: no documented COVID-19 index event	Typical, 1.47%	Typical, 0.80%	not scored: the model covers women (FHIR gender female)	Typical, 1.29%	Typical, 0.11%	Typical, 9.98%	Typical, 22.8%	Typical, 17.8%	lee.json
Nora	no baseline encounters: not scored	not scored: no encounters in the year before COVID-19 (insufficient history)	Typical, 1.73%	Typical, 1.10%	Typical, 5.10%	Typical, 1.10%	Typical, 0.16%	Typical, 13.7%	Typical, 24.8%	Typical, 20.8%	nora.json
Dana	Lyme demo (Dutchess County, NY)	not scored: no documented COVID-19 index event	Typical, 1.71%	Typical, 1.32%	Typical, 6.03%	Elevated, 3.48%	Typical, 0.18%	Typical, 12.2%	Typical, 20.1%	Typical, 14.0%	dana.json
Elena	long fatigue work-up, never coded: should score high	not scored: no documented COVID-19 index event	High, 16.9%	Typical, 3.96%	Typical, 9.33%	Typical, 2.24%	Typical, 0.19%	Elevated, 22.7%	Elevated, 33.3%	Elevated, 25.8%	elena.json
Omar	routine care: should score typical	not scored: no documented COVID-19 index event	Typical, 1.47%	Typical, 0.89%	not scored: the model covers women (FHIR gender female)	Typical, 1.23%	Typical, 0.11%	Typical, 8.54%	Typical, 16.5%	Typical, 11.1%	omar.json
Grace	already coded: not scored	not scored: no documented COVID-19 index event	not scored: a chronic fatigue or ME/CFS diagnosis is already recorded	Typical, 1.59%	Typical, 5.77%	Typical, 1.30%	Typical, 0.24%	Typical, 15.6%	Typical, 27.4%	Typical, 20.2%	grace.json
Sofia	years of widespread pain, negative rheumatology work-up, never coded	not scored: no documented COVID-19 index event	High, 16.7%	High, 30.9%	Typical, 12.4%	Elevated, 3.94%	High, 3.59%	High, 53.1%	High, 70.0%	not scored: an anxiety diagnosis is already recorded	sofia.json
Priya	Long COVID code before the COVID-19 index: not scored	not scored: a Long COVID diagnosis is already recorded	Typical, 2.69%	Typical, 1.75%	Typical, 6.39%	Typical, 1.40%	Typical, 0.18%	Typical, 14.9%	Typical, 19.4%	Typical, 14.0%	priya.json
Sam	Long COVID code (SNOMED) on the index date: not scored	not scored: a Long COVID diagnosis is already recorded	Typical, 1.82%	Typical, 0.99%	not scored: the model covers women (FHIR gender female)	Typical, 1.63%	Typical, 0.10%	Typical, 8.27%	Typical, 13.9%	Typical, 8.82%	sam.json
Theo	Long COVID code on day 90, the landmark day: not scored	not scored: a Long COVID diagnosis is already recorded	Typical, 2.29%	Typical, 1.03%	not scored: the model covers women (FHIR gender female)	Typical, 1.53%	Typical, 0.09%	Typical, 9.93%	Typical, 16.6%	Typical, 12.3%	theo.json
Yara	undated Long COVID code: not scored	not scored: a Long COVID diagnosis is already recorded	Typical, 2.19%	Typical, 1.19%	Typical, 5.45%	Typical, 1.38%	Typical, 0.15%	Typical, 13.0%	Typical, 21.3%	Typical, 16.7%	yara.json
Vera	COVID-19 508 days before, past day 455: not scored	not scored: more than 455 days since the COVID-19 index (outside the model's time window)	Typical, 1.87%	Typical, 1.27%	Typical, 6.39%	Typical, 1.79%	Typical, 0.14%	Typical, 10.2%	Typical, 14.9%	Typical, 9.20%	vera.json
Will	partial dates, a pre-era infection, and refuted or future Long COVID codes: should score (index 2023-12-05)	Typical, 0.49%	Typical, 1.51%	Typical, 0.93%	not scored: the model covers women (FHIR gender female)	Typical, 1.22%	Typical, 0.12%	Typical, 9.66%	Typical, 18.2%	Typical, 13.1%	will.json
Uma	no birth date: not scored by any model	not scored: birth date missing or unusable (age is required)	not scored: birth date missing or unusable (age is required)	not scored: birth date missing or unusable (age is required)	not scored: birth date missing or unusable (age is required)	not scored: birth date missing or unusable (age is required)	not scored: birth date missing or unusable (age is required)	not scored: birth date missing or unusable (age is required)	not scored: birth date missing or unusable (age is required)	not scored: birth date missing or unusable (age is required)	uma.json
Kai	16 at the COVID-19 index, 17 on the scoring date: not scored by any model	not scored: under 18 at the COVID-19 index (the model covers adults)	not scored: under 18 on the scoring date (the model covers adults)	not scored: under 18 on the scoring date (the model covers adults)	not scored: under 18 on the scoring date (the model covers adults)	not scored: under 18 on the scoring date (the model covers adults)	not scored: under 18 on the scoring date (the model covers adults)	not scored: under 18 on the scoring date (the model covers adults)	not scored: under 18 on the scoring date (the model covers adults)	not scored: under 18 on the scoring date (the model covers adults)	kai.json

A scorer passes a vector when, run with the vector's as_of date, its result has the same fields as the expected output (without as_of), every number agrees to 1e-9, and everything else is identical: status, reason, band, each feature value, and the drivers in order with their evidence. Beyond the demo patients, the vectors cover the eligibility rules: a Long COVID code before the index, on the index date, on day 90, or undated; an index more than 455 days back; partial dates, an infection before July 2, 2021, and refuted or future-dated codes; no birth date; and a patient under 18.

bash

# rebuild the bundles and one model's expected outputs with the reference scorer
python3 model/test_vectors/make_bundles.py model/longcovid.json

# check the JavaScript scorer and the CDS Hooks service against them
cd web && npm test

The bundles are FHIR R4 collection Bundles whose fullUrl values are name-based urn:uuid: UUIDs. Where the HL7 FHIR validator is installed: java -jar validator_cli.jar -version 4.0.1 model/test_vectors/bundles/*.json.

Versioning

Each model file carries its model_id and a semantic version. The CDS Hooks card names both, and so does /models/index.json.
Expected outputs are stored per version in model/test_vectors/expected/<model_id>-<version>/. The directories of superseded versions stay in the repository.
Model files are served with Cache-Control: public, max-age=300, so a cached copy is at most five minutes old.
python3 analysis/build_release_manifest.py --out <file> records each deployed model file's hash and version, the scorer hashes, and the contract version.

Directory in `model/test_vectors/expected/`	Outputs	Model file
`unm-anxiety-1.0.0/`	18	current: `model/anxiety.json`
`unm-chronicfatigue-1.0.0/`	18	current: `model/chronicfatigue.json`
`unm-depression-1.0.0/`	18	current: `model/depression.json`
`unm-endometriosis-1.0.0/`	18	current: `model/endometriosis.json`
`unm-fibromyalgia-1.0.0/`	18	current: `model/fibromyalgia.json`
`unm-longcovid-0.1.0/`	18	placeholder example: `model/longcovid.example.json`
`unm-longcovid-1.0.0/`	6	superseded: no model file has this version
`unm-longcovid-1.1.0/`	10	superseded: no model file has this version
`unm-longcovid-1.2.0/`	18	current: `model/longcovid.json`
`unm-lupus-1.0.0/`	18	current: `model/lupus.json`
`unm-lyme-1.0.0/`	18	current: `model/lyme.json`
`unm-migraine-1.0.0/`	18	current: `model/migraine.json`

web/test/scorer.test.js checks the JavaScript scorer against the 180 expected outputs whose model file is in model/ and skips the superseded ones.

Source code

Repository: github.com/undiagnosed-no-more (public at release).

Path	What it holds
`model/`	The model files; `web/tools/sync_models.sh` copies them to `web/web/models/`
`docs/interface/model_spec.md`	The model contract
`model/reference/score_fhir.py`	The Python reference scorer and, next to it, its tests
`web/web/shared/scorer.js`	The JavaScript scorer
`model/test_vectors/`	The bundle generator, the bundles, the expected outputs, and the SMART sandbox upload
`web/functions/cds-services/`	The CDS Hooks service (Cloudflare Pages Functions)
`web/web/smart/`, `web/web/epic/`, `web/web/athena/`	The SMART launch pages and the clinician app
`analysis/build_models_guide.py`	Generates the models guide, the spec page, the vectors, the examples, the registry, and the generated parts of this page
`web/test/`	The tests: `cd web && npm test`

Try the scorer

CDS Hooks service

1. Discovery

2. Request

3. Response

Errors and incomplete data

Authentication and logging

Send an example from this page

From the CDS Hooks sandbox

SMART on FHIR app

SMART Health IT sandbox

Epic

athenaOne

What the app reads

Scorer library

JavaScript

Python

The result

Model registry

Long COVID unm-longcovid 1.2.0, research

Chronic fatigue and ME/CFS unm-chronicfatigue 1.0.0, research

Fibromyalgia unm-fibromyalgia 1.0.0, research

Endometriosis unm-endometriosis 1.0.0, research

Lyme unm-lyme 1.0.0, research

Lupus unm-lupus 1.0.0, research

Migraine unm-migraine 1.0.0, research

Depression unm-depression 1.0.0, research

Anxiety unm-anxiety 1.0.0, research

Model contract

FHIR inputs

Test vectors

Versioning

Source code

Long COVID `unm-longcovid` 1.2.0, research

Chronic fatigue and ME/CFS `unm-chronicfatigue` 1.0.0, research

Fibromyalgia `unm-fibromyalgia` 1.0.0, research

Endometriosis `unm-endometriosis` 1.0.0, research

Lyme `unm-lyme` 1.0.0, research

Lupus `unm-lupus` 1.0.0, research

Migraine `unm-migraine` 1.0.0, research

Depression `unm-depression` 1.0.0, research

Anxiety `unm-anxiety` 1.0.0, research