Does AI Make Clinicians More Appropriately Confident? A Randomized Study in Preterm Birth Prediction

Condition(s) studied

Investigational drug(s) / intervention(s)

AI prediction (binary): AI decision support based on cervical ultrasound providing a binary classification (preterm birth before 37 weeks or term birth) in addition to standard clinical information.

AI risk estimate (%): AI decision support based on cervical ultrasound providing an estimate of preterm birth risk (%) in addition to standard clinical information.

Study summary

The goal of this randomized questionnaire-based study is to evaluate how different presentations of artificial intelligence (AI) decision support influence clinical judgment among medical doctors working in obstetrics and gynecology when assessing the risk of spontaneous preterm birth using clinical case vignettes with cervical ultrasound images. The study specifically compares two AI presentation formats: a binary classification (preterm vs term birth) and an individualized risk estimate of preterm birth.

The main questions it aims to answer are:

* Which AI presentation format leads to better alignment between clinicians' confidence and decision accuracy (diagnostic calibration)?
* Do different AI presentation formats lead to helpful or harmful changes in clinical decisions?

Participants will complete an online questionnaire in which they review clinical cases, make diagnostic and management decisions, rate their diagnostic confidence before and after seeing the AI output, and report their trust in the AI.

Eligibility

Primary outcome measure(s)

• Clinician diagnostic calibration (accuracy-confidence alignment) after AI exposure. — Immediately after AI exposure during a single questionnaire session (approximately 20 minutes).
  Agreement between post-AI decision correctness (0/1) and post-AI confidence rating (0-10) will be quantified using the Brier score. Confidence will be rescaled to 0-1 and squared differences between confidence and correctness will be averaged across cases to produce a participant-level score. Lower scores indicate better diagnostic calibration. Results will be compared between randomized arms.

Trial sites (18)

More Rigshospitalet, Denmark trials in Denmark

Other trials for the same condition