Assessment of Intelligent Algorithms for Detection, Diagnosis and Risk Stratification of Tumors of the Kidney and Prostate

Condition(s) studied

Investigational drug(s) / intervention(s)

Medical Reading: Participants will perform medical reading of abdominal CT for renal cancer and prostate mpMRI for prostate cancer, without access to AI output using the COMFORT platform.

AI assisted Medical reading: Participants will perform medical reading of abdominal CT for renal cancer and prostate mpMRI for prostate cancer, with access to AI output using the COMFORT platform.

Study summary

The COMFORT study will evaluate an artificial intelligence-based platform developed to support the clinical assessment of kidney and prostate cancer. The platform utilises imaging data to provide AI-generated outputs intended to assist clinicians in the assessment of suspected lesions.

The primary objective of the study is to assess the role of the COMFORT platform in the clinical decision making process of clinicians managing kidney or prostate cancer cases.

Secondary objectives include evaluation of the performance of AI models on anatomical segmentations, tumor characterisation (localisation, volumetric measurements, lesion classification, and scoring (e.g., Bosniak, PI-RADS, RENAL). Also the acceptability, usability, utility and trustworthiness of the AI models will be evaluated.

In addition, potential causes of privacy leakage associated with the COMFORT AI models will be evaluated as part of the platform's privacy assessment.

Eligibility

Primary outcome measure(s)

• Accuracy of the multimodal AI algorithms — Through study completion, an average of 6 months.
  Accuracy of the multimodal AI algorithms will be assessed by comparing model predictions with the clinician-reported reference standard for each suspected lesion. Accuracy will be calculated for each applicable cancer type, with bootstrap confidence intervals.
• Precision of the multimodal AI algorithms — Through study completion, an average of 6 months.
  Precision of the multimodal AI algorithms will be assessed by comparing model predictions with the clinician-reported reference standard for each suspected lesion. Precision will be calculated for each applicable cancer type, with bootstrap confidence intervals.
• F1-score of the multimodal AI algorithms — Through study completion, an average of 6 months.
  F1-score will be calculated to assess the classification performance of the multimodal AI algorithms against the clinician-reported reference standard for each suspected lesion. F1-score will be calculated for each applicable cancer type, with bootstrap confidence intervals.
• Sensitivity of the multimodal AI algorithms — Through study completion, an average of 6 months.
  Sensitivity (recall) will be calculated to assess the ability of the multimodal AI algorithms to correctly identify positive cases relative to the clinician-reported reference standard. Sensitivity will be calculated for each applicable cancer type, with bootstrap confidence intervals.
• Specificity of the multimodal AI algorithms — Through study completion, an average of 6 months.
  Specificity will be calculated to assess the ability of the multimodal AI algorithms to correctly identify negative cases relative to the clinician-reported reference standard. Specificity will be calculated for each applicable cancer type, with bootstrap confidence intervals.
• Area under the receiver operating characteristic curve (AUC-ROC) — Through study completion, an average of 6 months.
  AUC-ROC will be calculated to characterize the classification performance of the multimodal AI algorithms relative to the clinician-reported reference standard for each applicable cancer type, with bootstrap confidence intervals.
• Intersection over Union (IoU) — Through study completion, an average of 6 months.
  Intersection over Union (IoU) will be calculated for applicable object detection and segmentation tasks to assess agreement between AI-generated regions and the corresponding reference standard.
• Agreement between AI-generated stratification and the clinical reference standard — Through study completion, an average of 6 months.
  Agreement between model-generated lesion stratification and the clinician-reported reference standard will be evaluated using a confusion matrix and quantified using weighted kappa.
• Agreement between AI-generated PI-RADS score and the clinical reference standard — Through study completion, an average of 6 months.
  Agreement between the model-generated PI-RADS score and the clinician-reported reference standard will be evaluated using a confusion matrix and quantified using weighted kappa.
• Agreement between AI-generated Bosniak score and the clinical reference standard — Through study completion, an average of 6 months.
  Agreement between the model-generated Bosniak score and the clinician-reported reference standard will be evaluated using a confusion matrix and quantified using weighted kappa.
• Agreement between AI-generated RENAL score and the clinical reference standard — Through study completion, an average of 6 months.
  Agreement between the model-generated RENAL score and the clinician-reported reference standard will be evaluated using a confusion matrix and quantified using weighted kappa.
• Group-stratified performance of the multimodal AI algorithms — Through study completion, an average of 6 months.
  Performance metrics of the multimodal AI algorithms will be calculated separately for prespecified participant groups to assess differences in model performance across groups. Group-stratified metrics will be reported with bootstrap confidence intervals.
• Group-stratified calibration of the multimodal AI algorithms — Through study completion, an average of 6 months.
  Calibration metrics of the multimodal AI algorithms will be calculated separately for prespecified participant groups to assess model calibration across groups. Group-stratified calibration metrics will be reported with bootstrap confidence intervals.

Trial sites (6)

More Aristotle University Of Thessaloniki trials in Germany

Other trials for the same condition