P-280 Potential for improvement and current limitations of Artificial Intelligence (AI) for embryo selection: analysis of external validation data

Abstract Study question What are the prospects of improvement and the limitations of an AI system for embryo selection? Summary answer The predictive performance of AI can be enhanced by including additional factors, on top of embryo images, and by assessing images with centered blastocysts. What is known already We previously reported the external validation of IVFvision.ai, an AI algorithm that differentiates between Day-5 blastocysts with a positive or negative implantation outcome. IVFvision.ai had higher AUC and overall accuracy in predicting implantation compared to KIDScoreD5 and senior embryologists. Here we report a secondary analysis of external validation data, focusing on a) the improvement of the predictive ability of IVFvision.ai by incorporating data from additional sources, and b) the impact of the blastocyst image quality on the performance of IVFvision.ai. Study design, size, duration This is a secondary analysis of external validation data. External validation of IVFvision.ai was performed at a University IVF Clinic using 113 anonymised Embryoscope images of single D5 blastocyst transfers with known implantation outcome. Participants/materials, setting, methods The performance of IVFvision.ai and three senior Embryologists to correctly classify blastocysts according to implantation outcome were compared in images in which the whole blastocyst was visible (centred blastocysts, n = 62) vs images in which part of the blastocyst was not visible (off-centred blastocysts, n = 51). Logistic regression models were created: a) IVFvision alone, b) IVFvision+age, c) IVFvision+fertilisation_method, d) IVFvision+KIDScoreD5, e) IVFvision+age+Fertilisation_method+KIDScoreD5. The AUC of each model in predicting implantation was estimated using ROC curve analysis. Main results and the role of chance The AUC of IVFVision.ai (0.675 vs 0.432), Embryologist 1 (0.570 vs 0.390), Embryologist 2 (0.663 vs 0.448) and Embryologist 3 (0.628 vs 0.485) were higher for images with centered blastocysts compared to non-centered blastocysts, respectively. There was a progressive increase of AUC with the addition of more factors in the predictive models. a) IVFvision alone: AUC=0.675, b) IVFvision+age: AUC=0.675 c) IVFvision+KIDScoreD5: AUC=0.721 d) IVFvision+fertilisation_method=0.740, e) IVFvision+age+Fertilisation_method+KIDScoreD5=0.768. Limitations, reasons for caution The retrospective nature of the study and the small sample of the study raise the need for further prospective studies with a larger number of embryos. Wider implications of the findings The highest performance of IVFvision.ai is achieved in images with centred blastocysts, suggesting that implantation cannot be predicted accurately in images with non-centred blastocysts. In addition, we provide provide proof of concept that training AI systems using data from different sources, in addition to embryo images, may increase overall accuracy. Trial registration number not applicable