Explainable Artificial Intelligence for Deep Learning in Food

Abstract The integration of artificial intelligence (AI) in food systems is accelerating. Explainable AI can help in understanding AI, but the literature is fragmented in food systems. In light of regulatory imperatives such as the European Union's AI Act, this systematic review brings together current research on explainable AI in food, highlighting key patterns and gaps. We find that most studies use off-the-shelf explainable AI tools that fail to address the complexities of food data. Beyond model transparency, explainable AI offers broader value in model enhancement, supporting trust, and knowledge discovery. However, most studies do not adequately evaluate the explainable AI methods they use. Advancing explainable AI in food systems requires tailored and carefully evaluated approaches to ensure responsible and effective AI deployment. Domain and AI experts from the entire food system should collaborate on an evaluation framework in explainable AI for food to provide better guidance, tools, and evaluation.