Improving the Interpretability of fMRI Decoding Using Deep Neural Networks and Adversarial Robustness

Deep neural networks (DNNs) are being increasingly used to make predictions from functional magnetic resonance imaging (fMRI) data. However, they are widely seen as uninterpretable “black boxes,” as it can be difficult to discover what input information is used by the DNN in the process, something important in both cognitive neuroscience and clinical applications. A saliency map is a common approach for producing interpretable visualizations of the relative importance of input features for a prediction. However, methods for creating maps often fail due to DNNs being sensitive to input noise or by focusing too much on the input and too little on the model. It is also challenging to evaluate how well saliency maps correspond to the truly relevant input information, as ground truth is not always available. In this paper, we review a variety of methods for producing gradient-based saliency maps and present an adversarial training method we developed to make DNNs robust to input noise, with the goal of improving the quality of DNN saliency maps. We introduce two quantitative evaluation procedures for saliency map methods in fMRI, applicable whenever a DNN or linear model is being trained to decode some information from imaging data. We evaluate the procedures using synthetic data sets, where the complex activation structure is known, and on fMRI data from the Human Connectome Project using saliency maps produced for linear models and DNNs doing task decoding in both settings. Our key finding is that saliency maps produced with different methods vary widely in quality in both synthetic and Human Connectome Project fMRI data, even when those methods have similar prediction performance. We found that training both linear and non-linear models using adversarial noise increased the quality of their saliency maps. We also found that, while some linear models generate good saliency maps in the highly controlled, synthetic data, non-linear DNN methods generate better saliency maps in real-world fMRI data. Finally, our experiments give evidence that combining adversarial training with a complex, non-linear model can improve saliency map quality when compared to several methods commonly used in the literature.