Functional protein biomarkers based on distributions of expression levels in single-cell imaging data

Abstract Motivation The intra-tumor heterogeneity of protein expression is well recognized and may provide important information for cancer prognosis and predicting treatment responses. Analytic methods that account for spatial heterogeneity remain methodologically complex and computationally demanding for single-cell protein expression. For many functional proteins, single-cell expressions vary independently of spatial localization in a substantial proportion of the tumor tissues, and incorporation of spatial information may not affect the prognostic value of such protein biomarkers. Results We developed a new framework for using the distributions of functional single-cell protein expression levels as cancer biomarkers. The quantile functions of single-cell expressions are used to fully capture the heterogeneity of protein expression across all cancer cells. The quantile index (QI) biomarker is defined as an integral of an unspecified function which may depend linearly or nonlinearly on a tissue-specific quantile function. Linear and nonlinear versions of QI biomarkers based on single-cell expressions of ER, Ki67, TS, and CyclinD3 were derived and evaluated as predictors of progression-free survival or high mitotic index in a large breast cancer dataset. We evaluated performance and demonstrated the advantages of nonlinear QI biomarkers through simulation studies. Availability and implementation The associated R package Qindex is available at https://CRAN.R-project.org/package=Qindex and R package hyper.gam is available at https://github.com/tingtingzhan/hyper.gam. Examples of R code and detailed instructions could be found in vignette quantile-index-predictor (https://CRAN.R-project.org/package=hyper.gam/vignettes/applications.html#quantile-index-predictor).