User Testing a mHealth Behavioral Health Application for Hopi Youth during the COVID-19 Pandemic (Preprint)

BACKGROUND American Indian/Alaska Native (AI/AN) people represent a culturally diverse people group within the United States. AI/AN people experience some of the most severe health disparities in the US, including behavioral health disparities [1, 14]. Of the 9.7 million AI/AN people living in the US, 25% live on tribal lands [15], which experience some of the most significant disparities in access to mental health resources [16] and broadband infrastructure to support behavioral health telehealth [18]. To address mental healthcare disparities among AI/AN youth, we developed Amplifying Resilience Over Restricted Internet Access (ARORA)–a mobile health (mHealth) smartphone application that incorporates Southwestern AI/AN culture and values into its design while promoting mindfulness practices and community building. Originally co-designed with both Hopi and Navajo youth and adults, the goal of this study was to understand how well the app resonated with Hopi/Tewa youth and to develop a culturally-responsive mindfulness-based mHealth intervention. This study took place during the COVID-19 pandemic, presenting unique challenges for meeting with key stakeholders. OBJECTIVE This study aimed to (1) assess the potential effectiveness and acceptability of a mHealth app designed to support Hopi youth engaged in mindfulness practices in ways that supports their culture, values, and beliefs. This study also aimed to (2) identify points in which this mobile application could be improved to better fit the needs and preferences of Hopi youth surrounding mental health and wellness. METHODS We developed an initial prototype version of the ARORA mHealth app based on meetings with six Community Advisory Board (CAB) members as well as focus groups with potential stakeholders prior to this study [2]. ARORA included several individual mindfulness-focused activities, along with one multiplayer game, Superfly, designed to encourage collaboration and community among users. In Superfly, players must work together to collect digital butterflies using Augmented Reality (AR). They then combine their butterflies to form one larger and more complex butterfly. The mindfulness activities within the app include (1) a guided meditation activity; (2) a mindfulness walking activity; and (3) a calming breathing activity. Each of the mindfulness activities are completed individually and incorporate artwork or color schemes influenced by Hopi culture, as informed by our CAB. This study focused on the usability and acceptability of the app, evaluating how intuitive it was to operate. Participants were asked to identify key areas for improvement in future iterations. Participants were asked to interact with each component of ARORA’s features and activities, and then provide feedback both on the design and effectiveness of the app. All participants in this study were Hopi youth, who represented the primary stakeholders for the final version of ARORA. RESULTS This study involved nine participants ranging from ages 16 to 24 years. Of these participants, 1 was male and 8 were female; all were members of the Hopi Tribe. The results of this study included a quantitative assessment using a modified version of the end-user Mobile Application Rating (uMARS) scale [19]. The uMARS includes 11 questions related to emotional wellness and community and 5 additional questions related to cultural resonance, each of which is rated on a scale of 1(low) to 5 (high). Overall, the mean score across all questions was 3.71, suggesting that feedback on the app was greater than average. We also gathered qualitative data from open-ended focus group responses. We identified five open codes and 12 axial themes related to users’ perceptions of ARORA. The majority of qualitative feedback revealed that, while its use was generally enjoyable, the ARORA app could be modified to be more specific to Hopi culture, especially in its visual design. CONCLUSIONS In this evaluation and usability testing of the ARORA prototype, participants expressed interest and engagement in the mindfulness-centric activities, particularly those that held high cultural alignment. However, participants also identified spaces in which the app could improve, both in usability and cultural-groundedness, especially with the visual dimensions of the app.