Comparing automated analysis and human analysis of hedging in academic texts

Hedging is a metadiscourse device employed by academic writers to manage knowledge claims and establish writer-reader interaction in written discourse. Research writing involves a balance of fact and a writer’s personal evaluation and interpretation. This study compared automated analysis of hedging through Authorial Voice Analyzer (AVA) with a more traditional human analysis of hedging, to increase understanding of the relative strengths and weaknesses of the AVA versus human analysis of hedging in academic texts. An explanatory sequential mixed-methods design was used; quantitative analysis (Pearson correlation) was followed by qualitative analysis to understand the reasons for quantitative differences. AVA found a larger number of hedging items than the human analysis in the same academic writing corpus. However, qualitative analysis suggests that the AVA only considers frequency and does not take account of function. Since many hedging devices are multifunctional, AVA seems to overestimate the frequency of hedging by counting hedge markers as hedging even when they are used with propositional functions. Overall, automated analytic tools like AVA are useful for metadiscourse studies. However, unless used in combination with human analysis they are unlikely to effectively operate with multifunctional markers. The findings of this study offer validation of AVA and help raise awareness of how the tool can be used to evaluate hedging in academic texts.