Punctuation Restoration for Speech Transcripts using seq2seq Transformers

When creating text transcripts from spoken audio, Automatic Speech Recognition (ASR) systems need to infer appropriate punctuation in order to make the transcription more readable. This task, known as punctuation restoration, is challenging since punctuation is not explicitly stated in speech. Most recent works framed punctuation restoration as a classification task and used pre-trained encoder-based transformers, such as BERT, to perform it. In this work, we present an alternative approach, framing punctuation restoration as a sequence-to-sequence task and using T5, a pretrained encoder-decoder transformer model, as the basis of our implementation. Training our model on IWSLT 2012, a common punctuation restoration benchmark, we find its performance is comparable to state of the art classification-based systems with an F1 score of 80.7 on the test set. Furthermore, we argue that our approach might be more flexible in its ability to adapt to more complex types of outputs, such as predicting more than one punctuation mark in a row.