WaveVC: Speech and Fundamental Frequency Consistent Raw Audio Voice Conversion

Abstract Voice conversion (VC) is a task for changing the speech of a source speaker to the target voice style while preserving linguistic information of the source speech. Existing VC methods require a separate vocoder because they output mel-spectrogram. Therefore, the VC performance varies depending on the vocoder performance, and noisy speech can be generated due to problems such as train-test mismatch. In this paper, we propose a speech and fundamental frequency consistent raw audio voice conversion method called WaveVC. WaveVC does not require a separate vocoder because it performs VC directly on raw audio and is unaffected by vocoder performance. In addition, WaveVC uses speech loss and F0 loss to preserve content information and generate F0 consistent results. WaveVC shows high performance in both many-to-many VC and any-to-any VC, and the converted samples are available online.