A time-scale modification dataset with subjective quality labels
This work addresses the problem of evaluating TSM quality for audio researchers and engineers, but it is incremental as it focuses on dataset creation rather than a new method.
The paper tackles the lack of an effective objective quality measure for Time Scale Modification (TSM) by creating and analyzing a dataset with subjective quality labels, comprising 88 source files processed with six TSM methods and 20 with three additional methods, resulting in 42,529 ratings from 633 sessions.
Time Scale Modification (TSM) is a well-researched field; however, no effective objective measure of quality exists. This paper details the creation, subjective evaluation, and analysis of a dataset for use in the development of an objective measure of quality for TSM. Comprised of two parts, the training component contains 88 source files processed using six TSM methods at 10 time scales, while the testing component contains 20 source files processed using three additional methods at four time scales. The source material contains speech, solo harmonic and percussive instruments, sound effects, and a range of music genres. Ratings (42 529) were collected from 633 sessions using laboratory and remote collection methods. Analysis of results shows no correlation between age and quality of rating; expert and non-expert listeners to be equivalent; minor differences between participants with and without hearing issues; and minimal differences between testing modalities. A comparison of published objective measures and subjective scores shows the objective measures to be poor indicators of subjective quality. Initial results for a retrained objective measure of quality are presented with results approaching average root mean squared error loss and Pearson correlation values of subjective sessions. The labeled dataset is available at http://ieee-dataport.org/1987.