Interrater Reliability in the Application of the VPAS-PB: Analysis Among Experienced Judges
Objective
Phonetic-based perceptual-auditory analysis allows vocal quality to be described by listening to phonatory, supraglottal, tension, and vocal dynamics adjustments, offering an integrated view of the parameters that compounds a vocal profile. The Vocal Profile Analysis Scheme (VPAS) was developed for this purpose and, in its version adapted to Brazilian Portuguese (VPAS-PB), maintains the phonetic model of vocal quality proposed by Laver, incorporating linguistic and cultural adaptations. Verification of inter-rater reliability is essential to ensure the validity and reproducibility of the protocol in research, clinical, and forensic applications. The objective was to analyze the internal consistency of assessments performed with VPAS-PB, a perceptual-auditory assessment protocol for vocal quality adapted to Brazilian Portuguese.
Methods
Ten adult male subjects, Brazilian Portuguese native speakers, aged between 34 and 49 years, all residing in the same city, participated in the study. Male participants were chosen to control for gender-related acoustic variables, avoiding the need to form two distinct groups. The research corpus consisted of semi-spontaneous speech samples from these participants, who narrated significant stories from their lives. The samples were recorded simultaneously by a digital recorder and cell phone in a quiet environment. Vocal screening was performed by self-report and clinical listening by the researcher, with the aim of excluding cases with vocal changes, professional voice use, or hearing complaints. Three evaluators, trained in articulatory phonetics, analyzed the samples using the VPAS-PB protocol. Inter-evaluator reliability was calculated using Cronbach's alpha coefficient in IBM SPSS Statistics software, version 23.0, adopting a significance level of 5% (p<0.05).
Results
Cronbach's alpha values ranged from 0.60 to 0.94, indicating good to excellent internal consistency for most adjustments, with less agreement in the “stretched lips” and “falsetto” adjustments, probably because the corpus consisted only of male voices.
Conclusion
The VPAS-PB shows high inter-rater reliability and good internal consistency for most vocal adjustments. The study contributes to strengthening the application of the VPAS-PB in the Brazilian context and highlights the importance of well-structured phonetic protocols in the analysis of vocal quality.