Jenevora Williams 2017
How can we get behind the meaning of published research papers, and how much can we trust the experts to have found the truths?
We have to trust peer-reviewed scientific papers as the nearest thing we currently have to the facts. Many of the published findings are fascinating, and we’ve based a huge amount of our pedagogical practice on their outcomes. However, how close to the truth are these findings?
Many studies are just on one individual, or of a small number of singers. This gives us a picture of individual activity, not necessarily that of the majority.
Other studies are using a larger number of singers, but they are often friends, colleagues, or students of the researchers. Rarely are they absolutely top-level professionals. Several years ago, Pettersen measured the activity of neck muscles in singers. But is this necessary activity, an unrelated extra, or perhaps even an idiosyncratic bad habit of that singer? What are the practical applications for us as teachers? We can only know the answers if we take measurements from many other singers as well, or maybe even coach the singers through different manoeuvres and measure voice quality against muscular activity. Nevertheless, Pettersen’s findings are hugely important in our understanding of muscular activity and balance.
All too often results are inconclusive. Recent research by Vos et al. using 3D MRI scanning to show vocal tract configurations in resonance tuning (about to be published in JoV) has an enormous wealth of data, but is possibly in danger of any trends becoming buried in variance. It does give us a reliable protocol by which to make future measurements. One interesting point to note was that the singer herself (an international soloist) stated that she made certain manoeuvres, many of which were not actually observed in the measurements; this was also the case in Pettersen’s research. What we think we are doing is often not what we are actually doing.
Often the methods of measuring can have a disruptive effect on the singer. MRI scanning gives us amazing images of the vocal tract, but the singer has to lie supine, with the head immobilised, inside a very noisy and claustrophobic tube. Even nasendoscopic or EGG assessments can be disconcerting. These papers have validity, of course. They provide an insight into activities that we could only guess about previously; and without preliminary research we have no building blocks for subsequent investigations. So – how much do we trust the scientists? I’ve published scientific research into vocal behaviour and development – I’d like to think that the results have been interesting and relevant. Can we have faith in experts? Or is everything to be treated with caution?
We can separate 'observations' from 'significant' results. In order to be statistically significant, the data has to have enough numbers of participants and not too many variables. Then it can be shown that p<0.5. And we know that statistics can be selective, and can be manipulated to an extent. If there is no statistically significant result, then observations are also valid; we learn a great deal from these, but they are only 'truths' in relation to the individuals who are observed. Much reported research shows trends - these are also interesting observations but may not be statistically significant.
In all these cases, the researchers and the peer-review panel are fully aware of the circumstances of their results; the problem comes from the Chinese Whispers effect of reporting these, handing them on or possibly even misinterpreting them. How often do we read sensational statistics in the news, only to find that the journalist has misrepresented the data.
And one last thing – how often are we selective with the papers we cite, using only those that reinforce our personal beliefs and practices? There are many psychological biases, but confirmation bias and anchoring are so strong for our survival in a messy and unpredictable world of information. Much of the time we are unaware of these selective channels for our attention.
We have to trust science. We know that truths are often dismantled at a later date, but until we know otherwise, I'd much rather trust an expert than my own guesswork.