Notes by ff123
What I'm doing is just for fun. For those who are really serious about listening tests, there is a technical standard published by the International Telecommunication Union, Recommendation ITU-R BS.1116-1, "Methods for the Subjective Assessment of Small Impairments in Audio Systems Including Multichannel Sound Systems." This document can be ordered (takes just a few minutes) from the ITU website for CHF 22 (French Francs), which is about $14 US dollars (11/23/00). I recommend it highly.
The scope of ITU-R BS.1116-1 is as follows:
This Recommendation is intended for use in the assessment of systems which introduce impairments so small as to be undetectable without rigorous control o f the experimental conditions and appropriate statistical analysis.The preferred test method is described as follows:
The "double-blind triple-stimulus with hidden reference" method has been found to be especially sensitive, stable and to permit accurate detection of small impairments.
In the preferred and most sensitive form of this method, one subject at a time is involved and the selection of one of three stimuli ("A", "B", "C") is at the discretion of this subject. The known reference is always available as stimulus "A". The hidden reference and the object are simultaneously available but are "randomly" assigned to "B" and "C", depending on the trial.
The subject is asked to assess the impairments on "B" compared to "A", and "C" compared to "A", according to the continuous five-grade impairment scale. One of the stimuli, "B" or "C", should be indiscernible from stimulus "A"; the other one may reveal impairments. Any perceived differences between the reference and the other stimuli must be interpreted as an impairment
The PC ABX program works as follows: The known reference is always available as stimulus "B". The known object (test sample) is also always available as stimulus "A". Either the reference or the object is randomly assigned to "X", depending on the trial. The subject decides whether "X" corresponds to the reference "B" or to the test sample "A". This is different from the method described above. I don't know whether it is more or less sensitive.
For an example of a well-constructed and valid listening test, take a look at the MPEG Site and download one of the files detailing the results of subjective tests.