How I Conduct Listening Tests

notes by ff123

 

PC Noise
I use a PC, with a Silencer power supply and Silencer case fan from PC Power & Cooling. I also use a radial Molex CPU fan and SilentDrive hard drive enclosure from Silent Systems, which I ordered from Northwest Custom Computers. Even with these components, which are much quieter than in the average PC, noise is still audible and may affect my listening tests. To further mitigate the effects of PC noise, I have placed my computer on the carpeted floor beneath my table. I also use headphones to provide further attenuation.

Addendum (12/22/00): I have disabled all the fans in my computer. I found that I could no longer overclock my Celeron from 366 MHz to 550 MHz without Windows locking up, but 450 MHz seems to work just fine. Now we just have to wait to find how long it will run. I don't recommend this to other people, obviously. I'm kind of a nut for quiet PC operation, though, and the truly silent setup is sweet.

02/24/01: In the past two months, I have run the computer for days at a time without interruption or lockup. So I'd say it's pretty stable.

05/25/01: In the beginning of May, I noticed that over the preceding 2 or 3 weeks it had become harder and harder to perform a cold bootup of the computer. Windows kept locking up after a cold boot; however, once the computer warmed up, it was stable. I believe the extreme heat from five months of fanless operation had damaged the CPU. Also, when I visited QuietPC and checked into the Molex radial fans again, I noticed that they were not recommended for Celeron 366's. I bought a new Celeron 366 ($39) and a new, quiet CPU fan appropriate for the Celeron chip from PC Power & Cooling. The cold boot problem is gone. My new mode of operation will be to normally run with the CPU fan enabled, but to disable it during listening tests.

Sample rate conversion quality
Set to "best" from within Control Panel / Multimedia / Audio / Playback Advanced Properties / Performance

Sound Card
I have an M-Audio Audiophile 2496 soundcard (as of June 9, 2001). Before that I used a Soundblaster 64 AWE Gold (now serving duty as a lowly Gamepad port).

Headphones
I go straight from the line out (no amp) to headphones: Grado SR325's, as of Dec 9, 2000. Before this, I used Beyerdynamic DT990 Pro's. The Grado's have a brighter lower-treble than the Beyer's, which sounds better to me. The Beyers, on the other hand, have better mid and low bass. The Beyers could be described as warm and full, while the Grado's are dynamic and detailed. Although I love good bass (I own Nelson Reed 8-04B loudspeakers with accompanying subwoofer, which go down to 16Hz), I think the Grado's beat out the Beyer's overall. My high-frequency cutoff (16 or 17kHz) is about the same through either headphone set.

I plan to look into buying a headphone amp in the future.

The Headwize site has lively and interesting forums which I highly recommend if you're looking for a new pair of phones.

When I conduct tests
I normally conduct tests at night, after my family has gone to sleep. It's not ideal because I can hear them snoring, but it's the best I can do.

mp3 to wav decoder
I use the default Fraunhofer decoder within Winamp 2.73 to convert mp3's to wav's. In previous versions of Winamp, the Nitrane engine had a couple of serious problems with it (see David Robinson's mp3 decoder tests page). These problems were fixed as of version 2.666.

ABX
I use Arny Krueger's PC ABX program to conduct the equivalent of double-blind listening tests. I noticed that I hear a click whenever I start to play a sound, which can affect my test results (see my test of Radiohead's "Anyone Can Play Guitar"). To sidestep this problem, I insert half a second of silence to the beginning of clips, post-encoding. To save time, I don't do full runs (of 16 trials) on clips which have very obvious artifacting. 16 trials is recommended on the PC-ABX site to strike a balance between statistical reliability and listener fatigue. This is also the minimum number recommended in Sensory Evaluation Techniques, 3rd Ed. by Meilgaard, Civille, and Carr for "duo-trio" type tests, which is what PC-ABX would be classified as in this book. I perform all tests of the same clip at the same volume.

Meaning of ABX results
Listeners are presumed to have heard a difference between the original and encoded clip (95% confidence) with 12 correct trials out of 16. To be extra conservative, I would say that I definitely heard a difference if my score is 14 or above out of 16. If I don't score 12 or above out of 16, that doesn't mean there aren't differences that can't be heard by others. Also, it doesn't mean that I will never detect a difference -- I could be fatigued, or distracted, or I might be able to hear differences at a different volume or through a different audio system.

 

Return to ff123's home page