Listening Tests at Low Bitrates

Notes by ff123

 

Caveats: Please note that although I can hear pure tones up to a little over 16 kHz, I cannot hear the difference between a WAV ripped from CD and the same WAV brickwall filtered at 12 kHz (see my page, Test Your Hearing, Part 3). Take that into consideration as you evaluate what my results might mean. Also, as always, the music sample selection has a great influence on the outcome of codec listening tests.

Setup: Fanless computer operation. Grado SR325 Headphones. M-Audio Audiophile 2496 soundcard. MP3 decoder: in_mp3.dll from Winamp 2.76. WMA decoder: in_wm.dll from Winamp 2.6.

Conclusion: Coding Technologies touts mp3PRO at 64 kbit/s as having quality equivalent to regular mp3 at bitrates between 96 kbit/s and 128 kbit/s. Using one of Fraunhofer's best codecs at 96 kbit/s and 112 kbit/s (MMJB 6.1 "very high," or Alternate codec), I'd say mp3PRO quality weighs in nearer to 96 kbit/s equivalent than to 128 kbit/s equivalent. For people with good high-frequency hearing, mp3PRO may sound as good as or perhaps even better than 112 kbit/s regular mp3 in some cases (if they can hear 13 kHz and above in actual music, and heavily weight that part of the frequency spectrum). But for the most part, I would choose the regular mp3 at 112 kbit/s or even at 96 kbit/s, rather than mp3PRO or WMA8 at 64 kbit/s.

See what other people think about mp3PRO's sound quality.

 

This is an acoustic jazz track called "Tam's Jam" from Michael Garson' Serendipity (RR-20CD). It's not a particularly difficult piece to encode with mp3 at 128 kbit/s, which is part of the reason why I chose it -- I didn't want to use a "killer" sample. Even so, at low bitrates, I can hear problems with various codecs. The 13 second piece I used to test can be found on my Audio Samples Page. The file is called "tamsjam.flac" and has been losslessly compressed using FLAC. MP3Pro lies somewhere between 96 kbit/s and 112 kbit/s regular mp3 in quality for this sample (with my ears). It's possible that others may find mp3PRO to be better than the 112 kbit/s regular mp3 by virtue of its more extended frequency response (which I can't hear).

Encoder bitrate settings ABX results Comment
MusicMatch Jukebox 6.10.0165 mp3 (FhG Alternate codec) 112 kbit/s "very high" 15/16 Slight waver on the opening piano chords. some shifting of the image during the middle of the passage. Best of the codecs/settings tested.
RCA mp3PRO demo (version 1.0 build 84, Jun 12, 2001) 64 kbit/s extra high quality, allow intensity stereo 16/16 Some flanging. Second best
MusicMatch Jukebox 6.10.0165 mp3 (FhG Alternate codec) 96 kbit/s "very high" didn't bother Some collapsing of the stereo image. Wavering artifacts. Second worst.
Microsoft WMA (version 8.0.0.0302) 68 kbit/s   didn't bother Sounds very different from the others, with lots of fluttering artifacts. Worst of the bunch.

 

Ok, now for a killer sample: applaud.wav, which can be found on the Lame Quality and Test Information page. Applause is extremely difficult for encoders, and this is in some ways an unfair sample to choose, because it may not be representative of the type of music many people listen to.

Encoder bitrate settings ABX results Comment
MusicMatch Jukebox 6.10.0165 mp3 (FhG Alternate codec) 112 kbit/s "very high" didn't bother Sounds "noisier" (as in more hiss) than the original. Still, obviously the best.
RCA mp3PRO demo (version 1.0 build 84, Jun 12, 2001) 64 kbit/s extra high quality, allow intensity stereo didn't bother Very bad -- practically unlistenable. Metallic, blurred, yuck. Second worst.
MusicMatch Jukebox 6.10.0165 mp3 (FhG Alternate codec) 96 kbit/s "very high" didn't bother More nasty hiss, but it's confined to the center. Gives the impression that the stereo image has collapsed, although there are still sounds on the left and right. Still better than mp3PRO or WMA. Second best.
Microsoft WMA (version 8.0.0.0302) 68 kbit/s   didn't bother Gets progressively worse as the sample plays. By the end there are extraneous noises on the left and right that sound like a blaster battle from Star Wars. Worst.

 

Theme from the Deerhunter, featuring solo acoustic guitar. From a Telarc Sampler Disc. The 30 second piece I used to test can be found on my Audio Samples Page. The file is called "deerhunter.flac" and has been losslessly compressed using FLAC. This sample is transparent to me using normal mp3 at 96 kbit/s.

Encoder bitrate settings ABX results Comment
MusicMatch Jukebox 6.10.0165 mp3 (FhG Alternate codec) 112 kbit/s "very high" didn't try I already can't tell the difference at 96 kbit/s
RCA mp3PRO demo (version 1.0 build 84, Jun 12, 2001) 64 kbit/s extra high quality, allow intensity stereo didn't bother Weird sound at the beginning of the guitar plucks. The decay of the notes doesn't sound right. Worst.
MusicMatch Jukebox 6.10.0165 mp3 (FhG Alternate codec) 96 kbit/s "very high" 11/16 I can't tell the difference!
Microsoft WMA (version 8.0.0.0302) 68 kbit/s   16/16 I can hear a bit of "pumping," or an increase in the loudness of background sounds during the transients of the guitar plucks. Second worst.

 

A Rammstein (heavy metal) tune uploaded by cd-rw.org to his account at http://www.saunalahti.fi/~ljpp/lame/ as 03.pac. This is a complex sample with loads of high frequencies, distorted guitar, and a sound that pans from left to right.

Encoder bitrate settings ABX results Comment
MusicMatch Jukebox 6.10.0165 mp3 (FhG Alternate codec) 112 kbit/s "very high" 15/15 Oops, missed that last trial. Pretty close to the original, given my high freq limitations. Original has a section which distorts more than the encode. Best
RCA mp3PRO demo (version 1.0 build 84, Jun 12, 2001) 64 kbit/s extra high quality, allow intensity stereo didn't bother Sounds are blurred together, such that the guitar distortion is lost in the beginning section. Second worst, but not by much.
MusicMatch Jukebox 6.10.0165 mp3 (FhG Alternate codec) 96 kbit/s "very high" didn't bother Collapsing of stereo, but second best otherwise.
Microsoft WMA (version 8.0.0.0302) 68 kbit/s   didn't bother Same blurring of sounds as in mp3PRO. Ending section is worse, with chirping at the beginning of the downbeats. Worst.

 

This is from a 70's pop tune, Good Time Charlie's Got the Blues, by Danny O'Keefe. I thought it should have been simple enough, but none of the encoders came close to sounding like the original. The 13 second piece I used to test can be found on my Audio Samples Page. The file is called "charlies.flac" and has been losslessly compressed using FLAC. The normal mp3 at 112 kbit/s is probably "best" (it still sounds bad), but I can't decide how to rank the others, which all sound very bad.

Encoder bitrate settings ABX results Comment
MusicMatch Jukebox 6.10.0165 mp3 (FhG Alternate codec) 112 kbit/s "very high" didn't bother Sounds like tape flutter; an annoying gravelly sound during the singing; Probably the "best," though.
RCA mp3PRO demo (version 1.0 build 84, Jun 12, 2001) 64 kbit/s extra high quality, allow intensity stereo didn't bother Same metallic sound as the WMA. Part of a three-way tie for worst place.
MusicMatch Jukebox 6.10.0165 mp3 (FhG Alternate codec) 96 kbit/s "very high" didn't bother Same as the 112 kbit/s case, except more of it. Ties for worst place
Microsoft WMA (version 8.0.0.0302) 68 kbit/s   didn't bother metallic (sound of sharpening knives). Ties for worst place.

 

Electronica music (Rushing, by Moby): hihat and piano in the beginning. The crackling sounds you hear later are supposed to be there. The 15 second piece I used to test can be found on my Audio Samples Page. The file is called "rushing.flac" and has been losslessly compressed using FLAC.

Encoder bitrate settings ABX results Comment
MusicMatch Jukebox 6.10.0165 mp3 (FhG Alternate codec) 112 kbit/s "very high" 16/16 piano sounds wobbly in places. "Best."
RCA mp3PRO demo (version 1.0 build 84, Jun 12, 2001) 64 kbit/s extra high quality, allow intensity stereo didn't bother horrible pre-echo on the hihat; this part is worse than WMA8. The second part is better than WMA8. Tied for worst.
MusicMatch Jukebox 6.10.0165 mp3 (FhG Alternate codec) 96 kbit/s "very high" didn't bother much better than either mp3PRO or WMA8. However, piano sounds wobbly all the time. Second best.
Microsoft WMA (version 8.0.0.0302) 68 kbit/s   didn't bother hihat is smeared, but not nearly as badly as mp3PRO; metallic artifacting during the second part. Tied for worst.

 

Return to ff123's Home Page