Dogies.wav Listener Comments
Answers: 027 = WMA, bitrate = 132 106 = AAC, bitrate = 132 235 = MPC, bitrate = 143 740 = XING, bitrate = 128 854 = OGG, bitrate = 133 875 = LAME, bitrate = 128 -------------------------------------------------------------------- Listener 11 Here are my results (but noting that due to a bad cold my hearing is REALLY screwed up at the moment): dogies106.wav: 3.6 dogies235.wav: 5.0 dogies740.wav: 2.8 dogies854.wav: 5.0 dogies875.wav: 5.0 Thanks for doing this - results should be interesting... -------------------------------------------------------------------- Listener 9 (ff123) 027: 4.0 (15/16) slight noise pumping 106: 5.0 235: 5.0 740: 1.5 didn't bother: flanging in second half of sample 854: 3.5 (16/16) echoey sound in piano 875: 3.0 (16/16) wavery piano, low freq noise in second half of sample -------------------------------------------------------------------- Listener 1 (Garf) 106: 8/10 (focussing hihats) 8/10 (foussing background) I can hear it's lowpass filtered on the hihats and there is a difference in the instrument on the background (the one with the 'fuh' sound). Just relistened after hearing the others. This one phases too. I give this 2.5. There are 3 perceptible errors but they are not that annoying. 235: Sounds perfect to me. This is a 5.0 027: 9/10 Distortions (flanger?) on the hihats. I give this a 1.5. It's bad, but there exists worse. 740: 10/10 Total crap. I give it 1.0 and still think it's overrated. 854: 9/10 This one gets hissy instead of flangering/warbling/fluttering like like the others. Must be Vorbis point/phase stereo crap. The distortion is better, but I hate hissing, so it gets 1.5 too. 875: 9/10 It feels like the sound phases in or out. Yuck. Another 1.5 ------------------------------------------------------------------ Listener 5 (Hans Heijden) I spent about half an hour listening to the samples, no abx-ing, just comparing all in the winamp playlist. Then rated them. The ratings & short comment about what distinguished them from the original: 027 - 1.0 - serious noise pumping, pre-echo & lowpassing 106 - 4.5 - hiss slightly amplified 235 - 4.0 - guitar has 'ghost' sounds 740 - 1.0 - very waterly cymbals, pre-cho 854 - 2.0 - hissy, slightly blurred cymbals 875 - 3.0 - pre-echo Xing? - lqt - mpc - wma? - ogg - lame Don't worry, the encoders I figured out afterwards (of course I can't wait for days!) Well, mpc was quite good though bitrate was 142 k. I regret that pre-echo was heard with lame, what to do about that? Though I tried ogg for the first time yesterday (RC2) I was already pretty sure 854 would be ogg, with that added hiss. Good sample BTW, has a bit of everything, but shouldn't be longer! *** As you see, I mixed up wma and xing, because I don't use these for a long time and forgot their sound. Sheez how can you rate wma/027 so high, for me it's utter wrongness! Again it seems piano sound has never been critical to me. Now you said it, I hear that 'wavery' also a little in lame/875. Probably -ns-bass -8 is still not enough for that. ------------------------------------------------------------------ Listener 8 I just listened to the samples a few times. I'm not sure at all of my results yet, so i might be wrong. Well, here are my first impressions : dogies027 : 2.0 - WMA => it sounds as a metallic sound (thats why i thought about WMA). Some harmonics seem to miss + bad high frequency response dogies106 : 5.0 - AAC => i can't tell the difference dogies235 : 4.0 - LAME => there's something wrong but i cant tell what :/ dogies740 : 1.0 - XING => same as 027 one + a horrible flanger effect at 13''-14'' dogies854 : 4.0 - MP+ => the original background noise sounds higher dogies875 : 3.0 - OGG => pre-echo + a bit flanger effect ------------------------------------------------------------------ Listener 4 (r3mix) dogies027.wav 0.9 -> metal sounds in cymbals at 7s 9s 11s most obvious and very annoying. at the start also some bass distortion, but not realy annoying. clearly the worst imo. dogies106.wav 4.1 dogies235.wav 3.0 dogies740.wa v 2.0 again cymbal problems like in 027, less outspoken dogies854.wav 1.5 too harsh an overall sound most obvious when the drums kick in at 7s. it gives me an unpleasant listening sensation, like a certain pressure on my ears. I find myself wrinkling my forehead during the painful sensation. dogies875.wav 3.0 don't really like the overall sound I'm sorry ff123, just ran all samples 4-5 times and there is no such thing as "slightly annoying" if you start to focus. *** the ratings: I changed some to moderate aferward. just to scale in relation to the very bad 1st one. Then, the 4.1, sorry I should have done a ABX, but I think I can hear this sounds different but not annoying really, just different. then the 3.* was less agreable to listen too, hard to explain really, it's just probably some noise somewhere I do pick up but can't really define or focus on. It sounds compressed, but not annoying. ------------------------------------------------------------------ Listener 6 027: 3 - a little muffled, quiet xylophone at start (I think that's what it is..). 106: 4 - a bit better than 027. Just doesn't sound quite so dynamic. Probably the best to my ears. 235: 2-3 - quite warbly, gets a bit on my nerves. Otherwise sounds good. 740: 1 - The worst of the bunch. Warbly, too much zing and sounds muffled. 854: 3: A bit muffled, otherwise sounds okay. 875: 2: High end sounds muffled, a little warble. All tests done with a pair of Sennheiser 580s, with a sb live value as source. I found it a little hard to completely distinguish the differences with the type of music and the length of the sample. I wouldn't really like to listen to any of them except for the original.. Hope this helps. ------------------------------------------------------------------ Listener 3 (JohnV) Dogies027: ABX 16/16 Pre-echo with drums. First 1-2s not so obvious, but increasing all the time. At 5-6s clear pre-echo with drums (although thereīs slight "pre-echo-like" hiss in the original at 5-6s as well). At 7.5s "wooden" hit has also some pre-echo. Also thereīs somewhat harsh sound (or ringing) with the noisy background (hihat echo from 7s to the end). Scale = 2.7 ------------------------ Dogies106: ABX 16/16 Some ringing/harsh sound from 7s-end (noisy hihat in the background). Some pre-echo with drums. Scale = 3.0 ------------------------ Dogies235: ABX 16/16 Some ringing/harsh sound after 7s with hihat backgound echo. Only slight pre-echo with drums. Scale = 3.2 ------------------------ Dogies740: ABX - didnīt bother (16/16) Pre-echo problems all the way through the clip. From 7 to end bad warbling with hihat echo. Scale = 1.0 ------------------------ Dogies 854: ABX - 10/10 didnīt bother further Slight artifact at about 7s. From 7 to end background (hihat echo) sounds noisy. Not much pre-echo. Scale = 2.6 ------------------------ Dogies 875: ABX - didnīt bother (16/16) Pre-echo problems through out the clip. From 7s to the end backround hihat echo sound is warbling and/or ringing. Scale = 2.0 ------------------------ Ranking from best to worst: 1. Dogies235 2. Dogies106 3. Dogies027 4. Dogies854 5. Dogies875 6. Dogies740 It wasnīt easy to rank the first 4 although there are lots of differencies. Itīs not so clear which sounds (among the first 4) the most irritating and which is the best. Sometimes I notice more the pre-echo, sometimes more other problems... ------------------------------------------------------------------ Listener 14 Here are my ABX results for dogies: 027 - 5/16 106 - 10/16 235 - 8/16 740 - 13/16 first test & 16/16 second test 854 - 14/16 875 - 11/16 Comments: I listened in numerical order, straight through until my ears became fatigued during the 740 test. I had to take frequent breaks while testing thereafter. I repeated testing 740 at the end, because I thought the cymbals in 740 sound the worst to me, and I could do better on the ABX the second time around with rested ears. I hear an artifact on the first cymbal hit with 854. I really do believe that 027 has the best sound, not just because my ABX results indicate that. I knew I was guessing all 16 times while testing 027. 106, 235, and 875 all pretty much sound the same to me. The keyboard sounds the same to me in all 6 samples, and it wasn't until the cymbals started before I began to hear any differences. I had difficulty hearing differences because the keyboard impared my ability to judge how natural the cymbals sounded. Rating scale: 027, 106, 235, 875 - 5.0 740 - 3.0 854 - 4.5 ------------------------------------------------------------------ Listener 13 hmmm this is upsetting to me, but here are my results. dogies027 WMA? Dynamic lowpass?, 4.0 dogies106 sounds pretty good to me, 5.0 dogies235 sounds pretty good to me, 5.0 dogies740 sounds faintly underwaterish 3.8 dogies854 overemphasizes the hiss, 4.5 dogies875 sounds good 5.0 I thought I heard something in dogies875 but I just couldn't place it, and I might have been tricking myself I bet you anything I gave Xing a 5.0 ;-( I went back to listen to dogies740 again what I'm hearing, I think it's the only one owith it it's the most disconcerting, to me, of all possible artifacts even being so subtle, as it is to me, I still rated it 3.8 even though I can just barely barely barely make it out ------------------------------------------------------------------ Listener 15 dogies027.wav : 5.0 dogies106.wav : 5.0 dogies235.wav : 5.0 dogies740.wav : 5.0 dogies854.wav : 5.0 dogies875.wav : 4.0 (ABX: 13/16) In some of the first 5, I thought I could hear some distortion of the hi-hat, but I couldn't identify it consistently enough. In the 6th one, I was able to hear it consistently enough to identify it in ABX testing. I hope that subsequent test samples will be encoded at a lower bitrate, or that they'll be more challenging to the encoder so that those of us without "golden ears" aren't so frustrated. ------------------------------------------------------------------ Listener 7 ok, after listening to the samples here is my conclusion: 875 = 2 854 = 3 740 = 2 235 = 4 106 = 3 027 = 3 it becomes quite clear that none of them sound exactly like the original, but some samples are "bearable" listening to (this clip/song at least) ------------------------------------------------------------------ Listener 2 (Filburt) 027: 1.8, sounds reedy, cloudy, and a bit distant. Has a lot of added noise. Quite a departure from the original 106: 2.7, Fluttery, but keeps the audio reasonably intact. Higher frequencies are a bit weird but it's not terribly bothersome 235: 3.5, Overall sound is fairly close to original...but a bit of added hiss 740: 1.0, very little hiss, so little it's even less than the original. There's washed out tones, fluttering, and a bit of blippiness 854: 2.5, has a bit of flutter in the music..and a bit of reediness. Would be a 4 though except it is very noisy 875: 2.2, quite unstable, fluttery, almost watery. It feels as if only mids and high mids are really there. However it sounds better than sample 027 btw "reedy" is caused by pre-echo ------------------------------------------------------------------ Listener 12 Your listening test setup intrigued me, so I decided to participate. Here are my results of the 'dogies.wav' listening test. I'm probably not the most discriminating listener, as you can tell from the results, but for what it's worth, here they are: File Score Comments 027 5.0 106 4.8 bass notes seemed a bit weaker (12/16) 235 5.0 740 4.0 dull ride cymbal (16/16) 854 4.2 'hissy' ride cymbal (15/16) 875 4.0 shaky, fluttery beginning (16/16) I don't have ABX (I use Linux), so I wrote up a little randomizer to emulate the method. The x/16 scores are for all intents and purposes 'ABX equivalent'. ------------------------------------------------------------------ Listener 10 (2BDecided) file MOS ABX Hear problem? Description 027 4.5 16/16 fair noise at start 106 4.7 7/8 hard 1 cymbal hit pans right 235 3.7 8/8 easy unsteady sound 740 3.5 8/8 easy bass drum smeared, cymbals horrible! 854 4.0 8/8 fair/easy everything SLIGHTLY wrong (from bass drum to cymbal) 875 3.8 8/8 very easy unsteady at start I didn't have the codes in mind as I listened, but looking at the list in doggies.txt, 740 sounds like Xing, but I didn't hear a "WMA" sound in there - just proves how much I really know! ------------------------------------------------------------------ Listener 16 I think they all rank 5. Unless I post again with some ABX results put that into your stats then dunno if a flatline 5.0 would do much for your stats though Even if I spend serious time doing ABX tests none of those clips would ever have a prayer for dropping under 4.0 Repeat: score = 5 for all test clips in 128kbps test OK, I'll tell you what I listen with: I listen with cheap sony earbuds EQed for flatness (think you'd know how I do it if you've been following the board )