Notes by ff123
7/27/02: Test is closed. Results are here.
07/19/02
Instructions:
1. Visit the ABC/HR practice page, and download the practice test sample. You will need to do this in order to create the appropriate directories, obtain abchr.exe, flac.exe, oggenc.exe, and oggdec.exe. Familiarize yourself with how to perform an ABC/HR comparison.
2. Visit the Test Sample Download page, and obtain one of the twelve samples selected for this comparison. The web page will randomly select one for you. If you've already tested that sample, refresh the page and choose another.
3. email your results files to me at: miyaguch at eskimo dot com
Test Plan:
The first group test I would be interested in after the official release of Ogg Vorbis 1.0 is at 64 kbit/s (or the setting which produces files close to this bitrate).
1. The test would be conducted using my ABC/HR utility, which means that only people who have access to a Windows 32 environment would be able to participate.
2. I plan to use about a dozen 10 to 20 second samples of music. The samples would be from different genres of music and would not be specifically chosen for their ability to highlight deficiencies in the various encoders. That is, every effort will be made to choose representative samples.
3. Encoder participants: Ogg Vorbis is entered twice. the -b 64 setting allows it to compete on an equal footing with the other constant bitrate codecs, but the -q 0 setting is probably how most listeners will choose to use the codec.
a. Ogg Vorbis 1.0, using -b 64 --managed
b. Ogg Vorbis 1.0, using -q 0
c. MMJB 7.2 mp3pro 64
d. WMA8 at 64 kbit/s (Using WMP 7.1 to encode and Winamp 2.6 to decode)
e. Quicktime 6.0 MPEG-4 (AAC Low complexity) at 64 kbit/s
4. Average bitrates: average bitrates produced by the proposed settings will be verified and adjusted if necessary by encoding multiple CD's of various musical genres.
5. Time alignment and volume alignment: The files will be aligned in time by using the offset correction within the ABC/HR program. Volume alignment will be verified for one file before the test proper begins.
6. Clipping: No special actions will be taken to reduce artifacts caused by clipping due to the encode/decode process.
7. Pre-test training: I will prepare a web practice page walking a new user through the operation of the ABC/HR utility. There will be a pre-test training session to familiarize test participants with how a listening test is performed.
8. Interim group results: Interim group test results will be periodically made available. However, the first interim results will not necessarily be disclosed until a significant number of listeners have participated. This is to encourage people to participate. Of course, individual results are immediately available to each listener.
Return to ff123's Home Page