Of course, you can't design (or evaluate) a speaker on measurements alone; you have to listen to it. The challenge is to eliminate variables like listener bias, program content, room acoustics, speaker position, listener and volume level. Any of those factors can easily distort subjective listening tests, said Senior DSP Engineer Adrian Celestinos in his presentation on room acoustics.
At low frequencies, where wavelengths are close to room dimensions, acoustic reflections create room nodes, reinforcing certain bass sounds in some locations, and cancelling them in others. Room nodes can't be eliminated, but they can be mitigated.
The problems are different at mid and high frequencies. A room with an abundance of hard surfaces will have high reverberation time, compromising clarity, adding harshness and creating uneven response. A room full of sound-absorbing upholstery and carpeting will sound dead. As Celestinos notes, "Lateral reflections give a sense of spaciousness."
The Samsung Audio Lab has two listening rooms, one a reference listening room as defined by the ITU (International Telecommunications Union), and the other an average living room as defined by the IEC (International Electrotechnical Commission). The dimensions of the two rooms are calculated to distribute bass nodes as evenly as possible. And they contain sound absorbers and diffusers so that their acoustic properties meet ITU and IEC specs. This makes the results consistent and repeatable, and provides as much confidence as possible about how the speaker will perform in the field.
Other variables, such as product biases and listener experience, are minimized by the test procedures. Most tests involve four speakers, placed on a rotating platform behind an acoustically transparent curtain, playing 30- to 50-second musical clips with wide frequency and dynamic range. Listeners hear only one speaker at a time, in mono. After one clip, the next speaker under test rotates to the front, and the same clip plays again. Playback levels for each speaker are carefully matched. Devantier says the Lab would like to be able to do stereo listening tests by year-end.
Before participating in a subjective test for the first time, subjects undergo a hearing test, followed by listener training using Harman's How to Listen software. "With trained listeners, there's less variance in results, so we can test with fewer people," explains Elisabeth McMullin, Lead Acoustic Researcher - Competitive Benchmarking.
During her presentation, McMullin recounted a couple of recent subjective tests involving the new Radiant-360 R7. In one, participants used a rotary controller to compare different equalization curves. "We wanted to know if the R7 is as good as it can be," she explained. "Would EQ help its performance?" This yielded useful information; the panel's preferred settings have been implemented in the final product via a firmware update. In the second round of tests, students from the nearby California Arts Institute compared the R7 with three other speakers, ranking the speakers' performance on a scale of one to 10.
OUTSIDE THE BOX
Many other audio manufacturers, including Canadian brands like PSB and Paradigm, use the same basic methods as Samsung, combining anechoic measurements of on- and off-axis response with blind subjective testing. But Devantier has developed some unique twists for Samsung, notably an anechoic chamber and listening room specially designed for flat-panel TVs and soundbars.
Speakers like the R7 are measured in a conventional 360° chamber with the speaker placed in the centre. But the Samsung Audio Lab also has a 180° chamber designed for products that will be used on or near a wall. Five walls have sound-absorbing wedges. Engineers mount the product under test to a heavy horizontal door, then swing it shut and begin measurements. A test suite takes 300 measurements in 10 minutes using an array of 15 microphones on a pivoting boom that moves through a half-sphere. This lets engineers measure how products like TVs and soundbars will behave when set next to a wall. They can also attach a simulated tabletop below the TV or soundbar, so they can measure how it will perform when set on a credenza. Devantier says the company who built Samsung's anechoic chambers had never seen a design like this.
The Lab's second listening room allows for blind listening tests of TV sound systems. It contains a rotating six-foot wall that can accommodate TVs up to 65", and soundbars as well. "It's the only one like it in the world," Devantier says.
Last year, Samsung mandated the Audio Lab to test the performance of its premium 2015 TVs. The project was a success. "All TVs sold in North America will be tweaked here," Devantier says. "We're making our existing designs better through EQ."
During our tour, we got to hear the results for ourselves, comparing the sound of last year's top-of-the-line U9000 Ultra-High-Definition TV with this year's JS9500. The sound of the newer model was unmistakably better: smoother and less edgy, with better warmth and clarity.