WELCOME to the HEAD blog

open
[Translate to Français:] Four ears with different in-ear headsets

ANC Headset Testing

When making a phone call or using multimedia applications, using a headset has become widespread customary. With more and more people working from home, this trend will even pick up. Audio/video group applications such as MS Teams, GoToMeeting, and Zoom work best with high-quality headsets. Especially in mobile scenarios, adaptive noise cancellation (ANC) is a must-have feature. Headset makers use various ANC certification schemes from several application providers to qualify their products, and implementing new technologies requires more advanced technologies and optimization strategies, starting with using the proper artificial ear. The aim is to simulate the average human as precisely as possible in the lab and provide the most realistic test environment to address communication conditions and typical human behavior.

The right choice of artificial ear

The need for realistic measurements and high reproducibility of measurements conflict with each other: The more simplistic an ear simulator, the more reproducible the results are. However, measuring modern communication devices requires realistic simulations of the ear canal, the ear impedance, and the mechanical properties of the complex structure and geometry of the human ear. Noise-canceling headsets also require measurement equipment (especially the artificial impedance simulator microphone) with very low self-noise.
ITU-T has addressed these topics in its latest revision of Recommendation P.57, with two new types of artificial ears that model the concha and the ear canal entrance of human ears much more precisely than before. The recommendation also describes the requirements for low noise ear simulators, with an idle noise level below the diffuse-field hearing threshold.


The artificial head HMS II.3 LN- HEC, equipped with a type 4.4 artificial ear, fulfills both requirements ideally.

The anatomically shaped pinna simulator allows a natural fit, especially for intra-concha type headsets, since the concha bottom's shape and the ear canal's orientation are most human-like. With artificial ears that have a 90-degree ear canal orientation, they often can only be positioned by bending the artificial ear's rubber-made ear canal, leading to an unnatural fit.

[Translate to Français:] collage of HMS II.3 artificial heads with different types of ears

On the left, you see an HMS II.3 artificial head equipped with a type 3.3 artificial ear; on the right, you see an HMS II.3 LN-HEC fitted with a type 4.4 artificial ear that is superior for measuring intra-concha headsets

ANC Headset Test - Head Acoustics GmbH

See how much lower the noise floor of HMS II.3 LN HEC with a type 4.4 artificial ear (green) is compared to a traditional HMS II.3 with ITU-T type 3.3/3.4 artificial ear (blue) in relation to the human hearing threshold (grey)

Crucial setup requirements

It is essential to achieve almost symmetrical attenuation between the left and right ear, indicating correct positioning. And applying too much pressure to the headsets would not reflect the natural coupling to the human ears. We repeat any test with any headset at least five times. If the positioning is improper, the measurement will be imprecise and vague, especially at high frequencies.

ANC Performance

The insertion loss – the "noise damping" – is the most important measurement when determining ANC performance. There are three types of insertion loss: Passive loss means the damping a headset provides with ANC turned off, total loss is the passive damping of the headset plus ANC turned on, and active loss is the difference between the two as it states exactly how much more the ANC attenuates the exterior sound.

For such measurements, it is essential to simulate realistic background noise in a laboratory environment. But that is not enough: ANC systems react adaptively and adjust to the background noise situation with a certain delay. If we want to measure the attenuation in the adapted state, we condition the headset before the test with the background noise that is also used in the measurement.

[Translate to Français:] ANC headset test system set-up with ACQUA, labCORE, HMS II.3 LN HEC, 3PASSlab and labBGN devices, HMS II.5


This is how we set up our ANC headset test system. The analysis system consists of our ACQUA, labCORE, and HMS II.3 LN HEC as well as background noise simulation using our 3PASSlab and labBGN devices. The HMS II.5 artificial head can simulate an interfering talker for talk-through performance tests.

Some typical measurement results when conducting such tests are shown in the charts below, indicating total, active and passive insertion losses. Passive insertion loss for this intra-concha device is only marginal under 200 Hz (1) but can be improved in the frequency range from 30 Hz to 1 kHz by enabling the ANC functionality of this headset (2). However, enabling ANC may decrease the performance at higher frequencies (active insertion loss between 3 and 10 kHz in (3)).

ANC Headset Test - Head Acoustics GmbH

(1)

ANC Headset Test - Head Acoustics GmbH

(2)

ANC Headset Test - Head Acoustics GmbH

(3)

Passive, active, and total insertion loss, left/right, measured in a diffuse pink noise sound field for an intra-concha headset

The artificial ear's idle noise may influence the measurement if the headset's remaining noise is very low. The lower the self-noise of the headset, the more the artificial ear's idle noise will affect the measurement. The artificial ear’s idle noise may dominate the measured idle noise. Therefore, all types of ANC headset measurements should use an artificial ear that fulfills the low noise requirements as defined in the current ITU-T Recommendation P.57 and conforms to the impedance requirements specified for ITU-T type 4.3 and 4.4 artificial ears.

Talk-Through Performance

Many headsets provide a "talk-through" ANC mode to hear the environment better, for example, announcement systems. Of course, we also want to know how well this mode performs since manufacturers implement it in different ways into their ANC. You can see a typical test setup in the image above: An artificial head (head and torso simulator, or HATS) simulates the talker in a noisy environment. But how can we understand how well the talk-through mode works with ANC attenuating unwanted noise and letting a talker's voice through? We use a perceptual-based measure to determine the effort required to listen to a talker when wearing headsets. The ABLE (Assessment of Binaural Listening Effort) measure evaluates devices on a scale between 5 (no effort required) to 1 (no meaning understood with any feasible effort). It is a binaural method based on an average user's perception.

The table below shows an example of an intra-concha headset with ANC on, ANC off, and transparency mode turned on.

  • With ANC "on," the listening effort is high. We expect that since the ANC attenuates outside noises, including talkers: the loudness reduction of the outside noise is 20 phon.
  • With ANC "off," the listening effort is less, but the loudness reduction is less as well.
  • In "transparency mode," the listing effort is relatively low. We get a so-called Mean Opinion Score (MOS) of 3.0, which means it is much easier for the user to listen to an external talker. However, in this situation, the loudness reduction of the external noise is just 1,7 phon.

 

ANC on

ANC off

Transparency Mode

 

MOS-LE (ETSI TS 103 558 / ABLE)

1,9

2,4

3

MOS

Loudness Reduction

(ISO 532-1, Zwicker method)

20,1

15,7

1,7

phon

 

We measured the listening effort with ABLE for an intra-concha headset in different modes and corresponding loudness reduction using airplane noise.

Consequently, headset manufacturers leave it up to the users to choose the setting adequate for the situation. This measurement combination provides the most relevant information for headset manufacturers to implement the talk-through functionality the way they deem best.

Sound quality

Sound quality is vital for audio playback and speech communication as well. We measure sound quality physically correct: We use a human ear reproduction and an accurate simulation of the average human ear impedance, carefully positioning the headsets on the artificial head. For a perceptually correct measurement, we must choose the proper HATS equalization in Receiving direction. The most often-used equalization is the diffuse field equalization. When applied, the response characteristics measurement should be almost flat – of course, within certain limits and not considering the manufacturer's sound reproduction flavors. In the picture below, you can see a typical example of a frequency response characteristic measured with all hearable frequencies:

ANC Headset Test - Head Acoustics GmbH


Response characteristics for an intra-concha headset

The left and right ear characteristics match pretty well – that is good. We can see a roll-off in the high-frequency and a dip at 10 kHz. However, we cannot make final judgments about speech or audio sound quality based on this single measurement. That is why we use the perceptually motivated (based on actual human evaluation) procedure MDAQS for audio quality evaluation. MDAQS provides better insight into the true sound quality using simple-to-understand scores for timbre, distortion, immersiveness, and overall quality.

Intra Concha headsets

Timbre

Distortion

Immersiveness

Overall

H 1

2.5

2.9

2.6

2.3

H 2

4.7

3.4

4.3

4.4

H 3

4.4

3.4

3.4

4.1

H 4

4.7

3.3

3.9

4.3

 

Audio Quality (in MOS) measured with MDAQS for different intra- concha headsets with H 2 achieving the best result overall

Speech quality

A measure providing perceptually well-correlating results for speech quality in noise is 3QUEST, allowing to determine the speech quality, the noise performance, and the overall quality separately. Again, using a diffuse-field equalized artificial head equipped with a low-noise ear simulator correctly replicating the human ear impedance is the technical basis. You can see example results for one intra-concha headset with different background noises in the table below.

 

Road

Cafe

Train

S-MOS

3,4

3,4

3,7

N-MOS

3,2

3,2

4,6

G-MOS

2,8

2,8

3,6

 

We measured Speech Quality (S-MOS), Noise intrusiveness (N-MOS), and Overall Quality (G-MOS) with 3QUEST for an intra-concha headset and with different noises

The headset we measured achieves the best speech quality with train noise, while noise cancellation with road noise and café noise works not as well – but the speech quality with all background noises is relatively high.

Meaningful testing made real-life-proof

HEAD acoustics makes measurements for all use cases of ANC headset testing possible. ANC headset testing requires an artificial head with realistic artificial ears, low self-noise microphones, and correct equalization. In addition, a setup for realistic background noise simulation and appropriate background noises are required. These measures can be combined with a second HATS, simulating an external talker. High precision, consistency, and a holistic approach make our solution unique.

In-depth reading on some of the standards and recommendations mentioned above

Talk to us