This article highlights key considerations in measuring the quality control of audio products and devices incorporating audio, with an emphasis on manufacturing test strategy and objectives.
Creating the manufacturing test strategy for an audio device can be regarded as the need to balance costs – specifically the cost of test against the cost of failure. While simple in concept, this can be a far more complex equation involving several variables for both “cost of test” and “cost of failure.” From a cost-of-test perspective, the expense is measured in time — as defined by setup requirements and test methods — and money, the cost of the overall test system, including accessories and fixtures. Determining the cost of failure ranges from the expense of warranty repairs and product returns to the more imprecise, but still very real, impact to brand perception. In this article, we review these considerations for production-line measurement and then speak to applied strategies for efficient testing.
Manufacturing test objective
The goal of manufacturing test is to ensure that the products manufactured and shipped — whether to distributors, retail outlets, or directly to customers — perform as intended. However, uncontrolled variability or tolerances — whether in materials and components, or in assembly processes— can negatively impact the ability to achieve said objective. With this in mind, the key question is: what is the most effective way to test devices to ensure they meet specification?
If manufacturing test is viewed as a range, it is anchored on one end by absolute quality assurance via full functional testing of the device. In this approach, the test strategy is defined by the product’s functional specification and every product assembled is tested to ensure each element — connectors, speakers, microphones — work correctly and to specification. Such a test strategy is quite expensive, in terms of required equipment and fixturing, as well as the production time devoted to testing.
At the opposite end of the range is simply not testing the device or product at all. Such a strategy is likely in cases where the product is so inexpensive that investment in testing is not justified. In the case of such devices, the customer is effectively the quality control process. Field failures and customer returns are the primary indicators that there is a quality problem.
Between these two extremes – full functional test or no test at all — is the multifaceted assessment comparing the cost of test and the cost of failure. Depending on the value of a product (or brand), ensuring no sub-par products reach end customers may be a key factor in an overall product strategy. The determination of just how critical that is, or isn’t, defines the manufacturing quality standard and informs test strategy.
When and where to test
Modern electronic devices are rarely assembled and manufactured from only in-house materials, with the majority coming together as finished products from the materials, components, and sub-systems of many suppliers. If testing isn’t performed until the device is fully assembled, known as final test, any failures can lead to the added expense of re-work or even complete scrappage of a finished device. If components and sub-systems are tested but no final evaluation is performed, a faulty system may end up delivered to a customer. How can a product built completely from good components fail? Mis-assembly or damage during assembly are possible causes.
With the above in mind, answers to the question of “where to test?” can cover everything from each point in the supply chain and manufacturing process to not testing at all. Returning to the earlier cost analysis of test vs. failure that determined both the quality standard and test strategy, if that assessment determined that some production test must be performed, “where to test?” is often best answered by where failures are occurring. The quality teams of many manufacturing organizations start with extensive testing during a product’s development. Frequently referred to as Engineering, Design, and Production Validation stages (EVT, DVT, and PVT respectively), these phases are used to initially define, and subsequently optimize, where, and for what, testing should be performed. Besides confirming the performance of the design and qualifying components, this approach helps to determine if incoming material quality and in-house manufacturing processes call for more, or less, testing in specific areas.
The lab vs. the line
A common aspect of the transition from development to assembly is the need or desire to directly correlate measurement results from the manufacturing line with those from the product development lab. This creates inevitable tension since measurements made during product development — intended to validate and characterize system performance — usually take far more time to perform than can be tolerated, or afforded, within a production environment. An example of this challenge is trying to compare the free-field frequency response (lab) of a speaker to the same speaker’s response inside a compact test cell (production line). While direct comparison of lab and line measurements is often challenging, using similar test systems, or least systems sharing a common software and measurement methodology, can go a long way to reducing, or even eliminating, this tension.
Aside from the use of a common measurement platform, the best strategy to negotiating this potential conflict and closing any correlation gap in measurement results is the development of a test standard and the use of reference (aka, “golden”) units. With this approach, known good units are evaluated in the lab as well as on the production line (or lines). The results from both measurement sites are then compared and correlated. If test results cannot directly be compared, for example because the acoustic test fixtures are different, transfer functions can be created that adjust for the difference between different fixtures. It is worth noting that a separate transfer function for such comparison must be developed for each specific device model as a transfer function is only valid for a given product.
When evaluating the performance of a purely electronic device — power amplifier, headphone amplifier, DAC, ADC, or really any device with an electrical input or output — one thing to note is that even the simplest test will yield valuable data. That is, just measuring level, distortion, and noise with a single tone such as 1 kHz effectively yields a lot of information. A single-tone test can be performed easily and quickly, likely in as little as one second, and will functionally verify most of an electronic signal chain. It verifies that the power supply is functional, that the signal path and all electronic components in it are correctly installed and, by measuring the added spurious noise, verifies that many adjacent devices are not defective. Adding a few more test tones or points for practical purposes provides almost complete validation. A classic approach for a full-bandwidth electronic device is a test with points at 20 Hz, 1 kHz, and 20 kHz. Testing at additional frequency points will verify that output coupling filters are correct. In electronic devices, beyond basic functionality, a classic cause of fallout is variability or tolerance in the passive analog components (i.e., the capacitors, resistors, and inductors), particularly in the output filters used in power and headphone amplifiers. By choosing test frequencies at the corner points of the filters you can easily verify that these components are correct without having to perform a complete frequency sweep of the device. In addition, if you offset the frequencies in each channel, for example testing at 19.5 kHz in one channel while testing at 20.0 kHz in the other, you can simultaneously identify any crosstalk.
Acoustic devices – speakers, earphones
In the case of speakers, as well as earphones and balanced armature drivers, testing the device at one frequency provides only minimal utility. More commonly, and more effective in determining performance, these devices are swept across a range of frequencies of interest. To reduce test time in a manufacturing environment, and since uncorrelated noise is not a useful measurement on a speaker driver, the frequency sweeps can be very fast, with a few second-long sweep usually enough. When testing a speaker driver, it is useful to examine the acoustic (frequency) response of the device and the impedance function. The combined results of these two measurements will essentially reveal whether the system is performing to specification or not.
Returning to the discussion on where/when to test, incoming inspection of raw speaker drivers can be easily performed via an electrical-only measurement of impedance. Since any deviation in mass of the cone, or compliance of the suspension, will shift the impedance curve — particularly at the frequency of the resonant peak — this approach can be used to quickly eliminate devices with common assembly errors before they end up in a finished product. This method — measuring just impedance — significantly simplifies the incoming test requirement, as it removes the need for an acoustic test box or measurement microphone, and considerably simplifies test setup. For best measurement repeatability, impedance should be measured at small signal levels, where the motion of the driver is within the linear, pistonic range of motion. It is worth noting that this leads to a common disconnect between R&D and manufacturing, so it is worth remembering that this technique is focused on incoming, or pre-assembly, test, not necessarily final test. In the latter, drivers are usually measured at high signal levels, to test the driver through their full range of motion. However, the impedance curve of a driver at high signal levels cannot be compared to the typical small signal used when calculating Thiele-Small parameters, the more common measurement made in R&D.
Modern audio analyzers can usually measure all the important parameters of a speaker in a single fast sweep. With a measurement microphone and electrical connection, a speaker’s acoustic frequency response and distortion can be verified along with the impedance and resonance, rub and buzz or high-order harmonic distortion, and even polarity can all be checked in a single test.
Acoustic devices – microphones
A microphone is often tested just for frequency response, usually with a swept signal across the frequency range of interest. This can be used to digitally calibrate the sensitivity of the microphone for noise cancelling and phase data can also be captured for microphone beam forming. A second test is a seal test. For this measurement, the microphone is swept twice, with the acoustic port for the microphone closed the second time. This is to verify that the microphone is properly acoustically isolated from other parts of the device.
Measurement data & process improvement
Obviously, the core purpose of production line testing is to limit, or completely avoid, the possibility of a non-compliant product being shipped. However, the manufacturing test process has the potential to provide added value via the tracking and analysis of measurement results (rather than just recording pass or fail). The collection, storing, and analysis of test data offers a number of additional benefits. Field failures, and their failure mode, can be compared to measurements made during production to better understand how tests can be modified to check for a previously unknown failure mode. If there is a measurement that devices always pass, the measurement can possibly be removed, thus saving test time on the line. It is also possible a device passed final test, but only marginally, and the pass/fail limits should be adjusted. Even as devices pass, it is useful to examine trend data to detect drift in materials and tolerances to preempt future quality issues.
The technical details of the tests conducted on a production line are directly related to a specific product’s value, along with the company’s overall quality standard and brand. Is the product positioned as a high-end or luxury item? Or a value-oriented product? How is it priced? What is the cost of a field failure? Answers to such questions support the creation of both the strategy and budget for manufacturing test. Will every unit assembled receive a full functionality test? Or will only basic functionality be checked? Maybe only incoming test of common failure components? Will every unit be tested? Or will a sampling approach be used for manufacturing test?
With objectives, strategy, and budget established, the technical requirements for manufacturing test can be defined. With a minimal test budget, just stimulating a device with a single tone and recording output level and THD+N may be the answer. For higher value items, and an associated larger test budget, the approach may shift to using multitone, continuous sweep, or another fast measurement technique to more thoroughly characterize a device in various use scenarios. Ultimately, the technical requirements for manufacturing test are the product of overall quality standards, and the potential cost of field failures that defined those standards.