Deep Learning SoC Brings Always-on Neural Processing to Edge Devices

Article By : Sally Ward-Foxton

A brand-new core powers Syntiant’s second-generation SoC, designed for voice control of consumer devices...

AI accelerator startup Syntiant’s second-generation product for edge AI at ultra-low power is based on a new in-house designed processor core. Like Syntiant’s first generation product, the new SoC is designed for voice control applications with tight power budgets. It can run multiple neural networks simultaneously with a power budget of 1mW for always-on battery-powered devices such as smartphones. The new chip, the NDP120, uses the second-generation Syntiant Core 2 for AI acceleration, alongside a Tensilica HiFi3 DSP used for feature extraction, and an Arm Cortex-M0 core which manages the system. Syntiant, founded in 2017, was one of the first startups to bring an ultra-low power neural network processor to market based on its own neural network processing IP. The company announced last summer that it had shipped a million first-gen production devices to its customers, who are largely in consumer electronics. The company has raised a total of $65 million across three funding rounds. As of August, the company employed 66 staff at its base in Irvine, Calif. New core While the company’s first-generation core was designed for the absolute lowest possible power consumption, the Syntiant Core 2 offers more processing power and capacity for bigger neural networks, expanding to multiple concurrent networks running simultaneously, and caters for most common neural network architectures. It builds on the same near-memory compute architecture in which the neural engine is tightly coupled with the onboard SRAM.
Kurt Busch Syntiant
Kurt Busch (Image: Syntiant)
“The Syntiant Core 2 takes about three years of learning to build a very flexible core that can scale up to much larger applications,” Kurt Busch, CEO of Syntiant, told EE Times. “We’ve gone from [processing] a four-layer fully connected network in the Syntiant Core 1 to a very flexible neural network [processor] that can support convolutions, RNNs, LSTMs or almost all the popular neural network architectures today.” The new core offers 25x the tensor throughput of the first-generation core, and is much bigger, holding neural networks with up to 7 million parameters (up from half a million in the previous generation). This is sufficient to run multiple different neural networks concurrently, Busch said. For a typical audio AI application, this could include echo cancellation, beamforming, noise suppression, speech enhancement or speaker identification alongside voice command recognition. “The core is a native neural network processor, so there’s no translation needed between what you develop in Keras and what runs on the device,” Busch said. All major frameworks can port to the Syntiant Core 2 runtime. One of Syntiant’s design goals was to speed up time-to-product. The company’s design tools allow the user to explore power, latency, memory and parameter requirements on the Syntiant Core 2 interactively. The Syntiant Core 2 supports quantization to 1, 2, 4 and 8 bits with a new high-precision 16-bit inference mode. NDP120 Chip The NDP120 is Syntiant’s first SoC to use the Syntiant Core 2. While the first-generation chip had a hard-coded feature extractor, Syntiant’s second-generation product uses a Tensilica HiFi3 DSP for feature extraction. There is also an Arm Cortex-M0 core which manages the system.
Syntiant SoC NDP120
Syntiant has developed a brand-new core for its latest SoC, the NDP120, which can hold neural networks with up to 7 million parameters (Image: Syntiant)
The device supports up to 7 audio streams and is capable of far-field audio processing. “The NDP120 can bring the level of performance that you would typically find in a plugged-in smart speaker to a battery powered device, that’s really the goal for this product,” Busch said. “The NDP120 is primarily for audio [processing] as well as sensor fusion, with one or more of those sensors being a microphone… for far-field and noise clean up. It’s ideal for [AI-powered] noise reduction.” Syntiant’s second-generation product is aimed at the same market as for the previous generation – battery-powered consumer devices from laptops and smartphones to earbuds. While previous generation products enabled keyword spotting and wake word detection, the new chip is powerful enough to enable a bigger vocabulary of commands as well as real-time noise reduction in something as small as an earbud. While the Syntiant Core 2 is application-agnostic, the I/O on the NDP120 is geared up for audio. Future SoC products using the same core will focus on video processing, according to Busch, and the company is also working on a third-generation core which will add another degree of scalability. The Syntiant NDP120 is sampling now and is expected to ship in production volumes in summer 2021.

Leave a comment