The startup is bringing up its 2nd-gen silicon remotely. It's banking on a surge in voice-control demand due to Covid-19...
Despite the global pandemic and resulting shutdown of many businesses, it’s pretty much business as usual for many fabless semiconductor companies, with a few allowances made. Speaking to the CEO of AI processor company Syntiant, Kurt Busch, revealed that the company is working at full speed on customer engagements and testing the second generation of its silicon.
“It’s a very interesting time. When we first started to work from home, I was concerned… I assumed that current [customer] engagements would continue, maybe at a slower pace, but they would definitely continue,” Busch told EE Times. “I didn’t know if new engagements would work without face to face, but it turns out that we are able to get new engagements, with both customers and partners, remotely.”
Syntiant, an ultra-low power AI processor startup, received its second-generation silicon back from the fab recently, but was able to bring up samples and begin testing remotely. According to Busch, the company had its new silicon up and running in just one hour and thirteen minutes.
Syntiant’s setup uses cheap consumer boards, namely Raspberry Pis, as a network interface for the silicon under test, connecting to the chip via the SPI and other interfaces on the Raspberry Pi. Sensors in the setup are triggered remotely via the generation of audio, video, vibrations or other conditions to test the chip. A couple of dozen chips are being tested in this way, with engineers able to write software and debug it remotely.
“We can do 90% of the bring-up remotely. There are some things you have to do physical measurements for, and for that we send in one person, maintaining social distancing,” Busch said. “We have been able to bring up a pretty complicated piece of silicon all remotely; no-one had done that before in our company, this was totally new for us.”
Neural decision processor
Syntiant was one of the first to build a neural network processor for edge applications from the ground up (as opposed to using multi-core DSP or ARM cores). Its first-generation architecture is based on computation in or near memory, massively parallel operations and modest precision (Syntiant’s first generation chip can handle 4-bit or 8-bit calculations that are common in machine learning inference).
“What we’ve built is a data flow architecture where the multiply and accumulate is tightly coupled to the memory. So there’s very little memory movement going on inside of our device. All the memory is inside,” Busch explained, in an earlier conversation with EE Times. “We built this largely parallel architecture that’s also modest precision, so it’s sub-8-bits and the memory is tightly coupled with the multiply accumulate, to greatly reduce the memory consumption.”
On the software side, the compiler step is nixed. Most AI processor companies with novel architectures are building compilers that can transfer code from TensorFlow or Keras to something that runs on their processor, which can be a big challenge.
“The compiler step, trying to get your compiled code to actually match what you see in TensorFlow – there’s a compiler beauty contest going on to see whose compiler does the best job if it,” Busch said. “We totally bypass that – what you see in TensorFlow is what you get in silicon, we just take the weights and load them directly into the device.”
Syntiant started in the then-nascent AI space in 2017, with its co-founders filing 18 patents on its silicon architecture. An A-round led by Intel in October 2017, followed by a B-round led by Microsoft, raised $30 million. This was enough to get the company to tapeout within 5 months, and to get samples back within eight months.
The result is an extremely low-power chip. Compared to an Arm M4-based design, Busch said, Syntiant’s chip is 200x more power-efficient for voice workloads, with 20x the throughput. The chip consumes active power below 140µW while recognising words.
“Because we don’t have all the legacy processing, die size is smaller so we can compete on power, performance and cost, which almost never happens, getting to compete on all three axes,” Busch said. “But there is no such thing as a free lunch. [Our chip] only does deep learning. It doesn’t do anything else.”
Syntiant got its first production orders in September 2019 and is designed into mobile phones, smart speakers, earbuds and watches. More than 90% of these applications are using the NDP10x for voice recognition such as voice commands and wake word detection, while some are doing more general sound detection (such as gunshot or glass break). A few more are using the chip outside of the audio domain, Busch said, in sensor applications that use accelerometers, gas detectors, vibration sensors or infrared sensors for person detection.
“What we see in the state of the market today is very few companies have released production machine learning models,” said Busch. “We have a very small number of customers that have done it themselves and gotten production level code. You can get demo level code within an hour. But production level machine learning, especially wake words and in very noisy environments is a skillset not widely known in the marketplace.”
Syntiant therefore supplies production-ready, trained neural network models. The company also offers a training and development kit for the NDP10x, but with very few customers having built a production machine learning pipeline, very few are able to take advantage of it. Instead, Busch said, Syntiant has invested in data collection, cleaning and preparation to train its in-house models.
Syntiant’s first generation product is targeted squarely at battery-powered always-on voice control applications. Voice control is expected to see increasing demand as consumers are increasingly aware of the surfaces they touch during the current global pandemic, particularly in public spaces such as elevators.
A national survey (carried out on behalf of Syntiant) revealed that more than two thirds of Americans currently use voice to control their devices. This is more pronounced in the younger generations (for Generation Z and Millennials, the figure is 81%). For even younger consumers, Busch said, they will use voice control of household devices before they can read, and for them, voice will become the default interface for televisions, laptops and smartphones.
Busch also anticipates that voice control, done on the edge device without requiring an internet connection, to help open up the world of technology to populations that may not be able to read.
The second generation of Syntiant’s neural decision architecture, just back from the fab but due to be announced this fall, makes use of the company’s two years of learning.
“The chip itself, how it does processing is completely new,” Busch said. “It’s based on the same philosophy of computation in or near memory, massively parallel and modest precision, so it’s designed still from the ground up to do neural processing… it still runs the code from the previous chip, but it greatly expands what we can do with it.”
Syntiant expects to announce its second-generation silicon in Q4 2020.