Last year when Sony came to the CES to talk about its EV project called "Vision-S," it had a look of a marketing stunt. This year, Sony EV gambit seems more like a real car project...
Last year when Sony came to CES to talk about its EV project called “Vision-S,” it had a look of a marketing stunt. This year, Sony’s EV gambit seems more like a real car project.
The Japanese consumer electronics giant has a host of building blocks that make entertainment sing. Now the question is: Can Sony adapt this technology know-how to the automotive market and enter the realm of car safety?
Sony’s arsenal of technologies and media assets ranges from high-resolution imagers, time-of-flight (ToF) cameras, 360-degree spatial audio and display technologies to 5G connectivity, UI/UX for smartphones, AI and sensing technologies, PlayStation 5 and Columbia Pictures.
According to Sony, the car “represents a fusion of Sony’ technology and creativity” that will play a role in “the future of mobility, encompassing the evolution of safety, comfort, entertainment and adaptability.” As grand a plan as this sounds, it’s unclear whether this vision is what carmakers expect of Sony, or if they want to buy it from Sony.
Take, CMOS image sensors for example.
CMOS image sensors come first to mind, because imaging is one of the most important perception technologies in advanced driver-assist systems (ADAS) and driver monitoring systems (DMS). Second, ADAS image sensors in 2019-20 saw spectacular growth.
“It doubled in two years, reaching $500 million in 2020, and it is still expected to grow 19 percent CAGR in the five years to come,” Pierre Cambou, principal analyst at Yole Développement, told EE Times. Cambou was citing figures from “Status of CMOS Image Sensor Industry 2020 report” compiled by Yole Développement.
Given Sony’s CMOS image sensor success with smartphones, it might seem that Sony already dominates the automotive vision market.
On the ADAS image sensor market, Sony’s 11-percent market share is a distant third, trailing On Semiconductor, which commands 49 percent, and Omnivision at 19 percent.
The same ranking applies to image sensors used for DMS (to watch a vehicle’s interior). The DMS image-sensor market leader is, once again, On Semiconductor.
So, what’s going on here? Why is Sony trailing in automotive? Does Sony, perhaps, have a secret ace-in-the-hole plan — like following Apple into EVs?
Colin Barnden, lead analyst at Semicast Research, told EE Times: “Making cars is so easy that Tesla does it in a tent. The tricky part is making money by making cars, and then making reliable cars.”
Sony has not announced any plans to make cars.
Nevertheless, Sony is building an EV prototype called Vision-S (announced a year ago). The Vision-S was described then as an experiment to “explore the potential for next-generation mobility.”
A year later, the Vision-S project rolls on, now as a functioning real car project. Sony began testing it on roads in Austria last December, reported Kenichiro Yoshida, Sony’s CEO, last week at CES 2021.
Sony’s automotive ambition
As some industry observers suggested, Sony might well become a carmaker someday.
One telltale sign is its partnership with Magna, whose CEO noted its close collaboration with Sony on Vision-S, which has been going on behind the scenes since 2018. The deal doesn’t appear to end with development of the prototype. Barnden speculates that Magna will be a sub-contract manufacturer for Sony. Magna is also Barnden’s “top pick as sub-contract manufacturer for the Apple iCar.” The race for vehicle subcontract manufacturing is well under way — between Magna and Foxconn, in Barnden’s view.
But we digress. Setting speculation aside, let’s break down Sony’s current business in the automotive market.
Today, despite its potential, Sony as a chip supplier has yet to position itself as a powerhouse technology supplier in the auto market.
Nonetheless, Sony’s Vision-S promotional video illustrates potential opportunities. Vision-S, a sleek-looking four-seater, bristles with products and technologies that appear to be Sony’s.
The prototype comes with dashboard, digital instrument display and door-to-door touch-screen infotainment screen. Car Magazine said the Vision-S looks like “a blend of Tesla ‘s Model S, the Porsche Taycan and the Audio A7 (particularly at the rear with 21-inch wheels.”
Sony’s video suggests Vision-S has some autonomous driving capabilities. It has 40 sensors (Camera/ToF: 18; radar/ultrasonic: 18; lidar: 4), according to Sony’s auto website. with a personalized cabin complete with ToF and DMS cameras.
Lip-reading is coming
Sony has a vision and it claims to have the technology. Success might come down to execution.
Consider in-cabin monitoring.
Inside the cabin, Sony noted, “ToF camera sensors monitor the condition of the vehicle occupants. Facial expressions and gestures are used to determine the driver’s level of concentration and fatigue, and alerts are issued when necessary.”
As seen in many announcements at CES 2021, DMS is already becoming a given. Next in line is speech recognition, enabling the curious driver to discuss in-vehicle operations or ask the car a question to a car—such as “What’s that building up ahead?”
Now, Sony is taking a stab at “lip reading” features, intended to “reliably capture the driver’s speech intentions even in noisy situations,” according to the company.
In an interview with EE Times, Sony claimed that although some researchers have demonstrated the basic concept [of lip reading], no one has mastered it. Mark Hanson, vice president of technology and business innovation at Sony Semiconductor Solutions of America, articulated why lip-reading is important. Although voice might become a popular feature, the reality is that “with wind noise, there is virtually no way to enable speech recognition.” Hanson noted, “I only own two vehicles — both convertibles — and I’ve never been able to use speech recognition in my particular environment.”
Speech recognition, he added, is also tough when kids are screaming in the back of the car.
We asked if lip-reading might be a way to promote Sony’s in-cabin cameras. Not exactly. Sony’s plan is to license its lip-reading software to car OEMs as part of Sony’s strategy to serve as broadly as possible as a “technology service and solution” provider.
We asked if Sony is working with a particular automotive platform to process the lip-reading algorithm. Has Sony considered, for example, working with Qualcomm’s Snapdragon Automotive Platform which is already incorporated with Alexa capabilities?
Not exactly. “Lots of different implementations can be done,” said Hanson. Acknowledging that Sony’s effort is still in early days, it has no partnerships or collaborations it’s ready to announce.
But Hanson stressed a wide range of ways to enable this lip-reading software. These include running it on the in-vehicle processor system, embedding it inside cameras, or adding a daughter card. All the options mentioned above are expected to locally process speech.
Hanson also brought up the intelligent vision sensor Sony introduced last year. It comes with AI processing functionality. “You can run the algorithm on the sensor itself,” he explained. “So we could do some level of task there… Such an intelligent sensor could be programmed to identify the facial features, identify the lips, and then only send lip data into the model, to enable higher performance.”
A caveat is that Sony’s intelligent sensor Hanson mentioned isn’t yet inside any vehicles. The sensor, developed for consumer applications, is not automotive qualified.
Late to the automotive market
Sony’s big mistake is, most likely, that it came very late to the driver’s seat, perhaps because it was too enamored with its success in smartphone image sensors. It wasn’t until 2014 when Sony officially announced plans to commercialize image sensors for automotive use.
Nonetheless, over the last several years, Sony released a slew of highly advanced imagers that boast industry-first features. These include IMX224MQV in 2014, which the company touted as “the world’s highest sensitivity CMOS image sensor for automotive cameras.” Sony acquired Softkinetics Systems (Belgium) in 2015, which developed ToF technology for distance detection. Sony then renamed Softkinetics as Sony Depthsensing Solutions Holding.
Sony launched IMX390CQV in 2017, to deliver simultaneous LED flicker mitigation and high-quality HDR shooting. Later that year, Sony’s introduced IMX324, equipped with a 7.42 effective megapixel stacked CMOS image sensor. Sony said the new image sensor is compatible with Mobileye’s EyeA4 and EyeQ5.
Phil Amsrud, senior principal analyst at IHS Markit, agreed that Sony is “a latecomer” with limited automotive experience.
He observed, “On Semi became the dominant CMOS image sensor (CIS) supplier because of their automotive experience having been a spinoff from Motorola Semiconductor, so when they acquired Aptina, On Semi had both the automotive pedigree and the CIS products.”
What tripped up Sony
Sony’s own history in consumer electronics might have hindered its understanding of automakers’ needs.
Amsrud observed, “On Semi and OmniVision supported lower resolution CIS products so they were able to support a range of applications including very cost sensitive ones using <1MP.” In contrast, coming from the consumer segment with higher resolution devices, I think Sony focused more on the high-resolution automotive applications which were pretty limited at that time.”
Early entry into the lower resolution market enabled On Semi and Omnivision to secure a lion’s share of the volume market. That market has eventually grown into ADAS.
Yole’s Cambou concurred. “The dynamics in automotive is very different from smartphones.” He explained, “As ADAS was rolling out in mid-tier cars, the pressure on performance was not so high. Reliability and price are currently more important, which favor the incumbents On Semiconductor and Omnivision.”
Cambou added, “This pressure on price is also favoring the entry of new players such as Smartens Technologies on the Chinese market. This will also slow down future penetration of Sony.”
He went on: “At Yole, we believe Sony will get a significant share only if pressure to get high performance image sensor increases on the market. The race to autonomy could play a role. But today, apparently there is more money to be invested in high-performance computing (i.e. Tesla FSD) than in high-performing image sensors. Instead of quality, the market is driven by quantity.”
Cambou said, “This is not an environment favorable to Sony.”
What future holds for Sony
The potential for Sony as an EV company, if it goes that way, is exciting. But this remains a gamble for the Japanese consumer electronics giant. Looking through Vision-S, Sony’s eyes might be bigger than its stomach.
More industry observers are beginning to believe that Vision-S isn’t doomed to languish as just a Sony vanity project.
Numerous suppliers, in fact, unveiled at CES 2021 their relationships with Sony on Vision-S. AImotive, an ADAS and autonomy software company, revealed a collaboration with Sony to “advance its automated driving software stack with Sony’s Vision-S Prototype.”
An article posted by TechCrunch cited “an array of partners on Vision-S.” Among the collaborators are “Bosch and Continental, Hungarian automated driving startup AIMotive, software company Elektrobit Automotive, French automotive supplier Valeo, telecommunications giant Vodafone and German car parts maker ZF Group.”
The story concluded: “The collection of partners, which also includes mapping company HERE, Nvidia and Blackberry/QNX and Qualcomm, leaves little doubt that someday there will be a Sony car that consumers can buy.”
Sony’s ambition is clear. But just as clear is an important unanswered question. How will Sony’s semiconductor division make money in the automotive market if its parent is competing with car OEMs? Or, does Sony have a better plan — to use Vision-S as an eye-catching project that starts the conversation with OEMs and tier ones? If Sony doesn’t get caught up in its own hype, this is a twist that could end up helping its semiconductor division.