Exploring Next-Gen AI ‘from Atoms to Apps’

Article By : Rajiv Joshi

AI Compute Symposium explored how research efforts in various disciplines might converge to take the next level of artificial intelligence “from Atoms to Applications.”

Like most conferences last year, the 3rd IBM IEEE CAS/EDS AI Compute Symposium was also the symposium’s inaugural virtual event. The event explored the next-gen AI ‘from atoms to apps.’ Held over two days in October and supported by the IBM Academy of Technology, the symposium drew more than 650 participants from roughly 45 countries. They joined with the keynoters, presenters, and panelists to explore how research efforts in materials and hardware, compute paradigms, cognitive science, and security might converge to take the next level of artificial intelligence “From Atoms to Applications,” as the symposium’s promoters put it.
Atoms to Apps
AI Compute Symposium (Source: IBM)
Keynote speaker Mukesh V. Khare, vice president of IBM Research, opened the conference with an examination of “What’s Next in AI: Our Vision for the Future of AI Hardware.” We have reached a turning point in computation, Khare said. As the pandemic and the warming climate continue to present global challenges, the next generation of computers will define and respond to those and future crises. The IBM Research AI Hardware Center is developing a scalable computing platform that will use the power of AI and the flexibility of the hybrid cloud to create a virtually limitless pool of computing power and capabilities to find and implement solutions. Understand the human microbiome Keynoter Rob Knight of the University of California San Diego spoke about “Using AI to Understand the Human Microbiome and its Role in COVID-19.” The human microbiome — the collective microbiota that reside on or in human tissues and biofluids — harbors an immensely complicated collection of genes and cells that outnumber “our own” and are redefining the concept of what it means to be human. Grappling with data of this complexity requires AI techniques, said Knight, who is the director of the Center for Microbiome Innovation at UCSD. Knight highlighted the work of a partnership between the university and IBM called Using Artificial Intelligence for Healthy Living, presenting applications of cutting-edge techniques designed to understand the microbiome and discover how it changes with age and disease. The aim is to formulate diet, lifestyle, and medication strategies that maximize health throughout the human lifespan. In response to the Covid-19 pandemic, studies are under way to determine the microbiome’s role in disease progression among the population groups most impacted by SARS-CoV-2, including the elderly and those with specific microbiome-related co-morbidites. The techniques will be generally applicable to other illnesses and will help bridge the previously disparate fields of infectious and chronic disease research and treatment. Atomic Memory Some researchers are pursuing atomic-level materials to enable devices used in AI applications. In a talk titled “Atomic Memory: From Single Defects to Analog Switches and Computing,” professor Deji Akinwande of the University of Texas, Austin, focused on the discovery of the memory effect in atomically thin, 2D nanomaterials. The discovery could provide greater scientific understanding and advancements for engineering applications, Akinwande said. Non-volatile memory devices based on 2D materials are an application of defects and constitute a rapidly advancing field with rich physics that can be attributed to vacancies combined with metal diffusion. Akinwande’s talk highlighted his group’s pioneering work on monolayer memory (atomristors), which could enable applications including zero-power devices, non-volatile RF switches, and memristors for neuromorphic computing. Reverse engineering visual intelligence Researchers in brain and cognitive science are working to reverse engineer the human mind and its intelligent behavior. The field is in its infancy, as are forward-engineering approaches that aim to emulate human intelligence (HI) in artificial systems (AI). Yet the intelligence and cognitive flexibility apparent in human behavior are an existence proof that machines can be constructed to emulate and work alongside the human mind. James J. DiCarlo, co-director of the Quest for Intelligence project at the Massachusetts Institute of Technology and an investigator at MIT’s Institute for Brain Research, presented a paper on one aspect of this work: “Reverse Engineering Visual Intelligence.” DiCarlo predicted that the challenges of reverse engineering human intelligence would be solved by tightly combining the efforts of brain and cognitive scientists (hypothesis generation and data acquisition) with forward engineering to emulate intelligent behavior (hypothesis instantiation and data prediction). As this approach discovers the correct neural network models, those models not only will encapsulate our understanding of complex brain systems but will form the basis of next-generation computing and novel brain interfaces for therapeutic and augmentation goals, such as addressing brain disorders. The talk focused on visual object categorization and detection. DiCarlo noted that work in brain, cognitive, and computer science converged to create deep neural networks (DNNs) that could support such tasks. Those networks not only reach human performance for many images, but their internal workings are modeled after— and largely explain and predict — the internal workings of the primate visual system. The primate visual system (HI) still outperforms current-generation artificial deep neural networks (AI), however, suggesting that there are more clues for brain and cognitive science to uncover. The quest to understand human intelligence may be the most important scientific task of our age, and DiCarlo’s talk challenged the audience to engage with the researchers already exploring the frontier. ‘Pretraining’ & ‘Transfer learning’ Another presentation focused on “Exploring the Limits of Weakly Supervised Pretraining.” Laurens van der Maaten, research director at Facebook AI Research in New York, noted that state-of-the-art visual perception models for a wide range of tasks rely on supervised pretraining. ImageNet classification, the de facto pretraining task for those models, is now nearly 10 years old and is “small” by modern standards. Even so, relatively little is known about the behavior of pretraining with datasets that are multiple orders of magnitude larger. The reasons is obvious: Such datasets are difficult to collect and annotate. In his talk, van der Maaten discussed a study of transfer learning that looked at large convolutional networks trained to predict hashtags on billions of social media images. Extensive experimentation provided novel empirical data on the relationship between large-scale pretraining and transfer learning performance The data shows that training for large-scale hashtag prediction yielded excellent results: Improvements on several image classification and object detection tasks were shown, and the study recorded the highest ImageNet-1k single-crop, top-1 accuracy to date: 85.4% (97.6% top-5).  The future of AI on the edge Pamela Norton, CEO of Borsetta, opened the second day of the symposium with a keynote on “Securing the Future of AI on the Edge with Intelligent Trusted Chips.” Building the new edge economy requires us to ensure that the convergence of AI, new compute processing, and intelligent chips migrating to the edge will protect stakeholders’ privacy and security. Norton presented a framework for accomplishing this in a way that would create opportunities for all players in this emerging economy. Studies of neural plasticity — the ability of the biological nervous system to change — can offers insights for AI system design. The convergence of these fields is in its infancy however, so there is a critical knowledge gap in designing neuromorphic systems that can support on-device learning with heterogeneous plasticity. Neuro-inspired AI Dhireesha Kudithipudi, director of the Matrix AI Consortium and a professor at the University of Texas, San Antonio, addressed this topic in “Neuro-Inspired AI: Compact and resilient models for the Edge.” She presented a recent in-silico learning system from UTSA’s Neuromorphic Artificial Intelligence Lab (NAUI). Based on a CMOS/memristor architecture, the system emulates a biomimetic sequence memory algorithm inspired by the neocortex with structural and intrinsic plasticity mechanisms. The synthetic synapse representation supports dynamic synaptic pathways for compact memory. The structural plasticity in the synaptic pathways was emulated in the memristor’s physical behavior, and the synaptic modulation is achieved through a custom training scheme. Quantum circuits In the past few years, quantum computing has moved beyond the laboratory setting and has been accelerated through cloud access. This new computing paradigm uses the same physical rules that atoms follow to manipulate information. At this very fundamental level, quantum computers execute quantum circuits much as a computer executes logic circuits, but by using the physical phenomena of superposition, entanglement, and interference to implement mathematical calculations that are out of the reach of even the most advanced supercomputers. Jay Gambetta, an IBM fellow and vice president of IBM Quantum, presented a quantum roadmap in a talk titled “Quantum Circuits and the Future of Quantum Technology in the Cloud.” He gave an overview of the IBM Quantum effort to increase the device performance of superconducting qubit systems to produce quantum circuits with higher fidelity, explaining how IBM is linking the computational difficulty of these circuits toward quantum applications. Cutting-edge research, system, and software development are extending the frontier, bolstered by an engaged and growing quantum computing community. AI with photonic computing The final talk on day two looked at “Unlocking Transformative AI with Photonic Computing.” Recent large-scale AI models, such as OpenAI’s GPT-3 for natural language processing (NLP), open a range of opportunities that might have even more of an impact than those enabled by deep learning in the past decade. But training these models requires massive amounts of computing power. Laurent Daudet, CTO and co-founder of LightOn, presented the company’s take on how future AI hardware should be designed for addressing some of the hardest computing challenges, such as NLP, recommender systems, and big science. Daudet, who is also a professor at the University of Paris Diderot, described how LightOn’s Optical Processing Units (OPUs) can be seamlessly integrated into hybrid photonics / silicon pipelines implementing state-of-the-art machine learning algorithms. The symposium also accepted poster presentations and reviewed 44 entries. Four were awarded honors:
  • Tianyu Jia, Yuhao Ju, Russ Joseph, and Jie Gu, “NCPU: A Binary Neural Network that Emulates RISC-V CPU at the Conjunction of Neuromorphic and Von-Neumann Architectures,” Northeastern University, Boston
  • Seah Kim and Hasan Genc, “Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via Full-Stack Integration,” University of California, Berkeley
  • Sanjeev T. Chandrasekaran, Sumukh Bhanushali, Imon Banerjee, and Arindam Sanyal, “Towards Intelligent Wearable Health Monitors Using Reservoir Computing CMOS IC,” State University of New York at Buffalo
  • Minh Truong, “Data-Oriented Processing — Using-Memory with Emerging Memory Technology,” Carnegie Mellon University, Pittsburgh
The symposium closed with a panel discussion that asked, “What will be the currency of future AI computation?” Moderated by program co-chair Arvind Kumar, a manager and researcher at IBM’s T.J. Watson Research Center, the panel invited Norton, Kudithipudi, Sidney Tsai, Daudet, DiCarlo, and Kristan Temme to weigh in from the various perspectives of their expertise in security, analog AI, optical computing, brain science, and quantum computing. The initial points of view suggested benefits for each approach, but there was also consensus that no one approach would dominate, and heterogenous computing with secure translation between AI’s varied “currencies” would be needed. Technical questions were posed on topics such as the combination of approaches, emulation of Hebbian learning, plasticity and its importance, the longevity of current DNNs, and the role of quantum computing in AI. The panel ended with some questions from the audience seeking advice from the panelists for early research in the field. Find more on the symposium here. Once registered (free), you get to watch keynotes and lectures delivered at the conference. — This report was compiled by Rajiv Joshi, general chair; Arvind Kumar, program co-chair; Matt Ziegler, program co-chair; Xin Zhang, poster co-chair; and Krishnan Kailas, poster co-chair. Special thanks to Cindy Goldberg of IBM and Brittian Parkinson of CAS for the publicity help.

Leave a comment