Machine Learning applications, especially Deep Neural Networks (DNNs) have seen ubiquitous use in computer vision, speech recognition, and robotics. However, the growing complexity of DNN models have necessitated efficient hardware implementations. The key compute primitives of DNNs are matrix vector multiplications which leads to significant data movement between memory and processing units in today’s von Neumann systems. A promising alternative would be co-locating memory and processing elements, which can be further extended to performing computations inside the memory itself. We believe in-memory computing is a propitious candidate for future DNN accelerators since it mitigates the memory wall bottleneck. In this talk, I will discuss various in-memory computing primitives in both CMOS and emerging non-volatile memory (NVM) technologies. Subsequently, I will describe how such primitives can be incorporated in stand-alone machine learning accelerator architectures. Finally, I will focus on the challenges associated with designing such in-memory computing accelerators and explore future opportunities.

Computing is bottlenecked by data. Large amounts of application data overwhelm storage capability, communication capability, and computation capability of the modern machines we design today. As a result, many key applications' performance, efficiency and scalability are bottlenecked by data movement. We describe three major shortcomings of modern architectures in terms of 1) dealing with data, 2) taking advantage of the vast amounts of data, and 3) exploiting different semantic properties of application data. We argue that an intelligent architecture should be designed to handle data well. We show that handling data well requires designing architectures based on three key principles: 1) data-centric, 2) data-driven, 3) data-aware. We give several examples for how to exploit each of these principles to design a much more efficient and high performance computing system. We will especially discuss recent research that aims to fundamentally reduce memory latency and energy, and practically enable computation close to data, with at least two promising novel directions: 1) performing massively-parallel bulk operations in memory by exploiting the analog operational properties of memory, with low-cost changes, 2) exploiting the logic layer in 3D-stacked memory technology in various ways to accelerate important data-intensive applications. We discuss how to enable adoption of such fundamentally more intelligent architectures, which we believe are key to efficiency, performance, and sustainability. We conclude with some guiding principles for future computing architecture and system designs. Throughout the talk, we will point out how open source hardware can enable innovation in and adoption of the paradigms we introduce.

Processing-in-memory (PIM) has been a research topic for many years. However, it’s a concept that, until the end of Dennard’s Scaling, never had an opportunity to gain commercial traction. As new computing paradigms have emerged, as well as the reality that the memory bandwidth improvements have not kept up with improvements in computational performance (one of the artifacts from the growth of multi-core), PIM is gaining renewed interest. However, there are still challenges to making PIM a success. This talk will discuss the opportunities as well as some of the aforementioned challenges for PIM to become a new class of computation.

Under the pressure of big data applications and the long-standing memory-wall problem, processing-in-memory (PIM) is proposed to reduce data access delay in architecture design. In this talk, we conduct a thorough study of computer architecture design from a performance perspective. We start with Amdahl’s law and the data performance constraint of multicore processors, then introduce the concept of deep memory-storage hierarchy and focus on memory system optimizations; and we finalize the study with a compute-data integrated optimization methodology, named DataflowV, where PIM plays a pivoting role. We model memory systems performance using four different types of memory cycles which paves the way of establishing DataflowV. We will present the concept, modeling, and design of DataflowV and some of its implementation results under PIM environments.

With brain-inspired architecture and event driven operation, neuromorphic processors, such as IBM TrueNorth and Intel Loihi, have demonstrated outstanding energy efficiency in many applications. However, because of their unique spiking-based information encoding and communication, special computing models must be designed to fully explore the potential of these computing platforms. How to unitize the synapse plasticity provided by the hardware to realize online learning is another challenging problem. In this talk I will introduce some of our works on Spiking Neural Network (SNN) learning and inference and their implementation on the neuromorphic processors. Three different works will be discussed. First, I will present AnRAD, a bio-inspired detection framework that mimics the human decision-making and probabilistic inferences for anomaly detection, then I will introduce how to utilize neuron and synapse dynamics to detect spatiotemporal pattern and how this technique can reduce hardware cost and improve energy efficiency for sensor-based applications. Finally, I will present EMSTDP, which achieves gradient based backpropagation in spiking domain, and its implementation on the Intel Loihi processor.