Derek Odom, CEO Non-Von
As artificial intelligence (AI) technologies continue to advance and proliferate, the energy demands associated with current computing hardware (CPUs and GPUs) is also rapidly increasing. This growing energy consumption driven by AI leads to significant environmental challenges.  New AI focused chip designs, like Non-Von’s, are a large part of the solution and are the focus of this post.

Growing Energy Demands
The rise of AI applications, especially generative AI, has led to a dramatic increase in energy consumption, particularly within data centers that support these technologies. According to the International Energy Agency, electricity demand from data centers is expected to more than double by 2026, largely due to AI applications (TIME). This surge in energy needs poses significant challenges, especially when much of this energy is derived from fossil fuels, undermining global climate change efforts (McKinsey & Company).

The overwhelming energy requirements of AI can strain local infrastructures, resulting in brownouts and additional resource shortages, such as water (used to cool the data centers), exacerbating the urgency of addressing AI's energy consumption sustainably (TIME). The increasing energy demands point toward a steadily increasing carbon footprint for AI computing industry and the processors currently powering its growth.  These environmental impacts, and their associated economic and national security implications, highlight the need for mitigating innovative solutions.(Yale E360).

Environmental Challenges
The environmental consequences of AI's energy demands are substantial. Critics emphasize that the growing carbon footprint associated with current AI processors could further exacerbate climate change (Goldman Sachs). While AI holds the potential to optimize energy usage and reduce emissions, the current trajectory of energy consumption may outpace available resources in many regions (Yale E360). The challenge lies in ensuring that the expansion of AI technologies does not compromise environmental sustainability.

The Need for Sustainable Solutions
To tackle these pressing challenges, collaboration between the energy and data center sectors is vital. Developing sustainable solutions, such as transitioning to renewable energy sources and enhancing energy efficiency, is critical (McKinsey & Company). 

Interestingly, AI can also contribute to its own solution. By leveraging AI to optimize energy use across various industries, companies and researchers can enhance overall energy efficiency, emphasizing the need for energy-efficient AI chips (Yale E360).

Advantages of Non-Von’s Hardware/Software Solution
Non-Von’s chips and supporting software ecosystem are engineered to overcome the limitations of traditional computing hardware. These chips boast enhanced computing power and improved energy efficiency, directly addressing the energy consumption challenges posed by standard chips. Non-Von uses advanced parallel processing capabilities, allowing them to perform complex AI functions while minimizing energy use.  While easily implementing today’s popular models through its advanced compiler, Non-Von’s hardware also opens up computing to a whole new class of sparse, unstructured models.  These types of models can enable additional efficiency gains currently unrealized due to constraints in today’s hardware. 

By prioritizing power efficiency, Non-Von chips are crucial in meeting the increasing energy demands associated with AI technologies.

Future Implications
As the demand for AI technologies continues to grow, the environmental impact of standard chips will become increasingly unsustainable. Transitioning to energy-efficient hardware/software solutions like Non-Von’s is essential for mitigating these impacts and achieving sustainability goals.  Continued innovation and investment in AI chip technology is a necessity in aligning technological advancements with environmental sustainability. By emphasizing energy-efficient designs and harnessing AI to optimize energy use, the industry can ensure that the benefits of AI do not come at the expense of our planet's health.

Hope for a sustainable future with AI
The environmental impact of standard chips presents significant challenges as the world moves toward an AI-driven future. However, with innovative solutions like Non-Von’s, there is hope for a more sustainable technological landscape. By addressing energy consumption and prioritizing efficiency, we can harness the power of AI while protecting our environment for future generations (not to mention the economic benefits of reduced electrical requirements). Sign up for our newsletter at the bottom of our homepage (non-von.com) and become part of shaping the future of AI with us.

Sources:McKinsey & Company. "How data centers and the energy sector can sate AI’s hunger for power."
TIME. "How AI Is Fueling a Boom in Data Centers and Energy Demand." 
Goldman Sachs. "AI is poised to drive 160% increase in data center power demand." 
Yale E360. "As Use of A.I. Soars, So Does the Energy and Water It Requires."

Richard Granger, PhD, President Non-Von
Eli Bowen, PhD, CTO Non-Von

Non-Von chips reduce power costs by two orders of magnitude (100x) for the growing set of sparse, quantized AI systems.  Non-Von's power efficient chips both i) enable low-power untethered edge systems such as sensors, and ii) provide enormous power savings for server-level AI.
 
Current AI systems typically run on GPUs, first designed decades ago for screen graphics processing.  To work fast, GPUs rely on high power consumption, burdening current AI systems with immense electric budgets (1). Currently, many companies are scrambling to hire AI and data science coders, to construct or adapt large models, and then *lose money* on the resulting power bills.
 
Small parts of this burden can sometimes be alleviated with software changes alone.  Many companies are making systems sparse (reducing redundant data), and quantized or compacted (using lowest-precision encodings), such as BitNet and Hamiltonian Net (2).  
 
Orders of magnitude more savings arise by using non-GPU hardware that is directly fitted to these sparse and compacted software methods.  Some chips are designed for particular software examples, such as convolutional nets or transformers, but Non-Von's NV1 chip specifically targets the large and growing class of sparse compacted systems.
 
From first principles, Non-Von has designed a hardware-software solution called SCULPT - sparse compacted ultra low-power toolset for AI, "sculpting away" whatever the model does not need, refactoring it to take maximal advantage of Non-Von hardware.  Any existing model software can be SCULPTed with just a few lines of code, yielding large gains when implemented on Non-Von hardware.
 
By contrast, standard GPUs and von-Neumann architectures prevent efficient code execution, greatly adding to the costs of implementing and running large AI systems.  The Non-Von hardware-software SCULPT architecture allows AI designs that immediately run efficiently, with enormous cost savings.
 
Non-Von can take any trained model, including all common model development packages (Pytorch, ONNX, Tensorflow, Keras, SKLearn, and many more), and automatically convert it to a sparse compacted system implemented on silicon.  These can be printed as custom Non-Von silicon, or imprinted into Non-Von's mass-produced programmable AI accelerator boards.
 
The results are 100 times more power-efficient on tasks ranging from deep-learning-based visual recognition to transformer-based chatbots (3).  
 
 
 
1) AI electrical power usage:
   https://www.nature.com/articles/d41586-024-00478-x
   https://www.goldmansachs.com/insights/articles/AI-poised-to-drive-160-increase-in-power-demand    https://www.wsj.com/business/energy-oil/ai-is-about-to-boost-power-billswholl-take-heat-for-that-c527f27b?mod=panda_wsj_custom_topic_alert
   https://www.theatlantic.com/technology/archive/2024/03/ai-water-climate-microsoft/677602/ 
 
2) BitNet and Hamiltonian HNet (Quantization and sparsity)   
   https://huggingface.co/blog/1_58_llm_extreme_quantization
   https://arxiv.org/abs/2402.17764
   https://tinyurl.com/HNet2023  (AAAI 2023)
 
3) Hokenmeier et al. (2024) IEEE 33rd Microelectronics Design & Test Symposium (MDTS)
   doi.org/10.1109/MDTS61600.2024.10570137
   https://arxiv.org/abs/2409.19389
 
 

Derek Odom, CEO Non-Von
     The limits of current hardware prove particularly stark in the military and national security context. Not only are today’s tactical edge devices limited by the generally low computational capacity of current hardware, but any edge device on the battlefield also must limit its use of cloud communication, to evade enemy targeting of the device. Furthermore, the high power budgets of current edge AI hardware limits the duration of all types of autonomous devices from stand-alone sensors to air/land/sea/space unmanned vehicles.  These compute and power constraints often mean the ability to use advanced AI generally requires a cloud connection. This requires commanders to assess the risk of leveraging AI capabilities with the risk of compromise.  Together, these needs constitute an enormous demand for new approaches to AI at the edge.
 
     The recognition that today’s hardware technology must advance is not a novel idea.  Billions of dollars are pouring into developing advanced processors for AI applications from the edge to the data center.  But most of these "new" solutions are still just incremental fixes to existing technology.  The underlying architectural principal remains von Neumann’s 80-year-old design.  Right now, the industry reminds us of software companies that refuse to start over and instead build ever more convoluted solutions to advance their products. (Windows anyone?).
 
     We propose a different solution, a processor-and-software ecosystem intrinsically designed for AI.  We have constructed a new paradigm to break away from the constraining GPU world: a new chip architecture that enables efficiencies that the GPU ecosystem cannot take advantage of.  We start with the NV1 architecture, that already outperforms current edge GPUs on AI tasks [3], and encompasses a software ecosystem to enable algorithms to work far more effectively and efficiently than GPUs allow.  An innovative communication protocol allows Non-Von to minimize power usage, and data transport costs among nodes are vastly reduced by eliminating the address bus, through local target address matching [3].  Non-Von's new chips open the door for expanded classes of models, particularly suited to efficient edge processing.  Join our mailing list at the bottom of our home page (non-von.com), and help us build the future of AI. 
 
 
[1] “AI Is Poised to Drive 160% Increase in Data Center Power Demand.”
    https://www.goldmansachs.com/insights/articles/AI-poised-to-drive-160-increase-in-power-demand
    https://www.nature.com/articles/d41586-024-00478-x
    https://www.theatlantic.com/technology/archive/2024/03/ai-water-climate-microsoft/677602/
    https://www.wsj.com/business/energy-oil/ai-is-about-to-boost-power-billswholl-take-heat-for-that-c527f27b?mod=panda_wsj_custom_topic_alert
 
[2] Singh and Gill, “Edge AI.”
    https://www.sciencedirect.com/science/article/pii/S2667345223000196
   
[3] Hokenmeier et al. (2024) IEEE 33rd Microelectronics Design & Test Symposium (MDTS)
    doi.org/10.1109/MDTS61600.2024.10570137