Training GPT-3 Requires 700,000 L Clean Freshwater; What Else Didn’t You Know About LLMs?


TL;DR Breakdown

  • Concerns grow over the environmental impact of large language models (LLMs) like ChatGPT due to undisclosed high energy consumption and water usage.
  • Lack of transparency hampers meaningful studies on the carbon footprint and water consumption of LLMs.
  • Sustainable AI solutions require exploring smaller, climate-positive neural networks and promoting transparency in LLM operations.

As the use of large language models (LLMs) such as ChatGPT continues to grow, concerns are being raised about their environmental impact. These models require significant computing resources, resulting in high energy consumption and carbon emissions. Additionally, the water consumption associated with data centers powering these models remains undisclosed. The lack of transparency and the trend toward using bigger models for various applications are further exacerbating environmental issues. 

The carbon footprint of LLMs

Estimating the carbon footprint of LLMs is challenging because of the lack of transparency and the closed-source nature of many models. However, calculations based on similar models suggest large models like ChatGPT consume a substantial amount of energy. For instance, a comparable LLM called Bloom consumed between 1.2 million and 23.4 million KWH in January. These energy requirements far exceed the average consumption of a US household. Determining the actual carbon footprint is even more complex, as it depends on factors such as the grid’s cleanliness and the location and number of data centers used.

Besides energy consumption, data centers supporting LLMs consume significant amounts of water for cooling purposes. However, the exact water consumption remains undisclosed by companies like Microsoft. A recent study revealed that training GPT-3 in Microsoft’s US data centers required approximately 700,000 liters of clean freshwater. The water consumption would have tripled if training had taken place in Asian data centers. This lack of transparency raises concerns about the overall environmental impact of LLMs and their sustainability.

The need for sustainable AI solutions

Dr. Sasha Luccioni, a leading researcher in ethical AI, emphasizes the importance of developing sustainable AI solutions. She argues against the trend of using larger, less efficient models like ChatGPT for every application. This “one size fits all” approach results in increased energy consumption and computing requirements. Instead, smaller neural networks that have a positive climate impact should be explored for specific tasks. Reevaluating the current approach to AI model development and usage is necessary to mitigate the environmental concerns associated with LLMs.

Lack of transparency hinders meaningful studies

The closed-source nature of many LLMs impedes meaningful studies on their environmental impact. Researchers face challenges in accessing the necessary information to conduct comprehensive assessments. To address this issue, more transparency is needed regarding the infrastructure and operations of LLMs. Providing access to data about energy consumption, carbon emissions, and water usage would enable researchers to quantify and analyze the environmental consequences more accurately.

The increasing usage of large language models like ChatGPT raises significant concerns about their environmental impact. The high energy consumption and undisclosed water usage associated with these models contribute to carbon emissions and strain water resources. To develop sustainable AI solutions, it is crucial to explore smaller neural networks that have a positive climate impact for specific applications. Moreover, transparency regarding the infrastructure and environmental footprint of LLMs is essential for conducting meaningful studies and addressing these concerns effectively.

Share link:

John Palmer

John Palmer is an enthusiastic crypto writer with an interest in Bitcoin, Blockchain, and technical analysis. With a focus on daily market analysis, his research helps traders and investors alike. His particular interest in digital wallets and blockchain aids his audience.

Most read

Loading Most Read articles...

Stay on top of crypto news, get daily updates in your inbox

Related News

Subscribe to CryptoPolitan