More on AI Carbon Footprint

When it works, machine learning is amazing.  Over my own career, ML has grown (literally) from tiny toys to gargantuan globe spanning monsters.  And it turns out, with enough data and enough compute time, ML can do astonishing stuff.  Like translate human languages–which was considered theoretically impossible when I was a lad, but apparently is definitely possible.

There are many things to criticize about this technology.  It’s hardly fail proof.  It’s famously opaque.  It’s infamously biased.  And it sucks enormous resources, to the point where only the richest corporations can actually do it.

Worse, training large ML models sucks huge amounts of power, potentially emitting significant amounts of Carbon.  This has led to the emergence of a small industry dedicated to measuring and moderating the power consumption and emissions of AI.

This isn’t all that easy to do.  ML training is usually done with large clouds, which means that the actual resources are “virtual”, and not necessarily under the control of or visible to the modellers. Power consumption depends on the actual system configuration, which may or may not be easy to measure or control. In addition, ML training is a complicated and long running process, so resource consumption varies as it runs, so the total resource consumption can depend on details of the actual computation. 

In addition, the actual Carbon emissions depends not only on the power consumption but also on the source of the electricity and other factors such as Carbon offsets.  These factors vary, so it matters exactly which cloud the process runs on.

Current research is tackling two targets; ways to measure and predict resource consumption and emissions, and ways to optimize ML learning.

On the second front, the first optimization may be abstinence:  don’t do it unless you really need to.  Using low power chips and green data centers may mitigate impacts.  Other tweaks might include limiting computation to off peak times.

But you can’t optimize what you can’t measure.  And this stuff is pretty hard to measure [1].

This summer researchers at the Allen Institute and other institutions report on improved tools for measure “Carbon intensity” of cloud computations [2].  The basic idea is to incorporate location and time specific information, to better estimate the impact of cloud computation.

The researchers use historic information to identify times when impact will be relatively low, e.g., off peak hours when “extra” energy is available with little impact.  They suggest delaying computations to start at a low impact time.  And, for long running computations, “pause and restart” might be used to avoid high impact computations. (These strategies are quite familiar to old gray hairs, who used to have to run big jobs on campus time-shared mainframes. : – ) )

The ”start later” strategy actually works well for small jobs, which finish within a low impact window.  Pause and restart are trickier, but should work. Both of these are trading wall clock completion time against Carbon emissions.  Real time processes in general, and deadline driven processes can’t use these methods.  (E.g., we can’t wait several days for today’s weather forecast, can we?)

Overall, it is clear that cloud systems are going to need to provide not only general characteristics, but run time data on resource consumption and impacts. In particular, there will need to be predictive models for expected impacts, which is going to be hard, for sure.

But there is only so much optimization can do.  Using smaller ML models, or other methods entirely should be considered. 

““The first good practice is just to think before running an experiment,” she says. “Be sure that you really need machine learning for your problem.””

(Anne-Laure Ligozat quoted in [3]).

When I was a lad, we could try ML, in fact, try a munch of variations of ML, just to see what happened.  Those days are gone. 

---

1. Nesrine Bannour, Sahar Ghannay, Aurélie Névéol, and Anne-Laure Ligozat, Evaluating the carbon footprint of NLP methods: a survey and analysis of existing tools, in EMNLP, Workshop SustaiNLP. 2021: Punta Cana, Dominican Republic. https://hal.archives-ouvertes.fr/hal-03435068
2. Jesse Dodge, Taylor Prewitt, Remi Tachet des Combes, Erika Odmark, Roy Schwartz, Emma Strubell, Alexandra Sasha Luccioni, Noah A. Smith, Nicole DeCario, and Will Buchanan, Measuring the Carbon Intensity of AI in Cloud Instances, in 2022 ACM Conference on Fairness, Accountability, and Transparency. 2022, Association for Computing Machinery: Seoul, Republic of Korea. p. 1877–1894. https://facctconference.org/static/pdfs_2022/facct22-145.pdf
3. Matthew Hutson, Measuring AI’s Carbon Footprint, in IEEE Spectrum – Artificial Intelligence, June 26, 2022. https://spectrum.ieee.org/ai-carbon-footprint