Wouldn’t it be a good idea to bring a more energy efficient language into the ML world to reduce the insane costs a bit?[D]

Submitted by thedarklord176 t3_10vqxtp in MachineLearning

I know Python is the primary choice because it’s a simpler language for data scientists to use and a lot of ML libraries are made for Python. But, if you look at this, it is extremely inefficient with energy. And everyone knows big models like ChatGPT cost a ton to keep running. Maybe a more efficient but not too difficult language, like C#, is something we should consider giving more attention in ML?

0

Comments

You must log in or register to comment.

The-Last-Lion-Turtle t1_j7j1qxl wrote

Pytorch is written in C++ and CUDA.

Python is really just an interface, with a minimal contribution to the execution time.

55

thedarklord176 OP t1_j7j35wu wrote

But isn’t everything in Python from C? By that logic I’d think that would make no difference because it’s still Python. Not saying you’re wrong, I don’t work in AI I’m just curious

−46

username-requirement t1_j7j5ihu wrote

The critical factor to consider is whether the computation spends time in the python code or C/C++.

Many of the python language constructs are quite slow, and this is why libraries like numpy exist. The program spends relatively little time in the python code which is merely acting as an interpreted, rapid-to-modify "glue" between the compiled C/C++ library functions.

In the case of tensorflow and pytorch virtually all the computation is being done in C/C++ and python is basically acting as a highly flexible configuration language to do setup.

18

currentscurrents t1_j7j78vc wrote

All the computation is happening on the GPU. Python is just making a bunch of calls to the GPU drivers.

Researchers spend a lot of time making neural networks as fast as possible. If switching to another language would have given a substantial speed boost, they would have done it already.

12

blacksnowboader t1_j7j6y2y wrote

The answer is sort of but not really. A lot of the common packages in ML And data science are in python, but the computations happen in C/C++, Fortran, Scala and others to name a few.

3

PredictorX1 t1_j7izzeg wrote

Just to be clear, deep learning is the energy consumer. "Shallow" machine learning (logistic regression, multilayer perceptron, tree induction, etc.) and related technologies cost pennies to fit.

15

currentscurrents t1_j7j7dgu wrote

Call me when logistic regression can generate a realistic detailed digital painting by greg rutkowski.

−8

CKtalon t1_j7j14ab wrote

Most inference/mlops solutions don’t really use Python despite being used to develop the model.

Stuff like Nvidia’s Triton inference server is used for speed up.

5

Azmisov t1_j7n2491 wrote

No. All the energy intensive computations occur on dedicated hardware like GPU/TPU. These run a compiled instruction set that would not benefit from using a different language frontend. You have to tackle energy efficiency at the hardware level, and in this respect, the number of flops/watt has steadily gone up over the years. The ML tasks always grow to fill the extra computational efficiency though. At this point, progress in ML is the fruit of increased energy efficiency, not energy cost.

2

LanchestersLaw t1_j7mkxvx wrote

It matters way more what you doing than the language itself. I can easily make an infinite loop C program that uses more energy than a Haskell program.

1

MrSurname t1_j7ralcv wrote

You can minimize the energy costs by hosting the primary interface on a COBOL mainframe, with eleventeen node-processors.

1

MrEloi t1_j7kolg7 wrote

The 'busy' core stuff will be written in a low level high efficiency language.

0

Additional-Lack1978 t1_j7mkmh8 wrote

It’s not python, but the underlying amount of data, i.e. Tensor size and interdependence.

0

evanthebouncy t1_j7mntx2 wrote

Good points but python is NOT the problem.

0

mskogly t1_j7jds7k wrote

Perhaps when we can grow human-like brains and interface with them?

−3