[deleted]

High chance you're right

Would you say “coming up with architectures that randomly work / don’t work” is a shortcoming of your understanding or of the field in general?

I’m asking because I’m thinking about doing the opposite switch right now - ML interests me deeply and I’m currently in standard cloud development.

[deleted]

Try more applied fields!

Not in industry (PhD student), but used to do deep learning type research with a computer vision background. I'm now working on applying ML for computer systems. At least 90% of my actual research work is building the system, collecting the data, and deciding how to frame the problem; for that last 10%, essentially anything reasonable will do. The catch is that if you don't have a solid ML background, you might completely miss that last 10% and never realize!

It's much more satisfying since if you're working on applications, you likely have potential downstream use cases lined up (that's what collaborators are for!). You're also probably the only person working on this specific problem formulation, so you don't need to worry about beating SOTA and squeezing out that last 5% -- your data-driven algorithm is 2x better than a traditional approach, so does it really matter?

It's a property of the field in general - there is very little theory to guide neural architecture design, just some heuristics backed by trial-and-error experimentation. Deep learning models are fun, but in practice you spend a lot of your time trying to trick gradient descent into converging faster.

People can't do deep learning or AI without the tools to make them happen. Imagine how complicated the data collection methods at the scale of Terabytes of data and cleaning it. People also need to annotate data for it to work which requires software to get it done in a cost effective manner.

I agree with this. I used to do Machine Learning but shifted to working at an open-source company (Argilla), to be more involved with data quality instead of overfitting and building models without enough data.

> Coming up with architectures that randomly work/don't work, tuningparameters, waiting for days till the model is trained... the level ofuncertainty is just too high for me.

Good news! You say you work in computer vision. There's a high chance that in practice you'll mostly use off-the-shelf solutions and most of your actual time will be spent gathering data:)

You can do a lot in other subfields. Even basic churn predictions are something valuable for a lot of firms. Are you sure you’re not swept by the DL hype yourself?

I know exactly what you are talking about. Perhaps you could train a model to tell you what to do next? We have folks all the time believing that ML/DL will solve any problem… :-)

Haha not sure if this is that comforting

The model is ready, I'm here to collect data ;)

Do you mean roles such as data engineer / MLOps?

How would you call your position? Is it software engineering / data engineering or yet something else?

In case I'd like to dive into something along these lines, how such positions tend to be called?

I was wondering if this has something to do with lack of experience. And from what I've heard indeed the more you experiment with the models, the better you understand what helps when, to some extent.

The thing is, a neural network still remains a black box, as the number of parameters is too big to fully understand what is happening. It is an empirical study mostly. So you choose your architecture, test, change hyperparameters, test, change the architecture, test, change some other parameters, test, and so on. You can't be sure your model will work properly right away and it may take lots of iterations. With larger models which take long to train it might be overwhelming.

Of course, it might be different in your case, you can start with some toy examples, and if you still like it, I'd recommend playing with larger networks.

That was my first thought too. I'm about to finish my masters program and I spent the first half thinking that it was just hyperparameter tuning, until I sat down and learned the math and theory. Now it's so much more interesting and explainable. That random tuning is now much more calibrated from experience and understanding the theory. (As of now), I could easily make a career out of this, because it's not random and simple optimization. Plus, the field is so hot right now, that it's unreasonable to assume that what data scientists do now is what they will do in 5, 10, or 20 years

You can pivot to an adjacent role within ML like MLOps or Data Engineering. That’s what I did. I didn’t like waiting an entire day for a model to run and then having to fix it late at night so I can do another iteration throughout the night. So now I build machine learning platforms which is more traditional software engineering and comes with predictable outcomes. Your knowledge about ML is still valuable.

Staying up late to run another iteration sounds very familiar haha. Are you happy with your change or did it turn out that the grass is greener on the other side?

The whole thing seems very bitter lesson-y and I suspect in the future we'll have a very general architecture that learns to reconfigure itself for the data.

I dislike Computer Vision. It just annoys me somehow.

I've had jobs that were similar to what you describe. My current job contains less by way of tiny tweaks to massive DL models and more feature engineering and engineering in general which suits me better.

My slightly warm take is that DL at the coal face in industry feels very random, very time consuming, and as a result a bit demoralising. More power to you if you have the knack for it, and enjoy it, it's just not super my bag.

A lot of NLP specialists are having an existential crisis since chatgpt and now gpt4 have shown to solve pretty much any problem you throw at it. Some of us have used our knowledge of productising large language models to move into MLOps and data engineering where the outcome is more deterministic

[deleted]

What would you say are the key skills I should sharpen to work as a data engineer? (ofc I can google it, but I'm curious how it looks like from your experence)

I'm probably not the most qualified since I'm a PhD student, but in my experience it's generally an ML position where the description requires you to have some domain knowledge or a position title in the application domain that specifies that you're doing ML.

I actually went the other way. My background is FEDGOV hurricane scientist who now works at a popular weather app after teaching myself the ML part. We do a lot of statistical modeling in meteorology already (mainly bias correction and downscaling), and I made the decision to move away from dynamics-based research to statistics-based applications since it's about to take over our field as well. I'd argue meteorology is one of the original big data problems, but I guess that's a topic for a different day.

Anyway, there are a bunch of meteorology companies that are using AI/ML for a variety of things. Since you said you were doing computer vision, identifying things in satellite imagery (Sentinel-2, LandSat, maybe even GOES) is very popular right now. It doesn't get you away from the core issues of tuning models, but maybe a different focus might work for you...?

I would just warn you that the pay in the field of meteorology sucks relative to BIG TECH.

Manage people who still do machine learning. Motivating scientists isn't unlike the constant iterative grind of matching the right reward function to the right model.

Never quite made it to working on ML professionally. This week I'm a 'Data Engineer'.

I like how you phrased it "this week"

Well I started of doing my time in the Data Warehouse. I was hoping I could retire to the Data Lakehouse. Now it's being drained by a Data Pipeline and the rest is slowly floating off into The Cloud.

Amusingly they recently changed my team name to Data Integration and Engineering. The DIE team.

I’m happy with the change. MLOps is an exciting field right now. There’s a lot of companies that need people who knows ML and software engineering.

I jumped from ML ( 2 years) to MLoPs ( 1 year) then to backend engineering (1 year) . Now I am planning to come back to ML. Mostly worked with CV was bit bored since we mostly used off the shelf models, but other endeavours turned out to be more boring for me . It's my personal opinion, there's nothing wrong with the fields. I don't find it intellectually challenging. For me it was bit stressful because it seems there were lot of systems in place which created complexities. Coming back to ML, because I feel it satisfying ( although it has its cons) but I do enjoy it more than the other fields that I have tried.

I think "Machine Learning Engineer" or even just "Data Scientist". Most of the jobs out there are probably exactly like this. In fact there's a shortage of people who want to do the 'boring' stuff, compared to people who want to be researchers. If you are good at MLops and implementing solutions that you know will work, you're super valuable.

It is more software engineering working on the core package and creating educational content (videos, presentations etc.) about getting from no data to a decent baseline model. Combining both, really helps to understand what people struggle with.

1 - Data engineering and DevOps
2 - It's way less stressful than ML because you have really clear requirements ( I need to get data from a source in a certain target with those constraints ). This sometimes can be challenging due to business requirements (Time, consistency, and monitoring those pipelines) but I find it better than go into a project where I don't even know if it will be feasible or no.
3 - I was a good programmer before I got to ML, so for me it was like I switched back to what I used to do, so it was not a big deal. ( My curriculum was a lot of software engineering / managing networks and pure dev)

I went into a quant research position at a prop trading firm and am very happy about that decision. While unfortunately I have not been using any deep learning so far, it is a lot of stats and machine learning. Also there are some interesting applications of physics informed neural networks, which I want to look into at some point. It is definitely fun to work on problems now that just need to be solved in a creative way instead of continuously having to come up with new research ideas.