Again, you keep bringing up the same point - “art work being used without permission” - and I keep arguing that this is no different to a person looking at a piece of art as inspiration.

It’s perhaps more of a philosophical issue, and it also relates to my personal belief that DL models are closer to analogous to the brain than a lot of people imagine - but this is purely conjecture.

Yes, but if you took a 4 year old child who had never seen a painting before, showed them 10 paintings, and then asked them to make their own painting. Either, they’ll just scribble on the canvas randomly because they’re not competent enough to do anything, or they’ll end up making something very similar and nearly identical to those examples you’ve shown them.

You use the example of the programmer taking code off the internet… I’m not sure if you’re a programmer yourself, but you know that’s a meme right? The joke is that a big part of programming is finding the right stack overflow/blog/tutorial that has the code similar enough to what you need, and you change bits of it and incorporate it into your work.

AI models take the art, and add it to their training inputs.

It doesn't have perfect memory of the inputs - this can be demonstrated by the fact that model sizes are significantly smaller than the size of data used to train them. Similarly, 'own perception' is an interesting idea. What does it actually mean? I'd argue than in an ML model, utilising some random input when training, to allow for different outputs for the same input (e.g, how chat GPT can reply differently even if you ask it the exact same thing on two different occasions).

I'm not saying we should treat AI models as if they're human beings - I don't think an AI model should be able to hold a copyright for example, but the company thats trained that model should be able to.

Similarly, if the AI model were to output something VERY similar to some existing work, then I think that the company that owns said AI model should be taken to court.

So, while this is certainly true, for something to come under copy right it had to be pretty similar to whatever its copying.

For example, if I want to write a book about wizards in the UK fighting some big bad guy, that doesn't mean I'm infringing on the copy right of Harry Potter.

Similarly, I can write a pop song that discusses, idk, how much I like girls with big asses, and that doesn't infringe on the copyright of the (hundreds) of songs on the same topic.

Now, I do think that if an AI model output something that was too similar to some of its training material, and the company that owned that said AI went ahead and published it, then yeah the company should be sued for copyright infringement.

But, it is certainly possible for AI to output completely new things. Just look at the AI art that has been generated in recent month - it's certainly making new images based off what its learnt a good image should look like.

​

Also, on top of all this, its perfectly possible to ensure (or at lest, massively decrease probability of) outputting something similar to its inputs, by 'punishing' the model if it ever outputs something too similar to training inputs.

​

All this means that I don't think this issue is anywhere near as clear cut as a lot of the internet makes it out to be.

I mean, yes an AI model learnt from other people’s examples, but is that also not what humans do?

Chat GPT is a form of deep learning model, which is a subsection of a machine learning model. A machine learning (ML) model is one in which the decisions the model makes are based off a ‘training’ step rather than being physically encoded.

A simple example is a model that tried to distinguish between different breeds of flower. So, you give this model some information about each flower (petal length, colour, etc) as well as a ‘truth label’ (what a flower expert has said that flower is).

The model takes these numbers as inputs, these inputs are multiples by a set of numbers, have some numbers added to them, and then get passed to the output, and some value is decided as a cut off (eg, if output >5 it’s flower A, otherwise it’s flower B) If the model is wrong, all those numbers get changed a little bit, in a process known as stochastic gradient descent.

In a deep learning model, the inputs are multiplied, and then passed to a ‘hidden layer’ of nodes (often called neurons). Then these numbers are again multiplied by another set of numbers. This keeps going for multiple layers until you get to the output layer.

This is an over simplification, but is the basis of how things like chatGPT work. They simply look for patterns, and output the next word based on what they think matches the pattern.

What makes chat gpt pretty powerful is (mostly) it’s size. It contains 175 billion of those numbers that have to get updated while training, and so takes a long time + is very expensive to train

Very interesting, thanks for the information! Goes to show that scientific papers don’t always mean useable results!

Yeah, it’s certainly difficult. But it’s also complicated. For example, I believe ML models looking at certain cancer scans have higher accuracy than experts looking at the same scans. In this situation, if someone is told they have no cancer (by the scan) but it turns out they do, is the model really at fault?

I think the thing that should be done in the time being, is that models should have better uncertainty calibration (I.e, in the cancer scan example, if it says this person has an 80% chance of cancer, then if you were to take all scans that scored 80% chance, then 80% of them should have cancer, and 20% should not) and then a cutoff point at which point an expert will double check the scan (maybe anything more than a 1% ML output)

There’s probably definitions for what “high risk” is. Maybe for example “high risk” means 90% of people in that group overdose within 6 months. These definitions are obviously decided by the person creating the model, and so should be based on expert opinion. But predicting someone as “high risk” 86% of the time is pretty damn good, and it’s definitely a useful tool. However, it probably shouldn’t be the only tool. Doctors shouldn’t say “the ml model says you’re high risk, so no more drugs”, instead a discussion should be started with the patient at this point, and then the doctor can make a balanced decision based on the ml output, as well as the facts they’ve got from the patient.

Agreed. ML is the future, but it needs significant legislation to ensure its safe. ML probably should just be used as an aid, and not as a final truth.