>shouldn't the onus of delineating man from machine be on the side providing the AI chatbot?

It is.

Here's a very long read, but it will explain how OpenAI is building in watermarking for use by govt + themselves + maybe academia.

https://scottaaronson.blog/?p=6823

>'to watermark, instead of selecting the next token randomly, the idea will be to select it pseudorandomly, using a cryptographic pseudorandom function, whose key is known only to OpenAI. That won’t make any detectable difference to the end user, assuming the end user can’t distinguish the pseudorandom numbers from truly random ones. But now you can choose a pseudorandom function that secretly biases a certain score—a sum over a certain function g evaluated at each n-gram (sequence of n consecutive tokens), for some small n—which score you can also compute if you know the key for this pseudorandom function'

And why they wouldn't just stick it in a database of logs:

>'Some might wonder: if OpenAI controls the server, then why go to all the trouble to watermark? Why not just store all of GPT’s outputs in a giant database, and then consult the database later if you want to know whether something came from GPT? Well, the latter could be done, and might even have to be done in high-stakes cases involving law enforcement or whatever. But it would raise some serious privacy concerns: how do you reveal whether GPT did or didn’t generate a given candidate text, without potentially revealing how other people have been using GPT? The database approach also has difficulties in distinguishing text that GPT uniquely generated, from text that it generated simply because it has very high probability (e.g., a list of the first hundred prime numbers).'

It’s kind of like trying to detect if a student used a calculator on a math test.

The key point to take away is that it’s no longer useful to teach students how to do complex calculations in their head. What’s far more important are the fundamental concepts. Do they understand the formulas and when to apply them, do they understand how to use the tools available to them to achieve a goal.

The end goal has never been to write an essay, it’s just to convey information. Far more important than the essay itself is information being conveyed. What idea is the student trying to communicate.

It will take academia a while to adjust though. For many years teachers stuck with the mantra of “you won’t always have a calculator on you”. I’m sure some will cry “you won’t always have access to an LLM generative text engine”, but we all know that’s simply not true.

There's the issue of locally run LLMs. It's even possible low-scale with models like Pygmalion, but it would be an even bigger issue if there wasn't low-scale models, as nothing would stop the richer people from having a language learning model on the downlow, or as funny as it sounds, there might even emerge some sort of black-market of LLM

people are also seemingly very careless about what they put into LLMs

Ultimately you can work around any detector by just having an AI learn solely of the cheater’s work. They’ll pick up on your mannerisms, which steps you skip, which route you’re more likely to take (convert to other units or just take it on as is), and what not.

In the end, there will be no way to detect AI’s work from humans. You could literally make the AI be wrong, just enough times not to trip the system but still get an A.

All this does is illustrate how stupid homework is and how important in class learning and work is.

Traditional education will always fail in a world with AI because traditional education barely works today. To put it more accurately, it barely works at the expense of the students who push themselves to work in an idiotic system.

> Am I missing something in my assessment?

Something you may be missing is that these models are being made available to collect data from users to make them better. Is data valuable to competitors commonly shared by competitors?

Also ever peek at charts showing how GPU power is growing vs time? In perhaps a decade our pocket devices may be locally running the current models. Anything done now will not help in the long run. I expect the various efforts to detect AI writing and AI art will add up to nothing because the error rates will be too high to be useful.

To be fair, we're already a society built on cheating. All rich people already cheat. They used natural LLMs called "poor people" to do all their work for them and then just took credit for it. We call those people "CEOs".

Mostly thinking out loud here.

Education is already fundamentally busted on like 50 levels. Increased anti-cheating tech is so far from the original point of learning that it's not about learning anymore. Grading itself is already a necessary evil at best. And cheating being bad is a conclusion from that.

Okay, education, ideally, is for learning. In practice it has a double purpose of also being for certification. Which is to say, the examination and projects you're supposed to not cheat on are the point. This isn't really a good thing, but it may be a necessary one. Still, a lot of prestigious and/or exclusionary institutions are overwhelmingly about certification and not learning which clearly isn't.

More certification and stricter certification do not ensure more learning. In fact, they're often the opposite. In my own experience as a college student, I've had plenty of classes I cheated with super hard and ones I did honestly, and I've seen no correlation between classes I cheat on and classes where I don't learn much. I've had classes I did honestly, were challenging, got a great grade, and learned shit-all. I've also had classes I cheated on every single thing and learned a lot doing it.

In my experience, too, ChatGPT has been like a total godsend for learning. I ask questions to the professor, which needs to be done during the limited window of office hours, and they tell me to 'look over my notes and figure it out' cause they always do and they never help. I ask ChatGPT and they explain exactly what I want forever and I can ask about anything and I can do it at 2AM.

Anything degree that relies on writing assignments is up to being automated anyway.