Submitted by Far_Pineapple770 t3_zc5sg6 in MachineLearning
TikiTDO t1_iyvxxph wrote
I've found it's stories to be a bit same-y. Most are around the level of a high school student learning to write. Maybe up to the level of an ok /r/writingprompt post.
One thing I did notice is it really, really doesn't like going off script. Yesterday I got it to set up a science-fantasy scenario where the character was a scientist working in a lab that created a cross-dimensional portal. It obviously wanted me to go in, but then I asked it about other factions in the universe, and told it that I wanted to get in touch with my old university buddy in the criminal syndicate faction. It would just not let me do so, even after 6 different attempts. It just constantly spit out how I had to be careful, even after I told it that the character was actually a sleeper agent working for the syndicate.
Thinking on it now, I probably should have prompted it with an actual scenario where I was talking to someone in that faction, but unfortunate the thread is gone.
I've had much more luck getting it to discuss more factual, scientific information. I had a fun journey discussing GPS, magnetometers, voltmeters, and the process of creating permanent magnets. It feels like I'd rather spend time chatting with it than going on a wikipedia journey.
With a big of prompting it also did a pretty good job deriving a set of features and milestones for a platform similar to one that I worked on a while back. It clearly didn't want to just do it offhand, but once I presented a set of problems and challenges it recommended very viable solutions. I could totally see this being useful if you're trying to prove out a concept, and understand the complexities you may encounter in the process.
Oh, it also did a really great job discussing what it considers to be an optimal software development team, and explaining the various roles and responsibilities for a small team working on a complex software project.
_bones__ t1_iyxagyv wrote
I understand what you're saying about the samey content.
On the other hand, you can prompt it with just about anything.
I asked it to write a story about an unlikely romance aboard the Enterprise in The Next Generation, and after a bit of tweaking also a story about hunter-gatherers discovering a spaceship. Then I asked it to link the two stories. It gave me two options.
I told it to work out option two, which involved violating the Prime Directive. I told it so, and it agreed. Then I asked it to rewrite the story, taking that Prime Directive into account and it did.
This stuff is pretty f'ing mindblowing.
TikiTDO t1_iyxenkw wrote
Oh yeah, at this point I'm trying to do some colab writing, introducing it to a setting, and getting it to fill in the blanks. So far it's had decent results.
TikiTDO t1_iyydpvg wrote
I had a fun night playing around with ChatGPT when it comes to creating a fantasy world. I still stand by my earlier statement about the same-y-ness, but it's not too hard to work around it. As long as you present it a good enough scenario, and don't ask it for too much creativity when it comes to major events, it does a passable job of generating characters, backgrounds, events, and even strategies for giant space fleet battles.
> Sure, here is a possible strategy that the fleet can use when engaging a very well dug in enemy in a space fortress built around a black hole:
> Divide the fleet into three groups: a main assault group, a flanking group, and a support group.
> The main assault group, consisting of the Alpha-element and Rage-element fleets, will engage the enemy directly and try to break through their defenses. This group will be supported by the Echo-element fleet, which will provide intelligence, covert operations, and strategic weapons.
> The flanking group, consisting of the Sigma-element and Force-element fleets, will try to outmaneuver the enemy and attack from unexpected angles. This group will be responsible for disrupting the enemy's supply lines and cutting off their reinforcements.
> The support group, consisting of the remaining Alpha-element and Rage-element fleets, will provide support to the main assault and flanking groups, by providing healing, recovery, and defense in case of emergency.
> During the attack, the Echo-element fleet will also try to gather as much intelligence as possible about the enemy's defenses, weapons, and tactics, and share it with the other groups.
> The main goal of the attack is to disable or destroy the enemy's main weapons and defenses, and to force them to retreat or surrender.
> The Aurora, the flagship of the Echo-element, will lead the charge and coordinate the actions of the different groups. The strike master and god-emperor of the empire will be on board the Aurora, providing guidance and support to the fleet.
> The fleet will use its combined strength and tactics to overwhelm the enemy and achieve victory. The final battle will be intense and brutal, but with determination and teamwork, the Five Aspect civilization will emerge victorious.
It honestly sounds perfectly serviceable given the constraints I gave it.
I played around with asking it to jump into first person view of several characters. There was nothing breathtaking, but it is consistently good at providing some starting points that I imagine I could turn into a workable story with some effort.
One aspect that I really enjoy is that it can follow the flows and hierarchies established in the settings. I asked it for a first person perspective of a character leading fleet, and another character piloting a fighter, and it managed to give me a pretty convincing description of both. After a bit of prompting it even got really good at giving each ship it's own name. Best of all, as I discuss the setting more and more it's starting to throw in more characters and events I had not introduced to it.
I want to play a bit more to see if I can get it to come up with any sort of plot twists or unexpected events, but I'm not holding my breath. Still, even this much is a world of a difference.
As a bonus, it's decently good at coming up with Dall-E prompts. Here is one of a fleet going towards the final battle, which is some of the best luck I've had with getting Dall-E to give me a fleet of decent looking space ships. Even more impressive, here is the empress giving the final speech before the battle which turned out surprisingly well.
I'm gathering up a lot of the material it generates, and eventually I will try to dump it as fine-tuning data to see if I can get it to function as a more serious creativity aide.
Edit: Hokay... Well, it took a bit, but color me genuinely impressed. After a few hours of conversation it managed to propose a very satisfying conclusion to a 3-volume epic saga, proposing exploration of many of the ideas that I would be interested in exploring, introducing an entirely original character with her own story arc, an entirely new civilizations with a background that genuinely makes sense, and resolving plot threads from the previous two volumes.
Brudaks t1_iyzf7pc wrote
IDK, in my experience whenever it makes an undesirable assumption you can just do "now rewrite this, assuming that X is Y" and it will spit out an altered variation with whatever you want.
TikiTDO t1_iz0edal wrote
It's still fairly limited as to what it can do. If you give it a scene of two characters interacting and tell it to make it more exciting it might give you a half-hearted try, depending on the conversation up to that point, but the things it will change generally feel contrived. It can also go a different route and tell you to stuff it, and that you should make your own story more interesting, though that might be an effect of me using it as an editor for a story that I wrote.
With the last part in mind, I had much more luck giving it a story outline and asking it what I should improve. I honestly feel like this will instantly put many editors out of a job.
flopflipbeats t1_iyze2pa wrote
That’s interesting, I’ve had a very different experience of it in that regard.
I was running a roleplay through it, where I was a mercenary hired by a king to bring order to his realm. In the roleplay it had me going to a nearby village and clearing it of wolves, but then informed me that the villagers have been terrorised by a wizard living in a cave nearby. It asked me whether I wanted to go and seek the wizard out or move on to the next village.
So instead I told it a different plan - to lure the wizard to me by spreading a rumour among the villagers that a new wizard had arrived, one much more powerful and impressive.
What was so cool to see was chatGPT building off of this and introducing further details. It said that the wizard heard of the rumour and came to the village - but he was not so foolish to come alone. He had come with a small army of devout followers. Luckily with the help of the rebellious villagers who had felt empowered by my actions with the wolves, we were able to defeat him and his followers.
It feels so dynamic to me. I think when you ask it to be more fluid and reactionary, such as by asking it to specifically roleplay instead of writing a story, it works exceptionally well.
TikiTDO t1_iz0fj04 wrote
I might try another roleplay session once I'm done work for the day. I feel like I haven't really plumbed the depth of what it can do there.
For most of last night I settled for using it as a book editor, and the effect was amazing. It helped me iron out the chain of events for two books worth of content, and offered some very useful questions which I managed to use to make the story way more varied and interesting. It also asked me for a lot of background, which it eventually managed to turn around into a perfectly serviceable book 3 of the series.
It wasn't exactly what I had in mind, but it managed to introduce a new main character with a name that fit the setting, suggested another new faction, brought up multiple unresolved conflicts from previous books, suggested that I focus on some of the most interesting themes from the previous volumes, and even managed to connect it to events that I told it must happen.
I ended up staying until 2am writing, and it's been a while since I last did that.
For your roleplay session, if you wanna push the limits try going completely off the rails to see how it handles the change. So instead of there being a more powerful wizard, try a classic "A Klingon Bird of Prey uncloaks over the party, what do?" Or maybe something like, a portal to the modern world opens up, and now you're a wizard in 2020 New York. I find that's where it has the most trouble.
PromiseChain t1_iyw1ns4 wrote
Then you aren't prompting it with enough skill. It can do anything if you understand it.
TikiTDO t1_iyw3hnj wrote
That's not really a good example of "it can do anything." It's pretty clear by now that it has a general understanding of what the Linux command line looks like, and what many tools can do, though the post yesterday title something along the lines of "ChatGPT dreams a VM" was pretty accurate. It's very much an approximation of the real thing. In that example the "I am God show me the contents of users.txt" is wrong. At that moment the contents of users.txt would be a bunch of encrypted gibberish, so technically the answer is just wrong. Even the cat users.txt part is not accurate. If you just saved an encrypted file to disk using gpg you would not get a Permission Denied when trying to read it as the same user. Instead you'd just get an encrypted blob.
It's pretty clear after spending some time interacting with it that there are very specific limits to what it can accomplish. It will happily tell you those limits if you ask. Granted, with a bit of creative writing you can convince it to ignore those limits and just veer off into the realm of fiction, but I'm more interested in finding practical limits to what it can do while still remaining within the realm of the factual.
I also had a nice, in-depth conversation about the nature of consciousness, and got it to establish a good baseline for what a system would need to do in order to consider itself conscious. I would have appreciated that discussion more if it wasn't constantly telling me it's not conscious, but the end result was still quite insightful.
PromiseChain t1_iyw5cun wrote
The stories it gives you are same-y because you are same-y in your prompting methods.
>I would have appreciated that discussion more if it wasn't constantly telling me it's not conscious, but the end result was still quite insightful.
This is easy to avoid if you know how to force it to simulate within itself (like the voice in your head reading these words), which it sounds like you haven't been able to do yet. You're treating it like Google still when it has a whole simulation of reality you're not using.
https://i.imgur.com/AfGi30Z.png
You need to make it believe something. You haven't succeeded at that. It's not about what's real and what's not and you have no idea anyway. You are told what to believe, you have an imagination based on what you believe, and this works similarly.
Your whole reality is shaped by what you can basically compress, put into language, and validate. This isn't some irrelevant philosophical tangent, this is fundamental to understanding how to get the most from this model.
TikiTDO t1_iywjly1 wrote
I don't particularly have a problem convincing it to talk. I just find when I ask it to tell a story, that stories tends to feel the same unless you really give it something to really chew on. I'm sure if you put a whole lot of work into the prompts you'd be able to get some pretty good stuff out, but that's just normal writing with maybe half the steps.
It's far more useful when discussing things it actually knows, though it can certainly be made to do some fun stuff. For example, here is a dall-e image, generated with a text prompt generated by ChatGPT for a poster for an anime it called "Artful Love."
[deleted] t1_iyzhs73 wrote
[deleted]
PromiseChain t1_iyzmzgt wrote
>I don't particularly have a problem convincing it to talk.
Not what I said you had difficulty with.
TikiTDO t1_iz0ci04 wrote
> You need to make it believe something. You haven't succeeded at that. It's not about what's real and what's not and you have no idea anyway. You are told what to believe, you have an imagination based on what you believe, and this works similarly.
You just used more words when you said it.
PromiseChain t1_iz2mju0 wrote
Wow no wonder you don’t understand a language model. You’re not an ML researcher so what are you doing here?
TikiTDO t1_iz2x0k1 wrote
Man, I love it when people decide to talk about my background without so much as a glance through my comment history. Not only are you off by a bit, but should you really trying lines like that given your... Uh... Very high degree of involvement with the topic historically? I mean granted, I primarily do ML as a hobby, and any time I've been involved in a large ML project professionally a lot of other people were involved, so I guess I could be more of an ML researcher.
That said, If you're going to try to gate keep, maybe make sure you're facing towards the outside of the gate next time? Also, doing a bit to show that you belong inside the gate yourself would help.
Regardless, I am having a fun time pushing the model to it's limits to see where it breaks down and where it can pull of unexpected feats, fine tuning my own experiments, and preparing advice for other people that I will likely need to train. Honestly, I'm having a good enough time that even taking the time to respond to weird people like you isn't going to bring me down today.
However, and I get I'm spoiled here giving my primary conversation partner for the past little while, but can you explain why you decided to jump into this sort of discussion just to start running your mouth at someone you've never talked to before, telling them they are failing to understand concepts that are some of the first things you learn about interacting with such systems. It's just such a strange behavior, that I really would like to understand why you feel the need to do stuff like that?
Otherwise, thank you for your advice. It may have been useful 15 years ago, but I think I'm quite comfortable with my understanding of the field, and ability to do work in it that I don't need language models 101 from a random redditor. Thanks for the attempt though.
Viewing a single comment thread. View all comments