Viewing a single comment thread. View all comments

TFenrir t1_jdhqnb3 wrote

Reply to comment by light24bulbs in [N] ChatGPT plugins by Singularian2501

Are you working with the gpt4 api yet? I'm still working with 3.5-turbo so it isn't toooo crazy during dev, but I'm about to write a new custom agent that will be my first attempt at a few different improvements to my previous implementations - one of them namely is trying to use different models for different parts of the chain, conditionally. Eg - I want to experiment with using 3.5 for some mundane infernal scratch pad work, but switch to 4 if the confidence of the agent in success is low - that sort of thing.

I'm hoping I can have some success, but at the very least the pain will be educational.

1

light24bulbs t1_jdi5qau wrote

That's what I'm doing. Using 3.5 to take big documents and search them for answers, and then 4 to do the overall reasoning.

It's very possible. You can have gpt4 writing prompts to gpt 3.5 telling it to do things

3

TFenrir t1_jdicafu wrote

Awesome! Good to know it will work

1

light24bulbs t1_jdijmr3 wrote

My strategy was to have the outer LLM make a JSON object where one of the args is an instruction or question, and then pass that to the inner LLM wrapped in a template like "given the following document, <instruction>"

Works for a fair few general cases and it can get the context that ends up in the outer LLM down to a few sentences aka few tokens, meaning there's plenty of room for more reasoning and cost savings

1

TFenrir t1_jdim3vv wrote

That is a really good tip.

I'm using langchainjs (I can do python, but my js background is 10x python) - one of the things I want to play with more is getting consistent json output from a response - there is a helper tool I tried with a bud a while back when we were pairing... Typescript validator or something or other, that seemed to help.

Any tips with that?

1

light24bulbs t1_jditg4b wrote

Nope, I'm struggling along with you on that I'm afraid. That's why these new plugins will be nice.

Maybe we can make some money selling premium feature access to ours once we get it

2