Viewing a single comment thread. View all comments

xorbinant_ranchu t1_j4zmevh wrote

Reply to comment by DaLameLama in [D] Inner workings of the chatgpt memory by terserterseness

1 token ~= 4 chars in English 1 token ~= ¾ words 100 tokens ~= 75 words

https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them

I've had it reference much more than ~6000 words so there's definitely something else going on

14

LetterRip t1_j56kpcq wrote

It probably does a summarize if it is longer than the allowed input.

5