londons_explorer t1_jcj8p9y wrote on March 17, 2023 at 5:56 AM

Can we run things like this through github.com/OpenAI/evals?

They have now got a few hundred tests, which is a good way to gauge performance.

Taenk t1_jckzuxm wrote on March 17, 2023 at 4:23 PM

Sorry, I am not an expert, just an enthusiast, so this is a stupid question: Where can I see a list of these few hundred tests and is there some page where I can see comparisons between different models?

bo_peng OP t1_jcjuvg9 wrote on March 17, 2023 at 11:02 AM

Yeah that will be cool. You are welcome to try it and I can help.

The rwkv pip package: https://pypi.org/project/rwkv/