Viewing a single comment thread. View all comments

Ayakalam t1_j5loxsx wrote

Awesome tool!

Some qs, how do you actually index those papers exactly ? Do you use openalex or ?…

Secondly, are you using a library for semantic search ?

2

BullyMaguireJr OP t1_j5n3itx wrote

Thanks! I'm currently just using the arXiv API with OpenAI embeddings for semantic search.

1

Ayakalam t1_j5pa0uo wrote

Ok, but when you say you are “indexing papers”, what do you mean exactly ?

1

BullyMaguireJr OP t1_j5pq2uj wrote

Calculating embeddings for the paper using OpenAI's API.

1

Ayakalam t1_j5pvihn wrote

Ah, so you hand pick LLM papers and compute their embedding s essentially ?

1

BullyMaguireJr OP t1_j66d9um wrote

I just pull the latest Arxiv papers from their API and then compute their embeddings, index them for semantic search, etc.

1