Viewing a single comment thread. View all comments

Fit_Schedule5951 t1_iruexcg wrote on October 11, 2022 at 2:51 AM

Interesting, what's the latency vs openai python?

ggerganov OP t1_irv1b8s wrote on October 11, 2022 at 6:49 AM

Here is a comparison for Intel CPU:

Would be interesting to compare it on M1 when torch starts supporting F16.