Desticheq t1_j9xiv9l wrote
Reply to comment by theLastNenUser in [P] What are the latest "out of the box solutions" for deploying the very large LLMs as API endpoints? by johnhopiler
Well, in terms of "out-of-the-box," I'm not sure what else could be better. AWS, Azure or Google provide empty units basically, and you'd have to configure all the "Ops" stuff like network, security, load balancing, etc. That's not that difficult if you do it once in a while, but for a "test-it-and-forget-it" project it might be too difficult.
Viewing a single comment thread. View all comments