Viewing a single comment thread. View all comments

Philpax t1_jed8vd0 wrote

Deploying anything developed with Python to a end-user's machine

3

General-Wing-785 OP t1_jedl8nt wrote

Is this because of a lack of tools to deploy Python models/functions to the edge? Also when you say user’s machine, is it a browser or a native app?

1

Philpax t1_jee04jo wrote

It's just difficult to wrangle all of the dependencies; I want to be able to wrap an entire model in a complete isolated black box that I can call into with a C API or similar.

That is, I'd like something like https://github.com/ggerganov/llama.cpp/blob/master/llama.h without having to rewrite the entire model.

For my use cases, native would be good, but web would be a nice to have. (With enough magic, a native solution could be potentially compiled to WebAssembly?)

4