op_prabhuomkar
op_prabhuomkar OP t1_j5i7oyj wrote
Reply to comment by kkchangisin in [P] Benchmarking some PyTorch Inference Servers by op_prabhuomkar
Thank you for the feedback. I am looking forward to using the Triton's model analyzer possibly with different batch sizes and also FP16! Lets see how that goes :)
op_prabhuomkar OP t1_j5k0h1j wrote
Reply to comment by Ok_Two6167 in [P] Benchmarking some PyTorch Inference Servers by op_prabhuomkar
It’s actually easier to do for HTTP, will probably take that as a TODO. Thanks for the suggestion!