Viewing a single comment thread. View all comments

op_prabhuomkar OP t1_j5i7oyj wrote

Reply to comment by kkchangisin in [P] Benchmarking some PyTorch Inference Servers by op_prabhuomkar

Thank you for the feedback. I am looking forward to using the Triton's model analyzer possibly with different batch sizes and also FP16! Lets see how that goes :)

2

kkchangisin t1_j5if8hc wrote

Depending on how much time I have there just might be a PR coming your way 😀…

Triton is really a somewhat hidden gem - the implementation and toolkit surrounding it is pretty impressive!

2