faschu OP t1_j3q0sr7 wrote on January 10, 2023 at 7:21 AM

Reply to comment by Sylv__ in [Discussion]: Quantization in native pytorch for GPUs (Cuda)? by faschu

Thanks a lot for the detailed reply! I will try these suggestions.

faschu OP t1_j30wfpm wrote on January 5, 2023 at 7:48 AM

Reply to comment by jakderrida in [Discussion]: Quantization in native pytorch for GPUs (Cuda)? by faschu

Thanks for the reply. But do these three quantization techniques work on the GPU without TensorRT? The supported backends led me to believe they are not: https://pytorch.org/docs/stable/quantization.html#backend-hardware-support

faschu OP t1_j2xt5i6 wrote on January 4, 2023 at 6:24 PM

Reply to comment by _Arsenie_Boca_ in [Discussion]: Quantization in native pytorch for GPUs (Cuda)? by faschu

Thanks for the reply!

I personally find TensorRT hard to debug and I prefer to use it only in production when I'm absolutely sure that the model produces the desired results.

faschu t1_iucwopi wrote on October 30, 2022 at 11:24 AM

Reply to [N] Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI | Lex Fridman Podcast #333 by Singularian2501

Thanks for the nice episode. Don't see a point to criticize the talk they had, but instead enjoy all the interesting parts (such a the books, e.g.).