Viewing a single comment thread. View all comments

GPUaccelerated OP t1_iu4ve0v wrote

Reply to comment by mayiSLYTHERINyourbed in Do companies actually care about their model's training/inference speed? by GPUaccelerated

OK right. That's also a project with immense scale.

I guess the bigger the project, the more inference speed is required. But I've never heard about caring deeply about the ms in training. Mind sharing why that was important in that use case?

1

mayiSLYTHERINyourbed t1_iu7im0x wrote

Our use case was in biometrics, where the test sample would usually range in millions of images which needed to be matched simultaneously. Over here even accumulating 2-3ms over each batch or batch would lead to huge delay.

2