G_fucking_G t1_jdifa1c wrote on March 24, 2023 at 4:48 PM

Reply to [P] ChatGPT with GPT-2: A minimum example of aligning language models with RLHF similar to ChatGPT by liyanjia92

Very interesting.

Quick question. How long does training take? For:

SFT Model
Reward Model
RLHF

I saw you used one 3090Ti, so was it done in hours/days/weeks?

G_fucking_G t1_jczd46d wrote on March 20, 2023 at 6:37 PM

Reply to [D]: Vanishing Gradients and Resnets by Blutorangensaft

https://old.reddit.com/r/MachineLearning/comments/px3hzd/d_has_the_resnet_hypothesis_been_debunked/

The advantage of ResNets are most probably not the erasure of vanishing gradients but a smoothing of the loss landscape.

G_fucking_G t1_jc1tmli wrote on March 13, 2023 at 12:54 PM

Reply to comment by CashyJohn in [R] Introducing Ursa from Speechmatics | 25% improvement over Whisper by jplhughes

On which metric are you basing this on? I'm not deep in ASR but in the Whisper paper it is compared to word2vec 2.0 and whisper is better in most categories.