[R] ZerO Initialization: Initializing Neural Networks with only Zeros and Ones openreview.net Submitted by hardmaru t3_ys36do on November 11, 2022 at 6:28 AM in MachineLearning 47 comments 159
mikeful t1_ivxqhgi wrote on November 11, 2022 at 11:55 AM Neat. You could try to initialize them to 0.1 or 0.9 as it's unlikely that weights will stay at zero or one after training anyway. Permalink 1
Viewing a single comment thread. View all comments