Submitted by sharp7 t3_xu70v4 in MachineLearning
adam_jc t1_iqui2g0 wrote
you can do n-digit addition of positive integers as a sequence where each digit is a token, i.e.
the problem 946 + 82 = 1028 could be made into sequence of:
9 | 4 | 6 | + | 0 | 8 | 2 | = | 1 | 0 | 2 | 8
(you could also omit + and = tokens).
Andrej Karpathy uses this task in his minGPT repo.
edit: also in that repo he does character level training on a tiny dataset of Shakespeare writing
sharp7 OP t1_iquj21o wrote
Omg perfect thanks!
Viewing a single comment thread. View all comments