For tensorflow, you have to learn to use tensorflow datasets. https://www.tensorflow.org/datasets

You could also save your dataset as an hdf5 file using h5py, then use the tensorflow_io from_hdf5() to load your data. https://www.tensorflow.org/io

Hdf5 is the "traditional" (for lack of a better word) way of loading numpy data that is too big to fit in memory. The downside is that it is slow at random indexing, so people don't use it as much anymore for training networks.

Pytorch datasets are a little easier in my opinion.