Can you collect more data similar to hard examples?

People like to focus on the architecture or training techniques, but most real problems can be solved by collecting more relevant data.

If the loss remains high even after getting more data, two potential problems come to mind:

• There is not enough information in your data to correctly predict the target.
• Your model is not complex/expressive enough to properly estimate the target.