I feel like you're trying to re-invent regression residuals analysis in a NN setting? Where you're comparing your data points to the predicted values or class? There are a lot of tools on the regression diagnostics side, most of which are looking for things that don't really matter in an arbitrary non-linear curve fitting process like NN. So depends on what your need is for the error analysis.