You show very specific issues that if you count all of them, is influence.

I don't know

You don't know the capacity of what you're making until you make it though

^ Simply this

Error drives learning:

If Error ∝ (Target - Output)

Then you start your network with random weights (so the Output is random and error is large). When you pass the error back through the network, the weights are adjusted proportional to the error. Over time, the weights will settle where (Target - Output) is as low as possible.

This concept is true for any situation: if you're working with image data, no matter what architecture is used to produce the Output, you still compare it with Target (or 'label', Length of Pagrus for your case), and pass the Error back through the network to improve it iteratively.

Try building the simplest neuron: 1 input -> 1 output and use backpropagation to train until convergence.

For your assignment you could use a CNN (but a simple Feed-forward network would work too as you're just outputting 1 value for total length, so it's really a regression task) to get the Output, and train its weights which are internally shared (the window you shift across the image) which are trained the same way. You compute the output, compare it with the actual length of the Pagrus fish (you passed in as input), get the Error and the method above to improve the Network for the task applies.

I think it'll happen silently as an update to existing frameworks because there isn't much to optimize.

BN is used to reduce covariate shift, it just happened to regularize. Dropout as a regularizing technique didn't become big before ResNet (2014 vs. 2015).

I doubt what you're saying is true, that they're effectively the same. Try putting one after the other to see the effect. Two drop-out layers or BN layers in contrast have no problem co-existing.

edit: sorry what I mean is the variants of drop-out that work with CNNs (that don't have detrimental effects) haven't existed then.

Excellent, thanks!

Is it possible to sign-up for the newsletter to periodically email me on certain topics (using the semantic search, say "motor control for robotics") or keywords like "VR/Wifi"?

Yes. A "better" method makes less sense in context it seems.

When you replace the top layer and train the model, are the previous layers allowed to change?

Are you fixing the weights of the earlier layers?

It's hard to predict exactly, and requires one of two things:

a) Full-time research on new methods

b) Be the one with the breakthroughs

​

B) is hard and A) nobody I know pays for.

We're confined to jobs related to research or applying said research.

That'd lower the confidence scores but relatively, they'll still be just as false-confident.

I don't think it's nebulous. We infuse knowledge, bias, prior etc. like physics (in Lagrangian networks) all the time. I was just addressing his last point. There's no analytical solution for quality we can use as labels.

Networks can understand the difference between pretty and ugly semantically with tons of data, and tons of data only.

See that's the problem. We benefit from eons for evolution to imprint what quality is (i.e. what correlates with real life) the most genetically.

To tell a CNN about quality without using a CNN to analyze is either cyclical or redundant. I'm afraid.

How do you propose we define quality?