Submitted by MLNoober t3_xuogm3 in MachineLearning
pseudorandom_user t1_iqwoucu wrote
Assuming you mean how the weights/biasses are summed up, we don't. In neural networks the sum of weights is passed through a (usually) nonlinear activation function. Because of the nonlinearity we get the universal approximation theorem which says a 1 layer neural network is suffecient to model any continuous function, with some caveats.
Basically the nonlinearity in the activations is assumed to be suffecient for all applications, therefore it is not necessary to have a more complex structure.
However, when in comes to discontinuous functions things get more interesting, 1 layer NNs cannot, for example, model the XOR function. But with a more complex or Nonlinear construction this might be possible.
Viewing a single comment thread. View all comments