As an evolutionary learning guy, I'll say it's crazy this didn't already exist! Thanks for sharing. Is it based on any publications, or are you considering writing one?

I agree with your point, but playing devil's advocate, isn't it possible the AIs we end up creating may have a much different, "unnatural" type of consciousness? How do we know there isn't a "burst" of consciousness whenever ChatGPT (or its more advanced future offspring) answers a question? Even if we make AIs that closely imitate the human brain in silicon and can imagine, perceive, plan, dream, etc, theoretically we could just pause their state similarly to how ChatGPT pauses when not responding to a query. It's analogous to putting someone under anaesthesia.

This technique is similar to data augmentation, but with a specific focus on important samples. There may not be a specific name for this technique, but it could be considered a form of "strategic oversampling" or "strategic repetition" of important samples. By repeating these important samples in every batch, you are increasing their impact on the training process and potentially helping the neural network to converge to a better solution that takes these samples into account.

It's worth noting that this technique may not always be appropriate or necessary, and it could potentially lead to overfitting if not used carefully. However, in cases where there are a small number of important samples that have a disproportionate impact on the end application, repeating them in every batch can be a useful approach to ensure that the neural network learns to incorporate their information effectively.

:-P

What theory are you referring to when you say "theoretically"?

Right, even though language is a form of time series, in practice it doesn't use TSP methods. Transformers are not surprisingly being applied to TSP problems though.

The analysis relates to time series prediction problems. Isn't it fair to say vision and language do not fall under that umbrella?

Yeah, a solar panel would make sense. It doesn't need to convey the entire content of the article.

What's the benefit of neural nets being able to approximate analytic functions perfectly on (-inf, inf)? Standard neural nets can approximate to arbitrary accuracy on a bounded range, and training data will always be bounded. If you want to deal with unbounded ranges, there are various ways of doing symbolic regression that are designed for that.

If I'm understanding correctly, you're proposing each link (dendrite) could have a polynomial transfer function as a way to introduce additional nonlinearity. Is that correct?

First of all, there's the significantly increased computational costs (no free lunch). Second, what is it buying you? Neural nets as they're currently formulated can already approximate any function to arbitrary precision. Your method would do that in a different way, but it would be much more inefficient while not adding any additional expressive power. Making the activation function non-monotonic seems like a bad idea for obvious reasons (at least for typical neural nets), and making it more complex than a sigmoid seems pointless. The success of ReLU units relative to sigmoids shows that reducing the complexity of the activation function has benefits without significant drawbacks.

It's not a bad question, but I think there's a clear answer.