I know you said you are interested in MATLAB or Julia, but I'm interested in why not a python library? I mean a simple Google search would show lots of pytorch HFO solutions.

Matlab has a deep learning Toolbox that makes it easy and efficient to train any type of model. Including RNNs. Although, there is a good argument (and a famous paper) that anything you can do with RNN you can do better with CNN. Julia has deep learning libraries, but don't expect nearly the level of support and ease of use as Matlab. Matlab's DL is underrated.

Can you remember the title of the paper?

Almost all of my analysis scripts are in matlab, so it would be easier to do things locally.

https://fluxml.ai/Flux.jl/stable/models/recurrence/

"An empirical evaluation of generic convolutional and recurrent networks for sequence modeling"

Flux.jl is the most popular deep learning library in Julia. I've played around with it a little, it's quite nice and easy to use. It is amazing how much more elegant the implementations become in julia compared to python.

There is also the less known Lux.jl package that is essentially an explicitly parameterized Flux (less mature than Flux though).

Hessian-free second order will not likely work. There are reasons why everyone using gradient descent. The only working second order method seems K-FAC (disclaimer - I have no first hand experience) but as you will use Julia you will have to implement it from scratch, and it's highly non-trivial (as you can expect from method which work where other failed)

You can do Python locally tho

Ok, yeah that's what I was thinking. That totally makes sense. Good luck!

Can you not train in python and dump the results to a file and run analysis on that? Either you have to be an expert in the details of the implementation or you have to use the setup everyone else is using.

[deleted]

Consider that MATLAB can directly call Python functions and scripts. This is built into the core MATLAB, no extra toolbox or 3rd party code needed.