Submitted by Careful-Temporary388 t3_zs6rpk in MachineLearning
[removed]
Comments
LiquidDinosaurs69 t1_j16n0is wrote
I recommend PyTorch because stable baselines reinforcement learning library uses PyTorch. I’m using stable baselines3 to develop an automated Bitcoin trader. Stable baseline 3 is a pretty solid RL library.
djc1000 t1_j17bad6 wrote
Using rl to make a bitcoin reader is like orders of magnitude more difficult than trying to make an lstm for stock prices.
LiquidDinosaurs69 t1_j17h0yw wrote
Oh I should have read his post more carefully
djc1000 t1_j17hhkg wrote
I’m intrigued by your project though. I’ve been thinking about building a bitcoin trader with rl but never found the time to break metabolic inertia. Interested in having a collaborator?
LiquidDinosaurs69 t1_j1ef3xg wrote
Not really but I can send you GitHub links to my repos.
djc1000 t1_j1ezb1x wrote
Please do!
Careful-Temporary388 OP t1_j17iydm wrote
Why is that the case? What makes it more difficult?
Careful-Temporary388 OP t1_j17isqw wrote
Why'd you settle on RL? My understanding that LSTM was better for this sort of thing. Is that not the case? Sorry I'm fresh with all of this, probably asking a question that doesn't even make sense here. Is LSTM a form of RL? If so, what algorithms did you end up deciding on?
LiquidDinosaurs69 t1_j195usj wrote
RL is what you need to use when you want to learn to automatically act in an environment (buy and sell bitcoin in my case). Deep learning based RL requires a neural network for estimating the value of an action and for actor-critic methods there's also a policy network. So you can construct the neural networks that RL needs with an LSTM if you want.
I'm not sure if you want to create something that automatically figures out a strategy to buy and sell or if you just want to predict a stock price. If you just want to predict price then you don't need RL and just an LSTM will be sufficient.
I'm using stable baselines implementation of PPO (an RL algorithm). I'm using wavenet style stacked dilated convolutions as a feature extractor. It's not working though lol. I want to use an LSTM but stable baselines3 currently doesn't support it and I'm going to have to find a way to implement it myself.
Careful-Temporary388 OP t1_j1e28sh wrote
Thanks for explaining. I wish you luck my man! Instead of having the bot act, I was just going to have it predict price and then hardcode actions depending on the prediction. Really though, I feel like the net result is the same. There's so many people who have tried this and I'm yet to really hear of a success story, so I doubt there's much money in it unless you're creative with your data sources and your approach. Like finding a correlation somewhere that you can manipulate - for example - selling grain stocks when the weather is predicted to be a certain way for long periods of time.
LiquidDinosaurs69 t1_j1eexl1 wrote
I think some people have been able to conclusively show a profit with this actually. I know I’ve read at least one journal paper where they showed a profit. During my back testing I was actually able to get up to 2% profit per year which isn’t really worth it. Yeah I’ve heard using data from other sources like Twitter sentiments can work. Maybe weather would be useful too if you could find a way to use that data.
VeritaSimulacra t1_j16q158 wrote
Honestly TF 2 and PyTorch are pretty similar (that doesn’t stop people having their tribal fights), the real divide is TF/PyTorch vs Jax (flax equinox).
salomonbrothers t1_j16tt1g wrote
Honestly you can use both quite easily. TF has some nice documentation. You can follow TF's guide on time series forecasting here https://www.tensorflow.org/tutorials/structured_data/time_series
supersharklaser69 t1_j17a2o4 wrote
Tensorflow is dying, I’ll be surprised if it makes it into 2024. PyTorch is the obvious winner with Jax also picking up some niche following.
suflaj t1_j16i1ci wrote
Pytorch is easier to read and write and there are more resources for it. Tensorflow is easier to deploy and (sometimes) more performant.
Although Pytorch 2.0 is in testing and should be out soon. It makes up for whatever performance gap there is between the two apparently.
Regarding model performance it probably doesn't matter which one you choose, but looking at how your LSTM formulation of the solution is just plain wrong, Pytorch will be easier to use for more complex networks, although in practice to fully utilize resources you need to know both.
moist_buckets t1_j16hdbs wrote
Tensorflow with keras is the easiest way to get started. There are many tutorials on YouTube or online doing exactly what you mention. You could start by just following one.