Look-ahead cheating for current day stock values

[yacoubb]

When predicting tomorrow's open stock value in a real world situation, the only value you would have for today would be the stock's open price. Currently the model assumes knowledge of all open, high, low, close, volume information which likely gives it an unfair advantage.

[akildemir]

@yacoubb yeap that is right actually. Btw I just changed the way you trade and it makes 6.47x (was 3.84x originally) through 500 days with assuming MSE is 2.25.

[yacoubb]

Interesting, would you be willing to share the new trading algo?

[akildemir]

Yeap of course. But before I start, I realized that my algo can not be used in live situation with the current state of the network.

So the algorithm is given 500 days of prediction, we can determine the dips and peaks. And we buy in dip wait the actual price to come to the next peak, when it does, sell the shares and wait for the next dip. Day-to-day trading don't make good profit because the price doesn't change so much.(in real life situation, sometimes the fee that you pay to the exchange for each trade surpass your profit with this trade. So you may actually loose money.)

So then I realized, with the current state of the network, we couldn't make 500 days of consecutive prediction. For this to happen we needed 499 days of real price values. The reason I thought this algorithm is possible was that(btw it makes closer to 11x now. I wasn't correcting the price point according to the MSE previously), given the 50 days of real price value, you predict the next day open value and then using the network recursively(which means accepting this predicted value as a todays real price) you predict tomorrow price again and so on. But this wasn't what was happening. You needed a real price for each day to predict the tomorrow price.

Knowing the tomorrow price doesn't make so much difference with MSE 2.25. The price already moves 2% up and down in a day generally. So I think only way would be longer term predictions. 1 week, 2 weeks, 1 month maybe. For this to happen I tried it with weekly data so that we don't predict the tomorrow price but a week later price, but then training data went down 7x and MSE got crazy :D. (might still work if the network trained with more data.)

So using the predicted price recursively probably the way to go. But the net doesn't make prediction just based on the price as you stated in the first post. It needs open, close, high, low, volume values too. So those informations also needs to be predicted. So my idea is that now, predict the close, high, low, volume info the way open value is predicted. Then correct those values according to MSE(otherwise MSE would be growing exponentially for each prediction). Accept those values as real and predict the next day values and so on. And see what happens.

I would like to get your opinion o this method. how possible do you think is this? Also do you think increasing the nıumber of LSTM cells might help? I actually never worked on python and AI libraries before. So understanding your current code took me a while. I had done some AI in c++ but never used the libraries.