Refining ADAM optimizer in LSTM to improve the stock price prediction over HMM and GMM

Abstract Forecasting Stock prices is a critical challenge in financial markets due to their dynamic and volatile nature. These forecasts may be made more accurate by applying advanced machine learning methods. This study aims to enhance the accuracy of predicting a stock's next day's closing price by employing Long Short-Term Memory (LSTM) networks and performing a comparative analysis between LSTM networks and the Hidden Markov Model-Gaussian Mixture Model (HMM-GMM) approach. Along with Stochastic Gradient Descent and ADAM, two modified ADAM algorithms were also implemented to improve the performance of LSTM. This study also includes a detailed breakdown of the LSTM algorithm to better understand the intricacies of LSTM. Historical stock price data was used as input to train and validate the LSTM network. The model's predictions were then compared with the results obtained from the HMM-GMM method using the Mean Absolute Percentage Error (MAPE) as the evaluation metric. The results demonstrate that the LSTM with modified ADAM optimisers outperform the HMM-GMM model in accurately predicting the next day's closing prices. The value of this research lies in exploring the nuances of LSTM. This paper discusses the different gates inside LSTM, backpropagation about each gate, and various layers, along with their corresponding derivations and equations. This research has successfully increased the accuracy of forecasting a stock's subsequent day's closing price by using the Long Short-Term Memory (LSTM) networks. The improved accuracy of our model shows that LSTM can be used by investors to better predict stock prices. The detailed exploration of LSTM architecture presented in this study contributes to the broader understanding of advanced machine learning models.