Abstract
Hopfield neural network stands out from artificial neural network models for solving optimization problems due to its unique ability to rapidly converge on global solutions without requiring complex supervised learning steps, although the performance of this network depends mainly on the tuning of its hyperparameters. This study aims to address the complexity of hyperparameter optimization in Hopfield neural networks by framing it as a sequential decision problem. Unlike traditional optimization techniques, which often struggle with the dynamic and iterative nature of hyperparameter tuning, the proposed approach uses reinforcement learning (RL) principles to adapt and optimize in real time. By using RL, we can dynamically adjust hyperparameters based on feedback from the network’s performance, resulting in a more efficient and effective optimization process. The proposed approach significantly improves both the performance of the network and the execution time, thereby increasing the overall efficiency and effectiveness of the system. To demonstrate the effectiveness of the proposed approach, we applied it to the optimization of the tourist visit planning problem. The application of Hopfield neural networks, combined with reinforcement learning to optimize the hyperparameters of this network, has enabled the creation of a powerful model for optimizing tourist itineraries. This approach shows a clear improvement over traditional methods, maximizing the visitor experience and ensuring more efficient and enjoyable visits.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The data sets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
References
Ahn CW, Ramakrishna RS, Kang CG, Choi IC (2001) Shortest path routing algorithm using hopfield neural network. Electron Lett 37(19):1176–1178
Aiyer SVB, Niranjan M, Fallside F (1990) A theoretical investigation into the performance of the Hopfield model." IEEE transactions on neural networks 1.2: 204-215
Balaji B et al. Orl: Reinforcement learning benchmarks for online stochastic optimization problems. arXiv preprint arXiv:1911.10641 (2019)
Benchekroun Y, Senba H, Haddouch K A Novel approach to intelligent touristic visits using bing maps and genetic algorithms. International Conference on Digital Technologies and Applications. Cham: Springer Nature Switzerland, 2023
Bello I et al. Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:1611.09940 (2016)
Bouhouch A, Chakir L, El Qadi A (2016) Scheduling meeting solved by neural network and min-conflict heuristic. In: 2016 4th IEEE international colloquium on information science and technology (CiSt) (pp. 773-778). IEEE
Bouzerdoum Abdesselam, Pattison Tim R (1993) Neural network for quadratic optimization with bound constraints. IEEE Trans Neural Netw 4(2):293–304
Cao D et al. (2020) Reinforcement learning and its applications in modern power and energy systems: a review. J Modern Power Syst Clean Energy 8(6): 1029-1042
Comert S, Yazgan H, Turk G (2022) Hopfield neural network based on clustering algorithms for solving green vehicle routing problem. Int J Ind Eng Comput 13(4):573–586
Culioli JC et al. A neural network for explicitly bounded linear programming. Proceedings of international joint conference on neural networks. 1, 1990
El Alaoui M, El Moutaouakil K, Ettaouil M (2016) A multi-step method to calculate the equilibrium point of the Continuous Hopfield Networks: Application to the max-stable problem. Int J Comput Sci Inform Secur (IJCSIS) 14(6):216–221
Ettaouil, M., Lazaar, M., Elmoutaouakil, K., Haddouch, K. (2013). A new algorithm for optimization of the kohonen network architectures using the continuous Hopfield networks. wseas transactions on computers, 12(4), 155-163
Ettaouil, M., Loqman, C., Haddouch, K., Hami, Y.(2013). Maximal constraint satisfaction problems solved by continuous hopfield networks. WSEAS Transactions on Computers, 12(2), 29-40
Ettaouil M, Loqman C, Hami Y, Haddouch K (2012) Task assignment problem solved by continuous hopfield network. Int J Comput Sci Issues (IJCSI) 9(2):206
Ettaouil M, Loqman C, Haddouch K (2010) Job shop scheduling problem solved by the continuous hopfield networks. J Adv Res Comput Sci 2(1):31–47
Garmendia AI, Ceberio J, Mendiburu A (2022) Neural combinatorial optimization: a new player in the field. arXiv preprint arXiv:2205.01356
García L, Talaván PM, Yáñez J (2017) Improving the Hopfield model performance when applied to the traveling salesman problem. Soft Comput 21(14):3891–3905
Haddouch, K., El Moutaouakil, K. (2018). New Starting Point of the Continuous Hopfield Network. In International Conference on Big Data, Cloud and Applications (pp. 379-389). Springer, Cham
Haddouch K, Elmoutaoukil K, Ettaouil M (2016) Solving the weighted constraint satisfaction problems via the neural network approach. Int J Interact Multim Artif Intell 4(1):56–60
Hopfield JJ, Tank DW (1985) “Neural’’ computation of decisions in optimization problems. Biol Cybernet 52(3):141–152
Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci 79(8):2554–2558
Iranfar Arman, Zapater Marina, Atienza David (2021) Multiagent reinforcement learning for hyperparameter optimization of convolutional neural networks. IEEE Trans Comput-Aided Design Integrated Circ Syst 41(4):1034–1047
Jomaa HS, Grabocka J, Schmidt-Thieme L (2019) Hyp-rl: Hyperparameter optimization by reinforcement learning. arXiv preprint arXiv:1906.11527
Joudar NE, En-Naimani Z, Ettaouil M (2019) Using continuous Hopfield neural network for solving a new optimization architecture model of probabilistic self organizing map. Neurocomputing 344:82–91
Joya G, Atencia MA, Sandoval F (2002) Hopfield neural networks for optimization: study of the different dynamics. Neurocomputing 43(1-4): 219-237
Kennedy MP, Chua LO Neural networks for nonlinear programming. IEEE Transactions on Circuits and Systems 35.5 (1988): 554-562
Kumar GS et al. (2024) Differential privacy scheme using Laplace mechanism and statistical method computation in deep neural network for privacy preservation. Eng Appl Artificial Intell 128: 107399
Lin JS, Liu M, Huang NF (2000) The shortest path computation in MOSPF protocol using an annealed Hopfield neural network with a new cooling schedule. Information Sciences 129(1–4):17–30
Mazyavkina N et al. Reinforcement learning for combinatorial optimization: a survey. Comput Oper Res 134 (2021): 105400
Nithya TM et al (2024) TB-MFCC multifuse feature for emergency vehicle sound classification using multistacked CNN-Attention BiLSTM. Biomed Signal Process Control 88:105688
Rbihou, S., Haddouch, K. (2021, October). Comparative study between a neural network, approach metaheuristic and exact method for solving Traveling salesman Problem. In 2021 Fifth International Conference On Intelligent Computing in Data Sciences (ICDS) (pp. 1-5). IEEE
Rbihou, S., Joudar, N. E., En-Naimani, Z., Haddouch, K. (2023, February). Using Crank-Nicolson Scheme for Continuous Hopfield Network Equilibrium. In International Conference on Artificial Intelligence and Industrial Applications (pp. 201-210). Cham: Springer Nature Switzerland
Rbihou S, Joudar N-E, Haddouch K (2023) Parameter tuning of continuous Hopfield network applied to combinatorial optimization. Annals of Mathematics and Artificial Intelligence 1-19
Rbihou S, Haddouch K, El moutaouakil K (2024) Optimizing hyperparameters in Hopfield neural networks using evolutionary search. OPSEARCH 1-29
Shih H-S et al. (2004) A neural network approach to multiobjective and multilevel programming problems. Comput Math Appl 48.1-2: 95-108
Smith KA (1999) Neural networks for combinatorial optimization: a review of more than a decade of research. Informs J Comput 11(1):15–34
Talaván PM, Yáñez J (2008) The graph coloring problem: a neuronal network approach. Euro J Oper Res 191(1):100–111
Talaván PM, Yáñez J (2006) The generalized quadratic knapsack problem. A neuronal network approach. Neural Netw 19(4):416–428
Tan KC, Tang H, Ge SS (2005) On parameter settings of Hopfield networks applied to traveling salesman problems. IEEE Trans Circ Syst 52(5):994–1002
Talaván PM, Yáñez J (2005) A continuous Hopfield network equilibrium points algorithm. Comput Oper Res 32(8):2179–2196
Talaván PM, Yáñez J (2002) Parameter setting of the Hopfield network applied to TSP. Neural Netw 15(3):363–373
Wen UP, Lan KM, Shih HS (2009) A review of Hopfield neural networks for solving mathematical programming problems. Euro J Oper Res 198(3):675–687
Wu D et al. (2022) An improved teaching-learning-based optimization algorithm with reinforcement learning strategy for solving optimization problems. Comput Intell Neurosci 2022
Yang Li, Shami Abdallah (2020) On hyperparameter optimization of machine learning algorithms: Theory and practice. Neurocomputing 415:295–316
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no Conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Rbihou, S., Joudar, NE. & Haddouch, K. Optimizing parameter settings for hopfield neural networks using reinforcement learning. Evolving Systems 15, 2419–2440 (2024). https://doi.org/10.1007/s12530-024-09621-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12530-024-09621-5