TYPE Editorial

PUBLISHED 15 July 2024

DOI 10.3389/fnins.2024.1448517

Editorial: Information theory

OPEN ACCESS meets deep neural networks:
theory and applications
EDITED AND REVIEWED BY
Shuangming Yang,
Tianjin University, China

*CORRESPONDENCE
Anguo Zhang Anguo Zhang1*, Qichun Zhang2 and Kai Zhao3
anguo.zhang@hotmail.com
1
RECEIVED 13 June 2024 Institute of Microelectronics, University of Macau, Taipa, China, 2 School of Creative and Digital
ACCEPTED 27 June 2024
Industries, Buckinghamshire New University, Bradford, United Kingdom, 3 School of Automation,
PUBLISHED 15 July 2024
Chongqing University, Chongqing, China

CITATION
Zhang A, Zhang Q and Zhao K (2024) Editorial: KEYWORDS
Information theory meets deep neural
networks: theory and applications. artificial neural networks, information theory, information bottleneck, deep learning—
Front. Neurosci. 18:1448517. artificial intelligence, deep neural networks (DNNs)
doi: 10.3389/fnins.2024.1448517

COPYRIGHT
© 2024 Zhang, Zhang and Zhao. This is an
open-access article distributed under the
terms of the Creative Commons Attribution Editorial on the Research Topic
License (CC BY). The use, distribution or
reproduction in other forums is permitted, Information theory meets deep neural networks: theory and
provided the original author(s) and the applications
copyright owner(s) are credited and that the
original publication in this journal is cited, in
accordance with accepted academic practice.
No use, distribution or reproduction is We are delighted to introduce this Research Topic, titled “Information Theory
permitted which does not comply with these
terms.
Meets Deep Neural Networks: Theory and Applications”. Deep neural networks (DNNs)
have become a focal point in machine learning research, achieving impressive results
across various tasks. However, understanding their workings and mechanisms remains
challenging (Samek et al., 2021; Gawlikowski et al., 2023). Information theory, a
mathematical framework for representing and analyzing information, has been widely
applied to study the fundamental characteristics of data, such as structure and distribution.
In the context of DNNs, information theory has been instrumental in explaining and
optimizing their performance (Zhang and Li, 2019; Zhang et al., 2022, 2023). For instance,
the information bottleneck theory has shed light on the abstract representations of neural
networks, while entropy and mutual information have been used to evaluate model
complexity and generalization performance (Wu et al., 2023). This Research Topic aims to
explore the intersection of information theory and DNNs, two fields that have profoundly
impacted the understanding and advancement of neural networks and their applications.
The synergy between these disciplines offers promising avenues for developing more
efficient, robust, and interpretable AI systems. In this Research Topic, we present four
papers that illustrate the breadth and depth of research at this intersection, highlighting
innovative methodologies and their applications in various domains.
You and Wang proposed a novel approach to genealogy layout recognition.
Recognizing the significance of genealogies in cultural heritage, the authors introduced a
sublinear information bottleneck (SIB) for feature extraction and a two-stage deep learning
model combining SIB-ResNet and SIB-YOLOv5. This method surpassed existing state-
of-the-art techniques, offering promising results in identifying and localizing components
in genealogy images. This advancement not only aids in genealogy research but also in
preserving cultural heritage through improved recognition technologies.
Li and Peng addressed the challenges of synthetic aperture radar (SAR) automatic
target recognition (ATR). The study introduced a data augmentation technique
that mitigates SAR image noise and a weighted ResNet with residual strain
control. This approach not only enhances computational efficiency but also improves
recognition accuracy, significantly reducing training time and data requirements.

Frontiers in Neuroscience 01 frontiersin.org

Zhang et al. 10.3389/fnins.2024.1448517

The experimental results demonstrated the superior performance inspires further research and collaboration in this exciting
of this method, paving the way for more efficient SAR ATR systems. domain.
Alazeb et al. focused on shifting to the realm of robotic
environments and scene classification. The paper presented
a robust framework for multi-object detection and scene
Author contributions
understanding, leveraging advanced visual sensor technologies
AZ: Resources, Writing – original draft. QZ: Writing – review
and deep learning models. By integrating preprocessing, semantic
& editing. KZ: Writing – review & editing.
segmentation, feature extraction, and object recognition, the
proposed system achieved remarkable accuracy on standard
datasets such as PASCALVOC-12, Cityscapes, and Caltech 101. Funding
This work represented a significant step forward in enhancing the
capabilities of vision-based systems in various applications, from The author(s) declare financial support was received for the
autonomous driving to augmented reality. research, authorship, and/or publication of this article. This work
Finally, Chen et al. delved into the theoretical aspects of was supported by the National Natural Science Foundation of
neural network training. The authors propose a novel method China under Grant 62306001.
for adaptive learning rate estimation in restricted Boltzmann
machines (RBMs) using rectified linear units (ReLUs). By providing
mathematical expressions for adaptive learning step calculation,
Conflict of interest
this approach optimized the learning rate dynamically, improving
The authors declare that the research was conducted in the
the generalization ability and reducing the loss function more
absence of any commercial or financial relationships that could be
effectively than traditional methods. This theoretical contribution
construed as a potential conflict of interest.
offers valuable insights into the optimization of unsupervised
learning algorithms.
In conclusion, this Research Topic showcased the Publisher’s note
innovative research at the crossroads of information theory
and deep neural networks. The contributions presented All claims expressed in this article are solely those of the
here not only advance theoretical understanding but also authors and do not necessarily represent those of their affiliated
demonstrate practical applications that hold the potential organizations, or those of the publisher, the editors and the
to transform various fields. We extended our gratitude to reviewers. Any product that may be evaluated in this article, or
the authors for their exceptional work and to the reviewers claim that may be made by its manufacturer, is not guaranteed or
for their rigorous evaluation. We hope this Research Topic endorsed by the publisher.

References
Gawlikowski, J., Tassi, C. R. N., Ali, M., Lee, J., Humt, M., Feng, J., et al. (2023). Zhang, A., Li, X., Gao, Y., and Niu, Y. (2022). Event-driven intrinsic plasticity
A survey of uncertainty in deep neural networks. Artif. Intellig. Rev. 56, 1513–1589. for spiking convolutional neural networks. IEEE Trans. Neural Netw. Learn. Syst. 33,
doi: 10.1007/s10462-023-10562-9 1986–1995. doi: 10.1109/TNNLS.2021.3084955
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., and Müller, K.-R. (2021). Zhang, A., Shi, J., Wu, J., Zhou, Y., and Yu, W. (2023). Low latency and
Explaining deep neural networks and beyond: A review of methods and applications. sparse computing spiking neural networks with self-driven adaptive threshold
Proc, IEEE 109, 247–278. doi: 10.1109/JPROC.2021.3060483 plasticity. IEEE Trans. Neural Netw. Learn. Syst. 1–12. doi: 10.1109/TNNLS.2023.
3300514
Wu, J., Huang, Y., Gao, M., Gao, Z., Zhao, J., Shi, J., et al. (2023).
Exponential information bottleneck theory against intra-attribute variations for Zhang, W., and Li, P. (2019). Information-theoretic intrinsic plasticity for
pedestrian attribute recognition. IEEE Trans. Inform. Forens. Secur. 18, 5623–5635. online unsupervised learning in spiking neural networks. Front. Neurosci. 13:31.
doi: 10.1109/TIFS.2023.3311584 doi: 10.3389/fnins.2019.00031

Frontiers in Neuroscience 02 frontiersin.org