A Method for Enhancing Generalization of Adam by Multiple IntegrationsLong Jin, Han Nong, Liangming Chen et al.
The insufficient generalization of adaptive moment estimation (Adam) has hindered its broader application. Recent studies have shown that flat minima in loss landscapes are highly associated with improved generalization. Inspired by the filtering effect of integration operations on high-frequency signals, we propose multiple integral Adam (MIAdam), a novel optimizer that integrates a multiple integral term into Adam. This multiple integral term effectively filters out sharp minima encountered during optimization, guiding the optimizer towards flatter regions and thereby enhancing generalization capability. We provide a theoretical explanation for the improvement in generalization through the diffusion theory framework and analyze the impact of the multiple integral term on the optimizer's convergence. Experimental results demonstrate that MIAdam not only enhances generalization and robustness against label noise but also maintains the rapid convergence characteristic of Adam, outperforming Adam and its variants in state-of-the-art benchmarks.
2.8SEFeb 13, 2019
Apply SOA Paradigms in Cyber-Physical System to Enhance Interoperability: State-of-the-Art ReviewSu Zhang
The Cyber-Physical System (CPS) is considered to be the next generation of intelligent industrial automation systems that integrate computing, communication and control technologies. In CPS, the interoperability requirements between devices and processes are growing rapidly. However, the diversity and heterogeneity of technologies and standards between information technology (IT) and operational technology (OT) in industrial systems pose significant challenges. Service-Oriented Architecture (SOA) provides a software design pattern in which software components are connected only through messaging, which is consistent with the fundamentals of modularity and communication in distributed automation systems. SOA is considered to be a key technology for improving interoperability in next-generation industrial automation systems. This paper introduces the latest research advances in applying SOA paradigms to CPS to improve interoperability, including: Open Platform Communications Unified Architecture (OPC UA) and related extensions, implementing SOA at the manufacturing device level, integrating legacy systems in SOA-based automation systems, Plug-and-Play in SOA-based automation systems, dynamic configuration in SOA-based automation systems and the combination of SOA with Multi-Agent Systems and Evolvable Production Systems.