Nature

A dual enhanced stochastic gradient descent method with dynamic momentum and step size adaptation for improved optimization performance

In modern machine learning, optimization algorithms are crucial; they steer the training process by skillfully navigating through complex, high-dimensional loss landscapes. Among these, stochastic ...
Nature

Convergence analysis and application for high-order neural networks based on gradient descent learning algorithm via smooth regularization

Higher order networks (HONs) extend the traditional pairwise interactions between nodes to higher order interactions involving three or more nodes. This allows for the modeling of more complex ...