Weight Search Space with Riemannian Geometry for Acceleration of Backpropagation in a Multilayer Perceptron


The convergence of a Multilayer Perceptron (MLP) to a global minimum error in the backpropagation algorithm can be accelerated by introducing the representation of a weight search space, according to the Riemannian geometry, which considers the curvature of this space. The trajectory of the search vector at the training phase is modified by a “force” originated by this curvature. The main concept is developed from the Einstein's General Relativity theory, and the search space (or manifold) must be controlled by parameters like spatial density and scale factor, modifying the gradient descent in the backpropagation algorithm. Some preliminary results show the weights initialization much closer from the final solution, at few epochs, rather than the normal MLP, for an application related here.

  • Abstract
  • 1 Introduction
  • 2 The Weight Search Space
  • 3 The Riemannian Weight Space Model
  • 4 Obtaining the Metric Tensor
  • 5 Coordinates Transformation and the Gradient Expression
  • 6 Gradient Descent in the Riemannian Space
  • 7 Application and Results
  • 8 Conclusion
  • References

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In