Transactions on Cybernetics
Riemannian Adaptive Optimization Algorithm and its Application to Natural Language Processing
This article proposes a Riemannian adaptive optimization algorithm to optimize the parameters of deep neural networks. The algorithm is an extension of both AMSGrad in Euclidean space and RAMSGrad on a Riemannian manifold. The algorithm helps to resolv…