Conference publications

Abstracts

XX conference

Framework of the stochastic gradient descent and its applications

Nikulin V.N.

Russia, 610000, Kirov, Moskovskaya, 36, VyatGU, department MME

In the marketing task (assessment of the quality of the products by the customers) the numbers of the customers are in hundreds of thousands and the numbers of products are in tens of thousands. As a rule, the available data cover only a small percentage of the theoretically total information. Therefore, those data cannot be represented in a traditional form of 2D matrix and, as a consequence, standard factorization methods (such as principle component analysis) are not applicable in this task.

Note, also, that the risk function (or target function) includes a huge number of the regulation parameters and may become unstable if we are minimising it without taking into account the mutual dependence between the factor elements. As a solution to the problem, and in a direct correspondence with the fundamental concepts of the stochastic gradient descent [1], we are proposing to consider consequently all the components (or terms) of the target function, minimising them as a function of the particular parameters which are involved in the definition of this particular term. Compared to usual gradient-based optimisation, we are dealing here with two sets of parameters, and we should mix uniformly updates of these parameters in order to insure stability of the convergence process, because those parameters are dependent.

Three novel and mutually relevant methods are presented in this paper: 1) gradient-based matrix factorization with two adaptive learning rates and their automatic updates, including an explicit update formulas; 2) nonparametric criterion for the selection of the numbers of factors; and 3) nonnegative version of the gradient-based matrix factorization which doesn't require any extra computational costs in difference to the existing methods.

We demonstrate an effectiveness of the proposed methods to the forecasting task in the area of marketing, education (forecast how well particular student will complete required task) and bioinformatics [1].

Литература

1. Nikulin V., Huang T.-H., Ng S.-K., Rathnayake S. and McLachlan G.J. A Very Fast Algorithm for Matrix Factorization // Statistics and Probability Letters, том 81, год 2011. Стр. 773-782.

• view as PDF (in Russian) (PDF)