Konstantin Vorontsov: Additive regularization of topic models

September 11, 2014, Laboratory for Internet Studies hosted an academic seminar “Additive regularisation of topic models” with Konstantin Vorontsov. 

Konstantin Vorontsov - Doctor of Physics and Mathematics, Senior Researcher  Dorodnitsin Intitute Computing Centre of Russian Academy of Sciences, Associate Professor of the Department of Mathematical Methods of Forecasting at the Moscow State University, Deputy Head of the Department of Intelligent Systems at Moscow Institute of Physics and Technology and Professor of the Joint Department with Yandex at Higher School of Economics.

 

In his presentation, Prof. Vorontsov reviewed the problems of the ambiguity of stochastic matrix decomposition and its impact on results of topic modeling procedures. He reminded that the topic modeling task is in fact an ill-posed problem. Therefore, taking into account the relationship between the amount of a priori information and the stability of the algorithm, the additional a priori information is needed for a good fit. 

 

The new advanced approach ‘ARTM’ was presented, capable to significantly improve the results of topic modeling. The method is based on the application of the classic model of probabilistic latent semantic analysis (PLSA). And EM algorithm estimates the parameters of PLSA model with maximum likelihood approach.  Prof Vorontsov and his team had mathematically proved that the standard LDA model may be viewed as a special case of Additively regularized model. The report produced lively discussion. After that further collaboration in regard to the integration of BigARTМ into TopicMiner (software developed by LINIS) were outlined.