Regarding [1], in module dict_learning, function _update_dict, we update dictionay as such :

dictionary[:, k] = np.dot(R, code[k, :].T)

which corresponds to an update (following [1] notation) :

which should be :

in an online setting.
I think the line above should be changed into :

dictionary[:, k] = R[:, k]

Theoretically this should improve convergence. Using plot_faces_decompositions.py, it appears that this does not break convergence. I intend to benchmark convergence (although we lack dictionary convergence detection for the moment).

This would involve selecting over online and not online formulation (where I think the present derivation is correct in _update_dict).

This is a minor issue, but I think it would be a good idea to comment a bit more the function _update_dict, which is not easy to understand at first sight.

[1] Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2010). Online learning for matrix factorization and sparse coding. The Journal of Machine Learning Research, 11, 19-60.