Just a note to remember that we can arguably still greatly improve the Coordinate Descent (lasso / elastic net) convergence speed if we implement the following trick:

http://www-stat.stanford.edu/~tibs/ftp/strong.pdf