After reading the follow-up paper on "Bayesian" stochastic gradient a bit disappointed because you need to invert a #numfeat^2 matrix.