Estimators in the forest module (random forest and extra trees) and in the bagging module allows to compute the out-of-bag estimates of the performance of the forest.

A nice things to add would to allow the choice of the scoring function using the scorer interface. The oob_score parameter would be equal would be the string corresponding to the appropriate scorer.

Thus, you would have

oob_score : bool or string, (default=False)
        Whether to use out-of-bag samples to estimate
        the generalization error. The oob scoring function could be chosen
        by passing  a string (see model evaluation documentation) or
        a scorer callable object / function with signature
        ``scorer(estimator, X, y)``.