Questions here: Any example for situations when tests pass but examples fails (except for HTTP errors)? I can't recall such situation in the past year. Maybe we should rely on the tests to ensure the correctness of the code?
Another concern is time, 1 complete job takes over 40 minutes.

In PR #10700, CircleCI tests are OK but I think examples/covariance/plot_outlier_detection.py should have failed because I removed the _decision_function method from LOF and _decision_function is used in examples/covariance/plot_outlier_detection.py

@albertcthomas Thanks for the example. I suppose we should blame the example of using _decision_function?
I vote +0 here since Circle will needs more than 40min instead of around 10min now.

PR #10700 might not be a good example after all because some examples have been modified and in such a case I think Circle only tests the modified examples and not the other ones (including plot_outlier_detection.py). But is this the intended behavior? Event if this would mean that we could merge a PR breaking an example whereas the full build of the doc would have revealed the problem.

This was done like this in order for PR CIs to finish quicker as @qinhanmin2014 was saying. This is not perfect but covers very well the 95% of cases.

This is not really a problem because master always does a full build, so in the rare case when a change to an estimator does break an example, master fails and we ask nicely the original author of the PR to fix it.

FYI, you can add [doc build] in your commit message if you want the doc to build.

I am going to close this one, feel free to reopen if you disagree or can think of a better approach.

thanks @lesteve. This answered my question and helped me understand the intended behavior of Circle which can be useful when reviewing or working on PRs :). Thanks again @qinhanmin2014.