Use the condition system to elimate the existing global state

Hi @hadley I am interested in capturing the failures printed to STDOUT so I can do some further analysis in R. Looking at test_file, a lot of the functionality is already there (courtesy of SummaryReporter), but there are 2 problems:

• test_file doesn't return the failure reasons associated with each tests (what I'm looking for)
• a dataframe's flat structure isn't appropriate for reporting failures from multiple expectations

Ideally, there would be a reporter that returns a list instead of a dataframe. It's important enough to me that I'm considering making a PR for this, but this issue makes me wonder if I should wait until after refactoring. So I guess my question is: will the refactoring work you suggest make this type of reporting easier and is it worth it to mess with reporter methods at this point?

Thanks for a great package. Looking forward to updates in the future.

Yes, this refactoring will definitely do what you want. But it's unlikely that I'll have time to work on it before May at the earliest.

That's great! I have been trying to hack together some workaround for this feature without much success. I am in the process of thinking about how to start, but I'm not sure where to begin. Since this is a feature I could use, I would be happy to work on it with your suggestions.

@lentils here is some code to help you capture the output:

library(stringr)
library(data.table)
library(magrittr)

# run tests and save results
errors <- capture.output(results <- test_dir("##Your dir with tests##", reporter = 'summary'))

# parse errors
failure <- grep("Failure",errors) # get vectors with failure tests
failure <- sort(c(failure,grep("Failure",errors)+1,grep("Failure",errors)+2)) # add vectors with failure test descriptions
failure <- errors[failure]
failure <- str_replace_all(failure, "-+$","") %>% str_replace_all("^[0-9]+. \\w+ \\(at \\w+.R\\#\\d+\\): ", "")
failures <- data.table(Test=failure[seq(1,length(failure)-2,3)], Failure=failure[seq(2,length(failure)-1,3)], Details=failure[seq(3,length(failure),3)])
failures[,Details := str_extract(Details, "[0-9]+$")]

# parse results
results <- as.data.table(as.data.frame(results))
results <- results[, .(context, test, checks=nb, failed, time=real)]

Reporters have been cleaned up a lot - I no longer there's much point in refactoring to switch to using the condition system.