ReSharper C++ :: Catch test runner :: use custom .xml output file?

When using the Catch C++ test runner. Is it possible to specify the location of the .xml file for StackTrace of test result instead of writing to standard out?. This is possible to set / use with JetBrains TeamCity.

7 comments

Hello Richard,

Sorry, I'm not sure I understand the problem. The ReSharper test runner is intended to be used from the IDE. If you need an .xml file with the results, why not just run the test executable yourself?

0

Hi Igor,

When Catch is given the 'xml' argv it expects the standard out to be in XML format. Instead, have an option for it to expect a .xml to be generated from the test application and display the results of that.

Running the test executable yourself defeats the point / feature set of using R# test runner.

0

Sorry, I still do not understand. R++ passes the "-r xml" command line argument to the Catch test binary and reads the resulting output from stdout.

Do you have a test binary which writes the output to a file instead of stdout so ReSharper does not read it? Why not write the test output to stdout by default, and if you need to write it to a file use the "-o" command-line option?

The reason why R++ communicates with the test binary via stdout is to read the output as soon as possible to display the current progress.

0

and because "-r xml" is passed it expects all stdout to be valid XML. The Catch test binary is running services that also write to stdout. As a result get: 'test is inconclusive' - 'Cannot parse the output of mytestapp,exe -r xml -d yes .... Please verify that the output is valid XML'

0

So you mean you have console output just before Catch starts executing tests? Console output done during test execution is captured by Catch and should not break the structure of the XML file.

I've solved this problem for Boost.Test by redirecting output of the test executable to a temporary named pipe, I'll see if we can do the same thing for Catch - https://youtrack.jetbrains.com/issue/RSCPP-22157.

1

It works with one exception: Catch does not honor -o when invoked with --list-test-names-only. ReSharper uses --list-test-names-only to discover tests which it can't discover from source code only. It means that --list-test-names-only output will still be contaminated by non-Catch output, and ReSharper might show fake tests as the result, but at least the XML output will be correctly parsed. I've pinged Catch's author about this issue.

0

Please sign in to leave a comment.