Resharper NUnit 3 Parallelism

For NUnit 3 Does Resharper obey the NUnit Parallelizable attribute, e.g. if I set ParallelScope.Self, that says that the fixture can be executed in parallel with other fixtures

Similarly, if I change "Run upto x assemblies in parallel" to more than 1, and I mark assemblies with the NonParellizable attribute is it obeyed?

 

 

3 comments
Comment actions Permalink
Official comment

The "Run up to x assemblies in parallel" setting of R# is completely separate and is not affected by the concurrency settings of a particular UT framework. It is merely an indication to us that we should try to execute each test assembly separately, and have X concurrent runners.

As for [Parallelizable] and [NonParallelizable] attributes: how them being applied at different levels change the behavior of test execution depends on NUnit engine. We might, however, need to update the version of NUnit we use inside R#. (Usually changes made to NUnit are made to nunit.framework.dll, which always comes from project references, thus not requiring us to update other parts, so we a few versions behind. But this case might be different).

Comment actions Permalink

My experiments indicate that ReSharper does not, unfortunately, "obey" the NUnit parallelizable attributes, either way. Parallelism is controlled purely by the R# setting.

The underlying reason appears to be that ReSharper has its own test runner, as opposed to invoking an NUnit executable, meaning that it calls into the NUnit Framework directly and has its own mechanism for executing and evaluating tests.

My guess is that R# needs its own test runner in order to be able to visualize its "test sessions" window (test names, running time, result, ...). Of course that is unfortunate because we are entirely dependent upon this custom test runner to evolve, if we want to leverage new features in the framework.

Edit: I can confirm that, strangely enough, parellism does work at the level of test fixtures. That makes me think that the real problem is just how ReSharper handles individual test assemblies. NUnit 3 can only parallelize multiple assemblies if it knows about them all, but R# probably just passes assemblies into NUnit one by one, and spins up more NUnit processes depending on your "number of assemblies to run in parallel" setting.

I surely hope to see a reply from JetBrains here, what are their plans?

0
Comment actions Permalink

@Eugene Strizhok

Good to see an official reply!

On the matter at hand: my experiments yesterday clearly indicated that it is not possible to control which assemblies are run in parallel with the R# test runner. NUnit 3 console does this flawlessly, by regarding "[assembly: Parallelizable]" attributes.

However, in order to do this, NUnit console of course has to know about all the assemblies you want to test. Am I correct in assuming that R# just spins up NUnit processes for each assembly, one-by-one, until the maximum number of processes that you choose in the configuration setting have been reached?

0

Please sign in to leave a comment.