chevron-thin-right chevron-thin-left brand cancel-circle search youtube-icon google-plus-icon linkedin-icon facebook-icon twitter-icon toolbox download check linkedin phone twitter-old google-plus facebook profile-male chat calendar profile-male
0 votes
Today I upgraded Isolator from 6.2.5 to 7.0.6. The built in test runner fails for many of my tests due to it not picking up on the app.config in the test project (this bug has already been reported here: The tests failed on a NullReferenceException when trying to use a null reference returned by ConfigurationManager.GetSection().

Before I knew about the app.config bug in the runner, I tried to debug that exception. So I placed a breakpoint where the configuration is being accessed (in the class being tested) and clicked the Debug button of the failing test (in the Armadillo UI). The debug session reached that breakpoint and after seeing that it was null and realizing what the issue is, I clicked the Stop Debugging (Shift+F5) button, at which point Visual Studio crashed (and Watson uploaded a crash report).

I tried using the TypeMock test runner to debug other tests and Stop Debugging at other locations but couldn't get VS to crash. But I consistently (100% of the time) can get VS to crash if I Stop Debugging at that breakpoint when using the TypeMock test runner. If I use another runner (MSTest or ReSharper's) and Stop Debugging at the same point, VS doesn't crash.

Also, is there some way of telling the test runner to ignore certain tests or test classes (without causing other test runners to ignore it, i.e. not using [Ignore]) so that I can use it to find tests fail due to an actual bug and not the app.config issue? If not, is there a way to hide the Armadillo UI until the app.config issue is resolved?

asked by allon.guralnek (10.6k points)

3 Answers

0 votes
Okay, I just got a crash in a completely different location, unrelated to configuration. It seems stopping the test runner in certain locations triggers the crash.
answered by allon.guralnek (10.6k points)
0 votes

Regarding app.config , we'll add support for it in a future version.

Regarding the crush during debug, we will try to recreate this scenario.
It would help us if you could send us a code example which causes it.

Regarding Ignoring tests by Auto Runner, you can use [DontRun] attribute which could be found in TypeMock namespace.

Alex , Typemock support.
answered by alex (17k points)
0 votes

It turns out the crash is much simpler to reproduce than I thought. It in fact has nothing to do with any code you write. It has to do with the breakpoint that is inserted at the start of the method when you click the Armadillo "Debug" button. I am annoyed by it because I've already set the breakpoints I want hit and don't like having to hit F5 just because this breakpoint is inserted every time for no reason. In general I think it should be a separate button (maybe call it "Step Into" which in Visual Studio does exactly that - starts debugging and stops at the first line). Anyway, as soon as that breakpoint is hit I remove it and then hit F5 to get to my real breakpoints. I then find a bug in my code and want to edit it, and so I hit Shift+F5 to stop the debugging at which point Visual Studio crashes. If I don't remove the breakpoint then Visual Studio doesn't crash. My guess is that at the end of the test you try to remove that breakpoint but it no longer exists, which causes the crash. Am I correct?

Detailed repro steps:

1. Create a test method with at least one valid breakable statement.
2. Debug the test using the "Debug" button on the Armadillo UI.
3. A breakpoint will be created on the first breakable statement, and the test will break on it.
4. Remove that breakpoint.
5. Run (F5) the test to completion or stop (Shift+F5) the test. Visual Studio will crash.
answered by allon.guralnek (10.6k points)