Xamarin Studio's Test Runner: Everything is broken and nobody cares?

ThomasBandtThomasBandt Thomas BandtDEDeveloper Group Leader ✭✭

I'm using Xamarin on a daily basis since more than 3 years now, always on a Mac with MonoDevelop/Xamarin Studio. It's not as comprehensive as Visual Studio with ReSharper, but it does its job.

But one thing is really bothering me, and it doesn't seem to get any better: Support for automatic tests is totally broken from my point of view. Not in a pure technical sense, because you get nunit tests eventually to run somehow. It's mainly the details that make it nearly unusable to me.

  • I am not able to debug any test (within a PCL)
  • Starting the debugger will always result in a switch back from the "Unit Testing" layout to the "Solution" layout
  • I need to compile my library, close my file and to re-open it to get XS to recognize a new test I've just written.
  • All the tests within the solution are organized by their namespaces, which are fairly deep. Which means a lot of clicks to get to them.
  • I can't pick a single test from the "Test Results" pad and re-run it (just another detail)

After all even the default test runner of Visual Studio is way better than everything here. I can't imagine how people at Xamarin are developing software. Without tests? Or without Xamarin Studio? Both would be far from ideal ...

So, now that I did complain a lot, I am wondering if I am the only one who desperately is looking forward to some improvements here.

PS: I am using the stable channel, so everything is related to the latest stable bits.

Posts

  • ravensorbravensorb Shawn Anderson USMember ✭✭

    You bring up good points - I too would love to know what the road map is to improving overall unit testing in general (including support for newer nunit/xunit frameworks)

  • mattwardmattward Matt Ward GBMember Xamurai
    • I am not able to debug any test (within a PCL)

    This seems to work for me but I am probably doing something different. I created a PCL project and NUnit test project and was able to debug a test. Can you provide more information about your setup?

    • Starting the debugger will always result in a switch back from the "Unit Testing" layout to the "Solution" layout

    Xamarin Studio switches to the debugging layout. The Test Results pad, if you have it auto-hide seems to briefly appear and disappear. You can probably workaround this by having the Test Results pad visible in the debugging layout. How would you prefer this to work?

    • I need to compile my library, close my file and to re-open it to get XS to recognize a new test I've just written.

    This seems to work for me. I just need to compile the unit tests project for the test to be displayed in the Unit Tests window. Can you provide more information here so I can repro the problem?

    • All the tests within the solution are organized by their namespaces, which are fairly deep. Which means a lot of clicks to get to them.

    There is an open enhancement request for this problem.

    • I can't pick a single test from the "Test Results" pad and re-run it (just another detail)

    This was fixed recently. There is a Rerun test menu item on the right click menu for the Test Results pad. It looks like this currently will be available in Xamarin Studio 6.0.

  • ThomasBandtThomasBandt Thomas Bandt DEDeveloper Group Leader ✭✭

    This seems to work for me but I am probably doing something different. I created a PCL project and NUnit test project and was able to debug a test. Can you provide more information about your setup?

    Interesting! It's working now with the latest stable bits.

    Xamarin Studio switches to the debugging layout. The Test Results pad, if you have it auto-hide seems to briefly appear and disappear. You can probably workaround this by having the Test Results pad visible in the debugging layout. How would you prefer this to work?

    What I mean is something different: Choose "Debug" from the "View" menu, set a breakpoint in a test and start debugging that test by using the menu behind the checkmark-button. After that debugg session ends, the layout is set back to "Solution".

    This seems to work for me. I just need to compile the unit tests project for the test to be displayed in the Unit Tests window. Can you provide more information here so I can repro the problem?

    When compiling the test project the "checkmark button" left to the test doesn't appear. Even closing and opening the file again doesn't help, I nead to recompile the project.

    Compiling the project to be able to run a test is just slowing down the development process. It's little things like that that make me switch back to Visual Studio and ReSharper. A test should be available to run as soon as I mark it as a test (by adding the [Test]-attribute).

    There is an open enhancement request for this problem.

    And again, nobody cares. This request ist more than three years old.

  • mattwardmattward Matt Ward GBMember Xamurai

    Xamarin Studio switches to the solution layout.

    I can repro this in Xamarin Studio 5.9. Unfortunately the right click menu in the text editor does not seem to be working for me in 5.10. Debugging the test from the Unit Tests window In Xamarin Studio 5.10 does not switch to the solution layout, which it does in Xamarin Studio 5.9.

    I need to compile my library, close my file and to re-open it to get XS to recognize a new test I've just written.

    OK now I understand this. It looks like the unit test icons in the left hand gutter of the text editor do not appear when you expect them to. When adding a new test method I had to build and run the tests before the icon appeared.

    This feature has been improved in the Roslyn preview (Xamarin Studio 6.0) here Xamarin Studio uses information from the parser so the test icons are available as soon as you add the NUnit test fixture or test attributes.

  • ThomasBandtThomasBandt Thomas Bandt DEDeveloper Group Leader ✭✭

    So we're all looking forward to Xamarin Studio 6.

    I would really appreciate if this topic could receive some more love in general – maybe someone of you guys should start to work test-driven ;-).

  • GaluteGalute James Millen GBMember

    I haven't done it recently, but even when the unit test icons are not shown, a secondary click in the position the icon should be shows the run/debug etc menu, without having to go through the whole recompile, run tests loop.

    The magic button as it's known in our development team.

  • JonathanDibbleJonathanDibble Jonathan Dibble GBMember ✭✭

    I imagine Xamarin Studio's days are numbered, following the acquistion of Xamarin by Microsoft. Everyone will be moving over to Visual Studio

  • JohnHairJohnHair John Hair GBMember ✭✭✭✭

    @JonathanDibble, Xamarin Studio is also available on the Mac and I dont see Visual Studio for Mac happening anytime soon, if ever. I do see MS improving Xamarin Studio though, and maybe a rebrand of it later. Pure speculation on my part of course.

  • JonathanDibbleJonathanDibble Jonathan Dibble GBMember ✭✭

    Yes @JohnHair I use Xamarin Studio on a macbook. I think MS will concentrate on maintaining Visual Studio as a superior development IDE, leaving XS to stay frozen in time.

  • DaveHuntDaveHunt David Hunt USMember ✭✭✭✭✭

    @JonathanDibble said:
    Yes @JohnHair I use Xamarin Studio on a macbook. I think MS will concentrate on maintaining Visual Studio as a superior development IDE, leaving XS to stay frozen in time.

    Per the Xamarin team - "Don't worry – Xamarin Studio on Mac has a very bright future."

  • GaryMcGheeGaryMcGhee Gary McGhee AUMember

    A year later, this is still essentially true. I'm trying to run unit tests in Xamarin Studio that reference Xamarin.Mac and having lots of trouble. It doesn't seem to be a supported route.
    NUnitLite is open source (https://github.com/mono/guiunit) and I have been able to execute a test in the same assembly.
    It should be possible to fork and develop guiunit as required.

  • MichaelMcCoolMichaelMcCool Michael McCool GBMember ✭✭

    It doesn't work from Visual Studio either. We can't debug the unit tests, won't even run. Release is fine.

Sign In or Register to comment.