Deadlock in NUnit test

Hi,

In attempting to resolve an async issue I'm having, I've discovered some strange behavior in the NUnit test runner:

[Test]
public async void async_test_1()
{
    await SomeAsyncThing();
    System.Diagnostics.Debug.WriteLine("Finished execution.");
}

[Test]
public async Task async_test_2()
{
    await SomeAsyncThing();
    System.Diagnostics.Debug.WriteLine("Finished execution.");
}

private Task SomeAsyncThing()
{
    return Task.Run(() =>
    {
        System.Diagnostics.Debug.WriteLine("Executing.");
        return true;
    });
}

Running async_test_1 works fine. Running async_test_2 blocks. It outputs "Executing" and then hangs with this stack trace:

System.Threading.WaitHandle.WaitOne_internal () in 
System.Threading.WaitHandle.WaitOne (millisecondsTimeout=-1, exitContext=false) in /Developer/MonoTouch/Source/mono/mcs/class/corlib/System.Threading/WaitHandle.cs:379
System.Threading.ManualResetEventSlim.Wait (millisecondsTimeout=-1, cancellationToken=The vm is not suspended.) in /Developer/MonoTouch/Source/mono/mcs/class/corlib/System.Threading/ManualResetEventSlim.cs:189
System.Threading.Tasks.Task.WaitCore (millisecondsTimeout=-1, cancellationToken=The vm is not suspended., runInline=true) in /Developer/MonoTouch/Source/mono/mcs/class/corlib/System.Threading.Tasks/Task.cs:699
System.Threading.Tasks.Task.Wait (millisecondsTimeout=-1, cancellationToken=The vm is not suspended.) in /Developer/MonoTouch/Source/mono/mcs/class/corlib/System.Threading.Tasks/Task.cs:672
System.Threading.Tasks.Task.Wait () in /Developer/MonoTouch/Source/mono/mcs/class/corlib/System.Threading.Tasks/Task.cs:649

The only syntactic difference between the two is that test 1 is declared as void whilst test 2 is declared as returning Task.

Any thoughts?

Posts

  • mhutchmhutch Mikayla Hutchinson USXamarin Team Xamurai
    edited March 2014

    I don't think the version of nUnit included in XS supports async tests. See e.g. http://stackoverflow.com/a/20893562/116899

  • KentBoogaartKentBoogaart Kent Boogaart AUMember ✭✭

    According to this it should be fixed. I have asked for clarification on the NUnitLite google group, but need to wait for my post to be approved. Will post a link here once it is available.

    I suppose it's possible that MonoTouch.NUnitLite is not based on NUnitLite 0.9. That will be the next thing I have to chase down...

  • KentBoogaartKentBoogaart Kent Boogaart AUMember ✭✭

    Please see the google group post here. Michael, would you mind weighing in on this? It would be awesome if the Xamarin test runner could upgrade its NUnit dependency accordingly. It seems to me that its absolutely critical for the platform to be able to effectively test one's async code.

  • mhutchmhutch Mikayla Hutchinson USXamarin Team Xamurai

    I'm not sure, there are multiple pieces in play here maintained/shipped by different teams - the test framework and test runners bundled with Xamarin.iOS, Xamarin.Android and Xamarin Studio - and I don't know much about the details of any of them. I'd suggest you file enhancement bugs against the relevant products.

    Note that you can still test async code without the test runner supporting async test methods. You just have your test methods call the actual async methods and Wait() on the result.

  • KentBoogaartKentBoogaart Kent Boogaart AUMember ✭✭

    Actually, I tried calling Wait() and that blocks too. Thanks - I'll see if I can get the appropriate people involved in solving this.

  • KentBoogaartKentBoogaart Kent Boogaart AUMember ✭✭

    FWIW, I ended up hacking the runner that comes with Xamarin such that it supports async from the base up. Now everything works perfectly. I'd be happy to coordinate a fix if I could figure out who exactly to correspond with...but then I'm not sure there's much point given all the work going into NUnit 3.

  • mhutchmhutch Mikayla Hutchinson USXamarin Team Xamurai

    Which test runner is it exactly, and what was the fix?

  • KentBoogaartKentBoogaart Kent Boogaart AUMember ✭✭
    edited March 2014

    It's Touch.Unit (i.e. what you get via Add/New Project/C#/iOS/iOS Unit Tests Project). The problem was that its copy of NUnitLite was calling Wait() on the task returned by the test. That, of course, is a big no-no and was the underlying cause of the deadlocking. So the fix was to change the interface for AsyncInvocationRegion so that it is async:

    public abstract Task<object> WaitForPendingOperationsToComplete(object invocationResult);
    

    And, of course, everything flows from that. All code must be changed that calls this so that it is also async, all the way up to the UI. It took maybe 45 mins to change it over.

  • mhutchmhutch Mikayla Hutchinson USXamarin Team Xamurai

    Ah, it's probably best to make a pull request then.

  • ZakiZaki Zaki Merchant USMember

    @KentBoogaart, do you have your changes accessible anywhere?

    I'm currently trying to make this same fix, however my changes didn't lead me to the AsyncInvocationRegion. I instead end up awaiting the reflected method in TestMethodCommand as shown below and not requiring AsyncInvocationRegion all together. This as you mentioned flows back the chain to the UI with async calls.

    await (Task) Reflect.InvokeMethod(testMethod.Method, context.TestObject, arguments);

    This seems to work for some async tests, however for a lot of tests I get the error 'The operation couldn't be completed. (NSURLErrorDomain error -1005)', so I'm obviously doing something wrong. Any pointers would be really helpful.

  • KentBoogaartKentBoogaart Kent Boogaart AUMember ✭✭

    @Zaki: you can find my changes (as a single commit) here.

    However, I'd recommend ditching NUnit altogether and moving to XUnit instead. It supports async properly and is generally much easier to get up and running with. Use the latest pre-release bits on NuGet and see their docs for how to get set up.

  • ZakiZaki Zaki Merchant USMember

    @KentBoogaart : Thanks for that. Sadly I'm still getting the same error, so I guess its somehow not sitting well with the native rest client we've implemented using NSURLSession.

    I've looked at XUnit before, I'm not sure it has runners for all platforms though. Will give it another stab.

  • KentBoogaartKentBoogaart Kent Boogaart AUMember ✭✭

    @Zaki: yeah, check out the xamarin.xunit project (available via NuGet).

  • ZakiZaki Zaki Merchant USMember

    @KentBoogaart : does work well for iOS and Android, there isn't a mono console runner though. I've gone full circle now.

Sign In or Register to comment.