Random crash with SIGABRT

ovi_

Hi!
We get next random crashes from our users

Crashed Thread:        30  Thread Pool Worker

Exception Type:        EXC_BAD_ACCESS (SIGABRT)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000020
Exception Note:        EXC_CORPSE_NOTIFY

VM Regions Near 0x20:
--> 
    __TEXT                 0000000106039000-0000000106515000 [ 4976K] r-x/r-x SM=COW  /Applications/Workplace.app/Contents/MacOS/Workplace

Application Specific Information:
abort() called
...
Thread 30 Crashed:: Thread Pool Worker
0   libsystem_kernel.dylib          0x00007fff72b6d33a __pthread_kill + 10
1   libsystem_pthread.dylib         0x00007fff72c29e60 pthread_kill + 430
2   libsystem_c.dylib               0x00007fff72af4808 abort + 120
3   com.os33                        0x00000001063d7d0c altstack_handle_and_restore.cold.1 + 44
4   com.os33                        0x00000001060c6b9e altstack_handle_and_restore + 206
5   libobjc.A.dylib                 0x00007fff71873678 objc_release + 24
6   libobjc.A.dylib                 0x00007fff71873dba objc_autoreleasePoolPop + 175
7   com.apple.CoreFoundation        0x00007fff38b81b15 _CFAutoreleasePoolPop + 22
8   com.apple.Foundation            0x00007fff3b22d73e -[NSAutoreleasePool drain] + 126
9   com.os33                        0x0000000106065ced xamarin_thread_finish(void*) + 141
10  com.os33                        0x0000000106065a9c thread_end(_MonoProfiler*, unsigned long) + 28
11  com.os33                        0x00000001062aa6d5 mono_profiler_raise_thread_stopped + 53
12  com.os33                        0x00000001062e60bb mono_thread_detach_internal + 1355
13  com.os33                        0x00000001062ee43b start_wrapper_internal + 411
14  com.os33                        0x00000001062ee27e start_wrapper + 62
15  libsystem_pthread.dylib         0x00007fff72c2a109 _pthread_start + 148
16  libsystem_pthread.dylib         0x00007fff72c25b8b thread_start + 15

It's started to reproduce when we started build our Xamarin.Mac application in macOS Catalina with next environment:

=== Visual Studio Community 2019 for Mac ===

Version 8.6.4 (build 14)
Installation UUID: 1a985d9f-94a4-4de8-967f-3174cb0eb3dd
    GTK+ 2.24.23 (Raleigh theme)
    Xamarin.Mac 6.18.0.23 (d16-6 / 088c73638)

    Package version: 610000104

=== Mono Framework MDK ===

Runtime:
    Mono 6.10.0.104 (2019-12/5d03a6fe116) (64-bit)
    Package version: 610000104

=== Roslyn (Language Service) ===

3.6.0-3.20210.9+4eafdcb1bcbd8d3573f2ba6065e56d9b9ce4f8a3

=== NuGet ===

Version: 5.6.0.6591

=== Apple Developer Tools ===

Xcode 11.5 (16139)
Build 11E608c

=== Xamarin.Mac ===

Version: 6.18.2.1 (Visual Studio Community)
Hash: 29c4ea731
Branch: d16-6
Build date: 2020-05-26 17:03:04-0400

=== Operating System ===

Mac OS X 10.15.5
Darwin 19.5.0 Darwin Kernel Version 19.5.0
    Tue May 26 20:41:44 PDT 2020
    root:xnu-6153.121.2~2/RELEASE_X86_64 x86_64

Before that we builded on Mojave and everything was fine:

  === Visual Studio Community 2017 for Mac ===

Version 7.8.4 (build 1)
Installation UUID: ae120131-5d3b-4a43-a420-afba336c5e7c
    GTK+ 2.24.23 (Raleigh theme)
    Xamarin.Mac 5.0.0.0 ( / b40230c0)

    Package version: 518010003

=== Mono Framework MDK ===

Runtime:
    Mono 5.18.1.3 (2018-08/fdb26b0a445) (64-bit)
    Package version: 518010003

=== NuGet ===

Version: 4.8.2.5835

=== Apple Developer Tools ===

Xcode 10.1 (14460.46)
Build 10B61

=== Xamarin.Mac ===

Version: 5.8.0.0 (Visual Studio Community)
Hash: 0aa84521
Branch: d16-0
Build date: 2019-04-02 16:01:19-0400

=== Operating System ===

Mac OS X 10.14.6
Darwin 18.7.0 Darwin Kernel Version 18.7.0
    Tue Aug 20 16:57:14 PDT 2019
    root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64

I've tried some workarounds like

new System.Threading.Thread (() => 
{
  while (true) {
    System.Threading.Thread.Sleep (100);
    GC.Collect ();
  }
}).Start ();

to speed up crash reproducing but i couldn't yet.

AOT is enabled.
Can anyone help me? Maybe someone could point me a direction to look on?

Thanks in advance!

ChrisHamons

That's a tricky one. A full stack trace (all threads) might be helpful to look at.

Can you reproduce this? You could try changing your application to print the thread ID of tasks as they start to work, to try to pin down what part of your application is hitting this.

I'm surprised to see mono_profiler_raise_thread_stopped, do you have the profiler enabled / shipping? What build settings do you have enabled?

ovi_

I attached full crash file.
No, haven't reproduced yet.

Build settings:

We use next code to catch unhandled exceptions
AppDomain.CurrentDomain.UnhandledException += HandleException;
TaskScheduler.UnobservedTaskException += OnUnobservedTaskException;

In theory, it should prevent crashes. Or no?
And periodically i catch exception, write logs and my app is closed

ChrisHamons

So UnhandledException/UnobservedTaskException only gives you a first chance to look through MANAGED (c#) exceptions. Any crashes in:

• Apple API (AppKit, Foundation, etc)
• Runtime (Mono),
• Any other 3rd party native libraries

will not show up as a managed exception.

ChrisHamons

So the crash report has:

• Thread 0 (view layout causes convertPoint which calls a managed object, where we try to alloc a string, which causes the garbage collector to run (to get more memory).
• Thread 1 (Sgen), 2 (Finalizer) through 29 looks fine. Looks like you have a network connection, chrome (webview?), and Audio threads going.
• Thread 29 has a bunch of app specific networking API Code running.
• Thread 34 has some XPC code, called from C#, with something that is talking about protocol witness (Swift?!) in stack

It looks like there is some managed object that either is corrupted was over released that crashes later. The GC.Collect trick is a good one, I would have assumed it would have helped.