"Link all assemblies" strips my static library symbols for release builds

JoeFriedchickenJoeFriedchicken USMember
edited January 2014 in Xamarin.iOS

I have an iOS app that uses a few C++ static libraries. I use the -force_load option to ensure that my C# interop calls can find and use the static library entry points.

It seems like Xamarin is a bit overzealous when it goes to link-all for release builds. When I set the Linker behavior to Link all assemblies, my release builds cannot call into my static libraries while my debug builds can.

I added a few -v flags to the Additional mtouch arguments field and saw this line after the final link for release:

/usr/bin/strip -i -s "/path/to/my/app/obj/iPhone/release/mtouch-cache/Build/symbol-file" "/path/to/my/output/directory/MyApp.app/MyApp"

When I use nm -m on the resulting binary, I don't see my static library entry points for my interop calls.

Looking at the symbol-file, I see a few mono names (eg, _monotouch_release_managed_ref, _monotouch_create_managed_ref, and friends), and it seems like if a library needs to be force-loaded, its symbols should be in this file so that they're not stripped immediately after.

How can I keep the static library symbols that I need to force load?

  • Do they need to be in the symbol-file? What do I do to ensure that happens?
  • Should Xamarin inspect force_load libraries and add their external symbols to the symbol-file?

Posts

  • SebastienPouliotSebastienPouliot CAXamarin Team Xamurai

    Using "Link all" (versus the default "Link SDK") has no effect on native libraries. Both options are also identical in release and debug modes (except for the removal of the UI thread check) and only applies to managed code. IOW you're likely having the same issue if you switch back to the default Link SDK.

    OTOH the strip step is only done on the release builds (in part because of the extra time it takes). You can test if this is the issue by disabling it using the extra --nostrip argument in your project's options.

  • JoeFriedchickenJoeFriedchicken USMember
    edited January 2014

    Thanks for your response :-) I agree with the goal of your claim about Link all, that it should not have an effect on native libraries, but it appears you're wearing kernel colored glasses.

    Using "Link all" (versus the default "Link SDK") has no effect on native libraries. Both options are also identical in release and debug modes (except for the removal of the UI thread check) and only applies to managed code. IOW you're likely having the same issue if you switch back to the default Link SDK.

    If I use Don't link, then strip is never called, even for release builds. Link all and Link SDK however, do call strip, which has a disabling impact on the final release binary, even if it wasn't intended to.

    OTOH the strip step is only done on the release builds (in part because of the extra time it takes). You can test if this is the issue by disabling it using the extra --nostrip argument in your project's options.

    I do not observe this behavior in 7.0.4 or 7.0.6 (currently in the alpha channel) -- strip is called only for Link all and Link SDK release builds, but not for Don't link. From your description of Xamarin's intended behavior, it appears that this is either misunderstood and under-documented or a bug.

    After doing a Link all release build, I took a closer look at the symbol-file, and I saw a small subset of my static library's entry points. Somehow, Xamarin is able to determine that these are worth preserving, and I would like to know what I can do to get the rest of those symbols kept. Using the --nostrip option is undesirable because the binary size is much larger: 144 MB unstripped, 57 MB stripped.

  • SebastienPouliotSebastienPouliot CAXamarin Team Xamurai

    Heh, don't we all wear glasses ;-)

    Now I can only answer questions based on the facts written. Defining the problem correctly is key to solving them. Your premise was incorrect - it was misleading you (or anyone reading it) to search for a solution at the wrong place and I wanted to correct this. What's actually happening is:

    > Release builds strips my library symbols

    Quoting my answer (which was "Link all" and "Link SDK" are identical) to assert that "Don't link" is different does not make my answer wrong. But this is still looking into the wrong direction to find your solution.

    To make the whole thread consistent (even if only thru my glasses ;-) I'll explain why (even if it's not related to your issue):

    Don't link is special and generally never used for release since the final executable sizes are (for most applications) too large and Apple will refuse them in AppStore. OTOH it's very useful if you have issues with a Link (all/SDK) build (for comparison purposes).

    We do no run strip for Don't Link because we did not process/analyze the code in this case (so we can't build a symbol-file). Also anyone using this option should not be worried about application size (it will be huge since the full framework will be included) so stripping is not very important.

    Note: I agree this is undocumented and I'll look at rectifying this (likely add a build warning in such case).

    Using the --nostrip option is undesirable because the binary size is much larger: 144 MB unstripped, 57 MB stripped.

    Yes, using --nostrip can result in larger binaries. However, if such a build works, it confirms where and what the issue is.

    I took a closer look at the symbol-file, and I saw a small subset of my static library's entry points.

    Yes, the tools can detect some symbols (the static ones) that managed code requires (e.g. p/invokes) and ensure that the native (not managed) linker is aware of them. However we cannot detect dynamically used symbols (e.g. dlsym) or any dynamic requirements inside your native libraries. If you have such, extra, requirements you need to provide your own list (or disable stripping).

    That can be done using the [LinkWith] attribute or giving extra instructions to your build, i.e. --nosymbolstrip[=VALUE,...].

  • adamkempadamkemp USInsider, Developer Group Leader mod

    Don't link is special and generally never used for release since the final executable sizes are (for most applications) too large and Apple will refuse them in AppStore. OTOH it's very useful if you have issues with a Link (all/SDK) build (for comparison purposes).

    Can you elaborate on that? I've never heard of Apple rejecting an app for being too large. Also, one of our previous apps used "Don't Link" (out of necessity; the feature was broken at the time, and we could never get a working app out of it), but our app was still under 50MB. I just want to understand what possible consequences we might face if we have to resort to "Don't Link" for this app as well.

  • DaveHuntDaveHunt USMember ✭✭✭✭✭

    As of June 2013, the total app size limit is 2GB, however the actual executable file (app_name.app/app_name) cannot exceed 60MB.

  • SebastienPouliotSebastienPouliot CAXamarin Team Xamurai

    @AdamKemp like @DaveHunt said the native executable limit is 60MB. At least that was the original limit (when first enforced by Apple in February 2013). Since then someone told me it was bumped to 80MB (but I've not confirmed this). YMMV

    The final size (using Don't link) greatly depends on the number of BCL assemblies being referenced (we still build/ship only the assemblies you need, not the full framework).

    Even then the most common case I've seen was not related to Don't link but when IKVM was used. The IKVM assemblies are not linked (by default, since they are user code) but they contains the whole java classpath - plus all (or part of) the BCL and the application code.

    Note: If one of your application still requires this then I encourage you to file a bug report.

  • Heh, don't we all wear glasses ;-)

    Yes! I am, too. Here is my lens:

    As a Xamarin user, I should be able to incorporate static libraries into my app and also be able to Link all assemblies for both debug and release.

    Right now, this is broken.

    Yes, the tools can detect some symbols (the static ones) that managed code requires (e.g. p/invokes) and ensure that the native (not managed) linker is aware of them. However we cannot detect dynamically used symbols (e.g. dlsym) or any dynamic requirements inside your native libraries. If you have such, extra, requirements you need to provide your own list (or disable stripping).

    I use P/Invoke to call into my static libraries, but Xamarin detects less than 5% of the calls. Why would it miss so many DllImport attributes -- how can I make them more visible?

    That can be done using the [LinkWith] attribute or giving extra instructions to your build, i.e. --nosymbolstrip[=VALUE,...].

    I can do this, but I feel it's Xamarin's responsibility.

  • Yes, the tools can detect some symbols (the static ones) that managed code requires (e.g. p/invokes) and ensure that the native (not managed) linker is aware of them. However we cannot detect dynamically used symbols (e.g. dlsym) or any dynamic requirements inside your native libraries. If you have such, extra, requirements you need to provide your own list (or disable stripping).

    I use P/Invoke to call into my static libraries, but Xamarin detects less than 5% of the calls. Why would it miss so many DllImport attributes -- how can I make them more visible?

    Xamarin appears to look only two classes deep for DllImport attributes.

    In the pseudo code below, the symbol MyEntryPoint will be found but the symbol MyNestedEntryPoint will not be found.

    namespace MyNamespace
    {
       public sealed class StaticLibraryWrapper
       {
          internal sealed class NativeMethods
          {
             private NativeMethods() { }
    
             [DllImport(ImportName.DllName, EntryPoint="MyEntryPoint", CallingConvention=CallingConvention.Cdecl)]
             public static extern Int32 MyEntryPoint();
          }
    
          public static int MyEntryPoint()
          {
             status = NativeMethods.MyEntryPoint()
             return status;
          }
    
          public sealed class Nested
          {
             internal sealed class NestedNativeMethods
             {
                private NestedNativeMethods() { }
    
                [DllImport(ImportName.DllName, EntryPoint="MyNestedEntryPoint", CallingConvention=CallingConvention.Cdecl)]
                public static extern Int32 MyNestedEntryPoint();
             }
    
             public static int MyEntryPoint()
             {
                status = NestedNativeMethods.MyNestedEntryPoint()
                return status;
             }
          }
       }
    }
    
  • SebastienPouliotSebastienPouliot CAXamarin Team Xamurai

    That sounds like a DllImport scanning bug (not being recursive on nested types). I'll look into it.

  • SebastienPouliotSebastienPouliot CAXamarin Team Xamurai

    Yes, that's the issue. I filed a bug for this and it will be fixed in future versions. Thanks for the test case.

    The workaround, as you can guess, is to limit [DllImport] declarations to a type or a nested type (but not deeper that that).

  • Excellent :-D Thanks for your diligence.

Sign In or Register to comment.