Assembly.Load - is it possible?

StuartLodgeStuartLodge Stuart LodgeUSBeta ✭✭✭
edited March 2013 in Xamarin.iOS

This is a follow-up to last September's post http://stackoverflow.com/questions/12401588/can-you-use-assembly-load-in-monotouch


In MvvmCross I load plugins - which are basically just a formal way of doing IoC

These plugins are compiled and linked statically in the app - they are not downloaded at runtime.

Ideally I'd like to get references to the code inside these plugins using Assembly.Load - but I just can't get it to work.

I've tried:

  • Cirrious.MvvmCross.Plugins.Location.Touch (this version works on WP, WPF and RT)
  • Cirrious.MvvmCross.Plugins.Location.Touch.dll (this version works on Droid)
  • Cirrious.MvvmCross.Plugins.Location.Touch, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null

But I always get FileNotFoundException back.


If I reference a type in the DLL first (in normal code) so that the assembly is already loaded then the Assembly.Load statement succeeds - but otherwise no luck.


Am I just doomed to failure on this? Or is there some magic combination of assemblyname, filename and/or project setting that can make it work?

Stuart

Posts

  • HuGuangYuHuGuangYu Hu Guang Yu CNBeta

    I think the Xamarin Team just disabled this kind of work in Monotouch compiler to follow Apple's rule.

  • StuartLodgeStuartLodge Stuart Lodge USBeta ✭✭✭

    Thanks @Simon

    But just to be clear, this isn't about dynamically loading new modules - this is about loading assemblies that ship within the package - which should work according to @poupou in that StackOverflow answer (Unless I'm wrong, I think @poupou is a Xamarin coding genius - sometimes also known as @spouliot ?)

  • HuGuangYuHuGuangYu Hu Guang Yu CNBeta

    I think it's possible for monodroid, but not monotouch.

  • NicWiseNicWise Nic Wise NZInsider, University mod

    @stuartlodge

    I think you have to link it at compile/build/AOT time. You can't load an assembly by name, because everything is loaded - or if not, it can't BE loaded. I'd expect Assembly.Load to fail all the time - I'm not sure why it works even sometimes!

    MonoDroid can do it 'cos you are running a full mono framework with JITer, I think. Monotouch is NOT - there is no JIT, if it's not compiled into your app, it doesn't exist. I suspect the "use it first" thing registers that your assembly is in memory now (Even tho, technically, it was before) so Assembly.Load matches something it has registered, and doesn't check the disk. Same as if you did Assembly.Load twice in a normal framework - the second time would be "oh, I have it already, here you go"

    And yes @poupou is Sebastian, who, along with @rolf, is one of the main X.iOS developers (not sure if there is more - others do bits, eg @jstedfast - but I think those 2 are the 2 core guys. But then add in @kumpera, Marek Safar, Zoltan, Mark.... who tend to do the more low level "we put this in register X so we don't get a cache miss", GC, etc type stuff)

    I'm also sure I've missed more than one someone. If so, SORRY!

    @jonp and @eno are the same for X.Android.

  • StuartLodgeStuartLodge Stuart Lodge USBeta ✭✭✭

    Thanks @nicwise who is also @fastchicken - top work in making sure everyone gets this flagged in their inbox ;)

    I'm with you on 'I'm not sure why it works even sometimes!' - that's why I gave up on it back last September... However, recent comments from @dsplaisted have given me new ideas and hope... so I thought I'd check...

    Stuart

  • NicWiseNicWise Nic Wise NZInsider, University mod

    @stuartlodge FLAG EVERYONE IN THEIR INBOXES!

  • SebastienPouliotSebastienPouliot Sebastien Pouliot CAXamarin Team Xamurai

    @StuartLodge This has not changed recently. IOW you cannot load new (as in not AOT compiled) assemblies.

    Now Assembly.Load* methods (there's many and they all differs a bit) don't really load assemblies if they were already loaded (e.g. by the runtime).

    That's why it will works (Load will return non-null) if you had used a type from that assembly earlier. The runtime load the assembly structures and that will make the Assembly available when Load* is called. E.g.

    string filename = Path.GetFileName (GetType ().Assembly.Location);
    Assembly assembly = Assembly.LoadFile (filename);
    

    There must be a type reference between you main .exe and the .dll you want, right ? otherwise the assembly loader would not AOT (and copy) those assemblies inside the final application. Is there anything you known from the assemblies that could be used to jumpstart the runtime into loading them ? (before you call Assembly.Load)

    p.s. @poupou (forum, stackoverflow) is the same as @spouliot (twitter) and sebastien@x.com - OTOH I have no papers naming me a genius (maybe I should print this one ;-)

  • StuartLodgeStuartLodge Stuart Lodge USBeta ✭✭✭

    Thanks PouPou

    you cannot load new (as in not AOT compiled) assemblies.

    If I reference an assembly but never use it - and I have Linker set to SDK only - then does it not get AOT compiled?

    Is there anything you known from the assemblies that could be used to jumpstart the runtime into loading them ?

    Kind of - there is a known registration interface (which is what I get just after the assembly.load is called) - but other than that it only contain objects that get referenced by interface.

    This is for plugins - interface-driven components - where the basic idea is that there's a central PCL assembly which contains the interface and then separate assemblies containing the platform specific implementations (the 'real code').

    -

    I've no worries if this doesn't work - my workaround is to jumpstart them just like you suggest - I use a registration pattern - like:

        protected override void AddPluginsLoaders(MvxLoaderPluginRegistry registry)
        {
            registry.AddConventionalPlugin<Cirrious.MvvmCross.Plugins.Visibility.Touch.Plugin>();
            registry.AddConventionalPlugin<Cirrious.MvvmCross.Plugins.File.Touch.Plugin>();
            registry.AddConventionalPlugin<Cirrious.MvvmCross.Plugins.DownloadCache.Touch.Plugin>();
            base.AddPluginsLoaders(registry);
        }
    

    However, since I was changing this area in WP and WinRT, I thought I'd check up on this old xamios question too.

    @dsplaisted and I are talking (arguing nicely) about this area at the moment - we're trying to work out if there is a way that all PCLs+extension methods could be built in a similar way - if they can, then tools like a multi-platform-nuget could be built around them.

    Thanks for the info

    Stuart
    (wondering about making an app that provides proof-of-genius-certification)

  • SebastienPouliotSebastienPouliot Sebastien Pouliot CAXamarin Team Xamurai

    If I reference an assembly but never use it - and I have Linker set to SDK only - then does it not get AOT compiled?

    No, that's how the .NET compilers works, i.e. it's unrelated to the linker, AOT compiler or even Xamarin.iOS itself.

    E.g. the C# compiler will only include assembly references if they are used. IOW you can add -r: for every FX assemblies but if you only use mscorlib.dll then your .exe will only reference this single assembly.

    After compilation XS/MD will give to mtouch an initial list of assemblies (you can see it in the Build Log).

    mtouch will recursively analyze the .exe (and other supplied .dll) to find every assemblies that can be needed (that's even before the linker gets called).

    Later the linker might find out some assemblies are not needed (i.e. the code that referencing them was removed) and remove them from the final application.

    What's left will be AOT'ed and copied (stripped first for release builds) inside your application.

    In doubt you can check inside your .app directory so see which .dll made it out to the end :-)

  • StuartLodgeStuartLodge Stuart Lodge USBeta ✭✭✭
    edited March 2013

    While I'm distracting you anyway... and so I don't ask this again in another 6 months :)

    No, that's how the .NET compilers works, i.e. it's unrelated to the linker, AOT compiler or even Xamarin.iOS itself.

    What do you mean by '.NET compilers' here?

    As far I understand it from practical hacking... especially while investigating this question

    • if I reference a project inside one of the Studio Twins, but don't use any types from within the source project
    • then the .Net compiler will compile that referenced project
    • and the MSBuild/XBuild Twins will copy that assembly to the output folder
    • but when the .Net compiler compiles the referencing project then the output IL doesn't have any reference information in the '.assembly extern' section.

    But then I'm not sure which bit of the toolchain happens next?

    Obviously the 'packagers' for WinRT, WP, and XamDroid all choose to package the semi-referenced Assembly (Assembly.Load works in all those platforms). Also, PCL-Daniel seems really convinced that whatever MS's magic AoT cloud compiler is for WP will process these assemblies.

    ... but it seems XamTouch is different? And I guessed that's because the AOT compiler is only being run against the reference list (and ancestor reference list) from the 'main' compiled IL assembly?

    Is that roughly right?

    Stuart

  • SebastienPouliotSebastienPouliot Sebastien Pouliot CAXamarin Team Xamurai

    What do you mean by '.NET compilers' here?

    I mean .NET compilers (not IDE) and assembly references (not project references). E.g.

        $ cat hello.cs
        using System;
    
       class Program {
        static void Main ()
        {
            Console.WriteLine ("Hello");
        }
       }
       $ mcs hello.cs -r:System.dll
       $ monodis hello.exe
       .assembly extern mscorlib
       {
         .ver 4:0:0:0
         .publickeytoken = (B7 7A 5C 56 19 34 E0 89 ) // .z\V.4..
       }
       .assembly 'hello'
       {
       ...
    

    So asking a compiler to add a reference (e.g. to System.dll) won't automatically include it in the final binary (e.g. only mscorlib is referenced by hello.exe) unless it's required.

  • StuartLodgeStuartLodge Stuart Lodge USBeta ✭✭✭

    Thanks

    I still think this is a packaging/AOT difference rather than a compiler one - as the other 'packagers' and AOT compilers (e.g. WinPhone in the Marketplace) all claim to support these additional files.

    But doesn't matter... I'll keep using the code based registration for Touch and Mac - and will use the file based loading on the others.

    Thanks again :)

    Stuart

  • KirkKlobeKirkKlobe Kirk Klobe USMember, Beta

    We are also resorting to this code-based registration technique. There really should be a feature added to overcome this limitation. Perhaps an mtouch flag to force inclusion of an assembly?

  • JimBordenJimBorden Jim Borden USMember

    Has there been any movement on this in the past two years? I'm also facing this exact problem. I'm working on Couchbase Lite for .NET (https://github.com/couchbase/couchbase-lite-net) and I am in the process of factoring our various storage modes into extensible packages. There are three packages at the moment, one for SQLite, one for SQLCipher and a third for ForestDB (our in house KV storage library). These packages depend on the core package, but are added onto a user application. The problem I am having is since the core package has no way of knowing about them (references would cause circular deps) it cannot reference them explicitly. The types are found and instantiated through reflection and assembly loading. The problem is that since nobody actually uses them they are not compiled in as a reference anywhere and not included in the final application. This happens at a level beyond our control (since it will be a different developer using our library and adding in the packages they desire).

    This is my workaround:

    I have three "bait" packages that have the same assembly name as the real ones but have nothing in them except a dummy class, and I have an inaccessible class in the core library that references these three bait classes but doesn't copy the bait classes to the output directory. This forces the references to be written into the core library and thus the linker will pick up the real ones if they are present in the final directory (because the names are the same). I don't like this workaround and would prefer if there were another way.

  • MelbourneDeveloperMelbourneDeveloper Christian Findlay AUMember ✭✭✭
    edited February 13

    I am very interested in this topic. We need to load the UI dynamically at runtime. Currently, we are using a hack to call a hidden method called LoadFromXaml. I am pushing to get this method made public. However, some people are saying that they'd prefer to use compiled Xaml rather than parsing Xaml at runtime. This is a fair call, but if assemblies cannot be loaded at runtime dynamically, then the point is moot.

    Here is my thread on dynamically loading Xaml.
    https://forums.xamarin.com/discussion/87810/please-make-xamlloader-public

    Basically, without the ability to load assemblies dynamically, and without the ability to dynamically parse Xaml at runtime, we are stuck. Why doesn't the Xamarin / Xamarin Forms team stick their neck out and tell us if/when these features are going to become available - if at all?

    @JimBorden - have you heard anything yet?

  • JimBordenJimBorden Jim Borden USMember

    @MelbourneDeveloper I have not heard anything. At least 90% of the stuff I post here goes unviewed and unanswered...

    I gave up on hoping to do this automatically in the end and resorted to a manual registration method where the user must call a dummy method inside the assembly to guarantee that they are not stripped out.

    However, if you can make at least one solid reference to the library in question then you can use Assembly.Load.

  • MelbourneDeveloperMelbourneDeveloper Christian Findlay AUMember ✭✭✭

    @JimBorden said:
    However, if you can make at least one solid reference to the library in question then you can use Assembly.Load.

    This is a problem for us. We have at least 10 different systems, and each system has its own data model and user interface. If we can't load these dynamically, we'd either have to:

    a) Ship all customizations to all customers, or
    b) Create a different app package for each customer

    A) is simply unfeasible because it would mean a redeployment of the app every time a simple change is made. B) is unfeasible because it would be a store maintenance nightmare. We would event up with 10 different apps in each of the stores.

  • JimBordenJimBorden Jim Borden USMember

    I see your point. At the end of the day what is needed to succeed is that the reference exists in the managed assembly (even if it is not used) so that the AOT compiler will not skip it. Or find a way to invoke mtouch manually so that you can AOT the assemblies yourself and include them.

    Good luck getting a response though...I've pretty much given up on this forum ever responding to in depth technical stuff like this...of the 14 threads I've posted only 3 have received any response at all. Perhaps actual bug reports are needed to get any attention.

  • MelbourneDeveloperMelbourneDeveloper Christian Findlay AUMember ✭✭✭
    edited February 14

    @JimBorden
    @StuartLodge

    Assemblies can be loaded dynamically on Android. Here is some sample code.

    I created this class and compiled it in to a DLL called DynamicTypes.dll in a PCL project:

    namespace DynamicTypes
    {
        public class TestType
        {
            public int TestInt { get; set; }
        }
    }
    

    In an Android project, I added the DLL as an Android Asset, and added this method to the MainActivity. It runs and works correctly. The last line of this code returns 1.

        private void LoadTypeDynamically()
        {
            var byteList = new List<byte>();
            var assets = Assets;
            var stream = assets.Open("DynamicTypes.dll");
    
            using (var binaryReader = new BinaryReader(stream))
            {
                //We should be using this property to get the length of the file but this errors on Android
                //var length = binaryReader.BaseStream.Length;
    
                //Set up a while loop
                while (true)
                {
                    try
                    {
                        //Read a byte
                        var theByte = binaryReader.ReadByte();
    
                        //Add the byte to the list
                        byteList.Add(theByte);
                    }
                    catch (Exception ex)
                    {
                        //We have come to the end of the file.
                        //Note: it would be much better if we could use the length, or if there were an EOF property, but these are not supported so it appears we need to rely on handling an exception
                        break;
                    }
                }
    
                //Convert the list to an array
                var assemblyBytes = byteList.ToArray();
    
                //Load the assembly in to the app domain
                var assembly = AppDomain.CurrentDomain.Load(assemblyBytes);
    
                //Get a list of the types in the assembly
                var types = assembly.GetTypes();
    
                //Get the first type in the assembly
                var firstType = types.FirstOrDefault();
    
                //Create an instance of the type (TestType in my case)
                var testType = Activator.CreateInstance(firstType);
    
                //Get the first property on the type (TestInt in my case)
                var testIntProperty = firstType.GetProperties().FirstOrDefault();
    
                //Set the TestInt property to 1
                testIntProperty.SetValue(testType, 1);
    
                //Get the value of TestInt
                var value = testIntProperty.GetValue(testType);
            }
    
        }
    

    I will try the same code on the other platforms. I suspect that this will not work in Windows UWP.

    Note: This still doesn't prove that DLLs can be loaded dynamically on Android. This DLL is compiled in to the AndroidAsset package, but if you look at the way I'm loading the DLL, I'm optimistic that I'd be able to do the same thing with a DLL say loaded from an external Url.

  • MelbourneDeveloperMelbourneDeveloper Christian Findlay AUMember ✭✭✭
    edited February 14

    I have slightly refactored the above code and integrated it in to a test project that I have been working on. Please clone this repo:

    https://ChristianFindlay@bitbucket.org/ChristianFindlay/xamarin-forms-scratch.git

    Feel free to check out the sample. It uses dependency injection for each platform to handle dynamic loading. So, far I've only implemented it on Android, but I will see if it is possible to implement it on other platforms.

    If you have implementations for other platforms, please submit a pull request.

  • MelbourneDeveloperMelbourneDeveloper Christian Findlay AUMember ✭✭✭

    I have implemented the same for iOS and proven that DLLs can be loaded dynamically. If you do a pull on the repo above, you can see the code.

  • MelbourneDeveloperMelbourneDeveloper Christian Findlay AUMember ✭✭✭
    edited February 14

    Unfortunately, UWP seems to be the major sticking point. I can't find any plans anywhere for Microsoft to add a feature which allows UWP to load DLLs dynamically. This is bizarre when this can be done on even iOS devices. Please take the time to upvote this feature where I request that DLLs be allowed to be loaded dynamically in UWP:

    https://wpdev.uservoice.com/forums/110705-universal-windows-platform/suggestions/18145291-dynamically-load-assembly

  • JimBordenJimBorden Jim Borden USMember

    That is really odd that you can do that it seems to me. Is that dll AOT compiled? Also doesn't that violate Apple's developer rules about running code not shipped with the app if you are using the web version?

  • JamesLaveryJamesLavery James Lavery GBBeta, University ✭✭✭
    This is interesting (to say the least). Have you established that assemblies which were not distributed as application assets (i.e. downloaded separately) can be loaded?
  • MelbourneDeveloperMelbourneDeveloper Christian Findlay AUMember ✭✭✭

    @JamesLavery said:
    This is interesting (to say the least). Have you established that assemblies which were not distributed as application assets (i.e. downloaded separately) can be loaded?

    Absolutely. The sample code has an example of how to load DLLs over the web. The code is commented out but you can see how it is done.

  • MelbourneDeveloperMelbourneDeveloper Christian Findlay AUMember ✭✭✭

    @JimBorden said:
    That is really odd that you can do that it seems to me. Is that dll AOT compiled?

    What do you mean by AOT compiled? It's just a PCL library compiled in Visual Studio.

    Also doesn't that violate Apple's developer rules about running code not shipped with the app if you are using the web version?

    This is the really thorny question...

    I have asked the same question here:
    https://forums.xamarin.com/discussion/89711/shell-app-deployment

    My thinking on this, and I could be wrong is that Apple and Google allow certain APIs to be ran. When you register your app with them, they will scan your app to ensure that you are not running an API that they don't like. If we deploy a simple shell app in to the store, we are just deploying the Xamarin framework. There are no APIs in the Xamarin framework that Apple, or Google don't like. When a .NET Standard, or PCL library is loaded in to the AppDomain from an external source, that doesn't change. The APIs used are internal to the Xamarin framework - not to Apple or Google. The app doesn't magically get new capabilities or anything like that. To put it another way, loading external .NET assemblies at runtime doesn't allow the app to call extra iOS/Android APIs. So, I figure it's really not breaking any of Apple or Google's policies. But, please correct me if I am wrong.

    PS: This discussion would be best continued here:
    https://forums.xamarin.com/discussion/89711/shell-app-deployment#latest

  • JimBordenJimBorden Jim Borden USMember

    What do you mean by AOT compiled? It's just a PCL library compiled in Visual Studio.

    iOS has no capability to JIT (Just-In-Time) compile managed assemblies (well technically it can but it is forbidden by Apple). As such, every used assembly must be compiled to native code first (AOT, or Ahead-Of-Time) or you will get exceptions about trying to JIT on an AOT only platform. This is handled for you when you deploy a Xamarin.iOS (or other AOT platform) project but as far as I know simply building an assembly will just compile the IL. I faced this problem in spectacular fashion when trying to battle with mtouch over which binaries it should actually consider (I was trying to do an automatic registration of a DLL that I used with Assembly.Load, but since it was not referenced in any way the .NET compiler removed the reference and mtouch skipped it).

    I'll take a look at the other thread for the other portion of this conversation.

  • MelbourneDeveloperMelbourneDeveloper Christian Findlay AUMember ✭✭✭

    So, I guess the conclusion to this is that the only platform where dynamic loading of assemblies is Android. You can load assemblies dynamically on the iOS simulator, but it will not work on iOS. I guess there's an open question as to whether Google will continue to allow this functionality in future.

  • JamesLaveryJamesLavery James Lavery GBBeta, University ✭✭✭

    @MelbourneDeveloper We need to be able to load (for want of a better name) plugins for our Android app, so now I'm looking with interest at your code.

    First of all, I think there's a good improvement which can be made to avoid the byte-reading code. If you copy the Asset to the file system, you can use Assembly.LoadFrom like this:

    var PluginAssemblyLocalFile = CopyAssemblyToFile ("AndroidPlugin.dll");
    var PluginAssembly = Assembly.LoadFrom (PluginAssemblyLocalFile);
    

    Where CopyAssemblyToFile looks like this:

    private string CopyAssemblyToFile (string AssemblyToCopy)
    {
        Stream myInput = Assets.Open (AssemblyToCopy);
        string OutputFilename = Path.Combine (System.Environment.GetFolderPath (System.Environment.SpecialFolder.MyDocuments), AssemblyToCopy);
        Stream myOutput = new FileStream (OutputFilename, FileMode.Truncate);
    
        //transfer bytes from the inputfile to the outputfile 
        byte [] buffer = new byte [1024];
        int b = buffer.Length;
        int length;
        while ((length = myInput.Read (buffer, 0, b)) > 0) {
            myOutput.Write (buffer, 0, length);
        }
        //Close the streams 
        myOutput.Flush ();
        myOutput.Close ();
        myInput.Close ();
    
        return OutputFilename;
    }
    

    Using this, I'm able to load a plugin assembly and call methods from it.

    Now my challenge is that my plugin carries out barcode scanning using hardware-specific assemblies - i.e. it has dependent assemblies. When I call the Scan method on my loaded assembly, I get an exception saying that the dependent assembly cannot be found (no real surprise there).

    Any thoughts as to how we can achieve this?

  • MelbourneDeveloperMelbourneDeveloper Christian Findlay AUMember ✭✭✭

    I see. I don't really see why you'd want to store the assembly in the Assets. In my case, I was hoping to download the assembly over the web. But, it seems you've got some requirement to not load the bytes in to memory. Is that the case?

    I'm only making a wild guess, but perhaps when the assemblies were originally compiled, you specified a different assembly to the one being loaded? I mean, perhaps you linked the assembly to a given assembly, and then later changed the name slightly. Perhaps it has a different assembly Guid or something? I probably recommend removing the assembly Guid if you have one so that the linker isn't expecting a particular Guid.

    I guess, loading the assemblies in the right order is important as well. In extreme cases, if you are dealing with a lot of assemblies, you may need to create a method to calculate dependency ordering. You'd need some kind of bubble sort to do that.

  • JamesLaveryJamesLavery James Lavery GBBeta, University ✭✭✭
    Actually I'm only using assets as an easy way for now to get the assembly distributed wot the app for testing.

    The good news is that we have decided not to go down the dynamic load route, and just build with all assemblies in the app.
Sign In or Register to comment.