When in the build process should I obfuscate and where do I copy the results?

MichaelRumplerMichaelRumpler ATMember ✭✭✭✭✭

I need to obfuscate some of my dlls before creating the .apk. I already had this running for a few years, but it seems like
the build process changed lately and it does not work anymore.

I have a target in my .csproj for the obfuscation:

<Target Name="Obfuscate" AfterTargets="_CopyIntermediateAssemblies" Condition="'$(Configuration)' == 'Release'">
  <Exec Command="$(MSBuildProjectDirectory)\Obfuscate.bat $(MSBuildProjectDirectory)"
        WorkingDirectory="$(MonoAndroidLinkerInputDir)" />
</Target>

In the past the obfuscator worked in the bin\Release folder, but that folder doesn't contain all dependencies anymore.
After the obfuscation I used to copy the files to obj\Release\linksrc in order to get picked up by the linker.
I saw that obj\Release\90\linksrc does contain all the .dlls, so I changed the obfuscation to work in that folder instead
and copy the obfuscated files to bin\Release.

Now the obfuscation works, but the linker fails.

The "LinkAssemblies" task failed unexpectedly.\r
Mono.Linker.MarkException: Error processing method: 'System.Threading.Tasks.Task`1<System.Boolean> RoyalMobileApps.XF.Helpers.LauncherBase::LaunchConnection(RoyalDocumentLibrary.RoyalConnection,Xamarin.Forms.ToolbarItem,System.Collections.Generic.Dictionary`2<System.String,System.String>)' in assembly: 'RoyalMobileApps.XF.dll'
    ---> Mono.Cecil.ResolutionException: Failed to resolve RoyalMobileApps.XF.Helpers.LauncherBase/<LaunchConnection>d__1\r

I had obfuscation running before the linker in the past, but apparently this is a problem now, so I tried to start it AfterTargets="_LinkAssemblies".

Now both obfuscation and linking works, but the apk contains the unobfuscated files.

I added to my Obfuscation.bat that the obfuscated files will be copied to obj\Release\90\android\assets where I also found all dlls.
Again obfuscation and linking works, but I get an error in GenerateJavaStubs:

The "GenerateJavaStubs" task failed unexpectedly.\r
System.IO.FileNotFoundException: Could not load assembly 'netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. Perhaps it doesn't exist in the Mono for Android profile?\r
File name: 'netstandard.dll'\r
   at Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.Resolve(AssemblyNameReference reference, ReaderParameters parameters)\r
   at Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.Resolve(AssemblyNameReference reference)\r
   at Mono.Cecil.MetadataResolver.Resolve(TypeReference type)\r
   at Mono.Cecil.ModuleDefinition.Resolve(TypeReference type)\r
   at Mono.Cecil.TypeReference.Resolve()\r
   at Java.Interop.Tools.Cecil.TypeDefinitionRocks.<GetTypeAndBaseTypes>d__1.MoveNext()\r
   at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source, Func`2 predicate)\r
   at Java.Interop.Tools.Cecil.TypeDefinitionRocks.IsSubclassOf(TypeDefinition type, String typeName)\r
   at Java.Interop.Tools.JavaCallableWrappers.JavaTypeScanner.AddJavaTypes(List`1 javaTypes, TypeDefinition type)\r
   at Java.Interop.Tools.JavaCallableWrappers.JavaTypeScanner.GetJavaTypes(IEnumerable`1 assemblies, IAssemblyResolver resolver)\r
   at Java.Interop.Tools.JavaCallableWrappers.TypeNameMapGenerator..ctor(IEnumerable`1 assemblies, Action`2 logger)\r
   at Java.Interop.Tools.JavaCallableWrappers.TypeNameMapGenerator..ctor(IEnumerable`1 assemblies, Action`2 logMessage)\r
   at Xamarin.Android.Tasks.GenerateJavaStubs.WriteTypeMappings(List`1 types)\r
   at Xamarin.Android.Tasks.GenerateJavaStubs.Run(DirectoryAssemblyResolver res)\r
   at Xamarin.Android.Tasks.GenerateJavaStubs.Execute()\r
   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()\r
   at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()

One last try was to start obfuscation AfterTargets="_CopyIntermediateAssemblies" again and copy the files to obj\Release\90\android\assets too.
Unfortunately the linker failed with the same error I got above without copying the files to that folder.

So when should I run the obfuscator and where should it place the files?

Best Answer

  • MichaelRumplerMichaelRumpler AT ✭✭✭✭✭
    edited March 20 Accepted Answer

    It seems like ConfuserEx does not work (with .NET Standard?) anymore. When I tried with Babel, I copied the obfuscated dlls to the very same folders like I did with ConfuserEx and it worked out of the box. So I'm pretty sure that my obfuscation task is not the issue.

    The most recent ConfuserEx fork I found was ConfuserEx_Reborn from June 2017. But this fork also doesn't work for me. There is an open issue about updating dnlib. Dnlib is the component which reads/writes the dlls. I guess that would fix it (dnlib itself is still maintained). Unfortunately dnlib had some breaking changes between versions 2 and 3. As I know nothing about neither ConfuserEx nor dnlib's inner workings I couldn't get it running.

    I also tried Dotfuscator. The Community edition is free, but it may not be used for commercial apps and it does not do control flow obfuscation which makes it completely useless.
    I told them that I make around €40/month with my app and their response was, that I would probably qualify for a Entrepreneur license of Dotfuscator Professional which would cost €2,490/year. Guess what they can do with this offer.

    Then I looked at Babel for .NET. Don't judge their web site! It is really old and still uses http://, but their stuff works. They have a blog article from 2014 about Obfuscating Xamarin for Android. It is 5 years old, but it still works. And unlike their web site and documentation the program itself is still updated. The current version is 9.1.1.1 from January 31st, 2019. Their email support is also very fast. I got answers within hours.

    What I really like about Babel is, that you can merge multiple dlls into one and automatically make all types from the merged dlls internal and thus eligible for renaming. This feature does require an Enterprise license though. But prizes are also not from out of this world. A Standard license costs €115 and Enterprise €245. You pay that once and can use the current version forever (updates for one year also included).

    So I settled for a Babel Enterprise license.

    Although this all sounds like an advertisement I am not associated with Babel in any way. They didn't ask me to write this and don't pay me for it. I'm just a satisfied customer.

    /cc @OliverM @YorkGo

Answers

  • YorkGoYorkGo CNMember, Xamarin Team Xamurai

    The "LinkAssemblies" task failed unexpectedly.
    Mono.Linker.MarkException: Error processing method: ...

    Try using Custom Linker Configuration for your project. Create a myLink.xml in your project, set the Build Action to LinkDescription, add the following code to the myLink.xml file:

    <linker>
      <assembly fullname="RoyalMobileApps.XF">
        <type fullname="RoyalMobileApps.XF.Helpers.*" />
      </assembly>
    </linker>
    
  • MichaelRumplerMichaelRumpler ATMember ✭✭✭✭✭
    edited January 11

    Thanks for your answer, @YorkGo

    I link Sdk Assmblies Only anyway.
    However, I did configure the linker as you said. I do see in the build log, that the file is picked up, but I still get an error message for a type in that namespace.

    C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(2129,5):
        error MSB4018:
        The "LinkAssemblies" task failed unexpectedly.\r
        Mono.Linker.MarkException: Error processing method: 'System.Threading.Tasks.Task RoyalMobileApps.XF.Helpers.UI::IRVlHxlYxSdtPtFTbMtDHKxAvfmf(Xamarin.Forms.NavigationPage,Xamarin.Forms.Page)' in assembly: 'RoyalMobileApps.XF.dll'
          ---> Mono.Cecil.ResolutionException: Failed to resolve RoyalMobileApps.XF.Helpers.UI/<ReplaceNavigationStack>d__22\r
          at Mono.Linker.Steps.MarkStep.HandleUnresolvedType(TypeReference reference)\r
          at Mono.Linker.Steps.MarkStep.MarkType(TypeReference reference)\r
          at MonoDroid.Tuner.MonoDroidMarkStep.MarkType(TypeReference reference)\r
          at Mono.Linker.Steps.MarkStep.MarkWithResolvedScope(TypeReference type)\r
          at Mono.Linker.Steps.MarkStep.MarkIfType(CustomAttributeArgument argument)\r
          at Mono.Linker.Steps.MarkStep.MarkCustomAttributeArguments(CustomAttribute ca)\r
          at Mono.Linker.Steps.MarkStep.MarkCustomAttribute(CustomAttribute ca)\r
          at Mono.Linker.Steps.MarkStep.MarkCustomAttributes(ICustomAttributeProvider provider)\r
          at Mono.Linker.Steps.MarkStep.ProcessMethod(MethodDefinition method)\r
          at Mono.Linker.Steps.MarkStep.ProcessQueue()\r
          --- End of inner exception stack trace ---\r
          at Mono.Linker.Steps.MarkStep.ProcessQueue()\r
          at Mono.Linker.Steps.MarkStep.ProcessPrimaryQueue()\r
          at Mono.Linker.Steps.MarkStep.Process()\r
          at MonoDroid.Tuner.MonoDroidMarkStep.Process(LinkContext context)\r
          at Mono.Linker.Pipeline.Process(LinkContext context)\r
          at MonoDroid.Tuner.Linker.Process(LinkerOptions options, ILogger logger, LinkContext& context)\r
          at Xamarin.Android.Tasks.LinkAssemblies.Execute(DirectoryAssemblyResolver res)\r
          at Xamarin.Android.Tasks.LinkAssemblies.Execute()\r
          at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()\r
          at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()
    

    Same result with

    <AndroidLinkSkip>RoyalMobileApps.XF.Android;RoyalMobileApps.XF</AndroidLinkSkip>
    
  • MichaelRumplerMichaelRumpler ATMember ✭✭✭✭✭

    The references are loaded from e.g. C:\Users\micha.nuget\packages\newtonsoft.json\11.0.2\lib\netstandard2.0\Newtonsoft.Json.dll
    I'd have to include each and every NuGet package in the .crproj config file and update them every time I update a NuGet package. That is not feasible.

    I did try to copy all those files to one folder before obfuscation and run the obfuscation in AfterBuild. The obfuscation worked.
    The only folder where I found my compiled dlls at that stage is bin\Release. I copied the obfuscated files there, but then the linker failed again later. I also changed my Obfuscate.bat file to a MSBuild task so that everything is in one place. Here is the current target definition:

      <Target Name="AfterBuild" Condition="'$(Configuration)' == 'Release'">
        <!-- Set some variables -->
        <PropertyGroup>
          <ObfuscaterExe Condition="Exists('C:\ConfuserEx_bin\Confuser.CLI.exe')">C:\ConfuserEx_bin\Confuser.CLI.exe</ObfuscaterExe>
          <ObfuscaterExe Condition="Exists('D:\ConfuserEx_bin\Confuser.CLI.exe')">D:\ConfuserEx_bin\Confuser.CLI.exe</ObfuscaterExe>
          <ObfuscateWorkingDir>$(MSBuildProjectDirectory)\$(IntermediateOutputPath)ObfuscatorSource</ObfuscateWorkingDir>
          <ObfuscateTargetDir>$(ObfuscateWorkingDir)\Obfuscated</ObfuscateTargetDir>
        </PropertyGroup>
        <!-- Preparation -->
        <MakeDir Directories="$(ObfuscateTargetDir)" />
        <ItemGroup>
          <CompiledFiles Include="$(OutputPath)\*.*" />
        </ItemGroup>
        <Copy SourceFiles="@(_ResolveAssemblyReferenceResolvedFilesAbsolute)" DestinationFolder="$(ObfuscateWorkingDir)" />
        <Copy SourceFiles="@(CompiledFiles)" DestinationFolder="$(ObfuscateWorkingDir)" />
        <Copy SourceFiles="$(MSBuildProjectDirectory)\$(MSBuildProjectName).crproj" DestinationFolder="$(ObfuscateWorkingDir)" />
        <!-- Obfuscate -->
        <Exec Command="$(ObfuscaterExe) $(MSBuildProjectName).crproj" WorkingDirectory="$(ObfuscateWorkingDir)" />
        <!-- Copy results -->
        <ItemGroup>
          <ObfuscatedFiles Include="$(ObfuscateTargetDir)\*.*" />
        </ItemGroup>
        <Copy SourceFiles="@(ObfuscatedFiles)" DestinationFolder="$(OutputPath)" />
      </Target>
    

    The result is an exception in the linker:

    C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(2129,5):
    error MSB4018:
        The "LinkAssemblies" task failed unexpectedly.\r
        Mono.Linker.MarkException: Error processing method: 'RoyalDocumentLibrary.RoyalStore rBrtnAtUBbovgwfLEdoWLpUAxRbQ::jXBsLFVBuExGoFPHBUkGMeUtuEAp()' in assembly: 'RoyalTSD.dll'
        ---> Mono.Cecil.ResolutionException: Failed to resolve System.String AHzxviELuKuqQSIZBzcehaeALPPo::XvOoIlIvrpctyjmVtJjpBBkGfMQX\r
            at Mono.Linker.Steps.MarkStep.MarkField(FieldReference reference)\r
            at Mono.Linker.Steps.MarkStep.MarkInstruction(Instruction instruction)\r
            at Mono.Linker.Steps.MarkStep.MarkMethodBody(MethodBody body)\r
            at Mono.Linker.Steps.MarkStep.ProcessMethod(MethodDefinition method)\r
            at Mono.Linker.Steps.MarkStep.ProcessQueue()\r
            --- End of inner exception stack trace ---\r
            at Mono.Linker.Steps.MarkStep.ProcessQueue()\r
            at Mono.Linker.Steps.MarkStep.ProcessPrimaryQueue()\r
            at Mono.Linker.Steps.MarkStep.Process()\r
            at MonoDroid.Tuner.MonoDroidMarkStep.Process(LinkContext context)\r
            at Mono.Linker.Pipeline.Process(LinkContext context)\r
            at MonoDroid.Tuner.Linker.Process(LinkerOptions options, ILogger logger, LinkContext& context)\r
            at Xamarin.Android.Tasks.LinkAssemblies.Execute(DirectoryAssemblyResolver res)\r
            at Xamarin.Android.Tasks.LinkAssemblies.Execute()\r
            at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()\r
            at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()
    

    The class and method in question does exist in the obfuscated dll.

    As this is a release build, I don't create debug symbols. I set these properties:

    <DebugSymbols>False</DebugSymbols>
    <DebugType>none</DebugType>
    <Optimize>true</Optimize>
    

    But the .pdbs are still created in bin\Release. After obfuscation they don't match the dlls anymore, so I always get a warning:

    Xamarin.Android.Common.targets(513,5): warning : 
        Failed to read 'D:\RoyalFamily\RoyalApplications\RoyalTS\Mobile\Android\RoyalTSD\obj\Release\90\linksrc\RoyalTSD.dll' with debugging symbols. Retrying to load it without it. Error details are logged below.
        Mono.Cecil.Cil.SymbolsNotMatchingException: Symbols were found but are not matching the assembly\r
    

    Could this be the problem?

  • MichaelRumplerMichaelRumpler ATMember ✭✭✭✭✭

    As I run the obfuscation in AfterBuild now, I also needed to copy all obfuscated files to the bin\Release folders of all obfuscated projects or they wouldn't show up in the linksrc folder.

    I also tried with debug symbols, but that didn't work either.

  • ChdoulaChdoula JPMember ✭✭

    How do you obfuscate dlls anyway, can't find any working solution on internet. thanks

  • MichaelRumplerMichaelRumpler ATMember ✭✭✭✭✭

    @Chdoula said:
    How do you obfuscate dlls anyway, can't find any working solution on internet. thanks

    I evaluated obfuscators in 2016 and ended up using ConfuserEx (and later ConfuserEx-Reborn). Both are discontinued now, but it is open source, so you could add some stuff on your own.
    Maybe there are others available now, but I don't want to evaluate again (only if I'm sure that the end product works).

    I explained in detail what I did in this post.

  • ChdoulaChdoula JPMember ✭✭

    @MichaelRumpler Thanks for you reply, I already tried that but it doesn't seem to be working and its so complicated to me though its well explained, I ended up using Skater .NET from nuget , and obfuscating dll and copy them manually to release folder.

  • OliverMOliverM USMember ✭✭

    @MichaelRumpler I also get Linker-Errors when obfuscating with ConfuserEx. It also worked for several years. I think it first occured after upgrading my PCLs to .NET Standard recently.

    I also tried AndroidLinkSkip, but still Linker-Errors.

    Did you solve this problem?

  • MichaelRumplerMichaelRumpler ATMember ✭✭✭✭✭

    @OliverM said:
    Did you solve this problem?

    Not yet. I couldn't work on this in the last couple of weeks.

    I did look at Dotfuscator and how they do it. Not because I want to replace ConfuserEx, but just to see how they integrate it. But they have a 800 line .targets file for that so I didn't find the essence of it yet (never looked at a .targets file before).

  • MichaelRumplerMichaelRumpler ATMember ✭✭✭✭✭
    edited March 20 Accepted Answer

    It seems like ConfuserEx does not work (with .NET Standard?) anymore. When I tried with Babel, I copied the obfuscated dlls to the very same folders like I did with ConfuserEx and it worked out of the box. So I'm pretty sure that my obfuscation task is not the issue.

    The most recent ConfuserEx fork I found was ConfuserEx_Reborn from June 2017. But this fork also doesn't work for me. There is an open issue about updating dnlib. Dnlib is the component which reads/writes the dlls. I guess that would fix it (dnlib itself is still maintained). Unfortunately dnlib had some breaking changes between versions 2 and 3. As I know nothing about neither ConfuserEx nor dnlib's inner workings I couldn't get it running.

    I also tried Dotfuscator. The Community edition is free, but it may not be used for commercial apps and it does not do control flow obfuscation which makes it completely useless.
    I told them that I make around €40/month with my app and their response was, that I would probably qualify for a Entrepreneur license of Dotfuscator Professional which would cost €2,490/year. Guess what they can do with this offer.

    Then I looked at Babel for .NET. Don't judge their web site! It is really old and still uses http://, but their stuff works. They have a blog article from 2014 about Obfuscating Xamarin for Android. It is 5 years old, but it still works. And unlike their web site and documentation the program itself is still updated. The current version is 9.1.1.1 from January 31st, 2019. Their email support is also very fast. I got answers within hours.

    What I really like about Babel is, that you can merge multiple dlls into one and automatically make all types from the merged dlls internal and thus eligible for renaming. This feature does require an Enterprise license though. But prizes are also not from out of this world. A Standard license costs €115 and Enterprise €245. You pay that once and can use the current version forever (updates for one year also included).

    So I settled for a Babel Enterprise license.

    Although this all sounds like an advertisement I am not associated with Babel in any way. They didn't ask me to write this and don't pay me for it. I'm just a satisfied customer.

    /cc @OliverM @YorkGo

Sign In or Register to comment.