How to use Xamarin Profiler to track memory during automation testing

ThinhKKieuThinhKKieu USMember ✭✭

Hi all,

Our team develop an application in Xamarin Native. We want to profile memory to detect memory leak and improve quality of the app.

We already profiled and fixed memory leak issues on individual functions of the app by Xamarin Profiler. Now we want to stress test the app day by day, and track memory during stress testing to see if there are any problems related to memory.

We tried using Xamarin UITest to perform automation test, and Xamarin Profiler to monitor memory, but didn't succeed.

  • When Xamarin Profiler starts the app, Xamarin UITest cannot connect to it to take control
  • When Xamarin UITest starts the app, Xamarin Profiler stops monitoring the app

Are there anyone having experience on this situation before? Do you have any suggest to make Xamarin Profiler work during automation testing?

Posts

  • RodrigoMoyaRodrigoMoya ESXamarin Team Xamurai

    Nice that you want to do that, as we have an internal tool, based on Xamarin Profiler code, to do exactly this, a performance test on every commit on our CI setup, that we are looking at making available to everyone as we speak.

    So, until we make it publicly available, what you could do is, if you have a Xamarin UITest runner (that is, an .exe that you can run passing it the tests dll to run, like https://github.com/nunit/docs/wiki/Console-Runner) and set it up on your CI so that it gets profiled. For doing so, you can use the Mono runtime profiler, so, you could start your unit runner with:

    $ mono --profile=log:alloc,output=whatever.mlpd $unit-runner $unit-runner-arguments

    that would produce a MLPD in whatever.mlpd that you can run through mprof-report (part of your Mono installation) to produce a report.

    Of course, this only works on Mac, or if you install Mono framework on Windows. It would help you start profiling whatever test you want on every commit/pull request for instance. Also, I'd suggest you just profile a (or a set of) specific test/scenario, as profiling a full unit test suite would produce a lot of noise that would make it hard to know exactly what test is having problems.

    Our internal tool does more or less this, plus it generates a much better report, that, in our case, is stored on an Azure database and then displayed, along with every single report of every run on a PowerBI report. Of course, when we make it public, the DB+PowerBI part won't be done automatically, you would have to do it yourself, but as I said, the report it generates is much better than what mprof-report shows.

    This internal tool has helped us catching several performance problems with no work at all from the QA/development team, as the PowerBI report clearly shows problems (like an increase in memory allocations, for instance), so I'm pretty sure it will be very useful for you, so I'll try to accelerate the publishing of it, so that at least you can start testing it yourself.

  • ThinhKKieuThinhKKieu USMember ✭✭
    edited April 9

    @RodrigoMoya said:

    So, until we make it publicly available, what you could do is, if you have a Xamarin UITest runner (that is, an .exe that you can run passing it the tests dll to run, like github.com/nunit/docs/wiki/Console-Runner) and set it up on your CI so that it gets profiled. For doing so, you can use the Mono runtime profiler, so, you could start your unit runner with:

    $ mono --profile=log:alloc,output=whatever.mlpd $unit-runner $unit-runner-arguments

    that would produce a MLPD in whatever.mlpd that you can run through mprof-report (part of your Mono installation) to produce a report.

    I've checked on Windows which I already installed Mono framework on. But I got the problem:
    The 'log' profiler wasn't found in the main executable nor could it be loaded from 'mono-profiler-log'

    Have you faced with the problem before?

  • ThinhKKieuThinhKKieu USMember ✭✭

    Hi @RodrigoMoya,

    I used MAC OS to run the command line again, and profiled memory successfully. However, when I opened the .mlpd file by Xamarin Profiler, I only saw processes of Xamarin UITest and NUnit. There were no information related to our application.

    Do you have any ideas about it?

  • RodrigoMoyaRodrigoMoya ESXamarin Team Xamurai

    Right, on Windows I am not sure how to make it work, so good you moved to Mac.

    As for the MLPD containing only Xamarin UITest and NUnit information, what do you mean? As your tests are ran via Xamarin UI test/Nunit, it is normal that lots of classes/methods for that are shown, so maybe you can filter to see your code (i.e. type your namespace in the search box).

Sign In or Register to comment.