How to achieve advanced performance?

ShawnCastrianni.5092ShawnCastrianni.5092 US ✭✭✭
edited March 23 in SkiaSharp

@mattleibow I have been using SkiaSharp for a few months now and am having great success. However, I am quickly approaching the point where I need to speed up my rendering. Right now, I draw everything from scratch on every PaintCanvas call. I am pretty sure I need to stop doing that and instead break things up into layers where each layer is its own bitmap. A layer would only redraw itself from primitives when it changes. All other times, it just blasts its saved bitmap onto the main canvas.

I have never used SKBitmap, SKCanvas, SKPicture, etc. for performance reasons. I have just used SKBitmaps for images and only had the 1 main SKCanvas. Below is my attempt at putting all of my text primitives into a textLayer which is just an SKBitmap that is only created the first time. It definitely fixed my performance problem when panning and zooming, however, if I zoom in, I now get blurry text. This is expected since the text was originally created at zoom factor 1 in a bitmap. So when zooming, the text layer bitmap is just enlarged causing pixelation. This is different than what happens when I do NOT try to create a textLayer bitmap. If I redraw the text from primitives every time, the text stays sharp.

So what I want to do is to allow the blurry text while zooming. but when the user lifts his fingers off the screen, I want the text layer to regenerate itself from primitives to become sharp again and then save that new sharpen bitmap text layer until the user zooms again.

This is where I get stuck since when I force my layer to regenerate itself from text primitives, it still stays blurry. What am I missing? Can you fix my sample program below?

public class LayerPerformancePage : ContentPage
{
    private const int CELL_DIM = 15;

    private SKCanvasView _canvasV = null;
    private SKMatrix _m = SKMatrix.MakeIdentity();
    private SKMatrix _startPanM = SKMatrix.MakeIdentity();
    private SKMatrix _startPinchM = SKMatrix.MakeIdentity();
    private Point _startPinchAnchor = Point.Zero;
    private float _totalPinchScale = 1f;
    private float _screenScale;
    private SKBitmap _bitmap = null;
    private SKBitmap _textLayer = null;

    public LayerPerformancePage()
    {
        #if __ANDROID__
                _screenScale = ((Android.App.Activity)Forms.Context).Resources.DisplayMetrics.Density;
        #else
            _screenScale = (float)UIKit.UIScreen.MainScreen.Scale;
        #endif
        Title = "Layer Performance";
        _canvasV = new SKCanvasView();
        _canvasV.PaintSurface += HandlePaintCanvas;
        Content = _canvasV;
        //Load assets
        using (var stream = new SKManagedStream(ResourceLoader.GetEmbeddedResourceStream(this.GetType().Assembly, "landscape.jpg"))) {
            _bitmap = SKBitmap.Decode(stream);
        }
        //Interaction
        PanGestureRecognizer pgr = new PanGestureRecognizer();
        pgr.PanUpdated += HandlePan;
        _canvasV.GestureRecognizers.Add(pgr);
        PinchGestureRecognizer pngr = new PinchGestureRecognizer();
        pngr.PinchUpdated += HandlePinch;
        _canvasV.GestureRecognizers.Add(pngr);
    }

    private void HandlePan(object sender, PanUpdatedEventArgs puea)
    {
        Console.WriteLine(puea.StatusType + " (" + puea.TotalX + "," + puea.TotalY + ")");
        switch (puea.StatusType) {
        case GestureStatus.Started:
            _startPanM = _m;
            break;
        case GestureStatus.Running:
            float canvasTotalX = (float)puea.TotalX * _screenScale;
            float canvasTotalY = (float)puea.TotalY * _screenScale;
            SKMatrix canvasTranslation = SKMatrix.MakeTranslation(canvasTotalX, canvasTotalY);
            SKMatrix.Concat(ref _m, ref canvasTranslation, ref _startPanM);
            _canvasV.InvalidateSurface();
            break;
        default:
            _startPanM = SKMatrix.MakeIdentity();
            break;
        }
    }

    private void HandlePinch(object sender, PinchGestureUpdatedEventArgs puea)
    {
        Console.WriteLine(puea.Status + " (" + puea.ScaleOrigin.X + "," + puea.ScaleOrigin.Y + ") " + puea.Scale);
        Point canvasAnchor = new Point(puea.ScaleOrigin.X * _canvasV.Width * _screenScale,
                                       puea.ScaleOrigin.Y * _canvasV.Height * _screenScale);
        switch (puea.Status) {
        case GestureStatus.Started:
            _startPinchM = _m;
            _startPinchAnchor = canvasAnchor;
            _totalPinchScale = 1f;
            break;
        case GestureStatus.Running:
            _totalPinchScale *= (float)puea.Scale;
            SKMatrix canvasScaling = SKMatrix.MakeScale(_totalPinchScale, _totalPinchScale, (float)_startPinchAnchor.X, (float)_startPinchAnchor.Y);
            SKMatrix.Concat(ref _m, ref canvasScaling, ref _startPinchM);
            _canvasV.InvalidateSurface();
            break;
        default:
            _startPinchM = SKMatrix.MakeIdentity();
            _startPinchAnchor = Point.Zero;
            _totalPinchScale = 1f;
            //Force textLayer to regenerate
            _textLayer = null;
            _canvasV.InvalidateSurface();
            break;
        }
    }

    private void HandlePaintCanvas(object sender, SKPaintSurfaceEventArgs e)
    {
        e.Surface.Canvas.SetMatrix(_m);
        e.Surface.Canvas.Clear();
        using (var paint = new SKPaint()) {
            paint.TextSize = 10;
            paint.Color = SKColors.Red;
            paint.IsAntialias = true;
            paint.Style = SKPaintStyle.Fill;
            paint.TextAlign = SKTextAlign.Center;
            SKSize imgSize = new SKSize(_bitmap.Width, _bitmap.Height);
            SKRect aspectRect = SKRect.Create(e.Info.Width, e.Info.Height).AspectFit(imgSize);
            e.Surface.Canvas.DrawBitmap(_bitmap, aspectRect, paint);
            if (_textLayer == null) {
                _textLayer = new SKBitmap(e.Info);
                using (SKCanvas layerCanvas = new SKCanvas(_textLayer)) {
                    layerCanvas.Clear();
                    float curX = 0;
                    float curY = 0;
                    while (curX < e.Info.Width) {
                        while (curY < e.Info.Height) {
                            SKRect cell = new SKRect(curX, curY, curX + CELL_DIM, curY + CELL_DIM);
                            layerCanvas.DrawText("Hi", cell.MidX, cell.MidY, paint);
                            curY += CELL_DIM;
                        }
                        curY = 0;
                        curX += CELL_DIM;
                    }
                }
            }
            e.Surface.Canvas.DrawBitmap(_textLayer, e.Info.Rect);
        }
    }
}

Best Answers

Answers

  • @mattleibow Thanks so much for your time and response. Your enhanced example is exactly what I was looking for. However, I am not sure I understand how it works. Here are some questions I have:

    1. by using SKAutoCanvasRestore on the main canvas when drawing the background layer, why does the bitmap NOT get unpanned or unzoomed when the canvas matrix gets reset? I understand the setting the matrix and then drawing the landscape bitmap, but when the matrix gets reset, shouldn't that undo the pan and undo the zoom?
    2. Why do you NOT use the SKAutoCanvasRestore technique on the layerCanvas inside the if statement? Just because you didn't need to?
    3. I don't understand how drawing the cached _textLayer bitmap using the incremental matrix _currentTransformM works? Why would the incremental matrix line up with the bitmap which was drawn using the total matrix _m?

    I will attempt to use this technique to handle multiple layers in my real app and see if it solves my performance problem. Thanks again.

  • AjaySawantAjaySawant INMember ✭✭

    Hi Shawn,

    I have been using the above pinch and pan code and was able to perform the zoom on SKCanvasView, Thanks for sharing this code it was really helpful.

    Could you please guide me on how to achieve the following with this code.

    1) User should not be able to zoom below the specific scale. Like anything below scale 1 should not be allowed.
    2) User should be able to pan only within the canvas bound / area while in scaled mode. Currently this is free flow.
    3) Can I use GestureRecognizers and TouchEffect together, Currently it seems that only one can work at a time.

    Thanks in Advance
    Ajay Sawant

  • mattleibowmattleibow ZAXamarin Team Xamurai

    @AjaySawant I just wanted to comment on point 3:

    It is true that only one can work at a time, but this is configurable - to some extent. (besides enabling/disabling)

    The first case I think I need to mention is that there is a bug on iOS with Xamarin.Forms:
    https://github.com/xamarin/Xamarin.Forms/pull/990
    This has been fixed in master and should be available in the 2.3.6-pre1 releases

    On Android, only one can be used, but this can be changed in the event handler. Something like this could be done:

    private void OnTouch(object sender, SKTouchEventArgs e)
    {
        if (shouldTap) {
            // we aren't interested
            e.Handled = false;
        } else {
            // stop the tap from working - we may be drawing a line
            e.Handled = true;
        }
    }
    

    Even though I am using a shouldTap, this can be anything - such as a coordinate check or some allow-drawing state

    Finally, on macOS and Windows, both will work fine.

  • AjaySawantAjaySawant INMember ✭✭

    Hi Matthew,

    Thanks for you support, I will try out these options. Will you be able to give me some guidance with point 1 and 2 ?

    Thanks in Advance
    Ajay Sawant

Sign In or Register to comment.