NativeElement

mdizzymdizzy MIchael BrownUSMember ✭✭

Summary

This proposal is for a NativeElement and NativeRenderer that would accept graphics primitives defined in Xaml and render them cross platform in a view.
The idea is that users could use these graphic primitives to create custom controls for unified look and feel across platforms.

API Changes

I've created a PoC around defining graphics primitives using XAML. But of course there are some limitations. The biggest of which is that defining the primitives as UIViews would be a potential performance hit depending on the complexity of the drawing.

Thinking further, I think the concept could work, if there were a single view --which I'm calling NativeRenderElement-- That will be the entry point to providing access to native graphics rendering.

The proposed additions to the API to support this scenario would include

  1. NativeElement: the top level control that is responsible for rendering graphic primitives.
  2. NativeElementRenderers for each platform that understands the XAML based primitives and draws them according to the definition.
  3. Graphic primitive bindable objects that define what the Element should render (Ellipse, Rectangle, Path, Text, with stroke and fill and translating the "Brush" --solid color and gradient among others-- from WPF to Xamarin Forms)

I've provided a working sample that shows an EllipseView with a native renderer that works in iOS and Android. Also, I took the time to verify that hit testing works properly on both (tapping outside the ellipse but within the view's bounding rect is ignored).

Intended Use Case

The Native Element could be used to create complex controls that need custom rendering. Or for building a custom UI that has a branded look & feel.

0
0 votes

Open · Last Updated

Posts

  • ClintStLaurentClintStLaurent Clint StLaurent USUniversity ✭✭✭✭✭

    So effectively... NGraphics... But with binding and XAML support. That would be cool.

  • mdizzymdizzy MIchael Brown USMember ✭✭
    I actually was debating whether or not to use an intermediate library. Does Ngraphics support custom hit testing?
  • ClintStLaurentClintStLaurent Clint StLaurent USUniversity ✭✭✭✭✭
    edited May 27

    I don't know. Its a drawing library, not an 'interaction' library. So I doubt it. But if you use it to make a control. Then you can just assign a TapGestureRecognizer to the control. By default the areas of full transparent don't register as a hit.

  • ErlendAErlendA Erlend Angelsen NOMember ✭✭

    Skiasharp does native rendering of primitives

  • mdizzymdizzy MIchael Brown USMember ✭✭

    @ClintStLaurent said:
    I don't know. Its a drawing library, not an 'interaction' library. So I doubt it. But if you use it to make a control. Then you can just assign a TapGestureRecognizer to the control. By default the areas of full transparent don't register as a hit.

    That wasn't the default behavior I got. By default the custom view was responding to taps anywhere in it's bounding rectangle both on Android and iOS

  • mattleibowmattleibow Matthew Leibowitz ZAXamarin Team Xamurai

    @mdizzy This looks like a cool idea! I always liked the way Windows had the XAML graphics so that I didn't have to write code to get lines on the screen.

    Although I am biased (because I am working on the project), I would suggest having a look at SkiaSharp as @ErlendA suggested: https://github.com/mono/SkiaSharp

    This library works on iOS, Android, UWP, macOS, Windows Desktop and tvOS. The graphics API is 100% cross-platform, and there is even a PCL that you can code against.

    I also agree that creating a view for each graphics element will be quite an overhead. I have been hacking away on something, so I will share in hopes that you will be inspired by something: https://github.com/mattleibow/SkiaSharpGraphics. All the code actually lives in the portable project, and all the drawing logic can live there too. If you still want to use the native platforms, this can still be done, and you can use bait-and-switch to do the drawing on each platform.

    I also would like to bring to your attention other projects that may have bits that may interest you (I am not sure exactly what they all do, and if it will relate, but I have come across various comments and they appear to be doing some graphics):

  • DavidDancyDavidDancy David Dancy AUMember ✭✭✭✭

    @mdizzy, @mattleibow that looks very interesting.

  • mdizzymdizzy MIchael Brown USMember ✭✭
    > @mattleibow said:
    > @mdizzy This looks like a cool idea! I always liked the way Windows had the XAML graphics so that I didn't have to write code to get lines on the screen.
    >
    > Although I am biased (because I am working on the project), I would suggest having a look at SkiaSharp as @ErlendA suggested: https://github.com/mono/SkiaSharp

    Like I said, I have no qualms with leveraging an intermediate library. I just wanted to prove the feasibility of the concept.

    I'll do a second round and see where that gets me.
  • rogiheerogihee Rogier van der Hee NLMember ✭✭✭

    SkiaSharp would be an ideal candidate for this, but AFAIK there is no support for touch events on canvasses, so if you want to make the graphics more like controls you are sortof dead in the water.

  • mattleibowmattleibow Matthew Leibowitz ZAXamarin Team Xamurai

    @rogihee Touch events are coming soon: https://github.com/mono/SkiaSharp/pull/309
    I just need to do some more checks and make a few tweaks to the system. Right now there are some conflicts with Xamarin.Forms' gestures.

  • mattleibowmattleibow Matthew Leibowitz ZAXamarin Team Xamurai
    edited June 19

    @mdizzy I made a few extra commits to my tests: https://github.com/mattleibow/SkiaSharpGraphics

    I added a few features such as drawing lines, rectangles and ovals. Shapes can be nested and also do clipping. I am not sure how cool this is, but I kind of like it:

    <g:GraphicsCanvas>
        <g:Rectangle Top="20" Left="10" Width="120" Height="110" FillColor="Red" ClipChildren="true">
            <g:Oval Top="10" Left="10" Width="10" Height="10" StrokeColor="Blue" StrokeWidth="2" />
            <g:Oval Top="50" Left="60" Width="50" Height="100" StrokeColor="Green" StrokeWidth="6" />
        </g:Rectangle>
        <g:Line Top="5" Left="5" Width="200" Height="150" StrokeColor="Purple" StrokeWidth="3" />
    </g:GraphicsCanvas>
    
  • mdizzymdizzy MIchael Brown USMember ✭✭
    Sweet, I've actually carved out some time this week to work on this. I was looking at what it would take to use SkiaSharp. You gave me a great starting point.
  • rogiheerogihee Rogier van der Hee NLMember ✭✭✭

    @mattleibow Cool! Does it also supports Swipes?

    The conflicts should become a thing of the past soon: https://github.com/xamarin/Xamarin.Forms/pull/990

  • mattleibowmattleibow Matthew Leibowitz ZAXamarin Team Xamurai

    @rogihee It currently just passes all the raw point data to the user:

    https://github.com/mono/SkiaSharp/blob/master/source/SkiaSharp.Views.Forms/SkiaSharp.Views.Forms.Shared/SKTouchEventArgs.cs

    This should be enough to create a gesture recognizer.

  • mattleibowmattleibow Matthew Leibowitz ZAXamarin Team Xamurai
    edited June 19

    @mdizzy Great, I am really hoping to get this thing to a state where it is package-able and useful to the community. The nice thing about what I did was that this is just elements, there are no views, and, all the code can be 100% shared.

    Another thing, I am hoping to get something going and then start adding it to our new repository for things that relate to SkiaSharp: https://github.com/mono/SkiaSharp.Extended

    A declarative canvas would be a huge plus!

  • mdizzymdizzy MIchael Brown USMember ✭✭

    @mattleibow Sorry I went silent on this. Have had a few other priorities/fires to put out. Ready to circle back around on this. Is this in the SkiaSharp extended repo yet or is it still in a private branch?

  • mattleibowmattleibow Matthew Leibowitz ZAXamarin Team Xamurai

    You can start adding code right away: https://github.com/mono/SkiaSharp.Extended, just fork and PR.

    For communication on this, you can always create an issue there and keep all the info together.

  • ChaseFlorellChaseFlorell Chase Florell CAInsider, University mod
    edited July 26

    the downside to SkiaSharp is the sheer size of the library. It's hard to justify a 50MB bump.

  • mattleibowmattleibow Matthew Leibowitz ZAXamarin Team Xamurai

    @ChaseFlorell The size is not that large. Each platform architecture is ~4MB. So for iOS, you need arm64 and maybe arm7 (thus ~9MB). Android has arm7 and arm64 (again 9MB). Windows UWP has 3 platforms, ARM, x86, x64, (so ~14MB). tvOS only really has arm64 (so ~4MB).

    With iOS 11, they only have support for arm64, so this will further reduce the overall app size. Even with Android, arm7 may be all that is needed.

  • ChaseFlorellChaseFlorell Chase Florell CAInsider, University mod

    @mattleibow said:
    @ChaseFlorell The size is not that large. Each platform architecture is ~4MB. So for iOS, you need arm64 and maybe arm7 (thus ~9MB). Android has arm7 and arm64 (again 9MB). Windows UWP has 3 platforms, ARM, x86, x64, (so ~14MB). tvOS only really has arm64 (so ~4MB).

    With iOS 11, they only have support for arm64, so this will further reduce the overall app size. Even with Android, arm7 may be all that is needed.

    I don't want to derail this post, so this will be my last comment on the topic.

    For comparison, here's my app stats when using SkiaSharp, and after removing it in favour of a different approach.

    enter image description here

    enter image description here

    Don't get me wrong however, SkiaSharp is a fantastic tool and is super useful. I just found that for my 2D needs, it was too much of a sacrifice to justify. Also, the size increase wasn't nearly as noticeable on the iOS platform, it was Android that was too much to handle.

  • TobiasSchulz.9796TobiasSchulz.9796 Tobias Schulz DEMember ✭✭

    The last time I tried SkiaSharp a few months ago, there was one major problem regarding the .apk size: The managed dll in the nuget contains the native libraries as an embedded resource.

    As part of the build process, they are extracted by xamarin and placed as separate files in the .apk file, but they remain in the managed dll too!

    I don't know how to achieve this, but the managed dll needs to be stripped of these native library resources, and then the apk size increase would increase only half as much.

Sign In or Register to comment.