Experience with SkiaSharp as a UI Framework Renderer

BrettBarrettBrettBarrett USMember ✭✭

Thanks for working on the SkiaSharp c# bindings to Skia.

Unfortunately my project and plans with it had to be put on hold for now. Here was my experience which might be helpful or perhaps if someone reading would be so kind to point out something which I may have glossed over.

Backstory:

I'm a huge fan of C# and I'm a big fan of simplicity and streamlining designs. Right now there is no cross platform UI library available for .net core so I was going to try my own at giving back to the community to producing one, I have quite a bit of experience with WPF rendering and understand how it ticks at the most part so wanted to replicate some of that over and do it with 'simplicity' as a goal. I was very exciting to see SkiaSharp and immediately wheels started turning on how I could harness it. I'm also a big fan of Facebooks 'React' javascript library and how you render a UI directly in an 'OnRender' method building up a UI based on layers of components. Then utilizing efficient techniques to only render what has actually changed. I find this technique liberating over the time wasting approaches of using XAML which forces a developer to use 2 programming languages to achieve 1 task (The whole idea that designers would author XAML just turned out to be something that never happens in practice and is just a complicated mess that hurts more than it helps). So here goes my attempt at a cross platform UI library of which I've been wanting to write for a while.

To the point:

  1. SkiaSharp views were particularly helpful in getting the understanding needed to pull SkiaSharp off the curb.
  2. SkiaSharp documentation and community just isn't there yet. I considered giving back with trying to help with a 'Getting Started' guide and Skia 101 post but alas I had to abandon ship. Since to me such documentation is secondary to 'native' Skia Documentation which is the root major problem.
  3. Skia.... Everyone talks about how great Skia is and how its the prime option out there for cross platform rendering. Which is great but the problem here is that the Skia website, documentation, and resources is a landmind of badly put together seemingly internal information. You fish for hours for little crumbs of information hoping you can connect the dots at some point -> but never can. The website is seemingly built for Skia internal developers which already had Skia 101, 102, and 103 courses and fully up to speed on how to work with Skia. (Where can I sign up for those?!?!?!) Its basically a dead project if you would look at its resources, documentation, and community perspective. Outside of Google backing of course. There is simply no high level information on this topic available anywhere.
  4. You might then have the idea that since google utilizes it and is in use by chromium that you could use Chromium sources to try to learn how Skia is used... Bad mistake, don't even try. I lost weeks of my life trying to understand that overly complex source code from hell. I still don't fully understand how Chrome utilizes Skia to this day.
  5. Video card hardware vs CPU software rendering is a very complex topic. WPF is a case in point since they simply got video card rendering wrong. The way primitives were tessellated and rendered was simply inefficient and major bottleneck. The fact is that a lot of rendering is lines and text which don't specifically 'tessellate' well for rendering on a video card it is easy to get wrong and end up with worse performance than just CPU driven rendering. Unfortunately there is no information on how Skia achieves any hardware rendering magic and a deep dive in sources will leave you wishing you had just drank a 6 pack instead. There is simply no high level information on this topic available anywhere which would be neccisary to make the correct decisions on implementation of a UI Framework.
  6. SVG rendering, this was the item which mostly killed my project, since I was willing to wade through the horrors of the above as I knowingly was on the bleeding edge of development and hoping to provide a interesting simple crossplatform UI framework to the world... alas SVG rendering was the death knell. Since any UI today is built with SVG assets. E.g. you can easily get all the google material UI assets in SVG form. This is excellent, unfortunately you can't use any of that at this time with SkiaSharp. There are 3 possible approaches I tried, SkiaSharp 1.55 has a SVG implementation which basically doesn't work since its only implemented for simple shapes. There then is a reference from twitter from Miguel de Icaza https://github.com/gentledepp/SVG/tree/skiasharp which you have to look carefully for the SkiaSharp tree which there again is no documentation, monolith library which I couldn't even get to pull off the curb with exceptions being generated at trying to initialize the SkiaGraphics device based on a Desktop Software rendering SKSurface. It's simply still in development, debugging through sources for hours yields many 'TODO's' and questions all left through the source, indicating that its just not ready. Maybe some day it will be but given that there are no documentation on how to use that library and no community at all to speak with the non-SkiaSharp branch I'm not hopefull. Lastly there is an FFImageLoader library which has 'PreviewSVG' support. However the SVG support appears via code analysis the same implementation of SkiaSharp 1.55 which is just simple SVG support. -> Wont work.

I've exhausted all options at my disposal and writing a SkiaSharp SVG renderer just isn't an option since I'd rather work in the salt mines for 6 months then try to do that. I understand why it doesn't exist since SVG is just a beast and not so simple task. Its mostly frustrating because you know that google somewhere has to be rendering SVG in Skia for Chrome. It is possible and done right, but the code is so complex and seemingly deeply rooted in their Blink renderer that its just not usable.

That was my experience. Was quite a ride.

-Brett

Posts

  • You may find this project interesting https://github.com/AvaloniaUI/Avalonia a multi-platform .NET UI framework it uses SkiaSharp as one of its rendering back ends, the UI itself is defined using Xaml (very similar to WPF).

  • BrettBarrettBrettBarrett USMember ✭✭

    Thanks, yea I was aware and even seen posts where the developer of that project were looking for Skia/SkiaSharp support. I did give it another try but can't get the samples to build unfortunately after following the build instructions on the website.

    Admittedly on my original approach number 6, I did have an idea the other day I have yet to try. Which is:

    • Use a 'standard' SVG renderer to render to a bitmap. Perhaps the non-indevelopment SkiaSharp branch of this will do: https://github.com/vvvv/SVG (I don't yet know if I'll be able to get that into .net standard 2.0 when it is released)
    • Cache the bitmap and then display the bitmap with SkiaSharp.

    I'll need to track 'size' and 'scale' and trigger a re-render of the SVG as size/scale changes, but I think it should work. I'll update if I'm able to get that going.

  • MigueldeIcazaMigueldeIcaza USXamarin Team Xamurai

    You could try asking the specific question that you are struggling with.

  • BrettBarrettBrettBarrett USMember ✭✭
    edited January 2017

    Sorry for the long post. Will keep it short and specific in future. Will post a new thread on specific questions if I can't figure them out after some time working on it myself.

    FYI -> I spent 20 hours over the past week refactoring the SkiaSharp branch from here https://github.com/gentledepp/SVG/tree/skiasharp into something simpler to use. I've got several issues remaining but I'm rendering SVG at this point. The polish is always the 'longest' part though but once done I'll post a github link.

  • DanielLDanielL PLInsider ✭✭✭✭

    @BrettBarrett Do you think it's production ready? I am currently using this class for parsing SVGs in FFImageLoading, but it would be nice to have support for more advanced SVG format features.

  • mattleibowmattleibow ZAXamarin Team Xamurai

    I just want to add this link: https://www.nuget.org/packages/SkiaSharp.Svg

    It is a fairly simple, but still got many features.

  • voidvoid DKBeta ✭✭✭

    @BrettBarrett said:
    @DanielL , no that code is not production ready. The SVG implementation is about 75% complete, it's missing transform/matrix math support and missing correct text layout implementation, and several little missing 'TODOs' in the code base. Furthermore the way the author did all the 'Factory' design pattern the interface levels are sometimes 4-5 levels deep with some complex setup needed before you could even render, which is very confusing with no documentation, hints, or samples.

    I refactored it all for simplicity and now sitting at around 85% complete implementation. Please see the below PNG's, the original PNG is rendered with e.g. Illustrator and exported. The SkiaSharp PNG is rendered with SkiaSharp. As you can see its mostly the way there. I've got some text issues as I mentioned and issues with some elements not rendering in the correct location (e.g. transform issues) but I'll be working through that, I was very thankful to gentledepp for giving me such a large head start (although originally being frustrated with the code base) hence my inappropriate rant post which I wish I could delete.

    Original

    SkiaSharp Rendered

    I wished for a flutter.io like UI the in Forms forum here: (https://forums.xamarin.com/discussion/comment/247195/#Comment_247195)

  • DanielLDanielL PLInsider ✭✭✭✭

    @BrettBarrett Impressive! It looks very promising. I also tried to fork that project some time ago, but the project structure was too complicated to me (although libs features are quite impressive). Will you publish those changes as a fork, so others could also collaborate?

  • BrettBarrettBrettBarrett USMember ✭✭
    edited January 2017

    Thanks. I have it in a separate project with plans to do exactly that, unfortunately its not yet the silver bullet you're likely looking for though since I made some decisions to just have the code in a .net 4.6.1 project to significantly simplify the project structure. Since I would have had to spend a weeks to refactor all the stuff to be PCL compliant. Given that .net standard 2.0 is coming around Spring 2017 and I got a lot of development in front of me before even considering Android, IOS. I'm just waiting for .net standard 2.0 and hopefully everyone will jump on board and its 4.6.1 compatibility is as advertised and it will just work.

  • MigueldeIcazaMigueldeIcaza USXamarin Team Xamurai

    @BrettBarrett It looks like you picked up that SVG effort where it was left, that is very interesting.

    Have you published the result of your work somewhere?

  • BrettBarrettBrettBarrett USMember ✭✭
    edited February 2017

    As requested: https://github.com/brettclutch/Svg.SkiaSharp

    Here is the demo video showing it in action: https://www.screencast.com/t/VIOf2h9e1hDE

    I included sample code in the readme so someone coming in would have an idea what to do, I could spend additional time making it more even more user-friendly and polished if demand. I have basics going so I'm back on track with my UI framework project utilizing SkiaSharp now.

  • DylanBerryDylanBerry CAUniversity

    @wieslaw.soltes said:
    You may find this project interesting github.com/AvaloniaUI/Avalonia a multi-platform .NET UI framework it uses SkiaSharp as one of its rendering back ends, the UI itself is defined using Xaml (very similar to WPF).

    With Flutter now in beta, the .NET world is going to need something equally flexible to keep competitive. I would hope AvaloniaUI or a similar Skia-based UI framework gains some traction.

Sign In or Register to comment.