Dynamic texture using planetearth.workbook

linzlinz DEMember ✭✭

Trying to learn how to do dynamic texture in Urho.
How could I dynamically draw each frame something on the "Textures/moon.jpg"

Material moonMaterial = Material.FromImage ("Textures/moon.jpg");
var moonDiffuseTexture = moonMaterial.GetTexture(TextureUnit.Diffuse);

Could someone suggest
(a) How to get Image from the diffuse texture
(b) perhaps use SkiaSharp to draw a e.g. a red square on the image extracted from the texture
(c) Convert the new image into the new DrawnTexture
(d) Set moonMaterial's texture to the new DrawnTexture

=> what is the best way to code in the Workbook so that each frame, the change in texture image is updated

Thanks

Best Answer

  • EgorBoEgorBo BYXamarin Team ✭✭✭
    edited April 1 Accepted Answer

    @linz, I've created a small sample

    Here is the code:

        // Node + StaticModel
        var child = scene.CreateChild();
        var model = child.CreateComponent<StaticModel>();
        model.Model = CoreAssets.Models.Box;
        child.Position = new Vector3(0, 0, 5);
        child.Rotation = new Quaternion(15, 20, 00);
    
        // Load image to SkiaSharp
        var bitmap = SKBitmap.Decode(@"F:\d5fcb66f4adae94ed886fa04279398.jpg");
        var canvas = new SKCanvas(bitmap);
        canvas.DrawBitmap(bitmap, 0, 0);
        canvas.ResetMatrix();
        var font = SKTypeface.FromFamilyName("Arial");
        var brush = new SKPaint {
            Typeface = font,
            TextSize = 128,
            IsAntialias = true, 
            Color = new SKColor(255, 255, 0, 255)
        };
        canvas.DrawText("UrhoSharp + SkiaSharp", bitmap.Width / 7f, bitmap.Height / 1.4f, brush);
        canvas.Flush();
        var image = SKImage.FromBitmap(bitmap);
        var data = image.Encode(SKImageEncodeFormat.Jpeg, 90);
        var skiaImgBytes = data.ToArray();
    
        // Create UrhoSharp Texture2D
        Texture2D text = new Texture2D();
        text.Load(new MemoryBuffer(skiaImgBytes));
        child.Scale = new Vector3(1, text.Height / (float) text.Width, 1) * 2;
    
        var material = new Material();
        material.SetTexture(TextureUnit.Diffuse, text);
        material.SetTechnique(0, CoreAssets.Techniques.Diff, 0, 0);
        model.SetMaterial(material);
    

    More efficient way will be accessing SkiaSharp pixels directly:

    Texture2D text = new Texture2D();
    text.SetNumLevels(1);
    text.SetSize(bitmap.Width, bitmap.Height, Urho.Graphics.RGBAFormat, TextureUsage.Dynamic); //or Static
    text.SetData(0, 0, 0, bitmap.Width, bitmap.Height, bitmap.Bytes);
    

Answers

  • linzlinz DEMember ✭✭

    perhaps we could include a dynamic texture as one of the lessons of \graphics\tiny-renderer => perhaps how to use skiaSharp to draw a dynamic changing tattoo on the face of the African warrior?

  • MigueldeIcazaMigueldeIcaza USXamarin Team Xamurai

    It is a good request.

    I am no expert on this, but generally, people using 3D engines like this avoid pushing new textures continuously as it is expensive to reload and reset the internal graphics state.

    Usually what they do is that they load all the data and then hide/show different textures at different times, rather than doing the "use a new texture" on every update.

  • EgorBoEgorBo BYXamarin Team ✭✭✭
    edited March 31

    @linz there is a sample where I change dynamic textures very frequently (in order to render a video stream actually).
    So you can grab that code here: https://github.com/xamarin/urho-samples/tree/master/FaceDetection/Core
    The idea is:
    1) Create a texture with TextureUsage.Dynamic
    2) Use SetData with a raw image data every time you need to update the texture

  • linzlinz DEMember ✭✭

    @EgorBo
    I found the link that leads to the code you have suggested.

    I believe getting each video frame bitmap and dump that to a texture is less challenging then to take a texture bitmap, converts that into a format that is the most efficient under SkiaSharp for Vector Drawing, and then converts that back to a format that is more seamless to urhosharp. How could we manage this in async way especially if the texture map is large e.g. 2 to 4K instead of e.g. 256x256

    As @MigueldeIcaza has pointed out, I am more concern [due to lack of experience] of the feasibility and if it is possible to squeeze that out of SkiaSharp and UrhoSharp.

    Ideally we need two experts: SkiaSharp (Matthew Leibowitz) and UrhoSharp( @EgorBo) to address the need for efficient dynamic texture use case in cross-platform 3D engines using .NET.

    I understand that this is an unreasonable request. I believe as we are moving towards a mixed reality using e.g. Hololens, this request will become more common and more highly appreciated.

  • EgorBoEgorBo BYXamarin Team ✭✭✭
    edited April 1 Accepted Answer

    @linz, I've created a small sample

    Here is the code:

        // Node + StaticModel
        var child = scene.CreateChild();
        var model = child.CreateComponent<StaticModel>();
        model.Model = CoreAssets.Models.Box;
        child.Position = new Vector3(0, 0, 5);
        child.Rotation = new Quaternion(15, 20, 00);
    
        // Load image to SkiaSharp
        var bitmap = SKBitmap.Decode(@"F:\d5fcb66f4adae94ed886fa04279398.jpg");
        var canvas = new SKCanvas(bitmap);
        canvas.DrawBitmap(bitmap, 0, 0);
        canvas.ResetMatrix();
        var font = SKTypeface.FromFamilyName("Arial");
        var brush = new SKPaint {
            Typeface = font,
            TextSize = 128,
            IsAntialias = true, 
            Color = new SKColor(255, 255, 0, 255)
        };
        canvas.DrawText("UrhoSharp + SkiaSharp", bitmap.Width / 7f, bitmap.Height / 1.4f, brush);
        canvas.Flush();
        var image = SKImage.FromBitmap(bitmap);
        var data = image.Encode(SKImageEncodeFormat.Jpeg, 90);
        var skiaImgBytes = data.ToArray();
    
        // Create UrhoSharp Texture2D
        Texture2D text = new Texture2D();
        text.Load(new MemoryBuffer(skiaImgBytes));
        child.Scale = new Vector3(1, text.Height / (float) text.Width, 1) * 2;
    
        var material = new Material();
        material.SetTexture(TextureUnit.Diffuse, text);
        material.SetTechnique(0, CoreAssets.Techniques.Diff, 0, 0);
        model.SetMaterial(material);
    

    More efficient way will be accessing SkiaSharp pixels directly:

    Texture2D text = new Texture2D();
    text.SetNumLevels(1);
    text.SetSize(bitmap.Width, bitmap.Height, Urho.Graphics.RGBAFormat, TextureUsage.Dynamic); //or Static
    text.SetData(0, 0, 0, bitmap.Width, bitmap.Height, bitmap.Bytes);
    
  • linzlinz DEMember ✭✭

    Holly cow! @EgorBo I am speechless. Now I need to learn the code!

  • PierPier MXMember

    This really looks like Flash code, conceptually speaking.

    Are there any benchmarks that compare this to Adobe Air?

  • linzlinz DEMember ✭✭
    edited April 2

    @Pier could u elaborate?

    I raised the question because I am curious of how these two typical use cases in any 3D engine can be achieved in efficient way through 2 excellently .NETfy opensource libraries:
    (a) 2D (SkiaSharp) to 3D (UrhoSharp) : 2D Vector graphics to texture of 3D object
    (b) 3D (UrhoSharp) to 2D (SkiaSharp): 3D coordinate raycasted to 2D vector Graphics.

  • PierPier MXMember

    Hey @linz

    Since Flash used to be a cross platform graphical engine (and still is using Adobe Air) it seems to me that SkiaSharp + UrhoSharp could be used in the same way to target a number of platforms with the same code.

    Until SkiaSharp there wasn't a cross platform way of creating native rich visual experiences since you had to either move to some sort of game engine (and lose so much functionality) or manage 2 code bases.

    Man I wish I had a couple of weeks to dive into this.

  • linzlinz DEMember ✭✭

    @Pier Well said!
    There are a couple of comparable technologies trying to address cross platform.
    When you have time to dive in, you may realize it may be worth the time to benchmark. I did, I learn, I stay with the Xamarin family of technologies.

  • mattleibowmattleibow ZAXamarin Team Xamurai

    I am not too sure about the internals of Urho, but SkiaSharp can draw directly on the GPU. Instead of creating an raster image, you could maybe draw to a frame buffer - which both SkiaSharp and Urho can access.

    I don't yet have a sample of this, but hope to create one sometime.

  • linzlinz DEMember ✭✭

    @Pier when you have time, take a look on how @mattleibow squeeze cross platform from Vulkan, OpenGL(Angle).etc to make cross platform Vector Drawing (SkiaSharp) possible in Xamarin Technologies.

    @mattleibow thnx for your participation.
    Not sure if there is a possibility of double buffering. The dynamic texture is flickering, but at least with help from @EgorBo, I can continue to move on.

    => I truly think this 2D to 3D and 3D to 2D use cases discussed above is a Clear Winning differentiating advantage with respect to other .NET 3D engine. Perhaps even superior than that of Unity3D. Hope you could work with @EgoBo to squeeze some crazy performance out of this discussion.

    @SandyArmstrong this forum is the right place for cross-expert collaboration. The discussion does not fit neither skiasharp nor UrhoSharp . :-)

Sign In or Register to comment.