Forum Xamarin.Forms

OpenCV with Xamarin.Forms

I would like to use OpenCV in a Xamarin.Forms app. OpenCV already provides binaries for Android (as .so files) and iOS (as a framework). I don't need all of OpenCV, so I think that the best way to go about it is to use these provided binaries to create my own library (perhaps in C++) that exposes only the functionality I need. I could then create a C# wrapper for this library that could be used by my Xamarin.Forms app. However, I don't know if this is the best approach, and even if it is, I don't know how to go about it.

A similar question was asked here. However, the answer talks about the difficulty in binding all of OpenCV. I don't need (or want) that. I want to use the binaries provided by OpenCV to create my own library, and then use that. I think that would be easier than trying to bind all of OpenCV.

Tagged:

Answers

  • LandLuLandLu Member, Xamarin Team Xamurai

    You could build your own projects with OpenVC and expose any apis as you want without any problems. Then use binding library to convert this native project to the libraries can be consumed by Xamarin.
    But if you involve the whole OpenVC in your own project, it will not decrease the library's size even though you expose the fewer functionalities. I think it's better to use the bound libraries for Xamarin instead of your own. Anyway, it's all up to you.

  • softlionsoftlion FRBeta ✭✭✭
    edited May 2019

    Just use the whole OpenCV and activate the link all compiler option. Only required code will stay in your app.

  • ohgabbeyohgabbey Member ✭✭

    Hello. Have you found a solution to implement OpenCV in your Xamarin.Forms application?

  • CarlosAndersonCarlosAnderson USMember ✭✭

    @ohgabbey Yes, I found a way to make it work, but the problem is that the final binaries are very large (~ 200 MB). I haven't found a way to make this smaller, although I haven't tried activating the "link all" option as suggested above.

  • jtthjtth Member ✭✭
    > @CarlosAnderson said:
    > @ohgabbey Yes, I found a way to make it work, but the problem is that the final binaries are very large (~ 200 MB). I haven't found a way to make this smaller, although I haven't tried activating the "link all" option as suggested above.

    Hi Carlos could you share your implementation? We’re curious about using image processing tools from OpenCV in XF.
  • jtthjtth Member ✭✭
    > @softlion said:
    > Just use the whole OpenCV and activate the link all compiler option. Only required code will stay in your app.

    Can you elaborate? Would be interested in doing this
  • jtthjtth Member ✭✭
    edited April 2020
    > @CarlosAnderson said:
    > @ohgabbey Yes, I found a way to make it work, but the problem is that the final binaries are very large (~ 200 MB). I haven't found a way to make this smaller, although I haven't tried activating the "link all" option as suggested above.

    Hi Carlos could you share your implementation? We’re curious about using image processing tools from OpenCV in XF.


    > @softlion said:
    > Just use the whole OpenCV and activate the link all compiler option. Only required code will stay in your app.

    Can you elaborate? Would be interested in doing this


    > @ohgabbey said:
    > Hello. Have you found a solution to implement OpenCV in your Xamarin.Forms application?

    Any luck on this?
  • CarlosAndersonCarlosAnderson USMember ✭✭
    edited April 2020

    @jtth said:

    @CarlosAnderson said:
    @ohgabbey Yes, I found a way to make it work, but the problem is that the final binaries are very large (~ 200 MB). I haven't found a way to make this smaller, although I haven't tried activating the "link all" option as suggested above.

    Hi Carlos could you share your implementation? We’re curious about using image processing tools from OpenCV in XF.

    I have it working for Android, but I think the steps will be similar for iOS. I developed my solution based on these resources:

    Using Visual Studio 2019, I added a "Dynamic Shared Library (Android)" C++ project to my Xamarin.Forms solution. In this project, I wrote my own API that does the processing using C++ and OpenCV. I then wrapped those classes by functions that don't depend on OpenCV (for example, they return uchar* instead of cv::Mat).

    I then created a "Shared Project" (C#) to wrap the C++ API so that it may be called from C#. This project contains a class with a bunch of DllImport statements that bind to the C++ API.

    In my Xamarin.Forms project, I created an IVisionService that is implemented in the Xamarin.Android project a by VisionService class that makes calls to the C# wrapper. In this class, I also handle working with IntPtr (which is what I get back instead of uchar*) and converting it to an image I can use in Xamarin. In my case, I decided to work with SkiaSharp, so that my images are converted to SKBitmap.

    I hope this helps, and I'd be happy to help if you get stuck.

  • jtthjtth Member ✭✭
    > @CarlosAnderson said:
    > (Quote)
    > I have it working for Android, but I think the steps will be similar for iOS. I developed my solution based on these resources:
    >
    > * https://docs.microsoft.com/en-us/xamarin/cross-platform/cpp/
    > * https://github.com/xamcat/mobcat-samples/tree/master/cpp_with_xamarin
    > * https://channel9.msdn.com/Shows/XamarinShow/Bring-Some-C-to-Xamarin-Apps
    > * https://www.codeproject.com/Articles/18032/How-to-Marshal-a-C-Class
    >
    > Using Visual Studio 2019, I added a "Dynamic Shared Library (Android)" C++ project to my Xamarin.Forms solution. In this project, I wrote my own API that does the processing using C++ and OpenCV. I then wrapped those classes by functions that don't depend on OpenCV (for example, they return uchar* instead of cv::Mat).
    >
    > I then created a "Shared Project" (C#) to wrap the C++ API so that it may be called from C#. This project contains a class with a bunch of DllImport statements that bind to the C++ API.
    >
    > In my Xamarin.Forms project, I created an IVisionService that is implemented in the Xamarin.Android project a by VisionService class that makes calls to the C# wrapper. In this class, I also handle working with IntPtr (which is what I get back instead of uchar*) and converting it to an image I can use in Xamarin. In my case, I decided to work with SkiaSharp, so that my images are converted to SKBitmap.
    >
    > I hope this helps, and I'd be happy to help if you get stuck.


    Awesome! Wanting something that will work for both Android and iOS. For functions such as thresholding, masking, circleFit, etc. Would you be interested in putting up your Android project / wrapper in a github repo we could sample? Seems like we have our work cut out for us. Thanks!
  • MrAKMrAK Member

    https :/ /www. youtube .com/watch?v=pFv_1KeMZVs

    Are the steps you followed similar to this tutorial?

  • CarlosAndersonCarlosAnderson USMember ✭✭

    @MrAK said:
    https :/ /www. youtube .com/watch?v=pFv_1KeMZVs

    Are the steps you followed similar to this tutorial?

    Not really. They're using Java bindings to make it work. I coded everything related to OpenCV in C++ and exposed a simple API to interop with C#.

  • kapilbhudhiakapilbhudhia Member ✭✭

    @CarlosAnderson , I am trying to exactly what you have achieved, however I have a few questions, it would be great if you can help me out. I was wondering how did you get hold of the opencv native c++ (shared/shared) library for the arm achitecture? Did you compile it yourself from the OpenCV source? Also, what kind of linker flags did you use when compiling your own wrapper C++ code? Specifically, are you using the c++ stl library statically or as a shared dll? Can you let me know the version of the OpenCV that you managed to use and the ndk version for building your own wrapper c++ libraries?

    I know these are a whole lot of questions but I would really appreciate if you can throw some light on these queries, it would go a long way to make my life easier.

    Thanks and Regards,

    Kapil.

  • CarlosAndersonCarlosAnderson USMember ✭✭

    @kapilbhudhia

    I was wondering how did you get hold of the opencv native c++ (shared/shared) library for the arm achitecture? Did you compile it yourself from the OpenCV source?

    I didn't compile it myself. OpenCV provides compiled versions for Android here: https://opencv.org/releases/

    Also, what kind of linker flags did you use when compiling your own wrapper C++ code? Specifically, are you using the c++ stl library statically or as a shared dll?

    For STL, I used LLVM libc++ shared library (c++_shared). I also needed to enable C++ exceptions and Run-Time Type Information (flags -fexceptions and -frtti). I also added the flag -lm because I'm using some math.

    For the Configuration Type, I used Dynamic Library (.so) and for the Target API Level, I used Lollipop 5.0 - 5.0.2, (android-21).

    I also needed to specify the include directories as well as the full path to the OpenCV .so library. For example, I defined the environment variable %OPENCV_ANDROID% to the root location of the compiled libraries. Then, I specified the following include directory: %OPENCV_ANDROID%\sdk\native\jni\include, and the following library dependency (for ARM64): %OPENCV_ANDROID%\sdk\native\libs\arm64-v8a\libopencv_java4.so.

    Can you let me know the version of the OpenCV that you managed to use and the ndk version for building your own wrapper c++ libraries?

    I used OpenCV 4.2.0. For the Android NDK, the project was targeting android-21, but I believe I had up to android-27 installed on my machine.

    I know these are a whole lot of questions but I would really appreciate if you can throw some light on these queries, it would go a long way to make my life easier.

    No problem! I hope this helps.

  • kapilbhudhiakapilbhudhia Member ✭✭

    @CarlosAnderson , thanks a lot for the prompt and detailed reply, I would try compiling my wrapper C++ code based on the information you provided. Would report my progrees here.

  • kapilbhudhiakapilbhudhia Member ✭✭

    Dear @CarlosAnderson , did you have face any issues while building your wrapper library for ARM64, I get the following error for the (ARM)x64 platforms

    0>C:\Program Files (x86)\Android\android-sdk\ndk-bundle\toolchains\aarch64-linux-android-4.9\prebuilt\windows-x86_64/lib/gcc/aarch64-linux-android/4.9.x/../../../../aarch64-linux-android/bin\ld: cannot find -landroid_support

    I notice that this file is not available in the latest ndk-bundle for the 64 bit platforms. I also gather from various sources on google that the android_support library was removed at some point and is not needed anymore. Just wondering if you came across this issue.

    Also, could you please let me know the ndk-bundle version that you used for building your project, the android API version that you targeted is different to the ndk version you must have used.

    for ex, from the android official page - developer.android.com/ndk/downloads/revision_history

    Android NDK LTS, Revision r21e (January 2021)

    Android NDK, Revision r20b (June 2019)

    Android NDK, Revision r19c (January 2019)

    Android NDK, Revision r18b (September 2018)

    Android NDK, Revision r17c (June 2018)

    Android NDK, Revision 16b (December 2017)

    Android NDK, Revision 15c (July 2017)


    I believe I am using the latest version i.e. r21e, can you let me know which one did you use. With ndk-version 16b the support library problem disappears.

    Thanks and Regards,

    Kapil.

  • CarlosAndersonCarlosAnderson USMember ✭✭

    @kapilbhudhia

    Just wondering if you came across this issue.

    I don't remember coming across this issue.

    Also, could you please let me know the ndk-bundle version that you used for building your project, the android API version that you targeted is different to the ndk version you must have used.

    The exact version of the NDK I used was 20.1.5948944, which is probably r20b. It makes sense because I started this specific project back in early 2020.

Sign In or Register to comment.