Forum Cross Platform with Xamarin

Announcement:

The Xamarin Forums have officially moved to the new Microsoft Q&A experience. Microsoft Q&A is the home for technical questions and answers at across all products at Microsoft now including Xamarin!

To create new threads and ask questions head over to Microsoft Q&A for .NET and get involved today.

How-to Send Byte[] encoded as ANSI ?

Hi,

I have a mono service running that send utf-8 Byte[] to my MoSync client application, however when I try to read that data as char[] I get weird characters, is the encoding multibyte? should I save it on client side as wchar[]?

I also tried to send bytes with a different Encoding, GetEncoding(1252) it throws this error:
11-08 16:38:37.150: I/mono-stdout(15426): System.NotSupportedException: CodePage 1252 not supported

Any idea if it's possible to encode a Byte[] into ANSI before sending thru socket?

MoSync MAUI only understands ANSI, that is my biggest issue.

Posts

  • JonathanPryorJonathanPryor USXamarin Team Xamurai

    when I try to read that data as char[] I get weird characters.

    How are you reading the data as a char[]?

    is the encoding multibyte?

    Yes, UTF-8 is a multibyte encoding, taking between 1 and 3 bytes to encode a Unicode codepoint.

    should I save it on client side as wchar[]?

    I do not understand this question. A System.Char (C# char) is a 16-bit (2 byte) entity. C# does not have a wchar, nor does it need one (as a C# char is the same size as a WinAPI WCHAR).

    GetEncoding(1252) it throws this error:
    11-08 16:38:37.150: I/mono-stdout(15426): System.NotSupportedException: CodePage 1252 not supported

    You need to add the West internationalization assemblies to your application. There is also a setting in the Project Options/Options page to configure this.

    If you're running a release prior to Mono for Android 4.2.8, you may need to use a workaround.

    Any idea if it's possible to encode a Byte[] into ANSI before sending thru socket?

    The problem is that "ANSI" is a nearly meaningless term; all it means is the "local" non-Unicode, possibly multi-byte, encoding. In the United States, ASCII is the ANSI encoding. Depending on where you are in Europe, code pages 1250, 1252, 1253, or others may be the ANSI encoding. In Japan, the multi-byte Shift-JIS encoding is the ANSI encoding, while China may use the multi-byte Big5 encoding.

    In a networked setting, "ANSI" is meaningless: do you want the client's ANSI encoding or the server's ANSI encoding? These may not be the same.

    MoSync MAUI only understands ANSI, that is my biggest issue.

    You need to determine exactly which encoding MAUI is using -- latin1? latin3? UTF-8? ASCII? -- and make sure that your client app uses that encoding.

    Furthermore, if MAUI isn't using UTF-8, and you're using MAUI in a multi-lingual setting, you may want to configure MAUI so that it uses some Unicode encoding (UTF-8 suggested), otherwise all your clients may not be able to use their full character sets.

  • AdamSAdamS SIMember

    Hi Jonp,

    Thanks for the feedback, let me explain this in a bit more detail:

    I have an application written in C/CPP using the MoSync SDK that communicates with a service written in C# using the Mono for Android SDK. They communicate via Sockets, and transmit to each other UTF-8 XML files. Initially they sent each other ASCII encoded XML, but now the application needs to support chars like "à á ã ç õ" etc... I was trying to get the C# service to send me the data already encoded in ANSI/cp1252, however I got the error above, that I will try to fix again using the West internationalization assemblies.

    However since the proper way seems to be encoding in UTF-8 and send it back

    How are you reading the data as a char[]?

    I am reading this in C/MoSync side to a char array. This char array is then turn into a String with MoSync StringBuilder.

    I do not understand this question. A System.Char (C# char) is a 16-bit (2 byte) entity. C# does not have a wchar, nor does it need one (as a C# char is the same size as a WinAPI WCHAR).

    What I am saying is that I can save it to a wchar on the MoSync side... however that does not fix the decoding issue, because I will have to decode it client side.

    Furthermore, if MAUI isn't using UTF-8, and you're using MAUI in a multi-lingual setting, you may want to configure MAUI so that it uses some Unicode encoding (UTF-8 suggested), otherwise all your clients may not be able to use their full character sets.

    Unfortunately MAUI does not support unicode... http://www.mosync.com/content/native-ui-and-language-support

    My initial goal was to have the server encode it in a way that I would just memcopy the bytes and put it in a String no need to convert anything server side.

  • JonathanPryorJonathanPryor USXamarin Team Xamurai

    I'm confused, as our terminology differs, and there are a lot of duplicate constructs. :-)

    Specifically, which is the "client" and which is the "server" in this environment. I would normally say that the "client" is the Mobile/Mono for Android app, while the "server" would be the MoSync app, but you seem to be using opposite terminology.

    You also said:

    I am reading this in C/MoSync side to a char array. This char array is then turn into a String with MoSync StringBuilder.

    I'm assuming that this has nothing in common with the C# String and StringBuilder types; is there any documentation for the "MoSync StringBuilder"? The only "StringBuilder" I see on their site is from their forums (which contains a bug1), and uses the String type, and I don't know if MoSync's String can sanely deal with multi-byte strings. (Perhaps WString could be used?)

    In any event, I'm not familiar enough with MoSync to know how to make it work with your particular setup. I just know that, in general, UTF-8 should be preferred for network communications. (I also feel that if you have a library that can't sanely support UTF-8, in 2012, it should be dumped post-haste for one that does. It's 2012!)


    1. StringBuilder::appendFormat is:

      void StringBuilder::appendFormat(const char* format, ...)
      {
          char* formattedText;
          va_list args;
          va_start(args, format);
          vasprintf(&formattedText, format, args);
          va_end(args);
          append(formattedText);
      
          delete [] formattedText;
      }
      

      It should be free(formattedText), not delete[] formattedText, as delete[] is not required to use the same memory pool as malloc(3) and free(3), while vasprintf(3) documents that free(3) should be used. ↩︎

  • AdamSAdamS SIMember

    Thank you very much for all your feedback.

    Indeed it is the opposite. In this case the Mono Daemon is the "server", and the MoSync App is the Client.

    Mono is used to wrap a bunch of calls to a C DLL. that calls a bunch of devices, bluetooth printer, and a card reader, similar to the ones used in subways using Sockets... Initially interop processes where supposed to be used but for some reason (that is not documented and the person that decided has left the company) it was changed to sockets.

    I have been brought in to maintain/Fix several limitations of the applications, since it has to run on Android tablets and Psion Teklogix 7505 devices that use WinMobile 6.1 Pro...

    AFAIK MoSync was used because it can be deployed to both platforms... However I cannot tell you how much of a pain in the ass this architecture has been, there is no way to Debug MoSync on the device, there are many many limitations... it's the worst project I have been involved, it has serious Architecture and Implementation issues... however it's been in development for over 9 months and is in production in one country... I have proposed a rewrite but have been shutdown multiple times.

    That is the correct StringBuilder that is being used.

    I Agree in principle, to all you said and the application would have been done in Mono many months ago with much higher quality. However the application uses MAUI that only supports ANSI/cp1252, it can support other font's but you have to create bitmaps, etc... so no UTF-8 support.

    I Agree with you that UTF-8 is the standard encoding for communication, and it should have been used since the beginning, the project was actually using ASCII encoding, so guess what happened when I get this project to support multilingual by changing the encoding from ASCII to UTF-8, MoSync simply can't handle it and there are no utilities to convert it... So I had two options:

    1) Convert it to CP1252 from Mono and Send it in that Encoding.
    2) Send it in UTF-8, And convert it to CP1252 client side (Need to create Converter)

    Tried option 1 first, with the error that lead me to your forum.

    Now I am trying option 2, but I can't get the encoding to change even though I used a different way to parse the XML that supposedly does the conversion. In mosync mtxFeed was being used, now I was using mtxFeedProcess that is supposed to parse UTF-8 to ANSI.

Sign In or Register to comment.