Quick JSON serializer performance test (Json.NET vs ServiceStack)

2»

Posts

  • YSharpYSharp Cyril Jandia USMember ✭✭
    edited December 2013

    Added support for "__type" key in input JSON, to help deserializing into interfaces, as Sami mentioned.

    Committed @ ee1727546a, and NuGet updated.

    I believe this is what Sami was alluding to, as being supported by ServiceStack...

    With the following (excerpt from Program.cs):

        public interface ISomething
        {
            int Id { get; set; }
            // Notice how "Name" isn't introduced here yet, but
            // instead, only in the implementation class "Stuff"
            // below:
        }
    
        public class Stuff : ISomething
        {
            public int Id { get; set; }
            public string Name { get; set; }
        }
    
        public class StuffHolder
        {
            public IList<ISomething> Items { get; set; }
        }
    

    being correctly deserialized from the corresponding unit test:

            obj = UnitTest(@"{""Items"":[
                {
                    ""__type"": ""Test.Program+Stuff, Test, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"",
                    ""Id"": 123, ""Name"": ""Foo""
                },
                {
                    ""__type"": ""Test.Program+Stuff, Test, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"",
                    ""Id"": 456, ""Name"": ""Bar""
                }]}", s => new JsonParser().Parse<StuffHolder>(s));
            System.Diagnostics.Debug.Assert
            (
                obj is StuffHolder && ((StuffHolder)obj).Items.Count == 2 &&
                ((Stuff)((StuffHolder)obj).Items[1]).Name == "Bar"
            );
    

    Where, in this example,

    Test.Program+Stuff, Test, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
    

    is the "Test.exe" assembly-fully qualified type name for the class "Stuff" above.

    (Still passes the few other unit tests and all the speed tests without breaking so far.)

    Goodnight for now.

    Cheers,

  • SpartanSpartan Uu Ko USMember

    @SKall: Always good to know which is the better of the many alternatives we have. I know we have Json.net for PCL, do we have PCL for Service Stack? Would you be interested to run a benchmark using PCL code (if that makes any sense)?

  • SKallSKall Sami M. Kallio USMember ✭✭✭

    Performance will be the same whether you use it from PCL or non-PCL. The binaries are per platform but that doesn't stop you from using them from PCL's.

  • YSharpYSharp Cyril Jandia USMember ✭✭
    edited December 2013

    Further improved support for deserializing into generic (typed) dictionaries.

    Committed @ f9af92f256, and NuGet updated.

    Inspired / informed by this StackOverflow question (asked in April):

    http://stackoverflow.com/questions/16296158/best-way-to-serialize-deserialize-net-object-json-containing-dictionaryfoo

    For instance, with the following enum type and POCOs:

        public enum VendorID
        {
            Vendor0,
            Vendor1,
            Vendor2,
            Vendor3,
            Vendor4,
            Vendor5
        }
    
        public class SampleConfigItem
        {
            public int Id { get; set; }
            public string Content { get; set; }
        }
    
        public class SampleConfigData<TKey>
        {
            public Dictionary<TKey, object> ConfigItems { get; private set; }
        }
    

    Both two new unit tests pass:

            string configTestInputVendors = @"{
                ""ConfigItems"": {
                    ""Vendor1"": {
                        ""__type"": ""Test.Program+SampleConfigItem, Test, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"",
                        ""Id"": 100,
                        ""Content"": ""config content for vendor 1""
                    },
                    ""Vendor3"": {
                        ""__type"": ""Test.Program+SampleConfigItem, Test, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"",
                        ""Id"": 300,
                        ""Content"": ""config content for vendor 3""
                    }
                }
            }";
    
            obj = UnitTest(configTestInputVendors, s => new JsonParser().Parse<SampleConfigData<VendorID>>(s));
            System.Diagnostics.Debug.Assert
            (
                obj is SampleConfigData<VendorID> &&
                ((SampleConfigData<VendorID>)obj).ConfigItems.ContainsKey(VendorID.Vendor3) &&
                ((SampleConfigData<VendorID>)obj).ConfigItems[VendorID.Vendor3] is SampleConfigItem &&
                ((SampleConfigItem)((SampleConfigData<VendorID>)obj).ConfigItems[VendorID.Vendor3]).Id == 300
            );
    

    and:

            string configTestInputIntegers = @"{
                ""ConfigItems"": {
                    ""123"": {
                        ""__type"": ""Test.Program+SampleConfigItem, Test, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"",
                        ""Id"": 123000,
                        ""Content"": ""config content for key 123""
                    },
                    ""456"": {
                        ""__type"": ""Test.Program+SampleConfigItem, Test, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"",
                        ""Id"": 456000,
                        ""Content"": ""config content for key 456""
                    }
                }
            }";
    
            obj = UnitTest(configTestInputIntegers, s => new JsonParser().Parse<SampleConfigData<int>>(s));
            System.Diagnostics.Debug.Assert
            (
                obj is SampleConfigData<int> &&
                ((SampleConfigData<int>)obj).ConfigItems.ContainsKey(456) &&
                ((SampleConfigData<int>)obj).ConfigItems[456] is SampleConfigItem &&
                ((SampleConfigItem)((SampleConfigData<int>)obj).ConfigItems[456]).Id == 456000
            );
    

    Note this works as well if we have instead:

            public Dictionary<TKey, dynamic> ConfigItems { get; private set; }
    

    with .NET >= 4.0, to answer the original StackOverflow question.

    Of course, this kind of use case still relies on the "__type" attribute and .NET type name being encountered first at the beginning of the incoming JSON object - that which maps to the Dictionary<TKey, TValue>'s TValue in this example - as in:

    ...
    {
        "__type": "Test.Program+SampleConfigItem, Test, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
        "Id": 100,
        "Content": "config content for vendor 1"
    }
    ...
    

    per the same convention as when using ServiceStack.

    (And still passes the few other unit tests and all the speed tests without breaking so far.)

    Cheers,

  • YSharpYSharp Cyril Jandia USMember ✭✭
    edited December 2013

    And finally, this works as well (yet another unit test follows):

        // (legacy)
        public enum Status { Single, Married, Divorced }
    
        public class Person
        {
            public int Id { get; set; }
            public string Name { get; set; }
            public Status Status { get; set; }
            public string Address { get; set; }
            public IEnumerable<int> Scores { get; set; }
            public object Data { get; set; }
            public IDictionary<DateTime, string> History { get; set; }
        }
    
        // (added)
        public class Asset
        {
            public string Name { get; set; }
            public decimal Price { get; set; }
        }
    
        public class Owner : Person
        {
            public IList<Asset> Assets { get; set; }
        }
    
        public class Owners
        {
            public IDictionary<decimal, Owner> OwnerByWealth { get; set; }
            public IDictionary<Owner, decimal> WealthByOwner { get; set; }
        }
    

    Tested thru:

            obj = UnitTest(@"{
                ""OwnerByWealth"": {
                    ""15999.99"":
                        { ""Id"": 1,
                          ""Name"": ""Peter"",
                          ""Assets"": [
                            { ""Name"": ""Car"",
                              ""Price"": 15999.99 } ]
                        },
                    ""250000.05"":
                        { ""Id"": 2,
                          ""Name"": ""Paul"",
                          ""Assets"": [
                            { ""Name"": ""House"",
                              ""Price"": 250000.05 } ]
                        }
                },
                ""WealthByOwner"": [
                    { ""key"": { ""Id"": 1, ""Name"": ""Peter"" }, ""value"": 15999.99 },
                    { ""key"": { ""Id"": 2, ""Name"": ""Paul"" }, ""value"": 250000.05 }
                ]
            }", s => new JsonParser().Parse<Owners>(s));
            Owner peter, owner;
            System.Diagnostics.Debug.Assert
            (
                (obj is Owners) &&
                (peter = ((Owners)obj).WealthByOwner.Keys.
                    Where(person => person.Name == "Peter").FirstOrDefault()
                ) != null &&
                (owner = ((Owners)obj).OwnerByWealth[15999.99m]) != null &&
                (owner.Name == peter.Name) &&
                (owner.Assets.Count == 1) &&
                (owner.Assets[0].Name == "Car")
            );
    

    The only somehow interesting guy is this one, of course (along with its corresponding input JSON fragment):

            public IDictionary<Owner, decimal> WealthByOwner { get; set; }
    

    (Note: the above is supported at revision 1.9.5 (and up))

    Goodnight all,

  • YSharpYSharp Cyril Jandia USMember ✭✭

    Before I forget, just one remark I could have made earlier:

    if one wants to rely on JSON to transport applicative messages thru (strongly typed) DTOs, over the wire and back-and-forth their application client and the service tier, I do think it is important for us developers to have a reliable and flexible support for .NET's Dictionary<TKey, TValue> whenever we find useful to deserialize into it -

    if only for the very simple reason of its type safety coupled with its efficiency.

    (Hence the effort, here)

    Cheers,

  • SKallSKall Sami M. Kallio USMember ✭✭✭

    Cyril, as an alternative to manual JSON data you could use Json.NET and/or ServiceStack to serialize a DTO and then deserialize it with yours. The DTO's then should implement IEquatable so you can easily compare them.

    Sample DTO's:

    https://github.com/sami1971/SimplyMobile/blob/master/Core/Tests/TextSerializationTests/Dtos/Primitives.cs https://github.com/sami1971/SimplyMobile/blob/master/Core/Tests/TextSerializationTests/Dtos/DateTimeDto.cs

    The benefit of this approach is you can discover errors like rounding and inaccuracies. The latter DTO for example helped me to realize with ServiceStack you will need to set the date format to ISO8601 or otherwise you will lose accuracy on the time stamps.

    I used these two helper functions to do sanity checks on the DTO's:

            public static bool CanSerialize<T>(
                        ITextSerializer serializer, 
                        T item, 
                        ITextSerializer deserializer)
                    {
                            var text = serializer.Serialize (item);
                            var obj = deserializer.Deserialize<T>(text);
                            return obj.Equals(item);
                    }
    
            public static bool CanSerializeEnumerable<T>(
                      ITextSerializer serializer, 
                      IEnumerable<T> list, 
                      ITextSerializer deserializer)
            {
                var text = serializer.Serialize(list);
                var obj = deserializer.Deserialize<IEnumerable<T>>(text);
                return obj.SequenceEqual(list);
            }
    
  • YSharpYSharp Cyril Jandia USMember ✭✭
    edited December 2013

    Very good idea, Sami. Indeed, it's also pretty boring to hand craft those strings just for the sake of preparing test cases. Thanks for this helper, I'll plug something alike for my unit tests some time later.

    Cheers,

  • YSharpYSharp Cyril Jandia USMember ✭✭
    edited January 5

    Up to 26% speed performances improvement when deserializing dictionaries, and 5% to 10% with POCOs, at revision 1.9.9.

    (Corresponding benchmark's figures updated)

    I expect those to be also observable with the CLR on Android.

    Cheers,

  • SKallSKall Sami M. Kallio USMember ✭✭✭

    Confirmed that the new version compiles to WP8, no tests run.

    Mixed results for speed w/ Android. Dicos 10k was faster, 92% of the old test. Tiny 100k was slower, 118%. Highly nested was faster, 96%.

    Memory usage went considerable higher with Tiny & Highly Nested, Dicos was about the same.

    https://skydrive.live.com/embed?cid=CA84936FF490EEDC&resid=CA84936FF490EEDC%211465&authkey=AL3iccbYFakA-zw&em=2

  • SKallSKall Sami M. Kallio USMember ✭✭✭
    edited January 8

    Ran the same test (object Person) with native Java code using gson library.

    Serialization:

    Xam.:  2037ms (with ServiceStack.Text)
    gson: 10601ms
    

    Deserializing:

    Xam.:  1798ms (with deserializer from @YSharp)
    gson: 15571ms
    

    Anyone know faster JSON serializers for Dalvik?

  • YSharpYSharp Cyril Jandia USMember ✭✭
  • SKallSKall Sami M. Kallio USMember ✭✭✭

    I gave it a try and it doesn't seem to work on Android.

  • YSharpYSharp Cyril Jandia USMember ✭✭

    I'm technically clueless about those, but if that helps, it seems Gson and Json Smart are oft-reported to be among the best performers in the Java realm.

  • YSharpYSharp Cyril Jandia USMember ✭✭
    edited January 8

    Hi,

    I have a design question (note: indirectly related to the performance aspect, though).

    As stated in the goal section of the project page, although I'm not interested in making its API as configurable as some of its competitors, I'd still like to keep it as user-friendly (i.e., intuitive) and flexible as I can, for the most common and somehow less common use cases.

    While putting more thoughts into how to support anonymous (reference) types and nullable (value) types, if that's feasible without a too much significant loss in performances, I ran into an interesting edge case w.r.t. nullable types. Here it is.

    Okay, so I assume that one would expect this:

    public class Person
    {
        public string Name { get; set; }
        public DateTime? BirthDate { get; set; }
    }
    

    to pass the following:

    var john = new JsonParser().Parse<Person>(@"{ ""Name"": ""John Smith"", ""BirthDate"": null }");
    System.Diagnostics.Debug.Assert(!john.BirthDate.HasValue);
    

    (E.g., one may have the requirement of being able to parse data about historical figures about whom the exact birth date is simply not known, etc.)

    Then, likewise, a question is:

    and what about nullable types used for keys of dictionaries - i.e., what shape of JSON one is most likely to use to represent the "null" key for those?

    Say, given this (granted, rather technical) example:

    public class ReferentialTableColumnInfo<TColumn> where TColumn : struct
    {
        public string ColumnName { get; set; }
        public SmartDictionary<TColumn?, string> ColumnValuesDescriptions { get; set; }
    }
    

    Note the "SmartDictionary": it can't be .NET's standard IDictionary(TKey, TValue) implementation (and contract), as that one doesn't allow null for dictionary keys, even if those are of nullable (value) types (most likely an early design decision, for consistency with how the dictionary handle keys in the case of reference types).

    However, I consider the latter for just what it is: a design decision from Microsoft, and arguably an arbitrary one at that.

    But, both conceptually and technically speaking, nothing prevents anyone to come up one day with their own dictionary, which, for whatever reason/purpose, will need to accept null as a value (or signal of absence thereof) for its key, be it of a reference or value type.

    So, one may want to deserialize some input JSON into it, using:

    var someColumnInfo = new JsonParser().Parse<ReferentialTableColumnInfo<int?>>("...");
    

    My question is about that ellipsis "..."

    As we know, there are two most common (frequent) ways to represent dictionaries in JSON (and currently automatically detected as such by my parser, when provided with an IDictionary(TKey, TValue) generic type argument in the Parse method)...

    ... One way is quite verbose, but also the most general (and sometimes privileged as the default by some JSON serializers, like Microsoft's DataContractJsonSerializer) - it expects the dictionary to be serialized as a list (JSON array) of its key/value pairs, which themselves are expected to be serialized as exactly-two-property objects, with "key" and "value" properties, resp.:

    [ { "Key": "Graduation", "Value": "1989-06-09" }, { "Key": "Marriage", "Value": "2001-09-08" }, ... ]
    

    With my above ReferentialTableColumnInfo example there's no particular problem with this JSON encoding; we could parse without difficulty:

    "ColumnValuesDescriptions": [ { "Key": null, "Value": "The unknown value" }, { "Key": 1, "Value": "The unit" }, { "Key": 2, "Value": "The pair" } ... ] ...
    

    (assuming the parser is bug-free, the above C# code - var someColumnInfo = ... - should execute OK then, with: someColumnInfo[null] == "The unknown value")

    But the second common way to represent dictionaries in JSON is of course the one that uses the property of JSON object values being exactly that, precisely, when the host language is JavaScript itself - with objects which are nothing but hash maps (of strings to something else):

    { "Graduation": "1989-06-09", "Marriage": "2001-09-08", "FirstChild": "2002-07-26", ... }
    

    Now, we need to ask ourselves: in that second JSON encoding, what could be expected as a reasonable JSON representation of "null" for that "int?" key type?

    Should that be:

    "ColumnValuesDescriptions" : { "": "The unknown value", "1": "The unit", "2": "The pair", ... } ...
    

    (if only because JSON requires double quotes to enclose the key names in JSON objects)

    Or:

    "ColumnValuesDescriptions" : { "null": "The unknown value", "1": "The unit", "2": "The pair", ... } ...
    

    ?

    What are your thoughts (and/or experiences) in that edge case?

    Thanks,

  • YSharpYSharp Cyril Jandia USMember ✭✭
    edited January 9

    Never mind. On a second thought helped by some more googling (over SO, etc) I figured I should just go with a conservative approach for a first try at implementing this nullable types support.

    That is, to simply follow .NET's convention of disallowing nulls for keys of dictionary-like container types. I might revisit this option of supporting possible representations of null for dictionary keys, only if I see some feature requests in that direction.

    (That doesn't seem to be something which bothered many people in .NET development involving JSON processing anyway, so... I was maybe just overthinking it because of implementor bias while coding it; 'often better to step back, and stay pragmatic w.r.t. what people actually/usually need to code on whatever platform they use - especially when that saves you some work! ;)

  • BenDodsonBenDodson Ben Dodson USMember ✭✭

    Has anyone done an analysis of which libraries are leanest in terms of garbage generation? I seem to get more GCs using ServiceStack.Text than Json.Net. This makes sense to me, since ServiceStack's parsing is all string-based and not stream-based.

  • YSharpYSharp Cyril Jandia USMember ✭✭
    edited April 25

    @ BenDodson‌ :

    Hi Ben,

    I may/must have missed those, but I haven't found much on that specific topic alone. As for testing my own implementation and getting a gist of how it performs there, I've used that code pattern between lines #341 and #353 which does seem to give consistent results from one test + run to the others (they seem stable anyway).

    See for instance what Sami got reported, for two significantly different test cases (i.e., different input JSON shapes) :

    here and there.

    However, I suspect it's likely only approximate (or biased) to do it that way from within the app tier itself, as compared to measures collected using a .NET profiler worth of the name, and with more thorough test cases. Here, I still haven't taken the time to look into that seriously, although I am curious about it, too.

    I definitely agree that's the sort of interesting info that'll be most welcome to have, for some use cases.

    'HTH,

2»
Sign In or Register to comment.