Performance Stub: getting all subtype items from a list

Performance Stubs

These blog posts are simply times I wanted to identify the fastest method for accomplishing a particular goal. As I write the code, I like to make some light notes of alternatives while driving forward with the first implementation that makes it from my brain to my fingers.

When I get the chance, I can go back and flesh out the two versions and drop them into some basic Stopwatch timing to determine which is better in terms of raw speed. Factoring those results with clarity of code, I have a method I will likely choose the next time I need the same feature.

Goal

Given a particular IEnumerable, find all the elements that are of a particular type (subclass, in this case).

Method 1: Select As, Where Not Null

The as operator with convert between two types with one nice tweak. If the cast cannot happen, it results in null.

In this version, we massage the list by hitting all items with an as cast and then filter out the ones that didn’t cast successfully.

public static IEnumerable<SubSomething> SelectAsWhereNotNull(IEnumerable<Something> source) {
    return source.Select(item => item as SubSomething).Where(item => item != null);
}

Method 2: Where Is, Cast

There is also the is operator. Instead of returning a cast result, it simply returns a boolean for whether the instance can be cast as a given type. From there, we use another Linq extension method: Cast<TResult>.

In this version, we filter the original list to just those that can make the cast successfully, then Cast those items to the final desired type.

public static IEnumerable<SubSomething> WhereIsCast(IEnumerable<Something> source) {
    return source.Where(item => item is SubSomething).Cast<SubSomething>();
}

Test

I built up a list (n=1,000,000) of parent type Something but make every other item a SubSomething (which is a subclass of Something). Then, I run that list through both methods 1000 iterations each.

Results

Clarity

Neither method is particularly difficult to decipher. The Select/Where method takes a few more characters, but not enough to hurt.

Speed

It takes quite a list for these tests to even register with average ticks. That said, there is little gain to be had here over small lists. Once list lengths are sufficiently large, method 1 took the lead.

For n=1,000,000 and 1000 iterations (run on 2.4GHz AMD Phenom 9750 with 8GB RAM)
Average Method
1.0 ticks {IEnumerable<Something>}.Select(item => item as SubSomething).Where(item => item != null)
1.7 ticks {IEnumerable<Something>}.Where(item => item is SubSomething).Cast<SubSomething>()

Performance Testing Framework

The original source for the test framework code evolved from a StackOverflow answer about converting byte arrays to hexadecimal strings. –That code is available in a bitbucket repo. I have adapted it here for these two methods.– That code is available in the framework repo.

Rather than post another repo for each test, I am going to abstract out a testing framework for these posts. That repo will show up soon and I will update this post accordingly.

Update 2013-01-22

I finally refined the testing framework code I was using. While I won’t pretend this code couldn’t use some improvements, feel free to hack away with it. The code is available on GitHub.

To add a new case to an existing test:

  1. Add the new static method (Func<byte[], string>) to /Tests/ConvertByteArrayToHexString/Test.cs.
  2. Add that method’s name to the TestCandidates return value in that same class.
  3. Make sure you are running the input version you want, sentence or text, by toggling the comments in GenerateTestInput in that same class.
  4. Hit F5 and wait for the output (an HTML dump is also generated in the /bin folder).

To add a new test fixture entirely, just write up something that implements IPerformanceTest in that project’s namespace.

Getting dynamic ExpandoObject to serialize to JSON as expected

Serializing ExpandoObjects

I am currently creating a JSON API for a handful of upcoming Sierra Trading Post projects. When I found myself generating JSON for a stripped-down representation of a number of domain classes, all wrapped with some metadata, I turned to dynamic and things have been going quite well. Unfortunately, there was a hurdle to getting the JSON to look the way I wanted.

If you start playing around with serializing ExpandoObject to JSON, you will probably start finding a number of options. The easiest solution to find is the one that comes with .NET, JavaScriptSerializer under System.Web.Script.Serialization. It will happily serialize an ExpandoObject for you, but it probably won’t look the way you expect. Your searches will probably vary, but I found Newtonsoft’s JSON.NET, which handled ExpandoObject right out of the NuGet box. Then I stumbled on ServiceStack.Text (also “NuGettable”). While it does even weirder things than the .NET serializer with ExpandoObjects, it supposedly does them very fast.

Test code

dynamic test = new ExpandoObject();
test.x = "xvalue";
test.y = DateTime.Now;

BCL JavaScriptSerializer (using System.Web.Script.Serialization;)

JavaScriptSerializer javaScriptSerializer = new JavaScriptSerializer();
string jsonOfTest = javaScriptSerializer.Serialize(test);
// [{"Key":"x","Value":"xvalue"},{"Key":"y","Value":"\/Date(1314108923000)\/"}]

Not quite what I was looking for but it makes sense if you realize that ExpandoObject plays very nicely with IDictionary<string, object>. By using some code borrowed from StackOverflow (not the accepted answer, but I like it) and theburningmonk.com, you put together a custom serializer for ExpandoObject and you can get something more typical of what went into assembling the object.

/// <summary>
/// Allows JSON serialization of Expando objects into expected results (e.g., "x: 1, y: 2") instead of the default dictionary serialization.
/// </summary>
public class ExpandoJsonConverter : JavaScriptConverter {
    public override object Deserialize(IDictionary<string, object> dictionary, Type type, JavaScriptSerializer serializer) {
        // See source code link for this extension method at the bottom of this post (/Helpers/IDictionaryExtensions.cs)
        return dictionary.ToExpando();
    }
    public override IDictionary<string, object> Serialize(object obj, JavaScriptSerializer serializer) {
        var result = new Dictionary<string, object>();
        var dictionary = obj as IDictionary<string, object>;
        foreach (var item in dictionary)
            result.Add(item.Key, item.Value);
        return result;
    }
    public override IEnumerable<Type> SupportedTypes {
        get {
            return new ReadOnlyCollection<Type>(new Type[] { typeof(ExpandoObject) });
        }
    }
}

JavaScriptSerializer javaScriptSerializer = new JavaScriptSerializer();
javaScriptSerializer.RegisterConverters(new JavaScriptConverter[] { new ExpandoJsonConverter() });
jsonOfTest = javaScriptSerializer.Serialize(test);
// {"x":"xvalue","y":"\/Date(1314108923000)\/"}

Newtonsoft Json.NET

string jsonOfTest = Newtonsoft.Json.JsonConvert.SerializeObject(test);
// {"x":"xvalue","y":"\/Date(1314108923000-0600)\/"}

That worked exactly as I expected. If I can get JSON.NET to work consuming JSON under MonoTouch, it will make my life quite easy; more to come on that.

ServiceStack.Text

string jsonOfTest = JsonSerializer.SerializeToString(test);
// ["[x, xvalue]","[y, 8/23/2011 08:15:23 AM]"]

ServiceStack’s JSON serialization system does something similar to the .NET JavaScriptSerializer, but not quite the same. I haven’t spent enough time with ServiceStack to know how this syntax will work out on consumption by another deserializing system, but I suspect this may be something that only ServiceStack would handle correctly.

Unfortunately, the author of the project was nice enough to confirm that ServiceStack.Text does not currently afford the same extensibility as the .NET JavaScriptSerializer for overriding its default behavior in this situation. He did welcome a pull request which I will look into.

ServiceStack.Text also doesn’t appear to support deserializing into an ExpandoObject as this resulted in an empty object.

dynamic testDeserialization = ServiceStack.Text.JsonSerializer.DeserializeFromString<ExpandoObject>(jsonOfTest);

I haven’t confirmed if ServiceStack.Text deserializing works under MonoTouch yet. If it does, it would be worthwhile to have it running API JSON generation as well as the client-side JSON consumption since there is evidence it performs quite nicely.

UPDATE

I slapped together a new MonoTouch project in MonoDevelop and tossed in ServiceStack.Text’s DLLs with a few bits of code and confirmed it works great for a deserializing JSON into a pre-defined object.

public class TestClass {
    public int x { get; set; }
    public string x { get; set; }
}
TestClass result = ServiceStack.Text.JsonSerializer.DeserializeFromString<TestClass>("{\"x\":3,\"y\":\"something\"}");
Console.WriteLine("result: x={0}, y={1}", result.x, result.y);
// result: 3, something

UPDATE (2012-04-05)

I missed a blog entry on the author’s blog describing how to get Json.NET to output DateTime in different ways. For example, if you prefer the ISO-8601 output, you would be able to tell it to use the IsoDateTimeConverter. When I went to update the test project, the latest version of Json.NET (4.5 Release 1) now defaults to ISO-8601. Since I don’t want to risk break an existing API, I tweaked the DateFormatHandling to make the output match the old default. Examples are hard to find since this is such a new release, so I slapped one together and submitted it to the Json.NET docs (now on GitHub).

Newtonsoft.Json.JsonSerializerSettings settingsWithMicrosoftDateFormat = new Newtonsoft.Json.JsonSerializerSettings() { DateFormatHandling = Newtonsoft.Json.DateFormatHandling.MicrosoftDateFormat };
string jsonOfTest = Newtonsoft.Json.JsonConvert.SerializeObject(test, settingsWithMicrosoftDateFormat);
// {"x":"xvalue","y":"1333640263042-0600"}

Unfortunately, even Json.NET 4.5 is appending a timezone offset to the DateTime serialization that isn’t found in the .NET implementation. I’ll look into a custom implementation of DateTimeConverterBase and I have posted this as a question on StackOverflow.

Additional Notes

I haven’t played much with what problems may arise with the various representations of DateTime objects on the consumption side, but they definitely all handled it differently here.

Source Code

To get all the code in an ASP.NET MVC project (download, load solution, hit F5 [I hope]), check out the bitbucket.org repository for this post.

Subtleties with using Url.RouteUrl to get fully-qualified URLs

At some point I missed the Url.RouteUrl overload that took a protocol and returned an absolute URL based on the current context. It is quite handy when you are sending URLs out into the world (e.g., RSS feed link). I ended up using the less-handy overload that took an explicit host (same as the current, in this case) and passing it in. When someone pointed out the simpler overload, I did the obvious and deleted the host from the call. That didn’t quite work.

For those looking for a way to get a fully-qualified URL through the route system, the less-than-obvious answer is to call the overload for Url.RouteUrl that gets a URL with a different protocol (Url.RouteUrl(string, object, string)), passing in Request.Url.Scheme for the current protocol.

Url.RouteUrl("Default", new { action = "Index", controller = "Department", id = 1 }, Request.Url.Scheme);
// http://www.currentdomain.com/department/index/1

Say you want to send someone to a different subdomain in your app while using the same routes. There’s an overload for that: Url.RouteUrl(string, RouteValueDictionary, string, string). Combined with the above example, here’s how these all play out if you are currently handling a www.currentdomain.com request and your route table includes the fallback default ({controller}/{action}/{id}).

Url.RouteUrl("Default", new { action = "Index", controller = "Department", id = 1 });
// /department/index/1
Url.RouteUrl("Default", new { action = "Index", controller = "Department", id = 1 }, Request.Url.Scheme);
// http://www.currentdomain.com/department/index/1
Url.RouteUrl("Default", new RouteValueDictionary(new { action = "Index", controller = "Department", id = 1 }), "http", "sub.currentdomain.com");
// http://sub.currentdomain.com/department/index/1

Now if you switch between the two fully-qualified calls, you may try just deleting or adding the hostName parameter, respectively. One direction is a compile error, and one direction is a runtime oddity resulting in a hideous URL.

Url.RouteUrl("Default", new { action = "Index", controller = "Department", id = 1 }, "http", "sub.currentdomain.com");
// Compile error (expects a RouteValueDictionary)
Url.RouteUrl("Default", new RouteValueDictionary(new { action = "Index", controller = "Department", id = 1 }), "http", "sub.currentdomain.com");
// Eye-bleeding and incorrect route as it "serializes" the RouteValueDictionary.
// In my case, I ended up with something like this:
// http://www.currentdomain.com/current/route/1/?Count=3&Keys=System.Collections.Generic.Dictionary%602%2BKeyCollection%5BSystem.String%2CSystem.Object%5D&Values=System.Collections.Generic.Dictionary%602%2BValueCollection%5BSystem.String%2CSystem.Object%5D

On a side note, if your development environment uses localhost with a port and you use some web.config app setting for that URL change between development and production (“localhost:12345” vs “www.currentdomain.com”). You will want your host setting to be without the port. Url.RouteUrl will hiccup on your development environment if the port is part of the host name (it’s no longer just the host at that point).

Url.RouteUrl("Default", new RouteValueDictionary(new { action = "Index", controller = "Department", id = 1 }), "http", ConfigurationManager.AppSettings["hostwithport"]);
// http://localhost:12345:12345/department/index/1