For the last couple of weeks I have been working with client-side frameworks such as Kendo UI and Sencha ExtJS a lot. One of the requirements of my clients was server-side filtering, grouping and ordering of several grids and other components. The challenge here is that you don’t know exactly what information might be coming in: which field do you have to filter or sort, which direction, what operator do you have to use, etc. There is only one proper way to do this in C#: Expressions.
Update 2020-06-01: the library on which this post is based on is now available on Github.
At some point, every developer will use the incredible capabilities of LINQ. If you look under the bonnet, you’ll notice all of this is possible thanks to delegates and expressions.
The difference between expressions (Expression) and delegates (Func) is explained really well in this StackOverflow article. I’d like to simplify this by saying that Func is just a regular delegate whereas Expression is some kind of wrapper around the delegate with added metadata. Of course, there’s SO much more to it but that’s out of scope of this article.
With just one line of code, you can query a list with a complex filter – with the list being any kind of collection such as a database table or just a plain IEnumerable collection:
This kind of code works well if your queries are static and are unlikely to change. However, when queries are dynamic and not known upfront, you’ll have to get rid of this approach and go a different way, that is you’ll have to make it generic and reusable. As far as I can work out, expressions are the only way to go. In fact, when you look at the definition of the Where extension method, you’ll see it expects a delegate:
Knowing this you should not be surprised that the following code snippet is perfectly valid:
For every item in the collection, the where clause delegate will be executed. You can see this happening if you set a breakpoint on the delegate. This is all well and good, but this is still a static query. Enter PredicateBuilder!
Predicate Builder
One step forward to making dynamic queries is by using libraries such as PredicateBuilder. This library allows you to generate ‘semi-dynamic’ queries. To be more accurate: it also you to combine expressions using the And and Or extension methods.
Let’s take this original example:
The same result can be achieved with the PredicateBuilder:
The important thing to notice is how the PredicateBuilder brings it all together. You can define your queries independently from each other, after which you can tie them together using the AND or OR extension methods. This is incredibly nice if you want to get rid of these kind of constructions:
There’s just one problem with this: the filters are still static. You still have to write what you want to query. Especially when working in web applications, maintainability can be a serious issue as this requires manual synchronization between client side filters and server side filters. And this will undoubtedly cause bugs.
Therefore: enter (advanced) expressions!
Expressions to the rescue
In cases where you don’t know which field will be ordered or which field will be filtered by which value by the user in the browser, you’ll have to build this query up from the ground. To give you an idea of, here’s an example of a Web API that is used by the client-side frameworks:
The magic that needs to happen here is to convert and map all of these parameters into data structures that ORMs and subsequently databases can understand.
The first step is to hydrate the stringified JSON parameters, which they often are in the beforementioned frameworks. Data from the sort, group and filter arguments are formatted in JSON. I use the handy JSON.NET NuGet package to deserialize this information into CLR objects:
With this information, we can start creating the query. We know which property is targeted, what value is passed as the argument and how to compare the value to the property. The following code will do all of this, let me guide you through it.
In a filter expression, there are three variables: the property, the operator and the value . Using reflection we look for the property on the generic type and then an expression is created for that property with the correct operator and corresponding value.
Here is the code explained step by step:
1) ln 3: First you start of by creating the return value type: Expression.
2) ln 4: Next, you setup the left-hand side of the lambda expression: (x) =>. In many cases, one parameter will do but it could be possible that you want to define multiple parameter. So far this has been boiler plate code.
3) ln 5: Now the property of the generic T type is targeted: (x) => x.PROPERTY. If you have complex navigation properties (properties of properties, etc.), you can define a MemberExpression of a MemberExpression, which will result in (x) => x.PROPERTY.PROPERTYOFPROPERTY.
4) ln 6 to ln 11: The following lines will create the value to compare to. The TypeConverter will ensure that ‘apples’ can be compared to ‘apples’ (int to int, string to string, …) as the filter value is populated as a string in the filter object.
5) ln 13 to the end: At this point we have all components ready to be tied together. Depending on the type of operation, you’ll want to use GreaterThanOrEquals, Equals, NotEquals, etc… Add your logic here and then set that value to the Expression variable.
The last thing to do here is to convert the expression to a lambda:
Now you’re done, you’ve successfully created a dynamic query. The lambda variable can now be used anywhere where you would use lambda expressions otherwise.
Final words
Keep in mind that not all LINQ providers are equally powerful. For instance,the capabilities of Entity Framework are limited (although still impressive), so you cannot execute complex algorithms in the runtime of Entity Framework.
The same kind of code is perfectly valid for sorting and grouping, but there’s less complexity involved in the process as you don’t have to define operators and values. I really love to work with expressions, it’s very flexible and it allows you to generate code on the fly, which I think is amazing.
