C# Deserves A Better Message Passing API

The structured programming model of C# is decisive and straightforward. There is generally one or only a handful of idiomatically correct ways to facilitate a particular design need. In the case of API design, the language’s conception of access modifiers is fundamental. Exposures of behaviors to client code are controlled by applying public, private, or protected modifiers to individual methods. While this basic security mechanism, does allow us to build cleaner APIs, it does not strictly limit our ability to reach inside objects and work with their internal private methods. We can both interrogate and mutate the internal state of objects with no public API whatsoever.

In this post I’m going to show you how use reflection to add a feature to C# that mimics the simple message passing API that I like in Ruby. With this in place, we’ll be able to run any private method on any instance object with a more straightforward syntax.

First, we’ll give System.Object the ability to receive messages via an Extension Method called send.

namespace MessagePassingExtensions
{
    public static class Extensions
    {
        //PUBLIC API
        public static bool respond_to(this object o, string responder, params object[] paramSoak)
        {
            return respond_by(o, responder, paramSoak) != null;
        }
 
        public static object send(this object o, string responder, params object[] paramSoak)
        {
            if (!o.respond_to(responder, paramSoak))
            {
                var reason = string.Format("This object does not respond to messages sent to: {0} with parameters: {1}",
                                           responder,
                                           string.Join(", ", paramSoak));
                throw new MissingMethodException(reason);
            }
            var result = o.respond_by(responder, paramSoak).InvokeMember(responder, send_flags, null, o, paramSoak);
            return result;
        }
 
        //PRIVATE HELPERS
        private const BindingFlags send_flags = BindingFlags.InvokeMethod | BindingFlags.Instance |
                                                BindingFlags.Public | BindingFlags.NonPublic;
 
        private static Type respond_by(this object o, string responder, params object[] paramSoak)
        {
            var class_lookup_path = o.GetType().GetBaseTypes().Reverse().Concat(new[] { o.GetType() }).Reverse().ToList();
            return class_lookup_path.FirstOrDefault(t => t.GetMethods(send_flags)
                                                          .Any(m => m.Name == responder &&
                                                                    m.GetParameters().Count() == paramSoak.Count() &&
                                                                    m.GetParameters().All(p => paramSoak[p.Position].GetType().IsAssignableFrom(p.ParameterType))));
        }
 
        private static IEnumerable<Type> GetBaseTypes(this Type type)
        {
            if (type.BaseType == null) return type.GetInterfaces();
            return Enumerable.Repeat(type.BaseType, 1)
                             .Concat(type.GetInterfaces())
                             .Concat(type.GetInterfaces().SelectMany<Type, Type>(GetBaseTypes))
                             .Concat(type.BaseType.GetBaseTypes())
                             .Distinct();
        }
    }

This gives the API we need. Note that there are only 2 public methods added to the builtin object type:

respond_to() – allows you to ask an instance if it can respond to a particular message.

send() – allows you to send an instance a message.

Now let’s create a class that we can use as our test subject. We’ll use domain of the animal kingdom. First we need a base Animal:

public class Animal
{
    private bool Eat()
    {
        return true;
    }
}

Note that there is only a private method in here, an Eat() behavior. This symbol will not be visible to subclasses nor will it be available via the public API. Next, let’s derive a specific animal by inheriting from this base class. Say in this case, a Duck:

public class Duck: Animal
{
    private string Quack(int times)
    {
        var noises = new List<string>();
        for (var i = 0; i < times; i++)
        {
            noises.Add(MethodBase.GetCurrentMethod().Name);
        }
        return string.Join(" ", noises);
    }
 
    private string Quack()
    {
        return Quack(1);
    }
}

Note that Duck is an Animal, and that it only has a few private methods. It cannot see Animal.Eat(). Now, let’s test it out. The following Test class exercises the implementation of Object.send() and demonstrates that we’re able to run all of the private methods that we just created.

[TestClass]
public class SendTests
{
    [TestMethod]
    public void can_resolve_correct_signature_with_params()
    {
        var duck = new Duck();
        var result = duck.send("Quack", 2);
        Assert.AreEqual(result, "Quack Quack");
    }
 
    [TestMethod]
    public void can_resolve_correct_signature_with_no_params()
    {
        var duck = new Duck();
        var result = duck.send("Quack");
        Assert.AreEqual(result, "Quack");
    }
 
    [TestMethod]
    public void can_respond_to_messages_from_a_base_class()
    {
        var duck = new Duck();
        var result = duck.send("Eat");
        Assert.AreEqual(result, true);
    }
 
    [TestMethod]
    [ExpectedException(typeof(MissingMethodException), "You sent a message that I could not respond to")]
    public void blows_up_if_sent_message_cannot_be_responded_to()
    {
        var duck = new Duck();
        var result = duck.send("Bark");
    }
}

All of these tests pass. They are self-explanatory. I think that the most interesting one is can_respond_to_messages_from_a_base_class(), which shows that we can also get and run a private method from the underlying Animal class. By sending a message to a symbol on an instance object, we’re able to program against it in a looser style. This opens up a number of interesting ways to work with an object. Suppose for example you wanted to build an XML-driven execution system. By simply reading in instructions from an external xml source, we can emit and drive code execution that was not known at design time. Here’s an example of a declarative approach to driving execution.

DuckDriver.xml:


<?xml version="1.0" encoding="UTF-8" ?>
<Obj type="DynamicSpikes.Tests.Duck">
  <Run method="Quack" times="3" />
</Obj>

Given the above XML File, I can process and verify it like this:

[TestClass]
[DeploymentItem(@".\Assets\DuckDriver.xml")]
public class XmlDeclarativeTests
{
    [TestMethod]
    public void can_create_and_run_from_xml()
    {
        var xml = XDocument.Load(@".\DuckDriver.xml");
 
        var objects = xml.Root.Descendants().Where(elem => elem.Name.LocalName == "Obj").ToList();
        objects.ForEach(objElem =>
            {
                var runs = objElem.Descendants().Where(elem => elem.Name.LocalName == "Run").ToList();
                runs.ForEach(runElem =>
                    {
                        var type = Type.GetType(objElem.Attribute("type").Value);
                        var method = runElem.Attributes().Where(attr => attr.Name == "method").FirstOrDefault().Value;
                        var parameters = runElem.Attributes().Where(attr => attr.Name != "method").Select(attr => attr.Value).Cast<object>().ToArray();
 
                        var obj = Activator.CreateInstance(type);
                        var result = obj.send(method, parameters);
                        Assert.AreEqual(result, "Quack Quack Quack"); //<Run method="Quack" times="3" />
                    });
            });
    }
}

The test reads in the XML File, finds the Obj elements, finds the Run elements, and grabs the method name and parameters from the attributes. With this information, we have all we need to create the object and send it the message. This test passes as expected.

The ability to turn a markup into runnable code is not new, but the ease with which it can be done with the nuts and bolts of the C# language are impressive. The fact that we can easily marshal declarative instructions into an unknown but completely runnable instruction set is more than just a little bit awesome.

Namaste…

Introducing X DSL, A More Fluent XML Builder

A while back, I posted about using DynamicObjects to facilitate building Domain Specific Languages in C#. To the extent that we have to play within the syntactic sandbox that the language itself requires, we are still able to take advantage of what is available in the way of built in operators by changing their meaning. Specifically, with the X DSL, I’ve combined some simple operators and a dynamic chaining API in order to make a code syntax that is more representative of the XML we want it to build for us. Here is an example:

Suppose we want to build this XML Document:

<Customers>
  <Customer category="Foo" type="Bar">
    <Address>
      <Street>123 Sunny Drive</Street>
      <City>Austin</City>
      <State>TX</State>
    </Address>
    <Profile CreatedOn="2013-05-13T18:44:27.39154-05:00">
      <Theme>Aristo</Theme>
      <DefaultPage>http://www.google.com</DefaultPage>
    </Profile>
  </Customer>
</Customers>

With X, we would use code that looks like this to create it:

var x = X._.Customers()
            > X._.Customer(category: "Foo", type: "Bar")
                > X._.Address()
                    > X._.Street("123 Sunny Drive")
                    + X._.City("Austin")
                    + X._.State("TX")
                < X._.Profile(CreatedOn: DateTime.Now)
                    > X._.Theme("Aristo")
                    + X._.DefaultPage("http://www.google.com");

The goal with X, is to make the code structurally and semantically similar to that which its instructions create. Take a close look. You’ll note a few interesting characteristics. First, I’m using X._ to represent the pipeline continuations. This allows me to make operations over an unknown number of chained future resources as the expression continues. There are a number of novel concepts that this approach yields up. First, we’re able to create a pretty complex structural recipe in a single expression. This is effectively a single line of code. Next, I’ve selected symbols for the operations that are indicative of directional navigation through immerging hierarchal and sibling relations. Specifically, the dialect has the following:

> means “step down and create what follows”

< means “step up and create what follows”

+ means “stay at the current level and create what follows”

These operation constructs enable navigation and creation simultaneously. This takes care of our need to create Xml nodes and their inner texts. However, what about attributes? Well, you can see that those too are enabled, by leveraging C#’s named parameters capability.

Thus, when we say:

X._.Customer(category: “Foo”)

We get:

<Customer category=Foo”>

Pretty nice.

This elegant abuse of the language can get us closer to code/output isomorphism, wherein the code resembles that which it is creating (at least more closely than the builtin APIs .NET provides for this kind of thing). It was inspired by the Excellent Builder library in Ruby. However, there are notable differences in how operators are used here to provide the Xml as the output of a composite expression, rather than a statement chain, as it works with Builder. In any case, I believe there is a compelling case to close the representational gap between data and code. If there’s anything that Lisp can teach us it’s that data and code should be thought of as the same thing. The closer we can get to that ideal the simpler, and more powerful our tools become in allowing us to express our intent and have the code just carry it out.

The implementation of X is here and below for anyone wishing to play around with it:


public class X : DynamicObject
{
    List<XElement> xsiblings = new List<XElement>();
    XElement xwrapper;
    XElement xplace;

    Dictionary<string, Func<X>> customOperators = new Dictionary<string, Func<X>>();

    private Func<X> Down
    {
        get { return () => new X(xplace) { xplace = xplace }; }
    }

    private Func<X> Up
    {
        get
        {
            var parent = xplace.Parent ?? xplace;
            parent = parent.Parent ?? parent;
            return () => new X(parent) { xplace = xplace.Parent };
        }
    }

    private Func<X> Self
    {
        get { return () => new X(this);  }
    }

    public static dynamic _{ get { return New; } }

    public static dynamic New
    {
        get
        {
            Func<dynamic> exp = () =>
            {
                var b = new X();
                return (dynamic)b;
            };
            return exp.Invoke();
        }
    }

    protected void init()
    {
        customOperators.Add(">", () => Down());
        customOperators.Add("<", () => Up());
        customOperators.Add("+", () => Self());
        customOperators.Add("_", () => _);
    }

    public X(){
        xwrapper = new XElement("Wrapper");
        xplace = xwrapper;
        init();
    }

    public X(XElement xelem)
    {
        xwrapper = xelem;
        xplace = xwrapper.Descendants().LastOrDefault() ?? xwrapper;
        init();
    }

    public X(X builder)
    {
        xwrapper = builder.xwrapper;
        xplace = xwrapper.Descendants().LastOrDefault() ?? xwrapper;
        init();
    }

    public X(X builder1, X builder2)
    {
        builder1.xplace.Add(builder2.xwrapper);
        builder2.xplace = builder1.xplace;
        init();
    }

    public override string ToString()
    {
        var sb = new StringBuilder();
        var content = xwrapper.Descendants();
        foreach (var c in content)
            sb.Append(c.ToString());
        return sb.ToString();
    }

    public static X operator +(X c1, X c2)
    {
        c1.xwrapper.Add(c2.xwrapper.Descendants());
        return c1;
    }

    public static X operator >(X c1, X c2)
    {
        var xlast = c1.xplace.Descendants().LastOrDefault();
        xlast = xlast ?? c1.xplace;
        xlast.Add(c2.xwrapper.Descendants());
        return c1;
    }

    public static X operator <(X c1, X c2)
    {
        var xlast = c1.xplace.Parent.Descendants().LastOrDefault();
        xlast = xlast ?? c1.xplace;
        var parent = xlast.Parent ?? xlast;
        parent = parent.Parent ?? parent;
        parent.Add(c2.xwrapper.Descendants());
        return c1;
    }

    public override bool TryInvoke(InvokeBinder binder, object[] args, out object result)
    {
            return base.TryInvoke(binder, args, out result);
    }

    public override bool TryInvokeMember(InvokeMemberBinder binder, object[] args, out object result)
    {
        result = default(object);

        if (customOperators.ContainsKey(binder.Name))
            result = customOperators[binder.Name]();
        else //build with this call
        {
            var work = new Func<object>(() =>
            {
                var diff = args.Count() - binder.CallInfo.ArgumentNames.Count;
                var innerText = string.Empty;
                var argQueue = new Queue<object>(args);
                if (diff > 0)
                    diff.Times(i => innerText += argQueue.Dequeue().ToString());

                args = argQueue.ToArray();
                var parameters = new Dictionary<string, object>();
                binder.CallInfo.ArgumentNames.Count().Times(i =>
                {
                    parameters.Add(binder.CallInfo.ArgumentNames[i], args[i]);
                });

                XElement xelem = new XElement(binder.Name,
                    parameters.ToList().Select(param => new XAttribute(param.Key, param.Value)));
                xelem.Add(innerText);
                xplace = xelem;
                return xelem;
            });
            var output = work.DynamicInvoke() as XElement;
            xwrapper.Add(output);
            result = this;
        }
        return true;
    }

    public X this[string key]
    {
        get { return customOperators[key](); }
    }
}

Namaste…