NTLM Authentication in Java with JCifs

In enterprise software development contexts, one of the frequent needs we encounter is working with FileSystems remotely via CIFS, sometimes referred to as SMB.  If you are using Java in these cases, you’ll want JCifs, a pure Java CIFS implementation.  In this post, I’ll show you how to remotely connect to a Windows share on an Active Directory domain and read/write a file.

In your pom.xml place this dependency:

<dependency>
    <groupId>jcifs</groupId>
    <artifactId>jcifs</artifactId>
    <version>1.3.17</version>
</dependency>

Here is a simple class with a main, you can run to see how it works:

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.UnknownHostException;
import java.util.logging.Level;
import java.util.logging.Logger;

import jcifs.UniAddress;
import jcifs.smb.NtlmPasswordAuthentication;
import jcifs.smb.SmbException;
import jcifs.smb.SmbFile;
import jcifs.smb.SmbFileInputStream;
import jcifs.smb.SmbFileOutputStream;
import jcifs.smb.SmbSession;

public class Program {

	public static void main(String[] args) throws SmbException, UnknownHostException, Exception {
	    
        final UniAddress domainController = UniAddress.getByName("DOMAINCONTROLLERHOSTNAME");
		 
	    final NtlmPasswordAuthentication credentials = new NtlmPasswordAuthentication("DOMAIN.LOCAL", "USERNAME", "SECRET");
	 
	    SmbSession.logon(domainController, credentials);
	    
	    SmbFile smbFile = new SmbFile("smb://localhost/share/foo.txt", credentials);
	    
	    //write to file
	    new SmbFileOutputStream(smbFile).write("testing....and writing to a file".getBytes());
	    
	    //read from file
	    String contents = readFileContents(smbFile);
	    
	    System.out.println(contents);

	}

	private static String readFileContents(SmbFile sFile) throws IOException {

		BufferedReader reader = reader = new BufferedReader(
				new InputStreamReader(new SmbFileInputStream(sFile)));

		StringBuilder builder = new StringBuilder();
		
		String lineReader = null;
		while ((lineReader = reader.readLine()) != null) {
			builder.append(lineReader).append("\n");
		}
		return builder.toString();
	}

}

As you can see its quite trivial to reach out across your network and interact with Files and Directories in Windows/Samba Shares. Being able to authenticate via NTLM is convenient and tidy for this purpose, not to mention the FileSystem API is straight forward and powerful.

Enjoy the power..

Meditation on JavaScript’s SetTimeout(), Recursion, and Execution Context

JavaScript’s setTimout and setInterval functions are for scheduling future work. They provide a way to schedule either a single or recurring execution of a function. In this article, we’ll play around with some fun and interesting capabilities that can be had using setTimeout.

To get going we’ll need a few examples in place.

Here is a a simple Calculator object that we can use for our testing. Note that it has 2 methods (add and multiply). It also contains a piece of state (value) that it uses to track its computational output.

//an object with methods
function Calculator (seed) {
    return {
          value: seed,
          add: function(b) {
            value = value || 0; //guard with default accumulator for addition
            this.value += b;
            console.log(this.value);
            return this;
          },
          multiply: function(b) {
            value = value || 1; //guard with default accumulator for multiplication
            this.value *= b;
            console.log(this.value);
            return this;
          }
    };
}
var calculator = new Calculator();

//basic math fluency
var twentyFive = calculator.add(1 + 1)
                           .add(3)
                           .multiply(5)
                           .value;

Now, let’s have some fun by giving all JavaScript objects in the runtime the ability to defer their own method execution until a time in the future. We do this by extending the Object prototype for the whole JavaScript runtime.

This implementation is as follows. Note that, that it imbues all objects with a new ability, to run future functions with themselves as the reciever.

if (typeof Object.prototype.later !== 'function') {
	Object.prototype.later = function (msec, method) {
		var that = this,
			args = Array.prototype.slice.apply(arguments, [2]);
		if (typeof method === 'string') {
			method = that[method];
		}
		setTimeout(function() {
			method.apply(that, args);
		}, msec);
		return that;
	};
}

Source: Douglas Crockford.

This in effect shims all objects with an additional behavior called “later”. It changes the receiver of the function call in the future to the object from which later() was invoked.

The function expects these things:

Parameter 1: The number of milliseconds to wait before executing

Parameter 2: Either a function or a string function name to execute

Additional Optional Parameters: The arguments for the method to be executed in the future

The later method takes in the parameters and schedules the function (parameter 2) with its arguments (parameters 3,4,5,…) to be executed in the specified milliseconds (parameter 1). Note that we use a closure to refer to the receiver (“this”) as “that” during the setup and scheduling phase of its execution. This ensures that the “this” inside of scope of execution is the object from which the .later() method is called. This overrides the pointer in setTimeout() for “this” from the default “global” JavaScript object to the object from which later() is invoked. This simple but elegant replacing of the execution context object is perhaps one of the most powerful features of JavaScript. Now that we’ve got this extension to the Object prototype in place, all objects in our system, including Calculator will have this behavior.

So let’s have some fun with the calculator by exercising the Chaining API it inherited from Object. Here are some tricks taking advantage of the fluent later() method’s ability to drive a series of asynchronous calls over the object from which the expression chain originates. Since calculator’s prototype is Object, it can drive its own api. This example demonstrates several different ways to invoke the later() method in a single expression chain. Let’s turn a calculator into an adding machine.

calculator
	//can pass a method name as a string.  targeting calculator.add
	.later(100, "add", 1)

	//can pass a method reference directly from the object
	.later(1000, calculator.add, 1)

	//can pass an anonymous function.
	.later(100, function(argument) {
		this.value += argument;
	}, 1)

    //close back onto chain originator
    .later(999, function(argument) { calculator.value += argument; }, 1);

//start an async message loop
calculator.later(0, function loop(argument) {
    this.value += argument;
    console.log(this.value);
    this.later(0, loop, argument);
}, "Looping");

Now lets have some fun with arrays.  Since JavaScript arrays inherit their prototype from Object, and we extended Object’s prototype with the later() method, arrays now also have the later() method.

Process an Array Asychronously:

var sequence = [1,2,3,4,5];
sequence.forEach(function(n) {
    sequence.later(0, function(i) {
    	console.log("got: " + i);
    }, n);
});

This is cool..

Now lets replace the execution context object with a new Calculator.

Driving Execution Over an Array – Compute Factorial:

[1,2,3,4,5].forEach(function(n) {
    return this.multiply(n).value;
}, new Calculator());

Lets Add Laterness:

//scheduled execution
[1,2,3,4,5].forEach(function(n) {
    this.later(100, this.multiply, n);
}, new Calculator());

Decaying Scheduled Array Processing:

[1,2,3,4,5].forEach(function(n) {
    this.wait = (this.wait || 0) + 1000 ;
    this.later(this.wait, this.multiply, n);
}, new Calculator());

Randomly Scheduled Array Processing:

[1,2,3,4,5].forEach(function(n) {
    this.later(Math.floor((Math.random()*1000)+1), this.multiply, n);
}, new Calculator());

Namaste…

Making Parallax Animation Effects With JavaScript

The Term “Parallax” means a difference in the apparent position of an object viewed along two different lines of sight, and is measured by the angle of inclination between those two lines. The positional difference between objects creates a visual illusion that is specific to the position of the observer.  A simple everyday example of parallax can be seen in the dashboard of motor vehicles that use a needle-style speedometer gauge.  When viewed from directly in front, the speed may show exactly 60; but when viewed from the passenger seat the needle may appear to show a slightly different speed, due to the angle of viewing.  This effect can be exploited when presenting content to trick the eyes into seeing multiple forced perspectives in the same scene.  When animated, the effects become visually interesting to people. Recently, I began a series of experiments to learn how parallax works.  In this article, I’ll walk you through the basics and leave you with a working example of a parallax web banner.  The code in this writeup, is available here.

First, let’s layout what we want to accomplish.

  • Mountains (far texture) – We want to build a scene that uses a scrolling landscape to provide the feeling of panning or having it spin around you.  (Note that I’ve modified this image to seamlessly scroll.  Here’s the technique ref.)
  • Sun (back texture) – We want to have a sun fixed in the sky in a position similar to the direction that the light in the landscape is coming from.
  • Cloud (mid texture) – We want to have clouds moving across our sky.
  • Girl (close texture) – We want a central figure of a person (our girl) at the front to hold our users’ attention and give the illusion that she’s in the scene.

Preview Results:

Parallax In Action Screenshot
Screenshot

We’ll begin with these 4 images.  Each is considered a texture that will be layered onto our stage, which in this case will be an HTML5 Canvas. With these images in our project, we can now write the code to bring them together and animate the scene.

Textures
Texture Images To Be Used As Sprites

First, we’ll make our markup. This is a minimal HTML file, with only a canvas element that will serve as our rendering stage for the scene.

<html>
  <head>
    <link rel='stylesheet' type='text/css' href='https://cdnjs.cloudflare.com/ajax/libs/meyer-reset/2.0/reset.min.css' />
    <link rel='stylesheet' type='text/css' href='style.css' />
  </head>
  <body onload="init();">
  	<script src="pixi.js"></script>
    <script src="http://code.jquery.com/jquery-1.10.2.min.js"></script>
	  <script src="http://cdnjs.cloudflare.com/ajax/libs/underscore.js/1.4.4/underscore-min.js"></script>
    
    <script type='text/javascript' src='script.js'></script>

    <div id=container; align="center">
      <p id="caption">
        Its A Beautiful World
      </p>
      <canvas id="game-canvas" width="1024" height="512"></canvas>
    </div>
  </body>
</html>

Next, we’ll write the JavaScript to bring it to life.

function init(){

  var WIDTH = 1024;
  var HEIGHT = 512;
  var stage = new PIXI.Stage();

  // let pixi choose WebGL or canvas
  var renderer;
  var back, far, mid, close;

  // target render to something on dom
  renderer = PIXI.autoDetectRenderer(WIDTH, HEIGHT, document.getElementById("game-canvas"));

  //sun texture
  var backTexture = PIXI.Texture.fromImage("sun1.gif");
  back = new PIXI.Sprite(backTexture, WIDTH, HEIGHT);
  back.position.x = 20;
  back.position.y = 7;

  //mountain texture
  var farTexture = PIXI.Texture.fromImage("mountain-04.jpg");
  far = new PIXI.TilingSprite(farTexture, WIDTH, HEIGHT);
  far.position.x = 0;
  far.position.y = 0;
  far.tilePosition.x = 0;
  far.tilePosition.y = 0;

  //cloud texture
  var midTexture = PIXI.Texture.fromImage("cloud1.gif");
  mid = new PIXI.Sprite(midTexture, WIDTH, HEIGHT);
  mid.position.x = WIDTH - 40;
  mid.position.y = -10;

  //girl texture
  var closeTexture = PIXI.Texture.fromImage("girl_character.gif");
  close = new PIXI.Sprite(closeTexture, WIDTH, HEIGHT);
  close.position.x = 512 - 256;
  close.position.y = 15;

  //add textures to stage in order from back to front
  stage.addChild(far);
  stage.addChild(back);
  stage.addChild(mid);
  stage.addChild(close);

  //render stage
  renderer.render(stage);

  //start animation loop
  requestAnimFrame(update);

  //recursive animation looper
  function update() {

    //move the far sprite to the left slowly
    far.tilePosition.x -= 0.128;

    //move the mid sprite to the left a little faster
    mid.position.x -= 0.37;
    if (mid.position.x < 0 - 512)
      mid.position.x = WIDTH + 512;

    renderer.render(stage);

    requestAnimFrame(update);
  }
}

Note that I’ve commented each stanza in this script to help you understand what’s happening. The code flow is generally this:

  1. Set dimensions for the stage (HEIGHT/WIDTH variables)
  2. Instantiate the stage.
  3. Instantiate a renderer targeted to the canvas element in the DOM
  4. Create sprites for the textures. Note that the far texture is a TilingSprite, so it’s position is manipulated by using the tilePosition attribute instead of the position attribute like the regular sprites.
  5. Place the stage into the renderer.
  6. Then finally, we start the animation loop by feeding the recursive update callback to the requestAnimFrame function given to us by PIXI. A more thorough look at the update function is required:

    1. Since we want the far texture to scroll to the left slowly, we decrement by .128px its position on the x axis each time the update function is called.
    2. Since we want the mid texture to scroll to the left more quickly, we decrement by .37px (a larger number) its position on the x axis each time the update function is called.

When brought together, the effects can be visually interesting. I say interesting, because the effect can be pleasant or disorienting depending upon how you’ve positioned the sprites and how quickly they are moving. The basic approach that I’m showing here can be used as the foundation for side-scrolling video games. There are also many potential uses for Web Banners, Presentations, Data Visualization, Rich User Interfaces, and more.

Result:

Hopefully you find this useful, or if nothing else instructive. You can get my code here

Namaste..

XML Interoperability of Serialized Entities in Java and .NET

Abstract:

In order to exchange structured data directly between the platforms, we must be able to easily take the marshalled or serialized definition of the object and turn it into an object in memory.  There are standard ways of marshalling of objects to XML in both Java and .NET.  I have found it a little frustrating in the past when I’ve had to adopt large frameworks or external machinery in order to easily move structured data between the JVM and CLR.   It seems that we should be able to bring these worlds together in a simple set of OOTB idioms, while providing a convenient way (one liner) to move back and forth between object and stringified forms.   For this I have created a minimal helper class for both languages that does the following:

  • Provides a common API between languages for moving between XML string and Objects (entities)
  • Provides adaptation capabilities between canonical XML representations for both Java’s JAXB and .NET’s XmlSerializer
  • Provides a façade to the underlying language and framework mechanics for going between representations
  • Implementation of SerializationHelper.java
  • Implementation of SerializationHelper.cs

The Need for Interoperable Xml Representation of Entities in Java and .NET

Both the Java and .NET ecosystems provide many ways to work with XML, JSON, Binary, YAML, etc. serialization.  In this article I’m focused on the base case between the standard platform-level mechanisms for moving between XML and Object graphs in memory.  The Web Services stacks in both platforms are of course built on top of their respective XML binding or serialization standards.  The standards however differ, in some slight but important ways.  Here I do not seek to build a bullet proof general purpose adapter between languages.  I’ll leave that to the WS-* ppl.  However, I think there is a common and often overlooked ability to do marshalling with XML with little to no additional framework or specialized stack.  Here are some scenarios that make sense with this kind capability.

  • Intersystem Messaging
  • Transforming and Adapting Data Structures
  • Database stored and shared XML
  • Queue-based storage and shared XML
  • File-based storage and shared XML
  • Web Request/Response shared XML

The Specifications:

Java:

JAXB (Java XML Binding)

JSR: 222

.NET

XmlSerializer

Version >= .NET 2.0

First, we need to understand the default differences between the XML output by JAXB and XmlSerializer. To start we’ll create the same entity in both Java and C#. Then we can compare them.

The entity: DataObject

.NET Entity Class:

[Serializable]
public class DataObject
{
   public string Id { get; set; }
   public string Name { get; set; }
   public bool Processed { get; set; }
}

Java Entity Class:

public class DataObject implements Serializable {

	private String id;
	private String name;
	private boolean processed = false;

	public String getId() {
		return id;
	}

	public void setId(String id) {
		this.id = id;
	}

	public String getName() {
		return name;
	}

	public void setName(String name) {
		this.name = name;
	}

	public boolean isProcessed() {
		return processed;
	}

	public void setProcessed(boolean processed) {
		this.processed = processed;
	}
}

Java Entity XML:

<DataObject>
  <id>ea9b96a6-1f8a-4563-9a15-b1ec0ea1bc34</id>
  <name>blah</name>
  <processed>false</processed>
</DataObject>

.NET Entity XML:

<DataObject xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <Id>b3766011-a1ab-41bf-9ce2-8566fca5736f</Id>
  <Name>blah</Name>
  <Processed>false</Processed>
</DataObject>

The notable differences in the XML are these:

  • xsi and xsd namespaces are put in by .NET and not by Java
  • The casing of the element names are different.  In fact, they follow the style convention used to create the entity.  The property naming styles between the languages are as follows:
    • Java: CamelCase
    • .NET: PascalCase

Let’s have a look at how we can use a class called SerializationHelper to round-trip objects to xml and back objects. We want it to easily dehydrate (stringify) and rehydrate (objectify) data objects.

The implementation of this class in both Java and C# provides the following api:

String serialize(Object object)
Object deserialize(String str, Class klass)

This is useful for quickly reversing objects to XML and visaversa.

I’ll walk you through how to use it with some tests.

Round Tripping (Java Usage):

@Test
public void can_round_trip_a_pojo_to_xml() throws Exception
{
	SerializationHelper helper = new SerializationHelper();
	DataObject obj = buildDataObject();

	String strObj = helper.serialize(obj);

	DataObject obj2 = (DataObject) helper.deserialize(strObj, DataObject.class);

	Assert.isTrue(obj.getId().equals(obj2.getId()));
	Assert.isTrue(obj.getName().equals(obj2.getName()));

}

Round Tripping (C# Usage):

[TestMethod]
public void can_round_trip_a_poco_to_xml()
{
    SerializationHelper helper = new SerializationHelper();
    DataObject obj = BuildDataObject();

    string strObj = helper.serialize(obj);

    DataObject obj2 = (DataObject)helper.deserialize(strObj, typeof(DataObject));

    Assert.IsTrue(obj.Id.Equals(obj2.Id));
    Assert.IsTrue(obj.Name.Equals(obj2.Name));
}

No problem. A simple single line expression reverses the representation. Now lets see if we can move the stringified representations between runtimes to become objects.

Adapting .NET XML to Java (Java Usage):

@Test
public void can_materialize_an_object_in_java_from_net_xml() throws Exception
{
	SerializationHelper helper = new SerializationHelper();

	String netStrObj = Files.toString(new File("DOTNET_SERIALIZED_DATAOBJECT.XML"), Charsets.UTF_8);

	DataObject obj2 = (DataObject) helper.deserialize(netStrObj, DataObject.class);

	Assert.isTrue(obj2.getName().equals("blah"));
}

Behind the scenes here there is a StreamReaderDelegateunder the hood in the SerializationHelper that is intercepting the inbound XML and camel-casing the names before it attempts to bind them onto the DataObject instance directly.

SerializationHelper.java:

public class SerializationHelper {

	public String serialize(Object object) throws Exception{
		StringWriter resultWriter = new StringWriter();
		StreamResult result = new StreamResult( resultWriter );
		XMLStreamWriter xmlStreamWriter =
		           XMLOutputFactory.newInstance().createXMLStreamWriter(result);

		JAXBContext context = JAXBContext.newInstance(object.getClass());
		Marshaller marshaller = context.createMarshaller();
		marshaller.marshal(new JAXBElement(new QName(object.getClass().getSimpleName()), object.getClass(), object), xmlStreamWriter);

		String res = resultWriter.toString();
	    return res;
	}

	public Object deserialize(String str, Class klass) throws Exception{

        InputStream is = new ByteArrayInputStream(str.getBytes("UTF-8"));
        XMLStreamReader reader = XMLInputFactory.newInstance().createXMLStreamReader(is);
        reader = new CamelCaseTransfomingReaderDelegate(reader, klass);

		JAXBContext context = JAXBContext.newInstance(klass);
		Unmarshaller unmarshaller = context.createUnmarshaller();

		JAXBElement elem = unmarshaller.unmarshal(reader, klass);
		Object object = elem.getValue();

		return object;
	}

	//adapts to Java property naming style
	private static class CamelCaseTransfomingReaderDelegate extends StreamReaderDelegate {

		Class klass = null;

        public CamelCaseTransfomingReaderDelegate(XMLStreamReader xsr, Class klass) {
        	super(xsr);
        	this.klass = klass;
        }

        @Override
        public String getLocalName() {
            String nodeName = super.getLocalName();
            if (!nodeName.equals(klass.getSimpleName()))
            {
            	nodeName = nodeName.substring(0, 1).toLowerCase() +
            			   nodeName.substring(1, nodeName.length());
            }
            return nodeName.intern(); //NOTE: intern very important!..
        }
    }
}

Note the deserialize method is able to do just-in-time fixup of the property name xml nodes to ensure they meet the expection (a camelCased fieldname) of the default jaxb unmarshalling behavior.

Now to go from XML produced by the default JAXB serializer to .NET objects with the same api. To do this I’ll switch back to C# now.

Adapting Java XML to .NET (C# Usage):

[TestMethod]
public void can_materialize_an_object_in_net_from_java_xml()
{
    string javaStrObj = File.ReadAllText("JAVA_SERIALIZED_DATAOBJECT.XML");

    SerializationHelper helper = new SerializationHelper();

    DataObject obj2 = (DataObject)helper.deserialize(javaStrObj, typeof(DataObject));

    Assert.isTrue(obj2.getName().equals("blah"));
}

In this case, I’m using a custom XmlReader that adapts the XML from Java style property names to .NET style. The pattern in Java and .NET is roughly the same for adapting the XML into a consumable form. This is the convenience and power that using an intermediary stream reader gives you. It basically applies changes to the node names it needs to bind them to the correct property names. The nice thing is that this happens just-in-time, as the XML being deserialized into a local Object.

Here is the C# implementation of the same SerializationHelper api in .NET.

SerializationHelper.cs:

public class SerializationHelper
{

    public string serialize(object obj)
    {
        using (MemoryStream stream = new MemoryStream())
        {
            XmlSerializer xs = new XmlSerializer(obj.GetType());
            xs.Serialize(stream, obj);
            return Encoding.UTF8.GetString(stream.ToArray());
        }
    }

    public object deserialize(string serialized, Type type)
    {
        using (MemoryStream stream = new MemoryStream(Encoding.UTF8.GetBytes(serialized)))
        {
            using (var reader = new PascalCaseTransfomingReader(stream))
            {
                XmlSerializer xs = new XmlSerializer(type);
                return xs.Deserialize(reader);
            }
        }
    }

    private class PascalCaseTransfomingReader : XmlTextReader
    {
        public PascalCaseTransfomingReader(Stream input) : base(input) { }

        public override string this[string name]
        {
            get { return this[name, String.Empty]; }
        }

        public override string LocalName
        {
            get
            {
                // Capitalize first letter of elements and attributes.
                if (base.NodeType == XmlNodeType.Element ||
                    base.NodeType == XmlNodeType.EndElement ||
                    base.NodeType == XmlNodeType.Attribute)
                {
                    return base.NamespaceURI == "http://www.w3.org/2000/xmlns/" ?
                           base.LocalName : MakeFirstUpper(base.LocalName);
                }
                return base.LocalName;
            }
        }

        public override string Name
        {
            get
            {
                if (base.NamespaceURI == "http://www.w3.org/2000/xmlns/")
                    return base.Name;
                if (base.Name.IndexOf(":") == -1)
                    return MakeFirstUpper(base.Name);
                else
                {
                    // Turn local name into upper, not the prefix.
                    string name = base.Name.Substring(0, base.Name.IndexOf(":") + 1);
                    name += MakeFirstUpper(base.Name.Substring(base.Name.IndexOf(":") + 1));
                    return NameTable.Add(name);
                }
            }
        }

        private string MakeFirstUpper(string name)
        {
            if (name.Length == 0) return name;
            if (Char.IsUpper(name[0])) return name;
            if (name.Length == 1) return name.ToUpper();
            Char[] letters = name.ToCharArray();
            letters[0] = Char.ToUpper(letters[0]);
            return NameTable.Add(new string(letters));
        }

    }
}

I think it’s important to have a thorough understanding and good control of the basics of serialization. In some cases, we’re just consuming a serialized object from a message queue, a file, or a database. The ability to move entities between process and stack boundaries should be easy.

It should take only 1 line of code.

Dev flow with integrated SublimeREPL

Here is a short screencast that I made to demonstrate what I believe are some of the more useful features and techniques of working in Sublime Text 2 and the python repl.  Specifically, I wanted to show others who might need the dots connected to understand just what the intended usage flow of SublimeRepl is.

The following is covered:

  • Where to find your Sublime Text 2 keymap file.
    • How to add a keymapping
    • I’ll post my example below
  • Use the REPL to work with objects loaded from the open file buffer.
  • Use the built-in key mappings for transferring current file to the REPL.

KEYMAPPING CODE

{ "keys": ["f8"],
  "command": "repl_open",
  "caption": "Python",
  "mnemonic": "p",
  "args": {
              "type": "subprocess",
              "encoding": "utf8",
              "cmd": ["python", "-i", "-u", "$file"],
              "cwd": "$file_path",
              "syntax": "Packages/Python/Python.tmLanguage",
              "external_id": "python"
           }
}
debugging in two row layout
debugging in two row layout

Namaste…

Hacking C#’s Lambda Expressions Into Hash Rockets

c# loves RubyAs I move between C# and Ruby, I have found my brain’s internal syntax parser always needing to switch gears and repurpose its understanding of Fat Arrow, =>. In Ruby, it provides a visually salient means of expressing key => value pairing within a Hash. C# on the other hand uses it to indicate the opening of a lambda expression’s body block, x => x + y. Its notable that in other languages, such as Coffee Script, it has a similar meaning. In any case, the lines sometimes blur as I’m dreaming up new ways to make C# look and behave more like my favorite dynamic language.

In this post, I’m going to show you how to repurpose C#’s lambda expression syntax for creating key,value pairs.  My goal is be able create nestable enumerable graph structures with a syntax like this:

var rockets = __.Rocketize(
                               foo => "asdf",
                               bar => 42,
                               biz => new Business{ Name = "AMD" },
                               now => DateTime.Now,
                               fun => new Func(() => return new Awesome(source: "Joel Holder")),
                               sub => __.Rocketize(a => 'b',
                                                   c => 'd',
                                                   e => 'f'),
                               xml => File.ReadAllText(@"data.xml"),
                               web => new Uri("https://uberpwn.wordpress.com/"),
                               ___ => typeof(__),
                               tru => (2*2+3*3)/(5*5) == 1,
                               etc => "..."
                          );

First we need a few functions that leverage the Expression API to provide a means of taking in a series of lambda expressions. Internally, we will convert each expression’s AST into a named key and value of Func<> that  returns an optional state object.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
using System.Text;
using System.Threading;

namespace HashRocket
{
    public class __
    {
        public static IEnumerable<KeyValuePair<object, Func<object, object>>> Rocketize(params Expression<Func<object, object>>[] exprs)
        {
            return exprs.Select(expr =>
            {
                var key = expr.Parameters.FirstOrDefault() != null
                            ? expr.Parameters.FirstOrDefault().Name
                            : DateTime.Now.Ticks.ToString();
                return Rocketize(key, expr).First();
            });
        }
        public static IEnumerable<KeyValuePair<object, Func<object, object>>> Rocketize(object key, params Expression<Func<object, object>>[] exprs)
        {
            return exprs.Select(expr =>
            {
                var fn = expr.Compile();
                return new KeyValuePair<object, Func<object, object>>(key, fn);
            });
        }
    }
}

Now that we have this in place, we can run a few tests to show off the behavior. Note that I’ve opted for IEnumerables of KeyValuePair instead of a Dictionary or Hashtable. This just a personal preference, in that I wanted to support multiple objects with the same key within the data structure.

using System;
using System.Linq;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace HashRocket.Tests
{
    [TestClass]
    public class Tests
    {
        [TestMethod]
        public void Can_Convert_Lambda_Into_Kvp()
        {
            //arrange
            var testInput = "asdf";

            //act
            var rocket = __.Rocketize(input => input).First();

            //assert
            Assert.IsTrue(rocket.Key.Equals("input"));
            Assert.IsTrue(rocket.Value(testInput).Equals("asdf"));
        }

        [TestMethod]
        public void Can_Convert_Multiple_Lambdas_Into_Multiple_Kvps()
        {
            //arrange
            var testInputs = new object[] {"asdf","zxcv",2};

            //act
            var rockets = __.Rocketize(foo => foo + "qwer0",
                                       bar => bar + "qwer1",
                                       biz => biz + "qwer2").ToList();

            //assert
            for (var i = 0; i < rockets.Count; i++)
            {
                Assert.IsTrue(rockets[i].Value(testInputs[i]).Equals(testInputs[i] + "qwer" + i));
            }
        }
}

What’s surprisingly cool about this approach is that it becomes very easy create configuration objects with lambda syntax that can be passed directly into objects for initialization. If you’re familiar with this pattern in Ruby or JavaScript, you’ll appreciate the power and elegance it also affords to C#. To better understand the benefits and potential tradeoffs to using this trick, see Jeremy Skinner’s article on the topic.

Namaste..

C# Deserves A Better Message Passing API

The structured programming model of C# is decisive and straightforward. There is generally one or only a handful of idiomatically correct ways to facilitate a particular design need. In the case of API design, the language’s conception of access modifiers is fundamental. Exposures of behaviors to client code are controlled by applying public, private, or protected modifiers to individual methods. While this basic security mechanism, does allow us to build cleaner APIs, it does not strictly limit our ability to reach inside objects and work with their internal private methods. We can both interrogate and mutate the internal state of objects with no public API whatsoever.

In this post I’m going to show you how use reflection to add a feature to C# that mimics the simple message passing API that I like in Ruby. With this in place, we’ll be able to run any private method on any instance object with a more straightforward syntax.

First, we’ll give System.Object the ability to receive messages via an Extension Method called send.

namespace MessagePassingExtensions
{
    public static class Extensions
    {
        //PUBLIC API
        public static bool respond_to(this object o, string responder, params object[] paramSoak)
        {
            return respond_by(o, responder, paramSoak) != null;
        }
 
        public static object send(this object o, string responder, params object[] paramSoak)
        {
            if (!o.respond_to(responder, paramSoak))
            {
                var reason = string.Format("This object does not respond to messages sent to: {0} with parameters: {1}",
                                           responder,
                                           string.Join(", ", paramSoak));
                throw new MissingMethodException(reason);
            }
            var result = o.respond_by(responder, paramSoak).InvokeMember(responder, send_flags, null, o, paramSoak);
            return result;
        }
 
        //PRIVATE HELPERS
        private const BindingFlags send_flags = BindingFlags.InvokeMethod | BindingFlags.Instance |
                                                BindingFlags.Public | BindingFlags.NonPublic;
 
        private static Type respond_by(this object o, string responder, params object[] paramSoak)
        {
            var class_lookup_path = o.GetType().GetBaseTypes().Reverse().Concat(new[] { o.GetType() }).Reverse().ToList();
            return class_lookup_path.FirstOrDefault(t => t.GetMethods(send_flags)
                                                          .Any(m => m.Name == responder &&
                                                                    m.GetParameters().Count() == paramSoak.Count() &&
                                                                    m.GetParameters().All(p => paramSoak[p.Position].GetType().IsAssignableFrom(p.ParameterType))));
        }
 
        private static IEnumerable<Type> GetBaseTypes(this Type type)
        {
            if (type.BaseType == null) return type.GetInterfaces();
            return Enumerable.Repeat(type.BaseType, 1)
                             .Concat(type.GetInterfaces())
                             .Concat(type.GetInterfaces().SelectMany<Type, Type>(GetBaseTypes))
                             .Concat(type.BaseType.GetBaseTypes())
                             .Distinct();
        }
    }

This gives the API we need. Note that there are only 2 public methods added to the builtin object type:

respond_to() – allows you to ask an instance if it can respond to a particular message.

send() – allows you to send an instance a message.

Now let’s create a class that we can use as our test subject. We’ll use domain of the animal kingdom. First we need a base Animal:

public class Animal
{
    private bool Eat()
    {
        return true;
    }
}

Note that there is only a private method in here, an Eat() behavior. This symbol will not be visible to subclasses nor will it be available via the public API. Next, let’s derive a specific animal by inheriting from this base class. Say in this case, a Duck:

public class Duck: Animal
{
    private string Quack(int times)
    {
        var noises = new List<string>();
        for (var i = 0; i < times; i++)
        {
            noises.Add(MethodBase.GetCurrentMethod().Name);
        }
        return string.Join(" ", noises);
    }
 
    private string Quack()
    {
        return Quack(1);
    }
}

Note that Duck is an Animal, and that it only has a few private methods. It cannot see Animal.Eat(). Now, let’s test it out. The following Test class exercises the implementation of Object.send() and demonstrates that we’re able to run all of the private methods that we just created.

[TestClass]
public class SendTests
{
    [TestMethod]
    public void can_resolve_correct_signature_with_params()
    {
        var duck = new Duck();
        var result = duck.send("Quack", 2);
        Assert.AreEqual(result, "Quack Quack");
    }
 
    [TestMethod]
    public void can_resolve_correct_signature_with_no_params()
    {
        var duck = new Duck();
        var result = duck.send("Quack");
        Assert.AreEqual(result, "Quack");
    }
 
    [TestMethod]
    public void can_respond_to_messages_from_a_base_class()
    {
        var duck = new Duck();
        var result = duck.send("Eat");
        Assert.AreEqual(result, true);
    }
 
    [TestMethod]
    [ExpectedException(typeof(MissingMethodException), "You sent a message that I could not respond to")]
    public void blows_up_if_sent_message_cannot_be_responded_to()
    {
        var duck = new Duck();
        var result = duck.send("Bark");
    }
}

All of these tests pass. They are self-explanatory. I think that the most interesting one is can_respond_to_messages_from_a_base_class(), which shows that we can also get and run a private method from the underlying Animal class. By sending a message to a symbol on an instance object, we’re able to program against it in a looser style. This opens up a number of interesting ways to work with an object. Suppose for example you wanted to build an XML-driven execution system. By simply reading in instructions from an external xml source, we can emit and drive code execution that was not known at design time. Here’s an example of a declarative approach to driving execution.

DuckDriver.xml:


<?xml version="1.0" encoding="UTF-8" ?>
<Obj type="DynamicSpikes.Tests.Duck">
  <Run method="Quack" times="3" />
</Obj>

Given the above XML File, I can process and verify it like this:

[TestClass]
[DeploymentItem(@".\Assets\DuckDriver.xml")]
public class XmlDeclarativeTests
{
    [TestMethod]
    public void can_create_and_run_from_xml()
    {
        var xml = XDocument.Load(@".\DuckDriver.xml");
 
        var objects = xml.Root.Descendants().Where(elem => elem.Name.LocalName == "Obj").ToList();
        objects.ForEach(objElem =>
            {
                var runs = objElem.Descendants().Where(elem => elem.Name.LocalName == "Run").ToList();
                runs.ForEach(runElem =>
                    {
                        var type = Type.GetType(objElem.Attribute("type").Value);
                        var method = runElem.Attributes().Where(attr => attr.Name == "method").FirstOrDefault().Value;
                        var parameters = runElem.Attributes().Where(attr => attr.Name != "method").Select(attr => attr.Value).Cast<object>().ToArray();
 
                        var obj = Activator.CreateInstance(type);
                        var result = obj.send(method, parameters);
                        Assert.AreEqual(result, "Quack Quack Quack"); //<Run method="Quack" times="3" />
                    });
            });
    }
}

The test reads in the XML File, finds the Obj elements, finds the Run elements, and grabs the method name and parameters from the attributes. With this information, we have all we need to create the object and send it the message. This test passes as expected.

The ability to turn a markup into runnable code is not new, but the ease with which it can be done with the nuts and bolts of the C# language are impressive. The fact that we can easily marshal declarative instructions into an unknown but completely runnable instruction set is more than just a little bit awesome.

Namaste…

Introducing X DSL, A More Fluent XML Builder

A while back, I posted about using DynamicObjects to facilitate building Domain Specific Languages in C#. To the extent that we have to play within the syntactic sandbox that the language itself requires, we are still able to take advantage of what is available in the way of built in operators by changing their meaning. Specifically, with the X DSL, I’ve combined some simple operators and a dynamic chaining API in order to make a code syntax that is more representative of the XML we want it to build for us. Here is an example:

Suppose we want to build this XML Document:

<Customers>
  <Customer category="Foo" type="Bar">
    <Address>
      <Street>123 Sunny Drive</Street>
      <City>Austin</City>
      <State>TX</State>
    </Address>
    <Profile CreatedOn="2013-05-13T18:44:27.39154-05:00">
      <Theme>Aristo</Theme>
      <DefaultPage>http://www.google.com</DefaultPage>
    </Profile>
  </Customer>
</Customers>

With X, we would use code that looks like this to create it:

var x = X._.Customers()
            > X._.Customer(category: "Foo", type: "Bar")
                > X._.Address()
                    > X._.Street("123 Sunny Drive")
                    + X._.City("Austin")
                    + X._.State("TX")
                < X._.Profile(CreatedOn: DateTime.Now)
                    > X._.Theme("Aristo")
                    + X._.DefaultPage("http://www.google.com");

The goal with X, is to make the code structurally and semantically similar to that which its instructions create. Take a close look. You’ll note a few interesting characteristics. First, I’m using X._ to represent the pipeline continuations. This allows me to make operations over an unknown number of chained future resources as the expression continues. There are a number of novel concepts that this approach yields up. First, we’re able to create a pretty complex structural recipe in a single expression. This is effectively a single line of code. Next, I’ve selected symbols for the operations that are indicative of directional navigation through immerging hierarchal and sibling relations. Specifically, the dialect has the following:

> means “step down and create what follows”

< means “step up and create what follows”

+ means “stay at the current level and create what follows”

These operation constructs enable navigation and creation simultaneously. This takes care of our need to create Xml nodes and their inner texts. However, what about attributes? Well, you can see that those too are enabled, by leveraging C#’s named parameters capability.

Thus, when we say:

X._.Customer(category: “Foo”)

We get:

<Customer category=Foo”>

Pretty nice.

This elegant abuse of the language can get us closer to code/output isomorphism, wherein the code resembles that which it is creating (at least more closely than the builtin APIs .NET provides for this kind of thing). It was inspired by the Excellent Builder library in Ruby. However, there are notable differences in how operators are used here to provide the Xml as the output of a composite expression, rather than a statement chain, as it works with Builder. In any case, I believe there is a compelling case to close the representational gap between data and code. If there’s anything that Lisp can teach us it’s that data and code should be thought of as the same thing. The closer we can get to that ideal the simpler, and more powerful our tools become in allowing us to express our intent and have the code just carry it out.

The implementation of X is here and below for anyone wishing to play around with it:


public class X : DynamicObject
{
    List<XElement> xsiblings = new List<XElement>();
    XElement xwrapper;
    XElement xplace;

    Dictionary<string, Func<X>> customOperators = new Dictionary<string, Func<X>>();

    private Func<X> Down
    {
        get { return () => new X(xplace) { xplace = xplace }; }
    }

    private Func<X> Up
    {
        get
        {
            var parent = xplace.Parent ?? xplace;
            parent = parent.Parent ?? parent;
            return () => new X(parent) { xplace = xplace.Parent };
        }
    }

    private Func<X> Self
    {
        get { return () => new X(this);  }
    }

    public static dynamic _{ get { return New; } }

    public static dynamic New
    {
        get
        {
            Func<dynamic> exp = () =>
            {
                var b = new X();
                return (dynamic)b;
            };
            return exp.Invoke();
        }
    }

    protected void init()
    {
        customOperators.Add(">", () => Down());
        customOperators.Add("<", () => Up());
        customOperators.Add("+", () => Self());
        customOperators.Add("_", () => _);
    }

    public X(){
        xwrapper = new XElement("Wrapper");
        xplace = xwrapper;
        init();
    }

    public X(XElement xelem)
    {
        xwrapper = xelem;
        xplace = xwrapper.Descendants().LastOrDefault() ?? xwrapper;
        init();
    }

    public X(X builder)
    {
        xwrapper = builder.xwrapper;
        xplace = xwrapper.Descendants().LastOrDefault() ?? xwrapper;
        init();
    }

    public X(X builder1, X builder2)
    {
        builder1.xplace.Add(builder2.xwrapper);
        builder2.xplace = builder1.xplace;
        init();
    }

    public override string ToString()
    {
        var sb = new StringBuilder();
        var content = xwrapper.Descendants();
        foreach (var c in content)
            sb.Append(c.ToString());
        return sb.ToString();
    }

    public static X operator +(X c1, X c2)
    {
        c1.xwrapper.Add(c2.xwrapper.Descendants());
        return c1;
    }

    public static X operator >(X c1, X c2)
    {
        var xlast = c1.xplace.Descendants().LastOrDefault();
        xlast = xlast ?? c1.xplace;
        xlast.Add(c2.xwrapper.Descendants());
        return c1;
    }

    public static X operator <(X c1, X c2)
    {
        var xlast = c1.xplace.Parent.Descendants().LastOrDefault();
        xlast = xlast ?? c1.xplace;
        var parent = xlast.Parent ?? xlast;
        parent = parent.Parent ?? parent;
        parent.Add(c2.xwrapper.Descendants());
        return c1;
    }

    public override bool TryInvoke(InvokeBinder binder, object[] args, out object result)
    {
            return base.TryInvoke(binder, args, out result);
    }

    public override bool TryInvokeMember(InvokeMemberBinder binder, object[] args, out object result)
    {
        result = default(object);

        if (customOperators.ContainsKey(binder.Name))
            result = customOperators[binder.Name]();
        else //build with this call
        {
            var work = new Func<object>(() =>
            {
                var diff = args.Count() - binder.CallInfo.ArgumentNames.Count;
                var innerText = string.Empty;
                var argQueue = new Queue<object>(args);
                if (diff > 0)
                    diff.Times(i => innerText += argQueue.Dequeue().ToString());

                args = argQueue.ToArray();
                var parameters = new Dictionary<string, object>();
                binder.CallInfo.ArgumentNames.Count().Times(i =>
                {
                    parameters.Add(binder.CallInfo.ArgumentNames[i], args[i]);
                });

                XElement xelem = new XElement(binder.Name,
                    parameters.ToList().Select(param => new XAttribute(param.Key, param.Value)));
                xelem.Add(innerText);
                xplace = xelem;
                return xelem;
            });
            var output = work.DynamicInvoke() as XElement;
            xwrapper.Add(output);
            result = this;
        }
        return true;
    }

    public X this[string key]
    {
        get { return customOperators[key](); }
    }
}

Namaste…

Dynamic Domain Specific Languages in C# and the Quine

Wikipedia defines a quine as “a computer program that takes no input and produces a copy of its own source code as its only output”.  Some standard terms for this category of program are “self-replicating”, “self-reproducing”, and “self-copying” programs.  Quines are possible in any programming language.  The term quine, itself, was first coined in Douglas Hofstadter’s brilliant tome, “Gödel, Escher, Bach: An Eternal Golden Braid, in the honor of philosopher Willard Van Orman Quine (1908–2000), who made an extensive study of indirect self-reference, and in particular for the paradox-producing expression, known as Quine’s paradox.

A while back, I was on an experimental jag centered on seeing just how badly I could abuse .NET  dynamic dispatch capabilities.  Specifically, what kind of DSLs could we make with C# dynamic and what would be the nature of a Quine built to accommodate fully adhoc call chaining over dynamics.  Well, this led to a series of DSLs and built on the metaprogramming concepts that fell out this work.  In this article, I will introduce you to the first and simplest DSL in this series, EchoDSL.  Echo is not a pure quine, yet it is a simple framework for creating self-replicating code.

Here is a test demonstrating the usage of the Echo:


[TestMethod]
public void Testing_EchoDSL()
{
    var self_is = Echo._
                      .Hello("Joel").Your.Last.Name.Is["Holder"].You.Should.Record.This(foo: "bar")
                      .ToString();

    var should_be = ".Hello(\"Joel\").Your.Last.Name.Is[\"Holder\"].You.Should.Record.This(foo: \"bar\")";

    Assert.AreEqual(self_is, should_be);
}

As you can see we just kind of told Echo what it should repeat back to us, but the important thing is that we expressed that in source code.  The source is both legal C# and code that is obviously legal in the domain specific language itself.  To the point, in Echo, chained code that is syntactically legal in C#, it will happily convert to stringified source code and give back to you with a simple ToString() call.  The sheer meta”ness” of this may not be apparent on its face.  Let’s a try a more aggressive usage of Echo.


[TestMethod]
public void Testing_EchoDSL2()
{
    var self = Echo._
                   .In.Order.To.Create.An.Apple.Pie.From.Scratch(", ")
                   ["You "].Must.First.Invent.The.Universe
                   .You().Should().Record.This.too(foo: "bar")[""][""]
                   [" ... wait what just happened!. "]["magic must have occurred."].LOL.lfmao
                   .terms_can_comeIn_any_ORDER(question1: "is there anything it won't tolerate?",
                                               answer1: "apparently not",
                                               question2: "what is the meaning of life?",
                                               answer2: 42)
                   .And["please don't feed the monkies"]
                   .And("don't stare at the gorrilla")
                   .He.will(think: "you").are["challenging"].him
                   .Or.At_worst_he_will_think_you_are_a(suitable: "mate").Or.
                   I.Could.Go.On.To["other things"].like().monkey.chicken(".")["cow"]
                   .IMean("it")["!!!"].NoReally["..."].Ok.Fine("!")["  Go_Ahead"].
                   And["Do_Whatever you"].Want().to;

    var self_is = self.ToString();
    var should_be = ".In.Order.To.Create.An.Apple.Pie.From.Scratch(\", \")[\"You \"].Must.First.Invent.The.Universe.You().Should().Record.This.too(foo: \"bar\")[\"\"][\"\"][\" ... wait what just happened!. \"][\"magic must have occurred.\"].LOL.lfmao.terms_can_comeIn_any_ORDER(question1: \"is there anything it won't tolerate?\", answer1: \"apparently not\", question2: \"what is the meaning of life?\", answer2:  42).And[\"please don't feed the monkies\"].And(\"don't stare at the gorrilla\").He.will(think: \"you\").are[\"challenging\"].him.Or.At_worst_he_will_think_you_are_a(suitable: \"mate\").Or.I.Could.Go.On.To[\"other things\"].like().monkey.chicken(\".\")[\"cow\"].IMean(\"it\")[\"!!!\"].NoReally[\"...\"].Ok.Fine(\"!\")[\"  Go_Ahead\"].And[\"Do_Whatever you\"].Want().to";

    Assert.AreEqual(self_is, should_be);
}

Again, we get a faithful representation from the code of itself as it executes.  There are a number of interesting points of note locked within this basic paradigm.  First, I am writing the code I want to write while writing it, and yet the program is executing and writing my code for me as well.  Second, given that it will accept any legal c# syntax, I can simply babble into the DSL and it will babble back.  And, beyond these superficial aspects, there are some potentially powerful genetic programming capabilities.  Consider, adding some semantic meanings and key words into the DSL that do more than just spool.   The output of a dynamic DSL such as Echo could be presented to CodeDom to produce and execute whole new programs that are runtime generated copies of itself, with perhaps mild nondeterministic mutations randomly interspersed.   What would millions of successive iterations produce?

Here is the implementation of EchoDSL: (this code uses the int.Times() extension method shown here)


using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Dynamic;
using System.Xml.Linq;
using System.Text.RegularExpressions;

public class Echo : DynamicObject
{
    string history = "";

    Dictionary<string, Func<Echo>> customOperators = new Dictionary<string, Func<Echo>>();

    /// <summary>
    /// Fluent Echo
    /// </summary>
    /// <param name="key"></param>
    /// <returns></returns>
    public Echo this[string key]
    {
        get
        {
            RecordIndexerCall(key);
            if (customOperators.ContainsKey(key))
                return customOperators[key]();
            else
                return Self();
        }
    }

    /// <summary>
    /// Fluent method getter for resolving self
    /// </summary>
    private Func<Echo> Self
    {
        get { return () => this; }
    }

    /// <summary>
    /// Fluent Property used for spawning instances from static.
    /// </summary>
    public static dynamic New
    {
        get
        {
            Func<dynamic> exp = () =>
            {
                var b = new Echo();
                return (dynamic)b;
            };
            return exp.Invoke();
        }
    }

    public static dynamic _
    {
        get { return New; }
    }

    #region constructors and initialization

    protected void init()
    {
        customOperators.Add("self", () => Self());
        customOperators.Add("new", () => New);
        customOperators.Add("_", () => New);
    }

    public Echo()
    {
        //todo
        init();
    }

    public Echo(Echo Echo)
    {
        //todo
        init();
    }

    public Echo(Echo Echo1, Echo Echo2)
    {
        //todo
        init();
    }

    #endregion

    #region operator overloading

    public static Echo operator +(Echo c1, Echo c2)
    {
        c1.history += c2.history;
        return c1;
    }

    public static Echo operator -(Echo c1, Echo c2)
    {
        c2.history += c1.history;
        return c2;
    }

    #endregion

    #region overrides

    public override string ToString()
    {
        //echo output
        return history.ToString();
    }

    #endregion

    #region dynamic api

    public override bool TryGetMember(GetMemberBinder binder, out object result)
    {
        RecordPropertyCall(binder.Name);

        if (customOperators.ContainsKey(binder.Name))
            result = customOperators[binder.Name]();
        else
            result = Self();
        return true;
    }

    public override bool TryInvokeMember(InvokeMemberBinder binder, object[] args, out object result)
    {
        result = default(object);

        if (customOperators.ContainsKey(binder.Name))
            result = customOperators[binder.Name]();
        else //build with it
        {

            var namedArgs = new Dictionary<string, object>();
            var indexArgs = new LinkedList<object>();
            Action prepare = () =>
            {
                var diff = args.Count() - binder.CallInfo.ArgumentNames.Count;
                var argQueue = new Queue<object>(args);
                if (diff > 0)
                {
                    diff.Times(i => indexArgs.AddLast(argQueue.Dequeue()));
                }

                var shiftedArgs = argQueue.ToArray();

                binder.CallInfo.ArgumentNames.Count().Times(i =>
                {
                    namedArgs.Add(binder.CallInfo.ArgumentNames[i], shiftedArgs[i]);
                });
            };

            var work = new Func<object>(() =>
                {
                    return null; //something
                });

            prepare();

            RecordMethodCall(binder.Name, indexArgs.ToArray(), namedArgs);

            //feed engine/managed state
            var output = work();
            //...

            result = Self();
        }

        return true;
    }

    #endregion

    #region private helpers
    void RecordMethodCall(string name, object[] indexArgs, Dictionary<string, object> namedArgs)
    {
        //fix string args
        indexArgs.Count().Times(i => indexArgs[i] = indexArgs[i] is string ? "\"" + indexArgs[i] + "\"" : indexArgs[i]);
        var call = name + "(" + String.Join(", ", indexArgs.ToArray());
        if (indexArgs.Count() > 0)
            call += ", ";

        namedArgs.ToList().ForEach(namedArg =>
            {
                call += namedArg.Key + ": ";
                call += (namedArg.Value is string ? "\"" + namedArg.Value + "\"" : " " + namedArg.Value) + ", ";
            });
        call = Regex.Replace(call.Trim(), ",$", "");
        call += ")";
        history += "." + call;
    }
    void RecordIndexerCall(string name)
    {
        var call = "[\"" + name + "\"]";
        history += call;
    }
    void RecordPropertyCall(string name)
    {
        var call = name;
        history += "." + call;
    }
    #endregion
}

In following posts, I will extend off of what has been demonstrated to show how higher-order functions can be folded into these dynamic DSLs to provide dynamic execution, decisioning, and data mutation pipelines. Namaste…

C#-powered JavaScript Expression Evaluator

Here’s an interesting little class that I initially created to do dynamic arithmetic.  However, after getting the concept up and running, it turns out to be a lot more powerful.  The idea is to start up a hosted JavaScript runtime inside the CLR runtime and present it with string of JavaScript code for immediate inline execution.  The ExpressionEvaluator dutifully loads and evaluates the “scriptlet”, and then it returns the results back to .NET as a string, which can easily be coerced back into an instance of a native CLR Type.  In the original case, this allowed for a javascript-powered dynamic calculator from the statically typed C#.NET, VB.NET, or F# language runtimes.  However, there are more interested use cases for this technology, beyond what I demonstrate here, such as providing a user extensibility layer to the logic of a compiled application at runtime.  For example, allowing users to generate, extend, and mutate existing business rules could easily be achieved by allowing users to introduce script text directly into a captive execution sandbox.  User-specified business rules could be stored as simple text and spooled for execution in workflow engine’s, etc.  Another interesting use case, might be to hook up a REPL (Read Eval Print Loop) into a UI and allow users to issue any sort structured arithmetic expressions directly to the engine, having it respond back with answers in realtime.

Here is a simple unit test showing how to drive it:

using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace Unit.Core
{
 [TestClass]
 public class ExpressionEvaluatorTest
 {
 public TestContext TestContext { get; set; }

[TestMethod]
 public void EvaluateToStringTest()
 {
 var result = ExpressionEvaluator.EvaluateToString("(2+5) < 8");
 Assert.IsTrue(Convert.ToBoolean(result));
 }
 }
}

Here is the ExpressionEvaluator:

using System;
using System.CodeDom.Compiler;
using System.Globalization;
using System.Reflection;
using Microsoft.JScript;

namespace Core.Utils
{
 public static class ExpressionEvaluator
 {
 #region static members
 private static object _evaluator = GetEvaluator();
 private static Type _evaluatorType;
 private const string _evaluatorSourceCode =
 @"package Evaluator
 {
 class Evaluator
 {
 public function Eval(expr : String) : String
 {
 return eval(expr);
 }
 }
 }";

#endregion

#region static methods
 private static object GetEvaluator()
 {
 CompilerParameters parameters;
 parameters = new CompilerParameters();
 parameters.GenerateInMemory = true;

JScriptCodeProvider jp = new JScriptCodeProvider();
 CompilerResults results = jp.CompileAssemblyFromSource(parameters, _evaluatorSourceCode);

Assembly assembly = results.CompiledAssembly;
 _evaluatorType = assembly.GetType("Evaluator.Evaluator");

return Activator.CreateInstance(_evaluatorType);
 }

/// <summary>
 /// Executes the passed JScript Statement and returns the string representation of the result
 /// </summary>
 /// <param name="statement">A JScript statement to execute</param>
 /// <returns>The string representation of the result of evaluating the passed statement</returns>
 public static string EvaluateToString(string statement)
 {
 object o = EvaluateToObject(statement);
 return o.ToString();
 }

/// <summary>
 /// Executes the passed JScript Statement and returns the result
 /// </summary>
 /// <param name="statement">A JScript statement to execute</param>
 /// <returns>The result of evaluating the passed statement</returns>
 public static object EvaluateToObject(string statement)
 {
 lock (_evaluator)
 {
 return _evaluatorType.InvokeMember(
 "Eval",
 BindingFlags.InvokeMethod,
 null,
 _evaluator,
 new object[] { statement },
 CultureInfo.CurrentCulture
 );
 }
 }
 #endregion
 }
}