I remembered reading about JUnit Max by Kent Beck, a continuous testing plugin for Eclipse, that runs your unit tests in the background and unobtrusively tells you when a test fails, allowing you to do what you do best: write code. JUnit Max seems like a great thing, and now I needed the same thing for Visual Studio. A quick Google didn’t yield any add-ins, extensions or packages, so I decided to create one.

The result is Continuous Testing for Visual Studio, a small extension which runs your unit tests each time you build your solution, and reports failing tests to the error list so you can navigate to the line that failed and make the test pass. The extension significantly improves my workflow by removing a tedious manual step of running unit tests, so I encourage you to take it for a spin. Continuous Testing can be downloaded for Visual Studio 2008 and Visual Studio 2010. Future updates and versions will be announced on the Continuous Testing home page.

UPDATE Jun 17th, 2010: I’ve receive a lot of feedback through various solutions online. To be able to help you and/or improve Continuous Testing for Visual Studio, I need samples from you that reproduce the problems you are experiencing. Do not hesitate to leave a comment here, and provide your e-mail address when commenting, and you will receive a reply.

• Code Complete: A Practical Handbook of Software Construction by Steve McConnell
• Design patterns: Elements of reusable object-oriented software by Erich Gamma, Richard Helm, Ralph Johnson, and John M. Vlissides (the Gang of Four)
• The Pragmatic Programmer: From Journeyman to Master by Andrew Hunt and David Thomas
• The Mythical Man-Month: Essays on Software Engineering by Frederick P. Brooks
• Clean Code: A Handbook of Agile Software Craftsmanship by Robert C. Martin
• Refactoring: Improving the Design of Existing Code by Martin Fowler, Kent Beck, John Brant, William Opdyke, and Don Roberts

The version I put up includes several enhancements from the latest release:

• It uses FormatterServices.GetUninitializedObject and hence does not depend on a parameterless constructor or custom instance provider (but you can of course still create an instance provider if you want to control object initialization)
• The bug with copy semantics for already visited objects submitted by Walter Oesch has been fixed
• The bug with inherited fields found by Alex, and the patch submitted for it, has been incorporated

Bleeding edge Copyable can be found at http://github.com/havard/copyable. The clone URL is git://github.com/havard/copyable.git. Now go fix your own bugs! Or even better, enhance the framework.

What is MapReduce?

For those of you who don’t know what MapReduce is: MapReduce is a simplified interface for parallel data processing. MapReduce was initially described by the Google engineers Jeffrey Dean and Sanjay Ghemawat in the 2004 paper titled MapReduce: Simplified data processing on large clusters.

MapReduce processes data by splitting the processing in to a set of transformations (in functional programming, this is called the “map” function (it maps or transforms an input to an output)). The results of the transformations are then combined into a single result (in functional programming, this is called the “reduce” function (it reduces a set of values to a single value)). On a sidenote, Linq has equivalent functions, but the names are different, presumably to make them more familiar to people with SQL knowledge. In Linq, map is called Select, and reduce is called Aggregate.

Shortly put, to process a huge set of data, you split the data into chunks and process each chunk in parallel. This eventually creates a new set of intermediary results, which is reduced to a single result.

Implementing a minimalistic MapReduce in .NET 4.0

The signature of my MapReduce function is

static Task<TResult> Start<TInput, TPartial, TResult>(
  Func<TInput, TPartial> map, 
  Func<TPartial[], TResult> reduce, 
  params TInput[] inputs);</pre>

In other words, to start a MapReduce run, you supply a map function, a reduce function, and a set of inputs. Each input will be turned into an intermediate result (of type TPartial). Inputs are transformed concurrently. When all inputs are transformed, the reduce function is called to transform the partial results into a final result (of type TResult). Cool!

The map part is implemented by starting a task for each supplied input using Task.Factory.StartNew().

Task.Factory.StartNew(() => map(input));

The reduce part is implemented as a continuation of all the map tasks, meaning that the reduce task waits for all the map tasks to complete, and then executes. This is achieved using Task.Factory.ContinueWhenAll.

Task.Factory.ContinueWhenAll(
  mapTasks, 
  tasks => PerformReduce(reduce, tasks));

As you can see, the implementation is minimalistic and simple, and usage is likewise.

Here’s a simple example using MapReduce to calculate the root mean square (MSE) of a set of values:

var task = MapReduce.Start<int, int, double>(
  i => i * i,
  s => Math.Sqrt(s.Aggregate((a, b) => a + b) / 5),
  1, 2, 3, 4, 5);
// Wait for result
task.Wait();
// Prints 3.3166...
Console.WriteLine(task.Result);

Actual applications of MapReduce are of course far more interesting than this simple example.

Applications of MapReduce

MapReduce can essentially be applied to any problem where you need a number of things to be done in parallel. It can even be applied in cases where you don’t need a final result. Just return an arbitrary value as the result (or even better, implement a variant of my MapReduce which uses Action<T>).

A few obvious use cases:

• Distributed search
• Distributed sort
• Tokenization
• Indexing
• Log processing
• Machine learning
• General artificial intelligence
• General data mining
• Large scale image processing
• …

The list goes on and on, these are just a few things off the top of my head.

You can grab the source code for MapReduce here. Since this is done in .NET 4.0, it requires Visual Studio 2010 Beta 2 or later.

As usual, play around with it, have fun, and let me know if you find it useful!

BouncyCastle provides flexibility and control over your encryption approach, which comes at a cost. The BouncyCastle API might be a bit hard to cope with at first, but if you know encryption in general you should be able to find your way around the API without too much effort. This post will be focusing on RSA, since that was my original need, but it should be mentioned that BouncyCastle provides many other asymmetric (and symmetric) algorithms for which the usage is similar to what you find below.

Creating RSA keys

Creating RSA keys is a simple task. The method below lets you specify the key size in bits, and creates a key pair for you.

public AsymmetricCipherKeyPair GenerateKeys(int keySizeInBits)
{
  RsaKeyPairGenerator r = new RsaKeyPairGenerator();
  r.Init(new KeyGenerationParameters(new SecureRandom(),
    keySizeInBits));
  AsymmetricCipherKeyPair keys = r.GenerateKeyPair();
  return keys;
}

That’s all there is to it.

Encryption

Now that we have a key pair, we are ready to encrypt and decrypt using RSA. In the example below, we use a key (public or private) to encrypt a byte sequence. To encrypt a string, simply convert the string to a byte array using Encoding.GetBytes.

public byte[] Encrypt(byte[] data, AsymmetricKeyParameter key)
{
  RsaEngine e = new RsaEngine();
  e.Init(true, key);</p>

<p>int blockSize = e.GetInputBlockSize();</p>

<p>List<byte> output = new List<byte>();</p>

<p>for (int chunkPosition = 0; chunkPosition &lt; data.Length; 
    chunkPosition += blockSize)
  {
    int chunkSize = Math.Min(blockSize, data.Length - 
      (chunkPosition * blockSize));
    output.AddRange(e.ProcessBlock(data, chunkPosition,
      chunkSize));
  }
  return output.ToArray();
}

The approach above uses a list to gather output for the sake of simplicity. Note that the RSA engine can only process a limited block size at a time (block size depends on the key size). The approach above generalizes the RSA engine, and processes a data set of an arbitrary size.

The above method does not impose constraints on which key you use for encryption. Use the public key or the private key as you see fit for your solution.

Decryption

The Decrypt method is very similar to the Encrypt method:

public byte[] Decrypt(byte[] data, AsymmetricKeyParameter key)
{
  RsaEngine e = new RsaEngine();
  e.Init(false, key);</p>

<p>int blockSize = e.GetInputBlockSize();</p>

<p>List<byte> output = new List<byte>();</p>

<p>for (int chunkPosition = 0; chunkPosition &lt; data.Length;
    chunkPosition += blockSize)
  {
    int chunkSize = Math.Min(blockSize, data.Length - 
      (chunkPosition * blockSize));
    output.AddRange(e.ProcessBlock(data, chunkPosition,
      chunkSize));
  }
  return output.ToArray();
}

Again, it’s up to you which key you choose to use. If you want to use the common approach, encrypt using a symmetric cipher, hash the data, and sign the hash with your private key using the above Encrypt method. If you want to use another approach like encrypting the actual data using your private key, you are free to do so.

I hope this post helps those of you who want to apply RSA (or any other asymmetric cipher) to more subtle cases than those supported by the .NET framework.

The ViewDataDictionary that is passed to the HtmlHelper can be empty, or made to contain the data you want for your test.

public static HtmlHelper CreateHtmlHelper(ViewDataDictionary vd)
{
  var mockViewContext = new Mock<ViewContext>(
    new ControllerContext(
      new Mock<HttpContextBase>().Object,
      new RouteData(),
      new Mock<ControllerBase>().Object),
    new Mock<IView>().Object,
    vd,
    new TempDataDictionary());</p>

<p>var mockViewDataContainer = new Mock<IViewDataContainer>();
  mockViewDataContainer.Setup(v => v.ViewData)
    .Returns(vd);</p>

<p>return new HtmlHelper(mockViewContext.Object, 
    mockViewDataContainer.Object);
}

The rationale is in the common application of public-key cryptography, where:

• Encrypting with the public key ensures confidentiality, i.e. the process known as encryption in common tongue. Encrypting with a public key ensures that only the entity in possession of the private key can read the data.
• Encrypting with the private key ensures authenticity, i.e. the process known as signing in common tongue. There is no need to encrypt the entire data stream to ensure authenticity, so the common approach is to calculate a hash of the data and sign the hash instead.

To facilitate these patterns, the .NET public-key cryptography API is designed so that:

• Encrypt() encrypts with a public key, Decrypt() requires a private key.
• SignData() encrypts with a private key, but since that implies a signature, one must provide a hashing algorithm and a private key. VerifyData() uses a public key.

But I want to encrypt with my private key! Yes, this is what SignData() does, but it does so to just the hash calculated by the provided hashing algorithm, since that is the de-facto approach for signing, and implementing my own HashAlgorithm that passes in all the data is just wrong.

OK, these are the common uses, but why limit the API to that? There is no limitation in the RSA algorithm to my knowledge that prevents other uses than the two offered by RSACryptoServiceProvider. In fact, if I wanted to perform the traditional signing approach, I could just hash the data myself and encrypt it with my private key myself. Or even better, SignData() could be available to help me for convenience.

So, how do you apply RSA in .NET in an uncommon manner? Don’t use .NET’s cryptography API, but embrace an open source alternatives like BouncyCastle, which saves your day.

Strongbind vs traditional data binding

Traditional data binding is typically declared like this:

// Assume we have a binding source with 
// a description, and a text box
// This binds the Text property of the text 
// box to the Description property of the source
textBox.DataBindings.Add("Text", bindingSource, 
    "Description");

Obviously, many errors can arise from this declaration: Your control may not have a Text property, or you might have spelled it wrong. The same goes the Description property of your source.

The same declaration in Strongbind is written as follows:

using(BindingScope scope = new BindingScope())
{
    // Create bindables
    IBusinessObject bindableSource = 
        scope.CreateSource(bindingSource);
    TextBox bindableTarget = 
        scope.CreateTarget(textBox);</p>

<pre><code>// Declare bindings
Binder.Bind(bindableSource.Description)
    .To(bindableTarget.Text);
</code></pre>

<p>}

As you probably understand, the risk of mistyping is removed, and we get IntelliSense support out of the box.

Behind the scenes of Strongbind

To achieve this strongly typed data binding, Strongbind uses a technique known as proxying. Strongbind dynamically generates a proxy for your business object and your control, and uses the proxies to intercept the calls to the property getters during runtime to declare the data binding. Hence, you need to declare a bindable source and bindable target first to create the proxies, and then use these proxies during the binding declaration. You will get a runtime error if you try to use your real objects when declaring data bindings.

Limitations of Strongbind

Although Strongbind makes data binding a far more declarative process, the library does have its limitations.

Controls containing ActiveX components are not supported. If the control containing an ActiveX component is a custom control created by you, you can get around it by declaring an interface for the control and specifying that as the type to use when declaring your binding source:

// The control interface 
// (must inherit IBindableComponent)
public interface IBindableAXControl 
    : IBindableComponent 
{
    string Text { get; set; }
}</p>

<p>// Declare the binding target this way within a 
// binding scope
IBindableAXControl source = 
   scope.CreateTarget<IBindableAXControl>(control);

Also, binding to concrete binding sources with non-virtual properties is not supported. Again, the recommended workaround is to create an interface for your binding source and use that when declaring the data bindings. (You always want to create these interfaces, since decoupling your objects’ interfaces from their implementation is recommended for testability, maintainability, and is generally A Good Thing(tm).)

// The business object interface
public interface IBusinessObject
{
    string Description { get; set; }
}</p>

<p>// Declare the binding source this way within a 
// binding scope
IBusinessObject bindableSource = 
    scope.CreateSource<IBusinessObject>(source);

Apart from these two issues, which can be worked around in most cases, Strongbind should work flawlessly. If it does not, I would be happy to know.

Where do I get it?

Strongbind is an open source project hosted at Google Code. To get the latest version, check out the code from its repository.

Strongbind is still in an early development stage, so no releases have been created yet. I still encourage you to check out and start using the library as soon as possible, though. A beta will be released as soon as I feel comfortable doing it.

If you want to contribute to Strongbind, I would be happy to welcome you. Visit http://strongbind.googlecode.com for instructions.

Bind away!

JSON injection, or actually script tag injection, is a rather common technique that circumvents the XMLHTTPRequest limitation by dynamically injecting script tags into the calling page. A script tag can have any domain as its source, which means that cross-domain calls are possible. The technique is also referred to as JSON callbacks, although it really is not limited to JSON payloads. The technique is also referred to as JSONP, although the original JSONP is a bit more extensive than just callbacks using script injection.

It really is a neat technique without a cool term. JSONP is a term for a superset of JSON injection. A more precise term than JSON injection would be Javascript injection, but that’s already used to describe vulnerabilities in web pages where malicious Javascript code is injected through e.g. links from external sites. I hereby propose the term AJAST – Asynchronous Javascript And Script Tags. At least it’ll be the name of my implementation. If it doesn’t catch on for anything but that, we’ll even be a bit more confused than we already are. Now, what do we require to do AJAST?

AJAST requirements

The requirements an AJAST request lays on the server-side are the following:

1. The server must provide its services through HTTP GET requests.
2. The client must be able to supply the name of a callback function that the response will be wrapped in.
3. The server is expected to provide a response on the form callback(payload), where callback is the name of the callback function supplied by the client, and payload is the payload returned by the server. The payload can be XML, JSON, or any other form of data that the Javascript callback function can accept as a single argument.

These requirements are already fulfilled by many REST services, but they are still hard to use in an AJAST fashion due to client side challenges. The two main requirements for an AJAST library are:

1. Complete handling of requests. Nothing more than a URL and a callback should be needed to create a request.
2. Timeouts is a show stopper for AJAST. With script injection, it is difficult to know if a call completes, and from an AJAST usage perspective it is essentially impossible to create a decent solution without knowing if requests complete or not.

Security is another obvious challenge, although in my view it is a challenge for the Internet in general rather than AJAST in particular. Developers creating cross-domain applications should be aware of the security risks involved, and take measures to prevent security breaches accordingly. There is no silver bullet.

Although several examples for implementing an AJAST request are found around the web, I found no fully functional stand-alone implementations. Dan Theurer’s article on script requests provides code that can be used to create an implementation, but leaves the timeout problem unsolved. Toolkits such as Dojo also implement variations on the approach using IFrame requests, but they are a lot more hairy, and I really don’t want a framework (or parts of it) bloating the web site I am creating just to be able to do AJAST. I want a library that can perform just the task that I want it to perform, and perform it well.

An AJAST library

So, I decided to create my own AJAST library, OX.AJAST, complete with the following features:

• A fully encapsulated mechanism for making AJAST calls
  You simply supply a URL, the name of the callback parameter that will be appended to the URL, and a callback function.
• Support for timeouts
  Remote requests can of course time out, and time outs need to be handled. Apart from the obvious security challenges involved with using AJAST (note that I’m not saying they’re defects, they’re challenges for us developers to handle) , this is the hardest challenge for AJAST requests. Without the ability of specifying timeouts, we’re essentially in the dark with regards to whether or not a request will complete. OX.AJAST neatly supports timeouts by wrapping the supplied callbacks, putting on a timer, and checking for completion when the timer times out.
• Guarantee that the callback function will be called
  Whatever happens, and as a direct consequence of the timeout support, the library guarantees that the callback function will be called. For this reason, the callback function must accept two arguments, the first a boolean indicating if the request succeeded or not, the other a string containing the response from the call. If the first argument is false, the call may have timed out or failed.

Using the AJAST call function

There are two ways of using OX.AJAST. The simplest is to use the call function.

  // Create a function that will be called when the AJAST request completes
  function callCompleted(success, data)
  {
    if(!success)
      alert('Fail');
    else
      alert('Received: ' + data);
  }
  // Call a service
  OX.AJAST.call(
    'http://xampl.com/rest?arg=foo', 
    'callback', 
    callCompleted);

The call function will execute the request by appending &callback=wrapper to the URL and injecting a <script> tag with the final URL as the src attribute. This will add a call to the DOM as soon as the data is received, which the browser will execute.

The function called from the injected script tag is a wrapper around the callCompleted function provided to the call function. The wrapper function is created by the call function, and handles timeouts and deletion of the script tag after the callCompleted function completes. As mentioned, by using this wrapper, the AJAST library can guarantee that callCompleted will be called, which significantly eases the handling of asynchronous calls for users of the library.

The function also allows you to specify how long the request will wait for a response before it times out. The default timeout is 5 seconds. Finally, you can pass an argument specifying if you want the response to be automatically decoded from JSON before it is passed to your callback function.

// Call with a 10 second timeout, decode JSON response
OX.AJAST.call(
  'http://xampl.com/rest?arg=foo', 
  'callback', 
  callCompleted, 
  10000, 
  true);

As stated above, all callback functions must be on the form callbackfunction(success, data){}, where success indicates whether or not the asynchronous call succeeded, and data is any data that was received from the call. It is also important to note that data may be undefined, but success will always be true or false.

Using the AJAST broker

The AJAST broker encapsulates a common pattern for REST requests using HTTP GET. Many RESTful services found online typically use some kind of root URL of the form http://xampl.com/rest as the base URL for all their REST services. The query string determines which service is requested, as well as the arguments for the service.

For the services that follow this pattern, the AJAST library provides a Broker class that encapsulates the process of calling the REST services.

The example below shows how the request from the first example can be made using the broker.

// Create a broker object
var broker = new OX.AJAST.Broker(
  'http://xampl.com/rest', 
  'callback'
);
// Perform the same call using the broker
broker.call({arg: 'foo'}, callCompleted);

The broker also supports the specification of a timeout limit, automated JSON decoding, and also provides the option of passing a set of default arguments that will be passed with every request, such as a Flickr API key.

// Create a broker object
var broker = new OX.AJAST.Broker(
  'http://xampl.com/rest', 
  'callback', 
  true, // Decode JSON response
  10000, // Timeout in ms
  {APIKey : '123'} // Default parameters
);

Now let’s do something useful with it.

A real example: Flickr using AJAST

To keep the example as simple as possible, we’ll create the functions necessary for a page which fetches the most recent photos from Flickr.

Luckily, Flickr supports REST and JSON callbacks in a lovely manner, so we’ll use the broker for our calls.

function flickrGetRecent()
{
  // Create  a broker
  var broker = new OX.AJAST.Broker(
    'http://api.flickr.com/services/rest/', 
    'jsoncallback', 
    true, 
    10000,
    {api_key: 'YourVeryOwnFlickrApiKey', 
      format: 'json'});</p>

<p>// Perform the call
  broker.call(
    {method: 'flickr.photos.getRecent'}, 
    recentFetched);
}

We’ve told the broker to call a function named recentFetched when the recent photos have been fetched, so let’s implement that as well. To keep the example simple, we’ll just append the photos to the body of the document.

function recentFetched(success, rsp)
{
  // Check for failure
  if(!success || !rsp || rsp.stat != 'ok')
  {
    alert('Call failed');
    return;
  }</p>

<p>// For each photo...<br />
  for(i in rsp.photos.photo)
  {
    photo = rsp.photos.photo[i];</p>

<pre><code>// Create an img element
img = document.createElement('img');

// Set its source to a valid Flickr URL
img.setAttribute('src', 
  'http://farm' + (photo.farm || 1) + 
  '.static.flickr.com/' + photo.server + 
  '/' + photo.id + '_' + photo.secret + '_t.jpg');

// Append the element
document.body.appendChild(img);
</code></pre>

<p>}
}

Now we just have to call flickrGetRecent from somewhere in a document, and the most recent photos will be appended to the document. A full example can be seen here. Note that you will need a Flickr API key to test it.

As you can see, the OX.AJAST library is really easy to use, and enables you to do pure client-side REST service calls across domain boundaries with hardly any effort. I hope you find it useful. Drop a comment if you have problems or suggestions, or if you create improvements to it. Now start using AJAST!

The problem is that a relational database is not suited for storing an object-oriented model. An object is almost always a non-scalar value, meaning that it won’t fit well in a table row. Hence, we have to create a schema suitable for persisting our objects to a set of tables. This schema will be different for each type of object we have, so it’s a rather tedious task, but it solves the problem.

As always, we developers tend to dislike repetitive tasks, so we try to simplify and automate the extra work involved with creating an ORM. This has led to various design patterns such as DAO, and in recent years a set of fully automated ORM tools such as Hibernate (Java), NHibernate (.NET), SQLAlchemy (Python), and Propel (PHP) have become very popular. These tools offer a highly transparent solution to the ORM problem, at the addition of a cost that varies heavily with the nature of the ORM problem. Still, their presence moves us closer and closer to a solution – we are creating an abstraction that fully encapsulates the difference between a relational database as our storage for persistent objects, and the objects themselves. Hopefully, we will soon be able to create an ORM that induces a linear or perhaps even constant cost on our solution, regardless of the nature of the problem we are trying to solve.

But wait. The real problem we are trying to solve is how to persist the state of our objects, remember? The database is indeed a store, and hence a candidate solution, but using it introduces an impedance mismatch, which is another problem we need to solve. It really is like trying to fit a square block in a round hole.

Let’s stop trying to solve the wrong problem of creating an ORM, and start finding a solution to our initial and real problem of persisting our objects. Recent innovations such as LINQ has taken us a step in the right direction by making persistence an integral part of the language, but we’re still a long way from automated persistence for our objects. I am certain, though, that moving focus away from the challenges of an ORM and on to the challenge of persistence in general will take us further.

Let’s do that.

(And I’m sorry for reusing Dijkstra’s already overused phrase.)