First of all, I want to emphasize that I think both of the aforementioned tools are great efforts, and most certainly provide value to those using them.

Second, the reason for being a passive listener is that I believe debates of this kind are utterly meaningless and provide no value whatsoever; making and improving the tools at hand is what provides value. I will make an exception this time as this debate (among other things) discusses a (if not the) fundamental motivation for writing Continuous Testing for Visual Studio, and highlights one of the significantly differentiating aspects of the approach it takes.

Third, and most importantly, the statement from the Herding Code podcast is objectively true. Put in even clearer words: It is impossible for a Continuous Testing tool to do static analysis and achieve perfect and correct test minimization. Even near perfect test minimization is impossible. The only generally possible thing is simple dependency resolution. Don’t let anyone tell you otherwise.

That is why Continuous Testing for Visual Studio does not do static analysis¹, but relies purely on statistical heuristics to prioritize and give as immediate feedback as it possibly can, while still guaranteeing you the correct feedback.

Note the highlit words in the previous sentence. Immediate while correct. That is the core philosophy of Continuous Testing for Visual Studio. In the end, correct feedback matters infinitely much more than fast feedback when it comes to testing. Continuous Testing for Visual Studio tries to give you as immediate feedback as it can, while still giving you correct feedback.

It is up to you, dear reader, to try out the tools and find out which works best for you in your real world project. Try Continuous Testing for Visual Studio. While you do that, I’ll continue improving it.

–

¹ We obviously do static analysis to resolve dependencies etc., but not for optimizing away tests.

From now on, Continuous Testing for Visual Studio will be available in two flavors: A full-featured Professional version and a free version staying more or less as it is today.

Continuous Testing 2.0

Buy Professional

Download for VS2010

Download for VS 2008

The new Professional version features a number of improvements over the free version:

• Smart Test Run Prioritization records the outcome of each of your tests, and orders the next test runs to minimize your wait for feedback even further. Smart Test Run Prioritization will run your tests in order of most recently failing and most often failing. This provides a significant speed-up for larger projects.
• Abort on first failing test: You can choose to have Continuous Testing abort as soon as a test fails. This is useful for freeing up resources acquired by the test run as soon as feedback is available. This is another must-have feature in real world software development on large projects.
• Priority support: Send an e-mail and get a response within 24 hours.

The combined features of Continous Testing Professional make for a very powerful and efficient TDD workflow.

Ensuring the last failing test runs first and aborting as soon as a test fails enables you to do as much as possible of what you are supposed to do: Write code.

The free version will continue to stay free with at least the current features. Here are the most notable new features of the free version:

• A new integrated version of the test runners for MSTest and NUnit speeds up test runs. The executable test runners are still available. If you experience problems with running your tests using the integrated runners, try running your tests using the executable test runners instead.
• Continuous Testing can be told to ignore specific projects when scanning for tests. This is useful if you are working on a solution with many test projects, and only want to run some of those projects.
• Continuous Testing now has its own Output Window pane. Watch this pane to see what Continuous Testing is doing.

In addition, many improvements and bug fixes have made it into the release, ensuring a more stable and productive experience than before.

Happy testing and TDD-ing!

I remembered reading about JUnit Max by Kent Beck, a continuous testing plugin for Eclipse, that runs your unit tests in the background and unobtrusively tells you when a test fails, allowing you to do what you do best: write code. JUnit Max seems like a great thing, and now I needed the same thing for Visual Studio. A quick Google didn’t yield any add-ins, extensions or packages, so I decided to create one.

The result is Continuous Testing for Visual Studio, a small extension which runs your unit tests each time you build your solution, and reports failing tests to the error list so you can navigate to the line that failed and make the test pass. The extension significantly improves my workflow by removing a tedious manual step of running unit tests, so I encourage you to take it for a spin. Continuous Testing can be downloaded for Visual Studio 2008 and Visual Studio 2010. Future updates and versions will be announced on the Continuous Testing home page.

UPDATE Jun 17th, 2010: I’ve receive a lot of feedback through various solutions online. To be able to help you and/or improve Continuous Testing for Visual Studio, I need samples from you that reproduce the problems you are experiencing. Do not hesitate to leave a comment here, and provide your e-mail address when commenting, and you will receive a reply.