Before I left on vacation a couple of weeks ago I was working with another developer on a story that made use of some 3rd party services on a VM.  Before leaving I gave him my VM to use so we could avoid the unnecessary PITA of configuring  a new environment. 

Unfortunately now that I've come back I would like to run my VM again, but I can't since he is still running it and we can't share instances (isolated development environment). 

What I ended up doing was taking the VM image as I had it before I left and essentially cloned it.  By renaming the PC and changing the SIDs on the box I essentially have a cloned VM that will peacefully coexist on the same Windows network.

Here are the steps to take an existing VM joined to the domain and essentially clone it as a different machine.

1.       Unjoin the machine from the domain.  Reboot.

2.       Download NewSid utility from Microsoft (sysinternals).

3.       Run NewSid from the command line: newsid /a mynewpcname

4.       After reboot, login and rejoin the PC to the domain.

5.       Done

From then on the two VMs can coexist with one another on the same network and domain.

Test code is sometimes more difficult to maintain then production code for various reasons, but chief among these reasons is that developers just don't know how to organize their test code.  Poorly organized tests lead to poor performing tests, which leads to tests not being run as frequently as they should, which leads to more frequent and longer broken builds, which leads to fewer checkins, which leads to a loss of productivity, which leads to helpless little kittens dying in Africa*...

Test code should be organized along test types: unit, integration, functional, acceptance, and perhaps some others that tend to be more QA specific.  I often use unit and integration as the base primitives for my code bases.  Its not uncommon that I will have 2 test assemblies (unit and integration) for each assembly under test.  Essentially all my fast unit tests that I constantly run are in the unit test project, while all the slower tests (that hit the DB) go into the integration project.

Why the fine grained separation?  Why not just use one monolithic test project per solution? 

The reason we split test code up is the same reason we split up a regular application, for maintainability.  Well tested applications have an equal or greater number of lines of test code when compared to the actual application code.  If we had just one test project for an entire solution it just wouldn't be manageable for anything but the smallest of solutions.

By splitting the unit and integration (aka slow) tests into their own project it becomes much easier to run just the fast in memory unit tests all the time.  Categories are nice, and indeed useful for test segregation, however it still requires some filtering in the testrunner to only run the unit tests.  Splitting unit and integration tests into separate projects just makes things faster and reinforces the intent of each particular test area.

As the code base grows we will find that build times increase.  By having smaller test projects we may find we're not building as much code which helps us keep the TDD rhythm.  It also gives us the flexibility to move our core library to a separate solution if we wish, which may be beneficial if there is a lot of dependencies on that library project.  If we had one solution wide test project we wouldn't be able to do this.

Speed is King

Your unit tests need to be blazingly fast, anything else will lead to less effective tests.  Tests need to be run often to be relevant.  What you will find over time is that tests that hit external dependencies like databases or services are at least several times slower than a real unit test that has no external dependencies.  Several hundred tests that hit a database will absolutely kill your test speed.

The important distinction between integration and unit tests is speed, test speed should guide you where to put a particular test.  Slower tests go into the integration project, while the fast tests go into the unit test project.

The distinction between unit and integration tests isn't whether the SUT is completely isolated from its collaborators. Its OK to have unit tests that are using concrete collaborators, no where is there a rule requiring us to mock all collaborators.  Does the system really need a proliferation of interfaces that are all internal to the assembly itself?  I would argue in the general case no, you're probably wasting your time writing and maintaining too many interfaces.  Interfaces should be used as general extensibility points not only as mocking extension points.

The reason we call slow tests integration tests is that they usually have one thing in common, access to an external resource (like a database).  External calls outside the process are a magnitude slower than in process calls.  When choosing what type of test to write, its almost always preferable to choose to write a unit test.  Your unit tests are your first line of defense against defects.

When to run

With our tests organized and segregated between fast and slow, we can make further optimizations to our build pipeline.

Unit tests, specifically the fixture(s) that are relevant to what a developer is working on will get run over and over again while development takes place.  Before checkin, ideally several times a day, a developer will run all the unit tests.  This keeps the feedback on their progress immediate, which is only possible because we have fast unit tests.

Upon checkin our multiphase CI process takes the code, builds it, and runs all the unit tests.  This ensures a minimal amount of functionality about the software that should be good enough to prevent any blocking issues for other developers.  This build fails/passes the build and should always be kept green and fast.

The second phase of our build process then goes through and runs all of the slow integration tests as a secondary build.  Depending on the amount of hardware and number of tests this may happen on every build or at scheduled intervals; perhaps a nightly build process runs all the slow build targets, things like: integration tests, creating the installer, and API documentation.

Failing integration tests should ideally be rare, except for perhaps legacy systems that lack unit test coverage.  Broken integration tests almost always means that you are missing unit test coverage somewhere.  If possible your first step to fix a broken integration test is to write one or more failing unit tests and then make them green.  Once you have those unit tests in place, go back and run the integration tests before checking in.

Summary

With some organization and thought about where to place and how to write your tests you will be in a much better position a year down the road when your product's code base has really grown.  By keeping the unit tests fast we keep the build fast, which enables faster turnaround on checkins and features.  Do it for the kittens!

*I don't have any numbers to back up my claim about poor innocent kittens being harmed by poor performing unit tests, but it makes me take pause.  Can I save a kitten with this test?

Somehow I managed to find myself reading a very long rant on RoR and the RoR community by Zed, the creator of Mongrel.  He picks on lots of people, groups of people, technologies, and companies, but here's some of my favorite parts from his rant.

This is exactly what makes Rails a ghetto. A bunch of half-trained former PHP morons who never bother to sit down and really learn the computer science they were too good to study in college.

You hear that? The #1 money maker for 2008 years will be Rails cleanup. I’m not shitting you, it’s true and so just get over it and make the money.

If anyone had known Rails was that unstable they would have laughed in his face. Think about it further, this means that the creator of Rails in his flagship products could not keep them running for longer than 4 minutes on average. Repeat that to yourself. “He couldn’t keep his own servers running for longer than 4 minutes on average.” Assuming his statements are true (which we may never know) he basically duped us all.

What pisses me off is that I know they’re responsible (ThoughtWorks) for turning Ruby on Rails into the next Visual Basic.

After ThoughtWorks left the most recent project we revamped the team. We got rid of pair programming, cut down the number of tests, started cleaning more and more code out, got rid of their shitty tools, and we all started leaving at 6pm. What happens? We doubled our productivity with fewer people.

First it started on the fringe in start-ups and a few lonely places where it’s having mixed success (mostly due to the poor performance of the Ruby platform). Now it’s getting adopted internally at companies where of course it’ll get fucked up again and die off. After that it’ll move to the government sector where it will languish along with it’s new found buddy COBOL.

One of the topics brought up at the agile beer night was test fragility and how mocking frameworks can be a contributor to test smell.  Actually, I'm the one who brought up the point about mocking frameworks and fragility, but further examination revealed some disparity on the subject, specifically related to replay semantics.

I know that when I first started using a mocking framework I assumed it was best practice to use strict mocks.  I mean, why wouldn't you want to verify everything possible?  Apparently I'm not the only one to think this at one point or another.  I found this blog post by Derik Whittaker stating how he prefers strict mocks, it gets really interesting once you read through the comments.  Scott Bellware made the following comment:

Strict mocks is often a design smell.  It causes tests to express knowledge of an operation's internals, which is a violation of encapsulation of the system under test.

Any large code base that depends on strict mocking by default will suffer from unnecessary productivity loss due to breaking tests that have to be fixed where those tests suffer from inappropriate intimacy.

An API's internals are supposed to be allowed to change independently of its tests.  This is an essential quality of an agile codebase.

Strict mocking by default is a rather extreme stance.  Agile succeeds by finding those places where strictness can be slackened without causing any observable and categorical loss in integrity.  Opting for the strictest possible default runs contrary to this heuristic.

Through years of experience with mocking frameworks have I come to the realization that loose mocks should be your default.  Apparently the creator of Rhino Mocks, Ayende, agrees (emphasis is mine).

As an aside, I am deprecating CreateMock in favor of StrictMock. Using strict mocks by default was a bad design decision on my part.

One of the primary reasons mocks introduce fragility is that they require knowledge of the SUT that goes deeper than the public API.  You often need to know what methods on a collaborator will get called and setup some sensible return values that force your SUT through a particular branch of code. To me that's the very definition of tightly coupled.

So, how do we take advantage of a mocking framework, without making our tests fragile and resistive to change?  Unfortunately there's no way to completely avoid the mock object tax, but we can certainly minimize it.

One of the very first things we can do is setup our mocks in a shared fixture setup method. Sharing mock setup between tests is a great way to reduce the verbosity of each test.  A shared setup method will also setup the SUT with the appropriate mocks.

[SetUp]
public void SetUp()
{
    mocks = new MockRepository();
    userRepository = mocks.DynamicMock<IUserRepository>();
    purchaseService = mocks.DynamicMock<IPurchaseService>();
    sut = new Controller(userRepository, purchaseService);
}

You'll notice that the shared fixture setup I've created is using Rhino Mocks dynamic mocks.  You could instead use an auto mocking container to create the mocks for you to save some keystrokes and repetition.

With loose replay semantics (dynamic mocks in Rhino terminology) any method call during the replay state is accepted and if there is no special handling setup for this method a null or zero is returned. All the expected methods must be called if the object is to pass verification.

Too many times I've been burned by an over specified test using strict mocks.  With strict replay semantics only the methods that were explicitly recorded are accepted as valid. This means that any call that is not expected would cause an exception and fail the test. All the expected methods must be called if the object is to pass verification.  Depending on the mock this could be rather verbose leading to many expectations that have nothing to do with the actual test.

Tests should test only one thing.  You may have multiple assertions or expectations per test, but all of those assertions should have the same goal or theme.  I only want to setup the minimal amount of expectations on a mock that will verify a particular behavior in my SUT, anything more is over specified.  More to the point, keep your tests short and concise.

By using dynamic mocks rather than strict mocks, we are free to ignore the parts of the interaction between the SUT and the mock that we don't care about for our particular test.  We can also use setup result instead of expect if we can verify the interaction using a classicist approach.

[Test]
public void Should_show_cart_empty_message_if_cart_is_empty()
{
    SetupResult.For(purchaseService.GetCart()).Return(new ShoppingCart());
    mocks.ReplayAll();

    sut.ShowCart();

    Assert.AreEqual("You're shopping cart is empty", sut.Flash["warning"]);
}

Here I'm verifying behavior without explicit help of the mock framework, I'm only using the mocking framework to stub in my external dependencies and then verifying the expected behavior using a traditional assertion. 

If it weren't for the external service dependency I probably wouldn't be using a mocking framework here.  Mocking frameworks are most valuable at the edges of your components where things come together with the SUT, like databases, web services, and files.

The normal flow through the controller actually uses the purchase service for more than just grabbing the cart, it also uses it to grab the customer's billing information, but for this particular test I don't care whether the billing details get populated or not.  Here's the simple controller method we're testing:

public void ShowCart()
{
    PropertyBag["customer"] = userRepository.GetCustomerDetails(userID);
    PropertyBag["billing"] = purchaseService.GetBillingInfo(userID);

    ShoppingCart cart = purchaseService.GetCart();
    if (cart.IsEmpty)
    {
        Flash["warning"] = "You're shopping cart is empty";
    }
}

Lets say I now want to verify that the billing information gets populated when ShowCart is called.  Here the tables are turned, I don't really care about the call to GetCart() on the purchaseService.  I really only care that the property bag gets properly populated with billing information.

[Test]
public void Should_show_billing_info()
{
    SetupResult.For(purchaseService.GetBillingInfo(0))
        .IgnoreArguments()
        .Return(new BillingInfo());

    mocks.ReplayAll();

    sut.ShowCart();

    Assert.IsNotNull(sut.PropertyBag["billing"]);
    Assert.IsInstanceOfType(typeof(BillingInfo), sut.PropertyBag["billing"]);
}

Perhaps now we're concerned with populating the correct billing information, because as you might imagine, showing someone else's billing info would be a horrible security flaw.  So lets change our test to verify that using a mock.

[Test]
public void Should_show_billing_info()
{
    sut.CurrentUserID = 5;
    Expect.Call(purchaseService.GetBillingInfo(5))
        .Return(new BillingInfo());

    mocks.ReplayAll();

    sut.ShowCart();

    Assert.IsNotNull(sut.PropertyBag["billing"]);
    Assert.IsInstanceOfType(typeof(BillingInfo), sut.PropertyBag["billing"]);

    mocks.VerifyAll();
}

Just by changing from SetupResult.For to Expect.Call we are now explicitly checking that controller gets its billing information using the correct user ID.  We also added in a VerifyAll call.  Now if the purchaseService.GetBillingInfo method is called with anything other than 5, or not called, the test will fail.

By using dynamic mocks we've verified just enough to see if our SUT works without adding a bunch of noise or unneeded fragility to our tests.

I finally figured out why my code snippets are all so horribly formatted via RSS, and it has nothing to do with the different source code formatters I've been using and everything to do with dasBlog.  I grabbed the dasBlog source and found the problem is with dasBlog's HTML tidy class, but more importantly I also found in the source how to disable it.  The latest dasBlog source has an option to disable formatting.

In the latest version of dasBlog they've added a setting to disable RSS formatting. Notice the setting even mentions "may mess up pre whitespace."  Mess up pre whitespace?  Yeah, no kidding.

At least now I don't have to switch blogging platforms, as the upgrade to dasBlog 2.0 is a whole lot easier.  Here's the dasBlog source code that optionally disables the formatting:

if (siteConfig.HtmlTidyContent == false)
{
    item.Description = "<div>" + PreprocessItemContent(entry.EntryId, entry.Content) + "</div>";
}
else
{
    item.Description = ContentFormatter.FormatContentAsHTML(PreprocessItemContent(entry.EntryId, entry.Content));
}

After much trial and error I finally gave up and asked Ayende what he uses to post to his blog.  His posts are always nicely formatted - including code snippets in Google Reader

It's that code snippet part that I have problems with, so I'm going to give the Syntax Highlighter for WLW that Ayende uses a shot.  If it works for him, it should work for me. Lets give it another shot shall we?

[TestFixture]
public class UnitOfWorkTests
{
    [Test]
    public void Should_return_same_instance()
    {
        var c = new Customer { Address = new Address(), CustomerID = 1, Name = "Sneal" };

        UnitOfWork uow = new UnitOfWork(new StubDataStore());
        uow.Save(c);

        Assert.AreSame(c, uow.Single(new Customer { CustomerID = 1 }));
    }

    [Test]
    public void Can_save_instance()
    {
        var c = new Customer { Address = new Address(), CustomerID = 1, Name = "Sneal" };

        UnitOfWork uow = new UnitOfWork(new StubDataStore());
        uow.Save(c);       
    }
}

Well that was nice, and boy does this thing support a lot of programming languages (way more than I know).

I've been using Windows Live Writer for a while now and would never go back to directly posting via a webform, WLW is just too easy.

I brought the topic of code comments up the other day at lunch to get some other's opinions about method level and class level comments, more specifically C# XML comments.  I brought the subject up because we have a policy at work to comment all methods, public and private, which I'm a bit dubious about.

It was interesting to see how widely opinions varied on the subject, from everything should be commented to produce nice CHM files to my position that comments are a last resort to explain the code to the next developer.

The best code needs no commenting at all.

public string ConcatStrings(string lhs, string rhs)
{
    return lhs + rhs;
}

That's good code.  Why?  Because I finally figured out how to join to strings together!  That may be part of it, but I deem this method pretty decent for the additional following reasons:

1. Fluent name, the method name describes exactly what the method does.
2. Short method.  The method name describes everything the method does because its small.

I think one of the best and easiest techniques to producing self documenting code is to use the extract method refactoring.  Take some code, extract a well named method out of it, and the readability goes up.

Fluent interfaces and DSL's take readable code a step further, and I think that's why they've become so popular. That, and its so damn easy to write an internal DSL in the cool popular dynamic languages like Ruby.  Why write comments buried in source code when you can write beautiful code instead? 

Have you seen developers obsessed with comments to the point of idiocy?  I know I have where I began to suspect the developer actually enjoyed writing comments more than code, or maybe writing code was too difficult for them so they focused on the low hanging fruit of comments?  Have your secretary comment your code, we as programmers should focus on writing code, good code.

Comments are not for documenting who made changes or when.  Heard of source control comments?  Heard of blame?  No?  Go learn them now.  In my experience DBA's are the worst offenders of this horrible practice, there must be something in their paranoid nature that makes them do this... obsessively.  There's absolutely no reason to litter beautiful code with code change comments, doing so only muddles the clear waters of code.

Back to my original point, I wait until I'm ready to checkin to add my comments.  In fact adding comments is one of my checkin practices.  Adding comments before then is often a waste of time.  How often have you written a method, an entire class, or even an entire namespace that never sees the light of source control?  I've mercilessly axed more JavaDoc comments and classes than I care to remember... more than I probably ever checked in.

And then the other day Jeff Atwood had a blog post about comments, and he said very concisely what I was thinking at lunch.

When you've rewritten, refactored, and rearchitected your code a dozen times to make it easy for your fellow developers to read and understand -- when you can't possibly imagine any conceivable way your code could be changed to become more straightforward and obvious -- then, and only then, should you feel compelled to add a comment explaining what your code does.

If there ever were a take away about a discussion around code comments, that's it.  Well put Jeff.

If you wait until you're happy with a piece of code, when its truly done enough to checkin you'll write less comments and better code.  Remember code comments can be a code smell.

I'm trying out a new Live Writer code formatting add in. I've had mixed results with the others I've used.  It seems I can't find one that is:

1. Easy to setup and use.  Copy, paste, done.
2. Works on my blog page.
3. Also works in RSS readers like Google Reader.

Lets see how the Insert from Visual Studio addin works...

[TestFixture]
public class UnitOfWorkTests
{
    [Test]
    public void Should_return_same_instance()
    {
        var c = new Customer { Address = new Address(), CustomerID = 1, Name = "Sneal" };

        UnitOfWork uow = new UnitOfWork(new StubDataStore());
        uow.Save(c);

        Assert.AreSame(c, uow.Single(new Customer { CustomerID = 1 }));
    }

    [Test]
    public void Can_save_instance()
    {
        var c = new Customer { Address = new Address(), CustomerID = 1, Name = "Sneal" };

        UnitOfWork uow = new UnitOfWork(new StubDataStore());
        uow.Save(c);       
    }
}

If nothing else, at least it looks pretty from Live Writer.  Lets see when it gets out in the wild...

Specifically ID and CLASS attributes are case sensitive when it comes to CSS.  If you take a look at the HTML 4 spec you'll notice that both ID and CSS have [CS] annotation for case-sensitive. 

Although this has always been the case, very few developers know this as a fact.  Its obvious Microsoft missed this in the first few HTML 4 compatible versions of IE.  In fact it wasn't until IE 6, when run in strict mode, would IE in fact honor the case sensitive nature of the CLASS and ID attribute.

I think its because of the early IE 'quirks' mode implementation that very few developers actually know that ID and CLASS are in fact case sensitive. They may guess correctly, but very few actually know this from experience.  A simple quiz at work today confirmed my suspicion.

Well after today, I know from experience that CSS is case sensitive.  <div class="msg-Error"/>  is not the same as  <div class="msg-error"/>.

So in summary if your CSS is not getting applied as expected you might want to check the casing between your HTML and your CSS.

I committed an example class library (utility) which drives some JsUnit tests and gathers the results.  This is handy if you want the results inside of a C# application or an NUnit test.  Basically the JsUnit webform is posted to an ASHX handler running under the .NET 2.0 webserver. 

Its not real pretty or that usable in its current form, but it has some good starting points. I'm not sure it actually works since I lost the original source code and had to decompile it using Reflector.  It at least compiles...  Do whatever you like with the source, its available in my SVN repository here.