One of the topics brought up at the agile beer night was test fragility and how mocking frameworks can be a contributor to test smell. Actually, I'm the one who brought up the point about mocking frameworks and fragility, but further examination revealed some disparity on the subject, specifically related to replay semantics.
I know that when I first started using a mocking framework I assumed it was best practice to use strict mocks. I mean, why wouldn't you want to verify everything possible? Apparently I'm not the only one to think this at one point or another. I found this blog post by Derik Whittaker stating how he prefers strict mocks, it gets really interesting once you read through the comments. Scott Bellware made the following comment:
Strict mocks is often a design smell. It causes tests to express knowledge of an operation's internals, which is a violation of encapsulation of the system under test.
Any large code base that depends on strict mocking by default will suffer from unnecessary productivity loss due to breaking tests that have to be fixed where those tests suffer from inappropriate intimacy.
An API's internals are supposed to be allowed to change independently of its tests. This is an essential quality of an agile codebase.
Strict mocking by default is a rather extreme stance. Agile succeeds by finding those places where strictness can be slackened without causing any observable and categorical loss in integrity. Opting for the strictest possible default runs contrary to this heuristic.
Through years of experience with mocking frameworks have I come to the realization that loose mocks should be your default. Apparently the creator of Rhino Mocks, Ayende, agrees (emphasis is mine).
As an aside, I am deprecating CreateMock in favor of StrictMock. Using strict mocks by default was a bad design decision on my part.
One of the primary reasons mocks introduce fragility is that they require knowledge of the SUT that goes deeper than the public API. You often need to know what methods on a collaborator will get called and setup some sensible return values that force your SUT through a particular branch of code. To me that's the very definition of tightly coupled.
So, how do we take advantage of a mocking framework, without making our tests fragile and resistive to change? Unfortunately there's no way to completely avoid the mock object tax, but we can certainly minimize it.
One of the very first things we can do is setup our mocks in a shared fixture setup method. Sharing mock setup between tests is a great way to reduce the verbosity of each test. A shared setup method will also setup the SUT with the appropriate mocks.
[SetUp]
public void SetUp()
{
mocks = new MockRepository();
userRepository = mocks.DynamicMock<IUserRepository>();
purchaseService = mocks.DynamicMock<IPurchaseService>();
sut = new Controller(userRepository, purchaseService);
}
You'll notice that the shared fixture setup I've created is using Rhino Mocks dynamic mocks. You could instead use an auto mocking container to create the mocks for you to save some keystrokes and repetition.
With loose replay semantics (dynamic mocks in Rhino terminology) any method call during the replay state is accepted and if there is no special handling setup for this method a null or zero is returned. All the expected methods must be called if the object is to pass verification.
Too many times I've been burned by an over specified test using strict mocks. With strict replay semantics only the methods that were explicitly recorded are accepted as valid. This means that any call that is not expected would cause an exception and fail the test. All the expected methods must be called if the object is to pass verification. Depending on the mock this could be rather verbose leading to many expectations that have nothing to do with the actual test.
Tests should test only one thing. You may have multiple assertions or expectations per test, but all of those assertions should have the same goal or theme. I only want to setup the minimal amount of expectations on a mock that will verify a particular behavior in my SUT, anything more is over specified. More to the point, keep your tests short and concise.
By using dynamic mocks rather than strict mocks, we are free to ignore the parts of the interaction between the SUT and the mock that we don't care about for our particular test. We can also use setup result instead of expect if we can verify the interaction using a classicist approach.
[Test]
public void Should_show_cart_empty_message_if_cart_is_empty()
{
SetupResult.For(purchaseService.GetCart()).Return(new ShoppingCart());
mocks.ReplayAll();
sut.ShowCart();
Assert.AreEqual("You're shopping cart is empty", sut.Flash["warning"]);
}
Here I'm verifying behavior without explicit help of the mock framework, I'm only using the mocking framework to stub in my external dependencies and then verifying the expected behavior using a traditional assertion.
If it weren't for the external service dependency I probably wouldn't be using a mocking framework here. Mocking frameworks are most valuable at the edges of your components where things come together with the SUT, like databases, web services, and files.
The normal flow through the controller actually uses the purchase service for more than just grabbing the cart, it also uses it to grab the customer's billing information, but for this particular test I don't care whether the billing details get populated or not. Here's the simple controller method we're testing:
public void ShowCart()
{
PropertyBag["customer"] = userRepository.GetCustomerDetails(userID);
PropertyBag["billing"] = purchaseService.GetBillingInfo(userID);
ShoppingCart cart = purchaseService.GetCart();
if (cart.IsEmpty)
{
Flash["warning"] = "You're shopping cart is empty";
}
}
Lets say I now want to verify that the billing information gets populated when ShowCart is called. Here the tables are turned, I don't really care about the call to GetCart() on the purchaseService. I really only care that the property bag gets properly populated with billing information.
[Test]
public void Should_show_billing_info()
{
SetupResult.For(purchaseService.GetBillingInfo(0))
.IgnoreArguments()
.Return(new BillingInfo());
mocks.ReplayAll();
sut.ShowCart();
Assert.IsNotNull(sut.PropertyBag["billing"]);
Assert.IsInstanceOfType(typeof(BillingInfo), sut.PropertyBag["billing"]);
}
Perhaps now we're concerned with populating the correct billing information, because as you might imagine, showing someone else's billing info would be a horrible security flaw. So lets change our test to verify that using a mock.
[Test]
public void Should_show_billing_info()
{
sut.CurrentUserID = 5;
Expect.Call(purchaseService.GetBillingInfo(5))
.Return(new BillingInfo());
mocks.ReplayAll();
sut.ShowCart();
Assert.IsNotNull(sut.PropertyBag["billing"]);
Assert.IsInstanceOfType(typeof(BillingInfo), sut.PropertyBag["billing"]);
mocks.VerifyAll();
}
Just by changing from SetupResult.For to Expect.Call we are now explicitly checking that controller gets its billing information using the correct user ID. We also added in a VerifyAll call. Now if the purchaseService.GetBillingInfo method is called with anything other than 5, or not called, the test will fail.
By using dynamic mocks we've verified just enough to see if our SUT works without adding a bunch of noise or unneeded fragility to our tests.