Mark Nichols

This post is a recap of the podcast that Clark Sell and I just recorded at DeveloperSmackdown.com (Musing #12 – Are You Mocking Me?!). 

Recently, on a project, I needed to describe mocks and stubs, what the difference is, how and when they are used and their benefits.  There are quite a few blog posts out there that describe the differences but many of them are confusing and I needed a way to clarify to my team members so I created the following content.

Commonalities:

Mocks and stubs are both ways to fake out methods being tested so they think they are using a real system resource.  System resources could be internal application objects, objects that are provided by the OS, framework objects, etc.  The issue with all of those objects is that they are either expensive to use, take excessive time to create and use, are very hard to set up for use within a test or are difficult to access during a test run.  Ultimately, the problem you are trying to solve is to isolate and unit test a method and you shouldn’t care about any external objects for that test.  You should be able to assume that they will do what they are supposed to under the given test conditions.  Trying to isolate a method under test when you have to access a back-end system that includes network communications and multiple database updates is just too difficult (in my opinion).  Using a mechanism during the test that can easily simulate the call to a resource is much easier to manage.  Mocks and stubs provide that mechanism.

Both mocks and stubs can be implemented through the use of frameworks.  The frameworks are usually for and are named after mocks (Moq, RhinoMock, TypeMock, NMock…) but they can be used to generate stubs.  If you don’t want to use a framework, stubs are easier to create in a roll your own scenario since they can easily consist of a set of simple method call implementations that just turn around and return data.  Mocks are more difficult to create on your own since they have to track more information around what happened during a test as compared to just returning data.

Differences:

To put it in a single sentence, mocks are about behavior and stubs are about state.  What the hell does that mean you say!?  A unit test that uses a stub is interested in being able to make a call to an object and getting data back that will allow the method under test the ability to continue with a given set of values.  Those values are designed to flex the method’s abilities and see if it succeeds in all the proper test scenarios.  So, stubs provide state (data) back to the method under test and your unit test has all the information it needs within itself to determine success or failure.

Mocks on the other hand are about behavior.  Under the current conditions, did my method pass the right values? Did it make all of the right calls and ONLY the calls that were necessary? As a demonstration to my team, I created a simple application to show how this works.  I tested method that had the requirement of NOT making any unnecessary database updates if there are no changes in the data.  It called the database, got some data back, made some calculations and then had an unqualified database update at the end of the method.  In my unit test, I made the call and checked the return value of the method under test.  Everything seemed to be ok so the test passed right?  Well, it passed but it really shouldn’t have.  When I used a mocking framework to set up the test and created an expectation that under those conditions a database update was not warranted, IOW, I put a “verify” statement at the end of the test, the test now fails which is what should happen.  It would not pass until I changed my code to validate an update before it performs one.  The best part of all of this is that I didn’t need a real connection to a database to verify my code.  Even if I had used a real database instead of a mock, taken the time to set up the test data, ran the test, cleaned up all my tracks after the test, I still wouldn’t know that my method did the unnecessary update. 

Mocks support the fact that my unit test, on it’s own, cannot make the determination that the entire test was a success.  It needs to request that information from the mock as the last step in verifying that the test was truly a success.  Hopefully, the following helps to illustrate this.

Here you can see the steps involved with creating and using a stub to help perform the test.  The test creates the stub, creates the object being tested, and performs dependency injection to make the method being tested use the stub.  The method under test is called which, in turn, calls the stub.  The stub returns data allowing the method under test to do its calculations and then return to the test method.  The test method is then able to evaluate success or failure on its own and the testing is done.

With a mock, the main steps are similar although the “Create the Mock” step is more involved – expectations of the behavior have to be asserted that tell the mock what behavior (what calls, possible parameter values, etc.) should occur.  The various frameworks provide simple interfaces to set that up.  The big difference in the execution of the test is that for the unit test to be able to determine complete success or failure, it has to ask the mock.  Method calls could have been made properly, but then again, maybe not.  Unless we have a mechanism in place that will track that we don’t know for sure.  A mock can tell us this.

Frameworks:

The best part of using a framework is that it makes the creation of mocks AND stubs easier.  If you have an object that you have to use in a test and it has 50 public methods and 20 properties in its interface (and you will need an interface), it is way easier to use a framework that will automatically create concrete calls for you as compared to you trying to create them on your own.  So, yes, this means that even if you don’t need to test behavior, you can use a mocking framework to create stubs for your tests.  You just don’t set up expectations or call verify at the end.

Clark and I have both used Moq, NMock and RhinoMock.  Each has their own benefits and shortcomings.  I don’t have a recommendation since picking one over the other is much like picking out any application – requirements should drive the decision.  I used Moq most recently and was impressed because of the ability to use lamda statements to create succinct expectations however, on a previous project, I had very good luck with RhinoMock.

So just remember: stubs is state – mocks is behavior!

Here is a zipped version of the test project code that I mentioned above.  It has a dependency on the Moq Mocking Framework so if you want to try to run the tests you will need to download version 3.1 and add the reference to the test project.