Those who have the misfortune to have me on their code reviews see me repeat this often. Don't test implementation details.
What do I mean? In short, don't test something you are likely to refactor latter. Find a test point that isn't likely to change. All too often I see someone write a seemingly useful test, but it locks them into a design that shouldn't be locked in.
Mocks are the primary way I see this rule violated. Your code calls a function with some arguments, so you have your mock check for those. However there are often several different ways to use the function that in the end give the same result. Your tests should support changing implementations to see what is better.
For example, the standard file read function has a length argument, if you mock this, your mock needs to be powerful enough to handle calling it once with a buffer size of 1000 bytes, or twice with a 500 byte buffer. Both are equally valid, and depending one how you use the data and performance considerations one night be better. As hardware changes which is better may change. With good tests you can try many different read sizes to see if any make a difference in the real world.
Read the above again, but not with this in mind: Computers have been doing file IO for many years. We understand it well. It is very unlikely someone will come up with a new way to do file IO that is completely different from what your program supports today, and compellingly enough better that you would change your program to use it.
Contrast to your typical program. The internal API is very much in flux because nobody knows what the right way to solve the problem is. It seems to be taken as a matter of faith that you should break your tests up, but where do you break things up? Once you break something up you won't touch it that API again. Not that you can't touch it, just that anyone who does will be afraid of all the tests that break and give up instead.
One role of architecture is figure this out, but they only get a part of the work. Anytime architecture gives you a well defined interface, that is a great place to jump in with test doubles: at least you know that it is intended to be hard to change that API. Thus your tests are a little less fragile: you won't be making changes anyway. However often we want more granular isolation, and then test doubles stand in the way of change.
Figuring out the answer to this question is in fact one of the reasons I started this blog. I don't know what is right, but I'm starting to recognize a problem. Expect to see this topic explored in greater detail over time.
Note, don't take my comments on file IO as a statement that I believe you should mock file IO. There is actually a lot of debate on this. There are some people who say you never mock anything that you don't own both sides of. They will tell you never mock file IO.
Read the above again, but not with this in mind: Computers have been doing file IO for many years. We understand it well. It is very unlikely someone will come up with a new way to do file IO that is completely different from what your program supports today, and compellingly enough better that you would change your program to use it.
Contrast to your typical program. The internal API is very much in flux because nobody knows what the right way to solve the problem is. It seems to be taken as a matter of faith that you should break your tests up, but where do you break things up? Once you break something up you won't touch it that API again. Not that you can't touch it, just that anyone who does will be afraid of all the tests that break and give up instead.
One role of architecture is figure this out, but they only get a part of the work. Anytime architecture gives you a well defined interface, that is a great place to jump in with test doubles: at least you know that it is intended to be hard to change that API. Thus your tests are a little less fragile: you won't be making changes anyway. However often we want more granular isolation, and then test doubles stand in the way of change.
Figuring out the answer to this question is in fact one of the reasons I started this blog. I don't know what is right, but I'm starting to recognize a problem. Expect to see this topic explored in greater detail over time.
Note, don't take my comments on file IO as a statement that I believe you should mock file IO. There is actually a lot of debate on this. There are some people who say you never mock anything that you don't own both sides of. They will tell you never mock file IO.
No comments:
Post a Comment