In my current position one of the metrics we track is code coverage for our unit tests. When I started at the company we were using JUnit with Mockito and JaCoCo. This was a pretty good setup we got good coverage reports and Mockito makes the testing writing much easier.

One of the limitations of Mockito is that you can’t mock private methods or static methods. This presented an issue for us in reaching our desired level of coverage. We initially worked around some of the private method issues using reflection, but it wasn’t always ideal. The decision was made to use PowerMock. PowerMock solved all of our Mockito issues immediately. It was compatible with Mockito but gave us some new powerful features to allow us to get much better unit test coverage. Then we ran our Jacoco reports and found that the reporting no longer worked. Due to the way PowerMock uses byte code manipulation in order to mock static methods it is not compatible with JaCoCo and there is no plan for them to support measuring that.

So we figured no big deal and switched to Cobertura. The first problem with this change is that Cobertura 2.0.3 has a regression in it so it won’t report coverage with Powermock. We figured no big deal we will run the 1.9.4.1 release of it until they release a bug fix so we can update. Unfortunately the update has never come. You go to the roadmap for the site and you see it hasn’t been updated since November 7, 2013. There appears to be very little activity on the project, and we haven’t seen an update in 2 years. I see work going on in the github repository but there seems to be no attempt and doing any maintenance or fixing the issues for the existing users, and I can’t get a feel for when the 2.1 release is ever going to come out. A second issue is that the current version doesn’t support Java 8 and I would like to update to Java 8 in the very near future. At what point when dealing with open source software do you say the project is either dead or too inactive for us to rely on for business needs?

Cut to last week I was updating some libraries in the project and I wanted to upgrade from PowerMock 1.5.5 to PowerMock 1.6.1 and my coverage reports went to 0. So it seems our old version of Cobertura can’t handle the latest PowerMock. I did a test with Atlassian Clover and our coverage reports worked perfect and looked better than anything I have ever seen for a report. At that point I decided I had reached my breaking point with Cobertura and put in a request that we move to clover and buy some server licenses and work and in the meantime while waiting for approval I had to settle for PowerMock 1.5.6 until we can get approval to buy Clover licenses.

Going forward when someone suggests a new tool to fix an issue that we are having, I have to say that I am going to be looking into other things that tool drags along with it as I don’t want to be in the situation again where we are trading one problem for another.