Hi guys, in my last post: Why is it important to have a “Green Build” I did explain how important is to have a Green Build. In this post I will explain several reasons why it is so complicated to keep it green.
Before I start to enumerate several possible reasons I want to highlight the background of our system. We have a long way to go until we are truly agile. We have all the necessary skills within the Feature Squad to release a product but the organization is structured in different departments (System Build, Development, QA, etc.). This set up is responsible for the presence of silos. We are working toward an organization where silos will not exist but this is how organization works. On top of this the developer must wait several hours to get some feedback about what he did. He only knows if he broke something several hours later. Now with this information you will understand better the problems we are facing.
There are four different areas that I want to touch:
- Code changes
- Tests changes
- Build System
- Test System
Based on the previous topics I want to tell you a story of a project where I was involved. It was a complex; the technology was new and difficult and the teams were not all collocated. I mentioned before that the developers had to wait several hours to get the feedback to know if they had broken something. As a consequence the product was constantly broken. In this case the basic unit test coverage was not enough to prevent the developers from breaking functionality. These teams still have a long way to go to become truly Agile. One of the areas where we could see this was by looking at the frequency of their commits, some of the guys did not commit their changes for days in a row. Which resulted in a lot of time lost trying to merge their changes into the trunk. And above all this complexity they also had to deal with a complex version control system, allowing developers to make many mistakes while checking in.
The testing part was still acting like they were before, they were now part of the same team but they acted like they were still in the Silo’s. F.ex. new tests were still only written by the QA people, not by the developers who implemented the code. This meant that sometimes the test were failing for several days because the people writing the tests were not the ones that needed to fix the code. Another problem they faced was that when the functionality changed and the QA people were not notified that the tests would not be adapted and then fail for the wrong reason.
And finally we had some problems that showed up once in a while, but important enough to mention here. The build system did not belong to the development team, instead the build system belonged to a whole separate department. Which meant that when a problem related to the build occurred hey sometimes had to wait for several days before the build was green again.
Having the above mentioned problems meant that finding a root cause was extremely difficult.
I don’t have the perfect solution for all these problems, I have my beliefs and my opinions but I don’t know how to solve all issues in a short time. Since I do not want to finish this blog without any ideas I will give an example of another team we have where the results are extremely positive.
This small team has the ownership of everything. They own the build system, testing system, production system and whatever is needed for the work. They practice TDD, and there are no departments saying how they should work. They solve their own problems on daily basis, they do whatever is needed to make it “green”. They commit their code every couple of hours, they don’t go home if the code does not compile and all the test are green. I believe the biggest advantage of this team is the fact they know why a green build is so important. After failing so many times they understood why a green build is mandatory.
All organizations should aim for continuous improvement. However I truly believe the first step is to understand where we are and where we want to go. So if your organization has some problems, bigger or smaller, does not matter. I believe first you should realize which problems you have and define a plan where you want to go. Then just work with the rest of the organization to achieve those goals. Do not forget to empower your teams, they know what to do much better than you.
Please let me know what do you think.
Thanks,
Luis
Like this:
Like Loading...