Continuous Integration/Delivery Anti-Pattern – Script Wrapper

Continuous Integration – Anti Pattern

Emergence of on demand infrastructure and need for faster time to market drove the adoption of continuous integration and delivery practices in software organizations in last few years. Most of organizations today do have some level of maturity in CI & CD implementations but the journey is far from complete. One of the key success factors in implementing the CI/CD practices and tooling is how well designed they are. A well designed CI/CD pipeline is easily maintainable and extendable for future while a poorly designed might add more work than reduce from future. Today we will focus on one such common anti-pattern which is indicator of poor design and will lead to extra effort in maintenance.

One large script wrapped by the tooling

Imagine you have a large shell script of a few thousand lines. Now the team decides to implement a tool such as Jenkins instead of calling the script from command line. What is easiest way to retrofit the existing script to Jenkins way of doing things? Create a job with “execute shell” component and call the underlying script. Even in a greenfield implementation it is it possible that you start out with a small shell script within execute block and eventually grow to a large shell block within Jenkins. While this will definitely work, let’s look at pros and cons of such shift and lift migration:

• Pros
  • Simplest path to migration without much effort in development/testing etc.
  • If you switch to a different tool tomorrow – the migration is still simple as most of underlying logic is inside the shell script
• Cons
  • Script is not easily version controlled (You can use Jenkins job history but that is not very easily readable at script level)
  • Debugging a job basically means debugging the underlying large script – which is not very efficient as script is large.
  • If two jobs need slight change in same script – you might end up having multiple running versions of same script and over time they might become two very different scripts.
  • Reusability and modularity is very less

So what is a better solution? Let’s talk about generic solution and how can we implement it in the tool we are discussing (Jenkins).

Recommendations:

• Break large scripts into smaller chunks which do one only one thing (Single responsibility principle).
• These smaller chunks should be created as “Action/Step” in the tool and then interconnected to create a workflow which will accomplish same task as script.
• All of these smaller chunks should be version controlled – and over period of time breakup/parameterize to meet changes.

Within Jenkins it is possible to use a plugin such as “Managed Script Plugin” and break the large script into smaller version controlled script.

Overall result of breaking the large script into smaller actions is:

• Increased reusability across jobs – the same block is reused across multiple jobs.
• Quick to identify step that failed and hence better maintainability.
• Easy to create new workflows by mixing and matching steps.

We will look at more anti-patterns in coming posts. What kind of anti patterns do you see in CI/CD implementation in your enterprise?

Looking for help with building your CI/CD pipelines? learn why so many startups and enterprises consider as their CI/CD consulting & services provider.