We see many organizations are trying to implement DevSecOps. However, many of these implementations are not generating the expected benefits. “Gartner’s analysis cites that the actual code a developer writes is less than 10% of the final application.” – https://anchore.com/blog/gartners-12-things-to-get-right-for-successful-devsecops-a-study-in-devsecops-best-practices/.
This is making the security most critical and at the same time most challenging. The main problems that we see leading to the failure of DevSecOps are as follows:
Culture
The most important problem is the Culture change. Changing to New Way of Working is difficult to accept for a lot. Change is not natural for all. The organization must seriously look at Organization Change Management.
Organization needs to help the people to adopt the change. First and foremost, the mindset must be changed from “Security as an afterthought” to “Security first” mindset. This requires a lot of training and communication at various levels.
The next change is from “Project” to “Product” approach. Here the Product/Value Stream Team structure is required. This needs Organization Design change. Kotter’s Dual Operating Model is required. The Organization Structure can be Functional for governance, but execution should be in a Network structure. There is also need for Team based / Outcome based appraisals, moving away from Individual based / Output based appraisal. There must be a Cadence across the organization, breaking down the silos. We need Continuous Delivery – from “Silos” to “Cadence”. If there is a 2-week sprint, the appraisal also is 2 weeks, release cycle is also 2 weeks. Every support function across the organization should be in same cadence. The team also has one common objective of delivering quality product / service faster, secure and the team should be happy.
Today the changes are very frequent in the VUCA world. Customer needs changing, technology changing, competition is changing. To enable everyone to deliver Value, we need to change from “Requirement based” to “Collaborative Hypothesis based” Development. The Customer and the Service Provider collaboratively experiments on the hypothesis to come out with a solution which gets implemented. Thus, everyone is happy with it, the value being co-created.
This needs another aspect to be looked at. We cannot have big-bang changes. We do not need the “T” of Transformation, i.e., the big-bang transformation. We need “t” of transformation, i.e., evolutionary transformation. To be able to do this we need to re-architect our systems from “Monolithic” to “Microservices”. Everything must be done in smaller chunk more frequently. Creating smaller ones are easier, modifying smaller ones are easier, finding defects in small pieces are faster and throwing away small pieces are less costly. Securing smaller pieces also will be easier to manage.
This brings us to another need for change. We need to move from “Centralized” to “Distributed” systems. We need to give autonomy to each individual Product / Value Stream Team. This means the dependency among systems must be removed. This again helps in making the systems much secure. To achieve this, we need to refactor and re-architect our systems and needs planned time and resource. One week in a month for each practitioner should be spent on refactoring.
Skills Shortage
The next big challenge is Skills shortage. There is a huge gap of skills that is required in the existing practitioners. Key takeaways from the Global UpSkilling IT Report from DevOps Institute are:
- Insufficient IT resource skills are a huge challenge across the globe. Forty percent (40%) of survey respondents said that the resource and skill shortage is one of their top three challenges today. Additional research shows dramatic skill shortages within the technology and IT area globally.
- Addressing technical debt must be paired with addressing talent debt. While technical skills are a must-have skill priority, technology without human skills, will not accelerate innovation and transformation.
- Upskilling is a professional and organizational imperative. Continuous learning must be foundational for leaders and individuals and requires a mind-shift across leaders and individuals.
To tackle this problem of skills gap, we need to have to change from a “Building” mindset to “Learning” mindset. Organizations must foster this Learning mindset through various activities.
First, new skills must be acquired through Specialized External Trainings. Organizations must organize, sponsor, and provide time for such trainings from good, accredited institutes and good trainers.
The training and learning must continue through formal In-house training provided by experts who have gone through the Specialized External Trainings, to help practitioners be able to relate to their work and implement the learning.
The use of tools to learn “On-the-job” is also useful. For example, a tool like Deepfactor helps developers to learn about Security Vulnerabilities by unearthing them by running the tool. This not only helps in identifying vulnerabilities early in the day and improve the speed of delivery, it also makes the practitioners aware of the security vulnerabilities and what they need to do, thus, improving their Security knowledge.
Finally, creating “Guilds” for competency development and sharing of knowledge helps in increasing the productivity manyfold. This free form of non-deliverable knowledge sharing sessions are a good source of improve the knowledge of the practitioners, thus reducing the skill gap.
Practice Challenges
The next is the challenge of Frameworks and Practices. There are multiple Frameworks and Practices being implemented in the organization, each running in a different track. We can see Agile, SAFe, ITIL, DevOps, SRE, DevSecOps, SIAM, DataOps, AIOps/MLOps, and many more. The challenge here is that these are creating a silo of frameworks.
Organizations need to integrate and stitch together the learnings from each of the frameworks and practices to deliver the best value to the business and customers. We should not blindly follow each framework or practices independently but use them together by complementing each other for the common shared objective.
It is only when Security is baked-in the entire lifecycle, when everyone accepts that security is everyone’s responsibility and does the needful in their areas of work, we can improve the security posture of the organization. The Security SMEs needs to come as “Advisors”/ ”Consultants” to entire SDLC, providing the “Control Objectives” and let the team implement the “Controls” to meet those objectives. The SMEs should guide and handhold the teams in Security related aspects. It is not a question of Speed Vs Security; both has to go hand-in-hand.
Tools Challenges
The next big challenge is that of tools. There are Too many tools. There is “Automation Fatigue”. We need to use the right set of tools. We need to use more Opensource tools which can be easily plug-and-play through APIs. It will be a good idea to use a Toolchain and use Application Security Orchestration and Correlation tool to tie them all together.
We need to have automated tests to shorten feedback loops. Trunk based development with Security built in the entire CI/CD pipeline and having adequate Observability, even with respect to Security is crucial.
One more challenge with the Security Tools are Inadequate capabilities of tools. Product vendors needs to move up their game and deliver more suitable and better product. The false positives need to be further reduced in these tools.
Infrastructure Challenges
Last but not the least is the infrastructure challenge. First, many organizations are still holding on to the traditional legacy systems. It is not an IT decision to hold on or move to new technologies. The question is for business to decide whether they can compete with new startups and smaller nimble organizations with the existing infrastructure and technology.
It is important to understand that Complexity of Cloud is there before moving to Cloud. We need to Architect well before moving to the Cloud. It is just not a Lift and Shift. We also need to architect for Data Security. This is one area which still needs a lot of improvement. We need to understand the Regulatory and Compliance Challenges and factor those in our architecture and implementation. Here working closely with GRC SMEs will help in understand properly the needs and thus help in better compliance.
For details on various learning programs, please visit http://xellentro.com/devsecops-sre/