DevSecOps Guide to Bulletproof Your Development Workflow: part 1

Today, in an ever-changing software development world, security becomes more and more of a major concern and requirement. But it's not unusual to onboard a project where security remains one of the sacrificed topics. Why is this?

Is this due to the need to deliver faster and faster? Resource constraints? Difficulty to integrate in legacy code in a lot of time? Or maybe a lack in culture on the team?

Well, it's actually all of them, and the first step in order to change this is realising where exactly the problem relies. But let's break it down a little, shall we?

This is a BIG topic, so you're reading Part 1 in 2.

In Part 1, we will cover:

• The Security Silo
• DevSecOps to the rescue
• What we want to achieve
• The Shift Left approach
• Protecting your software (Analogy)
• Objective scenario
• Different kinds of scans
• Don't leave Observability behind

The Security Silo

To understand the genesis of DevSecOps, we have to understand its need. Going back a bit we realize what gave birth to DevOps: there was a need to cement the breach between everything Development and everything Operations, right?

We understood that taking a package or a bundle and send it to another team with a set of instructions to be executed was probably not the best approach in the long run. It was a huge momentum killer, between communication, artifact transfering and the numerous ways this could fail.

Maybe the instructions weren't clear... maybe some variables were left unconsidered or the context was missing. There were a plethora of ways this process could go wrong. Then, what happened when it went wrong? Well, time to repeat the process yet again, ain't it fun? Now let's have a quick round of applause for automation processes. Yikes...

With security, we're back to the same old problem. Security is siloed from everything else, and this can vary from insufficient security checks up to the extreme of having a completely different team, we probably have no contact with or confidence in, performing who knows what kind of tests on your code... or even have no security checks at all! I've seen it all, and I'm sure you have too.

But one thing seems to reveal itself to be the standard use case, which is security being a destination of the software development process when in truth, it should be a part of it.

DevSecOps coming to the rescue

Security is a requirement, it's fundamental, so it's got to be treated as such.

Enters DevSecOps!

Taking a look at the current software development landscape we can bookmark some universal concepts: quick-time to market, code quality and security. If you lack any of these, you are in big disadvantage over the companies who can guarantee all three.

So, what is DevSecOps?

DevSecOps is the inclusion of security in the development workflow, just like DevOps, it's more of a group of processes and tools that need to be employed in order to produce better software pieces, do it faster and of course, guaranteeing security right from the get go.

What do we want to achieve?

If you can guarantee code quality, fast delivery and security, your product wins. It's just like that. So, in order for us to get this brighter future for your Software Delivery Life Cycle, a couple of points have to be secured (haha get it?).

• Security testing cannot be sacrificed or siloed anymore;
• Security testing has to be streamlined and not be a destination of your software development workflow;
• Resources cannot be wasted by having unattached teams and not well defined concepts within your team;

The Shift Left approach

One of the biggest concepts you can think of when needing to streamline security in your software development workflow is the Shift Left concept.

Very simply put, Shifting Left means moving something to an earlier stage of a process. So, shifting security testing left means that we want to have this done as early as possible in our software development process.

With this we guarantee:

Faster delivery - Since we're not waiting for a team to send back a batch of results and then iterating over them (repeating the development over old goals, and losing time adding new features to the product) we're saving time.

By finding vulnerabilities early, you focus on a smaller chunk of code (rather than months of commits piled on each other) resolving these rather quickly.

Cost reduction - When you have vulnerabilities being found and consequently mitigated early in the software development process, you stop the rolling snow ball that will grow over time and that will hurt your companies' finances and reputation (harshly).

Security guaranteed early on - By automating the process and having scans over your code, you can guarantee that you have regular checks and that you can act as soon as possible over small batches of code, and demolish vulnerabilities very early in the process.

Of course when you have all of these in place you can imagine that your efficiency goes through the roof.

The "Protecting Your House" analogy

That was a lot of talk wasn't it? Let's get to the juice!

So, how can we start protecting our software? Or anything, really?

The answer might be obvious but it's quite simple actually. Let's see a specific example, starting with another question.

How would you protect your house?

You set up a wall or some fences around it so people and/or your nosy neighbours don't walk in.

You make sure the construction materials are of good quality, use doors and windows. That they have locks and are sturdy. Maybe you set up some security cameras around your property, right? Maybe you have some dogs roaming around, just in case...

Protecting your software is just the same thing, same logic: you find vulnerable spots that can be exploited and you start protecting them.

Here's an objective scenario

Let's look into an objective scenario so we can have something palpable to apply these concepts on.

Here's what we might work with, when we talk about software:

• Third-party libraries;
• Your own code (everything you copied from StackOverflow...we all know);
• A Dockerfile that is a blueprint for our containerised application;
• Our own infrastructure that can be version controlled and replicable;
• A group of Kubernetes resources that are API based;

Let's go one by one and see how we can start securing them.

Different kinds of scans

Third-party libraries

This is the code you have no direct control over, but there's a bunch of functionality that is being brought into your code and all of those imports that are bringing it are also injecting vulnerabilities into it. So you cannot forget about checking the full extent of what's being used.

For this we want to have Software Composition Analysis (SCA) in place.

SCA deals with the open-source code in your application. It is able to detect vulnerabilities in these components and ensure that licence compliance is respected.

Looking into the house analogy: you're decorating your living room and saw this perfect couch while strolling through a yard sale. It looks perfect, cozy and soft but you still need to check what it's all about, maybe it's full of termites...

Your own code

Now you want to look into your own code, the one you have full ownership over. I'm sure you don't want to be responsible for the next big breach of your company.

To scan your own code you can look into a couple of alternatives:

• Static Application Security Testing (SAST)
• Dynamic Application Security Testing (DAST)

For SAST scanning you have one of the best quick wins you can get when talking about scanning your own code. It's usually really easy to integrate which allows you to have it up and running in a matter of hours (maybe not even so!).

SAST looks into the code itself, and then holds it against a set of rules that are pre-defined for the tech stack you are using and if it finds one pattern that matches one of those rules, it will warn you about it.

Now you were brave enough to go on Youtube and build a chair with your own hands. You're really proud of yourself by the end of it (as you should) so you invite your parents for dinner, but maybe check the chair first to make sure it won't break as soon as someone sits on it.

For DAST, it's a bit different, this tool is used to also secure your code but it follows more of a Shift Right approach (which differs from the Shift Left approach) since its results come after you have something already deployed.

DAST does not know about your code, but it's purpose is to try to break your code like a hacker would, trying to simulate attacks and analysing what comes back.

At the end of the day, there's really not one over the other, you should use both of them in order to get a more robust code scanning. But their implementation timelines can be (and usually are) quite different. If you want something done to show your boss for tomorrows' daily then implementing SAST is the way to go. If you want to focus on reliability after you actually deploy something then give DAST a go.

Now, getting back to the chair... SAST knows common pitfalls of chair building so it knows just by looking at the chair that one of the legs is bound to break easily. On the other hand, DAST will sit on the chair in various ways and forms and try really hard to break it.

What a great example, right?

Container image scanning

We have checked our code and all the third-party tools we are using or, if you prefer, all the furniture you've built yourself and the one you bought.

Now, we have our app Dockerized and we took our code and put it inside a package, just like we put the furniture on the living room. It's time to check the living room itself, make sure it's not covered in mold as it will ruin everything.

Having that said, when you're using Docker to run your applications an enormous number of vulnerabilities comes from the base image itself. You always start with a base image and a lot of it is about checking what it brings.

Besides this, you have to check:

• Your software packaged inside the container;
• Additional configuration;
• Network settings;
• Among others...

Let's make sure all are checked.

Infrastructure-as-Code Scanning

It's time to check the structure of the house itself. We don't want a gust of wind knocking everything down after doing so much work.

Infrastructure-as-Code is a fundamental part, especially when you start scaling your infrastructure, and it becomes hard to manage manually.

With Infrastructure-as-Code we usually have a bunch of cloud resources with configuration that can be exploited. So things like:

• Overexposed CIDR blocks;
• Unnecessary open ports;
• Weak encryption algorithms, or missing entirely;
• Outdated services;
• Compliance violations;

Kubernetes Objects Scanning

Your containers are being orchestrated using Kubernetes. Kubernetes offers you a bunch of resources to manage your services and all of these can be twist and turned in some way.

These include:

• Deprecated API Versions;
• Role base access controls that are too permissive;
• Sensitive information exposed on ConfigMaps;

And last, but not least...

Don't leave Observability behind

We've seen a set of security scans you can employ in your software development process. It's time to set up some security cameras (aka observability).

This kinda falls out of scope and it's a shift right approach, but its so important in todays' landscape that I thought it should have a quick mention.

Bet on observability. When everything is in its place and up and running you need to know at all times what is happening with and within your system.

If there's an incident, you need to know about it in order to act upon it.

We learned the "what". What about the "how"?

On Part 1 of this article, you learned about what you need to protect your software, and also what you can use for that purpose.

On Part 2, we will dive into how you can do it, namely:

• Bulletproofing your development workflow
• Embracing DevOps and adapt
• Building culture
• Breaking the build

Stay tuned, because we'll be launching it very soon!

Meanwhile, take a look at this article on:
Reviving DevOps: Embracing a Product-Centric Approach (DevOps 2.0)