Interview: How Developers And AppSec Teams Can Build Secure Code By Design

In today’s digital-first world, time is of the essence. Businesses want to grow, and they want to grow fast. To empower that growth, developers are consistently being pushed to produce applications quickly—often to the detriment of security. While that method may boost short-term productivity, unsecure code can lead to devastating consequences further down the line, such as data breaches and compliance fines—both of which negatively impact productivity long-term. It’s clear that developers and application security teams need to build their code so that it’s secure by design, but the question remains: how?

During BlackHat 2022, we spoke with Renny Shen, Director of Product at Checkmarx, and Bryant Schuck, Senior Product Manager at Checkmarx, to find out how developers and application security teams can build secure code without compromising on productivity. Checkmarx is a leading application security testing provider that enables businesses to produce secure code, while unifying their AppSec processes within a single platform.

In this interview, we discussed the main challenges that developers and AppSec teams are facing when it comes to application development, why development teams should “shift left” and bake security into their development lifecycle rather than adding it on at the end, the importance of security training for developers, and how businesses can take their first steps towards application security maturity.

Could you give us an introduction to Checkmarx, your typical customers, and your main use cases?

Shen: Checkmarx is a leader in the application security testing market. Essentially, what we do is help our customers build their modern applications more securely. Given that life is online and businesses are online, we help them build their applications in a way that reduces their business risk.

Our customers range from smaller businesses all the way up to the largest enterprises. Basically, anybody doing business online, and anybody building applications online, because they have a vested interest to do that securely.

What are the main challenges that organizations are facing today when it comes to application development?

Shen: So, applications are getting very complex. There are increasingly more technologies, more building pieces, and so on that our customers are putting into their applications. And every time you add some new technology, and every time you do things a little bit differently, that increases your risk. Especially if it’s introducing a new attack vector.

On the other side, businesses need to move faster. This is all about them making money. So, everybody is being pushed to move faster, including developers, and this means that sometimes we’re not taking a step back and thinking about the implications of doing things a certain way, or what the security risks are. It’s just a cost of doing business.

But we want to help our customers mitigate those costs.

I’d like to touch on that a bit more. As you say, productivity has traditionally been the priority for developers, with security being built in afterwards, rather than throughout development. Why is it important for development teams to bake security into the development cycle?

Shen: There have been any number of studies around this concept of “shift left”. The idea is that you can find a vulnerability and you can fix it at any point, but it’s much more expensive to find it and fix it after an application has been pushed into production; after it goes live.

So, think about what goes into that. If you have an application, or an API, or part of an application that’s gone live, that might have already been under attack for who knows how long. So, there’s loss there: potential data breaches, things like that—there’s a cost there.

But if you think about what it takes to actually plug that gap or that vulnerability once you discover it in a running application, you have to go backwards. Security teams don’t always know who exactly was responsible for building that part of the application, because we have a lot of app developer teams contributing to an application nowadays. So, the security team has to figure out what part of the application is it, what developer teams are responsible for it, which developers have the subject matter expertise or the knowledge of that particular part, and then go back to them, find them, and then raise the vulnerability with them.

By this time, the developers themselves have already moved on; they’re already doing something else. So, they have to switch context back into this thing that they may have launched weeks, months, or even years ago, and figure out how it worked and how they’re going to fix the vulnerability. And then they have to go through the process of testing it and releasing it again.

And even the process of releasing it again might be simple, or there might be a hiccup, or there might be something that’s disruptive to end users. So again, there’s another cost there.

It’s 100 times more expensive or more costly to fix an issue in production than it was in the design phase. It’s 15 times more expensive to fix a problem in production than it was in the coding phase. And it’s 6.5 times more expensive to fix it in production than in the testing phase. So basically, as you move further along the software development lifecycle, the more expensive it gets to find and fix a problem. So, the earlier or further left you can do that in the software development lifecycle, the better.

Schuck: Just to add on to that, I always look at it as humanizing the process, too. If developers catch these earlier in their lifecycle, they learn how to resolve them so, when they go to write their next lines of code, they’re writing security baked in. If it comes at production time, it’s going to be a fire—there’s extreme pressure to go fix and that, so they’re not trying to learn about it; they’re trying to find that senior or that architect who knows how to fix it, they’ll give them the task, it will go in, and no one else gets to learn about it.

So, one of the big advantages when we’re working with developers is that we have a product called Codebashing, which allows developers to learn about security. That way, they can write secure code. You get that benefit, and you’re also actually training your developers about security.

How do you deliver that training? Does it go hand-in-hand with your testing services so that developers learn on the go, or do you offer it as more of a modular training course?

Schuck: It is modular, but we productize it. So, because we know this specific line of code, we’re going to tell you how to resolve it. But you can also go deeper—that’s really our thing, asking how deep you want to go. Do you just want to resolve the bug and fix the line of code? Do you want to learn about it, whatever it may be? You can always go deeper in our products. But the training is also available as separate modules, so, if a customer does want to do training courses or put a curriculum together for their development teams, they can do that also.

Shen: People learn differently. Like Bryant said, you can go off and you can build customized learning courses for your developers around security. Some people learn great that way; other people don’t. Some people just don’t have time to sit through courses. I hate courses, so, for me, that’s the beauty of it: this whole in-context, just-in-time enablement. When we highlight a vulnerability in your code or a bug in your code, there’s immediately available, contextualized learning about what that vulnerability was. So, just-in-time, you get to learn a little snackable piece of content that helps you learn about the thing that you’re actually fixing, without overwhelming you with hours and hours of training. I think that’s really powerful.

Schuck: And as our API research team learns about these new attack surfaces or what’s happening in the industry through their research, we can produce courses to get that learning out there as soon as possible. So, we’ve had API security courses in Codebashing for quite some time to allow our customers to start realizing this is coming, even before we built the Checkmarx API Security product inside our platform.

How does that training come together with the other components of the Checkmarx platform—including your newly launched API Security tool—to help organizations and developers to ensure they’re building applications in a secure way, without disrupting productivity?

Shen: There are a couple of things there. So, in terms of who uses our products, we think about both developers and application security teams. For application security teams, it’s the idea of having a unified platform so they don’t have to go off and manage one tool for every different part of their application; they can have a single platform, a single GUI, a single set of findings, one click kicking off multiple scans, and then the correlation. That’s what part of the power of our platform is: the ability to correlate findings across these different scans, in order to increase accuracy or actionability.

For developers, the key is to make it seamless, so that they don’t have to do something new. Developers don’t necessarily think in the form of vulnerabilities, so, when we find one during check in, we automatically log a JIRA ticket or a bug ticket, for example. That’s the context that developers think in: we’ve got a bug, it’s got this priority, and now we’ve got to go off and fix it. And we can just automatically log that for that developer while they’re working on that part of the application.

And then we also provide guided remediation. We integrate with their IDE, so, whatever tooling they’d like to use to code that application, we can guide them to the right spot to fix it in their IDE. We can provide them with information and—through Codebashing—learning in their IDE about security vulnerabilities.

So, it’s really about adapting security to the developers’ process and the developers’ mindset on how they want to fix things and making it easier for them to do it. If you think about it from the security perspective, that’s their goal as well: making it easier for all of the developers to build code more securely.

Schuck: And it’s also about adapting to technologies. API security is becoming one of the biggest attack vectors. And the Checkmarx API Security module that we’ve just released allows us to go out and build a new product or a new engine, and to go much deeper into that realm. We’ve always had API security through scanning source code, but now, recognizing the customer pain points and understanding what they need from the platform, we can just add that with a simple click of a button. And that’s really nice.

We know technology is going to continue to change. The security concerns of today aren’t going to be the same in five years. So, this allows us to continue to add products and security expertise, without requiring the customer to change their processes or people every time a new technology or a new attack surface comes out.

Shen: To that point, in terms of both technology and learning, it’s continuous and it’s a spectrum. By no means are we saying developers are bad at writing secure code. Some are really great. Some are new coders. So, you’ve got developers that are thinking about security best practices, and others for whom that’s completely foreign. But even for the ones who think about security best practices, new technology comes out that presents different risks. APIs are a good example of that, where there are different risks that they may not have thought about before when building their applications. And so, providing that just-in-time learning and providing that ability to find and fix those vulnerabilities in that new technology is a continuous evolution.

In the industry, we often hear that application security maturity is a journey. What does that journey look like, and how can organizations take their first steps on that path?

Schuck: The first thing you’ve got to start off with is visibility. You’ve got to know what you’re working with. That in itself scares a lot of enterprises and software engineers, because if they don’t know about it, they can kind of walk away at five o’clock and think everything’s okay. So, the first thing you’ve got to do is figure out where you are in that journey. And we always encourage you to start simple; maybe start with your high-risk projects or things that you know you need to protect, and just see where you are as a company. And then prioritize and fix that.

One thing that we did with the Checkmarx API Security product is, we wanted to get both of those views. We know developers are very important, we need to give them the dashboard of how many APIs, how many sensitive data APIs, and how many undocumented APIs there are, and then allow them to fix that day-to-day, just so they’re getting better over time. But they also face time constraints when trying to get new features out the door, so we also need to give leadership views of their whole list. So, we built an API inventory that allows them to look by project or application and see how many APIs there are and if they’re getting better, or if there’s a high-risk API that no one’s actually taking account for.

We also launched a product called Checkmarx Fusion that allows you to tie a lot of these static scanning tools together to give you that complete picture and try to identify where your highest risk spot is. Because—especially if you’re scanning a legacy project for the first time—there’s a lot out there that you need to fix.

With that visibility, you can drive bottom up and top down to just get better over time. And, from that, we see a lot of customers prioritizing and trying to figure out what to fix first or where the weakest spot is.

Shen: If we go back to an API-specific lens, I think a lot of a lot of organizations or developers think of themselves as either “API-first”, which is kind of the goal, or “code-first”, which is more traditional. From a security team perspective, most of the time, visibility depends on having proper API documentation created by the developers. If you’re a code-first organization, which is where most organizations start, the documentation is unreliable or not there at all; you focus on creating the code before you go through design. Obviously, that’s not a best practice, but that’s just the way it is for a lot of companies today. And so, security has to catch up.

So, that’s what Bryant is talking about: discovering and inventorying APIs in source code without requiring the customer to have a robust documentation process in place. So, we can start with companies with very immature API practices.

But then, even once you get to API-first organizations, one thing we forget is that a development organization has a lot of people in multiple teams, and those teams may be doing things differently. They may have a formal documentation process, that process may be top priority, but it may be a lower priority, and they may be sticking their documents in different places—things like that. That’s just the way people and organizations are. But we can come in after the fact and help them catch some of these things. We can say, “You may have this documentation, but you didn’t give it to us. It’s not in your central repo, but we found an API in your inventory and your source code,” and then we work with the customer in terms of shoring up their documentation in that area.

So, regardless of where a customer is in that maturity curve—whether code-first or API-first—I think we can help them improve their secure coding practices.

What would your final piece of advice be to developers and AppSec teams that are looking to build security into their application development processes?

Shen: I would say something different to the different audiences. For developers, I would say it doesn’t have to be hard, and it doesn’t have to be scary. Work with your security teams to look for a solution that makes it easier for you to find and fix problems, and to contextualize security in the way that you’re used to thinking about developing applications.

For security teams, it’s the converse of that. Security teams have to think about the pre-production development environment and the production environment. They’re going to have to fix things afterwards, but it’s less costly and less risky to fix things earlier. So, think about how to make security easier for your developers to consume and take action early on.

Schuck: For me, I’d say have a voice! My goal with our products is to give the developers and the AppSec teams a voice at the table. So, you can come to the meeting with data, and you can say, “This is why we shouldn’t release or why we need to fix XYZ,” or “I need an extra day or two to fix the security issue, because if this API gets attacked, this is the data that could be breached.”

Empowering our development teams is going to help that. Giving them discrete views that they can use to make it tangible—that’s going to help us align those priorities.

It’s not just about getting the feature out the door or releasing the product. It’s about making sure the company is safe and keeping that top of mind.

Thank you to Renny Shen and Bryant Schuck for taking part in this interview. You can find out more about Checkmarx’s application security testing platform via their website.

Expert Insights provides leading research, reviews, and interviews to help organizations make the right IT purchasing decisions with confidence.