The challenge is made more difficult by a fundamental fact about security: security is about negative requirements. That is, security requirements are often that a certain thing cannot be done, rather that it can be done. Such negative requirements are notoriously difficult to test for, and creating automated tests for these types of requirements is even more difficult. Yet, DevOps relies on the ability to create automated tests for everything.
Before I go any further, I should say a little about the lay of the land – the scope of what is encompassed by IT security. IT security is often thought of in terms of the levels of a “stack”, such as (starting from the bottom) network, OS, application platform, applications, and user (this is an over simplification). It is really, really important for applications to have a secure application platform, OS, and network: otherwise, the application cannot trust the services that it uses from those levels. In my discussion here, I am only going to talk about the application platform and up, because those are things that I know something about.
Application platforms are software systems that enable programmers to create customized applications. Examples of such systems include databases and application servers. It takes a-lot of time and effort to secure these platforms, and so it is advisable to create standard configurations that can be reused across all of your software projects. In the control-oriented parlance that is used by government agencies, you can have a fixed set of application platforms that meet the requirements of the applicable controls, and those controls can then be “inherited” by systems that use those platforms – assuming that the configurations are not changed. This approach is very “DevOps friendly” because no testing of the application platforms is required during development or deployment if the application platforms use the standard configuration that has already been verified as secure. The only issue then is when to upgrade the platform, when a new version is available: that’s not a simple issue but it is beyond the scope of the discussion here.
The undiscovered country – the application level
The next level of the stack is the application level, and that is where the fun begins. However, before worrying too much about application level security, you should ask yourself how much you are worried about a targeted attack. A targeted attack is one in which someone singles out your organization and undertakes to penetrate it, spending perhaps months to achieve their goal. Not every organization is a likely target for this level of attack. Organizations that are include those that handle large volumes of slightly sensitive information (for example, credit card numbers), or small volumes of highly sensitive information (for example, plans for nuclear weapons). If you are not in these categories, then your organization is probably safe from a targeted attack, and it is unlikely that someone will go to the trouble to discover the unique vulnerabilities in your custom application code.If your organization does store sensitive information, then you are a potential target at the application level, and you should be thinking about how to secure the processes used to code and deploy your custom software applications. From a DevOps perspective, the goal is to automate as much security testing as possible.
The most common approach to automated security testing of application code is to use code scanning tools, aka “static analysis”, or “static application security testing” (SAST). This is an old method, but it is continually refreshed to support the new application languages. Static analysis has its history in code quality analysis tools, going back to “lint” – a Unix tool that scans C language programs and finds potential bugs. Nowadays, one of the most widely used tools for code level security scanning is Fortify, but there are many others. And one caveat: static analysis does not work very well for dynamic languages such as Ruby and Python.
Static code scanning a crucial for security. It is not a silver bullet though. In fact, it has severe limitations: (1) it only finds a fraction of the vulnerabilities [http://samate.nist.gov/docs/SA_tool_effect_QoP.pdf]; and (2) static analysis generates a-lot of false positives – and by a-lot, I mean a-lot: I am talking about thousands of false positives for every actual problem found.
The false positive problem is not as bad as it sounds, because some tools allow a programmer to flag a false positive as having been checked, so that it is not generated when the tool is run again later, although doing that carries its own risks. The more severe problem is that static analysis only finds a fraction of the vulnerabilities. Consider the latest IT security debacle in the news, in which private photos of movie stars were released to the public. Many of these photos appear to have come from the personal Apple iCloud accounts of the actors, and there is evidence that those accounts were penetrated using brute force guessing of passwords (a “dictionary attack”) in the Find My iPhone web application. This web application was apparently vulnerable to this type of attack because it permitted the user to try passwords again and again, without limit. [http://theconversation.com/novice-mistake-may-have-been-the-cause-of-the-icloud-naked-celebrities-hack-31272] Static analysis would not have detected this type of error: it was due to an application logic choice made by the programmer.
There are other automated tools that can come to the rescue. For example, password cracking tools would have discovered the Find My iPhone flaw; but to know to use that tool requires familiarity with the range of security testing tools and what they are used for – there are so many tools, and one cannot blindly use them all. There are also “dynamic” tools that will try to penetrate a system while it is running, and those can be helpful. However, it is often difficult for these tools to know that something that seems ok is actually not ok. For example, how would a security scanning tool know that if an administrator logs in, that the administrator should not be able to read the data of users of the system? It cannot know, because such a constraint is a business rule, and tools cannot anticipate business rules.
The challenge here is quite large: any programmer can write a line of code that will subvert all of the security tools that are in place. Whether this is done intentionally or unintentionally (that is, whether it is an insider attack or merely an error), scanning tools cannot read the minds of programmers and product owners and deduce what is an intended action versus what is an inappropriate action. The only real solution here is to have programmers educate themselves about application security, so that they make fewer unintentional mistakes. There are voluminous online resources for learning about application security, such as https://www.owasp.org. (See in particular [https://www.owasp.org/index.php/Testing_Guide_Introduction]) The organization could also provide incentives for developers to obtain CSSLP certification. This is not a lightweight certification: it requires a-lot of work to obtain this, and it is highly worthwhile. Your team should also employ security analysis practices such as threat modeling: threat modeling is a collaborative session in which the team sits in a room and tries to come up with ways to penetrate the application, based solely on the application’s design and code. It is a thought experiment. Its value is that it gets everyone on the team thinking about security. Your architects or technical leads should also be thinking about secure design patterns, applying concepts such as compartmentalization, least privilege, separation of duties, and privileged contexts (for a discussion and design pattern, see my book High-Assurance Design, p. 219). They should also be choosing robust security frameworks so that application developers have a trustworthy toolset for implementing security functions such as authentication and authorization.
Before you undertake to secure all of your code, you should also think about where the risks really are. Generally, applications have only certain modules that need to perform sensitive operations. Identify those, and put your security testing attention and code review efforts on those. It is a waste of time to do threat modeling on parts of a system that do not do anything sensitive and do not connect to anything sensitive. A good starting point is to consider what the application’s “attack surface” is.
Another thing to consider is how secure your development environment is: what good is your application level security if a hacker can get into your development git repository – possibly in a public cloud protected only by a password that is itself not well protected – so that the hacker can inspect your code for vulnerabilities or possibly even insert some malicious code into it?
It is also important that security be treated as a first class application requirement. In an agile context, that means writing security stories that are put into the backlog. (Here is a good article: http://www.infoq.com/articles/managing-security-requirements-in-agile-projects) It also means adding security acceptance criteria to all stories that have security ramifications. Knowing enough to do that requires a focus on security, as a critical application concern, from the outset. It requires having an application security expert on the team, involved from day one. And it requires a product owner who understands the importance of security: for example, if there is a stakeholder team that communicates requirements and concerns to the product owner, there should be a security stakeholder on that team.
There is also the problem of external partners: other organizations that build pieces of a system that you then connect to and trust. Attacks often occur at the weakest link. In the attack on Target, in which millions of credit card numbers were stolen, the attack began with an attack on Fazio Mechanical, a heating, air conditioning and refrigeration company – a firm that Target had contracted. [http://krebsonsecurity.com/2014/02/email-attack-on-vendor-set-up-breach-at-target/] Target had no direct control over how secure Fazio’s systems and procedures were, yet Target gave Fazio access to Target’s systems. The lesson here is that if you have external partners, treat them as a potential weak link, and only give them the least privileged access that they need, and monitor their access with intrusion detection tools.
Deployment processes are application level code!
People often think of deployment processes as part of infrastructure and support, but DevOps changes that. An important aspect of DevOps is to implement all processes as code that can be put under configuration control and made automatically repeatable. That means that the people who create the deployment processes are writing code, and if this code is unique to the application, then it is best treated as application code, with all of the same quality considerations – including security. This means that all of the practices that you adopt with respect to writing secure code should apply to the deployment scripts and deployment configuration files.In a DevOps world, deployment does not stop at putting the code onto servers: deployment is about building the process for putting code onto servers, and that includes testing. That testing should include security testing. You do not want to burden your development build process with full blown security testing each time a developer checks code in. Instead, there should be a series of progressively thorough security testing the farther down you are in the continuous delivery pipeline. For example, the development build process should include basic static scanning, as well as other kinds of scanning by selected tools. Down the line there should be processes that are run periodically to perform script based penetration testing, password cracking, and “fuzzing”. Manual penetration testing should also be done on a periodic based – not merely at the end before release. Otherwise, there will be no time to fix problems within the scope of the agile timeline.
Designing all of these processes is a significant architectural effort. It is not possible to have a standard process that will be appropriate for all applications. The types of security testing that are appropriate depend on the nature of the application and on its design. Design of the security aspect of the testing pipeline is therefore a companion activity of the application design. That said, it is possible to reuse the security testing pipeline to a large degree if the basic application architecture is reused.
The user – the weakest link
In the Target security breach, one can blame the security procedures of Fazio, yet it turns out that Fazio was compromised through the use of an email malware attack. This implicates users of Fazio systems – users who might have naively clicked on malware attachments or phishing hyperlinks. In the end, users are the weakest link.The only way to mitigate against security mistakes by users is to educate them. Users who do not have sufficient knowledge of the ways that they might be tricked into compromising systems should not be given access to those systems: it is that simple. Just as driving or flying a plane requires expertise to do it safely, the same applies to using computer systems that manage sensitive information. If you want to see the range of ways that users can be tricked, check out my own taxonomy of social engineering methods in section 5.2 of this chapter.
In the final analysis, security is not a DevOps issue: it is a human issue.
Thanks for your valuable guidance. It is really good stuff.
ReplyDeleteDevOps Training in Chennai
DevOps Training Institute in Chennai