Automated Security Test Cases

A paradigm shift in security testing

Let me propose a radical new idea: Implement security test cases to test security controls in your application similar to how you test functional requirements. Yeah, I know there is nothing radical about it and this is not a new concept. But, lot of organizations have accepted Test Driven Development (TDD) practice and are extremely good at automating testing of functional requirements, but when it comes to security testing, it is still being offloaded as a point in time assessment to the security teams. Lets try to make automated security testing as common as implementing test cases for functional testing! Are you with me on this?

Benefits of implementing security test cases

  1. Build secure coding practice in the Development teams and makes them aware of security vulnerabilities and their remediation.
  2. Over a period of time, you build a strong suite of security test cases that get executed continuously in a pipeline providing continuous security assurance as opposed to point in time assessments.
  3. Security tools can’t detect logic and functional security vulnerabilities such as what roles are required to execute a capability, if there are authorizations based on dollar amounts, user registration flows etc.
  4. Most importantly this is “free” as in there is no licensing costs for expensive application security testing tools out on the market that at the most provide partial coverage and effectiveness compared to manual security assessments. I put “free” in quotes because you still need to expend time to implement security test cases.

What are common security test cases to implement?

Please see below for some examples of automated security test cases to implement across various categories.

Authentication test cases

  1. Verify that you are not able to login with invalid credentials
  2. Verify that you are not able to bypass multi factor authentication steps
  3. Verify that account is locked after 3 tries
  4. Is access to secure end points limited to authenticated users?
  5. Is the timeouts of authentication tokens working as expected?
  6. Is logout working as expected?

Authorization test cases

  1. Is access limited to users that have certain Role?
  2. Authorizations based on thresholds, verify is access is limited based on business logic rules
  3. Can the user assume a Role that they are not assigned to by sending a role on the request?
  4. Is the user authorized to access the object referenced in the URL? This is significant for REST end points where as best practice, IDs are specified in the URL. For e.g. if you have REST end point to retrieve account information, /rs/account/123456789, verify that the authenticated user is authorized to access account 123456789 as the end user can easily manipulate the account number to somebody else’s account number. This is called Direct Object Reference Vulnerability.

Input Validation and Output Encoding test cases

  1. Does the system reject invalid or malicious inputs?
  2. Is the system encoding user inputs before persisting to a sink? For e.g., sending in data containing SQL Injection pay load.

CSRF Protection test cases

  1. Verify that you are not able to perform state changing operations with out a CSRF token in the request

Miscellaneous test cases

  1. Verify is data is encrypted in transit with the use of https protocol
  2. Verify that TLS Certificate is valid
  3. Verify if “secure” flag and “http only” flag is set on cookies that contain sensitive data

How to implement security test cases?

Now that you are convinced to implement security test cases, how do you go about implementing them? There is no secret sauce to security test cases, Developers can use commonly available Open Source testing frameworks such as Selenium, Cucumber, JUnit, Mockito, Jersey Test Framework etc. for implementing security test cases. There are some open source projects such as Gauntlt that can give you a head start in implementing security test cases. In a future blog, I can provide some sample Cucumber test cases to demonstrate how easy it is to incorporate security testing in to your applications!

Advertisements

SAST: Static Code Analysis to identify Security Vulnerabilities

What is SAST?

SAST stands for Static Application Security Testing. Before ShiftLeft became fashionable, security companies were thinking wouldn’t it be better to identify security vulnerabilities while the Developer is writing the code as opposed to later in the process. What a futuristic thought?

Static Application Security Testing (SAST) tools analyze source code or compiled code to identify security vulnerabilities. The intent of SAST tools is to Shift Security Testing to the left and allow Developers to scan code, identify and fix vulnerabilities during the software development process. SAST solutions have been available in the industry for 10+ years. Fortify, AppScan, Checkmarx, Veracode are some of the leading commercial SAST providers. There are mature Open Source tools and frameworks such as SonarQube, FindSecurityBugs, PMD, that have plugins/rules to scan for security vulnerabilities and the tools allow you to implement custom rules applicable for the software languages and frameworks used in your organization.

How do SAST tools analyze the code?

SAST tools are very effective at identifying the use of insecure APIs.  For e.g.,

  1. Use of Dynamic SQL that could lead to SQL Injection vulnerabilities
  2. Insecure use of Encryption & Hashing APIs such as the use of weak encryption and hashing algorithms
  3. Information disclosure with logs and stack traces emitted to stdout, stderr
  4. Use of deprecated APIs that are vulnerable

The commercial SAST tools provide deeper analysis by building Data Flow Analysis wherein the tools build a model of untrusted inputs from Source (say HTML Form) to a Sink (say a Database). SAST tools than analyze the Data Flow models to identify if the untrusted input is properly sanitized with input validations and output encoding to defend against Injection and XSS type of attacks. The tools allow you to customize the sanitizers to account for your custom libraries to whittle down false positives.

The SAST tools either consume raw source code or Binary code for analyzing the code.

Open Source Tools are very good in identifying vulnerabilities in the first category but fall woefully short in doing complex Data Flow Analysis that span multiple modules.

Coverage and Effectiveness of SAST Tools

The advantage of using SAST tools is that they offer coverage across many categories and industry standards such as OWASP Top-10, SANS 25, PCI DSS, HIPAA, CWEs etc across many languages and frameworks. Some organizations use SAST as a mechanism to be compliant with regulatory requirements such as PCI DSS.

Most of the tools have coverage for Java, JavaScript, .NET, Mobile (iOS & Android), Python, Ruby etc. and the vendors keep adding support to new and upcoming languages such as Go.

You can think of SAST as offering coverage that is mile wide but only an inch deep! The tools are very noisy and produce false positives. Don’t be under the impression that you can drop these tools in your environment and run them on autopilot! This will only drive Developers away from adopting these tools and lose goodwill with them! You need to spend time and resources training the tools by creating customizations that apply for the languages, frameworks that are used in your organization! For e.g., if you have custom input validators, you need to train the tool to recognize these validators or else the tool would report false positives in injection vulnerabilities.

SAST Tools Evaluation Criteria

Given the number of commercial tool vendors, what are some of the considerations for evaluating and selecting a SAST tool? I would rate the tools based on the following criteria:

  1. Does the tool cover the languages and frameworks used in your organization?
  2. Does the tool cover industry vulnerabiltiy categories such as OWASP Top 10, SANS 25, PCI DSS etc?
  3. Is the tool customizable to limit false positives being reported?
  4. Does the tool provide mitigation guidance and is the guidance customizable?
  5. Does the tool integrate with Build Pipeline to automate static analysis during Build time and fail the build if based on preset policies?
  6. Does the scan performant? Especially, if the tool is integrated into the pipeline, the scans have to be performant. Does the tool allow incremental scans to speed up the scans?
  7. Does the tool have an IDE plugin that integrates with the Developer IDE and allows the Developer to scan code during Development and consume the scan results from your Build Pipeline scans?
  8. Is there a usable dashboard and reporting capability that allows your Security team to identify vulnerable applications and work with the application teams to mitigate the vulnerabilities?
  9. Does the vendor offer on-demand training or coaching is the Developer needs further assistance?
  10. Does the tool integrate with Bug tracking systems such as Jira and productivity tools such as Slack or ChatOps?
  11. Does the tool integrate with your Authentication and Authorization systems?

The vendors demonstrate their tools with readily available vulnerable applications such as WebGoat. OWASP Benchmark project also provided a suite of test cases and code to help evaluate these tools. The project also ranks open source SAST tools and commercial SAST tools although they do not call out the commercial tools due to licensing constraints.

In my opinion, the tools are highly tuned to work for test apps such as WebGoat and the best way to evaluate a tool is to throw a variety of applications that you have in your organization against the tool and see what you get!

In Conclusion

Given the coverage and effectiveness shortfall associated with SAST tools, is there a place for SAST tools in your Development process? I believe there is value in having SAST tools in your arsenal. But, instead of looking at SAST tools as the be all and end all, I would consider using SAST tools as guardrails that can help Developers avoid common security anti-patterns and adhere to secure coding practices!

If your organization does not currently use SAST tools, there is no reason for not adopting Open Source Tools that are free and make them available for your Developers today!

 

 

Authorization Checks in REST APIs

Lets take a look at sample REST APIs that allows Students to view their grades and professors to add/update grades.

If a student wants to view his/her grade in English, your system perhaps has a REST API like:

GET /student/{student_id}/english/grade

The Professor perhaps has access to the following REST APIs to be able to view and update any one of their students’ grades:

  1. View a student’s grade

GET /student/{student_id}/english/grade

2. Post a student’s grade

POST /student/{student_id}/english/grade

3. Update a student’s grade

PUT /student/{student_id}/english/grade

Of course there is a portal that the Student logs in to to view their grades and the professor logs in to post and update grades.

A savvy student is able to turn on HttpWatch to view the REST APIs being called by the portal. Lets say, the student A observes that when she clicks on a link to view her English grade, a REST API call is generated that looks like the following:

GET /student/1234/english/grade

What if the student is smart enough to replay this request with a different student id, 6789?

GET /student/6789/english/grade

If there are no proper Authorization checks in the APIs, the student A with id, “1234” is now able to view student B with id “6789” grade.

Even worse if if the student was able to update their own grade. How about the student try the following API?

PUT /student/1234/english/grade with payload “A”

or update someone else’s grade to “F”

PUT /student/6789/english/grade with payload “F”

However, a professor should be able to view, post or update anyone of their students’ grades. All of the above APIs are fair if they are issued on behalf of a professor.

So, every REST API call needs to enforce proper authorization to make sure that the user on whose behalf the API call is being made has access to the Resource on which they are taking action.

Since Java 6, the following Authorization annotations were added to make it easy to decorate methods with Authorization checks:

Annotation Types Summary
DeclareRoles Used by application to declare roles.
DenyAll Specifies that no security roles are allowed to invoke the specified method(s) – i.e that the methods are to be excluded from execution in the J2EE container.
PermitAll Specifies that all security roles are allowed to invoke the specified method(s) i.e that the specified method(s) are “unchecked”.
RolesAllowed Specifies the list of roles permitted to access method(s) in an application.
RunAs Defines the identity of the application during execution in a J2EE container.

However, these are annotations do not allow you to check if the user has access to a specific Resource ID. This is where, I find using Spring Security to be very helpful.

Spring Security provides a mechanism to do Data Level Authorizations using “hasPermission” annotation. Below is a reference from the documentation:

hasPermission(Object targetId, String targetType, Object permission) Returns true if the user has access to the provided target for the given permission. For example, hasPermission(1, ‘com.example.domain.Message’, ‘read’)

This makes it a bit easy to centralize logic for authorization checks. In the case of a REST Call such as POST /student/1234/english/grade “A” would translate in to a hasPermissions check:

hasPermission(“1234”, Grade, “update”) The method has access to the Principal on whose behalf the call is being. So, you have access to the Principal and the Target, and you can verify if the Principal has access to act up on the target ID. In the case of a student, this check returns a true only if the Principal and the Target ID are the same and also the permission is “view”. In the case of a professor, the check may be something like does the Target ID belong to a student that is currently enrolled in the Professor’s class.

If you are using Spring Security or some other framework, make sure you provide an easy way to apply authorization checks to ensure that the Principal is able to act on a given Resource.

Break Builds: Getting Devs to take action !

As your organization matures DevSecOps, enabling Developers to release features faster, as a security professional you are trying to assure that the code is free of security defects. As part of you Security team, you incorporated automated security tools in to the build pipeline such as Static Code Analysis Tools, Dynamic Code Analysis, Security Test Cases etc. But, you can only bring the horse to the water, but can you make it drink the water? In other words, how do you get your development teams to consume results reported by the security tools? That is the million dollar question.

There is an aspirational goal that no security defects should ever leak in to production. There is an ask from clients to prevent code with security defects from getting deployed to production. Why don’t you break the build and force the developers to take action and fix the defects reported by the security tools before any code is deployed to production?

The challenge is multi-fold.

  1. The error ratio, ration of false positives to false negatives is significantly high for security tools. This is especially true for Application Security, you have better results with Infrastructure Security. Some of the tools reports results that run in to literally 1000s of pages long that a Developer needs to triage to identify true positives that they can take action and fix. The tools lack the Business Context to alert if an issue is High or Low for a given attack vector. This is especially challenging if you are focused on Application Security. Would lack of Input validation indicate or out out escaping result in an exploitable vulnerability depends up on whether the data can be exploited by the attacker to compromise your system.
  2. The criterion you use to break a build gives you a false sense of security. Given the limitations of tools, there are whole bunch of false negatives that leak in to production giving the organization a false sense of security. The Developers can always point back and say the Build went through, hence there must be no security defects in the code.
  3. Given the negative testing involved with uncovering security defects, the current generation of tools are not capable of identifying all possible attack vectors and assuring defenses against the attacks. I would think AI plays a role in being able to identify security weaknesses in code that are exploitable.

One of the tools that is used to measure code quality, SonarQube has removed the capability that supports breaking builds based on criterion such as severity level of the issues, number of issues, etc. The team recommends that the better approach is for teams to look at the dashboard of issues, triage and identify real issues that are applicable to your code to be fixed. This did not go well with security minded folks, but in reality that is a pragmatic approach until the security tools mature and are able to report security issues that are exploitable as opposed simply identifying security code syntax that may or may not lead to a vulnerability.

Until then, there is no easy button for assuring application security in the DevOps pipeline. Security organizations would have to continuously tune security tools, follow-up with Developers to look at the results, add exploitable defects to the Backlog for the Devs to work on to have success at assuring application security in DevOps Pipeline.

I would like to hear from you. Please share what you have done to assure security in DevOps Build Pipelines. What kind of automated tools do you run? How do you make Developers take action on them? Were you successful in implementing automated enforcement gates that prevents Developers from deploying code with vulnerabilities identified by security tools in the DevOps Pipeline?

 

 

 

Newest OWASP Top 10 Release Candidate List is Out

OWASP Top 10 is updated every 3 years and the latest 2016 Release Candidate list of OWASP Top 10 Vulnerability Categories list is out. There two new categories of vulnerabilities that were added and that is the focus of this Blog Post.

OWASP Top 10 Project team added the following two new categories to the list:

  1. A7 – Insufficient Attack Protection
  2. A10 – Underprotected APIs

Lets talk about these two.

A7 – Insufficient Attack Protection

This category suggests that most applications have insufficient Preventive, Detective and Corrective Controls built in to the applications. Does your AppSec team provide common libraries, frameworks and services for Authentication, Authorization, Input Validations etc.? Do these frameworks log Security events if someone tries to circumvent these controls? Do you have actively monitor for these errors and respond if you receive alerts on these events? Are your applications self healing if they detect an attack in progress? These are some of the questions that this new category prods us to think.

As in most organizations, they rely upon Developers to implement sufficient Logging to be able to Detect malicious activity. But more often than not, the organization that does the monitoring is separate from Development Organizations and most likely are not aware that they need to monitor and respond to these Application Security Events. Most of the times, the monitoring and alerting is only at the Network and Infrastructure layers and not at the App layer. Hopefully, having this new category allows organizations to fortify SOC monitoring to include Application Security Alerts for monitoring.

There are a few critiques of this new category, especially, as this category proposes including a RASP (Realtime Application Security Protection) in your application stack that can detect and protect in real time as the exploit is in progress. The reason being the this category proposal came from Jeff Williams, the co-founder of Contrast Security, that offers RASP capability. Is there a conflict of interest? May be. But, does that stop me from exploring this new category and looking at ways to get better at application security? Definitely not.

I am not endorsing the criticism, but here is one such post to learn about the controversy on A7 – Insufficient Attack Protection. But, who does not like some drama in their chosen field?

http://www.skeletonscribe.net/2017/04/abusing-owasp.html

A10 – Underprotected APIs

If you do not have APIs in your organization, you must be hiding under a Rock or accidentally hitched on to the Mars Rover looking for Alien life. APIs are every where. Every company worth its salt is embracing micro-services architecture. So what is the problem?

All the OWASP Top 10 vulnerabilities typically associated with Web Applications apply to APIs as well. All APIs need to be Authenticated, Authorized, etc. etc. But, one area that we see vulnerabilities is Insecure Direct Object References, where Resource Ids are exposed in the URIs and an attacker can change the Resource Id to see if they are able to get to Resources that they are not authorized for. For e.g., if you have an API to retrieve your bank account information and the API is, /rs/account/{id}, an attacker will enumerate this API with different Account Ids to see if they accidentally get access to someone else’s account information. This is Insecure Direct Object Reference. Does the Authenticated user has access to the Resource being requested, in this Account information referred to by the Account Id, is Business Logic and it is very hard to verify outside of the context of your application.

How do mitigate this vulnerability?

API Developers needs to implement what we can call Data Level Authorization checks in the application code. Does the authenticated user has access to Account Id “xyz” check needs to be implemented before any Account information is sent back to the user. This check may vary based on the Resources and APIs that you deal with and needs to be built with in the context of the application.

Please share your thoughts on what you are seeing as your organization adopt APIs and Micro-Services architecture.

Error Handling: Prevent Information Disclosure

Are your developers handling unexpected server side errors? If server side error handling is weak, attackers can gain insights in to the technology stack being used in your applications and use that information to exploit known vulnerabilities in the technology stack as reported on National Vulnerability Database (NVD).

Typically un-handled server side errors bubble up as stack-traces, default error pages disclosing web server/app server names, version numbers etc. If an attacker were to know that your application is using Apache Structs 2, the recent OS Command Injection vulnerability disclosed in Apache Structs2 provides an easy attack vector for an attacker to compromise your application. Why would you let anyone know that you are using Apache Struts with lax error handling?

What are the common techniques used for error handling in applications?

Exceptions can happen in any tier of your application – UI/Mid-tier/Data-tier. Typically any modern language offers try/catch/finally to handle exceptions. Also, the web tier offers configuration to handle any uncaught exception and display generic error messages when un-handled exceptions are raised in code. Although configuring the web tier to handle un-handled exceptions provides you the safety net, educate your developers to handle exceptions in all tiers of the code. In addition to helping prevent information disclosure, having  proper exception handling logic allows your application to gracefully fail and provide debug information in the logs to help you triage those stubborn issues that typically only happen sporadically in production environments.

OWASP also provides guidance on exception handling that is worth giving a quick read.

No body wants to see stack-traces reported on their web sites, which is a bad user experience, but on top of that you are providing valuable information to attackers. This is an easy one to fix with proper education to your Development teams !

 

OWASP (Open Web Application Security Project)

I recently came across this Blog on AppSec related topics at https://appsecfordevs.com. I found the articles to be very pertinent and helpful to me. Please take a look at this Blog is you are in the Application Security domain.

AppSec For Devs

If you are new to Application Security domain, OWASP has a treasure trove of information on getting you started on path to a great career in application security as a Security Architect, Penetration Tester, Security Assurance Engineer etc. The OWASP Website is located @ https://www.owasp.org/index.php/Main_Page

Some of the main things that I learnt from the OWASP Website are:

  1. Top 10 Web Application Vulnerabilities that are published every 3 years. This list allows you to prioritize your dollars on what defenses you need to build for securing your web application assets. They have recently released the Release Candidate for 2016 OWASP Top 10. The additions to the 2016 list are: a) Automated detection and remediation of vulnerabilities. b) Lack of sufficient security controls in APIs (for e.g. REST APIs). I agree that most organizations that are re-architecting their sites using REST/CSA technologies are missing basic security controls such as authorizations, output escaping…

View original post 229 more words