Automated Security Test Cases

A paradigm shift in security testing

Let me propose a radical new idea: Implement security test cases to test security controls in your application similar to how you test functional requirements. Yeah, I know there is nothing radical about it and this is not a new concept. But, lot of organizations have accepted Test Driven Development (TDD) practice and are extremely good at automating testing of functional requirements, but when it comes to security testing, it is still being offloaded as a point in time assessment to the security teams. Lets try to make automated security testing as common as implementing test cases for functional testing! Are you with me on this?

Benefits of implementing security test cases

  1. Build secure coding practice in the Development teams and makes them aware of security vulnerabilities and their remediation.
  2. Over a period of time, you build a strong suite of security test cases that get executed continuously in a pipeline providing continuous security assurance as opposed to point in time assessments.
  3. Security tools can’t detect logic and functional security vulnerabilities such as what roles are required to execute a capability, if there are authorizations based on dollar amounts, user registration flows etc.
  4. Most importantly this is “free” as in there is no licensing costs for expensive application security testing tools out on the market that at the most provide partial coverage and effectiveness compared to manual security assessments. I put “free” in quotes because you still need to expend time to implement security test cases.

What are common security test cases to implement?

Please see below for some examples of automated security test cases to implement across various categories.

Authentication test cases

  1. Verify that you are not able to login with invalid credentials
  2. Verify that you are not able to bypass multi factor authentication steps
  3. Verify that account is locked after 3 tries
  4. Is access to secure end points limited to authenticated users?
  5. Is the timeouts of authentication tokens working as expected?
  6. Is logout working as expected?

Authorization test cases

  1. Is access limited to users that have certain Role?
  2. Authorizations based on thresholds, verify is access is limited based on business logic rules
  3. Can the user assume a Role that they are not assigned to by sending a role on the request?
  4. Is the user authorized to access the object referenced in the URL? This is significant for REST end points where as best practice, IDs are specified in the URL. For e.g. if you have REST end point to retrieve account information, /rs/account/123456789, verify that the authenticated user is authorized to access account 123456789 as the end user can easily manipulate the account number to somebody else’s account number. This is called Direct Object Reference Vulnerability.

Input Validation and Output Encoding test cases

  1. Does the system reject invalid or malicious inputs?
  2. Is the system encoding user inputs before persisting to a sink? For e.g., sending in data containing SQL Injection pay load.

CSRF Protection test cases

  1. Verify that you are not able to perform state changing operations with out a CSRF token in the request

Miscellaneous test cases

  1. Verify is data is encrypted in transit with the use of https protocol
  2. Verify that TLS Certificate is valid
  3. Verify if “secure” flag and “http only” flag is set on cookies that contain sensitive data

How to implement security test cases?

Now that you are convinced to implement security test cases, how do you go about implementing them? There is no secret sauce to security test cases, Developers can use commonly available Open Source testing frameworks such as Selenium, Cucumber, JUnit, Mockito, Jersey Test Framework etc. for implementing security test cases. There are some open source projects such as Gauntlt that can give you a head start in implementing security test cases. In a future blog, I can provide some sample Cucumber test cases to demonstrate how easy it is to incorporate security testing in to your applications!


Error Handling: Prevent Information Disclosure

Are your developers handling unexpected server side errors? If server side error handling is weak, attackers can gain insights in to the technology stack being used in your applications and use that information to exploit known vulnerabilities in the technology stack as reported on National Vulnerability Database (NVD).

Typically un-handled server side errors bubble up as stack-traces, default error pages disclosing web server/app server names, version numbers etc. If an attacker were to know that your application is using Apache Structs 2, the recent OS Command Injection vulnerability disclosed in Apache Structs2 provides an easy attack vector for an attacker to compromise your application. Why would you let anyone know that you are using Apache Struts with lax error handling?

What are the common techniques used for error handling in applications?

Exceptions can happen in any tier of your application – UI/Mid-tier/Data-tier. Typically any modern language offers try/catch/finally to handle exceptions. Also, the web tier offers configuration to handle any uncaught exception and display generic error messages when un-handled exceptions are raised in code. Although configuring the web tier to handle un-handled exceptions provides you the safety net, educate your developers to handle exceptions in all tiers of the code. In addition to helping prevent information disclosure, having ┬áproper exception handling logic allows your application to gracefully fail and provide debug information in the logs to help you triage those stubborn issues that typically only happen sporadically in production environments.

OWASP also provides guidance on exception handling that is worth giving a quick read.

No body wants to see stack-traces reported on their web sites, which is a bad user experience, but on top of that you are providing valuable information to attackers. This is an easy one to fix with proper education to your Development teams !