Words and works

A short while ago I've mentioned this blog to someone who read through posts and then came back, saying: "Nice ideas, but did you actually implement any of this?"

Here's what we've managed to implement at work, all or most of the ideas in these topics:

Code review tools and techniques

http://www.surrendercontrol.com/2013/05/crutches-and-static-code-analysis.html
http://www.surrendercontrol.com/2012/12/focused-code-reviews-followup.html

Application security for big web apps

http://www.surrendercontrol.com/2012/11/modern-web-application-security.html

Changing security culture

http://www.surrendercontrol.com/2012/12/changing-things-when-change-is-hard.html

Crutches and static code analysis

First this was going to be a blog, then a DD post, then a blog again...

A while ago I've read an article absolutely not about security but about how great it is to work in small friendly teams - http://pragprog.com/magazines/2012-12/agile-in-the-small

It contains an awesome quote:

"...most best practices are just crutches for having a heterogeneous skill mix in one’s team."

Please hold that quote in mind while I turn to the figures recently released by WhiteHat Security

(http://www.slideshare.net/jeremiahgrossman/whitehat-security-website-statistics-full-report-2013).

They say that 39% of their clients use some sort of source code analysis on their webapps. These customers experience (probably meaning 'discover') more vulnerabilities, resolve them *slower* and

have a *worse* remediation rate.

Why is this? If you have ever been a customer to a SCA salesman then you know. Their pitch goes like this:

"All you need to do is to run our magic tool with this 'best practice' configuration and fix all results. The tool does not require a person who understands app security to be involved. It's like a tester in a box. Even better, just use "OWASP top 20" (I call it "fake silver bullet") configuration, this is what everyone else does."

Typical outcomes: the tool finds a large amount of rather unimportant noise, rates the issues overly high just in case. Developers get tired fixing these often nonsensical results. You'd be amazed how many people run SCA (or web scanners) with the default config and then forward results to developer teams, their own or third party's. Eventually, the person running the magical scanner starts being treated as the boy who cried wolf too often.

Now, this is *not* a post against static analysis. Static analysis can be an awesome tool for vulnerability research, especially for C/C++ code (although everyone seems to be 'fuzzing kernels' instead) and maybe even in web apps. That is, if the tool you've got is capable of being used as a research helper, not a checkbox filler.

Unfortunately the reaction of SCA salesmen to such a request (not of all, but many) is usually "You want what? Write your own rules? And drill down on results? And specify sanitisers and stuff? Crazy! Let me find someone back at headquarters who knows what you're talking about…"

Very often, a few simple scripts involving minimal lexing/parsing, written for *your* specific web app (even without getting into ASTs, solvers and data taint analysis) can be way more useful in finding old and preventing new issues. Especially if they are run as commit hooks in your git repo.

Back to the 'best practices' quote - if you are a software vendor and you want to get real benefits from commercial SCA tools (I do not count compliance among the benefits), do two things: hire someone with a clue (about app sec, about development, about SCA) and get a tool which has configurable rules.

Otherwise don't even bother. It will be about as effective as, and much more expensive than, running an IDS or AV.

Scams in security testing

Dedicated to people who submit Web scanner results to their software vendors.

A while ago I stumbled upon a book on software testing. Not security, mind you, just plain normal software testing. By my favourite "techie" author Gerald Weinberg - Perfect software and other illusions about software testing. It's a great read for app security folks, as long as you are capable of making basic domain substitutions.

My favourite chapter in the book is "Testing scams", where the author follows up his earlier discussion of fallacies in testing with a list of outright scams by vendors promising to sell a magic testing tools. He says

"Here's the secret about tools: Good tools amplify effectiveness. If your testing effectiveness is negative, adding tools will only amplify the negativity. Any other claim a tool vendor makes is, most likely, some kind of scam."

I made a short summary of this chapter, with examples from security testing domain (mostly web, "dynamic" and source code, "static" scanners). Text in quote marks is from the book, apart from the obvious phrases.

1. "Tool demonstration is a scam" - where you are shown a perfect demo, of a scanner running on WebGoat. And when you try it on your own code, the scanner explodes with junk. Of course the demo was carefully designed and tuned to produce the best impression.

Subtype 1a: You are not allowed do your own PoC without vendor's helpful supervision. At the least, they will give you a spreadsheet with criteria to compare their product against others.

Note: If you are not capable of conducting your own PoC without asking vendors for help, you should not be buying any of those tools in the first place.

2. "With all these testimonials, it must be good" - where there isn't even a demo, but you get a pile of whitepapers with pseudo-test results (comparisons, certifications, endorsements). These docs usually "appear to contain information, but ... only identify charlatans who took a fee or some payment [in kind] for the used of their names".

As a test, try requesting personal testimonies from the customers involved, ideally see how they use the tool. If the vendor cannot produce a single customer who is excited about their product so much that they want to show you how wonderful it is, it's a crap tool.

3. "We scam you with our pricing" - where the vendor creates cognitive dissonance among previously scammed people. As a result, despite the expensive purchase being a failure on all levels, from purchasers to end users, they keep this fact to themselves.

Subtype 3a: "[is] discrediting competitive tools with disingeniousness, suggesting, 'With a price so low, how could those tools be any good?'"

4. "Our tool can read minds" - where the tool is presented as a complete replacement of a security specialist - testing even better than a human, and not requiring any human post-processing. In personal experience, this is one of the most common scams in security testing market, with scam #1 being the next most popular, and #3 and #4 reserved for very pricey tools (you know who you are).

A belief that a magic app security silver bullet exists is so deep, that when the tool (or the "cloud" service) quickly fails to deliver to its promises, their customer concludes that this was his/her own mistake and there is another magic mind-reading service elsewhere. Rinse and repeat.

Note: There is no silver bullet. Artificial Intelligence is hard. Turing was right.

5. "We promise that you don't have to do a thing" - where a "cloud" service promises that all the customer has to do is to point it to the web app, and the service will spit out actionable results with no false positives and no false negatives. Less experienced security teams or managers fall for this one quite often, since many of the services come with a promise of manual postprocessing of results by the most experienced analysts in the world (or close to that). Where this fails is lack of context for the testing. The vendor does not know your code, they do not know your context in terms of exploitability, mitigating controls, or impacts. They do not know what technologies and methodologies your developers use. What usually comes out of such services is slightly processed results of a scanner with generic settings.

Subtype 5a: - when the service vendor does the old "bait and switch" between its personnel involved in the sales process (gurus) and who you get once you pay (little to no experience button pushers in a cheap outsourced location).

Still with me? Here's the summary:

If someone promises you something for nothing, it is a scam (or they are Mother Theresa, that is, not a business person). Even if you are promised a magic tool in exchange for a lot of money (scam 3 above), this is still a promise of something for nothing. 

It is impossible to do good security testing other than by employing (in one way or another) people who know the context of your environment and code or who are willing to learn it.

Focused code reviews - a followup

I promised something more technical than book reviews, so here it goes.

Earlier I posted about how to limit the amount of code for day-to-day security reviews if the code base is huge. I took Confluence (I work for Atlassian) as an example. The application uses Webworks 2, and other frameworks. Source code is not entirely free or public, but you can get it if you have almost any kind of Confluence license. I will keep some details out of this example.

Here are some things to trigger security reviews on this codebase.

Java generalities

Monitor for these being added, but there is no urgent need to review code if any of these get removed by developers. The list in this section is Java generic (and incomplete) and can be used for other apps, the other sections are more Confluence-specific. You might not need to trigger on all of these strings. You can also try structures from the IntelliJ searches from another blog entry.

Class.forName
ZipFile
Statement
Math.random
sendRedirect
"SELECT "
java.sql.Statement
java.sql.Connection
executeQuery
Runtime.
java.lang.Runtime
getRequestURI
java.sql
BeanUtils.setProp
java.lang.reflect

...

Sanitizers

Monitor for disappearance of any sanitisers from your code. There are legitimate reasons for this - for example a sanitiser in a view disappears but the corresponding model starts escaping or filtering data.

htmlEncode

...others skipped...

Filters


Being a Webwork2 webapp, Confluence utilises a number of filters and interceptors. You can get a list of filters your application uses with something like

grep -Rh --include=*.xml "<filter-name" . |sed -e 's/<filter-name>//'|sed -e 's/<\/filter-name>//'|sed -e 's/^[ \t]*//' |sort |uniq

Review the list and decide which ones have important security function. Monitor any change mentioning interceptors (both in web.xml files and for any change of their source)

HeaderSanitisingFilter
SecurityFilter
...
SafeParametersInterceptor
PermissionCheckInterceptor
...

Annotations

Some of these are generic, some are Confluence specific. One way of getting a list of all annotations is

grep -Rh --include=*.java "^\s\+@" . |sed -e 's/^[ \t]*//'  |sort |uniq

Example of what to monitor for:

@AnonymousAllowed	adding
@GET	adding
@POST	adding
@HttpMethodRequired	any change
@ParameterSafe	removal
@Path	adding
@RequireSecurityToken	removal
...

XML config files (new endpoints)

Action mapping etc - they introduce new URL endpoints. Monitor for adding, not removal.

"<action name" 
...

Other XML

Any change mentioning your filters or interceptors in web.xml, for example

<filter-name>header-sanitiser
<filter-name>request-param-cleaner
<filter-name>login

<interceptor-ref name="params"/>
<interceptor-ref name="permissions"/>
<interceptor-ref name="xsrfToken"/>

<interceptor-stack name 

...

Files and path

Look for any change in files used to implement crucial security features - login, session management, authorisation, sanitizers, CSRF protection and so on. 

confluence-core/confluence-webapp/src/main/webapp/WEB-INF/web.xml
confluence-core/confluence/src/etc/standalone/tomcat/web.xml
confluence-core/confluence/src/java/com/atlassian/confluence/security/login/*
confluence-core/confluence/src/java/com/atlassian/confluence/rpc/auth/*
confluence-core/confluence/src/java/com/atlassian/confluence/security/*
...

Monitoring for any web.xml changes is probably an overkill, you will catch interesting stuff with the items from other sections above).

Modern Web application security: Facebook, Twitter, Etsy

In the past 6 months at least 3 big modern webapp companies published details of how they do application security. Etsy was the first, with Twitter and Facebook close second. The presos are at:

Etsy: http://www.slideshare.net/zanelackey/effective-approaches-to-web-application-security
Twitter: http://www.slideshare.net/xplodersuv/putting-your-robots-to-work-14901538
Facebook: http://www.slideshare.net/mimeframe/ruxcon-2012-15195589

Despite small differences caused by frameworks and technologies these companies use, they all do the same set of things:

Code reviews

Security team does regular code reviews, and does them in a smart way. They have set up triggers for reviews, automated by unit tests or simply grep scripts looking at (D)VCS commits, e.g. git. These scripts monitor for two different kinds of changes:

1. Any change in "important" files. These usually are the parts of the app source code that deal with CSRF protection, encryption, login, session mgmt, XSS encoding.
2. Any new instances of "potentially nasty" snippets of code anywhere in the code base. These include introduction of file system operations, process execution, HTML decoding calls.

The above can be mixed and matched in a number of ways. For example, on can also monitor for any new URI endpoints (this can also be done via dynamic scanning, see below), or for people explicitly disabling automatic protections for CSRF, XSS, if you have these protections in place.

Dynamic scans

Security team have set up a number of Web bots to periodically scan their app for "simple" security issues.

NB: Do not use commercial scanner monsters, they are geared to produce as many results as (inhumanly) possible and are much more willing to produce false positives to reduce false negatives. In other words, they would rather alert on 10 "possible" issues that turn out to be non-issues, than miss one. The sad part is that they still miss a lot anyway.

What you (and everyone, unless they are paid by weight of the report) need is minimal false positives even at the cost of missing a number of things. Some mathematical reasoning behind the idea can be gathered from a 1999 paper The Base-Rate Fallacy and its Implications for the Difficulty of Intrusion Detection by Axelsson, who calculated that the IDS's false positive rate should be 10^-5 (yes, 1/100 000) in order for the alerts to be actionable in a high traffic environment.

All three companies use scanner bots to monitor for regressions ("hey, we fixed an XSS here, let's make sure it does not reappear"), for detecting new URI (if they do not detect them in source code), and other similar tasks, check their presos for details.

Secure by default

They developed (and it is a good idea for everyone else to do the same) their own or adopted "secure by default" frameworks. These frameworks are nothing grand - they achieve simple and important outcomes: provide automatic output encoding for XSS, automatically assign CSRF tokens and so on. Remember code monitoring scripts earlier? They trigger a security review if any of these security frameworks are disabled or opted out of on a specific page.

Security headers

Headers such as

• X-Content-Type-Options
• X-Xss-Protection
• CSP headers 

have gained popularity. They require care to implement and here the approaches differ, see the original presos.

A nice touch is deploying CSP policies in monitoring mode, without blocking anything, and analysing the resulting alerts (slide 45 in the Facebook deck). Applying CSP in blocking mode to a large existing app is a huge task and is likely not to gain traction with your developers. The CSP candidate spec says:

Content Security Policy (CSP) is not intended as a first line of defense against content injection vulnerabilities. Instead, CSP is best used as defense-in-depth, to reduce the harm caused by content injection attacks.
There is often a non-trivial amount of work required to apply CSP to an existing web application. To reap the greatest benefit, authors will need to move all inline script and style out-of-line, for example into external scripts, because the user agent cannot determine whether an inline script was injected by an attacker.

Graphing stuff

There are many things that can be graphed or at least thresholded with security benefits - CSP alerts, increase in traffic containing HTML, number of failed login attempts,...

Summary

All of the measures in this post are help the security team and make the team deliver the most bang for the buck. My next post will be on how to use similar tools for security "evangelism" (I try avoiding this word, it is misguiding), or "get those developers to not release vulnerable software"