Re-signing VirusTotal samples to install them on a non-jailbroken iPhone

Sir Leigh Teabing: In which year did a Harvard scholar outrow an Oxford man at Henley? 
Robert Langdon: [reluctantly] Surely such a travesty has never occurred.

Why would one even want to do this? Let's say, you're doing some sort of zomg malware research and you want to verify something. You can either use a jailbroken phone, code signing is kinda moot there, or a real phone :)

VT samples of iPhone apps are sometimes encrypted, tough luck then. Most of the time they are already stripped of DRM. So you can download one and rename it to *.ipa. Then the fun begins.
I'll assume you already have a working Xcode setup and can sign and run a test app on your device. If not, Google it, there are bazillions of how-tos on that topic.

The best and most fool-proof application for re-signig is iReSign, or you can do the signing manually, using commands similar to ones here (codesign is the main one, duh) or here. IReSign requires you to chose the application to resign, the certificate and the profile. Entitlements can be usually omitted, the app will attempt it best at recovering the ones required. The ID field can be left unchanged if your certificate allows for * to be signed, otherwise change it to something that you can actually sign for.

Tips

• If in the process of re-signing VT ipas you get "Getting certificate ID's failed" from iReSign and the cert list is empty, don't worry, this is temporary after 16 Feb 2016 because Apple's root cert has expired. To save you some googling - http://stackoverflow.com/a/35399656 ("Show expired certificates" in keychain, then delete the old ones, install the new one).

• iReSign needs the correct provisioning profile that applies both to the ipa file being re-signed and your iPhone. It looks like setting them is a little contrived, what finally worked for me was using “iOS Team Provisioning Profile: *” from Xcode. Download it, or a similar one from developer centre. It massively depends on your particular setup, though. The simplest way to test yours is to create a base iOS project with Xcode, fiddle with settings (do not leave them on "Automatic") in "Build Settings"->"Code Signing" and try "Project"->"Run" for your dummy project. If it does install and run, use the same identity and provisioning profile for iReSign.

• You are likely to get (if you install via Xcode, iTunes install will silently fail) "Can't install application. The Info.plist for application at... specifies a CFBundleExecutable of ..., which is not executable" at some stage. This is because VirusTotal samples do not maintain Unix file permissions and the main executable needs to be, well, executable. Unpack and "chmod +x" then repack the ipa, either before or after re-signing: https://github.com/maciekish/iReSign/issues/45.
• Sometimes troubles with signing are due to a mismatch of the certificate in the provisioning profile and the keys you have. See here for how to check the DeveloperCertificates in the provisioning profile (and don't forget to split lines in your .pem to 63 char lengths). Then again, overall signing is described in a million places on the Internet :)

More Meta Than Regehr

Context

This is another meta-meta-meta-meta-model page to show up in Google search. John, you're welcome :D
 

More IDA Pro plugins for OS X - HexRaysCodeXplorer

Another little thing I did. Compiling was rather painless - mostly changing __LINUX__ to __MAC__ and recovering a few files from pre-6.9 version. It looks like the format of "custom view handlers" in Hex-Rays SDK has changed between IDA 6.8 and 6.9.

So the diff is here and the (IDA 6.8) plugin is here (32bit) + here (64bit).

IMMV - I have only tested that it compiles, displays context menu and C-Tree graph.

I'll maintain a branch for IDA Pro 6.8 OS X for a brief while at https://github.com/agelastic/HexRaysCodeXplorer/tree/OSX_IDA68

Compiling non-OSX IDA pro plugins on OS X, or sanity ala Einstein

"Insanity: doing the same thing over and over again and expecting different results." - Albert Einstein

Recently I was looking at more IDA Pro on OS X than normally, so I experimented with themes a bit, e.g. Consonance. Then I found IDASkins. Lo and behold, there are tons of Windows IDA versions for which it has been already precompiled in the github repo, but no OS X ones.

OS X compilation in cmake files in IDASkins is marked "untested" and it is, in fact, a bit buggy. After some fiddling I got the compilation to work, although one of the resulting plugins (one for Ida 64) crashes everything, no idea why :). I'm mostly looking at 32bit ARM anyways, so one plugin is good enough for me.

The diff that made the build work is in this gist. It contains mostly cosmetic changes, plus a weird fact that "if(APPLE)" didn't work properly in the cmake I had (from homebrew?). Plus I haven't used make before :) YMMV

Plugin for 32bit OS X IDA Pro 6.8 is here.

Oh and while I am compiling plugins, here are the two precompiled WWDC plugins for the same version of IDA Pro. No tricks in compiling it from source, just stick it into "plugins" dir of IDA SDK and have a universal or i386 libcapstone.a (the one from home-brew is x64 only, I believe).

Windows, so many windows

Refreshing my very rusty (since late 90s / early 00s!) knowledge of reversing on Windows

Currently reading:

• Practical Malware Analysis
• Windows Internals, Part 1 (6th Edition)
• What Makes It Page?: The Windows 7 (x64) Virtual Memory Manager  

Skimming:

• Operating Systems: Three Easy Pieces
• Trading and Exchanges: Market Microstructure for Practitioners (totally unrelated :) )

Need to re-read, it's been a over a year:

• Practical Reverse Engineering: x86, x64, ARM, Windows Kernel, Reversing Tools, and Obfuscation

Recommendations welcome!

Time flies like an arrow...

...and fruit flies like a banana.

A lot happened in the past year. I moved to San Francisco, survived being seriously hit by a car  - facing a long, maybe a year, recovery... While at the hospital, I figured out a few things about life, universe, and everything. I won't reiterate those things here - I might sound as a buddhist nut if I do.

I will try to start blogging again, slowly. Among plans - translate the best articles drops.wooyun.org has on Android (my current fancy and job).

PS. In case I never gave out a link to my Twitter, it's @agelastic. Still struggling with typing, so most posts are retweets :)

Big data analytics with grep and sed

This was written a couple of years ago, when a gigabyte of logs was reasonably "large scale". Nevertheless, the approach still works for many log monitoring tasks.

Part 1.

A few jobs ago I was on a team tasked with creating a system to monitor security events for a Swiss bank with 20k+ (all physical!) Unix hosts plus 60k Windows desktops and servers.

Hadoop did not exist yet outside of Yahoo research labs, so we had to do with MySQL, a bit of C and Perl. By the way, the resulting system kept chugging along for at least 5 years, if not 10, before finally succumbing to the likes of ArcSight.

We already had a centralised syslog setup, which produced tons of logs. To start with, we needed to come up with:

• a list of things to ignore, 
• a list of things to definitely alert on, 
• and an anomaly detection algorithm. 

This turned out simpler than it sounded (hurray for 20-20 hindsight!), thanks to some ideas borrowed from Marcus Ranum's "Artificial Ignorance" and "Never Before Seen"

Part 2.

Back at my current job, I'd spent a couple of hours digging into auth.log from our (then) new and shiny SaaS infrastructure manager node (~ 1 Gb), and it summed up nicely into 38k of "interesting" events. Once you've created your own stoplist, you can run the filtered output in a terminal in background. It contains 10-20 messages an hour:

• Manual, as opposed to scripted jobs, ssh logins,
• Unscheduled scripts,
• Adhoc fix work,
• Auth failures,
• Various unique ("never before seen") alarms.

I'm not including a sample of the filtered results - try this at home instead.

What I did was a dozen of iterations of

cat auth.* | grep -v -f stoplist.txt |sed -f sedcommands.txt |  sort | uniq -c | sort -r -n > stuff

starting with an empty stoplist and the following file sedcommands.txt:

s/^.\{15\}\ [a-z0-9.-]*\ //
s/\[[0-9]*\]://

This cuts off the initial part of the log line and summarises similar lines.  Looking at the sorted results, I add a few lines to the stoplist to get rid of the most uninteresting from security/anomaly point of view entries (usually top hits), rinse, repeat.

The final stoplist (slightly censored):

last message repeated .* times
Accepted publickey for .* from 59\.167\.XX\.XX
Accepted publickey for .* from 10\.65\..*
Accepted publickey for .* from 207\.223\.XX\.XX
session opened for user root by (uid=0)
session opened for user postgres by (uid=0)
session opened for user YYYYYYYY by (uid=0)
session opened for user www-data by (uid=0)
session closed for user
Received disconnect from .*: 11: disconnected by user
Accepted publickey for YYYYYYYY from 67\.221\.XX\.XX
USER=root ; COMMAND=/usr/lib/nagios/plugins/uc_.*
USER=root ; COMMAND=/usr/share/pyshared/pycm/bin/nagios/
USER=root ; COMMAND=/usr/lib/container-manager/.*
USER=root ; COMMAND=/opt/.*
USER=root ; COMMAND=/usr/bin/ZZZZZZ
PWD=/usr/share/pyshared/pycm/bin/vz ; USER=root ;
PWD=/usr/share/pyshared/pycm/bin/vz ; USER=root ;
PWD=/opt/unicorn-repl-tools/var/queue ; USER=root ; COMMAND=/bin/rm \./.*\.net
PWD=/opt/unicorn-repl-tools/var/queue ; USER=root ; COMMAND=/bin/rm \./.*\.com
PWD=/home/hal-uc-1 ; USER=root ; COMMAND=/usr/share/pyshared/pycm/
running '/sbin/halt' with root privileges on behalf of 'root'



This is only for illustration - do not copy the result, the point of this exercise is for you to come up with your own that is best fit for your environment. If you deal with extremely complex logfiles, you could throw in some AWK :)

In all, it took about 2 hours in total from getting the logs to producing the summary, including the remembering how this stuff was done.

Implicit security and waste-cutting

I have a nagging feeling that I've read some of this somewhere a long time ago. Probably in @mjranum's papers.

Imagine you have a product or a process. It kind of works, although you can already see the ways you can improve it by cutting some stuff here and there and changing the way it operates.

A random example: an old paper based process in a company (say, a telco) is replaced with a nice workflow all done on a web site. All information is available there for anyone immediately, all staff are happy, costs decrease, productivity increases 10-fold and so on. The process was concerned with customer contracts, so the information is kind of sensitive.

Bad hackers discover the web site, dump all the data, identity theft sky-rockets, the bank is massively fined and everyone is sad. Downsizing occurs, and so on.

Now what happened here? The old process was slow and cumbersome and low productivity. At the same time it had a number of implicit security measures built in simply by the nature of it being paper based. In order to steal a million paper contracts, one has to break-and-enter the bank facility, plus have a big van to haul all this stuff out. The loss would be immediately discovered (photocopying is not an option due to the time limitations of the heist).

Designers of the new process did not identify these implicit measures or implicit requirements because nobody thought about them. After all, the measures were implicit.
Some of the cost savings of that redesign came from (unintentional) dropping these implicit requirements or measures.

Why I am writing about this? To remind: when you are putting together a new project or re-engineer a process, check if you forgot to implement security that you did not even know was there. Stop for a moment and think about what weaknesses your product or process has, what or who may exploit these weaknesses and what the results will be. "What" could be random event, not necessarily a malicious person. In some places they call this risk management.

The funniest example is OpenSSL in Debian - http://research.swtch.com/openssl.

A less funny example is Vodafone AU customer database project which is more or less described above. It did have one password for all employees.http://www.smh.com.au/technology/security/mobile-security-outrage-private-details-accessible-on-net-20110108-19j9j.html

How I installed pandoc

Probably funny

Pandoc is a "swiss-army knife" utility to transform documents in various markup languages. I needed it for something, and this post is how I managed to get it installed. In retrospect, I could have used the installer from http://code.google.com/p/pandoc/downloads/list

$ brew install pandoc
Error: No available formula for pandoc
Searching taps...
#...Googling... Turns out it's a Haskell thing?

$ brew install haskell-platform
...
==> Installing haskell-platform dependency: apple-gcc42
# ZOMG?!
...
...A GFortran compiler is also included
# ooohkay...
...

$ cabal update
...
Note: there is a new version of cabal-install available.
To upgrade, run: cabal install cabal-install

$ cabal install cabal-install
# Those strange Haskell people ^_^...
#...downloading the Internet...

$ cabal install pandoc
# FINALLY WE ARE GETTING SOMEWHERE!
#...downloading the Internet the second time...

$ pandoc
-bash: pandoc: command not found
# WTF?!

$ export PATH=$PATH:~/.cabal/bin

DONE...

How to buy code audits

In the past couple of years I've commissioned quite a few external source code audits. It turns out this task is far from simple, despite what the friendly salesmen tell you. I've collected a few thoughts and learnings based on my experience. YMMV.

First of all, all the advice you need is in @mdowd's (et al.) interview http://www.sans.edu/research/security-laboratory/article/192 (question 3). Quote:

Word-of-mouth recommendations often convey the best real-world measure of experience. To cast a wider net though, you can use publications and industry recognition as a good measure of reputation. When approaching a company, you may also want to ask for bios on the auditors likely to perform your assessment. Next, you'll want to ask for a sample report from any auditors you're considering. The quality of this report is extremely important, because it's a large part of what you're paying for. The report should be comprehensive and include sections targeted at the developer, management, and executive levels. The technical content should be clear enough that any developer familiar with the language and platform can follow both the vulnerability details and the recommendations for addressing them. You also need to get some understanding of the audit process itself. Ask if they lean toward manual analysis or if it's more tool-driven. Ask for names and versions of any commercial tools. For proprietary tools, ask for some explanation of the capabilities, and what advantages their tools have over those on the market. You also want to be wary of any process that's overly tool driven. After all, you're paying a consultant for their expertise, not to simply click a button and hand you a report. If a good assessment was that easy, all software would be a lot more secure.

Word-of-mouth is indeed the best indicator of quality. All other criteria are substitutes and make the selection inherently more risky. As with many risks, there are some things you can do to control them.

1. Beware of "bait and switch". This is a technique common in outsourcing when you are promised the best and famous resources when negotiating, but after the contract is signed, you get monkeys. 

When engaging any company larger than a boutique, insist on interviewing and approving every person who is going to work on your code. Ask for hard numbers of their experience in code auditing - "How many days in the past 3 years had this person worked on code audits in <language X>?" This metric is good because there is no reason for the vendor not to share it. 

Try not to deal with a company that has just acquired another entity or has just been acquired itself. These people have more important issues to sort out than your code, even if they say otherwise.

2. Your boss will probably ask you for an objective metric of progress. We all know the audit is more of a research than assembly line work. It is still possible to produce some useful metrics. Insist on specify your own format of reports, the one that works for you. I use a variation of the following:

• Description of work done during the week, 1-2 pages.
• Which sections of code or which application modules have been audited.
• A summary of any results - positive findings (issues found and filed in our bug tracker) or negative (code looked at is ok security-wise).
• Other work done, e.g. documentation
• Details of each confirmed positive finding (security vulnerabilities found)
• Description
• Relevant places in code
• Screenshot of successful exploitation where applicable
• Recommendations on how to fix
• Proposed severity rating
• Issue number in our bug tracker if already filed.

This way you see not only findings but also coverage and the general direction of the review. After all, if nothing is found you want some reassurane that the vendor had looked in the right places.

3. Run some static/dynamic tools yourself before handing the contract to vendors. Veracode is cheap. Findbugs is free. 

A very popular "manual" code review methodology seems to be "grep and gripe." This consists of running an inhouse pattern matcher or a commercial tool, tracing the results to inputs and filing bugs. This way you get tons of issues like "insufficient entropy", "DOM XSS" and other simple cruft, yet business logic issues will never be uncovered. Do reiterate to the vendor (several times) that you value logic issues, not grep results.

A couple of posts on related topics - One way of grepping, Some IntelliJ magic, and On automated tools

4. Require the vendor to work with your developer team. In my experience, the very first thing the best auditors ask for is code access, and the very second - a meeting with developers, in order to verify the auditor's understanding of the code base. NB: ask developers after the meeting what they think of the auditor. If developers are not impressed, you'll probably be wasting your money. Oh, and if you're not in a good relationship with your developers, maybe look for a new job :)

Do weekly reviews of results in person or on the phone, where you and developers review and validate each finding - sometimes things do not really make sense or, on the contrary, may give rise to further exploration ideas. If you leave everything until the review is completed, more likely than not it won't be a satisfactory one.

5. All of the above should ideally be specified in a contract. As a bare minimum, get an acceptance clause in: if you do not accept the final report from the auditor, they do not get paid and have to fix it up for you for free.

Good luck :)

P.S. Someone asked for timeframes for a good audit. It's difficult to say because no "coverage" metrics really apply. At the same time there is anecdotal evidence that you can expect up to 8 kLoC to be eyeballed by an auditor a day (on average, on large code bases). Properly "read" is more like 800 LoC a day - at the same time, hopefully not more than 1/10 of your code base is relevant to security, especially if it's Java with its verbosity.