Friday, 4 March 2016

Fiddling with Nexus 4 boot image

TLDR; How to modify any system to set ro.debuggable=1 without rebuilding it from source. This setting will make any apk debuggable on the device.

Get the existing boot image off the phone

dd if=/dev/block/mmcblk0p6 of=/mnt/sdcard/boot.img # on the phone
adb pull /mnt/sdcard/boot.img # on your computer

/dev/block/mmcblk0p6 is Nexus 4's boot partition.

Install abootimg from The rest of the process below is stolen from this page.

Extract and unpack initrd

mkdir boot 
cd boot
abootimg -x /tmp/boot.img

mkdir initrd
cd initrd
cat ../initrd.img | gunzip | cpio -vid

Edit default.prop, setting anything you want, including ro.debuggable=1.

Repack initrd and boot image

cd initrd
find . | cpio --create --format='newc' | gzip > ../myinitrd.img
cd ..
abootimg --create myboot.img -f bootimg.cfg -k zImage -r myinitrd.img

Flash to phone

adb reboot-bootloader
fastboot flash boot myboot.img

Android Studio for refactoring obscure decompiled code

"It's in Foreign" @thegrugq

A while ago, I've been experimenting with using Android Studio for refactoring decompiled code.

  1. Export Java sources, from whatever decompiler works 
  2. "Import project" from sources in Android Studio 
  3. Use Shift-Fn-F6 to rename classes, methods etc 
What's best is that Studio (hurray for IntelliJ IDEA) is sometimes intelligently estimates types of variables and offers reasonably meaningful names:

Thursday, 3 March 2016

Using Proguard to deobfuscate code

TLDR; Optimisation features of Proguard can be useful for removing some "obfuscations" that add dead code and screw up control flow.

Proguard comes with Android Studio or can be installed from homebrew on Mac, that one version is newer:

$brew install proguard

Then use a generic script similar to this one:

-injars      <obfuscated jar>
-outjars     <result>
-libraryjars $HOME/Library/Android/sdk/platforms/android-19/android.jar ; or similar
-optimizationpasses 10 


-keepattributes *Annotation*

-keep public class * extends
-keep public class * extends
-keep public class * extends
-keep public class * extends android.content.BroadcastReceiver
-keep public class * extends android.content.ContentProvider
-keep public class * extends
-keep public class * extends android.view.View {
 public <init>(android.content.Context);
        public <init>(android.content.Context, android.util.AttributeSet);
 public <init>(android.content.Context, android.util.AttributeSet, int);
 public void set*(...);
-keepclasseswithmembers class * {
 public <init>(android.content.Context, android.util.AttributeSet);
-keepclasseswithmembers class * {
 public <init>(android.content.Context, android.util.AttributeSet, int);
-keepclassmembers class * extends android.content.Context {
 public void *(android.view.View);
 public void *(android.view.MenuItem);
-keepclassmembers class * implements android.os.Parcelable {
 static ** CREATOR;
-keepclassmembers class **.R$* {
 public static <fields>;
-keepclassmembers class * {
 @android.webkit.JavascriptInterface <methods>;
-keepclasseswithmembernames class * {
    native <methods>;
-keepclassmembers enum * {
    public static **[] values();
    public static ** valueOf(java.lang.String);

Run as proguard @deob.conf and you'll end up with a more readable version of your obfuscated code.

Thursday, 25 February 2016

Re-signing VirusTotal samples to install them on a non-jailbroken iPhone

Sir Leigh Teabing: In which year did a Harvard scholar outrow an Oxford man at Henley? 
Robert Langdon: [reluctantly] Surely such a travesty has never occurred.

Why would one even want to do this? Let's say, you're doing some sort of zomg malware research and you want to verify something. You can either use a jailbroken phone, code signing is kinda moot there, or a real phone :)

VT samples of iPhone apps are sometimes encrypted, tough luck then. Most of the time they are already stripped of DRM. So you can download one and rename it to *.ipa. Then the fun begins.
I'll assume you already have a working Xcode setup and can sign and run a test app on your device. If not, Google it, there are bazillions of how-tos on that topic.

The best and most fool-proof application for re-signig is iReSign, or you can do the signing manually, using commands similar to ones here (codesign is the main one, duh) or here. IReSign requires you to chose the application to resign, the certificate and the profile. Entitlements can be usually omitted, the app will attempt it best at recovering the ones required. The ID field can be left unchanged if your certificate allows for * to be signed, otherwise change it to something that you can actually sign for.


  • If in the process of re-signing VT ipas you get "Getting certificate ID's failed" from iReSign and the cert list is empty, don't worry, this is temporary after 16 Feb 2016 because Apple's root cert has expired. To save you some googling - ("Show expired certificates" in keychain, then delete the old ones, install the new one).
  • iReSign needs the correct provisioning profile that applies both to the ipa file being re-signed and your iPhone. It looks like setting them is a little contrived, what finally worked for me was using “iOS Team Provisioning Profile: *” from Xcode. Download it, or a similar one from developer centre. It massively depends on your particular setup, though. The simplest way to test yours is to create a base iOS project with Xcode, fiddle with settings (do not leave them on "Automatic") in "Build Settings"->"Code Signing" and try "Project"->"Run" for your dummy project. If it does install and run, use the same identity and provisioning profile for iReSign.

  • You are likely to get (if you install via Xcode, iTunes install will silently fail) "Can't install application. The Info.plist for application at... specifies a CFBundleExecutable of ..., which is not executable" at some stage. This is because VirusTotal samples do not maintain Unix file permissions and the main executable needs to be, well, executable. Unpack and "chmod +x" then repack the ipa, either before or after re-signing:
  • Sometimes troubles with signing are due to a mismatch of the certificate in the provisioning profile and the keys you have. See here for how to check the DeveloperCertificates in the provisioning profile (and don't forget to split lines in your .pem to 63 char lengths). Then again, overall signing is described in a million places on the Internet :)

Wednesday, 10 February 2016

More Meta Than Regehr


This is another meta-meta-meta-meta-model page to show up in Google search. John, you're welcome :D

Wednesday, 3 February 2016

More IDA Pro plugins for OS X - HexRaysCodeXplorer

Another little thing I did. Compiling was rather painless - mostly changing __LINUX__ to __MAC__ and recovering a few files from pre-6.9 version. It looks like the format of "custom view handlers" in Hex-Rays SDK has changed between IDA 6.8 and 6.9.

So the diff is here and the (IDA 6.8) plugin is here (32bit) + here (64bit).

IMMV - I have only tested that it compiles, displays context menu and C-Tree graph.

I'll maintain a branch for IDA Pro 6.8 OS X for a brief while at

Monday, 25 January 2016

Compiling non-OSX IDA pro plugins on OS X, or sanity ala Einstein

"Insanity: doing the same thing over and over again and expecting different results." - Albert Einstein

Recently I was looking at more IDA Pro on OS X than normally, so I experimented with themes a bit, e.g. Consonance. Then I found IDASkins. Lo and behold, there are tons of Windows IDA versions for which it has been already precompiled in the github repo, but no OS X ones.

OS X compilation in cmake files in IDASkins is marked "untested" and it is, in fact, a bit buggy. After some fiddling I got the compilation to work, although one of the resulting plugins (one for Ida 64) crashes everything, no idea why :). I'm mostly looking at 32bit ARM anyways, so one plugin is good enough for me.

The diff that made the build work is in this gist. It contains mostly cosmetic changes, plus a weird fact that "if(APPLE)" didn't work properly in the cmake I had (from homebrew?). Plus I haven't used make before :) YMMV

Plugin for 32bit OS X IDA Pro 6.8 is here.

Oh and while I am compiling plugins, here are the two precompiled WWDC plugins for the same version of IDA Pro. No tricks in compiling it from source, just stick it into "plugins" dir of IDA SDK and have a universal or i386 libcapstone.a (the one from home-brew is x64 only, I believe).

Friday, 9 October 2015

Time flies like an arrow...

...and fruit flies like a banana.

A lot happened in the past year. I moved to San Francisco, survived being seriously hit by a car  - facing a long, maybe a year, recovery... While at the hospital, I figured out a few things about life, universe, and everything. I won't reiterate those things here - I might sound as a buddhist nut if I do.

I will try to start blogging again, slowly. Among plans - translate the best articles has on Android (my current fancy and job).

PS. In case I never gave out a link to my Twitter, it's @agelastic. Still struggling with typing, so most posts are retweets :)

Tuesday, 2 September 2014

Big data analytics with grep and sed

This was written a couple of years ago, when a gigabyte of logs was reasonably "large scale". Nevertheless, the approach still works for many log monitoring tasks.

Part 1.

A few jobs ago I was on a team tasked with creating a system to monitor security events for a Swiss bank with 20k+ (all physical!) Unix hosts plus 60k Windows desktops and servers.

Hadoop did not exist yet outside of Yahoo research labs, so we had to do with MySQL, a bit of C and Perl. By the way, the resulting system kept chugging along for at least 5 years, if not 10, before finally succumbing to the likes of ArcSight.

We already had a centralised syslog setup, which produced tons of logs. To start with, we needed to come up with:
  • a list of things to ignore, 
  • a list of things to definitely alert on, 
  • and an anomaly detection algorithm. 
This turned out simpler than it sounded (hurray for 20-20 hindsight!), thanks to some ideas borrowed from Marcus Ranum's "Artificial Ignorance" and "Never Before Seen"

Part 2.

Back at my current job, I'd spent a couple of hours digging into auth.log from our (then) new and shiny SaaS infrastructure manager node (~ 1 Gb), and it summed up nicely into 38k of "interesting" events. Once you've created your own stoplist, you can run the filtered output in a terminal in background. It contains 10-20 messages an hour:
  • Manual, as opposed to scripted jobs, ssh logins,
  • Unscheduled scripts,
  • Adhoc fix work,
  • Auth failures,
  • Various unique ("never before seen") alarms.
I'm not including a sample of the filtered results - try this at home instead.

What I did was a dozen of iterations of

cat auth.* | grep -v -f stoplist.txt |sed -f sedcommands.txt |  sort | uniq -c | sort -r -n > stuff

starting with an empty stoplist and the following file sedcommands.txt:

s/^.\{15\}\ [a-z0-9.-]*\ //

This cuts off the initial part of the log line and summarises similar lines.  Looking at the sorted results, I add a few lines to the stoplist to get rid of the most uninteresting from security/anomaly point of view entries (usually top hits), rinse, repeat.

The final stoplist (slightly censored):

last message repeated .* times
Accepted publickey for .* from 59\.167\.XX\.XX
Accepted publickey for .* from 10\.65\..*
Accepted publickey for .* from 207\.223\.XX\.XX
session opened for user root by (uid=0)
session opened for user postgres by (uid=0)
session opened for user YYYYYYYY by (uid=0)
session opened for user www-data by (uid=0)
session closed for user
Received disconnect from .*: 11: disconnected by user
Accepted publickey for YYYYYYYY from 67\.221\.XX\.XX
USER=root ; COMMAND=/usr/lib/nagios/plugins/uc_.*
USER=root ; COMMAND=/usr/share/pyshared/pycm/bin/nagios/
USER=root ; COMMAND=/usr/lib/container-manager/.*
USER=root ; COMMAND=/opt/.*
USER=root ; COMMAND=/usr/bin/ZZZZZZ
PWD=/usr/share/pyshared/pycm/bin/vz ; USER=root ;
PWD=/usr/share/pyshared/pycm/bin/vz ; USER=root ;
PWD=/opt/unicorn-repl-tools/var/queue ; USER=root ; COMMAND=/bin/rm \./.*\.net
PWD=/opt/unicorn-repl-tools/var/queue ; USER=root ; COMMAND=/bin/rm \./.*\.com
PWD=/home/hal-uc-1 ; USER=root ; COMMAND=/usr/share/pyshared/pycm/
running '/sbin/halt' with root privileges on behalf of 'root'

This is only for illustration - do not copy the result, the point of this exercise is for you to come up with your own that is best fit for your environment. If you deal with extremely complex logfiles, you could throw in some AWK :)

In all, it took about 2 hours in total from getting the logs to producing the summary, including the remembering how this stuff was done.