It’s finally release o’clock, and this time around the focus has been on
improving quality. As it’s been a while since the last time we upgraded our
third-party dependencies, and I found myself tracking down a memory-leak in GLib
that had already been fixed upstream, I figured it was time to upgrade our
dependencies. So with this release I’m happy to announce that we’re now packing
the latest V8, GLib, Vala compiler, etc. Great care was also taken to eliminate
resource leaks, so you can attach to long-running processes without worrying
about memory allocations or OS handles piling up.
So in closing, let’s summarize the changes:
7.3.0:
core: upgrade to the latest V8, GLib, Vala, Android NDK, etc.
core: plug resource leaks
core: fix thread enumeration on Linux/x86-32
core: (arm64) improve function hooking by adding support for relocating LDRPC
with an FP/SIMD destination register
7.3.1:
core: build Android binaries with PIE like we used to
7.3.2:
core: add Script.setGlobalAccessHandler() for handling attempts to access
undeclared global variables, which is useful for building REPLs
7.3.3:
objc: convert Number to NSNumber when an object is expected
objc: add support for auto-converting to an array of objects, useful when
calling e.g. +[NSArray arrayWithObjects:count:]
7.3.4:
core: improve the unstable accessor API
core: fix the Duktape globals accessor logic so it’s only applied to reads
7.3.5:
core: improve hexdump() to support any NativePointer-conforming object
objc: fix handling of the L type
7.3.6:
core: fix regression in devkit header auto-generation logic
First off is the long-standing issue where multiple Frida clients attached to
the same process were forced to coordinate so none of them would call detach()
while one of the others was still using the session.
This was probably not a big deal for most users of Frida. However, we also had
the same issue if one running frida-server was shared by multiple clients.
You might have frida-trace running in one terminal while using the REPL in
another, both attached to the same process, and you wouldn’t then expect one of
them calling detach() to result in the other one getting kicked out.
Some of you may have tried this and observed that it works as expected, but this
was due to some crazy logic in frida-server that would keep track of how many
clients were interested in the same process, so it could ignore a detach()
call if other clients were still subscribed to the same session. It also had
some logic to clean up a certain client’s resources, e.g. scripts, if it
suddenly disconnected.
Starting with 8.0 we have moved the session awareness into the agent, and kept
the client-facing API the same, but with one little detail changed. Each call to
attach() will now get its own Session, and the injected agent is aware of it.
This means you can call detach() at any time, and only the scripts created in
your session will be destroyed. Also, if your session is the last one alive,
Frida will unload its agent from the target process.
That was the big change of this release, but we didn’t stop there.
One important feature of Frida’s scripts is that you can exchange messages with
them. A script may call send(message[, data]) to send a JSON-serializable
message, and optionally a binary blob of data next to it. The latter is so
you don’t have to spend CPU-cycles turning your binary data into text that you
include in the message.
It is also possible to communicate in the other direction, where the script
would call recv(callback) to get a callback when you post_message() to
it from your application. This allowed you to post a JSON-serializable message
to your script, but there was no support for sending a binary blob of data
next to it.
To address this shortcoming we renamed post_message() to post(), and gave it
an optional second argument allowing you to send a binary blob of data next to
it.
We also improved the C API by migrating from plain C arrays to GBytes,
which means we are able to minimize how many times we copy the data as it flows
through our APIs.
So in closing, let’s summarize the changes:
8.0.0:
core: add support for multiple parallel sessions
core: rename Script’s post_message() to post() and add support for passing
out-of-band binary data to the script
core: replace C arrays with GBytes to improve performance
core: fix heap corruption caused by use-after-free in libgee
core: fix multiple crashes
core: fix exports enumeration crash on macOS Sierra
core: add basic support for running on Valgrind
core: bump the macOS requirement to 10.9 so we can rely on libc++
node: update to the new 8.x API
python: update to the new 8.x API
swift: update to the new 8.x API
swift: upgrade to Swift 3
qml: update to the new 8.x API
clr: update to the new 8.x API
clr: plug leaks
8.0.1:
node: fix Script#post()
8.0.2:
core: fix deadlock when calling recv().wait() from our JS thread
8.0.3:
core: reduce Interceptor base overhead by up to 65%
core: minimize Interceptor GC churn in our V8 runtime, using the same
recycling and copy-on-write tricks as our Duktape runtime
core: speed up gum_process_get_current_thread_id() on macOS and iOS
It’s time for a release, and this time we have some big new things for those of
you building Frida-based tools, plus a few additional goodies. Let’s start with
the first part.
There’s no doubt that Frida’s JavaScript API is fairly low-level and only
meant to provide low-level building blocks that don’t pertain to just one
specific use-case. If your use-case involves grabbing screenshots on iOS, for
example, this is not functionality one would expect to find in Frida itself.
You may then wonder how different tools with common features are supposed to
share agent code with each other, and luckily the answer is not “copy paste”.
We have a growing ecosystem of Frida-specific libraries, like
frida-screenshot, frida-uikit, frida-trace, etc.
Perhaps some of you would be interested in APIs for instrumenting backend
software written in Java, .NET, Python, Ruby, or Perl, or perhaps you would want
to trace crypto APIs across different OSes and libraries, or some other cool
idea. I would then highly recommend that you publish your module to npm, perhaps
naming your module frida-$name to make it easy to discover.
Now you might be asking “but Frida does not support require(), how can I even
split my agent code into multiple files in the first place?”. I’m glad you
asked! This is where a handy little CLI tool called frida-compile enters
the picture.
You give it a .js file as input and it will take care of bundling up any other
files it depends on into just one file. But unlike a homegrown concatenation
solution using cat, the final result also gets an embedded source map, which
means filenames and line numbers in stack traces are meaningful. Modules are
also separated into separate closures so variables are contained and never
collide. You can also use the latest JavaScript syntax, like
arrow functions, destructuring, and generator functions, as it
compiles the code down to ES5 syntax for you. This means that your code also
runs on our Duktape-based runtime, which you are forced to use if you use Frida
on a jailed iOS device, or on a jailbroken iOS device running iOS >= 9.
In order to give you a short feedback loop while developing, frida-compile also
provides a watch mode through -w, so you get instant incremental builds as you
develop your agent.
Anyway, enough theory. Let’s look at how we can use an off-the-shelf web
application framework from npm, and inject that into any process.
First, make sure you have the latest version of Node.js installed. Next, create
an empty directory and paste this into a file named “package.json”:
{"name":"hello-frida","version":"1.0.0","scripts":{"prepublish":"npm run build","build":"frida-compile agent -o _agent.js","watch":"frida-compile agent -o _agent.js -w"},"devDependencies":{"express":"^4.14.0","frida-compile":"^2.0.6"}}
Then in agent.js, paste the following code:
'use strict';constexpress=require('express');constapp=express();app.get('/ranges',(req,res)=>{res.json(Process.enumerateRangesSync({protection:'---',coalesce:true}));}).get('/modules',(req,res)=>{res.json(Process.enumerateModulesSync());}).get('/modules/:name',(req,res)=>{try{res.json(Process.getModuleByName(req.params.name));}catch(e){res.status(404).send(e.message);}}).get('/modules/:name/exports',(req,res)=>{res.json(Module.enumerateExportsSync(req.params.name));}).get('/modules/:name/imports',(req,res)=>{res.json(Module.enumerateImportsSync(req.params.name));}).get('/objc/classes',(req,res)=>{if(ObjC.available){res.json(Object.keys(ObjC.classes));}else{res.status(404).send('Objective-C runtime not available in this process');}}).get('/threads',(req,res)=>{res.json(Process.enumerateThreadsSync());});app.listen(1337);
Install frida-compile and build your agent in one step:
$ npm install
Then load the generated _agent.js into a running process:
Sweet. We just built a process inspection REST API with 7 different endpoints in
fewer than 50 lines of code. What’s pretty cool about this is that we used an
off-the-shelf web application framework written for Node.js. You can actually
use any existing modules that rely on Node.js’ built-in net and http
modules. Like an FTP server, IRC client, or NSQ client.
So up until this release you could use Frida-specific modules like those
mentioned earlier. You could also use thousands of other modules from npm, as
most of them don’t do any I/O. Now with this release you also get access to
all net and http based modules, which opens up Frida for even more cool
use-cases.
In case you are curious how this was implemented, I added Socket.listen()
and Socket.connect() to Frida. These are minimal wrappers on top of GIO’s
SocketListener and SocketClient, which are already part of Frida’s
technology stack and used by Frida for its own needs. So that means our
footprint stays the same with no dependencies added. Because frida-compile
uses browserify behind the scenes, all we had to do was plug in our own
builtins for net and http. I simply ported the original net and http
modules from Node.js itself.
This release also brings some other goodies. One long-standing limitation with
NativeFunction is that calling a system API that requires you to read errno
(UNIX) or call GetLastError() (Windows) would be tricky to deal with. The
challenge is that Frida’s own code might clobber the current thread’s error
state between your NativeFunction call and when you try to read out the
error state.
Enter SystemFunction. It is exactly like NativeFunction, except that the
call returns an object wrapping the returned value and the error state right
afterwards. Here’s an example:
constopen=newSystemFunction(Module.findExportByName(null,'open'),'int',['pointer','int']);constO_RDONLY=0;constpath=Memory.allocUtf8String('/inexistent');constresult=open(path,O_RDONLY);console.log(JSON.stringify(result,null,2));/* * Which on Darwin typically results in the following output: * * { * "value": -1, * "errno": 2 * } * * Where 2 is ENOENT. */
This release also lets you read and modify this system error value from your
NativeCallback passed to Interceptor.replace(), which might come handy if
you are replacing system APIs. Note that you could already do this with
Interceptor.attach(), but that’s not an option in cases where you don’t want
the original function to get called.
Another big change worth mentioning is that our V8 runtime has been heavily
refactored. The code is now easier to understand and it is way less work to add
new features. Not just that, but our argument parsing is also handled by a
single code-path. This means that all of our APIs are much more resilient to bad
or missing arguments, so you get a JavaScript exception instead of having some
APIs do fewer checks and happily crash the target process in case you forgot an
argument.
Anyway, those are the highlights. Here’s a full summary of the changes:
8.1.0:
core: add Socket.listen() and Socket.connect()
core: add setImmediate() and clearImmediate()
core: improve set{Timeout,Interval}() to support passing arguments
core: fix performance-related bug in Interceptor’s dirty state logic
8.1.1:
core: add Script.nextTick()
8.1.2:
core: teach Socket.listen() and Socket.connect() about UNIX sockets
core: fix handling of this.errno / this.lastError replacement functions
core: add SystemFunction API to get errno / lastError on return
core: fix crash on close() during I/O with the Stream APIs
core: fix and consolidate argument handling in the V8 runtime
8.1.3:
core: temporarily disable Mapper on macOS in order to confirm whether this was
the root cause of reported stability issues
core: add .call() and .apply() to NativeFunction
objc: fix parsing of opaque struct types
8.1.4:
core: fix crash in the V8 runtime caused by invalid use of v8::Eternal
frida-repl: add batch mode support through -e and -q
8.1.5:
node: generate prebuilds for 6.0 (LTS) and 7.0 only
8.1.6:
node: generate prebuilds for 4.0 and 5.0 in addition to 6.0 and 7.0
8.1.7:
objc: fix infinite recursion when proxying some proxies
objc: add support for proxying non-NSObject instances
python: fix removal of signal callbacks that are member functions
8.1.8:
core: implement hooking of single-instruction ARM functions
core: plug leak in the handling of unhookable functions on some architectures
core: fix race condition in the handling of setTimeout(0) and
setImmediate() in the Duktape runtime
core: fix crash when processing tick callbacks in the Duktape runtime
core: fix lifetime issue in the Duktape runtime
core: fix the reported module sizes on Linux
core: fix crash when launching apps on newer versions of Android
core: fix handling of attempts to launch Android apps not installed
core: improve compatibility with different versions and flavors of Android by
detecting Dalvik and ART field offsets dynamically
core: fix unload issue on newer versions of Android, which resulted in only
the first attach() succeeding and subsequent attempts all timing out
core: move ObjC and Java into their own modules published to npm, and use
frida-compile to keep baking them into Frida’s built-in JS runtime
java: improve ART compatibility by detecting ArtMethod field offsets
dynamically
node: update dependencies
node: fix unhandled Promise rejection issues
8.1.9:
core: fix use-after-free caused by race condition on script unload
8.1.10:
core: make ApiResolver and DebugSymbol APIs preemptible to avoid deadlocks
8.1.11:
core: use a Mach exception handler on macOS and iOS, allowing us to reliably
catch exceptions in apps that already have a Mach exception handler of
their own
core: fix leak in InvocationContext copy-on-write logic in the Duktape
runtime, used when storing data on this across onEnter and onLeave
8.1.12:
core: fix Interceptor argument replacement issue in the V8 runtime,
resulting in the argument only being replaced the first time
Some big changes this time. We now use our Duktape-based JavaScript runtime
by default on all platforms, iOS app launching no longer piggybacks on Cydia
Substrate, and we are bringing some massive performance improvements. That, and
some bugfixes.
Let’s talk about Duktape first. Frida’s first JS runtime was based on V8,
and I’m really happy about that choice. It is however quite obvious that there
are use-cases where it is a bad fit.
Some systems, e.g. iOS, don’t allow RWX memory1,
and V8 won’t run without that. Another example is resource-constrained embedded
systems where there just isn’t enough memory. And, as reported by users from
time to time, some processes decide to configure their threads to have tiny
stacks. V8 is however quite stack-hungry, so if you hook a function called by
any of those threads, it won’t necessarily be able to enter V8, and your hooks
appear to be ignored2.
Another aspect is that V8 is way more expensive than Duktape for the native ⇔
JS transitions, so if your Frida agent is all about API hooks, and your hooks
are really small, you might actually be better off with Duktape. Garbage
collection is also more predictable with Duktape, which is good for hooking
time-sensitive code.
That said, if your agent is heavy on JavaScript, V8 will be way faster. It also
comes with native ES6 support, although this isn’t too big a deal since
non-trivial agents should be using frida-compile, which compiles your code
to ES5.
So the V8 runtime is not going away, and it will remain a first-class citizen.
The only thing that’s changing is that we pick Duktape by default, so that you
are guaranteed to get the same runtime on all platforms, with a high probability
that it’s going to work.
However, if your use-case is JS-heavy, all you have to do is call
Session#enable_jit() before the first script is created, and V8 will be used.
For our CLI tools you may pass –enable-jit to get the same effect.
That was Duktape. What’s the story about app launching and Substrate, then?
Well, up until now our iOS app launching was piggybacking on Substrate. This was
a pragmatic solution in order to avoid going into interoperability scenarios
where Frida and Substrate would both hook posix_spawn() in launchd and
xpcproxy, and step on each other.
It was however on my long-term TODO to fix this, as it added a lot of complexity
in other areas. E.g. an out-of-band callback mechanism so our Substrate plugin
could talk back to us at load time, having to manage temporary files, etc.
In addition to that, it meant we were depending on a closed source third-party
component, even though it was a soft-dependency only needed for iOS app
launching. But still, it was the only part of Frida that indirectly required
permanent modifications to the running system, and we really want to avoid
that.
Let’s have a look at how the new app launching works. Imagine that you ran
this on your host machine that’s got a jailbroken iOS device connected to it
over USB:
$ frida-trace -U -f com.atebits.Tweetie2 -i open
We’re telling it to launch Twitter’s iOS app and trace functions named open.
As a side-note, if you’re curious about the details, frida-trace is written in
Python and is less than 900 lines of code, so it might be a good way to
learn more about building your own tools on top of Frida. Or perhaps you’d
like to improve frida-trace? Even better!
The first part that it does is that it gets hold of the first USB device and
launches the Twitter app there. This boils down to:
Our launchd.js agent then intercept launchd’s __posix_spawn() and adds
POSIX_SPAWN_START_SUSPENDED, and signals back the identifier and PID.
This is the /usr/libexec/xpcproxy helper that will perform an exec()-style
transition to become the app.
We then inject our xpcproxy.js agent into this so it can hook
__posix_spawn() and add POSIX_SPAWN_START_SUSPENDED just like our launchd
agent did. This one will however also have POSIX_SPAWN_SETEXEC, so that
means it will replace itself with the app to be launched.
We resume() the xpcproxy process and wait for the exec to happen and the
process to be suspended.
At this point we let the device.spawn() return with the PID of the app that
was just launched. The app’s process has been created, and the main thread is
suspended at dyld’s entrypoint. frida-trace will then want to attach to it
so it can load its agent that hooks open. So it goes ahead and does something
similar to this:
session=device.attach(pid)script=session.create_script("""Interceptor.attach(Module.findExportByName(null, 'open'), { onEnter: function () { console.log('open()'); }});""")script.load()
Now that it has applied the instrumentation, it will ask Frida to resume
the process so the main thread can call main() and have some fun:
device.resume(pid)
Note that I did skip over a few details here, as the attach() operation is
actually a bit more complicated due to how uninitialized the process is, but
you can read more about that here.
Finally, let’s talk about footprint and performance. First, let’s examine how
much disk space is required when Frida is installed on an iOS device and is
in a fully operational state:
That’s the 64-bit version, which is only 1.87 MB xz-compressed. The 32-bit
version is obviously even smaller. Quite a few optimizations at play here:
We used to write the frida-helper binary out to a temporary file and spawn it.
The meat of the frida-helper program is now statically linked into
frida-server, and its entitlements have been boosted along with it. This
binary is only necessary when Frida is used as a plugin in an unknown process,
i.e. where we cannot make any guarantees about entitlements and code-signing.
In the frida-server case, however, it is able to guarantee that all such
constraints are met.
The library that we inject into processes to be instrumented,
frida-agent.dylib, is no longer written out to a temporary file. We use
our own out-of-process dynamic linker to map it from frida-server’s memory
and directly into the address space of the target process. These mappings
are made copy-on-write, so that means it is as memory-efficient as the old
dlopen() approach was.
V8 was disabled for the iOS binaries as it’s only really usable on old
jailbreaks where the kernel is patched to allow RWX pages.
(If V8 is important to your use-case, you can build it like this:
make server-ios FRIDA_DIET=no)
The iOS package has been split into two, “Frida” for 64-bit devices, and
“Frida for 32-bit devices” for old devices.
Getting rid of the Substrate dependency for iOS app launching also meant
we got rid of FridaLoader.dylib. This is however a very minor improvement.
Alright, so that’s disk footprint. How about memory usage?
Nice. How about performance? Let’s have a look:
Note that these measurements include the time spent communicating from the
macOS host to the iOS device over USB.
Enjoy!
1 Except if the process has an entitlement, although that’s
limited to just one region. ↩
2: It is technically possible to work around this by
having a per-thread side-stack that we switch to before calling into V8. We
did actually have this partially implemented in the past. Might be something
we should revive in the longer term. ↩
This time we’re kicking it up a notch. We’re bringing you stability
improvements and state of the art JavaScript support.
Let’s talk about the stability improvements first. We fixed a heap
corruption affecting all Linux users. This one was particularly hard to
track down, but rr saved the day. The other issue was a crash on
unload in the Duktape runtime, affecting all OSes.
Dependencies were also upgraded, so as of Frida 10.0.0 you can now enjoy
V8 6.0.124, released just days ago. We also upgraded Duktape to the
latest 2.1.x. The Duktape upgrade resulted in slight changes to the
bytecode semantics, which meant we had to break our API slightly.
Instead of specifying a script’s name at load time, it is now specified
when compiling it to bytecode, as this metadata is now included in the
bytecode. This makes a lot more sense, so it was a welcome change.
Beside V8 and Duktape we’re also using the latest GLib, Vala compiler,
etc. These upgrades also included JSON-GLib, which recently ditched
autotools in favor of Meson. This is excellent news, as we’re also
planning on moving to Meson down the road, so we’ve now done the
necessary groundwork for making this happen.
So that’s about it. This upgrade should not require any changes to
existing code – unless of course you are among the few using the
bytecode API.
Frida provides quite a few building blocks that make it easy to do portable
instrumentation across many OSes and architectures. One area that’s been lacking
has been in non-portable use-cases. While we did provide some primitives like
Memory.alloc(Process.pageSize) and Memory.patchCode(), making it possible to
allocate and modify in-memory code, there wasn’t anything to help you actually
generate code. Or copy code from one memory location to another.
Considering that Frida needs to generate and transform quite a bit of machine
code for its own needs, e.g. to implement Interceptor and Stalker, it should
come as no surprise that we already have C APIs to do these things across six
different instruction set flavors. Initially these APIs were so barebones that I
didn’t see much value in exposing them to JavaScript, but after many years of
interesting internal use-cases they’ve evolved to the point where the essential
bits are now covered pretty well.
So with 10.4 we are finally exposing all of these APIs to JavaScript. It’s also
worth mentioning that these new bindings are auto-generated, so future additions
will be effortless.
Let’s take a look at an example on x86:
vargetLivesLeft=Module.findExportByName('game-engine.so','get_lives_left');varmaxPatchSize=64;// Do not write out of bounds, may be// a temporary buffer!Memory.patchCode(getLivesLeft,maxPatchSize,function(code){varcw=newX86Writer(code,{pc:getLivesLeft});cw.putMovRegU32('eax',9999);cw.putRet();cw.flush();});
Which means we replaced the beginning of our target function with simply:
moveax,9999ret
I.e. assuming the return type is int, we just replaced the function body with
return 9999;.
As a side-note you could also use Memory.protect() to change the page
protection and then go ahead and write code all over the place, but
Memory.patchCode() is very handy because it also
ensures CPU caches are flushed;
takes care of code-signing corner-cases on iOS.
So that was a simple example. Let’s try something a bit crazier:
Though that’s quite a few hoops just to multiply 42 by 7, the idea is to
illustrate how calling functions, even back into JavaScript, and jumping to
labels, is actually quite easy.
Finally, let’s look at how to copy instructions from one memory location to
another. Doing this correctly is typically a lot more complicated than a
straight memcpy(), as some instructions are position-dependent and need to
be adjusted based on their new locations in memory. Let’s look at how we can
solve this with Frida’s new relocator APIs:
We just made our own replica of puts() in just a few lines of code. Neat!
Note that you can also insert your own instructions, and use skipOne() to
selectively skip instructions in case you want to do custom instrumentation.
(This is how Stalker works.)
Anyway, that’s the gist of it. You can find the brand new API references at:
Also note that Process.arch is convenient for determining which
writer/relocator to use. On that note you may wonder why there’s just a single
implementation for 32- and 64-bit x86. The reason is that these instruction sets
are so close that it made sense to have a unified implementation. This also
makes it easier to write somewhat portable code, as some meta register-names are
available. E.g. xax resolves to eax vs rax depending on the kind of
process you are in.
The midnight oil has been burning and countless cups of coffee have been
consumed here at NowSecure, and boy do we have news for you this time.
Continuing in the spirit of last release’ low-level bag of goodies, we’ll be
moving one level up the stack this time. We are going to introduce a brand new
way to use new CodeWriter APIs, enabling you to weave in your own instructions
into the machine code executed by any thread of your choosing. We’re talking
lazy dynamic recompilation on a per-thread basis, with precise control of the
compilation process.
But first a little background. Most people using Frida are probably using the
Interceptor API to perform inline hooking, and/or doing method swizzling or
replacement through the ObjC and Java APIs. The idea is typically to
modify some interesting API that you expect to be called, and be able to divert
execution to your own code in order to observe, augment, or fully replace
application behavior.
One drawback to such approaches is that code or data is modified, and such
changes can be trivially detected. This is fine though, as being invisible to
the hosting process’ own code is always going to be a cat and mouse game when
doing in-process instrumentation.
These techniques are however quite limited when trying to answer the question
of “behind this private API, which other APIs actually get called for a given
input?”. Or, when doing reversing and fuzzing, you might want to know where
execution diverges between two known inputs to a given function. Another example
is measuring code coverage. You could use Interceptor’s support for
instruction-level probes, first using a static analysis tool to find all the
basic blocks and then using Frida to put single-shot probes all over the place.
Enter Stalker. It’s not a new API, but it’s been fairly limited in what it
allowed you to do. Think of it as a per-thread code-tracer, where the thread’s
original machine code is dynamically recompiled to new memory locations in
order to weave in instrumentation between the original instructions.
It does this recompilation lazily, one basic-block at a time. Considering that
a lot of self-modifying code exists, it is careful about caching compiled blocks
in case the original code changes after the fact.
Stalker also goes to great lengths to recompile the code such that
side-effects are identical. E.g. if the original instruction is a CALL it will
make sure that the address of the original next instruction is what’s pushed on
the stack, and not the address of the next recompiled instruction.
Anyway, Stalker has historically been like a pet project inside of a pet
project. A lot of fun, but other parts of Frida received most of my attention
over the years. There have been some awesome exceptions though. Me and
@karltk did some fun pair-programming sessions many years ago when we
sat down and decided to get Stalker working well on hostile code. At some
later point I put together CryptoShark in order get people excited about its
potential. Some time went by and suddenly Stalker received a critical bug-fix
contributed by Eloi Vanderbeken. Early this year, Antonio Ken Iannillo
jumped on board and ported it to arm64. Then, very recently, Erik Smit
showed up and fixed a critical bug where we would produce invalid code for
REP-prefixed JCC instructions. Yay!
Stalker’s API has so far been really limited. You can tell it to follow a
thread, including the thread you’re in, which is useful in combination with
inline hooking, i.e. Interceptor. The only two things you could do was:
Tell it which events you’re interested in, e.g. call: true, which will
produce one event per CALL instruction. This means Stalker will add some
logging code before each such instruction, and that would log where the
CALL happened, its target, and its stack depth. The other event types are
very similar.
Add your own call probes for specific targets, giving you a synchronous
callback into JavaScript when a CALL is made to a specific target.
I’m super-excited to announce that we’ve just introduced a third thing you
can do with this API, and this one is a game changer. You can now customize
the recompilation process, and it’s really easy:
The transform callback gets called synchronously whenever a new basic block
is about to be compiled. It gives you an iterator that you then use to drive
the recompilation-process forward, one instruction at a time. The returned
Instruction tells you what you need to know about the instruction that’s
about to be recompiled. You then call keep() to allow Stalker to recompile
it as it normally would. This means you can omit this call if you want to skip
some instructions, e.g. because you’ve replaced them with your own code. The
iterator also allows you to insert your own instructions, as it exposes the full
CodeWriter API of the current architecture, e.g. X86Writer.
The example above determines where the application’s own code is in memory, and
adds a few extra instructions before every RET instruction in any code belonging
to the application itself. This code checks if eax contains a value between 60
and 90, and if it does, calls out to JavaScript to let it implement arbitrarily
complex logic. This callback can read and modify registers as it pleases.
What’s nice about this approach is that you can insert code into hot code-paths
and selectively call into JavaScript, making it easy to do really fast checks in
machine code but offload more complex tasks to a higher level language. You can
also Memory.alloc() and have the generated code write directly there, without
entering into JavaScript at all.
So that’s the big new thing in 10.5. Special thanks to @asabil who helped
shape this new API.
In closing, the only other big change is that the Instruction API now exposes
a lot more details of the underlying Capstone instruction. Stalker also
uses a lot less memory on both x86 and arm64, and is also more reliable. Lastly,
Process.setExceptionHandler() is now a documented API, along with our
SQLite API.
This component is really useful when dealing with jailed iOS and Android
devices, but is now also able to cover a lot of other scenarios.
Its environment variables are now gone, and have been replaced by an optional
configuration file. Because some apps may look for loaded libraries with “Frida”
in their name as part of their “anti-debug” defenses, we now allow you to rename
Gadget’s binary however you like. Along with this it now also supports three
different interaction types.
It can listen on a TCP port, like before, and it can also load scripts from the
filesystem and run fully autonomously. The latter part used to be really limited
but is now really flexible, as you can even tell it to load scripts from a
directory, where each script may have filters. This is pretty useful for
system-wide tampering, and should allow for even more interesting use-cases.
So without further ado, I would urge you all to check out the brand new docs
available here.
iOS users rejoice: Frida is now compatible with the latest Electra
jailbreak on iOS 11! At the time of this writing that means 1.0.4, but
now that Electra seems to have stabilized it should be safe to assume
that we’ll also be compatible with future updates. Just make sure you’re
not running anything older than 1.0.4.
In other news, our Android support has improved significantly over the
last releases, so if you’re even slightly behind: go grab the latest
10.7.x right away.
Get ready for a major upgrade. This time we have solved our three longest
standing limitations – all in one release.
Limitation #1: fork()
As a quick refresher, this old-school UNIX API clones the entire process and
returns the child’s process ID to the parent, and zero to the child. The child
gets its own copy of the parent’s address space, usually at a very small cost
due to copy-on-write.
Dealing with this one gets tricky once multiple threads are involved. Only the
thread calling fork() survives in the child process, so if any of the other
threads happen to be holding locks, those locks will still be held in the child,
and nobody is going to release them.
Essentially this means that any application doing both forking and
multi-threading will have to be really carefully designed. Even though most
applications that fork are single-threaded, Frida effectively makes them
multi-threaded by injecting its agent into them. Another aspect is
file-descriptors, which are shared, so those also have to be carefully managed.
I’m super-excited to announce that we are finally able to detect that a fork()
is about to happen, temporarily stop our threads, pause our communications
channel, and start things back up afterwards. You are then able to apply the
desired instrumentation to the child, if any, before letting it continue
running.
Limitation #2: execve(), posix_spawn(), CreateProcess(), and friends
Or in plain English: programs that start other programs, either by replacing
themselves entirely, e.g. execve(), or by spawning a child process, e.g.
posix_spawn() without POSIX_SPAWN_SETEXEC.
Just like after a fork() happened you are now able to apply instrumentation
and control when the child process starts running its first instructions.
Limitation #3: dealing with sudden process termination
An aspect that’s caused a lot of confusion in the past is how one might hand off
some data to Frida’s send() API, and if the process is about to terminate,
said data might not actually make it to the other side.
Up until now the prescribed solution was always to hook exit(), abort() etc.
so you can do a send() plus recv().wait() ping-pong to flush any data still
in transit. In retrospect this wasn’t such a great idea, as it makes it hard to
do this correctly across multiple platforms. We now have a way better solution.
So with that, let’s talk about the new APIs and features.
Child gating
This is how we address the first two. The Session object, the one providing
create_script(), now also has enable_child_gating() and
disable_child_gating(). By default Frida will behave just like before, and
you will have to opt-in to this new behavior by calling enable_child_gating().
From that point on, any child process is going to end up suspended, and you will
be responsible for calling resume() with its PID. The Device object now also
provides a signal named delivered which you should attach a callback to in
order to be notified of any new children that appear. That is the point where
you should be applying the desired instrumentation, if any, before calling
resume(). The Device object also has a new method named
enumerate_pending_children() which can be used to get a full list of pending
children. Processes will remain suspended and part of that list until they’re
resumed by you, or eventually killed.
So that’s the theory. Let’s have a look at a practical example, using
Frida’s Python bindings:
As for the third limitation, namely dealing with sudden process termination,
Frida will now intercept the most common process termination APIs and take
care of flushing any pending data for you.
However, for advanced agents that optimize throughput by buffering data and only
doing send() periodically, there is now a way to run your own code when the
process terminates, or the script is unloaded. All you need to do is to define
an RPC export named dispose. E.g.:
Building on the brand new fork()-handling in Frida, there is also a fully
reworked Android app launching implementation. The frida-loader-{32,64}.so
helper agents are now gone, and our behind-the-scenes Zygote instrumentation
is now leveraging the brand new child gating to do all of the heavy lifting.
This means you can also instrument Zygote for your own needs. Just remember to
enable_child_gating() and resume() any children that you don’t care about.
It’s time to overhaul the spawn() API and fix some rough edges in the spawn-
and child-gating APIs.
spawn()
Say you’re using Frida’s Python bindings, you’d currently do:
pid=device.spawn(["/bin/cat","/etc/passwd"])
Or to spawn an iOS app:
pid=device.spawn(["com.apple.mobilesafari"])
Well, that’s pretty much all you could do with that API really… except one
thing that wasn’t exposed by the Python and Node.js bindings. We’ll get to
that in a bit. Before we go there, let’s take a look at the underlying API
in frida-core, which these bindings expose to different languages:
That’s Vala code by the way, which is the language that frida-core is
written in. It’s a C#-like language that compiles to C, and it’s pretty awesome.
But I digress. The first method, spawn() is asynchronous, allowing the calling
thread to do other things while the call is in progress, whereas spawn_sync()
blocks until the operation completes.
Those two methods compile down to the following three C functions:
The first two constitute spawn(), where you’d call the first giving it a
callback, and once that callback gets called you’d call the second one,
spawn_finish(), giving it the GAsyncResult your callback was given.
The return value is the PID, or, in case it failed, the error out-argument
explains what went wrong. This is the GIO async pattern in case you’re
curious.
As for the third, spawn_sync(), this is what Frida’s Python bindings use.
Our Node.js bindings actually use the first two, as those bindings are fully
asynchronous. Someday it would be nice to also migrate our Python bindings to
be fully async, by integrating with the async/await support introduced in
Python 3.5.
Anyway, returning to the examples above, I mentioned there was something not
exposed. If you look closely at the frida-core API you’ll notice that there’s
the envp string-array. Peeking under the hood of the bindings, you’d realize
we did indeed not expose this, and we actually did this:
So that means we passed along whatever the Python process’ environment happened
to be. That’s definitely not good if the actual spawning happened on another
system entirely, like on a connected iOS or Android device. What made this
slightly less problematic was the fact that envp was ignored when spawning
iOS and Android apps, and only used when spawning regular programs.
Another issue with this old API is that the declaration, string[] envp, means
it isn’t nullable, which it would have been if the declaration had been:
string[]? envp. That means there is no way to distinguish between wanting to
spawn without any environment, which intuitively would mean “use defaults”, and
an empty environment.
As I was about to fix this aspect of the API, I realized that it was time to
also fix a few other long-standing issues with it, like being able to:
provide just a few extra environment variables on top of the defaults
set the working directory
customize stdio redirection
pass along platform-specific options
Up until this point we have always redirected stdio to our own pipes, and
streamed any output through the output signal on Device. There was also
Device.input() for writing to stdin. Those APIs are still the same, the
only difference is that we no longer do such redirection by default. Most of
you were probably not too bothered with this, though, as we didn’t implement
such redirection for iOS and Android apps. Starting with this release we do
however finally implement it for iOS apps.
By now you’re probably wondering what the new API looks like. Let’s have a look:
So going back to the Python examples at the beginning, those still work without
any changes. But, instead of:
device.spawn(["com.apple.mobilesafari"])
You can now also do:
device.spawn("com.apple.mobilesafari")
As the first argument is the program to spawn. You can still pass an argv
here and that will be used to set the argv option, meaning that argv[0] will
be used for the program argument. You can also do this:
And if you’d like to replace the entire environment instead of using defaults:
device.spawn("/bin/ls",envp={"CLICOLOR":"1"})
Though in most cases you probably only want to add/override a few environment
variables, which is now also possible:
device.spawn("/bin/ls",env={"CLICOLOR":"1"})
You might also want to use a different working directory:
device.spawn("/bin/ls",cwd="/etc")
Or perhaps you’d like to redirect stdio:
device.spawn("/bin/ls",stdio="pipe")
The stdio default value is inherit, as mentioned earlier.
We have now covered all of the SpawnOptions, except the last of them: aux.
This is a dictionary for platform-specific options. Setting such options is
pretty simple with the Python bindings: any keyword-argument not recognized
will end up in that dictionary.
For example, to launch Safari and tell it to open a specific URL:
So as you can see, the second argument is an object with options, and those not
recognized end up in the aux dictionary.
The rest
Let’s just summarize the remaining changes, starting with the Device class:
enumerate_pending_spawns() is now enumerate_pending_spawn() to be
grammatically correct.
The spawned signal has been renamed to spawn-added, and there is now also
spawn-removed.
The delivered signal has been renamed to child-added, and there is now
also child-removed.
The final change is that the Child class’ path, argv, and envp
properties are now all nullable. This is to be able to discern e.g. “no envp
provided” from “empty envp provided”.
So that’s about it. If you didn’t read about the Frida 10.8 release that
happened last week, make sure you go read about it here.
As some of you may have picked up on, there may be a book on Frida in the
works. In my day to day at NowSecure I spend a good chunk of time as a user
of Frida’s APIs, and consequently I’m often reminded of past design decisions
that I’ve since come to regret. Even though I did address most of them over the
years, some were so painful to address that I kept them on the backburner. Fast
forward to today, and the thought of publishing a book with all these in mind
got me thinking that it was time to bite the bullet.
This is why I’m stoked to announce Frida 12. We have finally reached a point
in Frida’s evolution where our foundations can be considered sufficiently stable
for a book to be written.
Let’s have a look at the changes.
CLI tools
One thing that caused a bit of confusion in the past was the fact that our
Python bindings also came with some CLI tools. Frida is a toolkit for building
tools, and even though we provide a few sample tools it should be up to you if
you want to have them installed.
Up until now this meant anyone building a tool using our Python bindings would
end up depending on colorama, prompt-toolkit, and pygments, because our
CLI tools happen to depend on those.
Well, that changes now. If you do:
$ pip install frida
You will now only get our Python bindings. Nothing more. And this package has
zero dependencies.
The CLI tools might still be useful to you, though, so to install those do:
$ pip install frida-tools
Convenience APIs in bindings
Something that seemed like a great idea at the time was having our language
bindings provide some convenience APIs on the Session object. The thinking
was that simple use-cases that only need to enumerate loaded modules and perhaps
a few memory ranges, to then read or write memory, wouldn’t have to load their
own agent. So both our Python and our Node.js bindings did this behind the
scenes for you.
Back then it was somewhat tedious to communicate with an agent as the rpc
API didn’t exist, but even so, it was a bad design decision. The JS APIs
are numerous and not all can be exposed without introducing new layers of
complexity. Another aspect is that every language binding would have to
duplicate such convenience APIs, or we would have to add core APIs that
bindings could expose. Both are terrible options, and cause confusion by
blurring the lines, ultimately confusing people new to Frida. Granted, it did
make things easier for some very simple use-cases, like memory dumping tools,
but for everybody else it just added bloat and confusion.
These APIs are now finally gone from our Python and Node.js bindings. The other
bindings are unaffected as they didn’t implement any such convenience APIs.
Node.js bindings
It’s been a few years since our Node.js bindings were written, and since then
Node.js has evolved a lot. It now supports ES6 classes, async / await, arrow
functions, Proxy objects, etc.
Just the Proxy support alone means we can simplify rpc use-cases like:
Some of you may also prefer writing your application in TypeScript, which
is an awesome productivity boost compared to old-fashioned JavaScript. Not only
do you get type checking, but if you’re using an editor like VS Code you
also get type-aware refactoring and amazing code completion.
However, for type checking and editor features to really shine, it is crucial to
have type definitions for your project’s dependencies. This is rarely ever an
issue these days, except for those of you using Frida’s Node.js bindings.
Up until now we didn’t provide any type definitions. This has finally been
resolved. Rather than augmenting our bindings with type definitions, I decided
to rewrite them in TypeScript instead. This means we also take advantage of
modern language features like ES6 classes and async / await.
We could have stopped there, but those of you using our Node.js bindings from
TypeScript would still find this a bit frustrating:
Here, the compiler knows nothing about which events exist on the Script
object, and what the callback’s signature is supposed to be for this particular
event. We’ve finally fixed this. The API now looks like this:
script.message.connect((message,data)=>{});
Voilà. Your editor can even tell you which events are supported and give you
proper type checking for the code in your callback. Sweet!
Interceptor
Something that’s caused some confusion in the past is the observation that
accessing this.context.pc from onEnter or onLeave would give you the
return address, and not the address of the instruction that you put the hook
on. This has finally been fixed. Also, this.context.sp now points at the
return address on x86, instead of the first argument. The same goes for
Stalker when using call probes.
As part of this refactoring breaking our backtracer implementations, I also
improved our default backtracer on Windows.
Tether?
You might have wondered why frida.get_usb_device() would give you a Device
whose type was ‘tether’. It is now finally ‘usb’ as you’d expect. So our
language bindings are finally consistent with our core API.
Changes in 12.0.1
core: fix argument access on 32-bit x86
core: update Stalker to the new CpuContext semantics
python: publish the correct README to PyPI
python: fix the Windows build system
Changes in 12.0.2
core: upgrade to Capstone’s next branch
core: fix the DbgHelp backtracer on Windows and update to latest DbgHelp
python: fix long description
java: fix hooking of java.lang.Class.getMethod()
Changes in 12.0.3
core: fix the iOS build system broken by Capstone upgrade
Massive changes under the hood this time. All of our dependencies have been
upgraded to the latest and greatest. Let’s have a look at the highlights.
V8 7.0
Frida’s V8 dependency was previously at 6.2.2, and has now been upgraded to
7.0.242. The move to such a new version means that the V8 debugger API is gone
and has been replaced with the new Inspector API, which the latest Node.js is
also using. One thing that’s pretty awesome about it is that it’s natively
supported by Google Chrome’s Inspector.
To start using it, just tell Frida to use V8, by calling session.enable_jit(),
and then session.enable_debugger().
Then in Google Chrome, right-click and choose “Inspect”, and click on the green
Node.js icon in the upper left corner of the Inspector. That’s it, you are now
debugging your Frida scripts. That means a nifty console with auto-completion,
pause/continue, stepping, breakpoints, profiling, and heap snapshots. What makes
it really convenient is that the server listens on your host machine, so you can
call enable_debugger() on a session representing a process on your
USB-tethered Android device, and it all works the same.
Here’s what it looks like:
Note however that V8 is currently not included in our prebuilt iOS binaries,
but it should be possible to get it running again on iOS now that it’s able to
run without RWX pages. We do however plan on bridging Duktape’s binary debugger
protocol behind the scenes so debugging “just works” with Duktape also, though
probably with a slightly reduced feature-set.
Process.id
It is often quite useful to know the PID of the process that your agent is
executing inside, especially in preloaded mode. So instead of requiring you
to use NativeFunction to call e.g. getpid(), this is now a lot simpler as
you can use the brand new Process.id property.
Anything else?
No other features, but some really nice bug-fixes. Thanks to mrmacete we are
now able to attach to com.apple.WebKit.* processes on iOS 11.x. And thanks to
viniciusmarangoni this release also packs a couple of goodies for frida-java,
fixing one Android 8.0 regression and adding the ability to properly instrument
system_server.
Let’s talk about iOS kernel introspection. It’s been a while since Frida got
basic support for introspection of the iOS kernel, but in the last months we
kept improving on that. Today’s release includes significant additions to our
Kernel API to work with recent 64-bit kernels.
Kernel base
You can get the kernel’s base address by reading the Kernel.base property.
Having the base allows for example to calculate the slid virtual address of
any symbol you already know from static analysis of the kernel cache.
Kernel memory search
The memory search API has been ported to the Kernel, so you can use
Kernel.scan() (or Kernel.scanSync()) in the same way you use Memory.scan()
(or Memory.scanSync()) in userland. This is a powerful primitive which,
combined with the recent bit mask feature, allows you to create your own symbol
finding code by searching for arm64 patterns.
KEXTs and memory ranges
With Kernel.enumerateModules() (or Kernel.enumerateModulesSync()) it’s now
possible to get the names and the offsets of all the KEXTs.
Kernel.enumerateModuleRanges() (or Kernel.enumerateModuleRangesSync()) is
the way to enumerate all the memory ranges defined by the Mach-O sections
belonging to a module (by name) filtering by protection. The result is similar
to what you can get in userland when calling Module.enumerateRanges() but it
also includes the section names.
Final notes
All Kernel APIs don’t rely on NativePointer because its size depends on the
user-space which doesn’t necessarily match the kernel space one. Instead all
addresses are represented as UInt64 objects.
All of this, plus the existing JavaScript interfaces for reading, writing, and
allocating kernel memory can provide a powerful starting point to build your own
kernel analysis or vulnerability research tools.
Note that this is to be considered experimental and messing with the kernel in
random ways can wildly damage your devices, so be careful, and happy hacking!
Troubleshooting
Problem: Kernel.available is false
The Kernel API is available if both of these conditions are met:
Your device is jailbroken
Frida is able to get a send right to the kernel task, either by traditional
task_for_pid (0) or by accessing the host special port 4 (which is what
modern jailbreaks are doing)
The recommended way to accomplish the latter is to attach to the system session,
i.e. PID 0, and load your scripts there.
Problem: can’t do much with my 32-bit kernel
Yes, that could improve in the future but 32-bit iOS is quite far down on the
list of priorities nowadays, but you’re very welcome to contribute and send PRs.
Problem: I was trying to do X and the kernel panicked
Don’t worry that’s normal. You can go to the
/private/var/mobile/Library/Logs/CrashReporter directory on your device, or
navigate to Settings -> Privacy -> Analytics -> Analytics Data, find your panic
log and figure out what you (or Frida) did wrong. Remember: the Kernel is always
right!
Problem: I unrecoverably damaged my device using Frida Kernel APIs
Sorry to hear, if the damage is at the hardware level and you can dedicate
enough time and money you can probably repair it yourself by following tutorials
at https://ifixit.com.
This is one packed release. So many things to talk about.
V8
The main story this time is V8. But first, a little background.
Frida uses the Duktape JavaScript runtime by default to provide
scriptable access to its instrumentation core, Gum. We were
originally only using V8, but as V8 used to depend on operating support
for RWX pages – i.e. executable pages of memory that are also
writable – we ended up adding a secondary JS runtime based on Duktape.
Due to this OS constraint with V8, and the fact that Duktape lacks
support for the latest JS syntax and features, I decided to make Duktape
our default runtime so that scripts written for one platform wouldn’t
need any syntactic changes to work on another. Duktape was basically the
lowest common denominator.
Fast forward to today, and V8 no longer depends on OS support for RWX
pages. It’s actually moving towards flipping between RW- and R-X by
default. By doing so it means it can be used on modern iOS jailbreaks,
and also on jailed iOS if the process is marked as being debugged,
so it is able to run unsigned code. But there’s more; V8 can now even run
JIT-less, which means Frida can use V8 on every single platform, and
users no longer have to frida-compile their agents to use the latest
JavaScript syntax and features. This last point only applies to trivial
agents, though, as being able to split a non-trivial agent into multiple
source files is still desirable. Plus, frida-compile makes it easy to
use TypeScript, which is highly recommended for any non-trivial
Frida agent.
So with all of that in mind, it was clearly time to upgrade our V8 to
the latest and greatest. As of this release we are running 7.6.48,
and we also have a much deeper integration with V8 than ever before.
Both C++ memory allocations and page-level allocations are now managed
by Frida, so we are able to hide these memory ranges from APIs like
Process.enumerateRanges(), and also avoid poisoning the application’s
own heap with allocations belonging to Frida. These details may not
sound all that significant, but are actually crucial for implementing
memory-dumping tools on top of Frida. Not only that, however, we also
interfere less with the process that is being observed. That means
there’s a smaller risk of it exhibiting a different behavior than when
it runs without instrumentation.
Runtime Selection
You may remember the session.enable_jit() API. It has finally been
deprecated, as you can now specify the desired runtime during script
creation. E.g. using our Python bindings:
Another significant change in this release is that Stalker no longer
depends on RWX pages on arm64, thanks to John Coates’ awesome
contribution. This means Stalker is finally much more accessible on iOS.
For those of you using Stalker on 64-bit Windows and stalking 32-bit
processes, it finally handles the WOW64 transitions on newer versions of
Windows. This brain-twisting improvement was contributed by Florian Märkl.
Module.load()
There are times when you might want to load your own shared library,
perhaps containing hooks written in C/C++. On most platforms you could
achieve this by using NativeFunction to call dlopen() (POSIX) or
LoadLibrary() (Windows). It’s a very different story on newer versions
of Android, however, as their dlopen()-implementation looks at the
caller and makes decisions based on it. One such decision is whether
the app is trying to access a private system API, which would make it
hard for them to later remove or break that API. So as of Android 8
the implementation will return NULL when this is the case. This was
a challenge that Frida solved for its own injector’s needs, but users
wanting to load their own library were basically on their own.
As of Frida 12.5, there’s a brand new JavaScript API that takes care of
all the platform-specific quirks for you:
We fixed so many Android-specific bugs in this release. For example,
Module.getExportByName() on an app-bundled library no longer results
in the library being loaded a second time at a different base address.
This bug alone is reason enough to make sure you’ve got all your devices
upgraded and running the latest release.
iOS
The iOS Chimera jailbreak is also supported thanks to
Francesco Tamagni’s awesome contributions.
All the rest
There’s also lots of other improvements across platforms.
In chronological order:
Child gating also works on older versions of Windows.
Thanks Fernando Urbano!
Executables are smaller on UNIX OSes as they no longer export any
dynamic symbols.
Frida’s agent and gadget no longer crash on 32-bit Linux when loaded
at a high address, i.e. where MSB is set.
Only one of the two frida-helper-{32,64} binaries for Linux/Android
is needed, and none of them for builds without cross-arch support.
This means smaller footprint and improved performance.
Linux/ARM64 is finally supported, with binaries uploaded as part of
the release process.
We now provide a hint about Magisk Hide when our early instrumentation
fails to instrument Zygote on Android.
Changes in 12.5.1
Script runtime can be specified per script, deprecating enable_jit().
Changes in 12.5.2
Gadget no longer crashes at script load time when using V8 on Linux
and Android. Huge thanks to Leon Jacobs for reporting and helping
track this one down.
Changes in 12.5.3
Android linker integration supports a lot more devices.
Android Java integration no longer crashes with “Invalid instruction”
on some arm64 devices. Kudos to Jake Van Dyke for reporting and
helping track this one down.
LineageOS 15.1 is supported after adding a missing SELinux rule.
Changes in 12.5.4
Hooks are no longer able to interfere with our V8 page allocator
integration.
Android stability improved greatly after plugging a hole in our libc
shim. Big thanks to Giovanni Rocca for reporting and helping track
this one down!
Changes in 12.5.5
Apple USB devices are properly detected on Windows. Thanks @xiofee!
Please do make sure to check out the NowSecure Connect 2019
conference that’s happening June 2019 in Washington DC. It will feature
multiple Frida-related presentations, and I am super-excited about the
line-up.
After a flurry of fixes across all platforms over the last few weeks,
I figured it was time to do another minor bump to call attention to
this release.
One particular fix is worth mentioning specifically. There was a long-
standing bug in our Android Java integration, where exception delivery
would intermittently result in the process crashing with
GetOatQuickMethodHeader() typically in the stack-trace. Shout-out to
Jake Van Dyke and Giovanni Rocca for helping track this one
down. This bug has been around for as long as ART has been supported,
so this fix is worth celebrating. 🎉
Our V8 runtime is also a lot more stable, child-gating works better
than ever before, Android device compatibility is much improved, etc.
So bottom line is that this is the most stable version of Frida ever
released – and now is the time to make sure you’re running Frida 12.6.
There’s only one new feature this time, but it’s a big one. We’re going to
address the elephant in the room: performance.
While Frida’s instrumentation core, Gum, is written in C and can be used from C,
most use-cases are better off using its JavaScript bindings.
There are however situations where performance becomes an issue. Even when using
our V8-based runtime, which means your JavaScript will be profiled while it’s
running and optimized based on where the hotspots are… (Which by the way is
amazing – V8 is truly an impressive feat of engineering!)
…there is a small price to pay for entering and leaving the JavaScript VM.
On an iPhone 5S this might amount to something like six microseconds if you
use Interceptor.attach() and only specify onEnter, and leave it empty.
This may not sound like a lot, but if a function is called a million times,
it’s going to amount to 6 seconds of added overhead. And perhaps the hook only
needs to do something really simple, so most of the time is actually spent on
entering and leaving the VM.
There’s also the same kind of issue when needing to pass a callback to an API,
where the API walks through potentially millions of items and needs to call the
callback for each of them. The callback might just look at one byte and collect
the few of the items that match a certain criteria.
Naively one could go ahead and use a NativeCallback to implement that callback,
but it quickly becomes apparent that this just doesn’t scale.
Or, you might be writing a fuzzer and needing to call a NativeFunction in a
tight loop, and the cost of entering/leaving the VM plus libffi just adds up.
Short of writing the whole agent in C, one could go ahead and build a native
library, and load it using Module.load(). This works but means it has to be
compiled for every single architecture, deployed to the target, etc.
Another solution is to use the X86Writer/Arm64Writer/etc. APIs to generate
code at runtime. This is also painful as there’s quite a bit of work required
for each architecture to be supported. But up until now this was the only
portable option for use in modules such as frida-java-bridge.
But now we finally have something much better. Enter CModule:
It takes the string of C source code and compiles it to machine code, straight
to memory. This is implemented using TinyCC, which means that this feature
only adds ~100 kB of footprint to Frida.
As you can see, any global functions are automatically exported as NativePointer
properties named exactly like in the C source code.
And, it’s fast:
(Measured on an Intel i7 @ 3.1 GHz.)
We can also use this new feature in conjunction with APIs like Interceptor:
(Note that this and the following examples use modern JavaScript features like
template literals, so they either need to be run on our V8 runtime, or compiled
using frida-compile.)
Yay. Though this last particular example actually writes to stdout of the
target process, which is fine for debugging but probably not all that useful.
We can however fix that by calling back into JavaScript. Let’s see what that
might look like:
That is however just a toy example: doing it this way will actually defeat the
purpose of writing the hooks in C to improve performance. A real implementation
might instead append to a GLib.Array after acquiring a GLib.Mutex, and
periodically flush the buffered data by calling back into JS.
And just like JavaScript functions can be called from C, we can also share data
between the two realms:
For now we don’t have any docs on the built-in C APIs, but you can browse the
headers in frida-gum/bindings/gumjs/runtime/cmodule to get an overview.
Drop function names into an Internet search engine to look up docs for the
non-Frida APIs such as GLib’s.
The intention is to only expose a minimal subset of the standard C library,
GLib, JSON-GLib, and Gum APIs; in order to minimize bloat and maximize
performance. Things we include should either be impossible to achieve by calling
into JS, or prohibitively expensive to achieve that way.
Think of the JS side as the operating system where the functions you plug into
it are system calls; and only use CModule for hooking hot functions or
implementing high-performance glue code like callbacks passed to
performance-sensitive APIs.
Also bear in mind that the machine code generated by TinyCC is not as efficient
as that of Clang or GCC, so computationally expensive algorithms might actually
be faster to implement in JavaScript. (When using our V8-based runtime.) But for
hooks and glue code this difference isn’t significant, and you can always
generate machine code using e.g. Arm64Writer and plug into your CModule if you
need to optimize an inner loop.
One important caveat is that all data is read-only, so writable globals should
be declared extern, allocated using e.g. Memory.alloc(), and passed in as
symbols through the constructor’s second argument. (Like we did with calls in
the last example.)
You might also need to initialize things and clean them up when the CModule gets
destroyed – e.g. because the script got unloaded – and we provide a couple of
lifetime hooks for such purposes:
constcm=newCModule(`#include <stdio.h>voidinit (void){ printf ("init\\n");}voidfinalize (void){ printf ("finalize\\n");}`);cm.dispose();// or wait until it gets GCed or script unloaded
Anyway, this post is getting long, but before we wrap up let’s look at how to
use CModule with the Stalker APIs:
This shows how you can implement both the transform callback and the callouts
in C, but you may also use a hybrid approach where you write the transform
callback in JS and only some of the callouts in C.
It is also worth noting that I rewrote ObjC.choose() to use CModule, and it is
now roughly 100x faster. When testing it on the login screen of the Twitter app
on an iPhone 6S, this went from taking ~5 seconds to now only ~50 ms.
So with that, I hope you’ll enjoy this release. Excited to see what kind of
things you will build with the new CModule API. One thing I’m really looking
forward to is improving our REPL to support loading a .c file next to the
.js, for rapid prototyping purposes.
Enjoy!
Changes in 12.7.0
Brand new CModule API powered by TinyCC. (You just read about it.)
TinyCC was improved to support Apple’s ABI on macOS/x86.
Stalker.exclude() is now exposed to JS to be able to mark specific memory
ranges as excluded. This is useful to improve performance and reduce noise.
Concurrent calls to Java.use() are now supported, thanks to a neat
contribution by gebing.
The hexdump() implementation was improved to clamp the length option to
the length of the ArrayBuffer, thanks to another neat contribution by
gebing.
TinyCC was improved to support Apple’s ABI on iOS/arm64.
ObjC.choose() was rewritten using CModule, and is now ~100x faster.
Changes in 12.7.2
CModule got some missing ref-counting APIs.
Changes in 12.7.3
CModule memory ranges are now properly cloaked.
The V8 garbage collector is now informed about externally allocated CModule
memory so it can make better decisions about when to GC.
Symbols attached to a CModule are now properly kept alive in the V8 runtime
also; and the CModule itself is not kept alive indefinitely (or until script
unload).
CModule.dispose() was added for eagerly cleaning up memory.
Get ready for an exciting new release. This time around we’re giving some long
overdue love to our Stalker engine. It’s been around for roughly ten years,
but it wasn’t until Frida 10.5 in late 2017 that we started unleashing its
massive potential.
Up until now we were able to Stalker.follow() existing threads and not only
observe them, but also mutate their instruction streams any way we’d like. It
could also be combined with Interceptor to instrument the current thread
between strategic points. This allowed us to build tools such as AirSpy.
But, what if we want to Stalker.follow() a NativeFunction call? This may seem
really simple, but reentrancy makes this really hard. It’s easy to end up
following execution inside e.g. our private heap, and end up needing to allocate
memory for the instrumentation itself… all kinds of fun scenarios that are
mind-boggling to reason about.
The way we dealt with this was to teach Stalker to exclude certain memory
ranges, so that if it sees a call going to such a location it will simply emit
a call instruction there instead of following execution. So what we did was to
automatically exclude frida-agent’s own memory range, and that way we didn’t
have to deal with any of the reentrancy madness.
We also took care to special-case attempts to Stalker.follow() the current
thread, so that we queued that work until we’re about to leave our runtime
and transition back into user code (or our main loop, in the case of the JS
thread).
That still left the big unanswered question of how to use Stalker in conjunction
with NativeFunction. We can now finally put that behind us:
By setting the traps: 'all' option on the NativeFunction, it will re-activate
Stalker when called from a thread where Stalker is temporarily paused because
it’s calling out to an excluded range – which is the case here because all of
frida-agent’s code is marked as excluded.
We can also achieve the same goal for Objective-C methods:
Yay. That said, these examples are barely scratching the surface of what’s
possible using Stalker. One of the really cool use-cases is in-process fuzzing,
which frida-fuzz is a great example of. There’s also a bunch of other
use-cases, such as reversing, measuring code coverage, fault injection for
testing purposes, hooking inline syscalls, etc.
So that’s the main story of this release. Would like to thank
@andreafioraldi for the great bug-reports and help testing these tricky
changes.
Wrapping up
One cool new feature worth mentioning is the new ArrayBuffer.wrap() API, which
allows you to conveniently and efficiently access memory regions as if they were
JavaScript arrays:
This means you can hand over direct memory access to JavaScript APIs without
needing to copy memory in/out of the runtime. The only drawback is that bad
pointers won’t result in a JS exception, and will crash the process.
We now also allow you to access the backing store of any ArrayBuffer, through
the new unwrap() method on ArrayBuffer. An example use-case for this is when
using an existing module such as frida-fs where you get an ArrayBuffer that
you then want to pass to native code.
Kudos to @DaveManouchehri for contributing the first draft of the
ArrayBuffer.wrap() API, and also big thanks to @CodeColorist for suggesting
and helping shape the unwrap() feature.
Changes in 12.8.0
Stalker reactivation working properly.
Stalker thread lifetime properly handled. Will also no longer crash when
following a thread to its death on i/macOS.
Safer garbage collection logic in Stalker.
Making mistakes in the Stalker transform callback which ends up throwing a JS
exception now results in Stalker.unfollow(), so the error doesn’t get
swallowed by the process crashing.
Robust support for Stalker transform calling unfollow().
Stalker support for older x86 CPUs without AVX2 support.
Support for disabling automatic Stalker queue drain.
NativeFunction is better at handling exceptions through the brand new
Interceptor unwinding API.
Java and ObjC APIs to specify NativeFunction options for methods through
clone(options), and blocks through the second argument to ObjC.Block().
ObjC class and protocol caching logic finally works. Thanks @gebing!
The prebuilt Python 3 extension for Windows finally supports all Python 3
versions >= 3.4 on Windows, just like on the other platforms.
ArrayBuffer wrap() and unwrap().
DebugSymbol API has better error-handling on Linux/Android.
Java integration no longer crashes in recompileExceptionClearForArm64() in
system processes on Android 10.
Our previous big release was all about Stalker. For those of you not yet
familiar with it, it’s basically a code tracing engine – allowing threads to be
followed, capturing every function, every block, even every instruction which is
executed. Beyond just tracing code, it also allows you to add and remove
instructions anywhere. It even uses advanced JIT tricks to make all of this
really fast.
That may still sound a little abstract, so let’s have a look at a couple of
examples. One way to use it is when you want to determine “what other functions
does this function call”. Or, perhaps you’d like to use Apple’s speech
synthesizer to announce the RAX register’s value at every RET instruction in
code belonging to the app? Here is how that can be done. This was one of the
demos at my r2con presentation back in 2017.
Up until now Stalker has only been available on Intel architectures and ARM64.
So I’m really excited to announce that Stalker is now also available on ARM32!
Yay! 🎉 I’m hopeful that this sorely missed Stalker backend will motivate a lot
of you to start building really cool things on top of Stalker. I feel like it
has a ton of potential beyond “just” code tracing. And combined with CModule
it’s become really easy to balance rapid prototyping and dynamic behavior with
performance.
There’s so much to talk about in this release. One of the other major changes is
that we have upgraded all of our dependencies. Most interesting among them is
probably V8, which we have upgraded to 8.4. This means you can use all of the
latest JavaScript language features such as optional chaining and nullish
coalescing operator without having to frida-compile your agent. That, and
performance improvements, another area where V8 just keeps on getting better and
better.
We’ve also just added support for Android 11 Developer Preview 4, and iOS/arm64e
apps are now fully supported even on jailed iOS. Things have improved so much
across all of our supported platforms. One thing in particular that I’d like to
highlight is that we have finally eliminated a long-standing resource leak
affecting our Duktape-based JS runtime – a bug that’s been around for as long as
we’ve been using Duktape as our default JS runtime.
Anyway, there’s really no easy way to dig into all of the areas where things
have improved, so definitely check out the changelog below.
Enjoy!
Changes in 12.9.0
Stalker now also available on ARM32. 🎉
Stalker JS integrations no longer clobber errno / LastError.
Stalker.follow() now reliable on x86 and ARM64, including when the target
thread is in a syscall on Windows.
Stalker is finally reliable on WoW64. Thanks @zuypt!
All dependencies have been updated to the latest and greatest. Most exciting
is V8 8.4, supporting the latest JavaScript language features.
Long-standing Duktape memory leak finally discovered and fixed. Thanks to
@disazoz for the bug-report that lead to this breakthrough.
Socket.connect() no longer leaks the file-descriptor (and associated memory)
on error. (Fixed by the GLib dependency upgrade.) Thanks for reporting,
@1215clf!
Kernel.read*() no longer leaks in the V8 runtime.
UNIX build system moved to Meson 0.54.
Windows build system moved to VS2019.
Node.js prebuilds provided for v14, in addition to v10 and v12.
Electron prebuilds provided for v8 and v9.
Fedora packages for F32.
Ubuntu packages for Ubuntu 20.04.
Python bindings no longer using any deprecated APIs.
Support for leanback-only Android apps. Thanks @5murfette!
iOS jailed spawn() w/o closure supported on arm64e. Thanks @mrmacete!
iOS usbmux pairing record plist parsing now also handles binary plists,
fixing a long-standing issue where Frida would reject a tethered iOS USB
device. Thanks @pachoo!
ObjC.choose() also supported on arm64e. Thanks @mrmacete!
ObjC.protocols enumeration finally working properly, and not just the first
time. Thanks for reporting, @CodeColorist!
Initial support for Android 11 Developer Preview. Thanks @abdawoud!
MUSL libc compatibility.
Support for older versions of glibc, so our binaries can run on a wide variety
of desktop Linux systems.
Libc shim also covers memalign() and supports newer GNU toolchains.
Exceptor’s POSIX backend is now detecting Thumb correctly on ARM32, which
would previously result in random crashes.
Exceptor no longer clobbers “rflags” (x86_64) and “cpsr” (ARM64) on i/macOS,
and provides write access to the native context.
Four essential i/macOS 64-bit syscalls added to the libc shim: read(),
write(), mmap(), munmap(). Thanks @mrmacete!
iOS binaries now signed with the “skip-library-validation” entitlement for
convenience. Thanks @elvanderb!
The frida-core Vala API bindings are no longer missing the frida.Error type.
Our scripts now allow messages to be post()ed to them while they are in the
LOADING state. This is useful when a script needs to make a synchronous
request during load(). Thanks @Gbps!
Gadget finally supports early instrumentation with the V8 runtime on 64-bit
ELF targets, where constructor functions would previously be run in the wrong
order. Thanks @tacesrever!
Support for ADB channels beyond just TCP in Device.open_channel().
Thanks @aemmitt-ns!
Many new instructions supported in the ArmWriter and ThumbWriter APIs.
Massive improvements to our ARM32 relocator implementations.
Linux module enumeration working when invoked through loader.
Linux symbol resolution improvements.
Better argument list handling in the V8 runtime, treating undefined the same
as in the Duktape runtime. Thanks @mrmacete!
CModule Stalker API is back in working order.
CModule runtime now exposes Thread.{get,set}_system_error().
CModule is now a stub on Linux/MIPS, instead of failing to compile due to
TinyCC not yet supporting MIPS.
Capstone configured to support ARMv8 A32 encodings.
Changes in 12.9.1
The Python bindings’ setup.py does the right thing for Python 3.x on macOS.
Changes in 12.9.2
Fruity (iOS USB) backend no longer emits a warning on stdio.
Changes in 12.9.3
Android 11 Developer Preview 4 is now supported. Thanks for the assist,
@enovella_!
Linux file monitoring is back in great shape.
ArmRelocator properly relocates ADD instructions involving the PC register.
ThumbRelocator properly handles IT blocks containing an unconditional branch.
This means Interceptor is able to hook even more tricky cases. Thanks
@bet4it!
Stalker ARM32 also supports clone syscalls in Thumb mode.
Stalker ARM32 now suppresses events around exclusive ops like the ARM64
backend.
Stalker ARM32 trust threshold support.
Improved error-handling in ObjC and Java bridges, to avoid crashing the
process on unsupported OSes.