Jean-Marie Mineau
parent
ba7130160e
commit
7f61637b64
GPG key ID:
B66AEEDA9B645AD2
5 changed files with 127 additions and 80 deletions
|
|
@ -1,4 +1,4 @@
|
|||
#import "../lib.typ": todo, APK, DEX, JAR, OAT, eg, ART, paragraph, jm-note, jfl-note
|
||||
#import "../lib.typ": todo, APK, DEX, JAR, OAT, SDK, eg, ART, jm-note, jfl-note
|
||||
|
||||
== Code Transformation <sec:th-trans>
|
||||
|
||||
|
|
@ -156,8 +156,7 @@ We saw in @sec:cl the class loading model of Android.
|
|||
When doing dynamic code loading, an application define a new `ClassLoader` that handle the new bytecode, and start accessing its classes using reflection.
|
||||
We also saw in @sec:cl that Android now use the multi-dex format, allowing it to handle any number of #DEX files in one classloader.
|
||||
Therefore, the simpler way to give access to the dynamically loaded code to static analysis tool is add the dex files to the application.
|
||||
This should not impact the classloading model as long as there is no class collision (we will explore this in @sec:th-class-collision) and as long as the original application appliaction did not try to access unaccessible classes.
|
||||
#jm-note[explain? maybe ref to section limitation]
|
||||
This should not impact the classloading model as long as there is no class collision (we will explore this in @sec:th-class-collision) and as long as the original application appliaction did not try to access unaccessible classes (we will develop this issue in @sec:th-limits).
|
||||
|
||||
#figure(
|
||||
image(
|
||||
|
|
@ -181,21 +180,20 @@ In @sec:th-trans-cl, we are adding new code.
|
|||
By doing so, we increase the probability of having class collisions:
|
||||
The developper may have reuse a helper class in both the dynamically loaded bytecode and the application, or an obfuscation process may have rename classes without checking for intersection between the two sources of bytecode.
|
||||
When loaded dynamically, the classes are in a different classloader, and the class resolution is resolved at runtime like we saw in @sec:cl-loading.
|
||||
We decided to restrain our scope to the use of class loader from the Android SDK.
|
||||
We decided to restrain our scope to the use of class loader from the Android #SDK.
|
||||
In the abscence of class collision, those class loader behave seamlessly and adding the classes to application maintains the behavior.
|
||||
|
||||
#jm-note[
|
||||
When we detect a collision, we rename one of the classes colliding in order to be able to differenciate both classes.
|
||||
When we detect a collision, we rename one of the colliding classes in order to be able to differenciate both classes.
|
||||
To avoid breaking the application, we then need to rename all references to this specific class, an be carefull not to modify references to the other class.
|
||||
To do so, we regroup each classes by the classloaders defining them, then, for each colliding class name and each classloader, we check the actual class used by the classloader.
|
||||
To do so, we regroup each classes by the classloaders defining them.
|
||||
Then, for each colliding class name and each classloader, we check the actual class used by the classloader.
|
||||
If the class has been renamed, we rename all reference to this class in the classes defined by this classloader.
|
||||
To find the class used by a classloader, we reproduce the behavior of the different classloaders of the Android SDK.
|
||||
To find the class used by a classloader, we reproduce the behavior of the different classloaders of the Android #SDK.
|
||||
This is an important step: remember that the delegation process can lead to situation where the class defined by a classloader is not the class that will be loaded when querying the classloader.
|
||||
The pseudo-code in @lst:renaming-algo show the three steps of this algorithm:
|
||||
- first we detect collision and rename classes definitions to remove the collisions
|
||||
- then we rename the reference to the colliding classes to make sure the right classes are called
|
||||
- ultimately, we merge the modified dexfiles of each class loaders into one android application
|
||||
][this is redundant an messy]
|
||||
- First we detect collision and rename classes definitions to remove the collisions.
|
||||
- Then we rename the reference to the colliding classes to make sure the right classes are called.
|
||||
- Ultimately, we merge the modified dexfiles of each class loaders into one android application.
|
||||
|
||||
#figure(
|
||||
```python
|
||||
|
|
@ -229,34 +227,7 @@ The pseudo-code in @lst:renaming-algo show the three steps of this algorithm:
|
|||
* #todo[interupting try blocks: catch block might expect temporary registers to still stored the saved value] ?
|
||||
*/
|
||||
|
||||
=== Limitations
|
||||
#h(2em)
|
||||
|
||||
#paragraph()[Custom Classloaders][
|
||||
#jfl-note[The first obvious limitation is that we do not know what custom classloaders do, so we cannot accuratly reproduce statically their behavior.][est ce que c'est une limite des 2 transformations proposées? j'ai l'impression que tu veux faire une 3ieme transformation]
|
||||
We elected to fallback to the behavior of the `BaseDexClassLoader`, which is the highest Android specific classloader in the inheritance hierarchy, and whose behavior is shared by all classloaders safe `DelegateLastClassLoader`.
|
||||
The current implementation of the #ART enforce some restrictions on the classloaders behavior to optimize the runtime performance by caching classes.
|
||||
This gives us some garanties that custom classesloaders will keep a some coherences will the classic classloaders.
|
||||
For instance, a class loaded dynamically must have the same name as the name used in `ClassLoader.loadClass()`.
|
||||
This make `BaseDexClassLoader` a good estimation for legitimate classloaders, however, an obfuscated application could use the techniques discussed in @sec:cl-cross-obf, in wich case our model would be entirelly wrong.
|
||||
]
|
||||
|
||||
#paragraph()[Multiple Classloaders for one `Method.invoke()`][
|
||||
#todo[explain the problem arrose each time a class is compared to another]
|
||||
Although we managed to handle call to different methods from one `Method.invoke()` site, we do not handle calling methods from different classloaders with colliding classes definition.
|
||||
The first reason is that it is quite challenging to compare classloaders statically.
|
||||
At runtime, each object has an unique identifier that can be used to compare them over the course of the same execution, but this identifier is reset each time the application starts.
|
||||
This means we cannot use this identifier in an `if` condition to differentiate the classloaders.
|
||||
Ideally, we would combine the hash of the loaded #DEX files, the classloader class and parent to make an unique, static identifier, but the #DEX files loaded by a classloader cannot be accessed at runtime without accessing the process memory at arbitrary locations.
|
||||
For some classloaders, the string representation returned by `Object.toString()` list the location of the loaded #DEX file on the file system.
|
||||
This is not the case for the commonly used `InMemoryClassLoader`.
|
||||
In addition, the #DEX files are often located in the application private folder, whose name is derived from the hash of the #APK itself.
|
||||
Because we modify the application, the path of the private folder also change, and so will the string representation of the classloaders.
|
||||
Checking the classloader of a classes can also have side-effect on classloaders that delegate to the main application classloader:
|
||||
because we inject the classes in the #APK, the classes of the classloader are now already in the main application classloader, which in most case will have priority on the other classloaders, and lead to the class beeing loaded by the application classloader instead of the original classloader.
|
||||
If we check for the classloader, we would need to considere such cases en rename each classes of each classloader before reinjecting them to the in the application.
|
||||
This would greatly increase the risk of breaking the application during its transformation.
|
||||
Instead, we elected to ignore the classloaders when selecting the method to invoque.
|
||||
This leads to potential invalid runtime behaviore, as the first method that matching the class name will be called, but the alternative methods from other classloader still appears in the new application, albeit in a block that might be flagged as dead-code by a sufficiently advenced static analyser.
|
||||
]
|
||||
|
||||
#jfl-note[Deporter ces limites en fin de chapitre? je lirai tout ca + tard]
|
||||
Now that we saw the transformations we want to make, we know the runtime information we need to do it.
|
||||
In the next section, we will propose a solution to collect those informations.
|
||||
|
|
|
|||
|
|
@ -1,23 +1,58 @@
|
|||
#import "@preview/diagraph:0.3.5": raw-render
|
||||
#import "../lib.typ": todo, SDK, API, ART, DEX, APK, JAR, ADB, jfl-note
|
||||
|
||||
== Collecting Runtime Information <sec:th-dyn>
|
||||
|
||||
#jfl-note[Figure dcrivant le process a mettre en avant?]
|
||||
@fig:th-process show the general idea of our process.
|
||||
To perform the transformations discribed in @sec:th-trans, we need information like the name and signature of the method called with reflection, or the actual bytecode loaded dynamically.
|
||||
We decided to collet those information through dynamic analysis.
|
||||
We saw in @sec:bg different contributions that collect this kind of information.
|
||||
In the end, we decided to keep the analysis as simple as possible, so we avoided using a custom Android build like DexHunter, and instead use Frida(see @sec:bg-frida) to instrument the application and intercept calls of the methods that interest us.
|
||||
@sec:th-fr-dcl present our approach to collect dynamically loaded bytecode, and @sec:th-fr-ref present our approach to collect the reflection data.
|
||||
Because using dynamic analysis raise the concern of coverage, we also need some interaction with application during the analysis.
|
||||
Ideally, a reverse engineer would do the interaction.
|
||||
Because we wanted to analyse many applications in a reasonable time, we replaced this engineer by an automated runner that simulates the interactions.
|
||||
We discuss this option in @sec:th-grod.
|
||||
|
||||
/*
|
||||
* APK -> DYN -> RUNTIME INFO
|
||||
* APK -> TRANSFO
|
||||
* RUNTIME INFO -> TRANSFO
|
||||
* TRANSFO -> APK'
|
||||
*/
|
||||
#figure(
|
||||
raw-render(
|
||||
```
|
||||
digraph {
|
||||
rankdir=LR
|
||||
|
||||
splines="ortho"
|
||||
|
||||
|
||||
In order to perform the transformations described in @sec:th-trans, we need information like the name and signature of the method called with reflection, or the actual bytecode loaded dynamically.
|
||||
We are doing those transformation specifically because those information are difficult to extract statically.
|
||||
Hence, we are using dynamic analysis to collect the runtime information we need.
|
||||
We use Frida(see @sec:bg-frida) to instrument the application and intercept calls of specific methods.
|
||||
APK [shape=parallelogram]
|
||||
"Automated Runner"
|
||||
"Reverse Engineer"
|
||||
"Dynamic Analysis" [shape=box]
|
||||
"Runtime Information" [shape=parallelogram]
|
||||
Transformation [shape=box]
|
||||
"APK'" [shape=parallelogram]
|
||||
|
||||
=== Collecting Bytecode Dynamically Loaded
|
||||
APK:c -> "Dynamic Analysis"
|
||||
"Automated Runner" -> "Dynamic Analysis" [style="dashed"]
|
||||
"Reverse Engineer" -> "Dynamic Analysis" [style="dashed"]
|
||||
"Dynamic Analysis" -> "Runtime Information"
|
||||
APK -> Transformation
|
||||
"Runtime Information" -> Transformation
|
||||
Transformation -> "APK'"
|
||||
}
|
||||
```,
|
||||
width: 100%,
|
||||
alt: (
|
||||
"A diagram showing the process to transform an application.",
|
||||
"Dotted arrows go from a \"Automated Runner\" and from \"Reverse Engineer\" to a box labeled \"Dynamic Analysis\", as well as plain arrow from \"APK\" to \"Dynamic Analysis\".",
|
||||
"An arrow goes from \"Dynamic Analysis\" to \"Runtime Information\", then from \"Runtime Information\" to a box labeled \"Transformation\".",
|
||||
"Another arrow goes from \"APK\" to \"Transformation\".",
|
||||
"Finally, an arrow goes from \"Transformation\" to \"APK'\"."
|
||||
).join(),
|
||||
),
|
||||
caption: [Process to add runtime information to an #APK],
|
||||
) <fig:th-process>
|
||||
|
||||
=== Collecting Bytecode Dynamically Loaded <sec:th-fr-dcl>
|
||||
|
||||
Initially, we considered instrumenting the constructor methods of the classloaders of the Android #SDK.
|
||||
However, this is a significant number of methods to instrument, and looking at older application, we realized that we missed the `DexFile` class.
|
||||
|
|
@ -29,10 +64,9 @@ As a reference, in 2015, DexHunter~@zhang2015dexhunter already noticed `DexFile.
|
|||
`DefineClass(..)` is still a good function to instrument, but it is a C++ native method that does not have a Java interface, making it harder to work with using Frida, and we want to avoid patching the source code of the #ART like DexHunter did.
|
||||
For this reason, we decided to hook `DexFile.openInMemoryDexFilesNative(..)` and `DexFile.openDexFileNative(..)` instead.
|
||||
Those methods takes as argument a list of Androis code files, either in the form of in memory byte arrays or file path, and a reference to the classloader associated to the code.
|
||||
The code files can have many format, usually #DEX files, or #APK / #JAR files containing #DEX files, but it can also be internal format like `.aot` #todo[check, aot explain somewhere?]. #todo[cf later to explain that only #DEX / #APK / #JAR are found?]
|
||||
Instrumenting those methods allows us to collect all the #DEX files loaded by the #ART and associate them to their classloaders.
|
||||
Instrumenting those methods allows us to collect all the code files loaded by the #ART and associate them to their classloaders.
|
||||
|
||||
=== Collecting Reflection Data
|
||||
=== Collecting Reflection Data <sec:th-fr-ref>
|
||||
|
||||
As described in @sec:th-trans-ref, they are 3 methods that we need to instrument to capture reflection calls: `Class.newInstance()`, `Constructor.newInstance(..)` and `Method.invoke(..)`.
|
||||
Because Java has polymorphism, we need not only the method name and defining class, but also the whole signature of the method.
|
||||
|
|
@ -49,29 +83,31 @@ In particullar, the location of the call in the bytecode has a different meaning
|
|||
It can either be the address of the bytecode instruction invoking the callee method in the instruction array of the caller method, or the line number of original source code that call the callee method.
|
||||
Fortunatelly, in the #SDK 34, Android introduced the `StackWalker` #API.
|
||||
This #API allow to programatically travel the current stack and retrieve informations from it, including the bytecode address of the instruction calling the callee methods.
|
||||
Considering that the line number is not a reliable information, we chose to use the new #API, despite the restriction that come with chosing such a recent Android version (it was released in october 2023, arround 2 years ago, and less than 50% of the current Android market share support this #API today #todo[archive ref https://gs.statcounter.com/android-version-market-share]).
|
||||
Considering that the line number is not a reliable information, we chose to use the new #API, despite the restriction that come with chosing such a recent Android version (it was released in october 2023, arround 2 years ago, and less than 50% of the current Android market share support this #API today#footnote[https://gs.statcounter.com/android-version-market-share/mobile-tablet/worldwide/#monthly-202401-202508]).
|
||||
|
||||
=== Application Execution
|
||||
=== Application Execution <sec:th-grod>
|
||||
|
||||
Dynamic analysis requires actually running the application.
|
||||
In order to test automatically multiple applications, we needed to simulate human interractions with the applications.
|
||||
We found a few tools available, #todo[ref les outils testé, peut etre mettre dans state of the art?].
|
||||
After some tests, the #jfl-note[most suitable][donne un peu des details des pbs des autres et que comme vu en @sec:bg les outils sont peu reutilisable] one we found were the Monkey, a standard Android tool from Google that generate random event, and GroddDroid #todo[ref].
|
||||
We chose to avoid the Monkey because we noticed that it will often trigger event that will close the application (events likes pressing the 'home' button, or openning the general setting drop-down menu at the top of the screen).
|
||||
GroddDroid different execution modes.
|
||||
We choosed to use the most simple one, that explore the application following a depth-first search algorithm.
|
||||
GroddDroid can do more advance explorations targetting suspicious section of the application en priority, but this require to perform heavy static analysis.
|
||||
We elected to avoid this option to keep the exploiration lightwight and limit the chance of crashing the analysis (we saw in @sec:rasta the issues brought by complexe analysis).
|
||||
Behind the scene, GroddDroid uses UI Automator to interact with the application, an standar Android API used intended for automatic testing.
|
||||
In @sec:bg we presented a few solution to explore an application dynamically.
|
||||
We first eliminated Sapienz, as it rely on an application instrumentation library called ELLA, that has not be updated since 9 years ago.
|
||||
We also chose to avoid the Monkey because we noticed that it will often trigger event that will close the application (events likes pressing the 'home' button, or openning the general setting drop-down menu at the top of the screen).
|
||||
Stoat and GroddDroid use UI Automator to interact with the application.
|
||||
UI Automator is a standard Android #API inteded for automatic testing.
|
||||
Both Soat and GroddDroid perfom additionnal analysis on the application to improve the exploration.
|
||||
In the end, we elected to use the most basic execution mode of GroddDroid that does not need this additionnal analysis.
|
||||
It explore the application following a depth-first search algorithm.
|
||||
We chose this option to keep the exploiration lightwight and limit the chance of crashing the analysis (we saw in @sec:rasta the issues brought by complexe analysis).
|
||||
It might be interesting in futur work to explore more complexe exploration techniques.
|
||||
|
||||
Because we are using Frida, we do not need to use a custom version of Android with a modified #ART or kernel like DexHunter of AppSpear.
|
||||
However, we decided to not inject Frida in the original application, so we need to have root access to directly run Frida in Android which is not a normal thing to have on Android.
|
||||
Because we are using Frida, we do not need to use a custom version of Android with a modified #ART or kernel like, however, we decided to not inject Frida in the original application.
|
||||
This means we need to have root access to directly run Frida in Android which is not a normal thing to have on Android.
|
||||
Because dynamic analysis can be slow, we also decided to run the applications on emulators.
|
||||
This makes its easier to run several analysis in parallel.
|
||||
The alternative would have been to run the application on actual smartphones, and would have required multiple phones to run the analysis in parallel.
|
||||
For simplicity, we choosed to use Google Android emulator for our experiment.
|
||||
We spawned multiple emulators, installed Frida on it, took a snapshot of the emulator before installing the application to analyse.
|
||||
Then we run the application for a minute #todo[check la valeur exacte] with GroddRunner, and at the end of the analysis, we reload the snapshot in case the application modified the system in some unforseen way.
|
||||
Then we run the application for a five minutes with GroddRunner, and at the end of the analysis, we reload the snapshot in case the application modified the system in some unforseen way.
|
||||
If at some point the emulator start responding for too long, we terminate it and restart it.
|
||||
|
||||
#todo[Droid donjon, dire qu'on est au niveau -1 de l'anti-evation]
|
||||
|
|
|
|||
49
5_theseus/5_limits.typ
Normal file
49
5_theseus/5_limits.typ
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
#import "../lib.typ": paragraph, ART, DEX, APK
|
||||
#import "../lib.typ": todo, jfl-note, jm-note
|
||||
|
||||
== Limitations <sec:th-limits>
|
||||
|
||||
#todo[Structure the section]
|
||||
|
||||
#paragraph()[Custom Classloaders][
|
||||
#jfl-note(side: right)[The first obvious limitation is that we do not know what custom classloadrs do, so we cannot accuratly reproduce statically their behavior.][est ce que c'est une limite des 2 transformations proposées? j'ai l'impression que tu veux faire une 3ieme transformation]
|
||||
We elected to fallback to the behavior of the `BaseDexClassLoader`, which is the highest Android specific classloader in the inheritance hierarchy, and whose behavior is shared by all classloaders safe `DelegateLastClassLoader`.
|
||||
The current implementation of the #ART enforce some restrictions on the classloaders behavior to optimize the runtime performance by caching classes.
|
||||
This gives us some garanties that custom classesloaders will keep a some coherences will the classic classloaders.
|
||||
For instance, a class loaded dynamically must have the same name as the name used in `ClassLoader.loadClass()`.
|
||||
This make `BaseDexClassLoader` a good estimation for legitimate classloaders, however, an obfuscated application could use the techniques discussed in @sec:cl-cross-obf, in wich case our model would be entirelly wrong.
|
||||
]
|
||||
|
||||
#paragraph()[Multiple Classloaders for one `Method.invoke()`][
|
||||
Although we managed to handle call to different methods from one `Method.invoke()` site, we do not handle calling methods from different classloaders with colliding classes definition.
|
||||
The first reason is that it is quite challenging to compare classloaders statically.
|
||||
At runtime, each object has an unique identifier that can be used to compare them over the course of the same execution, but this identifier is reset each time the application starts.
|
||||
This means we cannot use this identifier in an `if` condition to differentiate the classloaders.
|
||||
Ideally, we would combine the hash of the loaded #DEX files, the classloader class and parent to make an unique, static identifier, but the #DEX files loaded by a classloader cannot be accessed at runtime without accessing the process memory at arbitrary locations.
|
||||
For some classloaders, the string representation returned by `Object.toString()` list the location of the loaded #DEX file on the file system.
|
||||
This is not the case for the commonly used `InMemoryClassLoader`.
|
||||
In addition, the #DEX files are often located in the application private folder, whose name is derived from the hash of the #APK itself.
|
||||
Because we modify the application, the path of the private folder also change, and so will the string representation of the classloaders.
|
||||
Checking the classloader of a classes can also have side-effect on classloaders that delegate to the main application classloader:
|
||||
because we inject the classes in the #APK, the classes of the classloader are now already in the main application classloader, which in most case will have priority on the other classloaders, and lead to the class beeing loaded by the application classloader instead of the original classloader.
|
||||
If we check for the classloader, we would need to considere such cases en rename each classes of each classloader before reinjecting them to the in the application.
|
||||
This would greatly increase the risk of breaking the application during its transformation.
|
||||
Instead, we elected to ignore the classloaders when selecting the method to invoque.
|
||||
This leads to potential invalid runtime behaviore, as the first method that matching the class name will be called, but the alternative methods from other classloader still appears in the new application, albeit in a block that might be flagged as dead-code by a sufficiently advenced static analyser.
|
||||
]
|
||||
|
||||
#paragraph()[`ClassNotFoundException` may not be raised][
|
||||
In the very specific situation where the original application tries to access a class from dynamically loaded bytecode without actually accessing this bytecode, the patched application behavior will differ.
|
||||
The original application should raise a `ClassNotFoundException`, but in the patched application, the class will be accible and the exception will not be raised.
|
||||
In pactice, their is not a lot of reason to do such thing.
|
||||
One could be to check if the #APK as been tempered with, but their are easier ways to do thins, like checking the application signature.
|
||||
#jm-note[Exception oriented programming worth mentioning? ]
|
||||
Another would be to check if the class is already available, and if not, load it dynamically, in wich case it does not matter as code loaded dynamically is already present.
|
||||
In any cases, statically, because we remove neither the calls to the function that load the classes (like `ClassLoader.loadClass(..)`) nor the `try` / `catch` blocks, static analysis tools those can handle the original behavior should still be hable to access the old behavior.
|
||||
]
|
||||
|
||||
|
||||
#todo[
|
||||
- Use multidex: min SDK >= 21 (android 5.0, published in 2014, should be ok)
|
||||
- No support for OAT (platform dependent)
|
||||
]
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
#import "../lib.typ": todo
|
||||
|
||||
== Limits and Threat to Validity <sec-ttv>
|
||||
|
||||
#todo[redaction]
|
||||
|
||||
- Use multidex: min SDK >= 21 (android 5.0, published in 2014, should be ok)
|
||||
- No support for OAT (platform dependent)
|
||||
|
||||
|
|
@ -15,4 +15,4 @@
|
|||
#include("2_static_transformation.typ")
|
||||
#include("3_dynamic_data_collection.typ")
|
||||
#include("4_results.typ")
|
||||
#include("5_ttv.typ")
|
||||
#include("5_limits.typ")
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue