This commit is contained in:
parent
039970904e
commit
10df431972
9 changed files with 240 additions and 250 deletions
|
@ -5,25 +5,25 @@
|
||||||
#todo[Reflectif call? Reflection call?]
|
#todo[Reflectif call? Reflection call?]
|
||||||
|
|
||||||
In the previous chapter, we studied the static impact of class loaders.
|
In the previous chapter, we studied the static impact of class loaders.
|
||||||
Doing so, we ignored the main usage of class laoders by developpers: dynamic code loading.
|
Doing so, we ignored the main usage of class loaders by developers: dynamic code loading.
|
||||||
In this chapter, we tackle this issue, as well as the issue of reflection that often comes with dynamic code loading.
|
In this chapter, we tackle this issue, as well as the issue of reflection that often comes with dynamic code loading.
|
||||||
Dynamic code loading is the practice of loading at runtime bytecode that was not already part of the original bytecode of the association.
|
Dynamic code loading is the practice of loading at runtime bytecode that was not already part of the original bytecode of the association.
|
||||||
This bytecode can stored as assets of the application, downloaded from a remote server, or even generated algorithmically by the application.
|
This bytecode can be stored as assets of the application, downloaded from a remote server, or even generated algorithmically by the application.
|
||||||
This is a problem for analysis: when the bytecode is not already visible in the application, it cannot be analysed statically.
|
This is a problem for analysis: when the bytecode is not already visible in the application, it cannot be analysed statically.
|
||||||
Meanwhile, reflection is the action of using code to manipulate object representing structures of the code itself, like classes or methods.
|
Meanwhile, reflection is the action of using code to manipulate objects representing structures of the code itself, like classes or methods.
|
||||||
The main issue for analysis occures when it is used to call methods.
|
The main issue for analysis occurs when it is used to call methods.
|
||||||
A static analtysis will show calls to `Method.invoke()`, but not the actual method invoked.
|
A static analysis will show calls to `Method.invoke()`, but not the actual method invoked.
|
||||||
|
|
||||||
In both cases, static analysis falls short, as the information to analyse may be generated just it time for its use.
|
In both cases, static analysis falls short, as the information to analyse may be generated just in time for its use.
|
||||||
For such cases, dynamic analysis is a more appropriate approach.
|
For such cases, dynamic analysis is a more appropriate approach.
|
||||||
It can be used to collect the missing information while the application is running.
|
It can be used to collect the missing information while the application is running.
|
||||||
However, having this information does not means that the application can now be analysed in its entirety.
|
However, having this information does not mean that the application can now be analysed in its entirety.
|
||||||
Generic analysis tools rarely have an easy way to read additionnal information about an application before analysing, and when they do, it is not standard.
|
Generic analysis tools rarely have an easy way to read additional information about an application before analysing, and when they do, it is not standard.
|
||||||
The usual approach for hybrid analysis, analysis that mix static and dynamic analysis, is to select one specific static tool, and modify its code to take into accound the additionnal data collected by dynamic analysis.
|
The usual approach for hybrid analysis, analyses that mix static and dynamic analysis, is to select one specific static tool and modify its code to take into account the additional data collected by dynamic analysis.
|
||||||
In this chapter, we propose to modify the code of the application to add the information needed for analysis in a format that any analysis tool can use.
|
In this chapter, we propose to modify the code of the application to add the information needed for analysis in a format that any analysis tool can use.
|
||||||
|
|
||||||
We structured this chapter as follow: We first present an overview of our method in @sec:th-overview.
|
We structured this chapter as follows: We first present an overview of our method in @sec:th-overview.
|
||||||
We then present the transformations we apply to the application in @sec:th-trans, and the dynamic analysis we perform in @sec:th-dyn.
|
We then present the transformations we apply to the application in @sec:th-trans and the dynamic analysis we perform in @sec:th-dyn.
|
||||||
In @sec:th-res compare the results of different tools on the initial application versus the modified application.
|
In @sec:th-res compare the results of different tools on the initial application versus the modified application.
|
||||||
To complete this chapter, in @sec:th-limits we discuss the limits of our solution, as well as directions for futur works.
|
To complete this chapter, in @sec:th-limits we discuss the limits of our solution, as well as directions for future work.
|
||||||
Finally, we conclude in @sec:th-conclusion.
|
Finally, we conclude in @sec:th-conclusion.
|
||||||
|
|
|
@ -3,11 +3,11 @@
|
||||||
|
|
||||||
== Overview <sec:th-overview>
|
== Overview <sec:th-overview>
|
||||||
|
|
||||||
Our objectif is to make available some dynamic information to any analysis tool able to analyse an Android #APK.
|
Our objective is to make available some dynamic information to any analysis tool able to analyse an Android #APK.
|
||||||
To do so, we elected to follow the path of a few contributions we presented in @sec:bg such as DroidRA~@li_droidra_2016 and use instrumentation.
|
To do so, we elected to follow the path of a few contributions we presented in @sec:bg, such as DroidRA~@li_droidra_2016, and use instrumentation.
|
||||||
Contrary to DroidRA, which use static analysis to compute the values of string and from that the methods used by reflection, we chose to dynamic analysis.
|
Contrary to DroidRA, which uses static analysis to compute the values of strings and, from that, the methods used by reflection, we chose to use dynamic analysis.
|
||||||
This allows us to collect informations that are simply not available statically (#eg a string send from a remote command and control server).
|
This allows us to collect information that is simply not available statically (#eg a string sent from a remote command and control server).
|
||||||
The tradeoff beeing the lack of exhaustiveness: dynamic analysis is known to have code coverage issues.
|
The tradeoff being the lack of exhaustiveness: dynamic analysis is known to have code coverage issues.
|
||||||
|
|
||||||
#figure(
|
#figure(
|
||||||
raw-render(
|
raw-render(
|
||||||
|
@ -47,12 +47,12 @@ The tradeoff beeing the lack of exhaustiveness: dynamic analysis is known to hav
|
||||||
caption: [Process to add runtime information to an #APK],
|
caption: [Process to add runtime information to an #APK],
|
||||||
) <fig:th-process>
|
) <fig:th-process>
|
||||||
|
|
||||||
@fig:th-process summarize our process.
|
@fig:th-process summarises our process.
|
||||||
We first take an application that we analyse dynamically.
|
We first take an application that we analyse dynamically.
|
||||||
To improve code coverage, either an reverse engineer or an automated runner will interact with the application.
|
To improve code coverage, either a reverse engineer or an automated runner will interact with the application.
|
||||||
During this analysis, we use Frida to capture dynamic informations like the name of the methods called using reflection and bytecode loaded at runtime.
|
During this analysis, we use Frida to capture dynamic information like the names of the methods called using reflection and bytecode loaded at runtime.
|
||||||
This analysis described in @sec:th-dyn.
|
This analysis is described in @sec:th-dyn.
|
||||||
|
|
||||||
The data collected by this analysis is then combined to application, transforming the application into another one that can then be analyzed further.
|
The data collected by this analysis is then combined with the application, transforming the application into another one that can then be analysed further.
|
||||||
We present the details of this transformation in @sec:th-trans.
|
We present the details of this transformation in @sec:th-trans.
|
||||||
Since the transformation drives the data we need to collect, we have decided to place this section first in this chapter.
|
Since the transformation drives the data we need to collect, we have decided to place this section first in this chapter.
|
||||||
|
|
|
@ -2,21 +2,21 @@
|
||||||
|
|
||||||
== Code Transformation <sec:th-trans>
|
== Code Transformation <sec:th-trans>
|
||||||
|
|
||||||
In this section, we will see how we can transform the application code to make dynamic codeloading and reflexive calls more analysable by static analysis tools.
|
In this section, we will see how we can transform the application code to make dynamic code loading and reflective calls more analysable by static analysis tools.
|
||||||
|
|
||||||
=== Transforming Reflection <sec:th-trans-ref>
|
=== Transforming Reflection <sec:th-trans-ref>
|
||||||
|
|
||||||
|
|
||||||
In Android, reflection allows to instanciate a class, or call a method, without having this class or method appear in the bytecode.
|
In Android, reflection allows applications to instantiate a class or call a method without having this class or method appear in the bytecode.
|
||||||
Instead, the bytecode uses the generic classes `Class`, `Method` and `Constructor`, that represent any existing class, method or constructor.
|
Instead, the bytecode uses the generic classes `Class`, `Method` and `Constructor`, which represent any existing class, method or constructor.
|
||||||
Reflection often starts by retrieving the `Class` object representing the class to use.
|
Reflection often starts by retrieving the `Class` object representing the class to use.
|
||||||
This class is usually retrieved using a `ClassLoader` object (though they are other ways to get it).
|
This class is usually retrieved using a `ClassLoader` object (though there are other ways to get it).
|
||||||
Once the class is retrieved, it can be instanciated using the deprecated method `Class.newInstance()`, as shown in @lst:-th-expl-cl-new-instance, or a specific method can be retrieved.
|
Once the class is retrieved, it can be instanciated using the deprecated method `Class.newInstance()`, as shown in @lst:-th-expl-cl-new-instance, or a specific method can be retrieved.
|
||||||
The current approach to instanciate a class is to retrieve the specific `Constructor` object, then calling `Constructor.newInstance(..)` like in @lst:-th-expl-cl-cnstr.
|
The current approach to instantiate a class is to retrieve the specific `Constructor` object, then call `Constructor.newInstance(..)` like in @lst:-th-expl-cl-cnstr.
|
||||||
Similarly, to call a method, the `Method` object must be retrieved, then called using `Method.invoke(..)`, as shown in @lst:-th-expl-cl-call.
|
Similarly, to call a method, the `Method` object must be retrieved, then called using `Method.invoke(..)`, as shown in @lst:-th-expl-cl-call.
|
||||||
|
|
||||||
Although the process seems to differ between class instanciation and method call from the Java stand point, the runtime opperations are very similar.
|
Although the process seems to differ between class instantiation and method call from the Java standpoint, the runtime operations are very similar.
|
||||||
When instanciating an object with `Object obj = cst.newInstance("Hello Void")`, the constructor method `<init>(Ljava/lang/String;)V`, represented by the `Constructor` `cst`, is called on the object `obj`.
|
When instantiating an object with `Object obj = cst.newInstance("Hello Void")`, the constructor method `<init>(Ljava/lang/String;)V`, represented by the `Constructor` `cst`, is called on the object `obj`.
|
||||||
|
|
||||||
#figure(
|
#figure(
|
||||||
```java
|
```java
|
||||||
|
@ -24,7 +24,7 @@ When instanciating an object with `Object obj = cst.newInstance("Hello Void")`,
|
||||||
Class clz = cl.loadClass("com.example.Reflectee");
|
Class clz = cl.loadClass("com.example.Reflectee");
|
||||||
Object obj = clz.newInstance();
|
Object obj = clz.newInstance();
|
||||||
```,
|
```,
|
||||||
caption: [Instanciating a class using `Class.newInstance()`]
|
caption: [Instantiating a class using `Class.newInstance()`]
|
||||||
) <lst:-th-expl-cl-new-instance>
|
) <lst:-th-expl-cl-new-instance>
|
||||||
|
|
||||||
#figure(
|
#figure(
|
||||||
|
@ -32,7 +32,7 @@ When instanciating an object with `Object obj = cst.newInstance("Hello Void")`,
|
||||||
Constructor cst = clz.getDeclaredConstructor(String.class);
|
Constructor cst = clz.getDeclaredConstructor(String.class);
|
||||||
Object obj = cst.newInstance("Hello Void");
|
Object obj = cst.newInstance("Hello Void");
|
||||||
```,
|
```,
|
||||||
caption: [Instanciating a class using `Constructor.newInstance(..)`]
|
caption: [Instantiating a class using `Constructor.newInstance(..)`]
|
||||||
) <lst:-th-expl-cl-cnstr>
|
) <lst:-th-expl-cl-cnstr>
|
||||||
|
|
||||||
#figure(
|
#figure(
|
||||||
|
@ -45,15 +45,15 @@ When instanciating an object with `Object obj = cst.newInstance("Hello Void")`,
|
||||||
) <lst:-th-expl-cl-call>
|
) <lst:-th-expl-cl-call>
|
||||||
|
|
||||||
One of the main reasons to use reflection is to access classes that are not present in the application bytecode, nor are platform classes.
|
One of the main reasons to use reflection is to access classes that are not present in the application bytecode, nor are platform classes.
|
||||||
Indeed, the application will crash if the #ART encounter references to a class that is cannot be found by the current classloader.
|
Indeed, the application will crash if the #ART encounters references to a class that cannot be found by the current classloader.
|
||||||
This is often the case when dealing with classes from bytecode loaded dynamically.
|
This is often the case when dealing with classes from bytecode loaded dynamically.
|
||||||
|
|
||||||
To allow static analysis tools to analyse an application that use reflection, we want to replace the reflection call by the bytecode that actually call the method.
|
To allow static analysis tools to analyse an application that uses reflection, we want to replace the reflection call with the bytecode that actually calls the method.
|
||||||
In @sec:th-trans-cl, we deal with the issue of dynamic code loading so that the classes used are in fact present in the application.
|
In @sec:th-trans-cl, we deal with the issue of dynamic code loading so that the classes used are, in fact, present in the application.
|
||||||
|
|
||||||
A notable issue is that a specific reflection call can call different methods.
|
A notable issue is that a specific reflection call can call different methods.
|
||||||
@lst:th-worst-case-ref illustrates a worst case scenario where any method can be called at the same reflection call.
|
@lst:th-worst-case-ref illustrates a worst-case scenario where any method can be called at the same reflection call.
|
||||||
In those situation, we cannot garanty that we know all the methods that can be called (#eg the name of the method called could be retrieved from a remote server).
|
In those situations, we cannot guarantee that we know all the methods that can be called (#eg the name of the method called could be retrieved from a remote server).
|
||||||
In addition, the method we propose in @sec:th-dyn is a best effort approach to collect reflection data: like any dynamic analysis, it is limited by its code coverage.
|
In addition, the method we propose in @sec:th-dyn is a best effort approach to collect reflection data: like any dynamic analysis, it is limited by its code coverage.
|
||||||
|
|
||||||
#figure(
|
#figure(
|
||||||
|
@ -65,14 +65,14 @@ In addition, the method we propose in @sec:th-dyn is a best effort approach to c
|
||||||
caption: [A reflection call that can call any method]
|
caption: [A reflection call that can call any method]
|
||||||
) <lst:th-worst-case-ref>
|
) <lst:th-worst-case-ref>
|
||||||
|
|
||||||
To handle those situation, instead of entirely removing the reflection call, we can modify the application code to test if the `Method` (or `Constructor`) object match any expected method, and if yes, directly call the method.
|
To handle those situations, instead of entirely removing the reflection call, we can modify the application code to test if the `Method` (or `Constructor`) object matches any expected method, and if yes, directly call the method.
|
||||||
If the object does not match any expected method, the code can fallback to the original reflection call.
|
If the object does not match any expected method, the code can fall back to the original reflection call.
|
||||||
DroidRA~@li_droidra_2016 has a similar solution, except that reflective calls are always evaluated, and the static equivalent follow just after, guarded behind an opaque predicate that is always false at runtime.
|
DroidRA~@li_droidra_2016 has a similar solution, except that reflective calls are always evaluated, and the static equivalent follows just after, guarded behind an opaque predicate that is always false at runtime.
|
||||||
@lst:-th-expl-cl-call-trans demonstrate this transformation on @lst:-th-expl-cl-call:
|
@lst:-th-expl-cl-call-trans demonstrate this transformation on @lst:-th-expl-cl-call:
|
||||||
at line 25, the `Method` objet `mth` is checked using a method we generated and injected in the application (defined at line 2 in the listing).
|
at line 25, the `Method` object `mth` is checked using a method we generated and injected in the application (defined at line 2 in the listing).
|
||||||
This method check if the method name, (line 5), its parameters (lines 6-9), its return type (lines 10-11) and its declaring class (lines 13-14) match the expected method.
|
This method checks if the method name (line 5), its parameters (lines 6-9), its return type (lines 10-11) and its declaring class (lines 13-14) match the expected method.
|
||||||
If it is the case, the method is used directly (line 26) after casting the arguments and associated object into the types/classes we just checked.
|
If it is the case, the method is used directly (line 26) after casting the arguments and associated object into the types/classes we just checked.
|
||||||
If the check line 25 does not pass, the original reflectif call is made (line 28).
|
If the check line 25 does not pass, the original reflective call is made (line 28).
|
||||||
If we were to expect other possible methods to be called in addition to `myMethod`, we would add `else if` blocks between lines 26 and 27, with other check methods reflecting each potential method call.
|
If we were to expect other possible methods to be called in addition to `myMethod`, we would add `else if` blocks between lines 26 and 27, with other check methods reflecting each potential method call.
|
||||||
/*
|
/*
|
||||||
#jfl-note[It should be noted that we do the transformation at the bytecode level, the code in the listing correspond to the output of JADX][
|
#jfl-note[It should be noted that we do the transformation at the bytecode level, the code in the listing correspond to the output of JADX][
|
||||||
|
@ -82,24 +82,24 @@ If we were to expect other possible methods to be called in addition to `myMetho
|
||||||
] #todo[Ref to list of common tools?] reformated for readability.
|
] #todo[Ref to list of common tools?] reformated for readability.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
The method check is done in a separate method injected inside the application to avoid clutering the application too much.
|
The method check is done in a separate method injected inside the application to avoid cluttering the application too much.
|
||||||
Because Java (and thus Android) uses polymorphic methods, we cannot just check the method name and its class, but also the whole method signature.
|
Because Java (and thus Android) uses polymorphic methods, we cannot just check the method name and its class, but also the whole method signature.
|
||||||
We chose to limit the transformation to the specific instruction that call `Method.invoke(..)`.
|
We chose to limit the transformation to the specific instruction that calls `Method.invoke(..)`.
|
||||||
This drastically reduce the risks of breaking the application, but leads to a lot of type casting.
|
This drastically reduces the risks of breaking the application, but leads to a lot of type casting.
|
||||||
Indeed, the reflection call uses the generic `Object` class, but actual methods usually use specific classes (#eg `String`, `Context`, `Reflectee`) or scalar types (#eg `int`, `long`, `boolean`).
|
Indeed, the reflection call uses the generic `Object` class, but actual methods usually use specific classes (#eg `String`, `Context`, `Reflectee`) or scalar types (#eg `int`, `long`, `boolean`).
|
||||||
This means that the method parameters and object on which the method is called must be downcast to their actual type before calling the method, then the returned value must be upcasted back to an `Object`.
|
This means that the method parameters and object on which the method is called must be downcast to their actual type before calling the method, then the returned value must be upcast back to an `Object`.
|
||||||
Scalar types especially require special attention.
|
Scalar types especially require special attention.
|
||||||
Java (and Android) distinguish between scalar type and classes, and they cannot be mixed: a scalar cannot be cast into an `Object`.
|
Java (and Android) distinguish between scalar types and classes, and they cannot be mixed: a scalar cannot be cast into an `Object`.
|
||||||
However, each scalar type has an associated class that can be use when doing reflection.
|
However, each scalar type has an associated class that can be used when doing reflection.
|
||||||
For example, the scalar type `int` is associated with the class `Integer`, the method `Integer.valueOf()` can convert an `int` scalar to an `Integer` object, and the method `Integer.intValue()` convert back an `Integer` object to an `int` scalar.
|
For example, the scalar type `int` is associated with the class `Integer`, the method `Integer.valueOf()` can convert an `int` scalar to an `Integer` object, and the method `Integer.intValue()` converts back an `Integer` object to an `int` scalar.
|
||||||
Each time the method called by reflection used scalars, the scalar-object convertion must be made before calling it.
|
Each time the method called by reflection uses scalars, the scalar-object conversion must be made before calling it.
|
||||||
And finally, because the instruction following the reflection call expect an `Object`, the return value of the method must be cast into an `Object`.
|
And finally, because the instruction following the reflection call expects an `Object`, the return value of the method must be cast into an `Object`.
|
||||||
|
|
||||||
This back and forth between types might confuse some analysis tools.
|
This back and forth between types might confuse some analysis tools.
|
||||||
This could be improved in futur works by analysing the code around the reflection call.
|
This could be improved in future works by analysing the code around the reflection call.
|
||||||
For example, if the result of the reflection call is immediatly cast into the expected type (#eg in @lst:-th-expl-cl-call, the result is cast to a `String`), they should not be any need to cast it to Object in between.
|
For example, if the result of the reflection call is immediately cast into the expected type (#eg in @lst:-th-expl-cl-call, the result is cast to a `String`), there should be no need to cast it to Object in between.
|
||||||
Similarly, it is common to have the method parameter arrays generated just before the reflection call never be used again (This is due to `Method.invoke(..)` beeing a varargs method: the array can be generated by the compiler at compile time).
|
Similarly, it is common to have the method parameter arrays generated just before the reflection call and never be used again (This is due to `Method.invoke(..)` being a varargs method: the array can be generated by the compiler at compile time).
|
||||||
In those cases, the parameters could be used directly whithout the detour inside an array.
|
In those cases, the parameters could be used directly without the detour inside an array.
|
||||||
|
|
||||||
#figure(
|
#figure(
|
||||||
```java
|
```java
|
||||||
|
@ -142,31 +142,30 @@ In those cases, the parameters could be used directly whithout the detour inside
|
||||||
|
|
||||||
#jfl-note[Ici je pensais lire comment on tranforme le code qui load du code, mais on me parle de multi dex]
|
#jfl-note[Ici je pensais lire comment on tranforme le code qui load du code, mais on me parle de multi dex]
|
||||||
|
|
||||||
An application can dynamically import code from several format like #DEX, #APK, #JAR or #OAT, either stored in memory or in a file.
|
An application can dynamically import code from several formats like #DEX, #APK, #JAR or #OAT, either stored in memory or in a file.
|
||||||
Because it is an internal, platform dependant format, we elected to ignore the #OAT format.
|
Because it is an internal, platform-dependent format, we elected to ignore the #OAT format.
|
||||||
Practically, #JAR and #APK files are zip files containing #DEX files.
|
Practically, #JAR and #APK files are zip files containing #DEX files.
|
||||||
This means that we only need to find a way to integrate #DEX files to the application.
|
This means that we only need to find a way to integrate #DEX files into the application.
|
||||||
|
|
||||||
|
|
||||||
We saw in @sec:cl the class loading model of Android.
|
We saw in @sec:cl the class loading model of Android.
|
||||||
When doing dynamic code loading, an application define a new `ClassLoader` that handle the new bytecode, and start accessing its classes using reflection.
|
When doing dynamic code loading, an application defines a new `ClassLoader` that handles the new bytecode, and starts accessing its classes using reflection.
|
||||||
We also saw in @sec:cl that Android now use the multi-dex format, allowing it to handle any number of #DEX files in one classloader.
|
We also saw in @sec:cl that Android now use the multi-dex format, allowing it to handle any number of #DEX files in one classloader.
|
||||||
Therefore, the simpler way to give access to the dynamically loaded code to static analysis tool is add the dex files to the application.
|
Therefore, the simpler way to give access to the dynamically loaded code to static analysis tools is to add the dex files to the application.
|
||||||
This should not impact the classloading model as long as there is no class collision (we will explore this in @sec:th-class-collision) and as long as the original application appliaction did not try to access unaccessible classes (we will develop this issue in @sec:th-limits).
|
This should not impact the classloading model as long as there is no class collision (we will explore this in @sec:th-class-collision) and as long as the original application did not try to access inaccessible classes (we will develop this issue in @sec:th-limits).
|
||||||
|
|
||||||
#figure(
|
#figure(
|
||||||
image(
|
image(
|
||||||
"figs/dex_insertion.svg",
|
"figs/dex_insertion.svg",
|
||||||
width: 80%,
|
width: 80%,
|
||||||
alt: "A diagram showing a box labelled 'app.apk', a box labelled 'lib.jar', and single file ouside the boxes labelled 'lib.dex'. The lib.jar boxe contains the files classes.dex and classes2.dex. Inside the app.apk box, the files AndroidManifest.xml, resources.arsc, classes.dex, classes2.dex, classes3.dex and the folders lib, res and assets are circled by dashes and labelled 'original files', and, still inside app.apk, the files classes4.dex, classes5.dex and classes5.dex are circled by dashes and labelled 'Added Files'. Arrows go from lib.dex to classes4.dex, from the classes.dex inside lib.jar to classes5.dex inside app.apk and from classe2.dex inside lib.jar to classes6.dex inside app.apk"
|
alt: "A diagram showing a box labelled 'app.apk', a box labelled 'lib.jar', and a single file outside the boxes labelled 'lib.dex'. The lib.jar box contains the files classes.dex and classes2.dex. Inside the app.apk box, the files AndroidManifest.xml, resources.arsc, classes.dex, classes2.dex, classes3.dex and the folders lib, res and assets are circled by dashes and labelled 'original files', and, still inside app.apk, the files classes4.dex, classes5.dex and classes5.dex are circled by dashes and labelled 'Added Files'. Arrows go from lib.dex to classes4.dex, from the classes.dex inside lib.jar to classes5.dex inside app.apk and from classes2.dex inside lib.jar to classes6.dex inside app.apk"
|
||||||
),
|
),
|
||||||
caption: [Inserting #DEX files inside an #APK]
|
caption: [Inserting #DEX files inside an #APK]
|
||||||
) <fig:th-inserting-dex>
|
) <fig:th-inserting-dex>
|
||||||
|
|
||||||
In the end, we decided to *not* modify the original code that load the bytecode.
|
In the end, we decided to *not* modify the original code that loads the bytecode.
|
||||||
Statically, we already added the bytecode loaded dynamically, and most tools already ignore dynamic code loading.
|
We already added the bytecode loaded dynamically, and most tools already ignore dynamic code loading.
|
||||||
At runtime, although the bytecode is already present in the application, the application will still dynamically load the code.
|
At runtime, although the bytecode is already present in the application, the application will still dynamically load the code.
|
||||||
This ensure that the application keep working as intended even if the transformation we applied are incomplete.
|
This ensures that the application keeps working as intended, even if the transformation we applied is incomplete.
|
||||||
Specifically, to call dynamically loaded code, an application needs to use reflection, and we saw in @sec:th-trans-ref that we need to keep reflection calls, and in order to keep reflection calls, we need the classloader created when loading bytecode.
|
Specifically, to call dynamically loaded code, an application needs to use reflection, and we saw in @sec:th-trans-ref that we need to keep reflection calls, and in order to keep reflection calls, we need the classloader created when loading bytecode.
|
||||||
|
|
||||||
=== Class Collisions <sec:th-class-collision>
|
=== Class Collisions <sec:th-class-collision>
|
||||||
|
@ -174,22 +173,22 @@ Specifically, to call dynamically loaded code, an application needs to use refle
|
||||||
We saw in @sec:cl/*-obfuscation*/ that having several classes with the same name in the same application can be problematic.
|
We saw in @sec:cl/*-obfuscation*/ that having several classes with the same name in the same application can be problematic.
|
||||||
In @sec:th-trans-cl, we are adding new code.
|
In @sec:th-trans-cl, we are adding new code.
|
||||||
By doing so, we increase the probability of having class collisions:
|
By doing so, we increase the probability of having class collisions:
|
||||||
The developper may have reuse a helper class in both the dynamically loaded bytecode and the application, or an obfuscation process may have rename classes without checking for intersection between the two sources of bytecode.
|
The developer may have reused a helper class in both the dynamically loaded bytecode and the application, or an obfuscation process may have renamed classes without checking for intersection between the two sources of bytecode.
|
||||||
When loaded dynamically, the classes are in a different classloader, and the class resolution is resolved at runtime like we saw in @sec:cl-loading.
|
When loaded dynamically, the classes are in a different classloader, and the class resolution is resolved at runtime, like we saw in @sec:cl-loading.
|
||||||
We decided to restrain our scope to the use of class loader from the Android #SDK.
|
We decided to restrain our scope to the use of class loaders from the Android #SDK.
|
||||||
In the abscence of class collision, those class loader behave seamlessly and adding the classes to application maintains the behavior.
|
In the absence of class collision, those class loaders behave seamlessly and adding the classes to the application maintains the behaviour.
|
||||||
|
|
||||||
When we detect a collision, we rename one of the colliding classes in order to be able to differenciate both classes.
|
When we detect a collision, we rename one of the colliding classes in order to be able to differentiate between classes.
|
||||||
To avoid breaking the application, we then need to rename all references to this specific class, an be carefull not to modify references to the other class.
|
To avoid breaking the application, we then need to rename all references to this specific class and be careful not to modify references to the other class.
|
||||||
To do so, we regroup each classes by the classloaders defining them.
|
To do so, we regroup each class by the classloaders defining them.
|
||||||
Then, for each colliding class name and each classloader, we check the actual class used by the classloader.
|
Then, for each colliding class name and each classloader, we check the actual class used by the classloader.
|
||||||
If the class has been renamed, we rename all reference to this class in the classes defined by this classloader.
|
If the class has been renamed, we rename all references to this class in the classes defined by this classloader.
|
||||||
To find the class used by a classloader, we reproduce the behavior of the different classloaders of the Android #SDK.
|
To find the class used by a classloader, we reproduce the behaviour of the different classloaders of the Android #SDK.
|
||||||
This is an important step: remember that the delegation process can lead to situation where the class defined by a classloader is not the class that will be loaded when querying the classloader.
|
This is an important step: remember that the delegation process can lead to situations where the class defined by a classloader is not the class that will be loaded when querying the classloader.
|
||||||
The pseudo-code in @lst:renaming-algo show the three steps of this algorithm:
|
The pseudo-code in @lst:renaming-algo shows the three steps of this algorithm:
|
||||||
- First we detect collision and rename classes definitions to remove the collisions.
|
- First, we detect collisions and rename class definitions to remove the collisions.
|
||||||
- Then we rename the reference to the colliding classes to make sure the right classes are called.
|
- Then we rename the reference to the colliding classes to make sure the right classes are called.
|
||||||
- Ultimately, we merge the modified dexfiles of each class loaders into one android application.
|
- Ultimately, we merge the modified #DEX files of each class loader into one Android application.
|
||||||
|
|
||||||
#figure(
|
#figure(
|
||||||
```python
|
```python
|
||||||
|
@ -226,4 +225,4 @@ The pseudo-code in @lst:renaming-algo show the three steps of this algorithm:
|
||||||
#v(2em)
|
#v(2em)
|
||||||
|
|
||||||
Now that we saw the transformations we want to make, we know the runtime information we need to do it.
|
Now that we saw the transformations we want to make, we know the runtime information we need to do it.
|
||||||
In the next section, we will propose a solution to collect those informations.
|
In the next section, we will propose a solution to collect that information.
|
||||||
|
|
|
@ -2,81 +2,82 @@
|
||||||
|
|
||||||
== Collecting Runtime Information <sec:th-dyn>
|
== Collecting Runtime Information <sec:th-dyn>
|
||||||
|
|
||||||
To perform the transformations discribed in @sec:th-trans, we need information like the name and signature of the method called with reflection, or the actual bytecode loaded dynamically.
|
To perform the transformations described in @sec:th-trans, we need information like the name and signature of the method called with reflection, or the actual bytecode loaded dynamically.
|
||||||
We decided to collet those information through dynamic analysis.
|
We decided to collect that information through dynamic analysis.
|
||||||
We saw in @sec:bg different contributions that collect this kind of information.
|
We saw in @sec:bg different contributions that collect this kind of information.
|
||||||
In the end, we decided to keep the analysis as simple as possible, so we avoided using a custom Android build like DexHunter, and instead use Frida (see @sec:bg-frida) to instrument the application and intercept calls of the methods of interest.
|
In the end, we decided to keep the analysis as simple as possible, so we avoided using a custom Android build like DexHunter, and instead used Frida (see @sec:bg-frida) to instrument the application and intercept calls of the methods of interest.
|
||||||
@sec:th-fr-dcl present our approach to collect dynamically loaded bytecode, and @sec:th-fr-ref present our approach to collect the reflection data.
|
@sec:th-fr-dcl present our approach to collect dynamically loaded bytecode, and @sec:th-fr-ref present our approach to collect the reflection data.
|
||||||
Because using dynamic analysis raise the concern of coverage, we also need some interaction with application during the analysis.
|
Because using dynamic analysis raises the concern of coverage, we also need some interaction with the application during the analysis.
|
||||||
Ideally, a reverse engineer would do the interaction.
|
Ideally, a reverse engineer would do the interaction.
|
||||||
Because we wanted to analyse many applications in a reasonable time, we replaced this engineer by an automated runner that simulates the interactions.
|
Because we wanted to analyse many applications in a reasonable time, we replaced this engineer with an automated runner that simulates the interactions.
|
||||||
We discuss this option in @sec:th-grod.
|
We discuss this option in @sec:th-grod.
|
||||||
|
|
||||||
=== Collecting Bytecode Dynamically Loaded <sec:th-fr-dcl>
|
=== Collecting Bytecode Dynamically Loaded <sec:th-fr-dcl>
|
||||||
|
|
||||||
Initially, we considered instrumenting the constructor methods of the classloaders of the Android #SDK.
|
Initially, we considered instrumenting the constructor methods of the classloaders of the Android #SDK.
|
||||||
However, this is a significant number of methods to instrument, and looking at older application, we realized that we missed the `DexFile` class.
|
However, this is a significant number of methods to instrument, and looking at older applications, we realised that we missed the `DexFile` class.
|
||||||
`DexFile` is a now deprecated but still usable class that can be used to load bytecode dynamically.
|
`DexFile` is now deprecated but still usable class that can be used to load bytecode dynamically.
|
||||||
We initially missed this class because it is neither a `ClassLoader` class nor an #SDK class (anymore).
|
We initially missed this class because it is neither a `ClassLoader` class nor an #SDK class (anymore).
|
||||||
To avoid running into this kind of oversight again, we decided to look at the #ART source code an list all the places where the internal function used to parse bytecode are called.
|
To avoid running into this kind of oversight again, we decided to look at the #ART source code and list all the places where the internal functions used to parse bytecode are called.
|
||||||
We found that all those calls are from under either `DexFile.openInMemoryDexFilesNative(..)` or `DexFile.openDexFileNative(..)`, two hidden #API methods.
|
We found that all those calls are from under either `DexFile.openInMemoryDexFilesNative(..)` or `DexFile.openDexFileNative(..)`, two hidden #API methods.
|
||||||
As a reference, in 2015, DexHunter~@zhang2015dexhunter already noticed `DexFile.openDexFileNative(..)` (although in the end DexHunter intrument another function, `DefineClass(..)`).
|
As a reference, in 2015, DexHunter~@zhang2015dexhunter already noticed `DexFile.openDexFileNative(..)` (although in the end DexHunter intruments another function, `DefineClass(..)`).
|
||||||
`DefineClass(..)` is still a good function to instrument, but it is a C++ native method that does not have a Java interface, making it harder to work with using Frida, and we want to avoid patching the source code of the #ART like DexHunter did.
|
`DefineClass(..)` is still a good function to instrument, but it is a C++ native method that does not have a Java interface, making it harder to work with using Frida, and we want to avoid patching the source code of the #ART like DexHunter did.
|
||||||
For this reason, we decided to hook `DexFile.openInMemoryDexFilesNative(..)` and `DexFile.openDexFileNative(..)` instead.
|
For this reason, we decided to hook `DexFile.openInMemoryDexFilesNative(..)` and `DexFile.openDexFileNative(..)` instead.
|
||||||
Those methods takes as argument a list of Androis code files, either in the form of in memory byte arrays or file path, and a reference to the classloader associated to the code.
|
Those methods take as argument a list of Android code files, either in the form of in-memory byte arrays or file paths, and a reference to the classloader associated with the code.
|
||||||
Instrumenting those methods allows us to collect all the code files loaded by the #ART and associate them to their classloaders.
|
Instrumenting those methods allows us to collect all the code files loaded by the #ART and associate them with their classloaders.
|
||||||
|
|
||||||
=== Collecting Reflection Data <sec:th-fr-ref>
|
=== Collecting Reflection Data <sec:th-fr-ref>
|
||||||
|
|
||||||
As described in @sec:th-trans-ref, they are 3 methods that we need to instrument to capture reflection calls: `Class.newInstance()`, `Constructor.newInstance(..)` and `Method.invoke(..)`.
|
As described in @sec:th-trans-ref, there are 3 methods that we need to instrument to capture reflection calls: `Class.newInstance()`, `Constructor.newInstance(..)` and `Method.invoke(..)`.
|
||||||
Because Java has polymorphism, we need not only the method name and defining class, but also the whole signature of the method.
|
Because Java has polymorphism, we need not only the method name and defining class, but also the whole signature of the method.
|
||||||
In addition to that, in case there are several classes with the same name as the defining class, we also need the classloader of the defining class to distinguish it from the other classes.
|
In addition to that, in case there are several classes with the same name as the defining class, we also need the classloader of the defining class to distinguish it from the other classes.
|
||||||
|
|
||||||
_Where_ the reflection method is called is more difficult to find.
|
_Where_ the reflection method is called is more difficult to find.
|
||||||
In order to correctly modify the application, we need to know which specific call to a reflection method we intercepted.
|
In order to correctly modify the application, we need to know which specific call to a reflection method we intercepted.
|
||||||
Specifically, we need to known the caller method (once again, we need the method name, full signature, defining class and its classloader), and the speficic instruction that called the reflection method (in case the caller method call a reflection method several times).
|
Specifically, we need the caller method (once again, we need the method name, full signature, defining class and its classloader), and the exact instruction that called the reflection method (in case the caller method uses reflection several times in different sites).
|
||||||
This information is more difficult to collect than one would expect.
|
This information is more difficult to collect than one would expect.
|
||||||
Those information are stored in the stack, but before the #SDK 34, the stack was not directly accessible programmatically.
|
It is stored in the stack, but before the #SDK 34, the stack was not directly accessible programmatically.
|
||||||
Historically, when a reverse engineer needed to access the stack, they would trigger and catch an exception, get the stack from the exception.
|
Historically, when a reverse engineer needed to access the stack, they would trigger and catch an exception and get the stack from that exception.
|
||||||
The issue with this approach is that the data stored in the exception are meant for debbuging.
|
The issue with this approach is that data stored in exceptions is meant for debugging.
|
||||||
In particullar, the location of the call in the bytecode has a different meaning depending on the debug information encoded in the bytecode.
|
In particullar, the location of the call in the bytecode has a different meaning depending on the debug information encoded in the bytecode.
|
||||||
It can either be the address of the bytecode instruction invoking the callee method in the instruction array of the caller method, or the line number of original source code that call the callee method.
|
It can either be the address of the bytecode instruction invoking the callee method in the instruction array of the caller method, or the line number of the original source code that calls the callee method.
|
||||||
Fortunatelly, in the #SDK 34, Android introduced the `StackWalker` #API.
|
Fortunately, in the #SDK 34, Android introduced the `StackWalker` #API.
|
||||||
This #API allow to programatically travel the current stack and retrieve informations from it, including the bytecode address of the instruction calling the callee methods.
|
This #API allow to programatically travel the current stack and retrieve information from it, including the bytecode address of the instruction calling the callee methods.
|
||||||
Considering that the line number is not a reliable information, we chose to use the new #API, despite the restriction that come with chosing such a recent Android version (it was released in october 2023, arround 2 years ago, and less than 50% of the current Android market share support this #API today#footnote[https://gs.statcounter.com/android-version-market-share/mobile-tablet/worldwide/#monthly-202401-202508]).
|
Considering that the line number is not a reliable information, we chose to use the new #API, despite the restrictions that come with choosing such a recent Android version (it was released in October 2023, around 2 years ago, and less than 50% of the current Android market share supports this #API today#footnote[https://gs.statcounter.com/android-version-market-share/mobile-tablet/worldwide/#monthly-202401-202508]).
|
||||||
|
|
||||||
=== Application Execution <sec:th-grod>
|
=== Application Execution <sec:th-grod>
|
||||||
|
|
||||||
Dynamic analysis requires actually running the application.
|
Dynamic analysis requires actually running the application.
|
||||||
In order to test automatically multiple applications, we needed to simulate human interractions with the applications.
|
In order to test multiple applications automatically, we needed to simulate human interactions with the applications.
|
||||||
In @sec:bg we presented a few solution to explore an application dynamically.
|
In @sec:bg, we presented a few solutions to explore an application dynamically.
|
||||||
We first eliminated Sapienz, as it rely on an application instrumentation library called ELLA, that has not be updated since 9 years ago.
|
We first eliminated Sapienz, as it relies on an application instrumentation library called ELLA, which has not been updated for 9 years.
|
||||||
We also chose to avoid the Monkey because we noticed that it will often trigger event that will close the application (events likes pressing the 'home' button, or openning the general setting drop-down menu at the top of the screen).
|
We also chose to avoid the Monkey because we noticed that it often triggers events that close the application (events like pressing the 'home' button, or opening the general settings drop-down menu at the top of the screen).
|
||||||
Stoat and GroddDroid use UI Automator to interact with the application.
|
Stoat and GroddDroid use UI Automator to interact with the application.
|
||||||
UI Automator is a standard Android #API inteded for automatic testing.
|
UI Automator is a standard Android #API intended for automatic testing.
|
||||||
Both Soat and GroddDroid perfom additionnal analysis on the application to improve the exploration.
|
Both Soat and GroddDroid perform additional analysis on the application to improve the exploration.
|
||||||
In the end, we elected to use the most basic execution mode of GroddDroid that does not need this additionnal analysis.
|
In the end, we elected to use the most basic execution mode of GroddDroid that does not need this additional analysis.
|
||||||
It explore the application following a depth-first search algorithm.
|
It explores the application following a depth-first search algorithm.
|
||||||
We chose this option to keep the exploiration lightwight and limit the chance of crashing the analysis (we saw in @sec:rasta the issues brought by complexe analysis).
|
We chose this option to keep the exploration lightweight and limit the chance of crashing the analysis (we saw in @sec:rasta the issues brought by complex analysis).
|
||||||
It might be interesting in futur work to explore more complexe exploration techniques.
|
It might be interesting in future work to explore more advanced exploration techniques.
|
||||||
|
|
||||||
Because we are using Frida, we do not need to use a custom version of Android with a modified #ART or kernel like, however, we decided to not inject Frida in the original application.
|
Because we are using Frida, we do not need to use a custom version of Android with a modified #ART or kernel.
|
||||||
This means we need to have root access to directly run Frida in Android which is not a normal thing to have on Android.
|
However, we decided not to inject Frida into the original application.
|
||||||
|
This means we need to have root access to directly run Frida in Android, which is not a normal thing to have on Android.
|
||||||
Because dynamic analysis can be slow, we also decided to run the applications on emulators.
|
Because dynamic analysis can be slow, we also decided to run the applications on emulators.
|
||||||
This makes its easier to run several analysis in parallel.
|
This makes it easier to run several analyses in parallel.
|
||||||
The alternative would have been to run the application on actual smartphones, and would have required multiple phones to run the analysis in parallel.
|
The alternative would have been to run the application on actual smartphones, and would have required multiple phones to run the analysis in parallel.
|
||||||
For simplicity, we choosed to use Google Android emulator for our experiment.
|
For simplicity, we chose to use Google's Android emulator for our experiment.
|
||||||
We spawned multiple emulators, installed Frida on it, took a snapshot of the emulator before installing the application to analyse.
|
We spawned multiple emulators, installed Frida on them, took a snapshot of the emulator before installing the application to analyse.
|
||||||
Then we run the application for a five minutes with GroddRunner, and at the end of the analysis, we reload the snapshot in case the application modified the system in some unforseen way.
|
Then we run the application for five minutes with GroddRunner, and at the end of the analysis, we reload the snapshot in case the application modified the system in some unforeseen way.
|
||||||
If at some point the emulator start responding for too long, we terminate it and restart it.
|
If at some point an emulator stops responding for too long, we terminate it and restart it.
|
||||||
|
|
||||||
As we will see in @sec:th-dyn-failure, our experimental setup is quite naive and still requires improvement. #todo(strike(stroke: green)[Comment on dit proprement que c'est tout pété?])
|
As we will see in @sec:th-dyn-failure, our experimental setup is quite naive and still requires improvement. #todo(strike(stroke: green)[Comment on dit proprement que c'est tout pété?])
|
||||||
For example, it does not implement any anti-evasion techniques, which can be a significant issue when analysing malware.
|
For example, it does not implement any anti-evasion techniques, which can be a significant issue when analysing malware.
|
||||||
Nonetheless, the benefit of our implementation is that it only requires a #ADB connection to a phone with a rooted Android system to work.
|
Nonetheless, the benefit of our implementation is that it only requires an #ADB connection to a phone with a rooted Android system to work.
|
||||||
Of course, to analyse a specific application, a reverse engineer could use an actual smartphone and explore the application manually.
|
Of course, to analyse a specific application, a reverse engineer could use an actual smartphone and explore the application manually.
|
||||||
It would be a lot more stable than our automated batch analysis setup.
|
It would be a lot more stable than our automated batch analysis setup.
|
||||||
|
|
||||||
#v(2em)
|
#v(2em)
|
||||||
|
|
||||||
Now that we have both saw both the dynamic analysis setup and the transformation we want to perform on the #APKs, we put our proposed approach into practice.
|
Now that we saw both the dynamic analysis setup and the transformation we want to perform on the #APKs, we put our proposed approach into practice.
|
||||||
In the next section, we will run our dynamic analysis on #APKs and studdy the look at the data collected as well a the impact the instrumentation has on appications and different analysis tools.
|
In the next section, we will run our dynamic analysis on #APKs and study the data collected, as well as the impact the instrumentation has on applications and different analysis tools.
|
||||||
|
|
|
@ -9,35 +9,35 @@
|
||||||
|
|
||||||
#todo[better section name for @sec:th-res]
|
#todo[better section name for @sec:th-res]
|
||||||
|
|
||||||
To studdy the impact of our transformation on analysis tools, we reused applications from the dataset we sampled in @sec:rasta/*-dataset*/.
|
To study the impact of our transformation on analysis tools, we reused applications from the dataset we sampled in @sec:rasta/*-dataset*/.
|
||||||
Because we are running the application on a rescent version of Android (#SDK 34), we only took the most recent applications: the one collected in 2023.
|
Because we are running the application on a recent version of Android (#SDK 34), we only took the most recent applications: the one collected in 2023.
|
||||||
This represent #num(5000) applications over the #NBTOTALSTRING total of the initial dataset.
|
This represents #num(5000) applications over the #NBTOTALSTRING total of the initial dataset.
|
||||||
Among them, we could not retrieve 43 from Androzoo, leaving us with #num(dyn_res.all.nb) applications to test.
|
Among them, we could not retrieve 43 from Androzoo, leaving us with #num(dyn_res.all.nb) applications to test.
|
||||||
|
|
||||||
We will first look at the results of the dynamic analysis and look closer at the bytecode we intercepted.
|
We will first look at the results of the dynamic analysis and look at the bytecode we intercepted.
|
||||||
Then, we will studdy the impact the instrumentation have on static analysis tools, notably on their success rate, and we will finish with the analysis of an handcrafted application to check the instrumentation does in fact improve the results of analysis tools.
|
Then, we will study the impact the instrumentation has on static analysis tools, notably on their success rate, and we will finish with the analysis of a handcrafted application to check whether the instrumentation does, in fact, improve the results of analysis tools.
|
||||||
|
|
||||||
=== Dynamic Analysis Results <sec:th-dyn-failure>
|
=== Dynamic Analysis Results <sec:th-dyn-failure>
|
||||||
|
|
||||||
After running the dynamic analysis on our dataset the first time we realised our dynamic setup was quite fragile.
|
After running the dynamic analysis on our dataset the first time, we realised our dynamic setup was quite fragile.
|
||||||
We found that #mypercent(dyn_res.all.nb_failed_first_run, dyn_res.all.nb) of the execution failed with various errors.
|
We found that #mypercent(dyn_res.all.nb_failed_first_run, dyn_res.all.nb) of the executions failed with various errors.
|
||||||
The majority of those errors were related to faillures to connect to the Frida agent or start the activity from Frida.
|
The majority of those errors were related to failures to connect to the Frida agent or start the activity from Frida.
|
||||||
Some of those errors seamed to come from Frida, while other seamed related to the emulator failing to start the application.
|
Some of those errors seemed to come from Frida, while others seemed related to the emulator failing to start the application.
|
||||||
We found that relaunching the analysis for the applications that failled was the most simple way to fix those issues, and after 6 passes we went from #num(dyn_res.all.nb_failed_first_run) to #num(dyn_res.all.nb_failed) application that could not be analysed.
|
We found that relaunching the analysis for the applications that failed was the simplest way to fix those issues, and after 6 passes, we went from #num(dyn_res.all.nb_failed_first_run) to #num(dyn_res.all.nb_failed) applications that could not be analysed.
|
||||||
The remaining errors look more related to the application itself or Android, with #num(96) errors being a failure to install the application, and #num(110) other beeing a null pointer exception from Frida.
|
The remaining errors look more related to the application itself or Android, with #num(96) errors being a failure to install the application, and #num(110) others being a null pointer exception from Frida.
|
||||||
|
|
||||||
Infortunatly, although we managed to start the applications, we can see from the list of activity visited by GroddDroid that a majority (#mypercent(dyn_res.all.z_act_visited, dyn_res.all.nb - dyn_res.all.nb_failed)) of the application stopped before even starting one activity.
|
Unfortunately, although we managed to start the applications, we can see from the list of activities visited by GroddDroid that a majority (#mypercent(dyn_res.all.z_act_visited, dyn_res.all.nb - dyn_res.all.nb_failed)) of the applications stopped before even starting one activity.
|
||||||
Some applications do not have an activity, and are not intended to interact with a user, but those are clearly a minority and do not explain such a high number.
|
Some applications do not have any activities and are not intended to interact with a user, but those are clearly a minority and do not explain such a high number.
|
||||||
We expected some issue related to the use of an emulator, like the lack of x86_64 library in the applications, or contermesures aborting the application if the emulator is detected.
|
We expected some issues related to the use of an emulator, like the lack of x86_64 library in the applications, or contermesures aborting the application if the emulator is detected.
|
||||||
We manually looked at some applications, but did not found a notable pattern.
|
We manually looked at some applications, but did not find a notable pattern.
|
||||||
In some cases, the application was just broken -- for instance, an application was trying to load a native library that simply does not exists in the application.
|
In some cases, the application was just broken -- for instance, an application was trying to load a native library that simply does not exist in the application.
|
||||||
In other case, Frida is to blame: we found some cases where calling a method from Frida can confuse the #ART.
|
In other cases, Frida is to blame: we found some cases where calling a method from Frida can confuse the #ART.
|
||||||
`protected` methods needs to be called from the class that defined the method or one of its children calsses, but Frida might be considered by the #ART as an other class, leading to the #ART aborting the application.
|
`protected` methods need to be called from the class that defined the method or one of its child classes, but Frida might be considered by the #ART as another class, leading to the #ART aborting the application.
|
||||||
#todo[jfl was suppose to test a few other app #emoji.eyes]
|
#todo[jfl was suppose to test a few other app #emoji.eyes]
|
||||||
@tab:th-dyn-visited shows the number of applications that we analysed, if we managed to start at least one activity and if we intercepted code loading or reflection.
|
@tab:th-dyn-visited shows the number of applications that we analysed, if we managed to start at least one activity and if we intercepted code loading or reflection.
|
||||||
It also shows the average number of activities visited (when at least one activity was started).
|
It also shows the average number of activities visited (when at least one activity was started).
|
||||||
This average slightly higher than 1, which seems reasonable: a lot of applications do not need more than one activity, but some do and we did manage to explore at least some of those additionnal activities.
|
This average is slightly higher than 1, which seems reasonable: a lot of applications do not need more than one activity, but some do, and we did manage to explore at least some of those additional activities.
|
||||||
As shown in the table, even if the application fails to start an activity, some times it will still load external code or use reflection.
|
As shown in the table, even if the application fails to start an activity, sometimes it will still load external code or use reflection.
|
||||||
|
|
||||||
#figure({
|
#figure({
|
||||||
let nb_col = 7
|
let nb_col = 7
|
||||||
|
@ -76,21 +76,21 @@ As shown in the table, even if the application fails to start an activity, some
|
||||||
caption: [Summary of the dynamic exploration of the applications from the RASTA dataset collected by Androzoo in 2023]
|
caption: [Summary of the dynamic exploration of the applications from the RASTA dataset collected by Androzoo in 2023]
|
||||||
) <tab:th-dyn-visited>
|
) <tab:th-dyn-visited>
|
||||||
|
|
||||||
The high number of application that did not start an activity means that our result will be highly biaised.
|
The high number of applications that did not start an activity means that our results will be highly biased.
|
||||||
The code that might be loaded or method that might be called by reflection from inside activities is filtered out by the limit of or dynamic execution.
|
The code/method that might be loaded/called by reflection from inside activities is filtered out by the limit of or dynamic execution.
|
||||||
This biaised must be kept in mind when reading the next subsection that studdy the bytecode that we intercepted.
|
This bias must be kept in mind while reading the next subsection that studies the bytecode that we intercepted.
|
||||||
|
|
||||||
=== The Bytecode Loaded by Application <sec:th-code-collected>
|
=== The Bytecode Loaded by Application <sec:th-code-collected>
|
||||||
|
|
||||||
We collected a total of #nb_bytecode_collected files for #dyn_res.code_loading.nb application that we detected loading bytecode dynamicatlly.
|
We collected a total of #nb_bytecode_collected files for #dyn_res.code_loading.nb application that we detected loading bytecode dynamically.
|
||||||
#num(92) of them were loaded by a `DexClassLoader`, #num(547) were loaded by a `InMemoryDexClassLoader` and #num(1) was loaded by a `PathClassLoader`.
|
#num(92) of them were loaded by a `DexClassLoader`, #num(547) were loaded by a `InMemoryDexClassLoader`, and #num(1) was loaded by a `PathClassLoader`.
|
||||||
|
|
||||||
Once we compared the files, we found that we only collected #num(bytecode_hashes.len()) distinct files, and that #num(bytecode_hashes.at(0).at(0)) where identicals.
|
Once we compared the files, we found that we only collected #num(bytecode_hashes.len()) distinct files, and that #num(bytecode_hashes.at(0).at(0)) were identical.
|
||||||
Once we looked more in details, we found that most of those files are advertisement libraries.
|
Once we looked more in detail, we found that most of those files are advertisement libraries.
|
||||||
In total, we collected #num(nb_google) files containing Google ads librairies and #num(nb_facebook) files containing Facebook ads librairies.
|
In total, we collected #num(nb_google) files containing Google ads libraries and #num(nb_facebook) files containing Facebook ads libraries.
|
||||||
In addition, we found #num(nb_appsflyer) files containing code that we believe to be AppsFlyer, and company that provides "measurement, analytics, engagement, and fraud protection technologies".
|
In addition, we found #num(nb_appsflyer) files containing code that we believe to be AppsFlyer, a company that provides "measurement, analytics, engagement, and fraud protection technologies".
|
||||||
The remaining #num(nb_bytecode_collected - nb_google - nb_appsflyer - nb_facebook) files were custom code from high security applications (#ie banking, social security)
|
The remaining #num(nb_bytecode_collected - nb_google - nb_appsflyer - nb_facebook) files were custom code from high security applications (#ie banking, social security)
|
||||||
@tab:th-bytecode-hashes sumarize the information we collected about the most common bytecode files.
|
@tab:th-bytecode-hashes summarises the information we collected about the most common bytecode files.
|
||||||
|
|
||||||
#figure(
|
#figure(
|
||||||
table(
|
table(
|
||||||
|
@ -115,18 +115,18 @@ The remaining #num(nb_bytecode_collected - nb_google - nb_appsflyer - nb_faceboo
|
||||||
|
|
||||||
We took the applications associated with the #num(nb_bytecode_collected - nb_google - nb_appsflyer - nb_facebook) unique #DEX files we found to see the impact of our transformation.
|
We took the applications associated with the #num(nb_bytecode_collected - nb_google - nb_appsflyer - nb_facebook) unique #DEX files we found to see the impact of our transformation.
|
||||||
|
|
||||||
The applications where indeed obfuscated, making a manual analysis tedious.
|
The applications were indeed obfuscated, making a manual analysis tedious.
|
||||||
We did not found visible #DEX files or #APK files inside the applications, meaning the applications are either downloading or generating them from variables and assets at runtime.
|
We did not find visible #DEX files or #APK files inside the applications, meaning the applications are either downloading or generating them from variables and assets at runtime.
|
||||||
To estimate the scope of the code we made available, we use Androguard to generate the call graph of the applications, before and after the instrumentation.
|
To estimate the scope of the code we made available, we use Androguard to generate the call graph of the applications, before and after the instrumentation.
|
||||||
@tab:th-compare-cg shows the number of edges of those call graphs.
|
@tab:th-compare-cg shows the number of edges of those call graphs.
|
||||||
The columns before and after shows the total number of edges of the graphs, and the diff columns is the number of new edges detected (#ie the number of edges after instrumentation minus the number of edges before).
|
The columns before and after show the total number of edges of the graphs, and the diff column indicates the number of new edges detected (#ie the number of edges after instrumentation minus the number of edges before).
|
||||||
This number include edges from the bytecode loaded dynamically, as well as the call added to reflect reflection calls, and calls to "glue" methods (method like `Integer.intValue()` used to convert objects to scalar values, or calls to `T.check_is_Xxx_xxx(Method)` used to check if a `Method` object represent a known method).
|
This number include edges from the bytecode loaded dynamically, as well as the call added to reflect reflection calls, and calls to "glue" methods (method like `Integer.intValue()` used to convert objects to scalar values, or calls to `T.check_is_Xxx_xxx(Method)` used to check if a `Method` object represent a known method).
|
||||||
The last column, "Added Reflection", is the list of non-glue method calls found in the call graph of the instrumented application but neither in call graph of the original #APK, nor in the call graphes of the added bytecode files that we computed separately.
|
The last column, "Added Reflection", is the list of non-glue method calls found in the call graph of the instrumented application but neither in call graph of the original #APK, nor in the call graphes of the added bytecode files that we computed separately.
|
||||||
This correspond to the calls we added to represent reflection calls.
|
This corresponds to the calls we added to represent reflection calls.
|
||||||
|
|
||||||
The first application, #lower(compared_callgraph.at(0).sha256), is noticable.
|
The first application, #lower(compared_callgraph.at(0).sha256), is noticable.
|
||||||
The instrumented #APK has ten times more edges to its call graph than the original, and only one reflection call.
|
The instrumented #APK has ten times more edges to its call graph than the original, and only one reflection call.
|
||||||
This is consistant with the behaviour of a packer: the application load the main of its code at runtime and switch from the bootstrap code to the loaded code with a single reflection call.
|
This is consistent with the behaviour of a packer: the application loads the main part of its code at runtime and switches from the bootstrap code to the loaded code with a single reflection call.
|
||||||
|
|
||||||
#figure({
|
#figure({
|
||||||
let nb_col = 5
|
let nb_col = 5
|
||||||
|
@ -152,27 +152,27 @@ This is consistant with the behaviour of a packer: the application load the main
|
||||||
[#lower("5D2CD1D10ABE9B1E8D93C4C339A6B4E3D75895DE1FC49E248248B5F0B05EF1CE").slice(0, 10)...], table.cell(colspan: nb_col - 1)[_Instrumentation Crashed_],
|
[#lower("5D2CD1D10ABE9B1E8D93C4C339A6B4E3D75895DE1FC49E248248B5F0B05EF1CE").slice(0, 10)...], table.cell(colspan: nb_col - 1)[_Instrumentation Crashed_],
|
||||||
table.hline(),
|
table.hline(),
|
||||||
)},
|
)},
|
||||||
caption: [Edges added to the call graphes computed by Androguard by instrumenting the applications]
|
caption: [Edges added to the call graphs computed by Androguard by instrumenting the applications]
|
||||||
) <tab:th-compare-cg>
|
) <tab:th-compare-cg>
|
||||||
|
|
||||||
Unfortunately, our implementation of the transformation is imperfect and does fails some time, as illustrated by #lower("5D2CD1D10ABE9B1E8D93C4C339A6B4E3D75895DE1FC49E248248B5F0B05EF1CE") in @tab:th-compare-cg.
|
Unfortunately, our implementation of the transformation is imperfect and does fails sometime, as illustrated by #lower("5D2CD1D10ABE9B1E8D93C4C339A6B4E3D75895DE1FC49E248248B5F0B05EF1CE") in @tab:th-compare-cg.
|
||||||
However, over the #num(dyn_res.all.nb - dyn_res.all.nb_failed) applications whose dynamic analysis finished in our experiment, #num(nb_patched) were patched.
|
However, over the #num(dyn_res.all.nb - dyn_res.all.nb_failed) applications whose dynamic analysis finished in our experiment, #num(nb_patched) were patched.
|
||||||
The remaining #mypercent(dyn_res.all.nb - dyn_res.all.nb_failed - nb_patched, dyn_res.all.nb - dyn_res.all.nb_failed) failed either due to some quirk in the zip format of the #APK file, because of a bug in our implementation when exceeding the method reference limit in a single #DEX file, or in the case of #lower("5D2CD1D10ABE9B1E8D93C4C339A6B4E3D75895DE1FC49E248248B5F0B05EF1CE"), because the application reused the original application classloader to load new code instead of instanciated a new classes loader (a behavior we did not expected as not possible using only the #SDK, but enabled by hidden #APIs).
|
The remaining #mypercent(dyn_res.all.nb - dyn_res.all.nb_failed - nb_patched, dyn_res.all.nb - dyn_res.all.nb_failed) failed either due to some quirk in the zip format of the #APK file, because of a bug in our implementation when exceeding the method reference limit in a single #DEX file, or in the case of #lower("5D2CD1D10ABE9B1E8D93C4C339A6B4E3D75895DE1FC49E248248B5F0B05EF1CE"), because the application reused the original application classloader to load new code instead of instanciated a new classes loader (a behavior we did not expected as not possible using only the #SDK, but enabled by hidden #APIs).
|
||||||
Taking into accound the failure from both dynamic analysis and the instrumentation process, we have a #mypercent(dyn_res.all.nb - nb_patched, dyn_res.all.nb) failure rate.
|
Taking into account the failure from both dynamic analysis and the instrumentation process, we have a #mypercent(dyn_res.all.nb - nb_patched, dyn_res.all.nb) failure rate.
|
||||||
This is a reasonable failure rate, but we should keep in mind that it adds up to the failure rate of the other tools we want to use on the patched application.
|
This is a reasonable failure rate, but we should keep in mind that it adds up to the failure rate of the other tools we want to use on the patched application.
|
||||||
|
|
||||||
To check the impact on the finishing rate of or instrumentation, we then run the same experiment we run in @sec:rasta.
|
To check the impact on the finishing rate of our instrumentation, we then run the same experiment we ran in @sec:rasta.
|
||||||
We run the tools on the #APK before and after instrumentation, and compared the finishing rates in @fig:th-status-npatched-vs-patched (without taking into account #APKs we failed to patch#footnote[Due to an handling error during the experiment, the figure show the results for #nb_patched_rasta #APKs instead of #nb_patched.]).
|
We run the tools on the #APK before and after instrumentation, and compared the finishing rates in @fig:th-status-npatched-vs-patched (without taking into account #APKs we failed to patch#footnote[Due to a handling error during the experiment, the figure shows the results for #nb_patched_rasta #APKs instead of #nb_patched.]).
|
||||||
|
|
||||||
The finishing rate comparision is shown in @fig:th-status-npatched-vs-patched.
|
The finishing rate comparison is shown in @fig:th-status-npatched-vs-patched.
|
||||||
We can see that in most cases, the finishing rate either the same, or slightly lower for the instrumented application.
|
We can see that in most cases, the finishing rate is either the same or slightly lower for the instrumented application.
|
||||||
This is consistent with the fact that we add more bytecode to the application, hence adding more oportunities of failure during analysis.
|
This is consistent with the fact that we add more bytecode to the application, hence adding more opportunities for failure during analysis.
|
||||||
They are two notable exceptions: Saaf and IC3.
|
They are two notable exceptions: Saaf and IC3.
|
||||||
The finishing rate of IC3 which was previously reasibabe drop to 0 after our instrumentation, while the finishing rate of Saaf jump to 100%, which is extremely suspicious.
|
The finishing rate of IC3, which was previously reasonable, dropped to 0 after our instrumentation, while the finishing rate of Saaf jumped to 100%, which is extremely suspicious.
|
||||||
Analysing the logs of the analysis showed that both cases have the same origin: the bytecode generated by our instrumentation has version number of 37 (the version introduced by Android 7.0).
|
Analysing the logs of the analysis showed that both cases have the same origin: the bytecode generated by our instrumentation has a version number of 37 (the version introduced by Android 7.0).
|
||||||
Infortunately, neither the version of Apktool used by Saaf nor Dare (the tool used by IC3 to convert Dalvik bytecode to Java bytecode) recognize this version of bytecode, and thus failed to parse the #APK.
|
Unfortunately, neither the version of Apktool used by Saaf nor Dare (the tool used by IC3 to convert Dalvik bytecode to Java bytecode) recognises this version of bytecode, and thus failed to parse the #APK.
|
||||||
In the case of Dare and IC3, our experiment correctly identify this a crash.
|
In the case of Dare and IC3, our experiment correctly identifies this as a crash.
|
||||||
On the other hand, Saaf do not detect the issue with Apktool and pursue the analysis with no bytecode to analyse and return a valid return file, but for an empty application.
|
On the other hand, Saaf do not detect the issue with Apktool and pursues the analysis with no bytecode to analyse and returns a valid return file, but for an empty application.
|
||||||
|
|
||||||
#todo[alt text @fig:th-status-npatched-vs-patched]
|
#todo[alt text @fig:th-status-npatched-vs-patched]
|
||||||
#figure({
|
#figure({
|
||||||
|
@ -190,11 +190,11 @@ On the other hand, Saaf do not detect the issue with Apktool and pursue the anal
|
||||||
|
|
||||||
=== Example
|
=== Example
|
||||||
|
|
||||||
In this subsection, we use on our approach on a small #APK to look in more details into analysis of the transformed application.
|
In this subsection, we use our approach on a small #APK to look in more detail into the analysis of the transformed application.
|
||||||
We handcrafted this application for the purpose of demonstrating how this can improve help a reverse engineer in its work.
|
We handcrafted this application for the purpose of demonstrating how this can help a reverse engineer in their work.
|
||||||
Accordingly, this application is quite small and contains both dynamic code loading and reflection.
|
Accordingly, this application is quite small and contains both dynamic code loading and reflection.
|
||||||
We defined a method `Utils.source()` and `Utils.sink()` to model respectively a method that collect sensitive data and that exfiltrate data.
|
We defined a method `Utils.source()` and `Utils.sink()` to model a method that collects sensitive data and a method that exfiltrates data.
|
||||||
Those methods are the one we will use with Flowdroid to track data flows.
|
Those methods are the ones we will use with Flowdroid to track data flows.
|
||||||
|
|
||||||
#figure(
|
#figure(
|
||||||
```java
|
```java
|
||||||
|
@ -223,25 +223,25 @@ public class Main {
|
||||||
...
|
...
|
||||||
}
|
}
|
||||||
```,
|
```,
|
||||||
caption: [Code of the main class of the application showed by Jadx, before patching],
|
caption: [Code of the main class of the application, as shown by Jadx, before patching],
|
||||||
)<lst:th-demo-before>
|
)<lst:th-demo-before>
|
||||||
|
|
||||||
A first analysis of the contant of the application shows that the application contains one `Activity` that instanciate the class `Main` and call `Main.main()`.
|
A first analysis of the content of the application shows that the application contains one `Activity` that instantiates the class `Main` and calls `Main.main()`.
|
||||||
@lst:th-demo-before shows the most of the code of `Main` as returned by Jadx.
|
@lst:th-demo-before shows most of the code of `Main` as returned by Jadx.
|
||||||
We can see that the class contains another #DEX file encoded in base 64 and loaded in the `InMemoryDexClassLoader` `cl`.
|
We can see that the class contains another #DEX file encoded in base 64 and loaded in the `InMemoryDexClassLoader` `cl`.
|
||||||
A class is then loaded from this class loader, and two methods from this class loader are called.
|
A class is then loaded from this class loader, and two methods from this class loader are called.
|
||||||
The names of this class and methods are not directly accessible as they have been chipĥered and are decoded just before beeing used at runtime.
|
The names of this class and methods are not directly accessible as they have been chipĥered and are decoded just before being used at runtime.
|
||||||
Here, the encryption key is available statically, and in theorie, a verry good static analyser implementing Android `Cipher` #API could compute the actual methods called.
|
Here, the encryption key is available statically, and in theory, a very good static analyser implementing Android `Cipher` #API could compute the actual methods called.
|
||||||
However, we could easily imagine an application that gets this key from a remote command and control server.
|
However, we could easily imagine an application that gets this key from a remote command and control server.
|
||||||
In this case, it would be impossible to compute those methods with static analysis alone.
|
In this case, it would be impossible to compute those methods with static analysis alone.
|
||||||
When running Flowdroid on this application, it computed a callgraph of 43 edges on this application, an no data leaks.
|
When running Flowdroid on this application, it computed a call graph of 43 edges on this application, and no data leaks.
|
||||||
This is not particularly surprising considering the obfusctation methods used.
|
This is not particularly surprising considering the obfuscation methods used.
|
||||||
|
|
||||||
Then we run the dynamic analysis we described in @sec:th-dyn on the application and apply the transformation described in @sec:th-trans to add the dynamic informations to it.
|
Then we run the dynamic analysis we described in @sec:th-dyn on the application and apply the transformation described in @sec:th-trans to add the dynamic information to it.
|
||||||
This time, Flowdroid compute a larger callgraph of 76 edges, and does find a data leak.
|
This time, Flowdroid computes a larger call graph of 76 edges, and does find a data leak.
|
||||||
Indeed, when looking at the new application with Jadx, we notice a new class `Malicious`, and the code of `Main.main()` is now as shown in @lst:th-demo-after:
|
Indeed, when looking at the new application with Jadx, we notice a new class `Malicious`, and the code of `Main.main()` is now as shown in @lst:th-demo-after:
|
||||||
the method called in the loop is either `Malicious.get_data`, `Malicious.send_data()` or `Method.invoke()`.
|
the method called in the loop is either `Malicious.get_data`, `Malicious.send_data()` or `Method.invoke()`.
|
||||||
Although self explanatory, verifying the code of those methods indeed confirms that `get_data()` calls `Utils.source()` and `send_data()` calls `Utils.sink()`.
|
Although self-explanatory, verifying the code of those methods indeed confirms that `get_data()` calls `Utils.source()` and `send_data()` calls `Utils.sink()`.
|
||||||
|
|
||||||
#figure(
|
#figure(
|
||||||
```java
|
```java
|
||||||
|
@ -260,14 +260,14 @@ Although self explanatory, verifying the code of those methods indeed confirms t
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```,
|
```,
|
||||||
caption: [Code of `Main.main()` showed by Jadx, after patching],
|
caption: [Code of `Main.main()`, as shown by Jadx, after patching],
|
||||||
)<lst:th-demo-after>
|
)<lst:th-demo-after>
|
||||||
|
|
||||||
For an higher level view of the method, we can also look at its the call graph.
|
For a higher-level view of the method, we can also look at its call graph.
|
||||||
We used Androguard to generate the call graphes in @fig:th-cg-before and @fig:th-cg-after#footnote[We manually edited the generated .dot files for readability.].
|
We used Androguard to generate the call graphs in @fig:th-cg-before and @fig:th-cg-after#footnote[We manually edited the generated .dot files for readability.].
|
||||||
@fig:th-cg-before show the original call graph, and gives a good idea of the obfuscation methods used: we can see calls to `Main.decrypt(String)` that it self calls cryptographic #APIs, as well as calls to `ClassLoader.loadClass(String)`, `Class.getMethod(String, Class[])` and `Method.invoke(Object, Object[])`.
|
@fig:th-cg-before shows the original call graph, and gives a good idea of the obfuscation methods used: we can see calls to `Main.decrypt(String)` that itself calls cryptographic #APIs, as well as calls to `ClassLoader.loadClass(String)`, `Class.getMethod(String, Class[])` and `Method.invoke(Object, Object[])`.
|
||||||
This indicate relflection calls base on ciphered strings, but does not reveal what the method actually does.
|
This indicates reflection calls based on ciphered strings, but does not reveal what the method actually does.
|
||||||
In comparison, @fig:th-cg-after, the call graph after instrumentation, still shows the cryptographic and reflection calls, be also four new methods calls.
|
In comparison, @fig:th-cg-after, the call graph after instrumentation, still shows the cryptographic and reflection calls, as well as four new method calls.
|
||||||
In grey on the figure, we can see the glue methods (`T.check_is_Xxx_xxx(Method)`).
|
In grey on the figure, we can see the glue methods (`T.check_is_Xxx_xxx(Method)`).
|
||||||
Those methods are part of the instrumentation process presented in @sec:th-trans, but do not bring a lot to the analysis of the call graph.
|
Those methods are part of the instrumentation process presented in @sec:th-trans, but do not bring a lot to the analysis of the call graph.
|
||||||
In red on the figure however, we have the calls that were hidded by reflection in the first call graph, and thank to the bytecode of the methods called being injected in the application, we can also see that they call `Utils.source(String)` and `Utils.sink(String)`, the methods we defined for this application as source of confidential data and exfiltration method.
|
In red on the figure however, we have the calls that were hidded by reflection in the first call graph, and thank to the bytecode of the methods called being injected in the application, we can also see that they call `Utils.source(String)` and `Utils.sink(String)`, the methods we defined for this application as source of confidential data and exfiltration method.
|
||||||
|
@ -281,7 +281,7 @@ In red on the figure however, we have the calls that were hidded by reflection i
|
||||||
"",
|
"",
|
||||||
).join(),
|
).join(),
|
||||||
),
|
),
|
||||||
caption: [Call Graph of `Main.main()` view by Androguard before patching],
|
caption: [Call Graph of `Main.main()` generated by Androguard before patching],
|
||||||
) <fig:th-cg-before>
|
) <fig:th-cg-before>
|
||||||
|
|
||||||
#figure(
|
#figure(
|
||||||
|
@ -292,11 +292,11 @@ In red on the figure however, we have the calls that were hidded by reflection i
|
||||||
"",
|
"",
|
||||||
).join(),
|
).join(),
|
||||||
),
|
),
|
||||||
caption: [Call Graph of `Main.main()` view by Androguard after patching],
|
caption: [Call Graph of `Main.main()` generated by Androguard after patching],
|
||||||
) <fig:th-cg-after>
|
) <fig:th-cg-after>
|
||||||
|
|
||||||
#v(2em)
|
#v(2em)
|
||||||
|
|
||||||
To conclude, we showed that our approach indeed improves the results of analysis tools without impacting too much their finishing rate.
|
To conclude, we showed that our approach indeed improves the results of analysis tools without impacting their finishing rates too much.
|
||||||
Infortunately, we also noticed that our dynamic analysis is suboptimal, either due to our experimental setup or due to our solution to explore the applications.
|
Unfortunately, we also noticed that our dynamic analysis is suboptimal, either due to our experimental setup or due to our solution to explore the applications.
|
||||||
In the next section, we will present in more detail the limitation of our solution, as well as futur work that can be done to improve the contributions presented in this chapter.
|
In the next section, we will present in more detail the limitations of our solution, as well as future work that can be done to improve the contributions presented in this chapter.
|
||||||
|
|
|
@ -1,76 +1,77 @@
|
||||||
#import "../lib.typ": paragraph, ART, DEX, APK, eg
|
#import "../lib.typ": paragraph, ART, DEX, APK, eg
|
||||||
#import "../lib.typ": todo, jfl-note, jm-note
|
#import "../lib.typ": todo, jfl-note, jm-note
|
||||||
|
|
||||||
== Limitations and Futur Works <sec:th-limits>
|
== Limitations and Future Works <sec:th-limits>
|
||||||
|
|
||||||
The method we presented in this chapter has a number of underdeveloped aspects.
|
The method we presented in this chapter has a number of underdeveloped aspects.
|
||||||
In this section we will present those issues and potential avenues of improvement, related to the bytecode transformation, the dynamic analysis and DroidRA, a tool similar to our solution.
|
In this section, we will present those issues and potential avenues of improvement related to the bytecode transformation, the dynamic analysis and DroidRA, a tool similar to our solution.
|
||||||
|
|
||||||
=== Bytecode Transformation
|
=== Bytecode Transformation
|
||||||
|
|
||||||
#paragraph[Custom Classloaders][
|
#paragraph[Custom Classloaders][
|
||||||
The first obvious limitation of our bytecode transformation is that we do not know what custom classloadrs do, so we cannot accuratly reproduce statically their behavior.
|
The first obvious limitation of our bytecode transformation is that we do not know what custom classloaders do, so we cannot accurately reproduce statically their behaviour.
|
||||||
We elected to fallback to the behavior of the `BaseDexClassLoader`, which is the highest Android specific classloader in the inheritance hierarchy, and whose behavior is shared by all classloaders safe `DelegateLastClassLoader`.
|
We elected to fallback to the behaviour of the `BaseDexClassLoader`, which is the highest Android-specific classloader in the inheritance hierarchy, and whose behaviour is shared by all classloaders except `DelegateLastClassLoader`.
|
||||||
The current implementation of the #ART enforce some restrictions on the classloaders behavior to optimize the runtime performance by caching classes.
|
The current implementation of the #ART enforces some restrictions on the class loader's behaviour to optimise the runtime performance by caching classes.
|
||||||
This gives us some garanties that custom classesloaders will keep a some coherences will the classic classloaders.
|
This gives us some guarantees that custom class loaders will keep some coherence with the classic classloaders.
|
||||||
For instance, a class loaded dynamically must have the same name as the name used in `ClassLoader.loadClass()`.
|
For instance, a class loaded dynamically must have the same name as the name used in `ClassLoader.loadClass()`.
|
||||||
This make `BaseDexClassLoader` a good estimation for legitimate classloaders, however, an obfuscated application could use the techniques discussed in @sec:cl-cross-obf, in wich case our model would be entirelly wrong.
|
This makes `BaseDexClassLoader` a good approximation for legitimate class loaders.
|
||||||
|
However, an obfuscated application could use the techniques discussed in @sec:cl-cross-obf, in which case our model would be entirely wrong.
|
||||||
|
|
||||||
It would be interesting to expore if some form of static analysis like symbolic execution could be used to extract the behavior of an ad hoc class loader and be used to model the class used appropriately.
|
It would be interesting to explore if some form of static analysis, like symbolic execution, could be used to extract the behaviour of an ad hoc class loader and be used to model the class used appropriately.
|
||||||
A more reasonable approach however would be to improve the static analysis to intercept each calls of `loadClass()` of each class loaders, including implicite calls performed by the #ART.
|
A more reasonable approach would be to improve the static analysis to intercept each call of `loadClass()` of each class loader, including implicit calls performed by the #ART.
|
||||||
This would allow to collect a mapping $("class loader", "class name") -> "class"$ that can then be used when renaming colliding classes.
|
This would allow us to collect a mapping $("class loader", "class name") -> "class"$ that can then be used when renaming colliding classes.
|
||||||
]
|
]
|
||||||
|
|
||||||
#paragraph[Multiple Classloaders for one `Method.invoke()`][
|
#paragraph[Multiple Classloaders for one `Method.invoke()`][
|
||||||
Although we managed to handle call to different methods from one `Method.invoke()` site, we do not handle calling methods from different classloaders with colliding classes definition.
|
Although we managed to handle calls to different methods from one `Method.invoke()` site, we do not handle calling methods from different classloaders with colliding class definitions.
|
||||||
The first reason is that it is quite challenging to compare classloaders statically.
|
The first reason is that it is quite challenging to compare classloaders statically.
|
||||||
At runtime, each object has an unique identifier that can be used to compare them over the course of the same execution, but this identifier is reset each time the application starts.
|
At runtime, each object has a unique identifier that can be used to compare them over the course of the same execution, but this identifier is reset each time the application starts.
|
||||||
This means we cannot use this identifier in an `if` condition to differentiate the classloaders.
|
This means we cannot use this identifier in an `if` condition to differentiate the classloaders.
|
||||||
Ideally, we would combine the hash of the loaded #DEX files, the classloader class and parent to make an unique, static identifier, but the #DEX files loaded by a classloader cannot be accessed at runtime without accessing the process memory at arbitrary locations.
|
Ideally, we would combine the hash of the loaded #DEX files, the classloader class and parent to make a unique, static identifier, but the #DEX files loaded by a classloader cannot be accessed at runtime without accessing the process memory at arbitrary locations.
|
||||||
For some classloaders, the string representation returned by `Object.toString()` list the location of the loaded #DEX file on the file system.
|
For some classloaders, the string representation returned by `Object.toString()` lists the location of the loaded #DEX file on the file system.
|
||||||
This is not the case for the commonly used `InMemoryClassLoader`.
|
This is not the case for the commonly used `InMemoryClassLoader`.
|
||||||
In addition, the #DEX files are often located in the application private folder, whose name is derived from the hash of the #APK itself.
|
In addition, the #DEX files are often located in the application's private folder, whose name is derived from the hash of the #APK itself.
|
||||||
Because we modify the application, the path of the private folder also change, and so will the string representation of the classloaders.
|
Because we modify the application, the path of the private folder also changes, and so will the string representation of the classloaders.
|
||||||
Checking the classloader of a classes can also have side-effect on classloaders that delegate to the main application classloader:
|
Checking the classloader of a class can also have side effects on classloaders that delegate to the main application classloader:
|
||||||
because we inject the classes in the #APK, the classes of the classloader are now already in the main application classloader, which in most case will have priority on the other classloaders, and lead to the class beeing loaded by the application classloader instead of the original classloader.
|
because we inject the classes in the #APK, the classes of the classloader are now already in the main application classloader, which in most cases will have priority over the other classloaders, and lead to the class being loaded by the application classloader instead of the original classloader.
|
||||||
If we check for the classloader, we would need to considere such cases en rename each classes of each classloader before reinjecting them to the in the application.
|
If we check for the classloader, we would need to consider such cases and rename each class of each classloader before reinjecting them in the application.
|
||||||
This would greatly increase the risk of breaking the application during its transformation.
|
This would greatly increase the risk of breaking the application during its transformation.
|
||||||
Instead, we elected to ignore the classloaders when selecting the method to invoque.
|
Instead, we elected to ignore the classloaders when selecting the method to invoke.
|
||||||
This leads to potential invalid runtime behaviore, as the first method that matching the class name will be called, but the alternative methods from other classloader still appears in the new application, albeit in a block that might be flagged as dead-code by a sufficiently advenced static analyser.
|
This leads to potential invalid runtime behaviour, as the first method that matches the class name will be called, but the alternative methods from other class loaders still appear in the new application, albeit in a block that might be flagged as dead code by a sufficiently advanced static analyser.
|
||||||
]
|
]
|
||||||
|
|
||||||
#paragraph[`ClassNotFoundException` may not be raised][
|
#paragraph[`ClassNotFoundException` may not be raised][
|
||||||
In the very specific situation where the original application tries to access a class from dynamically loaded bytecode without actually accessing this bytecode (#eg by using the wrong class loader), the patched application behavior will differ.
|
In the very specific situation where the original application tries to access a class from dynamically loaded bytecode without actually accessing this bytecode (#eg by using the wrong class loader), the patched application behaviour will differ.
|
||||||
The original application should raise a `ClassNotFoundException`, but in the patched application, the class will be accible and the exception will not be raised.
|
The original application should raise a `ClassNotFoundException`, but in the patched application, the class will be accessible and the exception will not be raised.
|
||||||
In pactice, their is not a lot of reason to do such thing.
|
In practice, there is not a lot of reason to do such a thing.
|
||||||
One could be to check if the #APK as been tempered with, but their are easier ways to do thins, like checking the application signature.
|
One could be to check if the #APK has been tampered with, but there are easier ways to do this, like checking the application signature.
|
||||||
Another would be to check if the class is already available, and if not, load it dynamically, in wich case it does not matter as code loaded dynamically is already present.
|
Another would be to check if the class is already available, and if not, load it dynamically, in which case it does not matter, as code loaded dynamically is already present.
|
||||||
In any cases, statically, because we remove neither the calls to the function that load the classes (like `ClassLoader.loadClass(..)`) nor the `try` / `catch` blocks, static analysis tools those can handle the original behavior should still be hable to access the old behavior.
|
In any case, statically, because we remove neither the calls to the function that load the classes (like `ClassLoader.loadClass(..)`) nor the `try` / `catch` blocks, static analysis tools that can handle the original behaviour should still be able to access the old behaviour.
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
=== Dynamic Analysis
|
=== Dynamic Analysis
|
||||||
|
|
||||||
#paragraph[Anti Evasion][
|
#paragraph[Anti Evasion][
|
||||||
Our dynamic analysis does not permform any kind of anti-evasive technique.
|
Our dynamic analysis does not perform any kind of anti-evasive technique.
|
||||||
Any application implementing even basic evasion will detect our environment and will probably not load malicious bytecode.
|
Any application implementing even basic evasion will detect our environment and will probably not load malicious bytecode.
|
||||||
Running the dynamic analysis in a appropriate sandbox such as DroidDungeon should improve the results significantly.
|
Running the dynamic analysis in an appropriate sandbox such as DroidDungeon should improve the results significantly.
|
||||||
]
|
]
|
||||||
|
|
||||||
#paragraph[Code Coverage][
|
#paragraph[Code Coverage][
|
||||||
In @sec:th-dyn-failure, we saw that our dynamic analysis performed poorly.
|
In @sec:th-dyn-failure, we saw that our dynamic analysis performed poorly.
|
||||||
It may be due to our experimental setup, and it is possible that a better sandbox will fix the issue.
|
It may be due to our experimental setup, and it is possible that a better sandbox will fix the issue.
|
||||||
However their is a larger code coverage issue.
|
However, there is a larger code coverage issue.
|
||||||
We tried to manually analysed a few applications marked as malware on MalwareBazaar to test our method.
|
We tried to manually analyse a few applications marked as malware on MalwareBazaar to test our method.
|
||||||
Although we did confirm statically that the applications where using reflection and dynamic code loading, we did not managed to trigger this behavior at runtime, and other obfuscation technique make it verry difficult to determine statically the required condition to trigger them.
|
Although we did confirm that the applications were using reflection and dynamic code loading with a static analysis, we did not manage to trigger this behaviour at runtime, and other obfuscation techniques make it very difficult to determine statically the required condition to trigger them.
|
||||||
Thus, we believe that techniques to improve code coverage are indeed needed when analysing application.
|
Thus, we believe that techniques to improve code coverage are indeed needed when analysing applications.
|
||||||
This could mean better exploration techniques such as the one implemented by Stoat and GroddDroid, or more intrusive approched such as forced excecution.
|
This could mean better exploration techniques, such as the one implemented by Stoat and GroddDroid, or more intrusive approaches, such as forced execution.
|
||||||
]
|
]
|
||||||
|
|
||||||
=== Comparision to DroidRA
|
=== Comparison with DroidRA
|
||||||
|
|
||||||
It would be very interesting to compare our tool to DroidRA.
|
It would be very interesting to compare our tool to DroidRA.
|
||||||
DroidRA is a tool that compute reflection information using static analysis and patch the application to add those calls to the application.
|
DroidRA is a tool that computes reflection information using static analysis and patches the application to add those calls.
|
||||||
Beyond the classic comparison static vs dynamic, DroidRA has a similar goal and strategy to ours.
|
Beyond the classic comparison of static versus dynamic, DroidRA has a similar goal and strategy to ours.
|
||||||
Two notable comparison criteria would be the failure rate and the number of edges added to an application call graph.
|
Two notable comparison criteria would be the failure rate and the number of edges added to an application call graph.
|
||||||
The first criterion indicate how much the results can be used by other tools, while the second indicate how effective the approaches are.
|
The first criterion indicates how much the results can be used by other tools, while the second indicates how effective the approaches are.
|
||||||
|
|
|
@ -5,9 +5,9 @@
|
||||||
In this chapter, we presented a set of transformations to apply to an application to encode reflection calls and code loaded dynamically inside the application.
|
In this chapter, we presented a set of transformations to apply to an application to encode reflection calls and code loaded dynamically inside the application.
|
||||||
We also presented a dynamic analysis approach to collect the information needed to perform those transformations.
|
We also presented a dynamic analysis approach to collect the information needed to perform those transformations.
|
||||||
|
|
||||||
We then applied this method to applications a rescent subset of applications of our dataset from @sec:rasta.
|
We then applied this method to a recent subset of applications of our dataset from @sec:rasta.
|
||||||
When comparing the success rate of the tools of @sec:rasta on the applications before and after the transformation, we found that, in general, the success rate of those tools slightly decrease, with a few exceptions.
|
When comparing the success rate of the tools of @sec:rasta on the applications before and after the transformation, we found that, in general, the success rate of those tools slightly decreases, with a few exceptions.
|
||||||
We also showed that our transformation indeed allow static analysis tools to access and process those runtime information in their analysis.
|
We also showed that our transformation indeed allows static analysis tools to access and process that runtime information in their analysis.
|
||||||
However, a more in-depth look at the results of our dynamic analysis showed that our code coverage is lacking, and that the great majority of dynamically loaded code we intercepted is from generic advertisement and telemetry libraries.
|
However, a more in-depth look at the results of our dynamic analysis showed that our code coverage is lacking, and that the great majority of dynamically loaded code we intercepted is from generic advertisement and telemetry libraries.
|
||||||
|
|
||||||
#v(2em)
|
#v(2em)
|
||||||
|
@ -15,6 +15,6 @@ However, a more in-depth look at the results of our dynamic analysis showed that
|
||||||
#align(center, highlight-block(inset: 15pt, width: 75%, breakable: false, block(align(left)[
|
#align(center, highlight-block(inset: 15pt, width: 75%, breakable: false, block(align(left)[
|
||||||
#pb3: #pb3-text
|
#pb3: #pb3-text
|
||||||
#v(0.75em)
|
#v(0.75em)
|
||||||
We showed that intrumentation can be used to add the direct calls to method initially called using reflections, which, combined with the injection in the application of dynamically loaded bytecode, allows generic static analysis tools to acces previously unavailable code.
|
We showed that instrumentation can be used to add direct calls to methods initially called through reflections, which, combined with the injection in the application of dynamically loaded bytecode, allows generic static analysis tools to access previously unavailable code.
|
||||||
However, we also found that the dynamic analysis can be a significant bottleneck in this approach.
|
However, we also found that the dynamic analysis can be a significant bottleneck in this approach.
|
||||||
])))
|
])))
|
||||||
|
|
|
@ -7,10 +7,10 @@
|
||||||
|
|
||||||
#align(center, highlight-block(inset: 15pt, width: 75%, block(align(left)[
|
#align(center, highlight-block(inset: 15pt, width: 75%, block(align(left)[
|
||||||
Some applications use dynamic code loading and reflection calls that prevent static analysis tools from analysing the complete application.
|
Some applications use dynamic code loading and reflection calls that prevent static analysis tools from analysing the complete application.
|
||||||
Those behavior can be analyse with dynamic analysis, but the information collected is not enough analyse the application: most tools do not have a way to process this additionnal data.
|
Those behaviours can be analysed with dynamic analysis; however, the information collected is not enough to analyse the application: most tools do not have a way to process this additional data.
|
||||||
In this chapter, promose to use dynamic analysis to collect information related to dynamic code loading and reflection, and to encode this information in the bytecode of the application to allow further analysis.
|
In this chapter, we propose to use dynamic analysis to collect information related to dynamic code loading and reflection, and to encode this information in the bytecode of the application to allow further analysis.
|
||||||
We compared the results of analysis on application before and after the transformation, using tools like Flowdroid or Androguard, and found that the additional information is indeed processed by the tools.
|
We compared the results of analysis on applications before and after the transformation, using tools like Flowdroid or Androguard, and found that the additional information is indeed processed by the tools.
|
||||||
We also compared the finishing rate of the tools, using the same experiment as in @sec:rasta, and found that the finishing rate is generally only slightly negativelly impacted by the transformation.
|
We also compared the finishing rate of the tools, using the same experiment as in @sec:rasta, and found that the finishing rate is generally only slightly negatively impacted by the transformation.
|
||||||
])))
|
])))
|
||||||
|
|
||||||
#include("1_introduction.typ")
|
#include("1_introduction.typ")
|
||||||
|
|
11
main.typ
11
main.typ
|
@ -139,8 +139,6 @@
|
||||||
|
|
||||||
#todo[Normalize classloaders vs class loaders]
|
#todo[Normalize classloaders vs class loaders]
|
||||||
#todo[Normalize bullets/item: either end with a '.' or a ';']
|
#todo[Normalize bullets/item: either end with a '.' or a ';']
|
||||||
#todo[footnote numbering]
|
|
||||||
#todo[Check defence date]
|
|
||||||
|
|
||||||
#include("1_introduction/main.typ")
|
#include("1_introduction/main.typ")
|
||||||
#include("2_background/main.typ")
|
#include("2_background/main.typ")
|
||||||
|
@ -149,13 +147,4 @@
|
||||||
#include("5_theseus/main.typ")
|
#include("5_theseus/main.typ")
|
||||||
#include("6_conclusion/main.typ")
|
#include("6_conclusion/main.typ")
|
||||||
|
|
||||||
/*
|
|
||||||
* Take aways depuis l'intro
|
|
||||||
* puis résumé des contributions majeurs, un paragraphe par contrib
|
|
||||||
*
|
|
||||||
* future work plus haut niveau: reprandre les plus important et/ou des plus large: eg: quide web-base? flutter?
|
|
||||||
*/
|
|
||||||
|
|
||||||
#text(fill: luma(75%), lorem(500))
|
|
||||||
|
|
||||||
#bibliography("bibliography.bib")
|
#bibliography("bibliography.bib")
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue