This commit is contained in:
parent
039970904e
commit
10df431972
GPG key ID:
B66AEEDA9B645AD2
9 changed files with 240 additions and 250 deletions
|
|
@ -2,21 +2,21 @@
|
|||
|
||||
== Code Transformation <sec:th-trans>
|
||||
|
||||
In this section, we will see how we can transform the application code to make dynamic codeloading and reflexive calls more analysable by static analysis tools.
|
||||
In this section, we will see how we can transform the application code to make dynamic code loading and reflective calls more analysable by static analysis tools.
|
||||
|
||||
=== Transforming Reflection <sec:th-trans-ref>
|
||||
|
||||
|
||||
In Android, reflection allows to instanciate a class, or call a method, without having this class or method appear in the bytecode.
|
||||
Instead, the bytecode uses the generic classes `Class`, `Method` and `Constructor`, that represent any existing class, method or constructor.
|
||||
In Android, reflection allows applications to instantiate a class or call a method without having this class or method appear in the bytecode.
|
||||
Instead, the bytecode uses the generic classes `Class`, `Method` and `Constructor`, which represent any existing class, method or constructor.
|
||||
Reflection often starts by retrieving the `Class` object representing the class to use.
|
||||
This class is usually retrieved using a `ClassLoader` object (though they are other ways to get it).
|
||||
This class is usually retrieved using a `ClassLoader` object (though there are other ways to get it).
|
||||
Once the class is retrieved, it can be instanciated using the deprecated method `Class.newInstance()`, as shown in @lst:-th-expl-cl-new-instance, or a specific method can be retrieved.
|
||||
The current approach to instanciate a class is to retrieve the specific `Constructor` object, then calling `Constructor.newInstance(..)` like in @lst:-th-expl-cl-cnstr.
|
||||
The current approach to instantiate a class is to retrieve the specific `Constructor` object, then call `Constructor.newInstance(..)` like in @lst:-th-expl-cl-cnstr.
|
||||
Similarly, to call a method, the `Method` object must be retrieved, then called using `Method.invoke(..)`, as shown in @lst:-th-expl-cl-call.
|
||||
|
||||
Although the process seems to differ between class instanciation and method call from the Java stand point, the runtime opperations are very similar.
|
||||
When instanciating an object with `Object obj = cst.newInstance("Hello Void")`, the constructor method `<init>(Ljava/lang/String;)V`, represented by the `Constructor` `cst`, is called on the object `obj`.
|
||||
Although the process seems to differ between class instantiation and method call from the Java standpoint, the runtime operations are very similar.
|
||||
When instantiating an object with `Object obj = cst.newInstance("Hello Void")`, the constructor method `<init>(Ljava/lang/String;)V`, represented by the `Constructor` `cst`, is called on the object `obj`.
|
||||
|
||||
#figure(
|
||||
```java
|
||||
|
|
@ -24,7 +24,7 @@ When instanciating an object with `Object obj = cst.newInstance("Hello Void")`,
|
|||
Class clz = cl.loadClass("com.example.Reflectee");
|
||||
Object obj = clz.newInstance();
|
||||
```,
|
||||
caption: [Instanciating a class using `Class.newInstance()`]
|
||||
caption: [Instantiating a class using `Class.newInstance()`]
|
||||
) <lst:-th-expl-cl-new-instance>
|
||||
|
||||
#figure(
|
||||
|
|
@ -32,7 +32,7 @@ When instanciating an object with `Object obj = cst.newInstance("Hello Void")`,
|
|||
Constructor cst = clz.getDeclaredConstructor(String.class);
|
||||
Object obj = cst.newInstance("Hello Void");
|
||||
```,
|
||||
caption: [Instanciating a class using `Constructor.newInstance(..)`]
|
||||
caption: [Instantiating a class using `Constructor.newInstance(..)`]
|
||||
) <lst:-th-expl-cl-cnstr>
|
||||
|
||||
#figure(
|
||||
|
|
@ -45,15 +45,15 @@ When instanciating an object with `Object obj = cst.newInstance("Hello Void")`,
|
|||
) <lst:-th-expl-cl-call>
|
||||
|
||||
One of the main reasons to use reflection is to access classes that are not present in the application bytecode, nor are platform classes.
|
||||
Indeed, the application will crash if the #ART encounter references to a class that is cannot be found by the current classloader.
|
||||
Indeed, the application will crash if the #ART encounters references to a class that cannot be found by the current classloader.
|
||||
This is often the case when dealing with classes from bytecode loaded dynamically.
|
||||
|
||||
To allow static analysis tools to analyse an application that use reflection, we want to replace the reflection call by the bytecode that actually call the method.
|
||||
In @sec:th-trans-cl, we deal with the issue of dynamic code loading so that the classes used are in fact present in the application.
|
||||
To allow static analysis tools to analyse an application that uses reflection, we want to replace the reflection call with the bytecode that actually calls the method.
|
||||
In @sec:th-trans-cl, we deal with the issue of dynamic code loading so that the classes used are, in fact, present in the application.
|
||||
|
||||
A notable issue is that a specific reflection call can call different methods.
|
||||
@lst:th-worst-case-ref illustrates a worst case scenario where any method can be called at the same reflection call.
|
||||
In those situation, we cannot garanty that we know all the methods that can be called (#eg the name of the method called could be retrieved from a remote server).
|
||||
@lst:th-worst-case-ref illustrates a worst-case scenario where any method can be called at the same reflection call.
|
||||
In those situations, we cannot guarantee that we know all the methods that can be called (#eg the name of the method called could be retrieved from a remote server).
|
||||
In addition, the method we propose in @sec:th-dyn is a best effort approach to collect reflection data: like any dynamic analysis, it is limited by its code coverage.
|
||||
|
||||
#figure(
|
||||
|
|
@ -65,14 +65,14 @@ In addition, the method we propose in @sec:th-dyn is a best effort approach to c
|
|||
caption: [A reflection call that can call any method]
|
||||
) <lst:th-worst-case-ref>
|
||||
|
||||
To handle those situation, instead of entirely removing the reflection call, we can modify the application code to test if the `Method` (or `Constructor`) object match any expected method, and if yes, directly call the method.
|
||||
If the object does not match any expected method, the code can fallback to the original reflection call.
|
||||
DroidRA~@li_droidra_2016 has a similar solution, except that reflective calls are always evaluated, and the static equivalent follow just after, guarded behind an opaque predicate that is always false at runtime.
|
||||
To handle those situations, instead of entirely removing the reflection call, we can modify the application code to test if the `Method` (or `Constructor`) object matches any expected method, and if yes, directly call the method.
|
||||
If the object does not match any expected method, the code can fall back to the original reflection call.
|
||||
DroidRA~@li_droidra_2016 has a similar solution, except that reflective calls are always evaluated, and the static equivalent follows just after, guarded behind an opaque predicate that is always false at runtime.
|
||||
@lst:-th-expl-cl-call-trans demonstrate this transformation on @lst:-th-expl-cl-call:
|
||||
at line 25, the `Method` objet `mth` is checked using a method we generated and injected in the application (defined at line 2 in the listing).
|
||||
This method check if the method name, (line 5), its parameters (lines 6-9), its return type (lines 10-11) and its declaring class (lines 13-14) match the expected method.
|
||||
at line 25, the `Method` object `mth` is checked using a method we generated and injected in the application (defined at line 2 in the listing).
|
||||
This method checks if the method name (line 5), its parameters (lines 6-9), its return type (lines 10-11) and its declaring class (lines 13-14) match the expected method.
|
||||
If it is the case, the method is used directly (line 26) after casting the arguments and associated object into the types/classes we just checked.
|
||||
If the check line 25 does not pass, the original reflectif call is made (line 28).
|
||||
If the check line 25 does not pass, the original reflective call is made (line 28).
|
||||
If we were to expect other possible methods to be called in addition to `myMethod`, we would add `else if` blocks between lines 26 and 27, with other check methods reflecting each potential method call.
|
||||
/*
|
||||
#jfl-note[It should be noted that we do the transformation at the bytecode level, the code in the listing correspond to the output of JADX][
|
||||
|
|
@ -82,24 +82,24 @@ If we were to expect other possible methods to be called in addition to `myMetho
|
|||
] #todo[Ref to list of common tools?] reformated for readability.
|
||||
*/
|
||||
|
||||
The method check is done in a separate method injected inside the application to avoid clutering the application too much.
|
||||
The method check is done in a separate method injected inside the application to avoid cluttering the application too much.
|
||||
Because Java (and thus Android) uses polymorphic methods, we cannot just check the method name and its class, but also the whole method signature.
|
||||
We chose to limit the transformation to the specific instruction that call `Method.invoke(..)`.
|
||||
This drastically reduce the risks of breaking the application, but leads to a lot of type casting.
|
||||
We chose to limit the transformation to the specific instruction that calls `Method.invoke(..)`.
|
||||
This drastically reduces the risks of breaking the application, but leads to a lot of type casting.
|
||||
Indeed, the reflection call uses the generic `Object` class, but actual methods usually use specific classes (#eg `String`, `Context`, `Reflectee`) or scalar types (#eg `int`, `long`, `boolean`).
|
||||
This means that the method parameters and object on which the method is called must be downcast to their actual type before calling the method, then the returned value must be upcasted back to an `Object`.
|
||||
This means that the method parameters and object on which the method is called must be downcast to their actual type before calling the method, then the returned value must be upcast back to an `Object`.
|
||||
Scalar types especially require special attention.
|
||||
Java (and Android) distinguish between scalar type and classes, and they cannot be mixed: a scalar cannot be cast into an `Object`.
|
||||
However, each scalar type has an associated class that can be use when doing reflection.
|
||||
For example, the scalar type `int` is associated with the class `Integer`, the method `Integer.valueOf()` can convert an `int` scalar to an `Integer` object, and the method `Integer.intValue()` convert back an `Integer` object to an `int` scalar.
|
||||
Each time the method called by reflection used scalars, the scalar-object convertion must be made before calling it.
|
||||
And finally, because the instruction following the reflection call expect an `Object`, the return value of the method must be cast into an `Object`.
|
||||
Java (and Android) distinguish between scalar types and classes, and they cannot be mixed: a scalar cannot be cast into an `Object`.
|
||||
However, each scalar type has an associated class that can be used when doing reflection.
|
||||
For example, the scalar type `int` is associated with the class `Integer`, the method `Integer.valueOf()` can convert an `int` scalar to an `Integer` object, and the method `Integer.intValue()` converts back an `Integer` object to an `int` scalar.
|
||||
Each time the method called by reflection uses scalars, the scalar-object conversion must be made before calling it.
|
||||
And finally, because the instruction following the reflection call expects an `Object`, the return value of the method must be cast into an `Object`.
|
||||
|
||||
This back and forth between types might confuse some analysis tools.
|
||||
This could be improved in futur works by analysing the code around the reflection call.
|
||||
For example, if the result of the reflection call is immediatly cast into the expected type (#eg in @lst:-th-expl-cl-call, the result is cast to a `String`), they should not be any need to cast it to Object in between.
|
||||
Similarly, it is common to have the method parameter arrays generated just before the reflection call never be used again (This is due to `Method.invoke(..)` beeing a varargs method: the array can be generated by the compiler at compile time).
|
||||
In those cases, the parameters could be used directly whithout the detour inside an array.
|
||||
This could be improved in future works by analysing the code around the reflection call.
|
||||
For example, if the result of the reflection call is immediately cast into the expected type (#eg in @lst:-th-expl-cl-call, the result is cast to a `String`), there should be no need to cast it to Object in between.
|
||||
Similarly, it is common to have the method parameter arrays generated just before the reflection call and never be used again (This is due to `Method.invoke(..)` being a varargs method: the array can be generated by the compiler at compile time).
|
||||
In those cases, the parameters could be used directly without the detour inside an array.
|
||||
|
||||
#figure(
|
||||
```java
|
||||
|
|
@ -142,31 +142,30 @@ In those cases, the parameters could be used directly whithout the detour inside
|
|||
|
||||
#jfl-note[Ici je pensais lire comment on tranforme le code qui load du code, mais on me parle de multi dex]
|
||||
|
||||
An application can dynamically import code from several format like #DEX, #APK, #JAR or #OAT, either stored in memory or in a file.
|
||||
Because it is an internal, platform dependant format, we elected to ignore the #OAT format.
|
||||
An application can dynamically import code from several formats like #DEX, #APK, #JAR or #OAT, either stored in memory or in a file.
|
||||
Because it is an internal, platform-dependent format, we elected to ignore the #OAT format.
|
||||
Practically, #JAR and #APK files are zip files containing #DEX files.
|
||||
This means that we only need to find a way to integrate #DEX files to the application.
|
||||
|
||||
This means that we only need to find a way to integrate #DEX files into the application.
|
||||
|
||||
We saw in @sec:cl the class loading model of Android.
|
||||
When doing dynamic code loading, an application define a new `ClassLoader` that handle the new bytecode, and start accessing its classes using reflection.
|
||||
When doing dynamic code loading, an application defines a new `ClassLoader` that handles the new bytecode, and starts accessing its classes using reflection.
|
||||
We also saw in @sec:cl that Android now use the multi-dex format, allowing it to handle any number of #DEX files in one classloader.
|
||||
Therefore, the simpler way to give access to the dynamically loaded code to static analysis tool is add the dex files to the application.
|
||||
This should not impact the classloading model as long as there is no class collision (we will explore this in @sec:th-class-collision) and as long as the original application appliaction did not try to access unaccessible classes (we will develop this issue in @sec:th-limits).
|
||||
Therefore, the simpler way to give access to the dynamically loaded code to static analysis tools is to add the dex files to the application.
|
||||
This should not impact the classloading model as long as there is no class collision (we will explore this in @sec:th-class-collision) and as long as the original application did not try to access inaccessible classes (we will develop this issue in @sec:th-limits).
|
||||
|
||||
#figure(
|
||||
image(
|
||||
"figs/dex_insertion.svg",
|
||||
width: 80%,
|
||||
alt: "A diagram showing a box labelled 'app.apk', a box labelled 'lib.jar', and single file ouside the boxes labelled 'lib.dex'. The lib.jar boxe contains the files classes.dex and classes2.dex. Inside the app.apk box, the files AndroidManifest.xml, resources.arsc, classes.dex, classes2.dex, classes3.dex and the folders lib, res and assets are circled by dashes and labelled 'original files', and, still inside app.apk, the files classes4.dex, classes5.dex and classes5.dex are circled by dashes and labelled 'Added Files'. Arrows go from lib.dex to classes4.dex, from the classes.dex inside lib.jar to classes5.dex inside app.apk and from classe2.dex inside lib.jar to classes6.dex inside app.apk"
|
||||
alt: "A diagram showing a box labelled 'app.apk', a box labelled 'lib.jar', and a single file outside the boxes labelled 'lib.dex'. The lib.jar box contains the files classes.dex and classes2.dex. Inside the app.apk box, the files AndroidManifest.xml, resources.arsc, classes.dex, classes2.dex, classes3.dex and the folders lib, res and assets are circled by dashes and labelled 'original files', and, still inside app.apk, the files classes4.dex, classes5.dex and classes5.dex are circled by dashes and labelled 'Added Files'. Arrows go from lib.dex to classes4.dex, from the classes.dex inside lib.jar to classes5.dex inside app.apk and from classes2.dex inside lib.jar to classes6.dex inside app.apk"
|
||||
),
|
||||
caption: [Inserting #DEX files inside an #APK]
|
||||
) <fig:th-inserting-dex>
|
||||
|
||||
In the end, we decided to *not* modify the original code that load the bytecode.
|
||||
Statically, we already added the bytecode loaded dynamically, and most tools already ignore dynamic code loading.
|
||||
In the end, we decided to *not* modify the original code that loads the bytecode.
|
||||
We already added the bytecode loaded dynamically, and most tools already ignore dynamic code loading.
|
||||
At runtime, although the bytecode is already present in the application, the application will still dynamically load the code.
|
||||
This ensure that the application keep working as intended even if the transformation we applied are incomplete.
|
||||
This ensures that the application keeps working as intended, even if the transformation we applied is incomplete.
|
||||
Specifically, to call dynamically loaded code, an application needs to use reflection, and we saw in @sec:th-trans-ref that we need to keep reflection calls, and in order to keep reflection calls, we need the classloader created when loading bytecode.
|
||||
|
||||
=== Class Collisions <sec:th-class-collision>
|
||||
|
|
@ -174,22 +173,22 @@ Specifically, to call dynamically loaded code, an application needs to use refle
|
|||
We saw in @sec:cl/*-obfuscation*/ that having several classes with the same name in the same application can be problematic.
|
||||
In @sec:th-trans-cl, we are adding new code.
|
||||
By doing so, we increase the probability of having class collisions:
|
||||
The developper may have reuse a helper class in both the dynamically loaded bytecode and the application, or an obfuscation process may have rename classes without checking for intersection between the two sources of bytecode.
|
||||
When loaded dynamically, the classes are in a different classloader, and the class resolution is resolved at runtime like we saw in @sec:cl-loading.
|
||||
We decided to restrain our scope to the use of class loader from the Android #SDK.
|
||||
In the abscence of class collision, those class loader behave seamlessly and adding the classes to application maintains the behavior.
|
||||
The developer may have reused a helper class in both the dynamically loaded bytecode and the application, or an obfuscation process may have renamed classes without checking for intersection between the two sources of bytecode.
|
||||
When loaded dynamically, the classes are in a different classloader, and the class resolution is resolved at runtime, like we saw in @sec:cl-loading.
|
||||
We decided to restrain our scope to the use of class loaders from the Android #SDK.
|
||||
In the absence of class collision, those class loaders behave seamlessly and adding the classes to the application maintains the behaviour.
|
||||
|
||||
When we detect a collision, we rename one of the colliding classes in order to be able to differenciate both classes.
|
||||
To avoid breaking the application, we then need to rename all references to this specific class, an be carefull not to modify references to the other class.
|
||||
To do so, we regroup each classes by the classloaders defining them.
|
||||
When we detect a collision, we rename one of the colliding classes in order to be able to differentiate between classes.
|
||||
To avoid breaking the application, we then need to rename all references to this specific class and be careful not to modify references to the other class.
|
||||
To do so, we regroup each class by the classloaders defining them.
|
||||
Then, for each colliding class name and each classloader, we check the actual class used by the classloader.
|
||||
If the class has been renamed, we rename all reference to this class in the classes defined by this classloader.
|
||||
To find the class used by a classloader, we reproduce the behavior of the different classloaders of the Android #SDK.
|
||||
This is an important step: remember that the delegation process can lead to situation where the class defined by a classloader is not the class that will be loaded when querying the classloader.
|
||||
The pseudo-code in @lst:renaming-algo show the three steps of this algorithm:
|
||||
- First we detect collision and rename classes definitions to remove the collisions.
|
||||
If the class has been renamed, we rename all references to this class in the classes defined by this classloader.
|
||||
To find the class used by a classloader, we reproduce the behaviour of the different classloaders of the Android #SDK.
|
||||
This is an important step: remember that the delegation process can lead to situations where the class defined by a classloader is not the class that will be loaded when querying the classloader.
|
||||
The pseudo-code in @lst:renaming-algo shows the three steps of this algorithm:
|
||||
- First, we detect collisions and rename class definitions to remove the collisions.
|
||||
- Then we rename the reference to the colliding classes to make sure the right classes are called.
|
||||
- Ultimately, we merge the modified dexfiles of each class loaders into one android application.
|
||||
- Ultimately, we merge the modified #DEX files of each class loader into one Android application.
|
||||
|
||||
#figure(
|
||||
```python
|
||||
|
|
@ -226,4 +225,4 @@ The pseudo-code in @lst:renaming-algo show the three steps of this algorithm:
|
|||
#v(2em)
|
||||
|
||||
Now that we saw the transformations we want to make, we know the runtime information we need to do it.
|
||||
In the next section, we will propose a solution to collect those informations.
|
||||
In the next section, we will propose a solution to collect that information.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue