From c34eb1b838d0ab9756e3e736b02657fc3e365281 Mon Sep 17 00:00:00 2001 From: Jean-Marie 'Histausse' Mineau Date: Thu, 25 Sep 2025 03:27:04 +0200 Subject: [PATCH] wip, again and again --- 0_preamble/french_summary.typ | 4 +- 2_background/1_intro.typ | 13 ------ 2_background/2_android_bg.typ | 3 +- 2_background/3_problem_statements.typ | 22 ++++++---- 2_background/4_1_rasta.typ | 58 +++++++++++---------------- 2_background/4_soa.typ | 2 + 4_class_loader/2_classloading.typ | 2 +- 7 files changed, 43 insertions(+), 61 deletions(-) diff --git a/0_preamble/french_summary.typ b/0_preamble/french_summary.typ index fdbcaba..10dbc3d 100644 --- a/0_preamble/french_summary.typ +++ b/0_preamble/french_summary.typ @@ -52,7 +52,6 @@ Certaines contributions d'ingénierie inverse ont déjà proposé d'instrumenter Cette approche prometteuse motive notre troisième problématique: #pb3-text-fr -#todo[Expliquer que la suite est un résumé francais des chapitres] #todo[Bouger le résumé a la fin fr à la fin?] #[ @@ -614,8 +613,7 @@ Seuls #num(nb_bytecode_collected - nb_google - nb_appsflyer - nb_facebook) fichi Ces fichiers restants contiennent du code spécifique aux applications les utilisant, principalement des applications exigeant un niveau important de sécurité comme des applications banquaires ou d'assurance santé. Nous avons ensuite modifié les applications comme décrit précédemment, puis relancé les outils de notre première contribution sur les applications modifiées pour comparer leur taux de finition au taux sur les applications initiales. -En fonction des outils, le taux de finition est soit inchangé, soit légèrement plus faible pour les applications modifiées (a l'exeption de Saaf #todo[expliquer saaf] -) +En fonction des outils, le taux de finition est soit inchangé, soit légèrement plus faible pour les applications modifiées. Pour vérifier que notre approche fonctionne, nous avons créé une petite application de test utilisant du chargement dynamique et des appels réflexifs. @fig:th-demo-avant montre la classe principale de l'application. diff --git a/2_background/1_intro.typ b/2_background/1_intro.typ index 4f701a7..b17b1dd 100644 --- a/2_background/1_intro.typ +++ b/2_background/1_intro.typ @@ -31,16 +31,3 @@ We will then examine the state of the art related to those problem statements @s #todo[synthese a la fin de chaque section soa des problemes] #todo[Problematique avant soa] - - -#todo[ - plan: - - - 2.1 intro - - 2.2 bases d'Android et RE (completer un peu pour souligner les besoins qui menes au pbs) - - 2.3 Problématiques du RE (reprendre l'intro avec ce qui a été dit dans 2.2) - apktool et androguard sont réutilisé, ca fait supposé qu'il y a peut être un peu de réutilisation - on peut charger des classes, et dans le code d'android, on vois qu'en fait le classes loading est beaucoup plus important que ca - c'est connus que cl + statique + ref = nono, tout les outils présentes leurs solutions d'une certaine facons - - 2.4 State of the Art -] diff --git a/2_background/2_android_bg.typ b/2_background/2_android_bg.typ index 2cf4809..fca0842 100644 --- a/2_background/2_android_bg.typ +++ b/2_background/2_android_bg.typ @@ -2,7 +2,8 @@ == Android Background -#todo[Intro] +We begin this chapter with background information about Android and reverse engineering of Android applications. +We start with a description of Android applications and their execution environement, then list some usefull basic tools for reverse engineering, and finish with the bases of static analysis for Android. #include("2_1_android.typ") #include("2_2_tools.typ") diff --git a/2_background/3_problem_statements.typ b/2_background/3_problem_statements.typ index 4d111d0..15a508c 100644 --- a/2_background/3_problem_statements.typ +++ b/2_background/3_problem_statements.typ @@ -12,7 +12,7 @@ Depending on the situation, a reverse engineer might want to use those tools, or Unfortunately, they can be hard to use. And like we said previously, the fast evolution of Android can be a significant obstacle. The combinaison of those two point can lead a reverse engineer to spend a lot of time trying to use a tool without realising that tools does not work anymore. -Our first problem statement #pb1 focuses on this issue: #pb1-text. +Our first problem statement #pb1 focuses on this issue: #pb1-text Determining which tools are still usable today is a first step, but finding out what reasons make a tool stop working might help writing more resilient tools in the futur. We also presented dynamic code loading an obstacle for static analysis. @@ -21,12 +21,18 @@ However, class loading plays a much more important role in the #ART. Class loading originate from the Java ecosystem, and was ported to Android so that developers could keep writting application in Java. Despit that, Android made a lot of change to the original Java classes, and did not document those changes. Between static analysis general oversight of class loading, relegating it to dynamic analysis, and the lake of documentation of the actual behaviour of the #ART, the question of the impact of the class loading algorithm on static analysis can be ask. -Our secon problem statement #pb2 tries to anwser this question: #pb2-text. +Our second problem statement #pb2 aims to anwser this question: #pb2-text +Circling back to known limitations of static analysis, dynamic code loading and reflection are often used to obfuscate applications. +Dynamic code loading allows to hide bytecode from static analysis with relativelly low effort. +The bytecode can downloaded at runtime, stored in the application encrypted, hidden inside other files, generated at runtime, etc. +In a way, reflection allows to do the same thing, but for specific method calls: instead of the actual call, static analysis will see a call to the generic `Method.invoke()` method. +By contrast, it is relatively easy to find those the name of the method called or to intercept dynamically loaded bytecode using dynamic tools like Frida. +The issue that arrise then is what to do with the collected data. +Simply having it greatly helps a manual analysis, but it cannot be used directly by tools that perform static analyses. +There is no standard representation for runtime information, and there is simply no way to give a list of reflection sites and the associated method calls for most tools. +This means that in most cases, when a reverse engineer wants to improve static analysis with dynamic analysis, they need to modify the static tools to receive the additionnal runtime data. +Doing so requires both time and knowledge of the internals of the tools used. +Our third problem statement, #pb3, explore an alternative aproach that modify the application instead of the tool: #pb3-text -#todo[ - Problématiques du RE (reprendre l'intro avec ce qui a été dit dans 2.2) - apktool et androguard sont réutilisé, ca fait supposé qu'il y a peut être un peu de réutilisation - on peut charger des classes, et dans le code d'android, on vois qu'en fait le classes loading est beaucoup plus important que ca - c'est connus que cl + statique + ref = nono, tout les outils présentes leurs solutions d'une certaine facons -] +We will now explore the current state of the art for relevent contributions related to our problem statements. diff --git a/2_background/4_1_rasta.typ b/2_background/4_1_rasta.typ index cf088ea..da64b63 100644 --- a/2_background/4_1_rasta.typ +++ b/2_background/4_1_rasta.typ @@ -4,41 +4,26 @@ === Reusability of Static Analysis Tools -//== Android Reverse Engineering Techniques - -//#todo[swap with tool section ?] -// - -#todo[Refactor] - -==== Static Analysis +#pb1-text In the past fifteen years, the research community released many tools to detect or analyse malicious behaviors in applications. -Two main approaches can be distinguished: static and dynamic analysis~@Li2017. -Dynamic analysis requires to run the application in a controlled environment to observe runtime values and/or interactions with the operating system. -For example, an Android emulator with a patched kernel can capture these interactions but the modifications to apply are not a trivial task. -Such approach is limited by the required time to execute a limited part of the application with no guarantee on the obtained code coverage. -Dynamic analysis is also limited by evading techniques that may prevent the execution of malicious parts of the code. -As a consequence, a lot of efforts have been put in static approaches. //, which is the focus of this paper. - -Data-flow analysis is the subject of many contribution~@weiAmandroidPreciseGeneral2014 @titzeAppareciumRevealingData2015 @bosuCollusiveDataLeak2017 @klieberAndroidTaintFlow2014 @DBLPconfndssGordonKPGNR15 @octeauCompositeConstantPropagation2015 @liIccTADetectingInterComponent2015, the most notable tool being Flowdroid~@Arzt2014a. - -#todo[Describe the different contributions in relations to the issues they tackle, be more critical] - -A lot of those more advanced tools rely on common tools to interact with Android applications/#DEX bytecode@~@Li2017. -Reccuring examples of such support tools are Appktool (#eg Amandroid~@weiAmandroidPreciseGeneral2014, Blueseal~@shenInformationFlowsPermission2014, SAAF~@hoffmannSlicingDroidsProgram2013), Androguard (#eg Adagio~@gasconStructuralDetectionAndroid2013, Appareciumn~@titzeAppareciumRevealingData2015, Mallodroid~@fahlWhyEveMallory2012) or Soot (#eg Blueseal~@shenInformationFlowsPermission2014, DroidSafe~@DBLPconfndssGordonKPGNR15, Flowdroid~@Arzt2014a). - -The number of publication related to static analysis make can make it difficult to find the right tool for the right task. +The first steps to anwser this question is to list those previously published tools. +The number of publication related to static analysis can make it difficult to find the right tool for the right task. Li #etal~@Li2017 published a systematic literature review for Android static analysis before May 2015. They analysed 92 publications and classified them by goal, method used to solve the problem and underlying technical solution for handling the bytecode when performing the static analysis. In particular, they listed 27 approaches with an open-source implementation available. -Nevertheless, experiments to evaluate the reusability of the pointed out software were not performed. + +Interestingly, a lot of the tools listed rely on common tools to interact with Android applications/#DEX bytecode. +Reccuring examples of such support tools are Appktool (#eg Amandroid~@weiAmandroidPreciseGeneral2014, Blueseal~@shenInformationFlowsPermission2014, SAAF~@hoffmannSlicingDroidsProgram2013), Androguard (#eg Adagio~@gasconStructuralDetectionAndroid2013, Appareciumn~@titzeAppareciumRevealingData2015, Mallodroid~@fahlWhyEveMallory2012) or Soot (#eg Blueseal~@shenInformationFlowsPermission2014, DroidSafe~@DBLPconfndssGordonKPGNR15, Flowdroid~@Arzt2014a). +This strengthens our idea that behing able to reuse previous tools in important. +Those tools are built incrementally, on top of each other. + +Nevertheless, experiments to evaluate the reusability of the pointed out software were not performed by Li #etal #jfl-note[We believe that the effort of reviewing the literature for making a comprehensive overview of available approaches should be pushed further: an existing published approach with a software that cannot be used for technical reasons endanger both the reproducibility and reusability of research.][A mettre en avant?] -In the next section, we will look at the work that has been done to evaluate different analysis tools. +//Data-flow analysis is the subject of many contribution~@weiAmandroidPreciseGeneral2014 @titzeAppareciumRevealingData2015 @bosuCollusiveDataLeak2017 @klieberAndroidTaintFlow2014 @DBLPconfndssGordonKPGNR15 @octeauCompositeConstantPropagation2015 @liIccTADetectingInterComponent2015, the most notable tool being Flowdroid~@Arzt2014a. -==== Evaluating Static Analysis Tools - +We will now explore this direction further by looking at the work that has been done to evaluate different analysis tools. Works that perform benchmaks of tools follow a similar method. They start by selecting a set of tools with similar goals. Usually, those contribusions are comparing existing tools to their own, but some contributions do not introduce a new tool and focus on surveying the state of the art for some technique. @@ -49,9 +34,10 @@ Several factors can be considered to compare the results of the tools: the number of false positives, false negatives, or even the time it took to finish the analysis. Occasionally, the number of application a tool simply failled to analyse are also compared. -In @sec:bg-datasets we will look at the dataset used in the community to compare analysis tools, and in @sec:rasta-soa we will go through the contributions that benchmarked those tools #jm-note[to see if they can be used as an indication as to which tools can still be used today.] [Mettre en avant] +In @sec:bg-datasets we will look at the dataset used in the community to compare analysis tools. +Then in @sec:bg-bench> we will go through the contributions that benchmarked those tools #jm-note[to see if they can be used as an indication as to which tools can still be used today.][Mettre en avant] -===== Application Datasets +==== Application Datasets Research contributions often rely on existing datasets or provide new ones in order to evaluate the developed software. Raw datasets such as Drebin@Arp2014 contain few information about the provided applications. @@ -74,7 +60,7 @@ Currently, Androzoo contains more than 25 millions applications, that can be dow Androzoo also provide additionnal information about the applications, like the date the application was detected for the first time by Androzoo or the number of antivirus from VirusTotal that flaged the application as malicious. In addition to providing researchers with an easy access to real world applications, Androzoo make it a lot easier to share datasets for reproducibility: instead of sharing hundreds of #APK files, the list of SHA256 is enough. -===== Benchmarking +==== Benchmarking The few datasets composed of real-world application confirmed that some tools such as Amandroid~@weiAmandroidPreciseGeneral2014 and Flowdroid~@Arzt2014a are less efficient on real-world applications~@bosuCollusiveDataLeak2017 @luoTaintBenchAutomaticRealworld2022. Unfortunatly, those real-world applications datasets are rather small, and a larger number of applications would be more suitable for our goal, #ie evaluating the reusability of a variety of static analysis tools. @@ -158,10 +144,12 @@ DroidBench@Arzt2014a #v(2em) -Reaves #etal raised two major concern for the use of Android static analysis tools. +To summariz, Li #etal made a systematic literature review of static analysis for Android that listed 27 opensourced tools. +However, they did not tested those tools. +Reaves #etal did so for some of them and analysed the difficulty of using them. +They raised two major concern for the use of Android static analysis tools. First, they can be quite difficult to setup, and second, they appear to have difficulties analysing read-world applications. -This is problematic for a reverser engineer, not only do they need to invest a significan amont of work to setup a tool properly, they do not have any guarantees that the tool will actually manage to analyse the application they are investigating. -Hence our first problem statement #pb1: - -#pb1-text +This is problematic for a reverser engineer, not only do they need to invest a significant amont of work to setup a tool properly, they do not have any guarantees that the tool will actually manage to analyse the application they are investigating. +In @sec:rasta, we will try to setup the tools listed by Li #etal and test them on a large number of real-world applications to see wich can be used today. +We will also aim at identify what caracteristic of real-world applications make them harder to analyse. diff --git a/2_background/4_soa.typ b/2_background/4_soa.typ index 11b85a6..b214576 100644 --- a/2_background/4_soa.typ +++ b/2_background/4_soa.typ @@ -1,6 +1,8 @@ == State of the Art +This section focus on the state of the art related to our three probleme statements: the reusability of Android static analysis tools, the class loading mechanism of Android, and the use of instrumentation to encode information collected dynamically. + #include("4_1_rasta.typ") #include("4_2_classloader.typ") #include("4_3_theseus.typ") diff --git a/4_class_loader/2_classloading.typ b/4_class_loader/2_classloading.typ index 87c05ef..4d5e453 100644 --- a/4_class_loader/2_classloading.typ +++ b/4_class_loader/2_classloading.typ @@ -55,7 +55,7 @@ When used directly by #ART, the classes are usually stored in an application fil Another dotted line labeled instance singleton goes from bootClassLoader to BootClassLoader. " ) - gray -- Java-based, white -- Android-based + grey -- Java-based, white -- Android-based ], caption: [The class loading hierarchy of Android] )