#import "../lib.typ": etal
#import "X_var.typ": *

== Introduction

Android is the most used mobile operating system since 2014, and since 2017, it even surpasses Windows all platforms combined#footnote[https://gs.statcounter.com/os-market-share#monthly-200901-202304].
The public adoption of Android is confirmed by application developers, with 1.3 millions apps available in the Google Play Store in 2014, and 3.5 millions apps available in 2017#footnote[https://www.statista.com/statistics/266210]. 
Its popularity makes Android a prime target for malware developers. // For example, various applications have been shown to steal personal information@shanSelfhidingBehaviorAndroid2018.
Consequently, Android has also been an important subject for security research. 
In the past fifteen years, the research community released many tools to detect or analyze malicious behaviors in applications. Two main approaches can be distinguished: static and dynamic analysis@Li2017. 
Dynamic analysis requires to run the application in a controlled environment to observe runtime values and/or interactions with the operating system. 
For example, an Android emulator with a patched kernel can capture these interactions but the modifications to apply are not a trivial task. 
// Such approach is limited by the required time to execute a limited part of the application with no guarantee on the obtained code coverage. 
// For malware, dynamic analysis is also limited by evading techniques that may prevent the execution of malicious parts of the code. // To explain better if we restore these sentences about malware + evading.
As a consequence, a lot of efforts have been put in static approaches, which is the focus of this paper.  
 
The usual goal of a static analysis is to compute data flows to detect potential information leaks@weiAmandroidPreciseGeneral2014 @titzeAppareciumRevealingData2015 @bosuCollusiveDataLeak2017 @klieberAndroidTaintFlow2014 @DBLPconfndssGordonKPGNR15,@octeauCompositeConstantPropagation2015,@liIccTADetectingInterComponent2015 by analyzing the bytecode of an Android application.
The associated developed tools should support the Dalvik bytecode format, the multiplicity of entry points, the event driven architecture of Android applications, the interleaving of native code and bytecode, possibly loaded dynamically, the use of reflection, to name a few. 
All these obstacles threaten the research efforts. 
When using a more recent version of Android or a recent set of applications, the results previously obtained may become outdated and the developed tools may not work correctly anymore.


In this paper/*#footnote[This work was supported by the ANR Research under the Plan France 2030 bearing the reference ANR-22-PECY-0007.]*/, we study the reusability of open source static analysis tools that appeared between 2011 and 2017, on a recent Android dataset. 
The scope of our study is *not* to quantify if the output results are accurate for ensuring reproducibility, because all the studied static analysis tools have different goals in the end. 
On the contrary, we take as hypothesis that the provided tools compute the intended result but may crash or fail to compute a result due to the evolution of the internals of an Android application, raising unexpected bugs during an analysis. 
This paper intends to show that sharing the software artifacts of a paper may not be sufficient to ensure that the provided software would be reusable. 

Thus, our contributions are the following. 
We carefully retrieved static analysis tools for Android applications that were selected by Li #etal@Li2017 between 2011 and 2017. 
We contacted the authors, whenever possible, for selecting the best candidate versions and to confirm the good usage of the tools.
We rebuild the tools in their original environment and we plan to share our Docker images with this paper.
We evaluated the reusability of the tools by measuring the number of successful analysis of applications taken /*in the Drebin dataset@Arp2014 and */ in a custom dataset that contains more recent applications (#NBTOTALSTRING in total). 
The observation of the success or failure of these analysis enables us to answer the following research questions: 

/ RQ1: What Android static analysis tools that are more than 5 years old are still available and can be reused without crashing with a reasonable effort?
/ RQ2: How the reusability of tools evolved over time, especially when analyzing applications that are more than 5 years far from the publication of the tool?
/ RQ3: Does the reusability of tools change when analyzing goodware compared to malware?

/*
As a summary, the contributions of this paper are the following:

- We provide containers with a compiled version of all studied analysis tools, which ensures the reproducibility of our experiments and an easy way to analyze applications for other researchers. Additionally receipts for rebuilding such containers are provided.
- We provide a recent dataset of #NBTOTALSTRING applications balanced over the time interval 2010-2023.
- We point out which static analysis tools of Li #etal SLR paper@Li2017 can safely be used and we show that #resultunusable of evaluated tools are unusable (considering that a tool that fails more than 50% of time is unusable). In total, the success rate of the tools we could run is #resultratio on our dataset.
- We discuss the effect of applications features (date, size, SDK version, goodware/malware) on static analysis tools and the nature of the issues we found by studying statistics on the errors captured during our experiments.
*/

The paper is structured as follows. 
@sec:rasta-soa presents a summary of previous works dedicated to Android static analysis tools. 
@sec:rasta-methodology presents the methodology employed to build our evaluation process and @sec:rasta-xp gives the associated experimental results.  
// @sec:rasta-discussion investigates the reasons behind the observed failures of some of the tools. 
@sec:rasta-discussion discusses the limitations of this work and gives some takeaways for future contributions.
@sec:rasta-conclusion concludes the paper.


