This commit is contained in:
parent
a42e0dc058
commit
a1e6682b3c
3 changed files with 101 additions and 13 deletions
|
|
@ -206,8 +206,6 @@ To guarantee reproducibility, we published the results, datasets, Dockerfiles an
|
|||
- on Docker Hub as `histausse/rasta-<toolname>:icsr2024`.
|
||||
]
|
||||
|
||||
#todo[alt text for @fig:rasta-methodo-collection]
|
||||
|
||||
#figure(
|
||||
raw-render(```
|
||||
digraph {
|
||||
|
|
@ -259,7 +257,21 @@ To guarantee reproducibility, we published the results, datasets, Dockerfiles an
|
|||
"Pack": ("Drop2": align(center, block(inset: 1em)[Could Not Set Up\ in 4 days])),
|
||||
),
|
||||
width: 100%,
|
||||
alt: "",
|
||||
alt: (
|
||||
"A diagram showing our methodology. ",
|
||||
"On the left their is a text 'Tools from Li et al', and a arrow ",
|
||||
"from the text to a box labelled 'Search Tools'. ",
|
||||
"An arrow goes down from this box, to the text 'Drop', and ",
|
||||
"another to the right to a box labelled 'Select Tools'. ",
|
||||
"An arrow goes down from this box, to the text 'Drop', with the ",
|
||||
"text 'Uses Dynamic Analysis' right of the arrow. ",
|
||||
"Another arrows goes to the right of the box to a box labelled ",
|
||||
"'Select Source Version'. An arrow goest to the left of the box ",
|
||||
"to a box labelled 'Package'. An arrow goes down from this box ",
|
||||
"to the text 'Not Reusable', with the text 'Could Not Set up in ",
|
||||
"4 days' to the right of the arrow. Another arrow goest to the ",
|
||||
"right of the box, to the text 'Docker Images'"
|
||||
).join(),
|
||||
),
|
||||
caption: [Tool selection methodology overview],
|
||||
) <fig:rasta-methodo-collection>
|
||||
|
|
|
|||
|
|
@ -110,14 +110,26 @@ For the tools that we could run, #resultratio of analyses are finishing successf
|
|||
|
||||
=== #rq2: Size, #SDK and Date Influence
|
||||
|
||||
#todo[alt text for fig rasta-exit-evolution-java and rasta-exit-evolution-not-java]
|
||||
|
||||
#figure(stack(dir: ltr,
|
||||
[#figure(
|
||||
image(
|
||||
"figs/finishing-rate-by-year-of-java-based-tools.svg",
|
||||
width: 50%,
|
||||
alt: ""
|
||||
alt: (
|
||||
"A graph showing the finishing rate (from 0 to 100) on the y-axe ",
|
||||
"and the year the applications were first seen (from 2010 to ",
|
||||
"2023) on the x-axes. There is a line for each of the following ",
|
||||
"tools: anadroid, blueseal, dialdroid, didfail, droidsafe, ",
|
||||
"flowdroid, gator, ic3, ic3_fork, iccta, perfchecker and saaf. ",
|
||||
"Saaf, ic3, ic3_fork and gator starts in 2010 between 95% and ",
|
||||
"100%, blueseal around 90%, flowdroid 70%, didfail 60%, ",
|
||||
"perfchecker, iccta and dialdroid are around 45% and 55%, ",
|
||||
"gator 40%, anadroid 15%. They all drop over time, finishing ",
|
||||
"arround 75% for gator, 60% for ic3, 40% for perfecker, and ",
|
||||
"between 0 and 20% for all the rest. ",
|
||||
"Noticeably, saaf drop suddently between 2014 and 2017, and ",
|
||||
"ic3_fork start dropping after 2017."
|
||||
).join()
|
||||
),
|
||||
caption: [a) Java-based tools],
|
||||
supplement: none,
|
||||
|
|
@ -127,7 +139,18 @@ For the tools that we could run, #resultratio of analyses are finishing successf
|
|||
image(
|
||||
"figs/finishing-rate-by-year-of-non-java-based-tools.svg",
|
||||
width: 50%,
|
||||
alt: "",
|
||||
alt: (
|
||||
"A graph showing the finishing rate (from 0 to 100) on the y-axe ",
|
||||
"and the year the applications were first seen (from 2010 to ",
|
||||
"2023) on the x-axes. There is a line for each of the following ",
|
||||
"tools: adagio, amandroid, androguard, androguar_dad, apparecium ",
|
||||
"mallodroid, redexer, wognsen_et_al. Wognsen_et_al starts at 75% ",
|
||||
"in 2010, the other start arround 95 and 100%. Wognsen_et_al and ",
|
||||
"androguar_dad start dropping from the beggining, redexer start ",
|
||||
"dropping in 2020, the other are vaguely stable. At the end, in ",
|
||||
"2023, mallodroid is at 100%, apparecium 95%, amandroid 80%, ",
|
||||
"redexer 60%, androguard_dad 35%, wognsen_et_al 5%."
|
||||
).join()
|
||||
),
|
||||
caption: [b) Non-Java-based tools],
|
||||
supplement: none,
|
||||
|
|
@ -178,13 +201,23 @@ sqlite> SELECT apk1.first_seen_year, (COUNT(*) * 100) / (SELECT 20 * COUNT(*)
|
|||
|
||||
To compare the influence of the date, #SDK version and size of applications, we fixed one parameter while varying another.
|
||||
|
||||
#todo[Alt text for fig rasta-decorelation-size]
|
||||
#figure(stack(dir: ltr,
|
||||
[#figure(
|
||||
image(
|
||||
"figs/decorelation/finishing-rate-of-java-based-tool-by-bytecode-size-of-apks-detected-in-2022.svg",
|
||||
width: 50%,
|
||||
alt: ""
|
||||
alt: (
|
||||
"A graph showing the finishing rate (from 0 to 100) on the y-axe ",
|
||||
"and the bytecode size of the applications were first seen (from ",
|
||||
"0 to 40 MB) on the x-axes. There is a line for each of the ",
|
||||
"following tools: anadroid, blueseal, dialdroid, didfail, ",
|
||||
"droidsafe, flowdroid, gator, ic3, ic3_fork, iccta, perfchecker ",
|
||||
"and saaf. At 0MB, Gator start at 100%, ic3 at 90%, flowdroid ",
|
||||
"60%, blueseal and ic3_fork 40%, the other between 20 an 0%.",
|
||||
"At the end (40MB), ic3 finish at 50%, gator 35%, the other ",
|
||||
"between 0 and 20%. Except for local variations, they all ",
|
||||
"decrease with bytecode size."
|
||||
).join()
|
||||
),
|
||||
caption: [a) Java-based tools],
|
||||
kind: "sub-rasta-decorelation-size-2022",
|
||||
|
|
@ -194,7 +227,17 @@ To compare the influence of the date, #SDK version and size of applications, we
|
|||
image(
|
||||
"figs/decorelation/finishing-rate-of-non-java-based-tool-by-bytecode-size-of-apks-detected-in-2022.svg",
|
||||
width: 50%,
|
||||
alt: "",
|
||||
alt: (
|
||||
"A graph showing the finishing rate (from 0 to 100) on the y-axe ",
|
||||
"and the bytecode size of the applications were first seen (from ",
|
||||
"0 to 40 MB) on the x-axes. There is a line for each of the ",
|
||||
"following tools: adagio, amandroid, androguard, androguard_dad ",
|
||||
"apparecium, mallodroid, redexer, wognsen_et_al. At 0MB, ",
|
||||
"wognsen_et_al. starts at 25%, redexer 75%, the other between ",
|
||||
"95 and 100%. Apparecium, mallodroid, androguard, and adagio ",
|
||||
"stay stable over time, the other visibly drop: redexer finishes ",
|
||||
"at 40%, amandroid 35%, redexer and androguard_dad 0%."
|
||||
).join()
|
||||
),
|
||||
caption: [b) Non-Java-based tools],
|
||||
kind: "sub-rasta-decorelation-size-2022",
|
||||
|
|
|
|||
|
|
@ -129,13 +129,46 @@ Sometimes errors successfully caught and handled are logged anyway.
|
|||
Thus, it is impossible to accurately extract the error responsible for a failed execution.
|
||||
Therefore, we investigated the nature of errors globally, without distinction between error messages in a log.
|
||||
|
||||
#todo()[alt text for rasta-heatmap]
|
||||
|
||||
#figure(
|
||||
image(
|
||||
"figs/repartition-of-error-types-among-tools.svg",
|
||||
width: 100%,
|
||||
alt: "",
|
||||
alt: (
|
||||
"A greyscale heatmap. On the y-axis they are arror types, and ",
|
||||
" on the x-axis, the tools. Black means the error type correspond ",
|
||||
"to 100% of the error raised by the tool, white 0%. The heatmap is ",
|
||||
"mostly white, with a few black squares and a little more grey ones. ",
|
||||
"amandroid has a black square for java.lang.NullPointerException. ",
|
||||
"anadroid has grey squares for subprocess.CalledProcessError, ",
|
||||
"java.io.IOException and brut.androlib.AndrolibException. ",
|
||||
"androguard has grey squares for struct.error, KeyError and ",
|
||||
"androguard.core.bytecodes.dvm.InvalidInstruction. ",
|
||||
"androguard_dad has a black square for OSError. ",
|
||||
"apparecium has a black square for KeyError. ",
|
||||
"blueseal has a black square for java.lang.RuntimeException. ",
|
||||
"dialdroid has grey squares for ",
|
||||
"java.util.concurent.RejectedExecution, java.lang.StackOverflowError",
|
||||
"and com.google.common.util.concurrent.ExecutionError. ",
|
||||
"didfail has grey squares for java.util.concurent.RejectedExecution ",
|
||||
"and java.io.FileNotFoundException. ",
|
||||
"droidsafe has a dark grey square for ",
|
||||
"droidsafe.utils.CannotFindMethodException. ",
|
||||
"flowdroi has a grey square at java.lang.NullPointerException, and ",
|
||||
"other light grey squares. ",
|
||||
"gator has a grey square at java.lang.OutOfMemoryError, and other ",
|
||||
"light grey squares. ",
|
||||
"ic3 has a black square at jas.jasError. ",
|
||||
"ic3_fork has has a grey square at java.lang.RuntimeException. ",
|
||||
"iccta has grey squares at java.lang.RuntimeException, ",
|
||||
"java.lang.StackOverflowError and com.google.common.utils.concurrentExecutionError. ",
|
||||
"mallodroid has a black square for KeyError. ",
|
||||
"perfchecker has grey squares at java.lang.UnsatifiedLinkError and ",
|
||||
"java.lang.VerifyError. ",
|
||||
"redexer has no colored square at all. ",
|
||||
"saaf has grey squares for java.io.IOException, de.rub.syssec.saaf.model.analysis.AnalysisException and brut.androlib.AndrolibException. ",
|
||||
"wognsen_et_al has grey squares for brut.androlib.AndrolibException, ",
|
||||
"java.io.IOException and java.lang.ArithmeticException."
|
||||
).join()
|
||||
),
|
||||
caption: [Heatmap of the ratio of error reasons for all tools for the RASTA dataset],
|
||||
) <fig:rasta-heatmap>
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue