49 lines
5 KiB
Typst
49 lines
5 KiB
Typst
#import "../lib.typ": eg, jfl-note, MWE
|
|
|
|
== Recommendations <sec:rasta-reco>
|
|
|
|
In the light of our findings in @sec:rasta-failure-analysis and the issues we met while packaging the tools, we summarize some takeaways that we believe developers should follow to improve the success of reusing their software.
|
|
|
|
//*developer*: dire que a la lumiere de ces resultats, on peut pense que certain pbs peuvent être évité ou bien corrigé par l'utilisateur]
|
|
We understand software developped for research purposes are not and should not be held to the same standards as production sofwares.
|
|
However, research is incremental and it is not sustanable to start each tools from scratch.
|
|
It is critical to be able to build upon tools already published, and efforts should be made to allows that when releasing a tool.
|
|
|
|
Durint the packaging and testing of the tools we examined in our experiment, the most notable issues we encontered could have been avoided by following classical development best practices.
|
|
To make a tool easy to reuse, it should have a documentation with at least:
|
|
- Instructions about how to install the dependencies.
|
|
- Instructions about how to build the tool (if the tool need to be build).
|
|
- Instructions about how to use the tool (#eg command line arguments)
|
|
- Instructions about how to interpret the results of the tools (we only checked for the existance of the results in our experiment, but we found that some results can be quite obscure)
|
|
In addition to the documentation, a minimum working example with the expected result of the tools allows a potential user to check if everything is working as intended.
|
|
This #MWE have the additionnal benefit that is can serve as an example in the documentation.
|
|
|
|
Another best practice to follow is to pin the version of dependencies of the tools.
|
|
Many modern dependency management tools can handle that: for instance for python, poetry or uv generate a lock files with the exact version of the libraries to use, cargo does the same for rust, in java this can be an option in gradle, and dependencies in maven `pom.xml` files are usually the exact version.
|
|
For other dependencies that are not managed by a dependency manager -- for instance the java virtual machine tu use, the python interpreter, resource files -- the version to use sould be clearly documented.
|
|
Alternatively, tools like nixpkg can be used to pin every dependencies.
|
|
The worst case we encontered during our experiment was a tool whose documentation instructed to install the z3 dependencies with a simple `git clone`, whithout specifying the commit to use.
|
|
The z3 project being still actively maintained, the dependency installed was not compatible, and finding a compatible version required checking releases one by one.
|
|
Dependencies fetched with version control system should alway indicate the exact version to used (in the case of git, a commit, tag or release should be used).
|
|
|
|
We also found that interactions with the running environment can become verry problematic when the environment changes.
|
|
To minimized the issues, packaging the tool inside a docker container or even a virtual machine can ensure that future users have at least access to a working version of the tool.
|
|
|
|
Finally, when possible, continuous integration, tests and code reviews should be implemented to improve the reliability of the developped tool.
|
|
|
|
Concerning the actual code of the tool, more attention should be paid to error repporting.
|
|
When a tool failed to perform its analysis, is should be clear to the user, and the reason should be clearly reported.
|
|
In some cases, this may imply _not_ trying to recover from unrecoverable errors: this often leads to error seemingly unrelated to the initial issue.
|
|
This is often a problem in Java code where the developers are strongly encouraged to catch all exceptions, and in bash scripts that run several programs in a row without checking the exit statuses.
|
|
|
|
Good error repporting can allow futur user to solve issues encontered using the tools: for instance the log generated by Androguard's decompiler clearly show that the issue is file names exceeding the size limit.
|
|
This issue could easily fixed by changing the filenames used to store the results.
|
|
In contrast, the error generated by Flowdroid are so opaque that we have no idea how we could solve them.
|
|
|
|
And at last, an important remark concerns the libraries used by a tool.
|
|
We have seen two types of libraries:
|
|
- internal libraries manipulating internal data of the tool.
|
|
- external libraries that are used to manipulate the input data (APKs, bytecode, resources).
|
|
We observed during our manual investigations that external libraries are the ones leading to crashes because of variations in recent APKs (file format, unknown bytecode instructions, multi-DEX files).
|
|
We believe that the developer should provide enough documentation to make possible a later upgrade of these external libraries.
|
|
For example, old versions of apktool are the top most libraries raising errors, but breaking changes introduced by upgrading from v1.X versions to v2.X versions preventing use from upgrading apktool.
|