113 lines
3.4 KiB
Markdown
113 lines
3.4 KiB
Markdown
# Directory structure
|
|
* docker
|
|
Contains one directory per tool
|
|
Each tool directory should have a RASTA_VERSION file that contains the subdir with the tested version
|
|
* tester
|
|
A Python module to analyse the output of the tools, and detect errors
|
|
* envs
|
|
One file per tool, used to setup the ENV variables in the containers.
|
|
This env file MUST define a numerical TIMEOUT
|
|
|
|
# Installation
|
|
|
|
- Install docker
|
|
|
|
''''
|
|
apt install docker.io
|
|
''''
|
|
|
|
- Install singularity
|
|
|
|
''''
|
|
apt install singularity-ce
|
|
''''
|
|
|
|
# Scripts
|
|
|
|
## grunt-worker-launcher.sh
|
|
|
|
A script specifically designed to launch one instance on a cluster node. Typically, it would be passed to a batch command (on a cluster that is managed with slurm). This script is probably highly dependant on the cluster setup. There is little sense in manually launching this script.
|
|
|
|
## grunt-worker.py
|
|
|
|
Contains the bulk of the logic to:
|
|
|
|
- Obtain tasks (from a redis server). Here a task is a couple (APK, TOOL_NAME)
|
|
- check whether this task was already done
|
|
- create tmp dir
|
|
- Download the APK from AndroZoo
|
|
- run an analysis through a docker (`--docker`) or singularity (`--singularity`) container
|
|
- analyse the output of the analysis, and detect errors
|
|
- delete tmp dir
|
|
- save the results (into a couch database)
|
|
|
|
Also has a `--manual` mode, which is the simplest way to manually launch a task, in particular when coupled with the options to deactivate CouchDB (`--no-write-to-couch`) and Redis (`--no-mark-done`), and the option to not delete the tmp dir (`--keep-tmp-dir`).
|
|
|
|
## build_docker_images.sh
|
|
|
|
To batch create all Docker and Singularity images.
|
|
|
|
Parameter: the dir where the singularity files will be placed.
|
|
|
|
## launch-container.sh
|
|
|
|
- Called by grunt-worker.py.
|
|
- Can also be called manually to debug.
|
|
|
|
Parameters:
|
|
|
|
1. Mode: Either DOCKER or SINGULARITY
|
|
2. TOOL_NAME: for example, androguard or blueseal, etc
|
|
3. CONTAINER_IMG: Either the name of the Docker image or the path to the sif file (without the trailing .sif)
|
|
4. TMP_WORKDIR: a dir
|
|
5. APK_FILENAME: the name of the APK file provided in TMP_WORKDIR (This script does NOT download apks)
|
|
|
|
# How to run
|
|
|
|
1. Choose the tool(s) you want to build the docker/singularity image by editing the file `./build_docker_images.sh on the line tools=. For example, to build didfail, change the line like below. By default, the script builds the docker/singularity image of all tools.
|
|
|
|
'''
|
|
tools="didfail"
|
|
'''
|
|
|
|
|
|
2. Create Docker and Singularity images (around 16 minutes on a modern laptop)
|
|
`./build_docker_images.sh path_you_want_the_sif_files_in` for example:
|
|
|
|
'''
|
|
bash build_docker_images.sh ~/singularity
|
|
'''
|
|
|
|
3. Create a venv
|
|
|
|
'''
|
|
python3 -m venv rasta-venv
|
|
source rasta-venv/bin/activate
|
|
'''
|
|
|
|
4. Install necessary python package
|
|
|
|
'''
|
|
python3 -m pip install -r requirements.txt
|
|
'''
|
|
|
|
5. Launch one manual analysis
|
|
|
|
- 5.0: fill in the settings.ini file with your Androzoo api key:
|
|
|
|
'''
|
|
[AndroZoo]
|
|
apikey = your_api_key
|
|
'''
|
|
|
|
- 5.1: launch the singularity container on a given hash of Android application:
|
|
|
|
'''
|
|
./grunt-worker.py --base-dir /tmp/RASTA/ --no-mark-done --keep-tmp-dir --no-write-to-couch --manual --task didfail --sha APK_HASH --singularity --image-basedir SINGULARITY_IMAGE_DIRECTORY
|
|
'''
|
|
|
|
For example:
|
|
|
|
'''
|
|
./grunt-worker.py --base-dir /tmp/RASTA/ --no-mark-done --keep-tmp-dir --no-write-to-couch --manual --task didfail --sha 0003468487C29A71A5DA40F59E4F1F5DFF026126DD64BB58C572E30EE167C652 --singularity --image-basedir ~/singularity
|
|
'''
|