Performance
KICS Accuracy Benchmark¶
The values below were obtained after scanning 150 open source projects with KICS (1.2.x) covering the supported IaC technologies (c.f., Terraform, Ansible, Kubernetes, Docker, AWS Cloudformation).
IaC Technology | Query Accuracy1 | Query Coverage2 | Scanned IaC files | Number of Results | Average Scan Time (s) | Average Project Size (MB) |
---|---|---|---|---|---|---|
Terraform | 99.7% | 46% | 1176 | 709 | 6.6 | 33.4 |
Docker | 98.8% | 68% | 1017 | 5109 | 11 | 0.7 |
Kubernetes | 99.3% | 88.7% | 6089 | 21753 | 7 | 90 |
CloudFormation | 95% | 73% | 1769 | 5343 | 10.2 | 4.8 |
Ansible | 100% | 54% | 3367 | 1320 | 23.3 | 4.1 |
1 Query Accuracy = TP results / results
2 Query Coverage = Query with results / Queries
Global Measures¶
Measure | Value |
---|---|
Average Accuracy | 98.6% |
Total Number of Results | 34234 |
Average Query Coverage | 66.4% |
Total Scanned IaC Files | 13418 |
Average Scan Time (s) | 11.2 |
Average Project Size (MB) | 26.6 |
KICS Profiling¶
Running Kics with --profiling
flag will log the CPU/MEM metrics used for:
- Getting Queries
- Parsing Files
- Scanning Vulnerabilities
- Generating Reports
Keep in mind that profiling will periodically stop KICS to retrieve the wanted metrics, meaning KICS execution time will increase substantially.
CPU Profiling¶
Flag: --profiling CPU
9:43AM INF Scanning with Keeping Infrastructure as Code Secure dev
9:43AM INF Total CPU usage for get_queries: 6.56s <-
9:43AM INF Inspector initialized, number of queries=1385
9:43AM INF Total CPU usage for get_sources: 200.00ms <-
9:43AM INF Total CPU usage for inspect: 15.43s <-
9:43AM INF Results saved to file results/results.json fileName=results.json
9:43AM INF Results saved to file results/results.sarif fileName=results.sarif
9:43AM INF Results saved to file results/results.html fileName=results.html
9:43AM INF Total CPU usage for generate_report: 290.00ms <-
9:43AM INF Files scanned: 221
9:43AM INF Parsed files: 221
9:43AM INF Queries loaded: 1385
9:43AM INF Queries failed to execute: 0
9:43AM INF Inspector stopped
9:43AM INF Scan duration: 21.1476197s
¶
9:43AM INF Scanning with Keeping Infrastructure as Code Secure dev
9:43AM INF Total CPU usage for get_queries: 6.56s <-
9:43AM INF Inspector initialized, number of queries=1385
9:43AM INF Total CPU usage for get_sources: 200.00ms <-
9:43AM INF Total CPU usage for inspect: 15.43s <-
9:43AM INF Results saved to file results/results.json fileName=results.json
9:43AM INF Results saved to file results/results.sarif fileName=results.sarif
9:43AM INF Results saved to file results/results.html fileName=results.html
9:43AM INF Total CPU usage for generate_report: 290.00ms <-
9:43AM INF Files scanned: 221
9:43AM INF Parsed files: 221
9:43AM INF Queries loaded: 1385
9:43AM INF Queries failed to execute: 0
9:43AM INF Inspector stopped
9:43AM INF Scan duration: 21.1476197s
MEM Profiling¶
Flag: --profiling MEM
9:43AM INF Scanning with Keeping Infrastructure as Code Secure dev
9:43AM INF Total MEM usage for get_queries: 237.96MB <-
9:43AM INF Inspector initialized, number of queries=1385
9:43AM INF Total MEM usage for get_sources: 280.53MB <-
9:43AM INF Total MEM usage for inspect: 335.44MB <-
9:43AM INF Results saved to file results/results.json fileName=results.json
9:43AM INF Results saved to file results/results.sarif fileName=results.sarif
9:43AM INF Results saved to file results/results.html fileName=results.html
9:43AM INF Total MEM usage for generate_report: 333.38MB <-
9:43AM INF Files scanned: 221
9:43AM INF Parsed files: 221
9:43AM INF Queries loaded: 1385
9:43AM INF Queries failed to execute: 0
9:43AM INF Inspector stopped
9:43AM INF Scan duration: 21.1476197s