Studies & Publications
Reproducible cybersecurity research from OMARINO IT Services.
We publish technical studies with transparent methods, measured results and the evidence needed to repeat our work. Each publication distinguishes verified findings from untested capabilities and explains its limitations.
DEFENSIVE SECURITY RESEARCH · PROJECT 2026-001
Evaluating Automated Web Security Assessment Accuracy
A Reproducible Regression Study Using OMARINO AmanCheck
Publisher: OMARINO IT Services inh. Omar Zaror, Germany
Published: 3 November 2026 · Research: 2–3 November 2026
Format: English technical paper, open-access PDF and sanitized reproducibility package.
What we investigated
Automated security tools can mistake missing evidence for a vulnerability, confuse an indicator with proof, or assign an ineffective severity. We evaluated AmanCheck’s seven web-assessment probes against 32 controlled local scenarios: TLS/HTTPS, HTTP security headers, cookies, DNS evidence, public exposure, redirects/performance and technology fingerprinting.
The original baseline and a separately corrected research variant each ran three times per scenario, producing 192 recorded observations. A separate acquisition path captured raw HTTP, TLS and local DNS evidence. No customer or third-party infrastructure was scanned.
Measured results
| Measure | Original baseline | Corrected research variant |
| Precision / recall (31 binary claims) | 70.0% / 70.0% | 100.0% / 100.0% |
| True positives / false positives | 14 / 6 | 20 / 0 |
| False negatives / true negatives | 6 / 5 | 0 / 11 |
| Agreement including the abstention case | 19 / 32 | 32 / 32 |
| Severity agreement under the study rubric | 18 / 32 | 32 / 32 |
| Exact redirect-count assertions | 0 / 2 | 2 / 2 |
Interpretation: These are targeted regression-suite results. The fixes were developed against the same source-informed cases; there was no held-out test set. A perfect post-fix result does not establish perfect real-world scanner accuracy.
Key findings
- Certificate-chain and hostname errors were detected but assigned low severity in the baseline.
- Presence-only header checks accepted an ineffective framing value and empty Content Security Policy.
- Cookie parsing confused Comment text with SameSite evidence and misapplied a severity threshold.
- DNS placeholders reported missing controls without querying the relevant record types; the corrected probe abstains when evidence is unavailable.
- Public-exposure checks misclassified informational files and generic success pages, while missing selected chunked responses and GET-only exposure.
- Automatic redirect following concealed hops from the manual tracking loop.
- Broad technology-name matching produced false signals from ordinary prose.
Download the study and evidence
Download technical paper (PDF) Download reproducibility package (ZIP)
The sanitized package contains the selected baseline and corrected C# probes, an executable .NET laboratory, expected outcomes, raw CSV/JSON data, independent observations, metric definitions, patches, source hashes and SHA-256 integrity manifests. Original repository history, deployment configuration, private keys and the separate embargoed advisory are excluded.
Scope and limitations
The experiment ran on loopback using .NET / ASP.NET Core 10.0.14, SDK 10.0.403, on macOS Arm64. The original probe source was compiled unchanged for the baseline on that runtime. The full product and production deployment were not tested. A Docker runner is supplied but was not executed. DNSSEC, DKIM, browser policy enforcement, HTTPS-cookie cases and a production DNS resolver remain outside the validated scope.
The three repetitions establish consistency within this laboratory, not 96 independent configurations per version. Pooled precision and recall describe the heterogeneous fixture claims, including technology and redirect signals; they are not population-level vulnerability-detection estimates. The paper documents these limits and the next experiments.
Cite this publication
OMARINO IT Services inh. Omar Zaror (2026). Evaluating Automated Web Security Assessment Accuracy: A Reproducible Regression Study Using OMARINO AmanCheck. OMARINO Cybersecurity Research, Project 2026-001. Published 3 November 2026. https://www.omarino.de/studies-publications
Questions about the study or responsible vulnerability reporting? Contact OMARINO.