Applications: A First Public Release of Whole-Project Benchmarks
Single-file cases measure one thing at a time. Real systems are whole projects. The application shape gets its first public release: deployable, multi-language projects with real weaknesses planted across services.
A single-file test can measure whether a scanner spots a sink. It can never measure whether the scanner follows taint across a service boundary, a language change, or a queue, because there is no boundary in one file. That is the part that ships bugs to production, and it is the part a snippet benchmark cannot ask about. The application shape exists to close that gap, and it is now getting its first public release.
What an application release contains
Each application is a small, idiomatic project: two or three application-tier languages plus one or two cloud or infrastructure targets, a handful of files, and several real weaknesses planted across its services. Some carry a compound chain, a set of planted steps that only add up to an escalation once a tool connects them. Every project compiles as a unit under its native toolchains, with real dependency manifests, the way a repository you would clone does.
The release ships what you need to score any tool against it: public answer keys for the planted flows and for the chains, the same standard-library scorer, per-suite manifests, and checksums. The ground truth lives in the CSVs, never in the code a scanner reads.
Built and run before it ships
An application benchmark that does not run is not a benchmark, it is a guess with directories. So the application release gate builds each suite per language and framework, runs the deterministic gate suite over it, and a runtime gate builds and runs representative applications under Docker. Nothing publishes that did not first stand up.
An honest first cut
This is v0.1: the first public application corpus, at real scale across the languages. Generated all-combos at depth 10, it is expected to run to millions of files, a figure that firms up only once the release is produced. It is the beginning of the multi-file axis, not the end of it. Read what an application looks like in Whole Apps, Not Snippets, and how the surface composes in Compose the Benchmark to Your Stack.