A Label Has to Match the Code Under It
The next BenchProctor release adds eight weakness classes the corpus never covered, and generates each one only in the languages where it can honestly exist. A CWE label that names the wrong weakness costs the tool that names the right one.
On the standalone corpus, BenchProctor’s default scoring is strict about names. A finding counts only when it carries the CWE the case is labeled with, so the label is part of the answer. That makes a wrong label expensive in a specific way: the tool that correctly names the weakness in front of it is marked as a miss, because the key expected a neighbor’s number.
So ahead of the next release, every category was rendered and read against the CWE it claims. Three kinds of change came out of that read.
Eight weakness classes the corpus did not have
Some weaknesses a scanner is expected to find had no category at all. The next release adds them:
- Time-of-check to time-of-use races (CWE-367), incorrect default permissions (CWE-276), and an uncontrolled search path element (CWE-427), across all eleven code languages.
- Sensitive data left in the browser cache (CWE-525), across the web languages.
- Regular expressions with catastrophic worst-case matching (CWE-1333).
- Three Java EE classes: calling
System.exit()inside a server application (CWE-382), a critical public field without a final modifier (CWE-493), and data sent without transport encryption (CWE-5).
Each arrives the way every other category does: a vulnerable form and a safe twin rendered from the weakness itself, balanced, with nothing in a comment or a name that gives the label away.
Only where the language can have it
The regex class shows the rule best. Catastrophic backtracking is a property of the matching engine, not of the pattern alone. Go and Rust match in linear time by design. On the Java and Ruby versions the corpus builds against, the pattern shapes this class uses measured fast. C and Bash use POSIX matchers these patterns are not written for. A vulnerable case in any of those languages would carry a label its own runtime refutes, so the class is generated in Python, JavaScript, TypeScript, PHP, and C++, and nowhere else.
The class also carries a trap reviewers fall for: a length cap in front of the match looks like a fix, but these patterns are still catastrophically slow at the capped length, so a capped case is labeled vulnerable.
The same rule applies everywhere: where a language has no honest form of a weakness, the category is left out of that language rather than rendered as a look-alike under the wrong number.
The right number on the right code
Some labels needed correcting rather than adding. Reentrancy on the Solidity side now carries CWE-1265, the entry for unintended reentrant invocation through nested calls, instead of a broader workflow entry.
Every category added from here follows the same rule: narrow a category before rendering a neighbor’s shape under the wrong number.
The principle is the one in a wrong answer key punishes the tools that get it right. Naming the weakness is its own score for a reason, as one score hides the gap lays out, and that score is only as fair as the labels under it.