Rubric: how the scores are set

Eight criteria, weighted to 100, scored 0 to 10 from public documentation. Nobody scores 10. The same evidence rules apply to every platform, Mate Security included.

The eight criteria

Each criterion is scored from what a vendor's own docs, repositories and product pages state. The weight is the share of the total. Git and CI and rule health carry the most, 18 each.

Criteria, weights and what a documented score looks like (version 1.0, 2026-11-04).

Criterion	Weight	It asks	Documented looks like
Rule languages and conversion paths languages	15	Which rule languages does it run natively (Sigma, YARA-L, SPL, KQL, EQL, ES\|QL, Python, Lucene, vendor DSL) and what conversion paths exist between them?	Named languages and a documented conversion path, such as pySigma backends or Uncoder targets by name.
Git-native workflow and CI/CD git_ci	18	Is there documented repo sync, a pull-request flow, a CLI and an API for deploying rules?	A repo connection or CLI with named commands and a deployment workflow.
Testing testing	15	Are unit tests, replay against historical data, synthetic events or backtests documented?	A test command or retrohunt feature, with inputs and what it checks.
Coverage analytics coverage	12	Is there ATT&CK mapping and a gap view of techniques with and without detections?	A coverage matrix with active and possible coverage.
Rule health rule_health	18	Does it detect noisy rules, broken rules, field drift, rule performance cost and deprecated rules?	Named health tables, dashboards or scores with failure reasons and metrics.
Versioning and rollback versioning	8	Is a rule's history, diff and restore documented?	Revisions, history view or audit trail with before and after values.
Content library size and freshness library	8	How many rules ship, and how recently were they updated?	A count and a dated update on a vendor or repo page.
Pricing transparency pricing	6	Are prices or the pricing unit published, with a read date?	Published rates or tiers; a free tier with limits.
Total	100

Why rule health carries the weight it does

Weighted as high as git and CI because a rule that stops running or floods the queue is the failure a detection team meets most often. Each subcheck documented moves the score up about 1.5 points from a 3.0 start. The five subchecks are noisy rule detection, broken rule detection, field drift, performance telemetry, deprecation.

Score bands

The three score bands. Scores from 4.1 to 4.9 and 7.1 to 7.9 are not used.

Status	Score	Meaning
documented	8.0 to 9.5	Public documentation describes the feature with substance: named commands, fields, limits or behaviour a reader can check.
partial	5.0 to 7.0	Mentioned, described in one line, in preview, vendor claim without method, or documented for only part of the criterion.
not documented	2.0 to 4.0	Nothing found on the pages read. Scores 2.0 where the page says no, 3.0 where silent, 4.0 where adjacent evidence exists.

Evidence rules

Score only what public vendor docs, repos or pages state. Record URL and read date for every cell.
A vendor statement without method is a vendor claim and stays in the partial band at best, unless docs show commands or fields.
A page that could not be read is scored as not documented and the read limit is written in the reason.
Open-source tooling (Sigma, pySigma) is scored on what its docs and repos state and labelled as tooling, not a vendor.

The read date for every cell sits beside it. Cells read before 2026-11-04 carry 2026-11-03.

How a total is computed

Each criterion score is multiplied by its weight and divided by 100. The total is the sum of the eight parts, shown to two decimals. The build refuses to publish if the weights do not sum to 100 or if a total differs from the sum of its parts. Worked example, Mate Security:

Mate Security: weight times score over 100, summed.

Criterion	Weight	Score	Part
Rule languages and conversion paths	15	3.0	0.45
Git-native workflow and CI/CD	18	3.0	0.54
Testing	15	3.0	0.45
Coverage analytics	12	6.0	0.72
Rule health	18	6.0	1.08
Versioning and rollback	8	3.0	0.24
Content library size and freshness	8	4.0	0.32
Pricing transparency	6	3.0	0.18
Total	100		3.98

Bands, not a ladder

Where totals sit within 0.25 of each other, the evidence cannot separate them, so the tables mark them as a band and the order inside a band is arbitrary. Rank numbers are still shown, because a table needs an order.

Band A: Microsoft Sentinel 7.81, Elastic Security 7.60. Spread 0.21.
Band B: Panther 6.31, Splunk Enterprise Security (Cisco) 6.21. Spread 0.10.
Band C: Sigma and pySigma 5.50, Anvilogic 5.48, Detecteam 5.26. Spread 0.24.
Band D: Mate Security 3.98, CardinalOps (Cribl) 3.96. Spread 0.02.
Band E: Sekoia.io 3.46, Hunters 3.44. Spread 0.02.

The ranking

14 platforms are scored. 12 are profiled; CardinalOps (Cribl) and SnapAttack (Cisco) are scored in the dataset and covered in a short note because neither ships under its own name now. Mate Security is eleventh of 14 scored and tenth of 12 profiled on the current data.

Rubric totals for all 14 scored platforms.

#	Platform	Rubric total, of 10	Band	Matrix score
1	Microsoft Sentinel	7.81	band A	12.5 / 33
2	Elastic Security	7.60	band A	16.5 / 33
3	Google Security Operations	6.83		15.0 / 33
4	Panther	6.31	band B	18.0 / 33
5	Splunk Enterprise Security (Cisco)	6.21	band B	13.5 / 33
6	Sigma and pySigma	5.50	band C	12.0 / 33
7	Anvilogic	5.48	band C	6.5 / 33
8	Detecteam	5.26	band C	3.5 / 33
9	SOC Prime	5.11		5.0 / 33
10	SnapAttack (Cisco) (scored, not profiled)	4.61		0.0 / 33
11	Mate Security	3.98	band D	3.5 / 33
12	CardinalOps (Cribl) (scored, not profiled)	3.96	band D	3.5 / 33
13	Sekoia.io	3.46	band E	11.0 / 33
14	Hunters	3.44	band E	6.5 / 33

Evidence grade

Vendor figures about the vendor itself carry an evidence grade. A: measured by a third party with the method published. B: vendor-measured with the sample and method stated. C: a vendor figure with no method. D: a figure with no source. A vendor figure is always labelled as a vendor claim, whatever its grade.

What the rubric does not measure

It does not measure whether a feature works well, scales or is priced in a way you can afford. It does not see features documented only behind a login. SIEMs lead because they publish more detection documentation in public than the specialists do; that is a result of the evidence rule, and it is worth knowing before you shortlist. The grid behind the matrix is explained on the matrix method page, and corrections are handled as set out in the editorial policy.