YARA performance: atoms, fast strings and conditions

A single slow rule can make a scan of thousands of files crawl, because every rule in a compiled set shares the same scanning pass. Understanding how YARA searches makes the difference between a rule that costs microseconds and one that dominates the scan.

How scanning works

Scanning happens in two phases:

1. Pattern search. At compile time, YARA picks from every pattern a short fixed sequence of up to four bytes, the atom. All atoms of all rules are put into one Aho-Corasick automaton, and the data is scanned once. Wherever an atom occurs, YARA verifies the full pattern at that position.
2. Condition evaluation. For each rule, the condition is evaluated using the matches found, plus file properties and module data.

The cost of phase 1 depends on how often atoms occur in the data. An atom like E8 45 FF 0A is rare; an atom like 00 00 occurs millions of times in any binary, and each occurrence triggers a verification.

Good and bad atoms

YARA-X warns at compile time about some patterns that are likely to be slow; the editor displays those warnings, and the linter adds heuristics for the rest.

Writing fast strings

• At least four fixed bytes. For text, prefer distinctive strings of eight characters or more. For hex, keep at least one run of four consecutive fixed bytes, ideally at the start.
• Avoid common bytes. Runs of 00, FF, 90 (NOP), CC (int3) or spaces are everywhere.
• Wildcard only what varies. In code patterns, wildcard relative addresses and immediates that change between builds, not the opcodes.
• Use nocase, xor and base64 on long strings only. Each expands one pattern into many atoms.
• Anchor regexes with literals. /evil[0-9]{2}\.exe/ gives the atom evil; a class-only regex gives nothing.

rule Fast_Code_Pattern
{
    strings:
        // Four fixed bytes up front, wildcards only on the rel32 operand.
        $decrypt = { 8A 04 0E 34 ?? 88 04 0F 41 81 F9 ?? ?? ?? ?? 72 }
        $cfg = "cfg_version=" ascii
    condition:
        uint16(0) == 0x5A4D and filesize < 3MB and all of them
}

Cheap guards first

Conditions short-circuit from left to right. Put the checks that reject most files, cheaply, at the front:

import "pe"
import "math"

rule Ordered_Condition
{
    strings:
        $a = "ReflectiveLoader" ascii
    condition:
        uint16(0) == 0x5A4D and           // one integer read
        filesize < 4MB and                // no I/O at all
        $a and                            // already computed by the scan
        pe.number_of_sections > 3 and     // module data
        math.entropy(0, 0x1000) < 7.0     // bounded computation
}

A filesize limit also protects you from huge files that happen to contain a matching atom.

Loops and module functions

• Loop over matches (for any i in (1..#a)), never over every offset of the file (for any i in (0..filesize)).
• math.entropy, hash.* and other functions over (0, filesize) read the whole file; limit the range or guard it.
• pe.imports() and similar functions are cheap once the module has parsed the file, but parsing itself is skipped for files that fail an earlier uint16(0) == 0x5A4D check.

Measuring

Measure rather than guess:

• Time yr scan rules.yar corpus/ on a representative corpus before and after a change.
• Compare the whole rule set with and without a suspicious rule.
• Tools such as yaraQA check rule sets for known performance and logic issues.
• The rule pages of this library show a YARA Forge quality score that already reflects automated QA.

Checklist

1. Every pattern has four or more consecutive fixed bytes.
2. No pattern is made mostly of 00, FF, 90, CC or whitespace.
3. nocase, xor and base64 only on strings of eight characters or more.
4. Regexes contain a literal of three characters or more and no leading .*.
5. The condition starts with magic bytes and a filesize bound.
6. Loops iterate over matches; module functions use bounded ranges.

Paste your rule into the linter to check points 1–6 automatically, then test it in the editor. For how to test rules systematically, continue with testing and CI.