Testing YARA rules: samples, goodware and CI

A rule that has never been tested is a guess. Testing answers three questions: does it compile, does it match what it should, and does it stay silent on everything else? This guide sets up a simple routine you can run by hand and in CI.

Why test

• Syntax drift. A rule written for YARA 4 may fail to compile with YARA-X, or the other way round.
• Missed detections. A typo in a hex string or an overly strict condition makes a rule silently useless.
• False positives. Generic strings match installers, browsers and system DLLs. In production that means alert fatigue and lost trust.
• Performance regressions. One slow pattern degrades every scan (see performance).

Build a small test corpus

Keep two directories next to your rules:

• tests/positive/ — samples the rule must match. For malware, store them encrypted (for example in a password-protected archive) or reference them by hash and fetch them in a controlled environment.
• tests/goodware/ — clean files the rule must not match: system binaries from several OS versions, common installers, office documents, scripts. Larger is better; public goodware collections exist, and your own golden images are ideal.

For logic that doesn't require real malware, craft tiny synthetic files. A file that contains only your marker strings in the right place is enough to test a condition.

# A synthetic positive: MZ header + the two strings the rule expects
printf 'MZ' > tests/positive/synthetic.bin
head -c 200 /dev/zero >> tests/positive/synthetic.bin
printf 'LoaderInit\0GetProcAddress\0' >> tests/positive/synthetic.bin

Compile and format checks

The YARA-X command-line tool yr has the checks you need:

yr check rules/            # compile everything, report errors and warnings
yr fmt rules/example.yar   # rewrite a file in the canonical format
yr scan rules/example.yar tests/positive/   # must print every positive sample
yr scan rules/example.yar tests/goodware/   # must print nothing

With YARA 4.x, yarac compiles and yara -r scans recursively.

A test rule to start from

rule MAL_Test_Target
{
    meta:
        description = "Target rule used in the testing guide"
        author = "yararules.org"
        date = "2026-09-29"
    strings:
        $a = "LoaderInit" ascii
        $b = "GetProcAddress" ascii
    condition:
        uint16(0) == 0x5A4D and filesize < 1MB and all of them
}

The synthetic file above matches it; any clean executable that lacks LoaderInit does not. Try both in the online editor: paste the rule, drop the file on the test pane, and see the matched strings with their offsets.

Continuous integration

Run the same checks on every change. A minimal GitHub Actions workflow:

name: yara
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install YARA-X
        run: cargo install yara-x-cli
      - name: Compile
        run: yr check rules/
      - name: Positive samples must match
        run: |
          for f in tests/positive/*; do
            yr scan rules/ "$f" | grep -q . || { echo "no match: $f"; exit 1; }
          done
      - name: Goodware must not match
        run: |
          out=$(yr scan rules/ tests/goodware/ || true)
          if [ -n "$out" ]; then echo "$out"; exit 1; fi

Installing from crates.io builds YARA-X from source; you can also download a prebuilt release binary from the YARA-X releases page to speed the job up.

Track false positives over time

• Record every false positive with the file hash, the rule and the fix, then add the file to the goodware corpus so it never regresses.
• Prefer adding a $fp* exclusion or a tighter structural check over deleting a useful string.
• Consider a score meta field (0–100) so that consumers can weigh weak rules differently from high-confidence ones.

Version your rules

Keep metadata that makes rules maintainable: a stable id (a UUID), date of creation, modified date, author, reference, and sample hash values. The style guide lists common conventions. When a rule changes behaviour, update modified so downstream users know to re-test.

Quick checklist

1. yr check passes with no errors (and warnings reviewed).
2. Every positive sample matches.
3. The goodware corpus produces zero matches.
4. The linter shows no performance warnings, or each one is justified.
5. Metadata is complete and modified is current.