Tuning alert fatigue: signal scoring in practice

Alert fatigue is a detection engineering problem, not a tooling problem. Most SOC teams that suffer from it are running the same rules they shipped two years ago, applied uniformly to every endpoint, with every rule weighted identically. The result is a queue that is 60% noise - and analysts who stop looking carefully because looking carefully never changed anything.

Signal scoring is the fix. The idea is simple: not all detections are equal. A process injection on a developer laptop running a debugger is different from the same event on a finance workstation at 02:00. Signal scoring builds that context into the detection pipeline, so your alert queue reflects actual risk rather than raw rule matches.

What signal scoring looks like in practice

Signal scoring assigns a confidence value to each detection event, then adjusts that value based on context before it becomes an alert. The three main inputs are:

• Base confidence - how reliable is this rule? A Sigma rule for powershell.exe -EncodedCommand has a high true-positive rate on most estates; a rule for cmd.exe spawned by Word might be 40% noise depending on your environment.
• Environmental context - is this endpoint in a high-risk segment? Is it flagged for unusual activity this week? Is the account a service account or a real person?
• Temporal context - is this happening during business hours? Has the same detection fired three times in 10 minutes?

These inputs combine to produce an adjusted score. Alerts below a threshold go to a “review later” queue; alerts above a higher threshold get immediate attention.

A practical way to track this is a per-rule tuning table:

Rule	Base confidence	FP rate (30d)	Adjusted weight
Encoded PowerShell execution	0.80	12%	0.70
LSASS memory read	0.95	3%	0.92
Office spawning shell	0.75	22%	0.58
SSH lateral movement (internal)	0.90	8%	0.83
Scheduled task creation	0.55	45%	0.30

This table lives in a spreadsheet or your SIEM config, not in your head. Review it monthly. If a rule’s 30-day FP rate is above 30%, it needs tightening - narrower conditions, allowlisted hashes, or a parent process filter - before it earns a place in your primary queue.

Practical tuning steps

Start with your noisiest rules, not your newest. Pull the top 10 rules by alert volume from the last 30 days. For each one, work out the FP rate from your triage records. Anything above 25% false positives should be tightened before you touch anything else.

Use parent process conditions. The single most effective tuning technique on Windows is filtering by parent process. cmd.exe spawned by explorer.exe is different from cmd.exe spawned by winword.exe. Adding a ParentImage condition to office-spawn rules cuts FP rate by 40-60% on most estates without losing real detections.

Build an allowlist for known-good software. Developer tools, monitoring agents, and backup software generate large volumes of events that match generic rules. An allowlist file - hash-based where possible, path-based where necessary - stops you re-triaging the same benign processes every week.

Cluster before you alert. A single event rarely tells you much. Five events from the same host in a 15-minute window tells you a lot. A clustering step in your pipeline - whether that is a SIEM correlation rule or an AI clustering layer - means analysts see “five related events” as one finding rather than five separate notifications.

Here is an example Sigma rule with a tighter parent condition that illustrates the pattern:

title: Office Application Spawning Shell
status: stable
logsource:
  category: process_creation
  product: windows
detection:
  selection:
    ParentImage|endswith:
      - '\winword.exe'
      - '\excel.exe'
      - '\powerpnt.exe'
    Image|endswith:
      - '\cmd.exe'
      - '\powershell.exe'
      - '\wscript.exe'
      - '\cscript.exe'
  condition: selection
falsepositives:
  - Macro-enabled templates from trusted vendors (add to allowlist by hash)
level: high
tags:
  - attack.execution
  - attack.t1059.001

The ParentImage filter does most of the noise suppression. Without it, any cmd.exe launch triggers the rule. With it, you are targeting a specific, high-risk relationship.

When to automate and when to review

Not every alert warrants human triage. Alerts below a confidence threshold are often better handled by automated enrichment - pulling process trees, checking hash reputation, querying the last 90 days of telemetry for the same event - and then re-scoring before an analyst sees them.

This is where AI-assisted triage adds real value: not as a replacement for an analyst, but as a pre-triage step that turns a raw event into an enriched, explained finding. The analyst confirms or dismisses in seconds rather than spending 10 minutes reconstructing context from scratch.

The goal is not zero alerts. It is a queue where every alert represents a genuine decision point - something where human judgement matters and the context to make that judgement is already in front of you.

If you are working through a tuning backlog and want to see how LightEDR’s behavioural detection and AI triage layer handles signal scoring, get in touch.