Skip to main content

Data redaction

InfraSage masks secrets and personal data at three points. Each is a separate switch, because each protects a different boundary.

WhereWhat is maskedSwitchDefault
Ingestion gateway, before storageLog bodies, span and log attributes, Prometheus series labelsINGEST_REDACTION = off, secrets or piioff (pii in values-enterprise.yaml)
Alert evidenceSample log lines, log patterns and extracted fields on the alert, and so in Slack, email and pager messagesEVIDENCE_REDACTION=off turns it offon
CopilotTool results, the context bundle and replayed tool history sent to the modelREDACT_PROMPTS_ENABLED (also covers RCA prompts)on

What counts as secret or PII​

secrets masks values that are secrets by shape: PEM private keys, AWS access key ids, GitHub, Slack and Google tokens, JWTs, Bearer and Basic credentials, and any password=, token:, "api_key": "..." style pair. An attribute whose key names a credential (db.password, http.request.header.authorization, x-api-key, cookie) is masked whatever its value.

pii adds:

  • Card numbers, spaced or dashed, only with a real network's prefix and length and a valid Luhn check digit. Timestamps and most ids don't qualify. About one in ten ids that happen to have a card's shape pass Luhn by chance and are masked too.
  • IBANs that pass the ISO 13616 mod-97 check.
  • Email addresses, US SSNs (123-45-6789) and E.164 phone numbers (with the leading +).

Masked values become typed placeholders such as [CARD], [EMAIL] or [SECRET]. That way a reader can still tell what was there. Identity fields are never rewritten, because grouping, joins and trace lookups depend on them: service, environment, trace and span ids, status codes, route, host and Kubernetes names, and job, instance and le labels.

The gateway counts what it masks in infrasage_gateway_redacted_values_total{where="log_body|attribute|series_label"}.

Limits​

This is pattern matching, not a classifier. It won't find names, street addresses or free-text account numbers. If your logs carry those, remove them at the source. The collector recipe below does that before data leaves your network, and it is the stronger control of the two.

Redaction applies to data ingested after it is turned on. Rows already stored keep their values until retention removes them.

Collector recipe​

Run this in your OpenTelemetry Collector to mask data before it reaches InfraSage. Add your own attribute names to the pattern and your own formats to blocked_values. Checked against collector-contrib 0.115.

processors:
  # Deletes credential-named attributes outright.
  attributes/drop-credentials:
    actions:
      - pattern: "(?i).*(password|passwd|secret|token|api[_-]?key|authorization|cookie).*"
        action: delete
  # Masks card numbers and emails in every attribute that remains.
  redaction:
    allow_all_keys: true
    blocked_values:
      - "\\b(?:4[0-9]{12}(?:[0-9]{3})?|5[1-5][0-9]{14}|3[47][0-9]{13})\\b"   # Visa, Mastercard, Amex
      - "\\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Za-z]{2,}\\b"              # email
    summary: silent
  # The redaction processor reads attributes only; log bodies need transform.
  transform/log-bodies:
    log_statements:
      - context: log
        statements:
          - replace_pattern(body, "\\b(?:4[0-9]{12}(?:[0-9]{3})?|5[1-5][0-9]{14}|3[47][0-9]{13})\\b", "[CARD]")
          - replace_pattern(body, "\\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Za-z]{2,}\\b", "[EMAIL]")
          - replace_pattern(body, "(?i)(password|token|api[_-]?key)=\\S+", "$$1=[SECRET]")

service:
  pipelines:
    logs:
      processors: [attributes/drop-credentials, redaction, transform/log-bodies, batch]
    traces:
      processors: [attributes/drop-credentials, redaction, batch]

The collector's patterns don't check Luhn digits, so they mask more than InfraSage's pii mode does. Run both if you can: the collector keeps the data inside your network, and the gateway catches whatever the recipe misses.