Phase 0: Interrogating QRadar's API Before You Trust It

The rule I gave myself: don’t assume the API

#

The tempting way to start a pipeline like this is to read the QRadar API guide, decide “I will create Custom Rules over REST” and start coding the deployer. That is also how you build three phases on top of a capability that does not exist on your build.

Phase 0 of What The Sigma has one job: find out, empirically, what the live QRadar instance allows, then let that answer decide the architecture. Nothing downstream gets written against an assumption I have not tested on a real system.

The published docs already hint at the problem. QRadar’s /analytics/rules endpoint exposes GET, POST and DELETE on {id}, but it reads like a surface for managing existing rules, not authoring a Custom Rule Engine (CRE) rule from scratch. CRE rules involve a tree of tests and building blocks that the REST surface does not obviously let you construct. Meanwhile, /ariel/saved_searches supports creation directly.

That suggests a working assumption: treat the Ariel saved search as the deployable unit, not the CRE rule. A saved search holds the converted AQL, can be created through the API, and is what a “scheduled search + threshold” CRE rule consumes. But an assumption is not a finding, so I wrote a tool to test it.

The prober

#

src/probe_api.py is deliberately read-mostly. It calls the endpoints the pipeline depends on, records what the instance says, and writes the evidence to docs/phase0-api-probe.json. Each probe is paired with the question it answers, so the JSON is still readable months later:

PROBES = [
    ("system_about",  "GET", "/system/about",   "Does the SEC token authenticate at all?"),
    ("api_versions",  "GET", "/help/versions",   "Which API versions does this build accept?"),
    ("endpoints",     "GET", "/help/endpoints",  "Full endpoint inventory for capability checks."),
    ("ariel_databases","GET","/ariel/databases", "Are the Ariel event/flow tables reachable?"),
    ("saved_searches","GET", "/ariel/saved_searches", "Can saved searches be listed?"),
    ("analytics_rules","GET","/analytics/rules", "Can CRE rules be listed?"),
    ("offenses",      "GET", "/siem/offenses",   "Can offenses be read for validation?"),
    ("log_sources",   "GET", "/config/event_sources/log_source_management/log_sources",
     "Is the Windows/Sysmon log source registered?"),
]

Every probe is wrapped so it never raises - a 403 on one endpoint shouldn’t abort the sweep, because a 403 is itself a finding:

def probe(client, method, path):
    try:
        data = client.request(method, path, headers={"Range": "items=0-4"})
    except QRadarAPIError as exc:
        return {"ok": False, "status": exc.status, "error": exc.body[:300]}
    ...

The one write it’s allowed to make

#

There is one question you cannot answer by reading: can this API create a saved search? Listing them proves only read access. The prober therefore performs one self-cleaning write. It creates a throwaway saved search and immediately deletes it:

def probe_saved_search_write(client, prefix):
    """Answer the central Phase 0 question: can we create a saved search?"""
    payload = {
        "name": f"{prefix} phase0-probe-delete-me",
        "description": "Temporary object created by src/probe_api.py; safe to delete.",
        "aql": "SELECT * FROM events LAST 1 MINUTES",
        "database": "EVENTS",
        "is_shared": True,
    }
    try:
        created = client.create_saved_search(payload)
    except QRadarAPIError as exc:
        return {
            "create_supported": False,
            "status": exc.status,
            "implication": "Saved searches are not API-creatable on this build - "
                           "fall back to exporting AQL for manual import, or use "
                           "reference data + a pre-built CRE rule.",
        }
    # ... then delete what we just made, and record whether that worked too

The object name carries a prefix, [DaC] by default. Every object this pipeline creates in QRadar carries that prefix, so the lab can be cleaned up by name without touching anything created by hand. Even the probe’s throwaway object is easy to find.

make probe runs the whole thing and prints a status line per endpoint, then the saved-search verdict:

Probing 192.168.56.10 (API version 20.0)

  [ok  ] GET  /system/about
  [ok  ] GET  /help/versions
  [ok  ] GET  /ariel/saved_searches
  [ok  ] GET  /analytics/rules
  ...
  Testing saved-search creation ...
    create supported: True

Every call goes through one door

#

The prober, the deployer and the validator all share one thin REST client, src/qradar_client.py. Authentication, API versioning, TLS policy and error formatting live in exactly one method:

def request(self, method, path, **kwargs):
    url = f"{self.settings.base_url}/{path.lstrip('/')}"
    response = self.session.request(method, url, timeout=self.settings.timeout, **kwargs)
    if response.status_code >= 400:
        raise QRadarAPIError(method, path, response.status_code, response.text)
    ...

Auth is a SEC token in an HTTP header, and QRadar wants an explicit Version header on every request - get that wrong and the same call behaves differently between builds. Both are set once, in the session:

self.session.headers.update({
    "SEC": settings.token,
    "Accept": "application/json",
    "Version": settings.api_version,
})
self.session.verify = settings.verify_tls

Two lab realities are handled here on purpose rather than hidden. QRadar CE ships a self-signed certificate, so TLS verification defaults to off - but as the setting QRADAR_VERIFY_TLS, not a hardcoded verify=False, so it’s a lab decision you can see and reverse. And the SEC token never touches a tracked file; it comes from the environment (.env, gitignored, or GitHub Secrets in CI) and a missing one fails loudly at startup instead of surfacing as a confusing 401 three calls later.

Fallbacks, ranked, before I need them

#

The point of Phase 0 is to know the answer before the architecture depends on it. If the probe reports that saved-search creation is unavailable on a given build, the fallbacks are already chosen, in order of preference:

1. Export the AQL, import it through the UI. The pipeline stays the generator and the diffing tool. You lose the automation, you keep the versioning.
2. Push rule content into a reference set via /reference_data/sets and have a few hand-built CRE rules consume it. Automates the content, not the rule structure.
3. Drive the UI with browser automation. Fragile, last resort.

The deployer is written so switching between these options is a small change rather than a rewrite - the saved-search assumption lives in one payload builder and four client methods. More on that in the deployment post.

Lab sizing, honestly

#

Phase 0 also surfaced the boring blocker. Measured on my dev machine: 46 GB RAM, 14 cores, 113 GB free disk. QRadar CE 7.5.0 wants 24 GB RAM / 250 GB disk / 4 cores. RAM and CPU are fine; disk is the problem, 113 GB against a 250 GB requirement. The options are free up ~150 GB and run 7.5.0 (my preference, since 7.5.x is what IBM ships today and its API version 20.x is the one worth learning), drop to the lighter 7.3.3 branch with its older 9.x API, or rent a cloud VM for a short window. QRADAR_API_VERSION in .env is the one setting that changes between them.

That is the unglamorous truth of detection-as-code: the hardest dependency is not the code, but the 24 GB SIEM. It is also why the pipeline does as much as possible before that SIEM exists.

Next: how the conversion works: two pipelines and why the choice matters.