Query-Driven Subgroup Discovery

Finding the diverse subgroups that respond most to an intervention, within a search area defined by a query

Paper (ICDE 2026) Code on GitHub

Description

Understanding how changes in actions or policies impact different population segments (users, items, etc.) is central to data-driven decision-making, but it is time-consuming and labor-intensive. We introduce a novel framework that combines user-defined search areas with causal inference to automatically identify responsive subgroups, i.e., subgroups most likely to benefit from a specific action or intervention, by estimating the impact of interventions, while also promoting subgroup diversity to ensure broad and representative insights. To support this, we develop a formal problem definition and an efficient search strategy to navigate the large combinatorial space of possible subgroups. We propose efficient algorithms that use pretrained causal effect estimators to dynamically surface the top-K subgroups. Experiments across multiple datasets and estimation techniques demonstrate the scalability and effectiveness of our framework for large-scale analysis.


Overview

An analyst defines a search area with a query (e.g. WHERE age > 40), and the system returns the K subgroups inside it that respond most to a treatment, are diverse, and are each large enough to matter.

A causal effect estimator (e.g. a Causal Forest) is trained once, offline, and yields a tree of candidate subgroups. At query time, the system keeps the subgroups that fall inside the search area and picks the top K with one of four algorithms that balance responsiveness against diversity:

  • DiRe: weighted sum of responsiveness and diversity.
  • ReCoD: maximizes diversity among the most responsive subgroups.
  • DiCoR: maximizes responsiveness with a minimum diversity.
  • LoRe: picks the most responsive subgroups from a single tree level.

Key Results

We evaluate on synthetic, semi-synthetic and real-world (Criteo Uplift) datasets:

  • Pretrained Causal Forests find more responsive subgroups than Causal Trees, and training once offline avoids retraining for every query.
  • DiCoR reaches 94% of the optimal responsiveness found by exhaustive search, at the same diversity.
  • LoRe and DiCoR stay fast on datasets of up to 30M rows.

Code

The code and experiments are available on GitHub.

Key Persons

Antonis Mandamadiotis

Georgia Koutrika