Skip to content

Imbalanced Data

Pilz handles heavily imbalanced datasets without SMOTE or manual resampling. It samples each class independently and gives both classes equal total weight in the grouped data, which makes it naturally robust to skewed class distributions.

The Problem

Most ML algorithms struggle when one class dominates (e.g., 99% non-target, 1% target). Standard approaches:

  • SMOTE: Generates synthetic samples of the minority class
  • Class weights: Penalizes misclassifications of the minority class more heavily
  • Manual resampling: Down-sample majority or up-sample minority

Pilz avoids SMOTE and manual resampling; its own balancing is described below.

Independent Sampling

Because target and non-target are sampled independently — each from the pre-shuffled cache with .head(max_eval_fit) — both sides contribute equally to each node:

def read_akt_train(self, targer_filter, train_settings, akt_filters):
    target_df = self._get_pl_train_df(
        full_filters=full_filter_target,
        max_eval_fit=train_settings.max_eval_fit,
    )
    non_target_df = self._get_pl_train_df(
        full_filters=full_filters_non_target,
        max_eval_fit=train_settings.max_eval_fit,
    )

A dataset with 1% target and 99% non-target produces the same max_eval_fit rows for each — say 1000 target and 1000 non-target — regardless of the original proportions.

Equal-Weight Grouping

After sampling, the data is split into count and group sets. The group set is weighted so each class contributes equal total mass:

# Each class half carries total weight 0.5, regardless of how many rows it has.
w_target = 0.5 / max(1, target_group.height)
w_non_target = 0.5 / max(1, non_target_group.height)
group = pl.concat([target_group, non_target_group]).hstack(
    [
        pl.Series(
            "weight",
            np.concatenate(
                [
                    np.full(target_group.height, w_target),
                    np.full(non_target_group.height, w_non_target),
                ]
            ),
        ),
        pl.Series(
            "target_weight",
            np.concatenate(
                [
                    np.full(target_group.height, 2 * w_target),
                    np.zeros(non_target_group.height),
                ]
            ),
        ),
    ]
)

Each side gets total weight 0.5. If one side has more rows, each individual row on that side gets a proportionally smaller weight. Target rows additionally carry target_weight = 2 * w_target, while non-target rows get 0.0. This means the correlation tables always reflect a balanced view.

Per-Node Re-Balancing

The balancing happens at every node in the tree, not just once at the root:

def train_pilz(
    self,
    target_filter: Filter,
    path_filter: list[Filter],
    depth: str = "",
    parent_features: list[str] | None = None,
) -> list[Spore]:
    train_df = self.darkwing.read_akt_train(
        targer_filter=target_filter,
        train_settings=self.settings,
        akt_filters=path_filter,
    )

As the tree splits and data becomes purer, the remaining rows are re-sampled and re-balanced at each recursive call. A deep node that has only a few hundred target rows left in the original data still gets its max_eval_fit rows taken from the pre-shuffled cache.

The Score Function

The default leaf score (leaf_score="log_odds") is the Laplace-smoothed log-odds of the node's sampled rows, so small leaves shrink towards the balanced score 0 via the pseudo-count leaf_alpha (default 0.5):

def score(self, alpha: float | None = None) -> float:
    sum = self.non_target_df_size + self.target_df_size
    if sum == 0:
        return 0.0
    if alpha is None:  # legacy weakened_diff
        ...
    return math.log(
        (self.target_df_size + alpha) / (self.non_target_df_size + alpha)
    )

(Training passes alpha=leaf_alpha for leaf_score="log_odds" and None for the legacy formula.)

Because both sides are sampled to similar sizes, this score is meaningful regardless of the original class balance. A node with mostly target rows (after filtering) gets a high positive score. The legacy "weakened_diff" formula instead scales the signed target rate by a weakening factor 1 - ln(n) / n, bounding scores to [-1, 1].

Comparison

Approach Requires tuning? Works out of the box? Handles 99:1 imbalance?
SMOTE Yes (k neighbors) No Poor
Class weights Yes (ratio) No Moderate
Manual resampling Yes (ratio) No Moderate
Pilz No Yes Yes

Summary

Concept Description
Independent sampling Each class sampled separately with .head(max_eval_fit)
Equal weight Both sides get total weight 0.5 in the group set
Per-node balancing Every tree node re-samples from the pre-shuffled cache
No SMOTE needed The approach naturally handles any class ratio

Next Steps