Multi-Dimensional Splits¶

The n_dims parameter controls how Pilz finds correlations between features. This is the core innovation that makes Pilz special — it can split on multiple features simultaneously rather than one at a time.

What is n_dims?¶

n_dims defines how many features are combined in a single split evaluation:

Why Multi-Dimensional Matters¶

The Correlation Problem¶

When features correlate, their combination is more predictive than either alone:

Traditional vs Pilz¶

A traditional decision tree needs multiple sequential splits to capture a pairwise correlation. Pilz captures it in a single multi-dimensional cut:

How It Works¶

_find_winner() scores every single feature by calc_diff() and then delegates the combination search to the native pilz_rs.counter_batch. It returns (best_feat, winner_features), where best_feat is the winning categorized feature (or combination) and winner_features names its feature or combination:

def _find_winner(self, train_df: TrainDataframes):
    sorted_train_feats = sorted(
        zip(
            train_df.train_features,
            train_df.train_features_by_rep,
            train_df.feature_mask,
        ),
        key=lambda x: x[0].calc_diff(),
        reverse=True,
    )
    feature_mask = [usable for _, _, usable in sorted_train_feats]
    ...
    res = pilz_rs.counter_batch(
        target_codes, non_target_codes, label_counts,
        n_features, n_reps, dim1_scores,
        self.settings.n_dims,
        self.settings.calcs_per_dim,
        self.settings.min_split_improvement,
        self.settings.score_method,
        self.settings.smoothing,
        feature_mask,
        self.settings.pair_corr_filter,
        self.settings.pair_corr_min_main,
        self.settings.pair_corr_max_features,
    )
    if res.winner_corr is not None:
        best_feat = CombinedCategorizedFeature.from_fused(
            train_features=tuple(
                sorted_train_feats[f][1][res.winner_median_rep]
                for f in res.winner_features
            ),
            corr=res.winner_corr,
            score_method=self.settings.score_method,
            smoothing=self.settings.smoothing,
        )
        winner_features = [feat.feature.name for feat in best_feat._train_features]
    ...
    return best_feat, winner_features

counter_batch evaluates the dimension stages 2..=n_dims one after another over all repetitions, applies the Pair-Correlation Filter between candidates, and returns the winner (its feature combination, median repetition index and joint correlation table) plus the updated feature mask. The winner's correlation table is turned back into a CombinedCategorizedFeature via from_fused, which keeps the cuts and count series that were used during the native scoring.

Repeated Combination Evaluation¶

When n_rep is greater than one, Pilz keeps the categorized candidate for every repetition. This is important for multi-dimensional splits because the count rows are shuffled independently for each repetition. A combination must therefore only combine features from the same repetition.

For example, with two features and n_rep=3, Pilz evaluates:

(feature_a_rep1, feature_b_rep1)
(feature_a_rep2, feature_b_rep2)
(feature_a_rep3, feature_b_rep3)

It never combines (feature_a_rep1, feature_b_rep2). Each of the three combined candidates has its own cuts and count series, so the joint bin table is built from correctly aligned rows. The median candidate by calc_diff() is then used for that feature combination, preserving the stability behavior of single-feature repetition.

The same process applies to every dimension. With n_dims=3, every triplet is also built once per repetition before its median candidate is selected. The calcs_per_dim limit counts feature combinations, not the individual repetitions used to evaluate each combination.

Step 1: Score Individual Features¶

Each feature is scored by calc_diff(), delegated to the native pilz_rs.score. The scoring method is selected with the score_method training setting:

• "diff_all_<exp>" — dimension-neutral sum 0.5 * sum(|p - q| ** exp).
• "diff_dir_<exp>" (default "diff_dir_1.5") — dominant-side directional sum of |p - q| ** exp.
• "lift_dir_<exp>" — dominant-side, evidence-weighted sum of the log-likelihood lift n * |ln(n / E)| ** exp, with n the cell count and E the expected count under independence. At exp = 1.0 the sums are the directional parts of the G deviance.
• "hellinger" — Hellinger distance 0.5 * sum((sqrt(p) - sqrt(q)) ** 2).

The diff_* families aggregate the plain proportion difference p - q; the lift_dir_* family uses the cell counts and their expected values, and hellinger takes the square roots of the proportions. With smoothing > 0 the proportions and counts used by all scoring methods are Laplace-smoothed:

def calc_diff(self) -> float:
    if self._score is None:
        self._score = pilz_rs.score(self._corr, self.score_method)
    return self._score

Feature Sorting¶

Before building combinations, _find_winner() sorts all features by their calc_diff() score descending. The single best feature becomes sorted_train_feats[0]:

sorted_train_feats = sorted(
    zip(
        train_df.train_features,
        train_df.train_features_by_rep,
        train_df.feature_mask,
    ),
    key=lambda x: x[0].calc_diff(),
    reverse=True,
)
feature_mask = [usable for _, _, usable in sorted_train_feats]
best_feat = sorted_train_feats[feature_mask.index(True)][0]

The best usable feature (the first True in feature_mask) becomes sorted_train_feats[first_eligible][0]; features dropped by is_diff_to_low arrive masked and are never part of any candidate.

This ordering determines combination priority. The native search generates the combinations over the sorted feature list in the same order, so the strongest features always appear first:

When calcs_per_dim limits the number of combinations, pruning naturally affects the weaker feature combinations at the end of the iteration order. Key implications:

• The best feature (sorted_train_feats[0]) appears in the most combinations, giving it maximum coverage
• Weaker features may never be evaluated when calcs_per_dim cuts early
• Combination order is deterministic — always follows the sorted calc_diff() order within each _find_winner() call
• The native search evaluates the candidates in waves and applies the Pair-Correlation Filter between evaluations: a correlated candidate masks its 1D-weakest feature immediately, and later combinations containing that feature are skipped without consuming calcs_per_dim budget — the budget stretches further for the meaningful combinations

Step 2: Try Feature Combinations¶

For each dimension from 2 to n_dims, the native search generates the combinations over the sorted, still-unmasked features. Each combination is evaluated once per repetition and the median repetition's correlation table and score are used. Since features are pre-sorted by calc_diff(), combinations follow the same priority — (best, second_best) is evaluated before (best, weakest).

calcs_per_dim limits how many combinations are actually evaluated per dimension; the limit is checked after a wave of evaluations, so up to calcs_per_dim + 1 combinations are evaluated per dimension (the last wave is completed). Combinations containing a masked feature are skipped without counting against the budget (see Pair-Correlation Filter).

When a combination wins the split, _find_winner() does not rebuild it from scratch. It reconstructs a CombinedCategorizedFeature from the winner's native correlation table via from_fused, which carries the cuts and count series already used during scoring into the branch build that partitions the node (the corr table is the same structure calc_diff scores):

Step 3: Determine Branch Build¶

The winner's observed bin combinations are merged into branches by spore_diff (see Splitting Nodes into Branches):

The Combination Explosion¶

Higher n_dims values evaluate exponentially more combinations:

The calcs_per_dim Parameter¶

To keep training time bounded, calcs_per_dim limits how many combinations are tried per dimension. The limit is checked after evaluation, so up to calcs_per_dim + 1 combinations are evaluated per dimension (the wave that crosses the limit is completed):

calcs_per_dim: int | None = Field(
    description="Wieviele berechnungen sollen pro dimension gemacht werden",
    default=5000,
)

Practical Guidelines¶

When to Use Higher n_dims¶

Recommendations¶

Summary¶

Next Steps¶

• Splitting Nodes into Branches — How the winning split becomes recursive subtrees (spore_diff)
• Pair-Correlation Filter — How redundant features are masked per node
• Feature Categorization — How features are binned first
• Training Internals — Full algorithm reference