Splitting Nodes into Branches¶

A Pilz node is split along the winning feature combination: every observed bin combination of the winning split's correlation table becomes a branch, and the combinations that were never observed in the count sample are grouped into a single residual branch. Each branch is split again recursively until enough samples or the maximum depth is reached.

The number of branches is controlled by the spore_diff training setting (default 1.0).

The Branch Logic¶

Every node therefore produces:

• Observed branches — one recursive subtree per (merged) group of bin combinations that were observed in the count sample of the median repetition. With spore_diff: 0.0 each observed combination is its own branch (fan-out).
• Residual branch (.n) — everything that was NOT observed in the count sample, so every possible combination still reaches a spore. The residual is omitted when all combinations were observed.

The merge step is greedy:

1. Sort the observed combinations by diff — the regularized normalized difference (n*M - m*N - sqrt(n)*M - sqrt(m)*N - n*sqrt(M) - m*sqrt(N)) / (n*M + m*N) with target/non-target cell counts n/m and totals N/M, clamped to 0 on the dominating side. The square-root penalty terms keep underpopulated cells at 0 until they gather enough evidence.
2. Find the adjacent pair with the smallest diff distance.
3. While at least two observed branches remain and that distance stays below spore_diff, merge the pair. The merged diff is recomputed from the summed counts with the same regularized formula.
4. The residual branch is never merged into.

At spore_diff >= 1.0 (the default) all observed combinations collapse into a binary split on the winning feature combination — the classic behavior. At spore_diff == 0.0 the split fans out into one subtree per observed cell.

Build Branches¶

get_cell_branches() returns one Filter per (merged) observed group plus the residual filter. A single-cell Filter is the And of the per-feature bin conditions of the group; the residual combines every bin combination that was not observed. Because the bins cover all values of their feature (the missing bin renders as is null), the observed branches plus the residual partition the whole row space:

def get_cell_branches(self, spore_diff: float = 0.0):
    ...
    # sort observed groups by diff, merge the adjacent pair with the
    # smallest gap while len(groups) > 2 and gap < spore_diff
    while len(groups) > 2:
        ordered = sorted(groups, key=lambda g: g["diff"])
        gap, a, b = smallest_gap(ordered)
        if not gap < spore_diff:
            break
        merge(a, b)  # diff = _diff_from_counts(a.count + b.count, a.count_right + b.count_right)
    return branches, residual

Recursive Tree Building¶

The trainer continues the tree with one recursive call per branch:

def _train_pilz_branches(self, target_filter, path_filter, depth, parent_features, train_df):
    best_feat, winner_features = self._find_winner(train_df=train_df)
    branches, residual = best_feat.get_cell_branches(
        spore_diff=self.settings.spore_diff
    )
    # one subtree per observed branch as depth ".<i>" ...
    for i, (branch_filter, _) in enumerate(branches):
        spores += self.train_pilz(
            target_filter=target_filter,
            path_filter=path_filter + [branch_filter],
            depth=f"{depth}.{i}" if depth else str(i),
            parent_features=winner_features,
        )
    # ... plus the residual as ".n"
    if residual is not None and residual.combine is not TRUE:
        spores += self.train_pilz(..., depth="...n", ...)
    return spores

The depth string encodes the path as dot-separated segments (0, 0.3, 0.3.n), where each segment is a branch index and the trailing n marks the residual branch.

The spore_diff Parameter¶

spore_diff: float = Field(
    description="Branch-merge distance: the observed bin combinations of a split "
    "are sorted by their target/non-target difference and the adjacent pair with "
    "the smallest difference is merged (the difference is recomputed from the "
    "summed counts) as long as at least two branches remain and the difference is "
    "below spore_diff. 0 keeps every combination as its own branch (fan-out); "
    ">= 1 collapses the combinations into a binary split.",
    default=1.0, ge=0.0,
)

• spore_diff = 0.0 — fan-out: every observed combination is its own branch.
• 0 < spore_diff < 1 — intermediate: nearby combinations are merged, well separated ones stay separate.
• spore_diff >= 1.0 (default) — binary split on the winning combination.

Leaf Creation¶

Recursion stops when not enough samples remain or max depth is reached:

def is_final_size(self) -> bool:
    return (
        self.target_df_size < self.min_size
        or self.non_target_df_size < self.min_size
    )

def make_spore(self, path_filter, depth, train_df, feature_names=None):
    score = train_df.score()
    return [
        Spore(
            cond=[Cond.from_expr(fil.combine) for fil in path_filter],
            score=score,
            depth=depth,
            feature_names=feature_names,
        )
    ]

Fresh models store the path as structured Cond objects (rendered to SQL on demand via Spore.get_cuts()), not as pre-rendered cut strings, and record the winning feature combination of the parent node in feature_names.

The score is the signed difference between target and non-target rows at the leaf, (target_size - non_target_size) / n, scaled by a weakening factor so that sparsely populated leaves contribute a smaller magnitude:

def score(self) -> float:
    sum = self.non_target_df_size + self.target_df_size
    if sum == 0:
        return 0.0
    if sum == 1:
        factor = 0.5
    else:
        factor = 1.0 - math.log(sum) / sum
    diff = self.target_df_size - self.non_target_df_size
    return factor * diff / sum

A positive score means more target than non-target rows, 0.0 means a perfect balance. The magnitude grows with the number of rows in the leaf and only approaches 1.0 for large leaves.

Legacy Three-Way Split¶

Older Pilz models used a three-way split on every bin combination: each cell was classified by its diff into Left (mostly non-target), Neutral (unclear — continue splitting) and Right (mostly target):

• diff > neutral_faktor → Right (target)
• diff < -neutral_faktor → Left (non-target)
• -neutral_faktor <= diff <= neutral_faktor → Neutral (uncertain)

The corresponding code (_train_pilz_three_way, counter, get_left_right_filter) is legacy: it is kept unwired for parity checks until the spore_diff branch logic proves identical results, then removed. The neutral_faktor, fanout_cells and min_split_count settings no longer exist.

Summary¶

Next Steps¶

• Multi-Dimensional Splits — How the winning feature combination is found
• Downsampling — How training data is sampled at each node
• Training Internals — Full algorithm