Skip to content

Introduction to Pilz

What is Pilz?

Pilz (German for "mushroom" / "fungus") is a machine learning library for classification tasks. The name reflects its nature: like mushrooms are neither plants nor animals, Pilz is neither a traditional decision tree nor a neural network — it's something in between that captures the best of both worlds.

flowchart TB subgraph Nature P[Plants] --> M[Mushrooms/Fungi] A[Animals] --> M end subgraph ML NN[Neural Networks] --> Pilz[Pilz] DT[Decision Trees] --> Pilz end style M fill:#ccffcc style Pilz fill:#e0f0ff

Unlike traditional ML tools that create "black box" models, Pilz generates readable SQL rules that you can directly deploy to your database.

The core innovation of Pilz is its ability to handle feature correlations naturally through multi-dimensional cuts.

How It Works

Pilz evaluates feature combinations by their ability to separate target from non-target, with all data processing running through SQL. The algorithm is composed of several key concepts:

  • Feature Categorization — Every feature is binned into n_cat categories. Numerical features use quantile binning; categorical features are grouped by target rate similarity.
  • Multi-Dimensional Splits — Pilz splits on feature combinations directly, not one at a time. This captures correlations between features in a single split.
  • SQL-Native Architecture — All data processing runs through DuckDB SQL. Filters are SymPy boolean expressions translated to SQL WHERE clauses. Trained models generate CASE WHEN SQL.
  • Three-Way Splits — Each node splits into three branches: Left (non-target), Neutral (continue splitting), and Right (target).
  • Downsampling — At every node, Pilz independently samples target and non-target rows via ORDER BY RANDOM() LIMIT.
  • Imbalanced Data — Because target and non-target are sampled independently with equal total weight, Pilz handles skewed distributions without SMOTE or class weights.

Traditional vs Pilz

A traditional decision tree needs multiple sequential splits to capture a pairwise correlation. Pilz captures it in a single multi-dimensional cut:

flowchart TD subgraph "Traditional Tree - Needs Multiple Splits" T1[Data] --> T2{Contract=Monthly?} T2 -->|Yes| T3{No TechSupport?} T3 -->|Yes| T4[High Churn] T3 -->|No| T5[Medium Churn] T2 -->|No| T6[Low Churn] end subgraph "Pilz - Single Multi-Dimensional Cut" P1[Data] --> P2{Contract=Monthly AND TechSupport=No?} P2 -->|Yes| P3[High Churn, Score: 0.85] P2 -->|No| P4[Low Churn, Score: 0.15] end style P2 fill:#ccffcc

Comparison with Other Approaches

Aspect Neural Networks Traditional Trees Pilz
Feature correlations Learned implicitly Multiple splits Single cut
Interpretability Low Medium High (SQL)
Output format Opaque weights Tree structure SQL rules
Data preprocessing Often required Minimal Minimal
Imbalanced data Needs tuning Needs tuning Native support

Core Parameters

Parameter What it controls
n_dims How many features to combine in a single split
n_cat Number of bins per feature
max_depth Maximum tree depth
neutral_faktor Width of the neutral zone in three-way splits
max_eval_fit Rows sampled per node
calcs_per_dim Max combinations evaluated per dimension

When to Use Pilz

✓ Ideal For

  • Tabular data — Customer data, transactions, any structured data
  • Interpretability matters — You need to understand and deploy model logic
  • Feature correlations exist — Pilz captures them naturally
  • SQL-native deployment — Model output is runnable SQL

✗ Not Ideal For

  • Image/audio/video — Use neural networks
  • Very large datasets — May need tuning
  • No correlations — Simpler models may suffice

Next Steps

Pilz: Capturing feature correlations naturally through multi-dimensional cuts.