Best Practices¶

Guidelines for getting the best results with Pilz.

Data Preparation¶

Feature Types¶

Choose correct statistical types:

# Good examples

features:

  - name: status         # "active", "inactive" → categorial

    statistical: categorial

    type: string

  - name: age            # 0-100 → numerical

    statistical: numerical

    type: int

  - name: score          # 0.0-100.0 → numerical

    statistical: numerical

    type: float

Missing Values¶

Always handle missing values:

features:

  - name: income

    statistical: numerical

    type: float

    missing_value: 0  # Replace nulls with 0

If you do not set missing_value, nulls are not imputed — instead they become a separate "missing" category that training and eval handle via IS NULL conditions.

Data Quality¶

Check for outliers before training
Ensure target column has no missing values
Use consistent value formats (e.g., "Yes" vs "yes")

Training Settings¶

Starting Point¶

Begin with these safe defaults:

n: 1

max_depth: 100

n_dims: 3

n_cat: 3

n_rep: 5

Incrementally Increase Complexity¶

flowchart TD START[Start] --> B["Baseline: n=1, n_dims=3 (defaults)"] B --> R{Results OK?} R -->|No| I["Increase n=3-5"] I --> R2{Results OK?} R2 -->|No| M["Try n_dims=1 or 2"] M --> R3{Results OK?} R3 -->|No| D[Tune n_cat or calcs_per_dim] R -->|Yes| O[Optimize] R2 -->|Yes| O R3 -->|Yes| O style START fill:#e0f0ff style O fill:#ccffcc

Parameter Guidelines¶

|-----------|-----------|------------|-------|

| n | 1 | 10+ | More trees = more accurate, slower |

| n_dims | 1 | 3 | Higher = more combinations, slower |

| n_cat | 3 | 10 | Higher = more bins, risk of overfitting |

| n_rep | 1 | 10+ | Higher = more stable cuts, slower |

n_rep also applies to multi-dimensional combinations. With n_dims > 1, each feature combination is evaluated once per repetition using matching count rows before its median candidate is selected. Increasing both n_dims and n_rep can therefore increase training time substantially. Use calcs_per_dim to cap the number of feature combinations evaluated at each dimension.

Model Selection¶

Bias-Variance Trade-off¶

flowchart LR A[Start] --> B[Process] B --> C[End] style A fill:#e0f0ff style C fill:#ccffcc

flowchart LR subgraph Underfitting U1[Simple model] --> U2[High bias] U2 --> U3[Low variance] end subgraph Good G1[Balanced] --> G2[Balanced bias/variance] end subgraph Overfitting O1[Complex model] --> O2[Low bias] O2 --> O3[High variance] end style G1 fill:#ccffcc

When to Use Feature Combinations¶

Consider higher n_dims (feature combinations) when:

Single features do not separate the classes well
Domain knowledge suggests interactions
After establishing a baseline

Evaluation¶

Test Set¶

Always hold out test data
Don't use training data for evaluation
Use stratified sampling for class balance

Metrics¶

AUC: Overall discrimination ability
Accuracy: Simple but can be misleading for imbalanced data
Per-class accuracy: Important for multi-class

Deployment¶

Model Export¶

Keep models in version control
Document settings used
Test with sample predictions

SQL Production¶

Test SQL in staging before production
Monitor prediction distributions
Set up alerts for unusual patterns

Performance Checklist¶

[ ] Data cleaned and validated
[ ] Correct feature types set
[ ] Missing values handled
[ ] Started with baseline settings
[ ] Incremental tuning
[ ] Test set properly separated
[ ] AUC > 0.8 (or domain-specific)
[ ] SQL tested in staging
[ ] Model versioned