Example: Bank Term Deposit¶

This example demonstrates binary classification with a bank marketing dataset - predicting whether a client will subscribe to a term deposit.

Dataset¶

Source: Kaggle Bank Term Deposit via thedevastator/bank-term-deposit-predictions
Task: Predict term deposit subscription (yes/no)
Features: 15 predictors (demographics, campaign information, economic context) plus the y target
Classes: 2 (yes, no)
Training samples: 45,211
Test samples: 4,521

Quick Start¶

The config files for this example are in examples/bank_term_deposit/:

# 1. Download data (requires kagglehub)
pip install kagglehub
python3 -c "
import kagglehub
path = kagglehub.dataset_download('thedevastator/bank-term-deposit-predictions')
print(f'Downloaded to: {path}')
"

# 2. Point the settings to your downloaded data
#    Edit examples/bank_term_deposit/train_settings.yaml and update train_files:
#      train_files:
#        - <kagglehub_path>/train.csv
#    Edit examples/bank_term_deposit/eval_settings.yaml and update test_files:
#      test_files:
#        - <kagglehub_path>/test.csv

# 3. Train
pilz train \
  --datacard examples/bank_term_deposit/bank_term_deposit.yaml \
  --trainsettings examples/bank_term_deposit/train_settings.yaml

# 4. Evaluate
pilz eval \
  --datacard examples/bank_term_deposit/bank_term_deposit.yaml \
  --evalsettings examples/bank_term_deposit/eval_settings.yaml

Or use the provided script:

cd examples/bank_term_deposit
# After downloading data and updating train_files/test_files paths in the settings
bash run.sh

DataCard Structure¶

features:
  - name: y
    statistical: categorial
    type: str
  - name: job
    statistical: categorial
    type: str
  - name: marital
    statistical: categorial
    type: str
  - name: education
    statistical: categorial
    type: str
  - name: default
    statistical: categorial
    type: str
  - name: balance
    statistical: numerical
    type: int
  - name: housing
    statistical: categorial
    type: str
  - name: loan
    statistical: categorial
    type: str
  - name: contact
    statistical: categorial
    type: str
  - name: day
    statistical: numerical
    type: int
  - name: month
    statistical: categorial
    type: str
  - name: duration
    statistical: numerical
    type: int
  - name: campaign
    statistical: numerical
    type: int
  - name: pdays
    statistical: numerical
    type: int
  - name: previous
    statistical: numerical
    type: int
  - name: poutcome
    statistical: categorial
    type: str

target:
  feature_name: y
  values:
    - "yes"
    - "no"

infos:
  src: https://www.kaggle.com/datasets/thedevastator/bank-term-deposit-predictions
  licence: CC0 1.0 Deed
  date: 2023-12-18

Settings (Quick Start)¶

n: 5                # 5 trees per class (10 trees total)
out_folder: test
max_depth: 8
frac_eval_cat: 0.8
max_eval_fit: 500
min_eval_fit: 5
n_dims: 2           # Pairwise feature combinations
n_cat: 3            # 3 bins per numerical feature
calcs_per_dim: 2000
n_rep: 5            # Repetitions per feature
train_files:
  - /path/to/train.csv

The checked-in examples/bank_term_deposit/train_settings.yaml uses a machine-specific absolute train_files path instead of the /path/to/... placeholder.

in_folders:
  - test
out_folder: eval
test_files:
  - /path/to/test.csv
out_file: eval/scored.csv

Training Time¶

With quick-start settings on a modern laptop (Apple Silicon):

Training: ~10 seconds
Evaluation: < 1 second

Actual Results¶

Overall Accuracy: 90.5%¶

Per-Class Accuracy¶

Class	Accuracy
no	97.5%
yes	37.0%

The "no" class is very accurate (majority class with ~88% of samples). The "yes" class is harder to predict due to strong class imbalance and the complexity of predicting rare subscriber behavior.

Overall vs. minority trade-off¶

The plain argmax (different_target: max) maximizes overall accuracy but largely ignores the minority class. The youden combination (per-tree margins with majority fallback) trades overall accuracy for minority recall — measured on the same model family (n: 5, max_depth: 8):

Combination (`different_target`)	`min_eval_fit`	Overall	no	yes
`max`	5	90.5%	97.5%	37.0%
`youden`	5	82.0%	81.9%	82.3%
`max`	50	86.4%	91.3%	48.9%
`youden`	50	74.6%	73.4%	83.5%

The max / 5 row is the checked-in model; the other rows are separately measured single runs (expect ±1–2 points of seed variation).

For subscriber detection (the business goal) the youden row is the relevant operating point: 82% recall on future subscribers at 82% overall. Larger min_eval_fit leaves shift the same trade-off towards the minority without changing the combination method.

ROC Curve¶

Output Files¶

test/
├── yes/0.json    # Model for predicting "yes" subscription
├── ...           # Trees 1-4 per class
├── yes/4.json
├── no/0.json     # Model for predicting "no" subscription
├── ...           # Trees 1-4 per class
├── no/4.json
└── label_stats.json

eval/
├── yes_roc.html
├── no_roc.html
├── all_roc.html
├── multi_class_result.html
└── scored.csv

Key Findings¶

Duration is the strongest predictor
Longer calls strongly correlate with subscription
Very short calls (< 2 min) almost never convert
Previous campaign outcome matters
Previous success → high likelihood of repeat success
Previous non-contact = low likelihood
Contact type influences results
Cellular contacts outperform telephone
Month shows seasonal patterns
Subscriptions peak in certain months (e.g., May, June)

Tips¶

Quick Start Settings (current)¶

The checked-in settings (n: 5, max_depth: 8) give ~90.5% overall accuracy — driven by the majority class (previously ~75% with single shallow trees). For the minority class see the trade-off table above.

For Better Accuracy¶

Beyond the checked-in settings, these directions are worth trying in train_settings.yaml (gains are not guaranteed — single runs vary):

max_depth: 10       # Deeper trees
n_dims: 3           # Triple feature combinations
n_cat: 5            # Finer bins
calcs_per_dim: 4000 # More thorough search
max_eval_fit: 5000  # More training samples

For the minority class specifically, the youden combination and larger min_eval_fit leaves shift the operating point towards recall — see the trade-off table under Actual Results.

Dealing with Class Imbalance¶

The dataset has ~88% "no" and ~12% "yes":

Start with the quick-start settings to verify the pipeline
Increase max_depth and n_dims to capture subtle patterns for the minority class
Add more trees (n=5 or n=10) for ensemble stability
Monitor the "yes" class accuracy specifically - not just overall accuracy