27  GLMM Encoding

Generalized linear mixed models (GLMM) encoding Pargent et al. (2022) follows as an extension to target encoding which is laid out in detail at Chapter 23.

A hierarchical generalized linear model is fit, using no intercept.

27.2 Pros and Cons

27.2.1 Pros

  • No hyperparameters to tune, as shrinkage is automatically done.
  • Can deal with categorical variables with many levels
  • Can deal with unseen levels in a sensible way

27.2.2 Cons

  • Can be prone to overfitting

27.3 R Examples

TODO

find a better data set

We apply the smoothed GLMM encoder using the step_lencode_mixed() step.

library(recipes)
library(embed)

data(ames, package = "modeldata")

rec_target_smooth <- recipe(Sale_Price ~ Neighborhood, data = ames) |>
  step_lencode_mixed(Neighborhood, outcome = vars(Sale_Price)) |>
  prep()

rec_target_smooth |>
  bake(new_data = NULL)
# A tibble: 2,930 Γ— 2
   Neighborhood Sale_Price
          <dbl>      <int>
 1      145156.     215000
 2      145156.     105000
 3      145156.     172000
 4      145156.     244000
 5      190633.     189900
 6      190633.     195500
 7      322591.     213500
 8      322591.     191500
 9      322591.     236500
10      190633.     189000
# β„Ή 2,920 more rows

And we can pull out the values of the encoding like so.

rec_target_smooth |>
  tidy(1)
# A tibble: 29 Γ— 4
   level                value terms        id                 
   <chr>                <dbl> <chr>        <chr>              
 1 North_Ames         145156. Neighborhood lencode_mixed_Bp5vK
 2 College_Creek      201769. Neighborhood lencode_mixed_Bp5vK
 3 Old_Town           124154. Neighborhood lencode_mixed_Bp5vK
 4 Edwards            131021. Neighborhood lencode_mixed_Bp5vK
 5 Somerset           229563. Neighborhood lencode_mixed_Bp5vK
 6 Northridge_Heights 321519. Neighborhood lencode_mixed_Bp5vK
 7 Gilbert            190633. Neighborhood lencode_mixed_Bp5vK
 8 Sawyer             136956. Neighborhood lencode_mixed_Bp5vK
 9 Northwest_Ames     188401. Neighborhood lencode_mixed_Bp5vK
10 Sawyer_West        184085. Neighborhood lencode_mixed_Bp5vK
# β„Ή 19 more rows

27.4 Python Examples

We are using the ames data set for examples. {category_encoders} provided the BinaryEncoder() method we can use.

from feazdata import ames
from sklearn.compose import ColumnTransformer
from category_encoders.glmm import GLMMEncoder

ct = ColumnTransformer(
    [('glmm', GLMMEncoder(), ['MS_Zoning'])], 
    remainder="passthrough")

ct.fit(ames, y=ames[["Sale_Price"]].values.flatten())
ColumnTransformer(remainder='passthrough',
                  transformers=[('glmm', GLMMEncoder(), ['MS_Zoning'])])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
ct.transform(ames)
      glmm__MS_Zoning  ... remainder__Latitude
0           55180.830  ...              42.054
1             338.691  ...              42.053
2           55180.830  ...              42.053
3           55180.830  ...              42.051
4           55180.830  ...              42.061
...               ...  ...                 ...
2925        55180.830  ...              41.989
2926        55180.830  ...              41.988
2927        55180.830  ...              41.987
2928        55180.830  ...              41.991
2929        55180.830  ...              41.989

[2930 rows x 74 columns]