87  Up-Sampling

Upsampling is one of the conceptually easiest ways to handle imbalanced data. Like all other methods in this section, it is a supervised method, since it requires knowledge of the outcome. This also means that it can only be applied to the training data set, since we need the outcome. See also down-dampling, which is the opposite action.

The algorithm is quite simple. Tally the number of observations within each class, as well as keep track of which observations are associated with each class. The class with the most observations is then denoted as the majority class, The remaining classes are denoted as the minority classes. The observations of each minority class are then sampled with replacement to increase the number of observations in the class. Typically, to align with the number of observations with that of the majority class.

One could also modify this for a different threshold, say 80%, which would make all minority classes with less than 80% of the number of observations of the majority class, upsample until they have at least 80% of that of the majority class.

This action can drastically increase the number of observations depending on the number of minority classes and the ratio between them. A 90-10 split would result in a 80% increase in data, and a 90-5-5 split would result in a 170% increase. No data is being deleted in this modification as we are just adding more rows, which are duplicates of existing rows.

You could think of this as a stochastic version of case weights without the compactness.

Like all the methods in this section, this changes the distribution of the training data set, which is covered in Section 86.2. Just because these methods are simple doesn’t mean that they they will work well. Random down sampling have been shown to lead to poorly calibrated models (Goorbergh et al. 2022).

87.2 Pros and Cons

87.2.1 Pros

  • Computationally fast and simple.

87.2.2 Cons

  • Could give issues with space for large data sets.
  • Solves tasks that could be better handled with other methods.
  • Distorts the predicted probabilities, which may require recalibration afterwards.

87.3 R Examples

We will be using the ames data set for these examples. The MS_Zoning variable is imbalanced, so we take the two largest classes to get a clean two-class problem to work with. The same data set is used throughout this section so the methods can be compared against each other.

library(recipes)
library(themis)
library(modeldata)
library(dplyr)
data("ames")

ames_imbalance <- ames |>
  filter(MS_Zoning %in% c("Residential_Low_Density", "Residential_Medium_Density")) |>
  mutate(MS_Zoning = droplevels(MS_Zoning)) |>
  select(MS_Zoning, Lot_Area, Year_Built, Gr_Liv_Area, Full_Bath)

ames_imbalance |>
  count(MS_Zoning)
# A tibble: 2 Γ— 2
  MS_Zoning                      n
  <fct>                      <int>
1 Residential_Low_Density     2273
2 Residential_Medium_Density   462

We are using numeric predictors only because most of the other methods in this section require it. Up-sampling itself has no such restriction, since it copies whole rows.

{themis} provides step_upsample(), which has the implementation for up-sampling.

recipe(MS_Zoning ~ ., data = ames_imbalance) |>
  step_upsample(MS_Zoning) |>
  prep() |>
  bake(new_data = NULL) |>
  count(MS_Zoning)
# A tibble: 2 Γ— 2
  MS_Zoning                      n
  <fct>                      <int>
1 Residential_Low_Density     2273
2 Residential_Medium_Density  2273

87.4 Python Examples