Package 'mupet'

Title: Multiclass Performance Evaluation Toolkit
Description: Implementation of custom 'tidymodels' metrics for multi-class prediction models with a single negative class. Currently are implemented macro-average sensitivity and specificity as in Mortaz, Ebrahim (2020) "Imbalance accuracy metric for model selection in multi-class imbalance classification problems” <doi:10.1016/j.knosys.2020.106490> and a generalized weighted Youden index as in Li, D.L., Shen F., Yin Y., Peng J.X and Chen P.Y. (2013) “Weighted Youden index and its two-independent-sample comparison based on weighted sensitivity and specificity” <doi:10.3760/cma.j.issn.0366-6999.20123102>.
Authors: Lise Bellanger [aut], Manon Simonot [aut], Aymeric Stamm [aut, cre] (ORCID: <https://orcid.org/0000-0002-8725-3654>)
Maintainer: Aymeric Stamm <[email protected]>
License: MIT + file LICENSE
Version: 0.1.0
Built: 2026-05-22 18:48:42 UTC
Source: https://github.com/astamm/mupet

Help Index

Macro Average Sensitivity

Description

This function computes the macro-average sensitivity for a multi-class prediction model. It assumes that the negative class is the first one.

Usage

macro_average_sensitivity_vec(
  truth,
  estimate,
  estimator = NULL,
  na_rm = TRUE,
  case_weights = NULL,
  event_level = "first",
  ...
)

macro_average_sensitivity(data, ...)

## S3 method for class 'data.frame'
macro_average_sensitivity(
  data,
  truth,
  estimate,
  estimator = NULL,
  na_rm = TRUE,
  case_weights = NULL,
  event_level = "first",
  ...
)

Arguments

truth

The column identifier for the true class results (that is a factor).
estimate

The column identifier for the predicted class results (that is also factor).
estimator

One of: "binary", "macro", "macro_weighted", or "micro" to specify the type of averaging to be done.
na_rm

A logical value indicating whether NA values should be stripped before the computation proceeds.
case_weights

The optional column identifier for case weights.
event_level

A single string. Either "first" or "second" to specify which level of truth to consider as the "event". This argument is only applicable when estimator = "binary".
...

Currently unused.
data

Either a data.frame containing the columns specified by the truth and estimate arguments, or a table/matrix where the true class results should be in the columns of the table.

Value

A scalar storing the value of the macro-average sensitivity score.

Examples

fold1 <- subset(yardstick::hpc_cv, Resample == "Fold01")
macro_average_sensitivity_vec(fold1$obs, fold1$pred)
macro_average_sensitivity(fold1, obs, pred)

Macro Average Specificity

Description

This function computes the macro-average specificity for a multi-class prediction model. It assumes that the negative class is the first one.

Usage

macro_average_specificity_vec(
  truth,
  estimate,
  estimator = NULL,
  na_rm = TRUE,
  case_weights = NULL,
  event_level = "first",
  ...
)

macro_average_specificity(data, ...)

## S3 method for class 'data.frame'
macro_average_specificity(
  data,
  truth,
  estimate,
  estimator = NULL,
  na_rm = TRUE,
  case_weights = NULL,
  event_level = "first",
  ...
)

Arguments

truth

The column identifier for the true class results (that is a factor).
estimate

The column identifier for the predicted class results (that is also factor).
estimator

One of: "binary", "macro", "macro_weighted", or "micro" to specify the type of averaging to be done.
na_rm

A logical value indicating whether NA values should be stripped before the computation proceeds.
case_weights

The optional column identifier for case weights.
event_level

A single string. Either "first" or "second" to specify which level of truth to consider as the "event". This argument is only applicable when estimator = "binary".
...

Currently unused.
data

Either a data.frame containing the columns specified by the truth and estimate arguments, or a table/matrix where the true class results should be in the columns of the table.

Value

A scalar storing the value of the macro-average specificity score.

Examples

fold1 <- subset(yardstick::hpc_cv, Resample == "Fold01")
macro_average_specificity_vec(fold1$obs, fold1$pred)
macro_average_specificity(fold1, obs, pred)

Weighted Youden Index

Description

This function computes the weighted Youden index for a multi-class prediction model. It assumes that the negative class is the first one.

Usage

weighted_youden_index_vec(
  truth,
  estimate,
  sensitivity_weight = 0.5,
  estimator = NULL,
  na_rm = TRUE,
  case_weights = NULL,
  event_level = "first",
  ...
)

weighted_youden_index(data, ...)

## S3 method for class 'data.frame'
weighted_youden_index(
  data,
  truth,
  estimate,
  sensitivity_weight = 0.5,
  estimator = NULL,
  na_rm = TRUE,
  case_weights = NULL,
  event_level = "first",
  ...
)

Arguments

truth

The column identifier for the true class results (that is a factor).
estimate

The column identifier for the predicted class results (that is also factor).
sensitivity_weight

A scalar value specifying the weight to put on sensitivity. Defaults to 0.5 which puts equal weights to sensitivity and specificity.
estimator

One of: "binary", "macro", "macro_weighted", or "micro" to specify the type of averaging to be done.
na_rm

A logical value indicating whether NA values should be stripped before the computation proceeds.
case_weights

The optional column identifier for case weights.
event_level

A single string. Either "first" or "second" to specify which level of truth to consider as the "event". This argument is only applicable when estimator = "binary".
...

Currently unused.
data

Either a data.frame containing the columns specified by the truth and estimate arguments, or a table/matrix where the true class results should be in the columns of the table.

Value

A scalar storing the value of the weighted Youden index.

Examples

fold1 <- subset(yardstick::hpc_cv, Resample == "Fold01")
weighted_youden_index_vec(fold1$obs, fold1$pred)
weighted_youden_index(fold1, obs, pred)