Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upMulticlassAUROC: Implement a multi-class version of the AUROC metric #3304
Comments
|
It would be a great addition. @NumesSanguis would you be up for creating a PR that implements a MulticlassAUROC metric? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment


Create the metric MulticlassAUROC to allow for the AUROC metric to be used in multi-class problem settings. Or,
Expand the AUROC metric to support multi-class data, which would also directly solve this AUROC bug that instead gives a random value when used in multi-class problems: #3303
Motivation
AUROC is a useful metric for model performance, but the current AUROC metric is not made for multi-class problems.
An out of the box working multi-class AUROC metric would be great.
Pitch
According to this post on the PyTorch Lightning forum, implementation would just require wrapping existing functionality together.
Implementing this metric would lower the barrier of its usage, instead of needing to be aware of the different metrics available that could be combined to effectively get his metric.
Alternatives
sklearn.metrics.roc_auc_score.This slows down the training, however, due to the need of transferring prediction and target data to CPU.
Additional context