close
The Wayback Machine - https://web.archive.org/web/20200910015411/https://github.com/PyTorchLightning/pytorch-lightning/issues/3304
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MulticlassAUROC: Implement a multi-class version of the AUROC metric #3304

Open
NumesSanguis opened this issue Sep 1, 2020 · 1 comment
Open

Comments

@NumesSanguis
Copy link

@NumesSanguis NumesSanguis commented Sep 1, 2020

🚀 Feature

Create the metric MulticlassAUROC to allow for the AUROC metric to be used in multi-class problem settings. Or,
Expand the AUROC metric to support multi-class data, which would also directly solve this AUROC bug that instead gives a random value when used in multi-class problems: #3303

Motivation

AUROC is a useful metric for model performance, but the current AUROC metric is not made for multi-class problems.
An out of the box working multi-class AUROC metric would be great.

Pitch

According to this post on the PyTorch Lightning forum, implementation would just require wrapping existing functionality together.

Implementing this metric would lower the barrier of its usage, instead of needing to be aware of the different metrics available that could be combined to effectively get his metric.

Alternatives

Additional context

@SkafteNicki
Copy link
Contributor

@SkafteNicki SkafteNicki commented Sep 1, 2020

It would be a great addition. @NumesSanguis would you be up for creating a PR that implements a MulticlassAUROC metric?

@justusschock justusschock added this to To do in Metrics package via automation Sep 1, 2020
@justusschock justusschock added the Metrics label Sep 1, 2020
@SkafteNicki SkafteNicki moved this from To do to New Metrics in Metrics package Sep 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Metrics package
  
New Metrics
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.