Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: custom direction in new_metric() #311

Open
mikemahoney218 opened this issue Jul 6, 2022 · 1 comment
Open

Feature request: custom direction in new_metric() #311

mikemahoney218 opened this issue Jul 6, 2022 · 1 comment
Labels
feature a feature request or enhancement

Comments

@mikemahoney218
Copy link
Member

mikemahoney218 commented Jul 6, 2022

Feature

Apologies if this already exists and I've missed it.

It would be useful to allow custom direction options when creating new metrics. My current use-case for this behavior is implementing Geary's C in waywiser, which has an optimal value of 1; while this case is easy to engineer around (a metric C* exists such that C* = C - 1, letting direction = "zero") I think there may be other metrics which would benefit from being able to provide their own direction arguments.

@DavisVaughan DavisVaughan added the feature a feature request or enhancement label Aug 24, 2022
@DavisVaughan
Copy link
Member

DavisVaughan commented Aug 25, 2022

The biggest issue with a completely custom direction is that tune somehow has to know how to use it. In particular, tune_bayes() does different things depending on whether you are minimizing/maximizing your metric

https://github.com/tidymodels/tune/blob/e4b7485f6196c005d457f713c2bc5da5ffbfb63c/R/tune_bayes.R#L216

So does show_best() I believe

It wouldn't know how to handle a completely custom direction without some kind of constraints around it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature a feature request or enhancement
Projects
None yet
Development

No branches or pull requests

2 participants