Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch eval yaml configs to use evaluation_platform #1516

Merged
merged 3 commits into from
Mar 6, 2025

Conversation

wizeng23
Copy link
Contributor

@wizeng23 wizeng23 commented Mar 6, 2025

Description

evaluation_platform was renamed to evaluation_backend throughout our codebase in #1484. This causes an error in all our eval configs at main, which use the YAML config from HEAD but the Oumi package from PyPi (which hasn't picked up this rename).

This PR:

  • Renames evaluation_backend back to evaluation_platform to fix the breakage
  • Updates EvaluationTaskParams so that evaluation_backend is not a required field. This will cause a breakage on main, as our configs don't provide this value anymore. Also updates post_init logic in this class to expect one of the two fields to be populated, not none or both.
  • Fixes a test

After the next PyPI push, we can revert this change.

Tested that there's no errors both when using the PyPI package and using source.

Related issues

Towards OPE-1097

Before submitting

  • This PR only changes documentation. (You can ignore the following checks in that case)
  • Did you read the contributor guideline Pull Request guidelines?
  • Did you link the issue(s) related to this PR in the section above?
  • Did you add / update tests where needed?

@wizeng23 wizeng23 changed the title [WIP] Switch eval yaml configs to use evaluation_platform Switch eval yaml configs to use evaluation_platform Mar 6, 2025
@wizeng23 wizeng23 requested a review from nikg4 March 6, 2025 02:11
@wizeng23 wizeng23 merged commit dc72492 into main Mar 6, 2025
2 checks passed
@wizeng23 wizeng23 deleted the wizeng/evaluation_backend branch March 6, 2025 04:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants