Skip to content

Add OpenAI Reinforcement Fine-Tuning (RFT) Support #3060

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 19 commits into
base: main
Choose a base branch
from
Draft

Add OpenAI Reinforcement Fine-Tuning (RFT) Support #3060

wants to merge 19 commits into from
+1,541 −46

Conversation

anndvision
Copy link
Contributor

This PR implements support for OpenAI's Reinforcement Fine-Tuning (RFT) API.

Core Implementation:

  • OpenAI RFT configuration with graders, hyperparameters, and training options
  • Content block conversion from ContentBlockChatOutput to OpenAIReinforcementOutput
  • Multiple grader types: StringCheck, TextSimilarity, ScoreModel, LabelModel, Python, and Multi
  • Template system integration via TomlRelativePath

Testing

  • Unit tests for conversion logic with tool calls
  • Mock and live testing infrastructure
  • Multi-grader configurations with StringCheck and ScoreModel
  • Template variable resolution validation

TODO

  • UI
  • Python

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Aaron1011 Aaron1011 Awaiting requested review from Aaron1011

@virajmehta virajmehta Awaiting requested review from virajmehta

Copilot code review Copilot Awaiting requested review from Copilot Copilot will automatically review once the pull request is marked ready for review

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@anndvision