Skip to content

feat(generative_ai): add model evaluation sample #11122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jan 15, 2024
Merged

feat(generative_ai): add model evaluation sample #11122

merged 2 commits into from
Jan 15, 2024

Conversation

gericdong
Copy link
Contributor

Description

Fixes b/302952429

Add model evaluation sample for generative AI

Docs: https://cloud.google.com/vertex-ai/docs/generative-ai/models/evaluate-models

@gericdong gericdong requested review from a team as code owners January 12, 2024 15:15
@snippet-bot Snippet Bot
Copy link

snippet-bot bot commented Jan 12, 2024 •
edited
Loading

Here is the summary of changes.

You are about to add 1 region tag.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment

@product-auto-label product-auto-label bot added the samples Issues that are directly related to samples. label Jan 12, 2024
@msampathkumar
Copy link
Member

I see an error

google.auth.exceptions.RefreshError: ('invalid_scope: Invalid OAuth scope or ID token audience provided.', {'error': 'invalid_scope', 'error_description': 'Invalid OAuth scope or ID token audience provided.'})
______________________ test_gemini_safety_config_example _______________________
Traceback (most recent call last):
  File "/workspace/generative_ai/test_gemini_examples.py", line 110, in test_gemini_safety_config_example
    assert "scones" in text
    
AssertionError: assert 'scones' in ' there are many different ways to enjoy coffee. some people like it black, while others prefer to add milk or sugar. ...atter how you like to drink your coffee, there is one thing that is for sure: it is a delicious way to start your day.'

@msampathkumar msampathkumar added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 12, 2024
@kokoro-team kokoro-team removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 12, 2024
@msampathkumar msampathkumar merged commit 583de9f into GoogleCloudPlatform:main Jan 15, 2024
@msampathkumar msampathkumar mentioned this pull request Mar 20, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@msampathkumar msampathkumar msampathkumar approved these changes

Assignees

@m-strzelczyk m-strzelczyk

Labels
samples Issues that are directly related to samples.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@gericdong @msampathkumar @m-strzelczyk @kokoro-team