Skip to content

Diskcache implementation for llama state. #289

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Jun 6, 2023
Merged

Diskcache implementation for llama state. #289

merged 5 commits into from
Jun 6, 2023

Conversation

Maximilian-Winter
Copy link
Contributor

No description provided.

@gjmulder gjmulder added enhancement New feature or request performance labels May 30, 2023
@abetlen
Copy link
Owner

abetlen commented May 30, 2023

  1. Can you add diskcache to the setup.py and pyproject.toml dependencies?
  2. Is it possible to support both options? Ie have a filesysystem cache and a ram cache

@Maximilian-Winter
Copy link
Contributor Author

@abetlen Will add all the stuff later today!

@Maximilian-Winter
Copy link
Contributor Author

@abetlen Have added a LlamaCache base class and RAM and Disk child classes!

Jchang4
Jchang4 approved these changes Jun 6, 2023
View reviewed changes
Copy link

@Jchang4 Jchang4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you!

@abetlen abetlen merged commit 0f0b447 into abetlen:main Jun 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
1 more reviewer

@Jchang4 Jchang4 Jchang4 approved these changes

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
enhancement New feature or request performance
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Maximilian-Winter @abetlen @Jchang4 @gjmulder