feat(anthropic): Add image url support to anthropic #7961
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
1 Skipped Deployment
|
jacoblee93
reviewed
Apr 17, 2025
| media_type: match[1] ?? "", | ||
| data: match[2] ?? "", | ||
| // eslint-disable-next-line @typescript-eslint/no-explicit-any | ||
| } as any; |
There was a problem hiding this comment.
I'm just following the cast in the previous implementation.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR updates the
langchain-anthropicpackage to support image inputs provided as URLs, in addition to the existing base64 encoding support.According to the latest Anthropic documentation, Claude 3 models can accept image inputs via a URL source:
{"type": "image", "source": {"type": "url", "url": "..."}}.This change modifies the
_formatImageutility function inlibs/langchain-anthropic/src/utils/message_inputs.tsto detect if the input string is a valid URL. If it is, it formats the message payload accordingly; otherwise, it falls back to the base64 parsing logic.This resolves the limitation where LangchainJS previously only accepted base64-encoded images, allowing users to leverage the full capabilities of the Anthropic API for multimodal inputs more easily.