Allow to modify multiple request options in Python SDK (similar to JavaScript SDK) #367
Description
Description of the feature request:
I would like the Python SDK to allow the same equivalent functionality as the JavaScript SDK regarding the possibility of modifying the request options
JavaScript SDK (can add many custom request options)
const model = googleGenAIClient.getGenerativeModel(
{ model: "gemini-pro" },
{
customHeaders: {
"x-param": "<PARAMETER>",
},
baseUrl: env.PROXY_URL,
}
);PythonSDK (cannot add many custom request options)
import google.generativeai as genai
genai.configure(api_key="AIzaSyBW75RG6YjRor0pL1Eu0jWsRSd-rndTNIY")
model = genai.GenerativeModel('gemini-1.0-pro-latest')
response = model.generate_content("The opposite of hot is",
request_options= {
# Only accepts timeout parameter
"timeout": 10,
# Anything else gives the following error
# TypeError: GenerativeServiceClient.generate_content() got an unexpected keyword argument 'customHeaders'
# "customHeaders": {
# "x-param": "<PARAMETER>",
# },
# I would also like to modify the baseUrl
# "baseUrl": "http://localhost:8787/",
# TypeError: GenerativeServiceClient.generate_content() got an unexpected keyword argument 'baseUrl'
}
print(response.text)What problem are you trying to solve with this feature?
Some people have talked about how being able to add metadata to requests in these libraries is helpful for proxies, and even tried to implement the solution themselves as you can find below:
- See this issue Add support to pass metadata from GenerativeModel.generate_content and Chat:send_message googleapis/python-aiplatform#3490
- See this PR feat: add metadata field in _GenerativeModels class methods to pass the metadata down the method chain. googleapis/python-aiplatform#3491
I'm simply frustrated by the fact that it's possible to modify the request options in the JavaScript SDK, but not in the Python SDK.
If I can do it in the JavaScript SDK, why can't I do it in the Python SDK? 🤔
For the record, the Open AI Python SDK allows to do it by adding the parameter default_headers in the client configuration and extra_headers in the AI generation call:
from openai import OpenAI
client = OpenAI(
api_key="<KEY>",
base_url="http://localhost:8787",
default_headers={
"x-param": "parameter"
}
)
chat_completion = client.chat.completions.create(
messages=[
{
"role": "user",
"content": "Say this is a test",
}
],
model="<model>",
extra_headers={
"x-param": "parameter"
}
)
print(chat_completion.choices[0].message.content)Any other information you'd like to share?
The documentation also seems to have implemented something similar to customHeaders in the request options (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/nodejs/latest/vertexai/requestoptions), but when I installed the Python SDK, it didn't seem to accept customHeaders as explained above. At any rate, the current Python SDK is far behind the JavaScript SDK in this regard (JS SDK accepts other parameters like baseUrl—useful for proxies).
Additional context
Add any other context or screenshots about the feature request here.
Other relevant links in this conversation may be:
- There should be a RequestOptions TypedDict to explain what attributes are allowed. #223
- Improve request_options #297
- Add support for custom headers in vertexai.init function googleapis/python-aiplatform#3526
- https://github.com/MarkDaoust/generative-ai-python/blob/418a38c4a2ab921c0ab084157bfa54f08c96a574/google/generativeai/types/helper_types.py#L30
- https://github.com/google-gemini/generative-ai-python/pull/297/files/4d653f8a5266418912e9a4a378cb02f5b240b69b#