Skip to content

Add support for request options #29

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Jul 24, 2025
Merged

Add support for request options #29

merged 19 commits into from
Jul 24, 2025

Conversation

zyc9012
Copy link
Collaborator

@zyc9012 zyc9012 commented Apr 3, 2025

Resolves #27

This PR was initially for supporting the proxy feature natively. Unfortunately, HttpsProxyAgent isn't supported in some of the older NodeJS versions that this library promises to support.

As a fallback, this PR allows users to pass their own http.RequestOptions so they can set the agent to be the proxy agent they want.

@zyc9012 zyc9012 force-pushed the support-proxy branch 2 times, most recently from 4405244 to a60a11b Compare April 5, 2025 10:11
@zyc9012 zyc9012 changed the title Add support for proxy Add support for request options Apr 5, 2025
@zyc9012 zyc9012 requested review from aciddjus, Ovi and Copilot April 7, 2025 01:54
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.

Files not reviewed (1)
  • deno.json: Language not supported
Comments suppressed due to low confidence (1)

src/utils.ts:66

  • [nitpick] The shallow merge of requestOptions may not correctly combine nested objects (e.g., headers). Consider using a deep merge strategy if preserving or merging nested properties is desired.
return { ...base, ...config.requestOptions, ...(parameters.requestOptions as http.RequestOptions), };

@zyc9012 zyc9012 requested a review from Copilot April 7, 2025 02:09
@zyc9012 zyc9012 self-assigned this Apr 7, 2025
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 14 out of 15 changed files in this pull request and generated no comments.

Files not reviewed (1)
  • deno.json: Language not supported
Comments suppressed due to low confidence (1)

src/utils.ts:91

  • Since setEncoding('utf8') is called, the 'data' event returns strings rather than Buffers. Consider changing the type annotation to 'string' or removing it.
resp.on("data", (chunk: Buffer) => {

@zyc9012 zyc9012 requested a review from Copilot April 7, 2025 02:13
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 14 out of 15 changed files in this pull request and generated no comments.

Files not reviewed (1)
  • deno.json: Language not supported
Comments suppressed due to low confidence (2)

src/utils.ts:66

  • [nitpick] Consider using a deep merge for the headers property if both config.requestOptions and parameters.requestOptions include headers. This would allow combining header objects instead of one entirely overriding the other.
return { ...config.requestOptions, ...(parameters.requestOptions as http.RequestOptions), ...basicOptions };

tests/utils_test.ts:136

  • [nitpick] Consider refining the type conversion instead of using an 'as unknown as' cast for requestOptions. Updating the parameter type definitions could improve type safety.
const params = { q: "coffee", requestOptions: customOptions } as unknown as qs.ParsedUrlQueryInput;

Copy link

@Ovi Ovi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @zyc9012! 👍
Great work.

});
};

const req = https.get(options, handleResponse).on("error", handleError);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deno task test fails, whereas deno task test:ci passes, I assume this is because we are using https module and it's trying to hit localhost with secure protocol (https)

Suggested change
const req = https.get(options, handleResponse).on("error", handleError);
const protocolModule = options.protocol === "https" ? https : http;
const req = protocolModule.get(options, handleResponse).on("error", handleError);

Comment on lines +208 to +209
assertEquals(options.hostname, "serpapi.com");
assertEquals(options.port, 443);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make this dynamic based on the environment.

Comment on lines 27 to 34
? {
hostname: "localhost",
port: 3000,
}
: {
hostname: "serpapi.com",
port: 443,
};
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's include protocol here

Suggested change
? {
hostname: "localhost",
port: 3000,
}
: {
hostname: "serpapi.com",
port: 443,
};
? {
hostname: "localhost",
port: 3000,
protocol: "http",
}
: {
hostname: "serpapi.com",
port: 443,
protocol: "https",
};

Comment on lines 44 to 51
? {
hostname: "localhost",
port: 3000,
}
: {
hostname: "serpapi.com",
port: 443,
};
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's include protocol here

Suggested change
? {
hostname: "localhost",
port: 3000,
}
: {
hostname: "serpapi.com",
port: 443,
};
? {
hostname: "localhost",
port: 3000,
protocol: "http",
}
: {
hostname: "serpapi.com",
port: 443,
protocol: "https",
};

@zyc9012
Copy link
Collaborator Author

zyc9012 commented Jul 22, 2025

Thanks @Ovi 👍

I removed the test against localhost instead. As this is an open-source repo, not all developers have SerpApi running in localhost.

@zyc9012 zyc9012 requested a review from Ovi July 22, 2025 14:01
Copy link

@Ovi Ovi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @zyc9012! Works great! 👍
Just some minor tweaks.

@zyc9012 zyc9012 requested a review from Ovi July 23, 2025 04:24
Copy link

@Ovi Ovi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @zyc9012!
Works great! 🚀

@zyc9012 zyc9012 merged commit 3332d7f into master Jul 24, 2025
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature Request: Support HTTPS_PROXY / HTTP_PROXY / NO_PROXY environment variables for proxying
2 participants