-
Notifications
You must be signed in to change notification settings - Fork 668
fix: optionally keep user-provided base URL for pagination #2149
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Codecov Report
@@ Coverage Diff @@
## main #2149 +/- ##
=======================================
Coverage 95.46% 95.46%
=======================================
Files 81 81
Lines 5353 5362 +9
=======================================
+ Hits 5110 5119 +9
Misses 243 243
Flags with carried forward coverage won't be shown. Click here to find out more.
|
I wonder if this would be better to be a config option? As probably the most likely case when this occurs is that something is wrong with their config. |
do you mean by set the config as parameter in |
Thanks for the work here @iomarmochtar! Have a look at #1978 as well, which covers mostly the same topic. I agree with John we should at least not completely blindly follow either the server or the client-provided URL when there is a mismatch, hence my proposal in the issue above: I think we should issue a warning instead, unless explicitly configured to follow the base URL, and only then reconstruct the URL. Could we add that here? 🙇 Thanks again! For context: I actually would consider this approach a misconfiguration of the Instead I think they should use |
9f52060
to
c815b94
Compare
@iomarmochtar would you still like to work on this as an optional parameter as discussed above? |
thanks you for remind me @nejch ,almost forgot to continue it due office task. let me try in this weekend. |
c815b94
to
c0f35d4
Compare
push as requested, please review it @nejch @JohnVillalovos |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks again @iomarmochtar. I have a few small comments
c0f35d4
to
9c02633
Compare
Thanks @iomarmochtar. I have some ideas to reuse this elsewhere that I think are outside the scope of this PR, so I'll merge this as is, and I will do a little refactor after. |
Background
To enhance HTTP security our Gitlab is fronted by IAP for example
https://gitlab.tools.domain.io
but for internal communication from runner to Gitlab API we create another endpoint that protected by firewall rules so then it only can be accessed from certain IP only, for examplehttps://runner.gitlab.tools.domain.io/
it actually using the reverse proxy that in the upstream destination will setHost
header to the original one (gitlab.tools.domain.io
).But when we run a python script inside Gitlab pipeline to the endpoint (
runner.gitlab.tools.domain.io
) and pass argumentall=True
oriterator=True
for loop all the data to all page will returning the base url asgitlab.tools.domain.io
whereas it will causing the error due the request will be blocked by IAP.Expected
The base url will be persist same as the first time set in the next url so the request will be expected goes as the first page when using pagination iterrator (
all=True
).Fix
This MR will make sure the base in next url is the same. It's been tested in my case and working as expected.