Skip to content

Extracted code to expand an URI to UriExpander #35415

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 4, 2020

Conversation

lyrixx
Copy link
Member

@lyrixx lyrixx commented Jan 21, 2020

Q A
Branch? master
Bug fix? no
New feature? yes
Deprecations? no
Tickets
License MIT
Doc PR

When building a crawler we need to extract and to expand all links on a
web pages.

ATM, we need to create a DomDocument, attach the href, and ask for the
full URL.

This is a bit slow, and unecessary. This is why I extracted the minimal
code to expand the URL to its onw trait for better re-usability.

I benched (a specific part of) my application:

  • before: 2.16ms
  • after: 1.42ms

@nicolas-grekas nicolas-grekas added this to the next milestone Jan 21, 2020
@lyrixx
Copy link
Member Author

lyrixx commented Jan 27, 2020

@nicolas-grekas thanks for the review. I addressed your comments

@fabpot
Copy link
Member

fabpot commented Feb 4, 2020

@lyrixx Can you fix the code :)

@lyrixx lyrixx changed the title Extracted code to expand an URI to UriExpanderTrait Extracted code to expand an URI to UriExpander Feb 4, 2020
@fabpot
Copy link
Member

fabpot commented Feb 4, 2020

Thank you @lyrixx.

fabpot added a commit that referenced this pull request Feb 4, 2020
This PR was merged into the 5.1-dev branch.

Discussion
----------

Extracted code to expand an URI to `UriExpander`

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| Deprecations? | no
| Tickets       |
| License       | MIT
| Doc PR        |

When building a crawler we need to extract and to expand all links on a
web pages.

ATM, we need to create a DomDocument, attach the href, and ask for the
full URL.

This is a bit slow, and unecessary. This is why I extracted the minimal
code to expand the URL to its onw trait for better re-usability.

I benched (a specific part of) my application:

 * before: 2.16ms
 * after: 1.42ms

Commits
-------

0c499c6 Extracted code to expand an URI to `UriExpanderTrait`
@fabpot fabpot merged commit 0c499c6 into symfony:master Feb 4, 2020
@lyrixx lyrixx deleted the url-expander branch February 4, 2020 13:05
@nicolas-grekas nicolas-grekas modified the milestones: next, 5.1 May 4, 2020
@fabpot fabpot mentioned this pull request May 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants