Skip to content

improved in out degree performance #2757

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Dec 15, 2022
Merged

improved in out degree performance #2757

merged 3 commits into from
Dec 15, 2022

Conversation

swilly22
Copy link
Contributor

@swilly22 swilly22 commented Dec 9, 2022

#2753
MATCH (a) RETURN outDegree(a), inDegree(a)

Previous implementation would extract a's incoming / outgoing edges into an array just to get its length and soon after discard it which is a waste. This new implementation doesn't extracts any edges but simply counts.

FYI with @AviAvni new design of NxM Relation matrices (N #nodes, M #edges) we get rid of Multi edge arrays and can simplify this change to a straight forward row reduction.

Quick benchmark:
Compute out degree of a node with 1M outgoing edges

GRAPH.QUERY g "CREATE (a:A), (b:B)"
GRAPH.QUERY g "MATCH (a:A), (b:B) WITH a, b UNWIND range (0, 1000000) as x CREATE (a)-[:E]->(b)"
GRAPH.QUERY g "MATCH (a:A) RETURN outdegree(a)"

This PR 0.2ms
Current Master 20ms
100X improvement

@codecov
Copy link

codecov bot commented Dec 9, 2022

Codecov Report

Base: 92.29% // Head: 92.36% // Increases project coverage by +0.06% 🎉

Coverage data is based on head (66032f6) compared to base (c1fb459).
Patch coverage: 93.44% of modified lines in pull request are covered.

❗ Current head 66032f6 differs from pull request most recent head d982cee. Consider uploading reports for the commit d982cee to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2757      +/-   ##
==========================================
+ Coverage   92.29%   92.36%   +0.06%     
==========================================
  Files         264      264              
  Lines       25215    25254      +39     
==========================================
+ Hits        23273    23325      +52     
+ Misses       1942     1929      -13     
Impacted Files Coverage Δ
src/arithmetic/entity_funcs/entity_funcs.c 97.26% <87.50%> (-1.08%) ⬇️
src/graph/graph.c 98.31% <95.55%> (-0.24%) ⬇️
src/procedures/proc_sp_paths.c 95.70% <0.00%> (-1.44%) ⬇️
src/procedures/proc_ss_paths.c 90.69% <0.00%> (-1.17%) ⬇️
src/execution_plan/ops/op_aggregate.c 98.14% <0.00%> (-0.62%) ⬇️
src/commands/cmd_dispatcher.c 71.31% <0.00%> (-0.24%) ⬇️
src/commands/cmd_query.c 97.67% <0.00%> (-0.06%) ⬇️
src/resultset/formatters/resultset_replyverbose.c 38.79% <0.00%> (+23.27%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Collaborator

@iddm iddm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I understand, looks good enough to me, and quite a simple change in the code.

@iddm iddm self-requested a review December 12, 2022 08:00
iddm
iddm previously approved these changes Dec 12, 2022
// outgoing edges
//----------------------------------------------------------------------

// TODO: revisit once we get rid of MULTI-EDGE hack
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we anticipate any changes regarding this "TODO" in this PR or later?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably later down the road

raz-mon
raz-mon previously approved these changes Dec 15, 2022
removed unused `info`
@swilly22 swilly22 dismissed stale reviews from raz-mon and iddm via d982cee December 15, 2022 07:57
@swilly22 swilly22 merged commit 3268829 into master Dec 15, 2022
@swilly22 swilly22 deleted the in_out_degree branch December 15, 2022 08:02
raz-mon pushed a commit that referenced this pull request Dec 20, 2022
* improved in out degree performance

* Update graph.c

removed unused `info`
raz-mon pushed a commit that referenced this pull request Dec 20, 2022
* improved in out degree performance

* Update graph.c

removed unused `info`
AviAvni added a commit that referenced this pull request Dec 26, 2022
* Fix type mismatch error message (#2691)

* Add function SIType_ToMultipleTypeString()

* Test type mismatch

* Fix test_query_validation assert messages

* Remove support to not used types

* Remove last comma from list of types

* Fix type mismatch test

* work on comparison (#1923)

* work on comparison

* fix merge

* simplify

* add test

* fix test

* address review

* work on comparison (#1923)

* work on comparison

* fix merge

* simplify

* add test

* fix test

* address review

* More readable code

* Fix SIType_ToMultipleTypeString() ASSERT()

* SIType_ToMultipleTypeString() support to all types

* Iterate over SITypes using currentType

* Remove outdated comment

* Switch to a stack allocated buffer

* Use __builtin_popcount()

* Fix test hasLabels() mismatch type

* Use constant for buf size

* Revert "Use constant for buf size"

This reverts commit 69f34d5.

* Remove bytesWritten arg

* Use named constant for buffer size

* SIType_ToMultipleTypeString() use 3 loops

Co-authored-by: Avi Avni <avi.avni@gmail.com>

* timeout fix (#2742)

* timeout fix

* Added a test

* Fix printing of the cond-traverse and expand-into operations. (#2727)

* Fixed printing of cond-traverse and expand-into

* Added tests

* WIP addressing comments

* taking labels from corresponding AlgebraicExpression

* fix profile test

* Reverted previous changes

* Addressing comments

* WIP freeing memory

* Fix leak

* WIP leak fix

* Leak fix

* Fix crash.

* Small fix

* clean ups

Co-authored-by: Victor Polevoy <fx@thefx.co>
Co-authored-by: swilly22 <roi@redislabs.com>
Co-authored-by: Avi Avni <avi.avni@gmail.com>

* Inherit attribute set (#2751)

* inherit attribute-set when creating a new node

* fix build

* clean

Co-authored-by: swilly22 <roi@redislabs.com>

* replicate command run on main thread (#2754)

* replicate command run on main thread

* address review

* fix

* improved in out degree performance (#2757)

* improved in out degree performance

* Update graph.c

removed unused `info`

* bump version 2.10.5

* Fix test

* Update release.json (#2771)

adding focal to enterprise qa

* Update RS_VERSIONS (#2776)

add cluster versions for release testing

Co-authored-by: nafraf <nafraf@users.noreply.github.com>
Co-authored-by: Avi Avni <avi.avni@gmail.com>
Co-authored-by: Victor Polevoy <fx@thefx.co>
Co-authored-by: swilly22 <roi@redislabs.com>
Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com>
Co-authored-by: TomerHekmati <72793005+TomerHekmati@users.noreply.github.com>
AviAvni added a commit that referenced this pull request Dec 26, 2022
* UNWIND an expression that is not a list (#2650)

* _initList: Create list for not list values

* Add test_unwind_clause.py

* tck test unwinding null

* Fix unwind tests

* Fix test unwind null

Co-authored-by: Avi Avni <avi.avni@gmail.com>

* Allow update properties to NULL (#2665)

* Allow update properties to NULL

* Test remove null attribute  MERGE ON CREATE SET

* Fix Duplicate reports when matching relationship type :R|R (#2674)

* _QueryGraphAddEdge don't add duplicated reltypeIDs

* Prevent duplicated reltypes when no schema found

Co-authored-by: Avi Avni <avi.avni@gmail.com>

* add reference entities in unwind (#2692)

* fix crash using freed values (#2695)

* fix crash using freed values

* add comment

* bug fix validate MERGE SET property: issue: 2637 (#2684)

Co-authored-by: Avi Avni <avi.avni@gmail.com>

* fix crash of aggregation function in reduce (#2696)

* remove old parameter syntax (#2697)

* remove old parameter syntax

* fix test

* fix tck

* fix

* Fix optimize-label-scan (#2711)

* Replacement AlgebraicExpression fixed to original label

* Added test

* Added comments

* Update test_optimizations_plan.py

* Update optimize_label_scan.c

Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com>
Co-authored-by: Avi Avni <avi.avni@gmail.com>

* Updated readies (fix for getpy3 on xenial) (#2720)

* timeout fix (#2742)

* timeout fix

* Added a test

* replicate command run on main thread (#2754)

* replicate command run on main thread

* address review

* fix

* improved in out degree performance (#2757)

* improved in out degree performance

* Update graph.c

removed unused `info`

* bump version 2.8.21

* cp fixes

* Update release.json (#2771)

adding focal to enterprise qa

* leak fix

* leak fix2

* leak fix3

* Update RS_VERSIONS (#2776)

add cluster versions for release testing

* leak fix

* test fix

Co-authored-by: nafraf <nafraf@users.noreply.github.com>
Co-authored-by: Avi Avni <avi.avni@gmail.com>
Co-authored-by: OfirMos <82366525+OfirMos@users.noreply.github.com>
Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com>
Co-authored-by: Rafi Einstein <raffapen@outlook.com>
Co-authored-by: TomerHekmati <72793005+TomerHekmati@users.noreply.github.com>
pnxguide pushed a commit to CMU-SPEED/RedisGraph that referenced this pull request Mar 22, 2023
* improved in out degree performance

* Update graph.c

removed unused `info`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants