Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NullPointerException error reports #408

Open
sherif-fanous opened this issue May 5, 2024 · 1 comment
Open

NullPointerException error reports #408

sherif-fanous opened this issue May 5, 2024 · 1 comment

Comments

@sherif-fanous
Copy link

sherif-fanous commented May 5, 2024 •
edited
Loading

I've been getting these error reports and corresponding Prometheus alerts since setting up Piraeus Operator + Linstor on K8s a couple of days ago.

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
┊ Id                    ┊ Datetime            ┊ Node             ┊ Exception                                                                          ┊
╞═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡
┊ 6636AF5E-FAD29-000003 ┊ 2024-05-04 22:01:08 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636AF5E-FAD29-000004 ┊ 2024-05-04 22:01:17 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636AF5E-FAD29-000005 ┊ 2024-05-04 22:01:47 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636AF5E-FAD29-000006 ┊ 2024-05-04 22:02:48 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636AF5E-FAD29-000007 ┊ 2024-05-04 22:04:47 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636AF5E-FAD29-000008 ┊ 2024-05-04 22:09:48 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636AF5E-FAD29-000009 ┊ 2024-05-04 22:19:47 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636AF5E-FAD29-000010 ┊ 2024-05-04 22:49:47 ┊ S|talos-hybrid-1 ┊ NullPointerException                                                               ┊
┊ 6636C2CC-FAD29-000000 ┊ 2024-05-04 23:34:42 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636C2CC-FAD29-000001 ┊ 2024-05-04 23:34:54 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636C2CC-FAD29-000002 ┊ 2024-05-04 23:35:18 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636C2CC-FAD29-000003 ┊ 2024-05-04 23:35:20 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636C2CC-FAD29-000004 ┊ 2024-05-04 23:36:18 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636C2CC-FAD29-000005 ┊ 2024-05-04 23:38:20 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636C2CC-FAD29-000006 ┊ 2024-05-04 23:43:18 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636C2CC-FAD29-000007 ┊ 2024-05-04 23:53:19 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636C2CC-FAD29-000008 ┊ 2024-05-05 00:23:18 ┊ S|talos-hybrid-1 ┊ NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "... ┊
┊ 6636C2CC-FAD29-000009 ┊ 2024-05-05 01:23:18 ┊ S|talos-hybrid-1 ┊ NullPointerException                                                               ┊
┊ 6636C2CC-FAD29-000010 ┊ 2024-05-05 05:23:18 ┊ S|talos-hybrid-1 ┊ NullPointerException                                                               ┊
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

There are 2 different error reports as follows

ERROR REPORT 6636C2CC-FAD29-000008

============================================================

Application:                        LINBIT? LINSTOR
Module:                             Satellite
Version:                            1.27.1
Build ID:                           c6f8ceed9d50da2c4d37ae8ce20d09daf3046464
Build time:                         2024-04-25T11:12:13+00:00
Error time:                         2024-05-05 00:23:18
Node:                               talos-hybrid-1
Thread:                             DeviceManager

============================================================

Reported error:
===============

Category:                           RuntimeException
Class name:                         NullPointerException
Class canonical name:               java.lang.NullPointerException
Generated at:                       Method 'getTextLength', Source file 'Matcher.java', Line #1769

Error message:                      Cannot invoke "java.lang.CharSequence.length()" because "this.text" is null

Error context:
        An error occurred while processing resource 'Node: 'talos-hybrid-1', Rsc: 'pvc-07b05ee1-a95a-42d2-8c7c-252c4ee10988''
Call backtrace:

    Method                                   Native Class:Line number
    getTextLength                            N      java.util.regex.Matcher:1769
    reset                                    N      java.util.regex.Matcher:415
    <init>                                   N      java.util.regex.Matcher:252
    matcher                                  N      java.util.regex.Pattern:1134
    shellQuote                               N      com.linbit.utils.StringUtils:93
    accept                                   N      java.util.stream.ReferencePipeline$3$1:197
    forEachRemaining                         N      java.util.Spliterators$ArraySpliterator:992
    copyInto                                 N      java.util.stream.AbstractPipeline:509
    wrapAndCopyInto                          N      java.util.stream.AbstractPipeline:499
    evaluate                                 N      java.util.stream.AbstractPipeline:575
    evaluateToArrayNode                      N      java.util.stream.AbstractPipeline:260
    toArray                                  N      java.util.stream.ReferencePipeline:616
    toArray                                  N      java.util.stream.ReferencePipeline:622
    joinShellQuote                           N      com.linbit.utils.StringUtils:117
    exec                                     N      com.linbit.extproc.ExtCmd:111
    exec                                     N      com.linbit.extproc.ExtCmd:89
    getCurrentGID                            N      com.linbit.linstor.layer.drbd.utils.DrbdAdm:371
    allVlmsMetaDataNew                       N      com.linbit.linstor.layer.drbd.DrbdLayer:1708
    condInitialOrSkipSync                    N      com.linbit.linstor.layer.drbd.DrbdLayer:1614
    adjustDrbd                               N      com.linbit.linstor.layer.drbd.DrbdLayer:713
    processResource                          N      com.linbit.linstor.layer.drbd.DrbdLayer:250
    lambda$processResource$4                 N      com.linbit.linstor.core.devmgr.DeviceHandlerImpl:908
    processGeneric                           N      com.linbit.linstor.core.devmgr.DeviceHandlerImpl:949
    processResource                          N      com.linbit.linstor.core.devmgr.DeviceHandlerImpl:904
    processResources                         N      com.linbit.linstor.core.devmgr.DeviceHandlerImpl:370
    dispatchResources                        N      com.linbit.linstor.core.devmgr.DeviceHandlerImpl:217
    dispatchResources                        N      com.linbit.linstor.core.devmgr.DeviceManagerImpl:331
    phaseDispatchDeviceHandlers              N      com.linbit.linstor.core.devmgr.DeviceManagerImpl:1204
    devMgrLoop                               N      com.linbit.linstor.core.devmgr.DeviceManagerImpl:778
    run                                      N      com.linbit.linstor.core.devmgr.DeviceManagerImpl:672
    run                                      N      java.lang.Thread:840


END OF ERROR REPORT.

And

ERROR REPORT 6636C2CC-FAD29-000010

============================================================

Application:                        LINBIT? LINSTOR
Module:                             Satellite
Version:                            1.27.1
Build ID:                           c6f8ceed9d50da2c4d37ae8ce20d09daf3046464
Build time:                         2024-04-25T11:12:13+00:00
Error time:                         2024-05-05 05:23:18
Node:                               talos-hybrid-1
Thread:                             DeviceManager

============================================================

Reported error:
===============

Category:                           RuntimeException
Class name:                         NullPointerException
Class canonical name:               java.lang.NullPointerException
Generated at:                       <UNKNOWN>


Error context:
        An error occurred while processing resource 'Node: 'talos-hybrid-1', Rsc: 'pvc-07b05ee1-a95a-42d2-8c7c-252c4ee10988''
Call backtrace:

    Method                                   Native Class:Line number


END OF ERROR REPORT.

Both error reports pertain to PVC pvc-07b05ee1-a95a-42d2-8c7c-252c4ee10988 but this PVC doesn't even exist in my cluster. It might have been a test PVC I created earlier and then deleted.

Thanks
@ghernadi
Copy link
Contributor

ghernadi commented May 6, 2024

In case this issue persists after restarting this one satellite and possibly also the controller, I'd like to ask for TRACE logs as next step (using linstor c set-log-level TRACE --global) and send me the logs (or an SOS report using linstor sos dl) via email for example. (Feel free to disable TRACE logging with linstor c set-log-level INFO --global after you have collected the logs)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sherif-fanous @ghernadi