Skip to content

By @tomyouyou: Avoid a scary log exception when a closing connection runs into an exception during a command writer flush operation #14123

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 25, 2025

Conversation

michaelklishin
Copy link
Collaborator

This is #14121 by @tomyouyou with log message tweaks from me.

The goal of this change is to avoid a scary looking exception on a connection that's closing and cannot do much about a writer flush failure.

tomyouyou and others added 2 commits June 25, 2025 14:47
…ge number of exception logs.

When the client disconnects, flushing writer in the termination may result in a large number of exceptions due to the writer being closed.
The exceptions are as follows:

2025-06-24 17:56:06.661 [error] <0.1381.0> ** Generic server <0.1381.0> terminating, ** Last message in was {'$gen_cast',terminate}, ** When Server state == {ch, {conf,running,rabbit_framing_amqp_0_9_1,1, <0.1371.0>,<0.1379.0>,<0.1371.0>, <<"10.225.80.5:50760 -> 10.225.80.6:5673">>, {user,<<"rabbit_inside_user">>,[], [{rabbit_auth_backend_internal, #Fun<rabbit_auth_backend_internal.3.16580688>}]}, <<"/">>, <<"lzz.localdomain_rc.py_reply_89a60f0ef2114da2b3f150ca359ecf46">>, <0.1373.0>, [{<<"authentication_failure_close">>,bool,true}, {<<"connection.blocked">>,bool,true}, {<<"consumer_cancel_notify">>,bool,true}, {<<"need_notify_server_info_with_heartbeat">>,bool, true}], none,5,1800000,#{},infinity,1000000000}, {lstate,<0.1380.0>,false}, none,3, {1, [{pending_ack,2,<<"1">>,-576460618632, {resource,<<"/">>,queue, <<"lzz.localdomain_rc.py_reply_89a60f0ef2114da2b3f150ca359ecf46">>}, 1}], []}, undefined, #{<<"1">> =>, {{amqqueue, {resource,<<"/">>,queue, <<"lzz.localdomain_rc.py_reply_89a60f0ef2114da2b3f150ca359ecf46">>}, false,false,none, [{<<"x-expires">>,signedint,1800000}, {<<"x-queue-type">>,longstr,<<"classic">>}], <0.1385.0>,[],[],[],undefined,undefined,[],[], live,0,[],<<"/">>, #{user => <<"rabbit_inside_user">>, system_creation => 1750758840399767062, recover_on_declare => false, creator =>, {1750758936,"10.225.80.5",50760,"rc.py"}}, rabbit_classic_queue,#{}}, {false,5,false, [{zclient,tuple, {1750758936,"10.225.80.5",50760,"rc.py"}}]}}}, #{{resource,<<"/">>,queue, <<"lzz.localdomain_rc.py_reply_89a60f0ef2114da2b3f150ca359ecf46">>} =>, {1,{<<"1">>,nil,nil}}}, {state,none,30000,undefined}, false,1, {rabbit_confirms,undefined,#{}}, [],[],none,flow,[], {rabbit_queue_type, #{{resource,<<"/">>,queue, <<"lzz.localdomain_rc.py_reply_89a60f0ef2114da2b3f150ca359ecf46">>} =>, {ctx,rabbit_classic_queue, {rabbit_classic_queue,<0.1385.0>,#{}, #{<0.1385.0> => ok}, false}}}}, #Ref<0.2472179985.4173070337.136448>,false, {erlang,#Ref<0.2472179985.4173070337.136063>}, "rc.py",true,0,false,undefined,undefined,undefined, false}, ** Reason for termination == , ** {{shutdown,{writer,send_failed,closed}}, {gen_server,call,[<0.1379.0>,flush,infinity]}},
2025-06-24 17:56:06.665 [error] <0.1381.0>   crasher:, initial call: rabbit_channel:init/1, pid: <0.1381.0>, registered_name: [], exception exit: {{shutdown,{writer,send_failed,closed}}, {gen_server,call,[<0.1379.0>,flush,infinity]}}, in function  gen_server2:terminate/3 (gen_server2.erl, line 1172), ancestors: [<0.1378.0>,<0.1376.0>,<0.1369.0>,<0.1368.0>,<0.1169.0>, <0.1168.0>,<0.1167.0>,<0.1165.0>,<0.1164.0>,rabbit_sup, <0.249.0>], message_queue_len: 1, messages: [{'EXIT',<0.1378.0>,shutdown}], links: [<0.1378.0>], dictionary: [{msg_io_dt_cfg,{1750758936,2}}, {zext_options_dt_cfg,{1750758966,[]}}, {zlog_consumer_dt_cfg,{1750758936,false}}, {channel_operation_timeout,15000}, {rbt_trace_enable,true}, {process_name, {rabbit_channel, {<<"10.225.80.5:50760 -> 10.225.80.6:5673">>,1}}}, {counter_publish_size_dt_cfg,{1750758936,undefined}}, {peer_info, {"10.225.80.5",50760, "10.225.80.5:50760 -> 10.225.80.6:5673 - rc.py:3382128:dfe6ba8d-a42f-4ece-93df-11bff0410814", "rc.py",0}}, {peer_host_port_compname,{"10.225.80.5",50760,"rc.py"}}, {permission_cache_can_expire,false}, {debug_openv_dt_cfg,{1750758936,[]}}, {z_qref_type_dic, [{{resource,<<"/">>,queue, <<"lzz.localdomain_rc.py_reply_89a60f0ef2114da2b3f150ca359ecf46">>}, rabbit_classic_queue}]}, {zconsumer_num,1}, {virtual_host,<<"/">>}, {msg_size_for_gc,458}, {rand_seed, {#{max => 288230376151711743,type => exsplus, next => #Fun<rand.5.65977474>, jump => #Fun<rand.3.65977474>}, [20053568771696737|52030598835932017]}}, {top_queue_msg_dt_cfg, {1750758936, {0,0,0,undefined,false,false,undefined,undefined}}}], trap_exit: true, status: running, heap_size: 4185, stack_size: 28, reductions: 50613, neighbours:,
@michaelklishin michaelklishin merged commit 7ec3703 into main Jun 25, 2025
562 of 567 checks passed
@michaelklishin michaelklishin deleted the rabbitmq-server-14121 branch June 25, 2025 13:38
michaelklishin added a commit that referenced this pull request Jun 25, 2025
By @tomyouyou: Avoid a scary log exception when a closing connection runs into an exception during a command writer flush operation (backport #14123)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants