This is more or less doable currently by running multiple instances of coredns.

I think we should put more effort into making multi-threaded scaling better rather than paper over the issue with multiple processes.

Isn't that what this does? It allows multiple internal servers (same process) to run on the same address and port. Basically, in #5595 it was determined that the single socket used to read the queries was a bottleneck; this alleviates that, IIUC.

Let me tell you about our scenario.
We want to run СoreDNS as a DaemonSet in Kubernetes. This means that there is only one pod per node. We can't scale CoreDNS horizontally in that case. And vertically too, because no matter how many resources you add, the CoreDNS will not perform better (#5595)

This fix allows to scale CoreDNS vertically.

This is more or less doable currently by running multiple instances of coredns.

In general, I agree. But it is more difficult to control these instances.

If we run several processes in a container, it is not clear how to detect main process. If the instance goes down, who will restart it? We would like to have one process so that Kubernetes itself controls the lifecycle of our container.

Sorry, I clearly didn't read the code / proposal well enough. Setting up multiple listeners from the same process is the goal.

I think this is fine for now, at least until Go has better support for this.

Perhaps we can consider an alternate plugin name? “reuseport” is a bit confusing given that coredns already binds using the reuseport option. Perhaps “multiprocess” or “multiserver” or something.

Perhaps we can consider an alternate plugin name? “reuseport” is a bit confusing given that coredns already binds using the reuseport option. Perhaps “multiprocess” or “multiserver” or something.

"numsockets" would most clearly describe what it does, I think. It's not about multiple processes.

Hi!

Please take into account, that we also need additional changes in Caddy server - coredns/caddy#6 to address this issue - #5595 (comment) with performance degradation after server reload.

Maybe this should be a command line flag and not something reloadable?

Maybe this should be a command line flag and not something reloadable?

During reloading, the server restarts. When restarting, all servers start listening to a single socket because here: https://github.com/coredns/caddy/blob/master/caddy.go#L221, we have a map of server addresses to PacketConn.

With numsockets, we are starting multiple servers on the same address. Therefore, reloading is a a problem for us.

If we want this functionality with numsockets, we need to fix this somehow. It doesn't matter whether it is a plugin or a CLI flag.

Can anyone please take a look at this PR - coredns/caddy#6?

Or do you think this is enough? :)

whoami shows that the reading of the requests and the replies work properly still, and of course the restart test is critical. So I think that's probably good.

Shouldn't there be at least one failing test, since we haven't merged coredns/caddy#6 ? I had expected to see that failing, then merge the caddy change, then update the go.mod to point to the new caddy version, and see the test clear up. The fact that it's not failing now makes me think our testing is incomplete?

https://github.com/coredns/coredns/pull/6882/files#diff-33ef32bf6c23acb95f5902d7097b7a1d5128ca061167ec0716715b0b9eeaa5f6R146
In this PR I did replace like

// Temp replace. Will be removed from this PR when the caddy changes are merged
replace github.com/coredns/caddy v1.1.1 => github.com/glebkin/caddy v0.0.0-20240924070517-c7def4b62439

So when caddy changes will be merged, I will remove it from here and will update the caddy version

Codecov Report

Attention: Patch coverage is 32.29167% with 65 lines in your changes missing coverage. Please review.

Project coverage is 57.75%. Comparing base (93c57b6) to head (df5da4d).
Report is 1317 commits behind head on master.

@@            Coverage Diff             @@
##           master    #6882      +/-   ##
==========================================
+ Coverage   55.70%   57.75%   +2.05%     
==========================================
  Files         224      270      +46     
  Lines       10016    17371    +7355     
==========================================
+ Hits         5579    10033    +4454     
- Misses       3978     6726    +2748     
- Partials      459      612     +153     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Shmillerov if you happen to be at KubeCon NA in Salt Lake City next month, would be great if you want to talk about this in the CoreDNS maintainer track session. Otherwise I will cover it. Great stuff.

@johnbelamaric, thank you, but I can't participate :) I believe you will cover me!

@Shmillerov just FYI since I don't think you are on the CoreDNS Slack. We are hoping to do the following in the next few days:

• merge low risk PRs and cut a patch release
• merge this (and maybe others?) and cut a minor release