Skip to content

Fix out of sockets problem #99

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 20, 2023
Merged

Fix out of sockets problem #99

merged 3 commits into from
Mar 20, 2023

Conversation

BiffoBear
Copy link
Contributor

closes #93 Implements a method of tracking which Wiznet hardware sockets are assigned to instances of socket.socket().

This works best when the user creates a socket.socket() instance using the with statement.
If the socket is created using x = socket.socket() the user must call socket.close() to be sure that the hardware socket will be released.
When garbage collection occurs, socket.__del__() will release the socket, but this is not reliable.

Copy link
Member

@anecdata anecdata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested. Keeps code from initializing too many sockets, even when the sockets aren't used initially.

I'm not clear why only socket #0 can be used for DNS. There's a use case where sockets use IP address rather than host names, and this reduces the available sockets. I don't know whether it's better to reserve a DNS socket, or raise an exception if the application uses too many sockets (like other uses of sockets).

edit: I don't see the linkage that enforces DNS to use socket #0. In a test, it seems to be fine with other sockets:

*** Opening socket 3
* Opening W5k Socket, protocol=2
* DNS: Sending request packet...
* socket_available called on socket 3, protocol 2

If that's the case, then I lean toward allowing the user code to use all available sockets, and raise an exception if it uses too many (for whatever purpose).

P.S. raw socket I think is needed for ping, but I don't see that ping is implemented in this library

@BiffoBear
Copy link
Contributor Author

BiffoBear commented Feb 18, 2023

Tested. Keeps code from initializing too many sockets, even when the sockets aren't used initially.

I'm not clear why only socket #0 can be used for DNS. There's a use case where sockets use IP address rather than host names, and this reduces the available sockets. I don't know whether it's better to reserve a DNS socket, or raise an exception if the application uses too many sockets (like other uses of sockets).

edit: I don't see the linkage that enforces DNS to use socket #0. In a test, it seems to be fine with other sockets:

*** Opening socket 3
* Opening W5k Socket, protocol=2
* DNS: Sending request packet...
* socket_available called on socket 3, protocol 2

If that's the case, then I lean toward allowing the user code to use all available sockets, and raise an exception if it uses too many (for whatever purpose).

P.S. raw socket I think is needed for ping, but I don't see that ping is implemented in this library

Good question.

My thought process was:

  1. If all the hardware sockets are assigned to socket.socket objects, then calls to maintain the DHCP lease and to socket.gethostbyname() will always fail.
  2. It is reasonable for the user to assume that calls to maintain a DHCP lease and to socket.gethostbyname() should succeed, therefore keep one hardware socket free for these calls.
  3. If one hardware socket is to be reserved, let it be socket 0, because it has some unique features that might be useful in the future.

So, by leaving one socket free, these other calls will succeed.

edit: I don't see the linkage that enforces DNS to use socket #0. In a test, it seems to be fine with other sockets:

Currently DHCP and DNS modules both import socket and use that. There's PR for DHCP module to use direct calls to the Wiznet chip instead of the socket module. I'm working on doing the same for the DNS module at the moment.

If a call is from anything other than a socket.socket() it tries for hardware socket 0 first. If that is busy it will try the remaining sockets. DNS calls will not be restricted only to hardware socket 0.

This PR works with the current version of the Wiznet5k library, but is written with a future versioning mind, where DHCP and DNS make direct calls to the Wiznet hardware instead of instantiating a socket.socket().

I'm fully open to just allowing all the hardware sockets to instances of socket.socket if that is the preferred option.

Also, this PR might be shelved as a draft until the updates to DHCP and DNS have been merged.

@anecdata
Copy link
Member

I forgot about DHCP, which I think is also UDP, though I never understood why it's needed at the user level when all of the other CircuitPython networking environments don't. Somehow, DHCP doesn't seem to cut into the socket count in other environments either, but I don't know how that could be?

You have a lot planned, you may want to involve @brentru, or someone else with more knowledge of this library than me. I'm always happy to test, but not the best one for code reviews

@BiffoBear
Copy link
Contributor Author

I forgot about DHCP, which I think is also UDP, though I never understood why it's needed at the user level when all of the other CircuitPython networking environments don't. Somehow, DHCP doesn't seem to cut into the socket count in other environments either, but I don't know how that could be?

You have a lot planned, you may want to involve @brentru, or someone else with more knowledge of this library than me. I'm always happy to test, but not the best one for code reviews

If the WIZNET5K instance has is_dhcp=True then an instance of the DHCP class is created by WIZNET5K. Periodically the T1 and T2 timers have to be checked to see whether the DHCP lease needs to be renewed. It is strange that the user is expected to make these checks. They could be incorporated into one of the WIZNET5K methods to automate it.

The DHCP protocol doesn't take any action until the T1 timer (50% of the lease time) expires. Then it periodically attempts to renew the lease. This renewal attempt will definitely require a hardware socket, however if the lease is e.g. 24 hours, then you would have to wait for 12 hours with all the hardware sockets assigned for the problem to manifest itself. This may not show up in testing.

@brentru does pop in every now and then.

I greatly appreciate your comments and testing. Thank you 😃

@anecdata
Copy link
Member

BTW, I do think that the assumption that there will be a close or with exiting to free up the socket is reasonable. Anything else risks stealing away the socket from some further use.

# Conflicts:
#	adafruit_wiznet5k/adafruit_wiznet5k_socket.py
Copy link
Contributor

@FoamyGuy FoamyGuy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. I tested successfully with the simpleserver / client and the simpletest that uses requests library.

Thank you for this fix @BiffoBear!

@FoamyGuy FoamyGuy merged commit 6581863 into adafruit:main Mar 20, 2023
adafruit-adabot added a commit to adafruit/Adafruit_CircuitPython_Bundle that referenced this pull request Mar 21, 2023
Updating https://github.com/adafruit/Adafruit_CircuitPython_VL53L1X to 1.1.10 from 1.1.9:
  > Merge pull request adafruit/Adafruit_CircuitPython_VL53L1X#15 from jposada202020/adding_version_metadata
  > Add upload url to release action

Updating https://github.com/adafruit/Adafruit_CircuitPython_Wiznet5k to 2.4.0 from 2.0.0:
  > Merge pull request adafruit/Adafruit_CircuitPython_Wiznet5k#97 from BiffoBear/shorter_socket_exit_timeout
  > Merge pull request adafruit/Adafruit_CircuitPython_Wiznet5k#99 from BiffoBear/fix_out_of_sockets
  > Merge pull request adafruit/Adafruit_CircuitPython_Wiznet5k#101 from BiffoBear/fix_socket.close()_behaviour

Updating https://github.com/adafruit/Adafruit_CircuitPython_HID to 5.3.4 from 5.3.3:
  > Merge pull request adafruit/Adafruit_CircuitPython_HID#113 from dhalbert/fix-led-status
  > Add upload url to release action
  > Add .venv to .gitignore
  > Update .pylintrc for v2.15.5
  > Fix release CI files
  > Update pylint to 2.15.5
  > Updated pylint version to 2.13.0
  > Switching to composite actions

Updating https://github.com/adafruit/Adafruit_CircuitPython_Bundle/circuitpython_library_list.md to NA from NA:
  > Updated download stats for the libraries
@BiffoBear BiffoBear deleted the fix_out_of_sockets branch March 22, 2023 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

.socket() allows creation of more sockets than available
3 participants