| Age | Commit message (Collapse) | Author |
|---|
|
Now that we have a hash-set implementation we can use that
instead of a hash-table with a static value.
|
|
Notes:
Merged-By: ioquatix <[email protected]>
|
|
[Bug #21211]
Socket errors raised from background threads are hard to track down because
their backtrace starts from the spawned thread.
To solve this we can raise a new error with the old one as `cause`.
Notes:
Merged: https://github.com/ruby/ruby/pull/13041
|
|
Fix segfault crash observable with TCPSocket.open(nil, nil)
Notes:
Merged: https://github.com/ruby/ruby/pull/12934
|
|
[Bug #21179]
```
socket.rb:1046:in 'Socket::HostnameResolutionStore#get_addrinfo': can not access non-shareable objects in constant
Socket::HostnameResolutionStore::PRIORITY_ON_V6 by non-main ractor. (Ractor::IsolationError)
from socket.rb:724:in 'block in Socket.tcp_with_fast_fallback'
from socket.rb:720:in 'Socket.tcp_with_fast_fallback'
```
Notes:
Merged: https://github.com/ruby/ruby/pull/12896
|
|
|
|
to avoid conflicts with other functions.
This was pointed out in https://github.com/ruby/ruby/pull/11653#discussion_r1837356617 , but it was not fixed at that time.
Notes:
Merged-By: shioimm <[email protected]>
|
|
(#12678)
* Do not save ResolutionError if resolution succeeds for any address family
Socket with Happy Eyeballs Version 2 performs connection attempts and name resolution in parallel.
In the existing implementation, if a connection attempt failed for one address family while name resolution was still in progress for the other, and that name resolution later failed, the method would terminate with a name resolution error.
This behavior was intended to ensure that the final error reflected the most recent failure, potentially overriding an earlier error.
However, [Bug #21088](https://bugs.ruby-lang.org/issues/21088) made me realize that terminating with a name resolution error is unnatural when name resolution succeeded for at least one address family.
This PR modifies the behavior so that if name resolution succeeds for one address family, any name resolution error from the other is not saved.
This PR includes the following changes:
* Do not display select(2) as the system call that caused the raised error, as it is for internal processing
* Fix bug: Get errno with Socket::SO_ERROR in Windows environment with a workaround for tests not passing
Notes:
Merged-By: shioimm <[email protected]>
|
|
`free_fast_fallback_getaddrinfo_*` (#12661)
Ensure that `getaddrinfo_entry` and `getaddrinfo_shared` exist before free them in the main thread.
Notes:
Merged-By: shioimm <[email protected]>
|
|
Signed-off-by: Raul Gutierrez Segales <[email protected]>
Notes:
Merged: https://github.com/ruby/ruby/pull/12506
|
|
write(2) failure (#12457)
Rarely, there are cases where a write(2) call from a child thread
to notify the main thread of the completion of name resolution fails.
If this happens while the main thread is waiting in `rb_thread_fd_select`,
rb_thread_fd_select may not notice that the name resolution has completed and end up hanging.
This issue becomes a problem when there are no sockets currently being connected,
no addresses ready for immediate connection attempts,
and name resolution has already completed for one address family
while the main thread is waiting for the name resolution of the other address family.
(If name resolution is not completed for either address family,
the chances of write(2) failing in both child threads are likely low.)
To avoid this issue, a timeout is introduced to rb_thread_fd_select under the above conditions.
This way, even if the issue occurs,
the completion of name resolution should still be detected
in the subsequent `if (!resolution_store.is_all_finished) ...` block.
Notes:
Merged-By: shioimm <[email protected]>
|
|
Also, a topic about Socket::ResolutionError is added to NEWS
Notes:
Merged-By: shioimm <[email protected]>
|
|
Wrap `do_fast_fallback_getaddrinfo` with `rb_thread_prevent_fork`
Referencing PR #10864,
wrap `do_fast_fallback_getaddrinfo` with `rb_thread_prevent_fork`
to avoid fork safety issues.
`do_fast_fallback_getaddrinfo` internally uses getaddrinfo(3),
leading to fork safety issues, as described in PR #10864.
This change ensures that `do_fast_fallback_getaddrinfo`
is guarded by `rb_thread_prevent_fork`,
preventing fork during its execution and avoiding related issues.
Notes:
Merged-By: shioimm <[email protected]>
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/12350
|
|
Since it's a singleton method, it should be referenced as `Socket.tcp_fast_fallback=`,
not `Socket#tcp_fast_fallback=`.
Notes:
Merged: https://github.com/ruby/ruby/pull/12348
|
|
This change includes the following updates:
- Added an environment variable `RUBY_TCP_NO_FAST_FALLBACK` to control enabling/disabling fast_fallback
- Updated documentation and man pages
- Revised the implementation of Socket.tcp_fast_fallback= and Socket.tcp_fast_fallback, which previously performed dynamic name resolution of constants and variables. As a result, the following performance improvements were achieved:
(Case of 1000 executions of `TCPSocket.new` to the local host)
Rehearsal -----------------------------------------
before 0.031462 0.147946 0.179408 ( 0.249279)
after 0.031164 0.146839 0.178003 ( 0.346935)
-------------------------------- total: 0.178003sec
user system total real
before 0.027584 0.138712 0.166296 ( 0.233356)
after 0.025953 0.127608 0.153561 ( 0.237971)
Notes:
Merged-By: shioimm <[email protected]>
|
|
Any memory allocated with xmalloc needs to be matched with xfree rather
than plain free.
Ruby unfortunately redefines strdup to be ruby_strdup, which uses
xmalloc so needs to be xfreed. Previously these were mismatched.
This commit changes the copy to be an explicit ruby_strdup (to avoid
confusion) and the free to be xfree.
Notes:
Merged: https://github.com/ruby/ruby/pull/12311
|
|
* Use `rb_thread_fd_select` instead of select(2)
For fixing https://bugs.ruby-lang.org/issues/20932 .
`TCPSocket.new`, which internally uses select(2) for HEv2, can cause SEGV if the number of file descriptors exceeds `FD_SETSIZE`.
This change avoids that issue by replacing select(2) with `rb_thread_fd_select`, which is provided as part of Ruby's internal API.
---
This includes the following changes.
* rb_thread_fd_select does not need common pipe
Notes:
Merged-By: shioimm <[email protected]>
|
|
The following two commits fix the proper clearing of the Connection Attempt Delay in `TCPSocket.new`.
- https://github.com/ruby/ruby/pull/12087/commits/b2f610b0edf8f84d9d0fce038f3ebfa9f2cb0223
- https://github.com/ruby/ruby/pull/12223/commits/6f4efaec5352cdeb6cab86036a83f53c9fa5c3d2
The same fix will be applied to `Socket.tcp`.
Notes:
Merged-By: shioimm <[email protected]>
|
|
In some locations we were using shared->lock and in others
&shared->lock, and we were leaking the allocated memory.
Notes:
Merged: https://github.com/ruby/ruby/pull/12239
|
|
connection failure (#12223)
* Improve the conditions for clearing the Connection Attempt Delay upon connection failure
This change addresses a case that was overlooked in ruby/ruby#12087.
In the previous change, the Connection Attempt Delay was cleared at the point of a connection failure only if both of the following conditions were met:
- No other sockets were attempting a connection
- There were addresses still available to start a new connection
In this update, the second condition has been removed.
As a result, if name resolution succeeds after a connection failure and new addresses are obtained, it will be able to immediately attempt a connection to one of them.
If there are no sockets attempting a connection, no addresses available for connection, and name resolution has completed, an exception will still be raised as before.
---
Additionally, the following minor fixes have been made:
* Refactor: Remove unnecessary members
Notes:
Merged-By: shioimm <[email protected]>
|
|
`TCPSocket.new` with HEv2 uses three threads.
The last of these threads to exit closed pipes.
However, if pipes were open at the end of the main thread, they would leak.
This change avoids this by closing pipes at the end of the main thread.
Notes:
Merged-By: shioimm <[email protected]>
|
|
with `Socket.tcp_fast_fallback=`
The functions that `Socket.tcp` had are now also available in `TCPSocket.new`.
Notes:
Merged-By: shioimm <[email protected]>
|
|
With https://github.com/ruby/ruby/pull/12156,
the memory of the `struct fast_fallback_getaddrinfo_shared`
is now allocated even if there is only one address family.
This change will always free it when `TCPSocket.new` finishes.
Notes:
Merged-By: shioimm <[email protected]>
|
|
```
for (int i = 0; i < arg->family_size; i++) {
arg->getaddrinfo_entries[i] = allocate_fast_fallback_getaddrinfo_entry();
if (!(arg->getaddrinfo_entries[i])) rb_syserr_fail(errno, "calloc(3)");
```
If the allocation fails in the second interation, the memory allocated
in the first iteration would be leaked.
This change prevents the memory leak by allocating the memory in
advance.
(The struct name `fast_fallback_getaddrinfo_shared` might no longer be
good.)
Notes:
Merged: https://github.com/ruby/ruby/pull/12163
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/12161
|
|
* Do not save the last_error if there are no sockets waiting to be connected
In this implementation, the results of both name resolution and connection attempts are awaited using select(2).
When it returned, the implementation attempted to check for connections even if there were no sockets currently attempting to connect, treating the absence of connected sockets as a connection failure.
With this fix, it will no longer check for connections when there are no sockets waiting to be connected.
Additionally, the following minor fixes have been made:
* Handle failure of getsockopt(2) and removed unnecessary continue in the loop
* Tweak: Use common API to check in_progress_fds
* Safely call TCPServer.new in test
* Set empty writefds when there is no socket waiting to be connected
* Enable fast_fallback option
Notes:
Merged-By: shioimm <[email protected]>
|
|
`rb_thread_call_without_gvl2` is used to wait for the results of name resolution and connection attempts.
When there is only one address family to resolve, the necessary resources were not being passed to the UBF.
With this change, the handling of resources has been revised and organized to work consistently, whether there are two address families to resolve or only one.
Notes:
Merged-By: shioimm <[email protected]>
|
|
even if a system call error happens after the name resolution failure in the child thread.
pipe and write(2) are used to notify the main thread of the name resolution results from the child thread.
After name resolution is completed in the child thread, if the call to write(2) fails, the main thread retrieves the resolved addresses.
However, when name resolution failed, the corresponding error was not being saved in `last_error`.
With this change, name resolution failures will now be saved in last_error even if the write(2) call in the child thread fails.
Notes:
Merged-By: shioimm <[email protected]>
|
|
http://ci.rvm.jp/results/trunk-repeat50@ruby-sp2-noble-docker/5420911
```
/tmp/ruby/src/trunk-repeat50/ext/socket/ipsocket.c: In function ‘reallocate_connection_attempt_fds’:
/tmp/ruby/src/trunk-repeat50/ext/socket/ipsocket.c:292:62: warning: pointer ‘fds’ may be used after ‘realloc’ [-Wuse-after-free]
292 | for (int i = current_capacity; i < new_capacity; i++) fds[i] = -1;
| ^
/tmp/ruby/src/trunk-repeat50/ext/socket/ipsocket.c:288:9: note: call to ‘realloc’ here
288 | if (realloc(fds, new_capacity * sizeof(int)) == NULL) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
|
|
http://ci.rvm.jp/results/trunk_asan@ruby-sp1/5409001
```
=================================================================
==3263562==ERROR: AddressSanitizer: stack-use-after-return on address 0x735a8f190da8 at pc 0x735a6f58dabc bp 0x735a639ffd10 sp 0x735a639ffd08
READ of size 4 at 0x735a8f190da8 thread T211
=================================================================
```
Notes:
Merged-By: shioimm <[email protected]>
|
|
to suppress failing output in CI.
Notes:
Merged-By: shioimm <[email protected]>
|
|
http://ci.rvm.jp/results/trunk_asan@ruby-sp1/5408428
```
==3159643==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x796cf8f09041 at pc 0x6539bbf68ded bp 0x796cfadffcf0 sp 0x796cfadff4b8
READ of size 2 at 0x796cf8f09041 thread T13
#0 0x6539bbf68dec in strlen (/tmp/ruby/build/trunk_asan/ruby+0x18edec) (BuildId: cca267c7ae091060e1b82a6b4ed1aeaf00edebab)
```
Notes:
Merged: https://github.com/ruby/ruby/pull/12089
|
|
Do not wait Connection Attempt Delay without in progress fds
Reset Connection Attempt Delay when connection fails and there is no other socket connection in progress.
This is intended to resolve an issue that was temporarily worked around in Pull Request #12062.
`TCPServer::new` (used in tests such as `TestNetHTTP_v1_2_chunked#test_timeout_during_non_chunked_streamed_HTTP_session_write`) can only connect over either IPv6 or IPv4, depending on the environment.
Since HEv2 attempts to connect over IPv6 first, environments where IPv6 connections are unavailable return ECONNREFUSED immediately.
In such cases, the client should immediately retry the connection over IPv4.
However, HEv2 includes a specification for a "Connection Attempt Delay," where it waits 250ms after the previous connection attempt before starting the next one.
This delay causes Net::OpenTimeout (100ms) to be exceeded while waiting for the next connection attempt to start.
With this change, when a connection attempt fails, if there are sockets still attempting to connect and there are addresses yet to be tried, the Connection Attempt Delay will be resetted, allowing the next connection attempt to start immediately.
---
Additionally, the following minor fixes have been made:
- The `nfds` value used for select(2) is now reset with each wait.
Notes:
Merged-By: shioimm <[email protected]>
|
|
TCPSocket.new (#11653)
* Introduction of Happy Eyeballs Version 2 (RFC8305) in TCPSocket.new
This is an implementation of Happy Eyeballs version 2 (RFC 8305) in `TCPSocket.new`.
See https://github.com/ruby/ruby/pull/11653
1. Background
Prior to this implementation, I implemented Happy Eyeballs Version 2 (HEv2) for `Socket.tcp` in https://github.com/ruby/ruby/pull/9374.
HEv2 is an algorithm defined in [RFC 8305](https://datatracker.ietf.org/doc/html/rfc8305), aimed at improving network connectivity.
For more details on the specific cases that HEv2 helps, please refer to https://bugs.ruby-lang.org/issues/20108.
2. Proposal & Outcome
This proposal implements the same HEv2 algorithm in `TCPSocket.new`.
Since `TCPSocket.new` is used more widely than `Socket.tcp`, this change is expected to broaden the impact of HEv2's benefits.
Like `Socket.tcp`, I have also added `fast_fallback` keyword argument to `TCPSocket.new`.
This option is set to true by default, enabling the HEv2 functionality.
However, users can explicitly set it to false to disable HEv2 and use the previous behavior of `TCPSocket.new`.
It should be noted that HEv2 is enabled only in environments where pthreads are available.
This specification follows the approach taken in https://bugs.ruby-lang.org/issues/19965 , where name resolution can be interrupted.
(In environments where pthreads are not available, the `fast_fallback` option is ignored.)
3. Performance
Below is the benchmark of 100 requests to `www.ruby-lang.org` with the fast_fallback option set to true and false, respectively.
While there is a slight performance degradation when HEv2 is enabled, the degradation is smaller compared to that seen in `Socket.tcp`.
```
~/s/build ❯❯❯ ../install/bin/ruby ../ruby/test.rb
Rehearsal --------------------------------------------------------
fast_fallback: true 0.017588 0.097045 0.114633 ( 1.460664)
fast_fallback: false 0.014033 0.078984 0.093017 ( 1.413951)
----------------------------------------------- total: 0.207650sec
user system total real
fast_fallback: true 0.020891 0.124054 0.144945 ( 1.473816)
fast_fallback: false 0.018392 0.110852 0.129244 ( 1.466014)
```
* Update debug prints
Co-authored-by: Nobuyoshi Nakada <[email protected]>
* Remove debug prints
* misc
* Disable HEv2 in Win
* Raise resolution error with hostname resolution
* Fix to handle errors
* Remove warnings
* Errors that do not need to be handled
* misc
* Improve doc
* Fix bug on cancellation
* Avoid EAI_ADDRFAMILY for resolving IPv6
* Follow upstream
* misc
* Refactor connection_attempt_fds management
- Introduced allocate_connection_attempt_fds and reallocate_connection_attempt_fds for improved memory allocation of connection_attempt_fds
- Added remove_connection_attempt_fd to resize connection_attempt_fds dynamically.
- Simplified the in_progress_fds function to only check the size of connection_attempt_fds.
* Rename do_pthread_create to raddrinfo_pthread_create to avoid conflicting
---------
Co-authored-by: Nobuyoshi Nakada <[email protected]>
Notes:
Merged-By: shioimm <[email protected]>
|
|
With the introduction of Happy Eyeballs Version 2 to `Socket::tcp`, the following areas have been mainly enhanced:
- How the value specified for `connect_timeout` works
- How Socket.tcp operates with Happy Eyeballs Version 2
A description for the new option `fast_fallback` has been added in https://github.com/ruby/ruby/pull/11813.
Notes:
Merged: https://github.com/ruby/ruby/pull/11920
Merged-By: nobu <[email protected]>
|
|
Notes:
Merged-By: ioquatix <[email protected]>
|
|
Both `rb_ensure` and `udp_bind_internal` return `VALUE`.
|
|
Notes:
Merged-By: ioquatix <[email protected]>
|
|
This function may be a macro for optimization, which will be expanded
to `rb_ary_new_from_values`.
```
ext/socket/ancdata.c: In function ‘bsock_recvmsg_internal’:
ext/socket/ancdata.c:1648:1: error: embedding a directive within macro arguments is not portable
1648 | #if defined(HAVE_STRUCT_MSGHDR_MSG_CONTROL)
| ^
ext/socket/ancdata.c:1650:1: error: embedding a directive within macro arguments is not portable
1650 | #else
| ^
ext/socket/ancdata.c:1652:1: error: embedding a directive within macro arguments is not portable
1652 | #endif
| ^
```
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11844
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11844
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11844
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11796
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11796
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11796
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11813
Merged-By: nobu <[email protected]>
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/11611
|
|
|
|
[Feature #20590]
For better of for worse, fork(2) remain the primary provider of
parallelism in Ruby programs. Even though it's frowned uppon in
many circles, and a lot of literature will simply state that only
async-signal safe APIs are safe to use after `fork()`, in practice
most APIs work well as long as you are careful about not forking
while another thread is holding a pthread mutex.
One of the APIs that is known cause fork safety issues is `getaddrinfo`.
If you fork while another thread is inside `getaddrinfo`, a mutex
may be left locked in the child, with no way to unlock it.
I think we could reduce the impact of these problem by preventing
in for the most notorious and common cases, by locking around
`fork(2)` and known unsafe APIs with a read-write lock.
Notes:
Merged: https://github.com/ruby/ruby/pull/10864
|
