Age | Commit message (Collapse) | Author |
|
* hash.c, internal.h: support theap for small Hash.
Introduce RHASH_ARRAY (li_table) besides st_table and small Hash
(<=8 entries) are managed by an array data structure.
This array data can be managed by theap.
If st_table is needed, then converting array data to st_table data.
For st_table using code, we prepare "stlike" APIs which accepts hash value
and are very similar to st_ APIs.
This work is based on the GSoC achievement
by tacinight <[email protected]> and refined by ko1.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65454 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
Following how mjit_worker.c currently works, rb_f_system
now ensures the VM-wide waitpid lists is locked before
creating a new process via fork/vfork.
This ensures other rb_waitpid callers cannot steal work and
there are no possible race conditions from toggling
ruby_nocldwait without the use of atomics.
This sets us up for implementing MJIT process management
logic using normal Ruby APIs prepares us for VM-wide
asynchronous/event-base waitpid which can allow MJIT to
work without worker threads.
Take 2: set waitpid_state.pid on platforms w/o fork.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65437 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
http://mswinci.japaneast.cloudapp.azure.com/vc12-x64/ruby-trunk/log/20181030T003541Z.fail.html.gz
I have no chance of getting anything to work on proprietary platforms :<
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65436 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65435 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
Following how mjit_worker.c currently works, rb_f_system
now ensures the VM-wide waitpid lists is locked before
creating a new process via fork/vfork.
This ensures other rb_waitpid callers cannot steal work and
there are no possible race conditions from toggling
ruby_nocldwait without the use of atomics.
This sets us up for implementing MJIT process management
logic using normal Ruby APIs prepares us for VM-wide
asynchronous/event-base waitpid which can allow MJIT to
work without worker threads.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65434 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
rb_hrtime_t is a more pleasant type to use and this can make
future changes around sleeping/scheduling easier.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65182 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65039 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64988 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
* process.c (check_exec_redirect): use RARRAY_AREF() instead of
using RARRAY_PTR().
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64987 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
This is the safer option, as there seems to be cases where checking
waitpid_state.ret is insufficient in ensure. I'm not 100% sure
why this is, but this change was required for my work-in-progress
Thread::Light patch series, too...
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64782 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
Oddly, all existing test cases passed multiple times
before this patch (even with --jit-wait), so this seems
like a difficult failure to prove.
Fixes: r64576 ("process.c: simplify SIGCHLD-based waitpid")
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64577 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
Introduce a new rb_thread_sleep_interruptible that does not
execute interrupts before sleeping. Skipping the interrupt
check before sleep is required for out-of-GVL ruby_waitpid_all
to function properly when setting waitpid_state.ret
Now that ubf_select can be called by the gvl.timer thread
without recursive locking gvl.lock, we can safely use
rb_threadptr_interrupt to deal with waking up sleeping
processes,
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64576 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
It is identical to do_waitpid, and the win32 version will not
be needed for MJIT (since win32 does not suffer from the
waitpid(-1, ...) conflict where waits can get stolen.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64541 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
This re-reverts commit r64447.
The issue was machine side problem.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64493 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
This reverts commit r64407.
Maybe it breaks http://mswinci.japaneast.cloudapp.azure.com/vc12-x64/ruby-trunk/log/20180817T095734Z.fail.html.gz
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64447 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
* process.c (ruby_waitpid_all): nothing to do unless SIGCHLD is
available.
* signal.c (ruby_nocldwait): used only if SIGCHLD is available.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64407 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
Arbitrarily closing file descriptors on exec breaks use cases
where a Ruby process sets up a descriptor for non-Ruby children
to use. For example, the "rake foo" target may spawn any number
of subprocesses (Ruby or not) which depends on parsing the "FOO"
environment variable for out_fd:99 and writing to foo.out
FOO=out_fd:99 rake foo 99>>foo.out
Unfortunately, this introduced one incompatibility in
test/lib/test/unit.rb and it now requires explicitly setting
IO#close_on_exec=true
[ruby-core:88007] [Misc #14907]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64399 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
We do not need to rely on SIGVTALRM for non-sighandler wakeups.
This will reduce spurious wakeups in cases where sigwait_fd
is not grabbed again, soon.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64389 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
This reverts commit 194a6a2c68e9c8a3536b24db18ceac87535a6051 (r64203).
Race conditions which caused the original reversion will be fixed
in the subsequent commit.
[ruby-core:88360] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64352 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64275 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
getgroups(2) may return a GID list that includes duplicated GIDs.
The behavior is totaly depends on what OS is used.
This commit fixes the example of Process.groups so that the example
is independent of this OS-dependent features.
Additonaly, this commit adds the description of such system-dependent
characteristics of Process.groups.
[ruby-dev:50603] [Bug #14969]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64265 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
This wasn't intended in r64253.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64254 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
by sharing it with vm.c in internal.h.
vm.c: ditto
internal.h: ditto
mjit.h: share more.
mjit.c: make sure the third arguemnt is not used
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64253 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
[ruby-core:88306]
Revert "process.c: ensure th->interrupt lock is held when migrating"
This reverts commit 5ca416bdf6b6785cb20f139c2c514eda005fe42f (r64201)
Revert "process.c (rb_waitpid): reduce sigwait_fd bouncing"
This reverts commit 217bdd776fbeea3bfd0b9324eefbfcec3b1ccb3e (r64200).
Revert "test/ruby/test_thread.rb (test_thread_timer_and_interrupt): add timeouts"
This reverts commit 9f395f11202fc3c7edbd76f5aa6ce1f8a1e752a9 (r64199).
Revert "thread_pthread.c (native_sleep): reduce ppoll sleeps"
This reverts commit b3aa256c4d43d3d7e9975ec18eb127f45f623c9b (r64193).
Revert "thread.c (consume_communication_pipe): do not retry after short read"
This reverts commit 291a82f748de56e65fac10edefc51ec7a54a82d4 (r64185).
Revert "test/ruby/test_io.rb (test_race_gets_and_close): timeout each thread"
This reverts commit 3dbd8d1f66537f968f0461ed8547460b3b1241b3 (r64184).
Revert "thread_pthread.c (gvl_acquire_common): persist timeout across calls"
This reverts commit 8c2ae6e3ed072b06fc3cbc34fa8a14b2acbb49d5 (r64165).
Revert "test/ruby/test_io.rb (test_race_gets_and_close): use SIGABRT on timeout"
This reverts commit 931cda4db8afd6b544a8d85a6815765a9c417213 (r64135).
Revert "thread_pthread.c (gvl_yield): do ubf wakeups when uncontended"
This reverts commit 508f00314f46c08b6e9b0141c01355d24954260c (r64133).
Revert "thread_pthread.h (native_thread_data): split condvars on some platforms"
This reverts commit a038bf238bd9a24bf1e1622f618a27db261fc91b (r64124).
Revert "process.c (waitpid_nogvl): prevent conflicting use of sleep_cond"
This reverts commit 7018acc946882f21d519af7c42ccf84b22a46b27 (r64117).
Revert "thread_pthread.c (rb_sigwait_sleep): th may be 0 from MJIT"
This reverts commit 56491afc7916fb24f5c4dc2c632fb93fa7063992 (r64116).
Revert "thread*.c: waiting on sigwait_fd performs periodic ubf wakeups"
This reverts commit ab47a57a46e70634d049e4da20a5441c7a14cdec (r64115).
Revert "thread_pthread.c (gvl_destroy): make no-op on GVL bits"
This reverts commit 95cae748171f4754b97f4ba54da2ae62a8d484fd (r64114).
Revert "thread_pthread.c (rb_sigwait_sleep): fix uninitialized poll set in UBF case"
This reverts commit 4514362948fdb914c6138b12d961d92e9c0fee6c (r64113).
Revert "thread_pthread.c (rb_sigwait_sleep): re-fix [Bug #5343] harder"
This reverts commit 26b8a70bb309c7a367b9134045508b5b5a580a77 (r64111).
Revert "thread.c: move ppoll wrapper into thread_pthread.c"
This reverts commit 3dc7727d22fecbc355597edda25d2a245bf55ba1 (r64110).
Revert "thread.c: move ppoll wrapper before thread_pthread.c"
This reverts commit 2fa1e2e3c3c5c4b3ce84730dee4bcbe9d81b8e35 (r64109).
Revert "thread_pthread.c (ubf_select): refix [Bug #5343]"
This reverts commit 4c1ab82f0623eca91a95d2a44053be22bbce48ad (r64108).
Revert "thread_win32.c: suppress warnings by -Wsuggest-attribute"
This reverts commit 6a9b63e39075c53870933fbac5c1065f7d22047c (r64159).
Revert "thread_pthread: remove timer-thread by restructuring GVL"
This reverts commit 708bfd21156828526fe72de2cedecfaca6647dc1 (r64107).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64203 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
w->cond may be changed without our knowledge in waitpid_nogvl
without th->interrupt_lock
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64201 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
Once a thread has acquired sigwait_fd, hold onto it until
waitpid is complete. This prevents unnecessary migration
and atomic operations.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64200 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
We don't use pthreads cancellation ourselves and it's painful to
use correctly. Any cancelled threads would break
vm->living_threads, GVL, thread_sync.c, autoload, etc...
So don't bother caring; because we can't stop rogue extensions
from completely breaking the VM in other ways, either.
[ruby-core:88282] [Misc #14962]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64197 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) reportedly fails to compile
cf: https://rubyci.org/logs/rubyci.s3.amazonaws.com/centos6/ruby-trunk/log/20180726T093003Z.fail.html.gz
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64119 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
We see several occurrence of "diagnostic push/pop" so why not
make them macros. Tested on GCC8 / Clang 6.
Note that ruby.h is intentionally left untouched because we don't
want to introduce new public macros.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64118 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
We reuse sleep_cond for waitpid notifications as well as GVL
waiting. So we must take care to not hold onto sleep_cond
when we try to reacquire GVL.
[ruby-core:88183]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64117 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
To reduce resource use and reduce CI failure; remove
timer-thread. Single-threaded Ruby processes (including forked
children) will never see extra thread overhead. This prevents
glibc and jemalloc from going into multi-threaded mode and
initializing locks or causing fragmentation via arena explosion.
The GVL is implements its own wait-queue as a ccan/list to
permit controlling wakeup order. Timeslice under contention is
handled by a designated timer thread (similar to choosing a
"patrol_thread" for current deadlock checking).
There is only one self-pipe, now, as wakeups for timeslice are
done independently using condition variables. This reduces FD
pressure slightly.
Signal handling is handled directly by a Ruby Thread (instead
of timer-thread) by exposing signal self-pipe to callers of
rb_thread_fd_select, native_sleep, rb_wait_for_single_fd, etc...
Acquiring, using, and releasing the self-pipe is exposed via 4
new internal functions:
1) rb_sigwait_fd_get - exclusively acquire timer_thread_pipe.normal[0]
2) rb_sigwait_fd_sleep - sleep and wait for signal (and no other FDs)
3) rb_sigwait_fd_put - release acquired result from rb_sigwait_fd_get
4) rb_sigwait_fd_migrate - migrate signal handling to another thread
after calling rb_sigwait_fd_put.
rb_sigwait_fd_migrate is necessary for waitpid callers because
only one thread can wait on self-pipe at a time, otherwise a
deadlock will occur if threads fight over the self-pipe.
TRAP_INTERRUPT_MASK is now set for the main thread directly in
signal handler via rb_thread_wakeup_timer_thread.
Originally, I wanted to use POSIX timers
(timer_create/timer_settime) for this. Unfortunately, this
proved unfeasible as Mutex#sleep resumes on spurious wakeups and
test/thread/test_cv.rb::test_condvar_timed_wait failed. Using
pthread_sigmask to mask out SIGVTALRM fixed that test, but
test/fiddle/test_function.rb::test_nogvl_poll proved there'd be
some unavoidable (and frequent) incompatibilities from that
approach.
Finally, this allows us to drop thread_destruct_lock and
interrupt current ec directly.
We don't need to rely on vm->thread_destruct_lock or a coherent
vm->running_thread on any platform. Separate timer-thread for
time slice and signal handling is relegated to thread_win32.c,
now.
[ruby-core:88088] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64107 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
* process.c (before_fork_ruby, after_fork_ruby): used only if fork()
or daemon() is available.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64065 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64063 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
http://ci.rvm.jp/results/trunk-asserts-nopara@silicon-docker/1149270
```
/home/ko1/ruby/src/trunk-asserts-nopara/process.c: In function 'assert_close_on_exec':
/home/ko1/ruby/src/trunk-asserts-nopara/process.c:298:9: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result]
(void)write(2, m, sizeof(m) - 1);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
workaround from https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66425#c34
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64029 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
* include/ruby/ruby.h (UNREACHABLE_RETURN): UNREACHABLE at the end
of non-void functions.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64025 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63971 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
Not much we can do if writing to STDERR_FILENO fails.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63963 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
There's actually no need to close the pipes used by the
sleepy timer thread before forking, only to stop the timer
thread itself.
Instead, we only close the parent pipes in the child process,
either via close-on-exec flag or when reinitializing the timer
thread.
This change will be necessary when we allow
rb_wait_for_single_fd and rb_thread_fd_select to wait on the
timer_thread_pipe.normal[0] directly and eliminate timer thread.
I don't anticipate compatibility problems with this change
alone.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63960 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
We need to preserve "IGNORE" behavior from Ruby 2.5 and earlier.
We can't rely on SA_NOCLDWAIT any more, since we always need
system() and MJIT to work; so we fake that behavior using
dedicated reaper (currently in timer-thread).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63879 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
Non-parallel "make test-spec" caused
spec/ruby/core/process/wait2_spec.rb failures because mspec
uses "exec" in single-process mode, so there's no chance
the post-exec state could know about the MJIT child process
from its pre-exec state.
[ruby-core:87846] [Bug #14867]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63877 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
Reading win32/win32.c waitpid implementation, maybe waitpid(-1, ...)
on that platform will never conflict with mjit use of waitpid.
In any case, I've added WAITPID_USE_SIGCHLD macro to vm_core.h
so it can be easy for Linux/BSD users to test (hopefully!)
win32-compatible code.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63855 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
The change is unstable on Windows. Please re-commit it when it correctly
supports Windows.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63852 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
For systems with lossy SIGCHLD, an infinitely sleeping timer
thread needs to be aware of rb_waitpid callers in the first
place before it can check and reset polling status.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63829 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
Basically in win32, mjit.c seems to work directly on spawnvp
result while normal Ruby code wraps process handles to look
like *nix PIDs. I'm only guessing, here...
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63797 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
Some systems lack SIGCHLD or have incomplete SIGCHLD
implementations. So enable polling mode for them.
[ruby-core:87705] [Bug #14867]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63795 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
It's a bit redundant, but we optimize for platforms with
SIGCHLD, not without.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63790 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
We may be interrupted by another thread after setting ubf,
but before we re-acquire interrupt_lock again to sleep on
w->cond.
This should fix test_wait_interrupt in test/ruby/test_process.rb
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63789 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
I expect this to fix [Bug #14873]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63785 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63770 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
It is possible to have both MJIT and normal child processes
alive, so we cannot set ECHILD based on such a guess. We can
still elide waitpid(PID <= 0) calls if we have callers in
vm->waiting_pids, however.
For specs, ensure Process.waitall does not leak MJIT
PIDs to Rubyspace.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63764 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|