BBC: [WIN32] Blead breaks Error-0.17030 #23022

sisyphus · 2025-02-23T01:22:02Z

UPDATE: Version 0.17030 is a recent release, but the same issue arises with the previous version (0.17029).

Breakage begins with perl-5.41.3, and also afflicts perl-5.40.1.
The problem exists all the way through to current blead (commit 3df6082).

Test Summary Report
-------------------
t/08warndie.t                      (Wstat: 1280 (exited 5) Tests: 1 Failed: 0)
  Non-zero exit status: 5
  Parse errors: Bad plan.  You planned 21 tests but ran 1.
Files=15, Tests=43,  2 wallclock secs ( 0.05 usr +  0.00 sys =  0.05 CPU)
Result: FAIL
Failed 1/15 test programs. 0/43 subtests failed.
make: *** [makefile:886: test_dynamic] Error 255

The crash occurs after the 3rd iteration of the following while{} loop (located at line 46 of t/08warndie.t ):

    while( defined ( $_ = <$childh> ) ) { # Crashes here
        $s .= $_;
    }

The assignment$s .= $_ is attempted only 3 times - and those attempts are successful.
Yet, we never get to successfully exit the loop.

Perl -V output:

Summary of my perl5 (revision 5 version 41 subversion 3) configuration:

  Platform:
    osname=MSWin32
    osvers=10.0.22631.4037
    archname=MSWin32-x64-multi-thread
    uname=''
    config_args='undef'
    hint=recommended
    useposix=true
    d_sigaction=undef
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='gcc'
    ccflags ='-std=c99 -DWIN32 -DWIN64 -fdiagnostics-color=never -DPERL_TEXTMODE_SCRIPTS -DMULTIPLICITY -DPERL_IMPLICIT_SYS -DUSE_PERLIO -D__USE_MINGW_ANSI_STDIO -fwrapv -fno-strict-aliasing -mms-bitfields'
    optimize='-Os -falign-functions -falign-jumps -falign-labels -falign-loops -freorder-blocks -freorder-blocks-algorithm=stc -freorder-blocks-and-partition'
    cppflags='-DWIN32'
    ccversion=''
    gccversion='14.2.0'
    gccosandvers=''
    intsize=4
    longsize=4
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='long long'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='g++'
    ldflags ='-s -L"D:\perl-5.41.3-1420-ucrt-mcf\lib\MSWin32-x64-multi-thread\CORE" -L"C:\winlibs-ucrt-1420\mingw64\lib" -L"C:\winlibs-ucrt-1420\mingw64\x86_64-w64-mingw32\lib" -L"C:\winlibs-ucrt-1420\mingw64\lib\gcc\x86_64-w64-mingw32\14.2.0"'
    libpth=C:\winlibs-ucrt-1420\mingw64\lib C:\winlibs-ucrt-1420\mingw64\x86_64-w64-mingw32\lib C:\winlibs-ucrt-1420\mingw64\lib\gcc\x86_64-w64-mingw32\14.2.0 D:\_64\msys_1420\1.0\local\lib
    libs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    perllibs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    libc=-lucrt
    so=dll
    useshrplib=true
    libperl=libperl541.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs
    dlext=dll
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags='-shared -s -L"D:\perl-5.41.3-1420-ucrt-mcf\lib\MSWin32-x64-multi-thread\CORE" -L"C:\winlibs-ucrt-1420\mingw64\lib" -L"C:\winlibs-ucrt-1420\mingw64\x86_64-w64-mingw32\lib" -L"C:\winlibs-ucrt-1420\mingw64\lib\gcc\x86_64-w64-mingw32\14.2.0"'


Characteristics of this binary (from libperl):
  Compile-time options:
    HAS_LONG_DOUBLE
    HAS_TIMES
    HAVE_INTERP_INTERN
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_HASH_FUNC_SIPHASH13
    PERL_HASH_USE_SBOX32
    PERL_IMPLICIT_SYS
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_SAFE_PUTENV
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
    USE_THREAD_SAFE_LOCALE
  Built under MSWin32
  Compiled at Aug 30 2024 10:58:33
  @INC:
    D:/perl-5.41.3-1420-ucrt-mcf/site/lib/MSWin32-x64-multi-thread
    D:/perl-5.41.3-1420-ucrt-mcf/site/lib
    D:/perl-5.41.3-1420-ucrt-mcf/lib/MSWin32-x64-multi-thread
    D:/perl-5.41.3-1420-ucrt-mcf/lib

The text was updated successfully, but these errors were encountered:

jkeenan · 2025-02-23T11:23:33Z

@sisyphus, would you be able to run this program (from the root directory of a git checkout of Perl 5), suitably adapted for Win32?

perl Porting/bisect.pl \
--start=v5.40.0 \
--end=v5.41.3 \
--module=Error

twata1 · 2025-02-24T00:15:38Z

That test (t/08warndie.t) passes in perl 5.41.1 and 5.41.2 built using visual studio.

tonycoz · 2025-02-24T01:14:23Z

I did see

panic: free 2e7f3bf5cb8 from wrong pool, 2e7f18aab50!=2e7f3bbd070

so there's definitely a problem, debugging further.

tonycoz · 2025-02-24T03:46:27Z

Looks like this is caused by my fix to prevent double-frees of the RExC state interacting with Win32 fork emulation, backtrace:

Thread 8 hit Breakpoint 1, Perl_safesysfree (where=0x81a898) at ..\util.c:393
393                     Perl_croak_nocontext("panic: free %p from wrong pool, %p!=%p",
(gdb) bt
#0  Perl_safesysfree (where=0x81a898) at ..\util.c:393
#1  0x00007ff987e3b90e in release_RExC_state (my_perl=<optimized out>, vstate=0x81a898) at ..\regcomp.c:1374
#2  0x00007ff988026c0e in Perl_leave_scope (my_perl=my_perl@entry=0x35f5900, base=4) at ..\scope.c:1537
#3  0x00007ff987f46bd2 in Perl_dounwind (my_perl=my_perl@entry=0x35f5900, cxix=cxix@entry=-1) at ..\pp_ctl.c:1851
#4  0x00007ff987f97b0f in S_my_exit_jump (my_perl=my_perl@entry=0x35f5900)
    at C:/Users/Tony/dev/perl/git/perl/win32/perl.c:5523
#5  0x00007ff987f9ff3c in Perl_my_exit (my_perl=my_perl@entry=0x35f5900, status=<optimized out>)
    at C:/Users/Tony/dev/perl/git/perl/win32/perl.c:5403
#6  0x00007ff987f4b03a in Perl_pp_exit (my_perl=0x35f5900) at ..\pp_ctl.c:3720
#7  0x00007ff987faf98d in Perl_runops_debug (my_perl=0x35f5900) at ..\dump.c:3000
#8  0x00007ff988040868 in win32_start_child (arg=0x35f5900) at C:/Users/Tony/dev/perl/git/perl/win32/perlhost.h:1907
#9  0x00007ffa09a57374 in KERNEL32!BaseThreadInitThunk () from C:\WINDOWS\System32\kernel32.dll
#10 0x00007ffa0ad9cc91 in ntdll!RtlUserThreadStart () from C:\WINDOWS\SYSTEM32\ntdll.dll
#11 0x0000000000000000 in ?? ()

What I think is happening is:

there's some runtime compilation of a regexp, which allocates a RExC_state_t and pushes a SAVEDESTRUCTOR_X() onto the scope stack to clean it up
fork is called while the clean up is still on the scope stack, the scope stack duplication code in sv.c just duplicates the entry, since it can't distinguish between different SAVEDESTRUCTOR_X() cleanups.
the child pseudo-process exits, unwinds the scope stack which calls release_RExc_state() which tries to Safefree() the RExC_state from the wrong thread

Some of the other clean up done in release_RExC_state might be duplicated between the two threads too, possibly causing the strange while loop behaviour.

Working on a fix (giving RExC_state_t cleanup its own save type)

Fixes Perl#23022

bulk88 · 2025-02-25T17:31:04Z

Tonyc consider a clone vtable CB if your designing a new api, I never looked too deep at it, but the new "save [stack] onto mortal stack" API, which runs a user C CB with either a RCed SV*, or runs a user C CB with a size_t/void *. # 1 has easy behaviour what to do on a fork/clone. but what on earth is # 2 during a fork/clone? What is the opaque user supplied size_t var contents really mean? use-after-free/double RC++ mem leak/segv is almost guaranteed in the OS proc.

Fixes Perl#23022

Previous on Win32 this could cause a double-free of the RExC state if an emulated fork was done with the free of the state on the scope stack. Use a custom save type and prevent freeing in the cloned process to prevent the double-free. Fixes Perl#23022

tonycoz · 2025-04-07T04:00:01Z

please try #23179

sisyphus · 2025-04-07T07:12:47Z

please try #23179

I won't be able to test this until I return home (next Monday).

twata1 · 2025-04-13T05:12:44Z

please try #23179

I'm building and testing it now. (It will take a while.)

twata1 · 2025-04-13T08:13:56Z

All tests of Error-0.17030 with #23179 passed.

The build log for Error-0.17030 is attached.
build.log

Perl -V output:

C:\home\vega54111a>perl -V
Summary of my perl5 (revision 5 version 41 subversion 11) configuration:
  Snapshot of: 8194f79baa64d5f7bb5c1edbcf836e6c279f9a59
  Platform:
    osname=MSWin32
    osvers=6.3.9600
    archname=MSWin32-x64-multi-thread
    uname=''
    config_args='undef'
    hint=recommended
    useposix=true
    d_sigaction=undef
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='gcc'
    ccflags ='-std=c99 -DWIN32 -DWIN64 -DPERL_TEXTMODE_SCRIPTS -DMULTIPLICITY -DPERL_IMPLICIT_SYS -DUSE_PERLIO -D__USE_MINGW_ANSI_
STDIO -fwrapv -fno-strict-aliasing -mms-bitfields'
    optimize='-Os'
    cppflags='-DWIN32'
    ccversion=''
    gccversion='14.2.0'
    gccosandvers=''
    intsize=4
    longsize=4
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='long long'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='g++'
    ldflags ='-s -L"C:\WinLibs\Perl-5.41.11-23022-pseudo-fork-rexc\lib\CORE" -L"C:\winlibs-x86_64-posix-seh-gcc-14.2.0-llvm-19.1.1
-mingw-w64msvcrt-12.0.0-r2\mingw64\lib" -L"C:\winlibs-x86_64-posix-seh-gcc-14.2.0-llvm-19.1.1-mingw-w64msvcrt-12.0.0-r2\mingw64\x8
6_64-w64-mingw32\lib" -L"C:\winlibs-x86_64-posix-seh-gcc-14.2.0-llvm-19.1.1-mingw-w64msvcrt-12.0.0-r2\mingw64\lib\gcc\x86_64-w64-m
ingw32\14.2.0"'
    libpth=C:\winlibs-x86_64-posix-seh-gcc-14.2.0-llvm-19.1.1-mingw-w64msvcrt-12.0.0-r2\mingw64\lib C:\winlibs-x86_64-posix-seh-gc
c-14.2.0-llvm-19.1.1-mingw-w64msvcrt-12.0.0-r2\mingw64\x86_64-w64-mingw32\lib C:\winlibs-x86_64-posix-seh-gcc-14.2.0-llvm-19.1.1-m
ingw-w64msvcrt-12.0.0-r2\mingw64\lib\gcc\x86_64-w64-mingw32\14.2.0
    libs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -
lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    perllibs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luu
id -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    libc=-lmsvcrt
    so=dll
    useshrplib=true
    libperl=libperl541.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs
    dlext=dll
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags='-shared -s -L"C:\WinLibs\Perl-5.41.11-23022-pseudo-fork-rexc\lib\CORE" -L"C:\winlibs-x86_64-posix-seh-gcc-14.2.0-ll
vm-19.1.1-mingw-w64msvcrt-12.0.0-r2\mingw64\lib" -L"C:\winlibs-x86_64-posix-seh-gcc-14.2.0-llvm-19.1.1-mingw-w64msvcrt-12.0.0-r2\m
ingw64\x86_64-w64-mingw32\lib" -L"C:\winlibs-x86_64-posix-seh-gcc-14.2.0-llvm-19.1.1-mingw-w64msvcrt-12.0.0-r2\mingw64\lib\gcc\x86
_64-w64-mingw32\14.2.0"'


Characteristics of this binary (from libperl):
  Compile-time options:
    HAS_LONG_DOUBLE
    HAS_TIMES
    HAVE_INTERP_INTERN
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_HASH_FUNC_SIPHASH13
    PERL_HASH_USE_SBOX32
    PERL_IMPLICIT_SYS
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_SAFE_PUTENV
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
  Built under MSWin32
  Compiled at Apr 13 2025 13:41:56
  %ENV:
    PERL5LIB="C:\home\vega54111a\perl5\lib\perl5"
    PERL_LOCAL_LIB_ROOT="C:\home\vega54111a\perl5"
    PERL_MB_OPT="--install_base "C:\\home\\vega54111a\\perl5""
    PERL_MM_OPT="INSTALL_BASE=C:\\home\\vega54111a\\perl5"
  @INC:
    C:\home\vega54111a\perl5\lib\perl5/MSWin32-x64-multi-thread
    C:\home\vega54111a\perl5\lib\perl5
    C:/WinLibs/Perl-5.41.11-23022-pseudo-fork-rexc/site/lib
    C:/WinLibs/Perl-5.41.11-23022-pseudo-fork-rexc/lib

C:\home\vega54111a>

Thank you,

388bf71 (ignoring Perl#23022 for now) ensured that the RExC_state and its controlled pointers were freed once and only once, regardless of whether re_op_compile() returned normally or threw an exception. Unfortunately that free could happen very later, surviving well beyond when the regexp was compiled. Add an ENTER/LEAVE pair to ensure the cleanup is done immediately on a normal return.

sisyphus · 2025-04-14T02:55:58Z

LGTM, too.

Previous on Win32 this could cause a double-free of the RExC state if an emulated fork was done with the free of the state on the scope stack. Use a custom save type and prevent freeing in the cloned process to prevent the double-free. Fixes Perl#23022

388bf71 (ignoring Perl#23022 for now) ensured that the RExC_state and its controlled pointers were freed once and only once, regardless of whether re_op_compile() returned normally or threw an exception. Unfortunately that free could happen very later, surviving well beyond when the regexp was compiled. Add an ENTER/LEAVE pair to ensure the cleanup is done immediately on a normal return.

388bf71 (ignoring #23022 for now) ensured that the RExC_state and its controlled pointers were freed once and only once, regardless of whether re_op_compile() returned normally or threw an exception. Unfortunately that free could happen very later, surviving well beyond when the regexp was compiled. Add an ENTER/LEAVE pair to ensure the cleanup is done immediately on a normal return.

sisyphus added the Needs Triage label Feb 23, 2025

sisyphus mentioned this issue Feb 23, 2025

perl 5.40.1 build: Possibly Error test t/08warndie.t will fail StrawberryPerl/Perl-Dist-Strawberry#240

Open

jkeenan added BBC Blead Breaks CPAN - changes in blead broke a cpan module(s) distro-mswin32 labels Feb 23, 2025

jkeenan mentioned this issue Feb 23, 2025

Tests failing against perl-5.41 development versions shlomif/perl-error.pm#5

Open

tonycoz added a commit to tonycoz/perl5 that referenced this issue Feb 24, 2025

regcomp: handle cloning the rexc cleanup in the scope stack

17ccfbd

Fixes Perl#23022

shawnlaffan mentioned this issue Feb 25, 2025

Strawberry perl 5.40.2.1, 5.38.4.1, 5.34.3.1 StrawberryPerl/Perl-Dist-Strawberry#233

Draft

tonycoz added a commit to tonycoz/perl5 that referenced this issue Feb 25, 2025

regcomp: handle cloning the rexc cleanup in the scope stack

68e090b

Fixes Perl#23022

jkeenan removed the Needs Triage label Feb 25, 2025

tonycoz self-assigned this Apr 7, 2025

tonycoz mentioned this issue Apr 7, 2025

regcomp: handle cloning the rexc cleanup in the scope stack #23179

Merged

mauke closed this as completed in #23179 Apr 19, 2025

mauke closed this as completed in 0c859ae Apr 19, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BBC: [WIN32] Blead breaks Error-0.17030 #23022

BBC: [WIN32] Blead breaks Error-0.17030 #23022

sisyphus commented Feb 23, 2025 •

edited

Loading

jkeenan commented Feb 23, 2025

twata1 commented Feb 24, 2025

tonycoz commented Feb 24, 2025

tonycoz commented Feb 24, 2025

bulk88 commented Feb 25, 2025

tonycoz commented Apr 7, 2025

sisyphus commented Apr 7, 2025

twata1 commented Apr 13, 2025

twata1 commented Apr 13, 2025

sisyphus commented Apr 14, 2025

BBC: [WIN32] Blead breaks Error-0.17030 #23022

BBC: [WIN32] Blead breaks Error-0.17030 #23022

Comments

sisyphus commented Feb 23, 2025 • edited Loading

jkeenan commented Feb 23, 2025

twata1 commented Feb 24, 2025

tonycoz commented Feb 24, 2025

tonycoz commented Feb 24, 2025

bulk88 commented Feb 25, 2025

tonycoz commented Apr 7, 2025

sisyphus commented Apr 7, 2025

twata1 commented Apr 13, 2025

twata1 commented Apr 13, 2025

sisyphus commented Apr 14, 2025

sisyphus commented Feb 23, 2025 •

edited

Loading