Built v5.21.12 (v5.21.11-10-ga8f582b) using the following command line: ./Configure -des -Dusedevel -DDEBUGGING -Dcc=afl-gcc -Doptimize=-O2\ -g && AFL_HARDEN=1 make -j6 test-prep Bug found with AFL (http://lcamtuf.coredump.cx/afl) GDB output: perl: sv.c:2964: Perl_sv_2pv_flags: Assertion `((svtype)((sv)->sv_flags & 0xff)) != SVt_PVAV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVHV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVFM' failed. Program received signal SIGABRT, Aborted. [----------------------------------registers-----------------------------------] RAX: 0x0 RBX: 0x7fffffffe624 --> 0x736574006c726570 ('perl') RCX: 0xffffffffffffffff RDX: 0x6 RSI: 0x126d RDI: 0x126d RBP: 0x7ffff6ea9c67 --> 0x257325732500203a (': ') RSP: 0x7fffffffde68 --> 0x7ffff6d933e0 (<*__GI_abort+384>: mov rdx,QWORD PTR fs:0x10) RIP: 0x7ffff6d90165 (<*__GI_raise+53>: cmp rax,0xfffffffffffff000) R8 : 0x7ffff7fdc700 (0x00007ffff7fdc700) R9 : 0x5653203d21202929 (')) != SV') R10: 0x8 R11: 0x202 R12: 0xf3b378 ("((svtype)((sv)->sv_flags & 0xff)) != SVt_PVAV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVHV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVFM") R13: 0xf4fab0 ("Perl_sv_2pv_flags") R14: 0x7ffff6ea9c67 --> 0x257325732500203a (': ') R15: 0xb94 EFLAGS: 0x202 (carry parity adjust zero sign trap INTERRUPT direction overflow) [-------------------------------------code-------------------------------------] 0x7ffff6d9015b <*__GI_raise+43>: movsxd rdi,eax 0x7ffff6d9015e <*__GI_raise+46>: mov eax,0xea 0x7ffff6d90163 <*__GI_raise+51>: syscall => 0x7ffff6d90165 <*__GI_raise+53>: cmp rax,0xfffffffffffff000 0x7ffff6d9016b <*__GI_raise+59>: ja 0x7ffff6d90182 <*__GI_raise+82> 0x7ffff6d9016d <*__GI_raise+61>: repz ret 0x7ffff6d9016f <*__GI_raise+63>: nop 0x7ffff6d90170 <*__GI_raise+64>: test eax,eax [------------------------------------stack-------------------------------------] 0000| 0x7fffffffde68 --> 0x7ffff6d933e0 (<*__GI_abort+384>: mov rdx,QWORD PTR fs:0x10) 0008| 0x7fffffffde70 --> 0xf3b378 ("((svtype)((sv)->sv_flags & 0xff)) != SVt_PVAV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVHV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVFM") 0016| 0x7fffffffde78 --> 0x7ffff6eabc21 --> 0x706c6568007325 ('%s') 0024| 0x7fffffffde80 --> 0x7fffffffdea0 --> 0x3000000018 0032| 0x7fffffffde88 --> 0xb94 0040| 0x7fffffffde90 --> 0x7fffffffdf90 --> 0x7fffffffe624 --> 0x736574006c726570 ('perl') 0048| 0x7fffffffde98 --> 0x7ffff6dc41b6 (<__fxprintf+310>: lea rsp,[rbp-0x20]) 0056| 0x7fffffffdea0 --> 0x3000000018 [------------------------------------------------------------------------------] Legend: code, data, rodata, value Stopped reason: SIGABRT 0x00007ffff6d90165 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. System Info: Debian 7, Kernel 3.2.65-1+deb7u2 x86_64, GCC 4.9.2, libc 2.13-38+deb7u8

I minimized the attached test case with afl-tmin from 469 bytes to 50 bytes, however, the minimized test case didn't cause the assertion failure and caused GDB and Valgrind to hang. Reduced test case: map{s[][]o=~@0,$0[0]>map{0?{s()()}:0}0}<>__END__ 0 Hexdump: 0000000 616d 7b70 5b73 5b5d 6f5d 7e3d 3040 242c 0000010 5b30 5d30 6d3e 7061 307b 7b3f 2873 2829 0000020 7d29 303a 307d 3c7d 5f3e 455f 444e 5f5f 0000030 300a 0000032

On Thu, Apr 23, 2015 at 01:57:08PM -0700, Brian Carpenter via RT wrote: > I minimized the attached test case with afl-tmin from 469 bytes to 50 > bytes, however, the minimized test case didn't cause the assertion > failure and caused GDB and Valgrind to hang. I can't reproduce the failure with the original code (no assertion failure; valgrind and ASan ok). > Reduced test case: > map{s[][]o=~@0,$0[0]>map{0?{s()()}:0}0}<>__END__ > 0 That hangs because the <> is reading from stdin. -- All wight. I will give you one more chance. This time, I want to hear no Wubens. No Weginalds. No Wudolf the wed-nosed weindeers. -- Life of Brian

On Wed Apr 29 04:14:52 2015, davem wrote: > I can't reproduce the failure with the original code (no assertion > failure; valgrind and ASan ok). I can, with a plain build of blead. Most of the original code is red herrings, I was able to reduce it to this: % ./miniperl -e '@a = qw{x y}; for (@a) { s{.}{}go =~ @b }' miniperl: sv.c:2964: Perl_sv_2pv_flags: Assertion `((svtype)((sv)->sv_flags & 0xff)) != SVt_PVAV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVHV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVFM' failed. Aborted (core dumped) % Are you able to reproduce it that way? If not I may be able to make some time to dig more over the weekend. Hugo

On Wed Apr 29 12:07:25 2015, hv wrote: > On Wed Apr 29 04:14:52 2015, davem wrote: > > I can't reproduce the failure with the original code (no assertion > > failure; valgrind and ASan ok). > > I can, with a plain build of blead. Most of the original code is red > herrings, I was able to reduce it to this: > > % ./miniperl -e '@a = qw{x y}; for (@a) { s{.}{}go =~ @b }' > miniperl: sv.c:2964: Perl_sv_2pv_flags: Assertion `((svtype)((sv)- > >sv_flags & 0xff)) != SVt_PVAV && ((svtype)((sv)->sv_flags & 0xff)) != > SVt_PVHV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVFM' failed. > Aborted (core dumped) > % > > Are you able to reproduce it that way? If not I may be able to make > some time to dig more over the weekend. The problem seems to be this code in pp_regcomp: /* can't change the optree at runtime either */ /* PMf_KEEP is handled differently under threads to avoid these problems */ if (!RX_PRELEN(PM_GETRE(pm)) && PL_curpm) pm = PL_curpm; if (pm->op_pmflags & PMf_KEEP) { pm->op_private &= ~OPpRUNTIME; /* no point compiling again */ cLOGOP->op_first->op_next = PL_op->op_next; } before this starts pm point at the match op and PL_curpm points at the subst op. RX_PRELEN() is zero for rx, so pm is updated, and since the subst has the /o flag (PMf_KEEP) the op tree is rewritten to skip the regcomp for the second time around the loop, leaving the raw AV on the stack as the regexp for pp_match. Hugo's example doesn't need the /g flag: $ ./miniperl -e '@a = qw{x y}; for (@a) { s{.}{}o =~ @b }' miniperl: sv.c:2964: Perl_sv_2pv_flags: Assertion `((svtype)((sv)->sv_flags & 0xff)) != SVt_PVAV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVHV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVFM' failed. Aborted Tested on a non-threaded perl. Tony

On Wed, Apr 29, 2015 at 10:22:00PM -0700, Tony Cook via RT wrote: > On Wed Apr 29 12:07:25 2015, hv wrote: > > On Wed Apr 29 04:14:52 2015, davem wrote: > > > I can't reproduce the failure with the original code (no assertion > > > failure; valgrind and ASan ok). > > > > I can, with a plain build of blead. Most of the original code is red > > herrings, I was able to reduce it to this: > > > > % ./miniperl -e '@a = qw{x y}; for (@a) { s{.}{}go =~ @b }' > > miniperl: sv.c:2964: Perl_sv_2pv_flags: Assertion `((svtype)((sv)- > > >sv_flags & 0xff)) != SVt_PVAV && ((svtype)((sv)->sv_flags & 0xff)) != > > SVt_PVHV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVFM' failed. > > Aborted (core dumped) > > % > > > > Are you able to reproduce it that way? If not I may be able to make > > some time to dig more over the weekend. > > The problem seems to be this code in pp_regcomp: > > /* can't change the optree at runtime either */ > /* PMf_KEEP is handled differently under threads to avoid these problems */ > if (!RX_PRELEN(PM_GETRE(pm)) && PL_curpm) > pm = PL_curpm; > if (pm->op_pmflags & PMf_KEEP) { > pm->op_private &= ~OPpRUNTIME; /* no point compiling again */ > cLOGOP->op_first->op_next = PL_op->op_next; > } > > before this starts pm point at the match op and PL_curpm points at the subst op. > > RX_PRELEN() is zero for rx, so pm is updated, and since the subst has > the /o flag (PMf_KEEP) the op tree is rewritten to skip the regcomp for > the second time around the loop, leaving the raw AV on the stack as the > regexp for pp_match. This bug has in fact been present since 5.0 or before; it's just that a) a change in 5.18.0 made "$_ =~ @a", where @a is empty, match "@a" (i.e. "") rather than the string "0"; and b) an assertion added in 5.19.x made getting the string value of an AV an error. So I don't think that it needs to be a 5.22 blocker any more. The main issue is the mis-implementation and interaction between two regexp "features"; first that the 'o' flag in /$expr/o means that the pattern is only compiled once, and second that in /$expr/, if $expr evaluates to the empty string, then the last successful regex is used instead. It is documented in perlop that: In this case, only the C<g> and C<c> flags on the empty pattern are honored; the other flags are taken from the original pattern So in something like the following: /./o; $_ =~ @b; if @b is empty, then (since 5.18.0), it evaluates to the empty string, so the null pattern rule kicks in, and the /./ pattern is used instead. Then the "other flags are taken from the original pattern" rule kicks in, so the //o flag from the earlier successful match is honoured, so the optree of the current match ($_ =~ @b), i.e. the one *without* the /o flag, is manipulated to make it skip future compiles. Since that optree wasn't compiled with //o present, it doesn't have the extra regcmaybe op in the tree, which the code blindly assumes is the first child of the regcomp op, and so happily corrupts that tree. This is fixable, but first we need to agree on what the semantics should be, in particular, in the following: for (...) { /abc/o; /$expr/; } on executing /$expr/ if $expr evaluates to the null string, should it honour the //o of the previous successful match, and so skip evaluating $expr on subsequent iterations? This would be hard to implement. Conversely, should /$expr/o honour the //o flag even if $expr is null? My opinion is that, like c and g, the o flag should be honoured on the match op it appears on, not on the last successful match op, and we fix the docs (as well as the code). So I propose that: /abc/o; $foo =~ /$null/ matches $foo against "abc" each time round the loop (it currently crashes on the second iteration); and that: /abc/; $foo =~ /$null/o always only evaluates $null once; if that happens to be the empty string, then subsequent iterations will see the empty string each time and so each time use the last successful match, which may be /abc/ (it currently ignores the //o on the first iteration if $null is empty). Note that nothing in the test suite currently tests these cases as far as I can tell. -- "But Sidley Park is already a picture, and a most amiable picture too. The slopes are green and gentle. The trees are companionably grouped at intervals that show them to advantage. The rill is a serpentine ribbon unwound from the lake peaceably contained by meadows on which the right amount of sheep are tastefully arranged." -- Lady Croom, "Arcadia"

On Fri May 08 13:58:07 2015, davem wrote: > So I propose that: > > /abc/o; $foo =~ /$null/ > > matches $foo against "abc" each time round the loop (it currently > crashes > on the second iteration); and that: > > /abc/; $foo =~ /$null/o > > always only evaluates $null once; if that happens to be the empty > string, > then subsequent iterations will see the empty string each time and so > each > time use the last successful match, which may be /abc/ (it currently > ignores the //o on the first iteration if $null is empty). That makes sense to me. Or we could take /$null/ meaning 'use the last match' out the back and bury it. Tony

On Fri May 08 13:58:07 2015, davem wrote: > This bug has in fact been present since 5.0 or before; it's just that > a) a change in 5.18.0 made "$_ =~ @a", where @a is empty, match "@a" > (i.e. > "") rather than the string "0"; and b) an assertion added in 5.19.x > made > getting the string value of an AV an error. So I don't think that it > needs > to be a 5.22 blocker any more. I think I may be missing something here. It sounds like it's legal to say ($x =~ @a) to mean ($x =~ "@a") but that this violates an assertion and so crashes debugging builds. Is this only a non-blocker because it was already broken in 5.20, or for some other reason? My initial reaction is: "won't people running debugging builds be irritated that they can't debug code that is legal otherwise?" As I say, I might be missing something. I don't use debugging builds almost ever. > [...] > My opinion is that, like c and g, the o flag should be honoured on the > match op it appears on, not on the last successful match op, and we > fix > the docs (as well as the code). This seems fine to me as a v5.23 change. -- rjbs

On Mon, May 11, 2015 at 08:20:17PM -0700, Ricardo SIGNES via RT wrote: > On Fri May 08 13:58:07 2015, davem wrote: > > This bug has in fact been present since 5.0 or before; it's just that > > a) a change in 5.18.0 made "$_ =~ @a", where @a is empty, match "@a" > > (i.e. > > "") rather than the string "0"; and b) an assertion added in 5.19.x > > made > > getting the string value of an AV an error. So I don't think that it > > needs > > to be a 5.22 blocker any more. > > I think I may be missing something here. > > It sounds like it's legal to say ($x =~ @a) to mean ($x =~ "@a") but > that this violates an assertion and so crashes debugging builds. Is > this only a non-blocker because it was already broken in 5.20, or for > some other reason? No, $x =~ @a is perfectly legal and non-problematic (except that its semantics changed in 5.18.0, probably inadvertently), both on straight and debugging builds. The issue is that on unthreaded builds going back to 5.000, in something like for (....) { "a" =~ /./o; $foo =~ expr; } where expr evaluates (at least once) to the null string, the optree for expr will become corrupted, which may affect future iterations of the loop in unexpected ways. It just so happens that a new debugging assertion added before 5.20 now detects one of those unexpected things, whereas before it just silently failed. Having expr be an empty array is just a good way to trigger this case. This is why I don't think it should be a 5.22 blocker. -- I thought I was wrong once, but I was mistaken.

* Dave Mitchell <[email protected]> [2015-05-12T06:22:22] > No, $x =~ @a is perfectly legal and non-problematic (except that its > semantics changed in 5.18.0, probably inadvertently), both on straight and > debugging builds. > > The issue is that on unthreaded builds going back to 5.000, in something > like > [...] > This is why I don't think it should be a 5.22 blocker. Got it. Agreed, thanks. -- rjbs

* Tony Cook via RT <[email protected]> [2015-05-12 02:30]: > On Fri May 08 13:58:07 2015, davem wrote: > > So I propose that: > > > > /abc/o; $foo =~ /$null/ > > > > matches $foo against "abc" each time round the loop (it currently > > crashes > > on the second iteration); and that: > > > > /abc/; $foo =~ /$null/o > > > > always only evaluates $null once; if that happens to be the empty > > string, > > then subsequent iterations will see the empty string each time and so > > each > > time use the last successful match, which may be /abc/ (it currently > > ignores the //o on the first iteration if $null is empty). > > That makes sense to me. Me too. -- Aristotle Pagaltzis // <http://plasmasturm.org/>

perl -e 'sub C0000(){0}C0000 C0000' causes this assertion failure in v5.23.7 (v5.23.6-119-gbc5be89).

On Mon, May 11, 2015 at 05:29:05PM -0700, Tony Cook via RT wrote: > Or we could take /$null/ meaning 'use the last match' out the back and > bury it. How about we make a distinction between literal //, and /$null/? Then make only the latter issue a run-time deprecation warning if $null evaluates to the empty string? -- Red sky at night - gerroff my land! Red sky at morning - gerroff my land! -- old farmers' sayings #14

* Dave Mitchell <[email protected]> [2016-01-19T10:57:17] > On Mon, May 11, 2015 at 05:29:05PM -0700, Tony Cook via RT wrote: > > Or we could take /$null/ meaning 'use the last match' out the back and > > bury it. > > How about we make a distinction between literal //, and /$null/? Then make > only the latter issue a run-time deprecation warning if $null evaluates to > the empty string? So, I bet this was heavily debated back in the year 2011, for #96230... but I was wrong! (That's when qr// stopped acting like m//.) Then I guessed it was debated during the discussion of changing "split / /". Note that with split, the fix was to make "split /$onespace/" act like "split / /". Again, wrong! In review, though, I think this won't end up being worth it. Dave's suggestion (at https://rt.perl.org/Ticket/Display.html?id=124368#txn-1348595 ) seems sensible, and there's no pressing /need/ to break this behavior, even if it has occasionally seemed more trouble than it's worth. -- rjbs

Triggered in Perl v5.25.5-8-g3c42ae1 with the following: ./perl -e '$0=s0::;chop@0=~y:::' sv.c:2928: char *Perl_sv_2pv_flags(SV *const, STRLEN *const, const I32): Assertion `((svtype)((sv)->sv_flags & 0xff)) != SVt_PVAV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVHV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVFM' failed. Aborted

On Thu, 23 Apr 2015 09:31:36 GMT, [email protected] wrote: > Built v5.21.12 (v5.21.11-10-ga8f582b) using the following command > line: > > ./Configure -des -Dusedevel -DDEBUGGING -Dcc=afl-gcc -Doptimize=-O2\ > -g && AFL_HARDEN=1 make -j6 test-prep > > Bug found with AFL (http://lcamtuf.coredump.cx/afl) This link is now giving a 404. Can someone provide a better link so that we can learn about "AFL"? Thanks. > > GDB output: > perl: sv.c:2964: Perl_sv_2pv_flags: Assertion `((svtype)((sv)- > >sv_flags & 0xff)) != SVt_PVAV && ((svtype)((sv)->sv_flags & 0xff)) != > SVt_PVHV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVFM' failed. > > Program received signal SIGABRT, Aborted. > [---------------------------------- > registers-----------------------------------] > RAX: 0x0 > RBX: 0x7fffffffe624 --> 0x736574006c726570 ('perl') > RCX: 0xffffffffffffffff > RDX: 0x6 > RSI: 0x126d > RDI: 0x126d > RBP: 0x7ffff6ea9c67 --> 0x257325732500203a (': ') > RSP: 0x7fffffffde68 --> 0x7ffff6d933e0 (<*__GI_abort+384>: mov > rdx,QWORD PTR fs:0x10) > RIP: 0x7ffff6d90165 (<*__GI_raise+53>: cmp rax,0xfffffffffffff000) > R8 : 0x7ffff7fdc700 (0x00007ffff7fdc700) > R9 : 0x5653203d21202929 (')) != SV') > R10: 0x8 > R11: 0x202 > R12: 0xf3b378 ("((svtype)((sv)->sv_flags & 0xff)) != SVt_PVAV && > ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVHV && ((svtype)((sv)- > >sv_flags & 0xff)) != SVt_PVFM") > R13: 0xf4fab0 ("Perl_sv_2pv_flags") > R14: 0x7ffff6ea9c67 --> 0x257325732500203a (': ') > R15: 0xb94 > EFLAGS: 0x202 (carry parity adjust zero sign trap INTERRUPT direction > overflow) > [------------------------------------- > code-------------------------------------] > 0x7ffff6d9015b <*__GI_raise+43>: movsxd rdi,eax > 0x7ffff6d9015e <*__GI_raise+46>: mov eax,0xea > 0x7ffff6d90163 <*__GI_raise+51>: syscall > => 0x7ffff6d90165 <*__GI_raise+53>: cmp rax,0xfffffffffffff000 > 0x7ffff6d9016b <*__GI_raise+59>: ja 0x7ffff6d90182 > <*__GI_raise+82> > 0x7ffff6d9016d <*__GI_raise+61>: repz ret > 0x7ffff6d9016f <*__GI_raise+63>: nop > 0x7ffff6d90170 <*__GI_raise+64>: test eax,eax > [------------------------------------ > stack-------------------------------------] > 0000| 0x7fffffffde68 --> 0x7ffff6d933e0 (<*__GI_abort+384>: mov > rdx,QWORD PTR fs:0x10) > 0008| 0x7fffffffde70 --> 0xf3b378 ("((svtype)((sv)->sv_flags & 0xff)) > != SVt_PVAV && ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVHV && > ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVFM") > 0016| 0x7fffffffde78 --> 0x7ffff6eabc21 --> 0x706c6568007325 ('%s') > 0024| 0x7fffffffde80 --> 0x7fffffffdea0 --> 0x3000000018 > 0032| 0x7fffffffde88 --> 0xb94 > 0040| 0x7fffffffde90 --> 0x7fffffffdf90 --> 0x7fffffffe624 --> > 0x736574006c726570 ('perl') > 0048| 0x7fffffffde98 --> 0x7ffff6dc41b6 (<__fxprintf+310>: lea > rsp,[rbp-0x20]) > 0056| 0x7fffffffdea0 --> 0x3000000018 > [------------------------------------------------------------------------------ > ] > Legend: code, data, rodata, value > Stopped reason: SIGABRT > 0x00007ffff6d90165 in *__GI_raise (sig=<optimized out>) > at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 > 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or > directory. > > System Info: Debian 7, Kernel 3.2.65-1+deb7u2 x86_64, GCC 4.9.2, libc > 2.13-38+deb7u8 -- James E Keenan ([email protected])

I just checked http://lcamtuf.coredump.cx/afl and it loads just fine with no errors.

On Mon, 21 Nov 2016 23:14:13 GMT, [email protected] wrote: > I just checked http://lcamtuf.coredump.cx/afl and it loads just fine with > no errors. Yes, you are correct. For some reason in my browser the link was picking up the trailing ')', which resulted in the 404. Thank you very much. -- James E Keenan ([email protected])

On Sun, 25 Sep 2016 15:01:37 -0700, [email protected] wrote: > Triggered in Perl v5.25.5-8-g3c42ae1 with the following: > > ./perl -e '$0=s0::;chop@0=~y:::' > sv.c:2928: char *Perl_sv_2pv_flags(SV *const, STRLEN *const, const > I32): Assertion `((svtype)((sv)->sv_flags & 0xff)) != SVt_PVAV && > ((svtype)((sv)->sv_flags & 0xff)) != SVt_PVHV && ((svtype)((sv)- > >sv_flags & 0xff)) != SVt_PVFM' failed. > Aborted This is an unrelated issue. I've created #130198 to track it, with some analysis. Hugo

On Fri, 08 May 2015 13:58:07 -0700, davem wrote: > This is fixable, but first we need to agree on what the semantics > should > be, in particular, in the following: > > for (...) { > /abc/o; > /$expr/; > } > > on executing /$expr/ if $expr evaluates to the null string, should it > honour the //o of the previous successful match, and so skip > evaluating > $expr on subsequent iterations? This would be hard to implement. > > Conversely, should /$expr/o honour the //o flag even if $expr is null? > > My opinion is that, like c and g, the o flag should be honoured on the > match op it appears on, not on the last successful match op, and we > fix > the docs (as well as the code). > > So I propose that: > > /abc/o; $foo =~ /$null/ > > matches $foo against "abc" each time round the loop (it currently > crashes > on the second iteration); and that: It wasn't clear to me from your post, but this actually behaves differently between threaded and non-threaded builds - for threaded builds it behaves as you describe above - /o is honoured on the op with /o and isn't inherited by and empty match without the /o. > > /abc/; $foo =~ /$null/o > > always only evaluates $null once; if that happens to be the empty > string, > then subsequent iterations will see the empty string each time and so > each > time use the last successful match, which may be /abc/ (it currently > ignores the //o on the first iteration if $null is empty). > > Note that nothing in the test suite currently tests these cases as far > as > I can tell. The attached covers both these cases, I think. Tony

From 154ad615384c3333a75c9915621cac9ddcbd49aa Mon Sep 17 00:00:00 2001 From: Tony Cook <[email protected]> Date: Tue, 4 Jul 2017 11:44:06 +1000 Subject: (perl #124368) make /foo/o; /$null/ act consistently Previously the /o would be inherited by the second match if the first match was successful, but only on non-threaded builds. The op-tree rewriting done on non-threaded builds could also confuse the interpreter, possibly resulting in the match op receiving the argument intended for the regcomp op. --- pp_ctl.c | 10 ++-------- t/re/pat.t | 17 ++++++++++++++++- 2 files changed, 18 insertions(+), 9 deletions(-) diff --git a/pp_ctl.c b/pp_ctl.c index db5eabe..d903a4c 100644 --- a/pp_ctl.c +++ b/pp_ctl.c @@ -163,15 +163,9 @@ PP(pp_regcomp) /* handle the empty pattern */ if (!RX_PRELEN(PM_GETRE(pm)) && PL_curpm) { if (PL_curpm == PL_reg_curpm) { - if (PL_curpm_under) { - if (PL_curpm_under == PL_reg_curpm) { - Perl_croak(aTHX_ "Infinite recursion via empty pattern"); - } else { - pm = PL_curpm_under; - } + if (PL_curpm_under && PL_curpm_under == PL_reg_curpm) { + Perl_croak(aTHX_ "Infinite recursion via empty pattern"); } - } else { - pm = PL_curpm; } } diff --git a/t/re/pat.t b/t/re/pat.t index fb6d4c4..cf97ecd 100644 --- a/t/re/pat.t +++ b/t/re/pat.t @@ -23,7 +23,7 @@ BEGIN { skip_all('no re module') unless defined &DynaLoader::boot_DynaLoader; skip_all_without_unicode_tables(); -plan tests => 837; # Update this when adding/deleting tests. +plan tests => 843; # Update this when adding/deleting tests. run_tests() unless caller; @@ -138,6 +138,21 @@ sub run_tests { $null = ""; $xyz =~ /$null/; is($&, $xyz, $message); + + # each entry: regexp, match string, $&, //o match success + my @tests = + ( + [ "", "xy", "x", 1 ], + [ "y", "yz", "y", !1 ], + ); + for my $test (@tests) { + my ($re, $str, $matched, $omatch) = @$test; + $xyz =~ /x/o; + ok($str =~ /$re/, "$str matches /$re/"); + is($&, $matched, "on $matched"); + $xyz =~ /x/o; + is($str =~ /$re/o, $omatch, "$str matches /$re/o (or not)"); + } } { -- 2.1.4

On Mon, 03 Jul 2017 19:04:51 -0700, tonyc wrote: > On Fri, 08 May 2015 13:58:07 -0700, davem wrote: > > This is fixable, but first we need to agree on what the semantics > > should > > be, in particular, in the following: > > > > for (...) { > > /abc/o; > > /$expr/; > > } > > > > on executing /$expr/ if $expr evaluates to the null string, should it > > honour the //o of the previous successful match, and so skip > > evaluating > > $expr on subsequent iterations? This would be hard to implement. > > > > Conversely, should /$expr/o honour the //o flag even if $expr is > > null? > > > > My opinion is that, like c and g, the o flag should be honoured on > > the > > match op it appears on, not on the last successful match op, and we > > fix > > the docs (as well as the code). > > > > So I propose that: > > > > /abc/o; $foo =~ /$null/ > > > > matches $foo against "abc" each time round the loop (it currently > > crashes > > on the second iteration); and that: > > It wasn't clear to me from your post, but this actually behaves > differently between threaded and non-threaded builds - for threaded > builds it behaves as you describe above - /o is honoured on the op > with /o and isn't inherited by and empty match without the /o. > > > > > /abc/; $foo =~ /$null/o > > > > always only evaluates $null once; if that happens to be the empty > > string, > > then subsequent iterations will see the empty string each time and so > > each > > time use the last successful match, which may be /abc/ (it currently > > ignores the //o on the first iteration if $null is empty). > > > > Note that nothing in the test suite currently tests these cases as > > far > > as > > I can tell. > > The attached covers both these cases, I think. Applied as 3cb4cde3dd4d2af2f5065053905708bffa5168f9. Tony

Thank you for filing this report. You have helped make Perl better. With the release yesterday of Perl 5.28.0, this and 185 other issues have been resolved. Perl 5.28.0 may be downloaded via: https://metacpan.org/release/XSAWYERX/perl-5.28.0 If you find that the problem persists, feel free to reopen this ticket.