Skip to content

regex: Using "]]]]][\\" as a pattern should raise an error #14919

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
p5pRT opened this issue Sep 23, 2015 · 8 comments
Closed

regex: Using "]]]]][\\" as a pattern should raise an error #14919

p5pRT opened this issue Sep 23, 2015 · 8 comments
Labels
distro-Linux type-core

Comments

@p5pRT
Copy link

p5pRT commented Sep 23, 2015

Migrated from rt.perl.org#126141 (status was 'resolved')

Searchable as RT126141$
@p5pRT
Copy link
Author

p5pRT commented Sep 23, 2015

From [email protected]

Created by @Grimy

How to reproduce
----------------

perl5.23.4 -e 'eval {/$_/}, print "$_ ==> ", $@​ || "OK!\n" for "]]]]]]]]][\\", "]]]]][\\"'

Expected behavior
-----------------

]]]]]]]]][\ ==> Unmatched [ in regex; marked by <-- HERE in m/]]]]]]]]][ <-- HERE \/ at -e line 1.
]]]]][\ ==> Unmatched [ in regex; marked by <-- HERE in m/]]]]][ <-- HERE \/ at -e line 1.

Actual behavior
---------------

]]]]]]]]][\ ==> Unmatched [ in regex; marked by <-- HERE in m/]]]]]]]]][ <-- HERE \/ at -e line 1.
]]]]][\ ==> OK!

"]]]]][\\" fails to raise an error. Note that this does not happen if we remove
of modify the first pattern, "]]]]]]]]][\\". Thus, this bug seems to involve
some leftover state in the regex engine.

Other combinations of regexes can trigger this bug. It happens regularly when
trying millions of randomly generated regexes. All occurences have in common
the unclosed character class and the final backslash; other details vary.

Affected versions
-----------------

Current blead (2d9b5f1) is affected.
perl5.22.0 is affected.
perl5.20.3 is affected.
perl5.18.4 is affected.
perl5.16.3 is affected.
perl5.14.4 is affected.

Perl Info 

Flags:
    category=core
    severity=low

Site configuration information for perl 5.20.2:

Configured by Red Hat, Inc. at Fri Jun  5 10:33:59 UTC 2015.

Summary of my perl5 (revision 5 version 20 subversion 2) configuration:

  Platform:
    osname=linux, osvers=3.19.5-200.fc21.x86_64,
archname=x86_64-linux-thread-multi
    uname='linux buildvm-13.phx2.fedoraproject.org
3.19.5-200.fc21.x86_64 #1 smp mon apr 20 19:51:56 utc 2015 x86_64
x86_64 x86_64 gnulinux '
    config_args='-des -Doptimize=-O2 -g -pipe -Wall
-Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector-strong --param=ssp-buffer-size=4
-grecord-gcc-switches  -m64 -mtune=generic
-Dccdlflags=-Wl,--enable-new-dtags -Dlddlflags=-shared -O2 -g -pipe
-Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector-strong --param=ssp-buffer-size=4
-grecord-gcc-switches  -m64 -mtune=generic -Wl,-z,relro
-Dshrpdir=/usr/lib64 -DDEBUGGING=-g -Dversion=5.20.2
-Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red
Hat, Inc. -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5
-Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl
-Darchlib=/usr/lib64/perl5 -Dvendorarch=/usr/lib64/perl5/vendor_perl
-Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64
/usr/lib64 -Duseshrplib -Dusethreads -Duseithreads
-Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db
-Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio
-Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly
-Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto
-Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto
-Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto
-Dscriptdir=/usr/bin -Dusesitecustomize'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g -pipe -Wall -Werror=format-security
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong
--param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='5.1.1 20150422 (Red Hat 5.1.1-1)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib64 /lib64 /usr/lib64 /usr/local/lib /usr/lib
/lib/../lib64 /usr/lib/../lib64 /lib
    libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread
-lc -lgdbm_compat
    perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=libc-2.21.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.21'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef,
ccdlflags='-Wl,--enable-new-dtags'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall
-Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector-strong --param=ssp-buffer-size=4
-grecord-gcc-switches -m64 -mtune=generic -Wl,-z,relro
-L/usr/local/lib'

Locally applied patches:
    Fedora Patch1: Removes date check, Fedora/RHEL specific
    Fedora Patch3: support for libdir64
    Fedora Patch4: use libresolv instead of libbind
    Fedora Patch5: USE_MM_LD_RUN_PATH
    Fedora Patch6: Skip hostname tests, due to builders not being
network capable
    Fedora Patch7: Dont run one io test due to random builder failures
    Fedora Patch15: Define SONAME for libperl.so
    Fedora Patch16: Install libperl.so to -Dshrpdir value
    Fedora Patch22: Document Math::BigInt::CalcEmu requires
Math::BigInt (CPAN RT#85015)
    Fedora Patch25: Use stronger algorithm needed for FIPS in
t/op/crypt.t (RT#121591)
    Fedora Patch26: Make *DBM_File desctructors thread-safe (RT#61912)
    Fedora Patch27: Report inaccesible file on failed require (RT#123270)
    Fedora Patch28: Use stronger algorithm needed for FIPS in
t/op/taint.t (RT#123338)
    Fedora Patch29: Fix Errno.pm generation for GCC 5.0 (RT#123784)
    Fedora Patch30: Handle hexadecimal constants by h2ph (RT#123784)
    Fedora Patch31: Do not use -_h2ph_pre.ph from system at tests (RT#123784)
    Fedora Patch200: Link XS modules to libperl.so with EU::CBuilder on Linux
    Fedora Patch201: Link XS modules to libperl.so with EU::MM on Linux


@INC for perl 5.20.2:
    /usr/local/lib64/perl5
    /usr/local/share/perl5
    /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl
    /usr/lib64/perl5
    /usr/share/perl5
    .


Environment for perl 5.20.2:
    HOME=/home/grimy
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH=/usr/lib64:/home/grimy/repos/SDK/InstallationCentOS59/IGC/Bin:/home/grimy/repos/SDK/Produits/QT-FC5/lib
    LOGDIR (unset)
    PATH=/home/grimy/bin:/home/grimy/.nvim/scripts:/home/grimy/.autojump/bin:/usr/java/jdk1.8.0_31/bin:/home/grimy/repos/CoreTex/bin:/home/grimy/bin:/home/grimy/.nvim/scripts:/home/grimy/.autojump/bin:/usr/java/jdk1.8.0_31/bin:/home/grimy/repos/CoreTex/bin:/usr/local/bin:/bin:/usr/bin:/home/grimy/repos/SDK/InstallationCentOS59/IGC/Bin:/home/grimy/bin:/home/grimy/.gem/ruby/2.0.0/bin:/sbin:/home/grimy/repos/SDK/InstallationCentOS59/IGC/Bin:/home/grimy/bin:/home/grimy/.gem/ruby/2.0.0/bin:/sbin
    PERL_BADLANG (unset)
    SHELL=/usr/bin/fish

@p5pRT
Copy link
Author

p5pRT commented Sep 24, 2015

From @dcollinsn

I'm not able to reproduce this at all on blead. I see your system perl build parameters, but can you provide your minimal ./Configure command and perl -V that you're able to reproduce this with?

@p5pRT
Copy link
Author

p5pRT commented Sep 24, 2015

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Sep 24, 2015

From @khwilliamson

On Thu Sep 24 09​:22​:42 2015, dcollinsn@​gmail.com wrote​:

I'm not able to reproduce this at all on blead. I see your system perl
build parameters, but can you provide your minimal ./Configure command
and perl -V that you're able to reproduce this with?

I too found that my blead did not have the problem. It is compiled with -DEBUGGING. Then I tried a non-debugging blead, and it did reproduce. When something inconsistent like that happens, it's often found by running valgrind. And sure enough, it was reading beyond the end of a buffer.

I have a fix locally, but will audit regcomp.c for similar errors before pushing it.

--
Karl Williamson

@p5pRT
Copy link
Author

p5pRT commented Mar 4, 2016

From @khwilliamson

Thanks for finding this

Fixed by
ca76e4e
--
Karl Williamson

@p5pRT
Copy link
Author

p5pRT commented Mar 4, 2016

@khwilliamson - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented May 13, 2016

From @khwilliamson

Thank you for submitting this report. You have helped make Perl better.
 
With the release of Perl 5.24.0 on May 9, 2016, this and 149 other issues have been resolved.

Perl 5.24.0 may be downloaded via https://metacpan.org/release/RJBS/perl-5.24.0

@p5pRT
Copy link
Author

p5pRT commented May 13, 2016

@khwilliamson - Status changed from 'pending release' to 'resolved'

@p5pRT p5pRT closed this as completed May 13, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
distro-Linux type-core
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@p5pRT