Extract hardening CFLAGS to a special $hardenflags variable

This changes the automatic detection of -fstack-protector,
-D_FORTIFY_SOURCE, and -mbranch-protection to write to $hardenflags
instead of $XCFLAGS. The definition of $cflags is changed to
"$hardenflags $orig_cflags $optflags $debugflags $warnflags" to match.

Furthermore, these flags are _prepended_ to $hardenflags, rather than
appended.

The implications of doing this are as follows:

* If a CRuby builder specifies cflags="-mbranch-protection=foobar" at
  the ./configure script, and the configure script detects that
  -mbranch-protection=pac-ret is accepted, then GCC will be invoked as
  "gcc -mbranch-protection=pac-ret -mbranch-protection=foobar". Since
  the last flags take precedence, that means that user-supplied values
  of these flags in $cflags will take priority.
* Likewise, if a CRuby builder explicitly specifies
  "hardenflags=-mbranch-protection=foobar", because we _prepend_ to
  $hardenflags in our autoconf script, we will still invoke GCC as
  "gcc -mbranch-protection=pac-ret -mbranch-protection=foobar".
* If a CRuby builder specifies CFLAGS="..." at the configure line,
  automatic detection of hardening flags is ignored as before.
* C extensions will _also_ be built with hardening flags now as well
  (this was not the case by default before because the detected flags
  went into $XCFLAGS).

Additionally, as part of this work, I changed how the detection of
PAC/BTI in Context.S works. Rather than appending the autodetected
option to ASFLAGS, we simply compile a set of test programs with the
actual CFLAGS in use to determine what PAC/BTI settings were actually
chosen by the builder. Context.S is made aware of these choices through
some custom macros.

The result of this work is that:

* Ruby will continue to choose some sensible defaults for hardening
  options for the C compiler
* Distributors are able to specify CFLAGS that are consistent with their
  distribution and override these defaults
* Context.S will react to whatever -mbranch-protection is actually in
  use, not what was autodetected
* Extensions get built with hardening flags too.

[Bug #20154]
[Bug #20520]

2 files changed, 21 insertions, 0 deletions

diff --git a/tool/m4/ruby_append_option.m4 b/tool/m4/ruby_append_option.m4
index ff828d2162..98359fa1f9 100644
--- a/tool/m4/ruby_append_option.m4
+++ b/tool/m4/ruby_append_option.m4
@@ -3,3 +3,7 @@ AC_DEFUN([RUBY_APPEND_OPTION],
 	[# RUBY_APPEND_OPTION($1)
 	AS_CASE([" [$]{$1-} "],
 	[*" $2 "*], [], ['  '], [ $1="$2"], [ $1="[$]$1 $2"])])dnl
+AC_DEFUN([RUBY_PREPEND_OPTION],
+	[# RUBY_APPEND_OPTION($1)
+	AS_CASE([" [$]{$1-} "],
+	[*" $2 "*], [], ['  '], [ $1="$2"], [ $1="$2 [$]$1"])])dnl
diff --git a/tool/m4/ruby_try_cflags.m4 b/tool/m4/ruby_try_cflags.m4
index b74718fe5e..b397642aad 100644
--- a/tool/m4/ruby_try_cflags.m4
+++ b/tool/m4/ruby_try_cflags.m4
@@ -22,3 +22,20 @@ AC_DEFUN([RUBY_TRY_CFLAGS], [
 	AC_MSG_RESULT(no)],
 	[$4], [$5])
 ])dnl
+
+AC_DEFUN([_RUBY_TRY_CFLAGS_PREPEND], [
+    RUBY_WERROR_FLAG([
+    CFLAGS="$1 [$]CFLAGS"
+    AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[$4]], [[$5]])],
+	[$2], [$3])
+    ])dnl
+])dnl
+AC_DEFUN([RUBY_TRY_CFLAGS_PREPEND], [
+    AC_MSG_CHECKING([whether ]$1[ is accepted as CFLAGS])dnl
+    _RUBY_TRY_CFLAGS_PREPEND([$1],
+	[$2
+	AC_MSG_RESULT(yes)],
+	[$3
+	AC_MSG_RESULT(no)],
+	[$4], [$5])
+])dnl