Skip to content

Narrowing occurred during type inference of ZEND_ADD_ARRAY_ELEMENT #10008

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
FWDekker opened this issue Nov 27, 2022 · 2 comments
Closed

Narrowing occurred during type inference of ZEND_ADD_ARRAY_ELEMENT #10008

FWDekker opened this issue Nov 27, 2022 · 2 comments

Comments

@FWDekker
Copy link

Description

I receive the following warning when running a project of mine:

E_WARNING: Narrowing occurred during type inference of ZEND_ADD_ARRAY_ELEMENT. Please file a bug report on \
    https://github.com/php/php-src/issues {"code":2,"message":"Narrowing occurred during type inference of \
    ZEND_ADD_ARRAY_ELEMENT. Please file a bug report on https://github.com/php/php-src/issues", \
    "file":"<redacted>/dist/com/fwdekker/deathnotifier/Mediawiki.php","line":95} []

How to reproduce

The bug occurs in my open-source project Death Notifier. I tried to create a minimal working example by removing parts which I thought were not relevant for the bug, but each time that resulted in the bug not occurring. So instead, I think it's better to share the whole project.

The warning can be reproduced as follows:

  1. Download the version of my project in which the warning is triggered:
    $> git clone https://git.fwdekker.com/tools/death-notifier.git
    $> cd death-notifier
    $> git checkout 6fb88c5454
  2. Check whether all requirements from the README.md are met on your system. (composer.phar and npm on your path, and some PHP extensions.)
  3. Run npm ci and composer.phar install to install dependencies.
  4. Run npm run dev to build the application into the dist/ directory.
  5. Get a PHP server running so that you can serve PHP through your browser. (I'm using PhpStorm to tie it all together.)
  6. Open dist/api.php in your browser. (You can also open dist/index.html, but note that this will make requests to my server.)
  7. Check dist/.death-notifier.log, where you'll find the warning in the log file.

Code context

The warning mentions line 95 of Mediawiki.php. The full file is hosted in Gitea. Here is an excerpt with the function containing that line; I added a comment to indicate which one is line 95.

/**
 * Queries Wikipedia's API with continuation and returns its response as a JSON object.
 *
 * @param array<string, mixed> $params the query parameters to send to the API
 * @param string|null $continue_name the name of the continue parameter to follow, or `null` if no continuation
 * should be done
 * @return mixed[] the query's value of the `query` key as a JSON object
 * @throws Exception if the query fails
 */
private function api_query_continued(array $params, ?string $continue_name = null): array
{
    $query_params = array_merge(["action" => "query", "format" => "json"], $params);

    $response = [];
    $continue = null;
    $continue_value = null;
    do {
        $continue_params = $continue === null
            ? $query_params
            // >> The following line is line 95 <<
            : array_merge($query_params, ["continue" => $continue, $continue_name => $continue_value]);

        $new_response = $this->api_fetch($continue_params);
        $response = array_merge_recursive_distinct($response, $new_response);

        if (isset($response["batchcomplete"])) {
            $continue = null;
            $continue_value = null;
        } else if ($continue_name !== null) {
            $continue = $response["continue"]["continue"];
            $continue_value = $response["continue"][$continue_name];
        }
    } while ($continue !== null);

    return $response;
}

PHP Version

PHP 8.1.12

Operating System

Debian testing

@nielsdos
Copy link
Member

I can reproduce this on PHP-8.1 and higher.
The original code can be reproduced in context with all optimisations enabled.
I made a simplified, stand-alone reproducer that lets us reproduce the issue reliably on opcache.optimization_level=0x20:

<?php
function test()
{
    $bool_or_int = true;
    while (a()) {
        if ($bool_or_int !== true) {
            // The following line triggers narrowing during type inference of ZEND_ADD_ARRAY_ELEMENT.
            $string_key = "a";
            $array = ["bool_or_int" => $bool_or_int, $string_key => 123];
        }

        $bool_or_int = 0;
    }
}

This triggers narrowing for two ops: first ZEND_ADD_ARRAY_ELEMENT, and then ZEND_ASSIGN.

The type inference happens in the following order:

  • The ZEND_ADD_ARRAY_ELEMENT infers type 0x40e04080 (packed flag is set), arr_type=0 at this point because it hasn't been set by ZEND_INIT_ARRAY yet
  • The ZEND_INIT_ARRAY infers type 0x40804080
  • The ZEND_ADD_ARRAY_ELEMENT infers type 0x40e04080, arr_type=0x40804080, which does not have the packed flag set while the existing result of ZEND_ADD_ARRAY_ELEMENT has the packed flag set

This seems to occur because of the phi node introduced by the while loop. If I remove the loop the problem goes away.
This is a guess, but it seems like we should take into account already inferred information about ZEND_ADD_ARRAY_ELEMENT.

nielsdos added a commit to nielsdos/php-src that referenced this issue Jan 11, 2023
nielsdos added a commit to nielsdos/php-src that referenced this issue Sep 27, 2023
…_ARRAY_ELEMENT

This test triggers narrowing for two ops: first ZEND_ADD_ARRAY_ELEMENT,
and then ZEND_ASSIGN.

The type inference happens in the following order:
1) The ZEND_ADD_ARRAY_ELEMENT infers type 0x40e04080 (packed flag is set),
   arr_type=0 at this point because it hasn't been set by ZEND_INIT_ARRAY yet.
2) The ZEND_INIT_ARRAY infers type 0x40804080
3) The ZEND_ADD_ARRAY_ELEMENT infers type 0x40e04080, arr_type=0x40804080,
   which does not have the packed flag set while the existing result of
   ZEND_ADD_ARRAY_ELEMENT has the packed flag set.

This seems to occur because of the phi node introduced by the while
loop. If I remove the loop the problem goes away.

As Arnaud noted, this seems to be caused by a too wide type inference
for arr_type==0. We should keep the invariant that if x>=y then
key_type(x) >= key_type(y).
If we write the possible results down in a table we get:

```
arr_type           resulting key type
---------------    --------------------------------------------------------------------------
HASH_ONLY	-> MAY_BE_ARRAY_NUMERIC_HASH
PACKED_ONLY	-> MAY_BE_ARRAY_NUMERIC_HASH | MAY_BE_ARRAY_PACKED (== MAY_BE_ARRAY_KEY_LONG)
HASH || PACKED	-> MAY_BE_ARRAY_NUMERIC_HASH | MAY_BE_ARRAY_PACKED (== MAY_BE_ARRAY_KEY_LONG)
0		-> MAY_BE_ARRAY_NUMERIC_HASH | MAY_BE_ARRAY_PACKED (== MAY_BE_ARRAY_KEY_LONG)
```

As we can see, `HASH_ONLY > 0` but
`MAY_BE_ARRAY_NUMERIC_HASH < MAY_BE_ARRAY_NUMERIC_HASH | MAY_BE_ARRAY_PACKED`,
which violates the invariant.
Instead if we modify the zero case to have MAY_BE_ARRAY_NUMERIC_HASH instead,
we get the following table which satisfies the invariant.

```
arr_type           resulting key type
---------------    --------------------------------------------------------------------------
HASH_ONLY	-> MAY_BE_ARRAY_NUMERIC_HASH
PACKED_ONLY	-> MAY_BE_ARRAY_NUMERIC_HASH | MAY_BE_ARRAY_PACKED (== MAY_BE_ARRAY_KEY_LONG)
HASH || PACKED	-> MAY_BE_ARRAY_NUMERIC_HASH | MAY_BE_ARRAY_PACKED (== MAY_BE_ARRAY_KEY_LONG)
0		-> MAY_BE_ARRAY_NUMERIC_HASH
```
nielsdos added a commit that referenced this issue Sep 29, 2023
* PHP-8.1:
  Fix GH-10008: Narrowing occurred during type inference of ZEND_ADD_ARRAY_ELEMENT
  Fix type error on XSLTProcessor::transformToDoc return value with SimpleXML
nielsdos added a commit that referenced this issue Sep 29, 2023
* PHP-8.2:
  Fix GH-10008: Narrowing occurred during type inference of ZEND_ADD_ARRAY_ELEMENT
  Fix type error on XSLTProcessor::transformToDoc return value with SimpleXML
nielsdos added a commit that referenced this issue Sep 29, 2023
* PHP-8.3:
  Fix compile error with -Werror=incompatible-function-pointer-types and old libxml2
  Fix GH-10008: Narrowing occurred during type inference of ZEND_ADD_ARRAY_ELEMENT
  Fix type error on XSLTProcessor::transformToDoc return value with SimpleXML
nielsdos added a commit that referenced this issue Sep 29, 2023
…D_ADD_ARRAY_ELEMENT"

Although it passes CI on 8.1, it causes CI failures in the JIT on 8.2 and
higher.
See https://github.com/php/php-src/actions/runs/6357716718/job/17269225001

This reverts commit e72fc12.
nielsdos added a commit that referenced this issue Sep 29, 2023
* PHP-8.1:
  Revert "Fix GH-10008: Narrowing occurred during type inference of ZEND_ADD_ARRAY_ELEMENT"
nielsdos added a commit that referenced this issue Sep 29, 2023
* PHP-8.2:
  Revert "Fix GH-10008: Narrowing occurred during type inference of ZEND_ADD_ARRAY_ELEMENT"
nielsdos added a commit that referenced this issue Sep 29, 2023
* PHP-8.3:
  Revert "Fix GH-10008: Narrowing occurred during type inference of ZEND_ADD_ARRAY_ELEMENT"
@nielsdos
Copy link
Member

Reopening because it causes JIT failures in 8.2+, it works in 8.1 though strangely

@nielsdos nielsdos reopened this Sep 29, 2023
dstogov added a commit that referenced this issue Nov 2, 2023
* PHP-8.1:
  Fixed GH-10008: Narrowing occurred during type inference of ZEND_ADD_ARRAY_ELEMENT
dstogov added a commit that referenced this issue Nov 2, 2023
* PHP-8.2:
  Fixed GH-10008: Narrowing occurred during type inference of ZEND_ADD_ARRAY_ELEMENT
  ext/intl: change when the locale is invalid for the 8.1/8.2 serie.
@dstogov dstogov closed this as completed in 798b9d0 Nov 2, 2023
dstogov added a commit that referenced this issue Nov 2, 2023
* PHP-8.3:
  Fixed GH-10008: Narrowing occurred during type inference of ZEND_ADD_ARRAY_ELEMENT
  ext/intl: change when the locale is invalid for the 8.1/8.2 serie.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants