Pytest trying to check if custom argument is a file crashes due to filename being too long

[jdckmz]

I have a custom flag defined in conftest.py, and when I try to assign it to a value that is too long pytest crashes before ever getting to my code. This reproduces even if the flag isn't defined, and even if the current working directory is /.

Failing example:

/> pytest --xxxxx_flags=" --xxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx"     
Traceback (most recent call last):
  File "/home/ANT.AMAZON.COM/jdckmz/.local/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 188, in console_main
    code = main()
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 146, in main
    config = _prepareconfig(args, plugins)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 325, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/helpconfig.py", line 102, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1013, in pytest_cmdline_parse
    self.parse(args)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1301, in parse
    self._preparse(args, addopts=addopts)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1203, in _preparse
    self.hook.pytest_load_initial_conftests(
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1080, in pytest_load_initial_conftests
    self.pluginmanager._set_initial_conftests(
  File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 525, in _set_initial_conftests
    if anchor.exists():  # we found some file object
  File "/usr/lib/python3.8/pathlib.py", line 1407, in exists
    self.stat()
  File "/usr/lib/python3.8/pathlib.py", line 1198, in stat
    return self._accessor.stat(self)
OSError: [Errno 36] File name too long: '/--xxxxx_flags= --xxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx'

If I reduce the length of the flag, I get the expected behavior for my project, and this different and expected error from my pytest MVP:

/> pytest --xxxxx_flags=" --xxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxx"
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.10, pytest-7.0.0, pluggy-1.0.0
rootdir: /
plugins: flaky-3.7.0, colcon-core-0.10.0, cov-2.8.1
collected 0 items                                                                                                                                                          

============================================================================= warnings summary =============================================================================
home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/cacheprovider.py:433
  /home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: could not create cache path /.pytest_cache/v/cache/nodeids
    config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/stepwise.py:52
  /home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/stepwise.py:52: PytestCacheWarning: could not create cache path /.pytest_cache/v/cache/stepwise
    session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================================================================== 2 warnings in 0.01s ============================================================================
ERROR: file or directory not found: --xxxxx_flags= --xxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxx

I did a little digging into my version of pytest (7.0.0) to make sure I wasn't doing something wrong, but it looks like there is a blind call to pathlib.Path.exists() with a path constructed from the argument in __init__.py:

    #
    # Internal API for local conftest plugin handling.
    #
    def _set_initial_conftests(
        self, namespace: argparse.Namespace, rootpath: Path
    ) -> None:
    ...
    testpaths = namespace.file_or_dir
    foundanchor = False
    for testpath in testpaths:
            path = str(testpath)
            i = path.find("::")
            if i != -1:
                path = path[:i]
            anchor = absolutepath(current / path)
            if anchor.exists():  # this throws OSError which is never caught

It seems to me like there should be a try or something here, since in cases like mine the argument may not be a file at all, and that can cause OS level errors.

Operating System: Ubuntu 20.04 LTS

> pytest --version
pytest 7.0.0
> python3 --version
Python 3.8.10

> pip list
/usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
/usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
Package                       Version
----------------------------- --------------------
aiohttp                       3.8.1
aiosignal                     1.2.0
alabaster                     0.7.12
apturl                        0.5.2
argcomplete                   1.8.1
astroid                       2.9.3
async-timeout                 4.0.2
atomicwrites                  1.4.0
attrs                         21.4.0
autobahn                      17.10.1
Automat                       0.8.0
aws-requests-auth             0.4.3
awscli                        1.22.52
awscrt                        0.13.0
awsiotsdk                     1.9.0
Babel                         2.9.1
bcrypt                        3.2.0
beautifulsoup4                4.8.2
black                         22.1.0
blinker                       1.4
boto3                         1.20.52
botocore                      1.23.52
Brlapi                        0.7.0
cached-property               1.5.1
catkin-pkg-modules            0.5.2
cbor                          1.0.0
certifi                       2021.10.8
cffi                          1.15.0
chardet                       4.0.0
charset-normalizer            2.0.11
click                         8.0.3
cmakelang                     0.6.13
cmakelint                     1.4.2
colcon-argcomplete            0.3.3
colcon-bash                   0.4.2
colcon-cd                     0.1.1
colcon-cmake                  0.2.26
colcon-common-extensions      0.3.0
colcon-core                   0.10.0
colcon-defaults               0.2.6
colcon-devtools               0.2.3
colcon-library-path           0.2.1
colcon-metadata               0.2.5
colcon-notification           0.2.13
colcon-output                 0.2.12
colcon-package-information    0.3.3
colcon-package-selection      0.2.10
colcon-parallel-executor      0.2.4
colcon-pkg-config             0.1.0
colcon-powershell             0.3.7
colcon-python-setup-py        0.2.7
colcon-recursive-crawl        0.2.1
colcon-ros                    0.3.23
colcon-test-result            0.3.8
colcon-zsh                    0.4.0
colorama                      0.4.3
command-not-found             0.3
constantly                    15.1.0
control                       0.9.1
cov-core                      1.15.0
coverage                      4.5.2
cryptography                  36.0.1
cupshelpers                   1.0
cycler                        0.11.0
Cython                        0.29.14
dbus-python                   1.2.16
defer                         1.0.6
distlib                       0.3.4
distro                        1.4.0
distro-info                   0.23ubuntu1
docker                        5.0.3
docker-compose                1.25.0
dockerpty                     0.4.1
docopt                        0.6.2
docutils                      0.15.2
duplicity                     0.8.12.0
EasyCluster                   0.22.2
empy                          3.3.2
entrypoints                   0.3
evdev                         1.3.0
fasteners                     0.14.1
filelock                      3.7.1
filemagic                     1.6
flake8                        3.7.9
flaky                         3.7.0
fonttools                     4.29.1
frozenlist                    1.3.0
future                        0.18.2
gitdb                         4.0.9
gitdb2                        4.0.2
github.py                     0.5.0
GitPython                     3.1.26
gpg                           1.13.1-unknown
greenlet                      1.1.2
html5lib                      1.0.1
httplib2                      0.14.0
hyperlink                     19.0.0
idna                          3.3
ifcfg                         0.18
imagesize                     1.3.0
importlib-metadata            4.10.1
incremental                   16.10.1
influxdb                      5.3.1
iniconfig                     1.1.1
isort                         5.10.1
Jinja2                        3.0.3
jmespath                      0.10.0
jsonschema                    3.2.0
keyring                       18.0.1
keyrings.alt                  3.4.0
kiwisolver                    1.3.2
language-selector             0.1
lark-parser                   0.8.1
launchpadlib                  1.10.13
lazr.restfulclient            0.14.2
lazr.uri                      1.0.3
lazy-object-proxy             1.7.1
lockfile                      0.12.2
louis                         3.12.0
lxml                          4.5.0
lz4                           3.0.2+dfsg
macaroonbakery                1.3.1
Mako                          1.1.0
MarkupSafe                    2.0.1
matplotlib                    3.5.1
mccabe                        0.6.1
mock                          3.0.5
monotonic                     1.5
more-itertools                8.12.0
mpi4py                        3.0.3
msgpack                       1.0.3
multi-key-dict                2.0.3
multidict                     6.0.2
mypy-extensions               0.4.3
netifaces                     0.10.4
nose2                         0.9.1
notify2                       0.3
numpy                         1.22.2
oauthlib                      3.1.0
olefile                       0.46
packaging                     21.3
pandas                        1.4.0
paramiko                      2.9.2
pathspec                      0.9.0
pbr                           5.8.1
pexpect                       4.8.0
Pillow                        9.0.1
pip                           22.1.2
pipenv                        2022.6.7
platformdirs                  2.5.0
pluggy                        1.0.0
protobuf                      3.19.4
psutil                        5.8.0
ptyprocess                    0.7.0
py                            1.11.0
py-ubjson                     0.14.0
pyasn1                        0.4.8
pyasn1-modules                0.2.1
pybind11                      2.8.0
pycairo                       1.16.2
pycodestyle                   2.8.0
pycparser                     2.21
pycrypto                      2.6.1
pycups                        1.9.73
pydocstyle                    2.1.1
pydot                         1.4.1
pyelftools                    0.28
pyflakes                      2.1.1
Pygments                      2.11.2
PyGObject                     3.36.0
PyHamcrest                    1.9.0
PyJWT                         1.7.1
pylint                        2.12.2
pymacaroons                   0.13.0
PyNaCl                        1.5.0
pyOpenSSL                     19.0.0
pyparsing                     3.0.7
pypng                         0.0.20
PyQRCode                      1.2.1
PyQt5                         5.14.1
pyquaternion                  0.9.9
pyRFC3339                     1.1
pyrsistent                    0.15.5
pyserial                      3.5
pytest                        7.0.0
pytest-cov                    2.8.1
python-apt                    2.0.0+ubuntu0.20.4.7
python-dateutil               2.8.2
python-debian                 0.1.36ubuntu1
python-dotenv                 0.19.2
python-jenkins                1.7.0
python-magic                  0.4.16
python-snappy                 0.5.3
PyTrie                        0.2
pytz                          2021.3
pyxdg                         0.26
PyYAML                        5.3.1
reportlab                     3.5.34
requests                      2.27.1
requests-unixsocket           0.2.0
roman                         2.0.0
rosdistro-modules             0.9.0
rospkg-modules                1.4.0
rplidar                       0.9.2
rsa                           4.7.2
s3transfer                    0.5.1
scipy                         1.8.0
screen-resolution-extra       0.0.0
SecretStorage                 2.3.1
service-identity              18.1.0
setproctitle                  1.1.10
setuptools                    45.2.0
simplejson                    3.16.0
sip                           4.19.21
six                           1.16.0
smmap                         5.0.0
smmap2                        3.0.1
snowballstemmer               2.2.0
soupsieve                     1.9.5
Sphinx                        4.4.0
sphinx-autoapi                1.8.4
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-dotnetdomain    0.4
sphinxcontrib-golangdomain    0.2.0.dev0
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
sphinxcontrib-websupport      1.2.4
SQLAlchemy                    1.4.35
ssh-import-id                 5.10
tensorrt                      8.0.1.6
texttable                     1.6.2
toml                          0.10.2
tomli                         2.0.1
tripy                         1.0.0
Twisted                       18.9.0
txaio                         2.10.0
typed-ast                     1.5.2
typing_extensions             4.0.1
u-msgpack-python              2.1
ubuntu-advantage-tools        27.9
ubuntu-drivers-common         0.0.0
ufw                           0.36
unattended-upgrades           0.1
Unidecode                     1.3.2
urllib3                       1.26.8
usb-creator                   0.3.7
virtualenv                    20.14.1
virtualenv-clone              0.5.7
wadllib                       1.3.3
wcwidth                       0.1.8
webencodings                  0.5.1
websocket-client              1.2.3
wheel                         0.34.2
wrapt                         1.13.3
wsaccel                       0.6.2
xdot                          1.1
xkit                          0.0.0
xmltodict                     0.12.0
yarl                          1.7.2
zipp                          3.7.0
zope.interface                4.7.1
zstandard                     0.17.0

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

[nicoddemus]

Thanks for the report @jdckmz!

[PurityLake]

From what I can tell, if we have got to this point there is no pluginmanager has picked up the command line arg, so is there really anything we can do?

[jdckmz]

The args that I've added aren't part of a plugin but are part of pytest itself as I understand, documented here:
In conftest.py:

def pytest_addoption(parser):
    """pytest hook to add command line options."""

    group = parser.getgroup("Canvas Test Options")
    group.addoption(
        "--xxxxx_flags",
        default=None,
        help="Extra flags to pass to the launched process.",
    )

[PurityLake]

@jdckmz

Ok after some investigation, you need to make --xxxxx_flags into a pytest.fixture for it to not register as a path

[arichardson]

I am seeing the same thing, there is even a comment in _set_initial_conftests that suggests this could be an issue:

        """Load initial conftest files given a preparsed "namespace".

        As conftest files may add their own command line options which have
        arguments ('--my-opt somepath') we might get some false positives.
        All builtin and 3rd party plugins will have been loaded, however, so
        common options will not confuse our logic here.
        """

Maybe there should be a try/except around the exists call? Or alternatively run this logic after trying to parse the arguments?

[nicoddemus]

@arichardson funny you posted this today, as I did a workaround for that just yesterday in https://github.com/pytest-dev/pytest/pull/10988/files#diff-df52f8f6a3544754cc8ebdf903594738e68a18dc9ac3c959f646cf4705a9afed.

Seems however we should remove that guard against Python 3.7 and always do the try/except to also solves this.