pytest collects files when told not to

[CobaltCause]

I can reproduce this with a project set up as follows:

• pyproject.toml:

  [tool.pytest.ini_options]
  python_files = "*.py"
  testpaths = [
      "tests",
  ]

• src/playground/__init__.py exists and is empty

• src/playground/__main__.py:

  import sys
  
  
  def main():
      print("Hello, world!")
  
  
  sys.exit(main())

• tests does not exist

When I run pytest, I get this output:

============================= test session starts ==============================
platform linux -- Python 3.11.3, pytest-7.2.1, pluggy-1.0.0
rootdir: /home/charles/science/python/playground, configfile: pyproject.toml
plugins: cov-4.0.0
collected 0 items / 1 error

==================================== ERRORS ====================================
_________________ ERROR collecting src/playground/__main__.py __________________
src/playground/__main__.py:11: in <module>
    sys.exit(main())
E   SystemExit
------------------------------- Captured stdout --------------------------------
Hello, world!
=========================== short test summary info ============================
ERROR src/playground/__main__.py - SystemExit
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.03s ===============================

I would expect this to succesfully collect nothing and run no tests, but instead it fails because it runs __main__.py. Notably, if I remove python_files = "*.py" from pyproject.toml, it works. It seems like this shouldn't matter, though, because I'm already narrowing testpaths to tests.

[nicoddemus]

Hi @CobaltCause,

This is not really a bug, but an intended (albeit questionable) behavior:

The values of testpaths are actually globs, so globbing for tests in the root yields nothing. Given it finds nothing, pytest will behave as if called from the command-line without any parameters, which makes it search recursively from the current directory looking for python_files to collect.

pytest/src/_pytest/config/__init__.py

Lines 1382 to 1384 in 739408b

	args = []
	for path in testpaths:
	args.extend(sorted(glob.iglob(path, recursive=True)))

If you create the tests directory, then pytest will correctly search in that directory only.

I agree those 2 facts are surprising:

1. The fact that testpaths is a glob. This is documented but easy to overlook, probably we should add a glob to the example there.
2. pytest silently not finding anything, and then proceeding as usual.

I don't think we can do anything more for 1, but for 2 seems like we should at least emit a warning if testpaths is defined but does not match anything.

[CobaltCause]

Oh, interesting. Yeah, this is quite surprising behavior. Making a tests directory, even if empty, does make it work the way I wanted. Either a warning or an error and refusal to continue would have really helped me here, yeah. Thanks for the explanation!

[nicoddemus]

Created #11013 to issue the warning/error, closing this one then.

Thanks @CobaltCause for the report!