Fixes#11104.
See the issue for a description of the problem.
Now, we use the same logic for initial conftest paths as we do for
deciding the initial args, which was the idea behind checking
`namespace.file_or_dir` and `testpaths` previously.
This fixes the issue of `testpaths` being considered for initial
conftests even when it's not used for the args.
(Another issue in faeb16146b was that the
`testpaths` were not glob-expanded, this is also fixed.)