Closes #11706. Originally fixed in #11721, but then reverted in #12022 due to a regression in pytest-xdist. The regression was fixed on the pytest-xdist side in pytest-dev/pytest-xdist#1026.
Prepare release version 8.0.0rc2 (cherry picked from commit 97960bdd148972b2f26bd9b336163e590bbc4c6b)
Co-authored-by: Ran Benita <ran@unusedvar.com>