p53209761
/
test_ok2
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Adjust releasing infra/docs (#8795) 

* Adjust releasing infra/docs

Follow-up for #8150, see #7507

* Add suggestions
This commit is contained in:
Florian Bruhin 2021-06-28 12:24:48 +02:00 committed by GitHub
parent 6573107b97
commit c19f63d39d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 64 additions and 342 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

40
.github/workflows/release-on-comment.yml vendored
View File

@ -1,40 +0,0 @@
# part of our release process, see `release-on-comment.py`
name: release on comment
on:
issues:
types: [opened, edited]
issue_comment:
types: [created, edited]
# Set permissions at the job level.
permissions: {}
jobs:
build:
runs-on: ubuntu-latest
permissions:
contents: write
issues: write
if: (github.event.comment && startsWith(github.event.comment.body, '@pytestbot please')) || (github.event.issue && !github.event.comment && startsWith(github.event.issue.body, '@pytestbot please'))
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: "3.8"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install --upgrade setuptools tox
- name: Prepare release
run: |
tox -e release-on-comment -- $GITHUB_EVENT_PATH ${{ github.token }}

98
RELEASING.rst
View File

@ -14,60 +14,90 @@ Preparing: Automatic Method
~~~~~~~~~~~~~~~~~~~~~~~~~~~
We have developed an automated workflow for releases, that uses GitHub workflows and is triggered
by opening an issue.
by `manually running <https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow>`__
the `prepare-release-pr workflow <https://github.com/pytest-dev/pytest/actions/workflows/prepare-release-pr.yml>`__
on GitHub Actions.
Bug-fix releases
^^^^^^^^^^^^^^^^
The automation will decide the new version number based on the following criteria:
A bug-fix release is always done from a maintenance branch, so for example to release bug-fix
``5.1.2``, open a new issue and add this comment to the body::
- If the "major release" input is set to "yes", release a new major release
(e.g. 7.0.0 -> 8.0.0)
- If there are any ``.feature.rst`` or ``.breaking.rst`` files in the
``changelog`` directory, release a new minor release (e.g. 7.0.0 -> 7.1.0)
- Otherwise, release a bugfix release (e.g. 7.0.0 -> 7.0.1)
- If the "prerelease" input is set, append the string to the version number
(e.g. 7.0.0 -> 8.0.0rc1), if "major" is set, and "prerelease" is set to `rc1`)
@pytestbot please prepare release from 5.1.x
Bug-fix and minor releases
^^^^^^^^^^^^^^^^^^^^^^^^^^
Where ``5.1.x`` is the maintenance branch for the ``5.1`` series.
Bug-fix and minor releases are always done from a maintenance branch. First,
consider double-checking the ``changelog`` directory to see if there are any
breaking changes or new features.
The automated workflow will publish a PR for a branch ``release-5.1.2``
and notify it as a comment in the issue.
For a new minor release, first create a new maintenance branch from ``main``::
Minor releases
git fetch --all
git branch 7.1.x upstream/main
git push upstream 7.1.x
Then, trigger the workflow with the following inputs:
- branch: **7.1.x**
- major release: **no**
- prerelease: empty
Or via the commandline using `GitHub's cli <https://github.com/cli/cli>`__::
gh workflow run prepare-release-pr.yml -f branch=7.1.x -f major=no -f prerelease=
Where ``7.1.x`` is the maintenance branch for the ``7.1`` series. The automated
workflow will publish a PR for a branch ``release-7.1.0``.
Similarly, for a bug-fix release, use the existing maintenance branch and
trigger the workflow with e.g. ``branch: 7.0.x`` to get a new ``release-7.0.1``
PR.
Major releases
^^^^^^^^^^^^^^
1. Create a new maintenance branch from ``main``::
git fetch --all
git branch 5.2.x upstream/main
git push upstream 5.2.x
git branch 8.0.x upstream/main
git push upstream 8.0.x
2. Open a new issue and add this comment to the body::
2. Trigger the workflow with the following inputs:
@pytestbot please prepare release from 5.2.x
- branch: **8.0.x**
- major release: **yes**
- prerelease: empty
The automated workflow will publish a PR for a branch ``release-5.2.0`` and
notify it as a comment in the issue.
Or via the commandline::
Major and release candidates
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gh workflow run prepare-release-pr.yml -f branch=8.0.x -f major=yes -f prerelease=
1. Create a new maintenance branch from ``main``::
git fetch --all
git branch 6.0.x upstream/main
git push upstream 6.0.x
2. For a **major release**, open a new issue and add this comment in the body::
@pytestbot please prepare major release from 6.0.x
For a **release candidate**, the comment must be (TODO: `#7551 <https://github.com/pytest-dev/pytest/issues/7551>`__)::
@pytestbot please prepare release candidate from 6.0.x
The automated workflow will publish a PR for a branch ``release-6.0.0`` and
notify it as a comment in the issue.
The automated workflow will publish a PR for a branch ``release-8.0.0``.
At this point on, this follows the same workflow as other maintenance branches: bug-fixes are merged
into ``main`` and ported back to the maintenance branch, even for release candidates.
Release candidates
^^^^^^^^^^^^^^^^^^
To release a release candidate, set the "prerelease" input to the version number
suffix to use. To release a ``8.0.0rc1``, proceed like under "major releases", but set:
- branch: 8.0.x
- major release: yes
- prerelease: **rc1**
Or via the commandline::
gh workflow run prepare-release-pr.yml -f branch=8.0.x -f major=yes -f prerelease=rc1
The automated workflow will publish a PR for a branch ``release-8.0.0rc1``.
**A note about release candidates**
During release candidates we can merge small improvements into

261
scripts/release-on-comment.py
View File

@ -1,261 +0,0 @@
"""
This script is part of the pytest release process which is triggered by comments
in issues.
This script is started by the `release-on-comment.yml` workflow, which always executes on
`main` and is triggered by two comment related events:
* https://help.github.com/en/actions/reference/events-that-trigger-workflows#issue-comment-event-issue_comment
* https://help.github.com/en/actions/reference/events-that-trigger-workflows#issues-event-issues
This script receives the payload and a secrets on the command line.
The payload must contain a comment with a phrase matching this pseudo-regular expression:
@pytestbot please prepare (major )? release from <branch name>
Then the appropriate version will be obtained based on the given branch name:
* a major release from main if "major" appears in the phrase in that position
* a feature or bug fix release from main (based if there are features in the current changelog
folder)
* a bug fix from a maintenance branch
After that, it will create a release using the `release` tox environment, and push a new PR.
**Token**: currently the token from the GitHub Actions is used, pushed with
`pytest bot <pytestbot@gmail.com>` commit author.
"""
import argparse
import json
import os
import re
import traceback
from pathlib import Path
from subprocess import CalledProcessError
from subprocess import check_call
from subprocess import check_output
from subprocess import run
from textwrap import dedent
from typing import Dict
from typing import Optional
from typing import Tuple
from colorama import Fore
from colorama import init
from github3.repos import Repository
class InvalidFeatureRelease(Exception):
pass
SLUG = "pytest-dev/pytest"
PR_BODY = """\
Created automatically from {comment_url}.
Once all builds pass and it has been **approved** by one or more maintainers, the build
can be released by pushing a tag `{version}` to this repository.
Closes #{issue_number}.
"""
def login(token: str) -> Repository:
import github3
github = github3.login(token=token)
owner, repo = SLUG.split("/")
return github.repository(owner, repo)
def get_comment_data(payload: Dict) -> str:
if "comment" in payload:
return payload["comment"]
else:
return payload["issue"]
def validate_and_get_issue_comment_payload(
issue_payload_path: Optional[Path],
) -> Tuple[str, str, bool]:
payload = json.loads(issue_payload_path.read_text(encoding="UTF-8"))
body = get_comment_data(payload)["body"]
m = re.match(r"@pytestbot please prepare (major )?release from ([-_.\w]+)", body)
if m:
is_major, base_branch = m.group(1) is not None, m.group(2)
else:
is_major, base_branch = False, None
return payload, base_branch, is_major
def print_and_exit(msg) -> None:
print(msg)
raise SystemExit(1)
def trigger_release(payload_path: Path, token: str) -> None:
payload, base_branch, is_major = validate_and_get_issue_comment_payload(
payload_path
)
if base_branch is None:
url = get_comment_data(payload)["html_url"]
print_and_exit(
f"Comment {Fore.CYAN}{url}{Fore.RESET} did not match the trigger command."
)
print()
print(f"Precessing release for branch {Fore.CYAN}{base_branch}")
repo = login(token)
issue_number = payload["issue"]["number"]
issue = repo.issue(issue_number)
check_call(["git", "checkout", f"origin/{base_branch}"])
try:
version = find_next_version(base_branch, is_major)
except InvalidFeatureRelease as e:
issue.create_comment(str(e))
print_and_exit(f"{Fore.RED}{e}")
error_contents = ""
try:
print(f"Version: {Fore.CYAN}{version}")
release_branch = f"release-{version}"
run(
["git", "config", "user.name", "pytest bot"],
text=True,
check=True,
capture_output=True,
)
run(
["git", "config", "user.email", "pytestbot@gmail.com"],
text=True,
check=True,
capture_output=True,
)
run(
["git", "checkout", "-b", release_branch, f"origin/{base_branch}"],
text=True,
check=True,
capture_output=True,
)
print(f"Branch {Fore.CYAN}{release_branch}{Fore.RESET} created.")
# important to use tox here because we have changed branches, so dependencies
# might have changed as well
cmdline = ["tox", "-e", "release", "--", version, "--skip-check-links"]
print("Running", " ".join(cmdline))
run(
cmdline,
text=True,
check=True,
capture_output=True,
)
oauth_url = f"https://{token}:x-oauth-basic@github.com/{SLUG}.git"
run(
["git", "push", oauth_url, f"HEAD:{release_branch}", "--force"],
text=True,
check=True,
capture_output=True,
)
print(f"Branch {Fore.CYAN}{release_branch}{Fore.RESET} pushed.")
body = PR_BODY.format(
comment_url=get_comment_data(payload)["html_url"],
version=version,
issue_number=issue_number,
)
pr = repo.create_pull(
f"Prepare release {version}",
base=base_branch,
head=release_branch,
body=body,
)
print(f"Pull request {Fore.CYAN}{pr.url}{Fore.RESET} created.")
comment = issue.create_comment(
f"As requested, opened a PR for release `{version}`: #{pr.number}."
)
print(f"Notified in original comment {Fore.CYAN}{comment.url}{Fore.RESET}.")
except CalledProcessError as e:
error_contents = f"CalledProcessError\noutput:\n{e.output}\nstderr:\n{e.stderr}"
except Exception:
error_contents = f"Exception:\n{traceback.format_exc()}"
if error_contents:
link = f"https://github.com/{SLUG}/actions/runs/{os.environ['GITHUB_RUN_ID']}"
msg = ERROR_COMMENT.format(
version=version, base_branch=base_branch, contents=error_contents, link=link
)
issue.create_comment(msg)
print_and_exit(f"{Fore.RED}{error_contents}")
else:
print(f"{Fore.GREEN}Success.")
ERROR_COMMENT = """\
The request to prepare release `{version}` from {base_branch} failed with:
```
{contents}
```
See: {link}.
"""
def find_next_version(base_branch: str, is_major: bool) -> str:
output = check_output(["git", "tag"], encoding="UTF-8")
valid_versions = []
for v in output.splitlines():
m = re.match(r"\d.\d.\d+$", v.strip())
if m:
valid_versions.append(tuple(int(x) for x in v.split(".")))
valid_versions.sort()
last_version = valid_versions[-1]
changelog = Path("changelog")
features = list(changelog.glob("*.feature.rst"))
breaking = list(changelog.glob("*.breaking.rst"))
is_feature_release = features or breaking
if is_feature_release and base_branch != "main":
msg = dedent(
f"""
Found features or breaking changes in `{base_branch}`, and feature releases can only be
created from `main`:
"""
)
msg += "\n".join(f"* `{x.name}`" for x in sorted(features + breaking))
raise InvalidFeatureRelease(msg)
if is_major:
return f"{last_version[0]+1}.0.0"
elif is_feature_release:
return f"{last_version[0]}.{last_version[1] + 1}.0"
else:
return f"{last_version[0]}.{last_version[1]}.{last_version[2] + 1}"
def main() -> None:
init(autoreset=True)
parser = argparse.ArgumentParser()
parser.add_argument("payload")
parser.add_argument("token")
options = parser.parse_args()
trigger_release(Path(options.payload), options.token)
if __name__ == "__main__":
main()

7
tox.ini
View File

@ -149,13 +149,6 @@ deps =
towncrier
commands = python scripts/release.py {posargs}
[testenv:release-on-comment]
decription = do a release from a comment on GitHub
usedevelop = {[testenv:release]usedevelop}
passenv = {[testenv:release]passenv}
deps = {[testenv:release]deps}
commands = python scripts/release-on-comment.py {posargs}
[testenv:prepare-release-pr]
decription = prepare a release PR from a manual trigger in GitHub actions
usedevelop = {[testenv:release]usedevelop}