diff --git a/doc/RELEASING.md b/doc/RELEASING.md index 5942a60e3..25b88d3fb 100644 --- a/doc/RELEASING.md +++ b/doc/RELEASING.md @@ -11,6 +11,9 @@ Notes for hledger release managers and maintainers. **2025** [ ] Make releasing easier +[ ] Improve automation +[ ] Improve process docs +[ ] Track releasing and release engineering time explicitly, per release \ **2024** [x] Make releasing easier @@ -23,7 +26,7 @@ Notes for hledger release managers and maintainers. [x] Establish routine monthly release cadence, [ ] Make releasing easy -## Release types +## hledger release types hledger major releases happen each quarter, normally at the start of the third month (see [past releases](relnotes.md)). Bugfix releases follow when needed. @@ -48,6 +51,241 @@ Preview releases may happen in the other months if wanted. [Regression bounty]: http://hledger.org/regressionbounty +## hledger release artifacts / value chain + +Higher things depend on lower things. +Release readiness and the release process go from the bottom of this diagram to the top. + + +[![release diagram](RELEASING.png)](RELEASING.png) + + +## Release checklist + +This is the guide for doing a hledger release. +It should correspond to the diagram above, but includes more detail of waypoints, required artifacts, and relevant commands. +It should be updated and improved each time it is used. + +- [ ] product + - [ ] blocking defects resolved + - [ ] desired improvements landed and stabilised + - [ ] building and passing tests with current ghcs, deps, and stackage snapshots + - [ ] building and passing tests on all platforms + `just relbin` (or push to github `binaries[-*]` branch) +- [ ] product docs and metadata + - [ ] release branch + - [ ] version strings (in **/.version, */.version.m4, */package.yaml) + - [ ] cabal files x 4 (hledger*/hledger*.cabal) + `just relbranch VER`, + `just cabalfilestest` + - [ ] options help texts up to date (in CliOptions.hs, UIOptions.hs, WebOptions.hs) + - [ ] embedded manuals x 3 + - [ ] generaloptions macro (in doc/common.m4) + - [ ] tool specific options in manuals (hledger*/hledger*.m4.md > # Options) + - [ ] man page dates (*/.date.m4) + - [ ] man (hledger*/hledger*.1) + - [ ] info (hledger*/hledger*.info) + - [ ] text (hledger*/hledger*.txt) + `./Shake manuals -c` + - [ ] embedded tldr pages synced with upstream (doc/tldr/*) + `just tldr-diff` + - [ ] embedded asciinema demos (hledger/embeddedfiles/*.cast) + - [ ] shell completions (hledger/shell-completion/hledger-completion.bash) + `just completions`, commit any changes + - [ ] changelogs x 5 (**/CHANGES.md) + `just changelogs [-c]` + `just changelogs-finalise` +- [ ] release docs and artifacts + - [ ] draft binaries building started + `just relbin` + - [ ] hledger.org html manuals x 3 (site/src/VER/*.md) + `just site-manuals-snapshot VER` + update `site/Makefile`, `site/js/site.js`, `site/hledger.org.caddy` + - [ ] release notes @ hledger.org (doc/relnotes.md) + `just relnotes`, + select & transform with `md-issue-refs`, + uniquify issue refs, + (unwrap long lines, ?) + add author github nicks, + commit + - [ ] github binary install docs (doc/ghrelnotes.md) up to date and pre-tested + - [ ] hledger-install/hledger-install.sh up to date and pre-tested + (`./Shake hledger-install-version` ?), + check/update dep versions, + select/test snapshot version + - [ ] Install page (site/src/install.md) up to date and pre-tested + - [ ] draft announcement for chat / mail list (doc/ANNOUNCE) + - [ ] draft announcement for mastodon (doc/ANNOUNCE.masto) + - [ ] release tags + `just reltags` + - [ ] release binaries built from tag + `just relbin` + - [ ] Install page (site/src/install.md) --version examples match release binaries +- [ ] published + - [ ] relevant release branch work cherry-picked to master branch + changelogs + relnotes + hledger-install + announcements + - [ ] all packages uploaded correctly to hackage + `just hackageupload`, check versions, confirm, check all uploads successful + - [ ] master branch pushed to github + - [ ] new manuals published and rendering/redirecting correctly + - [ ] site repo pushed to github + - [ ] main and site repos auto-pulled to hledger.org, site rebuilt + `hledgerorgsh grep release.= /opt/hledger/site/out/js/site.js` + - [ ] https://www.hledger.org/js/site.js showing latest version + `curl -s https://hledger.org/js/site.js | grep release.=` + purge cache at https://dash.cloudflare.com/f629035917dd3b99b1e37ae20c15ff09/hledger.org/caching/configuration + - [ ] default manual urls redirecting to latest version + `hledgerorgsh sh -c 'systemctl stop caddy; systemctl start caddy'` + `curl -sI https://hledger.org/hledger.html | grep location` + - [ ] release branch pushed to github + - [ ] release tags pushed to github + `just reltags-push VER` + - [ ] github draft release with release binaries attached + should be autocreated by the above; if not, + (https://github.com/simonmichael/hledger/releases/new) + `just ghrelnotes-push` + (`just ghruns-download`, too slow) + instead: `just ghruns-open`, download to tmp/, unzip the unix ones + `just ghrelease-upload` + - [ ] github release published + review, + publish + - [ ] install instructions tested and working + - [ ] hledger-install + - [ ] stack + - [ ] cabal + - [ ] source checkout + - [ ] github release > How to install, each platform + - [ ] announced + - [ ] mail list(s) hledger@googlegroups.com, + haskell-cafe@googlegroups.com for major + - [ ] matrix + - [ ] irc + - [ ] mastodon + - [ ] pta forum +- [ ] cleanup and support + - [ ] review/polish relnotes, changelogs + review, edit + propagate + - [ ] new dev tag/versions/man dates in master + `j devtag VER.99` + - [ ]y pta.o project stats updated + - [ ] process notes updated/cleaned + - [ ] monitor/support/handle issues: + [issue tracker](https://github.com/simonmichael/hledger/issues?q=is%3Aopen+is%3Aissue), matrix, irc, mail list, forum, reddit + +After release, it's a good time for: + + - finance repo updates + - hledger.org doc updates + - pta.o doc updates + +## How to.. + +More procedure notes. + +### Check release readiness + +- Any blocking open issues ? +- Any blocking open PRs ? +- Any blocking items on ? +- Any blocking items in personal notes & backlogs ? + +### Check tools are up to date +- Check for consistent stackage snapshot(s) and extra deps used in + stack.yaml, Shake.hs, hledger-install.sh, bin scripts, tools scripts +- Shake binary is up to date + `./Shake.hs` +- `hpack --version` matches the one in `stack --version` + +### Run local tests +- `just test` +- `just doctest` +- `just haddocktest` + +### Run CI tests +- push to a PR, wait for green +- or push to `ci` branch, wait for green at +- or `just push` (pushes to `ci`, then to `master`) + +### Run release branch tests +- `just test` +- `stack exec -- hledger --version`, check version, hash, release date, no '+' +- `stack exec -- hledger help | tail`, check version, month matches release + +### Update the Install page +In site repo: +- update `install.md` + - query-replace OLD -> NEW in + - "current hledger release" + - CI binaries badges/links, including linux-static-arm32v7 if built + - "building from source" + - stack install command + - cabal install command + - query-replace OLD-brightgreen -> OLD-red + - only after release binaries are built (preferably after release is published): + update --version outputs (version, hash, date, but not platform) + - final output line from `hledger test` (run local build and in terminal for normal speed) + - Total count from `make functest` + - preview + - commit: `install: NEW` + +### Update hledger in stackage + +- monitor packaging status in lts and nightly: +- update as needed + +### Update RELEASING.png +- edit RELEASING.canvas in obsidian +- CMD-p > Export as image, don't show logo +- commit + +## Tips + +- Release, or practice releasing, often to improve the process. + +- Use and continually update RELEASING.md. + Document procedures and gotchas to save time and enable automation in future. + +- Also the diagram (RELEASING.canvas, made with Obsidian). + +- But don't document prematurely or in too much detail. + +- Make things a little better each time through: simpler, more reliable, better documented, more automated, easier, faster, cheaper, higher quality. + +- Optionally save this file as RELEASING2.md and update notes there until after release, if it's interfering with git branch switching. + +- Use and update scripts, in `Justfile`, `Shake.hs`, `tools/` etc. + +- Do all releases from a release branch. + +- Update dev changelogs frequently in master. Finalise changelogs in the release branch. Merge back to master after release. + (Related older doc: [CHANGELOGS](CHANGELOGS.md)) + +- All release binaries should be built from the release-tagged commit. + The binaries' --version output should match the release tag and release date. + +- Try to do only full releases including all four main hledger packages; partial releases add complexity. + +- Try to avoid pre-announcing a hard release date. + It will always take more time than you think, + if you go late you might miss your intended date in many timezones, + and there's no point adding unnecessary pressure. + +- The biggest potential time sinks are: + + - reviewing/relearning the process/docs/infrastructure + - updating/improving the process/docs/infrastructure + - preparing changelogs + - building binaries for all platforms + - troubleshooting github workflow issues + - followup work due to release mistakes, bugs in new features, or regressions + +- Hard/risky/intensive tasks should happen without time pressure; + during the final countdown, things should be easy. + ## Release manager activities These have complex interdependencies and sequencing constraints. @@ -66,7 +304,7 @@ Chunk, separate, routinise, document and automate them as far as possible. ## Glossary -Some terminology useful when precision is needed, eg in release scripts. +Some standard terminology, useful when precision is needed, eg in release scripts. ### General @@ -164,399 +402,3 @@ The branch named `master` in the hledger repo; the main line of hledger developm Branches named `MA.JOR-branch` in the hledger repo, eg `1.25-branch`. Releases and release previews are always made from a release branch. -## Release artifacts / value chain - -Higher things depend on lower things. -Release readiness and the release process go from the bottom of this diagram to the top. - - -[![release diagram](RELEASING.png)](RELEASING.png) - - -## Checklist -Waypoints and required artifacts. - -- [ ] product - - [ ] blocking defects resolved - - [ ] desired improvements landed and stabilised - - [ ] building and passing tests with current ghcs, deps, and stackage snapshots - - [ ] building and passing tests on all platforms - `just relbin` (or push to github `binaries[-*]` branch) -- [ ] product docs and metadata - - [ ] release branch - - [ ] version strings (in **/.version, */.version.m4, */package.yaml) - - [ ] cabal files x 4 (hledger*/hledger*.cabal) - `just relbranch VER`, - `just cabalfilestest` - - [ ] options help texts up to date (in CliOptions.hs, UIOptions.hs, WebOptions.hs) - - [ ] embedded manuals x 3 - - [ ] generaloptions macro (in doc/common.m4) - - [ ] tool specific options in manuals (hledger*/hledger*.m4.md > # Options) - - [ ] man page dates (*/.date.m4) - - [ ] man (hledger*/hledger*.1) - - [ ] info (hledger*/hledger*.info) - - [ ] text (hledger*/hledger*.txt) - `./Shake manuals -c` - - [ ] embedded tldr pages synced with upstream (doc/tldr/*) - `just tldr-diff` - - [ ] embedded asciinema demos (hledger/embeddedfiles/*.cast) - - [ ] shell completions (hledger/shell-completion/hledger-completion.bash) - `just completions`, commit any changes - - [ ] changelogs x 5 (**/CHANGES.md) - `just changelogs [-c]` - `just changelogs-finalise` -- [ ] release docs and artifacts - - [ ] draft binaries building started - `just relbin` - - [ ] hledger.org html manuals x 3 (site/src/VER/*.md) - `just site-manuals-snapshot VER` - update `site/Makefile`, `site/js/site.js`, `site/hledger.org.caddy` - - [ ] release notes @ hledger.org (doc/relnotes.md) - `just relnotes`, - select & transform with `md-issue-refs`, - uniquify issue refs, - unwrap long lines, - add author github nicks, - commit - - [ ] github binary install docs (doc/ghrelnotes.md) up to date and pre-tested - - [ ] hledger-install/hledger-install.sh up to date and pre-tested - (`./Shake hledger-install-version` ?), - check/update dep versions, - select/test snapshot version - - [ ] Install page (site/src/install.md) up to date and pre-tested - - [ ] draft announcement for chat / mail list (doc/ANNOUNCE) - - [ ] draft announcement for mastodon (doc/ANNOUNCE.masto) - - [ ] release tags - `just reltags` - - [ ] release binaries built from tag - `just relbin` - - [ ] Install page (site/src/install.md) --version examples match release binaries -- [ ] published - - [ ] relevant release branch work cherry-picked to master branch - changelogs - relnotes - hledger-install - announcements - - [ ] all packages uploaded correctly to hackage - `just hackageupload`, check versions, confirm, check all uploads successful - - [ ] master branch pushed to github - - [ ] new manuals published and rendering/redirecting correctly - - [ ] site repo pushed to github - - [ ] main and site repos auto-pulled to hledger.org, site rebuilt - `hledgerorgsh grep release.= /opt/hledger/site/out/js/site.js` - - [ ] https://www.hledger.org/js/site.js showing latest version - `curl -s https://hledger.org/js/site.js | grep release.=` - purge cache at https://dash.cloudflare.com/f629035917dd3b99b1e37ae20c15ff09/hledger.org/caching/configuration - - [ ] default manual urls redirecting to latest version - `hledgerorgsh sh -c 'systemctl stop caddy; systemctl start caddy'` - `curl -sI https://hledger.org/hledger.html | grep location` - - [ ] release branch pushed to github - - [ ] release tags pushed to github - `just reltags-push VER` - - [ ] github draft release with release binaries attached - should be autocreated by the above; if not, - (https://github.com/simonmichael/hledger/releases/new) - `just ghrelnotes-push` - (`just ghruns-download`, too slow) - instead: `just ghruns-open`, download to tmp/, unzip the unix ones - `just ghrelease-upload` - - [ ] github release published - review, - publish - - [ ] install instructions tested and working - - [ ] hledger-install - - [ ] stack - - [ ] cabal - - [ ] source checkout - - [ ] github release > How to install, each platform - - [ ] announced - - [ ] mail list(s) hledger@googlegroups.com, + haskell-cafe@googlegroups.com for major - - [ ] matrix - - [ ] irc - - [ ] mastodon - - [ ] pta forum -- [ ] cleanup and support - - [ ] review/polish relnotes, changelogs - review, edit - propagate - - [ ] new dev tag/versions/man dates in master - `j devtag VER.99` - - [ ]y pta.o project stats updated - - [ ] process notes updated/cleaned - - [ ] monitor/support/handle issues: - [issue tracker](https://github.com/simonmichael/hledger/issues?q=is%3Aopen+is%3Aissue), matrix, irc, mail list, forum, reddit - -After release, it's a good time for: - - - finance repo updates - - hledger.org doc updates - - pta.o doc updates - -## How-to notes - -More detailed procedure notes. - -### General tips - -- Release, or practice releasing, often to improve the process. - -- Use and continually update RELEASING.md. - Document procedures and gotchas to save time and enable automation in future. - -- Also the diagram (RELEASING.canvas, made with Obsidian). - -- But don't document prematurely or in too much detail. - -- Make things a little better each time through: simpler, more reliable, better documented, more automated, easier, faster, cheaper, higher quality. - -- Optionally save this file as RELEASING2.md and update notes there until after release, if it's interfering with git branch switching. - -- Use and update scripts, in `Justfile`, `Shake.hs`, `tools/` etc. - -- Do all releases from a release branch. - -- Update dev changelogs frequently in master. Finalise changelogs in the release branch. Merge back to master after release. - (Related older doc: [CHANGELOGS](CHANGELOGS.md)) - -- All release binaries should be built from the release-tagged commit. - The binaries' --version output should match the release tag and release date. - -- Try to do only full releases including all four main hledger packages; partial releases add complexity. - -- Try to avoid pre-announcing a hard release date. - It will always take more time than you think, - if you go late you might miss your intended date in many timezones, - and there's no point adding unnecessary pressure. - -- The biggest potential time sinks are: - - - reviewing/relearning the process/docs/infrastructure - - updating/improving the process/docs/infrastructure - - preparing changelogs - - building binaries for all platforms - - troubleshooting github workflow issues - - followup work due to release mistakes, bugs in new features, or regressions - -- Hard/risky/intensive tasks should happen without time pressure; - during the final countdown, things should be easy. - -### Check release readiness - -- Any blocking open issues ? -- Any blocking open PRs ? -- Any blocking items on ? -- Any blocking items in personal notes & backlogs ? - -### Prepare release - -In main repo, release branch: -1. Create/switch to release branch, update versions/dates/docs: `just relprep NEW` (single-version releases; for mixed-version releases, take more care) -1. If not the first release in this branch, cherry-pick changes from master: `magit l o ..master` (minor releases) -1. Update shell completions: `make -C hledger/shell-completion` -1. (Could start building/testing/fixing release binaries/workflows/caches here, it takes time: `just relbin`) -1. Update install script: `hledger-install/hledger-install.sh` -1. Update changelogs (`**/CHANGES.md`): `./Shake changelogs`, manually edit, `./Shake changelogs -c` -1. Update release notes (`doc/relnotes*`): `tools/relnotes.hs`, select & transform with `md-issue-refs`, uniquify issue refs, add github nicks to authors, commit\ - *TODO: - tools/relnotes.hs: last release's summary, headings, unwrap long lines - md-issue-refs: uniquify* -1. Update announcements (`doc/ANNOUNCE*`) (major releases) -1. Build/test release binaries: `just relbin`. Troubleshoot/repeat as needed. - -In site repo: -1. Update config in `hledger.org.caddy` (@oldmanpath, @unversionedmanpath, any new redirects) (major releases, usually) -1. [Update online manuals](#release-manuals): bump versions in `site/Makefile`, `site/js/site.js`, `make -C site snapshot-NEW` (major releases) - *TODO: snapshot: don't switch to master, don't discard uncommitted changes, record git hash in commit message, clarify late update procedure* -1. Update install page: `site/src/install.md` -1. Don't push yet. Keep in local branch if needed. - -In main repo, master: -1. Cherry-pick the hledger-install update, and other finished useful updates, from the release branch (maybe not release docs yet): `magit l o LASTREL..REL-branch` -1. [Bump version](#bump-master-to-next-version) in master (major releases) -1. Add a new dev tag on the "bump version to ..." commit (magit `t t REL.99`) - -### Publish release - -In main repo, release branch: -1. Build final release binaries (`just relbin`) and tag the release (`just reltag`) -1. Push release branch & tags (not more than 5 tags at once):\ - `git push github HEAD REL hledger-REL hledger-lib-REL`\ - `git push github HEAD hledger-ui-REL hledger-web-REL`\ - Don't push all tags (don't push the dev tag; if you do, manually delete the corresponding draft release.)\ -1. release.yml creates a draft release when the REL tag is pushed. Check/fix its content. - *TODO: fix release.yml* -1. Publish on hackage (final check): `just hackageupload` -1. Publish github release - -In main repo, master: -1. Push master: `just push` - -In site repo: -1. Push to github (& site): `git push github` or magit `P p` - -In hledger.org [cloudflare caching settings](https://dash.cloudflare.com/f629035917dd3b99b1e37ae20c15ff09/hledger.org/caching/configuration): -1. Custom Purge `https://hledger.org/js/site.js` (major release) - -On hledger.org VPS: (major release, usually) -1. Restart caddy to enable new redirects -1. Test https://hledger.org/hledger.html redirect -1. Test manuals are displaying and highlighting the new version -1. If needed, `make buildall` - -### Announce - -(major releases, others if needed) -1. Update hledger entry at https://plaintextaccounting.org/#pta-apps -1. hledger matrix & irc chats -1. PTA forum -1. hledger mail list (& optionally haskell-cafe) -1. mastodon with #hledger and #plaintextaccounting tags - -### Post release - -1. Cherry-pick any final useful updates from the release branch (eg release docs): `magit l o LASTREL..REL-branch` -1. Add/commit any process updates: `doc/RELEASING.md` -1. Monitor packaging status (stackage, brew, docker, linux, nix etc); keep install page updated -1. Monitor, follow up on issues, especially regressions; keep doc/REGRESSIONS.md updated - -### Up-to-date tools -- Check for consistent stackage snapshot(s) and extra deps used in - stack.yaml, Shake.hs, hledger-install.sh, bin scripts, tools scripts -- Shake binary is up to date - `./Shake.hs` -- `hpack --version` matches the one in `stack --version` - -### Local tests -- `just test` -- `just doctest` -- `just haddocktest` - -### CI tests -- push to a PR, wait for green -- or push to `ci` branch, wait for green at -- or `just push` (pushes to `ci`, then to `master`) - -### Release branch tests -- `just test` -- `stack exec -- hledger --version`, check version, hash, release date, no '+' -- `stack exec -- hledger help | tail`, check version, month matches release - -### Download release binaries -With all platform binary tests green on same commit -- clear out any old `hledger*` from local Downloads dir -- in each github binary job: in safari, right click the artifact, Download linked file -- optional: unpack and save the github binaries for local platform (mac-arm64) -- optional: also build native local binaries from that same commit: `just installallas VER` - -### Update Install page -In site repo: -- update `install.md` - - query-replace OLD -> NEW in - - "current hledger release" - - CI binaries badges/links, including linux-static-arm32v7 if built - - "building from source" - - stack install command - - cabal install command - - query-replace OLD-brightgreen -> OLD-red - - only after release binaries are built (preferably after release is published): - update --version outputs (version, hash, date, but not platform) - - final output line from `hledger test` (run local build and in terminal for normal speed) - - Total count from `make functest` - - preview - - commit: `install: NEW` - -### Update RELEASING.png -- edit RELEASING.canvas in obsidian -- CMD-p > Export as image, don't show logo -- commit - -### Monitor packaging status, update Install page - - homebrew - expect badge to update soon - - docker - expect/merge PR - - nix - expect badge to update after a few days; can check with `make nix-hledger-version` - - linux distros - once in a while, follow the links & search for newer versions, update - -### Update hledger in stackage - -- monitor packaging status in lts and nightly: -- update as needed - -### Update homebrew formula - -Usually automatic; contacts: chenrui, other past packagers, Athas on #haskell - -