diff --git a/CHANGELOGS.md b/CHANGELOGS.md index 550821cf3..8eff2e302 100644 --- a/CHANGELOGS.md +++ b/CHANGELOGS.md @@ -1,8 +1,28 @@ # Changelogs -Always maintain changelogs in master branch (not in release branches). +
+ +
-**Frequently**, especially after merging changes, and before cherry picking into release branch: +## How to prepare changelogs & release notes + +Changelogs: + +- always maintain changelogs in master branch, not in release branches +- ./Shake changelogs +- edit the new changelog items (identify, filter, move to correct changelog, deduplicate, rewrite, sort/group) + +Release notes: + +- add a new TOC entry and section in site/release-notes.md +- copy/rewrite/summarise package changelogs +- note any other items of interest +- list release contributors +- write release summary + +## Frequently + +especially after merging changes, and before cherry picking into release branch: - dry run: `./Shake changelogs -n` - add new changes: `./Shake changelogs` @@ -14,7 +34,10 @@ Always maintain changelogs in master branch (not in release branches). - format ([#ISSUE](https://github.com/simonmichael/hledger/issues/), AUTHOR) on its own line - commit: `./Shake changelogs -c` -**After cherry-picking** changes to a release branch: +## After cherry-picking + +changes to a release branch: + - in the master branch changelogs, move the corresponding changelog items under a pending release heading, creating that when necessary: ``` @@ -29,7 +52,7 @@ Always maintain changelogs in master branch (not in release branches). # LASTVER YYYY-MM-DD ``` -**At release:** +## At release - do final update/edits; check organisation, wording, formatting, issue links - replace "unreleased" with the release date diff --git a/COMMITS.md b/COMMITS.md index 36407d734..7a7efcd20 100644 --- a/COMMITS.md +++ b/COMMITS.md @@ -1,8 +1,12 @@ # COMMITS +
+ +
+ ## When committing / reviewing commits -Follow/encourage the [commit conventions](CONTRIBUTING.html#commit-messages). +Follow/encourage the commit conventions (see below). Here they are in brief: - Commit messages must begin with a prefix, one or more colon-terminated words indicating the [topic](CONTRIBUTING.html#topics). @@ -15,8 +19,97 @@ Here they are in brief: intended audience (which might be users, installers, packagers, and/or developers) what they need to know. -## When committing/pushing/merging: +## When committing / pushing / merging: + - run `bin/commitlint` before push, to check recent commits - or, run it automatically before each commit (`make installcommithook` to configure your local repo) - it also runs in CI on github for pull requests, etc. +## Commit conventions + +Since the 1.23 release cycle, we try to follow certain conventions for commit messages, to + +- encourage considered, focussed, well documented changes +- reduce the cost of code review, maintaining changelogs and release notes, and releasing +- increase our throughput (rate of shipping useful, reliable, documented, maintainable features) + +**hledger commit conventions:** + +1. Commit messages in hledger's main repo follow this pattern: + ``` + type: [optionaltopic:] summary + + [Optional description, more details here when needed.] + ``` + +2. Every top-level commit must have a type prefix, ending with a colon and optional space. + This indicates the change's intended audience and the general type of change. + Here are the current types: + + - **Changes visible to end users** (including users of hledger-web's HTTP API). + These will appear in release notes and changelogs: + + - `feat` - a new feature + - `imp` - an improvement to existing features + - `fix` - a bugfix + + - **Changes affecting packagers, builders, and library users**. + These will appear in changelogs: + + - `cha` - a generic package/lib change. Or, one of these specific types: + - `pkg` - something to do with the haskell packages, dependencies etc. + - `lib` - a change in the package's library API + - ...some other type that seems useful... + + - **Changes interesting only to hledger developers/documentors/debuggers**. + These will usually appear only in the commit history, not in changelogs or release notes: + + - `dev` - a generic developer change. Or, one of these specific types: + - `ref` - refactoring + - `cln` - cleanup + - `doc` - documentation-related + - `test` - tests-related + - `ci` - continuous integration-related + - ...some other type that seems useful... + + There's a bit of ambiguity/overlap between the cha/dev types and topics. + Eg the `doc` type indicates a boring doc change, but there's also a `doc` topic + which might be used for interesting doc changes, as in `feat:doc:...`. TBD. + +3. If this is a "breaking change", introducing a compatibility or + migration issue, the type is followed by `!`, and the issue + and advice to users are included in the description. + This will most often be seen with the end-user types, eg: + `feat!:`, `imp!:`, `fix!:`. + +4. If the first character of the commit message is `;`, this commit + (more precisely, the push ending with this commit) will be excluded + from the usual CI checks. Our CI tends to do a lot of building, so + you can use this to save energy and carbon emissions when pushing + harmless changes. + +5. A topic prefix, and maybe even a subtopic prefix, can be added + before the summary if useful. These are standard prefixes similar + to what I have been using for some time, see [topics](#topics). + They help with readability in the commit history, changelogs and release notes. + +6. Any relevant issues should be mentioned, often parenthesised at + the end of the summary: `(#NNNN)`. + +7. The summary, and description if any, communicate this change's + purpose as clearly as possible to its intended audience: + end users, builders/packagers/library users, or developers/debuggers. + The text (or its first sentence/first paragraphs) should be ready + for use in changelogs/release notes when applicable. + +Crafting good commit messages (and thereby good commits, good change +documentation, easier code review, faster merging) is an art and a +habit. Just use your best judgement and we'll check and polish +as part of CI and code review. Examples will be added here in due course. + +Related: + +- +- +- -> Commit messages + diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 2df36587f..d53710939 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -4,6 +4,10 @@ +This is a collection of old developer docs on all topics. +Gradually, these are moving to separate files/pages +and this doc is becoming a focussed guide for new contributors. + ## Quick Links | | | @@ -503,211 +507,6 @@ Relevant tools include: [log](https://ircbrowse.net/day/hledger/2015/10/11) --> -## Commit messages - -Since the 1.23 release cycle, we try to follow certain conventions for commit messages, to - -- encourage considered, focussed, well documented changes -- reduce the cost of code review, maintaining changelogs and release notes, and releasing -- increase our throughput (rate of shipping useful, reliable, documented, maintainable features) - -**hledger commit conventions:** - -1. Commit messages in hledger's main repo follow this pattern: - ``` - type: [optionaltopic:] summary - - [Optional description, more details here when needed.] - ``` - -2. Every top-level commit must have a type prefix, ending with a colon and optional space. - This indicates the change's intended audience and the general type of change. - Here are the current types: - - - **Changes visible to end users** (including users of hledger-web's HTTP API). - These will appear in release notes and changelogs: - - - `feat` - a new feature - - `imp` - an improvement to existing features - - `fix` - a bugfix - - - **Changes affecting packagers, builders, and library users**. - These will appear in changelogs: - - - `cha` - a generic package/lib change. Or, one of these specific types: - - `pkg` - something to do with the haskell packages, dependencies etc. - - `lib` - a change in the package's library API - - ...some other type that seems useful... - - - **Changes interesting only to hledger developers/documentors/debuggers**. - These will usually appear only in the commit history, not in changelogs or release notes: - - - `dev` - a generic developer change. Or, one of these specific types: - - `ref` - refactoring - - `cln` - cleanup - - `doc` - documentation-related - - `test` - tests-related - - `ci` - continuous integration-related - - ...some other type that seems useful... - - There's a bit of ambiguity/overlap between the cha/dev types and topics. - Eg the `doc` type indicates a boring doc change, but there's also a `doc` topic - which might be used for interesting doc changes, as in `feat:doc:...`. TBD. - -3. If this is a "breaking change", introducing a compatibility or - migration issue, the type is followed by `!`, and the issue - and advice to users are included in the description. - This will most often be seen with the end-user types, eg: - `feat!:`, `imp!:`, `fix!:`. - -4. If the first character of the commit message is `;`, this commit - (more precisely, the push ending with this commit) will be excluded - from the usual CI checks. Our CI tends to do a lot of building, so - you can use this to save energy and carbon emissions when pushing - harmless changes. - -5. A topic prefix, and maybe even a subtopic prefix, can be added - before the summary if useful. These are standard prefixes similar - to what I have been using for some time, see [topics](#topics). - They help with readability in the commit history, changelogs and release notes. - -6. Any relevant issues should be mentioned, often parenthesised at - the end of the summary: `(#NNNN)`. - -7. The summary, and description if any, communicate this change's - purpose as clearly as possible to its intended audience: - end users, builders/packagers/library users, or developers/debuggers. - The text (or its first sentence/first paragraphs) should be ready - for use in changelogs/release notes when applicable. - -Crafting good commit messages (and thereby good commits, good change -documentation, easier code review, faster merging) is an art and a -habit. Just use your best judgement and we'll check and polish -as part of CI and code review. Examples will be added here in due course. - -Related: - -- -- -- -> Commit messages - -## Pull requests - -Most contributed hledger code (and some of the project maintainer's code) -is submitted and reviewed via Github pull requests. -Here are some tips for contributing PRs to hledger. - -### Code review is important - -We aim to improve and sustain hledger's quality and maintainability over the long term. - -Many PRs, especially small ones, and even some big ones, can be merged quickly. -We love merging good PRs quickly. - -Some bigger or more risky PRs can require substantial review, discussion, changes, or re-submission. -Sometimes this is a bigger task than the coding. -Much valuable design, quality control, and knowledge sharing happens at this time. -Some PRs get rejected, but their discussion and exploration can still be a useful contribution. -We very much want to avoid wasted work, but it occasionally happens. -Our process is evolving and imperfect. -All of this is normal. - -We hope you'll see it as a golden opportunity to collaborate with experts, -share and receive knowledge, refine your design/documentation/code, -and practice real-world development and communication skills. -Patience and persistence pays. - -### The pull request - -A PR should have a clear purpose, documented in its description. Mention any #ISSUENOs addressed. - -Don't tackle too much at once. -Smaller/more focussed PRs can be reviewed quicker and accepted (or rejected) quicker. - -Consider showing a draft of documentation first (more on this below). - -### The commit(s) - -Commits should be easy to review. -Ideally each commit is complete, and has a single clear purpose, -which should be documented in the summary (and long description, if needed). -\#ISSUENOs can be mentioned in summary/description too when appropriate. - -Within the above constraint, fewer, larger commits are preferred. - -Keep in mind that commit messages are valuable documentation -for future developers and troubleshooters. -They are also the starting point for package changelogs and hledger release notes. -High-quality commit messages makes the release process quicker, and the resulting docs better. - -User-impacting commits should mention the user-visible changes, -and be described in user-relevant language. -Library-user-impacting commits, eg API changes, ideally will also -be called out, and can described in more technical language. -Commits affecting hledger internals are less important, -but you may notice some adhoc conventions if you browse the history. -In particular, you can optionally prefix the summary with short component codes (cf [Issues](#issues)) -to facilitate history reading and changelog/release note production. - -Rewrite and force-push your commits freely (rebase -i, push -f) to clean them up. -Unless we decide to squash the PR into one commit, -your commits will become part of hledger's history "for all time", -so think about future developers trying to understand them, git bisect, etc. - -Rebase your commits against latest master for easiest review. Especially if they start to conflict. - -We like to use some conventions in commit messages when it makes sense. These aren't mandatory, but appreciated: - -- prepend a [topic](#topics) prefix, eg `cli: ` or `journal: `, for clarity and to help with changelog production -- prepend a semicolon (`;`) to indicate commits that need not be mentioned in changelogs/release notes (as in the Emacs project) -- append a final `[ci skip]` line to indicate commits that need not trigger a CI build, to reduce carbon emissions from Travis. - -### The docs - -PRs should include appropriate updates to reference documentation, unless otherwise agreed. -Typically this means the manual source files (hledger*/hledger*.m4.md). -It can also involve -command line option names and descriptions, -other --help output, -hledger's commands list, -hledger-ui's help dialog, -hledger-web's help dialog, -etc. -Sometimes it means the developer docs, at least the ones in the main repo (READMEs). - -Reviewers can understand your PR more efficiently once proposed doc changes are provided, and may postpone it otherwise. -We are happy to help with the docs if needed - just ask. - -Updating rendered manuals (hledger.{1,info,txt,md,html}) is not required, and probably best avoided to reduce conflicts. -Updating other docs such as tutorials, how-tos, examples, or screenshots is not required, -though it's welcome (may be in a different repo). - -### Documentation first - -hledger follows documentation-driven design. -It is in fact highly effective, and highly recommended, -to write the new docs (help text/user manual/haddocks/developer README..) before writing any code. -You can share a rough draft on IRC, on the mail list, in an issue comment, -or in a "WIP" PR starting with just the proposed docs commit. - -This is often the quickest road to getting something merged into hledger. -hledger's many parts interact in surprisingly complex ways. -The documentation-driven working style lets us discuss, clarify and reach a good-enough consensus economically, -after which coding/review/acceptance can go quicker. - - -### Related ideas - -[Neil Mitchell’s Blog - The One PR Per Day Rule](https://neilmitchell.blogspot.com/2019/06/the-one-pr-per-day-rule.html) - - ## Tests About testing in the hledger project, as of 201809. @@ -1261,114 +1060,6 @@ Project documentation lives in a number of places: - `doc/notes.org` has some old developer notes - developer reports (profiles, benchmarks, coverage..) in doc/profs, sometimes published at hledger.org/profs -How to prepare changelogs & release notes - -Changelogs: - -- ./Shake changelogs -- edit the new changelog items (identify, filter, move to correct changelog, deduplicate, rewrite, sort/group) - -Release notes: - -- add a new TOC entry and section in site/release-notes.md -- copy/rewrite/summarise package changelogs -- note any other items of interest -- list release contributors -- write release summary - - -## Issues - -The hledger project\'s issue tracker is on github. It contains: - -- BUG issues - failures in some part of the hledger project (the main - hledger packages, docs, website..) -- WISH issues - feature proposals, enhancement requests -- uncategorised issues - we don\'t know what these are yet -- pull requests - proposed changes to code and docs - -Issues are also labelled according to their [topics](#topics), for organisation. - -Some loose conventions: - -- In bug titles, mention the hledger version in which the bug first appeared - (and avoid mentioning version numbers otherwise). - This allows searches like - [new issues in 1.22](https://github.com/simonmichael/hledger/issues?q=in%3Atitle+1.22+) - and - [regressions in 1.22](https://github.com/simonmichael/hledger/issues?q=in%3Atitle+1.22+label%3Aregression%21) - - -### Issue Urls - -- - show open BUG issues -- - show open WISH issues -- - show all issues, open or closed -- - show open pull requests -- - create a new issue - -### Labels - -Labels are used to categorise: - -- the issue\'s type: \"A BUG\" or \"A WISH\", in shades of red (The A - makes it appear as first label) -- relevant subsystems/topics, in light blue. More about this below. -- relevant platforms, in light purple -- resolution if not fixed: - \"closed:cant-reproduce/duplicate/invalid/wont-fix\", in dark grey -- \"bounty\", in bright yellow: issues with bountysource funding -- \"easy?\", in dim yellow: issues which are probably relatively easy - to fix -- \"imported\" etc., in white: miscellaneous information - -### Topics - -Short topic names, corresponding to hledger commands, input formats, output formats and other common themes, -are used to organise things in the hledger project. In particular, - -- They are used as space saving descriptive prefixes for [commit messages](#commit-messages) -- They can be used as prefixes for issue/PR titles -- Issues and PRs are labelled with them (the light blue labels). - -A more or less complete list can be seen at [open issues](#open-issues) -or in the issue tracker's labels list. - -### Custodians - -If you are interested in helping with a particular component for a -while, please add yourself as a custodian in Open Issues table above. -A custodian\'s job is to help manage the issues, rally the troops, and -drive the open issue count towards zero. The more custodians, the -better! By dividing up the work this way, we can scale and make forward -progress. - -### Milestones and Projects - -Milestones are used a little bit to plan releases. In 2017 we -experimented with projects, but in 2018 milestones are in favour again.. - -### Estimates - -You might see some experiments in estimate tracking, where some issue -names might have a suffix noting estimated and spent time. Basic format: -\[ESTIMATEDTOTALTASKTIME\|TIMESPENTSOFAR\]. Examples: \`\`\` \[2\] two -hours estimated, no time spent \[..\] half an hour estimated (a dot is -\~a quarter hour, as in timedot format) \[1d\] one day estimated (a day -is \~4 hours) \[1w\] one week estimated (a week is \~5 days or \~20 -hours) \[3\|2\] three hours estimated, about two hours spent so far -\[1\|1w\|2d\] first estimate one hour, second estimate one week, about -two days spent so far \`\`\` Estimates are always for the total time -cost (not time remaining). Estimates are not usually changed, a new -estimate is added instead. Numbers are very approximate, but better than -nothing. - -### Trello - -The [trello board](http://trello.hledger.org) (trello.hledger.org) is an -old collection of wishlist items. This should probably be considered -deprecated. - ## Funding My vision for the hledger project has always been for it to be "accountable" and "self-sustaining", possibly through new forms of incentivisation. @@ -1418,230 +1109,19 @@ and help us to experiment. You are encouraged to claim your bounties, though you can also choose to transfer them to a new issue of your choice. +## Commit messages + +See [COMMITS](COMMITS.html). + +## Issues + +See [ISSUES](ISSUES.html). + +## Pull Requests + +See [PULLREQUESTS](PULLREQUESTS.html). + ## Developer workflows -### Get developer tools +See [WORKFLOWS](WORKFLOWS.html). -Ensure [`stack`](https://haskell-lang.org/get-started) is installed -(or if you’re a [cabal](https://www.haskell.org/cabal/) expert, feel free to use that.) - -Ensure [`git`](https://git-scm.com) is installed. On Windows, it comes with stack. - -Here are some useful optional tools: - -- [GNU Make](https://www.gnu.org/software/make): to use the convenient [Make rules](#make). -- [`entr`](https://www.entrproject.org/) runs arbitrary commands when files change. -- [`ghcid`](https://hackage.haskell.org/package/ghcid) gives real-time GHC feedback as you make code changes. -- [`shelltestrunner`](https://hackage.haskell.org/package/shelltestrunner) runs hledger's functional tests. -- [`quickbench`](https://hackage.haskell.org/package/quickbench) measures and reports time taken by commands. -- [`hasktags`](https://hackage.haskell.org/package/hasktags) generates tag files for quick code navigation in editors like Emacs and vim. -- For browsing and editing Haskell code, popular tools include: Emacs, Vim, IDEA, VS Code, Atom.. - -Eg: - - stack install ghcid shelltestrunner hasktags - git clone https://github.com/simonmichael/quickbench; cd quickbench; stack install # must run in source dir - -### Get the code - - git clone https://github.com/simonmichael/hledger - cd hledger - -### Review code - -- review and discuss new [pull requests](http://prs.hledger.org) and commits on github -- build hledger and test the latest changes in your own repo -- read the existing [code docs and source](#quick-links) -- send feedback or discuss via [IRC or mail list](index.html#helpfeedback) - -### Build in place - -See also https://hledger.org/download.html#c.-build-the-development-version . - - stack build # hledger hledger-ui ... - -This fetches the required GHC version and haskell dependencies from the default stackage snapshot (configured in `stack.yaml`), -then builds all hledger packages. -This can take a while! To save time, you can build individual packages, eg just the CLI and TUI. - -Note stack does not fetch C libraries such as curses or terminfo, which you might need to install yourself, using your system's package manager. -In case of trouble, see [download](/download.html#link-errors). - -If you want to use an older snapshot/GHC for some reason, specify one of the older stack-*.yaml files: - - stack --stack-yaml stack8.2.yaml build - -### Run in place - - stack exec -- hledger # ARGS... - stack exec -- hledger-ui # ARGS... - stack exec -- which hledger - -### Build and install - -This builds and also copies the hledger executables to `~/.local/bin` or the Windows equivalent -(which you should [add to your `$PATH`](/download.html#b)). - - stack install # hledger hledger-ui ... - -### Run package tests - -Runs any HUnit/doctest/easytest tests defined by each hledger package. - - stack test # hledger ... - -### Run package benchmarks - -Runs any performance reports defined by each hledger package. - - stack bench # hledger ... - -### Run quickbench benchmarks - -Times the end-user commands in `bench.sh` using quickbench. - - make bench - -### Run functional tests - -Runs the shelltestrunner tests defined in hledger/test/, which test the hledger CLI. - - make functest - -### Run haddock tests - -Checks for anything that would break haddock doc generation. - - make haddocktest - -Checks for the unit-tests embedded in documentation. - - make doctest - -### Simulate Travis tests - -Locally runs tests similar to what we run on Travis CI. - - make travistest - -### Test with all supported GHC versions/stackage snapshots - - make allsnapshotstest - -### Use GHCI - -GHCI is GHC's [REPL](https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop), -useful for exploring and calling code interactively. - -If you try to run GHCI (or things based on it, like ghcid) -right after cloning the hledger repo, you might see an error about CPP macros, eg like -[on #961](https://github.com/simonmichael/hledger/issues/961#issuecomment-459283412). -To fix this, build the hledger packages once, eg `stack build`. - (Or `stack build hledger` might be enough, depending what you are doing.) - -#### Get a GHCI prompt for hledger-lib: - - cd hledger-lib; stack ghci hledger-lib - -Changing into the package directory isn't actually needed, but it -enables a custom .ghci script which sets a more useful short prompt. - -#### Get a GHCI prompt for hledger: - - cd hledger; stack ghci hledger - -#### Get a GHCI prompt for hledger-ui: - - cd hledger-ui; stack ghci hledger-ui - -#### Get a GHCI prompt for hledger-web: - - cd hledger-web; stack ghci hledger-web - -hledger-web also needs to find some things in its current directory (like the static/ directory). -This normally just works, if not please [send details](https://github.com/simonmichael/hledger/issues/274). - -### Add a test - -- identify what to test -- choose the test type: unit ? functional ? benchmark ? -- currently expected to pass or fail ? -- figure out where it goes -- write test, verify expected result -- get it committed - -### Fix a bug or add a feature - -- research, discuss, validate the issue/feature on irc/list/bug tracker -- look for related tests, run the tests and check they are passing -- add a test ? -- develop a patch -- include any related issue numbers in the patch name, eg: "fix for blah blah (#NNN)" -- get it committed - -### Get your changes accepted - -Follow the usual github workflow: - -- fork the main hledger repo on github, -- git clone it to your local machine, -- git commit, after (?) pulling and merging the latest upstream changes -- git push back to github, -- open a pull request on github, -- follow up on any discussion there. - -If you're new to this process, [help.github.com](https://help.github.com) may be useful. - -### Add yourself to the contributor list - -- after getting something into the master branch, read and sign the [contributor list & agreement](https://hledger.org/contributors.html). Or, [ask](index.html#help-feedback) to be added. -- give yourself a high five! - -### Work on docs - -Most docs tasks are handled by [Shake](#shake). - -#### List Shake rules: - - ./Shake - -#### Generate man/info/txt manuals (in hledger*/) and embed in hledger executables: - - ./Shake manuals - stack build - -#### Generate html manuals and the hledger website (in site/_site/): - - ./Shake website - -#### To remove all files generated by Shake: - - ./Shake Clean - -### Use ghcid for watching GHC/GHCI - -[ghcid](https://hackage.haskell.org/package/ghcid) is the most reliable and fastest way to see GHC's feedback, and optionally run tests or a GHCI command, as you edit. We run it via make, for convenience and to watch multiple packages rather than just one. Run `make help-ghcid` to list related rules. - -#### Watch for compile errors in hledger-lib and hledger: - - make ghcid - -#### Watch compile errors and the output of some hledger command: - - ghcid -c 'make ghci' -T ':main -f a.j bal --budget -N' - -### Use --file-watch for watching stack - -stack's --file-watch flag will re-run build/test/bench when source files or package.yaml/cabal files change. Eg: - - stack test hledger --file-watch - -If you find that adding --fast makes this any faster, please update this. - -### Use entr for watching arbitrary commands - -[entr](https://entrproject.org/) is the most robust cross-platform tool for watching files and running a command when they change. Note its first argument must be an executable program, to run a shell command or multiple commands use `bash -c "..."`. - -#### Rerun a single functional test as you change it: - - ls hledger/test/budget/budget.test | entr bash -c 'clear; COLUMNS=80 stack exec -- shelltest --execdir hledger/test/budget/budget.test -i12' diff --git a/ISSUES.md b/ISSUES.md new file mode 100644 index 000000000..f30b31a6b --- /dev/null +++ b/ISSUES.md @@ -0,0 +1,96 @@ +# Issues + +
+ +
+ +The hledger project\'s issue tracker is on github. It contains: + +- BUG issues - failures in some part of the hledger project (the main + hledger packages, docs, website..) +- WISH issues - feature proposals, enhancement requests +- uncategorised issues - we don\'t know what these are yet +- pull requests - proposed changes to code and docs + +Issues are also labelled according to their [topics](#topics), for organisation. + +Some loose conventions: + +- In bug titles, mention the hledger version in which the bug first appeared + (and avoid mentioning version numbers otherwise). + This allows searches like + [new issues in 1.22](https://github.com/simonmichael/hledger/issues?q=in%3Atitle+1.22+) + and + [regressions in 1.22](https://github.com/simonmichael/hledger/issues?q=in%3Atitle+1.22+label%3Aregression%21) + + +## Issue Urls + +- - show open BUG issues +- - show open WISH issues +- - show all issues, open or closed +- - show open pull requests +- - create a new issue + +## Labels + +Labels are used to categorise: + +- the issue\'s type: \"A BUG\" or \"A WISH\", in shades of red (The A + makes it appear as first label) +- relevant subsystems/topics, in light blue. More about this below. +- relevant platforms, in light purple +- resolution if not fixed: + \"closed:cant-reproduce/duplicate/invalid/wont-fix\", in dark grey +- \"bounty\", in bright yellow: issues with bountysource funding +- \"easy?\", in dim yellow: issues which are probably relatively easy + to fix +- \"imported\" etc., in white: miscellaneous information + +## Topics + +Short topic names, corresponding to hledger commands, input formats, output formats and other common themes, +are used to organise things in the hledger project. In particular, + +- They are used as space saving descriptive prefixes for [commit messages](#commit-messages) +- They can be used as prefixes for issue/PR titles +- Issues and PRs are labelled with them (the light blue labels). + +A more or less complete list can be seen at [open issues](#open-issues) +or in the issue tracker's labels list. + +## Custodians + +If you are interested in helping with a particular component for a +while, please add yourself as a custodian in Open Issues table above. +A custodian\'s job is to help manage the issues, rally the troops, and +drive the open issue count towards zero. The more custodians, the +better! By dividing up the work this way, we can scale and make forward +progress. + +## Milestones and Projects + +Milestones are used a little bit to plan releases. In 2017 we +experimented with projects, but in 2018 milestones are in favour again.. + +## Estimates + +You might see some experiments in estimate tracking, where some issue +names might have a suffix noting estimated and spent time. Basic format: +\[ESTIMATEDTOTALTASKTIME\|TIMESPENTSOFAR\]. Examples: \`\`\` \[2\] two +hours estimated, no time spent \[..\] half an hour estimated (a dot is +\~a quarter hour, as in timedot format) \[1d\] one day estimated (a day +is \~4 hours) \[1w\] one week estimated (a week is \~5 days or \~20 +hours) \[3\|2\] three hours estimated, about two hours spent so far +\[1\|1w\|2d\] first estimate one hour, second estimate one week, about +two days spent so far \`\`\` Estimates are always for the total time +cost (not time remaining). Estimates are not usually changed, a new +estimate is added instead. Numbers are very approximate, but better than +nothing. + +## Trello + +The [trello board](http://trello.hledger.org) (trello.hledger.org) is an +old collection of wishlist items. This should probably be considered +deprecated. + diff --git a/PULLREQUESTS.md b/PULLREQUESTS.md new file mode 100644 index 000000000..908ea2146 --- /dev/null +++ b/PULLREQUESTS.md @@ -0,0 +1,117 @@ +# Pull requests + +Most contributed hledger code (and some of the project maintainer's code) +is submitted and reviewed via Github pull requests. +Here are some tips for contributing PRs to hledger. + +## Code review is important + +We aim to improve and sustain hledger's quality and maintainability over the long term. + +Many PRs, especially small ones, and even some big ones, can be merged quickly. +We love merging good PRs quickly. + +Some bigger or more risky PRs can require substantial review, discussion, changes, or re-submission. +Sometimes this is a bigger task than the coding. +Much valuable design, quality control, and knowledge sharing happens at this time. +Some PRs get rejected, but their discussion and exploration can still be a useful contribution. +We very much want to avoid wasted work, but it occasionally happens. +Our process is evolving and imperfect. +All of this is normal. + +We hope you'll see it as a golden opportunity to collaborate with experts, +share and receive knowledge, refine your design/documentation/code, +and practice real-world development and communication skills. +Patience and persistence pays. + +## The pull request + +A PR should have a clear purpose, documented in its description. Mention any #ISSUENOs addressed. + +Don't tackle too much at once. +Smaller/more focussed PRs can be reviewed quicker and accepted (or rejected) quicker. + +Consider showing a draft of documentation first (more on this below). + +## The commit(s) + +Commits should be easy to review. +Ideally each commit is complete, and has a single clear purpose, +which should be documented in the summary (and long description, if needed). +\#ISSUENOs can be mentioned in summary/description too when appropriate. + +Within the above constraint, fewer, larger commits are preferred. + +Keep in mind that commit messages are valuable documentation +for future developers and troubleshooters. +They are also the starting point for package changelogs and hledger release notes. +High-quality commit messages makes the release process quicker, and the resulting docs better. + +User-impacting commits should mention the user-visible changes, +and be described in user-relevant language. +Library-user-impacting commits, eg API changes, ideally will also +be called out, and can described in more technical language. +Commits affecting hledger internals are less important, +but you may notice some adhoc conventions if you browse the history. +In particular, you can optionally prefix the summary with short component codes (cf [Issues](#issues)) +to facilitate history reading and changelog/release note production. + +Rewrite and force-push your commits freely (rebase -i, push -f) to clean them up. +Unless we decide to squash the PR into one commit, +your commits will become part of hledger's history "for all time", +so think about future developers trying to understand them, git bisect, etc. + +Rebase your commits against latest master for easiest review. Especially if they start to conflict. + +We like to use some conventions in commit messages when it makes sense. These aren't mandatory, but appreciated: + +- prepend a [topic](#topics) prefix, eg `cli: ` or `journal: `, for clarity and to help with changelog production +- prepend a semicolon (`;`) to indicate commits that need not be mentioned in changelogs/release notes (as in the Emacs project) +- append a final `[ci skip]` line to indicate commits that need not trigger a CI build, to reduce carbon emissions from Travis. + +## The docs + +PRs should include appropriate updates to reference documentation, unless otherwise agreed. +Typically this means the manual source files (hledger*/hledger*.m4.md). +It can also involve +command line option names and descriptions, +other --help output, +hledger's commands list, +hledger-ui's help dialog, +hledger-web's help dialog, +etc. +Sometimes it means the developer docs, at least the ones in the main repo (READMEs). + +Reviewers can understand your PR more efficiently once proposed doc changes are provided, and may postpone it otherwise. +We are happy to help with the docs if needed - just ask. + +Updating rendered manuals (hledger.{1,info,txt,md,html}) is not required, and probably best avoided to reduce conflicts. +Updating other docs such as tutorials, how-tos, examples, or screenshots is not required, +though it's welcome (may be in a different repo). + +## Documentation first + +hledger follows documentation-driven design. +It is in fact highly effective, and highly recommended, +to write the new docs (help text/user manual/haddocks/developer README..) before writing any code. +You can share a rough draft on IRC, on the mail list, in an issue comment, +or in a "WIP" PR starting with just the proposed docs commit. + +This is often the quickest road to getting something merged into hledger. +hledger's many parts interact in surprisingly complex ways. +The documentation-driven working style lets us discuss, clarify and reach a good-enough consensus economically, +after which coding/review/acceptance can go quicker. + + +## Related ideas + +[Neil Mitchell’s Blog - The One PR Per Day Rule](https://neilmitchell.blogspot.com/2019/06/the-one-pr-per-day-rule.html) + + diff --git a/WORKFLOWS.md b/WORKFLOWS.md new file mode 100644 index 000000000..f450e64c6 --- /dev/null +++ b/WORKFLOWS.md @@ -0,0 +1,231 @@ +# Developer workflows + +
+ +
+ +## Get developer tools + +Ensure [`stack`](https://haskell-lang.org/get-started) is installed +(or if you’re a [cabal](https://www.haskell.org/cabal/) expert, feel free to use that.) + +Ensure [`git`](https://git-scm.com) is installed. On Windows, it comes with stack. + +Here are some useful optional tools: + +- [GNU Make](https://www.gnu.org/software/make): to use the convenient [Make rules](#make). +- [`entr`](https://www.entrproject.org/) runs arbitrary commands when files change. +- [`ghcid`](https://hackage.haskell.org/package/ghcid) gives real-time GHC feedback as you make code changes. +- [`shelltestrunner`](https://hackage.haskell.org/package/shelltestrunner) runs hledger's functional tests. +- [`quickbench`](https://hackage.haskell.org/package/quickbench) measures and reports time taken by commands. +- [`hasktags`](https://hackage.haskell.org/package/hasktags) generates tag files for quick code navigation in editors like Emacs and vim. +- For browsing and editing Haskell code, popular tools include: Emacs, Vim, IDEA, VS Code, Atom.. + +Eg: + + stack install ghcid shelltestrunner hasktags + git clone https://github.com/simonmichael/quickbench; cd quickbench; stack install # must run in source dir + +## Get the code + + git clone https://github.com/simonmichael/hledger + cd hledger + +## Review code + +- review and discuss new [pull requests](http://prs.hledger.org) and commits on github +- build hledger and test the latest changes in your own repo +- read the existing [code docs and source](#quick-links) +- send feedback or discuss via [IRC or mail list](index.html#helpfeedback) + +## Build in place + +See also https://hledger.org/download.html#c.-build-the-development-version . + + stack build # hledger hledger-ui ... + +This fetches the required GHC version and haskell dependencies from the default stackage snapshot (configured in `stack.yaml`), +then builds all hledger packages. +This can take a while! To save time, you can build individual packages, eg just the CLI and TUI. + +Note stack does not fetch C libraries such as curses or terminfo, which you might need to install yourself, using your system's package manager. +In case of trouble, see [download](/download.html#link-errors). + +If you want to use an older snapshot/GHC for some reason, specify one of the older stack-*.yaml files: + + stack --stack-yaml stack8.2.yaml build + +## Run in place + + stack exec -- hledger # ARGS... + stack exec -- hledger-ui # ARGS... + stack exec -- which hledger + +## Build and install + +This builds and also copies the hledger executables to `~/.local/bin` or the Windows equivalent +(which you should [add to your `$PATH`](/download.html#b)). + + stack install # hledger hledger-ui ... + +## Run package tests + +Runs any HUnit/doctest/easytest tests defined by each hledger package. + + stack test # hledger ... + +## Run package benchmarks + +Runs any performance reports defined by each hledger package. + + stack bench # hledger ... + +## Run quickbench benchmarks + +Times the end-user commands in `bench.sh` using quickbench. + + make bench + +## Run functional tests + +Runs the shelltestrunner tests defined in hledger/test/, which test the hledger CLI. + + make functest + +## Run haddock tests + +Checks for anything that would break haddock doc generation. + + make haddocktest + +Checks for the unit-tests embedded in documentation. + + make doctest + +## Simulate Travis tests + +Locally runs tests similar to what we run on Travis CI. + + make travistest + +## Test with all supported GHC versions/stackage snapshots + + make allsnapshotstest + +## Use GHCI + +GHCI is GHC's [REPL](https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop), +useful for exploring and calling code interactively. + +If you try to run GHCI (or things based on it, like ghcid) +right after cloning the hledger repo, you might see an error about CPP macros, eg like +[on #961](https://github.com/simonmichael/hledger/issues/961#issuecomment-459283412). +To fix this, build the hledger packages once, eg `stack build`. + (Or `stack build hledger` might be enough, depending what you are doing.) + +### Get a GHCI prompt for hledger-lib: + + cd hledger-lib; stack ghci hledger-lib + +Changing into the package directory isn't actually needed, but it +enables a custom .ghci script which sets a more useful short prompt. + +### Get a GHCI prompt for hledger: + + cd hledger; stack ghci hledger + +### Get a GHCI prompt for hledger-ui: + + cd hledger-ui; stack ghci hledger-ui + +### Get a GHCI prompt for hledger-web: + + cd hledger-web; stack ghci hledger-web + +hledger-web also needs to find some things in its current directory (like the static/ directory). +This normally just works, if not please [send details](https://github.com/simonmichael/hledger/issues/274). + +## Add a test + +- identify what to test +- choose the test type: unit ? functional ? benchmark ? +- currently expected to pass or fail ? +- figure out where it goes +- write test, verify expected result +- get it committed + +## Fix a bug or add a feature + +- research, discuss, validate the issue/feature on irc/list/bug tracker +- look for related tests, run the tests and check they are passing +- add a test ? +- develop a patch +- include any related issue numbers in the patch name, eg: "fix for blah blah (#NNN)" +- get it committed + +## Get your changes accepted + +Follow the usual github workflow: + +- fork the main hledger repo on github, +- git clone it to your local machine, +- git commit, after (?) pulling and merging the latest upstream changes +- git push back to github, +- open a pull request on github, +- follow up on any discussion there. + +If you're new to this process, [help.github.com](https://help.github.com) may be useful. + +## Add yourself to the contributor list + +- after getting something into the master branch, read and sign the [contributor list & agreement](https://hledger.org/contributors.html). Or, [ask](index.html#help-feedback) to be added. +- give yourself a high five! + +## Work on docs + +Most docs tasks are handled by [Shake](#shake). + +### List Shake rules: + + ./Shake + +### Generate man/info/txt manuals (in hledger*/) and embed in hledger executables: + + ./Shake manuals + stack build + +### Generate html manuals and the hledger website (in site/_site/): + + ./Shake website + +### To remove all files generated by Shake: + + ./Shake Clean + +## Use ghcid for watching GHC/GHCI + +[ghcid](https://hackage.haskell.org/package/ghcid) is the most reliable and fastest way to see GHC's feedback, and optionally run tests or a GHCI command, as you edit. We run it via make, for convenience and to watch multiple packages rather than just one. Run `make help-ghcid` to list related rules. + +### Watch for compile errors in hledger-lib and hledger: + + make ghcid + +### Watch compile errors and the output of some hledger command: + + ghcid -c 'make ghci' -T ':main -f a.j bal --budget -N' + +## Use --file-watch for watching stack + +stack's --file-watch flag will re-run build/test/bench when source files or package.yaml/cabal files change. Eg: + + stack test hledger --file-watch + +If you find that adding --fast makes this any faster, please update this. + +## Use entr for watching arbitrary commands + +[entr](https://entrproject.org/) is the most robust cross-platform tool for watching files and running a command when they change. Note its first argument must be an executable program, to run a shell command or multiple commands use `bash -c "..."`. + +### Rerun a single functional test as you change it: + + ls hledger/test/budget/budget.test | entr bash -c 'clear; COLUMNS=80 stack exec -- shelltest --execdir hledger/test/budget/budget.test -i12'