diff --git a/BENCHMARKS.md b/BENCHMARKS.md new file mode 100644 index 000000000..65b681793 --- /dev/null +++ b/BENCHMARKS.md @@ -0,0 +1,152 @@ +# Benchmarks + +Benchmarks are standard performance measurements, +which we define using `bench` declarations in cabal files. +There is [one in hledger.cabal](https://github.com/simonmichael/hledger/blob/master/hledger/hledger.cabal#L228), +with related code and data files in [hledger/bench/](https://github.com/simonmichael/hledger/tree/master/hledger/bench). + +To run the standard hledger benchmark, use `stack bench hledger`. +This installs haskell dependencies (but not system dependencies) and rebuilds as needed, +then runs [hledger/bench/bench.hs](https://github.com/simonmichael/hledger/blob/master/hledger/bench/bench.hs), +which by default shows quick elapsed-time measurements for several operations on a standard data file: + +```shell +$ stack bench hledger +NOTE: the bench command is functionally equivalent to 'build --bench' +... +hledger-0.27: benchmarks +Running 1 benchmarks... +Benchmark bench: RUNNING... +Benchmarking hledger in /Users/simon/src/hledger/hledger with timeit +read bench/10000x1000x10.journal [1.57s] +print [1.29s] +register [1.92s] +balance [0.21s] +stats [0.23s] +Total: 5.22s +Benchmark bench: FINISH +``` + +bench.hs has some other modes, which you can use by compiling and running it directly. +`--criterion` reports more detailed and dependable measurements, but takes longer: + +```shell +$ cd hledger; stack exec -- ghc -ibench bench/bench && bench/bench --criterion +... +Linking bench/bench ... +Benchmarking hledger in /Users/simon/src/hledger/hledger with criterion +benchmarking read bench/10000x1000x10.journal +time 1.414 s (1.234 s .. 1.674 s) + 0.996 R² (0.989 R² .. 1.000 R²) +mean 1.461 s (1.422 s .. 1.497 s) +std dev 59.69 ms (0.0 s .. 62.16 ms) +variance introduced by outliers: 19% (moderately inflated) + +benchmarking print +time 1.323 s (1.279 s .. 1.385 s) + 1.000 R² (0.999 R² .. 1.000 R²) +mean 1.305 s (1.285 s .. 1.316 s) +std dev 17.20 ms (0.0 s .. 19.14 ms) +variance introduced by outliers: 19% (moderately inflated) + +benchmarking register +time 1.995 s (1.883 s .. 2.146 s) + 0.999 R² (0.998 R² .. NaN R²) +mean 1.978 s (1.951 s .. 1.995 s) +std dev 25.09 ms (0.0 s .. 28.26 ms) +variance introduced by outliers: 19% (moderately inflated) + +benchmarking balance +time 251.3 ms (237.6 ms .. 272.4 ms) + 0.998 R² (0.997 R² .. 1.000 R²) +mean 260.4 ms (254.3 ms .. 266.5 ms) +std dev 7.609 ms (3.192 ms .. 9.638 ms) +variance introduced by outliers: 16% (moderately inflated) + +benchmarking stats +time 325.5 ms (299.1 ms .. 347.2 ms) + 0.997 R² (0.985 R² .. 1.000 R²) +mean 329.2 ms (321.5 ms .. 339.6 ms) +std dev 11.08 ms (2.646 ms .. 14.82 ms) +variance introduced by outliers: 16% (moderately inflated) +``` + +`--simplebench` shows a table of elapsed-time measurements for the commands defined in [bench/default.bench](https://github.com/simonmichael/hledger/blob/master/hledger/bench/default.bench). +It can also show the results for multiple h/ledger executables side by side, if you tweak the bench.hs code. +Unlike the other modes, it does not link with the hledger code directly, but runs the "hledger" executable found in $PATH (so ensure that's the one you intend to test). + +```shell +$ cd hledger; stack exec -- ghc -ibench bench/bench && bench/bench --simplebench +Benchmarking /Users/simon/.local/bin/hledger in /Users/simon/src/hledger/hledger with simplebench and shell +Using bench/default.bench +Running 4 tests 1 times with 1 executables at 2015-08-23 16:58:59.128112 UTC: +1: hledger -f bench/10000x1000x10.journal print [3.27s] +1: hledger -f bench/10000x1000x10.journal register [3.65s] +1: hledger -f bench/10000x1000x10.journal balance [2.06s] +1: hledger -f bench/10000x1000x10.journal stats [2.13s] + +Summary (best iteration): + ++-----------------------------------------++---------+ +| || hledger | ++=========================================++=========+ +| -f bench/10000x1000x10.journal print || 3.27 | +| -f bench/10000x1000x10.journal register || 3.65 | +| -f bench/10000x1000x10.journal balance || 2.06 | +| -f bench/10000x1000x10.journal stats || 2.13 | ++-----------------------------------------++---------+ +``` + +bench's --simplebench mode is based on a standalone tool, [tools/simplebench.hs](https://github.com/simonmichael/hledger/blob/master/tools/simplebench.hs). +simplebench.hs is a generic benchmarker of one or more executables (specified on the command line) against one or more sets of command-line arguments (specified in a file). +It has a better command-line interface than bench.hs, so you may find it more convenient +for comparing multiple hledger versions, or hledger and ledger. Eg: + +```shell +$ stack exec -- ghc tools/simplebench +[1 of 1] Compiling Main ( tools/simplebench.hs, tools/simplebench.o ) +Linking tools/simplebench ... +``` +```shell +$ tools/simplebench -h +tools/simplebench -h +simplebench: at least one executable needed +bench [-f testsfile] [-n iterations] [-p precision] executable1 [executable2 ...] + +Run some functional tests with each of the specified executables, +where a test is "zero or more arguments supported by all executables", +and report the best execution times. + + -f testsfile --testsfile=testsfile file containing tests, one per line, default: bench.tests + -n iterations --iterations=iterations number of test iterations to run, default: 2 + -p precision --precision=precision show times with this precision, default: 2 + -v --verbose show intermediate results + -h --help show this help + +Tips: +- executables may have arguments if enclosed in quotes +- tests can be commented out with # +- results are saved in benchresults.{html,txt} +``` +```shell +cd hledger; $ ../tools/simplebench -f bench/default.bench hledger ledger +Using bench/default.bench +Running 4 tests 2 times with 2 executables at 2015-08-24 04:24:37.257068 UTC: + +Summary (best iteration): + ++-----------------------------------------++---------+--------+ +| || hledger | ledger | ++=========================================++=========+========+ +| -f bench/10000x1000x10.journal print || 3.24 | 0.43 | +| -f bench/10000x1000x10.journal register || 3.80 | 3.48 | +| -f bench/10000x1000x10.journal balance || 2.05 | 0.18 | +| -f bench/10000x1000x10.journal stats || 2.10 | 0.19 | ++-----------------------------------------++---------+--------+ +``` + +Finally, for quick, fine-grained performance measurements when troubleshooting or optimising, I use +[dev.hs](https://github.com/simonmichael/hledger/blob/master/dev.hs). + + + diff --git a/CODE.md b/CODE.md new file mode 100644 index 000000000..0875ef186 --- /dev/null +++ b/CODE.md @@ -0,0 +1,211 @@ +# Code + +hledger is a suite of applications, tools and libraries. +The main hledger code repository is [github.com/simonmichael/hledger](https://github.com/simonmichael/hledger) +(shortcut url `code.hledger.org`). +There are also various hledger addons maintained as separate projects with their own repos. + +## hledger packages + +Within the main repo, there are a number of separate cabal packages, +making it easier to pick and choose parts of hledger to install or to package. +They are: + +### hledger-lib + +[package](https://hackage.haskell.org/package/hledger-lib), +[code](https://github.com/simonmichael/hledger/tree/master/hledger-lib) + +Core data models, parsing, standard reports, and utilities. +Most data types are defined in [Hledger.Data.Types](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html), +while functions that operate on them are defined in Hledger.Data.TYPENAME. +Under [Hledger.Read](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Read.hs) +are parsers for the supported input formats. +Data files are parsed into a +[Journal](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Journal), +which contains a list of +[Transactions](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Transaction), +each containing multiple +[Postings](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Posting) +of some +[MixedAmount](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:MixedAmount) +(multiple +single-[CommoditySymbol](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:CommoditySymbol) +[Amounts](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Amount)) +to some +[AccountName](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:AccountName). +When needed, the Journal is further processed to derive a +[Ledger](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Ledger), +which contains summed +[Accounts](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Account). +In [Hledger.Reports](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Reports.html) +there are standard reports, which extract useful data from the Journal or Ledger. + +Here's a diagram of the main data model: + + +diagram + + + + +### hledger + +[package](https://hackage.haskell.org/package/hledger), +[code](https://github.com/simonmichael/hledger/tree/master/hledger), +[manual](https://hledger.org/hledger.html) + +hledger's command line interface, and command line options and utilities for other hledger tools. + +Try tracing the execution of a hledger command: + +1. [Hledger.Cli.Main:main](https://github.com/simonmichael/hledger/blob/master/hledger/Hledger/Cli/Main.hs#L302) +parses the command line to select a command, then +2. gives it to +[Hledger.Cli.Utils:withJournalDo](https://github.com/simonmichael/hledger/blob/master/hledger/Hledger/Cli/Utils.hs#L73), +which runs it after doing all the initial parsing. +3. Parsing code is under +[hledger-lib:Hledger.Read](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Read.hs), +eg [Hledger.Read.JournalReader](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Read/JournalReader.hs). +4. Commands extract useful information from the parsed data model using +[hledger-lib:Hledger.Reports](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Reports), +and +5. render in plain text for console output (or another output format, like CSV). +6. Everything uses the data types and utilities from +[hledger-lib:Hledger.Data](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Data) +and [hledger-lib:Hledger.Utils](https://github.com/simonmichael/hledger/blob/master/hledger-lib/Hledger/Utils.hs). + +### hledger-ui + +[package](https://hackage.haskell.org/package/hledger-ui), +[code](https://github.com/simonmichael/hledger/tree/master/hledger-ui), +[manual](https://hledger.org/hledger-ui.html) + +A terminal interface. + +### hledger-web + +[package](https://hackage.haskell.org/package/hledger-web), +[code](https://github.com/simonmichael/hledger/tree/master/hledger-web), +[manual](https://hledger.org/hledger-web.html) + +A web interface. +hledger-web starts a web server built with the yesod framework, +and (by default) opens a web browser view on it. +It reads the journal file(s) at startup and again whenever they change. +It can also write (append) new transactions to the journal file. + +There are two main views, which can be filtered with +[queries](https://hledger.org/hledger.html#queries): + +- [/journal](https://demo.hledger.org/journal), showing general journal entries (like `hledger print`) + +- [/register](https://demo.hledger.org/register?q=inacct:Expenses:Food), + showing transactions affecting an account (slightly different from + hledger's [register](https://hledger.org/hledger.html#register) command, which shows postings). + +There is also: + +- a sidebar (toggled by pressing `s`) showing the chart of accounts (like `hledger balance`) +- an [add form](https://demo.hledger.org/journal?add=1) for adding new transactions (press `a`) +- a help dialog showing quick help and keybindings (press `h` or click ?) + +Most of the action is in + +- [config/routes](https://github.com/simonmichael/hledger/tree/master/hledger-web/config/routes) +- [templates/default-layout-wrapper.hamlet](https://github.com/simonmichael/hledger/tree/master/hledger-web/templates/default-layout-wrapper.hamlet) +- [Foundation](https://github.com/simonmichael/hledger/tree/master/hledger-web/Foundation.hs) +- [Handler.*](https://github.com/simonmichael/hledger/tree/master/hledger-web/Handler) +- [static/hledger.js](https://github.com/simonmichael/hledger/tree/master/hledger-web/static/hledger.js) +- [static/hledger.css](https://github.com/simonmichael/hledger/tree/master/hledger-web/static/hledger.css) + +Handler module and function names end with R, like the yesod-generated route type they deal with. + +Dynamically generated page content is mostly inline hamlet. +Lucius/Julius files and widgets generally are not used, except for the default layout. + +Here are some ways to run it during development: + +- `yesod devel`: runs in developer mode, rebuilds automatically when config, template, static or haskell files change +(but only files in the hledger-web package): +```shell +$ (cd hledger-web; yesod devel) +``` + +- [yesod-fast-devel](https://hackage.haskell.org/package/yesod-fast-devel) + may be a good alternative, also reloads the browser page + +- `stack ghci`: runs the server in developer mode from GHCI. +Changes to static files like hledger.js will be visible on page reload; +to see other changes, restart it as shown. +```shell +$ (cd hledger-web; stack ghci hledger-web) +hledger-web> :main --serve # restart: ctrl-c, :r, enter, ctrl-p, ctrl-p, enter +``` + +- `make ghci-web`: runs the server in developer mode from GHCI, also +interprets the hledger-lib and hledger packages so that :reload picks +up changes in those packages too: +```shell +$ make ghci-web +ghci> :main --serve +``` +(This rule also creates symbolic links to hledger-web's `config`, `messages`, `static` and `templates` +directories, needed in developer mode, so it can run from the top directory. This may not work on Windows.) + +## Quality + +Relevant tools include: + +- unit tests (HUnit, make unittest) +- functional tests (shelltestrunner, make functest) +- performance tests (simplebench, make bench) +- documentation tests (make haddocktest + manual) +- ui tests (manual) +- installation tests (manual) +- code reviews + +## Code review + +- Code review party 2014/7/21-25: + [discussion](https://thread.gmane.org/gmane.comp.finance.ledger.hledger/1070) +- Dev sprint/party 2015/10/10: + [discussion](https://thread.gmane.org/gmane.comp.finance.ledger.hledger/1254) + + diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d53710939..770ef03ec 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -7,6 +7,9 @@ This is a collection of old developer docs on all topics. Gradually, these are moving to separate files/pages and this doc is becoming a focussed guide for new contributors. +If you are unexpectedly seeing this page after following a link, +the content probably moved to a separate page: +see the [Developer docs](dev.md). ## Quick Links @@ -243,885 +246,3 @@ See [Developer workflows](#developer-workflows). -## Make - -A Makefile is provided to make common developer tasks easy to remember, -and to insulate us a little from the ever-evolving Haskell tools ecosystem. -Using it is entirely optional, but recommended. -You'll need [GNU Make](https://www.gnu.org/software/make) installed. - -The Makefile contains a fair amount of obsolete cruft and needs cleanup. Some tasks (docs, website) are now handled by the [Shake](#shake) file instead. - -The Makefile is self-documenting. Run `make` to see a list of the main make rules: - -```shell -$ make -Makefile:37: -------------------- hledger make rules -------------------- -Makefile:39: make [help] -- list documented rules in this makefile. make -n RULE shows more detail. -Makefile:204: (INSTALLING:) -Makefile:206: make install -- download dependencies and install hledger executables to ~/.local/bin or equivalent (with stack) -Makefile:231: (BUILDING:) -Makefile:235: make build -- download dependencies and build hledger executables (with stack) -Makefile:304: make hledgerdev -- quickly build the hledger executable (with ghc and -DDEVELOPMENT) -... -``` - -To see what a make rule will do without actually doing it, use the `-n` flag: - -```shell -$ make build -n -stack build -``` -```shell -$ make test -n -(stack test \ - && echo pkgtest PASSED) || echo pkgtest FAILED -(stack exec hledger test \ - && echo builtintest PASSED) || echo builtintest FAILED -(COLUMNS=80 PATH=`pwd`/bin:/home/simon/src/hledger/bin:/home/simon/.local/bin:/home/simon/.cabal/bin:/opt/ghc/7.10.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/var/lib/gems/1.9.1/bin stack exec -- shelltest --execdir -- -j16 --hide-successes tests \ - && echo functest PASSED) || echo functest FAILED -``` - - -## Shake - -`Shake.hs` in the top directory complements the Makefile; it is used for some more complex tasks, such as building documentation and the web site. - -Compile it: - - ./Shake.hs # or, make Shake - -See help: - - ./Shake - - -## Code - -hledger is a suite of applications, tools and libraries. -The main hledger code repository is [github.com/simonmichael/hledger](https://github.com/simonmichael/hledger) -(shortcut url `code.hledger.org`). -There are also various hledger addons maintained as separate projects with their own repos. - -### hledger packages - -Within the main repo, there are a number of separate cabal packages, -making it easier to pick and choose parts of hledger to install or to package. -They are: - -#### hledger-lib - -[package](https://hackage.haskell.org/package/hledger-lib), -[code](https://github.com/simonmichael/hledger/tree/master/hledger-lib) - -Core data models, parsing, standard reports, and utilities. -Most data types are defined in [Hledger.Data.Types](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html), -while functions that operate on them are defined in Hledger.Data.TYPENAME. -Under [Hledger.Read](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Read.hs) -are parsers for the supported input formats. -Data files are parsed into a -[Journal](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Journal), -which contains a list of -[Transactions](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Transaction), -each containing multiple -[Postings](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Posting) -of some -[MixedAmount](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:MixedAmount) -(multiple -single-[CommoditySymbol](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:CommoditySymbol) -[Amounts](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Amount)) -to some -[AccountName](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:AccountName). -When needed, the Journal is further processed to derive a -[Ledger](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Ledger), -which contains summed -[Accounts](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Account). -In [Hledger.Reports](https://hackage.haskell.org/package/hledger-lib/docs/Hledger-Reports.html) -there are standard reports, which extract useful data from the Journal or Ledger. - -Here's a diagram of the main data model: - - -diagram - - - - -#### hledger - -[package](https://hackage.haskell.org/package/hledger), -[code](https://github.com/simonmichael/hledger/tree/master/hledger), -[manual](https://hledger.org/hledger.html) - -hledger's command line interface, and command line options and utilities for other hledger tools. - -Try tracing the execution of a hledger command: - -1. [Hledger.Cli.Main:main](https://github.com/simonmichael/hledger/blob/master/hledger/Hledger/Cli/Main.hs#L302) -parses the command line to select a command, then -2. gives it to -[Hledger.Cli.Utils:withJournalDo](https://github.com/simonmichael/hledger/blob/master/hledger/Hledger/Cli/Utils.hs#L73), -which runs it after doing all the initial parsing. -3. Parsing code is under -[hledger-lib:Hledger.Read](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Read.hs), -eg [Hledger.Read.JournalReader](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Read/JournalReader.hs). -4. Commands extract useful information from the parsed data model using -[hledger-lib:Hledger.Reports](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Reports), -and -5. render in plain text for console output (or another output format, like CSV). -6. Everything uses the data types and utilities from -[hledger-lib:Hledger.Data](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Data) -and [hledger-lib:Hledger.Utils](https://github.com/simonmichael/hledger/blob/master/hledger-lib/Hledger/Utils.hs). - -#### hledger-ui - -[package](https://hackage.haskell.org/package/hledger-ui), -[code](https://github.com/simonmichael/hledger/tree/master/hledger-ui), -[manual](https://hledger.org/hledger-ui.html) - -A terminal interface. - -#### hledger-web - -[package](https://hackage.haskell.org/package/hledger-web), -[code](https://github.com/simonmichael/hledger/tree/master/hledger-web), -[manual](https://hledger.org/hledger-web.html) - -A web interface. -hledger-web starts a web server built with the yesod framework, -and (by default) opens a web browser view on it. -It reads the journal file(s) at startup and again whenever they change. -It can also write (append) new transactions to the journal file. - -There are two main views, which can be filtered with -[queries](https://hledger.org/hledger.html#queries): - -- [/journal](https://demo.hledger.org/journal), showing general journal entries (like `hledger print`) - -- [/register](https://demo.hledger.org/register?q=inacct:Expenses:Food), - showing transactions affecting an account (slightly different from - hledger's [register](https://hledger.org/hledger.html#register) command, which shows postings). - -There is also: - -- a sidebar (toggled by pressing `s`) showing the chart of accounts (like `hledger balance`) -- an [add form](https://demo.hledger.org/journal?add=1) for adding new transactions (press `a`) -- a help dialog showing quick help and keybindings (press `h` or click ?) - -Most of the action is in - -- [config/routes](https://github.com/simonmichael/hledger/tree/master/hledger-web/config/routes) -- [templates/default-layout-wrapper.hamlet](https://github.com/simonmichael/hledger/tree/master/hledger-web/templates/default-layout-wrapper.hamlet) -- [Foundation](https://github.com/simonmichael/hledger/tree/master/hledger-web/Foundation.hs) -- [Handler.*](https://github.com/simonmichael/hledger/tree/master/hledger-web/Handler) -- [static/hledger.js](https://github.com/simonmichael/hledger/tree/master/hledger-web/static/hledger.js) -- [static/hledger.css](https://github.com/simonmichael/hledger/tree/master/hledger-web/static/hledger.css) - -Handler module and function names end with R, like the yesod-generated route type they deal with. - -Dynamically generated page content is mostly inline hamlet. -Lucius/Julius files and widgets generally are not used, except for the default layout. - -Here are some ways to run it during development: - -- `yesod devel`: runs in developer mode, rebuilds automatically when config, template, static or haskell files change -(but only files in the hledger-web package): -```shell -$ (cd hledger-web; yesod devel) -``` - -- [yesod-fast-devel](https://hackage.haskell.org/package/yesod-fast-devel) - may be a good alternative, also reloads the browser page - -- `stack ghci`: runs the server in developer mode from GHCI. -Changes to static files like hledger.js will be visible on page reload; -to see other changes, restart it as shown. -```shell -$ (cd hledger-web; stack ghci hledger-web) -hledger-web> :main --serve # restart: ctrl-c, :r, enter, ctrl-p, ctrl-p, enter -``` - -- `make ghci-web`: runs the server in developer mode from GHCI, also -interprets the hledger-lib and hledger packages so that :reload picks -up changes in those packages too: -```shell -$ make ghci-web -ghci> :main --serve -``` -(This rule also creates symbolic links to hledger-web's `config`, `messages`, `static` and `templates` -directories, needed in developer mode, so it can run from the top directory. This may not work on Windows.) - -### Quality - -Relevant tools include: - -- unit tests (HUnit, make unittest) -- functional tests (shelltestrunner, make functest) -- performance tests (simplebench, make bench) -- documentation tests (make haddocktest + manual) -- ui tests (manual) -- installation tests (manual) -- code reviews - -### Code review - -- Code review party 2014/7/21-25: - [discussion](https://thread.gmane.org/gmane.comp.finance.ledger.hledger/1070) -- Dev sprint/party 2015/10/10: - [discussion](https://thread.gmane.org/gmane.comp.finance.ledger.hledger/1254) - - -## Tests - -About testing in the hledger project, as of 201809. - -### Kinds of tests - -
-"Here, then, is a list of properties of tests. Not all tests need to exhibit all properties. However, no property should be given up without receiving a property of greater value in return. - -- Isolated — tests should return the same results regardless of the order in which they are run. -- Composable — if tests are isolated, then I can run 1 or 10 or 100 or 1,000,000 and get the same results. -- Fast — tests should run quickly. -- Inspiring — passing the tests should inspire confidence -- Writable — tests should be cheap to write relative to the cost of the code being tested. -- Readable — tests should be comprehensible for reader, invoking the motivation for writing this particular test. -- Behavioral — tests should be sensitive to changes in the behavior of the code under test. If the behavior changes, the test result should change. -- Structure-insensitive — tests should not change their result if the structure of the code changes. -- Automated — tests should run without human intervention. -- Specific — if a test fails, the cause of the failure should be obvious. -- Deterministic — if nothing changes, the test result shouldn’t change. -- Predictive — if the tests all pass, then the code under test should be suitable for production." ---[Kent Beck](https://medium.com/@kentbeck_7670/test-desiderata-94150638a4b3) -
- -1. Unit tests - - Unit tests exercise small chunks of functionality. In hledger, that - means a function. So, many of our functions have one or more unit - tests. These are mostly in hledger-lib, with a few in hledger. - - Our unit tests use the - [tasty](https://hackage.haskell.org/package/tasty) test runner, - [tasty-hunit](https://hackage.haskell.org/package/tasty-hunit) HUnit-style tests, - and some helpers from - [Hledger.Utils.Test](https://github.com/simonmichael/hledger/blob/master/hledger-lib/Hledger/Utils/Test.hs), - such as: - - - `tests` and `test` aliases for `testGroup` and `testCase` - - `assert*` helpers for constructing various kinds of assertions - - We would like our unit tests to be: - - - easy to read (clear, concise) - - easy to write (low boilerplate, low cognitive load) - - easy to maintain (easy to edit, easy to refactor, robust) - - easy to associate with the code under test (easy to view/jump - between code & test, easy to estimate coverage) - - and scalable (usable for all devs, easy to run and select, - suitable for small/large modules/packages). - - Here\'s the current pattern (let us know if you see a better way): - - ``` haskell - module Foo ( - ... - tests_Foo -- export this module's and submodules' tests - ) - where - import Hledger -- provides Hledger.Utils.Test helpers - import Bar -- submodules, providing tests_Bar etc. - import Baz - - functionA = ... - functionB = ... - functionC = ... - functionD = ... - - tests_Foo = tests "Foo" [ -- define tests at the end of each module - - -- a group of several named tests for functionA - tests "functionA" [ - test "a basic test" $ assertBool "" SOMEBOOL - ,test "a pretty equality test" $ SOMEEXPR @?= EXPECTEDVALUE - ,test "a pretty parsing test" $ assertParseEq PARSER INPUT EXPECTEDRESULT - ,test "a multiple assertions test" $ do - A @?= B - doSomeIO - C @?= D - ] - - -- a single test containing multiple unnamed assertions for functionB - ,test "functionB" $ do - assertBool "" BOOL - EXPR @?= VALUE - - ,tests_Foo -- aggregate submodule tests - ,tests_Bar - ] - ``` - - Here are - [some](https://github.com/simonmichael/hledger/blob/master/hledger-lib/Hledger/Data/Posting.hs#L296) - real-world - [examples](https://github.com/simonmichael/hledger/blob/master/hledger-lib/Hledger/Read/JournalReader.hs#L579). - - The unit tests are shipped as part of the hledger executable, and - can always be run via the [test](https://hledger.org/manual#test) - command (`hledger test`). - - Here\'s the quick way to run unit tests while developing:\ - `make ghcid-test` or `make ghcid-test-Some.Module`. - -2. Doc tests - - Like unit tests, but defined inside functions\' haddock - documentation, in the style of a GHCI transcript. These test - functionality, provide usage examples in the API docs, and test - those examples, all at once. They are a bit more finicky and slower - than unit tests. See - [doctest](https://hackage.haskell.org/package/doctest) for more. - - doctests [do not work on Mac with GHC - 8.4+](https://github.com/sol/doctest/issues/199), out of the box. - See - [ghc\#15105](https://ghc.haskell.org/trac/ghc/ticket/15105#comment:10) - for current status and a workaround. - -3. Functional tests - - Functional tests test the overall functioning of the program. For - hledger, that means running `hledger` with various inputs and - options and checking for the expected output. This exercises - functionality in the hledger and hledger-lib packages. We do this - with - [shelltestrunner](https://hackage.haskell.org/package/shelltestrunner). - Tests are defined in files named `*.test` under - [hledger/test/](https://github.com/simonmichael/hledger/tree/master/hledger/test), - grouped by *component* (command or topic name). - For more about these, see the README there. - -4. Code tests - - We have some tests aimed at testing eg code quality, generally - defined as make rules, such as: - - --------------------- ------------------------------------------------------------------------------------- - `make haddocktest` can haddock process all code docs without error - `make buildtest` does all code build warning free with the default GHC version & stackage snapshot - `make buildtestall` does the code build warning free with all supported GHC versions/stackage snapshots - --------------------- ------------------------------------------------------------------------------------- - - See below for examples. - -5. Package test suites - - Haskell tools like stack and cabal recognise test suites defined in - a package\'s cabal file (or package.yaml file). These can be run via - `stack test`, `cabal test` etc., and they are required to build and - pass by services like Stackage. Here are the currently hledger - package test suites: - - ------------- ------------ --------------------------------------------------------------- - package test suite what it runs - hledger-lib doctests doctests - hledger-lib easytests unit tests - hledger test builtin test command (hledger\'s + hledger-lib\'s unit tests) - hledger-ui - hledger-web - ------------- ------------ --------------------------------------------------------------- - -### Coverage - -This means how thoroughly the code is tested - both in breadth (are all -parts of the code tested at least a little ?) and in depth (are all -possible code paths, states, situations tested ?). - -Our current test coverage can be summarised like so: - - ------------- ------ ----- ------------ - package unit doc functional - hledger-lib X X X - hledger X X - hledger-ui - hledger-web - ------------- ------ ----- ------------ - -There are ways to generate detailed coverage reports for haskell unit -tests, at least. It would be useful to set this up for hledger. - -### How to run tests - -Run unit tests: - -``` example -$ make unittest -``` - -Run doctests: - -``` example -$ make doctest -``` - -Run functional tests (and unit tests, now): - -``` example -$ stack install shelltestrunner -$ make functest -``` - -Run the package tests (unit tests, maybe doctests, but not functional -tests) of all or selected packages. - -``` example -$ stack test [PKG] -``` - -Run \"default tests: package plus functional tests\": - -``` example -$ make test -``` - -Test generation of haddock docs: - -``` example -$ make haddocktest -``` - -Thorough test for build issues with current GHC: - -``` example -$ make buildtest -``` - -Thorough test for build issues with all supported GHC versions: - -``` example -$ make buildtestall -``` - -Run built-in hledger/hledger-lib unit tests via hledger command: - -``` example -$ hledger test # test installed hledger -$ stack build hledger && stack exec -- hledger test # test just-built hledger -$ hledger test --help -test [TESTPATTERN] [SEED] - Run the unit tests built in to hledger-lib and hledger, - printing results on stdout and exiting with success or failure. - Tests are run in two batches: easytest-based and hunit-based tests. - If any test fails or gives an error, the exit code will be non-zero. - If a pattern argument (case sensitive) is provided, only easytests - in that scope and only hunit tests whose name contains it are run. - If a numeric second argument is provided, it will set the randomness - seed for easytests. -``` - -Rebuild and rerun hledger/hledger-lib unit tests via ghcid: - -``` example -$ make ghcid-test -``` - -Rebuild and rerun only some tests via ghcid (see hledger test --help): - -``` example -$ make ghcid-test-TESTPATTERN -``` - -See all test-related make rules: - -``` example -$ make help-test -``` - - - - -## Benchmarks - -Benchmarks are standard performance measurements, -which we define using `bench` declarations in cabal files. -There is [one in hledger.cabal](https://github.com/simonmichael/hledger/blob/master/hledger/hledger.cabal#L228), -with related code and data files in [hledger/bench/](https://github.com/simonmichael/hledger/tree/master/hledger/bench). - -To run the standard hledger benchmark, use `stack bench hledger`. -This installs haskell dependencies (but not system dependencies) and rebuilds as needed, -then runs [hledger/bench/bench.hs](https://github.com/simonmichael/hledger/blob/master/hledger/bench/bench.hs), -which by default shows quick elapsed-time measurements for several operations on a standard data file: - -```shell -$ stack bench hledger -NOTE: the bench command is functionally equivalent to 'build --bench' -... -hledger-0.27: benchmarks -Running 1 benchmarks... -Benchmark bench: RUNNING... -Benchmarking hledger in /Users/simon/src/hledger/hledger with timeit -read bench/10000x1000x10.journal [1.57s] -print [1.29s] -register [1.92s] -balance [0.21s] -stats [0.23s] -Total: 5.22s -Benchmark bench: FINISH -``` - -bench.hs has some other modes, which you can use by compiling and running it directly. -`--criterion` reports more detailed and dependable measurements, but takes longer: - -```shell -$ cd hledger; stack exec -- ghc -ibench bench/bench && bench/bench --criterion -... -Linking bench/bench ... -Benchmarking hledger in /Users/simon/src/hledger/hledger with criterion -benchmarking read bench/10000x1000x10.journal -time 1.414 s (1.234 s .. 1.674 s) - 0.996 R² (0.989 R² .. 1.000 R²) -mean 1.461 s (1.422 s .. 1.497 s) -std dev 59.69 ms (0.0 s .. 62.16 ms) -variance introduced by outliers: 19% (moderately inflated) - -benchmarking print -time 1.323 s (1.279 s .. 1.385 s) - 1.000 R² (0.999 R² .. 1.000 R²) -mean 1.305 s (1.285 s .. 1.316 s) -std dev 17.20 ms (0.0 s .. 19.14 ms) -variance introduced by outliers: 19% (moderately inflated) - -benchmarking register -time 1.995 s (1.883 s .. 2.146 s) - 0.999 R² (0.998 R² .. NaN R²) -mean 1.978 s (1.951 s .. 1.995 s) -std dev 25.09 ms (0.0 s .. 28.26 ms) -variance introduced by outliers: 19% (moderately inflated) - -benchmarking balance -time 251.3 ms (237.6 ms .. 272.4 ms) - 0.998 R² (0.997 R² .. 1.000 R²) -mean 260.4 ms (254.3 ms .. 266.5 ms) -std dev 7.609 ms (3.192 ms .. 9.638 ms) -variance introduced by outliers: 16% (moderately inflated) - -benchmarking stats -time 325.5 ms (299.1 ms .. 347.2 ms) - 0.997 R² (0.985 R² .. 1.000 R²) -mean 329.2 ms (321.5 ms .. 339.6 ms) -std dev 11.08 ms (2.646 ms .. 14.82 ms) -variance introduced by outliers: 16% (moderately inflated) -``` - -`--simplebench` shows a table of elapsed-time measurements for the commands defined in [bench/default.bench](https://github.com/simonmichael/hledger/blob/master/hledger/bench/default.bench). -It can also show the results for multiple h/ledger executables side by side, if you tweak the bench.hs code. -Unlike the other modes, it does not link with the hledger code directly, but runs the "hledger" executable found in $PATH (so ensure that's the one you intend to test). - -```shell -$ cd hledger; stack exec -- ghc -ibench bench/bench && bench/bench --simplebench -Benchmarking /Users/simon/.local/bin/hledger in /Users/simon/src/hledger/hledger with simplebench and shell -Using bench/default.bench -Running 4 tests 1 times with 1 executables at 2015-08-23 16:58:59.128112 UTC: -1: hledger -f bench/10000x1000x10.journal print [3.27s] -1: hledger -f bench/10000x1000x10.journal register [3.65s] -1: hledger -f bench/10000x1000x10.journal balance [2.06s] -1: hledger -f bench/10000x1000x10.journal stats [2.13s] - -Summary (best iteration): - -+-----------------------------------------++---------+ -| || hledger | -+=========================================++=========+ -| -f bench/10000x1000x10.journal print || 3.27 | -| -f bench/10000x1000x10.journal register || 3.65 | -| -f bench/10000x1000x10.journal balance || 2.06 | -| -f bench/10000x1000x10.journal stats || 2.13 | -+-----------------------------------------++---------+ -``` - -bench's --simplebench mode is based on a standalone tool, [tools/simplebench.hs](https://github.com/simonmichael/hledger/blob/master/tools/simplebench.hs). -simplebench.hs is a generic benchmarker of one or more executables (specified on the command line) against one or more sets of command-line arguments (specified in a file). -It has a better command-line interface than bench.hs, so you may find it more convenient -for comparing multiple hledger versions, or hledger and ledger. Eg: - -```shell -$ stack exec -- ghc tools/simplebench -[1 of 1] Compiling Main ( tools/simplebench.hs, tools/simplebench.o ) -Linking tools/simplebench ... -``` -```shell -$ tools/simplebench -h -tools/simplebench -h -simplebench: at least one executable needed -bench [-f testsfile] [-n iterations] [-p precision] executable1 [executable2 ...] - -Run some functional tests with each of the specified executables, -where a test is "zero or more arguments supported by all executables", -and report the best execution times. - - -f testsfile --testsfile=testsfile file containing tests, one per line, default: bench.tests - -n iterations --iterations=iterations number of test iterations to run, default: 2 - -p precision --precision=precision show times with this precision, default: 2 - -v --verbose show intermediate results - -h --help show this help - -Tips: -- executables may have arguments if enclosed in quotes -- tests can be commented out with # -- results are saved in benchresults.{html,txt} -``` -```shell -cd hledger; $ ../tools/simplebench -f bench/default.bench hledger ledger -Using bench/default.bench -Running 4 tests 2 times with 2 executables at 2015-08-24 04:24:37.257068 UTC: - -Summary (best iteration): - -+-----------------------------------------++---------+--------+ -| || hledger | ledger | -+=========================================++=========+========+ -| -f bench/10000x1000x10.journal print || 3.24 | 0.43 | -| -f bench/10000x1000x10.journal register || 3.80 | 3.48 | -| -f bench/10000x1000x10.journal balance || 2.05 | 0.18 | -| -f bench/10000x1000x10.journal stats || 2.10 | 0.19 | -+-----------------------------------------++---------+--------+ -``` - -Finally, for quick, fine-grained performance measurements when troubleshooting or optimising, I use -[dev.hs](https://github.com/simonmichael/hledger/blob/master/dev.hs). - - - -## Version numbers - -Some places version numbers appear: - -- --version (and sometimes --help) output of all hledger* executables -- web manuals on hledger.org -- download page -- changelogs -- release notes -- release announcements -- hackage/stackage uris -- cabal tarball filenames -- platform-specific packages - -Some old version numbering goals: - -1. automation, robustness, simplicity, platform independence -2. cabal versions must be all-numeric -3. release versions can be concise (without extra .0's) -4. releases should have a corresponding VCS tag -5. development builds should have a precise version appearing in --version -6. development builds should generate cabal packages with non-confusing versions -7. there should be a way to mark builds/releases as alpha or beta -8. avoid unnecessary compiling and linking -9. minimise VCS noise and syncing issues (commits, unrecorded changes) - -Current version numbering policy: - -- We (should) follow - -- The "full release version" is ma.jor.minor, where minor is 0 for a - normal release or 1..n for bugfix releases. Each component is a - natural number (can be >= 10). Eg: 1.13 major release, 1.13.1 - bugfix release. - -- The "release version", which we prefer to use when possible, is - just ma.jor when minor is 0. Ie elide the dot zero. - -- The build version is ma.jor.minor+patches, where patches is the number - of patches applied in the current repo since the last release tag. - -- `hledger --version` shows the release version or build version as - appropriate. - -- Release tags in the VCS are like PKG-VERSION. Eg hledger-1.13, -- hledger-ui-1.13.1. - -Current process: - -- In each hledger package directory there's a `.version` file - containing its desired version number. - -- After changing a `.version` file: run `./Shake setversion` to - propagate the versions to all other places in the packages where - they should appear. This is not perfect (see Shake.hs) so review and - manually adjust the proposed changes before committing. Those - places include (you can also run these rules individually): - - - `PKG/package.yaml` contains the cabal package version declaration, - bounds on other hledger packages, and a CPP VERSION macro used in - `hledger/Hledger/Cli/Version.hs`. Changes in package.yaml will be - propagated to `PKG/PKG.cabal` on the next stack or Shake build, or - by `make gencabal`. - - - `PKG/.version.m4` contains the _version_ macro used in documentation source files (*.m4.md). It is updated by `./Shake setversion`. - - - `PKG/.date.m4` contains the _monthyear_ macro used in man pages. It is updated by `./Shake manuals`. - -- At release time: - - - `./Shake PKG/CHANGES.md-finalise` converts the topmost heading, if - it is an interim heading (just a commit hash), to a permanent - heading containing the version and today's date. - - - for each package being released, a PKG-VERSION git tag is created. - -- At major release time: - - - A new snapshot of the reference docs is added to the website, by - `./Shake site/doc/VERSION/.snapshot`, and added to the links in - `site/js/site.js`. - -## Sample journals - -Synthetic data files like `examples/100x100x10.journal` are useful for benchmarks and testing. -The numbers describe the number of transactions, number of accounts, and maximum account depth respectively. -They are generated by [`tools/generatejournal.hs`](https://github.com/simonmichael/hledger/blob/master/tools/generatejournal.hs). -They should get built automatically as needed, if not you can use `make samplejournals`: - -```shell -$ make samplejournals -ghc tools/generatejournal.hs -[1 of 1] Compiling Main ( tools/generatejournal.hs, tools/generatejournal.o ) -Linking tools/generatejournal ... -tools/generatejournal 100 100 10 >examples/100x100x10.journal -tools/generatejournal 1000 1000 10 >examples/1000x1000x10.journal -tools/generatejournal 1000 10000 10 >examples/1000x10000x10.journal -tools/generatejournal 10000 1000 10 >examples/10000x1000x10.journal -tools/generatejournal 10000 10000 10 >examples/10000x10000x10.journal -tools/generatejournal 100000 1000 10 >examples/100000x1000x10.journal -tools/generatejournal 3 5 5 >examples/ascii.journal -tools/generatejournal 3 5 5 --chinese >examples/chinese.journal -tools/generatejournal 3 5 5 --mixed >examples/mixed.journal -``` - - -## Docs - -### Four kinds of documentation - -20191209: needs update. See also doc/README. - - -
-"There is a secret that needs to be understood in order to write good -software documentation: there isn’t one thing called documentation, -there are four. They are: tutorials, how-to guides, explanation and -technical reference. They represent four different purposes or -functions, and require four different approaches to their creation." ---[Daniele Procida] (https://news.ycombinator.com/item?id=21289832) -
- -https://github.com/simonmichael/hledger/tree/master/doc - -Project documentation lives in a number of places: - -- `site/*.md` is the hledger.org website content, which is generated with hakyll[-std] and pandoc -- haddock documentation in the code appears on Hackage -- short blurbs: cabal files, module headers, HCAR, GSOC project, .. -- `doc/notes.org` has some old developer notes -- developer reports (profiles, benchmarks, coverage..) in doc/profs, sometimes published at hledger.org/profs - -## Funding - -My vision for the hledger project has always been for it to be "accountable" and "self-sustaining", possibly through new forms of incentivisation. -Classic non-monetary FOSS communities are a beautiful and precious thing. -Adding money can change their dynamic. -Yet, we would enjoy having a lot more issues resolved, and a faster rate of progress. -So we experiment, gently. - -Currently we use bounties as a way to encourage resolution of issues. -There are a few ways to do this: - -1. You or your organisation can offer a bounty simply by saying so on the issue. - -2. You can use Bountysource. A few hledger bounties have been completed there. - -3. You can use the new Open Collective process below. - -Issues with bounties of any kind are marked with the `bounty` label. -The Bounty Manager is @simonmichael. - -### New bounty process - -It currently looks like this, and will evolve: - -- Issues are marked as bounties by @simonmichael. Feel free to suggest additional issues which should receive the bounty label. - -- Bounties are paid from the hledger project's public Open Collective fund. - By contributing to the fund as an individual or organisation, you enable more bounties. - -- These OC bounties (unlike 1 and 2 above) have standard amounts. - These may be adjusted over time, depending eg on the state of our funds. - Our current bounty amounts are - - level 1: 10 USD - - level 2: 25 USD - - level 3: 50 USD - -- When you complete a bounty, submit an expense to Open Collective, - for whichever of the above bounty amounts you think appropriate, - based eg on time or expertise spent, how much you need it, - how much remains in our fund for other bounties, etc. - This will be reviewed by OC and (maybe ?) @simonmichael. - Successful claims, like donations, will appear in our public OC ledger. - -Our bounty amounts are small, and nothing like professional rates in most countries, -but they still establish a principle of sustainability, -and help us to experiment. -You are encouraged to claim your bounties, -though you can also choose to transfer them to a new issue of your choice. - -## Commit messages - -See [COMMITS](COMMITS.html). - -## Issues - -See [ISSUES](ISSUES.html). - -## Pull Requests - -See [PULLREQUESTS](PULLREQUESTS.html). - -## Developer workflows - -See [WORKFLOWS](WORKFLOWS.html). - diff --git a/DOCS.md b/DOCS.md new file mode 100644 index 000000000..ebba9086b --- /dev/null +++ b/DOCS.md @@ -0,0 +1,25 @@ +# Docs + +20191209: needs update. See also doc/README. + +## Four kinds of documentation + +
+"There is a secret that needs to be understood in order to write good +software documentation: there isn’t one thing called documentation, +there are four. They are: tutorials, how-to guides, explanation and +technical reference. They represent four different purposes or +functions, and require four different approaches to their creation." +--[Daniele Procida] (https://news.ycombinator.com/item?id=21289832) +
+ +https://github.com/simonmichael/hledger/tree/master/doc + +Project documentation lives in a number of places: + +- `site/*.md` is the hledger.org website content, which is generated with hakyll[-std] and pandoc +- haddock documentation in the code appears on Hackage +- short blurbs: cabal files, module headers, HCAR, GSOC project, .. +- `doc/notes.org` has some old developer notes +- developer reports (profiles, benchmarks, coverage..) in doc/profs, sometimes published at hledger.org/profs + diff --git a/EXAMPLES.md b/EXAMPLES.md new file mode 100644 index 000000000..7bc102fa3 --- /dev/null +++ b/EXAMPLES.md @@ -0,0 +1,31 @@ +# EXAMPLES + +## Collected examples + +Many example input files in journal and other formats can be found +in `examples/` in the `hledger` repo. + +## Sample journals + +Synthetic data files like `examples/100x100x10.journal` are useful for benchmarks and testing. +The numbers describe the number of transactions, number of accounts, and maximum account depth respectively. +They are generated by [`tools/generatejournal.hs`](https://github.com/simonmichael/hledger/blob/master/tools/generatejournal.hs). +They should get built automatically as needed, if not you can use `make samplejournals`: + +```shell +$ make samplejournals +ghc tools/generatejournal.hs +[1 of 1] Compiling Main ( tools/generatejournal.hs, tools/generatejournal.o ) +Linking tools/generatejournal ... +tools/generatejournal 100 100 10 >examples/100x100x10.journal +tools/generatejournal 1000 1000 10 >examples/1000x1000x10.journal +tools/generatejournal 1000 10000 10 >examples/1000x10000x10.journal +tools/generatejournal 10000 1000 10 >examples/10000x1000x10.journal +tools/generatejournal 10000 10000 10 >examples/10000x10000x10.journal +tools/generatejournal 100000 1000 10 >examples/100000x1000x10.journal +tools/generatejournal 3 5 5 >examples/ascii.journal +tools/generatejournal 3 5 5 --chinese >examples/chinese.journal +tools/generatejournal 3 5 5 --mixed >examples/mixed.journal +``` + + diff --git a/FUNDING.md b/FUNDING.md new file mode 100644 index 000000000..7c626449b --- /dev/null +++ b/FUNDING.md @@ -0,0 +1,49 @@ +# Funding + +My vision for the hledger project has always been for it to be "accountable" and "self-sustaining", possibly through new forms of incentivisation. +Classic non-monetary FOSS communities are a beautiful and precious thing. +Adding money can change their dynamic. +Yet, we would enjoy having a lot more issues resolved, and a faster rate of progress. +So we experiment, gently. + +Currently we use bounties as a way to encourage resolution of issues. +There are a few ways to do this: + +1. You or your organisation can offer a bounty simply by saying so on the issue. + +2. You can use Bountysource. A few hledger bounties have been completed there. + +3. You can use the new Open Collective process below. + +Issues with bounties of any kind are marked with the `bounty` label. +The Bounty Manager is @simonmichael. + +## New bounty process + +It currently looks like this, and will evolve: + +- Issues are marked as bounties by @simonmichael. Feel free to suggest additional issues which should receive the bounty label. + +- Bounties are paid from the hledger project's public Open Collective fund. + By contributing to the fund as an individual or organisation, you enable more bounties. + +- These OC bounties (unlike 1 and 2 above) have standard amounts. + These may be adjusted over time, depending eg on the state of our funds. + Our current bounty amounts are + - level 1: 10 USD + - level 2: 25 USD + - level 3: 50 USD + +- When you complete a bounty, submit an expense to Open Collective, + for whichever of the above bounty amounts you think appropriate, + based eg on time or expertise spent, how much you need it, + how much remains in our fund for other bounties, etc. + This will be reviewed by OC and (maybe ?) @simonmichael. + Successful claims, like donations, will appear in our public OC ledger. + +Our bounty amounts are small, and nothing like professional rates in most countries, +but they still establish a principle of sustainability, +and help us to experiment. +You are encouraged to claim your bounties, +though you can also choose to transfer them to a new issue of your choice. + diff --git a/MAKE.md b/MAKE.md new file mode 100644 index 000000000..c4374e6a1 --- /dev/null +++ b/MAKE.md @@ -0,0 +1,40 @@ +# Make + +A Makefile is provided to make common developer tasks easy to remember, +and to insulate us a little from the ever-evolving Haskell tools ecosystem. +Using it is entirely optional, but recommended. +You'll need [GNU Make](https://www.gnu.org/software/make) installed. + +The Makefile contains a fair amount of obsolete cruft and needs cleanup. Some tasks (docs, website) are now handled by the [Shake](#shake) file instead. + +The Makefile is self-documenting. Run `make` to see a list of the main make rules: + +```shell +$ make +Makefile:37: -------------------- hledger make rules -------------------- +Makefile:39: make [help] -- list documented rules in this makefile. make -n RULE shows more detail. +Makefile:204: (INSTALLING:) +Makefile:206: make install -- download dependencies and install hledger executables to ~/.local/bin or equivalent (with stack) +Makefile:231: (BUILDING:) +Makefile:235: make build -- download dependencies and build hledger executables (with stack) +Makefile:304: make hledgerdev -- quickly build the hledger executable (with ghc and -DDEVELOPMENT) +... +``` + +To see what a make rule will do without actually doing it, use the `-n` flag: + +```shell +$ make build -n +stack build +``` +```shell +$ make test -n +(stack test \ + && echo pkgtest PASSED) || echo pkgtest FAILED +(stack exec hledger test \ + && echo builtintest PASSED) || echo builtintest FAILED +(COLUMNS=80 PATH=`pwd`/bin:/home/simon/src/hledger/bin:/home/simon/.local/bin:/home/simon/.cabal/bin:/opt/ghc/7.10.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/var/lib/gems/1.9.1/bin stack exec -- shelltest --execdir -- -j16 --hide-successes tests \ + && echo functest PASSED) || echo functest FAILED +``` + + diff --git a/SHAKE.md b/SHAKE.md new file mode 100644 index 000000000..22f9870b1 --- /dev/null +++ b/SHAKE.md @@ -0,0 +1,13 @@ +# Shake + +`Shake.hs` in the top directory complements the Makefile; it is used for some more complex tasks, such as building documentation and the web site. + +Compile it: + + ./Shake.hs # or, make Shake + +See help: + + ./Shake + + diff --git a/TESTS.md b/TESTS.md new file mode 100644 index 000000000..a6f495d55 --- /dev/null +++ b/TESTS.md @@ -0,0 +1,269 @@ +# Tests + +About testing in the hledger project, as of 201809. + +## Kinds of tests + +
+"Here, then, is a list of properties of tests. Not all tests need to exhibit all properties. However, no property should be given up without receiving a property of greater value in return. + +- Isolated — tests should return the same results regardless of the order in which they are run. +- Composable — if tests are isolated, then I can run 1 or 10 or 100 or 1,000,000 and get the same results. +- Fast — tests should run quickly. +- Inspiring — passing the tests should inspire confidence +- Writable — tests should be cheap to write relative to the cost of the code being tested. +- Readable — tests should be comprehensible for reader, invoking the motivation for writing this particular test. +- Behavioral — tests should be sensitive to changes in the behavior of the code under test. If the behavior changes, the test result should change. +- Structure-insensitive — tests should not change their result if the structure of the code changes. +- Automated — tests should run without human intervention. +- Specific — if a test fails, the cause of the failure should be obvious. +- Deterministic — if nothing changes, the test result shouldn’t change. +- Predictive — if the tests all pass, then the code under test should be suitable for production." +--[Kent Beck](https://medium.com/@kentbeck_7670/test-desiderata-94150638a4b3) +
+ +1. Unit tests + + Unit tests exercise small chunks of functionality. In hledger, that + means a function. So, many of our functions have one or more unit + tests. These are mostly in hledger-lib, with a few in hledger. + + Our unit tests use the + [tasty](https://hackage.haskell.org/package/tasty) test runner, + [tasty-hunit](https://hackage.haskell.org/package/tasty-hunit) HUnit-style tests, + and some helpers from + [Hledger.Utils.Test](https://github.com/simonmichael/hledger/blob/master/hledger-lib/Hledger/Utils/Test.hs), + such as: + + - `tests` and `test` aliases for `testGroup` and `testCase` + - `assert*` helpers for constructing various kinds of assertions + + We would like our unit tests to be: + + - easy to read (clear, concise) + - easy to write (low boilerplate, low cognitive load) + - easy to maintain (easy to edit, easy to refactor, robust) + - easy to associate with the code under test (easy to view/jump + between code & test, easy to estimate coverage) + - and scalable (usable for all devs, easy to run and select, + suitable for small/large modules/packages). + + Here\'s the current pattern (let us know if you see a better way): + + ``` haskell + module Foo ( + ... + tests_Foo -- export this module's and submodules' tests + ) + where + import Hledger -- provides Hledger.Utils.Test helpers + import Bar -- submodules, providing tests_Bar etc. + import Baz + + functionA = ... + functionB = ... + functionC = ... + functionD = ... + + tests_Foo = tests "Foo" [ -- define tests at the end of each module + + -- a group of several named tests for functionA + tests "functionA" [ + test "a basic test" $ assertBool "" SOMEBOOL + ,test "a pretty equality test" $ SOMEEXPR @?= EXPECTEDVALUE + ,test "a pretty parsing test" $ assertParseEq PARSER INPUT EXPECTEDRESULT + ,test "a multiple assertions test" $ do + A @?= B + doSomeIO + C @?= D + ] + + -- a single test containing multiple unnamed assertions for functionB + ,test "functionB" $ do + assertBool "" BOOL + EXPR @?= VALUE + + ,tests_Foo -- aggregate submodule tests + ,tests_Bar + ] + ``` + + Here are + [some](https://github.com/simonmichael/hledger/blob/master/hledger-lib/Hledger/Data/Posting.hs#L296) + real-world + [examples](https://github.com/simonmichael/hledger/blob/master/hledger-lib/Hledger/Read/JournalReader.hs#L579). + + The unit tests are shipped as part of the hledger executable, and + can always be run via the [test](https://hledger.org/manual#test) + command (`hledger test`). + + Here\'s the quick way to run unit tests while developing:\ + `make ghcid-test` or `make ghcid-test-Some.Module`. + +2. Doc tests + + Like unit tests, but defined inside functions\' haddock + documentation, in the style of a GHCI transcript. These test + functionality, provide usage examples in the API docs, and test + those examples, all at once. They are a bit more finicky and slower + than unit tests. See + [doctest](https://hackage.haskell.org/package/doctest) for more. + + doctests [do not work on Mac with GHC + 8.4+](https://github.com/sol/doctest/issues/199), out of the box. + See + [ghc\#15105](https://ghc.haskell.org/trac/ghc/ticket/15105#comment:10) + for current status and a workaround. + +3. Functional tests + + Functional tests test the overall functioning of the program. For + hledger, that means running `hledger` with various inputs and + options and checking for the expected output. This exercises + functionality in the hledger and hledger-lib packages. We do this + with + [shelltestrunner](https://hackage.haskell.org/package/shelltestrunner). + Tests are defined in files named `*.test` under + [hledger/test/](https://github.com/simonmichael/hledger/tree/master/hledger/test), + grouped by *component* (command or topic name). + For more about these, see the README there. + +4. Code tests + + We have some tests aimed at testing eg code quality, generally + defined as make rules, such as: + + --------------------- ------------------------------------------------------------------------------------- + `make haddocktest` can haddock process all code docs without error + `make buildtest` does all code build warning free with the default GHC version & stackage snapshot + `make buildtestall` does the code build warning free with all supported GHC versions/stackage snapshots + --------------------- ------------------------------------------------------------------------------------- + + See below for examples. + +5. Package test suites + + Haskell tools like stack and cabal recognise test suites defined in + a package\'s cabal file (or package.yaml file). These can be run via + `stack test`, `cabal test` etc., and they are required to build and + pass by services like Stackage. Here are the currently hledger + package test suites: + + ------------- ------------ --------------------------------------------------------------- + package test suite what it runs + hledger-lib doctests doctests + hledger-lib easytests unit tests + hledger test builtin test command (hledger\'s + hledger-lib\'s unit tests) + hledger-ui + hledger-web + ------------- ------------ --------------------------------------------------------------- + +## Coverage + +This means how thoroughly the code is tested - both in breadth (are all +parts of the code tested at least a little ?) and in depth (are all +possible code paths, states, situations tested ?). + +Our current test coverage can be summarised like so: + + ------------- ------ ----- ------------ + package unit doc functional + hledger-lib X X X + hledger X X + hledger-ui + hledger-web + ------------- ------ ----- ------------ + +There are ways to generate detailed coverage reports for haskell unit +tests, at least. It would be useful to set this up for hledger. + +## How to run tests + +Run unit tests: + +``` example +$ make unittest +``` + +Run doctests: + +``` example +$ make doctest +``` + +Run functional tests (and unit tests, now): + +``` example +$ stack install shelltestrunner +$ make functest +``` + +Run the package tests (unit tests, maybe doctests, but not functional +tests) of all or selected packages. + +``` example +$ stack test [PKG] +``` + +Run \"default tests: package plus functional tests\": + +``` example +$ make test +``` + +Test generation of haddock docs: + +``` example +$ make haddocktest +``` + +Thorough test for build issues with current GHC: + +``` example +$ make buildtest +``` + +Thorough test for build issues with all supported GHC versions: + +``` example +$ make buildtestall +``` + +Run built-in hledger/hledger-lib unit tests via hledger command: + +``` example +$ hledger test # test installed hledger +$ stack build hledger && stack exec -- hledger test # test just-built hledger +$ hledger test --help +test [TESTPATTERN] [SEED] + Run the unit tests built in to hledger-lib and hledger, + printing results on stdout and exiting with success or failure. + Tests are run in two batches: easytest-based and hunit-based tests. + If any test fails or gives an error, the exit code will be non-zero. + If a pattern argument (case sensitive) is provided, only easytests + in that scope and only hunit tests whose name contains it are run. + If a numeric second argument is provided, it will set the randomness + seed for easytests. +``` + +Rebuild and rerun hledger/hledger-lib unit tests via ghcid: + +``` example +$ make ghcid-test +``` + +Rebuild and rerun only some tests via ghcid (see hledger test --help): + +``` example +$ make ghcid-test-TESTPATTERN +``` + +See all test-related make rules: + +``` example +$ make help-test +``` + + + + diff --git a/VERSIONNUMBERS.md b/VERSIONNUMBERS.md new file mode 100644 index 000000000..224adead9 --- /dev/null +++ b/VERSIONNUMBERS.md @@ -0,0 +1,82 @@ +# Version numbers + +Some places version numbers appear: + +- --version (and sometimes --help) output of all hledger* executables +- web manuals on hledger.org +- download page +- changelogs +- release notes +- release announcements +- hackage/stackage uris +- cabal tarball filenames +- platform-specific packages + +Some old version numbering goals: + +1. automation, robustness, simplicity, platform independence +2. cabal versions must be all-numeric +3. release versions can be concise (without extra .0's) +4. releases should have a corresponding VCS tag +5. development builds should have a precise version appearing in --version +6. development builds should generate cabal packages with non-confusing versions +7. there should be a way to mark builds/releases as alpha or beta +8. avoid unnecessary compiling and linking +9. minimise VCS noise and syncing issues (commits, unrecorded changes) + +Current version numbering policy: + +- We (should) follow + +- The "full release version" is ma.jor.minor, where minor is 0 for a + normal release or 1..n for bugfix releases. Each component is a + natural number (can be >= 10). Eg: 1.13 major release, 1.13.1 + bugfix release. + +- The "release version", which we prefer to use when possible, is + just ma.jor when minor is 0. Ie elide the dot zero. + +- The build version is ma.jor.minor+patches, where patches is the number + of patches applied in the current repo since the last release tag. + +- `hledger --version` shows the release version or build version as + appropriate. + +- Release tags in the VCS are like PKG-VERSION. Eg hledger-1.13, +- hledger-ui-1.13.1. + +Current process: + +- In each hledger package directory there's a `.version` file + containing its desired version number. + +- After changing a `.version` file: run `./Shake setversion` to + propagate the versions to all other places in the packages where + they should appear. This is not perfect (see Shake.hs) so review and + manually adjust the proposed changes before committing. Those + places include (you can also run these rules individually): + + - `PKG/package.yaml` contains the cabal package version declaration, + bounds on other hledger packages, and a CPP VERSION macro used in + `hledger/Hledger/Cli/Version.hs`. Changes in package.yaml will be + propagated to `PKG/PKG.cabal` on the next stack or Shake build, or + by `make gencabal`. + + - `PKG/.version.m4` contains the _version_ macro used in documentation source files (*.m4.md). It is updated by `./Shake setversion`. + + - `PKG/.date.m4` contains the _monthyear_ macro used in man pages. It is updated by `./Shake manuals`. + +- At release time: + + - `./Shake PKG/CHANGES.md-finalise` converts the topmost heading, if + it is an interim heading (just a commit hash), to a permanent + heading containing the version and today's date. + + - for each package being released, a PKG-VERSION git tag is created. + +- At major release time: + + - A new snapshot of the reference docs is added to the website, by + `./Shake site/doc/VERSION/.snapshot`, and added to the links in + `site/js/site.js`. +