diff --git a/site/contributing.md b/site/contributing.md index dd7538af3..74d587f78 100644 --- a/site/contributing.md +++ b/site/contributing.md @@ -30,294 +30,36 @@ and help transform world finance! ### Tester - Test installation on platforms you have access to -- Run the latest release or developer build - Test examples, advice, and links in the docs -- Report packaging, documentation, UX, functional bugs +- Run the latest release or developer build in daily use +- Run [tests](#tests) +- Run [benchmarks](#benchmarking) +- Report packaging, documentation, UX, functional, performance issues - Report and help analyse problems via irc/mail list/bug tracker -#### Report bugs - -If you found a similar bug report already in the tracker, add any new information there. +When reporting bugs, don't forget to search the tracker for a similar bug report. Otherwise, open a new bug by clicking "New issue", or . -#### Suggest enhancements - -Some enhancement requests land in the bug tracker; these will get the WISH tag to avoid obscuring bugs. -But for general brainstorming and idea capture, consider using - -- The [#hledger](http://irc.hledger.org) (irc.hledger.org) IRC channel on freenode - and the [mail list](http://list.hledger.org) (list.hledger.org) are excellent places for discussing and refining ideas. - Both are archived and linkable, so the idea won't be lost. The IRC channel is quick, the mail list has more readers. -- The [trello board](http://trello.hledger.org) (trello.hledger.org) is a categorised collection of wishlist items. - -#### Search the issue tracker - -The hledger project's issue tracker is on github. It contains: - -- BUG issues - failures in some part of the hledger project (the main hledger packages, docs, website..) -- WISH issues - feature proposals, enhancement requests -- uncategorised issues - we don't know what these are yet -- pull requests - proposed changes to code and docs - -Use these shortcut urls for quick access: - -- - show open BUG issues -- - show open WISH issues -- - show all issues, open or closed -- - show open pull requests -- - show issue number N -- - create a new issue - -Labels are used to categorise: - -- the issue's type: "A BUG" or "A WISH", in shades of red (The A makes it appear as first label) -- relevant subsystems/topics, in light blue -- relevant platforms, in light purple -- resolution if not fixed: "closed:cant-reproduce/duplicate/invalid/wont-fix", in dark grey -- "bounty", in bright yellow: issues with bountysource funding -- "easy?", in dim yellow: issues which are probably relatively easy to fix -- "imported" etc., in white: miscellaneous information - -Clicking blue topic labels is a good way to review issues in a topic you're interested in. - -Milestones are used to organise releases. Click a milestone to see issues fixed in/planned for that release. - -Time annotations: some issue names might have a suffix noting estimated and spent time. -Basic format: [ESTIMATEDTOTALTASKTIME|TIMESPENTSOFAR]. Examples: -``` -[2] two hours estimated, no time spent -[..] half an hour estimated (a dot is ~a quarter hour, as in timedot format) -[1d] one day estimated (a day is ~4 hours) -[1w] one week estimated (a week is ~5 days or ~20 hours) -[3|2] three hours estimated, about two hours spent so far -[1|1w|2d] first estimate one hour, second estimate one week, about two days spent so far -``` -Estimates are always for the total time cost (not time remaining). -Estimates are not usually changed, a new estimate is added instead. -Numbers are very approximate, but better than nothing. - -#### Run tests - -This command will install haskell dependencies (you might need to -install additional system dependencies yourself) and run the package -test suites. Currently these consist of hledger-lib's unit tests, -hledger's unit tests, and hledger-web's functional tests: - -```shell -$ stack test [PKG] -``` - -Run the hledger-lib and hledger unit tests as a built-in hledger command: -```shell -$ [stack exec] hledger test -``` - -Run the hledger functional tests: -```shell -$ stack install shelltestrunner # if not already done -$ make functest -``` - -Run both unit and functional tests: -```shell -$ make test -``` - -Test haddock doc generation: -```shell -$ make haddocktest -``` - - -#### Run benchmarks - -Benchmarks are standard performance measurements, -which we define using `bench` declarations in cabal files. -There is [one in hledger.cabal](https://github.com/simonmichael/hledger/blob/master/hledger/hledger.cabal#L228), -with related code and data files in [hledger/bench/](https://github.com/simonmichael/hledger/tree/master/hledger/bench). - -To run the standard hledger benchmark, use `stack bench hledger`. -This installs haskell dependencies (but not system dependencies) and rebuilds as needed, -then runs [hledger/bench/bench.hs](https://github.com/simonmichael/hledger/blob/master/hledger/bench/bench.hs), -which by default shows quick elapsed-time measurements for several operations on a standard data file: - -```shell -$ stack bench hledger -NOTE: the bench command is functionally equivalent to 'build --bench' -... -hledger-0.27: benchmarks -Running 1 benchmarks... -Benchmark bench: RUNNING... -Benchmarking hledger in /Users/simon/src/hledger/hledger with timeit -read bench/10000x1000x10.journal [1.57s] -print [1.29s] -register [1.92s] -balance [0.21s] -stats [0.23s] -Total: 5.22s -Benchmark bench: FINISH -``` - -bench.hs has some other modes, which you can use by compiling and running it directly. -`--criterion` reports more detailed and dependable measurements, but takes longer: - -```shell -$ cd hledger; stack exec -- ghc -ibench bench/bench && bench/bench --criterion -... -Linking bench/bench ... -Benchmarking hledger in /Users/simon/src/hledger/hledger with criterion -benchmarking read bench/10000x1000x10.journal -time 1.414 s (1.234 s .. 1.674 s) - 0.996 R² (0.989 R² .. 1.000 R²) -mean 1.461 s (1.422 s .. 1.497 s) -std dev 59.69 ms (0.0 s .. 62.16 ms) -variance introduced by outliers: 19% (moderately inflated) - -benchmarking print -time 1.323 s (1.279 s .. 1.385 s) - 1.000 R² (0.999 R² .. 1.000 R²) -mean 1.305 s (1.285 s .. 1.316 s) -std dev 17.20 ms (0.0 s .. 19.14 ms) -variance introduced by outliers: 19% (moderately inflated) - -benchmarking register -time 1.995 s (1.883 s .. 2.146 s) - 0.999 R² (0.998 R² .. NaN R²) -mean 1.978 s (1.951 s .. 1.995 s) -std dev 25.09 ms (0.0 s .. 28.26 ms) -variance introduced by outliers: 19% (moderately inflated) - -benchmarking balance -time 251.3 ms (237.6 ms .. 272.4 ms) - 0.998 R² (0.997 R² .. 1.000 R²) -mean 260.4 ms (254.3 ms .. 266.5 ms) -std dev 7.609 ms (3.192 ms .. 9.638 ms) -variance introduced by outliers: 16% (moderately inflated) - -benchmarking stats -time 325.5 ms (299.1 ms .. 347.2 ms) - 0.997 R² (0.985 R² .. 1.000 R²) -mean 329.2 ms (321.5 ms .. 339.6 ms) -std dev 11.08 ms (2.646 ms .. 14.82 ms) -variance introduced by outliers: 16% (moderately inflated) -``` - -`--simplebench` shows a table of elapsed-time measurements for the commands defined in [bench/default.bench](https://github.com/simonmichael/hledger/blob/master/hledger/bench/default.bench). -It can also show the results for multiple h/ledger executables side by side, if you tweak the bench.hs code. -Unlike the other modes, it does not link with the hledger code directly, but runs the "hledger" executable found in $PATH (so ensure that's the one you intend to test). - -```shell -$ cd hledger; stack exec -- ghc -ibench bench/bench && bench/bench --simplebench -Benchmarking /Users/simon/.local/bin/hledger in /Users/simon/src/hledger/hledger with simplebench and shell -Using bench/default.bench -Running 4 tests 1 times with 1 executables at 2015-08-23 16:58:59.128112 UTC: -1: hledger -f bench/10000x1000x10.journal print [3.27s] -1: hledger -f bench/10000x1000x10.journal register [3.65s] -1: hledger -f bench/10000x1000x10.journal balance [2.06s] -1: hledger -f bench/10000x1000x10.journal stats [2.13s] - -Summary (best iteration): - -+-----------------------------------------++---------+ -| || hledger | -+=========================================++=========+ -| -f bench/10000x1000x10.journal print || 3.27 | -| -f bench/10000x1000x10.journal register || 3.65 | -| -f bench/10000x1000x10.journal balance || 2.06 | -| -f bench/10000x1000x10.journal stats || 2.13 | -+-----------------------------------------++---------+ -``` - -bench's --simplebench mode is based on a standalone tool, [tools/simplebench.hs](https://github.com/simonmichael/hledger/blob/master/tools/simplebench.hs). -simplebench.hs is a generic benchmarker of one or more executables (specified on the command line) against one or more sets of command-line arguments (specified in a file). -It has a better command-line interface than bench.hs, so you may find it more convenient -for comparing multiple hledger versions, or hledger and ledger. Eg: - -```shell -$ stack exec -- ghc tools/simplebench -[1 of 1] Compiling Main ( tools/simplebench.hs, tools/simplebench.o ) -Linking tools/simplebench ... -``` -```shell -$ tools/simplebench -h -tools/simplebench -h -simplebench: at least one executable needed -bench [-f testsfile] [-n iterations] [-p precision] executable1 [executable2 ...] - -Run some functional tests with each of the specified executables, -where a test is "zero or more arguments supported by all executables", -and report the best execution times. - - -f testsfile --testsfile=testsfile file containing tests, one per line, default: bench.tests - -n iterations --iterations=iterations number of test iterations to run, default: 2 - -p precision --precision=precision show times with this precision, default: 2 - -v --verbose show intermediate results - -h --help show this help - -Tips: -- executables may have arguments if enclosed in quotes -- tests can be commented out with # -- results are saved in benchresults.{html,txt} -``` -```shell -cd hledger; $ ../tools/simplebench -f bench/default.bench hledger ledger -Using bench/default.bench -Running 4 tests 2 times with 2 executables at 2015-08-24 04:24:37.257068 UTC: - -Summary (best iteration): - -+-----------------------------------------++---------+--------+ -| || hledger | ledger | -+=========================================++=========+========+ -| -f bench/10000x1000x10.journal print || 3.24 | 0.43 | -| -f bench/10000x1000x10.journal register || 3.80 | 3.48 | -| -f bench/10000x1000x10.journal balance || 2.05 | 0.18 | -| -f bench/10000x1000x10.journal stats || 2.10 | 0.19 | -+-----------------------------------------++---------+--------+ -``` - -Finally, for quick, fine-grained performance measurements when troubleshooting or optimising, I use -[dev.hs](https://github.com/simonmichael/hledger/blob/master/dev.hs). - -#### Generate sample journal files - -Synthetic data files like `examples/100x100x10.journal` are useful for benchmarks and testing. -The numbers describe the number of transactions, number of accounts, and maximum account depth respectively. -They are generated by [`tools/generatejournal.hs`](https://github.com/simonmichael/hledger/blob/master/tools/generatejournal.hs). -They should be built as needed, if not you can use `make samplejournals` rule: - -```shell -$ make samplejournals -ghc tools/generatejournal.hs -[1 of 1] Compiling Main ( tools/generatejournal.hs, tools/generatejournal.o ) -Linking tools/generatejournal ... -tools/generatejournal 100 100 10 >examples/100x100x10.journal -tools/generatejournal 1000 1000 10 >examples/1000x1000x10.journal -tools/generatejournal 1000 10000 10 >examples/1000x10000x10.journal -tools/generatejournal 10000 1000 10 >examples/10000x1000x10.journal -tools/generatejournal 10000 10000 10 >examples/10000x10000x10.journal -tools/generatejournal 100000 1000 10 >examples/100000x1000x10.journal -tools/generatejournal 3 5 5 >examples/ascii.journal -tools/generatejournal 3 5 5 --chinese >examples/chinese.journal -tools/generatejournal 3 5 5 --mixed >examples/mixed.journal -``` +Enhancement requests are sometimes added to the tracker,but for these consider using +the IRC channel and mail list (see [Getting help](/docs.html#getting-help)). +Both are archived and linkable, so the idea won't be lost. +There is also a collection of wishes at the old [trello board](http://trello.hledger.org). ### Developer #### Do code review - review and discuss new [pull requests](http://prs.hledger.org) and commits on github -- set up for development and test the latest changes in your own repo +- build hledger and test the latest changes in your own repo - read the existing [code docs and source](#quick-links) -- send feedback or discuss via irc or list +- send feedback or discuss via [IRC or mail list](/docs.html#get-help) #### Build hledger -In short: get [`stack`](/download.html#b) and (except on Windows, where stack provides it) [`git`](http://git-scm.com), then: +1. get [`stack`](/download.html#b) and (except on Windows, where stack provides it) [`git`](http://git-scm.com), then: +2. `git clone http://github.com/simonmichael/hledger hledger && cd hledger && stack install` -`$ git clone http://github.com/simonmichael/hledger hledger && cd hledger && stack install` - -\ +
In more detail: **1. Get tools** @@ -331,15 +73,18 @@ the hledger docs assume stack. fetch the hledger source and submit changes. On Windows, stack will install this as well. -Optional: + **2. Get the source** @@ -358,66 +103,25 @@ This builds all the hledger packages, and installs executables in `$HOME/.local/bin/` (or the Windows equivalent), which you should [add to your `$PATH`](/download.html#b). +This can take a while! +To save time, you can build fewer [packages](/manual.html#official-add-ons), eg just the CLI: +```shell +$ stack install hledger +``` + +You can also build and run in place, without installing executables: +```shell +$ stack build; stack exec -- hledger [ARGS] +``` + Note stack fetches most required dependencies automatically, but not C libraries such as curses or terminfo, which you might need to install yourself, using your system's package manager. In case of trouble, see [download](/download.html#b). -Downloading and building can take a while! -You can build fewer [packages](/manual.html#official-add-ons) to save time (recommended). Eg to build just the CLI:\ -`stack install hledger` +
-You can also build and run in place, without installing executables:\ -`stack build; stack exec -- hledger [ARGS]` - - -#### Add a test - -- identify what to test -- choose the test type: unit ? functional ? benchmark ? -- currently expected to pass or fail ? -- figure out where it goes -- write test, verify expected result -- get it committed - -#### Use the Makefile - -A Makefile is provided to make common developer tasks easy to remember, -and to insulate us a little from the ever-evolving Haskell tools ecosystem. -Using it is entirely optional, but recommended. -You'll need [GNU Make](http://www.gnu.org/software/make) installed. - -The Makefile is self-documenting. Run `make` to see a list of the main make rules: - -```shell -$ make -Makefile:37: -------------------- hledger make rules -------------------- -Makefile:39: make [help] -- list documented rules in this makefile. make -n RULE shows more detail. -Makefile:204: (INSTALLING:) -Makefile:206: make install -- download dependencies and install hledger executables to ~/.local/bin or equivalent (with stack) -Makefile:231: (BUILDING:) -Makefile:235: make build -- download dependencies and build hledger executables (with stack) -Makefile:304: make hledgerdev -- quickly build the hledger executable (with ghc and -DDEVELOPMENT) -... -``` - -To see what a make rule will do without actually doing it, use the `-n` flag: - -```shell -$ make build -n -stack build -``` -```shell -$ make test -n -(stack test \ - && echo pkgtest PASSED) || echo pkgtest FAILED -(stack exec hledger test \ - && echo builtintest PASSED) || echo builtintest FAILED -(COLUMNS=80 PATH=`pwd`/bin:/home/simon/src/hledger/bin:/home/simon/.local/bin:/home/simon/.cabal/bin:/opt/ghc/7.10.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/var/lib/gems/1.9.1/bin stack exec -- shelltest --execdir -- -j16 --hide-successes tests \ - && echo functest PASSED) || echo functest FAILED -``` - -#### Use the REPL (GHCI) +#### Use GHCI These all work from the main hledger source directory (at least). @@ -470,6 +174,15 @@ $ make ghci-dev ``` --> +#### Add a test + +- identify what to test +- choose the test type: unit ? functional ? benchmark ? +- currently expected to pass or fail ? +- figure out where it goes +- write test, verify expected result +- get it committed + #### Fix a bug or add a feature - research, discuss, validate the issue/feature on irc/list/bug tracker @@ -494,13 +207,13 @@ If you're new to this process, [help.github.com](http://help.github.com) may be #### Add yourself to the contributor list -- after getting something into the master branch, read and sign the [contributor list & agreement](contributors.html) -- or, [ask](/docs.html#get-help) to be added +- after getting something into the master branch, read and sign the [contributor list & agreement](contributors.html). Or, [ask](/docs.html#get-help) to be added. +- give yourself a high five! ### Technical Writer - get familiar with the website and documentation online, review and test -- get familiar with the site/doc source files (see Makefile) +- get familiar with the site/doc source files (see [Shake.hs](#shake)) - get the latest hledger source - send patches with names prefixed with "doc: " (or "site: ") @@ -510,6 +223,7 @@ If you're new to this process, [help.github.com](http://help.github.com) may be - illustrations and diagrams - web design mockups for home page, site, hledger-web UI + ### Communicator Marketing and market understanding is vital. @@ -684,8 +398,10 @@ don't run make setversion. -## Project overview -### Mission, principles, goals +## More about... +### Project + +#### Mission, principles, goals The hledger project aims to produce: @@ -693,7 +409,7 @@ The hledger project aims to produce: - a useful library and toolbox for finance-minded haskell programmers - a successful, time-and-money-solvent project within a thriving ecosystem of financial software projects. -### Roles and activities +#### Roles and activities - newcomer/potential user - user @@ -708,6 +424,301 @@ The hledger project aims to produce: - communicator - project manager +### Issue tracking + +The hledger project's issue tracker is on github. It contains: + +- BUG issues - failures in some part of the hledger project (the main hledger packages, docs, website..) +- WISH issues - feature proposals, enhancement requests +- uncategorised issues - we don't know what these are yet +- pull requests - proposed changes to code and docs + +Use these shortcut urls for quick access: + +- - show open BUG issues +- - show open WISH issues +- - show all issues, open or closed +- - show open pull requests + + + +Labels are used to categorise: + +- the issue's type: "A BUG" or "A WISH", in shades of red (The A makes it appear as first label) +- relevant subsystems/topics, in light blue +- relevant platforms, in light purple +- resolution if not fixed: "closed:cant-reproduce/duplicate/invalid/wont-fix", in dark grey +- "bounty", in bright yellow: issues with bountysource funding +- "easy?", in dim yellow: issues which are probably relatively easy to fix +- "imported" etc., in white: miscellaneous information + +Clicking blue topic labels is a good way to review issues in a topic you're interested in. + +In 2017, milestones are not used much. Projects are being used to organise the roadmap. + +You might see some experiments in estimate tracking, where +some issue names might have a suffix noting estimated and spent time. +Basic format: [ESTIMATEDTOTALTASKTIME|TIMESPENTSOFAR]. Examples: +``` +[2] two hours estimated, no time spent +[..] half an hour estimated (a dot is ~a quarter hour, as in timedot format) +[1d] one day estimated (a day is ~4 hours) +[1w] one week estimated (a week is ~5 days or ~20 hours) +[3|2] three hours estimated, about two hours spent so far +[1|1w|2d] first estimate one hour, second estimate one week, about two days spent so far +``` +Estimates are always for the total time cost (not time remaining). +Estimates are not usually changed, a new estimate is added instead. +Numbers are very approximate, but better than nothing. + +The [trello board](http://trello.hledger.org) (trello.hledger.org) is a categorised collection of wishlist items, +this should probably be considered deprecated. + +### Make + +A Makefile is provided to make common developer tasks easy to remember, +and to insulate us a little from the ever-evolving Haskell tools ecosystem. +Using it is entirely optional, but recommended. +You'll need [GNU Make](http://www.gnu.org/software/make) installed. + +The Makefile is self-documenting. Run `make` to see a list of the main make rules: + +```shell +$ make +Makefile:37: -------------------- hledger make rules -------------------- +Makefile:39: make [help] -- list documented rules in this makefile. make -n RULE shows more detail. +Makefile:204: (INSTALLING:) +Makefile:206: make install -- download dependencies and install hledger executables to ~/.local/bin or equivalent (with stack) +Makefile:231: (BUILDING:) +Makefile:235: make build -- download dependencies and build hledger executables (with stack) +Makefile:304: make hledgerdev -- quickly build the hledger executable (with ghc and -DDEVELOPMENT) +... +``` + +To see what a make rule will do without actually doing it, use the `-n` flag: + +```shell +$ make build -n +stack build +``` +```shell +$ make test -n +(stack test \ + && echo pkgtest PASSED) || echo pkgtest FAILED +(stack exec hledger test \ + && echo builtintest PASSED) || echo builtintest FAILED +(COLUMNS=80 PATH=`pwd`/bin:/home/simon/src/hledger/bin:/home/simon/.local/bin:/home/simon/.cabal/bin:/opt/ghc/7.10.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/var/lib/gems/1.9.1/bin stack exec -- shelltest --execdir -- -j16 --hide-successes tests \ + && echo functest PASSED) || echo functest FAILED +``` + +### Shake + +`Shake.hs` in the top directory complements the Makefile; it is used for some more complex +scripts relating to documentation. + +### Tests + +Run the package tests of all (or selected) packages. +Does not include hledger's functional tests. +```shell +$ stack test [PKG] +``` + +Run some hledger unit tests via a built-in hledger command: +```shell +$ [stack exec] hledger test +``` + +Run hledger's functional tests: +```shell +$ stack install shelltestrunner +$ make functest +``` + +Run both unit and functional tests: +```shell +$ make test +``` + +Test generation of haddock docs: +```shell +$ make haddocktest +``` + + +### Benchmarking + +Benchmarks are standard performance measurements, +which we define using `bench` declarations in cabal files. +There is [one in hledger.cabal](https://github.com/simonmichael/hledger/blob/master/hledger/hledger.cabal#L228), +with related code and data files in [hledger/bench/](https://github.com/simonmichael/hledger/tree/master/hledger/bench). + +To run the standard hledger benchmark, use `stack bench hledger`. +This installs haskell dependencies (but not system dependencies) and rebuilds as needed, +then runs [hledger/bench/bench.hs](https://github.com/simonmichael/hledger/blob/master/hledger/bench/bench.hs), +which by default shows quick elapsed-time measurements for several operations on a standard data file: + +```shell +$ stack bench hledger +NOTE: the bench command is functionally equivalent to 'build --bench' +... +hledger-0.27: benchmarks +Running 1 benchmarks... +Benchmark bench: RUNNING... +Benchmarking hledger in /Users/simon/src/hledger/hledger with timeit +read bench/10000x1000x10.journal [1.57s] +print [1.29s] +register [1.92s] +balance [0.21s] +stats [0.23s] +Total: 5.22s +Benchmark bench: FINISH +``` + +bench.hs has some other modes, which you can use by compiling and running it directly. +`--criterion` reports more detailed and dependable measurements, but takes longer: + +```shell +$ cd hledger; stack exec -- ghc -ibench bench/bench && bench/bench --criterion +... +Linking bench/bench ... +Benchmarking hledger in /Users/simon/src/hledger/hledger with criterion +benchmarking read bench/10000x1000x10.journal +time 1.414 s (1.234 s .. 1.674 s) + 0.996 R² (0.989 R² .. 1.000 R²) +mean 1.461 s (1.422 s .. 1.497 s) +std dev 59.69 ms (0.0 s .. 62.16 ms) +variance introduced by outliers: 19% (moderately inflated) + +benchmarking print +time 1.323 s (1.279 s .. 1.385 s) + 1.000 R² (0.999 R² .. 1.000 R²) +mean 1.305 s (1.285 s .. 1.316 s) +std dev 17.20 ms (0.0 s .. 19.14 ms) +variance introduced by outliers: 19% (moderately inflated) + +benchmarking register +time 1.995 s (1.883 s .. 2.146 s) + 0.999 R² (0.998 R² .. NaN R²) +mean 1.978 s (1.951 s .. 1.995 s) +std dev 25.09 ms (0.0 s .. 28.26 ms) +variance introduced by outliers: 19% (moderately inflated) + +benchmarking balance +time 251.3 ms (237.6 ms .. 272.4 ms) + 0.998 R² (0.997 R² .. 1.000 R²) +mean 260.4 ms (254.3 ms .. 266.5 ms) +std dev 7.609 ms (3.192 ms .. 9.638 ms) +variance introduced by outliers: 16% (moderately inflated) + +benchmarking stats +time 325.5 ms (299.1 ms .. 347.2 ms) + 0.997 R² (0.985 R² .. 1.000 R²) +mean 329.2 ms (321.5 ms .. 339.6 ms) +std dev 11.08 ms (2.646 ms .. 14.82 ms) +variance introduced by outliers: 16% (moderately inflated) +``` + +`--simplebench` shows a table of elapsed-time measurements for the commands defined in [bench/default.bench](https://github.com/simonmichael/hledger/blob/master/hledger/bench/default.bench). +It can also show the results for multiple h/ledger executables side by side, if you tweak the bench.hs code. +Unlike the other modes, it does not link with the hledger code directly, but runs the "hledger" executable found in $PATH (so ensure that's the one you intend to test). + +```shell +$ cd hledger; stack exec -- ghc -ibench bench/bench && bench/bench --simplebench +Benchmarking /Users/simon/.local/bin/hledger in /Users/simon/src/hledger/hledger with simplebench and shell +Using bench/default.bench +Running 4 tests 1 times with 1 executables at 2015-08-23 16:58:59.128112 UTC: +1: hledger -f bench/10000x1000x10.journal print [3.27s] +1: hledger -f bench/10000x1000x10.journal register [3.65s] +1: hledger -f bench/10000x1000x10.journal balance [2.06s] +1: hledger -f bench/10000x1000x10.journal stats [2.13s] + +Summary (best iteration): + ++-----------------------------------------++---------+ +| || hledger | ++=========================================++=========+ +| -f bench/10000x1000x10.journal print || 3.27 | +| -f bench/10000x1000x10.journal register || 3.65 | +| -f bench/10000x1000x10.journal balance || 2.06 | +| -f bench/10000x1000x10.journal stats || 2.13 | ++-----------------------------------------++---------+ +``` + +bench's --simplebench mode is based on a standalone tool, [tools/simplebench.hs](https://github.com/simonmichael/hledger/blob/master/tools/simplebench.hs). +simplebench.hs is a generic benchmarker of one or more executables (specified on the command line) against one or more sets of command-line arguments (specified in a file). +It has a better command-line interface than bench.hs, so you may find it more convenient +for comparing multiple hledger versions, or hledger and ledger. Eg: + +```shell +$ stack exec -- ghc tools/simplebench +[1 of 1] Compiling Main ( tools/simplebench.hs, tools/simplebench.o ) +Linking tools/simplebench ... +``` +```shell +$ tools/simplebench -h +tools/simplebench -h +simplebench: at least one executable needed +bench [-f testsfile] [-n iterations] [-p precision] executable1 [executable2 ...] + +Run some functional tests with each of the specified executables, +where a test is "zero or more arguments supported by all executables", +and report the best execution times. + + -f testsfile --testsfile=testsfile file containing tests, one per line, default: bench.tests + -n iterations --iterations=iterations number of test iterations to run, default: 2 + -p precision --precision=precision show times with this precision, default: 2 + -v --verbose show intermediate results + -h --help show this help + +Tips: +- executables may have arguments if enclosed in quotes +- tests can be commented out with # +- results are saved in benchresults.{html,txt} +``` +```shell +cd hledger; $ ../tools/simplebench -f bench/default.bench hledger ledger +Using bench/default.bench +Running 4 tests 2 times with 2 executables at 2015-08-24 04:24:37.257068 UTC: + +Summary (best iteration): + ++-----------------------------------------++---------+--------+ +| || hledger | ledger | ++=========================================++=========+========+ +| -f bench/10000x1000x10.journal print || 3.24 | 0.43 | +| -f bench/10000x1000x10.journal register || 3.80 | 3.48 | +| -f bench/10000x1000x10.journal balance || 2.05 | 0.18 | +| -f bench/10000x1000x10.journal stats || 2.10 | 0.19 | ++-----------------------------------------++---------+--------+ +``` + +Finally, for quick, fine-grained performance measurements when troubleshooting or optimising, I use +[dev.hs](https://github.com/simonmichael/hledger/blob/master/dev.hs). + +#### Sample journal files + +Synthetic data files like `examples/100x100x10.journal` are useful for benchmarks and testing. +The numbers describe the number of transactions, number of accounts, and maximum account depth respectively. +They are generated by [`tools/generatejournal.hs`](https://github.com/simonmichael/hledger/blob/master/tools/generatejournal.hs). +They should get built automatically as needed, if not you can use `make samplejournals`: + +```shell +$ make samplejournals +ghc tools/generatejournal.hs +[1 of 1] Compiling Main ( tools/generatejournal.hs, tools/generatejournal.o ) +Linking tools/generatejournal ... +tools/generatejournal 100 100 10 >examples/100x100x10.journal +tools/generatejournal 1000 1000 10 >examples/1000x1000x10.journal +tools/generatejournal 1000 10000 10 >examples/1000x10000x10.journal +tools/generatejournal 10000 1000 10 >examples/10000x1000x10.journal +tools/generatejournal 10000 10000 10 >examples/10000x10000x10.journal +tools/generatejournal 100000 1000 10 >examples/100000x1000x10.journal +tools/generatejournal 3 5 5 >examples/ascii.journal +tools/generatejournal 3 5 5 --chinese >examples/chinese.journal +tools/generatejournal 3 5 5 --mixed >examples/mixed.journal +``` + ### Documentation Project documentation lives in a number of places: @@ -720,72 +731,53 @@ Project documentation lives in a number of places: ### Code -The hledger repo is hosted on Github, at . -You can also jump there via `code.hledger.org[/commits]`. +hledger is a suite of applications, tools and libraries. +The main hledger code repository is [github.com/simonmichael/hledger](http://github.com/simonmichael/hledger) +(aka `code.hledger.org`). +There are also various hledger addons maintained as separate projects with their own repos. -### Quality control +Within the main repo, there are a number of separate cabal packages, +making it easier to pick and choose parts of hledger to install or to package. +They are as follows: -Relevant tools include: +#### hledger-lib -- unit tests (HUnit, make unittest) -- functional tests (shelltestrunner, make functest) -- performance tests (simplebench, make bench) -- documentation tests (make haddocktest + manual) -- ui tests (manual) -- installation tests (manual) -- code reviews +[package](http://hackage.haskell.org/package/hledger-lib), +[exported modules](http://hackage.haskell.org/package/hledger-lib#modules), +[code](https://github.com/simonmichael/hledger/tree/master/hledger-lib) -### Code reviews +Core data models, parsing, standard reports, and utilities. +Most data types are defined in [Hledger.Data.Types](http://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html), +while functions that operate on them are defined in Hledger.Data.TYPENAME. +Under [Hledger.Read](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Read.hs) +are parsers for the supported input formats. +Data files are parsed into a +[Journal](http://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Journal), +which contains a list of +[Transactions](http://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Transaction), +each containing multiple +[Postings](http://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Posting) +of some +[MixedAmount](http://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:MixedAmount) +(multiple +single-[CommoditySymbol](http://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:CommoditySymbol) +[Amounts](http://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Amount)) +to some +[AccountName](http://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:AccountName). +When needed, the Journal is further processed to derive a +[Ledger](http://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Ledger), +which contains summed +[Accounts](http://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html#t:Account). +In [Hledger.Reports](http://hackage.haskell.org/package/hledger-lib/docs/Hledger-Reports.html) +there are standard reports, which extract useful data from the Journal or Ledger. -We have held one code review party, in July 2014, on the mail list and IRC channel. -Here's the original [proposal](http://article.gmane.org/gmane.comp.finance.ledger.hledger/1070) giving some motivation, and the discussion logs, note these are a good source of hledger development tips: - -- 2014/7/21-25 **hledger-web code & UI** - [mail thread](http://thread.gmane.org/gmane.comp.finance.ledger.hledger/1070), - [IRC log](http://hledger.org/static/irc-20140725-code-review.html) - -Dev sprint/party #2 was on 2015/10/10. - -- [announcement, report](http://thread.gmane.org/gmane.comp.finance.ledger.hledger/1254) -- [chat](http://ircbrowse.net/day/hledger/2015/10/10) - [log](http://ircbrowse.net/day/hledger/2015/10/11) - - - - - - - - - - - - - - - - - - - - -## Code architecture -### hledger - -There are two core cabal packages: - -**[hledger-lib](http://hackage.haskell.org/package/hledger-lib)** - data model, parsing, manipulation, standard reports -([github](https://github.com/simonmichael/hledger/tree/master/hledger-lib))\ -**[hledger](http://hackage.haskell.org/package/hledger)** - command line interface, reusable cli options & helpers -([github](https://github.com/simonmichael/hledger/tree/master/hledger)) - -Most data types are defined in [hledger-lib:Hledger.Data.Types](http://hackage.haskell.org/package/hledger-lib/docs/Hledger-Data-Types.html), -while functions that operate on them are defined in -hledger-lib:Hledger.Data.TYPENAME. Here's a diagram of the main data model: - - -hledger parses the journal file into a -[Journal](http://hackage.haskell.org/package/hledger-lib-0.23.2/docs/Hledger-Data-Types.html#t:Journal), -which contains a list of -[Transactions](http://hackage.haskell.org/package/hledger-lib-0.23.2/docs/Hledger-Data-Types.html#t:Transaction), -each containing multiple -[Postings](http://hackage.haskell.org/package/hledger-lib-0.23.2/docs/Hledger-Data-Types.html#t:Posting) -of some -[MixedAmount](http://hackage.haskell.org/package/hledger-lib-0.23.2/docs/Hledger-Data-Types.html#t:MixedAmount) -(multiple -single-[CommoditySymbol](http://hackage.haskell.org/package/hledger-lib-0.23.2/docs/Hledger-Data-Types.html#t:CommoditySymbol) -[Amounts](http://hackage.haskell.org/package/hledger-lib-0.23.2/docs/Hledger-Data-Types.html#t:Amount)) -to some -[AccountName](http://hackage.haskell.org/package/hledger-lib-0.23.2/docs/Hledger-Data-Types.html#t:AccountName). -Commands get and render -[Reports](http://hackage.haskell.org/package/hledger-lib-0.23.2/docs/Hledger-Reports.html) -from the Journal, or sometimes from a -[Ledger](http://hackage.haskell.org/package/hledger-lib-0.23.2/docs/Hledger-Data-Types.html#t:Ledger), -which contains -[Accounts](http://hackage.haskell.org/package/hledger-lib-0.23.2/docs/Hledger-Data-Types.html#t:Account) -representing the summed balances and other details of each account. +#### hledger -After surveying the packages, modules, and data types, try tracing the execution of a hledger command: +[package](http://hackage.haskell.org/package/hledger), +[exported modules](http://hackage.haskell.org/package/hledger#modules), +[code](https://github.com/simonmichael/hledger/tree/master/hledger), +[manual](/manual.html#hledger) -1. CLI stuff is in [hledger:Hledger.Cli](https://github.com/simonmichael/hledger/tree/master/hledger/Hledger/Cli). -2. [hledger:Hledger.Cli.Main:main](https://github.com/simonmichael/hledger/blob/master/hledger/Hledger/Cli/Main.hs#L179) +hledger's command line interface, and command line options and utilities for other hledger tools. + +Try tracing the execution of a hledger command: + +1. [Hledger.Cli.Main:main](https://github.com/simonmichael/hledger/blob/master/hledger/Hledger/Cli/Main.hs#L302) parses the command line to select a command, then -3. gives it to -[hledger:Hledger.Cli.Utils:withJournalDo](https://github.com/simonmichael/hledger/blob/master/hledger/Hledger/Cli/Utils.hs#L61), +2. gives it to +[Hledger.Cli.Utils:withJournalDo](https://github.com/simonmichael/hledger/blob/master/hledger/Hledger/Cli/Utils.hs#L73), which runs it after doing all the initial parsing. -4. Parsing code is under +3. Parsing code is under [hledger-lib:Hledger.Read](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Read.hs), -eg the -[hledger-lib:Hledger.Read.JournalReader](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Read/JournalReader.hs). -5. Commands extract useful information from the parsed data model using +eg [Hledger.Read.JournalReader](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Read/JournalReader.hs). +4. Commands extract useful information from the parsed data model using [hledger-lib:Hledger.Reports](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Reports), and -6. render it to the console. -7. Everything uses the types and data -utilities under -[hledger-lib:Hledger.Data](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Data), -and the general helpers from -[hledger-lib:Hledger.Utils](https://github.com/simonmichael/hledger/blob/master/hledger-lib/Hledger/Utils.hs) -and below. +5. render in plain text for console output (or another output format, like CSV). +6. Everything uses the data types and utilities from +[hledger-lib:Hledger.Data](https://github.com/simonmichael/hledger/tree/master/hledger-lib/Hledger/Data) +and [hledger-lib:Hledger.Utils](https://github.com/simonmichael/hledger/blob/master/hledger-lib/Hledger/Utils.hs). -### hledger-web +#### hledger-ui -hledger-web is in a third cabal package: +[package](http://hackage.haskell.org/package/hledger-ui), + +[code](https://github.com/simonmichael/hledger/tree/master/hledger-ui), +[manual](/manual.html#hledger-ui) -**[hledger-web](http://hackage.haskell.org/package/hledger-web)** - web interface -([github](https://github.com/simonmichael/hledger/tree/master/hledger-web)) +A curses-style text interface. -It is a single-executable web application using the -[yesod](http://yesodweb.com) framework. It runs a built-in web server -serving some views of the journal file, reading it at startup and -again whenever it changes. It can also append new transactions to the journal file. -There are two main views, which can be filtered with [query arguments](manual#query-arguments): +#### hledger-web + +[package](http://hackage.haskell.org/package/hledger-web), +[exported modules](http://hackage.haskell.org/package/hledger-web#modules), +[code](https://github.com/simonmichael/hledger/tree/master/hledger-web), +[manual](/manual.html#hledger-web) + +A web interface. +hledger-web starts a web server built with the yesod framework, +and (by default) opens a web browser view on it. +It reads the journal file(s) at startup and again whenever they change. +It can also write (append) new transactions to the journal file. + +There are two main views, which can be filtered with +[queries](/manual.html#queries): - [/journal](http://demo.hledger.org/journal), showing general journal entries (like `hledger print`) -- [/register](http://demo.hledger.org/register?q=inacct:Assets:Bank:Checking), +- [/register](http://demo.hledger.org/register?q=inacct:Expenses:Food), showing transactions affecting an account (slightly different from - `hledger register`, which shows postings). + hledger's [register](/manual.html#register) command, which shows postings). There is also: @@ -898,22 +885,67 @@ Most of the action is in - [static/hledger.js](https://github.com/simonmichael/hledger/tree/master/hledger-web/static/hledger.js) - [static/hledger.css](https://github.com/simonmichael/hledger/tree/master/hledger-web/static/hledger.css) -Handler module and function names end with R, like the Yesod-generated route type they deal with. +Handler module and function names end with R, like the yesod-generated route type they deal with. Dynamically generated page content is mostly inline hamlet. Lucius/Julius files and widgets generally are not used, except for the default layout. -The quickest way to test changes is `cd hledger-web; stack ghci hledger-web`, `:main --serve`, control-C, `:r`, repeat. -No linking is required, and changes to static files like hledger.js are visible after reloading a page. +Here are some ways to run it during development: -Another way is `yesod devel`, which rebuilds automatically when files -change, including config files, templates and static files (but only in the hledger-web package). +- `yesod devel`: runs in developer mode, rebuilds automatically when config, template, static or haskell files change +(but only files in the hledger-web package): +```shell +$ (cd hledger-web; yesod devel) +``` -A third way is `make autoweb`, if you can get it working (see the -makefile for instructions). This rebuilds automatically when haskell -files change in any of the hledger{-lib,,-web} packages. +- `stack ghci`: runs the server in developer mode from GHCI. +Changes to static files like hledger.js will be visible on page reload; +to see other changes, restart it as shown. +```shell +$ (cd hledger-web; stack ghci hledger-web) +hledger-web> :main --serve # restart: ctrl-c, :r, enter, ctrl-p, ctrl-p, enter +``` - +- `make ghci-web`: runs the server in developer mode from GHCI, also +interprets the hledger-lib and hledger packages so that :reload picks +up changes in those packages too: +```shell +$ make ghci-web +ghci> :main --serve +``` +(This rule also creates symbolic links to hledger-web's `config`, `messages`, `static` and `templates` +directories, needed in developer mode, so it can run from the top directory. This may not work on Windows.) + +#### hledger-api + +[package](http://hackage.haskell.org/package/hledger-api), + +[code](https://github.com/simonmichael/hledger/tree/master/hledger-api), +[manual](/manual.html#hledger-api) + +A web API server. Uses the servant framework. + +### Quality + +Relevant tools include: + +- unit tests (HUnit, make unittest) +- functional tests (shelltestrunner, make functest) +- performance tests (simplebench, make bench) +- documentation tests (make haddocktest + manual) +- ui tests (manual) +- installation tests (manual) +- code reviews + +### Code review + +- Code review party 2014/7/21-25: + [discussion](http://thread.gmane.org/gmane.comp.finance.ledger.hledger/1070) +- Dev sprint/party 2015/10/10: + [discussion](http://thread.gmane.org/gmane.comp.finance.ledger.hledger/1254) ## Quick links @@ -991,3 +1023,4 @@ libghc-hledger-lib-prof --> +