Simon Michael 10163f698a doc: devguide: how to run ghci (#274 )

2015-07-19 09:34:34 -07:00

32 KiB

Raw Blame History

Developer guide

Contributors of all levels are most welcome in the hledger project. This guide is action-oriented: below you’ll find useful links, then procedures, then general info.

Quick links

IRC	Join #hledger (chat log; see also #ledger)
Mail list	list.hledger.org (Gmane)
Twitter	#ledgercli, @LedgerTips
hledger-web demo	demo.hledger.org
Trello	backlog/wishlist planning board
Github	code.hledger.org commits, COMMITS (turn up your volume), unreleased commits, release branch commits build status (hydra) open bugs, all bugs, open issues, all issues, issues with bounties, issue bounties @ bountysource.com
Hackage	packages: hledger-lib, hledger, hledger-web, hledger-diff, hledger-interest, hledger-irr, hledger-vty, hledger-chart, hledger build history/GHC compatibility: hledger-lib, hledger, hledger-web outdated dependencies: hledger-lib, hledger, hledger-web reverse dependencies: hledger-lib, hledger download stats (BestHaskell)
Stackage	hledger entry, issues build status (jenkins), last build output: 7.4, 7.6, 7.8, 7.10
Debian	source packages haskell-hledger-lib, bugs, haskell-hledger, bugs, haskell-hledger-web, bugs binary packages testing: hledger, bugs, hledger-web, bugs unstable: hledger, bugs, hledger-web, bugs all: hledger popularity stats: hledger, hledger-web PTS help
Ubuntu	source packages haskell-hledger-lib, bugs, haskell-hledger, bugs, haskell-hledger-web, bugs binary packages hledger
Gentoo	hledger, hledger-web, hledger
Fedora	hledger, hledger, Haskell SIG

How to..

Here are some suggested procedures to help us all quickly (re)engage with the project and get things done.

Suggest enhancements

Suggestions and feature requests (aka wishes) are very welcome, but we don’t want them to pile up in the issue tracker obscuring bugs and higher-priority tasks. So before opening a github issue, first consider:

The #hledger (irc.hledger.org) IRC channel on freenode and the mail list (list.hledger.org) are excellent places for discussing and refining ideas. Both are archived and linkable, so the idea won’t be lost. IRC is quick (if I’m not online, leave a comment anyway), while the mail list has the most readers.
The trello board (trello.hledger.org) is a good place for storing and prioritising backlog and wishlist items of all kinds.
The bug tracker (bugs.hledger.org) on github is best used for issues with the existing product or docs. If you’re not sure if you’re reporting a bug or not, it’s fine to post it here.

When wishes land in the bug tracker, they get the WISH label, and these are excluded from the default view given by bugs.hledger.org.

Report problems

for quick help or if you’re not sure about the problem, you can ask on the #hledger (irc.hledger.org) IRC channel or the mail list (list.hledger.org). If #hledger does not respond quickly, you can leave the window open and check back later, or leave your email address.
check the open and closed issues in the bug tracker (bugs.hledger.org). Sometimes the problem has been fixed in git but not yet released.
report new issues in the bug tracker (shortcut: bugs.hledger.org/new)

Help with testing

review and test our documentation and web presence
download and test the binaries on your platform
test installing via cabal
use the tools and test functionality, usability, browser compatibility, ui layout etc.
check that hledger test reports no failures
run the developer tests
discuss/report problems via irc/mail list/bug tracker

Help with bug tracking

get to know the bug tracker and its contents
research and update issues
some convenient url shortcuts:
bugs.hledger.org
bugs.hledger.org/new
bugs.hledger.org/N

Set up for development

Download and install stack. This builds haskell projects and can also install GHC (and on windows, git) if needed.

Get the hledger repo:

$ git clone https://github.com/simonmichael/hledger.git
$ cd hledger

Install GNU Make and other optional extra tools (see the Makefile for a list):
```
$ stack install shelltestrunner hasktags profiteur hpack
```

Run make or stack --help to see some suggested commands:

$ make
Makefile:37: -------------------- hledger make rules --------------------
Makefile:39: make [help] -- list documented rules in this makefile. make -n RULE shows more detail. 
Makefile:204: (INSTALLING:)
Makefile:206: make install -- download dependencies and install hledger executables to ~/.local/bin or equivalent (with stack)
Makefile:231: (BUILDING:)
Makefile:235: make build -- download dependencies and build hledger executables (with stack)
Makefile:304: make hledgerdev -- quickly build the hledger executable (with ghc and -DDEVELOPMENT) 
...
$ stack --help
Available commands:
  build                    Build the project(s) in this directory/configuration
  install                  Build executables and install to a user path
  test                     Build and test the project(s) in this
                           directory/configuration
  bench                    Build and benchmark the project(s) in this
                           directory/configuration
...

Old instructions:

Get GHC and cabal-install installed. I recommend the stackage.org install guide. You can see which GHC versions are officially supported in the tested-with field in hledger.cabal and hledger-web.cabal. Older versions may also work.
Get git installed.
Get GNU Make installed (unless you don’t care about the Makefile’s conveniences). On some platforms the command will be eg gmake instead of make.

Get the hledger repo:

$ git clone https://github.com/simonmichael/hledger.git

You might want to install or upgrade some of these haskell developer tools. If you’re not sure, skip this step and return to it as needed. Be sure the cabal bin directory where these are installed (eg ~/.cabal/bin) is in your PATH.

$ cabal update
$ cabal install alex happy       # if you get alex/happy-related errors when building hledger
$ cabal install haddock          # needed to build hledger API docs
$ cabal install shelltestrunner  # needed to run hledger functional tests (may need latest git version)
$ cabal install hoogle hlint     # maybe useful for searching API docs and checking code

You’ll also want a comfortable code editor, preferably with Haskell support. (I use emacs + haskell-mode, or occasionally IntelliJ IDEA + one of the plugins).

Install haskell libs required by hledger:
```
$ cd hledger
$ cabal sandbox init   # optional
$ make installdeps     # or cabal install --only-dep ./hledger-lib ./hledger [./hledger-web]
```
This will install the required dependencies from Hackage. If you’re new to cabal, you can expect problems at this stage. The usual remedy is to ensure you start with a clean package db, eg by doing cabal sandbox init. You can simplify and speed up this step a lot by commenting out hledger-web in the PACKAGES list in the Makefile.
Build with cabal:
```
$ make cabalbuild
```
(Tip: make cabalCMD runs cabal CMD in each of the hledger packages).
Build with GHC:
```
$ make bin/hledgerdev
```
This builds hledger (and hledger-lib) with GHC directly, without using cabal, and as quickly as possible, without optimizations (the “dev” suffix is a reminder of this). I use and recommend this method for development, as it crosses package boundaries and ensures you are building the latest code. However it needs some files generated by cabal build, which is why we did that first.

Get your changes accepted

Follow the usual github workflow:

fork the main hledger repo on github,
git clone it to your local machine,
git commit, after (?) pulling and merging the latest upstream changes
git push back to github,
open a pull request on github,
follow up on any discussion there.

If you’re new to this process, help.github.com may be useful.

Improve the documentation

get familiar with the website and documentation online, review and test
get familiar with the site/doc source files (see Makefile)
set up for hledger development
send patches with names prefixed with “doc:” (or “site:”)

Use the REPL (GHCI)

These all work from the main hledger source directory (at least).

First, ensure all required dependencies are installed with these commands. (You might also need to install some system libs like terminfo or curses.)

$ stack test
$ stack bench

Get a GHCI prompt for hledger-lib:

$ stack ghci hledger-lib

Get a GHCI prompt for hledger:

$ stack ghci hledger

Get a GHCI prompt for hledger-web:

$ stack ghci hledger-web

hledger-web also needs to find some things in its current directory (like the static/ directory). This normally just works, if not please send details.

Get a GHCI prompt for hledger and hledger-lib:

$ make ghci

Get a GHCI prompt for hledger-web, hledger and hledger-lib:

$ make ghci-web

Run the tests

This command will install haskell dependencies (you might need to install additional system dependencies yourself) and run the package test suites. Currently these consist of hledger-lib’s unit tests, hledger’s unit tests, and hledger-web’s functional tests:

$ stack test [PKG]

Run the hledger-lib and hledger unit tests as a built-in hledger command:

$ [stack exec] hledger test

Run the hledger functional tests:

$ stack install shelltestrunner  # if not already done
$ make functest

Run both unit and functional tests:

$ make test

Test haddock doc generation:

$ make haddocktest

Add a test

identify what to test
choose the test type: unit ? functional ? benchmark ?
currently expected to pass or fail ?
figure out where it goes
write test, verify expected result
get it committed

Fix a bug or add a feature

research, discuss, validate the issue/feature on irc/list/bug tracker
look for related tests, run the tests and check they are passing
add a test ?
develop a patch
include any related issue numbers in the patch name, eg: “fix for blah blah (#NNN)”
get it committed

Become a contributor

after getting one or more patches committed, read and sign the contributor list & agreement
or, ask to be added

Do code review

review and discuss new pull requests and commits on github
set up for development and test the latest changes in your own repo
read the existing code docs and source
send feedback or discuss via irc or list

Help with packaging

package hledger for linux distros, macports, etc.
develop mac/windows installers
find and assist distro packagers/installer developers

Help with project management

clarify/update goals and principles
monitor, report on project progress and performance
research, compare and report on successful projects, related projects
identify collaboration opportunities
marketing, communication, outreach
release management, roadmap planning

Do a major release

review the release how-to in the developer guide
- and update as needed (make site-preview, http://localhost:8000/developer-guide.html#do-a-major-release)
clean working copy
- commit/stash/clear any pending changes in working copy
- merge any commits from other branches & working copies
- check out master, or release branch. Major releases are done in master if possible. If not, do as much of the below as is feasible in master, then start a release branch (git checkout -b X.Y)
ensure tests pass
- make unittest
- make functest
- make haddocktest
- make cabalfiletest
update dependencies
- check & fix any outdated upper bounds (dev guide -> quick links -> hackage)
update cabal files
- /hledger.cabal
  - descriptions
  - tested-with
  - file lists
    - data-files
    - extra-tmp-files
    - extra-source-files
    - exposed-modules
    - other-modules
update stack.yaml file
- resolver
- extra-deps
- flags
update docs
- haddocks
- */CHANGES
- doc/site/release-notes.md
- doc/contributors.md
- doc/manual.md (commands, options, –help, ledger compatibility..)
- doc/site/step-by-step.md
- doc/site/how-to-*
- doc/site/faq.md (ledger compatibility)
- doc/site/installing.md
- doc/site/download.md
- doc/developer-guide.md
- doc/ANNOUNCE
update version
- edit .version
- make setversion
- double-check & commit (cabal files, manual, download page..)
make tarballs/binaries
- ensure no packages are commented out in Makefile’s PACKAGES
- make cabalsdist
- [make windows binaries]
- [make mac binaries]
release tests
- make haddocktest
- make cabalfiletest
- cabal tarballs install into a clean directory without warnings
- cabal upload –dry reports no problems
tag
- make tagrelease
publish
- stack upload hledger-lib; stack upload hledger; stack upload hledger-web (or make cabalsdist hackageupload[-dry])
- [wait a day for it to appear in stackage nightly ?]
- ensure hackage is showing the latest haddocks
- check the hackage build matrix
- git push –tags
- deploy at demo.hledger.org
- [upload binaries to hledger.org]
- ensure the website is showing latest docs (download links, release notes, manual, how-tos, dev guide links, etc.)
announce
- review/close open issues in tracker
- email doc/ANNOUNCE to hledger, haskell-cafe, haskell, [ledger] lists
- tweet
- [blog]
- [reddit]
- update release notes with announcement link & short description
post-release
- handle problem reports, support requests

Do a minor release

Differences from a major release: work in a release branch, set PACKAGES only to the affected package(s), don’t run make setversion.

cleanup
- review working copies (laptop, server, website) & branches, commit pending changes
document
- /.cabal for affected package(s) (descriptions, tested-with, files..)
- */CHANGES for affected package(s)
- doc/site/release-notes.md
- doc/manual.md (commands, options, –help, ledger compatibility..)
- doc/site/step-by-step.md
- doc/site/how-to-*
test
- make unittest
- make functest
- make haddocktest
branch
- switch to release branch (git checkout X.Y)
version
- edit .version (don’t make setversion)
- manually bump version for affected package(s): cabal files, manual..
package
- set Makefile’s PACKAGES to affected package(s)
- make cabalsdist
test
- install from tarball(s) into a clean directory
tag
- make tagrelease
push
- git push –tags
upload
- make cabalupload
announce
- [email hledger]
- [tweet]

Project overview

Mission, principles, goals

The hledger project aims to produce:

a practical, accessible, dependable tool for end users
a useful library and toolbox for finance-minded haskell programmers
a successful, time-and-money-solvent project within a thriving ecosystem of financial software projects.

Roles and activities

newcomer/potential user
user
library user
field tester
bug wrangler
support
documentor
qa
developer
packager
communicator
project manager

Documentation

Project documentation lives in a number of places:

doc/*.md and doc/site/*.md form the hledger.org website, which is generated with hakyll[-std] and pandoc
haddock documentation in the code appears on Hackage
short blurbs: cabal files, module headers, HCAR, GSOC project, ..
doc/notes.org has some old developer notes
developer reports (profiles, benchmarks, coverage..) in doc/profs, sometimes published at hledger.org/profs

Code

The hledger repo is hosted on Github, at http://github.com/simonmichael/hledger. You can also jump there via code.hledger.org[/commits].

Quality control

Relevant tools include:

unit tests (HUnit, make unittest)
functional tests (shelltestrunner, make functest)
performance tests (simplebench, make bench)
documentation tests (make haddocktest + manual)
ui tests (manual)
installation tests (manual)
code reviews

Code reviews

We have held one code review party, in July 2014, on the mail list and IRC channel. Here’s the original proposal giving some motivation, and the discussion logs, note these are a good source of hledger development tips:

2014/7/21-25 hledger-web code & UI mail thread, IRC log

Implementation notes

hledger

There are two core cabal packages:

hledger-lib - data model, parsing, manipulation, standard reports (github)
hledger - command line interface, reusable cli options & helpers (github)

Most data types are defined in hledger-lib:Hledger.Data.Types, while functions that operate on them are defined in hledger-lib:Hledger.Data.TYPENAME. Here’s a diagram of the main data model:

hledger parses the journal file into a Journal, which contains a list of Transactions, each containing multiple Postings of some MixedAmount (multiple single-Commodity Amounts) to some AccountName. Commands get and render Reports from the Journal, or sometimes from a Ledger, which contains Accounts representing the summed balances and other details of each account.

After surveying the packages, modules, and data types, try tracing the execution of a hledger command:

CLI stuff is in hledger:Hledger.Cli.
hledger:Hledger.Cli.Main:main parses the command line to select a command, then
gives it to hledger:Hledger.Cli.Utils:withJournalDo, which runs it after doing all the initial parsing.
Parsing code is under hledger-lib:Hledger.Read, eg the hledger-lib:Hledger.Read.JournalReader.
Commands extract useful information from the parsed data model using hledger-lib:Hledger.Reports, and
render it to the console.
Everything uses the types and data utilities under hledger-lib:Hledger.Data, and the general helpers from hledger-lib:Hledger.Utils and below.

hledger-web

hledger-web is in a third cabal package:

hledger-web - web interface (github)

It is a single-executable web application using the yesod framework. It runs a built-in web server serving some views of the journal file, reading it at startup and again whenever it changes. It can also append new transactions to the journal file. There are two main views, which can be filtered with query arguments:

/journal, showing general journal entries (like hledger print)
/register, showing transactions affecting an account (slightly different from hledger register, which shows postings).

There is also:

a sidebar (toggled by pressing s) showing the chart of accounts (like hledger balance)
an add form for adding new transactions (press a)
a help dialog showing quick help and keybindings (press h or click ?)

Most of the action is in

Handler module and function names end with R, like the Yesod-generated route type they deal with.

Dynamically generated page content is mostly inline hamlet. Lucius/Julius files and widgets generally are not used, except for the default layout.

The quickest way to test changes is make ghciweb, :main --serve, control-C, :r, repeat. No linking is required, and changes to static files like hledger.js are visible after reloading a page.

Another way is yesod devel, which rebuilds automatically when files change, including config files, templates and static files (but only in the hledger-web package).

A third way is make autoweb, if you can get it working (see the makefile for instructions). This rebuilds automatically when haskell files change in any of the hledger{-lib,,-web} packages.

32 KiB Raw Blame History Unescape Escape