From 6e09a4897d4cedc07d01dec8cbfa6596e78c0064 Mon Sep 17 00:00:00 2001 From: Simon Michael Date: Mon, 2 Jul 2012 01:01:20 +0000 Subject: [PATCH] notes; start cleanup, add estimates, document process --- NOTES.org | 3053 +++++++++++++++++++++++++++-------------------------- 1 file changed, 1554 insertions(+), 1499 deletions(-) diff --git a/NOTES.org b/NOTES.org index eae9bb31f..327e19290 100644 --- a/NOTES.org +++ b/NOTES.org @@ -1,1493 +1,24 @@ hledger project notes -* backlog - -** errors -*** double quote matches everything ? 1h -*** text: duplicate runs -$ hledger test 'showTransaction$' -Cases: 6 Tried: 0 Errors: 0 Failures: 0([],"") -### Failure in: 0:showTransaction -show a balanced transaction, eliding last amount -expected: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n assets:checking\n\n" - but got: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n\n assets:checking \n\n\n" -Cases: 6 Tried: 1 Errors: 0 Failures: 1([],"") -### Failure in: 1:showTransaction -show a balanced transaction, no eliding -expected: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n assets:checking $-47.18\n\n" - but got: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n\n assets:checking $-47.18\n([],"") -\n\n" -Cases: 6 Tried: 2 Errors: 0 Failures: 2([],"") -### Failure in: 2:showTransaction -show an unbalanced transaction, should not elide -expected: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n assets:checking $-47.19\n\n" - but got: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n\n assets:checking $-47.19\n([],"") -\n\n" -Cases: 6 Tried: 3 Errors: 0 Failures: 3([],"") -### Failure in: 3:showTransaction -show an unbalanced transaction with one posting, should not elide -expected: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n\n" - but got: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n\n\n" -Cases: 6 Tried: 4 Errors: 0 Failures: 4([],"") -### Failure in: 4:showTransaction -show a transaction with one posting and a missing amount -expected: "2007/01/28 coopportunity\n expenses:food:groceries \n\n" - but got: "2007/01/28 coopportunity\n expenses:food:groceries \n\n\n" -Cases: 6 Tried: 5 Errors: 0 Failures: 5([],"") -### Failure in: 5:showTransaction -show a transaction with a priced commodityless amount -expected: "2010/01/01 x\n a 1 @ $2\n b \n\n" - but got: "2010/01/01 x\n a 1 @ $2\n\n b \n([],"") -\n\n" -Cases: 6 Tried: 6 Errors: 0 Failures: 6 - -*** web: stray bracket in journal edit form title -*** web: enter doesn't work in add form completing fields -**** research this dhtmlxcombo issue -*** parsing: decimal point/thousands separator confusion ? -<<< -2011/09/30 - a $1,000,000.00 - b - -2011/09/30 x - a $1,2 - b - -2011/09/30 y - a $1.2 - b ->>> hledger -f t print -2011/09/30 - a $1,000,000.00 - b $-1,000,000.00 - -2011/09/30 x - a $1.20 - b $-1.20 - -2011/09/30 y - a $1.20 - b $-1.20 - -*** parsing: recursive file includes cause a hang -echo "!include rec" > rec -hledger -f rec print -*** parsing: "could not balance" error does not show line number -*** parsing: extra noise with eg bad date parse errors -$ cat t.journal -200/1/99 x - a 1 - b -$ ./hledger.hs -f t.journal print -hledger.hs: could not parse journal data in t.journal -"t.journal" (line 1, column 9): -unexpected " " <- undesired -expecting digit <- noise -bad year number: 200 - -*** parsing: confusing error when journal lacks a final newline -$ cat - >t.j -2010/1/2 - a 1 - b -$ hledger -f t.j bal -hledger: could not parse journal data in t.j -"t.j" (line 3, column 3): -unexpected "b" -expecting comment or new-line - -*** add: default amount adds one decimal place when journal contains no decimals -*** add: excessive precision in default balancing amount -**** shelltest tests/add.test -- -t10 -**** find original justification or drop -*** add: learn decimal point/thousands separator from the journal and/or add session ? -Eg: comma is already used as thousands separator in the journal, but add -interprets it as decimal point giving a wrong default for amount 2 (though -the correct journal transaction is written in this case) - -$ hledger -f t add -Adding transactions to journal file "t". -To complete a transaction, enter . (period) at an account prompt. -To stop adding transactions, enter . at a date prompt, or control-d/control-c. -date, or . to end [2011/09/30]: -description []: z -account 1: a -amount 1: 1,000 -account 2: b -amount 2 [-1,0]: -account 3, or . to record: . -date, or . to end [2011/09/30]: . -$ cat t -; journal created 2011-09-30 by hledger - -2011/09/30 - a $1,000,000.00 - b - -2011/09/30 x - a $1,2 - b - -2011/09/30 y - a $1.2 - b - -2011/09/30 z - a 1,000 - b - -*** convert: 49 convert should report rules file parse errors better -*** 25 hledger in windows console does not print non-ascii characters -*** journalAddFile is called in reverse order of includes -*** darcs show authors cleanup -**** trygve -**** encoding -eg in text-mode emacs 24 -** documentation, marketing -*** document status flag better -*** finalise/link 2012 survey -*** developer notes & log -**** prune backlog -*** intro brainstorming - -hledger is a robust command-line accounting tool with a simple plain text data format. - -hledger is a reporting tool for accounting transactions stored in a simple human-editable text format. - -hledger is a computer program for easily tracking money, time, or other transactions, usually recorded in a general journal file with a simple human-editable markup format. - -hledger is primarily a reporting tool, but it can also help you add transactions to the journal, or convert from other data formats. - -hledger is a haskell port and friendly fork of John Wiegley's c++ ledger tool. - -hledger aims to be a reliable, practical, useful tool for (slightly geeky) users and a reusable library for haskell programmers interested in finance. - -hledger is quite simple in essence, aiming to be a reliable low-level parsing-and-reporting tool that doesn't get in your way. - -For some, it is a less complex, less expensive, more efficient alternative to Quicken or Quickbooks. - -hledger is available for free under the GNU General Public License. - -hledger reads plain text files (general journal, timelog, or CSV format) describing transactions (in money, time or other commodities) and prints the chart of accounts, account balances, or transactions you're interested in. - -hledger is a free program that helps you understand your finances, making calculations based on data stored a simple text file. If you prefer the command line and a text editor to a big gui application, hledger gives you the power of Quicken and Quickbooks without the complexity. - -Your financial data will outlive your financial software, so it should have longevity and accessibility. Its integrity is important to your peace of mind, so changes should be transparent and (if desired) version controlled. It may also be important to allow multiple authors to edit safely. A structured, easy-to-parse, human-friendly plain text format, as in the wiki world, provides a good balance of longevity, reliability, transparency and flexibility. - -hledger helps you track and understand your finances, making calculations based on data stored in a simple text file. If you prefer the command line and a text editor to a big gui application, hledger gives you the power of Quicken and Quickbooks without the complexity. - -Features: reads transactions in journal, timelog, or CSV format; handles multi-currency/multi-commodity transactions; prints the chart of accounts, account balances, or transactions you're interested in, quickly; scriptable. - -*** website -**** 2012 refresh -*** faq -**** life cycle of top-level accounts -For personal ledgers, when you're born, all accounts are at zero (one hopes) and as you live: - -1. Equity accounts accommodate your previous years of not maintaining accounts (fixed, probably negative) - -2. Expense accounts become more and more positive (unavoidably) - -3. Income accounts become more and more negative (on payday) - -4. Assets Accounts become more and more positive (in good times) - -5. Liability account become more positive (in good times, when you pay them off) and more negative (when you use them to buy things). - -When you die, Equity: and Income: will stand at large negative balances, -Expense: and Assets: will stand at large positive balances and Liabilities -will have to be paid (out of Assets) before your heirs get what's left. - -adapted from Ben Alexander, ledger-cli - -*** manual -**** make accessible from command line -*** add missing -**** faq -**** examples/how-tos -**** hledger/ledger comparison/feature matrix -**** blog posts -*** reduce -*** aesthetics -**** better screenshots/images -***** use highslide -*** liveness -**** show feeds on site ? -***** commits (darcsweb) -**** weekly repo activity summary on list -**** available feeds -*** screencasts -**** intro -***** intro to hledger -****** place in the world -****** basic installation -****** quick demo -****** where to go from here -***** installing hledger on windows -***** installing hledger on mac -***** installing hledger on unix -***** accessing hledger's support forums -****** website -****** mail list -****** irc channel -***** reporting a hledger bug -**** using -***** income/expense tracking -***** time tracking -***** downloading bank data -***** reconciling with bank statement -***** see time reports by day/week/month/project -***** get accurate numbers for client billing and tax returns -***** find unpaid invoices -**** developing -***** intro to hledger development -***** testing hleder -***** documenting hledger -***** a hledger coding example -***** a tour of hledger's code -**** ledger cooperation -*** developer docs -**** developer notes & log -**** roadmap -***** 1.0 -culmination of 0.x releases - stable/usable/documented -followup releases are 1.01, 1.02.. -GHC 6.12/HP 2010 primary platform -GHC 6.10/HP 2009 also supported if possible -GHC 6.8 might work for core features, but not officially supported -separate ledger package ? license ? -separate vty, web packages ? -support plugins ? -web: loli+hsp+hack+simpleserver/happstack, or yesod+hstringtemplate+wai+simpleserver/happstack ? -add: completion ? -chart: register charts ? -histogram: cleaned up/removed -complete user manual -binaries for all platforms ? - -***** 2.0 -development releases are.. 1.60, 1.61.. or 1.98.01, 1.98.02.. -separate ledger lib -plugins -Decimal -binaries for all platforms - -**** internal api docs -**** external api docs -**** DEVGUIDE -***** How to do anything that needs doing in the hledger project. -****** website & documentation -******* overview of hledger docs -******* how the site is built -******* convenience urls -list.hledger.org - mail list -bugs.hledger.org - issue tracker -bugs.hledger.org/1 - go to specific issue -bugs.hledger.org/new - create a new issue -hledger.org/{list,bugs}/* also works - -****** issue tracking -****** testing - -hledger's unit tests and a simple test runner are built in. They can -be run several ways: - -$ hledger test [PAT] -$ make unittest -$ make autotest - -They can also be built as a separate executable, in case needed for -cabal test. (?) This requires test-framework, which may not work on -windows. - -$ make unittest-standalone - -hledger's functional tests are a set of @shelltestrunner@ tests defined -by .test files in the tests\/ subdirectory. - -$ make functest - -Shell tests can also be defined as doctests, literal blocks embedded -in modules' haddock docs, though this is hardly used. For example: - -@ -$ bin\/hledger -f data\/sample.journal balance o - $1 expenses:food - $-2 income - $-1 gifts - $-1 salary --------------------- - $-1 -@ - -$ make doctest - -****** coding -****** funding process -******* donation blurb -If you like project or have -benefited from it, you can give back by making one-time or periodic -donations of any amount. This also allows me to offer further -enhancements, maintenance and support for this project. Thanks! -***** reference -****** unsafe things which may fail at runtime include.. -******* incomplete pattern matching -******* error -******* printf -******* read -**** functional programming -hledger is written in the Haskell programming language; -it demonstrates a pure functional implementation of ledger. -*** announcements -**** set up a template -*** misc automation issues -**** keep intro blurbs in sync -***** README file -***** hledger.hs module description -***** hledger.cabal description field (exclude home page link) -***** home page description (http://joyful.com/Hledger/editform) -***** mail list description (http://groups.google.com/groups/hledger -> edit welcome msg) -***** gmane description -***** darcsweb description -**** keep usage info in sync -***** Options.hs -***** MANUAL.md -*** presenting/live demos -*** develop funding process -**** license change ? -**** donate button, see chimoo guy -**** funding document 2009/01 -***** text -======= -funding -======= - -vision -====== - - How to grow the hledger project ? - - I'm looking for ways to fund active and sustainable hledger - development by me and others. - - A secondary goal is to develop new sustainable models and processes - for funding free software developers and other community projects. - - This is sometimes the point in a free sw project's development where - the project leader seemingly loses the plot, alienates contributors - and destroys the community's good-will dynamic. I've seen it many - times, but a few have succeeded and I want to be one of them - so - that I can eat, have a modicum of stability and do my best work in - service to the community. At worst, I'll look bad but the project - will still be out there. At best I'll live more easily and joyfully - while serving the cause of Financial Solvency! - - So I'm beginning by posting these notes and inviting your thoughts - - as much or as little as folks want to give. How could we do this - so that all benefit ? - -funding models -============== - Brainstorming some possible funding models & processes. - - * grants - - How to find possible grant sources ? - - * con - - * getting grant funding is a whole new field to study - * slow and time intensive, I imagine - - * donations - - Solicit donations. - - * pro - - * simple - - * con - - * often difficult - * donators do not feel a direct benefit - - * shareware - - Release the project under a non-free license, requiring commercial - users to pay the fee on an honour basis (eg). - - * pro - - * flexible, low administration, encourages trust - - * con - - * effectively closed-source ? would inhibit collaboration - * benefit is still indirect, only a proportion will pay - * enforcement/guilt may come into play - - * limited-time premium branch - - The funded version of hledger gets some desirable premium features - before the free version and is closed-source. Funders/customers pay - a fixed price for immediate access to the funded version. Yearly, a - new funded version is released and the old funded version is merged - into the free version. (To gain experience it could be done on a - smaller scale, eg monthly/quarterly.) - - * pro - - * all features reach community, predictably - * customers are also community funders - * customers receive direct benefit from paying - - * con - - * free sw developers compete/outshine the premium branch - - * bounties - - Some (or all) feature, bugfix, project management or other tasks are - published with a bounty attached. When the bounty is paid by one or - more funders, the task is performed and delivered. Or, bounty is paid - on completion of task (honour system). - - * pro - - * funders receive direct benefit - - * bounties using fundable.org (eg) - - A more organised form of the above, perhaps facilitating trust, - co-funding and larger bounties. - - * pro - - * proven process developed by others - - * con - - * fundable takes a cut - - * hosted service - - Offer hosted and managed ledgers, perhaps with premium features, for - a monthly fee - - * pro - - * proven model - * clear benefit to customers, especially non-technies - - * con - - * success of free/self-installed version competes with hosting service - * some will avoid web-hosting their financial data - - * customisation - - Offer per-user customisations, possibly to be merged in the trunk, - for a fee - - * support - - Offer user/developer support for a fee - - * training - - Offer application and/or financial training for a fee - - * profit sharing/tithing - - Each period (quarter, half-year, year), donate 10% (eg) to project - contributors and/or supporting projects - - * transparent funding - - Funding and usage of funds is published on the web as a ledger - - * opaque funding - - All funding and spending need not be made public - -strengths -========= - hledger has some aptitudes in this area: - - * hledger deals with money => hledger users will tend to have some money - * hledger's purpose is to increase financial success => users will feel its value to their bottom line - * hledger is a tool that can support project funding, eg by publishing community funding data - -weaknesses -========== - * hledger doesn't have a nice ui yet - * hledger has a limited featureset - * hledger requires work, eg data entry and chart of accounts maintenance - * hledger is geeky - * there is competition - * hledger has no compelling market niche (aside from payment-averse free software users) - -competitors/fellow niche inhabitants -==================================== - - * web apps - - * netsuite - * sql-ledger, ledgersmb - * wesabe - * ... - - * desktop apps - - * quickbooks - * quicken - * ms money - * grisbi - * gnucash - * excel - * ledger! - * ... - -***** responses -****** albino - have you considered talking to business who hate their financial sw and going from there -****** gwern - most haskellers have never heard of hledger, sounds arrogant or hubristic to talk of charging for it - -**** home edition -**** real-time project ledger -**** in-place transaction editing fund drive - -Fund drive: hledger-web in-place transaction editing - -Goal: I would like to raise $X or more to fund basic in-place -transaction editing for hledger-web. hledger-web is a web-based GUI for -hledger (and ledger), which are free/open-source accounting programs -providing a lean and efficient alternative to quicken, gnucash, mint.com -etc. - -Current hledger-web[1] has simple web forms for adding transactions and -for editing the whole journal, but there is no easy ui for editing a -single existing transaction. Such a ui is an important step towards making -hledger (and ledger) usable by non techies, which would greatly expand -these tools' applicability and potential user/contributor base. - -Plan: do the front-end javascript and backend haskell work required to -support: - -- click date, description, account or amount cells in a register view to make that cell editable -- tab moves to the next cell -- enter or click on save button updates the transaction in the journal, overwriting/rewriting the whole file -- tested in firefox/chrome/safari - -The proposed amount will fund about 10 hours of work, so the above -features must be implemented very expeditiously. Other improvements will -be tackled in a followup fund drive if this one succeeds (or in this one -if the funding goal is exceeded.) Those future items include: - -- history/content awareness, smart defaults and auto-completion wherever useful -- date picker widget -- ability to add/remove postings -- ability to edit metadata/tags -- ability to edit other transaction/posting fields -- ledger compatibility -- compatibility testing/fixes for all the major browsers -- edit conflict checking - don't overwrite concurrent external edits -- try harder to preserve existing file layout/co-exist better with external edits -- a similar ui for adding new transactions -- pleasant visual style - -Also, 10% of the amount raised will be tithed to three contributing -projects or developers (ledger and two others of my choice.) - -This project will go forward if - -[1] http://demo.hledger.org:5001 -** testing -*** test running improvements -**** stop on first failure -**** run tests in bottom up order -*** envision better test setup -**** every parser has a test and is easy to test -**** easy to run any single test or module's tests -**** tests run bottom up by default -**** test runner can select tests precisely eg by regexp -**** test runner stops at first failure by default - -*** documentation -**** site up, current ? -**** demo up, current ? -**** haddock building, current ? -**** doctests ? -*** unit -**** hunit -**** quickcheck -**** easier unit test development -*** functional -**** ledger file parsing tests -***** test all ledger file format features -***** clarify hledgerisms in file format - that hledger can read but ledger can't -**** ledger 3 baseline tests -**** MaybeSo subtotal rounding issue -I had a question about balance totals. Given this test data: - -$ cat test.dat -D $1,000.00 -P 2011-01-01 22:00:00-0800 TESTA $78.35 -P 2011-01-01 22:00:00-0800 TESTB $15.86 -P 2011-01-01 22:00:09-0800 TESTC $13.01 - -2011/01/01 Example - Assets:Brokerage:TESTA 188.424 TESTA @ $76.61 - Assets:Brokerage:TESTB 1,809.282 TESTB @ $15.60 - Assets:Brokerage:TESTC 384.320 TESTC @@ $5,000.00 - Assets:Brokerage:TESTC 5.306 TESTC @@ $68.18 - Equity:Opening Balances - -I'm a little bit surprised that the sub-accounts -reflect a difference from the top level account -w/re to rounding the last cent: - -$ ledger -V -f test.dat bal - $48,527.27 Assets:Brokerage - $14,763.02 TESTA - $28,695.21 TESTB - $5,069.03 TESTC - $-47,728.14 Equity:Opening Balances --------------------- - $799.13 - -Even if --no-rounding is passed in: - -$ ledger -V -f test.dat --no-rounding bal - $48,527.27 Assets:Brokerage - $14,763.02 TESTA - $28,695.21 TESTB - $5,069.03 TESTC - $-47,728.14 Equity:Opening Balances --------------------- - $799.13 - -Is there something off with how the data aboce is set up? Should I be -using be more place holders? -*** performance -**** speed, benchmark tests -**** memory usage -*** build & packaging -**** use -Wall and anything else useful -**** build with multiple ghc versions -**** cabal test -**** hackage upload -**** cabal install with: -***** ghc 6.8 -***** ghc 6.10.x -***** windows -***** linux -***** macos -***** no flags -***** happs flag -***** vty flag -*** field -**** talkback, auto bug reports -**** usability -**** download & usage stats -** packaging, installability -*** linux -***** debian/ubuntu packaging -*** mac -***** easy installer -***** easy startup -*** windows -***** easy installer -***** easy startup -** refactoring -*** makefile cleanups -*** make shell tests version independent -tests/no-such-file.test: rm -f $$; bin/hledger register -f $$; rm -f $$ -tests/no-such-file.test: rm -f $$; bin/hledger balance --no-total -f $$; rm -f $$ -tests/add.test: rm -f t$$.j; bin/hledger -f t$$.j add; rm -f t$$.j -tests/add.test: rm -f t$$.j; bin/hledger -f t$$.j add; rm -f t$$.j -tests/add.test: rm -f t$$.j; bin/hledger -f t$$.j add; rm -f t$$.j -tests/add.test: printf 'D $1000.00\n' >t$$.j; bin/hledger -f t$$.j add >/dev/null; cat t$$.j; rm -f t$$.j -tests/add.test: printf 'D $1000.0\n' >t$$.j; bin/hledger -f t$$.j add >/dev/null; cat t$$.j; rm -f t$$.j -tests/add.test: printf '2010/1/1\n a $1000.00\n b\n' >t$$.j; bin/hledger -f t$$.j add >/dev/null; cat t$$.j; rm -f t$$.j -tests/add.test: printf '2010/1/1\n a $1000.0\n b\n' >t$$.j; bin/hledger -f t$$.j add >/dev/null; cat t$$.j; rm -f t$$.j -tests/add.test: printf 'D $1000.0\nD £1,000.00\n' >t$$.j; bin/hledger -f t$$.j add; cat t$$.j; rm -f t$$.j -tests/add.test:rm -f nosuch.journal; bin/hledger -f nosuch.journal add; rm -f nosuch.journal -tests/add.test:## printf '\n\na\n1\nb\n' | bin/hledger -f /dev/null add -tests/add.test:# bin/hledger -f /dev/null add -tests/amount-layout-vertical.test:# bin/hledger -f - print -tests/amount-layout-vertical.test:# bin/hledger -f - register -tests/amount-layout-vertical.test:# bin/hledger -f - balance -tests/parse-posting-error-pos.test:# bin/hledger -f- stat -tests/null-accountname-component.test:# bin/hledger -f - balance -E -tests/include.test: mkdir -p b/c/d ; printf '2010/1/1\n (D) 1\n' >b/c/d/d.journal ; printf '2010/1/1\n (C) 1\n!include d/d.journal\n' >b/c/c.journal ; printf '2010/1/1\n (B) 1\n!include c/c.journal\n' >b/b.journal ; printf '2010/1/1\n (A) 1\n!include b/b.journal\n' >a.journal ; bin/hledger -f a.journal print; rm -rf a.journal b -tests/timelog-stack-overflow.test:#bin/hledger -f - balance -tests/precision.test:# bin/hledger -f - print --cost -tests/precision.test: bin/hledger -f - balance --cost -tests/timezone.test:# bin/hledger -f - balance --no-total --cost -tests/read-csv.test:rm -rf t.rules$$; printf 'date-field 0\ndate-format %%d/%%Y/%%m\ndescription-field 1\namount-field 2\ncurrency $\nbase-account assets:myacct\n' >t.rules$$; echo '10/2009/09,Flubber Co,50' | bin/hledger -f- print --rules-file t.rules$$; rm -rf t.rules$$ -tests/read-csv.test: printf 'base-account Assets:MyAccount\ndate-field 0\ndate-format %%d/%%Y/%%m\ndescription-field 1\nin-field 2\nout-field 3\ncurrency $\n' >$$.rules ; bin/hledger -f- print --rules-file $$.rules; rm -rf $$.rules -tests/read-csv.test:# rm -rf t.rules$$; printf 'date-fiel 0\ndate-format %%d/%%Y/%%m\ndescription-field 1\namount-field 2\ncurrency $\nbase-account assets:myacct\n' >t.rules$$; echo '10/2009/09,Flubber Co,50' | bin/hledger convert --rules-file t.rules$$; rm -rf t.rules$$ -tests/prices.test:# bin/hledger -f - print - -*** abstract DataSource -*** use matchers for command line too -**** design cli, backwards compatibility strategy -**** replace optsToFilterSpec -*** review/simplify apis -**** simplify option types -*** more modularity -**** packages/namespace -***** hledger-datasource? -**** plugin strategy -**** export lists -**** graph and reduce dependencies -*** switch to http-conduit -*** include latest jquery, jquery-url, minified and non -http://ajaxcssblog.com/jquery/url-read-request-variables/ -*** clarify levels of abstraction -**** web ui balance sheet view - data model, view layout -**** hledger web framework - define routes, handlers/views/actions/controllers/presenters, skins/styles.. -**** happstack - ? happstack api.. -**** hledger app platform - hledger.hs, Options, Utils, withLedgerDo.. -**** hledger lib - Ledger, TimeLog, Account, Transaction, Commodity.. -**** hledger dev platform - make build, ci, test, bench, prof, check, release.. -**** general libs - directory, parsec, regex-*, HUnit, time.. -**** cabal - hledger.cabal, hackage.. -**** ghc - ghc 6.8, 6.10.. -**** haskell 98 -**** unix/windows/mac platform -*** inspiration -http://community.haskell.org/~ndm/downloads/paper-hoogle_overview-19_nov_2008.pdf -> Design Guidelines -** features/wishlist -*** balancesheet real-world testing -*** narrow (editable) view to selected txns -*** optimise balance report -*** IIF/quickbooks writer -*** inacct: on command line -hledger -f demo.journal reg inacct:expenses:food:pets date:2010/8/25 -2010/08/25 catfood expenses:food:pets $10.00 $10.00 - assets:cash $-10.00 0 -*** support - in period expressions -*** more useful show instances -**** use simple derived show instances for Amount & above ? -**** show no of postings in account tree for Ledger ? -*** hledger-web release -*** don't recompile between autotest & functest -*** stats: list all journal files -*** search by commodity -*** account types -*** runtime templates for reports -*** generalise rules file ? -**** make it applicable to all formats ? -**** absorb its directives into journal format ? -*** support apostrophe digit group separator -*** detect .hs plugins -*** more powerful storage layer -**** Clint's filestore_proof_of_concept.dpatch - -New patches: - -[filestore-proof-of-concept -Clint Adams **20110901172739 - Ignore-this: 1991477c2b70d276665c52478dc54d3d - - This is a somewhat broken replacement of the traditional file - storage with a forced darcs repo. It assumes that the - darcs repo already exists since Data.FileStore refuses to - initialize a repository in an extant directory. It does not - handle any error conditions well. -] hunk ./hledger-lib/Hledger/Read.hs 104 - when (not exists) $ do - hPrintf stderr "No journal file \"%s\", creating it.\n" f - hPrintf stderr "Edit this file or use \"hledger add\" or \"hledger web\" to add transactions.\n" -- emptyJournal >>= writeFile f -+ emptyJournal >>= writeFileWithBackup f - - -- | Give the content for a new auto-created journal file. - emptyJournal :: IO String -hunk ./hledger-lib/Hledger/Utils.hs 40 - import Text.ParserCombinators.Parsec - import Text.Printf - import Text.RegexPR -+import System.FilePath (takeFileName, takeDirectory) -+import qualified Data.FileStore.Types as DFT -+import qualified Data.FileStore.Generic as DFG -+import Data.FileStore.Darcs (darcsFileStore) - -- import qualified Data.Map as Map - -- - -- import Prelude hiding (readFile,writeFile,getContents,putStr,putStrLn) -hunk ./hledger-lib/Hledger/Utils.hs 432 - -- | Apply a function the specified number of times. Possibly uses O(n) stack ? - applyN :: Int -> (a -> a) -> a -> a - applyN n f = (!! n) . iterate f -+ -+-- Store file in VCS; Data.FileStore takes care of only committing -+-- when necessary. -+ -+filestoreSave :: FilePath -> String -> IO () -+filestoreSave f t = DFT.save assumedRepo assumedFilename assumedAuthor logMessage t -+ where -+ assumedRepo = darcsFileStore (takeDirectory f) -+ assumedFilename = takeFileName f -+ assumedAuthor = (DFT.Author "Hledger Role" "hledger@fake") -+ logMessage = "Some kind of change committed by some part of the hledger suite" -+ -+writeFileWithBackup :: FilePath -> String -> IO () -+writeFileWithBackup = filestoreSave -+ -+-- modify existing file in filestore -+filestoreModify :: FilePath -> DFT.RevisionId -> String -> IO (Either DFT.MergeInfo ()) -+filestoreModify f lr t = DFG.modify assumedRepo assumedFilename lr assumedAuthor logMessage t -+ where -+ assumedRepo = darcsFileStore (takeDirectory f) -+ assumedFilename = takeFileName f -+ assumedAuthor = (DFT.Author "Hledger Role" "hledger@fake") -+ logMessage = "Some kind of change committed by some part of the hledger suite" -+ -+filestoreAppend :: FilePath -> String -> IO () -+filestoreAppend f t = do -+ lastrev <- DFT.latest assumedRepo assumedFilename -+ oldcontents <- DFT.retrieve assumedRepo assumedFilename (Just lastrev) -+ result <- filestoreModify f lastrev (oldcontents ++ "\n\n" ++ t) -+ either (\x -> putStrLn "Help, the append didn't work and I am failing miserably.") (\x -> return ()) result -+ where -+ assumedRepo = darcsFileStore (takeDirectory f) -+ assumedFilename = takeFileName f -hunk ./hledger-lib/hledger-lib.cabal 60 - ,containers - ,directory - ,filepath -+ ,filestore - ,mtl - ,old-locale - ,old-time -hunk ./hledger/Hledger/Cli/Add.hs 31 - import qualified Data.Set as Set - - import Hledger --import Prelude hiding (putStr, putStrLn, appendFile) --import Hledger.Utils.UTF8 (putStr, putStrLn, appendFile) -+import Prelude hiding (putStr, putStrLn) -+import Hledger.Utils.UTF8 (putStr, putStrLn) - import Hledger.Cli.Options - import Hledger.Cli.Register (postingsReportAsText) - import Hledger.Cli.Utils -hunk ./hledger/Hledger/Cli/Add.hs 194 - journalAddTransaction :: Journal -> CliOpts -> Transaction -> IO Journal - journalAddTransaction j@Journal{jtxns=ts} opts t = do - let f = journalFilePath j -- appendToJournalFile f $ showTransaction t -+ filestoreAppend f $ showTransaction t - when (debug_ opts) $ do - putStrLn $ printf "\nAdded transaction to %s:" f - putStrLn =<< registerFromString (show t) -hunk ./hledger/Hledger/Cli/Add.hs 200 - return j{jtxns=ts++[t]} - ---- | Append data to a journal file; or if the file is "-", dump it to stdout. --appendToJournalFile :: FilePath -> String -> IO () --appendToJournalFile f s = -- if f == "-" -- then putStr $ sep ++ s -- else appendFile f $ sep++s -- where -- -- appendFile means we don't need file locking to be -- -- multi-user-safe, but also that we can't figure out the minimal -- -- number of newlines needed as separator -- sep = "\n\n" -- -- sep | null $ strip t = "" -- -- | otherwise = replicate (2 - min 2 (length lastnls)) '\n' -- -- where lastnls = takeWhile (=='\n') $ reverse t -- - -- | Convert a string of journal data into a register report. - registerFromString :: String -> IO String - registerFromString s = do -hunk ./hledger/Hledger/Cli/Utils.hs 18 - journalSpecifiedFileIsNewer, - fileModificationTime, - openBrowserOn, -- writeFileWithBackup, -- writeFileWithBackupIfChanged, - readFileStrictly, - Test(TestList), - ) -hunk ./hledger/Hledger/Cli/Utils.hs 25 - import Control.Exception - import Data.List - import Data.Maybe --import Safe (readMay) - import System.Console.CmdArgs -hunk ./hledger/Hledger/Cli/Utils.hs 26 --import System.Directory (getModificationTime, getDirectoryContents, copyFile) -+import System.Directory (getModificationTime) - import System.Exit -hunk ./hledger/Hledger/Cli/Utils.hs 28 --import System.FilePath ((), splitFileName, takeDirectory) - import System.Info (os) - import System.Process (readProcessWithExitCode) - import System.Time (ClockTime, getClockTime, diffClockTimes, TimeDiff(TimeDiff)) -hunk ./hledger/Hledger/Cli/Utils.hs 123 - -- what not. - -- ::ShellExecute(NULL, "open", "www.somepage.com", NULL, NULL, SW_SHOWNORMAL); - ---- | Back up this file with a (incrementing) numbered suffix then ---- overwrite it with this new text, or give an error, but only if the text ---- is different from the current file contents, and return a flag ---- indicating whether we did anything. --writeFileWithBackupIfChanged :: FilePath -> String -> IO Bool --writeFileWithBackupIfChanged f t = do -- s <- readFile f -- if t == s then return False -- else backUpFile f >> writeFile f t >> return True -- ---- | Back up this file with a (incrementing) numbered suffix, then ---- overwrite it with this new text, or give an error. --writeFileWithBackup :: FilePath -> String -> IO () --writeFileWithBackup f t = backUpFile f >> writeFile f t -- - readFileStrictly :: FilePath -> IO String - readFileStrictly f = readFile f >>= \s -> Control.Exception.evaluate (length s) >> return s -hunk ./hledger/Hledger/Cli/Utils.hs 125 -- ---- | Back up this file with a (incrementing) numbered suffix, or give an error. --backUpFile :: FilePath -> IO () --backUpFile fp = do -- fs <- safeGetDirectoryContents $ takeDirectory $ fp -- let (d,f) = splitFileName fp -- versions = catMaybes $ map (f `backupNumber`) fs -- next = maximum (0:versions) + 1 -- f' = printf "%s.%d" f next -- copyFile fp (d f') -- --safeGetDirectoryContents :: FilePath -> IO [FilePath] --safeGetDirectoryContents "" = getDirectoryContents "." --safeGetDirectoryContents fp = getDirectoryContents fp -- ---- | Does the second file represent a backup of the first, and if so which version is it ? --backupNumber :: FilePath -> FilePath -> Maybe Int --backupNumber f g = case regexMatch ("^" ++ f ++ "\\.([0-9]+)$") g of -- Just (_, ((_,suffix):_)) -> readMay suffix -- _ -> Nothing - -*** web api -*** client-side ui -*** support -V ? -*** print: shouldn't support -M and --depth ? -*** in-place editing -**** http://stackoverflow.com/questions/640971/setfocus-to-textbox-from-javascript-after-just-creating-the-textbox-with-javascr -*** measure bug open times -http://code.google.com/p/support/wiki/IssueTrackerAPI -http://code.google.com/p/support/wiki/IssueTrackerAPIReference -*** wai-handler-webkit, wai-handler-launch -*** add: don't offer record txn option in account N prompt if it's not balanced yet -*** add: would be nice to create the journal file only if a txn is actually recorded -*** web: lose io-storage -*** web: use cli args as a permanent additional filter ? -*** web: should display virtual postings with () or [] -*** web: how to find out total spent in an account during a specific month -*** web: how to find out net worth, /register?q=assets+liabilities shows nothing -*** parsing: end directive may also be spelled end account; or end ends last directive -*** convert: clean up default rules file -*** convert: use default rules when converting stdin with no --rules -*** convert: consistent multi-field formats allowed for any field -*** convert: optionally generate single-entry txns -*** convert: optionally set final amount blank -*** convert: parse HH:MM[:SS] as an amount, converting to decimal hours -*** convert: ignore heading row -*** convert: warn and ignore unparsed rows ? -*** stats: list all included files -*** parsing: alias directives should be modified by account directives - -*** web: should take port from base-url when appropriate -*** add: rewrite a short description (trader) to the full description from the matched txn (trader joe's) ? -*** web: can't find out total income:7th total in june at http://localhost:5005/register?q=inacct%3Aassets%3Atreasury ! -*** web: update/remove browser startup -*** docs: simple hledger.org/topic help urls, like php.net -*** docs: --help-web and web ui help links that go to online help, with paragraph comments & chat -cf clients & profits interactive user guide, php.net, realworldhaskell etc. -*** parsing: accounts tree & metadata specification -; chart of accounts -; defines allowed account names, hierarchy, default sort order, -; and some metadata (account numbers, cf http://www.netmba.com/accounting/fin/accounts/chart/) -ACCOUNTS - assets ; :number: 1000 - cash ; :number: 1010 - HT7 ; :number: 1011 - jan ; :number: 1011.01 - feb ; :number: 1011.02 - mar ; :number: 1011.03 - RSG ; :number: 1012 - jan ; :number: 1012.01 - feb ; :number: 1012.02 - mar ; :number: 1012.03 - bank ; :number: 1020 - HT7 ; :number: 1021 - RSG ; :number: 1022 - reserve ; :number: 1023 - liabilities ; :number: 2000 - accounts payable ; :number: 2010 - BSG/GI/RSG ; :number: 2011 - equity ; :number: 3000 - opening balances ; :number: 3010 - income ; :number: 4000 - HT7 ; :number: 4010 - RSG ; :number: 4020 - expenses ; :number: 6000 - rent ; :number: 6010 - -or: http://furius.ca/beancount/examples/demo.ledger - -*** -X/--show-in-commodity -cf http://bugs.ledger-cli.org/show_bug.cgi?id=538 -*** cli, web: consistent rich query language, including and/or/parentheses -ledger's query syntax: http://ledger-cli.org/3.0/doc/ledger.1.html -**** draft 1: - -Filter patterns restrict the postings/transactions that are displayed. -(They often reduce the amount of processing work hledger has to do, as well.) -A pattern is a string or regular expression, usually with a prefix specifying the type of match to do. -The supported prefixes are: - -acct:PAT match postings affecting accounts whose name matches PAT -otheracct:PAT match the other postings in transactions with an acct match (like ledger's --related) -anyacct:PAT match all postings in transactions with an acct match (union of acct and otheracct) -desc:PAT match postings whose description matches PAT -status:PAT match postings whose cleared status matches PAT -code:PAT match postings whose transaction code matches PAT -tag:PAT match postings with a metadata tag whose name matches PAT -tag:TAG=PAT match postings with a metadata tag named TAG whose value matches PAT -from:DATE match postings on or after DATE (like --begin) -to:DATE match postings before DATE (like --end) -in:PERIOD match postings during PERIOD (like --period) -(or ? -begin:DATE match postings on or after DATE (like --begin) -end:DATE match postings before DATE (like --end) -period:PERIOD match postings during PERIOD (like --period) -) - -Prefixes have a short form which is their first letter, except for tag and anyacct. - -Prefix-less patterns are treated like acct: patterns, except by the -register command which treats them as otheracct: . - -Patterns containing whitespace must be enclosed in quotes. - -Matches are always case-insensitive. - -Matches are always substring matches (except for TAG); to match exactly, -wrap the pattern in ^ and $. - -A posting's date, status, code, etc. is usually (but not always) that of -its containing transaction. - -Any of these may be further prefixed with not: for an inverse match. - -Filter patterns may be combined with AND, OR, and parentheses. OR is -assumed by default. -(previously: -When you specify multiple filter patterns, hledger generally selects the -items which match: - - any of the account patterns AND any of the description patterns - -The print command selects transactions which - - match any of the description patterns AND have any postings matching any - of the positive account patterns AND have no postings matching any of - the negative account patterns -) - -*** journalAddTransaction should check txn balances -*** balance setting -ledger: You can accomplish "setting to the bank's view" with a transaction like this: - -2011-08-12 Sample - Assets:Checking = $200.00 - Equity:Adjustments - -This tells Ledger (Git/3.0) that your checking account's balance must be $200 -after this transaction is completed. It will put whatever amounts are -required to accomplish this into the Equity:Adjustments account. - -what about balance assertions ? - -*** web: optional fully self-contained production build (all support files embedded) -*** tools: fix slow make startup -*** tools: better ghci setup -*** docs: better intro, less wall-of-text, separate tutorial & reference sections -*** balance: --depth with --flat should show aggregate balances including the non-displayed deeper accounts -*** parsing: let's enforce positive price amounts for clarity, like ledger -eg 1€ @@ $-2 is not allowed -*** parsing: canonicalise account names to be case-insensitive ? -*** parsing: period expressions should allow interval at the end -eg support -p 'from 1/1 to 2/1 weekly' - -*** parsing: more flexible file including -currently only journals (not timelog files) can include, and only another journal -*** --no-elide -> --empty-parents ? Make it the default ? - $260.00 expenses - $260.00 rent -*** --empty -> --show-zero-accounts, --show-empty-parents ? Make it default ? -*** cli: better control of output format/layout -**** register --format, generalise --format -**** --wide ? window width sensitive ? -**** more tidy/consistent layout from print -**** --output-layout=ledger|traditional -**** --output-format=text|html|pdf -*** balance: try indenting amounts - $260.00 expenses - $260.00 rent -*** parsing: ignore/support ledger assert statements - ; Assertion directives Options - ; These can occur in many places: - ; ; Within an automated transaction, the assert is evaluated every time - ; ; a posting is matched, with the expression context set to the - ; ; matched posting. - ; = /Food/ - ; assert account("Expenses:Food").total >= $100 - ; 2010-06-12 Sample - ; Expenses:Food $100 - ; Assets:Checking - ; ; At file scope, the expression is evaluated within "global" scope. - ; assert account("Expenses:Food").total == $100 - ; ; At the top of a transction, the assertion's scope is the - ; ; transaction. After a posting, the scope is that posting. Note - ; ; however that account totals are only adjusted after successful - ; ; parsing of a transaction, which means that all the assertions below - ; ; are true, even though it appears as though the first posting should - ; ; affect the total immediately, which is not the case. - ; 2010-06-12 Sample 2 - ; assert account("Expenses:Food").total == $100 - ; Expenses:Food $50 - ; assert account("Expenses:Food").total == $100 - ; Assets:Checking - ; assert account("Expenses:Food").total == $100 - -*** parsing: better international number format support (#32) -*** web: : completes one account name component in add form account fields -*** web: auto-complete from substrings, not just prefixes -*** web: auto-complete accounts & amount as well as description -*** parsing: allow price record for null commodity, eg with quotes -P 2009/1/1 "" 0.5h -and why doesn't this work ? time.journal: -P 2010/9/27 h 1 -$ hledger -f time.journal bal -p aug -B - 1 - 17.75h work:jobs - 1 - 17.50h clearview - 1 60 clear glass thermal data - 0.25h admin:cheque issue - 12.75h backups/hosting - 2.00h cleanup - 1.00h move plan - 2.25h move prep - 4.00h testing - 3.50h speed - 1.50h barbara spellcheck issue - 0.50h installation report dates - 0.25h plan change issue - 1.00h planning/discussion - 0.50h speed issue - 0.25h tina quote low-e layout - 0.50h tina title 24 issue - 0.25h kcrw:admin:contract update:unbilled --------------------- - 1 - 17.75h - -*** --related -*** Double -> Decimal -*** parsing: safety check that effective date > actual (to catch eg 2009/12/30=1/4) -*** parsing: accept all real-world ledger files -As far as I know it currently accepts all ledger 2.6-era files. -Add support for ledger 3 file format as/when that stabilises. -It would be nice to optionally semi/automatically submit parse error reports when they happen -*** talkback feature -gather data on real-world installation & usage issues -simplify bug reporting/handling -improve reliability -*** upload feature -*** generation -**** move generatejournal.hs to a subcommand -*** anonymisation -**** payees -**** accounts -*** easier timelog formats -*** convert: generalise/reuse add's history awareness -*** parsing: per-posting effective/actual dates -*** web: better web ui/gui -*** nice standard financial reports -*** more automated bank data conversion -*** parse more file formats - gnucash, qif, ofx, csv, etc. -**** ofx reader -***** clint's code -Date: Sun, 18 Sep 2011 12:26:16 -0400 -From: Clint Adams -To: hledger@googlegroups.com -Subject: OFX conversion -Message-ID: <20110918162616.GA18874@softwarefreedom.org> -MIME-Version: 1.0 -User-Agent: Mutt/1.5.20 (2009-06-14) -X-Original-Sender: clint@softwarefreedom.org -X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: - domain of clint@softwarefreedom.org designates 216.27.154.199 as permitted - sender) smtp.mail=clint@softwarefreedom.org -Reply-To: hledger@googlegroups.com -Precedence: list -Mailing-list: list hledger@googlegroups.com; contact hledger+owners@googlegroups.com -List-ID: -X-Google-Group-Id: 895107692464 -List-Post: , -List-Help: , -List-Archive: -Sender: hledger@googlegroups.com -List-Subscribe: , -List-Unsubscribe: , -Content-Type: text/plain; charset=iso-8859-1 -Content-Disposition: inline -Content-Transfer-Encoding: 8bit - -This is definitely suboptimal but it seems to work on -the OFX 1.0.2 output from AmEx. - - - -{-# LANGUAGE Arrows, NoMonomorphismRestriction #-} -import Text.XML.HXT.Core -import Text.Printf (printf) - -import Data.List (groupBy) -import Data.List.Split (splitOn) - -import Data.Maybe (fromMaybe) - -import Data.Time.Calendar (Day (ModifiedJulianDay)) -import Data.Time.Format (formatTime) -import Data.Time.LocalTime (LocalTime (LocalTime), TimeOfDay (TimeOfDay)) -import Data.Time.Parse (strptime) - -import System.Locale (defaultTimeLocale) -import System.Process (readProcessWithExitCode) - -import Hledger.Cli.Format (FormatString (FormatField), Field (FieldNo)) -import Hledger.Cli.Convert - -normAmount :: String -> String -normAmount amt | amt == "" = "" - | otherwise = printf "%.2f" (read amt :: Double) - -compressWhitespace :: String -> String -compressWhitespace x = map head $ groupSpaces x - where groupSpaces "" = [""] - groupSpaces x = groupBy (\x y -> x==' ' && y==' ') x - -data Transaction = Transaction - { trnType, dtUser, dtPosted, trnAmt, fitId, refNum, name, memo :: String } - deriving (Show, Eq) - --- this doesn't get the timezone right -ofxDateParse :: String -> String -ofxDateParse x = formatTime defaultTimeLocale "%Y-%m-%d" (fst (fromMaybe (LocalTime (ModifiedJulianDay 100) (TimeOfDay 0 0 0), "") (strptime "%Y%m%d%H%M%S.%OS" x))) - -parseFakeXML string = readString [ withValidate no - , withRemoveWS yes - ] string - -atTag tag = deep (isElem >>> hasName tag) -text = getChildren >>> getText -textAtTag tag = atTag tag >>> text - -getTransactions = atTag "STMTTRN" >>> - proc l -> do - trnType <- textAtTag "TRNTYPE" -< l - dtUser <- textAtTag "DTUSER" -< l - dtPosted <- textAtTag "DTPOSTED" -< l - trnAmt <- textAtTag "TRNAMT" -< l - fitId <- textAtTag "FITID" -< l - refNum <- textAtTag "REFNUM" -< l - name <- textAtTag "NAME" -< l - memo <- textAtTag "MEMO" -< l - returnA -< Transaction - { trnType = trnType, - dtUser = ofxDateParse dtUser, - dtPosted = ofxDateParse dtPosted, - trnAmt = trnAmt, - fitId = fitId, - refNum = refNum, - name = name, - memo = memo } - -ofxrules = CsvRules { - dateField=Just 0, - dateFormat=Nothing, - statusField=Nothing, - codeField=Nothing, - descriptionField=[FormatField False Nothing Nothing (FieldNo 2)], - amountField=Just 1, - inField=Nothing, - outField=Nothing, - currencyField=Nothing, - baseCurrency=Nothing, - accountField=Nothing, - account2Field=Nothing, - effectiveDateField=Nothing, - baseAccount="Liabilities:American Express", - accountRules=[] -} - -txnToCsvRecord :: Transaction -> CsvRecord -txnToCsvRecord x = [dtUser x, normAmount (trnAmt x), compressWhitespace (name x) ++ "(" ++ refNum x ++ ")", fitId x, memo x] - -printTxnWithComment :: CsvRecord -> IO () -printTxnWithComment x = putStrLn ("; " ++ x !! 3 ++ " - " ++ x !! 4) >> printTxn False ofxrules x - -main = do - filecontents <- readFile "/tmp/ofx.ofx" - let splitfilecontents = splitOn "\n\n" filecontents - let ofxheader = head splitfilecontents - let ofxsgml = splitfilecontents !! 1 - (_, fakexml, _) <- readProcessWithExitCode "/usr/bin/sgml2xml" [] ofxsgml - - transes <- runX (parseFakeXML fakexml >>> getTransactions) - - let records = map txnToCsvRecord transes - mapM_ (printTxnWithComment) records - -**** qif reader -***** clint's code -Date: Tue, 25 Oct 2011 11:46:24 -0400 -From: Clint Adams -To: hledger@googlegroups.com -Cc: thomas@marketpsychdata.com, jjenning@fastmail.fm -Subject: Re: QIF parsing -Message-ID: <20111025154624.GA3097@softwarefreedom.org> -References: <20111006164952.GA734@softwarefreedom.org> -MIME-Version: 1.0 -In-Reply-To: <20111006164952.GA734@softwarefreedom.org> -User-Agent: Mutt/1.5.21 (2010-09-15) -X-Original-Sender: clint@softwarefreedom.org -X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: - domain of clint@softwarefreedom.org designates 207.86.247.70 as permitted - sender) smtp.mail=clint@softwarefreedom.org -Reply-To: hledger@googlegroups.com -Precedence: list -Mailing-list: list hledger@googlegroups.com; contact hledger+owners@googlegroups.com -List-ID: -X-Google-Group-Id: 895107692464 -List-Post: , -List-Help: , -List-Archive: -Sender: hledger@googlegroups.com -List-Subscribe: , -List-Unsubscribe: , -Content-Type: text/plain; charset=iso-8859-1 -Content-Disposition: inline -Content-Transfer-Encoding: 8bit -X-Truedomain-Domain: googlegroups.com -X-Truedomain-SPF: Neutral (mx4: 173.255.219.222 is neither permitted nor denied by domain of googlegroups.com) -X-Truedomain-DKIM: Pass -X-Truedomain-ID: 16FADD416626EE6BDC6CCBB61A94EA31 -X-Truedomain: Neutral - -I had to update my QIF converter for modern hledger; included below. - -Thomas, I didn't see your reply because I'm not subscribed to -this Google Group. I believe that QuickBooks uses OFX, not QIF, -so you'd be more interested in - -http://groups.google.com/group/hledger/browse_thread/thread/e03ccc655347ba72 - -or - -http://www.dingoskidneys.com/~jaredj/ - -------8<------- - -import Text.Parsec -import Text.Parsec.String - -import Control.Monad.State as State - -import System (getArgs) -import Data.List (groupBy) -import Data.Maybe (fromMaybe) -import qualified Data.Map as Map -import Text.Printf (printf) - -import Hledger.Cli.Format (FormatString (FormatField), Field (FieldNo)) -import Hledger.Cli.Convert - -qifFile :: GenParser Char st (String,[[TransactionDetail]]) -qifFile = do - skipMany newline - dtype <- typeHeader - newline - trans <- endBy1 transaction recordSep - return $ (dtype,trans) - -typeHeader :: GenParser Char st String -typeHeader = do - string "!Type:" - dataType - -dataType :: GenParser Char st String -dataType = do string "Cash" - <|> string "Bank" - <|> string "CCard" - <|> string "Invst" - <|> string "Oth A" - <|> string "Oth L" - <|> string "Invoice" - -transaction :: GenParser Char st [TransactionDetail] - -*** download via ofx protocol -*** parsing: more date syntax ? last nov, next friday, optional this, week of -*** parsing: more period syntax ? every N days, biweekly -*** cli: accept multiple journal file options -*** i18n -*** speed, memory usage -**** simple/more frequent benchmarking -*** clear, documented interfaces/surfaces -*** plugin architecture/modular packaging -**** goals -***** allow separately-packaged functionality to be discovered at run-time and integrated within the hledger ui. -Example: user installs hledger-ofx package from hackage, or adds Ofx.hs to their ~/.hledger/plugins/; -then "ofx" is among the commands listed by hledger --help, and/or is a new command available in -the web and vty interfaces, and/or is a new file format understood by the convert command. -**** issues to consider -***** what is the api for plugins ? -they'll want to import Ledger lib, to work with ledger data structures - -***** there are different kinds of "plugin". What could plugins provide ? -****** commands - for all uis, or for one or more of them (cli, web, vty..). A command may itself be a new ui. -****** import/export formats -****** skins/styles/templates for uis, eg the web ui ? -**** techniques to consider -***** running executables provided by plugins -a cli command plugin: cli execs the executable with same arguments -a web command plugin: web ui runs the executable as a subprocess and captures the output -***** linking plugins into main app with direct-plugins -simplification of plugins lib -main app needs to know the types used in plugin's interface -weakens type safety, avoiding runtime errors requires extra care -requires whole-program linking at plugin load time -plugins can be discovered by querying ghc for installed packages or modules in a known part of the hierarchy -maintained and keen to help -***** linking plugins into main app with plugins (original) -more complex than above -more type-safe/featureful ? -***** doing whatever xmonad does with dyre -***** interpreting plugins under control of main app with hint -ghci in an IO-like monad -types need converting, etc. -plugins may run more slowly -plugins can be discovered/loaded by module path or by loading files directly +* about +Simon keeps developer notes in this emacs org-mode outline, NOTES.org. +Currently it contains my backlog, activity log, and random notes and only +I use this file, though anyone may send a patch. +** backlog +Todos and wishlist items, roughly grouped by category and ordered by (my) +priority. Used for planning, and often as a more lightweight bug +tracker. Also contains.. +** estimates +I keep effort-to-complete estimates on backlog tasks as N[dwm] at the end +of the line following two spaces, meaning N hours, days, weeks or months, +assuming 4 hours in day, 5 days in a week, and 4 weeks in a month. +** burndown charts +At some point there might be a process to aggregate estimates, log them +(including deleted items as 0) on commit, and make burndown charts. + + +* routines +standard procedures/checklists for various activities * misc @@ -1517,7 +48,7 @@ ABILITY TO SEE. --Scott McCloud, Understanding Comics *** less is more *** code review/pair programming -** things I want to know +** things I want to know about money and time *** time where have I been spending my time in recent weeks ? where have I spent my time today ? @@ -1556,14 +87,14 @@ are there any cashflow, tax, budgetary problems looming ? [2:08pm] those would be a good start. How do I make those visual [2:09pm] well I guess the first step is a script to print them -** data representation -*** http://www.python.org/dev/peps/pep-0327/ -*** http://www.n-heptane.com/nhlab/repos/Decimal/ -*** http://www.n-heptane.com/nhlab/repos/Decimal/Money.hs -*** http://www2.hursley.ibm.com/decimal/ ** other docs *** http://en.wikibooks.org/wiki/Accounting *** http://books.google.com/books?id=4V8pZmpwmBYC&lpg=PP1&dq=analysis%20patterns&pg=PA95#v=onepage&q&f=false +*** data representation +**** http://www.python.org/dev/peps/pep-0327/ +**** http://www.n-heptane.com/nhlab/repos/Decimal/ +**** http://www.n-heptane.com/nhlab/repos/Decimal/Money.hs +**** http://www2.hursley.ibm.com/decimal/ *** lwn grumpy editor articles **** http://lwn.net/Articles/149383/ **** http://lwn.net/Articles/153043/ @@ -1721,14 +252,15 @@ http://stackoverflow.com/questions/6398996/good-haskell-source-to-read-and-learn *** bitcoin **** http://cryptome.org/0004/bitcoin-triple.htm **** http://forum.bitcoin.org/index.php?topic=2609.0 -** software +*** selinger article on currency & capital gains accounting +http://www.mscs.dal.ca/~selinger/accounting/tutorial.html#1.2 +** other software *** http://gnucash.org *** http://www.xtuple.com/postbooks *** http://weberp.org **** http://www.weberp.org/weberp/doc/Manual/ManualContents.php *** http://www.clientsandprofits.com -** selinger article on currency & capital gains accounting -http://www.mscs.dal.ca/~selinger/accounting/tutorial.html#1.2 +*** YNAB ** hledger feedback *** fabrice niessen +For me, what would be very useful for a 1.0 version would be: @@ -1767,7 +299,7 @@ itself reembursed by FFIS). We do not bother with any local tasks like income tax, vat or statistical filing, invoicing and the like. -** hamlet feedback +** my hamlet feedback *** thread data through nested templates with Reader monad *** HDString constructor for HamletData ? *** data type for non-RT Hamlet as well, or better, the same type for both @@ -1782,7 +314,7 @@ statistical filing, invoicing and the like. **** @ @ could recognise tuples automatically; drop ? **** why !: : for conditional attributes ? How about !? ? -** code snippets +** snippets let assertAccountsReportItemEqual ((ea1,ea2,ei,eamt), (aa1,aa2,ai,aamt)) = do assertEqual "full account name" ea1 aa1 @@ -3456,3 +1988,1526 @@ move *FromOpts into toOpts *** 5/17 tests, porting entriesReport to query *** 5/29 release *** 6/29 release +*** 6/30 announce, notes +*** 7/1 notes + + +* backlog +** errors +*** double quote matches everything ? 1 +*** web: stray bracket in journal edit form title 1 +*** web: enter doesn't work in add form completing fields 1d +**** research this dhtmlxcombo issue +*** parsing: decimal point/thousands separator confusion ? 1d +<<< +2011/09/30 + a $1,000,000.00 + b + +2011/09/30 x + a $1,2 + b + +2011/09/30 y + a $1.2 + b +>>> hledger -f t print +2011/09/30 + a $1,000,000.00 + b $-1,000,000.00 + +2011/09/30 x + a $1.20 + b $-1.20 + +2011/09/30 y + a $1.20 + b $-1.20 + +*** parsing: recursive file includes cause a hang 2 +echo "!include rec" > rec +hledger -f rec print +*** parsing: "could not balance" error does not show line number 1d +*** parsing: extra noise with eg bad date parse errors 1d +$ cat t.journal +200/1/99 x + a 1 + b +$ ./hledger.hs -f t.journal print +hledger.hs: could not parse journal data in t.journal +"t.journal" (line 1, column 9): +unexpected " " <- undesired +expecting digit <- noise +bad year number: 200 + +*** parsing: confusing error when journal lacks a final newline 1d +$ cat - >t.j +2010/1/2 + a 1 + b +$ hledger -f t.j bal +hledger: could not parse journal data in t.j +"t.j" (line 3, column 3): +unexpected "b" +expecting comment or new-line + +*** add: default amount adds one decimal place when journal contains no decimals 2d +*** add: excessive precision in default balancing amount 1d +**** shelltest tests/add.test -- -t10 +**** find original justification or drop +*** add: learn decimal point/thousands separator from the journal and/or add session ? 2d +Eg: comma is already used as thousands separator in the journal, but add +interprets it as decimal point giving a wrong default for amount 2 (though +the correct journal transaction is written in this case) + +$ hledger -f t add +Adding transactions to journal file "t". +To complete a transaction, enter . (period) at an account prompt. +To stop adding transactions, enter . at a date prompt, or control-d/control-c. +date, or . to end [2011/09/30]: +description []: z +account 1: a +amount 1: 1,000 +account 2: b +amount 2 [-1,0]: +account 3, or . to record: . +date, or . to end [2011/09/30]: . +$ cat t +; journal created 2011-09-30 by hledger + +2011/09/30 + a $1,000,000.00 + b + +2011/09/30 x + a $1,2 + b + +2011/09/30 y + a $1.2 + b + +2011/09/30 z + a 1,000 + b + +*** convert: 49 convert should report rules file parse errors better 1d +*** 25 hledger in windows console does not print non-ascii characters 3d +http://stackoverflow.com/questions/10779149/unicode-console-i-o-in-haskell-on-windows +http://hackage.haskell.org/trac/ghc/ticket/4471 +*** journalAddFile is called in reverse order of includes 2 +** documentation, marketing +*** developer notes +**** 2012/7 cleanup +***** quick cleanup finance onwards 2 +***** add some estimates 1 +**** review/prune backlog 1 +**** estimate summing 1d +***** research existing, ask in #orgmode +***** org-sum +**** burndown charts 2d +***** research existing tools +*** finalise/link 2012 survey 2 +*** document status flag better 1 +*** review/prune docs 1d +*** announcements +**** list +***** release +***** HCAR, twice yearly +****** update entry & process +*** short description +**** collect/clarify +hledger is a robust command-line accounting tool with a simple plain text data format. + +hledger is a reporting tool for accounting transactions stored in a simple human-editable text format. + +hledger is a computer program for easily tracking money, time, or other transactions, usually recorded in a general journal file with a simple human-editable markup format. + +hledger is primarily a reporting tool, but it can also help you add transactions to the journal, or convert from other data formats. + +hledger is a haskell port and friendly fork of John Wiegley's c++ ledger tool. + +hledger aims to be a reliable, practical, useful tool for (slightly geeky) users and a reusable library for haskell programmers interested in finance. + +hledger is quite simple in essence, aiming to be a reliable low-level parsing-and-reporting tool that doesn't get in your way. + +For some, it is a less complex, less expensive, more efficient alternative to Quicken or Quickbooks. + +hledger is available for free under the GNU General Public License. + +hledger reads plain text files (general journal, timelog, or CSV format) describing transactions (in money, time or other commodities) and prints the chart of accounts, account balances, or transactions you're interested in. + +hledger is a free program that helps you understand your finances, making calculations based on data stored a simple text file. If you prefer the command line and a text editor to a big gui application, hledger gives you the power of Quicken and Quickbooks without the complexity. + +Your financial data will outlive your financial software, so it should have longevity and accessibility. Its integrity is important to your peace of mind, so changes should be transparent and (if desired) version controlled. It may also be important to allow multiple authors to edit safely. A structured, easy-to-parse, human-friendly plain text format, as in the wiki world, provides a good balance of longevity, reliability, transparency and flexibility. + +hledger helps you track and understand your finances, making calculations based on data stored in a simple text file. If you prefer the command line and a text editor to a big gui application, hledger gives you the power of Quicken and Quickbooks without the complexity. + +Features: reads transactions in journal, timelog, or CSV format; handles multi-currency/multi-commodity transactions; prints the chart of accounts, account balances, or transactions you're interested in, quickly; scriptable. + +hledger is written in the Haskell programming language; +it demonstrates a pure functional implementation of ledger. + +*** medium intro blurb +**** collect/clarify +***** README file +***** hledger.hs module description +***** hledger.cabal description field (exclude home page link) +***** home page description (http://joyful.com/Hledger/editform) +***** mail list description (http://groups.google.com/groups/hledger -> edit welcome msg) +***** gmane description +***** darcsweb description + +**** keep in sync +***** refine process +*** command-line docs +**** keep usage info in sync +***** Options.hs +***** MANUAL.md +**** browse/search manual content 2d +*** feature list +**** full 1 +**** short 1 +*** manual +**** fix pre/toc overlap on manual 2 +**** clarify reference nature 1 +*** FAQ +**** create/highlight 1d +**** life cycle of top-level accounts +For personal ledgers, when you're born, all accounts are at zero (one hopes) and as you live: + +1. Equity accounts accommodate your previous years of not maintaining accounts (fixed, probably negative) + +2. Expense accounts become more and more positive (unavoidably) + +3. Income accounts become more and more negative (on payday) + +4. Assets Accounts become more and more positive (in good times) + +5. Liability account become more positive (in good times, when you pay them off) and more negative (when you use them to buy things). + +When you die, Equity: and Income: will stand at large negative balances, +Expense: and Assets: will stand at large positive balances and Liabilities +will have to be paid (out of Assets) before your heirs get what's left. + +adapted from Ben Alexander, ledger-cli + +*** website +**** review stats 1h +**** clean up stats 1d +**** refresh +*** progressive tutorial +**** plan, begin 1d +*** screencasts +**** brainstorm +***** intro +****** intro to hledger +******* place in the world +******* basic installation +******* quick demo +******* where to go from here +****** installing hledger on windows +****** installing hledger on mac +****** installing hledger on unix +****** accessing hledger's support forums +******* website +******* mail list +******* irc channel +****** reporting a hledger bug +***** using +****** income/expense tracking +****** time tracking +****** downloading bank data +****** reconciling with bank statement +****** see time reports by day/week/month/project +****** get accurate numbers for client billing and tax returns +****** find unpaid invoices +***** developing +****** intro to hledger development +****** testing hleder +****** documenting hledger +****** a hledger coding example +****** a tour of hledger's code +***** ledger cooperation +*** blog posts +*** examples/how-tos +*** hledger/ledger comparison/feature matrix 1d +*** improve aesthetics +**** embed screenshots in web docs +***** use highslide +*** improve liveness +**** show feeds on site ? +***** commits +**** cc/summarise repo activity to list ? +*** developer guide +**** clarify/merge developer guide 2h +**** How to do anything that needs doing in the hledger project. +***** website & documentation +****** overview of hledger docs +****** how the site is built +****** convenience urls +list.hledger.org - mail list +bugs.hledger.org - issue tracker +bugs.hledger.org/1 - go to specific issue +bugs.hledger.org/new - create a new issue +hledger.org/{list,bugs}/* also works + +***** issue tracking +***** testing + +hledger's unit tests and a simple test runner are built in. They can +be run several ways: + +$ hledger test [PAT] +$ make unittest +$ make autotest + +They can also be built as a separate executable, in case needed for +cabal test. (?) This requires test-framework, which may not work on +windows. + +$ make unittest-standalone + +hledger's functional tests are a set of @shelltestrunner@ tests defined +by .test files in the tests\/ subdirectory. + +$ make functest + +Shell tests can also be defined as doctests, literal blocks embedded +in modules' haddock docs, though this is hardly used. For example: + +@ +$ bin\/hledger -f data\/sample.journal balance o + $1 expenses:food + $-2 income + $-1 gifts + $-1 salary +-------------------- + $-1 +@ + +$ make doctest + +***** coding +***** funding process +****** donation blurb +If you like project or have +benefited from it, you can give back by making one-time or periodic +donations of any amount. This also allows me to offer further +enhancements, maintenance and support for this project. Thanks! +**** reference +***** unsafe things which may fail at runtime include.. +****** incomplete pattern matching +****** error +****** printf +****** read +*** api docs +*** darcs show authors +**** clean up output 2 +***** trygve +***** encoding +eg in text-mode emacs 24 +*** roadmap +**** review old +***** 1.0 +culmination of 0.x releases - stable/usable/documented +followup releases are 1.01, 1.02.. +GHC 6.12/HP 2010 primary platform +GHC 6.10/HP 2009 also supported if possible +GHC 6.8 might work for core features, but not officially supported +separate ledger package ? license ? +separate vty, web packages ? +support plugins ? +web: loli+hsp+hack+simpleserver/happstack, or yesod+hstringtemplate+wai+simpleserver/happstack ? +add: completion ? +chart: register charts ? +histogram: cleaned up/removed +complete user manual +binaries for all platforms ? + +***** 2.0 +development releases are.. 1.60, 1.61.. or 1.98.01, 1.98.02.. +separate ledger lib +plugins +Decimal +binaries for all platforms + +*** internal code docs +*** live demos/talks +** finance +*** develop funding process +**** license change ? +**** donate button, see chimoo guy +**** funding document 2009/01 +***** text +======= +funding +======= + +vision +====== + + How to grow the hledger project ? + + I'm looking for ways to fund active and sustainable hledger + development by me and others. + + A secondary goal is to develop new sustainable models and processes + for funding free software developers and other community projects. + + This is sometimes the point in a free sw project's development where + the project leader seemingly loses the plot, alienates contributors + and destroys the community's good-will dynamic. I've seen it many + times, but a few have succeeded and I want to be one of them - so + that I can eat, have a modicum of stability and do my best work in + service to the community. At worst, I'll look bad but the project + will still be out there. At best I'll live more easily and joyfully + while serving the cause of Financial Solvency! + + So I'm beginning by posting these notes and inviting your thoughts - + as much or as little as folks want to give. How could we do this + so that all benefit ? + +funding models +============== + Brainstorming some possible funding models & processes. + + * grants + + How to find possible grant sources ? + + * con + + * getting grant funding is a whole new field to study + * slow and time intensive, I imagine + + * donations + + Solicit donations. + + * pro + + * simple + + * con + + * often difficult + * donators do not feel a direct benefit + + * shareware + + Release the project under a non-free license, requiring commercial + users to pay the fee on an honour basis (eg). + + * pro + + * flexible, low administration, encourages trust + + * con + + * effectively closed-source ? would inhibit collaboration + * benefit is still indirect, only a proportion will pay + * enforcement/guilt may come into play + + * limited-time premium branch + + The funded version of hledger gets some desirable premium features + before the free version and is closed-source. Funders/customers pay + a fixed price for immediate access to the funded version. Yearly, a + new funded version is released and the old funded version is merged + into the free version. (To gain experience it could be done on a + smaller scale, eg monthly/quarterly.) + + * pro + + * all features reach community, predictably + * customers are also community funders + * customers receive direct benefit from paying + + * con + + * free sw developers compete/outshine the premium branch + + * bounties + + Some (or all) feature, bugfix, project management or other tasks are + published with a bounty attached. When the bounty is paid by one or + more funders, the task is performed and delivered. Or, bounty is paid + on completion of task (honour system). + + * pro + + * funders receive direct benefit + + * bounties using fundable.org (eg) + + A more organised form of the above, perhaps facilitating trust, + co-funding and larger bounties. + + * pro + + * proven process developed by others + + * con + + * fundable takes a cut + + * hosted service + + Offer hosted and managed ledgers, perhaps with premium features, for + a monthly fee + + * pro + + * proven model + * clear benefit to customers, especially non-technies + + * con + + * success of free/self-installed version competes with hosting service + * some will avoid web-hosting their financial data + + * customisation + + Offer per-user customisations, possibly to be merged in the trunk, + for a fee + + * support + + Offer user/developer support for a fee + + * training + + Offer application and/or financial training for a fee + + * profit sharing/tithing + + Each period (quarter, half-year, year), donate 10% (eg) to project + contributors and/or supporting projects + + * transparent funding + + Funding and usage of funds is published on the web as a ledger + + * opaque funding + + All funding and spending need not be made public + +strengths +========= + hledger has some aptitudes in this area: + + * hledger deals with money => hledger users will tend to have some money + * hledger's purpose is to increase financial success => users will feel its value to their bottom line + * hledger is a tool that can support project funding, eg by publishing community funding data + +weaknesses +========== + * hledger doesn't have a nice ui yet + * hledger has a limited featureset + * hledger requires work, eg data entry and chart of accounts maintenance + * hledger is geeky + * there is competition + * hledger has no compelling market niche (aside from payment-averse free software users) + +competitors/fellow niche inhabitants +==================================== + + * web apps + + * netsuite + * sql-ledger, ledgersmb + * wesabe + * ... + + * desktop apps + + * quickbooks + * quicken + * ms money + * grisbi + * gnucash + * excel + * ledger! + * ... + +***** responses +****** albino + have you considered talking to business who hate their financial sw and going from there +****** gwern + most haskellers have never heard of hledger, sounds arrogant or hubristic to talk of charging for it + +**** home edition +**** real-time project ledger +**** in-place transaction editing fund drive + +Fund drive: hledger-web in-place transaction editing + +Goal: I would like to raise $X or more to fund basic in-place +transaction editing for hledger-web. hledger-web is a web-based GUI for +hledger (and ledger), which are free/open-source accounting programs +providing a lean and efficient alternative to quicken, gnucash, mint.com +etc. + +Current hledger-web[1] has simple web forms for adding transactions and +for editing the whole journal, but there is no easy ui for editing a +single existing transaction. Such a ui is an important step towards making +hledger (and ledger) usable by non techies, which would greatly expand +these tools' applicability and potential user/contributor base. + +Plan: do the front-end javascript and backend haskell work required to +support: + +- click date, description, account or amount cells in a register view to make that cell editable +- tab moves to the next cell +- enter or click on save button updates the transaction in the journal, overwriting/rewriting the whole file +- tested in firefox/chrome/safari + +The proposed amount will fund about 10 hours of work, so the above +features must be implemented very expeditiously. Other improvements will +be tackled in a followup fund drive if this one succeeds (or in this one +if the funding goal is exceeded.) Those future items include: + +- history/content awareness, smart defaults and auto-completion wherever useful +- date picker widget +- ability to add/remove postings +- ability to edit metadata/tags +- ability to edit other transaction/posting fields +- ledger compatibility +- compatibility testing/fixes for all the major browsers +- edit conflict checking - don't overwrite concurrent external edits +- try harder to preserve existing file layout/co-exist better with external edits +- a similar ui for adding new transactions +- pleasant visual style + +Also, 10% of the amount raised will be tithed to three contributing +projects or developers (ledger and two others of my choice.) + +This project will go forward if + +[1] http://demo.hledger.org:5001 +** testing +*** test running improvements +**** test: duplicate runs +$ hledger test 'showTransaction$' +Cases: 6 Tried: 0 Errors: 0 Failures: 0([],"") +### Failure in: 0:showTransaction +show a balanced transaction, eliding last amount +expected: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n assets:checking\n\n" + but got: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n\n assets:checking \n\n\n" +Cases: 6 Tried: 1 Errors: 0 Failures: 1([],"") +### Failure in: 1:showTransaction +show a balanced transaction, no eliding +expected: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n assets:checking $-47.18\n\n" + but got: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n\n assets:checking $-47.18\n([],"") +\n\n" +Cases: 6 Tried: 2 Errors: 0 Failures: 2([],"") +### Failure in: 2:showTransaction +show an unbalanced transaction, should not elide +expected: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n assets:checking $-47.19\n\n" + but got: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n\n assets:checking $-47.19\n([],"") +\n\n" +Cases: 6 Tried: 3 Errors: 0 Failures: 3([],"") +### Failure in: 3:showTransaction +show an unbalanced transaction with one posting, should not elide +expected: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n\n" + but got: "2007/01/28 coopportunity\n expenses:food:groceries $47.18\n\n\n" +Cases: 6 Tried: 4 Errors: 0 Failures: 4([],"") +### Failure in: 4:showTransaction +show a transaction with one posting and a missing amount +expected: "2007/01/28 coopportunity\n expenses:food:groceries \n\n" + but got: "2007/01/28 coopportunity\n expenses:food:groceries \n\n\n" +Cases: 6 Tried: 5 Errors: 0 Failures: 5([],"") +### Failure in: 5:showTransaction +show a transaction with a priced commodityless amount +expected: "2010/01/01 x\n a 1 @ $2\n b \n\n" + but got: "2010/01/01 x\n a 1 @ $2\n\n b \n([],"") +\n\n" +Cases: 6 Tried: 6 Errors: 0 Failures: 6 + 1 +**** stop on first failure +**** run tests in bottom up order +*** envision better test setup +**** every parser has a test and is easy to test +**** easy to run any single test or module's tests +**** tests run bottom up by default +**** test runner can select tests precisely eg by regexp +**** test runner stops at first failure by default + +*** documentation +**** site up, current ? +**** demo up, current ? +**** haddock building, current ? +**** doctests ? +*** unit +**** hunit +**** quickcheck +**** easier unit test development +*** functional +**** ledger file parsing tests +***** test all ledger file format features +***** clarify hledgerisms in file format - that hledger can read but ledger can't +**** ledger 3 baseline tests +**** MaybeSo subtotal rounding issue +I had a question about balance totals. Given this test data: + +$ cat test.dat +D $1,000.00 +P 2011-01-01 22:00:00-0800 TESTA $78.35 +P 2011-01-01 22:00:00-0800 TESTB $15.86 +P 2011-01-01 22:00:09-0800 TESTC $13.01 + +2011/01/01 Example + Assets:Brokerage:TESTA 188.424 TESTA @ $76.61 + Assets:Brokerage:TESTB 1,809.282 TESTB @ $15.60 + Assets:Brokerage:TESTC 384.320 TESTC @@ $5,000.00 + Assets:Brokerage:TESTC 5.306 TESTC @@ $68.18 + Equity:Opening Balances + +I'm a little bit surprised that the sub-accounts +reflect a difference from the top level account +w/re to rounding the last cent: + +$ ledger -V -f test.dat bal + $48,527.27 Assets:Brokerage + $14,763.02 TESTA + $28,695.21 TESTB + $5,069.03 TESTC + $-47,728.14 Equity:Opening Balances +-------------------- + $799.13 + +Even if --no-rounding is passed in: + +$ ledger -V -f test.dat --no-rounding bal + $48,527.27 Assets:Brokerage + $14,763.02 TESTA + $28,695.21 TESTB + $5,069.03 TESTC + $-47,728.14 Equity:Opening Balances +-------------------- + $799.13 + +Is there something off with how the data aboce is set up? Should I be +using be more place holders? +*** performance +**** speed, benchmark tests +**** memory usage +*** build & packaging +**** use -Wall and anything else useful +**** build with multiple ghc versions +**** cabal test +**** hackage upload +**** cabal install with: +***** ghc 6.8 +***** ghc 6.10.x +***** windows +***** linux +***** macos +***** no flags +***** happs flag +***** vty flag +*** field +**** talkback, auto bug reports +**** usability +**** download & usage stats +** packaging, installability +*** linux +***** debian/ubuntu packaging +*** mac +***** easy installer +***** easy startup +*** windows +***** easy installer +***** easy startup +** refactoring +*** makefile cleanups +*** make shell tests version independent +tests/no-such-file.test: rm -f $$; bin/hledger register -f $$; rm -f $$ +tests/no-such-file.test: rm -f $$; bin/hledger balance --no-total -f $$; rm -f $$ +tests/add.test: rm -f t$$.j; bin/hledger -f t$$.j add; rm -f t$$.j +tests/add.test: rm -f t$$.j; bin/hledger -f t$$.j add; rm -f t$$.j +tests/add.test: rm -f t$$.j; bin/hledger -f t$$.j add; rm -f t$$.j +tests/add.test: printf 'D $1000.00\n' >t$$.j; bin/hledger -f t$$.j add >/dev/null; cat t$$.j; rm -f t$$.j +tests/add.test: printf 'D $1000.0\n' >t$$.j; bin/hledger -f t$$.j add >/dev/null; cat t$$.j; rm -f t$$.j +tests/add.test: printf '2010/1/1\n a $1000.00\n b\n' >t$$.j; bin/hledger -f t$$.j add >/dev/null; cat t$$.j; rm -f t$$.j +tests/add.test: printf '2010/1/1\n a $1000.0\n b\n' >t$$.j; bin/hledger -f t$$.j add >/dev/null; cat t$$.j; rm -f t$$.j +tests/add.test: printf 'D $1000.0\nD £1,000.00\n' >t$$.j; bin/hledger -f t$$.j add; cat t$$.j; rm -f t$$.j +tests/add.test:rm -f nosuch.journal; bin/hledger -f nosuch.journal add; rm -f nosuch.journal +tests/add.test:## printf '\n\na\n1\nb\n' | bin/hledger -f /dev/null add +tests/add.test:# bin/hledger -f /dev/null add +tests/amount-layout-vertical.test:# bin/hledger -f - print +tests/amount-layout-vertical.test:# bin/hledger -f - register +tests/amount-layout-vertical.test:# bin/hledger -f - balance +tests/parse-posting-error-pos.test:# bin/hledger -f- stat +tests/null-accountname-component.test:# bin/hledger -f - balance -E +tests/include.test: mkdir -p b/c/d ; printf '2010/1/1\n (D) 1\n' >b/c/d/d.journal ; printf '2010/1/1\n (C) 1\n!include d/d.journal\n' >b/c/c.journal ; printf '2010/1/1\n (B) 1\n!include c/c.journal\n' >b/b.journal ; printf '2010/1/1\n (A) 1\n!include b/b.journal\n' >a.journal ; bin/hledger -f a.journal print; rm -rf a.journal b +tests/timelog-stack-overflow.test:#bin/hledger -f - balance +tests/precision.test:# bin/hledger -f - print --cost +tests/precision.test: bin/hledger -f - balance --cost +tests/timezone.test:# bin/hledger -f - balance --no-total --cost +tests/read-csv.test:rm -rf t.rules$$; printf 'date-field 0\ndate-format %%d/%%Y/%%m\ndescription-field 1\namount-field 2\ncurrency $\nbase-account assets:myacct\n' >t.rules$$; echo '10/2009/09,Flubber Co,50' | bin/hledger -f- print --rules-file t.rules$$; rm -rf t.rules$$ +tests/read-csv.test: printf 'base-account Assets:MyAccount\ndate-field 0\ndate-format %%d/%%Y/%%m\ndescription-field 1\nin-field 2\nout-field 3\ncurrency $\n' >$$.rules ; bin/hledger -f- print --rules-file $$.rules; rm -rf $$.rules +tests/read-csv.test:# rm -rf t.rules$$; printf 'date-fiel 0\ndate-format %%d/%%Y/%%m\ndescription-field 1\namount-field 2\ncurrency $\nbase-account assets:myacct\n' >t.rules$$; echo '10/2009/09,Flubber Co,50' | bin/hledger convert --rules-file t.rules$$; rm -rf t.rules$$ +tests/prices.test:# bin/hledger -f - print + +*** abstract DataSource +*** use matchers for command line too +**** design cli, backwards compatibility strategy +**** replace optsToFilterSpec +*** review/simplify apis +**** simplify option types +*** more modularity +**** packages/namespace +***** hledger-datasource? +**** plugin strategy +**** export lists +**** graph and reduce dependencies +*** switch to http-conduit +*** include latest jquery, jquery-url, minified and non +http://ajaxcssblog.com/jquery/url-read-request-variables/ +*** clarify levels of abstraction +**** web ui balance sheet view - data model, view layout +**** hledger web framework - define routes, handlers/views/actions/controllers/presenters, skins/styles.. +**** happstack - ? happstack api.. +**** hledger app platform - hledger.hs, Options, Utils, withLedgerDo.. +**** hledger lib - Ledger, TimeLog, Account, Transaction, Commodity.. +**** hledger dev platform - make build, ci, test, bench, prof, check, release.. +**** general libs - directory, parsec, regex-*, HUnit, time.. +**** cabal - hledger.cabal, hackage.. +**** ghc - ghc 6.8, 6.10.. +**** haskell 98 +**** unix/windows/mac platform +*** inspiration +http://community.haskell.org/~ndm/downloads/paper-hoogle_overview-19_nov_2008.pdf -> Design Guidelines +** features/wishlist +*** register: for the love of god, show full descriptions +*** add: try wizards lib +*** separate --display and --limit expressions, eg for --depth +*** balancesheet real-world testing +*** narrow (editable) view to selected txns +*** optimise balance report +*** IIF/quickbooks writer +*** inacct: on command line +hledger -f demo.journal reg inacct:expenses:food:pets date:2010/8/25 +2010/08/25 catfood expenses:food:pets $10.00 $10.00 + assets:cash $-10.00 0 +*** support - in period expressions +*** more useful show instances +**** use simple derived show instances for Amount & above ? +**** show no of postings in account tree for Ledger ? +*** hledger-web release +*** don't recompile between autotest & functest +*** stats: list all journal files +*** search by commodity +*** account types +*** runtime templates for reports +*** generalise rules file ? +**** make it applicable to all formats ? +**** absorb its directives into journal format ? +*** support apostrophe digit group separator +*** detect .hs plugins +*** more powerful storage layer +**** Clint's filestore_proof_of_concept.dpatch + +New patches: + +[filestore-proof-of-concept +Clint Adams **20110901172739 + Ignore-this: 1991477c2b70d276665c52478dc54d3d + + This is a somewhat broken replacement of the traditional file + storage with a forced darcs repo. It assumes that the + darcs repo already exists since Data.FileStore refuses to + initialize a repository in an extant directory. It does not + handle any error conditions well. +] hunk ./hledger-lib/Hledger/Read.hs 104 + when (not exists) $ do + hPrintf stderr "No journal file \"%s\", creating it.\n" f + hPrintf stderr "Edit this file or use \"hledger add\" or \"hledger web\" to add transactions.\n" +- emptyJournal >>= writeFile f ++ emptyJournal >>= writeFileWithBackup f + + -- | Give the content for a new auto-created journal file. + emptyJournal :: IO String +hunk ./hledger-lib/Hledger/Utils.hs 40 + import Text.ParserCombinators.Parsec + import Text.Printf + import Text.RegexPR ++import System.FilePath (takeFileName, takeDirectory) ++import qualified Data.FileStore.Types as DFT ++import qualified Data.FileStore.Generic as DFG ++import Data.FileStore.Darcs (darcsFileStore) + -- import qualified Data.Map as Map + -- + -- import Prelude hiding (readFile,writeFile,getContents,putStr,putStrLn) +hunk ./hledger-lib/Hledger/Utils.hs 432 + -- | Apply a function the specified number of times. Possibly uses O(n) stack ? + applyN :: Int -> (a -> a) -> a -> a + applyN n f = (!! n) . iterate f ++ ++-- Store file in VCS; Data.FileStore takes care of only committing ++-- when necessary. ++ ++filestoreSave :: FilePath -> String -> IO () ++filestoreSave f t = DFT.save assumedRepo assumedFilename assumedAuthor logMessage t ++ where ++ assumedRepo = darcsFileStore (takeDirectory f) ++ assumedFilename = takeFileName f ++ assumedAuthor = (DFT.Author "Hledger Role" "hledger@fake") ++ logMessage = "Some kind of change committed by some part of the hledger suite" ++ ++writeFileWithBackup :: FilePath -> String -> IO () ++writeFileWithBackup = filestoreSave ++ ++-- modify existing file in filestore ++filestoreModify :: FilePath -> DFT.RevisionId -> String -> IO (Either DFT.MergeInfo ()) ++filestoreModify f lr t = DFG.modify assumedRepo assumedFilename lr assumedAuthor logMessage t ++ where ++ assumedRepo = darcsFileStore (takeDirectory f) ++ assumedFilename = takeFileName f ++ assumedAuthor = (DFT.Author "Hledger Role" "hledger@fake") ++ logMessage = "Some kind of change committed by some part of the hledger suite" ++ ++filestoreAppend :: FilePath -> String -> IO () ++filestoreAppend f t = do ++ lastrev <- DFT.latest assumedRepo assumedFilename ++ oldcontents <- DFT.retrieve assumedRepo assumedFilename (Just lastrev) ++ result <- filestoreModify f lastrev (oldcontents ++ "\n\n" ++ t) ++ either (\x -> putStrLn "Help, the append didn't work and I am failing miserably.") (\x -> return ()) result ++ where ++ assumedRepo = darcsFileStore (takeDirectory f) ++ assumedFilename = takeFileName f +hunk ./hledger-lib/hledger-lib.cabal 60 + ,containers + ,directory + ,filepath ++ ,filestore + ,mtl + ,old-locale + ,old-time +hunk ./hledger/Hledger/Cli/Add.hs 31 + import qualified Data.Set as Set + + import Hledger +-import Prelude hiding (putStr, putStrLn, appendFile) +-import Hledger.Utils.UTF8 (putStr, putStrLn, appendFile) ++import Prelude hiding (putStr, putStrLn) ++import Hledger.Utils.UTF8 (putStr, putStrLn) + import Hledger.Cli.Options + import Hledger.Cli.Register (postingsReportAsText) + import Hledger.Cli.Utils +hunk ./hledger/Hledger/Cli/Add.hs 194 + journalAddTransaction :: Journal -> CliOpts -> Transaction -> IO Journal + journalAddTransaction j@Journal{jtxns=ts} opts t = do + let f = journalFilePath j +- appendToJournalFile f $ showTransaction t ++ filestoreAppend f $ showTransaction t + when (debug_ opts) $ do + putStrLn $ printf "\nAdded transaction to %s:" f + putStrLn =<< registerFromString (show t) +hunk ./hledger/Hledger/Cli/Add.hs 200 + return j{jtxns=ts++[t]} + +--- | Append data to a journal file; or if the file is "-", dump it to stdout. +-appendToJournalFile :: FilePath -> String -> IO () +-appendToJournalFile f s = +- if f == "-" +- then putStr $ sep ++ s +- else appendFile f $ sep++s +- where +- -- appendFile means we don't need file locking to be +- -- multi-user-safe, but also that we can't figure out the minimal +- -- number of newlines needed as separator +- sep = "\n\n" +- -- sep | null $ strip t = "" +- -- | otherwise = replicate (2 - min 2 (length lastnls)) '\n' +- -- where lastnls = takeWhile (=='\n') $ reverse t +- + -- | Convert a string of journal data into a register report. + registerFromString :: String -> IO String + registerFromString s = do +hunk ./hledger/Hledger/Cli/Utils.hs 18 + journalSpecifiedFileIsNewer, + fileModificationTime, + openBrowserOn, +- writeFileWithBackup, +- writeFileWithBackupIfChanged, + readFileStrictly, + Test(TestList), + ) +hunk ./hledger/Hledger/Cli/Utils.hs 25 + import Control.Exception + import Data.List + import Data.Maybe +-import Safe (readMay) + import System.Console.CmdArgs +hunk ./hledger/Hledger/Cli/Utils.hs 26 +-import System.Directory (getModificationTime, getDirectoryContents, copyFile) ++import System.Directory (getModificationTime) + import System.Exit +hunk ./hledger/Hledger/Cli/Utils.hs 28 +-import System.FilePath ((), splitFileName, takeDirectory) + import System.Info (os) + import System.Process (readProcessWithExitCode) + import System.Time (ClockTime, getClockTime, diffClockTimes, TimeDiff(TimeDiff)) +hunk ./hledger/Hledger/Cli/Utils.hs 123 + -- what not. + -- ::ShellExecute(NULL, "open", "www.somepage.com", NULL, NULL, SW_SHOWNORMAL); + +--- | Back up this file with a (incrementing) numbered suffix then +--- overwrite it with this new text, or give an error, but only if the text +--- is different from the current file contents, and return a flag +--- indicating whether we did anything. +-writeFileWithBackupIfChanged :: FilePath -> String -> IO Bool +-writeFileWithBackupIfChanged f t = do +- s <- readFile f +- if t == s then return False +- else backUpFile f >> writeFile f t >> return True +- +--- | Back up this file with a (incrementing) numbered suffix, then +--- overwrite it with this new text, or give an error. +-writeFileWithBackup :: FilePath -> String -> IO () +-writeFileWithBackup f t = backUpFile f >> writeFile f t +- + readFileStrictly :: FilePath -> IO String + readFileStrictly f = readFile f >>= \s -> Control.Exception.evaluate (length s) >> return s +hunk ./hledger/Hledger/Cli/Utils.hs 125 +- +--- | Back up this file with a (incrementing) numbered suffix, or give an error. +-backUpFile :: FilePath -> IO () +-backUpFile fp = do +- fs <- safeGetDirectoryContents $ takeDirectory $ fp +- let (d,f) = splitFileName fp +- versions = catMaybes $ map (f `backupNumber`) fs +- next = maximum (0:versions) + 1 +- f' = printf "%s.%d" f next +- copyFile fp (d f') +- +-safeGetDirectoryContents :: FilePath -> IO [FilePath] +-safeGetDirectoryContents "" = getDirectoryContents "." +-safeGetDirectoryContents fp = getDirectoryContents fp +- +--- | Does the second file represent a backup of the first, and if so which version is it ? +-backupNumber :: FilePath -> FilePath -> Maybe Int +-backupNumber f g = case regexMatch ("^" ++ f ++ "\\.([0-9]+)$") g of +- Just (_, ((_,suffix):_)) -> readMay suffix +- _ -> Nothing + +*** web api +*** client-side ui +*** support -V ? +*** print: shouldn't support -M and --depth ? +*** in-place editing +**** http://stackoverflow.com/questions/640971/setfocus-to-textbox-from-javascript-after-just-creating-the-textbox-with-javascr +*** measure bug open times +http://code.google.com/p/support/wiki/IssueTrackerAPI +http://code.google.com/p/support/wiki/IssueTrackerAPIReference +*** wai-handler-webkit, wai-handler-launch +*** add: don't offer record txn option in account N prompt if it's not balanced yet +*** add: would be nice to create the journal file only if a txn is actually recorded +*** web: lose io-storage +*** web: use cli args as a permanent additional filter ? +*** web: should display virtual postings with () or [] +*** web: how to find out total spent in an account during a specific month +*** web: how to find out net worth, /register?q=assets+liabilities shows nothing +*** parsing: end directive may also be spelled end account; or end ends last directive +*** convert: clean up default rules file +*** convert: use default rules when converting stdin with no --rules +*** convert: consistent multi-field formats allowed for any field +*** convert: optionally generate single-entry txns +*** convert: optionally set final amount blank +*** convert: parse HH:MM[:SS] as an amount, converting to decimal hours +*** convert: ignore heading row +*** convert: warn and ignore unparsed rows ? +*** stats: list all included files +*** parsing: alias directives should be modified by account directives + +*** web: should take port from base-url when appropriate +*** add: rewrite a short description (trader) to the full description from the matched txn (trader joe's) ? +*** web: can't find out total income:7th total in june at http://localhost:5005/register?q=inacct%3Aassets%3Atreasury ! +*** web: update/remove browser startup +*** docs: simple hledger.org/topic help urls, like php.net +*** docs: --help-web and web ui help links that go to online help, with paragraph comments & chat +cf clients & profits interactive user guide, php.net, realworldhaskell etc. +*** parsing: accounts tree & metadata specification +; chart of accounts +; defines allowed account names, hierarchy, default sort order, +; and some metadata (account numbers, cf http://www.netmba.com/accounting/fin/accounts/chart/) +ACCOUNTS + assets ; :number: 1000 + cash ; :number: 1010 + HT7 ; :number: 1011 + jan ; :number: 1011.01 + feb ; :number: 1011.02 + mar ; :number: 1011.03 + RSG ; :number: 1012 + jan ; :number: 1012.01 + feb ; :number: 1012.02 + mar ; :number: 1012.03 + bank ; :number: 1020 + HT7 ; :number: 1021 + RSG ; :number: 1022 + reserve ; :number: 1023 + liabilities ; :number: 2000 + accounts payable ; :number: 2010 + BSG/GI/RSG ; :number: 2011 + equity ; :number: 3000 + opening balances ; :number: 3010 + income ; :number: 4000 + HT7 ; :number: 4010 + RSG ; :number: 4020 + expenses ; :number: 6000 + rent ; :number: 6010 + +or: http://furius.ca/beancount/examples/demo.ledger + +*** -X/--show-in-commodity +cf http://bugs.ledger-cli.org/show_bug.cgi?id=538 +*** cli, web: consistent rich query language, including and/or/parentheses +ledger's query syntax: http://ledger-cli.org/3.0/doc/ledger.1.html +**** draft 1: + +Filter patterns restrict the postings/transactions that are displayed. +(They often reduce the amount of processing work hledger has to do, as well.) +A pattern is a string or regular expression, usually with a prefix specifying the type of match to do. +The supported prefixes are: + +acct:PAT match postings affecting accounts whose name matches PAT +otheracct:PAT match the other postings in transactions with an acct match (like ledger's --related) +anyacct:PAT match all postings in transactions with an acct match (union of acct and otheracct) +desc:PAT match postings whose description matches PAT +status:PAT match postings whose cleared status matches PAT +code:PAT match postings whose transaction code matches PAT +tag:PAT match postings with a metadata tag whose name matches PAT +tag:TAG=PAT match postings with a metadata tag named TAG whose value matches PAT +from:DATE match postings on or after DATE (like --begin) +to:DATE match postings before DATE (like --end) +in:PERIOD match postings during PERIOD (like --period) +(or ? +begin:DATE match postings on or after DATE (like --begin) +end:DATE match postings before DATE (like --end) +period:PERIOD match postings during PERIOD (like --period) +) + +Prefixes have a short form which is their first letter, except for tag and anyacct. + +Prefix-less patterns are treated like acct: patterns, except by the +register command which treats them as otheracct: . + +Patterns containing whitespace must be enclosed in quotes. + +Matches are always case-insensitive. + +Matches are always substring matches (except for TAG); to match exactly, +wrap the pattern in ^ and $. + +A posting's date, status, code, etc. is usually (but not always) that of +its containing transaction. + +Any of these may be further prefixed with not: for an inverse match. + +Filter patterns may be combined with AND, OR, and parentheses. OR is +assumed by default. +(previously: +When you specify multiple filter patterns, hledger generally selects the +items which match: + + any of the account patterns AND any of the description patterns + +The print command selects transactions which + + match any of the description patterns AND have any postings matching any + of the positive account patterns AND have no postings matching any of + the negative account patterns +) + +*** journalAddTransaction should check txn balances +*** balance setting +ledger: You can accomplish "setting to the bank's view" with a transaction like this: + +2011-08-12 Sample + Assets:Checking = $200.00 + Equity:Adjustments + +This tells Ledger (Git/3.0) that your checking account's balance must be $200 +after this transaction is completed. It will put whatever amounts are +required to accomplish this into the Equity:Adjustments account. + +what about balance assertions ? + +*** web: optional fully self-contained production build (all support files embedded) +*** tools: fix slow make startup +*** tools: better ghci setup +*** docs: better intro, less wall-of-text, separate tutorial & reference sections +*** balance: --depth with --flat should show aggregate balances including the non-displayed deeper accounts +*** parsing: let's enforce positive price amounts for clarity, like ledger +eg 1€ @@ $-2 is not allowed +*** parsing: canonicalise account names to be case-insensitive ? +*** parsing: period expressions should allow interval at the end +eg support -p 'from 1/1 to 2/1 weekly' + +*** parsing: more flexible file including +currently only journals (not timelog files) can include, and only another journal +*** --no-elide -> --empty-parents ? Make it the default ? + $260.00 expenses + $260.00 rent +*** --empty -> --show-zero-accounts, --show-empty-parents ? Make it default ? +*** cli: better control of output format/layout +**** register --format, generalise --format +**** --wide ? window width sensitive ? +**** more tidy/consistent layout from print +**** --output-layout=ledger|traditional +**** --output-format=text|html|pdf +*** balance: try indenting amounts + $260.00 expenses + $260.00 rent +*** parsing: ignore/support ledger assert statements + ; Assertion directives Options + ; These can occur in many places: + ; ; Within an automated transaction, the assert is evaluated every time + ; ; a posting is matched, with the expression context set to the + ; ; matched posting. + ; = /Food/ + ; assert account("Expenses:Food").total >= $100 + ; 2010-06-12 Sample + ; Expenses:Food $100 + ; Assets:Checking + ; ; At file scope, the expression is evaluated within "global" scope. + ; assert account("Expenses:Food").total == $100 + ; ; At the top of a transction, the assertion's scope is the + ; ; transaction. After a posting, the scope is that posting. Note + ; ; however that account totals are only adjusted after successful + ; ; parsing of a transaction, which means that all the assertions below + ; ; are true, even though it appears as though the first posting should + ; ; affect the total immediately, which is not the case. + ; 2010-06-12 Sample 2 + ; assert account("Expenses:Food").total == $100 + ; Expenses:Food $50 + ; assert account("Expenses:Food").total == $100 + ; Assets:Checking + ; assert account("Expenses:Food").total == $100 + +*** parsing: better international number format support (#32) +*** web: : completes one account name component in add form account fields +*** web: auto-complete from substrings, not just prefixes +*** web: auto-complete accounts & amount as well as description +*** parsing: allow price record for null commodity, eg with quotes +P 2009/1/1 "" 0.5h +and why doesn't this work ? time.journal: +P 2010/9/27 h 1 +$ hledger -f time.journal bal -p aug -B + 1 + 17.75h work:jobs + 1 + 17.50h clearview + 1 60 clear glass thermal data + 0.25h admin:cheque issue + 12.75h backups/hosting + 2.00h cleanup + 1.00h move plan + 2.25h move prep + 4.00h testing + 3.50h speed + 1.50h barbara spellcheck issue + 0.50h installation report dates + 0.25h plan change issue + 1.00h planning/discussion + 0.50h speed issue + 0.25h tina quote low-e layout + 0.50h tina title 24 issue + 0.25h kcrw:admin:contract update:unbilled +-------------------- + 1 + 17.75h + +*** --related +*** Double -> Decimal +*** parsing: safety check that effective date > actual (to catch eg 2009/12/30=1/4) +*** parsing: accept all real-world ledger files +As far as I know it currently accepts all ledger 2.6-era files. +Add support for ledger 3 file format as/when that stabilises. +It would be nice to optionally semi/automatically submit parse error reports when they happen +*** talkback feature +gather data on real-world installation & usage issues +simplify bug reporting/handling +improve reliability +*** upload feature +*** generation +**** move generatejournal.hs to a subcommand +*** anonymisation +**** payees +**** accounts +*** easier timelog formats +*** convert: generalise/reuse add's history awareness +*** parsing: per-posting effective/actual dates +*** web: better web ui/gui +*** nice standard financial reports +*** more automated bank data conversion +*** parse more file formats - gnucash, qif, ofx, csv, etc. +**** ofx reader +***** clint's code +Date: Sun, 18 Sep 2011 12:26:16 -0400 +From: Clint Adams +To: hledger@googlegroups.com +Subject: OFX conversion +Message-ID: <20110918162616.GA18874@softwarefreedom.org> +MIME-Version: 1.0 +User-Agent: Mutt/1.5.20 (2009-06-14) +X-Original-Sender: clint@softwarefreedom.org +X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: + domain of clint@softwarefreedom.org designates 216.27.154.199 as permitted + sender) smtp.mail=clint@softwarefreedom.org +Reply-To: hledger@googlegroups.com +Precedence: list +Mailing-list: list hledger@googlegroups.com; contact hledger+owners@googlegroups.com +List-ID: +X-Google-Group-Id: 895107692464 +List-Post: , +List-Help: , +List-Archive: +Sender: hledger@googlegroups.com +List-Subscribe: , +List-Unsubscribe: , +Content-Type: text/plain; charset=iso-8859-1 +Content-Disposition: inline +Content-Transfer-Encoding: 8bit + +This is definitely suboptimal but it seems to work on +the OFX 1.0.2 output from AmEx. + + + +{-# LANGUAGE Arrows, NoMonomorphismRestriction #-} +import Text.XML.HXT.Core +import Text.Printf (printf) + +import Data.List (groupBy) +import Data.List.Split (splitOn) + +import Data.Maybe (fromMaybe) + +import Data.Time.Calendar (Day (ModifiedJulianDay)) +import Data.Time.Format (formatTime) +import Data.Time.LocalTime (LocalTime (LocalTime), TimeOfDay (TimeOfDay)) +import Data.Time.Parse (strptime) + +import System.Locale (defaultTimeLocale) +import System.Process (readProcessWithExitCode) + +import Hledger.Cli.Format (FormatString (FormatField), Field (FieldNo)) +import Hledger.Cli.Convert + +normAmount :: String -> String +normAmount amt | amt == "" = "" + | otherwise = printf "%.2f" (read amt :: Double) + +compressWhitespace :: String -> String +compressWhitespace x = map head $ groupSpaces x + where groupSpaces "" = [""] + groupSpaces x = groupBy (\x y -> x==' ' && y==' ') x + +data Transaction = Transaction + { trnType, dtUser, dtPosted, trnAmt, fitId, refNum, name, memo :: String } + deriving (Show, Eq) + +-- this doesn't get the timezone right +ofxDateParse :: String -> String +ofxDateParse x = formatTime defaultTimeLocale "%Y-%m-%d" (fst (fromMaybe (LocalTime (ModifiedJulianDay 100) (TimeOfDay 0 0 0), "") (strptime "%Y%m%d%H%M%S.%OS" x))) + +parseFakeXML string = readString [ withValidate no + , withRemoveWS yes + ] string + +atTag tag = deep (isElem >>> hasName tag) +text = getChildren >>> getText +textAtTag tag = atTag tag >>> text + +getTransactions = atTag "STMTTRN" >>> + proc l -> do + trnType <- textAtTag "TRNTYPE" -< l + dtUser <- textAtTag "DTUSER" -< l + dtPosted <- textAtTag "DTPOSTED" -< l + trnAmt <- textAtTag "TRNAMT" -< l + fitId <- textAtTag "FITID" -< l + refNum <- textAtTag "REFNUM" -< l + name <- textAtTag "NAME" -< l + memo <- textAtTag "MEMO" -< l + returnA -< Transaction + { trnType = trnType, + dtUser = ofxDateParse dtUser, + dtPosted = ofxDateParse dtPosted, + trnAmt = trnAmt, + fitId = fitId, + refNum = refNum, + name = name, + memo = memo } + +ofxrules = CsvRules { + dateField=Just 0, + dateFormat=Nothing, + statusField=Nothing, + codeField=Nothing, + descriptionField=[FormatField False Nothing Nothing (FieldNo 2)], + amountField=Just 1, + inField=Nothing, + outField=Nothing, + currencyField=Nothing, + baseCurrency=Nothing, + accountField=Nothing, + account2Field=Nothing, + effectiveDateField=Nothing, + baseAccount="Liabilities:American Express", + accountRules=[] +} + +txnToCsvRecord :: Transaction -> CsvRecord +txnToCsvRecord x = [dtUser x, normAmount (trnAmt x), compressWhitespace (name x) ++ "(" ++ refNum x ++ ")", fitId x, memo x] + +printTxnWithComment :: CsvRecord -> IO () +printTxnWithComment x = putStrLn ("; " ++ x !! 3 ++ " - " ++ x !! 4) >> printTxn False ofxrules x + +main = do + filecontents <- readFile "/tmp/ofx.ofx" + let splitfilecontents = splitOn "\n\n" filecontents + let ofxheader = head splitfilecontents + let ofxsgml = splitfilecontents !! 1 + (_, fakexml, _) <- readProcessWithExitCode "/usr/bin/sgml2xml" [] ofxsgml + + transes <- runX (parseFakeXML fakexml >>> getTransactions) + + let records = map txnToCsvRecord transes + mapM_ (printTxnWithComment) records + +**** qif reader +***** clint's code +Date: Tue, 25 Oct 2011 11:46:24 -0400 +From: Clint Adams +To: hledger@googlegroups.com +Cc: thomas@marketpsychdata.com, jjenning@fastmail.fm +Subject: Re: QIF parsing +Message-ID: <20111025154624.GA3097@softwarefreedom.org> +References: <20111006164952.GA734@softwarefreedom.org> +MIME-Version: 1.0 +In-Reply-To: <20111006164952.GA734@softwarefreedom.org> +User-Agent: Mutt/1.5.21 (2010-09-15) +X-Original-Sender: clint@softwarefreedom.org +X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: + domain of clint@softwarefreedom.org designates 207.86.247.70 as permitted + sender) smtp.mail=clint@softwarefreedom.org +Reply-To: hledger@googlegroups.com +Precedence: list +Mailing-list: list hledger@googlegroups.com; contact hledger+owners@googlegroups.com +List-ID: +X-Google-Group-Id: 895107692464 +List-Post: , +List-Help: , +List-Archive: +Sender: hledger@googlegroups.com +List-Subscribe: , +List-Unsubscribe: , +Content-Type: text/plain; charset=iso-8859-1 +Content-Disposition: inline +Content-Transfer-Encoding: 8bit +X-Truedomain-Domain: googlegroups.com +X-Truedomain-SPF: Neutral (mx4: 173.255.219.222 is neither permitted nor denied by domain of googlegroups.com) +X-Truedomain-DKIM: Pass +X-Truedomain-ID: 16FADD416626EE6BDC6CCBB61A94EA31 +X-Truedomain: Neutral + +I had to update my QIF converter for modern hledger; included below. + +Thomas, I didn't see your reply because I'm not subscribed to +this Google Group. I believe that QuickBooks uses OFX, not QIF, +so you'd be more interested in + +http://groups.google.com/group/hledger/browse_thread/thread/e03ccc655347ba72 + +or + +http://www.dingoskidneys.com/~jaredj/ + +------8<------- + +import Text.Parsec +import Text.Parsec.String + +import Control.Monad.State as State + +import System (getArgs) +import Data.List (groupBy) +import Data.Maybe (fromMaybe) +import qualified Data.Map as Map +import Text.Printf (printf) + +import Hledger.Cli.Format (FormatString (FormatField), Field (FieldNo)) +import Hledger.Cli.Convert + +qifFile :: GenParser Char st (String,[[TransactionDetail]]) +qifFile = do + skipMany newline + dtype <- typeHeader + newline + trans <- endBy1 transaction recordSep + return $ (dtype,trans) + +typeHeader :: GenParser Char st String +typeHeader = do + string "!Type:" + dataType + +dataType :: GenParser Char st String +dataType = do string "Cash" + <|> string "Bank" + <|> string "CCard" + <|> string "Invst" + <|> string "Oth A" + <|> string "Oth L" + <|> string "Invoice" + +transaction :: GenParser Char st [TransactionDetail] + +*** download via ofx protocol +*** parsing: more date syntax ? last nov, next friday, optional this, week of +*** parsing: more period syntax ? every N days, biweekly +*** cli: accept multiple journal file options +*** i18n +*** speed, memory usage +**** simple/more frequent benchmarking +*** clear, documented interfaces/surfaces +*** plugin architecture/modular packaging +**** goals +***** allow separately-packaged functionality to be discovered at run-time and integrated within the hledger ui. +Example: user installs hledger-ofx package from hackage, or adds Ofx.hs to their ~/.hledger/plugins/; +then "ofx" is among the commands listed by hledger --help, and/or is a new command available in +the web and vty interfaces, and/or is a new file format understood by the convert command. +**** issues to consider +***** what is the api for plugins ? +they'll want to import Ledger lib, to work with ledger data structures + +***** there are different kinds of "plugin". What could plugins provide ? +****** commands - for all uis, or for one or more of them (cli, web, vty..). A command may itself be a new ui. +****** import/export formats +****** skins/styles/templates for uis, eg the web ui ? +**** techniques to consider +***** running executables provided by plugins +a cli command plugin: cli execs the executable with same arguments +a web command plugin: web ui runs the executable as a subprocess and captures the output +***** linking plugins into main app with direct-plugins +simplification of plugins lib +main app needs to know the types used in plugin's interface +weakens type safety, avoiding runtime errors requires extra care +requires whole-program linking at plugin load time +plugins can be discovered by querying ghc for installed packages or modules in a known part of the hierarchy +maintained and keen to help +***** linking plugins into main app with plugins (original) +more complex than above +more type-safe/featureful ? +***** doing whatever xmonad does with dyre +***** interpreting plugins under control of main app with hint +ghci in an IO-like monad +types need converting, etc. +plugins may run more slowly +plugins can be discovered/loaded by module path or by loading files directly + +