Commit Graph

115 Commits

Author SHA1 Message Date
Simon Michael
e119941648 rename "uncleared" status to "unmarked" and --uncleared to --unmarked (#564)
See the issue and linked mail list discussion. Ambiguity between the
uncleared state, and the "not cleared" --uncleared flag causes confusion
and friction. At this point it seems best to break with Ledger and
past hledger, pick a new name and drop --uncleared to put an end to it.
2017-06-15 19:16:39 -07:00
Simon Michael
aca7c0d93b lib: clarify journalBalanceTransactions 2017-05-30 13:57:52 -07:00
Justin Le
f47df67167 mapping function over journal amounts 2017-02-04 16:38:18 -08:00
Mykola Orliuk
7437c96ff6 Make hledger-rewrite tool suitable for re-factoring original journals (#490)
* cli: fix bug in pivot for postings without tag

Without this fix for postings without tag query checked effective
account which is always empty text ("").

* rewrite: inherit dates, change application order

For budgeting it is important to inherit actual date of posting if it
differs from date of transaction. These dates will be added
as a separate line of comment.

More natural order of rewrites is when result of first defined one is
available for all next rewrites.

* rewrite: factor out Hledger.Data.AutoTransaction

* rewrite: add diff output

With this option you can modify your original files without loosing
inter-transaction comments etc. I.e. you can run:

hledger-rewrite --diff Agency \
  --add-posting 'Expenses:Taxes  *0.17' \
  | patch

As result multiple files should be updated.
Also it is nice to review your changes using colordiff instead of
patch.

* lib: track source lines range for journal

* doc: auto entries and diff output for rewrite
2017-01-20 07:33:24 -08:00
Mykola Orliuk
015b764d00 Preserve implicit amounts and prices the way user wrote them in output of print command (#471)
* Remember original postings during infer and pivot

This includes such functions like:
- inferFromAssignment
- inferBalancingAmount
- inferBalancingPrices
- pivotPosting

* Use original postings for hledger print

- Introduce "--explicit" option for "print" command which brings back
  old behavior when every inferred number being printed.
- Make "print" by default print original postings without inferred
  amounts. But use effective account name to have effect from aliases.
- Instruct shell tests with an new expected output or to use
  --explicit option when inferred amounts are checked.

Resolves simonmichael/hledger#442
2017-01-13 07:25:44 -08:00
Moritz Kiefer
d236f7b237 Fix a few spaceleaks (#413) 2017-01-12 16:24:53 -08:00
Simon Michael
3ae8712bbc rename data/ to examples/ 2017-01-08 07:26:13 -08:00
Johannes Gerer
45401e538e Balance Assignments and accounts resetting (#438)
* Changed behavior of `readJournalFiles` to be identical to `readJournalFile` for singleton lists

* Balance Assertions have to be simple Amounts

* Add 'isAssignment' and 'assignmentPostings' to Hledger.Data.Posting and Transaction

* Implemented 'balanceTransactionUpdate', a more general version of 'balanceTransaction' that takes an update function

* Fixed test cases.

* Implemented balance assignment ("resetting a balance")

* Add assertions to show function

* updated the comments

* numbering is not needed in journalCheckBalanceAssertions

* remove prices before balance checks

* rename functions
2016-12-10 07:04:48 -08:00
Simon Michael
2f4dde3699 lib: debug utils cleanup 2016-09-06 08:31:53 -07:00
Simon Michael
72c39470d6 lib: non-journal formats now produce transaction ids #394
Transactions are now numbered consistently during journal finalisation,
rather than just in the journal reader. Also transaction knot-tying has been
moved out of journalBalanceTransactions.
2016-08-14 12:44:19 -07:00
Simon Michael
7eeb5e01ee ui: I toggles balance assertions
This was a bit tricky.
2016-07-06 15:08:57 -07:00
Simon Michael
479a4138fe journal: better errors and debug output for balance assertions
With --debug=2, better information about assertions is printed.
Balance assertion errors now have a more standard and parseable layout.
The asserted balance is now shown with the diff, let's see if that's better.
2016-07-05 14:12:46 -07:00
Simon Michael
579ab45d0a lib: clarify balance assertion error again
"calculated" is better, "actual" implies the account's actual real-world
balance which might be something else again.
2016-06-07 09:23:54 -07:00
Simon Michael
7f3ae224e2 lib: refactor journal utils 2016-06-01 20:47:27 -07:00
Simon Michael
509f55864d ui: indicate real mode and toggle it with R key (#354)
There is a limitation/bug: disabling real mode in the transaction screen
won't show the non-real postings if it was entered from a real-mode
register screen.
2016-06-01 12:30:39 -07:00
Simon Michael
90c9735b7a lib: textification: descriptions & codes
Slightly higher (with small files) and lower (with large files) maximum
residency, and slightly quicker for all.

hledger -f data/100x100x10.journal stats
<<ghc: 42858472 bytes, 84 GCs, 193712/269608 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.015 elapsed), 0.016 MUT (0.042 elapsed), 0.011 GC (0.119 elapsed) :ghc>>
<<ghc: 42891776 bytes, 84 GCs, 190816/260920 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.004 elapsed), 0.017 MUT (0.025 elapsed), 0.010 GC (0.015 elapsed) :ghc>>

hledger -f data/1000x1000x10.journal stats
<<ghc: 349575240 bytes, 681 GCs, 1396425/4091680 avg/max bytes residency (7 samples), 11M in use, 0.000 INIT (0.000 elapsed), 0.137 MUT (0.146 elapsed), 0.050 GC (0.057 elapsed) :ghc>>
<<ghc: 349927568 bytes, 681 GCs, 1397825/4097248 avg/max bytes residency (7 samples), 11M in use, 0.000 INIT (0.000 elapsed), 0.126 MUT (0.133 elapsed), 0.050 GC (0.057 elapsed) :ghc>>

hledger -f data/10000x1000x10.journal stats
<<ghc: 3424029496 bytes, 6658 GCs, 11403141/41077288 avg/max bytes residency (11 samples), 111M in use, 0.000 INIT (0.000 elapsed), 1.278 MUT (1.310 elapsed), 0.493 GC (0.546 elapsed) :ghc>>
<<ghc: 3427418064 bytes, 6665 GCs, 11127869/37790168 avg/max bytes residency (11 samples), 109M in use, 0.000 INIT (0.001 elapsed), 1.212 MUT (1.229 elapsed), 0.466 GC (0.519 elapsed) :ghc>>

hledger -f data/100000x1000x10.journal stats
<<ghc: 34306546248 bytes, 66727 GCs, 77030638/414617944 avg/max bytes residency (14 samples), 1012M in use, 0.000 INIT (0.000 elapsed), 12.965 MUT (13.164 elapsed), 4.771 GC (5.447 elapsed) :ghc>>
<<ghc: 34340246056 bytes, 66779 GCs, 76983178/416011480 avg/max bytes residency (14 samples), 1011M in use, 0.000 INIT (0.008 elapsed), 12.666 MUT (12.836 elapsed), 4.595 GC (5.175 elapsed) :ghc>>
2016-05-24 19:00:58 -07:00
Simon Michael
770dcee742 lib: textification: comments and tags
No change.

hledger -f data/100x100x10.journal stats
<<ghc: 42859576 bytes, 84 GCs, 193781/269984 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.001 elapsed), 0.016 MUT (0.020 elapsed), 0.009 GC (0.011 elapsed) :ghc>>
<<ghc: 42859576 bytes, 84 GCs, 193781/269984 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.001 elapsed), 0.015 MUT (0.018 elapsed), 0.009 GC (0.013 elapsed) :ghc>>

hledger -f data/1000x1000x10.journal stats
<<ghc: 349576344 bytes, 681 GCs, 1407388/4091680 avg/max bytes residency (7 samples), 11M in use, 0.000 INIT (0.000 elapsed), 0.124 MUT (0.130 elapsed), 0.047 GC (0.055 elapsed) :ghc>>
<<ghc: 349576280 bytes, 681 GCs, 1407388/4091680 avg/max bytes residency (7 samples), 11M in use, 0.000 INIT (0.000 elapsed), 0.126 MUT (0.132 elapsed), 0.049 GC (0.058 elapsed) :ghc>>

hledger -f data/10000x1000x10.journal stats
<<ghc: 3424030664 bytes, 6658 GCs, 11403359/41071624 avg/max bytes residency (11 samples), 111M in use, 0.000 INIT (0.000 elapsed), 1.207 MUT (1.228 elapsed), 0.473 GC (0.528 elapsed) :ghc>>
<<ghc: 3424030760 bytes, 6658 GCs, 11403874/41077288 avg/max bytes residency (11 samples), 111M in use, 0.000 INIT (0.002 elapsed), 1.234 MUT (1.256 elapsed), 0.470 GC (0.520 elapsed) :ghc>>

hledger -f data/100000x1000x10.journal stats
<<ghc: 34306547448 bytes, 66727 GCs, 76805504/414629288 avg/max bytes residency (14 samples), 1009M in use, 0.000 INIT (0.003 elapsed), 12.615 MUT (12.813 elapsed), 4.656 GC (5.291 elapsed) :ghc>>
<<ghc: 34306547320 bytes, 66727 GCs, 76805504/414629288 avg/max bytes residency (14 samples), 1009M in use, 0.000 INIT (0.009 elapsed), 12.802 MUT (13.065 elapsed), 4.774 GC (5.441 elapsed) :ghc>>
2016-05-24 19:00:57 -07:00
Simon Michael
58c3362908 lib: textification: saved journal source
Slightly worse on small files, better on large ones.

hledger -f data/100x100x10.journal stats
<<ghc: 39305392 bytes, 77 GCs, 196354/268584 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.007 elapsed), 0.014 MUT (0.027 elapsed), 0.011 GC (0.111 elapsed) :ghc>>
<<ghc: 39307728 bytes, 77 GCs, 196909/270248 avg/max bytes residency (3 samples), 2M in use, 0.001 INIT (0.010 elapsed), 0.015 MUT (0.028 elapsed), 0.012 GC (0.116 elapsed) :ghc>>

hledger -f data/1000x1000x10.journal stats
<<ghc: 314285912 bytes, 612 GCs, 2064811/6597608 avg/max bytes residency (7 samples), 16M in use, 0.000 INIT (0.002 elapsed), 0.126 MUT (0.134 elapsed), 0.059 GC (0.069 elapsed) :ghc>>
<<ghc: 314271368 bytes, 612 GCs, 2070227/6628024 avg/max bytes residency (7 samples), 16M in use, 0.000 INIT (0.000 elapsed), 0.137 MUT (0.143 elapsed), 0.059 GC (0.068 elapsed) :ghc>>

hledger -f data/10000x1000x10.journal stats
<<ghc: 3070033264 bytes, 5965 GCs, 12699294/62962464 avg/max bytes residency (10 samples), 124M in use, 0.000 INIT (0.000 elapsed), 1.245 MUT (1.300 elapsed), 0.498 GC (0.558 elapsed) :ghc>>
<<ghc: 3070006752 bytes, 5973 GCs, 12687314/62848920 avg/max bytes residency (10 samples), 124M in use, 0.000 INIT (0.003 elapsed), 1.257 MUT (1.281 elapsed), 0.496 GC (0.554 elapsed) :ghc>>

hledger -f data/100000x1000x10.journal stats
<<ghc: 30753465088 bytes, 59763 GCs, 117723618/666643528 avg/max bytes residency (14 samples), 1589M in use, 0.000 INIT (0.002 elapsed), 12.536 MUT (12.793 elapsed), 5.978 GC (7.155 elapsed) :ghc>>
<<ghc: 30753367256 bytes, 59811 GCs, 117723236/666627528 avg/max bytes residency (14 samples), 1590M in use, 0.001 INIT (0.012 elapsed), 12.923 MUT (13.169 elapsed), 5.981 GC (6.860 elapsed) :ghc>>
2016-05-24 19:00:57 -07:00
Simon Michael
2538d14ea7 lib: textification begins! account names
The first of several conversions from String to (strict) Text, hopefully
reducing space and time usage.

This one shows a small improvement, with GHC 7.10.3 and text-1.2.2.1:

hledger -f data/100x100x10.journal stats
string: <<ghc: 39471064 bytes, 77 GCs, 198421/275048 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.001 elapsed), 0.015 MUT (0.020 elapsed), 0.010 GC (0.014 elapsed) :ghc>>
text:   <<ghc: 39268024 bytes, 77 GCs, 197018/270840 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.002 elapsed), 0.016 MUT (0.022 elapsed), 0.009 GC (0.011 elapsed) :ghc>>

hledger -f data/1000x100x10.journal stats
string: <<ghc: 318555920 bytes, 617 GCs, 2178997/7134472 avg/max bytes residency (7 samples), 16M in use, 0.000 INIT (0.001 elapsed), 0.129 MUT (0.136 elapsed), 0.067 GC (0.077 elapsed) :ghc>>
text:   <<ghc: 314248496 bytes, 612 GCs, 2074045/6617960 avg/max bytes residency (7 samples), 16M in use, 0.000 INIT (0.003 elapsed), 0.137 MUT (0.145 elapsed), 0.067 GC (0.079 elapsed) :ghc>>

hledger -f data/10000x100x10.journal stats
string: <<ghc: 3114763608 bytes, 6026 GCs, 18858950/75552024 avg/max bytes residency (11 samples), 201M in use, 0.000 INIT (0.000 elapsed), 1.331 MUT (1.372 elapsed), 0.699 GC (0.812 elapsed) :ghc>>
text:   <<ghc: 3071468920 bytes, 5968 GCs, 14120344/62951360 avg/max bytes residency (9 samples), 124M in use, 0.000 INIT (0.003 elapsed), 1.272 MUT (1.349 elapsed), 0.513 GC (0.578 elapsed) :ghc>>

hledger -f data/100000x100x10.journal stats
string: <<ghc: 31186579432 bytes, 60278 GCs, 135332581/740228992 avg/max bytes residency (13 samples), 1697M in use, 0.000 INIT (0.008 elapsed), 14.677 MUT (15.508 elapsed), 7.081 GC (8.074 elapsed) :ghc>>
text:   <<ghc: 30753427672 bytes, 59763 GCs, 117595958/666457240 avg/max bytes residency (14 samples), 1588M in use, 0.000 INIT (0.008 elapsed), 13.713 MUT (13.966 elapsed), 6.220 GC (7.108 elapsed) :ghc>>
2016-05-24 19:00:49 -07:00
Simon Michael
0f5ee154c4 lib: simplify parsers; cleanups (#275)
The journal/timeclock/timedot parsers, instead of constructing (opaque)
journal update functions which are later applied to build the journal,
now construct the journal directly (by modifying the parser state). This
is easier to understand and debug. It also removes any possibility of
the journal updates being a space leak. (They weren't, in fact memory
usage is now slightly higher, but that will be addressed in other ways.)

Also:

Journal data and journal parse info have been merged into one type (for
now), and field names are more consistent.

The ParsedJournal type alias has been added to distinguish being-parsed
and finalised journals.

Journal is now a monoid.

stats: fixed an issue with ordering of include files

journal: fixed an issue with ordering of included same-date transactions

timeclock: sessions can no longer span file boundaries (unclocked-out
sessions will be auto-closed at the end of the file).

expandPath now throws a proper IO error (and requires the IO monad).
2016-05-23 00:44:19 -07:00
Simon Michael
7f5e09096f lib: rename JournalContext to JournalParseState 2016-05-18 20:57:34 -07:00
Simon Michael
fdd7feefdb lib: make Journal a monoid 2016-05-18 15:08:50 -07:00
Simon Michael
9304aa1100 journal: commodity format directive overrides inferred style (#295)
The commodity directive's format subdirective can now be used to
override the inferred style for a commodity, eg to increase or decrease
the precision. This doesn't fix the root cause of #295 but is at least a
good workaround.
2016-05-08 09:07:08 -07:00
Simon Michael
2c0ef877eb journal: parse and store commodity formats 2016-05-08 09:06:54 -07:00
Simon Michael
207922a023 lib: Commodity now has a CommoditySymbol and style 2016-05-08 08:07:27 -07:00
Simon Michael
525ca6041d lib: tweak balance assertion message 2016-05-04 09:08:53 -07:00
Simon Michael
588f36d662 rename timelog format to timeclock 2016-04-12 21:13:19 -07:00
Simon Michael
31a754544a journal: parse account directives
We now parse account directives, like Ledger's. We don't do anything
with them yet. The default parent account feature must now be spelled
"apply account"/"end apply account".
2016-04-04 11:35:39 -07:00
Simon Michael
931a1d5b9e hlint 2016-02-21 02:37:59 -08:00
Simon Michael
c9937db10e lib: show txn's file position in assertion errors 2016-02-19 22:48:59 -08:00
Simon Michael
5da355c06f lib: more ergonomic balance assertion errors 2016-02-10 07:40:18 -08:00
Simon Michael
96e1ca7ea1 lib: refactor amount canonicalisation
Amount display style canonicalisation code and terminology has been
clarified a bit. Individual amounts still have styles; from these we
derive the standard "commodity styles". In user docs, we might call
these "commodity formats" since a Ledger-compatible commodity directive
would use the "format" keyword.
2015-11-24 01:40:10 -08:00
Simon Michael
2feace32dd lib: number transactions as they are read
And allow looking up transactions by their read order (index),
or the previous/next transactions in the sequence.
2015-10-29 20:12:46 -07:00
Simon Michael
71921135f6 include P amounts in canonicalisation (fixes #131)
Since market price amounts didn't contribute to the canonical commodity
styles, they were being reset to the null style. And this propagated to
the reported amounts when -V was in effect, causing much confusion.
Now, market prices contribute to canonicalisation and the expected
styles are preserved even with -V.

cf https://github.com/simonmichael/hledger/issues/131#issuecomment-133545140
2015-10-11 16:07:31 -07:00
Simon Michael
94094252be rename historical prices to market prices
Simpler and clearer. We now have "transaction prices" (recorded as part
of transaction amounts) and "market prices" (recorded with P
directives). Both are matters of historical record, also this avoids
confusion with the balance command's "historical balances".
2015-08-09 16:20:02 -07:00
Simon Michael
040d00e8fb also canonicalise historical price amounts
So that when we convert amounts to market value, the result will have
the canonical style of the target commodity.
2015-08-09 15:12:16 -07:00
Simon Michael
7a050d65c8 bs/is/cf: recognise "debt..." as a synonym for "liabilities..." 2015-07-12 12:32:53 -07:00
Simon Michael
61e4034de5 Journal's Show instance reported one too many accounts 2015-06-28 14:14:56 -07:00
Simon Michael
d1f63334ee handle pending status correctly, add --pending (#250)
A transaction/posting status of ! (pending) was effectively equivalent
to * (cleared). Now it's a separate state, not matched by --cleared.
The new Ledger-compatible --pending flag matches it, and so does
--uncleared. The equivalent search queries are now status:*, status:!
and status: (the old status:1 and status:0 spellings are deprecated).

Since we interpret --uncleared and status: as "any state except cleared",
it's not currently possible to match things which are neither cleared
nor pending.
2015-05-16 11:51:35 -07:00
Simon Michael
077e3c6a02 journal: re-add non-regex aliases, as default (#252)
The regex account aliases added in 0.24 trip up people switching between
hledger and Ledger. (Also they are currently slow).

This change makes the old non-regex aliases the default; they are
unsurprising, useful, and pretty close in functionality to Ledger's.

The new regex aliases are also available; they must be enclosed in
forward slashes. Ledger effectively ignores these, which is ok.

Also clarify docs, refactor, and use the same parser for alias
directives and alias options
2015-05-14 13:01:50 -07:00
Simon Michael
70d87613f2 some cleanup of debug trace helpers 2015-05-14 13:01:49 -07:00
Simon Michael
257ce57383 fix balance accumulation across assertions (#195)
A sequence of balance assertions asserting first one commodity, then
another, then the first again, was not working.
2015-01-10 22:17:19 -08:00
Julien Moutinho
c65fea2b4b lib: fix combineJournalUpdates folding order
NOTE: this is important to correctly build JournalContext
NOTE: currently a list reverse must done at the end,
      maybe using a Data.Queue would be more efficient.
2014-12-11 04:37:10 +01:00
Simon Michael
e892fdc6d5 parsing: aliases now match by regular expression
alias match patterns (the part left of the =) are now case-insensitive
regular expressions matching anywhere in the account name. The
replacement string (the part right of the =) can replace multiple
matches within the account name. The replacement string does not yet
support any of the usual syntax like backreferences.
2014-10-24 15:05:10 -07:00
gwern
d1618aaca8 strip trailing whitespace from all Haskell files 2014-10-14 18:48:24 -07:00
Julien Moutinho
a6190420b2 data: add source location to transactions 2014-08-07 16:38:44 +02:00
Simon Michael
b3da48aaa0 identify the commodity in balance assertion errors
Now that balance assertions are checking only a single commodity, it can
be confusing. Eg say all your amounts are in dollars, an assertion like
"= 0" checked the dollar balance in hledger 0.23 but always succeeds in
hledger 0.24. When an assertion fails, we now report which commodity was
checked to help troubleshooting.
2014-07-17 15:23:03 -07:00
Simon Michael
19d42f8c99 clarify failed balance assertion message 2014-07-15 06:55:46 -07:00
Simon Michael
c31710d942 look harder for decimal point & digit groups (fixes #196)
Amount display styles have been reworked a bit; they are now calculated
after journal parsing, not during it. This allows the fix for #196:
we now search through the amounts until a decimal point is detected,
instead of just looking at the first one; likewise for digit groups.
Digit groups are now implemented with a better type.

Digit group size detection has been improved a little:
1000,000 now gives group sizes [3,4,4,...], not [3,3,...], and
10,000 gives groups sizes [3,3,...] not [3,2,2,..].
(To get [3,2,2,...] you'd use eg 00,00,000.)

There are still some old (or new ?) issues; I don't think we handle
inconsistent decimal points & digit groups too well. But for now all
tests pass.
2014-07-02 23:26:16 -07:00
Simon Michael
330dfe6891 commodity & amount style parser cleanups, part 2 2014-07-02 17:41:33 -07:00