slaesvuo/hledger
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9,735 Commits 1 Branch 0 Tags 88 MiB
c827cbcc3e
Commit Graph

645 Commits

Author SHA1 Message Date
Moritz Kiefer
4141067428 Replace Parsec with Megaparsec (see #289) (#366) 
* Replace Parsec with Megaparsec (see #289)

This builds upon PR #289 by @rasendubi

* Revert renaming of parseWithState to parseWithCtx

* Fix doctests

* Update for Megaparsec 5

* Specialize parser to improve performance

* Pretty print errors

* Swap StateT and ParsecT

This is necessary to get the correct backtracking behavior, i.e. discard
state changes if the parsing fails.
 2016-07-29 08:57:10 -07:00
Simon Michael
f3bf98bfae lib: parentheses trying to resolve IDE warning 2016-05-26 15:51:59 -07:00
Simon Michael
90c9735b7a lib: textification: descriptions & codes 
Slightly higher (with small files) and lower (with large files) maximum
residency, and slightly quicker for all.

hledger -f data/100x100x10.journal stats
<<ghc: 42858472 bytes, 84 GCs, 193712/269608 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.015 elapsed), 0.016 MUT (0.042 elapsed), 0.011 GC (0.119 elapsed) :ghc>>
<<ghc: 42891776 bytes, 84 GCs, 190816/260920 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.004 elapsed), 0.017 MUT (0.025 elapsed), 0.010 GC (0.015 elapsed) :ghc>>

hledger -f data/1000x1000x10.journal stats
<<ghc: 349575240 bytes, 681 GCs, 1396425/4091680 avg/max bytes residency (7 samples), 11M in use, 0.000 INIT (0.000 elapsed), 0.137 MUT (0.146 elapsed), 0.050 GC (0.057 elapsed) :ghc>>
<<ghc: 349927568 bytes, 681 GCs, 1397825/4097248 avg/max bytes residency (7 samples), 11M in use, 0.000 INIT (0.000 elapsed), 0.126 MUT (0.133 elapsed), 0.050 GC (0.057 elapsed) :ghc>>

hledger -f data/10000x1000x10.journal stats
<<ghc: 3424029496 bytes, 6658 GCs, 11403141/41077288 avg/max bytes residency (11 samples), 111M in use, 0.000 INIT (0.000 elapsed), 1.278 MUT (1.310 elapsed), 0.493 GC (0.546 elapsed) :ghc>>
<<ghc: 3427418064 bytes, 6665 GCs, 11127869/37790168 avg/max bytes residency (11 samples), 109M in use, 0.000 INIT (0.001 elapsed), 1.212 MUT (1.229 elapsed), 0.466 GC (0.519 elapsed) :ghc>>

hledger -f data/100000x1000x10.journal stats
<<ghc: 34306546248 bytes, 66727 GCs, 77030638/414617944 avg/max bytes residency (14 samples), 1012M in use, 0.000 INIT (0.000 elapsed), 12.965 MUT (13.164 elapsed), 4.771 GC (5.447 elapsed) :ghc>>
<<ghc: 34340246056 bytes, 66779 GCs, 76983178/416011480 avg/max bytes residency (14 samples), 1011M in use, 0.000 INIT (0.008 elapsed), 12.666 MUT (12.836 elapsed), 4.595 GC (5.175 elapsed) :ghc>>
 2016-05-24 19:00:58 -07:00
Simon Michael
a1b68009da lib: fix tests; always try parsing stdin as journal 2016-05-24 19:00:58 -07:00
Simon Michael
770dcee742 lib: textification: comments and tags 
No change.

hledger -f data/100x100x10.journal stats
<<ghc: 42859576 bytes, 84 GCs, 193781/269984 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.001 elapsed), 0.016 MUT (0.020 elapsed), 0.009 GC (0.011 elapsed) :ghc>>
<<ghc: 42859576 bytes, 84 GCs, 193781/269984 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.001 elapsed), 0.015 MUT (0.018 elapsed), 0.009 GC (0.013 elapsed) :ghc>>

hledger -f data/1000x1000x10.journal stats
<<ghc: 349576344 bytes, 681 GCs, 1407388/4091680 avg/max bytes residency (7 samples), 11M in use, 0.000 INIT (0.000 elapsed), 0.124 MUT (0.130 elapsed), 0.047 GC (0.055 elapsed) :ghc>>
<<ghc: 349576280 bytes, 681 GCs, 1407388/4091680 avg/max bytes residency (7 samples), 11M in use, 0.000 INIT (0.000 elapsed), 0.126 MUT (0.132 elapsed), 0.049 GC (0.058 elapsed) :ghc>>

hledger -f data/10000x1000x10.journal stats
<<ghc: 3424030664 bytes, 6658 GCs, 11403359/41071624 avg/max bytes residency (11 samples), 111M in use, 0.000 INIT (0.000 elapsed), 1.207 MUT (1.228 elapsed), 0.473 GC (0.528 elapsed) :ghc>>
<<ghc: 3424030760 bytes, 6658 GCs, 11403874/41077288 avg/max bytes residency (11 samples), 111M in use, 0.000 INIT (0.002 elapsed), 1.234 MUT (1.256 elapsed), 0.470 GC (0.520 elapsed) :ghc>>

hledger -f data/100000x1000x10.journal stats
<<ghc: 34306547448 bytes, 66727 GCs, 76805504/414629288 avg/max bytes residency (14 samples), 1009M in use, 0.000 INIT (0.003 elapsed), 12.615 MUT (12.813 elapsed), 4.656 GC (5.291 elapsed) :ghc>>
<<ghc: 34306547320 bytes, 66727 GCs, 76805504/414629288 avg/max bytes residency (14 samples), 1009M in use, 0.000 INIT (0.009 elapsed), 12.802 MUT (13.065 elapsed), 4.774 GC (5.441 elapsed) :ghc>>
 2016-05-24 19:00:57 -07:00
Simon Michael
c89c33b36e lib: textification: parse stream 
10% more allocation, but 35% lower maximum residency, and slightly quicker.

hledger -f data/100x100x10.journal stats
<<ghc: 39327768 bytes, 77 GCs, 196834/269496 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.010 elapsed), 0.020 MUT (0.092 elapsed), 0.014 GC (0.119 elapsed) :ghc>>
<<ghc: 42842136 bytes, 84 GCs, 194010/270912 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.009 elapsed), 0.016 MUT (0.029 elapsed), 0.012 GC (0.120 elapsed) :ghc>>

hledger -f data/1000x1000x10.journal stats
<<ghc: 314291440 bytes, 612 GCs, 2070776/6628048 avg/max bytes residency (7 samples), 16M in use, 0.000 INIT (0.000 elapsed), 0.128 MUT (0.144 elapsed), 0.059 GC (0.070 elapsed) :ghc>>
<<ghc: 349558872 bytes, 681 GCs, 1397597/4106384 avg/max bytes residency (7 samples), 11M in use, 0.000 INIT (0.004 elapsed), 0.124 MUT (0.133 elapsed), 0.047 GC (0.053 elapsed) :ghc>>

hledger -f data/10000x1000x10.journal stats
<<ghc: 3070026824 bytes, 5973 GCs, 12698030/62951784 avg/max bytes residency (10 samples), 124M in use, 0.000 INIT (0.002 elapsed), 1.268 MUT (1.354 elapsed), 0.514 GC (0.587 elapsed) :ghc>>
<<ghc: 3424013128 bytes, 6658 GCs, 11405501/41071624 avg/max bytes residency (11 samples), 111M in use, 0.000 INIT (0.001 elapsed), 1.343 MUT (1.406 elapsed), 0.511 GC (0.573 elapsed) :ghc>>

hledger -f data/100000x1000x10.journal stats
<<ghc: 30753387392 bytes, 59811 GCs, 117615462/666703600 avg/max bytes residency (14 samples), 1588M in use, 0.000 INIT (0.000 elapsed), 12.068 MUT (12.238 elapsed), 6.015 GC (7.190 elapsed) :ghc>>
<<ghc: 34306530696 bytes, 66727 GCs, 76806196/414629312 avg/max bytes residency (14 samples), 1009M in use, 0.000 INIT (0.010 elapsed), 14.357 MUT (16.370 elapsed), 5.298 GC (6.534 elapsed) :ghc>>
 2016-05-24 19:00:57 -07:00
Simon Michael
58c3362908 lib: textification: saved journal source 
Slightly worse on small files, better on large ones.

hledger -f data/100x100x10.journal stats
<<ghc: 39305392 bytes, 77 GCs, 196354/268584 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.007 elapsed), 0.014 MUT (0.027 elapsed), 0.011 GC (0.111 elapsed) :ghc>>
<<ghc: 39307728 bytes, 77 GCs, 196909/270248 avg/max bytes residency (3 samples), 2M in use, 0.001 INIT (0.010 elapsed), 0.015 MUT (0.028 elapsed), 0.012 GC (0.116 elapsed) :ghc>>

hledger -f data/1000x1000x10.journal stats
<<ghc: 314285912 bytes, 612 GCs, 2064811/6597608 avg/max bytes residency (7 samples), 16M in use, 0.000 INIT (0.002 elapsed), 0.126 MUT (0.134 elapsed), 0.059 GC (0.069 elapsed) :ghc>>
<<ghc: 314271368 bytes, 612 GCs, 2070227/6628024 avg/max bytes residency (7 samples), 16M in use, 0.000 INIT (0.000 elapsed), 0.137 MUT (0.143 elapsed), 0.059 GC (0.068 elapsed) :ghc>>

hledger -f data/10000x1000x10.journal stats
<<ghc: 3070033264 bytes, 5965 GCs, 12699294/62962464 avg/max bytes residency (10 samples), 124M in use, 0.000 INIT (0.000 elapsed), 1.245 MUT (1.300 elapsed), 0.498 GC (0.558 elapsed) :ghc>>
<<ghc: 3070006752 bytes, 5973 GCs, 12687314/62848920 avg/max bytes residency (10 samples), 124M in use, 0.000 INIT (0.003 elapsed), 1.257 MUT (1.281 elapsed), 0.496 GC (0.554 elapsed) :ghc>>

hledger -f data/100000x1000x10.journal stats
<<ghc: 30753465088 bytes, 59763 GCs, 117723618/666643528 avg/max bytes residency (14 samples), 1589M in use, 0.000 INIT (0.002 elapsed), 12.536 MUT (12.793 elapsed), 5.978 GC (7.155 elapsed) :ghc>>
<<ghc: 30753367256 bytes, 59811 GCs, 117723236/666627528 avg/max bytes residency (14 samples), 1590M in use, 0.001 INIT (0.012 elapsed), 12.923 MUT (13.169 elapsed), 5.981 GC (6.860 elapsed) :ghc>>
 2016-05-24 19:00:57 -07:00
Simon Michael
102b76c17f lib: textification: commodity symbols 
hledger -f data/100x100x10.journal stats
<<ghc: 39288536 bytes, 77 GCs, 196608/269560 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.001 elapsed), 0.016 MUT (0.028 elapsed), 0.009 GC (0.012 elapsed) :ghc>>
<<ghc: 39290808 bytes, 77 GCs, 196608/269560 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.003 elapsed), 0.015 MUT (0.021 elapsed), 0.009 GC (0.011 elapsed) :ghc>>

hledger -f data/1000x100x10.journal stats
<<ghc: 314268960 bytes, 612 GCs, 2143219/6826152 avg/max bytes residency (7 samples), 16M in use, 0.000 INIT (0.009 elapsed), 0.135 MUT (0.151 elapsed), 0.065 GC (0.178 elapsed) :ghc>>
<<ghc: 314254512 bytes, 612 GCs, 2072377/6628024 avg/max bytes residency (7 samples), 16M in use, 0.000 INIT (0.000 elapsed), 0.130 MUT (0.134 elapsed), 0.064 GC (0.075 elapsed) :ghc>>

hledger -f data/10000x100x10.journal stats
<<ghc: 3070016592 bytes, 5965 GCs, 13138220/64266016 avg/max bytes residency (10 samples), 128M in use, 0.000 INIT (0.000 elapsed), 1.272 MUT (1.322 elapsed), 0.527 GC (0.595 elapsed) :ghc>>
<<ghc: 3069989896 bytes, 5973 GCs, 12687877/62848920 avg/max bytes residency (10 samples), 124M in use, 0.000 INIT (0.002 elapsed), 1.295 MUT (1.324 elapsed), 0.511 GC (0.570 elapsed) :ghc>>

hledger -f data/100000x100x10.journal stats
<<ghc: 30753448072 bytes, 59763 GCs, 121502982/673169248 avg/max bytes residency (14 samples), 1640M in use, 0.000 INIT (0.007 elapsed), 12.421 MUT (12.672 elapsed), 6.240 GC (7.812 elapsed) :ghc>>
<<ghc: 30753350528 bytes, 59811 GCs, 117616668/666703600 avg/max bytes residency (14 samples), 1588M in use, 0.001 INIT (0.011 elapsed), 13.209 MUT (13.683 elapsed), 6.137 GC (7.117 elapsed) :ghc>>
 2016-05-24 19:00:57 -07:00
Simon Michael
2538d14ea7 lib: textification begins! account names 
The first of several conversions from String to (strict) Text, hopefully
reducing space and time usage.

This one shows a small improvement, with GHC 7.10.3 and text-1.2.2.1:

hledger -f data/100x100x10.journal stats
string: <<ghc: 39471064 bytes, 77 GCs, 198421/275048 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.001 elapsed), 0.015 MUT (0.020 elapsed), 0.010 GC (0.014 elapsed) :ghc>>
text:   <<ghc: 39268024 bytes, 77 GCs, 197018/270840 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.002 elapsed), 0.016 MUT (0.022 elapsed), 0.009 GC (0.011 elapsed) :ghc>>

hledger -f data/1000x100x10.journal stats
string: <<ghc: 318555920 bytes, 617 GCs, 2178997/7134472 avg/max bytes residency (7 samples), 16M in use, 0.000 INIT (0.001 elapsed), 0.129 MUT (0.136 elapsed), 0.067 GC (0.077 elapsed) :ghc>>
text:   <<ghc: 314248496 bytes, 612 GCs, 2074045/6617960 avg/max bytes residency (7 samples), 16M in use, 0.000 INIT (0.003 elapsed), 0.137 MUT (0.145 elapsed), 0.067 GC (0.079 elapsed) :ghc>>

hledger -f data/10000x100x10.journal stats
string: <<ghc: 3114763608 bytes, 6026 GCs, 18858950/75552024 avg/max bytes residency (11 samples), 201M in use, 0.000 INIT (0.000 elapsed), 1.331 MUT (1.372 elapsed), 0.699 GC (0.812 elapsed) :ghc>>
text:   <<ghc: 3071468920 bytes, 5968 GCs, 14120344/62951360 avg/max bytes residency (9 samples), 124M in use, 0.000 INIT (0.003 elapsed), 1.272 MUT (1.349 elapsed), 0.513 GC (0.578 elapsed) :ghc>>

hledger -f data/100000x100x10.journal stats
string: <<ghc: 31186579432 bytes, 60278 GCs, 135332581/740228992 avg/max bytes residency (13 samples), 1697M in use, 0.000 INIT (0.008 elapsed), 14.677 MUT (15.508 elapsed), 7.081 GC (8.074 elapsed) :ghc>>
text:   <<ghc: 30753427672 bytes, 59763 GCs, 117595958/666457240 avg/max bytes residency (14 samples), 1588M in use, 0.000 INIT (0.008 elapsed), 13.713 MUT (13.966 elapsed), 6.220 GC (7.108 elapsed) :ghc>>
 2016-05-24 19:00:49 -07:00
Simon Michael
097c9e09b6 lib: cleanups 2016-05-23 12:51:15 -07:00
Simon Michael
0f5ee154c4 lib: simplify parsers; cleanups (#275) 
The journal/timeclock/timedot parsers, instead of constructing (opaque)
journal update functions which are later applied to build the journal,
now construct the journal directly (by modifying the parser state). This
is easier to understand and debug. It also removes any possibility of
the journal updates being a space leak. (They weren't, in fact memory
usage is now slightly higher, but that will be addressed in other ways.)

Also:

Journal data and journal parse info have been merged into one type (for
now), and field names are more consistent.

The ParsedJournal type alias has been added to distinguish being-parsed
and finalised journals.

Journal is now a monoid.

stats: fixed an issue with ordering of include files

journal: fixed an issue with ordering of included same-date transactions

timeclock: sessions can no longer span file boundaries (unclocked-out
sessions will be auto-closed at the end of the file).

expandPath now throws a proper IO error (and requires the IO monad).
 2016-05-23 00:44:19 -07:00
Simon Michael
7f5e09096f lib: rename JournalContext to JournalParseState 2016-05-18 20:57:34 -07:00
Simon Michael
ae490d738f lib: Hledger.Read notes 2016-05-18 16:42:25 -07:00
Simon Michael
7e84b4d643 lib: rename get/setIndex, add increment fn 2016-05-18 15:06:16 -07:00
Simon Michael
84097b75c7 journal: can now include timeclock/timedot files (#320) 
journal files can now include journal, timeclock or timedot files (but
not yet CSV files). Also timeclock/timedot files no longer support
default year directives.

The Hledger.Read.* modules have been reorganised for better reuse.
Hledger.Read.Utils has been renamed Hledger.Read.Common and holds
low-level parsers & utilities; high-level read utilities have moved to
Hledger.Read.
 2016-05-17 19:46:54 -07:00
Simon Michael
ec507955e1 journal: simplify parse error 2016-05-17 10:06:51 -07:00
Simon Michael
9c130e1850 lib: journal: hlint cleanups! 
Hopefully still backward compatible.
 2016-05-13 22:09:39 -07:00
Simon Michael
16ee07cc52 remove temp type 2016-05-13 20:44:21 -07:00
Simon Michael
ddefec412f journal: also allow one-line commodity directives 2016-05-08 20:56:34 -07:00
Simon Michael
b583cb327d lib: fix a warning 2016-05-08 20:55:59 -07:00
Simon Michael
2c0ef877eb journal: parse and store commodity formats 2016-05-08 09:06:54 -07:00
Simon Michael
207922a023 lib: Commodity now has a CommoditySymbol and style 2016-05-08 08:07:27 -07:00
Simon Michael
e7a1eb627f journal: export parsing helpers, fix warning 2016-05-06 16:58:07 -07:00
Simon Michael
9a424d77fc lib: more thorough journal reloading 
Reloading a journal should now reload all included files as well.
 2016-05-02 20:07:53 -07:00
Simon Michael
856c0b3042 lib: fix bracketed posting dates, parser cleanup (#304) 
Bracketed posting dates were fragile; they worked only if you wrote full
10-character dates. Also some semantics were a bit unclear. Now they
should be robust, and have been documented more clearly. This is a
legacy undocumented Ledger syntax, but it improves compatibility and
might be preferable to the more verbose "date:" tags if you write
posting dates often (as I do).

Internally, bracketed posting dates are no longer considered to be tags.
Journal comment, tag, and posting date parsers have been reworked, all
with doctests. Also the journal parser types generally have been
tightened up and clarified, making it much easier to know how to combine
and run them. There's now

-- | A parser of strings with generic user state, monad and return type.
type StringParser u m a = ParsecT String u m a

-- | A string parser with journal-parsing state.
type JournalParser m a = StringParser JournalContext m a

-- | A journal parser that runs in IO and can throw an error mid-parse.
type ErroringJournalParser a = JournalParser (ExceptT String IO) a

and corresponding convenience functions (and short aliases) for running them.
 2016-04-28 13:34:57 -07:00
Simon Michael
259e7bfbe3 lib: add a doctest suite 2016-04-28 13:34:57 -07:00
Simon Michael
9946e7df88 lib: organise JournalReader a bit 2016-04-28 13:34:57 -07:00
Simon Michael
df4b82740b lib: make the supported format names accessible 2016-04-18 16:43:57 -07:00
Simon Michael
588f36d662 rename timelog format to timeclock 2016-04-12 21:13:19 -07:00
Simon Michael
31a754544a journal: parse account directives 
We now parse account directives, like Ledger's. We don't do anything
with them yet. The default parent account feature must now be spelled
"apply account"/"end apply account".
 2016-04-04 11:35:39 -07:00
Simon Michael
a9afd7bcbe lib: slightly better journal/time format detection 
The Journal, Timelog and Timedot readers' detectors now check
each line in the sample data, not just the first one. I think
the sample data is only about 30 chars right now, but even so
this fixed a format detection issue I was seeing.
 2016-02-19 23:02:10 -08:00
Simon Michael
70863ae40b lib: timedot allow indenting 2016-02-19 22:58:08 -08:00
Simon Michael
4b4a4bacf7 lib: timedot parse order fix 2016-02-19 22:57:43 -08:00
Simon Michael
0adcdf21f8 lib: timedot parsing fix 2016-02-19 22:57:06 -08:00
Simon Michael
b26dd3d9b0 lib: fix timedot comments 2016-02-19 22:55:30 -08:00
Simon Michael
06b54bf05e lib: timedot format, convenient for time logging 
Timedot is a plain text format for logging dated, categorised
quantities (eg time), supported by hledger.  It is convenient for
approximate and retroactive time logging, eg when the real-time
clock-in/out required with a timeclock file is too precise or too
interruptive.  It can be formatted like a bar chart, making clear at a
glance where time was spent.
 2016-02-19 17:55:57 -08:00
Simon Michael
a2b989d898 journal: support ledger3-compatible "apply account" 2016-01-29 07:39:26 -08:00
Simon Michael
0ee2ff7ab2 lib: fix off-by-one in transaction index 2015-10-30 10:41:42 -07:00
Simon Michael
2feace32dd lib: number transactions as they are read 
And allow looking up transactions by their read order (index),
or the previous/next transactions in the sequence.
 2015-10-29 20:12:46 -07:00
Simon Michael
d24b1b96f7 lib: parser cleanups 2015-10-17 12:09:03 -07:00
Simon Michael
bc43036117 lib: use consistent p suffix for parsers 2015-10-17 11:51:45 -07:00
Simon Michael
8214a6ccef dangit.. simplify combineJournalUpdates for real 2015-10-13 11:54:15 -07:00
Simon Michael
3be88ad6b5 simplify combineJournalUpdates 
I really don't see why that extra x parameter is needed or works..
rewrite it in simpler form.

I also might be introducing breakage for older GHC's by using
unconditionally <$>, but I'm not seeing that for some reason
(tested back to ghc 7.6).
 2015-10-13 11:49:51 -07:00
Simon Michael
939f7184c9 journal: make "end comment" optional, like ledger 
The "comment" directive longer requires an "end comment", and will
extend to the end of the file(s) without it.
 2015-10-07 16:41:49 -07:00
Simon Michael
a2f344f6d9 journal: require dates to begin with a digit 2015-10-04 08:41:01 -10:00
Simon Michael
dafa8efada note possible combineJournalUpdates inefficiency 2015-09-26 16:01:27 -10:00
Simon Michael
5048d3bf06 lib: memoise accountNameApplyAliases too ? 
This adds a accountNameApplyAliasesMemo, which memoises the result of
applying a set of aliases (simple and regex) to an account name. In
theory this should reduce more repetitive work, but in practice it
doesn't seem to make a difference, so it's unused for now.
 2015-09-26 15:58:12 -10:00
Simon Michael
591abefe9e clarify modifiedaccountnamep 2015-09-24 14:23:52 -10:00
Simon Michael
2b339667e2 Merge branch 'perf-polyparse' (early part) 2015-08-13 13:10:10 -07:00
Simon Michael
42d452f99c abstract parsec's SourcePos so as to derive NFData 
The NFData instance helps us time things with criterion.
 2015-08-13 12:56:15 -07:00
Simon Michael
94094252be rename historical prices to market prices 
Simpler and clearer. We now have "transaction prices" (recorded as part
of transaction amounts) and "market prices" (recorded with P
directives). Both are matters of historical record, also this avoids
confusion with the balance command's "historical balances".
 2015-08-09 16:20:02 -07:00
Simon Michael
ba6269981b begin reviving journalreader unit tests 2015-06-11 10:16:46 -07:00
Simon Michael
d1769d9d62 journal: allow journal entries with no postings 
We now parse, and also print, posting-less journal entries, as I
proposed on the lists.

These are not real General Journal entries/transactions, but here is my
rationale:

- Ledger and beancount parse them
- if we parse them, we should print them
- they provide a natural way to record and report non-transaction events
- most of all, they permit more gradual introduction and learning of the concepts.
  Eg a beginner can keep a simple journal even before learning about accounts and postings.
 2015-06-11 10:16:46 -07:00
Simon Michael
d1f63334ee handle pending status correctly, add --pending (#250) 
A transaction/posting status of ! (pending) was effectively equivalent
to * (cleared). Now it's a separate state, not matched by --cleared.
The new Ledger-compatible --pending flag matches it, and so does
--uncleared. The equivalent search queries are now status:*, status:!
and status: (the old status:1 and status:0 spellings are deprecated).

Since we interpret --uncleared and status: as "any state except cleared",
it's not currently possible to match things which are neither cleared
nor pending.
 2015-05-16 11:51:35 -07:00
Simon Michael
077e3c6a02 journal: re-add non-regex aliases, as default (#252) 
The regex account aliases added in 0.24 trip up people switching between
hledger and Ledger. (Also they are currently slow).

This change makes the old non-regex aliases the default; they are
unsurprising, useful, and pretty close in functionality to Ledger's.

The new regex aliases are also available; they must be enclosed in
forward slashes. Ledger effectively ignores these, which is ok.

Also clarify docs, refactor, and use the same parser for alias
directives and alias options
 2015-05-14 13:01:50 -07:00
Simon Michael
70d87613f2 some cleanup of debug trace helpers 2015-05-14 13:01:49 -07:00
Simon Michael
7c3379db63 journal: allow whitespace following a comment directive 2015-05-09 13:25:31 -07:00
Simon Michael
52ce7c1d93 make recent timelog commit ghc 7.8 compatible 2015-04-29 07:10:13 -07:00
Simon Michael
bb71b077a2 fix runExceptT breakage from recent PR (#248) 2015-04-29 07:08:33 -07:00
Simon Michael
5102eca9c3 timelog: support the description field (fix #247) 2015-04-28 13:54:36 -07:00
Simon Michael
fb37e99bc8 lib: simple dates must start with non-space; docs 2015-04-28 13:50:58 -07:00
Simon Hengel
964a410b24 hledger-lib: Update for base-compat-0.8.0 (see #245) 2015-04-23 15:41:59 +08:00
Simon Michael
f8a24ccead fix parseTime warnings with time 1.5+ (#239) 2015-03-29 16:12:54 -07:00
Simon Michael
f75849cdd6 fix ghc 7.10 Applicative import warnings (#239) 
Still needed CPP, despite using base-compat.
 2015-03-29 16:09:41 -07:00
Simon Michael
8e50395b7c ErrorT -> ExceptT, handle mtl <2.2.1 && >=2.2.1 (#239) 2015-03-29 14:16:42 -07:00
Simon Michael
e60eb71467 adapt to GHC-7.10's time-1.5 (#239) 2015-03-27 15:42:32 -07:00
Julien Moutinho
af56ced3b0 lib: add eof parsing checks 2015-01-11 09:45:55 -08:00
Simon Michael
e794412a8f fix JournalReader ctx warning 2014-12-27 15:24:02 -08:00
Simon Michael
9c68944c79 journal, csv: comment lines can also start with * 
As in Ledger. This means you can embed emacs org/outline-mode nodes in
your journal file and manipulate it like an outline.
 2014-12-27 14:41:28 -08:00
Julien Moutinho
c65fea2b4b lib: fix combineJournalUpdates folding order 
NOTE: this is important to correctly build JournalContext
NOTE: currently a list reverse must done at the end,
      maybe using a Data.Queue would be more efficient.
 2014-12-11 04:37:10 +01:00
Simon Michael
1708f0b441 csv: try to preserve order of same-day transactions 
If the CSV records appear to have been in reverse date order,
we'll now reverse them all before also sorting by transaction date,
so that the original order of same-day transactions is preserved.
We detect this using a simple heuristic: if the first converted
transaction's date is later than the last's.
 2014-12-02 11:16:51 -08:00
Simon Michael
733a7b12ef csv: include path is relative to current (close #198) 2014-12-02 10:50:31 -08:00
Julien Moutinho
cf28985cf2 lib: move from Text.ParserCombinators.Parsec to Text.Parsec 
NOTE: required to use liftIO in includedirective
SEE: http://www.vex.net/~trebla/haskell/parsec-generally.xhtml#IO
 2014-11-20 10:08:30 +01:00
Gergely Risko
6e4a7dff8c Add support for ledger3 style multi-line comments 2014-10-26 19:20:46 +01:00
Simon Michael
e892fdc6d5 parsing: aliases now match by regular expression 
alias match patterns (the part left of the =) are now case-insensitive
regular expressions matching anywhere in the account name. The
replacement string (the part right of the =) can replace multiple
matches within the account name. The replacement string does not yet
support any of the usual syntax like backreferences.
 2014-10-24 15:05:10 -07:00
Simon Michael
bfedf367c4 export Regexp types, disambiguate CsvReader's 2014-10-24 14:30:49 -07:00
gwern
d1618aaca8 strip trailing whitespace from all Haskell files 2014-10-14 18:48:24 -07:00
Simon Michael
22279978af revert an unnecessary "fix" in efe7c7a that broke tests 2014-09-13 21:28:48 -07:00
Simon Michael
efe7c7a029 journal: dates should be followed by whitespace or newline 2014-09-06 09:20:22 -07:00
Simon Michael
c485017ca6 journal: require consistent separators in a Y/M/D date 
Date separators can be /, - or . but only one kind should be used in
each date. Eg we now reject 2014-8/1.
 2014-08-08 07:29:02 -07:00
Simon Michael
3e27f4fb8b journal: add p suffix to date[time] parsers 2014-08-08 07:29:02 -07:00
Simon Michael
d0ad571321 fix manual url in default CSV rules file 2014-08-07 13:15:40 -07:00
Julien Moutinho
a6190420b2 data: add source location to transactions 2014-08-07 16:38:44 +02:00
Simon Michael
40ab1e17f6 amounts cleanups, and support zeros with commodity 2014-07-28 18:45:13 -07:00
Simon Michael
b3da48aaa0 identify the commodity in balance assertion errors 
Now that balance assertions are checking only a single commodity, it can
be confusing. Eg say all your amounts are in dollars, an assertion like
"= 0" checked the dollar balance in hledger 0.23 but always succeeds in
hledger 0.24. When an assertion fails, we now report which commodity was
checked to help troubleshooting.
 2014-07-17 15:23:03 -07:00
Simon Michael
3a16e6cfc7 mostly replace slow regexpr with regex-tdfa (fixes #189) 2014-07-06 14:03:28 -07:00
Simon Michael
c31710d942 look harder for decimal point & digit groups (fixes #196) 
Amount display styles have been reworked a bit; they are now calculated
after journal parsing, not during it. This allows the fix for #196:
we now search through the amounts until a decimal point is detected,
instead of just looking at the first one; likewise for digit groups.
Digit groups are now implemented with a better type.

Digit group size detection has been improved a little:
1000,000 now gives group sizes [3,4,4,...], not [3,3,...], and
10,000 gives groups sizes [3,3,...] not [3,2,2,..].
(To get [3,2,2,...] you'd use eg 00,00,000.)

There are still some old (or new ?) issues; I don't think we handle
inconsistent decimal points & digit groups too well. But for now all
tests pass.
 2014-07-02 23:26:16 -07:00
Simon Michael
9b4b85b4bf commodity & amount style parser cleanups 2014-07-02 11:23:30 -07:00
Simon Michael
0c3148ac7b add an --ignore-assertions flag 
Can be helpful when reading Ledger files, where assertions may have
different semantics; or for getting some answers from your journal
to help you fix your assertions.

Could be called --no-assertions, but this might create surprise when it
has an effect contrary to --no-new-accounts.

I had to add another flag throughout the parsers & journal read
functions, ok for now.
 2014-07-01 18:26:37 -07:00
Simon Michael
cf3d21afef csv and general reader fixes, cleanups 
- The CSV reader no longer writes a "(stdin).rules" file when reading
  from stdin.

- Selection of reader(s) is now smarter when input is coming from stdin.
  Previously, all readers were considered applicable for stdin.  This
  meant that when reading a CSV file from stdin, the journal and timelog
  readers were always tried first, and if the CSV file was unparseable,
  you'd see the first (journal) reader's error instead of the CSV
  reader's.  Now, the readers do some basic content sniffing when
  reading stdin, so it generally tries only the one right reader and
  we'll see the right errors.

- The read system now has more debug output.
 2014-05-09 17:55:32 -07:00
Simon Michael
4740c7082e csv: allow an empty first name in fields list (fixes #178) 2014-05-03 15:05:35 -07:00
Simon Michael
dedd26bbf5 csv: don't count fields in skipped lines (fixes #177) 2014-05-03 14:54:15 -07:00
Simon Michael
70ad5b26ff journal: really don't parse + before an amount as part of the commodity (fixes #181 harder) 2014-04-30 11:28:47 -07:00
Simon Michael
8341b4a8d3 journal: don't parse + before an amount as part of the commodity (fixes #181) 2014-04-30 11:21:01 -07:00
Simon Michael
da0ce23144 don't let default commodity limit the max precision (fixes #169) 2014-03-26 17:25:59 -07:00
Simon Michael
4b4715ab76 parsing: don't accept . as a valid amount 2014-03-01 10:26:20 -08:00
Simon Michael
762a1b4d74 parsing: comments can also start with # in col 0 2014-02-27 11:47:36 -08:00
Simon Michael
d158d401d9 fix some breakages found by Hydra 2014-02-05 19:30:01 -08:00
Simon Michael
5223bc5c41 lib: rename and expose more journal file parsers 2014-02-05 18:55:38 -08:00
Simon Michael
dc8b687056 extra: hledger-rewrite script 2014-02-05 13:03:06 -08:00
Simon Michael
3cf53661f3 new debug helpers; --debug=N sets debugLevel 
The debug level set by `--debug[=N]` is now available to pure and
startup code as debugLevel, using unsafePerformIO.

`dbg LABEL ...` is now the go-to helper for tracing values on the
console; it produces output when the debug level is non-zero. `dbgExit`
is similar but exits immediately, avoiding further output. The
`dbgshow`, `dbgppshow` and `dbgpprint` variants allow control over the
pretty-printing method and required debug level, allowing more control
over what is displayed when.

Other cleanups: lstrace -> ltrace, pdbgAt -> pdbg, tracewith -> traceWith.
 2013-12-06 13:35:50 -08:00
Simon Michael
f9656a21af print: comment positions (same line or next line) are now preserved 2013-09-10 10:32:49 -07:00
Simon Michael
eff1d3f1a5 csv reader: add the include directive, useful for factoring out common rules used with multiple CSV files 2013-08-03 20:53:41 -07:00
Dmitry Astapov
ed58d815d6 Fix for multiple field assignments in CSV parsing 2013-06-19 08:30:33 +01:00
Simon Michael
080eb866ec web: clean up language extensions a bit, make autoweb works again 2013-06-04 18:23:55 -07:00
Simon Michael
44545d6ec7 parsing: update a csv reader error message 2013-06-01 12:38:58 -07:00
Simon Michael
a26ab926d8 parsing: don't fail when a csv amount has trailing whitespace (fixes #113) 2013-06-01 12:38:13 -07:00
Simon Michael
56a37aabcf small cleanup 2013-05-30 15:17:21 -07:00
Simon Michael
0be986fcb9 parsing: actually parse balance assertions 
Parser unit test is commented out until HTF tests are fixed.
 2013-05-28 16:22:30 -07:00
Simon Michael
78837c66a6 parsing: fix test breakage due to new csv rules format (fixes #102) 2013-04-12 14:59:28 -07:00
Simon Michael
616a25979a CSV reader version 2 with new rules syntax 
At long last. The main change is a new rules file format that aims to
be more powerful and more intuitive than v1 (hledger 0.19.x and older).
Existing rules files will need to be adapted manually to the new format.
 2013-03-29 22:56:55 +00:00
Simon Michael
af5c0e80e0 parsing: read files in universal newline mode 
Line endings other than the unix style, are now accepted in journal and rules files.
 2013-03-29 18:46:10 +00:00
Simon Michael
7b6a59123d eolof parser matches end of line or end of file 2013-03-29 18:42:00 +00:00
Simon Michael
69f2491224 export journal file code parser 2013-02-24 21:07:54 +00:00
Simon Michael
621a91807e rename actual/effective dates to primary/secondary 
The command-line flag is now --date2. Alternate spellings --effective and --aux-date
are accepted for compatibility.
 2012-12-06 04:43:41 +00:00
Simon Michael
869ff2528a accept normal simple date syntax in posting dates, give an error on bad date syntax here 2012-12-06 03:23:56 +00:00
Simon Michael
20e68f23a8 support effective dates on postings as well 
We now preserve all dates while reporting, instead of overwriting one with the other.
 2012-12-06 02:41:37 +00:00
Simon Michael
e75abc4625 parse posting dates, better comment/tag handling, begin using HTF for unit testing 2012-12-06 00:28:23 +00:00
Simon Michael
4aafeb32e6 refactor: clean up Posting construction 2012-12-06 00:03:07 +00:00
Joachim Breitner
f0372bdd93 Also parse ! as a modifier, like * 2012-12-03 21:38:05 +00:00
Simon Michael
6eda8c4bbf csv reader: append ".rules" to the original file name instead of replacing its extension 2012-11-26 01:56:39 +00:00
Simon Michael
afb4fb0356 csv reader: parse parenthesised amounts as negative 2012-11-26 01:56:01 +00:00
Simon Michael
8b4a99c4d5 79: convert: add a skip-lines directive (Magnus Henoch) 2012-11-18 18:21:52 +00:00
Simon Michael
1c6a7d8a6f rename the commodityless amount constructor to num 2012-11-20 02:22:20 +00:00
Simon Michael
45d4a38cc7 refactor: setAmountPrecision -> withPrecision 2012-11-19 23:24:04 +00:00
Simon Michael
64180b18ef refactor: clarify that price amounts have only a single commodity 2012-11-19 23:17:55 +00:00
Simon Michael
8c06553e3b refactor: make amount price a non-maybe, reducing noise 2012-11-19 22:39:08 +00:00
Simon Michael
4567e91409 refactor: move amount display settings out of commodity, simplify amount construction 2012-11-19 21:20:10 +00:00
Simon Michael
1c1058be4e 70: ignore ledger's fixed lot price declarations 2012-11-17 02:40:27 +00:00
Simon Michael
28baf926f7 parsing: ignore ledger-style balance assertions ("= BAL" after a posting's amount) 2012-11-02 16:50:36 +00:00
Simon Michael
5b9c5459be web: make hledger[-lib] compatible with all of yesod's language extensions 
Useful when building all of hledger-web at once.
 2012-11-14 17:25:02 +00:00
Simon Michael
2a4d89bb27 expose more utilities from CsvReader 2012-05-29 21:00:49 +00:00
Simon Michael
0b96a767b4 support tilde (~) in journal and rules file paths 2012-05-30 08:36:01 +00:00
Simon Michael
776ad2a098 remove ensureRulesFile debug trace 2012-05-30 08:36:34 +00:00
Simon Michael
34318906d8 register, print: fix reverse ordering of same-day transactions 2012-06-29 18:06:58 +00:00
Simon Michael
33b7c91e20 combine journal updates with foldl' instead of foldr 
on general principles, even though it makes no measurable difference
 2012-05-30 10:19:08 +00:00
Simon Michael
9a239789fa apply journal updates in the order they were parsed, not the reverse (!) 
Fixes the order of include files in stats command.
 2012-05-30 10:16:37 +00:00
Simon Michael
50068221c3 fix build warnings in all GHC versions 2012-03-29 21:19:35 +00:00
Simon Michael
b77ba66fb1 haddock fix 2012-05-15 01:51:14 +00:00
Simon Michael
9524e6d927 more metadata -> tag renaming in the parser 2012-05-28 00:24:49 +00:00
Simon Michael
1062e2f9a4 clean up reader selection, don't write a csv rules file on journal parse error 2012-05-28 18:40:36 +00:00
Simon Michael
cb905a741c new query by tag implementation 2012-05-28 00:27:55 +00:00
Simon Michael
2fb2aea056 rename metadata fields to tags 2012-05-27 22:59:06 +00:00
Simon Michael
3ee6a351f3 big query/tests/show refactoring 
- use new query system for command line too, filterspec is no more
- move unit tests near the code they test, run them in bottom up order, add more
- more precise Show instances, used for debugging not ui
 2012-05-27 18:14:20 +00:00
Simon Michael
56cf9b21cb fix txn & posting comment parsing & printing; better unit tests 2012-05-15 01:49:05 +00:00
Simon Michael
a58a5f96ad more robust comments & metadata, on postings too, and document as "tags" 2012-05-14 18:52:22 +00:00
Simon Michael
88212f26e8 simplify journal parser names 2012-05-09 15:34:05 +00:00
xiaoruoruo
ce83876700 metadata parse and show for transaction 2012-04-08 19:13:56 +00:00
Simon Michael
8492f6cae4 fix unicode handling on GHC >= 7.2, unify utf8 IO compatibility layer 
tests pass again from GHC 6.12.3 to 7.4.1
 2012-03-29 19:06:31 +00:00
Simon Michael
d4451ce5e3 read system cleanup, require conversion rules from a file to simplify API 2012-03-24 18:08:11 +00:00
Simon Michael
e396c0dc8d push csv rule and format string types down 2012-03-24 01:58:34 +00:00
Simon Michael
6eb7ad28e1 refactor/beef up readJournal/readJournalFile 2012-03-23 16:21:41 +00:00
Simon Michael
4d7a809c4a cleanups and early code for csv reader based on convert 2012-03-10 21:55:48 +00:00
Simon Michael
381ebea35c show correct line number for posting parse errors (#67) 2011-10-04 23:39:53 +00:00
Simon Michael
8d62452260 parsing: historical price directives may contain a numeric time zone, like ledger; note that the time is ignored 2011-06-01 01:50:04 +00:00
Simon Michael
946e5ffcbc parsing: parse time of day more carefully 2011-05-31 22:45:54 +00:00
Simon Michael
1f24e025da parsing: make bad date parse error more reliable 2011-05-31 19:49:37 +00:00
Simon Michael
a44aa8d7d2 refactor: remove unnecessary LANGUAGE CPP pragmas 2011-07-17 23:29:57 +00:00
Simon Michael
1273f02a9a more balance/amount clarifications and a test 2011-08-30 11:37:36 +00:00
Simon Michael
957c349780 --alias command-line option 2011-08-05 00:05:39 +00:00
Simon Michael
30b7448f45 parsing: alias/end aliases directives, like ledger's but a bit more powerful 2011-08-04 08:45:18 +00:00
Simon Michael
4015e02097 account directive should preserve virtual/balanced virtual posting type 2011-08-02 23:29:13 +00:00
Simon Michael
345c2343b6 refactor ledgerDirective 2011-08-04 07:49:10 +00:00
Johann Klähn
8e7d20e4d5 parsing: support pop as well as end tag 2011-05-31 17:41:12 +00:00
Simon Michael
75f0886ad4 parsing: make the leading ! in directives optional, like ledger 2011-08-02 23:28:53 +00:00
Simon Michael
5084280879 imports, api cleanup 2011-05-28 04:11:44 +00:00
Simon Michael
8980dd9336 parser cleanup 2011-01-19 23:27:44 +00:00
Simon Michael
6a845eb5e4 i18n: always read and write UTF-8, ignoring current locale (#34) 
Also remove a bunch of CPP conditions.
Uses the GHC-version-independent UTF-8 layer from pandoc.
 2011-01-21 01:24:51 +00:00
Simon Michael
5f1ebc694a parsing: assume current year when transaction dates have no year and there is no default year 2011-03-11 18:45:57 +00:00
Simon Michael
1c7ad7f421 parsing: more careful validation of eg leap years in transaction dates (only) 2011-04-22 13:55:42 +00:00
Simon Michael
d3663b818e parsing: better international number format support (#32) 
This allows period and comma to be used for decimal point and digit group
separator or vice versa, and also flexible digit groups.  See
http://en.wikipedia.org/wiki/Decimal_separator .  Digit group separators
are possibly not worth the trouble and might not stay.
 2011-01-19 12:32:18 +00:00
Simon Michael
dd9e78a67a reorganize tests 2010-12-27 20:26:22 +00:00
Simon Michael
e03ada3bd3 parsing: support @@ syntax specifying total price 2011-01-15 02:04:53 +00:00
Simon Michael
631215d5d7 parsing: accept (but ignore) c++ ledger's C commodity conversion directive 2010-11-12 23:55:40 +00:00
Simon Michael
d7617fe154 parsing: ignore cleared flags at the start of postings, parse account name correctly 2010-11-13 20:20:04 +00:00
Simon Michael
811e71aba7 price precisions don't influence display precision; balancing is based on display precision (#23) 
Like ledger, price amounts are now "unobserved", ie their precision does
not affect the canonical display precisions used when displaying amounts,
and transaction balancing is done based on display precision, ie amounts
are considered to balance if their sum appears to be zero when using the
canonical display precision.
 2010-11-14 22:44:37 +00:00
Simon Michael
77519e2c1d remove debug code breaking reports 2010-11-14 18:10:48 +00:00
Simon Michael
e707f97847 parsing: parse transaction and posting metadata, though we don't use it yet 2010-11-13 22:17:32 +00:00
Simon Michael
17d5acf64b refactor: more reader cleanups 2010-11-15 07:18:35 +00:00
Simon Michael
a3a7a346ac uniquify reader module names 2010-11-15 07:01:46 +00:00
Simon Michael
d6c2cf6a90 parsing: save final parse context with journal 2010-11-13 15:03:40 +00:00
Simon Michael
7714bab58d web: offer a choice of files to edit when there are multiple (included) files 2010-09-24 01:56:11 +00:00
Simon Michael
78db98366f parsing: support D default commodity directive 2010-11-12 23:54:21 +00:00
Simon Michael
8429df0f32 remember all included file paths, if any 2010-09-22 23:02:19 +00:00
Simon Michael
96036e56a0 docs: comments 2010-09-22 21:51:25 +00:00
Simon Michael
d132f5e45a ledgerInclude cleanup 2010-09-22 21:52:04 +00:00
Ben Boeckel
5e576e9ff0 fix-recursive-includes 
Fix recursive includes beyond depth 3. Also add a test case for the bug.
 2010-09-02 00:30:57 +00:00
Simon Michael
d29b393ca2 fix a failing test, handle unicode better in errors 2010-09-05 18:18:50 +00:00
Simon Michael
ce7e155934 parsing: fix obscured date parse errors with parsec 3; require split 
With parsec 3, invalid date errors were not being reported properly.
This should be more robust.
 2010-09-03 23:22:58 +00:00
Simon Michael
63de5e0127 parsing: fix wrong parse error locations within postings 2010-07-08 02:51:17 +00:00
Simon Michael
3e9ad0ae13 parsing: don't let trailing whitespace in a timelog description mess up layout 2010-07-08 03:08:28 +00:00
Simon Michael
7d7159609b refactor: much renaming of ledger -> journal, hopefully the right amount 2010-07-13 06:30:06 +00:00
Simon Michael
8a64792ba7 smarter file reading: detect (or specify) intended data format and show appropriate error messages 
New data reader modules need to provide just reader :: Reader, which is
the format name, a detector predicate, and a parser.
 2010-06-25 14:56:48 +00:00
Simon Michael
1ec1f7c4ea more file reading cleanup, try each known format in turn 
Currently this is just the journal and timelog file formats.  This is more
scalable, but when things go wrong there is no longer just a single parse
error. For now we'll just show the first error, the one from the journal
file parser.
 2010-05-31 01:15:18 +00:00
Simon Michael
a848a835a2 clean up and combine I/O and parsing under Hledger.Read.* 
This facilitates adding readers for new data formats. Timelog parsing is temporarily broken.
 2010-05-30 19:11:58 +00:00