Commit Graph

1839 Commits

Author SHA1 Message Date
Stephen Morgan
0df3a4714f lib: When performing a summary posting report without depth limiting,
report exclusive balances rather than inclusive balances (#1568).
2021-06-11 16:53:39 -10:00
Simon Michael
b81f8f768d ;csv: amount-setting notes, doc improvements from reddit discussion
https://www.reddit.com/r/plaintextaccounting/comments/nxu1ss/hledger_parsing_csv_with_negative_amount_in_debit/
2021-06-11 16:30:43 -10:00
Simon Michael
665fec83cd
Merge pull request #1560 from Xitian9/rationalisevaluation
Clean up valuation functions, and make clear which to use where.
2021-06-07 19:02:44 -10:00
Stephen Morgan
0f1837816d lib,cli,ui,web: Add check balancednoautoconversion command, which checks that
transactions are balanced possibly using explicit prices, but without
inferring any prices. This is included in --strict mode.

Renames check autobalanced to check balancedwithautoconversion.
2021-06-07 18:58:58 -10:00
Stephen Morgan
0a019e2167 lib,cli,web,bin: Replace journalSelectingAmountFromOpts with journalApplyValuationFromOpts.
This also has the effect of allowing valuation in more reports, for
example the transactionReport.
2021-06-08 14:55:05 +10:00
Stephen Morgan
68e975adf1 lib,cli,ui,web: Remove unused LANGUAGE pragmas. 2021-06-07 17:33:54 -10:00
Stephen Morgan
55308e1ca8 lib,cli,ui,web: Remove unnecessary CPP when dropping support for GHC 8.2. 2021-06-07 17:33:54 -10:00
Stephen Morgan
c827cbcc3e lib: Refactor to eliminate use of printf. 2021-06-03 23:23:54 -10:00
Stephen Morgan
e80bb37b1c lib: Remove unused String utility functions. 2021-06-03 23:23:54 -10:00
Stephen Morgan
0b419adba2 lib: Remove unused Text utility functions. 2021-06-03 23:23:54 -10:00
Stephen Morgan
bf22c3efdd lib,cli: Replace concat(Top|Bottom)Padded with textConcat(Top|Bottom)Padded. 2021-06-03 23:23:54 -10:00
Stephen Morgan
0e59fee251 lib,cli: Export Text.Tabular from Text.Tabular.AsciiWide, clean up import lists. 2021-06-03 23:23:54 -10:00
Simon Michael
1e2ff1315b Don't infer a txn price with same-sign amounts (#1551) 2021-06-02 15:22:57 -10:00
Stephen Morgan
6c2ab5c940 lib: When matching an account query against a posting, don't try to
match against the same posting twice, in cases when poriginal is
Nothing.
2021-05-16 12:43:13 -10:00
Stephen Morgan
940b2c6ab9 lib: Create mixedAmountApplyValuationAfterSumFromOptsWith for doing any valuation needed after summing amounts. 2021-05-13 22:18:52 +10:00
Stephen Morgan
6fb3dfdbb2 lib: Create journalApplyValuationFromOpts.
This does costing and valuation on a journal, and is meant to replace
most direct calls of costing and valuation. The exception is for reports
which require amounts to be summed before valuation is applied, for
example a historical balance report with --value=end.
2021-05-13 19:52:32 +10:00
Stephen Morgan
dc16451de0 lib: Remove unused (amount|mixedAmount|posting|transaction)ApplyCostValuation functions. 2021-05-07 22:25:19 +10:00
Stephen Morgan
53611be6e9 lib,ui: Do all cost conversion and price stripping in journalSelectingAmountFromOpts. 2021-05-07 21:51:15 +10:00
Stephen Morgan
d1467d4766 lib: When inferring total prices, use the precision of toamount, rather than NaturalPrecision. 2021-05-02 21:14:36 +10:00
Stephen Morgan
686a0871a9 lib: Make sure fromamount and toamount have opposite signs when inferring prices.
Also change priceInferrerFor so that it would give the correct
(negative) prices when fromamount and toamount have the same sign.
2021-05-02 21:14:36 +10:00
Stephen Morgan
0078f1a520 lib: Infer prices correctly even when there are only balance assignments. 2021-05-01 09:45:29 -10:00
Stephen Morgan
7cb621b82f lib,cli,ui,bin: Remove unnecessary normalisedMixedAmount, replace
normaliseMixedAmountSquashPricesForDisplay with mixedAmountStripPrices.
2021-05-01 09:45:29 -10:00
Stephen Morgan
4780a02e5a lib,cli: Remove showAmounts*B functions, replacing them entirely with showMixedAmount*B functions. 2021-05-01 09:45:29 -10:00
Stephen Morgan
5e7b69356f lib: Change internal representation of MixedAmount to use a strict Map
instead of a list of Amounts. No longer export Mixed constructor, to
keep API clean (if you really need it, you can import it directly from
Hledger.Data.Types). We also ensure the JSON representation of
MixedAmount doesn't change: it is stored as a normalised list of
Amounts.

This commit improves performance. Here are some indicative results.

hledger reg -f examples/10000x1000x10.journal
- Maximum residency decreases from 65MB to 60MB (8% decrease)
- Total memory in use decreases from 178MiB to 157MiB (12% decrease)

hledger reg -f examples/10000x10000x10.journal
- Maximum residency decreases from 69MB to 60MB (13% decrease)
- Total memory in use decreases from 198MiB to 153MiB (23% decrease)

hledger bal -f examples/10000x1000x10.journal
- Total heap usage decreases from 6.4GB to 6.0GB (6% decrease)
- Total memory in use decreases from 178MiB to 153MiB (14% decrease)

hledger bal -f examples/10000x10000x10.journal
- Total heap usage decreases from 7.3GB to 6.9GB (5% decrease)
- Total memory in use decreases from 196MiB to 185MiB (5% decrease)

hledger bal -M -f examples/10000x1000x10.journal
- Total heap usage decreases from 16.8GB to 10.6GB (47% decrease)
- Total time decreases from 14.3s to 12.0s (16% decrease)

hledger bal -M -f examples/10000x10000x10.journal
- Total heap usage decreases from 108GB to 48GB (56% decrease)
- Total time decreases from 62s to 41s (33% decrease)

If you never directly use the constructor Mixed or pattern match against
it then you don't need to make any changes. If you do, then do the
following:

- If you really care about the individual Amounts and never normalise
  your MixedAmount (for example, just storing `Mixed amts` and then
  extracting `amts` as a pattern match, then use should switch to using
  [Amount]. This should just involve removing the `Mixed` constructor.
- If you ever call `mixed`, `normaliseMixedAmount`, or do any sort of
  amount arithmetic (+), (-), then you should replace the constructor
  `Mixed` with the function `mixed`. To extract the list of Amounts, use
  the function `amounts`.
- If you ever call `normaliseMixedAmountSquashPricesForDisplay`, you can
  replace that with `mixedAmountStripPrices`. (N.B. this does something
  slightly different from `normaliseMixedAmountSquashPricesForDisplay`,
  but I don't think there's any use case for squashing prices and then
  keeping the first of the squashed prices around. If you disagree let
  me know.)
- Any remaining calls to `normaliseMixedAmount` can be removed, as that
  is now the identity function.
2021-05-01 09:45:29 -10:00
Simon Michael
483fa9682e cli: ANSI color is now also disabled by -o/--output-file (#1533)
ANSI color on stdout (not stderr) is now disabled if the
-o/--output-file option is detected (and its value is not "-").

Added outputFileOption, and more advice in comments.
2021-04-17 16:51:08 -10:00
Simon Michael
fe846a0c7f ;lib: comments 2021-04-17 16:06:20 -10:00
Simon Michael
21a49b9479 ;lib: note caveats with useColor* 2021-04-14 06:23:21 -10:00
Simon Michael
6298722ade lib: debug output checks for color support on stderr, not stdout
This is more accurate.
useColor is replaced by useColorOnStdout, useColorOnStderr.
2021-04-14 06:23:21 -10:00
Simon Michael
2b04b76448 lib: debug output now respects --color/NO_COLOR/ANSI support
Debug output now selects color or monochrome in the same way as normal
output.
2021-04-14 06:23:21 -10:00
Simon Michael
5e57cfb43e lib: useColor, colorOption helpers usable anywhere 2021-04-14 06:23:21 -10:00
Simon Michael
0f4e462ae7 add a Show instance for AmountDisplayOpts, WideBuilder
For debug logging.
2021-04-13 17:19:22 -10:00
Stephen Morgan
28fd7c65f2 lib: When displaying amounts on a single line, always display at least one amount, even if that would exceed the requested maximum width. Addresses #1526. 2021-04-13 15:29:58 -10:00
Simon Michael
f5121129ec timeclock: print shows timeclock amounts with just 2 decimals, like pre-1.21 (#1527) 2021-04-12 18:40:33 -10:00
Stephen Morgan
493ed14629 lib,test: Make sure historical postings have prices stripped if necessary in PostingsReport (see issue #1522). 2021-04-11 18:31:30 -10:00
Stephen Morgan
f6feef7f80 lib,cli: Strip prices in MultiBalanceReport and PostingsReport whenever
we know we won't need them.

Knowing whether we need them is accomplished by pulling the "show-costs"
option used by the Close command up into ReportOpts.
2021-04-11 18:31:30 -10:00
Stephen Morgan
b7a2479186 lib: Use uniform naming for stripping prices.
Creates a new function amountStripPrices, and renames removePrices to
postingStripPrices.
2021-04-11 18:31:30 -10:00
Stephen Morgan
278153effa lib: Add more efficient toEncoding for custom ToJSON declarations. 2021-04-10 17:52:33 -10:00
Stephen Morgan
a529207ae7 lib: Do not round Decimal before applying JSON representation if
existing representation is small enough.

Previously the JSON representation of Decimal was rounded to 10 points
of precision before serialising. This sometimes results in an
unnecessary increase of precision.
2021-04-10 17:52:33 -10:00
Stephen Morgan
edbb713cf8 lib,cli: Fix ledgerDateSpan, so that in takes transaction dates into account. 2021-04-03 13:54:23 -10:00
Stephen Morgan
5fbb67b893 lib,cli: Move reportPeriodName to Hledger.Reports.ReportOptions, use it
for HTML and CSV output for compound balance reports.
2021-04-03 13:43:37 -10:00
Eric Mertens
48d558fc7a Tolerate spaces in amount fields in CSV files 2021-03-26 16:39:24 -07:00
Stephen Morgan
b6e20dea13 lib,test: Simplify the JSON representation of AmountPrecision.
It now uses the same JSON representation as Maybe Word8. This means that
the JSON serialisation is now broadly compatible with that used before the
commit f6fa76bba7, differing only in
how it handles numbers outside Word8 and that it can now produce null
for NaturalPrecision.
2021-03-25 15:47:34 -07:00
Stephen Morgan
4cb9dfb5b8 lib: Properly escape quotes in csv output. 2021-03-25 09:41:42 -07:00
Stephen Morgan
4609e79f2c lib,cli,ui,web: A number of AccountName and Journal functions which are
supposed to produce unique sorted use Sets internally to be slightly
more efficient. There is also a new function journalCommodities.
2021-03-23 11:26:30 -07:00
Stephen Morgan
7fe58f1346 lib: More efficiently check whether Amounts are or look zero.
Comparing two Quantity (either with (==) or compare) does a lot of
normalisation (calling roundMax) which is unnecessary if we're comparing
to zero. Do things more directly to save work.

For reg -f examples/10000x10000x10.journal, this results in
- A 12% reduction in heap allocations, from 70GB to 62GB
- A 14% reduction in (profiled) time, from 79s to 70s
Results for bal -f examples/10000x10000x10.journal are of the same order
of magnitude.
2021-03-23 11:26:30 -07:00
Stephen Morgan
522c8a6ad3 lib: In sorting account names, perform lookups on HashSets and HashMaps,
rather than lists. This is probably not an enormous performance sink in real
situations, but it takes a huge amount of time and memory in our
benchmarks (specifically 10000x10000x10.journal).

For bal -f examples/10000x10000x10.journal, this results in
- A 23% reduction in heap allocation, from 27GiB to 21GiB
- A 33% reduction in (profiled) time running, from 26.5s to 17.9s
2021-03-23 11:26:30 -07:00
Stephen Morgan
13589aca2e doc: Remove incorrect haddock comment. 2021-03-23 11:26:30 -07:00
Stephen Morgan
7488140608 lib: Do not call showAmount twice for every posting.
For print -f examples/10000x10000x10.journal, this results in
- A 7.7% reduction in heap allocations, from 7.6GB to 7.1GB.
2021-03-21 07:10:27 -07:00
Stephen Morgan
7aa3d3e760 lib,cli: Some efficiency improvements in register reports.
Strip prices after valuing postings in PostingsReport.
Use renderRow interface for Register report.

For reg -f examples/10000x10000x10.journal, this results in:
- Heap allocations decreasing by 55%, from 68.6GB to 31.2GB
- Resident memory decreasing by 75%, from 254GB to 65GB
- Total (profiled) time decreasing by 55%, from 37s to 20s
2021-03-21 07:10:27 -07:00
Stephen Morgan
d54e276658 lib: Split showMixedAmountB into showMixedAmountB and showAmountsB, the
former being a simple wrapper around the latter.

This removes the need for the showNormalised option, as showMixedAmountB
will always showNormalised and showAmountsB will never do so.

We also strip prices from MixedAmount before displaying if not displaying prices.
2021-03-21 07:10:27 -07:00