imp: set display style, natural precision on valued amounts (fix #2105, precisiongeddon)

Cost/value conversion now applies the standard display style, and
sets the display precision equal to the internal decimal precision
(or 8 if the decimal appears to be infinite).
This means value reports and especially `print -V` now show amounts
with more accurate and standard style and precision.

New tests have been added describing and explaining various
style/precision behaviours in print cost/value reports.

hledger-lib/Hledger/Data/Amount.hs

@@ -91,7 +91,9 @@ module Hledger.Data.Amount (
   amountSetPrecision,
   withPrecision,
   amountSetFullPrecision,
-  -- amountInternalPrecision,
+  amountSetFullPrecisionUpTo,
+  amountInternalPrecision,
+  amountDisplayPrecision,
   setAmountInternalPrecision,
   withInternalPrecision,
   setAmountDecimalPoint,
@@ -375,17 +377,36 @@ amountSetPrecision p a@Amount{astyle=s} = a{astyle=s{asprecision=p}}
 
 -- | Increase an amount's display precision, if needed, to enough decimal places
 -- to show it exactly (showing all significant decimal digits, without trailing zeros).
--- If the amount's display precision is unset, it is will be treated as precision 0.
+-- If the amount's display precision is unset, it will be treated as precision 0.
 amountSetFullPrecision :: Amount -> Amount
 amountSetFullPrecision a = amountSetPrecision p a
   where
     p                = max displayprecision naturalprecision
     displayprecision = asprecision $ astyle a
-    naturalprecision = Precision . decimalPlaces . normalizeDecimal $ aquantity a
+    naturalprecision = Precision $ amountInternalPrecision a
+-- XXX Is that last sentence correct ?
+-- max (Precision n) NaturalPrecision is NaturalPrecision.
+-- Would this work instead ?
+-- amountSetFullPrecision a = amountSetPrecision (Precision p) a
+--   where p = max (amountDisplayPrecision a) (amountInternalPrecision a)
 
--- -- | Get an amount's internal Decimal precision (not display precision).
--- amountInternalPrecision :: Amount -> Word8
--- amountInternalPrecision = decimalPlaces . normalizeDecimal . aquantity
+-- | Similar to amountSetPrecision, but with an upper limit (up to 255).
+-- And always sets an explicit Precision.
+-- Useful for showing a not-too-verbose approximation of amounts with infinite decimals.
+amountSetFullPrecisionUpTo :: Word8 -> Amount -> Amount
+amountSetFullPrecisionUpTo n a = amountSetPrecision (Precision p) a
+  where p = min n $ max (amountDisplayPrecision a) (amountInternalPrecision a)
+
+-- | How many internal decimal digits are stored for this amount ?
+amountInternalPrecision :: Amount -> Word8
+amountInternalPrecision = decimalPlaces . normalizeDecimal . aquantity
+
+-- | How many decimal digits will be displayed for this amount ?
+amountDisplayPrecision :: Amount -> Word8
+amountDisplayPrecision a =
+  case asprecision $ astyle a of
+    Precision n      -> n
+    NaturalPrecision -> amountInternalPrecision a
 
 -- | Set an amount's internal precision, ie rounds the Decimal representing
 -- the amount's quantity to some number of decimal places.

hledger-lib/Hledger/Data/Balancing.hs

@@ -461,7 +461,10 @@ journalBalanceTransactions bopts' j' =
     j@Journal{jtxns=ts} = journalNumberTransactions j'
     -- display precisions used in balanced checking
     styles = Just $
-      journalCommodityStylesWith HardRounding  -- txn balancedness will be checked using commodity display precisions
+      -- Use all the specified commodity display precisions, with hard rounding, when checking txn balancedness.
+      -- XXX Problem, those precisions will also be used when inferring balancing amounts;
+      -- it would be better to give those the precision of the amount they are balancing.
+      journalCommodityStylesWith HardRounding
       j
     bopts = bopts'{commodity_styles_=styles}
       -- XXX ^ The commodity directive styles and default style and inferred styles

hledger-lib/Hledger/Data/Valuation.hs

@@ -47,6 +47,7 @@ import Hledger.Data.Types
 import Hledger.Data.Amount
 import Hledger.Data.Dates (nulldate)
 import Text.Printf (printf)
+import Data.Decimal (normalizeDecimal, DecimalRaw (decimalPlaces))
 
 
 ------------------------------------------------------------------------------
@@ -183,22 +184,29 @@ mixedAmountValueAtDate priceoracle styles mc d = mapMixedAmount (amountValueAtDa
 -- valuation date.)
 --
 -- The returned amount will have its commodity's canonical style applied,
--- but with the precision adjusted to show all significant decimal digits
--- up to a maximum of 8. (experimental)
+-- (with soft display rounding).
 --
 -- If the market prices available on that date are not sufficient to
 -- calculate this value, the amount is left unchanged.
+--
 amountValueAtDate :: PriceOracle -> M.Map CommoditySymbol AmountStyle -> Maybe CommoditySymbol -> Day -> Amount -> Amount
 amountValueAtDate priceoracle styles mto d a =
   case priceoracle (d, acommodity a, mto) of
     Nothing           -> a
     Just (comm, rate) ->
-      -- setNaturalPrecisionUpTo 8 $  -- XXX force higher precision in case amount appears to be zero ?
-                                      -- Make default display style use precision 2 instead of 0 ?
-                                      -- Leave as is for now; mentioned in manual.
-      styleAmounts styles
       nullamt{acommodity=comm, aquantity=rate * aquantity a}
 
+      -- Manage style and precision of the new amount. Initially:
+      --  rate is a Decimal with the internal precision of the original market price declaration.
+      --  aquantity is a Decimal with a's internal precision.
+      --  The resulting internal precision will be larger than both (their sum ?).
+      --  The display precision will be that of nullamt (0).
+      -- Now apply the standard display style for comm
+      & styleAmounts styles
+      -- and set the display precision to rate's (internal) precision
+      & amountSetPrecision (Precision $ decimalPlaces $ normalizeDecimal rate)
+      -- see also print-styles.test, valuation2.test
+
 -- | Calculate the gain of each component amount, that is the difference
 -- between the valued amount and the value of the cost basis (see
 -- mixedAmountApplyValuation).

hledger/test/journal/valuation.test

@@ -105,7 +105,8 @@ $ hledger -f- reg -V
 2000-01-01                      (a)                        €120.00       €120.00
 
 
-# ** 8. print -V affects posting amounts but not balance assertions.
+# ** 8. print -V affects posting amounts, but not balance assertions
+# (causing it to show a failing balance assertion).
 <
 P 2000/1/1 $ €1.20
 2000/1/1
@@ -113,7 +114,7 @@ P 2000/1/1 $ €1.20
 
 $ hledger -f- print -V
 2000-01-01
-    (a)            €120 = $100
+    (a)         €120.00 = $100
 
 >=0
 

hledger/test/journal/valuation2.test

@@ -57,24 +57,14 @@ $ hledger -f- print -x --value=now,Z
 
 # ** 6. request commodity C - uses reverse of C->B price.
 # There's nothing setting C display style, so the default style is used,
-# which shows no decimal digits.
-# And because that makes it display as zero, the commodity symbol
-# and sign are not shown either.
+# but the precision is increased to show all significant decimal digits
+# (otherwise it would show C0).
 $ hledger -f- print -x --value=now,C
 2019-06-01
-    a              C0
-    b              C0
+    a            C0.5
+    b           C-0.5
 
 >=
-# # There's nothing setting C display style, so the default style is used,
-# # but the precision is increased to show the decimal digit
-# # (otherwise it would show C0).
-# $ hledger -f- print -x --value=now,C
-# 2019-06-01
-#     a            C0.5
-#     b           C-0.5
-#
-# >=
 
 # ** 7. request commodity D - chains B->A, A->D prices
 $ hledger -f- print -x --value=now,D

hledger/test/print/print-style.test

@@ -1,4 +1,4 @@
-# * print amount style tests
+# * print command's amount styling
 #
 # Here's an overview of historical behaviour.
 # See the tests below for examples.
@@ -33,7 +33,7 @@
 #  | 1.31.1 --round=all  | hard | hard | hard | hard    |
 
 
-# Four print style tests. In these, basic styling is applied
+# ** Four print style tests. In these, basic styling is applied
 # to all amounts (the commodity symbol moves to the left),
 # and precision styling is applied as described below.
 <
@@ -141,3 +141,140 @@ $ hledger -f- print -x
     d               0
 
 >=
+
+# ** Some style/precision behaviours with cost and value, with explanations as of 2023-10.
+
+# ** 8. Costs normally don't affect display precisions,
+# so why is it showing the B amounts with 4 decimal digits instead of the default 0 ?
+# Summary: a's calculated cost has 4 digits, and so also must the inferred b amount.
+#
+# Some implementation-level explanation, for the record:
+#
+# In journalFinalise,
+#
+# journalStyleAmounts infers A and B display styles and A precision 0,
+# and applies the style but not the precision to the A amount.
+# 
+# journalBalanceTransactions infers b's amount from a's cost,
+# which it calculates as 0.1234 B.
+# 
+# journalInferCommodityStyles infers commodity styles again,
+# now inferring B precision 4 from b's amount.
+# 
+# entriesReport converts a's amount to cost, calculating 0.1234 B
+# with journalToCost, which does not re-apply styles or precisions.
+#
+# print does no rounding and shows a's cost 0.1234 B and b's amount 0.1234 B.
+
+<
+2023-01-01
+  a    1 A @ 0.1234 B
+  b
+
+$ hledger -f- print -x -B
+2023-01-01
+    a        0.1234 B
+    b       -0.1234 B
+
+>=
+
+# ** 9. Why is it showing the B amounts with 4 decimal digits here ?
+# Summary: the a value is calculated from a nullamt with 0 decimal digits,
+# then re-styled again with the 4 digit B precision inferred from b's amount.
+# This works out right, in this case.
+#
+# Details:
+#
+# journalStyleAmounts infers A display style and A precision 0 from 10 A,
+# and applies the style but not the precision to all A amounts.
+#
+# journalBalanceTransactions infers b's amount from a's cost,
+# which it calculates as 0.1234 B.
+# 
+# journalInferCommodityStyles infers commodity styles again,
+# now inferring B precision 4 from b's amount.
+# 
+# journalInferMarketPricesFromTransactions infers a 0.1234 B price for A.
+# 
+# entriesReport converts 10 A to value using that price,
+# with amountValueAtDate, which calculates from nullamt (would show as B0)
+# then re-applies the inferred B style (symbol on the right)
+# and sets the price's (0.1234) numeric precision (4) as the display precision,
+# showing 0.1234 B.
+# 
+# print does no rounding, and shows a's value 0.1234 B and b's amount 0.1234 B.
+
+<
+2023-01-01
+  a    1 A @ 0.1234 B
+  b
+
+$ hledger -f - print --infer-market-prices -V
+2023-01-01
+    a        0.1234 B
+    b       -0.1234 B
+
+>=
+
+# ** 10. What if a different style/precision is specified for B, eg more digits
+# than the price ? We still expect the calculated amount to have the price's precision,
+# as above.
+# XXX
+# 2023-01-01
+#     a         0.1234B
+#     b       -0.12340B
+<
+2023-01-01
+  a    1 A @ 0.1234 B
+  b
+
+$ hledger -f - print --infer-market-prices -V -c '0.123456B'
+2023-01-01
+    a         0.1234B
+    b        -0.1234B
+
+>=
+
+# ** 11. Printing A's value in B, using an inferred reverse market price,
+# with no B amount written in the journal to infer a display precision from.
+# In this case B is displayed with the default style (symbol on the left)
+# and the numeric precision of the inferred price 0.5 (1 decimal  digit).
+# (#2105)
+<
+P 2023-01-01 B 2A
+
+2023-01-01
+  a  1A
+  b
+
+$ hledger -f- print -X B
+2023-01-01
+    a            B0.5
+    b           B-0.5
+
+>=
+
+# ** 12. What if the inferred reverse market price has infinite decimal digits ?
+<
+P 2023-01-01 B 3.00A
+
+2023-01-01
+  a  1A
+  b
+
+# current:
+# That propagates to the calculated value, causing it to be displayed with the
+# maximum 255 decimal digits.
+$ hledger -f- print -X B
+2023-01-01
+    a     B0.3333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333
+    b    B-0.3333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333
+
+# preferred:
+# The reverse price is given the same precision as in the forward price declaration
+# it was inferred from - 2 digits in this example, and that in turn affects the
+# calculated value.
+#2023-01-01
+#    a     B0.33
+#    b    B-0.33
+