slaesvuo/hledger
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: areg: Make sure the true original transaction is made available in

Browse Source 
AccountTransactionsReport.

Only a limited number of journal transformations are allowed in
accountTransactionReports due to the need to include the original
transaction. Document these.

Make sure to remove currency and amount queries from the reportq, so
they do not cause problems after valuation.

Avoid confusion by calculating transaction date at one point, and
passing that down with the transaction.
This commit is contained in:
Stephen Morgan 2021-08-02 13:59:20 +10:00 committed by Simon Michael
parent b7e40a9e63
commit ae3c7e8756
2 changed files with 66 additions and 64 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

124
hledger-lib/Hledger/Reports/AccountTransactionsReport.hs
View File

@ -23,10 +23,10 @@ module Hledger.Reports.AccountTransactionsReport (
) )
where where
import Data.List (mapAccumR, nub, partition, sortOn) import Data.List (mapAccumR, nub, partition, sortBy)
import Data.List.Extra (nubSort) import Data.List.Extra (nubSort)
import Data.Maybe (catMaybes) import Data.Maybe (catMaybes)
import Data.Ord (Down(..)) import Data.Ord (Down(..), comparing)
import Data.Text (Text) import Data.Text (Text)
import qualified Data.Text as T import qualified Data.Text as T
import Data.Time.Calendar (Day, addDays) import Data.Time.Calendar (Day, addDays)
@ -95,11 +95,11 @@ triCommodityBalance c = filterMixedAmountByCommodity c . triBalance
accountTransactionsReport :: ReportSpec -> Journal -> Query -> AccountTransactionsReport accountTransactionsReport :: ReportSpec -> Journal -> Query -> AccountTransactionsReport
accountTransactionsReport rspec@ReportSpec{_rsReportOpts=ropts} j thisacctq = items accountTransactionsReport rspec@ReportSpec{_rsReportOpts=ropts} j thisacctq = items
where where
-- a depth limit should not affect the account transactions report -- A depth limit should not affect the account transactions report; it should show all transactions in/below this account.
-- seems unnecessary for some reason XXX -- Queries on currency or amount are also ignored at this stage; they are handled earlier, before valuation.
reportq = simplifyQuery $ And [depthlessq, periodq, excludeforecastq (forecast_ ropts)] reportq = simplifyQuery $ And [aregisterq, periodq, excludeforecastq (forecast_ ropts)]
where where
depthlessq = filterQuery (not . queryIsDepth) $ _rsQuery rspec aregisterq = filterQuery (not . queryIsCurOrAmt) . filterQuery (not . queryIsDepth) $ _rsQuery rspec
periodq = Date . periodAsDateSpan $ period_ ropts periodq = Date . periodAsDateSpan $ period_ ropts
-- Except in forecast mode, exclude future/forecast transactions. -- Except in forecast mode, exclude future/forecast transactions.
excludeforecastq (Just _) = Any excludeforecastq (Just _) = Any
@ -107,46 +107,41 @@ accountTransactionsReport rspec@ReportSpec{_rsReportOpts=ropts} j thisacctq = it
And [ Not . Date $ DateSpan (Just . addDays 1 $ _rsDay rspec) Nothing And [ Not . Date $ DateSpan (Just . addDays 1 $ _rsDay rspec) Nothing
, Not generatedTransactionTag , Not generatedTransactionTag
] ]
symq = filterQuery queryIsSym reportq amtq = filterQuery queryIsCurOrAmt $ _rsQuery rspec
realq = filterQuery queryIsReal reportq queryIsCurOrAmt q = queryIsSym q || queryIsAmt q
statusq = filterQuery queryIsStatus reportq
journalValuation = \j' -> journalApplyValuationFromOptsWith rspec j' priceoracle -- Note that within this functions, we are only allowed limited
where priceoracle = journalPriceOracle (infer_value_ $ _rsReportOpts rspec) j -- transformation of the transaction postings: this is due to the need to
-- pass the original transactions into accountTransactionsReportItem.
-- sort by the transaction's register date, for accurate starting balance -- Generally, we either include a transaction in full, or not at all.
transactions = -- Do some limited filtering and valuing of the journal's transactions:
ptraceAtWith 5 (("ts4:\n"++).pshowTransactions) -- - filter them by the account query if any,
. sortOn (Down . transactionRegisterDate reportq thisacctq) -- - discard amounts not matched by the currency and amount query if any,
. jtxns -- - then apply valuation if any.
. ptraceAtWith 5 (("ts3:\n"++).pshowTransactions.jtxns) acctJournal =
-- maybe convert these transactions to cost or value ptraceAtWith 5 (("ts3:\n"++).pshowTransactions.jtxns)
. journalValuation -- maybe convert these transactions to cost or value
-- If we haven't yet filtered by reportq, do so now. . journalApplyValuationFromOpts rspec
$ (if txn_dates_ ropts then id else filterJournalTransactions reportq) journalAcctTxns . ptraceAtWith 5 (("ts2:\n"++).pshowTransactions.jtxns)
-- these are not yet filtered by tdate, we want to search them all for priorps -- apply any cur:SYM filters in reportq
journalAcctTxns = . (if queryIsNull amtq then id else filterJournalAmounts amtq)
-- accountTransactionsReportItem will keep transactions of any date which -- only consider transactions which match thisacctq (possibly excluding postings
-- have any posting inside the report period. -- which are not real or have the wrong status)
-- Should we also require that transaction date is inside the report period ? . traceAt 3 ("thisacctq: "++show thisacctq)
-- Should we be filtering by reportq here to apply other query terms (?) $ ptraceAtWith 5 (("ts1:\n"++).pshowTransactions.jtxns)
-- Make it an option for now. j{jtxns = filter (matchesTransaction thisacctq . relevantPostings) $ jtxns j}
(if txn_dates_ ropts then filterJournalTransactions reportq else id) where
. ptraceAtWith 5 (("ts2:\n"++).pshowTransactions.jtxns) relevantPostings
-- apply any cur:SYM filters in reportq | queryIsNull realq && queryIsNull statusq = id
. (if queryIsNull symq then id else filterJournalAmounts symq) | otherwise = filterTransactionPostings . simplifyQuery $ And [realq, statusq]
-- keep just the transactions affecting this account (via possibly realness or status-filtered postings) realq = filterQuery queryIsReal reportq
. traceAt 3 ("thisacctq: "++show thisacctq) statusq = filterQuery queryIsStatus reportq
. ptraceAtWith 5 (("ts1:\n"++).pshowTransactions.jtxns)
. filterJournalTransactions thisacctq
$ filterJournalPostings (And [realq, statusq]) j
startbal startbal
| balanceaccum_ ropts == Historical = sumPostings priorps | balanceaccum_ ropts == Historical = sumPostings priorps
| otherwise = nullmixedamt | otherwise = nullmixedamt
where where
priorps = dbg5 "priorps" . journalPostings . journalValuation $ priorps = dbg5 "priorps" . journalPostings $ filterJournalPostings priorq acctJournal
filterJournalPostings priorq journalAcctTxns
priorq = dbg5 "priorq" $ And [thisacctq, tostartdateq, datelessreportq] priorq = dbg5 "priorq" $ And [thisacctq, tostartdateq, datelessreportq]
tostartdateq = tostartdateq =
case mstartdate of case mstartdate of
@ -155,37 +150,46 @@ accountTransactionsReport rspec@ReportSpec{_rsReportOpts=ropts} j thisacctq = it
mstartdate = queryStartDate (date2_ ropts) reportq mstartdate = queryStartDate (date2_ ropts) reportq
datelessreportq = filterQuery (not . queryIsDateOrDate2) reportq datelessreportq = filterQuery (not . queryIsDateOrDate2) reportq
items = accountTransactionsReportItems reportq thisacctq startbal maNegate transactions items =
accountTransactionsReportItems reportq thisacctq startbal maNegate
-- sort by the transaction's register date, for accurate starting balance
. ptraceAtWith 5 (("ts4:\n"++).pshowTransactions.map snd)
. sortBy (comparing $ Down . fst)
. map (\t -> (transactionRegisterDate reportq thisacctq t, t))
$ jtxns acctJournal
pshowTransactions :: [Transaction] -> String pshowTransactions :: [Transaction] -> String
pshowTransactions = pshow . map (\t -> unwords [show $ tdate t, T.unpack $ tdescription t]) pshowTransactions = pshow . map (\t -> unwords [show $ tdate t, T.unpack $ tdescription t])
-- | Generate transactions report items from a list of transactions, -- | Generate transactions report items from a list of transactions,
-- using the provided user-specified report query, a query specifying -- using the provided user-specified report query, a query specifying
-- which account to use as the focus, a starting balance, a sign-setting -- which account to use as the focus, a starting balance, and a sign-setting
-- function and a balance-summing function. -- function.
accountTransactionsReportItems :: Query -> Query -> MixedAmount -> (MixedAmount -> MixedAmount) -> [Transaction] -> [AccountTransactionsReportItem] -- Each transaction is accompanied by the date that should be shown for it
-- in the report, which is not necessarily the transaction date; it is
-- the earliest of the posting dates which match both thisacctq and reportq,
-- otherwise the transaction's date if there are no matching postings.
accountTransactionsReportItems :: Query -> Query -> MixedAmount -> (MixedAmount -> MixedAmount)
-> [(Day, Transaction)] -> [AccountTransactionsReportItem]
accountTransactionsReportItems reportq thisacctq bal signfn = accountTransactionsReportItems reportq thisacctq bal signfn =
catMaybes . snd . catMaybes . snd . mapAccumR (accountTransactionsReportItem reportq thisacctq signfn) bal
mapAccumR (accountTransactionsReportItem reportq thisacctq signfn) bal
accountTransactionsReportItem :: Query -> Query -> (MixedAmount -> MixedAmount) -> MixedAmount -> Transaction -> (MixedAmount, Maybe AccountTransactionsReportItem) accountTransactionsReportItem :: Query -> Query -> (MixedAmount -> MixedAmount) -> MixedAmount
accountTransactionsReportItem reportq thisacctq signfn bal torig = balItem -> (Day, Transaction) -> (MixedAmount, Maybe AccountTransactionsReportItem)
accountTransactionsReportItem reportq thisacctq signfn bal (d, torig)
-- 201407: I've lost my grip on this, let's just hope for the best -- 201407: I've lost my grip on this, let's just hope for the best
-- 201606: we now calculate change and balance from filtered postings, check this still works well for all callers XXX -- 201606: we now calculate change and balance from filtered postings, check this still works well for all callers XXX
| null reportps = (bal, Nothing) -- no matched postings in this transaction, skip it
| otherwise = (b, Just (torig, tacct{tdate=d}, numotheraccts > 1, otheracctstr, a, b))
where where
tacct@Transaction{tpostings=reportps} = torig{tdate=transactionRegisterDate reportq thisacctq torig} tacct@Transaction{tpostings=reportps} = filterTransactionPostings reportq torig
balItem = case reportps of (thisacctps, otheracctps) = partition (matchesPosting thisacctq) reportps
[] -> (bal, Nothing) -- no matched postings in this transaction, skip it numotheraccts = length $ nub $ map paccount otheracctps
_ -> (b, Just (torig, tacct, numotheraccts > 1, otheracctstr, a, b)) otheracctstr | thisacctq == None = summarisePostingAccounts reportps -- no current account ? summarise all matched postings
where | numotheraccts == 0 = summarisePostingAccounts thisacctps -- only postings to current account ? summarise those
(thisacctps, otheracctps) = partition (matchesPosting thisacctq) reportps | otherwise = summarisePostingAccounts otheracctps -- summarise matched postings to other account(s)
numotheraccts = length $ nub $ map paccount otheracctps a = signfn . maNegate $ sumPostings thisacctps
otheracctstr | thisacctq == None = summarisePostingAccounts reportps -- no current account ? summarise all matched postings b = bal `maPlus` a
| numotheraccts == 0 = summarisePostingAccounts thisacctps -- only postings to current account ? summarise those
| otherwise = summarisePostingAccounts otheracctps -- summarise matched postings to other account(s)
a = signfn . maNegate $ sumPostings thisacctps
b = bal `maPlus` a
-- | What is the transaction's date in the context of a particular account -- | What is the transaction's date in the context of a particular account
-- (specified with a query) and report query, as in an account register ? -- (specified with a query) and report query, as in an account register ?

6
hledger/test/journal/amounts-and-commodities.test
View File

@ -324,10 +324,8 @@ $ hledger -f- register -V -B cur:A "amt:<5"
2021-01-01 (a) 1000 B 1000 B 2021-01-01 (a) 1000 B 1000 B
>= >=
# 24. aregister report cur: and amt: query matches currency before # 24. aregister report cur: and amt: query matches currency before valuation/cost
# valuation/cost (note that aregister currently only tests queries other than $ hledger -f- aregister a -V -B cur:A "amt:<5"
# "cur:" and the account query when given the --txn-dates option. Who knows why.
$ hledger -f- aregister a -V -B cur:A "amt:<5" --txn-dates
Transactions in a and subaccounts: Transactions in a and subaccounts:
2021-01-01 a 1000 B 1000 B 2021-01-01 a 1000 B 1000 B
>= >=