This can be useful to override defaults in scripts.
These flags will now toggle when repeated on the command line:
--invert
--transpose
-r/--related
-%/--percent
-E/--empty
-N/--no-total
-T/--row-total
-A/--average
-S/--sort-amount
Since hledger 1.25, "every Nth day of month" period rules with N > 28
could be off by a couple of days if given certain forecast start dates.
Eg `~ every 31st day of month` with `--forecast='2023-03-30..'`.
Breaking change: previously timeclock descriptions could contain
semicolons. Now a semicolon in the description will end it and
start a comment (which may contain tags).
I found at least one user for whom this would be a breaking change
(they generate forecast txns, and have auto posting rules, but don't
want the latter applied to the former). I guess it's better to keep
things as they were for now: if you need auto postings on your
forecast txns you must use two flags, --forecast --auto.
When doing a type: match we now also check the original unaliased,
unpivoted posting, as when doing an acct: match. This is effectively
how things worked with the older account type detection in hledger <1.27.
Previously, similarity completely outweighed recency, so a
slightly-more-similar transaction would always be selected no matter
how old it was. Now similarity and recency are more balanced,
and it should produce the desired transaction more often.
There is also new debug output (at debug level 1) for
troubleshooting.
Boolean queries are now prefixed with an 'expr:' prefix, making them
completely separable from old queries and making the addition of them a
little more migration proof.
The tests are updated accordingly, changes made to the tests previously
are removed and extra cautious documentation is also removed.
This commit changes some of the functions in the Query module and
changes the overall way to parse queries. Instead of using the words''
split function, this commit starts to fully parse the query, as it's
seen as a type of expression.
AND, OR, NOT, and space operators can be used. The space operator
simulates the behaviour from before, leaving a minimal amount of tests
that need to be adjusted to comply to the new behaviour.
Eg, where previously -p 'monthly from 1/15' or -M -b 1/15 would always
adjust the report start date to 1/1,
unless you used the special -p 'every 15th day of month from 1/15' form,
now the start date will not be adjusted. (It is still adjusted if
the report date is not specified explicitly, eg inferred from the journal).
This keeps behaviour consistent between report periods and periodic transactions.
'in' period expressions, like 'in 2023-01', are a grey area; they
do specify a start date (2023-01-01), although they look a bit implicit.
So previously, -p 'weekly in 2023-01' would adjust the start date to
the preceding monday (2022-12-26), but now it will start exactly on
2023-01-01 (a sunday, which also causes ugly verbose column headings).
To ensure monday based weeks and simple report headings here,
you would have to explicitly specific a start date that is a monday,
eg -p 'weekly from 2022-12-26 to 2023-02'.
You can now write both @/@@ costs and corresponding equity conversion postings
in a transaction at any time, not just when using --infer-costs or --infer-equity.
hledger will recognise the redundancy and ignore it.
One thing has become more strict: hledger now requires conversion postings
to occur in adjacent pairs; an odd number of them is not allowed.
(Conversion postings are postings to accounts of type `V`/`Conversion`,
or named `equity:conversion`, `equity:trade`, `equity:trading`,
or subaccounts of these.)
And, --infer-costs now works in transactions with an implicit amount
(inferring costs from equity now happens after transaction balancing,
not before).
Previously, the accounts passed to account directives would be stripped
of their surrounding brackets, but the required behaviour is to have
account directives plain reject bracketed accounts. This change ensures
that accounts in account directives may not start with a bracket
character.
Currently an account name like "a:(aa)" will not have (aa) unbracketed.
However, this seems reasonable since the full name is unbracketed and
thus will not be confused with virtual or virtual-balanced posting.
"" is equivalent to writing no symbol.
But it's now possible to declare market prices for the no-symbol
commodity, which occurs eg with timedot data.
P 2022-01-01 "" $100
Hledger.Utils.Debug's "trace or log" functions are now controlled as
follows: to enable logging, append ",logging" to the program name at
startup (using withProgName). This also works when running in GHCI.
And they log to PROGNAME.log, not debug.log.
All (hopefully) debug logging in the hledger packages is now "trace or
log" capable.
This means that hledger-ui should now log all debug output to
./hledger-ui.log, with none of it appearing on the console.
Moved from Hledger.Utils.Debug to Hledger.Utils.Print:
pshow
pshow'
pprint
pprint'
colorOption
useColorOnStdout
useColorOnStderr
outputFileOption
hasOutputFile
Previously, CSV date-times with a different time zone from yours
(with or without explicit timezones in the CSV) could give off-by-one
dates, because the CSV timezone was ignored.
Now,
1. you can use the `timezone` rule to indicate which other
timezone a CSV is implicitly using
2. CSV date-times with a timezone - whether declared by rule or
parsed with %Z - are localised to the system time zone
(or another set with the TZ environment variable).
dlog has been replaced by more reliable functions for debug-logging
to a file, useful for debugging TUI apps like hledger-ui:
dlogTrace
dlogTraceAt
dlogAt
dlog0
dlog1
dlog2
dlog3
dlog4
dlog5
dlog6
dlog7
dlog8
dlog9
Monochrome pprint' and pshow' have been added.
New dependency: deepseq
This replaces the old journal*AccountQuery with the new Type query. This
enables uniform treatment of account type, and fixes a subtle bug
(#1921).
Note that cbcsubreportquery no longer takes Journal as an argument.
hledger check recentassertions (or flycheck-hledger if you enable this
check) will complain if any balance-asserted account does not have a
balance assertion within 7 days before its latest posting. This aims
to prevent the situation where you are regularly updating your
journal, but forgetting to check your balances against the real world,
eventually requiring you to dig back through months of data to find
the error.
transaction prices.
When given --infer-costs, hledger will now separately infer transaction
prices for different prices. Given a pair of adjacent conversion
postings, hledger will check if there is a single posting with a
transaction price which matches both the amounts. If so, it associates
those conversion postings to that priced post.
If it can't find any transaction price postings which match, it will
find the first non-transaction price post which matches one of the two
amounts, and will add a transaction price to that, and associate them.
When given --infer-equity, hledger will change transaction prices to balancing equity postings. This introduces the inverse operation, --infer-costs, which will match balancing equity postings and transform them into a transaction price, allowing --cost to work properly with them. This is only a partial inverse as it needs to use some heuristics to match the postings which will not work in complicated cases.
Specifically, when hledger finds exactly two conversion postings in a transaction (by default, subaccounts of equity:conversion or equity:trad(e|ing)), it will find the first posting in the transaction whose amount is negative one of the conversion posting amounts, and inserts the corresponding transaction price.
May also fix#1154, #1033, #708, #536, #73: testing is needed.
This aims to solve all problems where misconfigured locales lead to
parsers failing on utf8-encoded data. This should hopefully avoid
encoding issues, but since it fundamentally alters how encoding is dealt
with it may lead to unexpected outcomes. Widespread testing on a number
of different platforms would be useful.
And remove the last vestiges of older more complex behaviour.
ordereddates now always checks all transactions in each file,
unaffected by a query. (But still affected by --date2).
By default, all account declarations and all transactions are searched;
but when there's a query involving transaction fields,
account declarations unrelated to the matched transactions are not searched.
added:
queryIsCode
queryIsTransactionRelated
parseAndFinaliseJournal' has been removed. In the unlikely event you
needed it in your code, you can replace it with:
parseAndFinaliseJournal' parser iopts fp t =>
initialiseAndParseJournal parser iopts fp t
>>= liftEither . journalApplyAliases (aliasesFromOpts iopts)
>>= journalFinalise iopts fp t
Some parsers have been generalised from JournalParser to TextParser.
This increases composability and avoids some ugly case handling. We
re-export runExceptT in Hledger.Read.
The final return types of the following functions has been changed from
IO (Either String a) to ExceptT String IO a. If this causes a problem,
you can get the old behaviour by calling runExceptT on the output:
readJournal, readJournalFiles, readJournalFile
Or, you can use the easy functions readJournal', readJournalFiles', and
readJournalFile', which assume default options and return in the IO
monad.
total prices. (#1813)
Also reduce duplication for inferring market prices (previously it was
done separately in both Hledger.Data.Journal and
Hledger.Cli.Commands.Prices), and remove *TotalPriceToUnitPrice
functions, since unit prices cannot represent all total prices.
Add a helper function numDigitsInt to get the number of digits in an
integer, which has a surprising number of ways to get it wrong.
This results in big speedups in cases when we have many transaction prices, like in examples/10000x1000x10.journal. This can be disabled with the show_costs_ option in ReportOpts.
^assets?(:.+)?:(cash|bank|che(ck|que?)(ing)?|savings?|current)(:|$)
This should have no effect outside the cash report.
The rule for auto-detecting "Cash" (liquid assets) from account names
has changed in this release - see [Account types]. This is to simplify
how account types are detected and configured. If you use the cashflow
report without explicitly declaring accounts as Cash, you might notice a
change, and might need to add explicit account type: tags.
This requires checking parent accounts for any new accounts introduced by auto postings which do not exist in the original journal.
Also refactor journalFinalise to only call journalPostingsAddAccountTags once, and use fewer intermediate variables.
type:TYPES, where TYPES is any of the (case insensitive) letters
ALERXCV, matches accounts by their declared or inferred type.
(See https://hledger.org/hledger.html#account-types.)
This should work with most commands, eg:
hledger bal type:al
hledger reg type:x
API changes:
Journal has a new jaccounttypes map.
The journalAccountType lookup function makes it easy to check an account's type.
The journalTags and journalInheritedTags functions look up an account's tags.
Functions like journalFilterPostings and journalFilterTransactions,
and new matching functions matchesAccountExtra, matchesPostingExtra
and matchesTransactionExtra, use these to allow more powerful matching
that is aware of account types and tags.
Accounts, postings, and transactions can now all be filtered by the
tags in an account's declaration. In particular it's now possible to
more reliably select accounts by type, using their type: tag rather
than their name:
account myasset ; type:Asset
account myliability ; type:Liability
$ hledger accounts tag:type=^a
myasset
Accounts inherit tags from their parents.
API changes:
A finalised Journal has a new jdeclaredaccounttags field
for easy lookup of account tags.
Query.matchesTaggedAccount is a tag-aware version of matchesAccount.
directives.
Previously, you declare an account type with the following format:
account assets A
This has been deprecated since 1.13, and should now be declared with one
of:
account assets ; type:A
account assets ; type:asset
Previously the helper functions splitspan and splitspan' would calculate
each span from the start point of the previous span. This meant we had
to be very careful not to lose any relevant information (e.g. what day
of the week it was) about the original start date. We now calculate each
span from the original start date, so there's no risk of losing
information. This simplifies many of the calculations.
parsers to allow for arbitrary numbers of periods in relative dates.
We now accept smart dates like “in 5 days, 5 weeks ahead, in -6 months, 2 quarters ago”.
Introduce --infer-equity option which will generate conversion postings.
--cost will override --infer-equity.
This means there will no longer be unbalanced transactions, but will be
offsetting conversion postings to balance things out. For example.
2000-01-01
a 1 AAA @@ 2 BBB
b -2 BBB
When converting to cost, this is treated the same as before.
When used with --infer-equity, this is now treated as:
2000-01-01
a 1 AAA
equity:conversion:AAA-BBB:AAA -1 AAA
equity:conversion:AAA-BBB:BBB 2 BBB
b -2 BBB
There is a new account type, Conversion/V, which is a subtype of Equity/E.
The first account declared with this type, if any, is used as the base account
for inferred equity postings in conversion transactions, overriding the default
"equity:conversion".
API changes:
Costing has been changed to ConversionOp with three options:
NoConversionOp, ToCost, and InferEquity.
The first correspond to the previous NoCost and Cost options, while the
third corresponds to the --infer-equity flag. This converts transactions with costs
(one or more transaction prices) to transactions with equity:conversion postings.
It is in ConversionOp because converting to cost with -B/--cost and inferring conversion
equity postings with --infer-equity are mutually exclusive.
Correspondingly, the cost_ record of ReportOpts has been changed to
conversionop_.
This also removes show_costs_ option in ReportOpts, as its functionality
has been replaced by the richer cost_ option.
Together with -E, this shows a balance for both used and declared
accounts (excluding empty parent accounts, which are usually not
wanted in list-mode reports).
This is somewhat consistent with --declared in the accounts and payees
commands, except for the leaf account restriction.
The idea of this is to be able to see a useful "complete" balance
report, even when you don't have transactions in all of your declared
accounts yet. I mainly want this for hledger-ui, but there's no harm
in exposing it in the balance CLI as well.
Together with -E, this allows showing a balance for all accounts, both
used and declared. I mainly want this for hledger-ui, but there's no
harm in exposing it in the balance command as well. This is somewhat
consistent with the accounts and payees commands.
This allows more control over how multicommodity amounts are displayed.
In addition to the default single-line display, and the recent commodity
column display, we now have multi-line display. This is controlled by
the --layout option, which has possible values "wide", "tall", and
"bare". The --commodity-column option has been hidden, but is equivalent
to --layout=bare.
squash
Replace showPosting with a wrapper around postingAsLines.
The functions textConcat(Top|Bottom)Padded are no longer used anywhere
in the code base, and can be removed if desired.
This produces slightly different output for showPosting, in particular
it no longer displays the transaction date. However, this has been
marked as ‘for debugging only’ for a while, and is only used in
hledger-check-fancy assertions. The output there is still acceptable.
Combining valuation with filtration is subtle and error-prone (see e.g. #1625).
We have to do in in both MultiBalanceReport and PostingsReport, where it
is done in slightly different ways. This refactors this functionality
into separate functions which are called in both reports, for uniform
behaviour.
(SourcePos, SourcePos).
This has been marked for possible removal for a while. We are keeping
strictly more information. Possible edge cases arise with Timeclock and
CsvReader, but I think these are covered.
The particular motivation for getting rid of this is that
GenericSourcePos is creating some awkward import considerations for
little gain. Removing this enables some flattening of the module
dependency tree.
Hledger.Data.Balancing.
Both Hledger.Data.Transaction and Hledger.Data.Journal are massive
module with many things in them. Placing the balancing functions, which
are conceptually related, into a separate module helps keep things more
modular.
It also reduces the risk of import cycles, as right now balancing
functions cannot depend on any functions defined outside of
Hledger.Data.Transaction or Hledger.Data.Journal, respectively, if those
modules require basic transaction or journal functions.
mixedAmount(Looks|Is)Zero now operate directly on the MixedAmount,
rather than converting them to a list of amounts first.
mixedAmountCost no longer reconstructs the entire MixedAmount when there
are amounts with no cost.
transactionCheckBalanced only checks if signs are okay if sums are not
okay. It also only traverses the list of postings once when picking real
and balanced virtual postings.
rawOptsTo* in hledger-lib now takes a day as an argument, and does not
live in the IO monad, since it's now pure.
This is so that we can run tests containing future transactions that
won't fail as soon as ‘the future’ actually arrives.
There are no modules which depend on Hledger.Data.Commodity which don't
also depend on Hledger.Data.Amount. Though Hledger.Data.Amount is a very
large module and might be broken up, Hledger.Data.Commodity only defines
three very small functions which are used, and so can be combined with
little cost.
The forecast period begins on:
- the start date supplied to the `--forecast` argument, if present
- otherwise, the later of
- the report start date if specified with -b/-p/date:
- the day after the latest normal (non-periodic) transaction in the journal, if any
- otherwise today.
It ends on:
- the end date supplied to the `--forecast` argument, if present
- otherwise the report end date if specified with -e/-p/date:
- otherwise 180 days (6 months) from today.
Note that the previous behaviour did not quite match the documentation,
so this also acts as a bug fix for #1665.
POSIXTime.
This eliminates old-time, which has been deprecated for a while, from
our dependencies.
This introduces a slight incompatibility, as a small number of functions
now take/return POSIXTime instead of ClockTime. Generally you will be
using the current time, in which case you should use getPOSIXTime from
Data.Time.Clock.POSIX instead of getClockTime.
utcTimeToClockTime has been removed, as it is now equivalent to
utcTimeToPOSIXSeconds from Data.Time.Clock.POSIX.
We can't filter out empty commodity strings since that is a legitimate
group. Simultaneously, we should only include the empty commodity if it
is explicitly used (part of a posting) and not generated as part of
`Amounts.amounts`
A gain report will report on unrealised gains by looking at the
difference between the valuation of an amount (by default, --value=end),
and the valuation of the cost of the amount.
Add documentation and sample output for `--commodity-column` behavior
and functional tests e.g single-period balance, yearly balance, and
yearly budget
This adds the `--commodity-column` option that displays each commodity
on a separate line and the commodities themselves as a separate column.
The initial design considerations are at
simonmichael.hledger.issues.1559
The single-period balance report with `--commodity-column` does not
interoperate with custom formats.
rather than as a postprocessing step. (#1638)
This allows us to have a uniform procedure for balancing transactions,
whether they are normal transactions or forecast transactions, including
dealing with balance assignments, balance assertions, and auto postings.
To get the previous behaviour, you can call with the additional query "not📅tomorrow-".
This is because we are preparing to move forecast_ to InputOpts, where
it is not accessible to AccountTransactionsReport.
We officially support GHC 8.6+ (and 8.8+ for hledger-web) now.
Hackage matrix builder shows all packages building successfully
with GHC 8.4+, somehow, so we'll adjust the base bound to
allow that but prevent any attempts to build with older GHCs,
AccountTransactionsReport.
Only a limited number of journal transformations are allowed in
accountTransactionReports due to the need to include the original
transaction. Document these.
Make sure to remove currency and amount queries from the reportq, so
they do not cause problems after valuation.
Avoid confusion by calculating transaction date at one point, and
passing that down with the transaction.
valuation after all reportq filtering is done (in both the historical
and recent postings), and make sure filtering is done at the correct
point based on whether --txn-dates is set. (#1634)
style amounts according to that argument. journalAddForecast and
journalTransform now return an Either String Journal.
This improves efficiency, as we no longer have to restyle all amounts in
the journal after generating auto postings or periodic transactions.
Changing the return type of journalAddForecast and journalTransform
reduces partiality.
To get the previous behaviour for modifyTransaction, use modifyTransaction mempty.
The only semantic difference is that we now apply
journalApplyCommodityStyles before running journalCheckAccountsDeclared
and journalCheckCommoditiesDeclared.
