dropped journalPrices
renamed Price to AmountPrice, AKA "transaction price"
renamed MarketPrice to PriceDirective.
added new MarketPrice (more pure form of PriceDirective without the amount style information)
Prices is now a more efficient data structure, but not used yet.
This feature turns out to be quite involved, as valuation interacts
with the many report variations. Various bugs/specs have been
fixed/clarified relating to register's running total, balance totals
etc. Eg register's total should now be the sum of the posting amount
values, not the values of the original sums. Current level of support
has been documented.
When valuing at transaction date, we once again do early valuation of
all posting amounts, to get more correct results. variants. This means
--value-at=t can be slower than other valuation modes when there are
many transactions and many prices. This could be revisited for
optimisation when things are more settled.
Instead of converting all journal amounts to value early on, we now
convert just the report amounts to value, before rendering.
This was basically how it originally worked (for the balance command),
but now it's built in to the four basic reports used by print,
register, balance and their variants - Entries, Postings, Balance,
MultiBalance - each of which now has its own xxValue helper.
This should mostly fix -V's performance when there are many
transactions and prices (the price lookups could still be optimised),
and allow more flexibility for report-specific value calculations.
+------------------------------------------++-----------------+-------------------+--------------------------+
| || hledger.999.pre | hledger.999.1sort | hledger.999.after-report |
+==========================================++=================+===================+==========================+
| -f examples/1000x1000x10.journal bal -V || 1.08 | 0.96 | 0.76 |
| -f examples/2000x1000x10.journal bal -V || 1.65 | 1.05 | 0.73 |
| -f examples/3000x1000x10.journal bal -V || 2.43 | 1.58 | 0.84 |
| -f examples/4000x1000x10.journal bal -V || 4.39 | 1.96 | 0.93 |
| -f examples/5000x1000x10.journal bal -V || 7.75 | 2.99 | 1.07 |
| -f examples/6000x1000x10.journal bal -V || 11.21 | 3.72 | 1.16 |
| -f examples/7000x1000x10.journal bal -V || 16.91 | 4.72 | 1.19 |
| -f examples/8000x1000x10.journal bal -V || 27.10 | 9.83 | 1.40 |
| -f examples/9000x1000x10.journal bal -V || 39.73 | 15.00 | 1.51 |
| -f examples/10000x1000x10.journal bal -V || 50.72 | 25.61 | 2.15 |
+------------------------------------------++-----------------+-------------------+--------------------------+
There's one new limitation, not yet resolved: -V once again can pick a
valuation date in the future, if no report end date is specified and
the journal has future-dated transactions. We prefer to avoid that,
but reports currently are pure and don't have access to today's date.
-V is still quite a bit slower than no -V, but not as much as before:
+===========================================================++=======+
| hledger.999.pre -f examples/10000x10000x10.journal bal || 5.20 |
| hledger.999.pre -f examples/10000x10000x10.journal bal -V || 57.20 |
| hledger.999 -f examples/10000x10000x10.journal bal || 5.34 |
| hledger.999 -f examples/10000x10000x10.journal bal -V || 17.50 |
+-----------------------------------------------------------++-------+
I needed to be more careful about ordering, as johannesgerer's original
code was, and the tests missed it. I think I have it now.
Found the PR whose code I have been reworking, it was #438.
Going with option 1b from the issue: calculated and asserted amounts
are compared exactly, disregarding display precision.
But now balance assertion failure messages show those exact amounts at
full precision, avoiding confusion.
Surprisingly, balance assertions were checking to maximum precision,
which meant it was possible, with a display-precision-limiting
commodity directive, to have a failing assertion with the error
message showing asserted and actual amounts that looked the same.
Now we round the calculated account balance (but not the asserted
balance) to display precision before comparing. This should ensure
assertions always behave as you would expect from visual inspection.
This should eventually include accountnames from transaction
modifiers (if `--auto` is enabled), or periodic transactions (if
`--forecast` is enabled).
A different approach: instead of converting to unit prices and fiddling
with the display precision, just multiply the total prices by the same
multiplier (and keep them positive).
This seems a little more natural. I'm not sure if one of these will be
more robust than the other.
Divide/multiply amounts *and* their total price, if they have one.
Helpful for keeping transactions balanced when transaction modifiers are
multiplying amounts.
Transaction modifier multipliers have never multiplied total-priced amounts
correctly (and prior to hledger 1.10, this could generate unbalanced
transactions).
Now, the generated postings in this situation will have unit prices,
and an extra digit of display precision. This helps ensure that
the modified transaction will remain balanced. I'm not sure yet if
it's guaranteed.
Noticed by peti: showTransaction could sometimes hide the last posting's
amount even if one of the other posting amounts was already implcit,
producing invalid transaction output.
* journal: Get rid of `journalFinalise` and use granular functions
Complete the process started in 53b3e2bd. This gets rid of the
`journalFinalise` function and uses the smaller steps, in order to
have more granular control.
* journal: Change order of operations in finalization
We want to make sure that we add the filepath after the order is
reversed, so the added filepath is on the head and not the tail (as it
would be if it were reversed after it was added).
* journal: Refine granular finalization functions
This commit fixes two of the granular finalization functions:
1. Rename `journalSetTime` to `journalSetLastReadTime` and improve
documentation.
2. Remove `journalSetFilePath`. It's redundant with `journalAddFile`
currently in `Hledger.Read.Common`. The only difference between the
functions is where the file is added (we keep the one in which it
is added to the tail), so we change the position vis-a-vis
reversal.
`journalFinalise` is only used in the `parseAndFinaliseJournal`
functions, but it needs to be run differently at different stages when
transaction modifiers are applied. This change breaks it into smaller
functions, and uses those smaller parts in `parseAndFinaliseJournal`
as needed.
Currently `journalFinalise` always reverses the order of
entries. However, if there are automated transactions, we might need
to run it twice. This adds a boolean flag to make reordering
optional. This will be used in the `parseAndFinaliseJournal`
functions.
These commands now detect the account types declared by account directives.
Whenever such declarations are not present, built-in regular expressions
are used, as before.
Previously you had to use one of the standard english account names
(assets, liabilities..) for top-level accounts, if you wanted to use
the bs/bse/cf/is commands.
Now, account directives can specify which of the big five categories
an account belongs to - asset, liability, equity, revenue or expense -
by writing one of the letters A, L, E, R or X two or more spaces after
the account name (where the numeric account code used to be).
This might change. Some thoughts influencing the current syntax:
- easy to type and read
- does not require multiple lines
- does not depend on any particular account numbering scheme
- allows more types later if needed
- still anglocentric, but only a little
- could be treated as syntactic sugar for account tags later
- seems to be compatible with (ignored by) current Ledger
The current design permits unlimited account type declarations anywhere
in the account tree. So you could declare a liability account somewhere
under assets, and maybe a revenue account under that, and another asset
account even further down. In such cases you start to see oddities like
accounts appearing in multiple places in a tree-mode report. In theory
the reports will still behave reasonably, but this has not been tested
too hard. In any case this is clearly too much freedom. I have left it
this way, for now, in case it helps with:
- modelling contra accounts ?
- multiple files. I suspect the extra expressiveness may come in handy
when combining multiple files with account type declarations,
rewriting account names, apply parent accounts etc.
If we only allowed type declarations on top-level accounts, or
only allowed a single account of each type, complications seem likely.
- In anticipation of megaparsec 7, which removes support for stacks of
include files (as far as I can tell)
- Intended for the 'StateT Journal' layer of the parser
- A stack of include files would be better in a 'ReaderT' layer, but I
don't want to add another layer to the parser
- Intended for detecting cycles of include files
- Potential issue: for proper error messages for include file cycles,
we must remember to provide the filepath of the root journal file via
the initial journal state passed to a 'JournalParser'; I imagine
that we may forget to do so because in all other cases it is okay
not to do so.
A bunch of account sorting changes that got intermingled.
First, account codes have been dropped. They can still be parsed and
will be ignored, for now. I don't know if anyone used them.
Instead, account display order is now controlled by the order of account
directives, if any. From the mail list:
I'd like to drop account codes, introduced in hledger 1.9 to control
the display order of accounts. In my experience,
- they are tedious to maintain
- they duplicate/compete with the natural tendency to arrange account
directives to match your mental chart of accounts
- they duplicate/compete with the tree structure created by account
names
and it gets worse if you think about using them more extensively,
eg to classify accounts by type.
Instead, I plan to just let the position (parse order) of account
directives determine the display order of those declared accounts.
Undeclared accounts will be displayed after declared accounts,
sorted alphabetically as usual.
Second, the various account sorting modes have been implemented more
widely and more correctly. All sorting modes (alphabetically, by account
declaration, by amount) should now work correctly in almost all commands
and modes (non-tabular and tabular balance reports, tree and flat modes,
the accounts command). Sorting bugs have been fixed, eg #875.
Only the budget report (balance --budget) does not yet support sorting.
Comprehensive functional tests for sorting in the accounts and balance
commands have been added. If you are confused by some sorting behaviour,
studying these tests is recommended, as sorting gets tricky.
Custom Show instances were obscuring important details in test failure
output again. The best policy seems to be: stick with default derived
Show instances as far as possible, but when necessary customize them
to conform to haskell syntax so pretty-show can do its thing (eg when
they contain Day values, cf https://github.com/haskell/time/issues/101).
This makes skipping/unskipping tests easier, and improves readability
a bit.
Note it's also possible to just write the test name with no preceding
function, when the type is constrained (see Journal.hs).
Amount's default show instance hid important details, making eg test
failures hard to understand. Showing full detail required increasing
the debug level which was inconvenient.
Now it has a single show instance which shows more information, is
fairly compact, and is pretty-printable with pretty-show.
Ellipses (..) in the output indicate where fields are
- not shown in full detail, and/or
- shown in pseudo syntax (double quoted) to work with pretty-show.
ghci> usd 1
OLD:
Amount {acommodity="$", aquantity=1.00, ..}
NEW:
Amount {acommodity = "$", aquantity = 1.00, aprice = NoPrice, astyle = AmountStyle "L False 2 Just '.' Nothing..", amultiplier = False}
MixedAmount's show instance is unchanged, but showMixedAmountDebug
is affected by this change:
ghci> putStrLn $ showMixedAmountDebug $ Mixed [usd 1]
OLD:
Mixed [Amount {acommodity="$", aquantity=1.00, aprice=, astyle=AmountStyle {ascommodityside = L, ascommodityspaced = False, asprecision = 2, asdecimalpoint = Just '.', asdigitgroups = Nothing}}]
NEW:
Mixed [Amount {acommodity="$", aquantity=1.00, aprice=, astyle=AmountStyle "L False 2 Just '.' Nothing.."}]
This removes transactionModifierToFunction's extra query parameter;
the rewrite command sets it in the TransactionModifier instead, which
I think is equivalent. I had to change one functional test, but it
seems correct now, so perhaps it wasn't working right before ?
For Data/Dates.hs in particular:
- Changed `SimpleTextParser` to `TextParser m` for all parsers
- Changed `string` to the case-insensitive `string'` to match the
behaviour of `T.toLower` found in `parsePeriodExpr`
- export `periodexprp` for "direct" use
base-compat-batteries provides the same API across more ghc versions
than base-compat does, at the cost of more dependencies. Eg it exports
Prelude.Compat ((<>)) with ghc 7.10/base 4.8, which we expect.
My belief is that several of our deps already require it so the added
cost is not too great. We should probably go back to base-compat when
possible though, eg when we stop supporting ghc 7.10.
The new version of our package set apparently contains both base-compat and
base-compat-batteries in its transitive closure. This breaks the doctest suite,
which just imports everything into scope when the tests are run, thereby making
module names like Prelude.Compat ambiguous.
It now works slightly differently. Eg:
- <unbudgeted>'s subaccounts are hidden by default
- --show-unbudgeted shows all unbudgeted accounts, including subaccounts of budgeted parents
- --show-unbudgeted doesn't affect the grouping under <unbudgeted>
IMHO it's a nice simplification and increase in consistency, while still meeting the original intent.
A commodity directive that doesn't specify the decimal point character
increases ambiguity and the chance of misparsing numbers, especially
as it overrides all style information inferred from the journal amounts.
In some cases it caused amounts with a decimal point to be parsed as if
with a digit group separator so 1.234 became 1234.
We could augment it with extra info from the journal amounts, when available,
but it would still be possible to be ambiguous, and that won't be obvious.
A commodity directive is what we recommend to nail down the style.
It seems the simple and really only way to do this reliably is to require
an explicit decimal point character. Most folks probably do this already.
Unfortunately, it makes another potential incompatiblity with ledger and
beancount journals. But the error message will be clear and easy to
work around.
Inferred amounts now have the appropriate standard amount style applied.
And when checking for balanced transactions, amount styles declared with
commodity directives are also used (previously only inferred amount styles were).
Now we have:
-- | Sorted unique account names posted to by this journal's transactions.
journalAccountNamesUsed :: Journal -> [AccountName]
-- | Sorted unique account names implied by this journal's transactions -
-- accounts posted to and all their implied parent accounts.
journalAccountNamesImplied :: Journal -> [AccountName]
-- | Sorted unique account names declared by account directives in this journal.
journalAccountNamesDeclared :: Journal -> [AccountName]
-- | Sorted unique account names declared by account directives or posted to
-- by transactions in this journal.
journalAccountNamesDeclaredOrUsed :: Journal -> [AccountName]
-- | Sorted unique account names declared by account directives, or posted to
-- or implied as parents by transactions in this journal.
journalAccountNamesDeclaredOrImplied :: Journal -> [AccountName]
-- | Convenience/compatibility alias for journalAccountNamesImpliedOrUsed.
journalAccountNames :: Journal -> [AccountName]
accountsFromPostings is currently doing excessive work when adding up
postings in each account. It sorts (accountName, amount) tuples which
cause amounts in them to be compared. There is no need to look at amount
here at all since subsequent summing up and counting does not depend on
order. It is enough to sort by accountname only.
Went through similar pieces of code, made them all look uniform.
- Simplify doctests for periodexpr.
- Besides consuming leading space consume ending space for periodexpr also.
- Drop implicit option (def, def) behaviour of periodexpr. I.e. disallow
hledger reg -p '' and auto-transaction with heading just '~'.
- Slightly re-factor periodexpr.
- Ensure that reportinginterval doesn't consume trailing space.
Useful if we'll start disallowing periods like "every1stjan2009-".
This is very helpful for periodic transactions, because in budget mode
you need to ensure that no periodic transactions extend past the end
of the journal, and in forecast mode you need to make sure that all
periodic transactions are strictly after the end of the journal.
Some of these demonstrate that runPeriodicTransaction could generate
transactions ouside of requested DateSpan. This happens because
runPeriodicTransaction uses splitSpan internally, and splitSpan always
generates dateSpans that fully cover original DateSpan, extending
beyound left/right boundary if necessary. This is ok if transactions
are generated for budgeting purpose, but during forecasting care should
be taken to check that all generated transactions are happening past
the end of the real journal.
Currently only lower-case account names are supported, both on the
command line, and in the journal (in periodic transactions):
This works:
$ hledger balance -p nov
This does not:
$ hledger balance -p Nov
First transaction will parse, second will not:
```
cat every-month.journal ~/devel/haskell/darcs-get/hledger/examples
~ aug to sep
assets
expenses $1
~ Aug to Sep
assets
expenses $2
```
$../bin/hledger-budget bal -f every-month.journal
hledger-budget: Failed to parse "Aug to Sep": date parse error ()
This commit fixes both cases.
Add "fixed" to the regex used to exclude asset accounts from cashflow
reports. Assumes accounts are set up something like:
assets:fixed assets:equipment
assets:fixed assets:vehicles
assets:fixed assets:buildings
Older megaparsec is still supported.
Also cleans up our custom parser types,
and some text (un)packing is done in different places
(possible performance impact).
The new entry effectively adds a loan which is placed in the checking account.
This loan is then closed by the "pay off" transaction (which was already
present).
This is mainly to be used as a test point for the -H option; to make
sure -H does not show empty accounts.
All previous tests were changed to reflect the new change.
The documentation of the journal module was updated too.
See the issue and linked mail list discussion. Ambiguity between the
uncleared state, and the "not cleared" --uncleared flag causes confusion
and friction. At this point it seems best to break with Ledger and
past hledger, pick a new name and drop --uncleared to put an end to it.
Related to #563, when rendering a transaction, we reserve two more chars
of width so that amounts remain aligned when there are posting flags.
Affects hledger-ui's transaction screen, print, hledger-rewrite etc.
This affects hledger-ui's transaction screen, print, hledger-rewrite etc.
A pending flag on postings is now displayed, just like a cleared flag.
Also there will now be a space between flag and account name.
And it's slightly better at aligning amounts, due to now considering
virtual posting parentheses/brackets or something.
When generating a new posting as a multiple of an existing posting,
support conversion to a different commodity. For example, postings in
hours can be used to generate postings in USD.
Automatic transactions generated from rewrite rules use the commodity,
amount style, and transaction price if the rewrite defines a commodity.
The balance command now shows negative amounts in red, when it thinks
ANSI codes are supported, ie when TERM is not "dumb" and stdout is not
being redirected or piped somewhere.
* added showMarketPrice and Hledger.Data.MarketPrice module
* showMarketPrice implemented using showDate
* attempted to add tests to Hledger.Data.MarketPrice
* moved MarketPrice test to Hledger.Read.JournalReader; fixed documentation on MarketPrice; added MarketPrice module to package.yaml
* cli: fix bug in pivot for postings without tag
Without this fix for postings without tag query checked effective
account which is always empty text ("").
* rewrite: inherit dates, change application order
For budgeting it is important to inherit actual date of posting if it
differs from date of transaction. These dates will be added
as a separate line of comment.
More natural order of rewrites is when result of first defined one is
available for all next rewrites.
* rewrite: factor out Hledger.Data.AutoTransaction
* rewrite: add diff output
With this option you can modify your original files without loosing
inter-transaction comments etc. I.e. you can run:
hledger-rewrite --diff Agency \
--add-posting 'Expenses:Taxes *0.17' \
| patch
As result multiple files should be updated.
Also it is nice to review your changes using colordiff instead of
patch.
* lib: track source lines range for journal
* doc: auto entries and diff output for rewrite
* bin: ignore chart and dupes addons
* lib: fix txnTieKnot
Ensure that postings refers to their transaction rather than original
one.
This should allow compiler perform destructive update and/or allow
garbage collecting old transaction.
This assumes a "PAYEE | NOTE" convention in the description field,
similar to Beancount's journal syntax. If the description has no pipe
character, payee and note are the same as the full description.
* Add implicit tags code/desc/payee for --pivot
Additionally allow using of transaction inherited tags.
* Use original posting in query by account name
To be able to query on individual postings and by account name it is
useful to have access to original account name (before pivot).
Especially this is useful when all postings within transaction gets
the same pivot name due.
As a side effect we'll match by alias.
Note: to query on amt it usually expected to see matches with inferred
amounts.
* Remember original postings during infer and pivot
This includes such functions like:
- inferFromAssignment
- inferBalancingAmount
- inferBalancingPrices
- pivotPosting
* Use original postings for hledger print
- Introduce "--explicit" option for "print" command which brings back
old behavior when every inferred number being printed.
- Make "print" by default print original postings without inferred
amounts. But use effective account name to have effect from aliases.
- Instruct shell tests with an new expected output or to use
--explicit option when inferred amounts are checked.
Resolvessimonmichael/hledger#442
* Changed behavior of `readJournalFiles` to be identical to `readJournalFile` for singleton lists
* Balance Assertions have to be simple Amounts
* Add 'isAssignment' and 'assignmentPostings' to Hledger.Data.Posting and Transaction
* Implemented 'balanceTransactionUpdate', a more general version of 'balanceTransaction' that takes an update function
* Fixed test cases.
* Implemented balance assignment ("resetting a balance")
* Add assertions to show function
* updated the comments
* numbering is not needed in journalCheckBalanceAssertions
* remove prices before balance checks
* rename functions
ie, when viewing a "current" period (the current day/week/month/quarter/year),
it will be moved to enclose the current date, if needed, whenever the system date changes.
When we don't know a file's format, instead of choosing a subset of
readers based on content sniffing, now we just try them all.
Also, LedgerReader is now used only as a last resort,
as it's not yet competitive with JournalReader.
Transactions are now numbered consistently during journal finalisation,
rather than just in the journal reader. Also transaction knot-tying has been
moved out of journalBalanceTransactions.
* Replace Parsec with Megaparsec (see #289)
This builds upon PR #289 by @rasendubi
* Revert renaming of parseWithState to parseWithCtx
* Fix doctests
* Update for Megaparsec 5
* Specialize parser to improve performance
* Pretty print errors
* Swap StateT and ParsecT
This is necessary to get the correct backtracking behavior, i.e. discard
state changes if the parsing fails.
With --debug=2, better information about assertions is printed.
Balance assertion errors now have a more standard and parseable layout.
The asserted balance is now shown with the diff, let's see if that's better.
There is a limitation/bug: disabling real mode in the transaction screen
won't show the non-real postings if it was entered from a real-mode
register screen.
The journal/timeclock/timedot parsers, instead of constructing (opaque)
journal update functions which are later applied to build the journal,
now construct the journal directly (by modifying the parser state). This
is easier to understand and debug. It also removes any possibility of
the journal updates being a space leak. (They weren't, in fact memory
usage is now slightly higher, but that will be addressed in other ways.)
Also:
Journal data and journal parse info have been merged into one type (for
now), and field names are more consistent.
The ParsedJournal type alias has been added to distinguish being-parsed
and finalised journals.
Journal is now a monoid.
stats: fixed an issue with ordering of include files
journal: fixed an issue with ordering of included same-date transactions
timeclock: sessions can no longer span file boundaries (unclocked-out
sessions will be auto-closed at the end of the file).
expandPath now throws a proper IO error (and requires the IO monad).
The commodity directive's format subdirective can now be used to
override the inferred style for a commodity, eg to increase or decrease
the precision. This doesn't fix the root cause of #295 but is at least a
good workaround.
Bracketed posting dates were fragile; they worked only if you wrote full
10-character dates. Also some semantics were a bit unclear. Now they
should be robust, and have been documented more clearly. This is a
legacy undocumented Ledger syntax, but it improves compatibility and
might be preferable to the more verbose "date:" tags if you write
posting dates often (as I do).
Internally, bracketed posting dates are no longer considered to be tags.
Journal comment, tag, and posting date parsers have been reworked, all
with doctests. Also the journal parser types generally have been
tightened up and clarified, making it much easier to know how to combine
and run them. There's now
-- | A parser of strings with generic user state, monad and return type.
type StringParser u m a = ParsecT String u m a
-- | A string parser with journal-parsing state.
type JournalParser m a = StringParser JournalContext m a
-- | A journal parser that runs in IO and can throw an error mid-parse.
type ErroringJournalParser a = JournalParser (ExceptT String IO) a
and corresponding convenience functions (and short aliases) for running them.
We now parse account directives, like Ledger's. We don't do anything
with them yet. The default parent account feature must now be spelled
"apply account"/"end apply account".
Amount display style canonicalisation code and terminology has been
clarified a bit. Individual amounts still have styles; from these we
derive the standard "commodity styles". In user docs, we might call
these "commodity formats" since a Ledger-compatible commodity directive
would use the "format" keyword.
Since market price amounts didn't contribute to the canonical commodity
styles, they were being reset to the null style. And this propagated to
the reported amounts when -V was in effect, causing much confusion.
Now, market prices contribute to canonicalisation and the expected
styles are preserved even with -V.
cf https://github.com/simonmichael/hledger/issues/131#issuecomment-133545140
print now always right-aligns the amounts in an entry, even when they
are wider than 12 characters.
If there is a price, it's considered part of the amount for
right-alignment. Maybe it would be nicer to put amounts and prices in
separate columns ? That will get a little complicated, needs more
discussion/design.
Also some cleanup of postingAsLines.
The print command wasn't lining up amounts with wide chars in account
names, fixed it properly this time. Transaction and Posting's Show instances
should also be wide-char-aware now.
Simple (non-multicolumn) balance reports containing wide characters
should now align correctly (in apps and fonts that show wide chars as
double width). Likewise, the print command.
Wide characters, eg chinese/japanese/korean characters, are typically
rendered wider than latin characters. In some applications (eg gnome
terminal or osx terminal) and fonts (eg monaco) they are exactly double
width. This is a start at making hledger aware of this. A register
report containing wide characters (in descriptions, account names, or
commodity symbols) should now align its columns correctly, when viewed
with a suitable font and application.
This adds a accountNameApplyAliasesMemo, which memoises the result of
applying a set of aliases (simple and regex) to an account name. In
theory this should reduce more repetitive work, but in practice it
doesn't seem to make a difference, so it's unused for now.
Roughly speaking, the time to apply regular expression account aliases
was O(aliases x transactions), and should now be O(aliases x accounts).
Also, the constant factor was reduced a lot by the recent commit
memoising toRegex. So now, regex aliases should be "free" like simple
aliases - use as many as you want, the slowdown shouldn't be noticeable.
hledger-lib-0.24's "track the commodity of zero amounts when
possible (useful eg for hledger-web's multi-commodity charts)" preserved
the commodity when normalising a zero mixed amount, but not the amount
style. This showed up as occasionally incorrect amount style (commodity
symbol placement, decimal point character, etc.) in balance reports with
certain journals, like this:
$ hledger bal
€3000.00 a <------ not using the canonical € style
4000,58€ 1
-1000,58€ D
-3000,00€ e
--------------------
0
I thought this would require a big rewrite of amount arithmetic, but it
seems that just being a little more careful is enough. When normalising
a mixed amount containing multiple zeros in the same commodity, we now
preserve the last zero with its amount style, instead of replacing them
all with a new one.
The register screen is now like the register view in hledger-web (and
other accounting systems), rather than hledger's register command.
This means:
- it shows transactions affecting a particular current account, rather
than postings matching a pattern.
- Each line represents a whole transaction.
- The account field shows the *other* account being transacted with.
When there is more than one, they are all listed, abbreviated and
marked with "(split)".
- The amount field shows the effect of the transaction on the current
account; positive for an inflow to this account, negative for an
outflow.
- The balance field should usually show the current account's historic
balance as of the transaction date, even when you change the report
start date. (Not working yet - currently it always shows the running
total).
- Transactions are listed most recent first, currently.
The balance command's --format option (in single-column mode) can now
adjust the rendering of multi-line strings, such as amounts with multiple
commodities. To control this, begin the format string with one of:
%_ - renders on multiple lines, bottom-aligned (the default)
%^ - renders on multiple lines, top-aligned
%, - render on one line, comma-separated
Also the final total (and the line above it) now adapt themselves to a
custom format.
The --format option's OutputFormat type was named confusingly like the
--output-format option. It has been renamed StringFormat to distinguish
it from StorageFormat (aka the data file format, referenced by
--output-format). Related code and types have been consolidated.
Also the (single-column) balance report's item rendering has had
some cleanup.
so we can benchmark things more easily with criterion.
As well as NFData, the Generic instance and a bunch more GHC extensions
seemed necessary. This is a little scary, impact unknown.
Simpler and clearer. We now have "transaction prices" (recorded as part
of transaction amounts) and "market prices" (recorded with P
directives). Both are matters of historical record, also this avoids
confusion with the balance command's "historical balances".
Initial support of market value reporting and currency conversion,
similar in spirit to Ledger's. The balance command now has a -V/--value
flag that converts all the reported amounts using their "default market
price". That is the latest market price (P directive, formerly called
"historical prices") found in the journal for their commodity that is on
or before the report end date.
Unlike Ledger, hledger's -V only uses the market prices recorded with P
directives, ignoring transaction prices recorded as part of posting
amounts (which -B/--cost uses). Using -B and -V together is allowed.
We don't do a good job of calculating good-looking unit prices when the
commodity display precisions are low. Eg when a journal doesn't use any
decimal places, any inferred unit prices are shown by the print command
also with no decimal places, which makes them look wrong.
Now inferred unit prices always have a minimum display precision of 2,
which helps a bit. Could do better.
When a transaction posts to two commodities without specifying the
conversion price, we generate a price which makes it balance
(cf http://hledger.org/manual.html#prices).
Until now, these generated prices were always shown with full precision
(all available decimal digits) so that a manual calculation with the
displayed numbers would agree.
If there's just one posting in the commodity being priced, we can use an
exact total price and the precision is no problem.
But if there are multiple postings in the commodity being priced, we
must show the averaged unit price. This can be an irrational number,
which with our current Decimal-based implementation would display an
excessive 255 decimal digits. So in this case we now set the price's
display precision to the sum of the (max) display precisions of the
commodities involved. An example:
hledgerdev -f- print
<<<
1/1
c C 10.00
c C 11.00
d D -320.00
>>>
2015/01/01
c C 10.00 @ D 15.2381
c C 11.00 @ D 15.2381
d D -320.00
>>>=0
There might still be cases where this will show more price decimal
places than necessary. For now, YAGNI.
A transaction/posting status of ! (pending) was effectively equivalent
to * (cleared). Now it's a separate state, not matched by --cleared.
The new Ledger-compatible --pending flag matches it, and so does
--uncleared. The equivalent search queries are now status:*, status:!
and status: (the old status:1 and status:0 spellings are deprecated).
Since we interpret --uncleared and status: as "any state except cleared",
it's not currently possible to match things which are neither cleared
nor pending.
The regex account aliases added in 0.24 trip up people switching between
hledger and Ledger. (Also they are currently slow).
This change makes the old non-regex aliases the default; they are
unsurprising, useful, and pretty close in functionality to Ledger's.
The new regex aliases are also available; they must be enclosed in
forward slashes. Ledger effectively ignores these, which is ok.
Also clarify docs, refactor, and use the same parser for alias
directives and alias options
NOTE: this is important to correctly build JournalContext
NOTE: currently a list reverse must done at the end,
maybe using a Data.Queue would be more efficient.
Eg recognise that 2014/11/30-2014/12/1 can be abbreviated to
2014/11/30d, similarly 2014/12/31-2015/1/1. Doesn't handle feb 29th
correctly, so eg 2000/2/28-2000/3/1 is wrongly abbreviated to
2000/2/28d.
alias match patterns (the part left of the =) are now case-insensitive
regular expressions matching anywhere in the account name. The
replacement string (the part right of the =) can replace multiple
matches within the account name. The replacement string does not yet
support any of the usual syntax like backreferences.
Previously, a depth:0 query produced an empty report (since there are no
level zero accounts). Now, it aggregates all data into one summary item
with account name "...".
This makes it easier to see the kind of data Gwern was looking for from
register-csv (net worth over time). Eg this shows one line per month
summarising the total of assets and liabilities:
hledger register-csv -- -MHE ^assets ^liabilities depth:0
Single and multi-column balance reports behave similarly.
hledger has represented quantities with floating point (Double) until
now. While this has been working fine in practice, the time has come to
upgrade our number representation to something more principled: Decimal,
for now. As a bonus, this brings a ~30% speed boost to most reports.
We'll keep the old representation(s) around for a while, selectable via
hledger-lib cabal flag, for research/testing/benchmarking purposes. To
build with the old Double representation: cabal install -fdouble
hledger-lib hledger hledger-web
In periodic multicolumn balance reports, column headings now show a more
compact description of common periods (years, half-years, quarters,
months, weeks) for better readability and screen space efficiency.
- show a background color for future and less-than-zero regions
- show points for transactions, not all line corners
- hovering over point shows balance, date, posted amount and transaction
- clicking a point scrolls towards that date
Now that balance assertions are checking only a single commodity, it can
be confusing. Eg say all your amounts are in dollars, an assertion like
"= 0" checked the dollar balance in hledger 0.23 but always succeeds in
hledger 0.24. When an assertion fails, we now report which commodity was
checked to help troubleshooting.
To avoid totally breaking the layout of multicolumn balance reports when
there are multi-commodity balances, these are now rendered on a single
line, comma separated. Simple balance reports still use the old vertical
format for now.
Amount display styles have been reworked a bit; they are now calculated
after journal parsing, not during it. This allows the fix for #196:
we now search through the amounts until a decimal point is detected,
instead of just looking at the first one; likewise for digit groups.
Digit groups are now implemented with a better type.
Digit group size detection has been improved a little:
1000,000 now gives group sizes [3,4,4,...], not [3,3,...], and
10,000 gives groups sizes [3,3,...] not [3,2,2,..].
(To get [3,2,2,...] you'd use eg 00,00,000.)
There are still some old (or new ?) issues; I don't think we handle
inconsistent decimal points & digit groups too well. But for now all
tests pass.
This change means you can make assertions on a multi-commodity account
balance (asserting one commodity at a time). On the flip side, you can
no longer assert the complete balance of an account (new unexpected
commodities will not be detected.) We might restore that ability later,
using the == syntax.
Can be helpful when reading Ledger files, where assertions may have
different semantics; or for getting some answers from your journal
to help you fix your assertions.
Could be called --no-assertions, but this might create surprise when it
has an effect contrary to --no-new-accounts.
I had to add another flag throughout the parsers & journal read
functions, ok for now.
- The CSV reader no longer writes a "(stdin).rules" file when reading
from stdin.
- Selection of reader(s) is now smarter when input is coming from stdin.
Previously, all readers were considered applicable for stdin. This
meant that when reading a CSV file from stdin, the journal and timelog
readers were always tried first, and if the CSV file was unparseable,
you'd see the first (journal) reader's error instead of the CSV
reader's. Now, the readers do some basic content sniffing when
reading stdin, so it generally tries only the one right reader and
we'll see the right errors.
- The read system now has more debug output.
Changes include:
- flat mode now shows exclusive (subaccount-excluding) balances.
This is a deviation from ledger, but seems simpler and clearer
for users and implementors across the various modes.
- in flat mode, --depth now aggregates deeper accounts at the
depth limit, rather than just excluding them from the report.
This is more useful.
- in flat mode, --empty no longer shows parent accounts with
no postings.
- more tests, more debug output, clearer code
Two new multi-column balance report modes show ending balance per
period: `--cumulative`, starting from 0, and `--historical`, starting
from the historical starting balance.
The balance command's specification has been clarified and consolidated
in the Balance.hs haddock. Reports.hs has also had haddock updates. The
old AccountsReport type is now BalanceReport, still used by
single-column balance report. The new MultiBalanceReport type is used by
the multi-column reports.
Amounts and journal values are often rendered too verbosely in debug
output, obscuring what's important. Here we try adjusting the level
of detail in the Show instance based on the global debug level.
Needs more work.
