(SourcePos, SourcePos).
This has been marked for possible removal for a while. We are keeping
strictly more information. Possible edge cases arise with Timeclock and
CsvReader, but I think these are covered.
The particular motivation for getting rid of this is that
GenericSourcePos is creating some awkward import considerations for
little gain. Removing this enables some flattening of the module
dependency tree.
rawOptsTo* in hledger-lib now takes a day as an argument, and does not
live in the IO monad, since it's now pure.
This is so that we can run tests containing future transactions that
won't fail as soon as ‘the future’ actually arrives.
The forecast period begins on:
- the start date supplied to the `--forecast` argument, if present
- otherwise, the later of
- the report start date if specified with -b/-p/date:
- the day after the latest normal (non-periodic) transaction in the journal, if any
- otherwise today.
It ends on:
- the end date supplied to the `--forecast` argument, if present
- otherwise the report end date if specified with -e/-p/date:
- otherwise 180 days (6 months) from today.
Note that the previous behaviour did not quite match the documentation,
so this also acts as a bug fix for #1665.
POSIXTime.
This eliminates old-time, which has been deprecated for a while, from
our dependencies.
This introduces a slight incompatibility, as a small number of functions
now take/return POSIXTime instead of ClockTime. Generally you will be
using the current time, in which case you should use getPOSIXTime from
Data.Time.Clock.POSIX instead of getClockTime.
utcTimeToClockTime has been removed, as it is now equivalent to
utcTimeToPOSIXSeconds from Data.Time.Clock.POSIX.
rather than as a postprocessing step. (#1638)
This allows us to have a uniform procedure for balancing transactions,
whether they are normal transactions or forecast transactions, including
dealing with balance assignments, balance assertions, and auto postings.
style amounts according to that argument. journalAddForecast and
journalTransform now return an Either String Journal.
This improves efficiency, as we no longer have to restyle all amounts in
the journal after generating auto postings or periodic transactions.
Changing the return type of journalAddForecast and journalTransform
reduces partiality.
To get the previous behaviour for modifyTransaction, use modifyTransaction mempty.
The only semantic difference is that we now apply
journalApplyCommodityStyles before running journalCheckAccountsDeclared
and journalCheckCommoditiesDeclared.
In Amount, aismultiplier is a boolean flag that will always be False,
except for in TMPostingRules, where it indicates whether the posting
rule is a multiplier. It is therefore unnecessary in the vast majority
of cases. This posting pulls this flag out of Amount and puts it into
TMPostingRule, so it is only kept around when necessary.
This changes the parsing of journals somewhat. Previously you could
include an * before an amount anywhere in a Journal, and it would
happily parse and set the aismultiplier flag true. This will now fail
with a parse error: * is now only acceptable before an amount within an
auto posting rule.
Any usage of the library in which the aismultiplier field is read or set
should be removed. If you truly need its functionality, you should
switch to using TMPostingRule.
This changes the JSON output of Amount, as it will no longer include
aismultiplier.
transactions are balanced possibly using explicit prices, but without
inferring any prices. This is included in --strict mode.
Renames check autobalanced to check balancedwithautoconversion.
instead of a list of Amounts. No longer export Mixed constructor, to
keep API clean (if you really need it, you can import it directly from
Hledger.Data.Types). We also ensure the JSON representation of
MixedAmount doesn't change: it is stored as a normalised list of
Amounts.
This commit improves performance. Here are some indicative results.
hledger reg -f examples/10000x1000x10.journal
- Maximum residency decreases from 65MB to 60MB (8% decrease)
- Total memory in use decreases from 178MiB to 157MiB (12% decrease)
hledger reg -f examples/10000x10000x10.journal
- Maximum residency decreases from 69MB to 60MB (13% decrease)
- Total memory in use decreases from 198MiB to 153MiB (23% decrease)
hledger bal -f examples/10000x1000x10.journal
- Total heap usage decreases from 6.4GB to 6.0GB (6% decrease)
- Total memory in use decreases from 178MiB to 153MiB (14% decrease)
hledger bal -f examples/10000x10000x10.journal
- Total heap usage decreases from 7.3GB to 6.9GB (5% decrease)
- Total memory in use decreases from 196MiB to 185MiB (5% decrease)
hledger bal -M -f examples/10000x1000x10.journal
- Total heap usage decreases from 16.8GB to 10.6GB (47% decrease)
- Total time decreases from 14.3s to 12.0s (16% decrease)
hledger bal -M -f examples/10000x10000x10.journal
- Total heap usage decreases from 108GB to 48GB (56% decrease)
- Total time decreases from 62s to 41s (33% decrease)
If you never directly use the constructor Mixed or pattern match against
it then you don't need to make any changes. If you do, then do the
following:
- If you really care about the individual Amounts and never normalise
your MixedAmount (for example, just storing `Mixed amts` and then
extracting `amts` as a pattern match, then use should switch to using
[Amount]. This should just involve removing the `Mixed` constructor.
- If you ever call `mixed`, `normaliseMixedAmount`, or do any sort of
amount arithmetic (+), (-), then you should replace the constructor
`Mixed` with the function `mixed`. To extract the list of Amounts, use
the function `amounts`.
- If you ever call `normaliseMixedAmountSquashPricesForDisplay`, you can
replace that with `mixedAmountStripPrices`. (N.B. this does something
slightly different from `normaliseMixedAmountSquashPricesForDisplay`,
but I don't think there's any use case for squashing prices and then
keeping the first of the squashed prices around. If you disagree let
me know.)
- Any remaining calls to `normaliseMixedAmount` can be removed, as that
is now the identity function.
Exceptions are for dealing with the pamount field, which is really just
dealing with an unnormalised list of amounts.
This creates an API for dealing with MixedAmount, so we never have to
access the internals outside of Hledger.Data.Amount.
Also remove a comment, since it looks like #1207 has been resolved.
aquantity.
Journal entries still require a positive @@ price, but now the sign is
set after parsing, rather than when converting in amountToCost.
The reason for this change is that, if we're going to perform arithmetic
on Amount with TotalCost, then the presence of aquantity=0 means that
amountToCost would render the total cost as 0, because signum 0 == 0.
This makes journal entries like the following impossible to balance:
2000-01-01
a 0 @@ 10 A
b -10 A
Error messages for these four are now a bit fancier and more
consistent. But not yet optimised for machine readability.
Cf #1436.
Added to hledger-lib: chomp1, linesPrepend[2].
Command-line account aliases now also affect transactions read
from these formats (not just journal format).
lib: journalApplyAliases, transactionApplyAliases, postingApplyAliases
helpers have been added.
Journal keeps a new piece of parsing state, a decimal mark character,
which can optionally be set to force the number format expected by all
amount parsers.
We now accept (but still ignore) a fixed or nonfixed ({=} or {}) lot
price following a posting amount, and it may appear before or after a
transaction price (@ or @@). And it may no longer appear after a
balance assertion.
Also: fixedlotpricep renamed to lotpricep, now also parses non-fixed
lot prices. A bit of amount parsers cleanup.
orgstruct-mode was dropped from org 9.2, and I shouldn't have been
forcing it on anyway.
The new config allows its "replacement", outshine-mode, to do similar
code folding when you press tab on any of the lines matching
outline-regexp. But only if you patch it as mentioned at
https://github.com/alphapapa/outshine/issues/77.
Enable it by, eg: (add-hook 'haskell-mode-hook 'outshine-mode)
The include directive now tries just one reader, based on the file
extension and defaulting to journal, like the rest of hledger.
(It doesn't yet handle a reader prefix.)
Reader-finding utilities have moved from Hledger.Read to
Hledger.Read.JournalReader so the include directive can use them.
Reader changes:
- rExperimental flag removed
- old rParser renamed to rReadFn
- new rParser field provides the actual parser.
This seems to require making Reader a higher-kinded type, unfortunately.
Renamed: writeValidJournal -> writeJournalTextIfValidAndChanged
Added comments clarifying line ending behaviour of:
add, import, appendToJournalFileOrStdout, readFilePortably,
writeFileWithBackupIfChanged, writeJournalTextIfValidAndChanged
Summary of current behaviour:
- hledger add and import commands will append with (at least some)
unix line endings, possibly causing the file to have mixed line
endings
- hledger-web edit and upload forms will write the file with
the current system's native line endings, ie changing all
line endings if the file previously used foreign line endings.
Hledger.Util.Tests helpers have been cleaned up, and test names are
now shown.
Tests have been cleaned up a bit. Some groups of unnamed tests have
been collapsed into a single named test containing a sequence of
assertions. The test command counts named tests, not assertions, so
the reported unit test count has dropped from 199 to 188.
easytest is not actively maintained and requires an old version of
hedgehog which does not support base-compat 0.11 & ghc 8.8.
This is still using the old easytest helpers, and not displaying test
names properly.
And if they did, the stats command would now throw an error.
Changed:
journalApplyCommodityStyles
journalInferCommodityStyles
commodityStylesFromAmounts
fail is moving out of Monad and into it's own MonadFail class.
This will be enforced in GHC 8.8 (I think).
base-compat/base-compat-batteries 0.11.0 have adapted to this,
and are approaching stackage nightly
(https://github.com/commercialhaskell/stackage/issues/4802).
hledger is now ready to build with base-compat-batteries 0.11.0, once
all of our deps do (eg aeson). We are still compatible with the older
0.10.x and GHC 7.10.3 as well.
For now we are using both fails:
- new fail (from Control.Monad.Fail), used in our parsers, imported
via base-compat-batteries Control.Monad.Fail.Compat to work with
older GHC versions.
- old fail (from GHC.Base, exported by Prelude, Control.Monad,
Control.Monad.State.Strict, Prelude.Compat, ...), used in easytest's
Test, since I couldn't find their existing fail implementation to update.
To reduce (my) confusion, these are imported carefully, consistently,
and qualified everywhere as Fail.fail and Prelude.fail, with clashing
re-exports suppressed, like so:
import Prelude hiding (fail)
import qualified Prelude (fail)
import Control.Monad.State.Strict hiding (fail)
import "base-compat-batteries" Prelude.Compat hiding (fail)
import qualified "base-compat-batteries" Control.Monad.Fail.Compat as Fail
dropped journalPrices
renamed Price to AmountPrice, AKA "transaction price"
renamed MarketPrice to PriceDirective.
added new MarketPrice (more pure form of PriceDirective without the amount style information)
Prices is now a more efficient data structure, but not used yet.
Hopefully this is will do it. This restores the past behaviour:
- parsing prices in balance assertions/assignments
- ignoring them in assertions
- using them in assignments
- and printing them
and clarifies tests and docs.
- parse a period expression by first extracting words separated by
single spaces, then by "re-parsing" this text with 'periodexprp'
- this way, the period expression parsers do not need to know about
the single- or double-space rules
* journal: Get rid of `journalFinalise` and use granular functions
Complete the process started in 53b3e2bd. This gets rid of the
`journalFinalise` function and uses the smaller steps, in order to
have more granular control.
* journal: Change order of operations in finalization
We want to make sure that we add the filepath after the order is
reversed, so the added filepath is on the head and not the tail (as it
would be if it were reversed after it was added).
* journal: Refine granular finalization functions
This commit fixes two of the granular finalization functions:
1. Rename `journalSetTime` to `journalSetLastReadTime` and improve
documentation.
2. Remove `journalSetFilePath`. It's redundant with `journalAddFile`
currently in `Hledger.Read.Common`. The only difference between the
functions is where the file is added (we keep the one in which it
is added to the tail), so we change the position vis-a-vis
reversal.
`journalFinalise` is only used in the `parseAndFinaliseJournal`
functions, but it needs to be run differently at different stages when
transaction modifiers are applied. This change breaks it into smaller
functions, and uses those smaller parts in `parseAndFinaliseJournal`
as needed.
Previously we ran if `--auto` was set. But this adds a small
performance hit if `--auto` becomes default. Now we only run twice if
there are transactionModifiers AND `--auto` is set. So even if auto is
specified, there will be no penalty if there are no modifiers.
Currently, automated transactions are added before the journal is
finalized. This means that no inferred values will be picked up. We
change the procedure, if `auto_` is set, to
1. first run `journalFinalise` without assertion checking (assertions
might be wrong until automated transactions), but with reordering
2. Insert transaction modifiers
3. Run `journalFinalise` again, this time with assertion checking as
set in the options, and without reordering.
If `auto_` is not set, all works as before.
Closes: #893
Previously you had to use one of the standard english account names
(assets, liabilities..) for top-level accounts, if you wanted to use
the bs/bse/cf/is commands.
Now, account directives can specify which of the big five categories
an account belongs to - asset, liability, equity, revenue or expense -
by writing one of the letters A, L, E, R or X two or more spaces after
the account name (where the numeric account code used to be).
This might change. Some thoughts influencing the current syntax:
- easy to type and read
- does not require multiple lines
- does not depend on any particular account numbering scheme
- allows more types later if needed
- still anglocentric, but only a little
- could be treated as syntactic sugar for account tags later
- seems to be compatible with (ignored by) current Ledger
The current design permits unlimited account type declarations anywhere
in the account tree. So you could declare a liability account somewhere
under assets, and maybe a revenue account under that, and another asset
account even further down. In such cases you start to see oddities like
accounts appearing in multiple places in a tree-mode report. In theory
the reports will still behave reasonably, but this has not been tested
too hard. In any case this is clearly too much freedom. I have left it
this way, for now, in case it helps with:
- modelling contra accounts ?
- multiple files. I suspect the extra expressiveness may come in handy
when combining multiple files with account type declarations,
rewriting account names, apply parent accounts etc.
If we only allowed type declarations on top-level accounts, or
only allowed a single account of each type, complications seem likely.
We previously had another parser type, 'type ErroringJournalParser =
ExceptT String ...' for throwing parse errors without the possibility of
backtracking. This parser type was removed under the assumption that it
would be possible to write our parser without this capability. However,
after a hairy backtracking bug, we would now prefer to have the option
to prevent backtracking.
- Define a 'FinalParseError' type specifically for the 'ExceptT' layer
- Any parse error can be raised as a "final" parse error
- Tracks the stack of include files for parser errors, anticipating the
removal of the tracking of stacks of include files in megaparsec 7
- Although a stack of include files is also tracked in the 'StateT
Journal' layer of the parser, it seems easier to guarantee correct
error messages in the 'ExceptT FinalParserError' layer
- This does not make the 'StateT Journal' stack redundant because the
'ExceptT FinalParseError' stack cannot be used to detect cycles of
include files
- Don't immediately throw custom parse errors into 'ParsecT'; rather,
just construct and return them
- This anticipates the re-implementation of an 'ExceptT' layer of the
parser, which should be able throw custom parse errors
- In anticipation of megaparsec 7, which removes support for stacks of
include files (as far as I can tell)
- Intended for the 'StateT Journal' layer of the parser
- A stack of include files would be better in a 'ReaderT' layer, but I
don't want to add another layer to the parser
- Intended for detecting cycles of include files
- Potential issue: for proper error messages for include file cycles,
we must remember to provide the filepath of the root journal file via
the initial journal state passed to a 'JournalParser'; I imagine
that we may forget to do so because in all other cases it is okay
not to do so.
This makes skipping/unskipping tests easier, and improves readability
a bit.
Note it's also possible to just write the test name with no preceding
function, when the type is constrained (see Journal.hs).
