This makes skipping/unskipping tests easier, and improves readability
a bit.
Note it's also possible to just write the test name with no preceding
function, when the type is constrained (see Journal.hs).
Same-line & next-line comments of transactions, postings, etc.
are now parsed a bit more precisely. Previously parsing no comment
gave the same result as an empty comment (a single newline); now
it gives an empty string.
Also, and perhaps as a consequence of the above, when there's no
same-line comment but there is a next-line comment, we'll insert an
empty first line, otherwise next-line comments would get moved up to
the same line when rendered.
Some doctests have been added.
This removes transactionModifierToFunction's extra query parameter;
the rewrite command sets it in the TransactionModifier instead, which
I think is equivalent. I had to change one functional test, but it
seems correct now, so perhaps it wasn't working right before ?
I was negligent and did not test enough. This should ignore
transaction comments in auto posting rules more safely.
It also adds support for trailing comments on the first line of auto
posting rules, which previously were misparsed as part of the query.
'fail' will just terminate the current parse branch, whereas here
we have encountered a definite error. Also bring the code to
get the current working directory inside 'getFilePaths', as it
logically belongs there.
Field names are supposed to be case insensitive, but a field assignment like
fields ...,Transaction_Date,...
date %Transaction_Date
was failing, because of the capitalised letters. Fixed now.
- expands the set of expected tokens when e.g. parsing the invalid
posting `account $1 a`
- whitespace can affect parse errors because of the longest match rule
where errors that occur later take precedence over those that occur
earlier
- inline `spaceamountormissingp` into `postingp`
- combine `rightsymbolamountp` and `nosymbolamountp`
- the multiplier symbol '*' for an amount must now always preceed a sign '-'
[breaking change]
- make amount parser labels more generic to simplify error messages
For Data/Dates.hs in particular:
- Changed `SimpleTextParser` to `TextParser m` for all parsers
- Changed `string` to the case-insensitive `string'` to match the
behaviour of `T.toLower` found in `parsePeriodExpr`
- export `periodexprp` for "direct" use
base-compat-batteries provides the same API across more ghc versions
than base-compat does, at the cost of more dependencies. Eg it exports
Prelude.Compat ((<>)) with ghc 7.10/base 4.8, which we expect.
My belief is that several of our deps already require it so the added
cost is not too great. We should probably go back to base-compat when
possible though, eg when we stop supporting ghc 7.10.
The new version of our package set apparently contains both base-compat and
base-compat-batteries in its transitive closure. This breaks the doctest suite,
which just imports everything into scope when the tests are run, thereby making
module names like Prelude.Compat ambiguous.
We don't need to import Data.Monoid because Prelude.Compat exports "<>"
already. In fact, importing that module causes build failures:
Hledger/Read/Common.hs:725:62: error:
Ambiguous occurrence ‘<>’
It could refer to either ‘Sem.<>’,
imported from ‘Prelude.Compat’ at Hledger/Read/Common.hs:97:1-39
(and originally defined in ‘Data.Semigroup’)
or ‘Data.Monoid.<>’,
imported from ‘Data.Monoid’ at Hledger/Read/Common.hs:110:1-18
Fixes https://github.com/simonmichael/hledger/issues/794.
- Rationale:
- The information necessary for applying exponents to a number is more
explicitly represented in the inputs to `fromRawNumber` than in the outputs
- This way, `exponentp` may simply return an `Int`
- Purpose: to reduce the verbosity of the previous implementation
- Split off `AmbiguousNumber` into its own type
- Introduce a function `AmbiguousNumber -> RawNumber` explicitly capturing the
disambiguation logic
- Reduce the number of remaining constructors in `RawNumber` to just two,
`WithSeparator` and `NoSeparator`
- The choice to distinguish by the presence of digit separators is motivated
by the need for this information later on when disallowing exponents on
numbers with digit separators
- Extracts the handling of signs out of `fromRawNumber` and into `signp` itself
- Rationale: The sign can be applied independently from the logic in
`fromRawNumber`
A commodity directive that doesn't specify the decimal point character
increases ambiguity and the chance of misparsing numbers, especially
as it overrides all style information inferred from the journal amounts.
In some cases it caused amounts with a decimal point to be parsed as if
with a digit group separator so 1.234 became 1234.
We could augment it with extra info from the journal amounts, when available,
but it would still be possible to be ambiguous, and that won't be obvious.
A commodity directive is what we recommend to nail down the style.
It seems the simple and really only way to do this reliably is to require
an explicit decimal point character. Most folks probably do this already.
Unfortunately, it makes another potential incompatiblity with ledger and
beancount journals. But the error message will be clear and easy to
work around.
Trailing whitespace in the replacement part of a regular expression
account alias is now significant. Eg, slightly flattening some bank
accounts: --alias '/:somebank:/=somebank '
Older megaparsec is still supported.
Also cleans up our custom parser types,
and some text (un)packing is done in different places
(possible performance impact).
See the issue and linked mail list discussion. Ambiguity between the
uncleared state, and the "not cleared" --uncleared flag causes confusion
and friction. At this point it seems best to break with Ledger and
past hledger, pick a new name and drop --uncleared to put an end to it.
When generating a new posting as a multiple of an existing posting,
support conversion to a different commodity. For example, postings in
hours can be used to generate postings in USD.
Automatic transactions generated from rewrite rules use the commodity,
amount style, and transaction price if the rewrite defines a commodity.
* added showMarketPrice and Hledger.Data.MarketPrice module
* showMarketPrice implemented using showDate
* attempted to add tests to Hledger.Data.MarketPrice
* moved MarketPrice test to Hledger.Read.JournalReader; fixed documentation on MarketPrice; added MarketPrice module to package.yaml
* cli: fix bug in pivot for postings without tag
Without this fix for postings without tag query checked effective
account which is always empty text ("").
* rewrite: inherit dates, change application order
For budgeting it is important to inherit actual date of posting if it
differs from date of transaction. These dates will be added
as a separate line of comment.
More natural order of rewrites is when result of first defined one is
available for all next rewrites.
* rewrite: factor out Hledger.Data.AutoTransaction
* rewrite: add diff output
With this option you can modify your original files without loosing
inter-transaction comments etc. I.e. you can run:
hledger-rewrite --diff Agency \
--add-posting 'Expenses:Taxes *0.17' \
| patch
As result multiple files should be updated.
Also it is nice to review your changes using colordiff instead of
patch.
* lib: track source lines range for journal
* doc: auto entries and diff output for rewrite
* Changed behavior of `readJournalFiles` to be identical to `readJournalFile` for singleton lists
* Balance Assertions have to be simple Amounts
* Add 'isAssignment' and 'assignmentPostings' to Hledger.Data.Posting and Transaction
* Implemented 'balanceTransactionUpdate', a more general version of 'balanceTransaction' that takes an update function
* Fixed test cases.
* Implemented balance assignment ("resetting a balance")
* Add assertions to show function
* updated the comments
* numbering is not needed in journalCheckBalanceAssertions
* remove prices before balance checks
* rename functions
When we don't know a file's format, instead of choosing a subset of
readers based on content sniffing, now we just try them all.
Also, LedgerReader is now used only as a last resort,
as it's not yet competitive with JournalReader.
This reader is used by default for files with suffix .ledger or .l,
and tried along with the other readers for files of unknown type.
Currently only the bare minimum of the raw parsed data is used:
transaction dates/descriptions and posting accounts/amounts,
with the rest being ignored.
Amounts are parsed the same way as in the hledger journal format.
Malformed amounts might be ignored instead of error-reported.
* csv rules: Show prettier parsing errors
This goes from
hledger: user error ("ParseError {errorPos = SourcePos {sourceName = \"foo.csv.rules\",
sourceLine = Pos 20, sourceColumn = Pos 1} :| [], errorUnexpected =
fromList [Tokens (' ' :| \"\")], errorExpected = fromList [Label ('b' :| \"lank or comment
line\"),EndOfInput], errorCustom = fromList []}")
to
hledger: user error (foo.csv.rules:20:1:
unexpected space
expecting blank or comment line or end of input
)
* csv rules: Fix parsing of empty field values
A single line containing `account1 ` (note the space at the end) should
parse as assignment of the empty string to account1. At least it did
until commit 4141067.
The problem is that megaparsec's `space` parses multiple space
characters as opposed to parsec. So in the example above it would
incorrectly consume the newline.
This commit also adds a new test case for this bug.
Timeclock transaction ids now count up rather than down.
Also, remove old code for appending timeclock transactions to journal transactions,
a holdover from the days when both were allowed in one file.
Transactions are now numbered consistently during journal finalisation,
rather than just in the journal reader. Also transaction knot-tying has been
moved out of journalBalanceTransactions.
* Replace Parsec with Megaparsec (see #289)
This builds upon PR #289 by @rasendubi
* Revert renaming of parseWithState to parseWithCtx
* Fix doctests
* Update for Megaparsec 5
* Specialize parser to improve performance
* Pretty print errors
* Swap StateT and ParsecT
This is necessary to get the correct backtracking behavior, i.e. discard
state changes if the parsing fails.
The journal/timeclock/timedot parsers, instead of constructing (opaque)
journal update functions which are later applied to build the journal,
now construct the journal directly (by modifying the parser state). This
is easier to understand and debug. It also removes any possibility of
the journal updates being a space leak. (They weren't, in fact memory
usage is now slightly higher, but that will be addressed in other ways.)
Also:
Journal data and journal parse info have been merged into one type (for
now), and field names are more consistent.
The ParsedJournal type alias has been added to distinguish being-parsed
and finalised journals.
Journal is now a monoid.
stats: fixed an issue with ordering of include files
journal: fixed an issue with ordering of included same-date transactions
timeclock: sessions can no longer span file boundaries (unclocked-out
sessions will be auto-closed at the end of the file).
expandPath now throws a proper IO error (and requires the IO monad).
journal files can now include journal, timeclock or timedot files (but
not yet CSV files). Also timeclock/timedot files no longer support
default year directives.
The Hledger.Read.* modules have been reorganised for better reuse.
Hledger.Read.Utils has been renamed Hledger.Read.Common and holds
low-level parsers & utilities; high-level read utilities have moved to
Hledger.Read.
Bracketed posting dates were fragile; they worked only if you wrote full
10-character dates. Also some semantics were a bit unclear. Now they
should be robust, and have been documented more clearly. This is a
legacy undocumented Ledger syntax, but it improves compatibility and
might be preferable to the more verbose "date:" tags if you write
posting dates often (as I do).
Internally, bracketed posting dates are no longer considered to be tags.
Journal comment, tag, and posting date parsers have been reworked, all
with doctests. Also the journal parser types generally have been
tightened up and clarified, making it much easier to know how to combine
and run them. There's now
-- | A parser of strings with generic user state, monad and return type.
type StringParser u m a = ParsecT String u m a
-- | A string parser with journal-parsing state.
type JournalParser m a = StringParser JournalContext m a
-- | A journal parser that runs in IO and can throw an error mid-parse.
type ErroringJournalParser a = JournalParser (ExceptT String IO) a
and corresponding convenience functions (and short aliases) for running them.
We now parse account directives, like Ledger's. We don't do anything
with them yet. The default parent account feature must now be spelled
"apply account"/"end apply account".
The Journal, Timelog and Timedot readers' detectors now check
each line in the sample data, not just the first one. I think
the sample data is only about 30 chars right now, but even so
this fixed a format detection issue I was seeing.
Timedot is a plain text format for logging dated, categorised
quantities (eg time), supported by hledger. It is convenient for
approximate and retroactive time logging, eg when the real-time
clock-in/out required with a timeclock file is too precise or too
interruptive. It can be formatted like a bar chart, making clear at a
glance where time was spent.
I really don't see why that extra x parameter is needed or works..
rewrite it in simpler form.
I also might be introducing breakage for older GHC's by using
unconditionally <$>, but I'm not seeing that for some reason
(tested back to ghc 7.6).
This adds a accountNameApplyAliasesMemo, which memoises the result of
applying a set of aliases (simple and regex) to an account name. In
theory this should reduce more repetitive work, but in practice it
doesn't seem to make a difference, so it's unused for now.
Simpler and clearer. We now have "transaction prices" (recorded as part
of transaction amounts) and "market prices" (recorded with P
directives). Both are matters of historical record, also this avoids
confusion with the balance command's "historical balances".
We now parse, and also print, posting-less journal entries, as I
proposed on the lists.
These are not real General Journal entries/transactions, but here is my
rationale:
- Ledger and beancount parse them
- if we parse them, we should print them
- they provide a natural way to record and report non-transaction events
- most of all, they permit more gradual introduction and learning of the concepts.
Eg a beginner can keep a simple journal even before learning about accounts and postings.
A transaction/posting status of ! (pending) was effectively equivalent
to * (cleared). Now it's a separate state, not matched by --cleared.
The new Ledger-compatible --pending flag matches it, and so does
--uncleared. The equivalent search queries are now status:*, status:!
and status: (the old status:1 and status:0 spellings are deprecated).
Since we interpret --uncleared and status: as "any state except cleared",
it's not currently possible to match things which are neither cleared
nor pending.
The regex account aliases added in 0.24 trip up people switching between
hledger and Ledger. (Also they are currently slow).
This change makes the old non-regex aliases the default; they are
unsurprising, useful, and pretty close in functionality to Ledger's.
The new regex aliases are also available; they must be enclosed in
forward slashes. Ledger effectively ignores these, which is ok.
Also clarify docs, refactor, and use the same parser for alias
directives and alias options
