Commit Graph

850 Commits

Author SHA1 Message Date
Simon Michael
69be1d4ef7 ;dev: csv: refactor, clarify 2023-05-11 15:44:56 -10:00
Simon Michael
70d4c0c638 ;dev: csv: refactor, clarify 2023-05-11 15:35:05 -10:00
Simon Michael
755c3d3dbb ;dev: csv: refactor 2023-05-11 15:34:31 -10:00
Simon Michael
bbecd611f1 imp: timedot: make one multi-posting transaction per date line (#1754)
Descriptions and comments are now more straightforward and similar to
journal format.
2023-05-03 19:18:35 -10:00
Simon Michael
50349f81f7 imp: timeclock: support comments and tags (fix #1220)
Breaking change: previously timeclock descriptions could contain
semicolons. Now a semicolon in the description will end it and
start a comment (which may contain tags).
2023-05-03 00:05:11 -10:00
Simon Michael
f7f86a709b imp: timedot: fix day description/comment parsing; parse posting comments/tags 2023-05-02 17:03:03 -10:00
Simon Michael
1b19f3d330 dev: timedot: clarify, cleanup 2023-05-02 17:02:30 -10:00
Simon Michael
a771c8fc19 imp: revert 2b5194238 imp: generate auto postings on forecast transactions by default (#2027)
I found at least one user for whom this would be a breaking change
(they generate forecast txns, and have auto posting rules, but don't
want the latter applied to the former). I guess it's better to keep
things as they were for now: if you need auto postings on your
forecast txns you must use two flags, --forecast --auto.
2023-04-30 08:08:26 -10:00
Simon Michael
7f713f6a44 imp: Generated/modified txns/postings are now tagged only with --verbose-tags 2023-04-29 18:25:11 -10:00
Simon Michael
2b5194238b imp: generate auto postings on forecast transactions by default 2023-04-29 16:00:14 -10:00
Chris Lemaire
cddbae6467 queries: Prefix boolean queries with expr:
Boolean queries are now prefixed with an 'expr:' prefix, making them
completely separable from old queries and making the addition of them a
little more migration proof.

The tests are updated accordingly, changes made to the tests previously
are removed and extra cautious documentation is also removed.
2023-03-27 10:29:26 -10:00
Simon Michael
c790aa6145 ;dev: lib: also build with GHC 9.6.1; add base-compat 2023-03-14 10:42:48 -10:00
Simon Michael
fa70f160ae imp: partial/inferred dates are flexible, full dates are not (#1982)
DateSpans are now now aware of exact/flexible dates.
2023-02-17 07:24:19 -10:00
Simon Michael
5537a251f3 imp: journal: periodic txns need not start on an interval boundary
Eg, ~ monthly from 1/15 now works, instead of giving an error message.
2023-02-17 07:24:19 -10:00
Simon Michael
7a9b0fd94c feat: check: the tags check checks tag names 2023-02-16 11:56:22 -10:00
Simon Michael
c51ebe7b8c imp: journal: remove the "adjacent conversion postings" check again
I find that single conversion postings are sometimes needed, eg with
paypal.
2023-02-01 13:30:21 -10:00
Simon Michael
de330bdfce imp: journal: check for adjacent conversion postings only in strict mode
This avoids breaking existing journals if, say, they happen to have
a single "equity:trading" posting somewhere.
2023-01-25 19:00:29 -10:00
Simon Michael
76f8eaf91a imp: journal: ignore redundant costs; --infer-costs more robustly
You can now write both @/@@ costs and corresponding equity conversion postings
in a transaction at any time, not just when using --infer-costs or --infer-equity.
hledger will recognise the redundancy and ignore it.

One thing has become more strict: hledger now requires conversion postings
to occur in adjacent pairs; an odd number of them is not allowed.
(Conversion postings are postings to accounts of type `V`/`Conversion`,
or named `equity:conversion`, `equity:trade`, `equity:trading`,
or subaccounts of these.)

And, --infer-costs now works in transactions with an implicit amount
(inferring costs from equity now happens after transaction balancing,
not before).
2023-01-25 19:00:29 -10:00
Chris Lemaire
43c9f018dc journal: Account directives reject account names with brackets
Previously, the accounts passed to account directives would be stripped
of their surrounding brackets, but the required behaviour is to have
account directives plain reject bracketed accounts. This change ensures
that accounts in account directives may not start with a bracket
character.
2023-01-24 13:40:01 -10:00
Simon Michael
9497472e64
Merge branch 'master' into 1984-source-name-and-position-for-forecast 2023-01-23 16:04:00 -10:00
Chris Lemaire
4ada2a3fbc journal: Fully unbracket AccountNames in account directives
Currently an account name like "a:(aa)" will not have (aa) unbracketed.
However, this seems reasonable since the full name is unbracketed and
thus will not be confused with virtual or virtual-balanced posting.
2023-01-23 12:43:13 -10:00
Chris Lemaire
9443fe0e0d journal: json: Add source positions to forecast transactions 2023-01-21 22:30:30 +01:00
Simon Michael
5b27554f0a dev: note why inferring costs from equity before balancing 2023-01-19 20:57:02 -10:00
Simon Michael
cf469c080d lib: journalInferCosts -> journalInferCostsFromEquity 2023-01-19 20:25:41 -10:00
Simon Michael
28eb8be4fa lib: BalancingOpts{infer_transaction_prices_ -> infer_balancing_costs_} 2023-01-19 20:21:31 -10:00
Simon Michael
dfea6eaee2 lib: Hledger.Data.Journal: journalAddPricesFromEquity -> journalInferCosts 2023-01-19 09:17:56 -10:00
Simon Michael
dfebf3174c imp: csv: check assigned account names are valid (parseable) (#1978) 2023-01-11 21:42:47 -10:00
Simon Michael
a9b63bb694 fix: csv: skip header lines before attempting to parse records (#1967) 2022-12-27 12:21:20 -10:00
Simon Michael
4b36d852b4 imp: journal: ignore ((valuation expressions)) after amounts (ledger compat) 2022-12-22 20:06:28 -10:00
Simon Michael
1ea2bcc83f imp: journal: also parse (lot notes) in amounts (ledger compat)
and rename lotpricep -> lotcostp
and instrument some amount parsers for debugging with megaparsec's dbg
2022-12-22 19:07:40 -10:00
Simon Michael
aa54c3273a dev: lib: clarify some amount parsers; describe Ledger lot notation
amountpwithmultiplier -> amountp'
amountpnolotpricesp   -> amountnobasisp
amountwithoutpricep   -> simpleamountp
priceamountp          -> costp
2022-12-22 09:15:43 -10:00
Simon Michael
b1e08ffe46 imp: journal: python directive also consumes empty lines (ledger compat) 2022-12-22 02:30:29 -10:00
Simon Michael
85e9fbcf7d imp: journal: allow an empty double-quoted commodity symbol
"" is equivalent to writing no symbol.
But it's now possible to declare market prices for the no-symbol
commodity, which occurs eg with timedot data.

P 2022-01-01 "" $100
2022-12-22 01:53:06 -10:00
Simon Michael
a070875fe6 imp: journal: ignore end apply year, end apply tag, drop "pop" (ledger compat) 2022-12-22 01:50:27 -10:00
Simon Michael
a5d6b42322 imp: journal: ignore end apply fixed, end apply tag directives (ledger compat) 2022-12-22 01:12:56 -10:00
Simon Michael
af5645ebb7 imp: journal: ignore python, eval directives (ledger compat) 2022-12-22 00:50:43 -10:00
Simon Michael
2e77586d1b imp: journal: ignore value directives (ledger compat) 2022-12-22 00:38:51 -10:00
Simon Michael
fb7134c9a0 imp: journal: ignore tag subdirectives (ledger compat) 2022-12-22 00:34:33 -10:00
Simon Michael
c58b9e6ac9 imp: journal: accept apply year as another synonym for Y (ledger compat) 2022-12-22 00:27:35 -10:00
Simon Michael
2357a7a750 dev: journal: ignore CLI flags in journal 2022-12-21 23:40:43 -10:00
Simon Michael
abd104ccee imp: journal: ignore more ledger directives (ledger compat)
apply fixed, apply tag, assert, bucket, A, capture, check, define, expr

And reorder the directivep parsers.
2022-12-21 23:04:58 -10:00
Simon Michael
e09ab0936a imp: journal: accept @ as a prefix of directives (ledger compat) 2022-12-21 22:56:32 -10:00
Simon Michael
95d7679f6d imp: journal: accept year as a synonym for Y (ledger compat) 2022-12-21 22:27:08 -10:00
Simon Michael
826a7cfb2a imp: journal: ignore payee subdirectives 2022-12-21 22:09:15 -10:00
Simon Michael
bf6a2a7ccf imp: journal: ignore commodity subdirectives other than format 2022-12-21 22:08:54 -10:00
Simon Michael
ace185f7d2 ;doc: update old manual links 2022-12-10 18:56:47 -10:00
Simon Michael
b50d60cfea ;doc: csv, timeclock, timedot: clarify comment lines (#1953) 2022-12-06 10:38:50 -10:00
Simon Michael
78771e900b dev: lib: support debug logging to file in a few more places for hledger-ui 2022-11-06 09:01:12 -10:00
Simon Michael
988c164ec8 imp: debug logging improvements; hledger-ui logs to hledger-ui.log only
Hledger.Utils.Debug's "trace or log" functions are now controlled as
follows: to enable logging, append ",logging" to the program name at
startup (using withProgName). This also works when running in GHCI.
And they log to PROGNAME.log, not debug.log.

All (hopefully) debug logging in the hledger packages is now "trace or
log" capable.

This means that hledger-ui should now log all debug output to
./hledger-ui.log, with none of it appearing on the console.
2022-11-03 16:07:54 -10:00
Simon Michael
7fd25809e8 dev: fix customErrorBundlePretty import warnings 2022-10-07 07:43:28 -10:00
Simon Michael
de5a97600c imp: lib: Hledger.Read.Common: rename/add amount parsing helpers
removed:
 amountp'
 mamountp'

added:
 parseamount
 parseamount'
 parsemixedamount
 parsemixedamount'
2022-10-07 06:34:34 -10:00
Simon Michael
01387548e7 feat: csv: intra-day-reversed compensates when days' txns are reversed
As in eg vanguard CSV.
2022-10-06 22:21:55 -10:00
Simon Michael
15b2e7d586 fix: csv: ignore extra whitespace in account rule when detecting virtual postings
Reported by CruxOfTheB in chat.
2022-10-03 07:50:23 -10:00
Simon Michael
3b24d9465b imp: csv: new timezone rule; convert zoned date-times to local dates (#1936)
Previously, CSV date-times with a different time zone from yours
(with or without explicit timezones in the CSV) could give off-by-one
dates, because the CSV timezone was ignored.

Now,

1. you can use the `timezone` rule to indicate which other
   timezone a CSV is implicitly using

2. CSV date-times with a timezone - whether declared by rule or
   parsed with %Z - are localised to the system time zone
   (or another set with the TZ environment variable).
2022-10-01 14:50:35 -10:00
Simon Michael
bfb632dd8e dev: lib: Hledger.Data.Amount: drop amount, use nullamt always 2022-08-23 12:16:15 +01:00
Simon Michael
c80c72d7cd dev: lib, cli, bin: enable/fix name shadowing warnings
And a few other cleanups.
2022-08-23 12:16:15 +01:00
Simon Michael
b32b130045 dev: lib: fix a failing doctest 2022-08-18 05:08:10 +01:00
Simon Michael
3003a18c94 dev: lib: hlint improvements 2022-08-16 09:13:30 +01:00
Simon Michael
a3096d0549 dev: journalFilePath, include: show (unknown) instead of nothing
Probably not user-visible, but might improve debug logging slightly.
2022-08-16 09:13:30 +01:00
Simon Michael
1f08a8a94e fix: fix multi-file account display order; improve file read logging (#1909) 2022-08-16 09:13:30 +01:00
Simon Michael
07e3dca735 imp: at --debug 5, log account declarations info while parsing (#1909) 2022-08-16 09:13:30 +01:00
Simon Michael
3d6e363461 lib: save account directive positions, for troubleshooting (#1909) 2022-08-16 09:13:30 +01:00
Simon Michael
b46aa1330f fix: don't reverse account display order with include files (#1909) 2022-08-05 13:44:56 +01:00
Simon Michael
09d66c5fc1 fix: account display order is now tracked across all files (fix #1909) 2022-08-04 19:05:52 +01:00
Simon Michael
f135001b87 ;dev: improve followingcommentp' doc 2022-07-28 09:14:13 +01:00
Simon Michael
147856e3bb imp: errors: timeclock, csv error improvements 2022-07-23 02:35:52 +01:00
Stephen Morgan
c54fb4d514 imp: cost: Allow matching equity conversion equity postings to
transaction prices.

When given --infer-costs, hledger will now separately infer transaction
prices for different prices. Given a pair of adjacent conversion
postings, hledger will check if there is a single posting with a
transaction price which matches both the amounts. If so, it associates
those conversion postings to that priced post.

If it can't find any transaction price postings which match, it will
find the first non-transaction price post which matches one of the two
amounts, and will add a transaction price to that, and associate them.
2022-07-15 12:32:38 +01:00
Stephen Morgan
07d1b01287 imp: cost: Allow inferring cost from conversion equity postings with --infer-costs.
When given --infer-equity, hledger will change transaction prices to balancing equity postings. This introduces the inverse operation, --infer-costs, which will match balancing equity postings and transform them into a transaction price, allowing --cost to work properly with them. This is only a partial inverse as it needs to use some heuristics to match the postings which will not work in complicated cases.

Specifically, when hledger finds exactly two conversion postings in a transaction (by default, subaccounts of equity:conversion or equity:trad(e|ing)), it will find the first posting in the transaction whose amount is negative one of the conversion posting amounts, and inserts the corresponding transaction price.
2022-07-15 12:32:38 +01:00
Simon Michael
7d8fbe2dc0 dev: errors: update doctests 2022-07-13 19:00:00 +01:00
Simon Michael
1c67d0860e imp: errors: more error prettification 2022-07-13 03:59:44 +01:00
Simon Michael
dd14f9c822 imp: lib: Hledger.Utils.Parse: export HledgerParseErrors 2022-06-29 09:58:56 +01:00
Stephen Morgan
9155d679fe fix!: Revert "fix!: utf-8: Use with-utf8 to ensure all files are read and written with utf8 encoding. (#1619)"
This reverts commit e233f001c5.

This would break at least some people's workflow. A lighter touch is
probably sufficient.
2022-06-01 09:35:18 +10:00
Simon Michael
a9779b2377 ref: move journal checking/pretty errors down further, to Hledger.Data
now at Hledger.Data.JournalChecks*, Hledger.Data.Errors
2022-05-21 18:29:13 -10:00
Simon Michael
307f723b0a lib: simpler, more consistent names for check functions
API changes:
journalCheckAccountsDeclared
journalCheckCommoditiesDeclared
journalCheckPayeesDeclared
->
journalCheckAccounts
journalCheckCommodities
journalCheckPayees
2022-05-21 18:29:13 -10:00
Simon Michael
65e913b7c5
Merge pull request #1834 from Xitian9/utf8
Use with-utf8 and don't use Data.Text.IO.
2022-05-21 17:50:59 -10:00
Stephen Morgan
e233f001c5 fix!: utf-8: Use with-utf8 to ensure all files are read and written with utf8 encoding. (#1619)
May also fix #1154, #1033, #708, #536, #73: testing is needed.

This aims to solve all problems where misconfigured locales lead to
parsers failing on utf8-encoded data. This should hopefully avoid
encoding issues, but since it fundamentally alters how encoding is dealt
with it may lead to unexpected outcomes. Widespread testing on a number
of different platforms would be useful.
2022-05-22 13:12:19 +10:00
Stephen Morgan
15a5d5d38b
Merge pull request #1814 from Xitian9/csverror
imp: csv: Give an error if unable to substitute csv templates. (#1803)
2022-05-22 11:35:39 +10:00
Simon Michael
db26456e1c imp: consolidate ordereddates check under Hledger.Read.Checks too
And remove the last vestiges of older more complex behaviour.
ordereddates now always checks all transactions in each file,
unaffected by a query. (But still affected by --date2).
2022-05-09 00:05:39 -10:00
Simon Michael
810a868c88 ref: consolidate most checks under Hledger.Read.Checks
and error message helpers in Hledger.Read.Error.
2022-05-09 00:04:33 -10:00
Simon Michael
a00dc04e0d imp: check: uniqueleafnames: use the standard error format (#1436)
Hledger.Read.Common:
export makePostingErrorExcerpt
2022-04-25 02:56:59 -10:00
Simon Michael
7e45ab338f imp: check: show column ranges properly (#1436) 2022-04-25 02:56:59 -10:00
Simon Michael
66d0beea5e imp: check: ordereddates: use the standard error format (#1436)
Hledger.Read.Common:
export makeTransactionErrorExcerpt
2022-04-25 02:56:59 -10:00
Simon Michael
792434ca7d imp: check: payees: use the standard error format (#1436) 2022-04-25 02:56:59 -10:00
Simon Michael
c7e8f58c33 imp: check: mark all error columns when that's preferable (#1436)
Undeclared commodity errors now mark the whole amount and assertion
region, since locating the exact position of commodity symbols is
difficult.
2022-04-25 02:56:59 -10:00
Simon Michael
8f4405e628 dev: check: refactor error messages (#1436) 2022-04-25 02:56:59 -10:00
Simon Michael
6ec0e05c42 imp: check: commodities: use the standard error format (#1436) 2022-04-25 02:56:59 -10:00
Simon Michael
4b3644d780 imp: check: accounts: print a standardised error message like megaparsec's (#1436)
added:
formatExcerptLikeMegaparsec
2022-04-25 02:56:59 -10:00
Simon Michael
5ab7b9f643 ref: HledgerParseErrors type alias 2022-04-25 02:56:59 -10:00
Simon Michael
2f28e1b0a7 ref: rename CustomErr -> HledgerParseErrorData
Verbose, but use every chance to clarify the complicated parse error
situation.
2022-04-25 02:56:59 -10:00
Simon Michael
53332ee6a5 stack: re-enable hledger-web with ghc 9.2 2022-04-15 15:07:17 -10:00
Stephen Morgan
c48d98c515 imp: csv: Substitute empty string if csv template fails. (#1803) 2022-03-29 18:03:33 +11:00
Stephen Morgan
8968733630 cln!: Clean up journal parsing.
parseAndFinaliseJournal' has been removed. In the unlikely event you
needed it in your code, you can replace it with:
parseAndFinaliseJournal' parser iopts fp t =>
initialiseAndParseJournal parser iopts fp t
>>= liftEither . journalApplyAliases (aliasesFromOpts iopts)
>>= journalFinalise iopts fp t

Some parsers have been generalised from JournalParser to TextParser.
2022-03-26 15:35:19 -10:00
Simon Michael
a2f6b30fbd cln: silence some warnings/lints 2022-03-26 08:34:21 -10:00
Stephen Morgan
603b2e9f09 ref: Use ExceptT String IO a instead of IO (Either String a).
This increases composability and avoids some ugly case handling. We
re-export runExceptT in Hledger.Read.

The final return types of the following functions has been changed from
IO (Either String a) to ExceptT String IO a. If this causes a problem,
you can get the old behaviour by calling runExceptT on the output:
readJournal, readJournalFiles, readJournalFile

Or, you can use the easy functions readJournal', readJournalFiles', and
readJournalFile', which assume default options and return in the IO
monad.
2022-03-25 14:23:27 -10:00
Simon Michael
b584774a03 ci: fix a typo I fat-fingered 2022-03-18 11:03:30 -10:00
Simon Michael
2b2d643333 test: errors: make tests runnable 2022-03-18 06:59:33 -10:00
Stephen Morgan
e6bf04fce3 fix: aliases: Make sure we can escape forward slashes in account
aliases, but otherwise the regular expression handler handles escapes.
2022-03-16 19:01:03 -10:00
Stephen Morgan
a65e64115a fix: alias: Allow escaped characters in regular expression aliases. (#982) 2022-03-10 17:00:03 -10:00
Stephen Morgan
19ed6d3f00 cln!: sourcepos: Use megaparsec-supplied sourcePosPretty.
Change showSourcePos to sourcePosPretty, and showSourcePosPair to
sourcePosPairPretty.
2022-03-10 16:56:28 -10:00
Stephen Morgan
2f47ae05c6 fix: csv: Allow unicode in field references for csv. (#1809) 2022-02-06 14:16:17 -10:00