Commit Graph

864 Commits

Author SHA1 Message Date
Simon Michael
5a72b9e9ea feat:print: add --round option for more control of precisions (#2085) 2023-10-18 13:56:24 +01:00
Simon Michael
c13c13ab1f lib!: use styleAmounts in more places; add rounding strategies 2023-10-18 13:56:24 +01:00
Simon Michael
94ab8b1ed7 dev: journalApplyCommodityStyles -> journalStyleAmounts 2023-10-18 13:56:24 +01:00
Simon Michael
f8027abb44 dev:print: refactor, add AmountStyle "rounding strategy"
Changes to enable more control of "rounding" behaviour
(ie, choosing display precisions for amounts).

This reverts 1.31's change of asprecision, making it a non-Maybe
again, and adds a new asrounding field providing more control over how
a target display precision is applied to existing amounts (two options
for now, more later). Functionality is in an interim state (reports do
no rounding).
2023-10-18 13:56:24 +01:00
Simon Michael
9ead8a64e1 fix:timedot: parse unitful quantities more accurately (fix #2096)
A quantity with a unit like "15m" was being parsed internally as
0.249999999... rather than 0.25 (and since hledger 1.21, printed that
way also). Now we round such quantities to two places during parsing,
to get exact quarter-hour amounts.
2023-10-17 12:05:10 +01:00
Simon Michael
e8da1f9f25 fix: balance-assigned amounts affect styles again (fix #2091) 2023-10-14 08:38:20 +01:00
bobobo1618
9fb5740045 Add support for negating a Matcher
https://github.com/simonmichael/hledger/issues/2054
2023-10-05 10:22:01 +01:00
Simon Michael
97be1646f1 dev: AmountStyle: make setting the precision optional
This simplifies the code for styling amounts with or without precision.
But it complicates the semantics (Nothing is useful only when setting style).
Not sure if it's the best way.
2023-09-02 06:46:14 +01:00
Simon Michael
85845e51b2 dev: AmountStyle: rename, reorder fields more mnemonically
Since this type is about to change anyway.
2023-09-02 06:46:14 +01:00
Simon Michael
5f1bf0404a imp:check: more intuitive "balanced..." check names
balancedwithautoconversion -> autobalanced
balancednoautoconversion   -> balanced
2023-08-22 08:38:52 +01:00
Simon Michael
1cfcb7fa76 ;cln: journalAddInferredEquityPostings -> journalInferEquityFromCosts 2023-07-04 13:21:55 -10:00
Simon Michael
e832c344c1 dev: comment 2023-05-20 17:29:06 -10:00
Simon Michael
029b59093b feat: csv: rules files can be read directly; data file can be specified
CSV rules files can now be read directly, eg you have the option of
writing `hledger -f foo.csv.rules CMD`. By default this will read data
from foo.csv in the same directory.  But you can also specify a
different data file with a new `source FILE` rule. This has some
convenience features:

- If the data file does not exist, it is treated as empty, not an
  error.

- If FILE is a relative path, it is relative to the rules file's
  directory. If it is just a file name with no path, it is relative
  to ~/Downloads/.

- If FILE is a glob pattern, the most recently modified matched file
  is used.

This helps remove some of the busywork of managing CSV downloads.
Most of your financial institutions's default CSV filenames are
different and can be recognised by a glob pattern.  So you can put a
rule like `source Checking1*.csv` in foo-checking.csv.rules,
periodically download CSV from Foo's website accepting your browser's
defaults, and then run `hledger import checking.csv.rules` to import
any new transactions. The next time, if you have done no cleanup, your
browser will probably save it as something like Checking1-2.csv, and
hledger will still see that because of the * wild card. You can choose
whether to delete CSVs after import, or keep them for a while as
temporary backups, or archive them somewhere.
2023-05-19 09:09:21 -10:00
Simon Michael
577e4b6347 fix!: csv: skip now counts non-blank lines more robustly (fix #2024)
Inner empty lines were not being skipped automatically, contrary to
docs. Now all empty lines are skipped automatically, and the `skip`
rule is needed only for non-empty lines, as intended.

This may be a breaking change: it's possible that the `skip` count
might need to be adjusted in some CSV rules files.
2023-05-11 17:06:12 -10:00
Simon Michael
69be1d4ef7 ;dev: csv: refactor, clarify 2023-05-11 15:44:56 -10:00
Simon Michael
70d4c0c638 ;dev: csv: refactor, clarify 2023-05-11 15:35:05 -10:00
Simon Michael
755c3d3dbb ;dev: csv: refactor 2023-05-11 15:34:31 -10:00
Simon Michael
bbecd611f1 imp: timedot: make one multi-posting transaction per date line (#1754)
Descriptions and comments are now more straightforward and similar to
journal format.
2023-05-03 19:18:35 -10:00
Simon Michael
50349f81f7 imp: timeclock: support comments and tags (fix #1220)
Breaking change: previously timeclock descriptions could contain
semicolons. Now a semicolon in the description will end it and
start a comment (which may contain tags).
2023-05-03 00:05:11 -10:00
Simon Michael
f7f86a709b imp: timedot: fix day description/comment parsing; parse posting comments/tags 2023-05-02 17:03:03 -10:00
Simon Michael
1b19f3d330 dev: timedot: clarify, cleanup 2023-05-02 17:02:30 -10:00
Simon Michael
a771c8fc19 imp: revert 2b5194238 imp: generate auto postings on forecast transactions by default (#2027)
I found at least one user for whom this would be a breaking change
(they generate forecast txns, and have auto posting rules, but don't
want the latter applied to the former). I guess it's better to keep
things as they were for now: if you need auto postings on your
forecast txns you must use two flags, --forecast --auto.
2023-04-30 08:08:26 -10:00
Simon Michael
7f713f6a44 imp: Generated/modified txns/postings are now tagged only with --verbose-tags 2023-04-29 18:25:11 -10:00
Simon Michael
2b5194238b imp: generate auto postings on forecast transactions by default 2023-04-29 16:00:14 -10:00
Chris Lemaire
cddbae6467 queries: Prefix boolean queries with expr:
Boolean queries are now prefixed with an 'expr:' prefix, making them
completely separable from old queries and making the addition of them a
little more migration proof.

The tests are updated accordingly, changes made to the tests previously
are removed and extra cautious documentation is also removed.
2023-03-27 10:29:26 -10:00
Simon Michael
c790aa6145 ;dev: lib: also build with GHC 9.6.1; add base-compat 2023-03-14 10:42:48 -10:00
Simon Michael
fa70f160ae imp: partial/inferred dates are flexible, full dates are not (#1982)
DateSpans are now now aware of exact/flexible dates.
2023-02-17 07:24:19 -10:00
Simon Michael
5537a251f3 imp: journal: periodic txns need not start on an interval boundary
Eg, ~ monthly from 1/15 now works, instead of giving an error message.
2023-02-17 07:24:19 -10:00
Simon Michael
7a9b0fd94c feat: check: the tags check checks tag names 2023-02-16 11:56:22 -10:00
Simon Michael
c51ebe7b8c imp: journal: remove the "adjacent conversion postings" check again
I find that single conversion postings are sometimes needed, eg with
paypal.
2023-02-01 13:30:21 -10:00
Simon Michael
de330bdfce imp: journal: check for adjacent conversion postings only in strict mode
This avoids breaking existing journals if, say, they happen to have
a single "equity:trading" posting somewhere.
2023-01-25 19:00:29 -10:00
Simon Michael
76f8eaf91a imp: journal: ignore redundant costs; --infer-costs more robustly
You can now write both @/@@ costs and corresponding equity conversion postings
in a transaction at any time, not just when using --infer-costs or --infer-equity.
hledger will recognise the redundancy and ignore it.

One thing has become more strict: hledger now requires conversion postings
to occur in adjacent pairs; an odd number of them is not allowed.
(Conversion postings are postings to accounts of type `V`/`Conversion`,
or named `equity:conversion`, `equity:trade`, `equity:trading`,
or subaccounts of these.)

And, --infer-costs now works in transactions with an implicit amount
(inferring costs from equity now happens after transaction balancing,
not before).
2023-01-25 19:00:29 -10:00
Chris Lemaire
43c9f018dc journal: Account directives reject account names with brackets
Previously, the accounts passed to account directives would be stripped
of their surrounding brackets, but the required behaviour is to have
account directives plain reject bracketed accounts. This change ensures
that accounts in account directives may not start with a bracket
character.
2023-01-24 13:40:01 -10:00
Simon Michael
9497472e64
Merge branch 'master' into 1984-source-name-and-position-for-forecast 2023-01-23 16:04:00 -10:00
Chris Lemaire
4ada2a3fbc journal: Fully unbracket AccountNames in account directives
Currently an account name like "a:(aa)" will not have (aa) unbracketed.
However, this seems reasonable since the full name is unbracketed and
thus will not be confused with virtual or virtual-balanced posting.
2023-01-23 12:43:13 -10:00
Chris Lemaire
9443fe0e0d journal: json: Add source positions to forecast transactions 2023-01-21 22:30:30 +01:00
Simon Michael
5b27554f0a dev: note why inferring costs from equity before balancing 2023-01-19 20:57:02 -10:00
Simon Michael
cf469c080d lib: journalInferCosts -> journalInferCostsFromEquity 2023-01-19 20:25:41 -10:00
Simon Michael
28eb8be4fa lib: BalancingOpts{infer_transaction_prices_ -> infer_balancing_costs_} 2023-01-19 20:21:31 -10:00
Simon Michael
dfea6eaee2 lib: Hledger.Data.Journal: journalAddPricesFromEquity -> journalInferCosts 2023-01-19 09:17:56 -10:00
Simon Michael
dfebf3174c imp: csv: check assigned account names are valid (parseable) (#1978) 2023-01-11 21:42:47 -10:00
Simon Michael
a9b63bb694 fix: csv: skip header lines before attempting to parse records (#1967) 2022-12-27 12:21:20 -10:00
Simon Michael
4b36d852b4 imp: journal: ignore ((valuation expressions)) after amounts (ledger compat) 2022-12-22 20:06:28 -10:00
Simon Michael
1ea2bcc83f imp: journal: also parse (lot notes) in amounts (ledger compat)
and rename lotpricep -> lotcostp
and instrument some amount parsers for debugging with megaparsec's dbg
2022-12-22 19:07:40 -10:00
Simon Michael
aa54c3273a dev: lib: clarify some amount parsers; describe Ledger lot notation
amountpwithmultiplier -> amountp'
amountpnolotpricesp   -> amountnobasisp
amountwithoutpricep   -> simpleamountp
priceamountp          -> costp
2022-12-22 09:15:43 -10:00
Simon Michael
b1e08ffe46 imp: journal: python directive also consumes empty lines (ledger compat) 2022-12-22 02:30:29 -10:00
Simon Michael
85e9fbcf7d imp: journal: allow an empty double-quoted commodity symbol
"" is equivalent to writing no symbol.
But it's now possible to declare market prices for the no-symbol
commodity, which occurs eg with timedot data.

P 2022-01-01 "" $100
2022-12-22 01:53:06 -10:00
Simon Michael
a070875fe6 imp: journal: ignore end apply year, end apply tag, drop "pop" (ledger compat) 2022-12-22 01:50:27 -10:00
Simon Michael
a5d6b42322 imp: journal: ignore end apply fixed, end apply tag directives (ledger compat) 2022-12-22 01:12:56 -10:00
Simon Michael
af5645ebb7 imp: journal: ignore python, eval directives (ledger compat) 2022-12-22 00:50:43 -10:00
Simon Michael
2e77586d1b imp: journal: ignore value directives (ledger compat) 2022-12-22 00:38:51 -10:00
Simon Michael
fb7134c9a0 imp: journal: ignore tag subdirectives (ledger compat) 2022-12-22 00:34:33 -10:00
Simon Michael
c58b9e6ac9 imp: journal: accept apply year as another synonym for Y (ledger compat) 2022-12-22 00:27:35 -10:00
Simon Michael
2357a7a750 dev: journal: ignore CLI flags in journal 2022-12-21 23:40:43 -10:00
Simon Michael
abd104ccee imp: journal: ignore more ledger directives (ledger compat)
apply fixed, apply tag, assert, bucket, A, capture, check, define, expr

And reorder the directivep parsers.
2022-12-21 23:04:58 -10:00
Simon Michael
e09ab0936a imp: journal: accept @ as a prefix of directives (ledger compat) 2022-12-21 22:56:32 -10:00
Simon Michael
95d7679f6d imp: journal: accept year as a synonym for Y (ledger compat) 2022-12-21 22:27:08 -10:00
Simon Michael
826a7cfb2a imp: journal: ignore payee subdirectives 2022-12-21 22:09:15 -10:00
Simon Michael
bf6a2a7ccf imp: journal: ignore commodity subdirectives other than format 2022-12-21 22:08:54 -10:00
Simon Michael
ace185f7d2 ;doc: update old manual links 2022-12-10 18:56:47 -10:00
Simon Michael
b50d60cfea ;doc: csv, timeclock, timedot: clarify comment lines (#1953) 2022-12-06 10:38:50 -10:00
Simon Michael
78771e900b dev: lib: support debug logging to file in a few more places for hledger-ui 2022-11-06 09:01:12 -10:00
Simon Michael
988c164ec8 imp: debug logging improvements; hledger-ui logs to hledger-ui.log only
Hledger.Utils.Debug's "trace or log" functions are now controlled as
follows: to enable logging, append ",logging" to the program name at
startup (using withProgName). This also works when running in GHCI.
And they log to PROGNAME.log, not debug.log.

All (hopefully) debug logging in the hledger packages is now "trace or
log" capable.

This means that hledger-ui should now log all debug output to
./hledger-ui.log, with none of it appearing on the console.
2022-11-03 16:07:54 -10:00
Simon Michael
7fd25809e8 dev: fix customErrorBundlePretty import warnings 2022-10-07 07:43:28 -10:00
Simon Michael
de5a97600c imp: lib: Hledger.Read.Common: rename/add amount parsing helpers
removed:
 amountp'
 mamountp'

added:
 parseamount
 parseamount'
 parsemixedamount
 parsemixedamount'
2022-10-07 06:34:34 -10:00
Simon Michael
01387548e7 feat: csv: intra-day-reversed compensates when days' txns are reversed
As in eg vanguard CSV.
2022-10-06 22:21:55 -10:00
Simon Michael
15b2e7d586 fix: csv: ignore extra whitespace in account rule when detecting virtual postings
Reported by CruxOfTheB in chat.
2022-10-03 07:50:23 -10:00
Simon Michael
3b24d9465b imp: csv: new timezone rule; convert zoned date-times to local dates (#1936)
Previously, CSV date-times with a different time zone from yours
(with or without explicit timezones in the CSV) could give off-by-one
dates, because the CSV timezone was ignored.

Now,

1. you can use the `timezone` rule to indicate which other
   timezone a CSV is implicitly using

2. CSV date-times with a timezone - whether declared by rule or
   parsed with %Z - are localised to the system time zone
   (or another set with the TZ environment variable).
2022-10-01 14:50:35 -10:00
Simon Michael
bfb632dd8e dev: lib: Hledger.Data.Amount: drop amount, use nullamt always 2022-08-23 12:16:15 +01:00
Simon Michael
c80c72d7cd dev: lib, cli, bin: enable/fix name shadowing warnings
And a few other cleanups.
2022-08-23 12:16:15 +01:00
Simon Michael
b32b130045 dev: lib: fix a failing doctest 2022-08-18 05:08:10 +01:00
Simon Michael
3003a18c94 dev: lib: hlint improvements 2022-08-16 09:13:30 +01:00
Simon Michael
a3096d0549 dev: journalFilePath, include: show (unknown) instead of nothing
Probably not user-visible, but might improve debug logging slightly.
2022-08-16 09:13:30 +01:00
Simon Michael
1f08a8a94e fix: fix multi-file account display order; improve file read logging (#1909) 2022-08-16 09:13:30 +01:00
Simon Michael
07e3dca735 imp: at --debug 5, log account declarations info while parsing (#1909) 2022-08-16 09:13:30 +01:00
Simon Michael
3d6e363461 lib: save account directive positions, for troubleshooting (#1909) 2022-08-16 09:13:30 +01:00
Simon Michael
b46aa1330f fix: don't reverse account display order with include files (#1909) 2022-08-05 13:44:56 +01:00
Simon Michael
09d66c5fc1 fix: account display order is now tracked across all files (fix #1909) 2022-08-04 19:05:52 +01:00
Simon Michael
f135001b87 ;dev: improve followingcommentp' doc 2022-07-28 09:14:13 +01:00
Simon Michael
147856e3bb imp: errors: timeclock, csv error improvements 2022-07-23 02:35:52 +01:00
Stephen Morgan
c54fb4d514 imp: cost: Allow matching equity conversion equity postings to
transaction prices.

When given --infer-costs, hledger will now separately infer transaction
prices for different prices. Given a pair of adjacent conversion
postings, hledger will check if there is a single posting with a
transaction price which matches both the amounts. If so, it associates
those conversion postings to that priced post.

If it can't find any transaction price postings which match, it will
find the first non-transaction price post which matches one of the two
amounts, and will add a transaction price to that, and associate them.
2022-07-15 12:32:38 +01:00
Stephen Morgan
07d1b01287 imp: cost: Allow inferring cost from conversion equity postings with --infer-costs.
When given --infer-equity, hledger will change transaction prices to balancing equity postings. This introduces the inverse operation, --infer-costs, which will match balancing equity postings and transform them into a transaction price, allowing --cost to work properly with them. This is only a partial inverse as it needs to use some heuristics to match the postings which will not work in complicated cases.

Specifically, when hledger finds exactly two conversion postings in a transaction (by default, subaccounts of equity:conversion or equity:trad(e|ing)), it will find the first posting in the transaction whose amount is negative one of the conversion posting amounts, and inserts the corresponding transaction price.
2022-07-15 12:32:38 +01:00
Simon Michael
7d8fbe2dc0 dev: errors: update doctests 2022-07-13 19:00:00 +01:00
Simon Michael
1c67d0860e imp: errors: more error prettification 2022-07-13 03:59:44 +01:00
Simon Michael
dd14f9c822 imp: lib: Hledger.Utils.Parse: export HledgerParseErrors 2022-06-29 09:58:56 +01:00
Stephen Morgan
9155d679fe fix!: Revert "fix!: utf-8: Use with-utf8 to ensure all files are read and written with utf8 encoding. (#1619)"
This reverts commit e233f001c5.

This would break at least some people's workflow. A lighter touch is
probably sufficient.
2022-06-01 09:35:18 +10:00
Simon Michael
a9779b2377 ref: move journal checking/pretty errors down further, to Hledger.Data
now at Hledger.Data.JournalChecks*, Hledger.Data.Errors
2022-05-21 18:29:13 -10:00
Simon Michael
307f723b0a lib: simpler, more consistent names for check functions
API changes:
journalCheckAccountsDeclared
journalCheckCommoditiesDeclared
journalCheckPayeesDeclared
->
journalCheckAccounts
journalCheckCommodities
journalCheckPayees
2022-05-21 18:29:13 -10:00
Simon Michael
65e913b7c5
Merge pull request #1834 from Xitian9/utf8
Use with-utf8 and don't use Data.Text.IO.
2022-05-21 17:50:59 -10:00
Stephen Morgan
e233f001c5 fix!: utf-8: Use with-utf8 to ensure all files are read and written with utf8 encoding. (#1619)
May also fix #1154, #1033, #708, #536, #73: testing is needed.

This aims to solve all problems where misconfigured locales lead to
parsers failing on utf8-encoded data. This should hopefully avoid
encoding issues, but since it fundamentally alters how encoding is dealt
with it may lead to unexpected outcomes. Widespread testing on a number
of different platforms would be useful.
2022-05-22 13:12:19 +10:00
Stephen Morgan
15a5d5d38b
Merge pull request #1814 from Xitian9/csverror
imp: csv: Give an error if unable to substitute csv templates. (#1803)
2022-05-22 11:35:39 +10:00
Simon Michael
db26456e1c imp: consolidate ordereddates check under Hledger.Read.Checks too
And remove the last vestiges of older more complex behaviour.
ordereddates now always checks all transactions in each file,
unaffected by a query. (But still affected by --date2).
2022-05-09 00:05:39 -10:00
Simon Michael
810a868c88 ref: consolidate most checks under Hledger.Read.Checks
and error message helpers in Hledger.Read.Error.
2022-05-09 00:04:33 -10:00
Simon Michael
a00dc04e0d imp: check: uniqueleafnames: use the standard error format (#1436)
Hledger.Read.Common:
export makePostingErrorExcerpt
2022-04-25 02:56:59 -10:00
Simon Michael
7e45ab338f imp: check: show column ranges properly (#1436) 2022-04-25 02:56:59 -10:00
Simon Michael
66d0beea5e imp: check: ordereddates: use the standard error format (#1436)
Hledger.Read.Common:
export makeTransactionErrorExcerpt
2022-04-25 02:56:59 -10:00
Simon Michael
792434ca7d imp: check: payees: use the standard error format (#1436) 2022-04-25 02:56:59 -10:00
Simon Michael
c7e8f58c33 imp: check: mark all error columns when that's preferable (#1436)
Undeclared commodity errors now mark the whole amount and assertion
region, since locating the exact position of commodity symbols is
difficult.
2022-04-25 02:56:59 -10:00
Simon Michael
8f4405e628 dev: check: refactor error messages (#1436) 2022-04-25 02:56:59 -10:00
Simon Michael
6ec0e05c42 imp: check: commodities: use the standard error format (#1436) 2022-04-25 02:56:59 -10:00