Commit Graph

726 Commits

Author SHA1 Message Date
Simon Michael
65e913b7c5
Merge pull request #1834 from Xitian9/utf8
Use with-utf8 and don't use Data.Text.IO.
2022-05-21 17:50:59 -10:00
Stephen Morgan
e233f001c5 fix!: utf-8: Use with-utf8 to ensure all files are read and written with utf8 encoding. (#1619)
May also fix #1154, #1033, #708, #536, #73: testing is needed.

This aims to solve all problems where misconfigured locales lead to
parsers failing on utf8-encoded data. This should hopefully avoid
encoding issues, but since it fundamentally alters how encoding is dealt
with it may lead to unexpected outcomes. Widespread testing on a number
of different platforms would be useful.
2022-05-22 13:12:19 +10:00
Stephen Morgan
15a5d5d38b
Merge pull request #1814 from Xitian9/csverror
imp: csv: Give an error if unable to substitute csv templates. (#1803)
2022-05-22 11:35:39 +10:00
Simon Michael
db26456e1c imp: consolidate ordereddates check under Hledger.Read.Checks too
And remove the last vestiges of older more complex behaviour.
ordereddates now always checks all transactions in each file,
unaffected by a query. (But still affected by --date2).
2022-05-09 00:05:39 -10:00
Simon Michael
810a868c88 ref: consolidate most checks under Hledger.Read.Checks
and error message helpers in Hledger.Read.Error.
2022-05-09 00:04:33 -10:00
Simon Michael
a00dc04e0d imp: check: uniqueleafnames: use the standard error format (#1436)
Hledger.Read.Common:
export makePostingErrorExcerpt
2022-04-25 02:56:59 -10:00
Simon Michael
7e45ab338f imp: check: show column ranges properly (#1436) 2022-04-25 02:56:59 -10:00
Simon Michael
66d0beea5e imp: check: ordereddates: use the standard error format (#1436)
Hledger.Read.Common:
export makeTransactionErrorExcerpt
2022-04-25 02:56:59 -10:00
Simon Michael
792434ca7d imp: check: payees: use the standard error format (#1436) 2022-04-25 02:56:59 -10:00
Simon Michael
c7e8f58c33 imp: check: mark all error columns when that's preferable (#1436)
Undeclared commodity errors now mark the whole amount and assertion
region, since locating the exact position of commodity symbols is
difficult.
2022-04-25 02:56:59 -10:00
Simon Michael
8f4405e628 dev: check: refactor error messages (#1436) 2022-04-25 02:56:59 -10:00
Simon Michael
6ec0e05c42 imp: check: commodities: use the standard error format (#1436) 2022-04-25 02:56:59 -10:00
Simon Michael
4b3644d780 imp: check: accounts: print a standardised error message like megaparsec's (#1436)
added:
formatExcerptLikeMegaparsec
2022-04-25 02:56:59 -10:00
Simon Michael
5ab7b9f643 ref: HledgerParseErrors type alias 2022-04-25 02:56:59 -10:00
Simon Michael
2f28e1b0a7 ref: rename CustomErr -> HledgerParseErrorData
Verbose, but use every chance to clarify the complicated parse error
situation.
2022-04-25 02:56:59 -10:00
Simon Michael
53332ee6a5 stack: re-enable hledger-web with ghc 9.2 2022-04-15 15:07:17 -10:00
Stephen Morgan
c48d98c515 imp: csv: Substitute empty string if csv template fails. (#1803) 2022-03-29 18:03:33 +11:00
Stephen Morgan
8968733630 cln!: Clean up journal parsing.
parseAndFinaliseJournal' has been removed. In the unlikely event you
needed it in your code, you can replace it with:
parseAndFinaliseJournal' parser iopts fp t =>
initialiseAndParseJournal parser iopts fp t
>>= liftEither . journalApplyAliases (aliasesFromOpts iopts)
>>= journalFinalise iopts fp t

Some parsers have been generalised from JournalParser to TextParser.
2022-03-26 15:35:19 -10:00
Simon Michael
a2f6b30fbd cln: silence some warnings/lints 2022-03-26 08:34:21 -10:00
Stephen Morgan
603b2e9f09 ref: Use ExceptT String IO a instead of IO (Either String a).
This increases composability and avoids some ugly case handling. We
re-export runExceptT in Hledger.Read.

The final return types of the following functions has been changed from
IO (Either String a) to ExceptT String IO a. If this causes a problem,
you can get the old behaviour by calling runExceptT on the output:
readJournal, readJournalFiles, readJournalFile

Or, you can use the easy functions readJournal', readJournalFiles', and
readJournalFile', which assume default options and return in the IO
monad.
2022-03-25 14:23:27 -10:00
Simon Michael
b584774a03 ci: fix a typo I fat-fingered 2022-03-18 11:03:30 -10:00
Simon Michael
2b2d643333 test: errors: make tests runnable 2022-03-18 06:59:33 -10:00
Stephen Morgan
e6bf04fce3 fix: aliases: Make sure we can escape forward slashes in account
aliases, but otherwise the regular expression handler handles escapes.
2022-03-16 19:01:03 -10:00
Stephen Morgan
a65e64115a fix: alias: Allow escaped characters in regular expression aliases. (#982) 2022-03-10 17:00:03 -10:00
Stephen Morgan
19ed6d3f00 cln!: sourcepos: Use megaparsec-supplied sourcePosPretty.
Change showSourcePos to sourcePosPretty, and showSourcePosPair to
sourcePosPairPretty.
2022-03-10 16:56:28 -10:00
Stephen Morgan
2f47ae05c6 fix: csv: Allow unicode in field references for csv. (#1809) 2022-02-06 14:16:17 -10:00
Stephen Morgan
73925ae965 fix: types: Ensure auto postings can match against and be matched by type: queries.
This requires checking parent accounts for any new accounts introduced by auto postings which do not exist in the original journal.

Also refactor journalFinalise to only call journalPostingsAddAccountTags once, and use fewer intermediate variables.
2022-02-03 15:32:00 -10:00
Stephen Morgan
a16c88b1b1 fix: Make sure cash and conversion account types are correctly
determined when there are no explicit declarations.
2022-02-03 15:32:00 -10:00
Simon Michael
45127dc5f5 feat: new type: query for easy matching by account type
type:TYPES, where TYPES is any of the (case insensitive) letters
ALERXCV, matches accounts by their declared or inferred type.
(See https://hledger.org/hledger.html#account-types.)
This should work with most commands, eg:

    hledger bal type:al
    hledger reg type:x

API changes:

Journal has a new jaccounttypes map.
The journalAccountType lookup function makes it easy to check an account's type.
The journalTags and journalInheritedTags functions look up an account's tags.
Functions like journalFilterPostings and journalFilterTransactions,
and new matching functions matchesAccountExtra, matchesPostingExtra
and matchesTransactionExtra, use these to allow more powerful matching
that is aware of account types and tags.
2022-01-31 16:44:13 -10:00
Simon Michael
885a4fcfec ref: journalFinalise: clarify sequence, top to bottom 2022-01-31 16:44:13 -10:00
Simon Michael
56be63e6f1 feat: queries can now match account tags (#1817)
Accounts, postings, and transactions can now all be filtered by the
tags in an account's declaration. In particular it's now possible to
more reliably select accounts by type, using their type: tag rather
than their name:

    account myasset       ; type:Asset
    account myliability   ; type:Liability

    $ hledger accounts tag:type=^a
    myasset

Accounts inherit tags from their parents.

API changes:
A finalised Journal has a new jdeclaredaccounttags field
for easy lookup of account tags.
Query.matchesTaggedAccount is a tag-aware version of matchesAccount.
2022-01-30 09:47:52 -10:00
Simon Michael
4ed80d4069 ;doc: update journalFinalise haddock 2022-01-30 09:47:52 -10:00
Stephen Morgan
df4949ad1a imp!: journal: Remove deprecated account type code syntax from account
directives.

Previously, you declare an account type with the following format:
account assets  A
This has been deprecated since 1.13, and should now be declared with one
of:
account assets  ; type:A
account assets  ; type:asset
2022-01-28 12:38:38 -10:00
Stephen Morgan
55a022a941 fix: strict: Make sure forecast transactions and auto postings are
checked with --strict. (#1810)
2022-01-26 07:57:13 -10:00
Stephen Morgan
57d055b643 fix: cost: Add inferred equity postings during journal finalisation, so
they can be matched by queries.
2022-01-16 18:37:18 -10:00
Stephen Morgan
5006e26995 imp: check: Do not check commodities for zero amounts with empty
commodity. (#1767)
2022-01-16 08:11:02 -10:00
Stephen Morgan
8eedbbbe87 imp: cost: Generate totally balanced conversion postings for amounts with costs.
Introduce --infer-equity option which will generate conversion postings.
--cost will override --infer-equity.

This means there will no longer be unbalanced transactions, but will be
offsetting conversion postings to balance things out. For example.

2000-01-01
  a   1 AAA @@ 2 BBB
  b  -2 BBB

When converting to cost, this is treated the same as before.
When used with --infer-equity, this is now treated as:

2000-01-01
  a                               1 AAA
  equity:conversion:AAA-BBB:AAA  -1 AAA
  equity:conversion:AAA-BBB:BBB   2 BBB
  b                              -2 BBB

There is a new account type, Conversion/V, which is a subtype of Equity/E.
The first account declared with this type, if any, is used as the base account
for inferred equity postings in conversion transactions, overriding the default
"equity:conversion".

API changes:

Costing has been changed to ConversionOp with three options:
NoConversionOp, ToCost, and InferEquity.
The first correspond to the previous NoCost and Cost options, while the
third corresponds to the --infer-equity flag.  This converts transactions with costs
(one or more transaction prices) to transactions with equity:conversion postings.
It is in ConversionOp because converting to cost with -B/--cost and inferring conversion
equity postings with --infer-equity are mutually exclusive.

Correspondingly, the cost_ record of ReportOpts has been changed to
conversionop_.

This also removes show_costs_ option in ReportOpts, as its functionality
has been replaced by the richer cost_ option.
2021-12-21 10:50:13 -10:00
Simon Michael
0d83bdf6d7 cln: csv: small rename 2021-12-08 16:57:53 -10:00
Simon Michael
1fee70b51f cln: hlint cleanups 2021-11-21 19:33:21 -10:00
Lawrence
fb817ffa55 imp: journal: support decimal-mark directive in journal files
To match csv-rule and override amount-style as described in
https://github.com/simonmichael/hledger/issues/1670
2021-11-18 22:17:07 -10:00
Stephen Morgan
e35d0b7865 fix: csv: Successfully parse empty csv file. (#1183) 2021-11-18 20:50:02 -10:00
Stephen Morgan
87a7a586d4 fix: csv: Handle multiple zero amounts in postings in csv files. (#1733) 2021-11-18 20:48:55 -10:00
Simon Michael
41bde20095 timedot: a D default commodity (and style) is applied to timedot aounts
This means they can be priced and converted.
2021-11-05 23:34:46 -10:00
Stephen Morgan
1bc04685b7 pkg: Drop base-compat-batteries dependency.
Our supported stackage versions are now new enough that we don't need
any of the compatibility features anymore.
2021-10-31 07:56:07 -10:00
Stephen Morgan
4cfd3cb590 lib!: Remove GenericSourcePos, and replace it with either SourcePos or
(SourcePos, SourcePos).

This has been marked for possible removal for a while. We are keeping
strictly more information. Possible edge cases arise with Timeclock and
CsvReader, but I think these are covered.

The particular motivation for getting rid of this is that
GenericSourcePos is creating some awkward import considerations for
little gain. Removing this enables some flattening of the module
dependency tree.
2021-09-20 08:38:33 -10:00
Stephen Morgan
df71d2ddd5 dev: Move journal and text parsing utilities into more appropriate places, to reduce reliance on Hledger.Read.Common. 2021-09-19 17:10:38 -10:00
Stephen Morgan
315e92c08b pkg!: Break functions related to balancing out into a separate module,
Hledger.Data.Balancing.

Both Hledger.Data.Transaction and Hledger.Data.Journal are massive
module with many things in them. Placing the balancing functions, which
are conceptually related, into a separate module helps keep things more
modular.

It also reduces the risk of import cycles, as right now balancing
functions cannot depend on any functions defined outside of
Hledger.Data.Transaction or Hledger.Data.Journal, respectively, if those
modules require basic transaction or journal functions.
2021-09-19 17:10:38 -10:00
Malte Brandy
e31eb58ada lib: Allow multiline comments in csv rules 2021-09-18 12:43:49 -10:00
Simon Michael
326d6e8dae ref: ReportOptions{infer_value_} renamed to infer_prices_
For more consistency with the flag name (--infer-market-prices).
And BalancingOpts{infer_prices_} is now infer_transaction_prices_.
2021-09-18 12:12:31 -10:00
Stephen Morgan
020dd15be4 ref: Simplify parsing of commodity-style option. 2021-09-18 11:41:53 -10:00