Commit Graph

180 Commits

Author SHA1 Message Date
Simon Michael
8951e11584 fix:import: don't erase .latest files when there are no new txns [#2444]
Fixes a regression caused by the #2125 fix.
2025-08-25 19:25:52 +01:00
Simon Michael
e2bb24e902 dev: make LatestDatesForFile showable 2025-08-25 19:24:38 +01:00
Simon Michael
db7783b16d dev: cleanups 2025-08-14 12:54:40 +01:00
Simon Michael
bfbef4bcbb dev: refactor PrefixedFilePath 2025-08-14 12:37:11 +01:00
Simon Michael
820a44eb07 imp:lib:Hledger.Utils.Debug: simpler, more consistent dbg* names 2025-05-21 22:54:00 -10:00
Simon Michael
151b36a6f2 ;dev: haddock edits 2025-05-16 09:42:55 -10:00
Simon Michael
bca4b39c58 lib: defaultJournalSafely: don't strip newline 2025-04-25 15:42:47 -10:00
Simon Michael
f0f048326e lib: defaultJournalSafely wasn't very safe; now catches error calls 2025-04-25 15:21:11 -10:00
Simon Michael
255e79c550 lib: Hledger.Read: more ways to read the default journal
defaultJournalWith, defaultJournalSafely, defaultJournalSafelyWith
2025-04-22 18:50:42 -10:00
Simon Michael
b7509030c5 lib: Hledger.Data.Dates: parsedateM -> parsedate
M makes me think of monad and is awkward to type for this common helper; Maybe is expected for a parser.
2025-04-22 10:40:41 -10:00
Simon Michael
f37a4a7dcb lib: refactor, export getHomeSafe 2025-04-20 10:52:50 -10:00
Simon Michael
9340b73aae imp: improve/format errors for various failures [#2367]
These now call error' and show errors in the standard style:

- reading a nonexistent data file
- reading an unsafe dotted file name on windows
- web: using --socket on windows
- demo: demo not found
- demo: error while running asciinema
- diff: bad arguments
- print --match: no match found
- register --match: no match found
- roi: no investment transactions found
2025-04-11 08:06:47 -10:00
Simon Michael
d47107ab7a ;dev:readJournal: doc 2025-03-06 05:39:23 -10:00
Simon Michael
874771d47c dev:readJournal: assume default file name "-" rather than "(string)" [#2328]
"-" implies data from standard input so "string" is perhaps more
correct, but I think this is a harmless simplification and it
makes `files` output consistent when run by `run`.
2025-03-06 05:13:37 -10:00
Joschua Kesper
5114962b2a feat:csv: add an encoding rule, allowing non-UTF8 CSV to be read [#2319]
Previously, hledger could read CSV files containing non-ascii
characters only if they are UTF8-encoded.  Now there is a new CSV
rule, encoding ENCODING, which allows reading CSV files with other
encodings.

This adds a dependency on the encoding library, which supports fewer
encodings than text-icu but does not require a third-party C library.
To avoid build issues on various platforms, we require version 0.10+.

This adds some use of the ImplicitParams language extension, required
by encoding's API, but only in a small code region.

This also changes the type of Reader's rReadFn; it now takes
a `Handle` rather than a `Text`, allowing more flexibility.
2025-02-15 14:48:30 -10:00
Simon Michael
1048791810 imp:csv: ignore whitespace, show error position when reading .latest files 2024-12-05 16:35:18 -10:00
Simon Michael
243a96b656 dev: clarify how readJournalFiles defers checks, and use a dedicated flag 2024-05-02 21:36:32 -10:00
Simon Michael
ea58a7ccd7 imp: check: check ordereddates before balance assertions, when both are enabled 2024-05-02 21:11:04 -10:00
Simon Michael
55401282a0 imp: run checks in a well-defined order; and tweak that order
Now commodities are checked before accounts, and tags before recentassertions.
Also some check doc cleanups.
2024-04-26 18:59:45 -10:00
Simon Michael
c7dd1fcc2f ;imp:cli:errors: make file-not-found error more format agnostic 2024-02-26 10:25:22 -10:00
Simon Michael
f6a50923c2 imp: interpret glob patterns in $LEDGER_FILE
For example, LEDGER_FILE=2???.journal now works.
2024-02-22 07:32:37 -10:00
Simon Michael
9ccc1d3fa5 fix:add,import,web: really avoid filenames ending with . on Windows [#1056]
On MS Windows, trying to add or import or web add to a file whose name
ends with a dot could cause data loss, so in 2019 I made this raise an
error instead (in Hledger.Read.ensureJournalFileExists).

But, the logic was backward, so it did not do the check on Windows.
Now it does.

Also I have removed mention of this from add's doc; currently it's
not documented anywhere. It's obscure, but maybe this is not ideal.
2024-01-04 17:35:22 -10:00
Simon Michael
c6a580ff3b fix:import: save each file's latest dates, separately (#2125) 2023-12-07 03:41:31 -10:00
Simon Michael
d1635a55f8 lib: move readFileStrictly to Hledger.Utils.IO 2023-12-07 03:41:31 -10:00
Simon Michael
d717732928 ;doc: Hledger.Read: cleanups (#2113) 2023-11-16 23:37:20 -10:00
Simon Michael
037613abab ;doc: Hledger.Read: cleanups (#2113) 2023-11-16 23:28:14 -10:00
Simon Michael
e92ab28cce imp:reading: better timing of strict checks and .latest writing (#2113)
Strict checks now run only once, at end of the high level read operation,
and not for each individual file; this fixes some spurious --strict failures,
like account declarations not affecting a sibling file as they should.

And .latest file writing now happens as the last step, after passing
strict checks. This is mainly for the import command, but it also
means that hledger print --new now does not update .latest files
if strict checks are failing.

The file reading API has been improved and documented in more detail.
2023-11-16 21:48:43 -10:00
Simon Michael
029b59093b feat: csv: rules files can be read directly; data file can be specified
CSV rules files can now be read directly, eg you have the option of
writing `hledger -f foo.csv.rules CMD`. By default this will read data
from foo.csv in the same directory.  But you can also specify a
different data file with a new `source FILE` rule. This has some
convenience features:

- If the data file does not exist, it is treated as empty, not an
  error.

- If FILE is a relative path, it is relative to the rules file's
  directory. If it is just a file name with no path, it is relative
  to ~/Downloads/.

- If FILE is a glob pattern, the most recently modified matched file
  is used.

This helps remove some of the busywork of managing CSV downloads.
Most of your financial institutions's default CSV filenames are
different and can be recognised by a glob pattern.  So you can put a
rule like `source Checking1*.csv` in foo-checking.csv.rules,
periodically download CSV from Foo's website accepting your browser's
defaults, and then run `hledger import checking.csv.rules` to import
any new transactions. The next time, if you have done no cleanup, your
browser will probably save it as something like Checking1-2.csv, and
hledger will still see that because of the * wild card. You can choose
whether to delete CSVs after import, or keep them for a while as
temporary backups, or archive them somewhere.
2023-05-19 09:09:21 -10:00
Simon Michael
c790aa6145 ;dev: lib: also build with GHC 9.6.1; add base-compat 2023-03-14 10:42:48 -10:00
Simon Michael
988c164ec8 imp: debug logging improvements; hledger-ui logs to hledger-ui.log only
Hledger.Utils.Debug's "trace or log" functions are now controlled as
follows: to enable logging, append ",logging" to the program name at
startup (using withProgName). This also works when running in GHCI.
And they log to PROGNAME.log, not debug.log.

All (hopefully) debug logging in the hledger packages is now "trace or
log" capable.

This means that hledger-ui should now log all debug output to
./hledger-ui.log, with none of it appearing on the console.
2022-11-03 16:07:54 -10:00
Simon Michael
c80c72d7cd dev: lib, cli, bin: enable/fix name shadowing warnings
And a few other cleanups.
2022-08-23 12:16:15 +01:00
Simon Michael
1f08a8a94e fix: fix multi-file account display order; improve file read logging (#1909) 2022-08-16 09:13:30 +01:00
Stephen Morgan
9155d679fe fix!: Revert "fix!: utf-8: Use with-utf8 to ensure all files are read and written with utf8 encoding. (#1619)"
This reverts commit e233f001c5.

This would break at least some people's workflow. A lighter touch is
probably sufficient.
2022-06-01 09:35:18 +10:00
Simon Michael
a9779b2377 ref: move journal checking/pretty errors down further, to Hledger.Data
now at Hledger.Data.JournalChecks*, Hledger.Data.Errors
2022-05-21 18:29:13 -10:00
Stephen Morgan
e233f001c5 fix!: utf-8: Use with-utf8 to ensure all files are read and written with utf8 encoding. (#1619)
May also fix #1154, #1033, #708, #536, #73: testing is needed.

This aims to solve all problems where misconfigured locales lead to
parsers failing on utf8-encoded data. This should hopefully avoid
encoding issues, but since it fundamentally alters how encoding is dealt
with it may lead to unexpected outcomes. Widespread testing on a number
of different platforms would be useful.
2022-05-22 13:12:19 +10:00
Simon Michael
810a868c88 ref: consolidate most checks under Hledger.Read.Checks
and error message helpers in Hledger.Read.Error.
2022-05-09 00:04:33 -10:00
Simon Michael
d60e1464d5 imp: fix MonadIO import (#1847) 2022-03-25 14:49:56 -10:00
Stephen Morgan
31c7c5d1ca fix: Clean up build failures due to redundant imports and duplicate
function.
2022-03-26 11:37:32 +11:00
Stephen Morgan
603b2e9f09 ref: Use ExceptT String IO a instead of IO (Either String a).
This increases composability and avoids some ugly case handling. We
re-export runExceptT in Hledger.Read.

The final return types of the following functions has been changed from
IO (Either String a) to ExceptT String IO a. If this causes a problem,
you can get the old behaviour by calling runExceptT on the output:
readJournal, readJournalFiles, readJournalFile

Or, you can use the easy functions readJournal', readJournalFiles', and
readJournalFile', which assume default options and return in the IO
monad.
2022-03-25 14:23:27 -10:00
Stephen Morgan
df71d2ddd5 dev: Move journal and text parsing utilities into more appropriate places, to reduce reliance on Hledger.Read.Common. 2021-09-19 17:10:38 -10:00
Stephen Morgan
8274da81fc cln: tests: Remove test and tests, which are just aliases for testCase
and testGroup.

Replacing these removes a layer of indirection, and reduces the need to
depend on Hledger.Utils.Test.
2021-08-30 16:32:19 -10:00
Stephen Morgan
21e62ffcbd cln: hlint: Remove unless and $> warnings. 2021-08-27 06:13:56 -10:00
Stephen Morgan
22db5c4a3f cln: hlint: Remove warnings to use library list functions. 2021-08-27 06:13:56 -10:00
Stephen Morgan
c07ad29a87 imp!: forecast: Implements more intuitive logic for the forecast interval. (#1648)
The forecast period begins on:
- the start date supplied to the `--forecast` argument, if present
- otherwise, the later of
  - the report start date if specified with -b/-p/date:
  - the day after the latest normal (non-periodic) transaction in the journal, if any
- otherwise today.
It ends on:
- the end date supplied to the `--forecast` argument, if present
- otherwise the report end date if specified with -e/-p/date:
- otherwise 180 days (6 months) from today.

Note that the previous behaviour did not quite match the documentation,
so this also acts as a bug fix for #1665.
2021-08-26 20:32:30 -10:00
Arjen Langebaerd
3426030a91 feat: added commodity style commandline option 2021-08-17 22:05:29 -10:00
Stephen Morgan
7ed2a0aa9b lib!: lib: Remove aismultiplier from Amount.
In Amount, aismultiplier is a boolean flag that will always be False,
except for in TMPostingRules, where it indicates whether the posting
rule is a multiplier. It is therefore unnecessary in the vast majority
of cases. This posting pulls this flag out of Amount and puts it into
TMPostingRule, so it is only kept around when necessary.

This changes the parsing of journals somewhat. Previously you could
include an * before an amount anywhere in a Journal, and it would
happily parse and set the aismultiplier flag true. This will now fail
with a parse error: * is now only acceptable before an amount within an
auto posting rule.

Any usage of the library in which the aismultiplier field is read or set
should be removed. If you truly need its functionality, you should
switch to using TMPostingRule.

This changes the JSON output of Amount, as it will no longer include
aismultiplier.
2021-07-22 19:06:33 -10:00
Stephen Morgan
2df8ad781e
imp: cli: Don't show extra double quotes in missing journal file error. (#1601) 2021-07-13 10:41:51 -10:00
Stephen Morgan
55308e1ca8 lib,cli,ui,web: Remove unnecessary CPP when dropping support for GHC 8.2. 2021-06-07 17:33:54 -10:00
Stephen Morgan
e63138ef7d lib,cli: Assorted fixes for older GHC. 2021-01-02 15:08:09 +11:00
Stephen Morgan
e3ec01c3c6 lib,cli,ui: Use Text for showDate and related. 2021-01-02 15:08:09 +11:00