Commit Graph

955 Commits

Author SHA1 Message Date
g. nicholas d'andrea
e678e09704 feat: accounts: add Gain (G) account type as subtype of Revenue [#2522]
Add a new account type Gain with single-letter code G as a subtype of
Revenue, similar to how Cash is a subtype of Asset and Conversion is a
subtype of Equity. This enables tracking capital gains/losses separately
while still including them in income statements and close --retain.

Usage: account revenues:capital  ; type: G

- type:G matches only Gain accounts
- type:R matches both Revenue and Gain (subtype matching)
- Auto-detection from account names matching:
  ^(income|revenue)s?:(capital[- ]?)?(gains?|loss(es)?)(:|$)
  e.g. income:gains, revenue:capital-gains, income:losses
2026-01-08 19:18:13 -10:00
Simon Michael
c0971c1c09 dev: cleanup 2025-12-16 00:18:28 -10:00
Simon Michael
48620c8e8c imp:journal:include: don't read file attributes unnecessarily
When including a literal path, don't use the Glob library at all.
Glob seems to read attributes of all files in a directory,
which disturbs build tools like tup which detect dependencies
based on filesystem operations.
2025-12-15 21:54:53 -10:00
Simon Michael
ebaabe4305 imp:journal: fix a slight pessimisation of include directives
Since 1.50.3, canonicalizePath was being called wastefully when
processing journals with many nested include files and/or many matches
for include glob paths. On a slow filesystem, with unusually
many includes, this might have been quite noticeable.

Now we canonicalise each file path just once as it is encountered,
avoiding the wasted IO work.
2025-12-10 19:21:59 -10:00
Simon Michael
665e2d0a55 fix:journal:include: relative includes from a symlinked file work again [#2503] 2025-12-08 09:36:13 -10:00
Simon Michael
12234e0b7e ;fix:journal: including an unreadable file now shows a clearer error
showing the problem include directive (previously the line number was
off by one). Likewise for other IO errors like when resolving ~ and
a home directory can't be found.
2025-12-05 02:34:08 -10:00
Simon Michael
8cd113389a fix:journal:include: drop 1.50's exclusion of glob-matched dot paths [#2498]
1.50* attempted to work around Glob's implicit searching of non-top-level dot dirs.
This was overzealous; it meant that journal's include completely
excluded paths involving a glob and a dot dir or dot file anywhere in the path.

Now, the pre-1.50 behaviour is restored:
`*` and `**` won't implicitly match dot files or top-level dot directories.
They will implicitly search non-top-level dot directories, as before (#Glob#49).
2025-12-04 05:54:54 -10:00
Simon Michael
81744d81a1 fix:journal:include: fix some regressions with glob matching in 1.50-1.50.3
Before 1.50, journal's include directive's handling of glob patterns (*, **, ?, etc.)
had these limitations:

- ** always searched intermediate dot directories
- ** matched only directories, not files

In 1.50-1.50.3, it had different limitations, some unintended:

- it ignored all dot files, dot dirs, and symbolic links to dot dirs,
  even when explicitly mentioned in the pattern (unless using --old-glob)
- it showed symbolic links dereferenced, eg in `hledger files` output

Now it has fewer limitations, mainly this:

- it ignores all dot files and dot dirs, even when explicitly mentioned (unless using --old-glob)

Ie it no longer ignores symbolic links to dot dirs, and it no longer shows symbolic links dereferenced.
Also: including the current file is now always harmless, whether using a glob pattern or not.

Internally, file paths in the "include file stack" (jincludefilestack) are now just absolute,
but not canonicalised; showing symbolic links un-dereferenced in output and error messages seems
generally more useful. This might affect output elsewhere also.
(Those paths are still canonicalised on the fly when checking for include cycles,
not so efficiently: each time an include directive is parsed, all the current parent files
and all the new glob-matched include files will be re-canonicalised.
Hopefully this is unnoticeable.)
2025-12-01 11:28:51 -08:00
Simon Michael
00f6a832d4 fix:journal: consistent error message when include has no argument 2025-12-01 11:00:42 -08:00
Simon Michael
356e2ba88a fix:journal: repair 1.50's journal reading slowness [#2493]
Since 1.50, sourceFilePath, which does IO operations, was being called for every item in the journal.
On my machine this was causing a ~40% slowdown,
but probably it could be more depending on storage system.

Now it's once again called only once per include directive.
Speed seems slightly better now than 1.43 for some reason
(eg: 13k txns/s -> 8k txns/s -> 14k txns/s).
2025-11-15 21:22:36 -10:00
Simon Michael
2f007c93d2 dev: switch all qualifed imports to ImportQualifiedPost style 2025-09-29 19:28:59 -10:00
Simon Michael
e51e6c4e3d ;dev: remove old prototype 2025-09-29 18:35:20 -10:00
Simon Michael
5db4ee7420 dev:Hledger.Utils.IO: readHandlePortably, readHandlePortably' -> hGetContentsPortably 2025-09-26 01:08:43 -10:00
Simon Michael
9351c70f74 fix:csv: respect encoding rule when rules file is input file [#2465] 2025-09-26 00:34:01 -10:00
Simon Michael
6d164b48a0 imp:journal: in amounts, also allow and ignore empty {} (or {{}}) 2025-09-11 07:33:22 +01:00
Simon Michael
7ac0fa1aaa dev:TimeclockReader, Timeclock: refactor/reindent [#2417] 2025-09-01 08:27:01 +01:00
Simon Michael
e37caf0f27 imp:journal: better error when attempting to include a rules file 2025-09-01 08:26:44 +01:00
Simon Michael
efe1d11edb fix:timeclock: --old-timeclock also affects included files [#2417]
This required changing the Reader type and passing InputOpts down to
journal's include directive parser.
2025-09-01 08:26:44 +01:00
Simon Michael
5a3e34cc55 imp:timeclock: syntax is more robust and featureful
The default timeclock parser (ie when not using --old-timeclock) has
the following changes, related to issues such as
[#2141], [#2365], [#2400], [#2417]:

- semicolon now always starts a comment; timeclock account names can't include semicolons
  (though journal account names still can)
- clock-in and clock-out entries now have different syntax
- clock-ins now require an account name
- clock-outs now can have a comment and tags
- the doc has been rewritten, and now mentions the --old-timeclock flag

- lib: accountnamep and modifiedaccountnamep now take a flag to allow semicolons or not
2025-08-31 10:58:37 +01:00
Simon Michael
97899f9a9b feat:csv: support data generating scripts
and rewrite the rules reader.
2025-08-28 21:38:38 +01:00
Simon Michael
b22f94007f dev: source, archive, rules reader: improve docs 2025-08-28 21:38:38 +01:00
Simon Michael
e796a00fc4 dev:import: drop archiving of original data
just archive clean data
2025-08-28 21:38:38 +01:00
Simon Michael
b64ddfe813 dev:rules reader: drop "fall back to reading latest archived" 2025-08-28 21:38:38 +01:00
Simon Michael
c60ec90756 dev:import: improve buggy detection of import command 2025-08-28 21:38:38 +01:00
Simon Michael
c515fedf70 feat:csv: support data cleaning scripts 2025-08-28 21:38:38 +01:00
Simon Michael
cb1d6a71a6 dev:import:archive: fix bugs in new code
Too hard to rebase
2025-08-14 19:22:52 +01:00
Simon Michael
7dfe2d84e7 dev:import: fix debug message 2025-08-14 17:52:54 +01:00
Simon Michael
88b451d6eb imp: when source rule finds no files, read the latest archived 2025-08-14 14:23:53 +01:00
Simon Michael
3dec0a8944 dev: indentation 2025-08-14 14:23:53 +01:00
Simon Michael
76dc6d089a feat:import:archive: archive data files, and process oldest first 2025-08-14 12:54:40 +01:00
Simon Michael
db7783b16d dev: cleanups 2025-08-14 12:54:40 +01:00
Simon Michael
bfbef4bcbb dev: refactor PrefixedFilePath 2025-08-14 12:37:11 +01:00
Simon Michael
e69c72a6c7 dev: include: revert wrong error position fix; refactor
Errors in the main file are being reported a few lines too high,
due to the setOffset in includedirectivep.

It seems reverting this should have restored the original bug with
wrong line number in certain include error messages, but I can't find
that right now.
2025-07-28 11:57:46 +01:00
Simon Michael
c48c41bcd2 dev:timeclock: cleanups 2025-07-18 07:17:45 -07:00
Simon Michael
b7e35f84a2 imp: include: add hidden --old-glob flag to restore old dot behaviour
This disables the workaround for Glob#49, allowing glob patterns to
find dot files and traverse dot directories again (sometimes too much).
2025-07-17 08:00:08 -07:00
Simon Michael
5ec770badd imp: include: more flexible **; show the correct line in read errors 2025-07-16 06:52:19 -07:00
Simon Michael
b35dcfda28 imp: include: give a clearer error for trying to include a csv file
And note that other read errors may be unclear when occurring in included files.
2025-07-16 06:52:19 -07:00
Simon Michael
8215f19baa dev: include: cleanup 2025-07-16 06:52:19 -07:00
Simon Michael
3741f9f030 fix: include: report read failures with correct line number 2025-07-16 06:52:19 -07:00
Simon Michael
2dcfe22c89 imp: include: report ** without / as an error, for clarity 2025-07-16 06:52:19 -07:00
Simon Michael
460ae28826 imp: include: globs exclude current file; more cleanup 2025-07-16 06:52:19 -07:00
Simon Michael
b4a1add267 imp: include: more robust tests and glob pattern handling
This switches from filepattern back to Glob, which is more powerful.
New notes, implementation, workarounds and tests.
2025-07-16 06:52:19 -07:00
Simon Michael
1046f652b1 dev: PrefixedFilePath cleanups
And some helpers that weren't needed after all, but maybe in future
2025-07-16 06:52:19 -07:00
Simon Michael
28f60bcf92 dev: includedirectivep: refactor 2025-07-11 20:34:50 -07:00
Simon Michael
3a03927018 imp: include: show including file path in debug output 2025-07-11 20:12:13 -07:00
Simon Michael
0add2e90db imp: include: glob patterns always exclude the current file
Eg include **/*.journal is less likely no complain
2025-07-11 19:36:17 -07:00
Simon Michael
08017366b5 imp: file reading: demote some debug=6 output to level 7 2025-07-11 13:48:58 -07:00
Simon Michael
536589e2c2 imp: include: improve cycle and read failure error messages 2025-07-11 13:36:47 -07:00
Simon Michael
b71e001c51 imp: include: more robust ** patterns, and ignore dotted directories
** now ignores anything under dotted directories, ie directories whose
name begins with a dot. Eg .git/, foo/.secret/, etc.

Switched from Glob to filepattern lib.
2025-07-11 13:36:47 -07:00
Simon Michael
b1f416dee7 dev: parseIncludedFile: doc cleanup 2025-07-11 13:01:54 -07:00