diff --git a/hledger-ui/hledger-ui.1 b/hledger-ui/hledger-ui.1 index 6f68ff1ef..9222632cc 100644 --- a/hledger-ui/hledger-ui.1 +++ b/hledger-ui/hledger-ui.1 @@ -7,7 +7,7 @@ hledger\-ui \- terminal interface (TUI) for \f[CR]hledger\f[R], a robust, friendly plain text accounting app. .SH SYNOPSIS -\f[CR]hledger\-ui [OPTS] [QUERYARGS]\f[R] +\f[CR]hledger\-ui [OPTS] [QUERYARGS]\f[R] .PD 0 .P .PD @@ -15,7 +15,7 @@ or .PD 0 .P .PD -\f[CR]hledger ui \-\- [OPTS] [QUERYARGS]\f[R] +\f[CR]hledger ui [OPTS] [QUERYARGS]\f[R] .SH DESCRIPTION This manual is for hledger\[aq]s terminal interface, version 1.43.99. See also the hledger manual for common concepts and file formats. diff --git a/hledger-ui/hledger-ui.info b/hledger-ui/hledger-ui.info index bf47a6cfc..051f4aaac 100644 --- a/hledger-ui/hledger-ui.info +++ b/hledger-ui/hledger-ui.info @@ -16,7 +16,7 @@ plain text accounting app. 'hledger-ui [OPTS] [QUERYARGS]' or -'hledger ui -- [OPTS] [QUERYARGS]' +'hledger ui [OPTS] [QUERYARGS]' This manual is for hledger's terminal interface, version 1.43.99. See also the hledger manual for common concepts and file formats. @@ -565,22 +565,22 @@ with the UI unresponsive.  Tag Table: Node: Top221 -Node: OPTIONS1872 -Node: MOUSE8758 -Node: KEYS9090 -Node: SCREENS14094 -Node: Menu screen14834 -Node: Cash accounts screen15150 -Node: Balance sheet accounts screen15511 -Node: Income statement accounts screen15847 -Node: All accounts screen16232 -Node: Register screen16595 -Node: Transaction screen19038 -Node: Error screen20613 -Node: WATCH MODE20979 -Node: --watch problems21877 -Node: ENVIRONMENT23230 -Node: BUGS23463 +Node: OPTIONS1869 +Node: MOUSE8755 +Node: KEYS9087 +Node: SCREENS14091 +Node: Menu screen14831 +Node: Cash accounts screen15147 +Node: Balance sheet accounts screen15508 +Node: Income statement accounts screen15844 +Node: All accounts screen16229 +Node: Register screen16592 +Node: Transaction screen19035 +Node: Error screen20610 +Node: WATCH MODE20976 +Node: --watch problems21874 +Node: ENVIRONMENT23227 +Node: BUGS23460  End Tag Table diff --git a/hledger-ui/hledger-ui.txt b/hledger-ui/hledger-ui.txt index 5d724b6c7..249d596d0 100644 --- a/hledger-ui/hledger-ui.txt +++ b/hledger-ui/hledger-ui.txt @@ -6,9 +6,9 @@ NAME plain text accounting app. SYNOPSIS - hledger-ui [OPTS] [QUERYARGS] + hledger-ui [OPTS] [QUERYARGS] or - hledger ui -- [OPTS] [QUERYARGS] + hledger ui [OPTS] [QUERYARGS] DESCRIPTION This manual is for hledger's terminal interface, version 1.43.99. See diff --git a/hledger-web/hledger-web.1 b/hledger-web/hledger-web.1 index 28061c333..39e4e58ed 100644 --- a/hledger-web/hledger-web.1 +++ b/hledger-web/hledger-web.1 @@ -7,7 +7,7 @@ hledger\-web \- web interface and API for \f[CR]hledger\f[R], a robust, friendly plain text accounting app. .SH SYNOPSIS -\f[CR]hledger\-web [OPTS] [QUERY]\f[R] +\f[CR]hledger\-web [OPTS] [QUERY]\f[R] .PD 0 .P .PD @@ -15,7 +15,7 @@ or .PD 0 .P .PD -\f[CR]hledger web \-\- [OPTS] [QUERY]\f[R] +\f[CR]hledger web [OPTS] [QUERY]\f[R] .SH DESCRIPTION This manual is for hledger\[aq]s web interface, version 1.43.99. See also the hledger manual for common concepts and file formats. @@ -84,7 +84,7 @@ Flags: \-\-base\-url=BASEURL set the base url (default: http://IPADDR:PORT) \-\-test run hledger\-web\[aq]s tests and exit. hspec test runner args may follow a \-\-, eg: hledger\-web \-\-test - \-\- \-\-help + \-\-help .EE .PP By default hledger\-web listens only on IP address \f[CR]127.0.0.1\f[R], diff --git a/hledger-web/hledger-web.info b/hledger-web/hledger-web.info index 90f3ebc70..1c121381a 100644 --- a/hledger-web/hledger-web.info +++ b/hledger-web/hledger-web.info @@ -16,7 +16,7 @@ plain text accounting app. 'hledger-web [OPTS] [QUERY]' or -'hledger web -- [OPTS] [QUERY]' +'hledger web [OPTS] [QUERY]' This manual is for hledger's web interface, version 1.43.99. See also the hledger manual for common concepts and file formats. @@ -96,7 +96,7 @@ Flags: --base-url=BASEURL set the base url (default: http://IPADDR:PORT) --test run hledger-web's tests and exit. hspec test runner args may follow a --, eg: hledger-web --test - -- --help + --help By default hledger-web listens only on IP address '127.0.0.1', which be accessed only from the local machine. @@ -528,15 +528,15 @@ We welcome bug reports in the hledger issue tracker  Tag Table: Node: Top223 -Node: OPTIONS2581 -Node: PERMISSIONS11482 -Node: EDITING UPLOADING DOWNLOADING12632 -Node: RELOADING13647 -Node: JSON API14214 -Node: DEBUG OUTPUT19863 -Node: Debug output20015 -Node: ENVIRONMENT20533 -Node: BUGS20769 +Node: OPTIONS2578 +Node: PERMISSIONS11476 +Node: EDITING UPLOADING DOWNLOADING12626 +Node: RELOADING13641 +Node: JSON API14208 +Node: DEBUG OUTPUT19857 +Node: Debug output20009 +Node: ENVIRONMENT20527 +Node: BUGS20763  End Tag Table diff --git a/hledger-web/hledger-web.txt b/hledger-web/hledger-web.txt index 48916e440..f5889a6bb 100644 --- a/hledger-web/hledger-web.txt +++ b/hledger-web/hledger-web.txt @@ -6,9 +6,9 @@ NAME plain text accounting app. SYNOPSIS - hledger-web [OPTS] [QUERY] + hledger-web [OPTS] [QUERY] or - hledger web -- [OPTS] [QUERY] + hledger web [OPTS] [QUERY] DESCRIPTION This manual is for hledger's web interface, version 1.43.99. See also @@ -73,7 +73,7 @@ OPTIONS --base-url=BASEURL set the base url (default: http://IPADDR:PORT) --test run hledger-web's tests and exit. hspec test runner args may follow a --, eg: hledger-web --test - -- --help + --help By default hledger-web listens only on IP address 127.0.0.1, which be accessed only from the local machine. diff --git a/hledger/hledger.1 b/hledger/hledger.1 index a0646085f..afe973509 100644 --- a/hledger/hledger.1 +++ b/hledger/hledger.1 @@ -20,11 +20,6 @@ or .PD 0 .P .PD -or -.PD 0 -.P -.PD -\f[CR]hledger ADDONCMD [OPTS] \-\- [ADDONOPTS] [ADDONARGS]\f[R] .SH DESCRIPTION hledger is a robust, user\-friendly, cross\-platform set of programs for tracking money, time, or any other commodity, using double\-entry @@ -283,7 +278,7 @@ Are all commodities declared with a \f[CR]commodity\f[R] directive ? .IP \[bu] 2 Are all commodity conversions declared explicitly ? .PP -You can use the check command to run individual checks \-\- the ones +You can use the check command to run individual checks \- the ones listed above and some more. .SH Commands hledger provides various subcommands for getting things done. @@ -318,31 +313,27 @@ terminal, run \f[CR]hledger CMD \-h\f[R]. Eg: \f[CR]hledger bal \-h\f[R]. .SS Add\-on commands In addition to the built\-in commands, you can install \f[I]add\-on -commands\f[R]: programs or scripts named \[dq]hledger\-SOMETHING\[dq], -which will also appear in hledger\[aq]s commands list. -If you used the hledger\-install script, you will have several add\-ons -installed already. -Some more can be found in hledger\[aq]s bin/ directory, documented at +commands\f[R], which will also appear in hledger\[aq]s commands list. +Some of these can be installed as separate packages; others can be found +in hledger\[aq]s bin/ directory, documented at https://hledger.org/scripts.html. .PP -More precisely, add\-on commands are programs or scripts in your -shell\[aq]s PATH, whose name starts with \[dq]hledger\-\[dq] and ends -with no extension or a recognised extension (\[dq].bat\[dq], -\[dq].com\[dq], \[dq].exe\[dq], \[dq].hs\[dq], \[dq].js\[dq], -\[dq].lhs\[dq], \[dq].lua\[dq], \[dq].php\[dq], \[dq].pl\[dq], -\[dq].py\[dq], \[dq].rb\[dq], \[dq].rkt\[dq], or \[dq].sh\[dq]), and (on -unix and mac) which has executable permission for the current user. +Add\-on commands are programs or scripts in your shell\[aq]s PATH, whose +name starts with \[dq]hledger\-\[dq] and ends with no extension or a +recognised extension (\[dq].bat\[dq], \[dq].com\[dq], \[dq].exe\[dq], +\[dq].hs\[dq], \[dq].js\[dq], \[dq].lhs\[dq], \[dq].lua\[dq], +\[dq].php\[dq], \[dq].pl\[dq], \[dq].py\[dq], \[dq].rb\[dq], +\[dq].rkt\[dq], or \[dq].sh\[dq]), and (on unix and mac) which has +executable permission for the current user. .PP -You can run add\-on commands using hledger, much like built\-in -commands: -\f[CR]hledger ADDONCMD [\-\- ADDONCMDOPTS] [ADDONCMDARGS]\f[R]. -But note the double hyphen argument, required before add\-on\-specific -options. -Eg: \f[CR]hledger ui \-\- \-\-watch\f[R] or -\f[CR]hledger web \-\- \-\-serve\f[R]. -If this causes difficulty, you can always run the add\-on directly, -without using \f[CR]hledger\f[R]: \f[CR]hledger\-ui \-\-watch\f[R] or -\f[CR]hledger\-web \-\-serve\f[R]. +You can run add\-on commands directly: \f[CR]hledger\-ui \-\-watch\f[R]. +.PP +Or you can run them with hledger, like built\-in commands: +\f[CR]hledger ui \-\-watch\f[R]. +In this case hledger\[aq]s config file will be used, so you can set +custom options for the addon there. +(Before hledger 1.50, an \f[CR]\-\-\f[R] argument was needed before +addon options, but not any more.) .SH Options Run \f[CR]hledger \-h\f[R] to see general command line help. Options can be written either before or after the command name. @@ -448,6 +439,11 @@ Usually hledger accepts any unambiguous flag prefix, eg you can write \f[CR]\-\-tl\f[R] instead of \f[CR]\-\-tldr\f[R] or \f[CR]\-\-dry\f[R] instead of \f[CR]\-\-dry\-run\f[R]. .PP +You can combine short flags which don\[aq]t take arguments, eg you can +write \f[CR]\-MAST\f[R] instead of \f[CR]\-M \-A \-S \-T\f[R]. +Flags requiring an argument can\[aq]t be combined in this way +(\f[CR]\-If FILE\f[R] won\[aq]t work). +.PP If the same option appears more than once in a command line, usually the last (right\-most) wins. Similarly, if mutually exclusive flags are used together, the @@ -911,8 +907,7 @@ the effect on all your reports. To troubleshoot the effect of config files, run with \f[CR]\-\-debug\f[R] or \f[CR]\-\-debug 8\f[R]. .PP -The config file feature was added in hledger 1.40 and is considered -\f[I]experimental\f[R]. +The config file feature was added in hledger 1.40. .SS Shell completions If you use the bash or zsh shells, you can optionally set up context\-sensitive autocompletion for hledger command lines. @@ -1184,7 +1179,7 @@ If you are using the \f[CR]less\f[R] pager, hledger automatically appends a number of options to the \f[CR]LESS\f[R] variable to enable ANSI colour and a number of other conveniences. (At the time of writing: \-\-chop\-long\-lines \-\-hilite\-unread -\-\-ignore\-case \-\-mouse \-\-no\-init \-\-quit\-at\-eof +\-\-ignore\-case \-\-no\-init \-\-quit\-at\-eof \-\-quit\-if\-one\-screen \-\-RAW\-CONTROL\-CHARS \-\-shift=8 \-\-squeeze\-blank\-lines \-\-use\-backslash ). If these don\[aq]t work well, you can set your preferred options in the @@ -3312,29 +3307,57 @@ You can pull in the content of additional files by writing an include directive, like this: .IP .EX -include FILEPATH +include SOMEFILE .EE .PP -Only journal files can include, and only journal, timeclock or timedot -files can be included (not CSV files, currently). +This has the same effect as if SOMEFILE\[aq]s content was inlined at +this point. +(With any include directives in SOMEFILE processed similarly, +recursively.) .PP -If the file path does not begin with a slash, it is relative to the -current file\[aq]s folder. +Only journal files can include other files. +They can include journal, timeclock or timedot files, but not CSV files. .PP -A tilde means home directory, eg: \f[CR]include \[ti]/main.journal\f[R]. +If the file path begins with a tilde, that means your home directory: +\f[CR]include \[ti]/main.journal\f[R]. .PP -The path may contain glob patterns to match multiple files, eg: -\f[CR]include *.journal\f[R]. +If it begins with a slash, it is an absolute path: +\f[CR]include /home/user/main.journal\f[R]. +Otherwise it is relative to the including file\[aq]s folder: +\f[CR]include ../finances/main.journal\f[R]. .PP -There is limited support for recursive wildcards: \f[CR]**/\f[R] (the -slash is required) matches 0 or more subdirectories. -It\[aq]s not super convenient since you have to avoid include cycles and -including directories, but this can be done, eg: -\f[CR]include */**/*.journal\f[R]. +Also, the path may have a file type prefix to force a specific file +format, overriding the file extension(s) (as described in Data formats): +\f[CR]include timedot:notes/2023.md\f[R]. .PP -The path may also be prefixed to force a specific file format, -overriding the file extension (as described in Data formats): -\f[CR]include timedot:\[ti]/notes/2023*.md\f[R]. +The path may contain glob patterns to match multiple files. +hledger\[aq]s globs are similar to zsh\[aq]s: \f[CR]?\f[R] to match any +character; \f[CR][a\-z]\f[R] to match any character in a range; +\f[CR]*\f[R] to match zero or more characters that aren\[aq]t a path +separator (like \f[CR]/\f[R]); \f[CR]**\f[R] to match zero or more +subdirectories and/or zero or more characters at the start of a file +name; etc. +Also, hledger\[aq]s globs always exclude the including file itself. +So, you can do +.IP \[bu] 2 +\f[CR]include *.journal\f[R] to include all other journal files in the +current directory (excluding dot files) +.IP \[bu] 2 +\f[CR]include **.journal\f[R] to include all other journal files in this +directory and below (excluding dot directories/files) +.IP \[bu] 2 +\f[CR]include timelogs/2???.timedot\f[R] to include all timedot files +named like a year number. +.PP +There is a limitation: hledger\[aq]s globs always exclude paths +involving dot files or dot directories. +This is a workaround for unavoidable dot directory traversal; you can +disable it and revert to older behaviour with the +\f[CR]\-\-old\-glob\f[R] flag, for now. +.PP +If you are using many, or deeply nested, include files, and have an +error that\[aq]s hard to pinpoint: a good troubleshooting command is +\f[CR]hledger files \-\-debug=6\f[R] (or 7). .SS \f[CR]P\f[R] directive The \f[CR]P\f[R] directive declares a market price, which is a conversion rate between two commodities on a certain date. @@ -3542,6 +3565,9 @@ Note these generated postings are temporary, existing only for the duration of the report, and only when \f[CR]\-\-auto\f[R] is used; they are not saved in the journal file by hledger. .PP +The postings can contain the special string \f[CR]%account\f[R] which +will be expanded to the account name of the matched account. +.PP Generated postings\[aq] amounts can depend on the matched posting\[aq]s amount. So auto postings can be useful for, eg, adding tax postings with a @@ -4222,34 +4248,68 @@ will look for rules in \f[CR]foo.csv.rules\f[R]. Or, you can tell it to read the rules file, with \f[CR]\-f foo.csv.rules\f[R], and it will look for data in \f[CR]foo.csv\f[R] (since 1.30). -.PP These are mostly equivalent, but the second method provides some extra features. For one, the data file can be missing, without causing an error; it is just considered empty. -And, you can specify a different data file by adding a \[dq]source\[dq] -rule: +.PP +For more flexibility, add a \f[CR]source\f[R] rule, which lets you +specify a different data file: .IP .EX source ./Checking1.csv .EE .PP +If the file does not exist, it is just considered empty, without raising +an error. +.PP If you specify just a file name with no path, hledger will look for it -in your system\[aq]s downloads directory (\f[CR]\[ti]/Downloads\f[R], -currently): +in the \f[CR]\[ti]/Downloads\f[R] folder: .IP .EX source Checking1.csv .EE .PP -And if you specify a glob pattern, hledger will read the most recent of -the matched files (useful with repeated downloads): +You can use a glob pattern, to avoid specifying the file name exactly: .IP .EX source Checking1*.csv .EE .PP +This has another benefit: if the pattern matches multiple files, hledger +will read the newest (most recently modified) one. +This avoids problems if you have downloaded a file multiple times +without cleaning up. +.PP +All this enables a convenient workflow where can you just download CSV +files, then run \f[CR]hledger import rules/*\f[R]. +.PP See also \[dq]Working with CSV > Reading files specified by rule\[dq]. +.PP +The \f[CR]archive\f[R] rule adds a few more features to +\f[CR]source\f[R]; see below. +.SS \f[CR]archive\f[R] +Adding the \f[CR]archive\f[R] rule to your rules file affects importing +or reading files specified by \f[CR]source\f[R]: +.IP \[bu] 2 +After successfully importing, \f[CR]import\f[R] will move the data file +to an archive directory (\f[CR]data/\f[R] next to the rules file, +auto\-created), renamed to +\f[CR]RULESFILEBASENAME.DATAFILEMODDATE.DATAFILEEXT\f[R]. +Archiving data files is optional, but it can be useful for +troubleshooting, detecting variations in your banks\[aq] CSV data, +regenerating entries with improved rules, etc. +.IP \[bu] 2 +\f[CR]import\f[R] will pick the oldest of \f[CR]source\f[R] glob +matches, rather than the newest. +So if you have multiple versions of a download, repeated imports will +process them in chronological order. +.IP \[bu] 2 +For commands other than \f[CR]import\f[R], when the \f[CR]source\f[R] +path or glob pattern matches no files, hledger will try to read the +latest archived data file instead. +This is convenient for working with the downloaded data again, even +after it has been imported. .SS \f[CR]encoding\f[R] .IP .EX @@ -4366,6 +4426,9 @@ date\-format %Y\-%h\-%d # Note the time and junk must be fully parsed, though only the date is used. date\-format %\-m/%\-d/%Y %l:%M %p some other junk .EE +.PP +Note currently there is no locale awareness for things like +\f[CR]%b\f[R], and setting LC_TIME won\[aq]t help. .SS \f[CR]timezone\f[R] .IP .EX @@ -6110,12 +6173,13 @@ can optionally use \[dq]smart date\[dq] syntax. Smart dates can be written with english words, can be relative, and can have parts omitted. Missing parts are inferred as 1, when needed. -Smart dates can be interpreted as dates or periods depending on context. +Smart dates can be interpreted as dates or periods depending on the +context. .PP Examples: .PP \f[CR]2004\-01\-01\f[R], \f[CR]2004/10/1\f[R], \f[CR]2004.9.1\f[R], -\f[CR]20240504\f[R] : +\f[CR]20240504\f[R], \f[CR]2024Q1\f[R] : .PD 0 .P .PD @@ -6123,10 +6187,17 @@ Exact dates. The year must have at least four digits, the month must be 1\-12, the day must be 1\-31, the separator can be \f[CR]\-\f[R] or \f[CR]/\f[R] or \f[CR].\f[R] or nothing. +The q can be upper or lower case and the quarter number must be 1\-4. .TP \f[CR]2004\-10\f[R] start of month .TP +\f[CR]2004q3\f[R] +start of third quarter of 2004 +.TP +\f[CR]q3\f[R] +start of third quarter of current year +.TP \f[CR]2004\f[R] start of year .TP @@ -6226,7 +6297,7 @@ For example, if the journal\[aq]s last transaction is on february 20th, of february. .IP \[bu] 2 \f[CR]hledger register \-\-monthly \-\-end 2/14\f[R] also will end the -report at the end of february. +report at the end of february (overriding the requested end date). .IP \[bu] 2 \f[CR]hledger register \-\-monthly \-\-begin 1/5 \-\-end 2/14\f[R] will end the report on march 4th [1]. @@ -6565,7 +6636,7 @@ accounts matching \f[CR]liabilities\f[R] to depth 3, all other accounts to depth 1. .PP If an account is matched by more than one regular expression depth -argument then the more specific one will used. +argument then the more specific one will be used. For example, if \f[CR]\-\-depth assets=1 \-\-depth assets:bank:savings=2\f[R] is provided, then \f[CR]assets:bank:savings\f[R] will be collapsed to depth @@ -6975,7 +7046,7 @@ amount to a cash account\[dq]. .PD Like \f[CR]expr:\f[R], but when used with transaction\-oriented commands like \f[CR]print\f[R], it matches the transaction only if all postings -are matched by all of QUERYEXPR. +are matched by all of QUERYEXPR (and there is at least one posting). .PD 0 .P .PD @@ -8041,8 +8112,8 @@ $ hledger \-f\- print \-\-value=end date:2000/01\-2000/03 (a) 2 B .EE .PP -With no report period specified, that shows the value as of the last day -of the journal (2000\-03\-01): +With no report period specified, the latest transaction date or price +date is used as valuation date (2000\-04\-01): .IP .EX $ hledger \-f\- print \-\-value=end @@ -8056,8 +8127,7 @@ $ hledger \-f\- print \-\-value=end (a) 3 B .EE .PP -Show the current value (the 2000\-04\-01 price is still in effect -today): +The value today is the same (the 2000\-04\-01 price is still in effect): .IP .EX $ hledger \-f\- print \-\-value=now @@ -8580,10 +8650,6 @@ eg \f[CR]\-s4\f[R] to play at 4x original speed or \f[CR]\-s.5\f[R] to play at half speed. The default speed is 2x. .PP -Other asciinema options can be added following a double dash, eg -\f[CR]\-\- \-i.1\f[R] to limit pauses or \f[CR]\-\- \-h\f[R] to list -asciinema\[aq]s other options. -.PP During playback, several keys are available: SPACE to pause/unpause, . to step forward (while paused), CTRL\-c quit. .PP @@ -8852,7 +8918,7 @@ Runs hledger\-ui (if installed). Runs hledger\-web (if installed). .SH Data entry commands .SS add -Record new transactions with interactive prompting in the console. +Add new transactions to a journal file, with interactive prompting. .IP .EX Flags: @@ -8906,6 +8972,9 @@ default commodity with a \f[CR]D\f[R] directive, you might expect It does not do this; we assume that if you are using a \f[CR]D\f[R] directive you prefer not to see the commodity symbol repeated on amounts in the journal. +.IP \[bu] 2 +\f[CR]add\f[R] creates entries in journal format; it won\[aq]t work with +timeclock or timedot files. .PP Examples: .IP \[bu] 2 @@ -8981,7 +9050,7 @@ $ hledger import bank1\-checking.csv bank1\-savings.csv .EX $ hledger import *.csv .EE -.SS Import preview +.SS Import dry run It\[aq]s useful to preview the import by running first with \f[CR]\-\-dry\-run\f[R], to sanity check the range of dates being imported, and to check the effect of your conversion rules if converting @@ -9004,7 +9073,7 @@ You could also run this repeatedly to see the effect of edits to your conversion rules: .IP .EX -$ watchexec \-\- \[aq]hledger import \-\-dry\-run bank.csv | hledger \-f\- \-I print unknown\[aq] +$ watchexec \-\- \[dq]hledger import \-\-dry\-run bank.csv | hledger \-f\- \-I print unknown\[dq] .EE .PP Once the conversion and dates look good enough to import to your @@ -9153,38 +9222,58 @@ journal\[aq]s canonical commodity styles, as declared by amounts. .PP Related: CSV > Amount decimal places. +.SS Import archiving +When importing from a CSV rules file +(\f[CR]hledger import bank.rules\f[R]), you can use the archive rule to +enable automatic archiving of the data file. +After a successful import, the data file (specified by +\f[CR]source\f[R]) will be moved to an archive folder (\f[CR]data/\f[R], +next to the rules file, auto\-created), and renamed similar to the rules +file, with a date. +This can be useful for troubleshooting, detecting variations in your +banks\[aq] CSV data, regenerating entries with improved rules, etc. +.PP +The \f[CR]archive\f[R] rule also causes \f[CR]import\f[R] to handle +\f[CR]source\f[R] glob patterns differently: when there are multiple +matched files, it will pick the oldest, not the newest. .SS Import special cases -If you have a download whose file name varies, you could rename it to a -fixed name after each download. -Or you could use a CSV \f[CR]source\f[R] rule with a suitable glob -pattern, and import from the .rules file instead of the data file. -.PP -Here\[aq]s a situation where you would need to run \f[CR]import\f[R] -with care: say you download \f[CR]bank.csv\f[R], but forget to import it -or delete it. -And next month you download it again. -This time your web browser may save it as \f[CR]bank (2).csv\f[R]. -So now each of these may have data not included in the other. -And a \f[CR]source\f[R] rule with a glob pattern would match only the -most recent file. -So in this case you should import from each one in turn, in the correct -order, taking care to use the same filename each time: -.IP -.EX -$ hledger import bank.csv -$ mv \[aq]bank (2).csv\[aq] bank.csv -$ hledger import bank.csv -.EE -.PP +.SS Deduplication Here are two kinds of \[dq]deduplication\[dq] which \f[CR]import\f[R] -does not handle (and generally should not, since these can happen -legitimately in financial data): +does not handle (and should not, because these can happen legitimately +in financial data): .IP \[bu] 2 Two or more of the new CSV records are identical, and generate identical new journal entries. .IP \[bu] 2 A new CSV record generates a journal entry identical to one(s) already in the journal. +.SS Varying file name +If you have a download whose file name varies, you could rename it to a +fixed name after each download. +Or you could use a CSV \f[CR]source\f[R] rule with a suitable glob +pattern, and import from the .rules file. +.SS Multiple versions +Say you download \f[CR]bank.csv\f[R], import it, but forget to delete it +from your downloads folder. +The next time you download it, your web browser will save it as (eg) +\f[CR]bank (2).csv\f[R]. +The source rule\[aq]s glob patterns are for just this situation: instead +of specifying \f[CR]source bank.csv\f[R], specify +\f[CR]source bank*.csv\f[R]. +Then \f[CR]hledger \-f bank.rules CMD\f[R] or +\f[CR]hledger import bank.rules\f[R] will automatically pick the newest +matched file (\f[CR]bank (2).csv\f[R]). +.PP +Alternately, what if you download, but forget to import or delete, then +download again ? +Now each of \f[CR]bank.csv\f[R] and \f[CR]bank (2).csv\f[R] might +contain data that\[aq]s not in the other, and not in your journal. +In this case, it\[aq]s best to import each of them in turn, oldest first +(otherwise, overlap detection could cause new records to be skipped). +Enabling import archiving ensures this. +Then \f[CR]hledger import bank.rules; hledger import bank.rules\f[R] +will import and archive first \f[CR]bank.csv\f[R], then +\f[CR]bank (2).csv\f[R]. .SH Basic report commands .SS accounts List the account names used or declared in the journal. @@ -9813,7 +9902,7 @@ full account name, or a distinctive substring that matches uniquely. .PP Transactions involving subaccounts of this account will also be shown. \f[CR]aregister\f[R] ignores depth limits, so its final total will -always match a balance report with similar arguments. +always match a historical balance report with similar arguments. .PP Any additional arguments form a query which will filter the transactions shown. @@ -12212,10 +12301,10 @@ spaces between account and amount. More: .IP .EX -$ hledger rewrite \-\- [QUERY] \-\-add\-posting \[dq]ACCT AMTEXPR\[dq] ... -$ hledger rewrite \-\- \[ha]income \-\-add\-posting \[aq](liabilities:tax) *.33\[aq] -$ hledger rewrite \-\- expenses:gifts \-\-add\-posting \[aq](budget:gifts) *\-1\[dq]\[aq] -$ hledger rewrite \-\- \[ha]income \-\-add\-posting \[aq](budget:foreign currency) *0.25 JPY; diversify\[aq] +$ hledger rewrite [QUERY] \-\-add\-posting \[dq]ACCT AMTEXPR\[dq] ... +$ hledger rewrite \[ha]income \-\-add\-posting \[aq](liabilities:tax) *.33\[aq] +$ hledger rewrite expenses:gifts \-\-add\-posting \[aq](budget:gifts) *\-1\[dq]\[aq] +$ hledger rewrite \[ha]income \-\-add\-posting \[aq](budget:foreign currency) *0.25 JPY; diversify\[aq] .EE .PP Argument for \f[CR]\-\-add\-posting\f[R] option is a usual posting of @@ -12253,14 +12342,14 @@ It indicates the query by which you want to match the posting to add new ones. .IP .EX -$ hledger rewrite \-\- \-f input.journal \-f rewrite\-rules.journal > rewritten\-tidy\-output.journal +$ hledger rewrite \-f input.journal \-f rewrite\-rules.journal > rewritten\-tidy\-output.journal .EE .PP This is something similar to the commands pipeline: .IP .EX -$ hledger rewrite \-\- \-f input.journal \[aq]\[ha]income\[aq] \-\-add\-posting \[aq](liabilities:tax) *.33\[aq] \[rs] - | hledger rewrite \-\- \-f \- expenses:gifts \-\-add\-posting \[aq]budget:gifts *\-1\[aq] \[rs] +$ hledger rewrite \-f input.journal \[aq]\[ha]income\[aq] \-\-add\-posting \[aq](liabilities:tax) *.33\[aq] \[rs] + | hledger rewrite \-f \- expenses:gifts \-\-add\-posting \[aq]budget:gifts *\-1\[aq] \[rs] \-\-add\-posting \[aq]assets:budget *1\[aq] \[rs] > rewritten\-tidy\-output.journal .EE @@ -12273,7 +12362,7 @@ To use this tool for batch modification of your journal files you may find useful output in form of unified diff. .IP .EX -$ hledger rewrite \-\- \-\-diff \-f examples/sample.journal \[aq]\[ha]income\[aq] \-\-add\-posting \[aq](liabilities:tax) *.33\[aq] +$ hledger rewrite \-\-diff \-f examples/sample.journal \[aq]\[ha]income\[aq] \-\-add\-posting \[aq](liabilities:tax) *.33\[aq] .EE .PP Output might look like: @@ -12604,10 +12693,10 @@ help: command\-specific options must go after the command (it\[aq]s fine to put common options there too: \f[CR]hledger CMD OPTS ARGS\f[R]) .IP \[bu] 2 -running add\-on executables directly simplifies command line parsing -(\f[CR]hledger\-ui OPTS ARGS\f[R]) +you can run addon commands via hledger (\f[CR]hledger ui [ARGS]\f[R]) or +directly (\f[CR]hledger\-ui [ARGS]\f[R]) .IP \[bu] 2 -enclose \[dq]problematic\[dq] args in single quotes +enclose \[dq]problematic\[dq] arguments in single quotes .IP \[bu] 2 if needed, also add a backslash to hide regular expression metacharacters from the shell @@ -13038,19 +13127,13 @@ We welcome bug reports in the hledger issue tracker .PP Some known issues and limitations: .PP -The need to precede add\-on command options with \f[CR]\-\-\f[R] when -invoked from hledger is awkward. -(See Command options, Constructing command lines.) -.PP A system locale with a suitable text encoding must be configured to work with non\-ascii data. (See Text encoding, Troubleshooting.) .PP -On Microsoft Windows, depending whether you are running in a CMD window -or a Cygwin/MSYS/Mintty window and how you installed hledger, non\-ascii -characters and colours may not be supported, and the tab key may not be -supported by \f[CR]hledger add\f[R]. -(Running in a WSL window should resolve these.) +On Microsoft Windows, depending what kind of terminal window you use, +non\-ascii characters, ANSI text formatting, and/or the add +command\[aq]s TAB key for completion, may not be supported. .PP When processing large data files, hledger uses more memory than Ledger. .SS Troubleshooting diff --git a/hledger/hledger.info b/hledger/hledger.info index 847b77e4b..8f2b95518 100644 --- a/hledger/hledger.info +++ b/hledger/hledger.info @@ -17,8 +17,6 @@ version). 'hledger' or 'hledger COMMAND [OPTS] [ARGS]' -or -'hledger ADDONCMD [OPTS] -- [ADDONOPTS] [ADDONARGS]' hledger is a robust, user-friendly, cross-platform set of programs for tracking money, time, or any other commodity, using double-entry @@ -334,26 +332,23 @@ File: hledger.info, Node: Add-on commands, Up: Commands 3.1 Add-on commands =================== -In addition to the built-in commands, you can install _add-on commands_: -programs or scripts named "hledger-SOMETHING", which will also appear in -hledger's commands list. If you used the hledger-install script, you -will have several add-ons installed already. Some more can be found in -hledger's bin/ directory, documented at -https://hledger.org/scripts.html. +In addition to the built-in commands, you can install _add-on commands_, +which will also appear in hledger's commands list. Some of these can be +installed as separate packages; others can be found in hledger's bin/ +directory, documented at https://hledger.org/scripts.html. - More precisely, add-on commands are programs or scripts in your -shell's PATH, whose name starts with "hledger-" and ends with no -extension or a recognised extension (".bat", ".com", ".exe", ".hs", -".js", ".lhs", ".lua", ".php", ".pl", ".py", ".rb", ".rkt", or ".sh"), -and (on unix and mac) which has executable permission for the current -user. + Add-on commands are programs or scripts in your shell's PATH, whose +name starts with "hledger-" and ends with no extension or a recognised +extension (".bat", ".com", ".exe", ".hs", ".js", ".lhs", ".lua", ".php", +".pl", ".py", ".rb", ".rkt", or ".sh"), and (on unix and mac) which has +executable permission for the current user. - You can run add-on commands using hledger, much like built-in -commands: 'hledger ADDONCMD [-- ADDONCMDOPTS] [ADDONCMDARGS]'. But note -the double hyphen argument, required before add-on-specific options. -Eg: 'hledger ui -- --watch' or 'hledger web -- --serve'. If this causes -difficulty, you can always run the add-on directly, without using -'hledger': 'hledger-ui --watch' or 'hledger-web --serve'. + You can run add-on commands directly: 'hledger-ui --watch'. + + Or you can run them with hledger, like built-in commands: 'hledger ui +--watch'. In this case hledger's config file will be used, so you can +set custom options for the addon there. (Before hledger 1.50, an '--' +argument was needed before addon options, but not any more.)  File: hledger.info, Node: Options, Next: Output, Prev: Commands, Up: Top @@ -461,6 +456,10 @@ General help flags: Usually hledger accepts any unambiguous flag prefix, eg you can write '--tl' instead of '--tldr' or '--dry' instead of '--dry-run'. + You can combine short flags which don't take arguments, eg you can +write '-MAST' instead of '-M -A -S -T'. Flags requiring an argument +can't be combined in this way ('-If FILE' won't work). + If the same option appears more than once in a command line, usually the last (right-most) wins. Similarly, if mutually exclusive flags are used together, the right-most wins. (When flags are mutually exclusive, @@ -906,8 +905,7 @@ scripts/applications, in ways that will surprise you later. 4. To troubleshoot the effect of config files, run with '--debug' or '--debug 8'. - The config file feature was added in hledger 1.40 and is considered -_experimental_. + The config file feature was added in hledger 1.40.  File: hledger.info, Node: Shell completions, Prev: Config files, Up: Options @@ -1089,10 +1087,10 @@ above. If you are using the 'less' pager, hledger automatically appends a number of options to the 'LESS' variable to enable ANSI colour and a number of other conveniences. (At the time of writing: -chop-long-lines --hilite-unread -ignore-case -mouse -no-init -quit-at-eof --quit-if-one-screen -RAW-CONTROL-CHARS -shift=8 -squeeze-blank-lines --use-backslash ). If these don't work well, you can set your preferred -options in the 'HLEDGER_LESS' variable, which will be used instead. +-hilite-unread -ignore-case -no-init -quit-at-eof -quit-if-one-screen +-RAW-CONTROL-CHARS -shift=8 -squeeze-blank-lines -use-backslash ). If +these don't work well, you can set your preferred options in the +'HLEDGER_LESS' variable, which will be used instead.  File: hledger.info, Node: HTML output, Next: CSV / TSV output, Prev: Text output, Up: Output format @@ -3227,27 +3225,49 @@ File: hledger.info, Node: include directive, Next: P directive, Prev: decimal You can pull in the content of additional files by writing an include directive, like this: -include FILEPATH +include SOMEFILE - Only journal files can include, and only journal, timeclock or -timedot files can be included (not CSV files, currently). + This has the same effect as if SOMEFILE's content was inlined at this +point. (With any include directives in SOMEFILE processed similarly, +recursively.) - If the file path does not begin with a slash, it is relative to the -current file's folder. + Only journal files can include other files. They can include +journal, timeclock or timedot files, but not CSV files. - A tilde means home directory, eg: 'include ~/main.journal'. + If the file path begins with a tilde, that means your home directory: +'include ~/main.journal'. - The path may contain glob patterns to match multiple files, eg: -'include *.journal'. + If it begins with a slash, it is an absolute path: 'include +/home/user/main.journal'. Otherwise it is relative to the including +file's folder: 'include ../finances/main.journal'. - There is limited support for recursive wildcards: '**/' (the slash is -required) matches 0 or more subdirectories. It's not super convenient -since you have to avoid include cycles and including directories, but -this can be done, eg: 'include */**/*.journal'. + Also, the path may have a file type prefix to force a specific file +format, overriding the file extension(s) (as described in Data formats): +'include timedot:notes/2023.md'. - The path may also be prefixed to force a specific file format, -overriding the file extension (as described in Data formats): 'include -timedot:~/notes/2023*.md'. + The path may contain glob patterns to match multiple files. +hledger's globs are similar to zsh's: '?' to match any character; +'[a-z]' to match any character in a range; '*' to match zero or more +characters that aren't a path separator (like '/'); '**' to match zero +or more subdirectories and/or zero or more characters at the start of a +file name; etc. Also, hledger's globs always exclude the including file +itself. So, you can do + + * 'include *.journal' to include all other journal files in the + current directory (excluding dot files) + * 'include **.journal' to include all other journal files in this + directory and below (excluding dot directories/files) + * 'include timelogs/2???.timedot' to include all timedot files named + like a year number. + + There is a limitation: hledger's globs always exclude paths involving +dot files or dot directories. This is a workaround for unavoidable dot +directory traversal; you can disable it and revert to older behaviour +with the '--old-glob' flag, for now. + + If you are using many, or deeply nested, include files, and have an +error that's hard to pinpoint: a good troubleshooting command is +'hledger files --debug=6' (or 7).  File: hledger.info, Node: P directive, Next: payee directive, Prev: include directive, Up: Journal @@ -3468,6 +3488,9 @@ posting. Note these generated postings are temporary, existing only for the duration of the report, and only when '--auto' is used; they are not saved in the journal file by hledger. + The postings can contain the special string '%account' which will be +expanded to the account name of the matched account. + Generated postings' amounts can depend on the matched posting's amount. So auto postings can be useful for, eg, adding tax postings with a standard percentage. AMOUNT can be: @@ -4089,6 +4112,7 @@ https://github.com/simonmichael/hledger/tree/master/examples/csv. * CSV rules cheatsheet:: * source:: +* archive:: * encoding:: * separator:: * skip:: @@ -4151,7 +4175,7 @@ The following kinds of rule can appear in the rules file, in any order. evaluated.  -File: hledger.info, Node: source, Next: encoding, Prev: CSV rules cheatsheet, Up: CSV +File: hledger.info, Node: source, Next: archive, Prev: CSV rules cheatsheet, Up: CSV 9.2 'source' ============ @@ -4159,31 +4183,70 @@ File: hledger.info, Node: source, Next: encoding, Prev: CSV rules cheatsheet, If you tell hledger to read a csv file with '-f foo.csv', it will look for rules in 'foo.csv.rules'. Or, you can tell it to read the rules file, with '-f foo.csv.rules', and it will look for data in 'foo.csv' -(since 1.30). +(since 1.30). These are mostly equivalent, but the second method +provides some extra features. For one, the data file can be missing, +without causing an error; it is just considered empty. - These are mostly equivalent, but the second method provides some -extra features. For one, the data file can be missing, without causing -an error; it is just considered empty. And, you can specify a different -data file by adding a "source" rule: + For more flexibility, add a 'source' rule, which lets you specify a +different data file: source ./Checking1.csv + If the file does not exist, it is just considered empty, without +raising an error. + If you specify just a file name with no path, hledger will look for -it in your system's downloads directory ('~/Downloads', currently): +it in the '~/Downloads' folder: source Checking1.csv - And if you specify a glob pattern, hledger will read the most recent -of the matched files (useful with repeated downloads): + You can use a glob pattern, to avoid specifying the file name +exactly: source Checking1*.csv + This has another benefit: if the pattern matches multiple files, +hledger will read the newest (most recently modified) one. This avoids +problems if you have downloaded a file multiple times without cleaning +up. + + All this enables a convenient workflow where can you just download +CSV files, then run 'hledger import rules/*'. + See also "Working with CSV > Reading files specified by rule". - -File: hledger.info, Node: encoding, Next: separator, Prev: source, Up: CSV + The 'archive' rule adds a few more features to 'source'; see below. -9.3 'encoding' + +File: hledger.info, Node: archive, Next: encoding, Prev: source, Up: CSV + +9.3 'archive' +============= + +Adding the 'archive' rule to your rules file affects importing or +reading files specified by 'source': + + * After successfully importing, 'import' will move the data file to + an archive directory ('data/' next to the rules file, + auto-created), renamed to + 'RULESFILEBASENAME.DATAFILEMODDATE.DATAFILEEXT'. Archiving data + files is optional, but it can be useful for troubleshooting, + detecting variations in your banks' CSV data, regenerating entries + with improved rules, etc. + + * 'import' will pick the oldest of 'source' glob matches, rather than + the newest. So if you have multiple versions of a download, + repeated imports will process them in chronological order. + + * For commands other than 'import', when the 'source' path or glob + pattern matches no files, hledger will try to read the latest + archived data file instead. This is convenient for working with + the downloaded data again, even after it has been imported. + + +File: hledger.info, Node: encoding, Next: separator, Prev: archive, Up: CSV + +9.4 'encoding' ============== encoding ENCODING @@ -4210,7 +4273,7 @@ rules. Eg: 'encoding iso-8859-1'.  File: hledger.info, Node: separator, Next: skip, Prev: encoding, Up: CSV -9.4 'separator' +9.5 'separator' =============== You can use the 'separator' rule to read other kinds of @@ -4235,7 +4298,7 @@ inferred automatically, and you won't need this rule.  File: hledger.info, Node: skip, Next: date-format, Prev: separator, Up: CSV -9.5 'skip' +9.6 'skip' ========== skip N @@ -4254,7 +4317,7 @@ required to be valid CSV.  File: hledger.info, Node: date-format, Next: timezone, Prev: skip, Up: CSV -9.6 'date-format' +9.7 'date-format' ================= date-format DATEFMT @@ -4280,10 +4343,13 @@ date-format %Y-%h-%d # Note the time and junk must be fully parsed, though only the date is used. date-format %-m/%-d/%Y %l:%M %p some other junk + Note currently there is no locale awareness for things like '%b', and +setting LC_TIME won't help. +  File: hledger.info, Node: timezone, Next: newest-first, Prev: date-format, Up: CSV -9.7 'timezone' +9.8 'timezone' ============== timezone TIMEZONE @@ -4312,7 +4378,7 @@ For others, use numeric format: +HHMM or -HHMM.  File: hledger.info, Node: newest-first, Next: intra-day-reversed, Prev: timezone, Up: CSV -9.8 'newest-first' +9.9 'newest-first' ================== hledger tries to ensure that the generated transactions will be ordered @@ -4335,8 +4401,8 @@ newest-first  File: hledger.info, Node: intra-day-reversed, Next: decimal-mark, Prev: newest-first, Up: CSV -9.9 'intra-day-reversed' -======================== +9.10 'intra-day-reversed' +========================= If CSV records within a single day are ordered opposite to the overall record order, you can add the 'intra-day-reversed' rule to improve the @@ -4354,7 +4420,7 @@ intra-day-reversed  File: hledger.info, Node: decimal-mark, Next: fields list, Prev: intra-day-reversed, Up: CSV -9.10 'decimal-mark' +9.11 'decimal-mark' =================== decimal-mark . @@ -4372,7 +4438,7 @@ misparsed numbers.  File: hledger.info, Node: fields list, Next: Field assignment, Prev: decimal-mark, Up: CSV -9.11 'fields' list +9.12 'fields' list ================== fields FIELDNAME1, FIELDNAME2, ... @@ -4417,7 +4483,7 @@ field (and generating a balance assertion).  File: hledger.info, Node: Field assignment, Next: Field names, Prev: fields list, Up: CSV -9.12 Field assignment +9.13 Field assignment ===================== HLEDGERFIELD FIELDVALUE @@ -4451,7 +4517,7 @@ comment note: %somefield - %anotherfield, date: %1  File: hledger.info, Node: Field names, Next: if block, Prev: Field assignment, Up: CSV -9.13 Field names +9.14 Field names ================ Note the two kinds of field names mentioned here, and used only in @@ -4500,7 +4566,7 @@ happens when you assign values to them:  File: hledger.info, Node: date field, Next: date2 field, Up: Field names -9.13.1 date field +9.14.1 date field ----------------- Assigning to 'date' sets the transaction date. @@ -4508,7 +4574,7 @@ Assigning to 'date' sets the transaction date.  File: hledger.info, Node: date2 field, Next: status field, Prev: date field, Up: Field names -9.13.2 date2 field +9.14.2 date2 field ------------------ 'date2' sets the transaction's secondary date, if any. @@ -4516,7 +4582,7 @@ File: hledger.info, Node: date2 field, Next: status field, Prev: date field,  File: hledger.info, Node: status field, Next: code field, Prev: date2 field, Up: Field names -9.13.3 status field +9.14.3 status field ------------------- 'status' sets the transaction's status, if any. @@ -4524,7 +4590,7 @@ File: hledger.info, Node: status field, Next: code field, Prev: date2 field,  File: hledger.info, Node: code field, Next: description field, Prev: status field, Up: Field names -9.13.4 code field +9.14.4 code field ----------------- 'code' sets the transaction's code, if any. @@ -4532,7 +4598,7 @@ File: hledger.info, Node: code field, Next: description field, Prev: status f  File: hledger.info, Node: description field, Next: comment field, Prev: code field, Up: Field names -9.13.5 description field +9.14.5 description field ------------------------ 'description' sets the transaction's description, if any. @@ -4540,7 +4606,7 @@ File: hledger.info, Node: description field, Next: comment field, Prev: code  File: hledger.info, Node: comment field, Next: account field, Prev: description field, Up: Field names -9.13.6 comment field +9.14.6 comment field -------------------- 'comment' sets the transaction's comment, if any. @@ -4558,7 +4624,7 @@ or a year-less date, will be ignored.  File: hledger.info, Node: account field, Next: amount field, Prev: comment field, Up: Field names -9.13.7 account field +9.14.7 account field -------------------- Assigning to 'accountN', where N is 1 to 99, sets the account name of @@ -4576,7 +4642,7 @@ or "income:unknown").  File: hledger.info, Node: amount field, Next: currency field, Prev: account field, Up: Field names -9.13.8 amount field +9.14.8 amount field ------------------- There are several ways to set posting amounts from CSV, useful in @@ -4634,7 +4700,7 @@ different situations.  File: hledger.info, Node: currency field, Next: balance field, Prev: amount field, Up: Field names -9.13.9 currency field +9.14.9 currency field --------------------- 'currency' sets a currency symbol, to be prepended to all postings' @@ -4647,7 +4713,7 @@ amount.  File: hledger.info, Node: balance field, Prev: currency field, Up: Field names -9.13.10 balance field +9.14.10 balance field --------------------- 'balanceN' sets a balance assertion amount (or if the posting amount is @@ -4665,7 +4731,7 @@ and currency.  File: hledger.info, Node: if block, Next: Matchers, Prev: Field names, Up: CSV -9.14 'if' block +9.15 'if' block =============== Rules can be applied conditionally, depending on patterns in the CSV @@ -4720,7 +4786,7 @@ if ,,,,  File: hledger.info, Node: Matchers, Next: if table, Prev: if block, Up: CSV -9.15 Matchers +9.16 Matchers ============= There are two kinds of matcher: @@ -4772,7 +4838,7 @@ expressions" in the hledger manual  File: hledger.info, Node: Multiple matchers, Next: Match groups, Up: Matchers -9.15.1 Multiple matchers +9.16.1 Multiple matchers ------------------------ When an if block has multiple matchers, each on its own line, @@ -4791,7 +4857,7 @@ and the date field contains "2025-01-01". _Added in 1.42._  File: hledger.info, Node: Match groups, Prev: Multiple matchers, Up: Matchers -9.15.2 Match groups +9.16.2 Match groups ------------------- _Added in 1.32_ @@ -4819,7 +4885,7 @@ if %account1 liabilities:family:(expenses:.*)  File: hledger.info, Node: if table, Next: balance-type, Prev: Matchers, Up: CSV -9.16 'if' table +9.17 'if' table =============== "if tables" are an alternative to if blocks; they can express many @@ -4880,7 +4946,7 @@ atm transaction fee,expenses:business:banking,deductible? check it  File: hledger.info, Node: balance-type, Next: include, Prev: if table, Up: CSV -9.17 'balance-type' +9.18 'balance-type' =================== Balance assertions generated by assigning to balanceN are of the simple @@ -4903,7 +4969,7 @@ balance-type ==*  File: hledger.info, Node: include, Next: Working with CSV, Prev: balance-type, Up: CSV -9.18 'include' +9.19 'include' ============== include RULESFILE @@ -4926,7 +4992,7 @@ include categorisation.rules  File: hledger.info, Node: Working with CSV, Next: CSV rules examples, Prev: include, Up: CSV -9.19 Working with CSV +9.20 Working with CSV ===================== Some tips: @@ -4952,7 +5018,7 @@ Some tips:  File: hledger.info, Node: Rapid feedback, Next: Valid CSV, Up: Working with CSV -9.19.1 Rapid feedback +9.20.1 Rapid feedback --------------------- It's a good idea to get rapid feedback while creating/troubleshooting @@ -4968,7 +5034,7 @@ output.  File: hledger.info, Node: Valid CSV, Next: File Extension, Prev: Rapid feedback, Up: Working with CSV -9.19.2 Valid CSV +9.20.2 Valid CSV ---------------- Note that hledger will only accept valid CSV conforming to RFC 4180, and @@ -4989,7 +5055,7 @@ permissive CSV parser like python's csv lib.  File: hledger.info, Node: File Extension, Next: Reading CSV from standard input, Prev: Valid CSV, Up: Working with CSV -9.19.3 File Extension +9.20.3 File Extension --------------------- To help hledger choose the CSV file reader and show the right error @@ -5009,7 +5075,7 @@ rule if needed.  File: hledger.info, Node: Reading CSV from standard input, Next: Reading multiple CSV files, Prev: File Extension, Up: Working with CSV -9.19.4 Reading CSV from standard input +9.20.4 Reading CSV from standard input -------------------------------------- You'll need the file format prefix when reading CSV from stdin also, @@ -5020,7 +5086,7 @@ $ cat foo.dat | hledger -f ssv:- print  File: hledger.info, Node: Reading multiple CSV files, Next: Reading files specified by rule, Prev: Reading CSV from standard input, Up: Working with CSV -9.19.5 Reading multiple CSV files +9.20.5 Reading multiple CSV files --------------------------------- If you use multiple '-f' options to read multiple CSV files at once, @@ -5031,7 +5097,7 @@ will be used for all the CSV files.  File: hledger.info, Node: Reading files specified by rule, Next: Valid transactions, Prev: Reading multiple CSV files, Up: Working with CSV -9.19.6 Reading files specified by rule +9.20.6 Reading files specified by rule -------------------------------------- Instead of specifying a CSV file in the command line, you can specify a @@ -5060,7 +5126,7 @@ most recent.  File: hledger.info, Node: Valid transactions, Next: Deduplicating importing, Prev: Reading files specified by rule, Up: Working with CSV -9.19.7 Valid transactions +9.20.7 Valid transactions ------------------------- After reading a CSV file, hledger post-processes and validates the @@ -5079,7 +5145,7 @@ $ hledger -f file.csv print | hledger -f- print  File: hledger.info, Node: Deduplicating importing, Next: Setting amounts, Prev: Valid transactions, Up: Working with CSV -9.19.8 Deduplicating, importing +9.20.8 Deduplicating, importing ------------------------------- When you download a CSV file periodically, eg to get your latest bank @@ -5109,7 +5175,7 @@ CSV data. See:  File: hledger.info, Node: Setting amounts, Next: Amount signs, Prev: Deduplicating importing, Up: Working with CSV -9.19.9 Setting amounts +9.20.9 Setting amounts ---------------------- Continuing from amount field above, here are more tips for @@ -5176,7 +5242,7 @@ amount-setting:  File: hledger.info, Node: Amount signs, Next: Setting currency/commodity, Prev: Setting amounts, Up: Working with CSV -9.19.10 Amount signs +9.20.10 Amount signs -------------------- There is some special handling making it easier to parse and to reverse @@ -5206,7 +5272,7 @@ its absolute value, ie discard its sign.  File: hledger.info, Node: Setting currency/commodity, Next: Amount decimal places, Prev: Amount signs, Up: Working with CSV -9.19.11 Setting currency/commodity +9.20.11 Setting currency/commodity ---------------------------------- If the currency/commodity symbol is included in the CSV's amount @@ -5254,7 +5320,7 @@ that would trigger the prepending effect, which we don't want here.  File: hledger.info, Node: Amount decimal places, Next: Referencing other fields, Prev: Setting currency/commodity, Up: Working with CSV -9.19.12 Amount decimal places +9.20.12 Amount decimal places ----------------------------- When you are reading CSV data, eg with a command like 'hledger -f @@ -5283,7 +5349,7 @@ want, add a 'commodity' directive to specify them.  File: hledger.info, Node: Referencing other fields, Next: How CSV rules are evaluated, Prev: Amount decimal places, Up: Working with CSV -9.19.13 Referencing other fields +9.20.13 Referencing other fields -------------------------------- In field assignments, you can interpolate only CSV fields, not hledger @@ -5320,7 +5386,7 @@ if something  File: hledger.info, Node: How CSV rules are evaluated, Next: Well factored rules, Prev: Referencing other fields, Up: Working with CSV -9.19.14 How CSV rules are evaluated +9.20.14 How CSV rules are evaluated ----------------------------------- Here's how to think of CSV rules being evaluated. If you get a @@ -5380,7 +5446,7 @@ hledger command the user specified.  File: hledger.info, Node: Well factored rules, Prev: How CSV rules are evaluated, Up: Working with CSV -9.19.15 Well factored rules +9.20.15 Well factored rules --------------------------- Some things than can help reduce duplication and complexity in rules @@ -5396,7 +5462,7 @@ files:  File: hledger.info, Node: CSV rules examples, Prev: Working with CSV, Up: CSV -9.20 CSV rules examples +9.21 CSV rules examples ======================= * Menu: @@ -5409,7 +5475,7 @@ File: hledger.info, Node: CSV rules examples, Prev: Working with CSV, Up: CSV  File: hledger.info, Node: Bank of Ireland, Next: Coinbase, Up: CSV rules examples -9.20.1 Bank of Ireland +9.21.1 Bank of Ireland ---------------------- Here's a CSV with two amount fields (Debit and Credit), and a balance @@ -5462,7 +5528,7 @@ imported into a journal file.  File: hledger.info, Node: Coinbase, Next: Amazon, Prev: Bank of Ireland, Up: CSV rules examples -9.20.2 Coinbase +9.21.2 Coinbase --------------- A simple example with some CSV from Coinbase. The spot price is @@ -5489,7 +5555,7 @@ $ hledger print -f coinbase.csv  File: hledger.info, Node: Amazon, Next: Paypal, Prev: Coinbase, Up: CSV rules examples -9.20.3 Amazon +9.21.3 Amazon ------------- Here we convert amazon.com order history, and use an if block to @@ -5547,7 +5613,7 @@ $ hledger -f amazon-orders.csv print  File: hledger.info, Node: Paypal, Prev: Amazon, Up: CSV rules examples -9.20.4 Paypal +9.21.4 Paypal ------------- Here's a real-world rules file for (customised) Paypal CSV, with some @@ -6040,18 +6106,25 @@ In hledger's user interfaces (though not in the journal file), you can optionally use "smart date" syntax. Smart dates can be written with english words, can be relative, and can have parts omitted. Missing parts are inferred as 1, when needed. Smart dates can be interpreted as -dates or periods depending on context. +dates or periods depending on the context. Examples: - '2004-01-01', '2004/10/1', '2004.9.1', '20240504' : + '2004-01-01', '2004/10/1', '2004.9.1', '20240504', '2024Q1' : Exact dates. The year must have at least four digits, the month must be 1-12, the day must be 1-31, the separator can be '-' or '/' or '.' or -nothing. +nothing. The q can be upper or lower case and the quarter number must +be 1-4. '2004-10' start of month +'2004q3' + + start of third quarter of 2004 +'q3' + + start of third quarter of current year '2004' start of year @@ -6168,7 +6241,7 @@ intervals, so that the last subperiod has the same length as the others. * 'hledger register --monthly' will end the report at the end of february. * 'hledger register --monthly --end 2/14' also will end the report at - the end of february. + the end of february (overriding the requested end date). * 'hledger register --monthly --begin 1/5 --end 2/14' will end the report on march 4th [1]. @@ -6394,8 +6467,8 @@ so '--depth assets=2 --depth liabilities=3 --depth 1' would collapse: * all other accounts to depth 1. If an account is matched by more than one regular expression depth -argument then the more specific one will used. For example, if '--depth -assets=1 --depth assets:bank:savings=2' is provided, then +argument then the more specific one will be used. For example, if +'--depth assets=1 --depth assets:bank:savings=2' is provided, then 'assets:bank:savings' will be collapsed to depth 2 rather than depth 1. This is because 'assets:bank:savings' matches at level 3 in the account name, while 'assets' matches at level 1. The same would be true with @@ -6800,7 +6873,7 @@ File: hledger.info, Node: all query, Prev: any query, Up: Boolean queries *'all:'QUERYEXPR''* Like 'expr:', but when used with transaction-oriented commands like 'print', it matches the transaction only if all postings are matched by -all of QUERYEXPR. +all of QUERYEXPR (and there is at least one posting). So, 'hledger print all:'cash and amt:0'' means "show transactions where all postings involve a cash account and have a zero amount". Or, 'hledger print all:'cash or checking'' means "show transactions @@ -7906,8 +7979,8 @@ $ hledger -f- print --value=end date:2000/01-2000/03 2000-02-01 (a) 2 B - With no report period specified, that shows the value as of the last -day of the journal (2000-03-01): + With no report period specified, the latest transaction date or price +date is used as valuation date (2000-04-01): $ hledger -f- print --value=end 2000-01-01 @@ -7919,8 +7992,8 @@ $ hledger -f- print --value=end 2000-03-01 (a) 3 B - Show the current value (the 2000-04-01 price is still in effect -today): + The value today is the same (the 2000-04-01 price is still in +effect): $ hledger -f- print --value=now 2000-01-01 @@ -8239,9 +8312,6 @@ write its number or a prefix or substring of its title. Tips: eg '-s4' to play at 4x original speed or '-s.5' to play at half speed. The default speed is 2x. - Other asciinema options can be added following a double dash, eg '-- --i.1' to limit pauses or '-- -h' to list asciinema's other options. - During playback, several keys are available: SPACE to pause/unpause, . to step forward (while paused), CTRL-c quit. @@ -8538,7 +8608,7 @@ File: hledger.info, Node: add, Next: add and balance assertions, Up: Data ent 26.1 add ======== -Record new transactions with interactive prompting in the console. +Add new transactions to a journal file, with interactive prompting. Flags: --no-new-accounts don't allow creating new accounts @@ -8579,6 +8649,8 @@ or press control-d or control-c to exit. 'add' to add this symbol for you. It does not do this; we assume that if you are using a 'D' directive you prefer not to see the commodity symbol repeated on amounts in the journal. + * 'add' creates entries in journal format; it won't work with + timeclock or timedot files. Examples: @@ -8661,17 +8733,18 @@ $ hledger import *.csv * Menu: -* Import preview:: +* Import dry run:: * Overlap detection:: * First import:: * Importing balance assignments:: * Import and commodity styles:: +* Import archiving:: * Import special cases::  -File: hledger.info, Node: Import preview, Next: Overlap detection, Up: import +File: hledger.info, Node: Import dry run, Next: Overlap detection, Up: import -26.4.1 Import preview +26.4.1 Import dry run --------------------- It's useful to preview the import by running first with '--dry-run', to @@ -8689,7 +8762,7 @@ $ hledger import --dry-run bank.csv | hledger -f- -I print unknown You could also run this repeatedly to see the effect of edits to your conversion rules: -$ watchexec -- 'hledger import --dry-run bank.csv | hledger -f- -I print unknown' +$ watchexec -- "hledger import --dry-run bank.csv | hledger -f- -I print unknown" Once the conversion and dates look good enough to import to your journal, perhaps with some manual fixups to follow, you would do the @@ -8698,7 +8771,7 @@ actual import: $ hledger import bank.csv  -File: hledger.info, Node: Overlap detection, Next: First import, Prev: Import preview, Up: import +File: hledger.info, Node: Overlap detection, Next: First import, Prev: Import dry run, Up: import 26.4.2 Overlap detection ------------------------ @@ -8823,7 +8896,7 @@ $ hledger print --new -f bank.csv >> $LEDGER_FILE that and send a pull request.)  -File: hledger.info, Node: Import and commodity styles, Next: Import special cases, Prev: Importing balance assignments, Up: import +File: hledger.info, Node: Import and commodity styles, Next: Import archiving, Prev: Importing balance assignments, Up: import 26.4.5 Import and commodity styles ---------------------------------- @@ -8835,38 +8908,83 @@ directives or inferred from the journal's amounts. Related: CSV > Amount decimal places.  -File: hledger.info, Node: Import special cases, Prev: Import and commodity styles, Up: import +File: hledger.info, Node: Import archiving, Next: Import special cases, Prev: Import and commodity styles, Up: import -26.4.6 Import special cases +26.4.6 Import archiving +----------------------- + +When importing from a CSV rules file ('hledger import bank.rules'), you +can use the archive rule to enable automatic archiving of the data file. +After a successful import, the data file (specified by 'source') will be +moved to an archive folder ('data/', next to the rules file, +auto-created), and renamed similar to the rules file, with a date. This +can be useful for troubleshooting, detecting variations in your banks' +CSV data, regenerating entries with improved rules, etc. + + The 'archive' rule also causes 'import' to handle 'source' glob +patterns differently: when there are multiple matched files, it will +pick the oldest, not the newest. + + +File: hledger.info, Node: Import special cases, Prev: Import archiving, Up: import + +26.4.7 Import special cases --------------------------- -If you have a download whose file name varies, you could rename it to a -fixed name after each download. Or you could use a CSV 'source' rule -with a suitable glob pattern, and import from the .rules file instead of -the data file. +* Menu: - Here's a situation where you would need to run 'import' with care: -say you download 'bank.csv', but forget to import it or delete it. And -next month you download it again. This time your web browser may save -it as 'bank (2).csv'. So now each of these may have data not included -in the other. And a 'source' rule with a glob pattern would match only -the most recent file. So in this case you should import from each one -in turn, in the correct order, taking care to use the same filename each -time: +* Deduplication:: +* Varying file name:: +* Multiple versions:: -$ hledger import bank.csv -$ mv 'bank (2).csv' bank.csv -$ hledger import bank.csv + +File: hledger.info, Node: Deduplication, Next: Varying file name, Up: Import special cases - Here are two kinds of "deduplication" which 'import' does not handle -(and generally should not, since these can happen legitimately in -financial data): +26.4.7.1 Deduplication +...................... + +Here are two kinds of "deduplication" which 'import' does not handle +(and should not, because these can happen legitimately in financial +data): * Two or more of the new CSV records are identical, and generate identical new journal entries. * A new CSV record generates a journal entry identical to one(s) already in the journal. + +File: hledger.info, Node: Varying file name, Next: Multiple versions, Prev: Deduplication, Up: Import special cases + +26.4.7.2 Varying file name +.......................... + +If you have a download whose file name varies, you could rename it to a +fixed name after each download. Or you could use a CSV 'source' rule +with a suitable glob pattern, and import from the .rules file. + + +File: hledger.info, Node: Multiple versions, Prev: Varying file name, Up: Import special cases + +26.4.7.3 Multiple versions +.......................... + +Say you download 'bank.csv', import it, but forget to delete it from +your downloads folder. The next time you download it, your web browser +will save it as (eg) 'bank (2).csv'. The source rule's glob patterns +are for just this situation: instead of specifying 'source bank.csv', +specify 'source bank*.csv'. Then 'hledger -f bank.rules CMD' or +'hledger import bank.rules' will automatically pick the newest matched +file ('bank (2).csv'). + + Alternately, what if you download, but forget to import or delete, +then download again ? Now each of 'bank.csv' and 'bank (2).csv' might +contain data that's not in the other, and not in your journal. In this +case, it's best to import each of them in turn, oldest first (otherwise, +overlap detection could cause new records to be skipped). Enabling +import archiving ensures this. Then 'hledger import bank.rules; hledger +import bank.rules' will import and archive first 'bank.csv', then 'bank +(2).csv'. +  File: hledger.info, Node: Basic report commands, Next: Standard report commands, Prev: Data entry commands, Up: Top @@ -9538,7 +9656,7 @@ matches uniquely. Transactions involving subaccounts of this account will also be shown. 'aregister' ignores depth limits, so its final total will always -match a balance report with similar arguments. +match a historical balance report with similar arguments. Any additional arguments form a query which will filter the transactions shown. Note some queries will disturb the running balance, @@ -11922,10 +12040,10 @@ two spaces between account and amount. More: -$ hledger rewrite -- [QUERY] --add-posting "ACCT AMTEXPR" ... -$ hledger rewrite -- ^income --add-posting '(liabilities:tax) *.33' -$ hledger rewrite -- expenses:gifts --add-posting '(budget:gifts) *-1"' -$ hledger rewrite -- ^income --add-posting '(budget:foreign currency) *0.25 JPY; diversify' +$ hledger rewrite [QUERY] --add-posting "ACCT AMTEXPR" ... +$ hledger rewrite ^income --add-posting '(liabilities:tax) *.33' +$ hledger rewrite expenses:gifts --add-posting '(budget:gifts) *-1"' +$ hledger rewrite ^income --add-posting '(budget:foreign currency) *0.25 JPY; diversify' Argument for '--add-posting' option is a usual posting of transaction with an exception for amount specification. More precisely, you can use @@ -11966,12 +12084,12 @@ $ rewrite-rules.journal transactions you usually write. It indicates the query by which you want to match the posting to add new ones. -$ hledger rewrite -- -f input.journal -f rewrite-rules.journal > rewritten-tidy-output.journal +$ hledger rewrite -f input.journal -f rewrite-rules.journal > rewritten-tidy-output.journal This is something similar to the commands pipeline: -$ hledger rewrite -- -f input.journal '^income' --add-posting '(liabilities:tax) *.33' \ - | hledger rewrite -- -f - expenses:gifts --add-posting 'budget:gifts *-1' \ +$ hledger rewrite -f input.journal '^income' --add-posting '(liabilities:tax) *.33' \ + | hledger rewrite -f - expenses:gifts --add-posting 'budget:gifts *-1' \ --add-posting 'assets:budget *1' \ > rewritten-tidy-output.journal @@ -11988,7 +12106,7 @@ File: hledger.info, Node: Diff output format, Next: rewrite vs print --auto, To use this tool for batch modification of your journal files you may find useful output in form of unified diff. -$ hledger rewrite -- --diff -f examples/sample.journal '^income' --add-posting '(liabilities:tax) *.33' +$ hledger rewrite --diff -f examples/sample.journal '^income' --add-posting '(liabilities:tax) *.33' Output might look like: @@ -12367,9 +12485,9 @@ described in OPTIONS, here are some tips that might help: * command-specific options must go after the command (it's fine to put common options there too: 'hledger CMD OPTS ARGS') - * running add-on executables directly simplifies command line parsing - ('hledger-ui OPTS ARGS') - * enclose "problematic" args in single quotes + * you can run addon commands via hledger ('hledger ui [ARGS]') or + directly ('hledger-ui [ARGS]') + * enclose "problematic" arguments in single quotes * if needed, also add a backslash to hide regular expression metacharacters from the shell * to see how a misbehaving command line is being parsed, add @@ -12790,18 +12908,12 @@ We welcome bug reports in the hledger issue tracker Some known issues and limitations: - The need to precede add-on command options with '--' when invoked -from hledger is awkward. (See Command options, Constructing command -lines.) - A system locale with a suitable text encoding must be configured to work with non-ascii data. (See Text encoding, Troubleshooting.) - On Microsoft Windows, depending whether you are running in a CMD -window or a Cygwin/MSYS/Mintty window and how you installed hledger, -non-ascii characters and colours may not be supported, and the tab key -may not be supported by 'hledger add'. (Running in a WSL window should -resolve these.) + On Microsoft Windows, depending what kind of terminal window you use, +non-ascii characters, ANSI text formatting, and/or the add command's TAB +key for completion, may not be supported. When processing large data files, hledger uses more memory than Ledger. @@ -12851,394 +12963,399 @@ See hledger and Ledger for full details.  Tag Table: Node: Top208 -Node: PART 1 USER INTERFACE4363 -Node: Input4502 -Node: Text encoding5594 -Node: Data formats6343 -Node: Standard input8077 -Node: Multiple files8466 -Node: Strict mode9203 -Node: Commands10037 -Node: Add-on commands11319 -Node: Options12537 -Node: Special characters19492 -Node: Escaping shell special characters20442 -Node: Escaping on Windows21686 -Node: Escaping regular expression special characters22419 -Node: Escaping add-on arguments23406 -Node: Escaping in other situations24435 -Node: Using a wild card25394 -Node: Unicode characters25773 -Node: Regular expressions27194 -Node: hledger's regular expressions30453 -Node: Argument files32094 -Node: Config files32797 -Node: Shell completions36098 -Node: Output36587 -Node: Output destination36778 -Node: Output format37336 -Node: Text output39122 -Node: Box-drawing characters40101 -Node: Colour40601 -Node: Paging41187 -Node: HTML output42713 -Node: CSV / TSV output43131 -Node: FODS output43385 -Node: Beancount output44189 -Node: Beancount account names45690 -Node: Beancount commodity names46231 -Node: Beancount virtual postings46878 -Node: Beancount metadata47194 -Node: Beancount costs47974 -Node: Beancount operating currency48390 -Node: SQL output48840 -Node: JSON output49631 -Node: Commodity styles50448 -Node: Debug output51335 -Node: Environment52167 -Node: PART 2 DATA FORMATS52824 -Node: Journal52967 -Node: Journal cheatsheet55445 -Node: Comments61696 -Node: Transactions62640 -Node: Dates63777 -Node: Simple dates63929 -Node: Posting dates64545 -Node: Status65632 -Node: Code67398 -Node: Description67733 -Node: Payee and note68420 -Node: Transaction comments69511 -Node: Postings70027 -Node: Debits and credits71190 -Node: The two space delimiter71800 -Node: Account names72365 -Node: Amounts74169 -Node: Decimal marks75198 -Node: Digit group marks76302 -Node: Commodity76937 -Node: Costs78054 -Node: Balance assertions80306 -Node: Assertions and ordering81554 -Node: Assertions and multiple files82273 -Node: Assertions and costs83441 -Node: Assertions and commodities84088 -Node: Assertions and subaccounts85747 -Node: Assertions and status86407 -Node: Assertions and virtual postings86827 -Node: Assertions and auto postings87192 -Node: Assertions and precision88067 -Node: Assertions and hledger add88551 -Node: Posting comments89299 -Node: Transaction balancing89839 -Node: Tags92047 -Node: Querying with tags93341 -Node: Displaying tags94140 -Node: When to use tags ?94536 -Node: Tag names95200 -Node: Special tags95753 -Node: Directives97318 -Node: Directives and multiple files98775 -Node: Directive effects99720 -Node: account directive102876 -Node: Account comments104326 -Node: Account error checking104985 -Node: Account display order106522 -Node: Account types107720 -Node: alias directive110995 -Node: Basic aliases112206 -Node: Regex aliases113081 -Node: Combining aliases114128 -Node: Aliases and multiple files115582 -Node: end aliases directive116365 -Node: Aliases can generate bad account names116733 -Node: Aliases and account types117566 -Node: commodity directive118458 -Node: Commodity directive syntax120045 -Node: Commodity error checking121694 -Node: decimal-mark directive122169 -Node: include directive122748 -Node: P directive123824 -Node: payee directive124858 -Node: tag directive125480 -Node: Periodic transactions126092 -Node: Periodic rule syntax128246 -Node: Periodic rules and relative dates129069 -Node: Two spaces between period expression and description!129846 -Node: Auto postings130807 -Node: Auto postings and multiple files133967 -Node: Auto postings and dates134372 -Node: Auto postings and transaction balancing / inferred amounts / balance assertions134813 -Node: Auto posting tags135659 -Node: Auto postings on forecast transactions only136554 -Node: Other syntax137024 -Node: Balance assignments137796 -Node: Balance assignments and costs139324 -Node: Balance assignments and multiple files139746 -Node: Bracketed posting dates140169 -Node: D directive140867 -Node: apply account directive142640 -Node: Y directive143507 -Node: Secondary dates144495 -Node: Star comments145980 -Node: Valuation expressions146672 -Node: Virtual postings146971 -Node: Other Ledger directives148595 -Node: Other cost/lot notations149357 -Node: CSV152198 -Node: CSV rules cheatsheet154352 -Node: source156379 -Node: encoding157379 -Node: separator158421 -Node: skip159074 -Node: date-format159724 -Node: timezone160567 -Node: newest-first161693 -Node: intra-day-reversed162406 -Node: decimal-mark163006 -Node: fields list163486 -Node: Field assignment165294 -Node: Field names166513 -Node: date field167845 -Node: date2 field168009 -Node: status field168204 -Node: code field168394 -Node: description field168582 -Node: comment field168799 -Node: account field169356 -Node: amount field170074 -Node: currency field172913 -Node: balance field173321 -Node: if block173844 -Node: Matchers175371 -Node: Multiple matchers177361 -Node: Match groups178169 -Node: if table179062 -Node: balance-type181125 -Node: include181952 -Node: Working with CSV182521 -Node: Rapid feedback183073 -Node: Valid CSV183656 -Node: File Extension184532 -Node: Reading CSV from standard input185267 -Node: Reading multiple CSV files185653 -Node: Reading files specified by rule186129 -Node: Valid transactions187526 -Node: Deduplicating importing188351 -Node: Setting amounts189580 -Node: Amount signs192107 -Node: Setting currency/commodity193172 -Node: Amount decimal places194548 -Node: Referencing other fields195805 -Node: How CSV rules are evaluated196913 -Node: Well factored rules199630 -Node: CSV rules examples200120 -Node: Bank of Ireland200318 -Node: Coinbase201915 -Node: Amazon203098 -Node: Paypal204940 -Node: Timeclock212690 -Node: Timedot215515 -Node: Timedot examples218992 -Node: PART 3 REPORTING CONCEPTS221269 -Node: Time periods221433 -Node: Report start & end date221706 -Node: Smart dates223182 -Node: Report intervals225125 -Node: Date adjustments225699 -Node: Start date adjustment225919 -Node: End date adjustment226822 -Node: Period headings227567 -Node: Period expressions228500 -Node: Period expressions with a report interval230405 -Node: More complex report intervals230853 -Node: Multiple weekday intervals232969 -Node: Depth233980 -Node: Queries235815 -Node: Query types238487 -Node: acct query238862 -Node: amt query239173 -Node: code query239870 -Node: cur query240065 -Node: desc query240671 -Node: date query240854 -Node: date2 query241250 -Node: depth query241541 -Node: note query241877 -Node: payee query242143 -Node: real query242424 -Node: status query242629 -Node: type query242869 -Node: tag query243402 -Node: Negative queries244031 -Node: not query244213 -Node: Space-separated queries244500 -Node: Boolean queries245188 -Node: expr query246506 -Node: any query247186 -Node: all query247639 -Node: Queries and command options248185 -Node: Queries and account aliases248633 -Node: Queries and valuation248958 -Node: Pivoting249320 -Node: Generating data251596 -Node: Forecasting253396 -Node: --forecast254052 -Node: Inspecting forecast transactions255153 -Node: Forecast reports256486 -Node: Forecast tags257595 -Node: Forecast period in detail258215 -Node: Forecast troubleshooting259303 -Node: Budgeting260374 -Node: Amount formatting260934 -Node: Commodity display style261178 -Node: Rounding263019 -Node: Trailing decimal marks263624 -Node: Amount parseability264557 -Node: Cost reporting266166 -Node: Recording costs266997 -Node: Reporting at cost268724 -Node: Equity conversion postings269489 -Node: Inferring equity conversion postings272134 -Node: Combining costs and equity conversion postings273276 -Node: Requirements for detecting equity conversion postings274501 -Node: Infer cost and equity by default ?276023 -Node: Value reporting276460 -Node: -V Value277396 -Node: -X Value in specified commodity277723 -Node: Valuation date278073 -Node: Finding market price279033 -Node: --infer-market-prices market prices from transactions280413 -Node: Valuation commodity283457 -Node: --value Flexible valuation284890 -Node: Valuation examples286733 -Node: Interaction of valuation and queries288865 -Node: Effect of valuation on reports289582 -Node: PART 4 COMMANDS297480 -Node: Help commands300269 -Node: commands300455 -Node: demo300663 -Node: help301897 -Node: User interface commands303602 -Node: repl303813 -Node: Examples306077 -Node: run306635 -Node: Examples 2309050 -Node: ui310074 -Node: web310211 -Node: Data entry commands310339 -Node: add310600 -Node: add and balance assertions313075 -Node: add and balance assignments313799 -Node: import314360 -Node: Import preview315418 -Node: Overlap detection316366 -Node: First import319252 -Node: Importing balance assignments320447 -Node: Import and commodity styles321502 -Node: Import special cases321940 -Node: Basic report commands323275 -Node: accounts323576 -Node: codes326222 -Node: commodities327244 -Node: descriptions328001 -Node: files328461 -Node: notes328758 -Node: payees329270 -Node: prices330182 -Node: stats331074 -Node: tags332815 -Node: Standard report commands334352 -Node: print334657 -Node: print explicitness337471 -Node: print amount style338391 -Node: print parseability339629 -Node: print other features340548 -Node: print output format341509 -Node: aregister344794 -Node: aregister and posting dates349347 -Node: register350248 -Node: Custom register output357489 -Node: balancesheet358674 -Node: balancesheetequity363639 -Node: cashflow368974 -Node: incomestatement373787 -Node: Advanced report commands378636 -Node: balance378844 -Node: balance features384265 -Node: Simple balance report386368 -Node: Balance report line format388178 -Node: Filtered balance report390538 -Node: List or tree mode391057 -Node: Depth limiting392570 -Node: Dropping top-level accounts393337 -Node: Showing declared accounts393847 -Node: Sorting by amount394577 -Node: Percentages395431 -Node: Multi-period balance report396138 -Node: Balance change end balance398890 -Node: Balance report modes400527 -Node: Calculation mode401206 -Node: Accumulation mode401910 -Node: Valuation mode403011 -Node: Combining balance report modes404355 -Node: Budget report406385 -Node: Using the budget report408685 -Node: Budget date surprises410961 -Node: Selecting budget goals412325 -Node: Budgeting vs forecasting413273 -Node: Balance report layout414950 -Node: Wide layout416155 -Node: Tall layout418560 -Node: Bare layout419866 -Node: Tidy layout421930 -Node: Balance report output423474 -Node: Some useful balance reports424248 -Node: roi425508 -Node: Spaces and special characters in --inv and --pnl427755 -Node: Semantics of --inv and --pnl428481 -Node: IRR and TWR explained430568 -Node: Chart commands433979 -Node: activity434160 -Node: Data generation commands434657 -Node: close434863 -Node: close --clopen437426 -Node: close --close439600 -Node: close --open440124 -Node: close --assert440374 -Node: close --assign440701 -Node: close --retain441380 -Node: close customisation442237 -Node: close and balance assertions443881 -Node: close examples445403 -Node: Retain earnings445640 -Node: Migrate balances to a new file446143 -Node: More detailed close examples447505 -Node: rewrite447727 -Node: Re-write rules in a file450299 -Node: Diff output format451609 -Node: rewrite vs print --auto452882 -Node: Maintenance commands453596 -Node: check453815 -Node: Basic checks454897 -Node: Strict checks455918 -Node: Other checks456855 -Node: Custom checks458607 -Node: diff459062 -Node: setup460270 -Node: test463137 -Node: PART 5 COMMON TASKS464040 -Node: Getting help464273 -Node: Constructing command lines465182 -Node: Starting a journal file466020 -Node: Setting LEDGER_FILE467404 -Node: Setting opening balances468662 -Node: Recording transactions471984 -Node: Reconciling472709 -Node: Reporting475098 -Node: Migrating to a new file479212 -Node: BUGS479661 -Node: Troubleshooting480629 +Node: PART 1 USER INTERFACE4307 +Node: Input4446 +Node: Text encoding5538 +Node: Data formats6287 +Node: Standard input8021 +Node: Multiple files8410 +Node: Strict mode9147 +Node: Commands9981 +Node: Add-on commands11263 +Node: Options12314 +Node: Special characters19464 +Node: Escaping shell special characters20414 +Node: Escaping on Windows21658 +Node: Escaping regular expression special characters22391 +Node: Escaping add-on arguments23378 +Node: Escaping in other situations24407 +Node: Using a wild card25366 +Node: Unicode characters25745 +Node: Regular expressions27166 +Node: hledger's regular expressions30425 +Node: Argument files32066 +Node: Config files32769 +Node: Shell completions36037 +Node: Output36526 +Node: Output destination36717 +Node: Output format37275 +Node: Text output39061 +Node: Box-drawing characters40040 +Node: Colour40540 +Node: Paging41126 +Node: HTML output42645 +Node: CSV / TSV output43063 +Node: FODS output43317 +Node: Beancount output44121 +Node: Beancount account names45622 +Node: Beancount commodity names46163 +Node: Beancount virtual postings46810 +Node: Beancount metadata47126 +Node: Beancount costs47906 +Node: Beancount operating currency48322 +Node: SQL output48772 +Node: JSON output49563 +Node: Commodity styles50380 +Node: Debug output51267 +Node: Environment52099 +Node: PART 2 DATA FORMATS52756 +Node: Journal52899 +Node: Journal cheatsheet55377 +Node: Comments61628 +Node: Transactions62572 +Node: Dates63709 +Node: Simple dates63861 +Node: Posting dates64477 +Node: Status65564 +Node: Code67330 +Node: Description67665 +Node: Payee and note68352 +Node: Transaction comments69443 +Node: Postings69959 +Node: Debits and credits71122 +Node: The two space delimiter71732 +Node: Account names72297 +Node: Amounts74101 +Node: Decimal marks75130 +Node: Digit group marks76234 +Node: Commodity76869 +Node: Costs77986 +Node: Balance assertions80238 +Node: Assertions and ordering81486 +Node: Assertions and multiple files82205 +Node: Assertions and costs83373 +Node: Assertions and commodities84020 +Node: Assertions and subaccounts85679 +Node: Assertions and status86339 +Node: Assertions and virtual postings86759 +Node: Assertions and auto postings87124 +Node: Assertions and precision87999 +Node: Assertions and hledger add88483 +Node: Posting comments89231 +Node: Transaction balancing89771 +Node: Tags91979 +Node: Querying with tags93273 +Node: Displaying tags94072 +Node: When to use tags ?94468 +Node: Tag names95132 +Node: Special tags95685 +Node: Directives97250 +Node: Directives and multiple files98707 +Node: Directive effects99652 +Node: account directive102808 +Node: Account comments104258 +Node: Account error checking104917 +Node: Account display order106454 +Node: Account types107652 +Node: alias directive110927 +Node: Basic aliases112138 +Node: Regex aliases113013 +Node: Combining aliases114060 +Node: Aliases and multiple files115514 +Node: end aliases directive116297 +Node: Aliases can generate bad account names116665 +Node: Aliases and account types117498 +Node: commodity directive118390 +Node: Commodity directive syntax119977 +Node: Commodity error checking121626 +Node: decimal-mark directive122101 +Node: include directive122680 +Node: P directive124898 +Node: payee directive125932 +Node: tag directive126554 +Node: Periodic transactions127166 +Node: Periodic rule syntax129320 +Node: Periodic rules and relative dates130143 +Node: Two spaces between period expression and description!130920 +Node: Auto postings131881 +Node: Auto postings and multiple files135167 +Node: Auto postings and dates135572 +Node: Auto postings and transaction balancing / inferred amounts / balance assertions136013 +Node: Auto posting tags136859 +Node: Auto postings on forecast transactions only137754 +Node: Other syntax138224 +Node: Balance assignments138996 +Node: Balance assignments and costs140524 +Node: Balance assignments and multiple files140946 +Node: Bracketed posting dates141369 +Node: D directive142067 +Node: apply account directive143840 +Node: Y directive144707 +Node: Secondary dates145695 +Node: Star comments147180 +Node: Valuation expressions147872 +Node: Virtual postings148171 +Node: Other Ledger directives149795 +Node: Other cost/lot notations150557 +Node: CSV153398 +Node: CSV rules cheatsheet155564 +Node: source157591 +Node: archive159012 +Node: encoding160103 +Node: separator161146 +Node: skip161799 +Node: date-format162449 +Node: timezone163394 +Node: newest-first164520 +Node: intra-day-reversed165233 +Node: decimal-mark165835 +Node: fields list166315 +Node: Field assignment168123 +Node: Field names169342 +Node: date field170674 +Node: date2 field170838 +Node: status field171033 +Node: code field171223 +Node: description field171411 +Node: comment field171628 +Node: account field172185 +Node: amount field172903 +Node: currency field175742 +Node: balance field176150 +Node: if block176673 +Node: Matchers178200 +Node: Multiple matchers180190 +Node: Match groups180998 +Node: if table181891 +Node: balance-type183954 +Node: include184781 +Node: Working with CSV185350 +Node: Rapid feedback185902 +Node: Valid CSV186485 +Node: File Extension187361 +Node: Reading CSV from standard input188096 +Node: Reading multiple CSV files188482 +Node: Reading files specified by rule188958 +Node: Valid transactions190355 +Node: Deduplicating importing191180 +Node: Setting amounts192409 +Node: Amount signs194936 +Node: Setting currency/commodity196001 +Node: Amount decimal places197377 +Node: Referencing other fields198634 +Node: How CSV rules are evaluated199742 +Node: Well factored rules202459 +Node: CSV rules examples202949 +Node: Bank of Ireland203147 +Node: Coinbase204744 +Node: Amazon205927 +Node: Paypal207769 +Node: Timeclock215519 +Node: Timedot218344 +Node: Timedot examples221821 +Node: PART 3 REPORTING CONCEPTS224098 +Node: Time periods224262 +Node: Report start & end date224535 +Node: Smart dates226011 +Node: Report intervals228134 +Node: Date adjustments228708 +Node: Start date adjustment228928 +Node: End date adjustment229831 +Node: Period headings230612 +Node: Period expressions231545 +Node: Period expressions with a report interval233450 +Node: More complex report intervals233898 +Node: Multiple weekday intervals236014 +Node: Depth237025 +Node: Queries238863 +Node: Query types241535 +Node: acct query241910 +Node: amt query242221 +Node: code query242918 +Node: cur query243113 +Node: desc query243719 +Node: date query243902 +Node: date2 query244298 +Node: depth query244589 +Node: note query244925 +Node: payee query245191 +Node: real query245472 +Node: status query245677 +Node: type query245917 +Node: tag query246450 +Node: Negative queries247079 +Node: not query247261 +Node: Space-separated queries247548 +Node: Boolean queries248236 +Node: expr query249554 +Node: any query250234 +Node: all query250687 +Node: Queries and command options251269 +Node: Queries and account aliases251717 +Node: Queries and valuation252042 +Node: Pivoting252404 +Node: Generating data254680 +Node: Forecasting256480 +Node: --forecast257136 +Node: Inspecting forecast transactions258237 +Node: Forecast reports259570 +Node: Forecast tags260679 +Node: Forecast period in detail261299 +Node: Forecast troubleshooting262387 +Node: Budgeting263458 +Node: Amount formatting264018 +Node: Commodity display style264262 +Node: Rounding266103 +Node: Trailing decimal marks266708 +Node: Amount parseability267641 +Node: Cost reporting269250 +Node: Recording costs270081 +Node: Reporting at cost271808 +Node: Equity conversion postings272573 +Node: Inferring equity conversion postings275218 +Node: Combining costs and equity conversion postings276360 +Node: Requirements for detecting equity conversion postings277585 +Node: Infer cost and equity by default ?279107 +Node: Value reporting279544 +Node: -V Value280480 +Node: -X Value in specified commodity280807 +Node: Valuation date281157 +Node: Finding market price282117 +Node: --infer-market-prices market prices from transactions283497 +Node: Valuation commodity286541 +Node: --value Flexible valuation287974 +Node: Valuation examples289817 +Node: Interaction of valuation and queries291961 +Node: Effect of valuation on reports292678 +Node: PART 4 COMMANDS300576 +Node: Help commands303365 +Node: commands303551 +Node: demo303759 +Node: help304852 +Node: User interface commands306557 +Node: repl306768 +Node: Examples309032 +Node: run309590 +Node: Examples 2312005 +Node: ui313029 +Node: web313166 +Node: Data entry commands313294 +Node: add313555 +Node: add and balance assertions316129 +Node: add and balance assignments316853 +Node: import317414 +Node: Import dry run318493 +Node: Overlap detection319441 +Node: First import322327 +Node: Importing balance assignments323522 +Node: Import and commodity styles324577 +Node: Import archiving325011 +Node: Import special cases325836 +Node: Deduplication326054 +Node: Varying file name326545 +Node: Multiple versions326929 +Node: Basic report commands328036 +Node: accounts328337 +Node: codes330983 +Node: commodities332005 +Node: descriptions332762 +Node: files333222 +Node: notes333519 +Node: payees334031 +Node: prices334943 +Node: stats335835 +Node: tags337576 +Node: Standard report commands339113 +Node: print339418 +Node: print explicitness342232 +Node: print amount style343152 +Node: print parseability344390 +Node: print other features345309 +Node: print output format346270 +Node: aregister349555 +Node: aregister and posting dates354119 +Node: register355020 +Node: Custom register output362261 +Node: balancesheet363446 +Node: balancesheetequity368411 +Node: cashflow373746 +Node: incomestatement378559 +Node: Advanced report commands383408 +Node: balance383616 +Node: balance features389037 +Node: Simple balance report391140 +Node: Balance report line format392950 +Node: Filtered balance report395310 +Node: List or tree mode395829 +Node: Depth limiting397342 +Node: Dropping top-level accounts398109 +Node: Showing declared accounts398619 +Node: Sorting by amount399349 +Node: Percentages400203 +Node: Multi-period balance report400910 +Node: Balance change end balance403662 +Node: Balance report modes405299 +Node: Calculation mode405978 +Node: Accumulation mode406682 +Node: Valuation mode407783 +Node: Combining balance report modes409127 +Node: Budget report411157 +Node: Using the budget report413457 +Node: Budget date surprises415733 +Node: Selecting budget goals417097 +Node: Budgeting vs forecasting418045 +Node: Balance report layout419722 +Node: Wide layout420927 +Node: Tall layout423332 +Node: Bare layout424638 +Node: Tidy layout426702 +Node: Balance report output428246 +Node: Some useful balance reports429020 +Node: roi430280 +Node: Spaces and special characters in --inv and --pnl432527 +Node: Semantics of --inv and --pnl433253 +Node: IRR and TWR explained435340 +Node: Chart commands438751 +Node: activity438932 +Node: Data generation commands439429 +Node: close439635 +Node: close --clopen442198 +Node: close --close444372 +Node: close --open444896 +Node: close --assert445146 +Node: close --assign445473 +Node: close --retain446152 +Node: close customisation447009 +Node: close and balance assertions448653 +Node: close examples450175 +Node: Retain earnings450412 +Node: Migrate balances to a new file450915 +Node: More detailed close examples452277 +Node: rewrite452499 +Node: Re-write rules in a file455059 +Node: Diff output format456360 +Node: rewrite vs print --auto457630 +Node: Maintenance commands458344 +Node: check458563 +Node: Basic checks459645 +Node: Strict checks460666 +Node: Other checks461603 +Node: Custom checks463355 +Node: diff463810 +Node: setup465018 +Node: test467885 +Node: PART 5 COMMON TASKS468788 +Node: Getting help469021 +Node: Constructing command lines469930 +Node: Starting a journal file470775 +Node: Setting LEDGER_FILE472159 +Node: Setting opening balances473417 +Node: Recording transactions476739 +Node: Reconciling477464 +Node: Reporting479853 +Node: Migrating to a new file483967 +Node: BUGS484416 +Node: Troubleshooting485129  End Tag Table diff --git a/hledger/hledger.txt b/hledger/hledger.txt index 95847641a..b62d2e049 100644 --- a/hledger/hledger.txt +++ b/hledger/hledger.txt @@ -9,9 +9,6 @@ SYNOPSIS hledger or hledger COMMAND [OPTS] [ARGS] - or - hledger ADDONCMD [OPTS] -- [ADDONOPTS] [ADDONARGS] - DESCRIPTION hledger is a robust, user-friendly, cross-platform set of programs for tracking money, time, or any other commodity, using double-entry ac- @@ -195,7 +192,7 @@ Input o Are all commodity conversions declared explicitly ? - You can use the check command to run individual checks -- the ones + You can use the check command to run individual checks - the ones listed above and some more. Commands @@ -224,25 +221,23 @@ Commands nal, run hledger CMD -h. Eg: hledger bal -h. Add-on commands - In addition to the built-in commands, you can install add-on commands: - programs or scripts named "hledger-SOMETHING", which will also appear - in hledger's commands list. If you used the hledger-install script, - you will have several add-ons installed already. Some more can be - found in hledger's bin/ directory, documented at - https://hledger.org/scripts.html. + In addition to the built-in commands, you can install add-on commands, + which will also appear in hledger's commands list. Some of these can + be installed as separate packages; others can be found in hledger's + bin/ directory, documented at https://hledger.org/scripts.html. - More precisely, add-on commands are programs or scripts in your shell's - PATH, whose name starts with "hledger-" and ends with no extension or a - recognised extension (".bat", ".com", ".exe", ".hs", ".js", ".lhs", - ".lua", ".php", ".pl", ".py", ".rb", ".rkt", or ".sh"), and (on unix - and mac) which has executable permission for the current user. + Add-on commands are programs or scripts in your shell's PATH, whose + name starts with "hledger-" and ends with no extension or a recognised + extension (".bat", ".com", ".exe", ".hs", ".js", ".lhs", ".lua", + ".php", ".pl", ".py", ".rb", ".rkt", or ".sh"), and (on unix and mac) + which has executable permission for the current user. - You can run add-on commands using hledger, much like built-in commands: - hledger ADDONCMD [-- ADDONCMDOPTS] [ADDONCMDARGS]. But note the double - hyphen argument, required before add-on-specific options. Eg: hledger - ui -- --watch or hledger web -- --serve. If this causes difficulty, - you can always run the add-on directly, without using hledger: - hledger-ui --watch or hledger-web --serve. + You can run add-on commands directly: hledger-ui --watch. + + Or you can run them with hledger, like built-in commands: hledger ui + --watch. In this case hledger's config file will be used, so you can + set custom options for the addon there. (Before hledger 1.50, an -- + argument was needed before addon options, but not any more.) Options Run hledger -h to see general command line help. Options can be writ- @@ -344,6 +339,10 @@ Options Usually hledger accepts any unambiguous flag prefix, eg you can write --tl instead of --tldr or --dry instead of --dry-run. + You can combine short flags which don't take arguments, eg you can + write -MAST instead of -M -A -S -T. Flags requiring an argument can't + be combined in this way (-If FILE won't work). + If the same option appears more than once in a command line, usually the last (right-most) wins. Similarly, if mutually exclusive flags are used together, the right-most wins. (When flags are mutually exclu- @@ -716,8 +715,7 @@ Options 4. To troubleshoot the effect of config files, run with --debug or --debug 8. - The config file feature was added in hledger 1.40 and is considered ex- - perimental. + The config file feature was added in hledger 1.40. Shell completions If you use the bash or zsh shells, you can optionally set up con- @@ -834,7 +832,7 @@ Output If you are using the less pager, hledger automatically appends a number of options to the LESS variable to enable ANSI colour and a number of other conveniences. (At the time of writing: --chop-long-lines - --hilite-unread --ignore-case --mouse --no-init --quit-at-eof + --hilite-unread --ignore-case --no-init --quit-at-eof --quit-if-one-screen --RAW-CONTROL-CHARS --shift=8 --squeeze-blank-lines --use-backslash ). If these don't work well, you can set your preferred options in the HLEDGER_LESS variable, which will @@ -2508,27 +2506,51 @@ Journal You can pull in the content of additional files by writing an include directive, like this: - include FILEPATH + include SOMEFILE - Only journal files can include, and only journal, timeclock or timedot - files can be included (not CSV files, currently). + This has the same effect as if SOMEFILE's content was inlined at this + point. (With any include directives in SOMEFILE processed similarly, + recursively.) - If the file path does not begin with a slash, it is relative to the - current file's folder. + Only journal files can include other files. They can include journal, + timeclock or timedot files, but not CSV files. - A tilde means home directory, eg: include ~/main.journal. + If the file path begins with a tilde, that means your home directory: + include ~/main.journal. - The path may contain glob patterns to match multiple files, eg: include - *.journal. + If it begins with a slash, it is an absolute path: include + /home/user/main.journal. Otherwise it is relative to the including + file's folder: include ../finances/main.journal. - There is limited support for recursive wildcards: **/ (the slash is re- - quired) matches 0 or more subdirectories. It's not super convenient - since you have to avoid include cycles and including directories, but - this can be done, eg: include */**/*.journal. + Also, the path may have a file type prefix to force a specific file + format, overriding the file extension(s) (as described in Data for- + mats): include timedot:notes/2023.md. - The path may also be prefixed to force a specific file format, overrid- - ing the file extension (as described in Data formats): include time- - dot:~/notes/2023*.md. + The path may contain glob patterns to match multiple files. hledger's + globs are similar to zsh's: ? to match any character; [a-z] to match + any character in a range; * to match zero or more characters that + aren't a path separator (like /); ** to match zero or more subdirecto- + ries and/or zero or more characters at the start of a file name; etc. + Also, hledger's globs always exclude the including file itself. So, + you can do + + o include *.journal to include all other journal files in the current + directory (excluding dot files) + + o include **.journal to include all other journal files in this direc- + tory and below (excluding dot directories/files) + + o include timelogs/2???.timedot to include all timedot files named like + a year number. + + There is a limitation: hledger's globs always exclude paths involving + dot files or dot directories. This is a workaround for unavoidable dot + directory traversal; you can disable it and revert to older behaviour + with the --old-glob flag, for now. + + If you are using many, or deeply nested, include files, and have an er- + ror that's hard to pinpoint: a good troubleshooting command is hledger + files --debug=6 (or 7). P directive The P directive declares a market price, which is a conversion rate be- @@ -2709,21 +2731,24 @@ Journal the duration of the report, and only when --auto is used; they are not saved in the journal file by hledger. + The postings can contain the special string %account which will be ex- + panded to the account name of the matched account. + Generated postings' amounts can depend on the matched posting's amount. - So auto postings can be useful for, eg, adding tax postings with a + So auto postings can be useful for, eg, adding tax postings with a standard percentage. AMOUNT can be: - o a number with no commodity symbol, like 2. The matched posting's + o a number with no commodity symbol, like 2. The matched posting's commodity symbol will be added to this. - o a normal amount with a commodity symbol, like $2. This will be used + o a normal amount with a commodity symbol, like $2. This will be used as-is. - o an asterisk followed by a number, like *2. This will multiply the + o an asterisk followed by a number, like *2. This will multiply the matched posting's amount (and total price, if any) by the number. - o an asterisk followed by an amount with commodity symbol, like *$2. - This multiplies and also replaces the commodity symbol with this new + o an asterisk followed by an amount with commodity symbol, like *$2. + This multiplies and also replaces the commodity symbol with this new one. Some examples: @@ -2758,38 +2783,38 @@ Journal assets:checking $20 Note that depending fully on generated data such as this has some draw- - backs - it's less portable, less future-proof, less auditable by oth- + backs - it's less portable, less future-proof, less auditable by oth- ers, and less robust (eg your balance assertions will depend on whether - you use or don't use --auto). An alternative is to use auto postings + you use or don't use --auto). An alternative is to use auto postings in "one time" fashion - use them to help build a complex journal entry, - view it with hledger print --auto, and then copy that output into the + view it with hledger print --auto, and then copy that output into the journal file to make it permanent. Auto postings and multiple files An auto posting rule can affect any transaction in the current file, or - in any parent file or child file. Note, currently it will not affect + in any parent file or child file. Note, currently it will not affect sibling files (when multiple -f/--file are used - see #1212). Auto postings and dates - A posting date (or secondary date) in the matched posting, or (taking - precedence) a posting date in the auto posting rule itself, will also + A posting date (or secondary date) in the matched posting, or (taking + precedence) a posting date in the auto posting rule itself, will also be used in the generated posting. Auto postings and transaction balancing / inferred amounts / balance asser- tions Currently, auto postings are added: - o after missing amounts are inferred, and transactions are checked for + o after missing amounts are inferred, and transactions are checked for balancedness, o but before balance assertions are checked. - Note this means that journal entries must be balanced both before and + Note this means that journal entries must be balanced both before and after auto postings are added. This changed in hledger 1.12+; see #893 for background. - This also means that you cannot have more than one auto-posting with a - missing amount applied to a given transaction, as it will be unable to + This also means that you cannot have more than one auto-posting with a + missing amount applied to a given transaction, as it will be unable to infer amounts. Auto posting tags @@ -2798,11 +2823,11 @@ Journal o generated-posting:= QUERY - shows this was generated by an auto post- ing rule, and the query - o _generated-posting:= QUERY - a hidden tag, which does not appear in + o _generated-posting:= QUERY - a hidden tag, which does not appear in hledger's output. This can be used to match postings generated "just now", rather than generated in the past and saved to the journal. - Also, any transaction that has been changed by auto posting rules will + Also, any transaction that has been changed by auto posting rules will have these tags added: o modified: - this transaction was modified @@ -2811,24 +2836,24 @@ Journal tion was modified "just now". Auto postings on forecast transactions only - Tip: you can can make auto postings that will apply to forecast trans- - actions but not recorded transactions, by adding tag:_generated-trans- - action to their QUERY. This can be useful when generating new journal + Tip: you can can make auto postings that will apply to forecast trans- + actions but not recorded transactions, by adding tag:_generated-trans- + action to their QUERY. This can be useful when generating new journal entries to be saved in the journal. Other syntax - hledger journal format supports quite a few other features, mainly to - make interoperating with or converting from Ledger easier. Note some - of the features below are powerful and can be useful in special cases, - but in general, features in this section are considered less important - or even not recommended for most users. Downsides are mentioned to + hledger journal format supports quite a few other features, mainly to + make interoperating with or converting from Ledger easier. Note some + of the features below are powerful and can be useful in special cases, + but in general, features in this section are considered less important + or even not recommended for most users. Downsides are mentioned to help you decide if you want to use them. Balance assignments - Ledger-style balance assignments are also supported. These are like - balance assertions, but with no posting amount on the left side of the - equals sign; instead it is calculated automatically so as to satisfy - the assertion. This can be a convenience during data entry, eg when + Ledger-style balance assignments are also supported. These are like + balance assertions, but with no posting amount on the left side of the + equals sign; instead it is calculated automatically so as to satisfy + the assertion. This can be a convenience during data entry, eg when setting opening balances: ; starting a new journal, set asset account balances @@ -2846,15 +2871,15 @@ Journal expenses:misc The calculated amount depends on the account's balance in the commodity - at that point (which depends on the previously-dated postings of the - commodity to that account since the last balance assertion or assign- + at that point (which depends on the previously-dated postings of the + commodity to that account since the last balance assertion or assign- ment). - Downsides: using balance assignments makes your journal less explicit; + Downsides: using balance assignments makes your journal less explicit; to know the exact amount posted, you have to run hledger or do the cal- - culations yourself, instead of just reading it. Also balance assign- + culations yourself, instead of just reading it. Also balance assign- ments' forcing of balances can hide errors. These things make your fi- - nancial data less portable, less future-proof, and less trustworthy in + nancial data less portable, less future-proof, and less trustworthy in an audit. Balance assignments and costs @@ -2869,31 +2894,31 @@ Journal (a) $1 @ 2 = $1 @ 2 Balance assignments and multiple files - Balance assignments handle multiple files like balance assertions. - They see balance from other files previously included from the current + Balance assignments handle multiple files like balance assertions. + They see balance from other files previously included from the current file, but not from previous sibling or parent files. Bracketed posting dates - For setting posting dates and secondary posting dates, Ledger's brack- + For setting posting dates and secondary posting dates, Ledger's brack- eted date syntax is also supported: [DATE], [DATE=DATE2] or [=DATE2] in - posting comments. hledger will attempt to parse any square-bracketed - sequence of the 0123456789/-.= characters in this way. With this syn- - tax, DATE infers its year from the transaction and DATE2 infers its + posting comments. hledger will attempt to parse any square-bracketed + sequence of the 0123456789/-.= characters in this way. With this syn- + tax, DATE infers its year from the transaction and DATE2 infers its year from DATE. - Downsides: another syntax to learn, redundant with hledger's + Downsides: another syntax to learn, redundant with hledger's date:/date2: tags, and confusingly similar to Ledger's lot date syntax. D directive D AMOUNT - This directive sets a default commodity, to be used for any subsequent - commodityless amounts (ie, plain numbers) seen while parsing the jour- - nal. This effect lasts until the next D directive, or the end of the + This directive sets a default commodity, to be used for any subsequent + commodityless amounts (ie, plain numbers) seen while parsing the jour- + nal. This effect lasts until the next D directive, or the end of the current file. - For compatibility/historical reasons, D also acts like a commodity di- - rective (setting the commodity's decimal mark for parsing and display + For compatibility/historical reasons, D also acts like a commodity di- + rective (setting the commodity's decimal mark for parsing and display style for output). So its argument is not just a commodity symbol, but a full amount demonstrating the style. The amount must include a deci- mal mark (either period or comma). Eg: @@ -2908,23 +2933,23 @@ Journal Interactions with other directives: - For setting a commodity's display style, a commodity directive has + For setting a commodity's display style, a commodity directive has highest priority, then a D directive. - For detecting a commodity's decimal mark during parsing, decimal-mark + For detecting a commodity's decimal mark during parsing, decimal-mark has highest priority, then commodity, then D. - For checking commodity symbols with the check command, a commodity di- + For checking commodity symbols with the check command, a commodity di- rective is required (hledger check commodities ignores D directives). - Downsides: omitting commodity symbols makes your financial data less - explicit, less portable, and less trustworthy in an audit. It is usu- - ally an unsustainable shortcut; sooner or later you will want to track - multiple commodities. D is overloaded with functions redundant with + Downsides: omitting commodity symbols makes your financial data less + explicit, less portable, and less trustworthy in an audit. It is usu- + ally an unsustainable shortcut; sooner or later you will want to track + multiple commodities. D is overloaded with functions redundant with commodity and decimal-mark. And it works differently from Ledger's D. apply account directive - This directive sets a default parent account, which will be prepended + This directive sets a default parent account, which will be prepended to all accounts in following entries, until an end apply account direc- tive or end of current file. Eg: @@ -2946,10 +2971,10 @@ Journal Account names entered via hledger add or hledger-web are not affected. - Account aliases, if any, are applied after the parent account is + Account aliases, if any, are applied after the parent account is prepended. - Downsides: this can make your financial data less explicit, less + Downsides: this can make your financial data less explicit, less portable, and less trustworthy in an audit. Y directive @@ -2959,7 +2984,7 @@ Journal year YEAR apply year YEAR - The space is optional. This sets a default year to be used for subse- + The space is optional. This sets a default year to be used for subse- quent dates which don't specify a year. Eg: Y2009 ; set default year to 2009 @@ -2980,38 +3005,38 @@ Journal Downsides: omitting the year (from primary transaction dates, at least) makes your financial data less explicit, less portable, and less trust- - worthy in an audit. Such dates can get separated from their corre- - sponding Y directive, eg when evaluating a region of the journal in - your editor. A missing Y directive makes reports dependent on today's + worthy in an audit. Such dates can get separated from their corre- + sponding Y directive, eg when evaluating a region of the journal in + your editor. A missing Y directive makes reports dependent on today's date. Secondary dates A secondary date is written after the primary date, following an equals - sign: DATE1=DATE2. If the year is omitted, the primary date's year is + sign: DATE1=DATE2. If the year is omitted, the primary date's year is assumed. When running reports, the primary (left side) date is used by default, but with the --date2 flag (--aux-date or--effective also work, - for Ledger users), the secondary (right side) date will be used in- + for Ledger users), the secondary (right side) date will be used in- stead. - The meaning of secondary dates is up to you. Eg it could be "primary - is the bank's clearing date, secondary is the date the transaction was + The meaning of secondary dates is up to you. Eg it could be "primary + is the bank's clearing date, secondary is the date the transaction was initiated, if different". In practice, this feature usually adds confusion: - o You have to remember the primary and secondary dates' meaning, and + o You have to remember the primary and secondary dates' meaning, and follow that consistently. - o It splits your bookkeeping into two modes, and you have to remember + o It splits your bookkeeping into two modes, and you have to remember which mode is appropriate for a given report. - o Usually your balance assertions will work with only one of these + o Usually your balance assertions will work with only one of these modes. - o It makes your financial data more complicated, less portable, and + o It makes your financial data more complicated, less portable, and less clear in an audit. - o It interacts with every feature, creating an ongoing cost for imple- + o It interacts with every feature, creating an ongoing cost for imple- mentors. o It distracts new users and supporters. @@ -3019,38 +3044,38 @@ Journal o Posting dates are simpler and work better. So secondary dates are officially deprecated in hledger, remaining only - as a Ledger compatibility aid; we recommend using posting dates in- + as a Ledger compatibility aid; we recommend using posting dates in- stead. Star comments - Lines beginning with * (star/asterisk) are also comment lines. This + Lines beginning with * (star/asterisk) are also comment lines. This feature allows Emacs users to insert org headings in their journal, al- lowing them to fold/unfold/navigate it like an outline when viewed with org mode. - Downsides: another, unconventional comment syntax to learn. Decreases - your journal's portability. And switching to Emacs org mode just for - folding/unfolding meant losing the benefits of ledger mode; nowadays - you can add outshine mode to ledger mode to get folding without losing + Downsides: another, unconventional comment syntax to learn. Decreases + your journal's portability. And switching to Emacs org mode just for + folding/unfolding meant losing the benefits of ledger mode; nowadays + you can add outshine mode to ledger mode to get folding without losing ledger mode's features. Valuation expressions - Ledger allows a valuation function or value to be written in double + Ledger allows a valuation function or value to be written in double parentheses after an amount. hledger ignores these. Virtual postings A posting with parentheses around the account name, like (some:account) - 10, is called an unbalanced virtual posting. These postings do not - participate in transaction balancing. (And if you write them without - an amount, a zero amount is always inferred.) These can occasionally - be convenient for special circumstances, but they violate double entry - bookkeeping and make your data less portable across applications, so + 10, is called an unbalanced virtual posting. These postings do not + participate in transaction balancing. (And if you write them without + an amount, a zero amount is always inferred.) These can occasionally + be convenient for special circumstances, but they violate double entry + bookkeeping and make your data less portable across applications, so many people avoid using them at all. - A posting with brackets around the account name ([some:account]) is - called a balanced virtual posting. The balanced virtual postings in a + A posting with brackets around the account name ([some:account]) is + called a balanced virtual posting. The balanced virtual postings in a transaction must add up to zero, just like ordinary postings, but sepa- - rately from them. These are not part of double entry bookkeeping ei- + rately from them. These are not part of double entry bookkeeping ei- ther, but they are at least balanced. An example: 2022-01-01 buy food with cash, update budget envelope subaccounts, & something else @@ -3061,13 +3086,13 @@ Journal [assets:checking:available] $10 ; <- (something:else) $5 ; <- this is not required to balance - Ordinary postings, whose account names are neither parenthesised nor - bracketed, are called real postings. You can exclude virtual postings + Ordinary postings, whose account names are neither parenthesised nor + bracketed, are called real postings. You can exclude virtual postings from reports with the -R/--real flag or a real:1 query. Other Ledger directives These other Ledger directives are currently accepted but ignored. This - allows hledger to read more Ledger files, but be aware that hledger's + allows hledger to read more Ledger files, but be aware that hledger's reports may differ from Ledger's if you use these. apply fixed COMM AMT @@ -3088,7 +3113,7 @@ Journal value EXPR --command-line-flags - See also https://hledger.org/ledger.html for a detailed hledger/Ledger + See also https://hledger.org/ledger.html for a detailed hledger/Ledger syntax comparison. Other cost/lot notations @@ -3100,12 +3125,12 @@ Journal o expresses a conversion rate, as in hledger - o when buying, also creates a lot that can be selected at selling + o when buying, also creates a lot that can be selected at selling time o (@) UNITCOST and (@@) TOTALCOST (virtual cost) - o like the above, but also means "this cost was exceptional, don't + o like the above, but also means "this cost was exceptional, don't use it when inferring market prices". o {=UNITCOST} and {{=TOTALCOST}} (fixed price) @@ -3115,10 +3140,10 @@ Journal o {UNITCOST} and {{TOTALCOST}} (lot price) - o can be used identically to @ UNITCOST and @@ TOTALCOST, also cre- + o can be used identically to @ UNITCOST and @@ TOTALCOST, also cre- ates a lot - o when selling, combined with @ ..., selects an existing lot by its + o when selling, combined with @ ..., selects an existing lot by its cost basis. Does not check if that lot is present. o [YYYY/MM/DD] (lot date) @@ -3149,8 +3174,8 @@ Journal o expresses a cost without creating a lot, as in hledger - o when buying (acquiring) or selling (disposing of) a lot, and com- - bined with {...}: is not used except to document the cost/selling + o when buying (acquiring) or selling (disposing of) a lot, and com- + bined with {...}: is not used except to document the cost/selling price o {UNITCOST} and {{TOTALCOST}} @@ -3167,10 +3192,10 @@ Journal o expresses the selling price for transaction balancing - o {}, {YYYY-MM-DD}, {"LABEL"}, {UNITCOST, "LABEL"}, {UNITCOST, + o {}, {YYYY-MM-DD}, {"LABEL"}, {UNITCOST, "LABEL"}, {UNITCOST, YYYY-MM-DD, "LABEL"} - o when selling, other combinations of date/cost/label, like the + o when selling, other combinations of date/cost/label, like the above, are accepted for selecting the lot. Currently, hledger @@ -3182,28 +3207,28 @@ Journal o and rejects the rest. CSV - hledger can read transactions from CSV (comma-separated values) files. - More precisely, it can read DSV (delimiter-separated values), from a + hledger can read transactions from CSV (comma-separated values) files. + More precisely, it can read DSV (delimiter-separated values), from a file or standard input. Comma-separated, semicolon-separated and - tab-separated are the most common variants, and hledger will recognise - these three automatically based on a .csv, .ssv or .tsv file name ex- + tab-separated are the most common variants, and hledger will recognise + these three automatically based on a .csv, .ssv or .tsv file name ex- tension or a csv:, ssv: or tsv: file path prefix. (To learn about producing CSV or TSV output, see Output format.) - Each CSV file must be described by a corresponding rules file. This - contains rules describing the CSV data (header line, fields layout, - date format etc.), how to construct hledger transactions from it, and - how to categorise transactions based on description or other attrib- + Each CSV file must be described by a corresponding rules file. This + contains rules describing the CSV data (header line, fields layout, + date format etc.), how to construct hledger transactions from it, and + how to categorise transactions based on description or other attrib- utes. - By default, hledger expects this rules file to be named like the CSV - file, with an extra .rules extension added, in the same directory. Eg - when asked to read foo/FILE.csv, hledger looks for foo/FILE.csv.rules. + By default, hledger expects this rules file to be named like the CSV + file, with an extra .rules extension added, in the same directory. Eg + when asked to read foo/FILE.csv, hledger looks for foo/FILE.csv.rules. You can specify a different rules file with the --rules option. - At minimum, the rules file must identify the date and amount fields, - and often it also specifies the date format and how many header lines + At minimum, the rules file must identify the date and amount fields, + and often it also specifies the date format and how many header lines there are. Here's a simple CSV file and a rules file for it: Date, Description, Id, Amount @@ -3219,70 +3244,102 @@ CSV expenses:unknown 10.23 income:unknown -10.23 - There's an introductory Tutorial: Import CSV data on hledger.org, and - more CSV rules examples below, and a larger collection at + There's an introductory Tutorial: Import CSV data on hledger.org, and + more CSV rules examples below, and a larger collection at https://github.com/simonmichael/hledger/tree/master/examples/csv. CSV rules cheatsheet The following kinds of rule can appear in the rules file, in any order. (Blank lines and lines beginning with # or ; or * are ignored.) - source optionally declare which file to read data + source optionally declare which file to read data from - encoding optionally declare which text encoding the + encoding optionally declare which text encoding the data has - separator declare the field separator, instead of rely- + separator declare the field separator, instead of rely- ing on file extension skip skip one or more header lines at start of file date-format declare how to parse CSV dates/date-times - timezone declare the time zone of ambiguous CSV + timezone declare the time zone of ambiguous CSV date-times - newest-first improve txn order when: there are multiple + newest-first improve txn order when: there are multiple records, newest first, all with the same date - intra-day-reversed improve txn order when: same-day txns are in + intra-day-reversed improve txn order when: same-day txns are in opposite order to the overall file - decimal-mark declare the decimal mark used in CSV amounts, + decimal-mark declare the decimal mark used in CSV amounts, when ambiguous - fields list name CSV fields for easy reference, and op- + fields list name CSV fields for easy reference, and op- tionally assign their values to hledger fields - Field assignment assign a CSV value or interpolated text value + Field assignment assign a CSV value or interpolated text value to a hledger field if block conditionally assign values to hledger fields, or skip a record or end (skip rest of file) if table conditionally assign values to hledger fields, using compact syntax - balance-type select which type of balance assertions/as- + balance-type select which type of balance assertions/as- signments to generate include inline another CSV rules file - Working with CSV tips can be found below, including How CSV rules are + Working with CSV tips can be found below, including How CSV rules are evaluated. source - If you tell hledger to read a csv file with -f foo.csv, it will look - for rules in foo.csv.rules. Or, you can tell it to read the rules - file, with -f foo.csv.rules, and it will look for data in foo.csv - (since 1.30). + If you tell hledger to read a csv file with -f foo.csv, it will look + for rules in foo.csv.rules. Or, you can tell it to read the rules + file, with -f foo.csv.rules, and it will look for data in foo.csv + (since 1.30). These are mostly equivalent, but the second method pro- + vides some extra features. For one, the data file can be missing, + without causing an error; it is just considered empty. - These are mostly equivalent, but the second method provides some extra - features. For one, the data file can be missing, without causing an - error; it is just considered empty. And, you can specify a different - data file by adding a "source" rule: + For more flexibility, add a source rule, which lets you specify a dif- + ferent data file: source ./Checking1.csv + If the file does not exist, it is just considered empty, without rais- + ing an error. + If you specify just a file name with no path, hledger will look for it - in your system's downloads directory (~/Downloads, currently): + in the ~/Downloads folder: source Checking1.csv - And if you specify a glob pattern, hledger will read the most recent of - the matched files (useful with repeated downloads): + You can use a glob pattern, to avoid specifying the file name exactly: source Checking1*.csv + This has another benefit: if the pattern matches multiple files, + hledger will read the newest (most recently modified) one. This avoids + problems if you have downloaded a file multiple times without cleaning + up. + + All this enables a convenient workflow where can you just download CSV + files, then run hledger import rules/*. + See also "Working with CSV > Reading files specified by rule". + The archive rule adds a few more features to source; see below. + + archive + Adding the archive rule to your rules file affects importing or reading + files specified by source: + + o After successfully importing, import will move the data file to an + archive directory (data/ next to the rules file, auto-created), re- + named to RULESFILEBASENAME.DATAFILEMODDATE.DATAFILEEXT. Archiving + data files is optional, but it can be useful for troubleshooting, de- + tecting variations in your banks' CSV data, regenerating entries with + improved rules, etc. + + o import will pick the oldest of source glob matches, rather than the + newest. So if you have multiple versions of a download, repeated im- + ports will process them in chronological order. + + o For commands other than import, when the source path or glob pattern + matches no files, hledger will try to read the latest archived data + file instead. This is convenient for working with the downloaded + data again, even after it has been imported. + encoding encoding ENCODING @@ -3362,36 +3419,39 @@ CSV # Note the time and junk must be fully parsed, though only the date is used. date-format %-m/%-d/%Y %l:%M %p some other junk + Note currently there is no locale awareness for things like %b, and + setting LC_TIME won't help. + timezone timezone TIMEZONE - When CSV contains date-times that are implicitly in some time zone + When CSV contains date-times that are implicitly in some time zone other than yours, but containing no explicit time zone information, you - can use this rule to declare the CSV's native time zone, which helps + can use this rule to declare the CSV's native time zone, which helps prevent off-by-one dates. - When the CSV date-times do contain time zone information, you don't - need this rule; instead, use %Z in date-format (or %z, %EZ, %Ez; see + When the CSV date-times do contain time zone information, you don't + need this rule; instead, use %Z in date-format (or %z, %EZ, %Ez; see the formatTime link above). In either of these cases, hledger will do a time-zone-aware conversion, localising the CSV date-times to your current system time zone. If you prefer to localise to some other time zone, eg for reproducibility, you - can (on unix at least) set the output timezone with the TZ environment + can (on unix at least) set the output timezone with the TZ environment variable, eg: $ TZ=-1000 hledger print -f foo.csv # or TZ=-1000 hledger import foo.csv - timezone currently does not understand timezone names, except "UTC", - "GMT", "EST", "EDT", "CST", "CDT", "MST", "MDT", "PST", or "PDT". For + timezone currently does not understand timezone names, except "UTC", + "GMT", "EST", "EDT", "CST", "CDT", "MST", "MDT", "PST", or "PDT". For others, use numeric format: +HHMM or -HHMM. newest-first hledger tries to ensure that the generated transactions will be ordered chronologically, including same-day transactions. Usually it can - auto-detect how the CSV records are ordered. But if it encounters CSV + auto-detect how the CSV records are ordered. But if it encounters CSV where all records are on the same date, it assumes that the records are - oldest first. If in fact the CSV's records are normally newest first, + oldest first. If in fact the CSV's records are normally newest first, like: 2022-10-01, txn 3... @@ -3405,9 +3465,9 @@ CSV newest-first intra-day-reversed - If CSV records within a single day are ordered opposite to the overall - record order, you can add the intra-day-reversed rule to improve the - order of journal entries. Eg, here the overall record order is newest + If CSV records within a single day are ordered opposite to the overall + record order, you can add the intra-day-reversed rule to improve the + order of journal entries. Eg, here the overall record order is newest first, but same-day records are oldest first: 2022-10-02, txn 3... @@ -3425,10 +3485,10 @@ CSV decimal-mark , - hledger automatically accepts either period or comma as a decimal mark - when parsing numbers (cf Amounts). However if any numbers in the CSV - contain digit group marks, such as thousand-separating commas, you - should declare the decimal mark explicitly with this rule, to avoid + hledger automatically accepts either period or comma as a decimal mark + when parsing numbers (cf Amounts). However if any numbers in the CSV + contain digit group marks, such as thousand-separating commas, you + should declare the decimal mark explicitly with this rule, to avoid misparsed numbers. fields list @@ -3437,17 +3497,17 @@ CSV A fields list (the word fields followed by comma-separated field names) is optional, but convenient. It does two things: - 1. It names the CSV field in each column. This can be convenient if - you are referencing them in other rules, so you can say %SomeField + 1. It names the CSV field in each column. This can be convenient if + you are referencing them in other rules, so you can say %SomeField instead of remembering %13. - 2. Whenever you use one of the special hledger field names (described - below), it assigns the CSV value in this position to that hledger - field. This is the quickest way to populate hledger's fields and + 2. Whenever you use one of the special hledger field names (described + below), it assigns the CSV value in this position to that hledger + field. This is the quickest way to populate hledger's fields and build a transaction. - Here's an example that says "use the 1st, 2nd and 4th fields as the - transaction's date, description and amount; name the last two fields + Here's an example that says "use the 1st, 2nd and 4th fields as the + transaction's date, description and amount; name the last two fields for later reference; and ignore the others": fields date, description, , amount, , , somefield, anotherfield @@ -3457,35 +3517,35 @@ CSV o There must be least two items in the list (at least one comma). - o Field names may not contain spaces. Spaces before/after field names + o Field names may not contain spaces. Spaces before/after field names are optional. o Field names may contain _ (underscore) or - (hyphen). - o Fields you don't care about can be given a dummy name or an empty + o Fields you don't care about can be given a dummy name or an empty name. - If the CSV contains column headings, it's convenient to use these for - your field names, suitably modified (eg lower-cased with spaces re- + If the CSV contains column headings, it's convenient to use these for + your field names, suitably modified (eg lower-cased with spaces re- placed by underscores). - Sometimes you may want to alter a CSV field name to avoid assigning to - a hledger field with the same name. Eg you could call the CSV's "bal- - ance" field balance_ to avoid directly setting hledger's balance field + Sometimes you may want to alter a CSV field name to avoid assigning to + a hledger field with the same name. Eg you could call the CSV's "bal- + ance" field balance_ to avoid directly setting hledger's balance field (and generating a balance assertion). Field assignment HLEDGERFIELD FIELDVALUE - Field assignments are the more flexible way to assign CSV values to + Field assignments are the more flexible way to assign CSV values to hledger fields. They can be used instead of or in addition to a fields list (see above). - To assign a value to a hledger field, write the field name (any of the - standard hledger field/pseudo-field names, defined below), a space, - followed by a text value on the same line. This text value may inter- - polate CSV fields, referenced either by their 1-based position in the - CSV record (%N) or by the name they were given in the fields list + To assign a value to a hledger field, write the field name (any of the + standard hledger field/pseudo-field names, defined below), a space, + followed by a text value on the same line. This text value may inter- + polate CSV fields, referenced either by their 1-based position in the + CSV record (%N) or by the name they were given in the fields list (%CSVFIELD), and regular expression match groups (\N). Some examples: @@ -3498,26 +3558,26 @@ CSV Tips: - o Interpolation strips outer whitespace (so a CSV value like " 1 " be- + o Interpolation strips outer whitespace (so a CSV value like " 1 " be- comes 1 when interpolated) (#1051). - o Interpolations always refer to a CSV field - you can't interpolate a + o Interpolations always refer to a CSV field - you can't interpolate a hledger field. (See Referencing other fields below). Field names - Note the two kinds of field names mentioned here, and used only in + Note the two kinds of field names mentioned here, and used only in hledger CSV rules files: - 1. CSV field names (CSVFIELD in these docs): you can optionally name - the CSV columns for easy reference (since hledger doesn't yet auto- + 1. CSV field names (CSVFIELD in these docs): you can optionally name + the CSV columns for easy reference (since hledger doesn't yet auto- matically recognise column headings in a CSV file), by writing arbi- trary names in a fields list, eg: fields When, What, Some_Id, Net, Total, Foo, Bar - 2. Special hledger field names (HLEDGERFIELD in these docs): you must - set at least some of these to generate the hledger transaction from - a CSV record, by writing them as the left hand side of a field as- + 2. Special hledger field names (HLEDGERFIELD in these docs): you must + set at least some of these to generate the hledger transaction from + a CSV record, by writing them as the left hand side of a field as- signment, eg: date %When @@ -3532,7 +3592,7 @@ CSV currency $ comment %Foo %Bar - Here are all the special hledger field names available, and what hap- + Here are all the special hledger field names available, and what hap- pens when you assign values to them: date field @@ -3555,7 +3615,7 @@ CSV commentN, where N is a number, sets the Nth posting's comment. - You can assign multi-line comments by writing literal \n in the code. + You can assign multi-line comments by writing literal \n in the code. A comment starting with \n will begin on a new line. Comments can contain tags, as usual. @@ -3567,99 +3627,99 @@ CSV Assigning to accountN, where N is 1 to 99, sets the account name of the Nth posting, and causes that posting to be generated. - Most often there are two postings, so you'll want to set account1 and - account2. Typically account1 is associated with the CSV file, and is - set once with a top-level assignment, while account2 is set based on + Most often there are two postings, so you'll want to set account1 and + account2. Typically account1 is associated with the CSV file, and is + set once with a top-level assignment, while account2 is set based on each transaction's description, in conditional rules. - If a posting's account name is left unset but its amount is set (see - below), a default account name will be chosen (like "expenses:unknown" + If a posting's account name is left unset but its amount is set (see + below), a default account name will be chosen (like "expenses:unknown" or "income:unknown"). amount field - There are several ways to set posting amounts from CSV, useful in dif- + There are several ways to set posting amounts from CSV, useful in dif- ferent situations. - 1. amount is the oldest and simplest. Assigning to this sets the + 1. amount is the oldest and simplest. Assigning to this sets the amount of the first and second postings. In the second posting, the - amount will be negated; also, if it has a cost attached, it will be + amount will be negated; also, if it has a cost attached, it will be converted to cost. - 2. amount-in and amount-out work exactly like the above, but should be - used when the CSV has two amount fields (such as "Debit" and + 2. amount-in and amount-out work exactly like the above, but should be + used when the CSV has two amount fields (such as "Debit" and "Credit", or "Inflow" and "Outflow"). Whichever field has a - non-zero value will be used as the amount of the first and second + non-zero value will be used as the amount of the first and second postings. Here are some tips to avoid confusion: - o It's not "amount-in for posting 1 and amount-out for posting 2", - it is "extract a single amount from the amount-in or amount-out + o It's not "amount-in for posting 1 and amount-out for posting 2", + it is "extract a single amount from the amount-in or amount-out field, and use that for posting 1 and (negated) for posting 2". - o Don't use both amount and amount-in/amount-out in the same rules + o Don't use both amount and amount-in/amount-out in the same rules file; choose based on whether the amount is in a single CSV field or spread across two fields. - o In each record, at most one of the two CSV fields should contain - a non-zero amount; the other field must contain a zero or noth- + o In each record, at most one of the two CSV fields should contain + a non-zero amount; the other field must contain a zero or noth- ing. - o hledger assumes both CSV fields contain unsigned numbers, and it + o hledger assumes both CSV fields contain unsigned numbers, and it automatically negates the amount-out values. - o If the data doesn't fit these requirements, you'll probably need + o If the data doesn't fit these requirements, you'll probably need an if rule (see below). 3. amountN (where N is a number from 1 to 99) sets the amount of only a - single posting: the Nth posting in the transaction. You'll usually - need at least two such assignments to make a balanced transaction. + single posting: the Nth posting in the transaction. You'll usually + need at least two such assignments to make a balanced transaction. You can also generate more than two postings, to represent more com- - plex transactions. The posting numbers don't have to be consecu- - tive; with if rules, higher posting numbers can be useful to ensure + plex transactions. The posting numbers don't have to be consecu- + tive; with if rules, higher posting numbers can be useful to ensure a certain order of postings. - 4. amountN-in and amountN-out work exactly like the above, but should - be used when the CSV has two amount fields. This is analogous to + 4. amountN-in and amountN-out work exactly like the above, but should + be used when the CSV has two amount fields. This is analogous to amount-in and amount-out, and those tips also apply here. 5. Remember that a fields list can also do assignments. So in a fields - list if you name a CSV field "amount", that counts as assigning to - amount. (If you don't want that, call it something else in the + list if you name a CSV field "amount", that counts as assigning to + amount. (If you don't want that, call it something else in the fields list, like "amount_".) - 6. The above don't handle every situation; if you need more flexibil- + 6. The above don't handle every situation; if you need more flexibil- ity, use an if rule to set amounts conditionally. See "Working with - CSV > Setting amounts" below for more on this and on amount-setting + CSV > Setting amounts" below for more on this and on amount-setting generally. currency field - currency sets a currency symbol, to be prepended to all postings' - amounts. You can use this if the CSV amounts do not have a currency + currency sets a currency symbol, to be prepended to all postings' + amounts. You can use this if the CSV amounts do not have a currency symbol, eg if it is in a separate column. currencyN prepends a currency symbol to just the Nth posting's amount. balance field - balanceN sets a balance assertion amount (or if the posting amount is + balanceN sets a balance assertion amount (or if the posting amount is left empty, a balance assignment) on posting N. balance is a compatibility spelling for hledger <1.17; it is equivalent to balance1. - You can adjust the type of assertion/assignment with the balance-type + You can adjust the type of assertion/assignment with the balance-type rule (see below). - See the Working with CSV tips below for more about setting amounts and + See the Working with CSV tips below for more about setting amounts and currency. if block - Rules can be applied conditionally, depending on patterns in the CSV - data. This allows flexibility; in particular, it is how you can cate- - gorise transactions, selecting an appropriate account name based on - their description (for example). There are two ways to write condi- - tional rules: "if blocks", described here, and "if tables", described + Rules can be applied conditionally, depending on patterns in the CSV + data. This allows flexibility; in particular, it is how you can cate- + gorise transactions, selecting an appropriate account name based on + their description (for example). There are two ways to write condi- + tional rules: "if blocks", described here, and "if tables", described below. - An if block is the word if and one or more "matcher" expressions (can + An if block is the word if and one or more "matcher" expressions (can be a word or phrase), one per line, starting either on the same or next line; followed by one or more indented rules. Eg, @@ -3675,11 +3735,11 @@ CSV RULE RULE - If any of the matchers succeeds, all of the indented rules will be ap- - plied. They are usually field assignments, but the following special + If any of the matchers succeeds, all of the indented rules will be ap- + plied. They are usually field assignments, but the following special rules may also be used within an if block: - o skip - skips the matched CSV record (generating no transaction from + o skip - skips the matched CSV record (generating no transaction from it) o end - skips the rest of the current CSV file. @@ -3705,39 +3765,39 @@ CSV Matchers There are two kinds of matcher: - 1. A whole record matcher is simplest: it is just a word, single-line - text fragment, or other regular expression, which hledger will try + 1. A whole record matcher is simplest: it is just a word, single-line + text fragment, or other regular expression, which hledger will try to match case-insensitively anywhere within the CSV record. Eg: whole foods. - 2. A field matcher has a percent-prefixed CSV field number or name be- + 2. A field matcher has a percent-prefixed CSV field number or name be- fore the pattern. Eg: %3 whole foods or %description whole foods. hledger will try to match the pattern just within the named CSV field. When using these, there's two things to be aware of: - 1. Whole record matchers don't see the exact original record; they see - a reconstruction of it, in which values are comma-separated, and - quotes enclosing values and whitespace outside those quotes are re- + 1. Whole record matchers don't see the exact original record; they see + a reconstruction of it, in which values are comma-separated, and + quotes enclosing values and whitespace outside those quotes are re- moved. Eg when reading an SSV record like: 2023-01-01 ; "Acme, Inc. " ; 1,000 the whole record matcher sees instead: 2023-01-01,Acme, Inc. ,1,000 2. Field matchers expect either a CSV field number, or a CSV field name - declared with fields. (Don't use a hledger field name here, unless - it is also a CSV field name.) A non-CSV field name will cause the - matcher to match against "" (the empty string), and does not raise - an error, allowing easier reuse of common rules with different CSV + declared with fields. (Don't use a hledger field name here, unless + it is also a CSV field name.) A non-CSV field name will cause the + matcher to match against "" (the empty string), and does not raise + an error, allowing easier reuse of common rules with different CSV files. You can also prefix a matcher with ! (and optional space) to negate it. - Eg ! whole foods, ! %3 whole foods, !%description whole foods will + Eg ! whole foods, ! %3 whole foods, !%description whole foods will match if "whole foods" is NOT present. Added in 1.32. - The pattern is, as usual in hledger, a POSIX extended regular expres- - sion that also supports GNU word boundaries (\b, \B, \<, \>) and noth- - ing else. If you have trouble with it, see "Regular expressions" in + The pattern is, as usual in hledger, a POSIX extended regular expres- + sion that also supports GNU word boundaries (\b, \B, \<, \>) and noth- + ing else. If you have trouble with it, see "Regular expressions" in the hledger manual (https://hledger.org/hledger.html#regular-expres- sions). @@ -3746,28 +3806,28 @@ CSV o By default they are OR'd (any of them can match). - o Matcher lines beginning with & (or &&, since 1.42) are AND'ed with + o Matcher lines beginning with & (or &&, since 1.42) are AND'ed with the matcher above (all in the AND'ed group must match). - o Matcher lines beginning with & ! (since 1.41, or && !, since 1.42) + o Matcher lines beginning with & ! (since 1.41, or && !, since 1.42) are first negated and then AND'ed with the matcher above. - You can also combine multiple matchers one the same line separated by + You can also combine multiple matchers one the same line separated by && (AND) or && ! (AND NOT). Eg %description amazon && %date 2025-01-01 - will match only when the description field contains "amazon" and the + will match only when the description field contains "amazon" and the date field contains "2025-01-01". Added in 1.42. Match groups Added in 1.32 Matchers can define match groups: parenthesised portions of the regular - expression which are available for reference in field assignments. + expression which are available for reference in field assignments. Groups are enclosed in regular parentheses (( and )) and can be nested. - Each group is available in field assignments using the token \N, where - N is an index into the match groups for this conditional block (e.g. + Each group is available in field assignments using the token \N, where + N is an index into the match groups for this conditional block (e.g. \1, \2, etc.). - Example: Warp credit card payment postings to the beginning of the + Example: Warp credit card payment postings to the beginning of the billing period (Month start), to match how they are presented in state- ments, using posting dates: @@ -3781,8 +3841,8 @@ CSV account1 \1 if table - "if tables" are an alternative to if blocks; they can express many - matchers and field assignments in a more compact tabular format, like + "if tables" are an alternative to if blocks; they can express many + matchers and field assignments in a more compact tabular format, like this: if,HLEDGERFIELD1,HLEDGERFIELD2,... @@ -3793,21 +3853,21 @@ CSV The first character after if is taken to be this if table's field sepa- - rator. It is unrelated to the separator used in the CSV file. It + rator. It is unrelated to the separator used in the CSV file. It should be a non-alphanumeric character like , or | that does not appear - anywhere else in the table (it should not be used in field names or + anywhere else in the table (it should not be used in field names or matchers or values, and it cannot be escaped with a backslash). - Each line must contain the same number of separators; empty values are - allowed. Whitespace can be used in the matcher lines for readability - (but not in the if line, currently). You can use the comment lines in - the table body. The table must be terminated by an empty line (or end + Each line must contain the same number of separators; empty values are + allowed. Whitespace can be used in the matcher lines for readability + (but not in the if line, currently). You can use the comment lines in + the table body. The table must be terminated by an empty line (or end of file). - An if table like the above is interpreted as follows: try all of the + An if table like the above is interpreted as follows: try all of the lines with matchers; whenever a line with matchers succeeds, assign all of the values on that line to the corresponding hledger fields; If mul- - tiple lines match, later lines will override fields assigned by the + tiple lines match, later lines will override fields assigned by the earlier ones - just like the sequence of if blocks would behave. If table presented above is equivalent to this sequence of if blocks: @@ -3838,10 +3898,10 @@ CSV balance-type Balance assertions generated by assigning to balanceN are of the simple - = type by default, which is a single-commodity, subaccount-excluding + = type by default, which is a single-commodity, subaccount-excluding assertion. You may find the subaccount-including variants more useful, - eg if you have created some virtual subaccounts of checking to help - with budgeting. You can select a different type of assertion with the + eg if you have created some virtual subaccounts of checking to help + with budgeting. You can select a different type of assertion with the balance-type rule: # balance assertions will consider all commodities and all subaccounts @@ -3857,9 +3917,9 @@ CSV include include RULESFILE - This includes the contents of another CSV rules file at this point. - RULESFILE is an absolute file path or a path relative to the current - file's directory. This can be useful for sharing common rules between + This includes the contents of another CSV rules file at this point. + RULESFILE is an absolute file path or a path relative to the current + file's directory. This can be useful for sharing common rules between several rules files, eg: # someaccount.csv.rules @@ -3876,42 +3936,42 @@ CSV Some tips: Rapid feedback - It's a good idea to get rapid feedback while creating/troubleshooting + It's a good idea to get rapid feedback while creating/troubleshooting CSV rules. Here's a good way, using entr from eradman.com/entrproject: $ ls foo.csv* | entr bash -c 'echo ----; hledger -f foo.csv print desc:SOMEDESC' - A desc: query (eg) is used to select just one, or a few, transactions - of interest. "bash -c" is used to run multiple commands, so we can - echo a separator each time the command re-runs, making it easier to + A desc: query (eg) is used to select just one, or a few, transactions + of interest. "bash -c" is used to run multiple commands, so we can + echo a separator each time the command re-runs, making it easier to read the output. Valid CSV - Note that hledger will only accept valid CSV conforming to RFC 4180, + Note that hledger will only accept valid CSV conforming to RFC 4180, and equivalent SSV and TSV formats (like RFC 4180 but with semicolon or tab as separators). This means, eg: o Values may be enclosed in double quotes, or not. Enclosing in single quotes is not allowed. (Eg 'A','B' is rejected.) - o When values are enclosed in double quotes, spaces outside the quotes + o When values are enclosed in double quotes, spaces outside the quotes are not allowed. (Eg "A", "B" is rejected.) - o When values are not enclosed in quotes, they may not contain double + o When values are not enclosed in quotes, they may not contain double quotes. (Eg A"A, B is rejected.) - If your CSV/SSV/TSV is not valid in this sense, you'll need to trans- - form it before reading with hledger. Try using sed, or a more permis- + If your CSV/SSV/TSV is not valid in this sense, you'll need to trans- + form it before reading with hledger. Try using sed, or a more permis- sive CSV parser like python's csv lib. File Extension - To help hledger choose the CSV file reader and show the right error - messages (and choose the right field separator character by default), - it's best if CSV/SSV/TSV files are named with a .csv, .ssv or .tsv + To help hledger choose the CSV file reader and show the right error + messages (and choose the right field separator character by default), + it's best if CSV/SSV/TSV files are named with a .csv, .ssv or .tsv filename extension. (More about this at Data formats.) - When reading files with the "wrong" extension, you can ensure the CSV - reader (and the default field separator) by prefixing the file path + When reading files with the "wrong" extension, you can ensure the CSV + reader (and the default field separator) by prefixing the file path with csv:, ssv: or tsv:: Eg: $ hledger -f ssv:foo.dat print @@ -3920,29 +3980,29 @@ CSV if needed. Reading CSV from standard input - You'll need the file format prefix when reading CSV from stdin also, + You'll need the file format prefix when reading CSV from stdin also, since hledger assumes journal format by default. Eg: $ cat foo.dat | hledger -f ssv:- print Reading multiple CSV files - If you use multiple -f options to read multiple CSV files at once, - hledger will look for a correspondingly-named rules file for each CSV - file. But if you specify a rules file with --rules, that rules file + If you use multiple -f options to read multiple CSV files at once, + hledger will look for a correspondingly-named rules file for each CSV + file. But if you specify a rules file with --rules, that rules file will be used for all the CSV files. Reading files specified by rule Instead of specifying a CSV file in the command line, you can specify a - rules file, as in hledger -f foo.csv.rules CMD. By default this will - read data from foo.csv in the same directory, but you can add a source - rule to specify a different data file, perhaps located in your web + rules file, as in hledger -f foo.csv.rules CMD. By default this will + read data from foo.csv in the same directory, but you can add a source + rule to specify a different data file, perhaps located in your web browser's download directory. This feature was added in hledger 1.30, so you won't see it in most CSV - rules examples. But it helps remove some of the busywork of managing + rules examples. But it helps remove some of the busywork of managing CSV downloads. Most of your financial institutions's default CSV file- - names are different and can be recognised by a glob pattern. So you - can put a rule like source Checking1*.csv in foo-checking.csv.rules, + names are different and can be recognised by a glob pattern. So you + can put a rule like source Checking1*.csv in foo-checking.csv.rules, and then periodically follow a workflow like: 1. Download CSV from Foo's website, using your browser's defaults @@ -3950,45 +4010,45 @@ CSV 2. Run hledger import foo-checking.csv.rules to import any new transac- tions - After import, you can: discard the CSV, or leave it where it is for a - while, or move it into your archives, as you prefer. If you do noth- - ing, next time your browser will save something like Checking1-2.csv, - and hledger will use that because of the * wild card and because it is + After import, you can: discard the CSV, or leave it where it is for a + while, or move it into your archives, as you prefer. If you do noth- + ing, next time your browser will save something like Checking1-2.csv, + and hledger will use that because of the * wild card and because it is the most recent. Valid transactions After reading a CSV file, hledger post-processes and validates the gen- erated journal entries as it would for a journal file - balancing them, - applying balance assignments, and canonicalising amount styles. Any - errors at this stage will be reported in the usual way, displaying the + applying balance assignments, and canonicalising amount styles. Any + errors at this stage will be reported in the usual way, displaying the problem entry. There is one exception: balance assertions, if you have generated them, - will not be checked, since normally these will work only when the CSV - data is part of the main journal. If you do need to check balance as- + will not be checked, since normally these will work only when the CSV + data is part of the main journal. If you do need to check balance as- sertions generated from CSV right away, pipe into another hledger: $ hledger -f file.csv print | hledger -f- print Deduplicating, importing - When you download a CSV file periodically, eg to get your latest bank - transactions, the new file may overlap with the old one, containing + When you download a CSV file periodically, eg to get your latest bank + transactions, the new file may overlap with the old one, containing some of the same records. The import command will (a) detect the new transactions, and (b) append just those transactions to your main journal. It is idempotent, so you - don't have to remember how many times you ran it or with which version - of the CSV. (It keeps state in a hidden .latest.FILE.csv file.) This + don't have to remember how many times you ran it or with which version + of the CSV. (It keeps state in a hidden .latest.FILE.csv file.) This is the easiest way to import CSV data. Eg: # download the latest CSV files, then run this command. # Note, no -f flags needed here. $ hledger import *.csv [--dry] - This method works for most CSV files. (Where records have a stable + This method works for most CSV files. (Where records have a stable chronological order, and new records appear only at the new end.) - A number of other tools and workflows, hledger-specific and otherwise, + A number of other tools and workflows, hledger-specific and otherwise, exist for converting, deduplicating, classifying and managing CSV data. See: @@ -3997,16 +4057,16 @@ CSV o https://plaintextaccounting.org -> data import/conversion Setting amounts - Continuing from amount field above, here are more tips for amount-set- + Continuing from amount field above, here are more tips for amount-set- ting: 1. If the amount is in a single CSV field: a. If its sign indicates direction of flow: - Assign it to amountN, to set the Nth posting's amount. N is usu- + Assign it to amountN, to set the Nth posting's amount. N is usu- ally 1 or 2 but can go up to 99. b. If another field indicates direction of flow: - Use one or more conditional rules to set the appropriate amount + Use one or more conditional rules to set the appropriate amount sign. Eg: # assume a withdrawal unless Type contains "deposit": @@ -4014,15 +4074,15 @@ CSV if %Type deposit amount1 %Amount - 2. If the amount is in two CSV fields (such as Debit and Credit, or In + 2. If the amount is in two CSV fields (such as Debit and Credit, or In and Out): a. If both fields are unsigned: - Assign one field to amountN-in and the other to amountN-out. - hledger will automatically negate the "out" field, and will use + Assign one field to amountN-in and the other to amountN-out. + hledger will automatically negate the "out" field, and will use whichever field value is non-zero as posting N's amount. b. If either field is signed: - You will probably need to override hledger's sign for one or the + You will probably need to override hledger's sign for one or the other field, as in the following example: # Negate the -out value, but only if it is not empty: @@ -4030,12 +4090,12 @@ CSV if %amount1-out [1-9] amount1-out -%amount1-out - c. If both fields can contain a non-zero value (or both can be + c. If both fields can contain a non-zero value (or both can be empty): - The -in/-out rules normally choose the value which is - non-zero/non-empty. Some value pairs can be ambiguous, such as 1 + The -in/-out rules normally choose the value which is + non-zero/non-empty. Some value pairs can be ambiguous, such as 1 and none. For such cases, use conditional rules to help select the - amount. Eg, to handle the above you could select the value con- + amount. Eg, to handle the above you could select the value con- taining non-zero digits: fields date, description, in, out @@ -4048,8 +4108,8 @@ CSV Use the unnumbered amount (or amount-in and amount-out) syntax. 4. If the CSV has only balance amounts, not transaction amounts: - Assign to balanceN, to set a balance assignment on the Nth posting, - causing the posting's amount to be calculated automatically. balance + Assign to balanceN, to set a balance assignment on the Nth posting, + causing the posting's amount to be calculated automatically. balance with no number is equivalent to balance1. In this situation hledger is more likely to guess the wrong default account name, so you may need to set that explicitly. @@ -4065,20 +4125,20 @@ CSV o If an amount value is parenthesised: it will be de-parenthesised and sign-flipped: (AMT) becomes -AMT - o If an amount value has two minus signs (or two sets of parentheses, + o If an amount value has two minus signs (or two sets of parentheses, or a minus sign and parentheses): they cancel out and will be removed: --AMT or -(AMT) becomes AMT - o If an amount value contains just a sign (or just a set of parenthe- + o If an amount value contains just a sign (or just a set of parenthe- ses): - that is removed, making it an empty value. "+" or "-" or "()" becomes + that is removed, making it an empty value. "+" or "-" or "()" becomes "". - It's not possible (without preprocessing the CSV) to set an amount to + It's not possible (without preprocessing the CSV) to set an amount to its absolute value, ie discard its sign. Setting currency/commodity - If the currency/commodity symbol is included in the CSV's amount + If the currency/commodity symbol is included in the CSV's amount field(s): 2023-01-01,foo,$123.00 @@ -4097,7 +4157,7 @@ CSV 2023-01-01,foo,USD,123.00 You can assign that to the currency pseudo-field, which has the special - effect of prepending itself to every amount in the transaction (on the + effect of prepending itself to every amount in the transaction (on the left, with no separating space): fields date,description,currency,amount @@ -4106,7 +4166,7 @@ CSV expenses:unknown USD123.00 income:unknown USD-123.00 - Or, you can use a field assignment to construct the amount yourself, + Or, you can use a field assignment to construct the amount yourself, with more control. Eg to put the symbol on the right, and separated by a space: @@ -4117,38 +4177,38 @@ CSV expenses:unknown 123.00 USD income:unknown -123.00 USD - Note we used a temporary field name (cur) that is not currency - that + Note we used a temporary field name (cur) that is not currency - that would trigger the prepending effect, which we don't want here. Amount decimal places - When you are reading CSV data, eg with a command like hledger -f - foo.csv print, hledger will infer each commodity's decimal precision - (and other commodity display styles) from the amounts - much as when + When you are reading CSV data, eg with a command like hledger -f + foo.csv print, hledger will infer each commodity's decimal precision + (and other commodity display styles) from the amounts - much as when reading a journal file without commodity directives (see the link). - Note, the commodity styles are not inferred from the numbers in the + Note, the commodity styles are not inferred from the numbers in the original CSV data; rather, they are inferred from the amounts generated by the CSV rules. When you are importing CSV data with the import command, eg hledger im- - port foo.csv, there's another step: import tries to make the new en- - tries conform to the journal's existing styles. So for each commodity + port foo.csv, there's another step: import tries to make the new en- + tries conform to the journal's existing styles. So for each commodity - let's say it's EUR - import will choose: 1. the style declared for EUR by a commodity directive in the journal 2. otherwise, the style inferred from EUR amounts in the journal - 3. otherwise, the style inferred from EUR amounts generated by the CSV + 3. otherwise, the style inferred from EUR amounts generated by the CSV rules. - TLDR: if import is not generating the precisions or styles you want, + TLDR: if import is not generating the precisions or styles you want, add a commodity directive to specify them. Referencing other fields - In field assignments, you can interpolate only CSV fields, not hledger - fields. In the example below, there's both a CSV field and a hledger - field named amount1, but %amount1 always means the CSV field, not the + In field assignments, you can interpolate only CSV fields, not hledger + fields. In the example below, there's both a CSV field and a hledger + field named amount1, but %amount1 always means the CSV field, not the hledger field: # Name the third CSV field "amount1" @@ -4160,7 +4220,7 @@ CSV # Set comment to the CSV amount1 (not the amount1 assigned above) comment %amount1 - Here, since there's no CSV amount1 field, %amount1 will produce a lit- + Here, since there's no CSV amount1 field, %amount1 will produce a lit- eral "amount1": fields date,description,csvamount @@ -4168,7 +4228,7 @@ CSV # Can't interpolate amount1 here comment %amount1 - When there are multiple field assignments to the same hledger field, + When there are multiple field assignments to the same hledger field, only the last one takes effect. Here, comment's value will be be B, or C if "something" is matched, but never A: @@ -4179,27 +4239,27 @@ CSV How CSV rules are evaluated Here's how to think of CSV rules being evaluated. If you get a confus- - ing error while reading a CSV file, it may help to try to understand + ing error while reading a CSV file, it may help to try to understand which of these steps is failing: - 1. Any included rules files are inlined, from top to bottom, depth - first (scanning each included file for further includes, recur- + 1. Any included rules files are inlined, from top to bottom, depth + first (scanning each included file for further includes, recur- sively, before proceeding). - 2. Top level rules (date-format, fields, newest-first, skip etc) are + 2. Top level rules (date-format, fields, newest-first, skip etc) are read, top to bottom. "Top level rules" means non-conditional rules. - If a rule occurs more than once, the last one wins; except for + If a rule occurs more than once, the last one wins; except for skip/end rules, where the first one wins. - 3. The CSV file is read as text. Any non-ascii characters will be de- + 3. The CSV file is read as text. Any non-ascii characters will be de- coded using the text encoding specified by the encoding rule, other- wise the system locale's text encoding. - 4. Any top-level skip or end rule is applied. skip [N] immediately - skips the current or next N CSV records; end immediately skips all + 4. Any top-level skip or end rule is applied. skip [N] immediately + skips the current or next N CSV records; end immediately skips all remaining CSV records (not normally used at top level). - 5. Now any remaining CSV records are processed. For each CSV record, + 5. Now any remaining CSV records are processed. For each CSV record, in file order: o Is there a conditional skip/end rule that applies for this record @@ -4208,33 +4268,33 @@ CSV ber of CSV records, then continue at 5. Otherwise... - o Do some basic validation on this CSV record (eg, check that it + o Do some basic validation on this CSV record (eg, check that it has at least two fields). o For each hledger field (date, description, account1, etc.): - 1. Get the field's assigned value, first searching top level as- - signments, made directly or by the fields rule, then assign- - ments made inside succeeding if blocks. If there are more + 1. Get the field's assigned value, first searching top level as- + signments, made directly or by the fields rule, then assign- + ments made inside succeeding if blocks. If there are more than one, the last one wins. - 2. Compute the field's actual value (as text), by interpolating - any %CSVFIELD references within the assigned value; or by + 2. Compute the field's actual value (as text), by interpolating + any %CSVFIELD references within the assigned value; or by choosing a default value if there was no assignment. - o Generate a hledger transaction from the hledger field values, + o Generate a hledger transaction from the hledger field values, parsing them if needed (eg from text to an amount). - This is all done by the CSV reader, one of several readers hledger can + This is all done by the CSV reader, one of several readers hledger can use to read transactions from an input file. When all input files have - been read successfully, their transactions are passed to whichever + been read successfully, their transactions are passed to whichever hledger command the user specified. Well factored rules - Some things than can help reduce duplication and complexity in rules + Some things than can help reduce duplication and complexity in rules files: - o Extracting common rules usable with multiple CSV files into a com- + o Extracting common rules usable with multiple CSV files into a com- mon.rules, and adding include common.rules to each CSV's rules file. o Splitting if blocks into smaller if blocks, extracting the frequently @@ -4242,8 +4302,8 @@ CSV CSV rules examples Bank of Ireland - Here's a CSV with two amount fields (Debit and Credit), and a balance - field, which we can use to add balance assertions, which is not neces- + Here's a CSV with two amount fields (Debit and Credit), and a balance + field, which we can use to add balance assertions, which is not neces- sary but provides extra error checking: Date,Details,Debit,Credit,Balance @@ -4285,13 +4345,13 @@ CSV assets:bank:boi:checking EUR-5.0 = EUR126.0 expenses:unknown EUR5.0 - The balance assertions don't raise an error above, because we're read- - ing directly from CSV, but they will be checked if these entries are + The balance assertions don't raise an error above, because we're read- + ing directly from CSV, but they will be checked if these entries are imported into a journal file. Coinbase - A simple example with some CSV from Coinbase. The spot price is - recorded using cost notation. The legacy amount field name conve- + A simple example with some CSV from Coinbase. The spot price is + recorded using cost notation. The legacy amount field name conve- niently sets amount 2 (posting 2's amount) to the total cost. # Timestamp,Transaction Type,Asset,Quantity Transacted,Spot Price Currency,Spot Price at Transaction,Subtotal,Total (inclusive of fees and/or spread),Fees and/or Spread,Notes @@ -4313,7 +4373,7 @@ CSV Amazon Here we convert amazon.com order history, and use an if block to gener- - ate a third posting if there's a fee. (In practice you'd probably get + ate a third posting if there's a fee. (In practice you'd probably get this data from your bank instead, but it's an example.) "Date","Type","To/From","Name","Status","Amount","Fees","Transaction ID" @@ -4365,7 +4425,7 @@ CSV expenses:fees $1.00 Paypal - Here's a real-world rules file for (customised) Paypal CSV, with some + Here's a real-world rules file for (customised) Paypal CSV, with some Paypal-specific rules, and a second rules file included: "Date","Time","TimeZone","Name","Type","Status","Currency","Gross","Fee","Net","From Email Address","To Email Address","Transaction ID","Item Title","Item ID","Reference Txn ID","Receipt ID","Balance","Note" @@ -4516,12 +4576,12 @@ CSV Timeclock The time logging format of timeclock.el, as read by hledger. - hledger can read time logs in timeclock format. As with Ledger, these - are (a subset of) timeclock.el's format, containing clock-in and - clock-out entries as in the example below. The date is a simple date. - The time format is HH:MM[:SS][+-ZZZZ]. Seconds and timezone are op- - tional. The timezone, if present, must be four digits and is ignored - (currently the time is always interpreted as a local time). Lines be- + hledger can read time logs in timeclock format. As with Ledger, these + are (a subset of) timeclock.el's format, containing clock-in and + clock-out entries as in the example below. The date is a simple date. + The time format is HH:MM[:SS][+-ZZZZ]. Seconds and timezone are op- + tional. The timezone, if present, must be four digits and is ignored + (currently the time is always interpreted as a local time). Lines be- ginning with # or ; or *, and blank lines, are ignored. i 2015/03/30 09:00:00 some account optional description after 2 spaces ; optional comment, tags: @@ -4533,12 +4593,12 @@ Timeclock o 2015/04/02 14:00:00 o 2015/04/02 15:00:00 another:account - hledger treats each clock-in/clock-out pair as a transaction posting - some number of hours to an account. Entries are paired by the account - name if the same name is given for a clock-in/clock-out pair. If no - name is given for a clock-out, then it is paired with the most recent - clock-in entry. If the session spans more than one day, it is split - into several transactions, one for each day. For the above time log, + hledger treats each clock-in/clock-out pair as a transaction posting + some number of hours to an account. Entries are paired by the account + name if the same name is given for a clock-in/clock-out pair. If no + name is given for a clock-out, then it is paired with the most recent + clock-in entry. If the session spans more than one day, it is split + into several transactions, one for each day. For the above time log, hledger print generates these journal entries: $ hledger -f t.timeclock print @@ -4574,13 +4634,13 @@ Timeclock perhaps the extras in ledgerutils.el o or use the old ti and to scripts in the ledger 2.x repository. These - rely on a "timeclock" executable which I think is just the ledger 2 + rely on a "timeclock" executable which I think is just the ledger 2 executable renamed. Timedot - timedot format is hledger's human-friendly time logging format. Com- - pared to timeclock format, it is more convenient for quick, approxi- - mate, and retroactive time logging, and more human-readable (you can + timedot format is hledger's human-friendly time logging format. Com- + pared to timeclock format, it is more convenient for quick, approxi- + mate, and retroactive time logging, and more human-readable (you can see at a glance where time was spent). A quick example: 2023-05-01 @@ -4599,59 +4659,59 @@ Timedot (per:admin:finance) 0 A timedot file contains a series of transactions (usually one per day). - Each begins with a simple date (Y-M-D, Y/M/D, or Y.M.D), optionally be + Each begins with a simple date (Y-M-D, Y/M/D, or Y.M.D), optionally be followed on the same line by a transaction description, and/or a trans- action comment following a semicolon. After the date line are zero or more time postings, consisting of: - o An account name - any hledger-style account name, optionally in- + o An account name - any hledger-style account name, optionally in- dented. - o Two or more spaces - required if there is an amount (as in journal + o Two or more spaces - required if there is an amount (as in journal format). o A timedot amount, which can be o empty (representing zero) - o a number, optionally followed by a unit s, m, h, d, w, mo, or y, - representing a precise number of seconds, minutes, hours, days + o a number, optionally followed by a unit s, m, h, d, w, mo, or y, + representing a precise number of seconds, minutes, hours, days weeks, months or years (hours is assumed by default), which will be - converted to hours according to 60s = 1m, 60m = 1h, 24h = 1d, 7d = + converted to hours according to 60s = 1m, 60m = 1h, 24h = 1d, 7d = 1w, 30d = 1mo, 365d = 1y. - o one or more dots (period characters), each representing 0.25. - These are the dots in "timedot". Spaces are ignored and can be + o one or more dots (period characters), each representing 0.25. + These are the dots in "timedot". Spaces are ignored and can be used for grouping/alignment. - o Added in 1.32 one or more letters. These are like dots but they - also generate a tag t: (short for "type") with the letter as its - value, and a separate posting for each of the values. This pro- - vides a second dimension of categorisation, viewable in reports + o Added in 1.32 one or more letters. These are like dots but they + also generate a tag t: (short for "type") with the letter as its + value, and a separate posting for each of the values. This pro- + vides a second dimension of categorisation, viewable in reports with --pivot t. - o An optional comment following a semicolon (a hledger-style posting + o An optional comment following a semicolon (a hledger-style posting comment). - There is some flexibility to help with keeping time log data and notes + There is some flexibility to help with keeping time log data and notes in the same file: o Blank lines and lines beginning with # or ; are ignored. - o After the first date line, lines which do not contain a double space + o After the first date line, lines which do not contain a double space are parsed as postings with zero amount. (hledger's register reports will show these if you add -E). - o Before the first date line, lines beginning with * (eg org headings) - are ignored. And from the first date line onward, Emacs org mode + o Before the first date line, lines beginning with * (eg org headings) + are ignored. And from the first date line onward, Emacs org mode heading prefixes at the start of lines (one or more *'s followed by a - space) will be ignored. This means the time log can also be a org + space) will be ignored. This means the time log can also be a org outline. Timedot files don't support directives like journal files. So a common - pattern is to have a main journal file (eg time.journal) that contains - any needed directives, and then includes the timedot file (include + pattern is to have a main journal file (eg time.journal) that contains + any needed directives, and then includes the timedot file (include time.timedot). Timedot examples @@ -4759,21 +4819,21 @@ Timedot PART 3: REPORTING CONCEPTS Time periods Report start & end date - Most hledger reports will by default show the full time period repre- - sented by the journal. The report start date will be the earliest + Most hledger reports will by default show the full time period repre- + sented by the journal. The report start date will be the earliest transaction or posting date, and the report end date will be the latest transaction, posting, or market price date. Often you will want to see a shorter period, such as the current month. - You can specify a start and/or end date with the -b/--begin, -e/--end, - or -p/--period options, or a date: query argument, described below. + You can specify a start and/or end date with the -b/--begin, -e/--end, + or -p/--period options, or a date: query argument, described below. All of these accept the smart date syntax, also described below. End dates are exclusive; specify the day after the last day you want to see in the report. When dates are specified by multiple options, the last (right-most) op- - tion wins. And when date: queries and date options are combined, the + tion wins. And when date: queries and date options are combined, the report period will be their intersection. Examples: @@ -4801,22 +4861,27 @@ Time periods -b and -e) Smart dates - In hledger's user interfaces (though not in the journal file), you can - optionally use "smart date" syntax. Smart dates can be written with - english words, can be relative, and can have parts omitted. Missing - parts are inferred as 1, when needed. Smart dates can be interpreted - as dates or periods depending on context. + In hledger's user interfaces (though not in the journal file), you can + optionally use "smart date" syntax. Smart dates can be written with + english words, can be relative, and can have parts omitted. Missing + parts are inferred as 1, when needed. Smart dates can be interpreted + as dates or periods depending on the context. Examples: - 2004-01-01, 2004/10/1, 2004.9.1, 20240504 : - Exact dates. The year must have at least four digits, the month must - be 1-12, the day must be 1-31, the separator can be - or / or . or - nothing. + 2004-01-01, 2004/10/1, 2004.9.1, 20240504, 2024Q1 : + Exact dates. The year must have at least four digits, the month must + be 1-12, the day must be 1-31, the separator can be - or / or . or + nothing. The q can be upper or lower case and the quarter number must + be 1-4. 2004-10 start of month + 2004q3 start of third quarter of 2004 + + q3 start of third quarter of current year + 2004 start of year 10/1 or oct or october @@ -4914,7 +4979,7 @@ Time periods ary. o hledger register --monthly --end 2/14 also will end the report at the - end of february. + end of february (overriding the requested end date). o hledger register --monthly --begin 1/5 --end 2/14 will end the report on march 4th [1]. @@ -5113,8 +5178,8 @@ Depth o all other accounts to depth 1. If an account is matched by more than one regular expression depth ar- - gument then the more specific one will used. For example, if --depth - assets=1 --depth assets:bank:savings=2 is provided, then as- + gument then the more specific one will be used. For example, if + --depth assets=1 --depth assets:bank:savings=2 is provided, then as- sets:bank:savings will be collapsed to depth 2 rather than depth 1. This is because assets:bank:savings matches at level 3 in the account name, while assets matches at level 1. The same would be true with the @@ -5365,7 +5430,7 @@ Queries all:'QUERYEXPR' Like expr:, but when used with transaction-oriented commands like print, it matches the transaction only if all postings are matched by - all of QUERYEXPR. + all of QUERYEXPR (and there is at least one posting). So, hledger print all:'cash and amt:0' means "show transactions where all postings involve a cash account and have a zero amount". Or, hledger print all:'cash or checking' means "show transactions which @@ -6294,8 +6359,8 @@ Value reporting 2000-02-01 (a) 2 B - With no report period specified, that shows the value as of the last - day of the journal (2000-03-01): + With no report period specified, the latest transaction date or price + date is used as valuation date (2000-04-01): $ hledger -f- print --value=end 2000-01-01 @@ -6307,7 +6372,7 @@ Value reporting 2000-03-01 (a) 3 B - Show the current value (the 2000-04-01 price is still in effect today): + The value today is the same (the 2000-04-01 price is still in effect): $ hledger -f- print --value=now 2000-01-01 @@ -6609,10 +6674,7 @@ Help commands eg -s4 to play at 4x original speed or -s.5 to play at half speed. The default speed is 2x. - Other asciinema options can be added following a double dash, eg -- - -i.1 to limit pauses or -- -h to list asciinema's other options. - - During playback, several keys are available: SPACE to pause/unpause, . + During playback, several keys are available: SPACE to pause/unpause, . to step forward (while paused), CTRL-c quit. Examples: @@ -6624,7 +6686,7 @@ Help commands This command is experimental: there aren't many useful demos yet. help - Show the hledger user manual with info, man, or a pager. With a (case + Show the hledger user manual with info, man, or a pager. With a (case insensitive) TOPIC argument, try to open it at that section heading. Flags: @@ -6633,23 +6695,23 @@ Help commands -p show the manual with $PAGER or less (less is always used if TOPIC is specified) - This command shows the hledger manual built in to your hledger exe- - cutable. It can be useful when offline, or when you prefer the termi- + This command shows the hledger manual built in to your hledger exe- + cutable. It can be useful when offline, or when you prefer the termi- nal to a web browser, or when the appropriate hledger manual or viewers are not installed properly on your system. - By default it chooses the best viewer found in $PATH, trying in this - order: info, man, $PAGER, less, more, stdout. (If a TOPIC is speci- - fied, $PAGER and more are not tried.) You can force the use of info, - man, or a pager with the -i, -m, or -p flags. If no viewer can be - found, or if running non-interactively, it just prints the manual to + By default it chooses the best viewer found in $PATH, trying in this + order: info, man, $PAGER, less, more, stdout. (If a TOPIC is speci- + fied, $PAGER and more are not tried.) You can force the use of info, + man, or a pager with the -i, -m, or -p flags. If no viewer can be + found, or if running non-interactively, it just prints the manual to stdout. - When using info, TOPIC can match either the full heading or a prefix. + When using info, TOPIC can match either the full heading or a prefix. If your info --version is < 6, you'll need to upgrade it, eg with 'brew install texinfo' on mac. - When using man or less, TOPIC must match the full heading. For a pre- + When using man or less, TOPIC must match the full heading. For a pre- fix match, you can write 'TOPIC.*'. Examples @@ -6661,7 +6723,7 @@ Help commands User interface commands repl - Start an interactive prompt, where you can run any of hledger's com- + Start an interactive prompt, where you can run any of hledger's com- mands. Data files are parsed just once, so the commands run faster. Flags: @@ -6669,21 +6731,21 @@ User interface commands This command is experimental and could change in the future. - hledger repl starts a read-eval-print loop (REPL) where you can enter - commands interactively. As with the run command, each input file (or + hledger repl starts a read-eval-print loop (REPL) where you can enter + commands interactively. As with the run command, each input file (or each input file/input options combination) is parsed just once, so com- - mands will run more quickly than if you ran them individually at the + mands will run more quickly than if you ran them individually at the command line. Also like run, the input file(s) specified for the repl command will be - the default input for all interactive commands. You can override this - temporarily by specifying an -f option in particular commands. But - note that commands will not see any changes made to input files (eg by + the default input for all interactive commands. You can override this + temporarily by specifying an -f option in particular commands. But + note that commands will not see any changes made to input files (eg by add) until you exit and restart the REPL. The command syntax is the same as with run: - o enter one hledger command at a time, without the usual hledger first + o enter one hledger command at a time, without the usual hledger first word o empty lines and comment text from # to end of line are ignored @@ -6692,7 +6754,7 @@ User interface commands o type exit or quit or control-D to exit the REPL. - While it is running, the REPL remembers your command history, and you + While it is running, the REPL remembers your command history, and you can navigate in the usual ways: o Keypad or Emacs navigation keys to edit the current command line @@ -6703,9 +6765,9 @@ User interface commands o TAB to complete file paths. - Generally repl command lines should feel much like the normal hledger - CLI, but you may find differences. repl is a little stricter; eg it - requires full command names or official abbreviations (as seen in the + Generally repl command lines should feel much like the normal hledger + CLI, but you may find differences. repl is a little stricter; eg it + requires full command names or official abbreviations (as seen in the commands list). The commands and help commands, and the command help flags (CMD --tldr, @@ -6714,7 +6776,7 @@ User interface commands You can type control-C to cancel a long-running command (but only once; typing it a second time will exit the REPL). - And in most shells you can type control-Z to temporarily exit to the + And in most shells you can type control-Z to temporarily exit to the shell (and then fg to return to the REPL). Examples @@ -6744,8 +6806,8 @@ User interface commands ... run - Run a sequence of hledger commands, provided as files or command line - arguments. Data files are parsed just once, so the commands run + Run a sequence of hledger commands, provided as files or command line + arguments. Data files are parsed just once, so the commands run faster. Flags: @@ -6755,52 +6817,52 @@ User interface commands You can use run in three ways: - o hledger run -- CMD1 -- CMD2 -- CMD3 - read commands from the command + o hledger run -- CMD1 -- CMD2 -- CMD3 - read commands from the command line, separated by -- - o hledger run SCRIPTFILE1 SCRIPTFILE2 - read commands from one or more + o hledger run SCRIPTFILE1 SCRIPTFILE2 - read commands from one or more files o cat SCRIPTFILE1 | hledger run - read commands from standard input. run first loads the input file(s) specified by LEDGER_FILE or by -f op- tions, in the usual way. Then it runs each command in turn, each using - the same input data. But if you want a particular command to use dif- - ferent input, you can specify an -f option within that command. This + the same input data. But if you want a particular command to use dif- + ferent input, you can specify an -f option within that command. This will override (not add to) the default input, just for that command. Each input file (more precisely, each combination of input file and in- - put options) is parsed only once. This means that commands will not - see any changes made to these files, until the next run. But the com- - mands will run more quickly than if run individually (typically about + put options) is parsed only once. This means that commands will not + see any changes made to these files, until the next run. But the com- + mands will run more quickly than if run individually (typically about twice as fast). Command scripts, whether in a file or written on the command line, have a simple syntax: - o each line may contain a single hledger command and its arguments, + o each line may contain a single hledger command and its arguments, without the usual hledger first word o empty lines are ignored o text from # to end of line is a comment, and ignored - o you can use single or double quotes to quote arguments when needed, + o you can use single or double quotes to quote arguments when needed, as on the command line - o these extra commands are available: echo TEXT prints some text, and + o these extra commands are available: echo TEXT prints some text, and exit or quit ends the run. - On unix systems you can use #!/usr/bin/env hledger run in the first - line of a command file to make it a runnable script. If that gives an + On unix systems you can use #!/usr/bin/env hledger run in the first + line of a command file to make it a runnable script. If that gives an error, use #!/usr/bin/env -S hledger run. It's ok to use the run command recursively within a command script. - You may find some differences in behaviour between run command lines - and normal hledger command lines. run is a little stricter; eg it re- - quires full command names or official abbreviations (as seen in the - commands list), and command options must be written after the command + You may find some differences in behaviour between run command lines + and normal hledger command lines. run is a little stricter; eg it re- + quires full command names or official abbreviations (as seen in the + commands list), and command options must be written after the command name. Examples @@ -6808,8 +6870,8 @@ User interface commands hledger -f some.journal run -- balance assets --depth 2 -- balance liabilities -f /some/other.journal --depth 3 --transpose -- stats - This would load some.journal, run balance assets --depth 2 on it, then - run balance liabilities --depth 3 --transpose on /some/other.journal, + This would load some.journal, run balance assets --depth 2 on it, then + run balance liabilities --depth 3 --transpose on /some/other.journal, and finally run stats on some.journal Run commands from standard input: @@ -6846,35 +6908,35 @@ User interface commands Data entry commands add - Record new transactions with interactive prompting in the console. + Add new transactions to a journal file, with interactive prompting. Flags: --no-new-accounts don't allow creating new accounts - Many hledger users edit their journals directly with a text editor, or - generate them from CSV. For more interactive data entry, there is the - add command, which prompts interactively on the console for new trans- - actions, and appends them to the main journal file (which should be in - journal format). Existing transactions are not changed. This is one - of the few hledger commands that writes to the journal file (see also + Many hledger users edit their journals directly with a text editor, or + generate them from CSV. For more interactive data entry, there is the + add command, which prompts interactively on the console for new trans- + actions, and appends them to the main journal file (which should be in + journal format). Existing transactions are not changed. This is one + of the few hledger commands that writes to the journal file (see also import). To use it, just run hledger add and follow the prompts. You can add as - many transactions as you like; when you are finished, enter . or press + many transactions as you like; when you are finished, enter . or press control-d or control-c to exit. Features: - o add tries to provide useful defaults, using the most similar (by de- - scription) recent transaction (filtered by the query, if any) as a + o add tries to provide useful defaults, using the most similar (by de- + scription) recent transaction (filtered by the query, if any) as a template. o You can also set the initial defaults with command line arguments. o Readline-style edit keys can be used during data entry. - o The tab key will auto-complete whenever possible - accounts, pay- - ees/descriptions, dates (yesterday, today, tomorrow). If the input + o The tab key will auto-complete whenever possible - accounts, pay- + ees/descriptions, dates (yesterday, today, tomorrow). If the input area is empty, it will insert the default value. o A parenthesised transaction code may be entered following a date. @@ -6883,17 +6945,20 @@ Data entry commands o If you make a mistake, enter < at any prompt to go one step backward. - o Input prompts are displayed in a different colour when the terminal + o Input prompts are displayed in a different colour when the terminal supports it. Notes: o If you enter a number with no commodity symbol, and you have declared - a default commodity with a D directive, you might expect add to add - this symbol for you. It does not do this; we assume that if you are - using a D directive you prefer not to see the commodity symbol re- + a default commodity with a D directive, you might expect add to add + this symbol for you. It does not do this; we assume that if you are + using a D directive you prefer not to see the commodity symbol re- peated on amounts in the journal. + o add creates entries in journal format; it won't work with timeclock + or timedot files. + Examples: o Record new transactions, saving to the default journal file: @@ -6958,7 +7023,7 @@ Data entry commands $ hledger import *.csv - Import preview + Import dry run It's useful to preview the import by running first with --dry-run, to sanity check the range of dates being imported, and to check the effect of your conversion rules if converting from CSV. Eg: @@ -6974,7 +7039,7 @@ Data entry commands You could also run this repeatedly to see the effect of edits to your conversion rules: - $ watchexec -- 'hledger import --dry-run bank.csv | hledger -f- -I print unknown' + $ watchexec -- "hledger import --dry-run bank.csv | hledger -f- -I print unknown" Once the conversion and dates look good enough to import to your jour- nal, perhaps with some manual fixups to follow, you would do the actual @@ -7104,27 +7169,23 @@ Data entry commands Related: CSV > Amount decimal places. + Import archiving + When importing from a CSV rules file (hledger import bank.rules), you + can use the archive rule to enable automatic archiving of the data + file. After a successful import, the data file (specified by source) + will be moved to an archive folder (data/, next to the rules file, + auto-created), and renamed similar to the rules file, with a date. + This can be useful for troubleshooting, detecting variations in your + banks' CSV data, regenerating entries with improved rules, etc. + + The archive rule also causes import to handle source glob patterns dif- + ferently: when there are multiple matched files, it will pick the old- + est, not the newest. + Import special cases - If you have a download whose file name varies, you could rename it to a - fixed name after each download. Or you could use a CSV source rule - with a suitable glob pattern, and import from the .rules file instead - of the data file. - - Here's a situation where you would need to run import with care: say - you download bank.csv, but forget to import it or delete it. And next - month you download it again. This time your web browser may save it as - bank (2).csv. So now each of these may have data not included in the - other. And a source rule with a glob pattern would match only the most - recent file. So in this case you should import from each one in turn, - in the correct order, taking care to use the same filename each time: - - $ hledger import bank.csv - $ mv 'bank (2).csv' bank.csv - $ hledger import bank.csv - + Deduplication Here are two kinds of "deduplication" which import does not handle (and - generally should not, since these can happen legitimately in financial - data): + should not, because these can happen legitimately in financial data): o Two or more of the new CSV records are identical, and generate iden- tical new journal entries. @@ -7132,6 +7193,29 @@ Data entry commands o A new CSV record generates a journal entry identical to one(s) al- ready in the journal. + Varying file name + If you have a download whose file name varies, you could rename it to a + fixed name after each download. Or you could use a CSV source rule + with a suitable glob pattern, and import from the .rules file. + + Multiple versions + Say you download bank.csv, import it, but forget to delete it from your + downloads folder. The next time you download it, your web browser will + save it as (eg) bank (2).csv. The source rule's glob patterns are for + just this situation: instead of specifying source bank.csv, specify + source bank*.csv. Then hledger -f bank.rules CMD or hledger import + bank.rules will automatically pick the newest matched file (bank + (2).csv). + + Alternately, what if you download, but forget to import or delete, then + download again ? Now each of bank.csv and bank (2).csv might contain + data that's not in the other, and not in your journal. In this case, + it's best to import each of them in turn, oldest first (otherwise, + overlap detection could cause new records to be skipped). Enabling im- + port archiving ensures this. Then hledger import bank.rules; hledger + import bank.rules will import and archive first bank.csv, then bank + (2).csv. + Basic report commands accounts List the account names used or declared in the journal. @@ -7690,7 +7774,7 @@ Standard report commands Transactions involving subaccounts of this account will also be shown. aregister ignores depth limits, so its final total will always match a - balance report with similar arguments. + historical balance report with similar arguments. Any additional arguments form a query which will filter the transac- tions shown. Note some queries will disturb the running balance, caus- @@ -9738,10 +9822,10 @@ Data generation commands More: - $ hledger rewrite -- [QUERY] --add-posting "ACCT AMTEXPR" ... - $ hledger rewrite -- ^income --add-posting '(liabilities:tax) *.33' - $ hledger rewrite -- expenses:gifts --add-posting '(budget:gifts) *-1"' - $ hledger rewrite -- ^income --add-posting '(budget:foreign currency) *0.25 JPY; diversify' + $ hledger rewrite [QUERY] --add-posting "ACCT AMTEXPR" ... + $ hledger rewrite ^income --add-posting '(liabilities:tax) *.33' + $ hledger rewrite expenses:gifts --add-posting '(budget:gifts) *-1"' + $ hledger rewrite ^income --add-posting '(budget:foreign currency) *0.25 JPY; diversify' Argument for --add-posting option is a usual posting of transaction with an exception for amount specification. More precisely, you can @@ -9771,12 +9855,12 @@ Data generation commands actions you usually write. It indicates the query by which you want to match the posting to add new ones. - $ hledger rewrite -- -f input.journal -f rewrite-rules.journal > rewritten-tidy-output.journal + $ hledger rewrite -f input.journal -f rewrite-rules.journal > rewritten-tidy-output.journal This is something similar to the commands pipeline: - $ hledger rewrite -- -f input.journal '^income' --add-posting '(liabilities:tax) *.33' \ - | hledger rewrite -- -f - expenses:gifts --add-posting 'budget:gifts *-1' \ + $ hledger rewrite -f input.journal '^income' --add-posting '(liabilities:tax) *.33' \ + | hledger rewrite -f - expenses:gifts --add-posting 'budget:gifts *-1' \ --add-posting 'assets:budget *1' \ > rewritten-tidy-output.journal @@ -9788,7 +9872,7 @@ Data generation commands To use this tool for batch modification of your journal files you may find useful output in form of unified diff. - $ hledger rewrite -- --diff -f examples/sample.journal '^income' --add-posting '(liabilities:tax) *.33' + $ hledger rewrite --diff -f examples/sample.journal '^income' --add-posting '(liabilities:tax) *.33' Output might look like: @@ -10086,10 +10170,10 @@ Constructing command lines o command-specific options must go after the command (it's fine to put common options there too: hledger CMD OPTS ARGS) - o running add-on executables directly simplifies command line parsing - (hledger-ui OPTS ARGS) + o you can run addon commands via hledger (hledger ui [ARGS]) or di- + rectly (hledger-ui [ARGS]) - o enclose "problematic" args in single quotes + o enclose "problematic" arguments in single quotes o if needed, also add a backslash to hide regular expression metachar- acters from the shell @@ -10471,17 +10555,12 @@ BUGS Some known issues and limitations: - The need to precede add-on command options with -- when invoked from - hledger is awkward. (See Command options, Constructing command lines.) - - A system locale with a suitable text encoding must be configured to + A system locale with a suitable text encoding must be configured to work with non-ascii data. (See Text encoding, Troubleshooting.) - On Microsoft Windows, depending whether you are running in a CMD window - or a Cygwin/MSYS/Mintty window and how you installed hledger, non-ascii - characters and colours may not be supported, and the tab key may not be - supported by hledger add. (Running in a WSL window should resolve - these.) + On Microsoft Windows, depending what kind of terminal window you use, + non-ascii characters, ANSI text formatting, and/or the add command's + TAB key for completion, may not be supported. When processing large data files, hledger uses more memory than Ledger.