;doc: update command docs

hledger/Hledger/Cli/Commands/Aregister.md

@@ -12,6 +12,8 @@ Flags:
                             balance.
      --no-elide             don't show only 2 commodities per amount
      --cumulative           show running total from report start date
+  -H --historical           show historical running total/balance (includes
+                            postings before report start date) (default)
      --invert               display all amounts with reversed sign
      --heading=YN           show heading row above table: yes (default) or no
   -w --width=N              set output width (default: terminal width or

hledger/Hledger/Cli/Commands/Aregister.txt

@@ -10,6 +10,11 @@ Flags:
                             date. Warning: this can show a wrong running
                             balance.
      --no-elide             don't show only 2 commodities per amount
+     --cumulative           show running total from report start date
+  -H --historical           show historical running total/balance (includes
+                            postings before report start date) (default)
+     --invert               display all amounts with reversed sign
+     --heading=YN           show heading row above table: yes (default) or no
   -w --width=N              set output width (default: terminal width or
                             $COLUMNS). -wN,M sets description width as well.
      --align-all            guarantee alignment across all lines (slower)
@@ -20,8 +25,9 @@ Flags:
 
 aregister shows the overall transactions affecting a particular account
 (and any subaccounts). Each report line represents one transaction in
-this account. Transactions before the report start date are always
+this account. Transactions before the report start date are included in
-included in the running balance (--historical mode is always on).
+the running balance (--historical mode is the default). You can suppress
+this behaviour using the --cumulative option.
 
 This is a more "real world", bank-like view than the register command
 (which shows individual postings, possibly from multiple accounts, not
@@ -71,9 +77,14 @@ visual discontinuities as column widths are adjusted. If you want to
 ensure perfect alignment, at the cost of more time and memory, use the
 --align-all flag.
 
+By default, aregister shows a heading above the data. However, when
+reporting in a language different from English, it is easier to omit
+this heading and prepend your own one. For this purpose, use the
+--heading=no option.
+
 This command also supports the output destination and output format
 options. The output formats supported are txt, csv, tsv (Added in 1.32),
-and json.
+html, fods (Added in 1.41) and json.
 
 aregister and posting dates
 

hledger/Hledger/Cli/Commands/Balance.txt

@@ -909,10 +909,6 @@ run the balance command with the extra option
 --base-url=http://localhost:5000. You can also produce relative links,
 like --base-url="some/path" or --base-url="".)
 
-The balance reports' HTML output currently does not indent tree mode
-reports properly (#1846). So in HTML balance reports, use list mode for
-now (it is the default).
-
 Some useful balance reports
 
 Some frequently used balance options/reports are:

hledger/Hledger/Cli/Commands/Balancesheet.md

@@ -13,8 +13,6 @@ Flags:
                             market price fluctuations)
      --gain                 show unrealised capital gain/loss (historical
                             balance value minus cost basis)
-     --budget               show sum of posting amounts compared to budget
-                            goals defined by periodic transactions
      --count                show the count of postings
      --change               accumulate amounts from column start to column
                             end (in multicolumn reports)

hledger/Hledger/Cli/Commands/Balancesheet.txt

@@ -12,8 +12,6 @@ Flags:
                             market price fluctuations)
      --gain                 show unrealised capital gain/loss (historical
                             balance value minus cost basis)
-     --budget               show sum of posting amounts compared to budget
-                            goals defined by periodic transactions
      --count                show the count of postings
      --change               accumulate amounts from column start to column
                             end (in multicolumn reports)

hledger/Hledger/Cli/Commands/Balancesheetequity.md

@@ -15,8 +15,6 @@ Flags:
                             market price fluctuations)
      --gain                 show unrealised capital gain/loss (historical
                             balance value minus cost basis)
-     --budget               show sum of posting amounts compared to budget
-                            goals defined by periodic transactions
      --count                show the count of postings
      --change               accumulate amounts from column start to column
                             end (in multicolumn reports)

hledger/Hledger/Cli/Commands/Balancesheetequity.txt

@@ -13,8 +13,6 @@ Flags:
                             market price fluctuations)
      --gain                 show unrealised capital gain/loss (historical
                             balance value minus cost basis)
-     --budget               show sum of posting amounts compared to budget
-                            goals defined by periodic transactions
      --count                show the count of postings
      --change               accumulate amounts from column start to column
                             end (in multicolumn reports)

hledger/Hledger/Cli/Commands/Cashflow.md

@@ -15,8 +15,6 @@ Flags:
                             market price fluctuations)
      --gain                 show unrealised capital gain/loss (historical
                             balance value minus cost basis)
-     --budget               show sum of posting amounts compared to budget
-                            goals defined by periodic transactions
      --count                show the count of postings
      --change               accumulate amounts from column start to column
                             end (in multicolumn reports) (default)

hledger/Hledger/Cli/Commands/Cashflow.txt

@@ -14,8 +14,6 @@ Flags:
                             market price fluctuations)
      --gain                 show unrealised capital gain/loss (historical
                             balance value minus cost basis)
-     --budget               show sum of posting amounts compared to budget
-                            goals defined by periodic transactions
      --count                show the count of postings
      --change               accumulate amounts from column start to column
                             end (in multicolumn reports) (default)

hledger/Hledger/Cli/Commands/Import.txt

@@ -6,123 +6,156 @@ Flags:
      --catchup              just mark all transactions as already imported
      --dry-run              just show the transactions to be imported
 
-This command detects new transactions in each FILE argument since it was
+This command detects new transactions in one or more data files
-last run, and appends them to the main journal.
+specified as arguments, and appends them to the main journal.
 
-Or with --dry-run, it just print the transactions that would be added.
+You can import from any input file format hledger supports, but
+CSV/SSV/TSV files, downloaded from financial institutions, are the most
+common import source.
 
-Or with --catchup, it just marks all of the FILEs' current transactions
+The import destination is the default journal file, or another specified
-as already imported.
+in the usual way with $LEDGER_FILE or -f/--file. It should be in journal
+format.
 
-This is one of the few hledger commands that writes to the journal file
+Examples:
-(see also add). It only appends; existing data will not be changed.
 
-The input files are specified as arguments, so to import one or more CSV
+$ hledger import bank1-checking.csv bank1-savings.csv
-files to your main journal, you will run hledger import bank.csv or
-perhaps hledger import *.csv.
 
-Note you can import from any file format, though CSV files are the most
+$ hledger import *.csv
-common import source, and these docs focus on that case. The target file
-(main journal) should be in journal format.
 
-Date skipping
+Import preview
 
-import tries to import only the transactions which are new since the
+It's useful to preview the import by running first with --dry-run, to
-last import, ignoring any that it has seen in previous runs. So if your
+sanity check the range of dates being imported, and to check the effect
-bank's CSV includes the last three months of data, you can download and
+of your conversion rules if converting from CSV. Eg:
-import it every month (or week, or day) and only the new transactions
-will be imported each time.
 
-It works as follows: for each imported FILE,
+$ hledger import bank.csv --dry-run
 
--   It tries to read the latest date previously seen, from .latest.FILE
+The dry run output is valid journal format, so hledger can re-parse it.
-    in the same directory
+If the output is large, you could show just the uncategorised
--   Then it processes FILE, ignoring transactions on or before that date
+transactions like so:
 
-And after a successful import, unless --dry-run was used, it updates the
+$ hledger import --dry-run bank.csv | hledger -f- -I print unknown
-.latest.FILE(s) for next time. This is a simple system that works for
-most real-world CSV files; it assumes the following are true, or true
-enough:
 
-1.  the name of the input file is stable across successive downloads
+You could also run this repeatedly to see the effect of edits to your
-2.  new items always have the newest dates
+conversion rules:
-3.  item dates are stable across downloads
-4.  the order of same-date items is stable across downloads.
 
-Tips:
+$ watchexec -- 'hledger import --dry-run bank.csv | hledger -f- -I print unknown'
 
--   To help ensure a stable file name, remember you can use a CSV rules
+Once the conversion and dates look good enough to import to your
-    file as an input file.
+journal, perhaps with some manual fixups to follow, you would do the
+actual import:
 
--   If you have a bank whose CSV dates or ordering occasionally change,
+$ hledger import bank.csv
-    you can reduce the chance of this happening in new transactions by
-    importing more often. (If it happens in old transactions, that's
-    harmless.)
 
-Note this is just one kind of "deduplication": not reprocessing the same
+Overlap detection
-dates across successive runs. import doesn't detect other kinds of
-duplication, such as the same transaction appearing multiple times
-within a single run, or a new transaction that looks identical to a
-transaction already in the journal. (Because these can happen
-legitimately in real-world data.)
 
-Here's a situation where you need to run import with care: say you
+Reading CSV files is built in to hledger, and not specific to import; so
-download but forget to import bank.1.csv, and a week later you download
+you could also import by doing hledger -f bank.csv print >>$LEDGER_FILE.
-bank.2.csv with some overlapping data. You should not process both of
-these as a single import (hledger import bank.1.csv bank.2.csv), because
-the overlapping transactions would not be deduplicated. Instead, import
-one file at a time, using the same filename each time:
 
-$ mv bank.1.csv bank.csv; hledger import bank.csv
+But import is easier and provides some advantages. The main one is that
-$ mv bank.2.csv bank.csv; hledger import bank.csv
+it avoids re-importing transactions it has seen on previous runs. This
+means you don't have to worry about overlapping data in successive
+downloads of your bank CSV; just download and import as often as you
+like, and only the new transactions will be imported each time.
 
-Normally you don't need to think about .latest.* files, but you can
+We don't call this "deduplication", as it's generally not possible to
-create or modify them to catch up to a certain date, or delete them to
+reliably detect duplicates in bank CSV. Instead, import remembers the
-mark all transactions as new. Their format is a single ISO-format
+latest date processed previously in each CSV file (saving it in a hidden
-YYYY-MM-DD date, optionally repeated on multiple lines, meaning "I have
+file), and skips any records prior to that date. This works well for
-seen the transactions before this date, and this many of them on this
+most real-world CSV, where:
-date".
 
-hledger print --new also uses and updates these .latest.* files, but it
+1.  the data file name is stable (does not change) across imports
-is less often used.
+2.  the item dates are stable across imports
+3.  the order of same-date items is stable across imports
+4.  the newest items have the newest dates
 
-Related: CSV > Working with CSV > Deduplicating, importing.
+(Occasional violations of 2-4 are often harmless; you can reduce the
+chance of disruption by downloading and importing more often.)
 
-Import testing
+Overlap detection is automatic, and shouldn't require much attention
+from you, except perhaps at first import (see below). But here's how it
+works:
 
-With --dry-run, the transactions that will be imported are printed to
+-   For each FILE being imported from:
-the terminal, without updating your journal or state files. The output
-is valid journal format, like the print command, so you can re-parse it.
-Eg, to see any importable transactions which CSV rules have not
-categorised:
 
-$ hledger import --dry bank.csv | hledger -f- -I print unknown
+    1.  hledger reads a file named .latest.FILE file in the same
+        directory, if any. This file contains the latest record date
+        previously imported from FILE, in YYYY-MM-DD format. If multiple
+        records with that date were imported, the date is repeated on N
+        lines.
 
-or (live updating):
+    2.  hledger reads records from FILE. If a latest date was found in
+        step 1, any records before that date, and the first N records on
+        that date, are skipped.
 
-$ ls bank.csv* | entr bash -c 'echo ====; hledger import --dry bank.csv | hledger -f- -I print unknown'
+-   After a successful import from all FILEs, without error and without
+    --dry-run, hledger updates each FILE's .latest.FILE for next time.
 
-Note: when importing from multiple files at once, it's currently
+If this goes wrong, it's relatively easy to repair:
-possible for some .latest files to be updated successfully, while the
+
-actual import fails because of a problem in one of the files, leaving
+-   You'll notice it before import when you preview with
-them out of sync (and causing some transactions to be missed). To
+    import --dry-run.
-prevent this, do a --dry-run first and fix any problems before the real
+-   Or after import when you try to reconcile your hledger account
-import.
+    balances with your bank.
+-   hledger print -f FILE.csv will show all recently downloaded
+    transactions. Compare these with your journal. Copy/paste if needed.
+-   Update your conversion rules and print again, if needed.
+-   You can manually update or remove the .latest file, or use
+    import --catchup FILE.
+-   Download and import more often, eg twice a week, at least while you
+    are learning. It's easier to review and troubleshoot when there are
+    fewer transactions.
+
+First import
+
+The first time you import from a file, when no corresponding .latest
+file has been created yet, all of the records will be imported.
+
+But perhaps you have been entering the data manually, so you know that
+all of these transactions are already recorded in the journal. In this
+case you can run hledger import --catchup once. This will create a
+.latest file containing the latest CSV record date, so that none of
+those records will be re-imported.
+
+Or, if you know that some but not all of the transactions are in the
+journal, you can create the .latest file yourself. Eg, let's say you
+previously recorded foobank transactions up to 2024-10-31 in the
+journal. Then in the directory where you'll be saving foobank.csv, you
+would create a .latest.foobank.csv file containing
+
+2024-10-31
+
+Or if you had three foobank transactions recorded with that date, you
+would repeat the date that many times:
+
+2024-10-31
+2024-10-31
+2024-10-31
+
+Then hledger import foobank.csv [--dry-run] will import only the newer
+records.
 
 Importing balance assignments
 
-Entries added by import will have their posting amounts made explicit
+Journal entries added by import will have all posting amounts made
-(like hledger print -x). This means that any balance assignments in
+explicit (like print -x).
-imported files must be evaluated; but, imported files don't get to see
-the main file's account balances. As a result, importing entries with
-balance assignments (eg from an institution that provides only balances
-and not posting amounts) will probably generate incorrect posting
-amounts. To avoid this problem, use print instead of import:
 
-$ hledger print IMPORTFILE [--new] >> $LEDGER_FILE
+This means that any balance assignments in the imported entries would
+need to be evaluated. But this generally isn't possible, as the main
+file's account balances are not visible during import. So try to avoid
+generating balance assignments with your CSV rules, or importing from a
+journal that contains balance assignments. (Balance assignments are best
+avoided anyway.)
 
-(If you think import should leave amounts implicit like print does,
+But if you must use them, eg because your CSV includes only balances:
-please test it and send a pull request.)
+you can import with print, which leaves implicit amounts implicit.
+(print can also do overlap detection like import, with the --new flag):
+
+$ hledger print --new -f bank.csv >> $LEDGER_FILE
+
+(If you think import should preserve implicit balances, please test that
+and send a pull request.)
 
 Import and commodity styles
 
@@ -131,3 +164,31 @@ journal's canonical commodity styles, as declared by commodity
 directives or inferred from the journal's amounts.
 
 Related: CSV > Amount decimal places.
+
+Import special cases
+
+If you have a download whose file name varies, you could rename it to a
+fixed name after each download. Or you could use a CSV source rule with
+a suitable glob pattern, and import from the .rules file instead of the
+data file.
+
+Here's a situation where you would need to run import with care: say you
+download bank.csv, but forget to import it or delete it. And next month
+you download it again. This time your web browser may save it as
+bank (2).csv. So now each of these may have data not included in the
+other. And a source rule with a glob pattern would match only the most
+recent file. So in this case you should import from each one in turn, in
+the correct order, taking care to use the same filename each time:
+
+$ hledger import bank.csv
+$ mv 'bank (2).csv' bank.csv
+$ hledger import bank.csv
+
+Here are two kinds of "deduplication" which import does not handle (and
+generally should not, since these can happen legitimately in financial
+data):
+
+-   Two or more of the new CSV records are identical, and generate
+    identical new journal entries.
+-   A new CSV record generates a journal entry identical to one(s)
+    already in the journal.

hledger/Hledger/Cli/Commands/Incomestatement.md

@@ -13,8 +13,6 @@ Flags:
                             market price fluctuations)
      --gain                 show unrealised capital gain/loss (historical
                             balance value minus cost basis)
-     --budget               show sum of posting amounts compared to budget
-                            goals defined by periodic transactions
      --count                show the count of postings
      --change               accumulate amounts from column start to column
                             end (in multicolumn reports) (default)

hledger/Hledger/Cli/Commands/Incomestatement.txt

@@ -13,8 +13,6 @@ Flags:
                             market price fluctuations)
      --gain                 show unrealised capital gain/loss (historical
                             balance value minus cost basis)
-     --budget               show sum of posting amounts compared to budget
-                            goals defined by periodic transactions
      --count                show the count of postings
      --change               accumulate amounts from column start to column
                             end (in multicolumn reports) (default)