to Hledger.Cli.
Since the git hash changes whenever any file in the repository changes,
this means Hledger.Cli.Version needs to be recompiled all the time.
Since it is at the bottom of the module hierarchy, this means that the
whole hledger package needs to be recompiled. We instead move the
TemplateHaskell splice to one of the top modules, so much less needs to
be recompiled.
Note: Ghc seems to be able to get out of most of the recompiling a lot
of the time (due to caching?), but this makes things more reliable.
This fixes a reported Stored XSS vulnerability in toBloodhoundJson by
encoding the user-controlled values in this payload into base64 and
parsing them with atob.
In my exploration of the vulnerability with various payloads I and
others crafted, it would appear that this is the only available XSS in
hledger-web in relation to stored accounts and transaction details. If
there is other parts of the UI which may contain user-controlled data,
they should be examined for similar things. In this instance,
protections provided by yesod and other libraries worked fine, but in a
bit of code that hledger-web was generating, the user could insert a
</Script> tag (which is valid HTML and equivalent to </script> but not
caught by the T.Replace that existed in toBloodhoundJson) in order to
switch out of a script context, allowing the parser to be reset, and for
arbitrary JavaScript to run.
The real fix is a bit more involved, but produces much better results:
Content-Security-Policy headers should be introduced, and using
sha256-<hash of script> or a different algorithm, they should be marked
as trusted in the header. This way, if the (in-browser) parser and
hledger-web generator disagree on the source code of the script, the
script won't run. Note that this would still be susceptible to attacks
that involve changing the script by escaping from the string inside it
or something similar to that, which can be avoided additionally by using
either the method used in this commit, or a proper JSON encoder.
The second approach has the advantage of preventing further XSS, to the
extent specified above, in practice, a combination of both should be
used, b64 for embedded data and the CSP sha256-hash script-src over
everything else, which will eliminate all injected or malformed script
blocks (via CSP), in combination with eliminating any HTML closing tags
which might occur in stored data (via b64).
This vulnerability appears to have been first introduced when
autocompletion was added in hledger-web, git tag hledger-0.24, commit
hash: ec51d28839
Test payload: </Script><svg onload=alert(1)//>
Closes#1525
rather than as a postprocessing step. (#1638)
This allows us to have a uniform procedure for balancing transactions,
whether they are normal transactions or forecast transactions, including
dealing with balance assignments, balance assertions, and auto postings.
We officially support GHC 8.6+ (and 8.8+ for hledger-web) now.
Hackage matrix builder shows all packages building successfully
with GHC 8.4+, somehow, so we'll adjust the base bound to
allow that but prevent any attempts to build with older GHCs,
style amounts according to that argument. journalAddForecast and
journalTransform now return an Either String Journal.
This improves efficiency, as we no longer have to restyle all amounts in
the journal after generating auto postings or periodic transactions.
Changing the return type of journalAddForecast and journalTransform
reduces partiality.
To get the previous behaviour for modifyTransaction, use modifyTransaction mempty.
This is done to be more consistent with future field naming conventions,
and to make automatic generation of lenses simpler. See discussion in
\#1545.
rsOpts -> _rsReportOpts
rsToday -> _rsDay
rsQuery -> _rsQuery
rsQueryOpts -> _rsQueryOpts
reportq from the ReportSpec, rather than being supplied as a separate
option.
This is the same approach used by the other reports, e.g. EntryReport,
PostingReport, MultiBalanceReport. This reduces code duplication, as
previously the reportq had to be separately tweaked in each of 5
different places.
If you call accountTransactionreport, there is no need to separately
derive the report query.
functions to AccountTransactionsReport.
If you use transactionsReport, you should either use entryReport if you
don't require a running total, or using accountTransactionsReport with
thisacctq as Any or None (depending on what you want included in the
running total).
This produces identical results. The only fields used in the TransactionReport were torig, split, and amt.
- torig is the same as in entriesReport
- since transactionsReport calls accountTransactionsReportItems with
None as thisacctq, all accounts are considered ‘other accounts’, so
amt is always zero, and ($if not split && not (mixedAmountLooksZero amt)) never fires.
- So the only thing used is torig, and we can just get that from the
entriesReport.
In each info manual's DIR-ENTRY metadata, it's best to use just the
filename, with no subdirectory. This should facilitate working info
manuals in packaged versions of hledger, eg in nix.
I don't remember the detail of why I added those and I'm not going to
spend a couple of hours retesting; hopefully no other significant
workflows will be affected.
I have left the subdirectory paths in the dir file - this is intended
for hledger developers and they hopefully don't cause a problem there.
This continues the work begun by #1585. It moves each info manual's
directory metadata to the top of the file, where it needs to be.
I couldn't see a way to do this with pandoc/makeinfo, so the Shake
recipe prepends it from a separate file.
With this metadata in place, the hledger info manuals can be added to
an Info dir file using install-info. Eg:
$ install-info hledger/hledger.info /usr/local/share/info/dir
$ info
Secondly, the Shake "manuals"/"infomanuals" recipes now also generate
an Info dir file in each package directory, so you can also just add
the package directories to your INFOPATH. Eg:
$ export INFOPATH=~/src/hledger/hledger:~/src/hledger/hledger-ui:~/src/hledger/hledger-web:$INFOPATH
$ info
transactions are balanced possibly using explicit prices, but without
inferring any prices. This is included in --strict mode.
Renames check autobalanced to check balancedwithautoconversion.
instead of a list of Amounts. No longer export Mixed constructor, to
keep API clean (if you really need it, you can import it directly from
Hledger.Data.Types). We also ensure the JSON representation of
MixedAmount doesn't change: it is stored as a normalised list of
Amounts.
This commit improves performance. Here are some indicative results.
hledger reg -f examples/10000x1000x10.journal
- Maximum residency decreases from 65MB to 60MB (8% decrease)
- Total memory in use decreases from 178MiB to 157MiB (12% decrease)
hledger reg -f examples/10000x10000x10.journal
- Maximum residency decreases from 69MB to 60MB (13% decrease)
- Total memory in use decreases from 198MiB to 153MiB (23% decrease)
hledger bal -f examples/10000x1000x10.journal
- Total heap usage decreases from 6.4GB to 6.0GB (6% decrease)
- Total memory in use decreases from 178MiB to 153MiB (14% decrease)
hledger bal -f examples/10000x10000x10.journal
- Total heap usage decreases from 7.3GB to 6.9GB (5% decrease)
- Total memory in use decreases from 196MiB to 185MiB (5% decrease)
hledger bal -M -f examples/10000x1000x10.journal
- Total heap usage decreases from 16.8GB to 10.6GB (47% decrease)
- Total time decreases from 14.3s to 12.0s (16% decrease)
hledger bal -M -f examples/10000x10000x10.journal
- Total heap usage decreases from 108GB to 48GB (56% decrease)
- Total time decreases from 62s to 41s (33% decrease)
If you never directly use the constructor Mixed or pattern match against
it then you don't need to make any changes. If you do, then do the
following:
- If you really care about the individual Amounts and never normalise
your MixedAmount (for example, just storing `Mixed amts` and then
extracting `amts` as a pattern match, then use should switch to using
[Amount]. This should just involve removing the `Mixed` constructor.
- If you ever call `mixed`, `normaliseMixedAmount`, or do any sort of
amount arithmetic (+), (-), then you should replace the constructor
`Mixed` with the function `mixed`. To extract the list of Amounts, use
the function `amounts`.
- If you ever call `normaliseMixedAmountSquashPricesForDisplay`, you can
replace that with `mixedAmountStripPrices`. (N.B. this does something
slightly different from `normaliseMixedAmountSquashPricesForDisplay`,
but I don't think there's any use case for squashing prices and then
keeping the first of the squashed prices around. If you disagree let
me know.)
- Any remaining calls to `normaliseMixedAmount` can be removed, as that
is now the identity function.
Prior to this commit,
- hledger still builds with GHC 8.0
- hledger-ui does if you use the build plan specified by stack8.0.yaml,
but you are likely to hit problems if you let cabal pick one
(https://github.com/jtdaugherty/vty/issues/198 and others)
- hledger-web might, if you could find the right build plan
The hassles are enough and GHC 8.0 is old enough (first released in
2016) that I'm letting it go; 8.2 is the new minimum version for all
hledger packages.
This allows a bunch of cleanups to conditional imports, which I leave
for later.
Also, updated the tested-with minor versions.
This was caused by an circular reference in an untested code path
in getViewData, added some months ago (lines 211-212 in 260283e2:
"lib,cli,ui,web: Introduce ReportSpec, which holds ReportOpts, the day of the report, and the parsed Query.").
This kind of bug unfortunately seems to be easy to create in Haskell,
and sometimes quite tedious to track down.
For this particular case, we should add a suitable test to the hledger-web
test suite, and/or set up a pre-release human test script, like hledger-ui's.
The bug was very obvious to the first downloader of hledger-web 1.20,
but it seems over the last three months nobody used a hledger-web built
from master. Providing and encouraging use of nightly builds could also help.
Replaces the partial HashMap.! with List.elemIndex.
(Although the HashMap is more algorithmically efficient,
its API changed recently, possibly complicating building,
and efficiency is just not going to be an issue here.)
* web: Put session file in $XDG_DATA_DIR.
* web: Use $XDG_CACHE_HOME instead of $XDG_DATA_HOME.
* web: Force minimum version of directory (needed for xdg utilities).
* web: Cancel changes to hledger-web.cabal
Co-authored-by: Félix Sipma <felix.sipma@no-log.org>
This is an API change, but it seems better than having additional
colour-supporting variants and trying to avoid duplicated code.
I stopped short of changing showAmount, so cshowAmount still exists.
Decimal 0.5.1+ changed to banker's rounding (round to nearest even
number), and math-functions 0.3.3.0 (used by roi) fixed various
precision-related issues. Now we require the latest versions of these.
This was causing some functional test failures when building with old
GHCs/snapshots.
Renamed: writeValidJournal -> writeJournalTextIfValidAndChanged
Added comments clarifying line ending behaviour of:
add, import, appendToJournalFileOrStdout, readFilePortably,
writeFileWithBackupIfChanged, writeJournalTextIfValidAndChanged
Summary of current behaviour:
- hledger add and import commands will append with (at least some)
unix line endings, possibly causing the file to have mixed line
endings
- hledger-web edit and upload forms will write the file with
the current system's native line endings, ie changing all
line endings if the file previously used foreign line endings.
Remove a bunch of trailing whitespace with M-x whitespace-cleanup.
Except for examples showing hledger output, which might be tested
with shelltest some day, so stripping that whitespace might be problematic.
[ci skip]
This commit introduces IPv6 support (and thus closes#1145).
It also allows using local hostnames as a parameter for --host.
For this, multiple things needed to be changed:
- checkWebOpts is dropped, as the supplied parameter is checked later
by Network.Socket.getAddrInfo
- defbaseurl needs to check if : is used in the host, as this indicates
the usage of an IPv6 address. In this case, the host needs to be
wrapped in [] for the base URL
- To allow opening such a modified base URL, runHostPortFullUrl is used
instead of runhostPortUrl, as it allows opening arbitrary URLs instead
of a path prefixed with http://127.0.0.1
As checking the host for validity is postponed until the webserver tries
to start, an invalid hostname leads to an exception caused by
Network.Socket.getAddrInfo.
This is still fine, as hledger-web won't start in an undefined state, but
will terminate with a nonzero exit code.
This commit adds the --socket option to use hledger-web over an AF_UNIX socket
file.
It allows running multiple instances of hledger-web on the same system without
having to manually choose a port for each instance, which is helpful for running
individual instances for multiple users. In this scenario, the socket path is
predictable, as it can be derived from the username.
It also introduces the following dependencies:
- network
- Used to create the unix domain socket
- unix-compat
- Used to identify if the socket file is still a socket, to reduce the risk
of deleting a file when cleaning up the socket
I think all the non-content changes are due to newer
doc tools and harmless, including this one hopefully:
-.B \f[C]--watch\f[R]
+\f[B]\f[CB]--watch\f[B]\f[R]
[ci skip]
It seems like show (toJSON "a") used to give "\"a\"" instead of
"String \"a\"". I haven't found the root cause, it's possible
that this fix won't work if built with older libs.
- Modified the cors option to require a String
- Moved the logic to build the cors policy to WebOptions.hs
- Specify the --cors "*" example in the cors option help
- Added utf8-string dependency to convert a String into a ByteString
in JournalReader.hs. If you still need this, feel free to work on
those errors. But hopefully not, because dropping base 4.8 should
permit some code cleanups.
dropped journalPrices
renamed Price to AmountPrice, AKA "transaction price"
renamed MarketPrice to PriceDirective.
added new MarketPrice (more pure form of PriceDirective without the amount style information)
Prices is now a more efficient data structure, but not used yet.
These Ledger-ish spellings were dropped over the years, to improve
--help's layout. Now they work again, but are considered hidden flags
so --help doesn't show them automatically (but I decided to mention
them manually in --date2's description).
A single transaction can be added by PUT to /add.
(I read that PUT, not POST, should be used to create;
perhaps the web add form should also use PUT ?)
As with the web form, the `add` capability is required (and enabled by
default).
Here's how to test with curl:
$ curl -s http://127.0.0.1:5000/add -X PUT -H 'Content-Type: application/json' --data-binary @in.json; echo
New readJsonFile/writeJsonFile helpers in Hledger.Web.Json
are handy for generating test data. Eg:
>>> writeJsonFile "in.json" (head $ jtxns samplejournal)
./Shake setversion will now propagate the version number in each
package's .version file, to all the places in that package where it's
needed (ie the defs.m4 file and multiple places in the package.yaml
file. Also it now updates upper bounds on other hledger packages
to the next major version. (The new lower/upper bounds may not always
be what's desired, manual review required.)
[ci skip]
