slaesvuo/hledger
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
10,608 Commits 1 Branch 0 Tags 88 MiB
603b2e9f09
Commit Graph

249 Commits

Author SHA1 Message Date
Simon Michael
13e6c7dfdf lib: drop "assertion" alias for nullassertion 
Perhaps we'll stick with the null* convention. Fixes a warning in Transaction.hs.
 2019-12-09 06:08:03 -08:00
Simon Michael
a0b92e02b9 Revert "remove old fail compatibility imports that are obsolete ?" 
They are needed with ghc <8.4, not for fail but for <> I think.
Try cleaning up again another time.
This reverts commit 318ce7eb07.
 2019-12-01 11:32:51 -08:00
Simon Michael
318ce7eb07 remove old fail compatibility imports that are obsolete ? 2019-12-01 08:31:50 -08:00
Simon Michael
2b2a0b3cf8 ;tests: use "test" alias for testCase everywhere 
Easier to type and read, and a little clearer to my eyes;
"testCase" implies a single case, but it can contain many assertions.
 2019-11-28 15:33:33 -08:00
Simon Michael
b36f6df110 tests: port all unit tests to tasty, second pass (#1090) 
Hledger.Util.Tests helpers have been cleaned up, and test names are
now shown.

Tests have been cleaned up a bit. Some groups of unnamed tests have
been collapsed into a single named test containing a sequence of
assertions. The test command counts named tests, not assertions, so
the reported unit test count has dropped from 199 to 188.
 2019-11-27 13:17:34 -08:00
Dmitry Astapov
aa051a05cf lib: support generation of (un)balanced virtual postings in csv reader 2019-11-23 10:28:10 -08:00
Mykola Orliuk
097486a247 ;lib: re-enable doctests 2019-11-19 03:58:56 -08:00
Dmitry Astapov
c4d650e6ad lib: balance-style -> balance-type 2019-11-13 09:06:25 -08:00
Dmitry Astapov
39bc183340 lib: csv reader gets balance-style directive 2019-11-13 09:06:25 -08:00
Simon Michael
fb3c1a132c csv: fix parsing of whitespace on line after an if block (fix #1120) 2019-11-13 05:41:32 -08:00
Dmitry Astapov
94f912e558 lib: better resolution of csv field aliases ("balance" vs "balance1", etc) 2019-11-12 12:20:09 -08:00
Simon Michael
c4d7cc198a csv: finalise and check journals generated from CSV (#1000) 
Invalid transactions generated from CSV will now be rejected.
I updated some csv tests to avoid this, except for 21, which
probably needs more cleanup.
 2019-11-07 18:58:12 -08:00
Dmitry Astapov
fc001da1ef lib: expenses:unknown is a much better default account name 2019-11-05 21:16:42 +00:00
Dmitry Astapov
c66ccc5cee lib: do not try to balance transaction in csv reader 2019-11-05 21:16:42 +00:00
Dmitry Astapov
95ec5715cc lib: better compatibility code in csv reader 2019-11-05 21:16:42 +00:00
Dmitry Astapov
d62f84bec2 lib: csv reader does not generate postings if account="" 2019-11-05 21:16:42 +00:00
Dmitry Astapov
8d24a40150 lib: csv parser fills out amounts on all postings, if possible 2019-11-05 21:16:42 +00:00
Dmitry Astapov
637741a755 lib: amount1 is no longer magical 2019-11-05 21:16:42 +00:00
Dmitry Astapov
977592e49c lib: typo fix, "expense:unknown" -> "expenses:unknown" 2019-11-05 21:16:42 +00:00
Dmitry Astapov
3c7d5d466d lib, doc, test: csv parser gains "end" command for "if" block 2019-11-05 21:16:42 +00:00
Dmitry Astapov
f92590f92c lib: allow zero postings in csv reader 2019-11-05 21:16:42 +00:00
Dmitry Astapov
1ab8631264 lib: change default rules file text to mention "amount1" 2019-11-05 21:16:42 +00:00
Dmitry Astapov
32cbe4c7b3 lib: better inference for unknown account names in csv parser 2019-11-05 21:16:42 +00:00
Dmitry Astapov
b5d4918c16 lib: allow recursive interpolation of fields with underscores in names 2019-11-05 21:16:42 +00:00
Dmitry Astapov
38db7eb24d lib: implement skip end in csv reader 2019-11-05 21:16:42 +00:00
Dmitry Astapov
b2ba1086b6 lib: fixed validation rules for minimul viable csv rules to include account1 etc 2019-11-05 21:16:42 +00:00
Dmitry Astapov
f1ab107400 lib, doc: extended "skip" in "if" body to "skip N" 2019-11-05 21:16:42 +00:00
Dmitry Astapov
f108b52171 lib: fix for #1001 - empty field assignment consumes next line 2019-11-05 21:16:42 +00:00
Dmitry Astapov
5dac141a7e lib: implement conditional line skips in csv (fixes #1076) 2019-11-05 21:16:42 +00:00
Dmitry Astapov
f74df08e98 lib: fix accidental switch of income:unknown and expence:unknown in csv parser 2019-11-05 21:16:42 +00:00
Dmitry Astapov
77fa81ea4d lib: fix error message formatting (header displayed twice) 2019-11-05 21:16:42 +00:00
Dmitry Astapov
825b9ce5b3 lib: fixed amount vs amount1 conflict detection in csv parser 2019-11-05 21:16:42 +00:00
Dmitry Astapov
881422cd0f lib: fix recursive interpolation in CSV parser for fields with dashes 2019-11-05 21:16:42 +00:00
Dmitry Astapov
e4476dd2f1 lib: more robust "unknown" account assignment in csv parser 2019-11-05 21:16:42 +00:00
Dmitry Astapov
28ca65b99a lib: more explicit conditions for using legacy csv parser 2019-11-05 21:16:42 +00:00
Dmitry Astapov
e4add6df83 lib: fix for multiline descriptions in csv (fixes #841, #416) 2019-11-05 21:16:42 +00:00
Dmitry Astapov
9aab476d53 lib: csv parser supports up to 9 postings. Fixes #570, #627 2019-11-05 21:16:42 +00:00
Simon Michael
eff1b31c86 csv: allow csv records with varying lengths, padding with empties 
Sometimes trailing empty fields are omitted entirely (including the
commas) in CSV records. (I see this in exported Google spreadsheets.)
Now we don't raise an error in this case, instead we automatically pad
any "short" records with empty fields. Not yet well tested.
 2019-10-07 14:54:36 -10:00
Simon Michael
f92b2fe6ef ;lib: csv: refactor: clarify, get rid of some IO/ExceptT 
Rule parsing doesn't need IO (since we are doing expandIncludes as a
pre-parsing step).
 2019-09-14 03:45:37 -07:00
Simon Michael
3c05662ce2 ;lib: csv: clarify confusing imports 2019-09-13 18:11:40 -07:00
Simon Michael
46caa1d4ef ;lib: csv: rename error-prone default rules 2019-09-13 17:51:14 -07:00
Simon Michael
499e20c0b2 lib, cli, ui: start using Control.Monad.Fail, allow base-compat 0.11 
fail is moving out of Monad and into it's own MonadFail class.
This will be enforced in GHC 8.8 (I think).

base-compat/base-compat-batteries 0.11.0 have adapted to this,
and are approaching stackage nightly
(https://github.com/commercialhaskell/stackage/issues/4802).

hledger is now ready to build with base-compat-batteries 0.11.0, once
all of our deps do (eg aeson). We are still compatible with the older
0.10.x and GHC 7.10.3 as well.

For now we are using both fails:

- new fail (from Control.Monad.Fail), used in our parsers, imported
  via base-compat-batteries Control.Monad.Fail.Compat to work with
  older GHC versions.

- old fail (from GHC.Base, exported by Prelude, Control.Monad,
  Control.Monad.State.Strict, Prelude.Compat, ...), used in easytest's
  Test, since I couldn't find their existing fail implementation to update.

To reduce (my) confusion, these are imported carefully, consistently,
and qualified everywhere as Fail.fail and Prelude.fail, with clashing
re-exports suppressed, like so:

import Prelude hiding (fail)
import qualified Prelude (fail)
import Control.Monad.State.Strict hiding (fail)
import "base-compat-batteries" Prelude.Compat hiding (fail)
import qualified "base-compat-batteries" Control.Monad.Fail.Compat as Fail
 2019-09-08 17:13:47 -07:00
Simon Michael
0a3af8d9ee csv: strip outer whitespace when interpolating CSV values (#1051) 
This removes a potential snag in amount field assignments, and
hopefully is harmless and acceptable otherwise.
 2019-06-14 16:43:12 -07:00
Simon Michael
d2b3185062 csv: don't show invalid inter-field spaces in CSV error messages 
Errors involving a record like:

2000-01-01,a,"1"

displayed the record with extra spaces:

the CSV record is:       "2000-01-01", "a", "1"

which was not accurate or valid RFC-4180.
 2019-06-14 16:17:40 -07:00
Simon Michael
90af360792 csv: fix parse error printing, broken since 1.11 (#1038) 2019-06-06 23:10:31 -07:00
Simon Michael
61e8ccbf88 csv: accept a balance field assignment instead of an amount (#1000) 2019-04-03 10:46:26 -07:00
Dmitry Astapov
3c3200fa66 lib: csv parser support for 'balance2', 'balance' renamed to 'balance1' 2019-03-15 16:18:43 -07:00
Simon Michael
c5df73910f csv: update some parse errors which weren't in human format 
cf
https://www.reddit.com/r/plaintextaccounting/comments/axekse/hledger_how_to_create_curved_brackets_with_rule/
 2019-03-05 06:23:11 -08:00
Simon Michael
6f36702141 lib: rename, reposition tprecedingcomment field 2019-01-05 07:35:54 +00:00
Samuel May
e57ef9e9a9 lib: journal: Add support for exact assertions 2018-10-22 09:55:14 -07:00
Samuel May
cde91fc5f4 lib: Make BalanceAssertion a full datatype 
Note: simplifies/moves whitespace parsing out of the balance assertion
parser.
 2018-10-22 09:55:14 -07:00
Alex Chen
3d2584d869 lib: switch to megaparsec 7 2018-09-30 20:15:12 -06:00
Simon Michael
23bdac41d9 csv: merge lucamolteni's cassava/custom separators (squashed) (#829) 
commit 5ba464de761b298e50d57a8b7d14bc28adb30d5d
Author: Luca Molteni <volothamp@gmail.com>
Date:   Fri Sep 7 17:54:12 2018 +0200

    Fix CI 2

commit f060ae9449f4b61a915b0ed4629fc1ba9b66fb4a
Author: Luca Molteni <volothamp@gmail.com>
Date:   Fri Sep 7 17:30:08 2018 +0200

    Fix CI build

commit af0719a33b9b72ad244ae80198d881a1f7145e9d
Author: Luca Molteni <volothamp@gmail.com>
Date:   Fri Sep 7 17:19:01 2018 +0200

    Fix rebase

commit 1a24ddfa54dfb4ff1326e1a51005ffa82d3dc3c8
Author: Luca Molteni <volothamp@gmail.com>
Date:   Fri Aug 10 16:25:24 2018 +0200

    Fixed some GHC warnings

commit 1ac43398a359b5925ef71f53347698f1c6c510ef
Author: Luca Molteni <volothamp@gmail.com>
Date:   Fri Aug 10 16:14:49 2018 +0200

    Fix .cabal

commit 422456b925d8aa4ab3e869f51e98c2b1c3dcde0a
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sun Jul 1 22:56:20 2018 +0200

    Removed to-do list

commit 1118b762e4fd15c4fe7ba48ba86676706ea3a5a5
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sun Jul 1 22:53:28 2018 +0200

    Better test

commit 1146ed0941655668bf7684f18aa15c5f4b9b20c2
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sun Jul 1 15:32:28 2018 +0200

    Fix parsing

commit 4fc2374b2b81802990da30c96756aab54d77399c
Author: Luca Molteni <volothamp@gmail.com>
Date:   Thu Jun 21 22:11:11 2018 +0200

    Parsing of separator

commit f7a61737f1ad4460ba20ca9b2e86eb21468abb33
Author: Luca Molteni <volothamp@gmail.com>
Date:   Thu Jun 21 14:29:23 2018 +0200

    Almost separator in options

commit ac8841cf3b9c80914bc3271ad9b9ff4ae9ba48a7
Author: Luca Molteni <volothamp@gmail.com>
Date:   Thu Jun 21 14:16:59 2018 +0200

    Separator in parseCSV

commit 92a8b9f6ba77ea4237f769641e03029ac88542ea
Author: Luca Molteni <volothamp@gmail.com>
Date:   Thu Jun 21 13:30:41 2018 +0200

    separator option

commit ec417a81ae625647cf35e61776cdf02bdb2c6aea
Author: Luca Molteni <volothamp@gmail.com>
Date:   Thu Jun 21 10:45:26 2018 +0200

    Removed one qualified import

commit 8b2f386c2f780adcd34cff3de7edceacc1d325a7
Author: Luca Molteni <volothamp@gmail.com>
Date:   Wed Jun 20 14:01:12 2018 +0200

    Removed string conversions

commit a14d0e099e28a286bb81770cfc9cb8f5c7e5cf1f
Author: Luca Molteni <volothamp@gmail.com>
Date:   Wed Jun 20 10:23:20 2018 +0200

    custom delimiter in cassava

commit 694d48e2bc1ada0037b90367c017f3082f68ed45
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sun Jun 10 17:51:54 2018 +0200

    Use Text.getContents - remove UTF-8 compatibility library

commit a7ada2cc60033ebdd796ca34cc2ec69a4f387843
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sun Jun 10 17:49:34 2018 +0200

    todo list

commit 58ec47d3987909f6bace50e3e647e30dadd5bf03
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sun Jun 10 17:45:22 2018 +0200

    CSV test now has unicode characters

commit b7851e94c3f1683b63ec7250a12bcde3b7bed691
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sun Jun 10 16:59:39 2018 +0200

    Use decode from Text

commit 79f59fd28ccaca08fcd718fcd8d00b1c1d65d7e1
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sun Jun 10 13:28:57 2018 +0200

    Use Text and Lazy Bytestring

commit 470c9bcb8dc00669beb4ef0303a1e7d9f7aecc89
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sat Jun 9 15:30:22 2018 +0200

    Use megaparsec error

commit f978848ba249ef4f67b855bea5d4e549290c205c
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sat Jun 9 15:22:07 2018 +0200

    Renamed qualify and remove Parsec

commit 152587fde204c43a55798d212e43f37cd3038c2e
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sat Jun 9 15:12:36 2018 +0200

    Use cassava mega parsec

commit cf281577a3d3a071196484a6fc8485f2ea1f7d67
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sat Jun 9 14:01:47 2018 +0200

    Removed Data.Vector

commit 1272e8e758369d8cc5778029a705b277355a5029
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sat Jun 9 12:16:18 2018 +0200

    Removed Parsec ParseError

commit ae07f043135a19307fd65b281ade37a74c76acb2
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sat Jun 9 12:06:14 2018 +0200

    Type sinonim for ParsecError

commit 8e15b253c11bd1c0c35a7641aeb18aa54e0ba9b0
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sat Jun 9 11:16:08 2018 +0200

    Replaced with typeclasses

commit 1ed46f9c175603611325f3d377004e4b85f29377
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sat Jun 9 11:01:33 2018 +0200

    Replaced Text/CSV with Cassava

commit 362f4111b5854145703174b976fc7acbd71b8783
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sat Jun 9 10:34:37 2018 +0200

    Use cassava parsin instead of Text/CSV

commit 83e678e371618687cf7c15a4e2cfa67f570b6b64
Author: Luca Molteni <volothamp@gmail.com>
Date:   Sat Jun 9 08:22:51 2018 +0200

    Text CSV error messages

commit f922df71d274beeacab9fb2530b16c97f005cc08
Author: Luca Molteni <volothamp@gmail.com>
Date:   Fri Jun 8 21:45:20 2018 +0200

    Better types

commit edd130781c84790a53bff2283e6041eb8232e7cf
Author: Luca Molteni <volothamp@gmail.com>
Date:   Fri Jun 8 21:34:59 2018 +0200

    Conversion to Text CSV type

commit 0799383214483018ad2d977a3c8022414959c2b2
Author: Luca Molteni <volothamp@gmail.com>
Date:   Fri Jun 8 16:06:21 2018 +0200

    First function with cassava

commit e92aeb151ff527b383ff3d0ced7764e81b71af82
Author: Luca Molteni <volothamp@gmail.com>
Date:   Fri Jun 8 13:47:34 2018 +0200

    Added cassava as dependency

commit 5ea005c558a3939af7e5f0cd735a9b4da931228e
Author: Luca Molteni <volothamp@gmail.com>
Date:   Fri Jun 8 13:18:47 2018 +0200

    Better .gitignore for multi idea modules
 2018-09-07 12:34:51 -07:00
Simon Michael
91b111b60d rename easytests_* to tests_* 2018-09-06 13:13:26 -07:00
Simon Michael
51618adc37 tests: use is in a few more places 2018-09-04 13:39:21 -07:00
Simon Michael
d1cff46ba7 tests: port CsvReader tests to easytest 2018-09-03 15:30:52 -07:00
Simon Michael
d778a92561 tests: export HUnit/EasyTest from Hledger.Utils.Test; more helpers 2018-08-18 15:19:59 +01:00
Simon Michael
fe73f2a6b5 csv: interpolated field names in values are now case insensitive 
Field names are supposed to be case insensitive, but a field assignment like

fields  ...,Transaction_Date,...
date %Transaction_Date

was failing, because of the capitalised letters. Fixed now.
 2018-07-22 22:26:24 +01:00
Simon Michael
d5430e7ddf clean up debug helpers (api change) 2018-07-16 15:28:58 +01:00
Alex Chen
c5561f25f1 lib: switch parsers to the custom error type 
Also add an explicit export list to `hledger-lib/Hledger/Utils/Parse.hs`
in order to re-export the custom error type (for convenience).
 2018-06-11 13:57:30 -06:00
Simon Michael
0ce9c5728a switch to base-compat-batteries to fix ghc 7.10 support (#794) 
base-compat-batteries provides the same API across more ghc versions
than base-compat does, at the cost of more dependencies. Eg it exports
Prelude.Compat ((<>)) with ghc 7.10/base 4.8, which we expect.
My belief is that several of our deps already require it so the added
cost is not too great. We should probably go back to base-compat when
possible though, eg when we stop supporting ghc 7.10.
 2018-06-04 17:32:42 -07:00
Peter Simons
6db7f800ee hledger-lib: fix doctest suite after recent package updates 
The new version of our package set apparently contains both base-compat and
base-compat-batteries in its transitive closure. This breaks the doctest suite,
which just imports everything into scope when the tests are run, thereby making
module names like Prelude.Compat ambiguous.
 2018-06-04 21:41:15 +02:00
Alex Chen
b245ec7b3d lib: remove the megaparsec compatability module 2018-05-22 12:16:46 -07:00
Simon Michael
f2d3b3e1d4 csv: handle the other double negation cases: -(N), ((N)) (fix #736) 2018-04-18 07:39:33 -07:00
Dmitry Astapov
ecf49b1e4b lib: auto postings generated before amount inference and balance checks (#729) 2018-04-17 14:33:32 -07:00
Everett Hildenbrandt
c69fbff6d9 CsvReader: print more information about amount-in/out errors 2018-04-15 16:52:52 -07:00
Moritz Kiefer
d7b68fbd7d Use skipMany/skipSome for parsing spacenonewline 
This avoids allocating the list of space characters only to then
discard it.
 2018-03-25 22:59:05 +01:00
Simon Michael
13855d6b53 csv: no csv records is no longer an error 2018-02-01 09:41:03 -08:00
Simon Michael
81e964502b lib: refactor low-level text file reading 2018-01-04 16:17:25 -08:00
Nadrieril
9e6e06033f lib: Allow balance-only entries in csv reader 2017-12-14 18:26:29 -08:00
jeevcat
3a9ea65b99 Rewrite of BalanceAssertion type to track its source position. 
Fixes #481.
 2017-10-16 13:25:03 -07:00
Simon Michael
16c07c537b csv: also switch which posting is converted to cost 
After switching the postings, make sure it's the second which is converted
to cost.
 2017-08-15 08:40:52 -07:00
Simon Michael
ce5724fee1 csv: switch the order of generated postings, so account1 is first 
This is subjective, but simplifies things as we'll be adding more than
two postings. Free-form entry templates will allow custom orderings
in future.
 2017-08-15 08:16:49 -07:00
Simon Michael
d7d5f8a064 add support for megaparsec 6 (fixes #594) 
Older megaparsec is still supported.
Also cleans up our custom parser types,
and some text (un)packing is done in different places
(possible performance impact).
 2017-07-27 19:20:46 -07:00
Simon Michael
b3b33336b4 csv: show the "creating/using rules file" message only with --debug 2017-07-06 14:07:46 +01:00
Simon Michael
e94f04311d csv: fix multiple includes in one rules file 2017-07-05 16:08:13 +01:00
Simon Michael
6614aab5d7 csv: add newest-first directive for more robust same-day ordering 2017-07-05 15:24:17 +01:00
Simon Michael
e119941648 rename "uncleared" status to "unmarked" and --uncleared to --unmarked (#564) 
See the issue and linked mail list discussion. Ambiguity between the
uncleared state, and the "not cleared" --uncleared flag causes confusion
and friction. At this point it seems best to break with Ledger and
past hledger, pick a new name and drop --uncleared to put an end to it.
 2017-06-15 19:16:39 -07:00
Simon Michael
5e00d2f31c csv: refactor (#548) 2017-05-14 18:17:56 +09:00
Nicolas Wavrant
309d1ccd59 CsvReader: simplify '-' signs if there exist more than one (#548) 
It prevents hledger to crash later, as it fails to read
amount strings containing more than one '-'

Fix #524
 2017-05-14 09:34:33 +09:00
Simon Michael
48a84cdd58 lib: note an issue with journalfieldnames ordering 2017-04-19 09:05:21 -07:00
Dmitry Astapov
c4c4be5fad Fix parsing of blank/empty balances + testcases (#539) 2017-04-18 09:07:30 -07:00
Dmitry Astapov
451f9d7307 "balance" parser for csv reader (#537) 
* "balance" parser for csv reader

* Some docs for
 2017-04-14 14:52:03 -07:00
Simon Michael
ea1f19c71e lib: csv: refactor, allow writing different rules text 2017-02-06 02:34:18 -08:00
Simon Michael
9cfb7bf6af lib: update readJournalFromCsv doc 2017-02-06 01:45:32 -08:00
Simon Michael
a1411768e5 csv: create default rules file only if CSV reader succeeds 2016-11-22 13:59:31 -08:00
Simon Michael
1f2276c100 lib: mark ledger reader as experimental, don't use automatically 2016-11-20 10:42:12 -08:00
Simon Michael
b6ff170688 lib: simplify format detection, avoid ledger reader by default 
When we don't know a file's format, instead of choosing a subset of
readers based on content sniffing, now we just try them all.
Also, LedgerReader is now used only as a last resort,
as it's not yet competitive with JournalReader.
 2016-11-18 13:24:57 -08:00
Simon Michael
3ddc9d7432 lib: clarify file format detectors 2016-11-16 13:25:33 -08:00
Hans-Peter Deifel
ae73c525d8 Fix csv rules parsing (#407) 
* csv rules: Show prettier parsing errors

This goes from

  hledger: user error ("ParseError {errorPos = SourcePos {sourceName = \"foo.csv.rules\",
  sourceLine = Pos 20, sourceColumn = Pos 1} :| [], errorUnexpected =
  fromList [Tokens (' ' :| \"\")], errorExpected = fromList [Label ('b' :| \"lank or comment
  line\"),EndOfInput], errorCustom = fromList []}")

to

  hledger: user error (foo.csv.rules:20:1:
  unexpected space
  expecting blank or comment line or end of input
  )

* csv rules: Fix parsing of empty field values

A single line containing `account1 ` (note the space at the end) should
parse as assignment of the empty string to account1. At least it did
until commit 4141067.

The problem is that megaparsec's `space` parses multiple space
characters as opposed to parsec. So in the example above it would
incorrectly consume the newline.

This commit also adds a new test case for this bug.
 2016-09-25 12:56:28 -07:00
Simon Michael
72c39470d6 lib: non-journal formats now produce transaction ids #394 
Transactions are now numbered consistently during journal finalisation,
rather than just in the journal reader. Also transaction knot-tying has been
moved out of journalBalanceTransactions.
 2016-08-14 12:44:19 -07:00
Simon Michael
4022f5cb61 lib, web: fix some warnings after megaparsec change 2016-07-29 09:55:02 -07:00
Moritz Kiefer
4141067428 Replace Parsec with Megaparsec (see #289) (#366) 
* Replace Parsec with Megaparsec (see #289)

This builds upon PR #289 by @rasendubi

* Revert renaming of parseWithState to parseWithCtx

* Fix doctests

* Update for Megaparsec 5

* Specialize parser to improve performance

* Pretty print errors

* Swap StateT and ParsecT

This is necessary to get the correct backtracking behavior, i.e. discard
state changes if the parsing fails.
 2016-07-29 08:57:10 -07:00
Simon Michael
f3bf98bfae lib: parentheses trying to resolve IDE warning 2016-05-26 15:51:59 -07:00
Simon Michael
90c9735b7a lib: textification: descriptions & codes 
Slightly higher (with small files) and lower (with large files) maximum
residency, and slightly quicker for all.

hledger -f data/100x100x10.journal stats
<<ghc: 42858472 bytes, 84 GCs, 193712/269608 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.015 elapsed), 0.016 MUT (0.042 elapsed), 0.011 GC (0.119 elapsed) :ghc>>
<<ghc: 42891776 bytes, 84 GCs, 190816/260920 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.004 elapsed), 0.017 MUT (0.025 elapsed), 0.010 GC (0.015 elapsed) :ghc>>

hledger -f data/1000x1000x10.journal stats
<<ghc: 349575240 bytes, 681 GCs, 1396425/4091680 avg/max bytes residency (7 samples), 11M in use, 0.000 INIT (0.000 elapsed), 0.137 MUT (0.146 elapsed), 0.050 GC (0.057 elapsed) :ghc>>
<<ghc: 349927568 bytes, 681 GCs, 1397825/4097248 avg/max bytes residency (7 samples), 11M in use, 0.000 INIT (0.000 elapsed), 0.126 MUT (0.133 elapsed), 0.050 GC (0.057 elapsed) :ghc>>

hledger -f data/10000x1000x10.journal stats
<<ghc: 3424029496 bytes, 6658 GCs, 11403141/41077288 avg/max bytes residency (11 samples), 111M in use, 0.000 INIT (0.000 elapsed), 1.278 MUT (1.310 elapsed), 0.493 GC (0.546 elapsed) :ghc>>
<<ghc: 3427418064 bytes, 6665 GCs, 11127869/37790168 avg/max bytes residency (11 samples), 109M in use, 0.000 INIT (0.001 elapsed), 1.212 MUT (1.229 elapsed), 0.466 GC (0.519 elapsed) :ghc>>

hledger -f data/100000x1000x10.journal stats
<<ghc: 34306546248 bytes, 66727 GCs, 77030638/414617944 avg/max bytes residency (14 samples), 1012M in use, 0.000 INIT (0.000 elapsed), 12.965 MUT (13.164 elapsed), 4.771 GC (5.447 elapsed) :ghc>>
<<ghc: 34340246056 bytes, 66779 GCs, 76983178/416011480 avg/max bytes residency (14 samples), 1011M in use, 0.000 INIT (0.008 elapsed), 12.666 MUT (12.836 elapsed), 4.595 GC (5.175 elapsed) :ghc>>
 2016-05-24 19:00:58 -07:00
Simon Michael
770dcee742 lib: textification: comments and tags 
No change.

hledger -f data/100x100x10.journal stats
<<ghc: 42859576 bytes, 84 GCs, 193781/269984 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.001 elapsed), 0.016 MUT (0.020 elapsed), 0.009 GC (0.011 elapsed) :ghc>>
<<ghc: 42859576 bytes, 84 GCs, 193781/269984 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.001 elapsed), 0.015 MUT (0.018 elapsed), 0.009 GC (0.013 elapsed) :ghc>>

hledger -f data/1000x1000x10.journal stats
<<ghc: 349576344 bytes, 681 GCs, 1407388/4091680 avg/max bytes residency (7 samples), 11M in use, 0.000 INIT (0.000 elapsed), 0.124 MUT (0.130 elapsed), 0.047 GC (0.055 elapsed) :ghc>>
<<ghc: 349576280 bytes, 681 GCs, 1407388/4091680 avg/max bytes residency (7 samples), 11M in use, 0.000 INIT (0.000 elapsed), 0.126 MUT (0.132 elapsed), 0.049 GC (0.058 elapsed) :ghc>>

hledger -f data/10000x1000x10.journal stats
<<ghc: 3424030664 bytes, 6658 GCs, 11403359/41071624 avg/max bytes residency (11 samples), 111M in use, 0.000 INIT (0.000 elapsed), 1.207 MUT (1.228 elapsed), 0.473 GC (0.528 elapsed) :ghc>>
<<ghc: 3424030760 bytes, 6658 GCs, 11403874/41077288 avg/max bytes residency (11 samples), 111M in use, 0.000 INIT (0.002 elapsed), 1.234 MUT (1.256 elapsed), 0.470 GC (0.520 elapsed) :ghc>>

hledger -f data/100000x1000x10.journal stats
<<ghc: 34306547448 bytes, 66727 GCs, 76805504/414629288 avg/max bytes residency (14 samples), 1009M in use, 0.000 INIT (0.003 elapsed), 12.615 MUT (12.813 elapsed), 4.656 GC (5.291 elapsed) :ghc>>
<<ghc: 34306547320 bytes, 66727 GCs, 76805504/414629288 avg/max bytes residency (14 samples), 1009M in use, 0.000 INIT (0.009 elapsed), 12.802 MUT (13.065 elapsed), 4.774 GC (5.441 elapsed) :ghc>>
 2016-05-24 19:00:57 -07:00
Simon Michael
c89c33b36e lib: textification: parse stream 
10% more allocation, but 35% lower maximum residency, and slightly quicker.

hledger -f data/100x100x10.journal stats
<<ghc: 39327768 bytes, 77 GCs, 196834/269496 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.010 elapsed), 0.020 MUT (0.092 elapsed), 0.014 GC (0.119 elapsed) :ghc>>
<<ghc: 42842136 bytes, 84 GCs, 194010/270912 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.009 elapsed), 0.016 MUT (0.029 elapsed), 0.012 GC (0.120 elapsed) :ghc>>

hledger -f data/1000x1000x10.journal stats
<<ghc: 314291440 bytes, 612 GCs, 2070776/6628048 avg/max bytes residency (7 samples), 16M in use, 0.000 INIT (0.000 elapsed), 0.128 MUT (0.144 elapsed), 0.059 GC (0.070 elapsed) :ghc>>
<<ghc: 349558872 bytes, 681 GCs, 1397597/4106384 avg/max bytes residency (7 samples), 11M in use, 0.000 INIT (0.004 elapsed), 0.124 MUT (0.133 elapsed), 0.047 GC (0.053 elapsed) :ghc>>

hledger -f data/10000x1000x10.journal stats
<<ghc: 3070026824 bytes, 5973 GCs, 12698030/62951784 avg/max bytes residency (10 samples), 124M in use, 0.000 INIT (0.002 elapsed), 1.268 MUT (1.354 elapsed), 0.514 GC (0.587 elapsed) :ghc>>
<<ghc: 3424013128 bytes, 6658 GCs, 11405501/41071624 avg/max bytes residency (11 samples), 111M in use, 0.000 INIT (0.001 elapsed), 1.343 MUT (1.406 elapsed), 0.511 GC (0.573 elapsed) :ghc>>

hledger -f data/100000x1000x10.journal stats
<<ghc: 30753387392 bytes, 59811 GCs, 117615462/666703600 avg/max bytes residency (14 samples), 1588M in use, 0.000 INIT (0.000 elapsed), 12.068 MUT (12.238 elapsed), 6.015 GC (7.190 elapsed) :ghc>>
<<ghc: 34306530696 bytes, 66727 GCs, 76806196/414629312 avg/max bytes residency (14 samples), 1009M in use, 0.000 INIT (0.010 elapsed), 14.357 MUT (16.370 elapsed), 5.298 GC (6.534 elapsed) :ghc>>
 2016-05-24 19:00:57 -07:00
Simon Michael
2538d14ea7 lib: textification begins! account names 
The first of several conversions from String to (strict) Text, hopefully
reducing space and time usage.

This one shows a small improvement, with GHC 7.10.3 and text-1.2.2.1:

hledger -f data/100x100x10.journal stats
string: <<ghc: 39471064 bytes, 77 GCs, 198421/275048 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.001 elapsed), 0.015 MUT (0.020 elapsed), 0.010 GC (0.014 elapsed) :ghc>>
text:   <<ghc: 39268024 bytes, 77 GCs, 197018/270840 avg/max bytes residency (3 samples), 2M in use, 0.000 INIT (0.002 elapsed), 0.016 MUT (0.022 elapsed), 0.009 GC (0.011 elapsed) :ghc>>

hledger -f data/1000x100x10.journal stats
string: <<ghc: 318555920 bytes, 617 GCs, 2178997/7134472 avg/max bytes residency (7 samples), 16M in use, 0.000 INIT (0.001 elapsed), 0.129 MUT (0.136 elapsed), 0.067 GC (0.077 elapsed) :ghc>>
text:   <<ghc: 314248496 bytes, 612 GCs, 2074045/6617960 avg/max bytes residency (7 samples), 16M in use, 0.000 INIT (0.003 elapsed), 0.137 MUT (0.145 elapsed), 0.067 GC (0.079 elapsed) :ghc>>

hledger -f data/10000x100x10.journal stats
string: <<ghc: 3114763608 bytes, 6026 GCs, 18858950/75552024 avg/max bytes residency (11 samples), 201M in use, 0.000 INIT (0.000 elapsed), 1.331 MUT (1.372 elapsed), 0.699 GC (0.812 elapsed) :ghc>>
text:   <<ghc: 3071468920 bytes, 5968 GCs, 14120344/62951360 avg/max bytes residency (9 samples), 124M in use, 0.000 INIT (0.003 elapsed), 1.272 MUT (1.349 elapsed), 0.513 GC (0.578 elapsed) :ghc>>

hledger -f data/100000x100x10.journal stats
string: <<ghc: 31186579432 bytes, 60278 GCs, 135332581/740228992 avg/max bytes residency (13 samples), 1697M in use, 0.000 INIT (0.008 elapsed), 14.677 MUT (15.508 elapsed), 7.081 GC (8.074 elapsed) :ghc>>
text:   <<ghc: 30753427672 bytes, 59763 GCs, 117595958/666457240 avg/max bytes residency (14 samples), 1588M in use, 0.000 INIT (0.008 elapsed), 13.713 MUT (13.966 elapsed), 6.220 GC (7.108 elapsed) :ghc>>
 2016-05-24 19:00:49 -07:00
Simon Michael
0f5ee154c4 lib: simplify parsers; cleanups (#275) 
The journal/timeclock/timedot parsers, instead of constructing (opaque)
journal update functions which are later applied to build the journal,
now construct the journal directly (by modifying the parser state). This
is easier to understand and debug. It also removes any possibility of
the journal updates being a space leak. (They weren't, in fact memory
usage is now slightly higher, but that will be addressed in other ways.)

Also:

Journal data and journal parse info have been merged into one type (for
now), and field names are more consistent.

The ParsedJournal type alias has been added to distinguish being-parsed
and finalised journals.

Journal is now a monoid.

stats: fixed an issue with ordering of include files

journal: fixed an issue with ordering of included same-date transactions

timeclock: sessions can no longer span file boundaries (unclocked-out
sessions will be auto-closed at the end of the file).

expandPath now throws a proper IO error (and requires the IO monad).
 2016-05-23 00:44:19 -07:00
Simon Michael
7f5e09096f lib: rename JournalContext to JournalParseState 2016-05-18 20:57:34 -07:00
Simon Michael
84097b75c7 journal: can now include timeclock/timedot files (#320) 
journal files can now include journal, timeclock or timedot files (but
not yet CSV files). Also timeclock/timedot files no longer support
default year directives.

The Hledger.Read.* modules have been reorganised for better reuse.
Hledger.Read.Utils has been renamed Hledger.Read.Common and holds
low-level parsers & utilities; high-level read utilities have moved to
Hledger.Read.
 2016-05-17 19:46:54 -07:00
Simon Michael
bc43036117 lib: use consistent p suffix for parsers 2015-10-17 11:51:45 -07:00
Simon Michael
42d452f99c abstract parsec's SourcePos so as to derive NFData 
The NFData instance helps us time things with criterion.
 2015-08-13 12:56:15 -07:00
Simon Michael
d1f63334ee handle pending status correctly, add --pending (#250) 
A transaction/posting status of ! (pending) was effectively equivalent
to * (cleared). Now it's a separate state, not matched by --cleared.
The new Ledger-compatible --pending flag matches it, and so does
--uncleared. The equivalent search queries are now status:*, status:!
and status: (the old status:1 and status:0 spellings are deprecated).

Since we interpret --uncleared and status: as "any state except cleared",
it's not currently possible to match things which are neither cleared
nor pending.
 2015-05-16 11:51:35 -07:00
Simon Michael
70d87613f2 some cleanup of debug trace helpers 2015-05-14 13:01:49 -07:00
Simon Hengel
964a410b24 hledger-lib: Update for base-compat-0.8.0 (see #245) 2015-04-23 15:41:59 +08:00
Simon Michael
f8a24ccead fix parseTime warnings with time 1.5+ (#239) 2015-03-29 16:12:54 -07:00
Simon Michael
f75849cdd6 fix ghc 7.10 Applicative import warnings (#239) 
Still needed CPP, despite using base-compat.
 2015-03-29 16:09:41 -07:00
Simon Michael
8e50395b7c ErrorT -> ExceptT, handle mtl <2.2.1 && >=2.2.1 (#239) 2015-03-29 14:16:42 -07:00
Simon Michael
e60eb71467 adapt to GHC-7.10's time-1.5 (#239) 2015-03-27 15:42:32 -07:00
Julien Moutinho
af56ced3b0 lib: add eof parsing checks 2015-01-11 09:45:55 -08:00
Simon Michael
9c68944c79 journal, csv: comment lines can also start with * 
As in Ledger. This means you can embed emacs org/outline-mode nodes in
your journal file and manipulate it like an outline.
 2014-12-27 14:41:28 -08:00
Simon Michael
1708f0b441 csv: try to preserve order of same-day transactions 
If the CSV records appear to have been in reverse date order,
we'll now reverse them all before also sorting by transaction date,
so that the original order of same-day transactions is preserved.
We detect this using a simple heuristic: if the first converted
transaction's date is later than the last's.
 2014-12-02 11:16:51 -08:00
Simon Michael
733a7b12ef csv: include path is relative to current (close #198) 2014-12-02 10:50:31 -08:00
Julien Moutinho
cf28985cf2 lib: move from Text.ParserCombinators.Parsec to Text.Parsec 
NOTE: required to use liftIO in includedirective
SEE: http://www.vex.net/~trebla/haskell/parsec-generally.xhtml#IO
 2014-11-20 10:08:30 +01:00
Simon Michael
bfedf367c4 export Regexp types, disambiguate CsvReader's 2014-10-24 14:30:49 -07:00
Simon Michael
d0ad571321 fix manual url in default CSV rules file 2014-08-07 13:15:40 -07:00
Julien Moutinho
a6190420b2 data: add source location to transactions 2014-08-07 16:38:44 +02:00
Simon Michael
40ab1e17f6 amounts cleanups, and support zeros with commodity 2014-07-28 18:45:13 -07:00
Simon Michael
3a16e6cfc7 mostly replace slow regexpr with regex-tdfa (fixes #189) 2014-07-06 14:03:28 -07:00
Simon Michael
0c3148ac7b add an --ignore-assertions flag 
Can be helpful when reading Ledger files, where assertions may have
different semantics; or for getting some answers from your journal
to help you fix your assertions.

Could be called --no-assertions, but this might create surprise when it
has an effect contrary to --no-new-accounts.

I had to add another flag throughout the parsers & journal read
functions, ok for now.
 2014-07-01 18:26:37 -07:00
Simon Michael
cf3d21afef csv and general reader fixes, cleanups 
- The CSV reader no longer writes a "(stdin).rules" file when reading
  from stdin.

- Selection of reader(s) is now smarter when input is coming from stdin.
  Previously, all readers were considered applicable for stdin.  This
  meant that when reading a CSV file from stdin, the journal and timelog
  readers were always tried first, and if the CSV file was unparseable,
  you'd see the first (journal) reader's error instead of the CSV
  reader's.  Now, the readers do some basic content sniffing when
  reading stdin, so it generally tries only the one right reader and
  we'll see the right errors.

- The read system now has more debug output.
 2014-05-09 17:55:32 -07:00
Simon Michael
4740c7082e csv: allow an empty first name in fields list (fixes #178) 2014-05-03 15:05:35 -07:00
Simon Michael
dedd26bbf5 csv: don't count fields in skipped lines (fixes #177) 2014-05-03 14:54:15 -07:00
Simon Michael
3cf53661f3 new debug helpers; --debug=N sets debugLevel 
The debug level set by `--debug[=N]` is now available to pure and
startup code as debugLevel, using unsafePerformIO.

`dbg LABEL ...` is now the go-to helper for tracing values on the
console; it produces output when the debug level is non-zero. `dbgExit`
is similar but exits immediately, avoiding further output. The
`dbgshow`, `dbgppshow` and `dbgpprint` variants allow control over the
pretty-printing method and required debug level, allowing more control
over what is displayed when.

Other cleanups: lstrace -> ltrace, pdbgAt -> pdbg, tracewith -> traceWith.
 2013-12-06 13:35:50 -08:00
Simon Michael
eff1d3f1a5 csv reader: add the include directive, useful for factoring out common rules used with multiple CSV files 2013-08-03 20:53:41 -07:00
Dmitry Astapov
ed58d815d6 Fix for multiple field assignments in CSV parsing 2013-06-19 08:30:33 +01:00
Simon Michael
080eb866ec web: clean up language extensions a bit, make autoweb works again 2013-06-04 18:23:55 -07:00
Simon Michael
44545d6ec7 parsing: update a csv reader error message 2013-06-01 12:38:58 -07:00
Simon Michael
a26ab926d8 parsing: don't fail when a csv amount has trailing whitespace (fixes #113) 2013-06-01 12:38:13 -07:00
Simon Michael
78837c66a6 parsing: fix test breakage due to new csv rules format (fixes #102) 2013-04-12 14:59:28 -07:00
Simon Michael
616a25979a CSV reader version 2 with new rules syntax 
At long last. The main change is a new rules file format that aims to
be more powerful and more intuitive than v1 (hledger 0.19.x and older).
Existing rules files will need to be adapted manually to the new format.
 2013-03-29 22:56:55 +00:00
Simon Michael
621a91807e rename actual/effective dates to primary/secondary 
The command-line flag is now --date2. Alternate spellings --effective and --aux-date
are accepted for compatibility.
 2012-12-06 04:43:41 +00:00
Simon Michael
4aafeb32e6 refactor: clean up Posting construction 2012-12-06 00:03:07 +00:00
Simon Michael
6eda8c4bbf csv reader: append ".rules" to the original file name instead of replacing its extension 2012-11-26 01:56:39 +00:00
Simon Michael
afb4fb0356 csv reader: parse parenthesised amounts as negative 2012-11-26 01:56:01 +00:00
Simon Michael
8b4a99c4d5 79: convert: add a skip-lines directive (Magnus Henoch) 2012-11-18 18:21:52 +00:00
Simon Michael
64180b18ef refactor: clarify that price amounts have only a single commodity 2012-11-19 23:17:55 +00:00
Simon Michael
4567e91409 refactor: move amount display settings out of commodity, simplify amount construction 2012-11-19 21:20:10 +00:00
Simon Michael
2a4d89bb27 expose more utilities from CsvReader 2012-05-29 21:00:49 +00:00
Simon Michael
776ad2a098 remove ensureRulesFile debug trace 2012-05-30 08:36:34 +00:00
Simon Michael
1062e2f9a4 clean up reader selection, don't write a csv rules file on journal parse error 2012-05-28 18:40:36 +00:00
Simon Michael
2fb2aea056 rename metadata fields to tags 2012-05-27 22:59:06 +00:00
Simon Michael
88212f26e8 simplify journal parser names 2012-05-09 15:34:05 +00:00
Simon Michael
8492f6cae4 fix unicode handling on GHC >= 7.2, unify utf8 IO compatibility layer 
tests pass again from GHC 6.12.3 to 7.4.1
 2012-03-29 19:06:31 +00:00
Simon Michael
d4451ce5e3 read system cleanup, require conversion rules from a file to simplify API 2012-03-24 18:08:11 +00:00
Simon Michael
e396c0dc8d push csv rule and format string types down 2012-03-24 01:58:34 +00:00
Simon Michael
6eb7ad28e1 refactor/beef up readJournal/readJournalFile 2012-03-23 16:21:41 +00:00
Simon Michael
4d7a809c4a cleanups and early code for csv reader based on convert 2012-03-10 21:55:48 +00:00