doc: add a README for the functional tests, linked from contrib guide
This commit is contained in:
		
							parent
							
								
									4444b8bc8f
								
							
						
					
					
						commit
						4fffe74408
					
				| @ -774,6 +774,7 @@ About testing in the hledger project, as of 201809. | |||||||
|     Tests are defined in files under |     Tests are defined in files under | ||||||
|     [tests/](https://github.com/simonmichael/hledger/tree/master/tests), |     [tests/](https://github.com/simonmichael/hledger/tree/master/tests), | ||||||
|     grouped by *component* (command or topic name). |     grouped by *component* (command or topic name). | ||||||
|  |     For more about these, see the README there. | ||||||
| 
 | 
 | ||||||
| 4.  Code tests | 4.  Code tests | ||||||
| 
 | 
 | ||||||
|  | |||||||
							
								
								
									
										78
									
								
								tests/README.md
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										78
									
								
								tests/README.md
									
									
									
									
									
										Normal file
									
								
							| @ -0,0 +1,78 @@ | |||||||
|  | hledger's functional tests. | ||||||
|  | See also [Contributor Guide: Tests]. | ||||||
|  | 
 | ||||||
|  | These mainly test the hledger CLI and (indirectly) hledger-lib. | ||||||
|  | They are organised roughly by [component]. | ||||||
|  | 
 | ||||||
|  | Running these requires [shelltestrunner]. | ||||||
|  | Older test files are in [format 1]; newer ones use [format 3] (preferred). | ||||||
|  | Some tests invoke unix commands so will not run in a Windows CMD shell. | ||||||
|  | 
 | ||||||
|  | [Contributor Guide: Tests]: https://hledger.org/CONTRIBUTING.html#tests | ||||||
|  | [component]: https://hledger.org/CONTRIBUTING.html#components | ||||||
|  | [shelltestrunner]: https://github.com/simonmichael/shelltestrunner#readme | ||||||
|  | [format 1]: https://github.com/simonmichael/shelltestrunner#format-1 | ||||||
|  | [format 3]: https://github.com/simonmichael/shelltestrunner#format-3 | ||||||
|  | 
 | ||||||
|  | 
 | ||||||
|  | Run them all (also builds hledger): | ||||||
|  | 
 | ||||||
|  |     make functest | ||||||
|  | 
 | ||||||
|  | See how the Makefile is invoking shelltestrunner: | ||||||
|  | 
 | ||||||
|  |     $ make functest -n | ||||||
|  |     stack build --fast hledger | ||||||
|  |     (COLUMNS=80 stack exec -- shelltest --execdir -j16 --hide-successes --exclude=/_ -w `stack exec -- which hledger` tests \ | ||||||
|  |             && echo functest PASSED) || (echo functest FAILED; false) | ||||||
|  | 
 | ||||||
|  | These are the most important: | ||||||
|  | 
 | ||||||
|  | - `COLUMNS=80` makes output independent of your terminal width. | ||||||
|  | - `--execdir` runs each test within its own directory. | ||||||
|  | - ``-w `stack exec -- which hledger` `` ensures you are testing the hledger executable that was just built. | ||||||
|  | - `-j16` runs tests in parallel which is much faster. | ||||||
|  | 
 | ||||||
|  | Run only the tests matching a regular expression: | ||||||
|  | 
 | ||||||
|  |     $ COLUMNS=80 shelltest --execdir -w `stack exec -- which hledger` tests -i balance-assertions.*19 | ||||||
|  |     :tests/journal/balance-assertions.test:19: [OK] | ||||||
|  | 
 | ||||||
|  |              Test Cases  Total       | ||||||
|  |      Passed  1           1           | ||||||
|  |      Failed  0           0           | ||||||
|  |      Total   1           1           | ||||||
|  | 
 | ||||||
|  | Run only the tests in one file: | ||||||
|  | 
 | ||||||
|  |     $ COLUMNS=80 shelltest --execdir -w `stack exec -- which hledger` tests/cli/query-args.test | ||||||
|  |     :tests/cli/query-args.test:1: [OK] | ||||||
|  |     :tests/cli/query-args.test:2: [OK] | ||||||
|  |     :tests/cli/query-args.test:3: [OK] | ||||||
|  | 
 | ||||||
|  |              Test Cases  Total       | ||||||
|  |      Passed  3           3           | ||||||
|  |      Failed  0           0           | ||||||
|  |      Total   3           3           | ||||||
|  | 
 | ||||||
|  | Run a test [repeatedly](http://eradman.com/entrproject/) as its file is changed: | ||||||
|  | 
 | ||||||
|  |     $ ls tests/cli/query-args.test | entr bash -c "COLUMNS=80 shelltest --execdir -w `stack exec -- which hledger` tests/cli/query-args.test -i1" | ||||||
|  |     :tests/cli/query-args.test:1: [OK] | ||||||
|  | 
 | ||||||
|  |              Test Cases  Total       | ||||||
|  |      Passed  1           1           | ||||||
|  |      Failed  0           0           | ||||||
|  |      Total   1           1           | ||||||
|  |     :tests/cli/query-args.test:1: [OK] | ||||||
|  | 
 | ||||||
|  |              Test Cases  Total       | ||||||
|  |      Passed  1           1           | ||||||
|  |      Failed  0           0           | ||||||
|  |      Total   1           1           | ||||||
|  |       C-c C-c | ||||||
|  | 
 | ||||||
|  | More shelltestrunner options: | ||||||
|  | 
 | ||||||
|  |     $ shelltest --help | ||||||
|  | 
 | ||||||
		Loading…
	
		Reference in New Issue
	
	Block a user