zebreus/lix
0
0
Fork 0
forked from lix-project/lix
Code Pull requests 1 Activity
lix/tests/functional/lang
History
eldritch horrors 86a1121d16 use byte indexed locations for PosIdx 
we now keep not a table of all positions, but a table of all origins and
their sizes. position indices are now direct pointers into the virtual
concatenation of all parsed contents. this slightly reduces memory usage
and time spent in the parser, at the cost of not being able to report
positions if the total input size exceeds 4GiB. this limit is not unique
to nix though, rustc and clang also limit their input to 4GiB (although
at least clang refuses to process inputs that are larger, we will not).

this new 4GiB limit probably will not cause any problems for quite a
while, all of nixpkgs together is less than 100MiB in size and already
needs over 700MiB of memory and multiple seconds just to parse. 4GiB
worth of input will easily take multiple minutes and over 30GiB of
memory without even evaluating anything. if problems *do* arise we can
probably recover the old table-based system by adding some tracking to
Pos::Origin (or increasing the size of PosIdx outright), but for time
being this looks like more complexity than it's worth.

since we now need to read the entire input again to determine the
line/column of a position we'll make unsafeGetAttrPos slightly lazy:
mostly the set it returns is only used to determine the file of origin
of an attribute, not its exact location. the thunks do not add
measurable runtime overhead.

notably this change is necessary to allow changing the parser since
apparently nothing supports nix's very idiosyncratic line ending choice
of "anything goes", making it very hard to calculate line/column
positions in the parser (while byte offsets are very easy).

(cherry picked from commit 5d9fdab3de0ee17c71369ad05806b9ea06dfceda)
Change-Id: Ie0b2430cb120c09097afa8c0101884d94f4bbf34
2024-03-18 16:12:46 +01:00
..
dir1
dir2
dir3
dir4
readDir
symlink-resolution
binary-data
data
empty.exp
eval-fail-abort.err.exp
eval-fail-abort.nix
eval-fail-addDrvOutputDependencies-empty-context.err.exp
eval-fail-addDrvOutputDependencies-empty-context.nix
eval-fail-addDrvOutputDependencies-multi-elem-context.err.exp
eval-fail-addDrvOutputDependencies-multi-elem-context.nix
eval-fail-addDrvOutputDependencies-wrong-element-kind.err.exp
eval-fail-addDrvOutputDependencies-wrong-element-kind.nix
eval-fail-assert.err.exp
eval-fail-assert.nix
eval-fail-attr-name-type.err.exp
eval-fail-attr-name-type.nix
eval-fail-bad-string-interpolation-1.err.exp
eval-fail-bad-string-interpolation-1.nix
eval-fail-bad-string-interpolation-2.err.exp
eval-fail-bad-string-interpolation-2.nix
eval-fail-bad-string-interpolation-3.err.exp
eval-fail-bad-string-interpolation-3.nix
eval-fail-bad-string-interpolation-4.err.exp libexpr: fix elided value counting in printer 2024-03-14 01:52:19 -06:00
eval-fail-bad-string-interpolation-4.nix
eval-fail-blackhole.err.exp
eval-fail-blackhole.nix
eval-fail-call-primop.err.exp
eval-fail-call-primop.nix
eval-fail-deepseq.err.exp
eval-fail-deepseq.nix
eval-fail-dup-dynamic-attrs.err.exp
eval-fail-dup-dynamic-attrs.nix
eval-fail-dupAttr-deep.err.exp diagnose duplicated attrs at correct path 2024-03-16 22:12:49 +01:00
eval-fail-dupAttr-deep.nix diagnose duplicated attrs at correct path 2024-03-16 22:12:49 +01:00
eval-fail-dupAttr-inherit.err.exp report inherit attr errors at the duplicate name 2024-03-18 16:12:45 +01:00
eval-fail-dupAttr-inherit.nix diagnose duplicated attrs at correct path 2024-03-16 22:12:49 +01:00
eval-fail-dupAttr.err.exp diagnose duplicated attrs at correct path 2024-03-16 22:12:49 +01:00
eval-fail-dupAttr.nix diagnose duplicated attrs at correct path 2024-03-16 22:12:49 +01:00
eval-fail-duplicate-traces.err.exp Merge pull request #10066 from 9999years/print-all-frames 2024-03-09 10:17:26 -07:00
eval-fail-duplicate-traces.nix
eval-fail-eol-1.err.exp match line endings used by parser and error reports 2024-03-18 16:12:46 +01:00
eval-fail-eol-1.nix match line endings used by parser and error reports 2024-03-18 16:12:46 +01:00
eval-fail-eol-2.err.exp match line endings used by parser and error reports 2024-03-18 16:12:46 +01:00
eval-fail-eol-2.nix match line endings used by parser and error reports 2024-03-18 16:12:46 +01:00
eval-fail-eol-3.err.exp match line endings used by parser and error reports 2024-03-18 16:12:46 +01:00
eval-fail-eol-3.nix match line endings used by parser and error reports 2024-03-18 16:12:46 +01:00
eval-fail-foldlStrict-strict-op-application.err.exp Merge pull request #10066 from 9999years/print-all-frames 2024-03-09 10:17:26 -07:00
eval-fail-foldlStrict-strict-op-application.nix
eval-fail-fromTOML-timestamps.err.exp
eval-fail-fromTOML-timestamps.nix
eval-fail-hashfile-missing.err.exp
eval-fail-hashfile-missing.nix
eval-fail-infinite-recursion-lambda.err.exp
eval-fail-infinite-recursion-lambda.nix
eval-fail-list.err.exp
eval-fail-list.nix
eval-fail-missing-arg.err.exp
eval-fail-missing-arg.nix
eval-fail-mutual-recursion.err.exp Merge pull request #10066 from 9999years/print-all-frames 2024-03-09 10:17:26 -07:00
eval-fail-mutual-recursion.nix
eval-fail-nonexist-path.err.exp
eval-fail-nonexist-path.nix
eval-fail-not-throws.err.exp
eval-fail-not-throws.nix
eval-fail-path-slash.err.exp
eval-fail-path-slash.nix
eval-fail-print-limit-list.err.exp libexpr: fix elided value counting in printer 2024-03-14 01:52:19 -06:00
eval-fail-print-limit-list.nix libexpr: fix elided value counting in printer 2024-03-14 01:52:19 -06:00
eval-fail-print-limit-set.err.exp libexpr: fix elided value counting in printer 2024-03-14 01:52:19 -06:00
eval-fail-print-limit-set.nix libexpr: fix elided value counting in printer 2024-03-14 01:52:19 -06:00
eval-fail-recursion.err.exp
eval-fail-recursion.nix
eval-fail-remove.err.exp
eval-fail-remove.nix
eval-fail-scope-5.err.exp
eval-fail-scope-5.nix
eval-fail-seq.err.exp
eval-fail-seq.nix
eval-fail-set-override.err.exp
eval-fail-set-override.nix
eval-fail-set.err.exp
eval-fail-set.nix
eval-fail-substring.err.exp
eval-fail-substring.nix
eval-fail-to-path.err.exp
eval-fail-to-path.nix
eval-fail-toJSON.err.exp
eval-fail-toJSON.nix
eval-fail-undeclared-arg.err.exp
eval-fail-undeclared-arg.nix
eval-fail-using-set-as-attr-name.err.exp
eval-fail-using-set-as-attr-name.nix
eval-okay-any-all.exp
eval-okay-any-all.nix
eval-okay-arithmetic.exp
eval-okay-arithmetic.nix
eval-okay-attrnames.exp
eval-okay-attrnames.nix
eval-okay-attrs.exp
eval-okay-attrs.nix
eval-okay-attrs2.exp
eval-okay-attrs2.nix
eval-okay-attrs3.exp
eval-okay-attrs3.nix
eval-okay-attrs4.exp
eval-okay-attrs4.nix
eval-okay-attrs5.exp
eval-okay-attrs5.nix
eval-okay-attrs6.exp
eval-okay-attrs6.nix
eval-okay-autoargs.exp
eval-okay-autoargs.flags
eval-okay-autoargs.nix
eval-okay-backslash-newline-1.exp
eval-okay-backslash-newline-1.nix
eval-okay-backslash-newline-2.exp
eval-okay-backslash-newline-2.nix
eval-okay-builtins-add.exp
eval-okay-builtins-add.nix
eval-okay-builtins.exp
eval-okay-builtins.nix
eval-okay-callable-attrs.exp
eval-okay-callable-attrs.nix
eval-okay-catattrs.exp
eval-okay-catattrs.nix
eval-okay-closure.exp
eval-okay-closure.exp.xml
eval-okay-closure.nix
eval-okay-comments.exp
eval-okay-comments.nix
eval-okay-concat.exp
eval-okay-concat.nix
eval-okay-concatmap.exp
eval-okay-concatmap.nix
eval-okay-concatstringssep.exp
eval-okay-concatstringssep.nix
eval-okay-context-introspection.exp
eval-okay-context-introspection.nix
eval-okay-context.exp
eval-okay-context.nix
eval-okay-curpos.exp
eval-okay-curpos.nix
eval-okay-deepseq.exp
eval-okay-deepseq.nix
eval-okay-delayed-with-inherit.exp
eval-okay-delayed-with-inherit.nix
eval-okay-delayed-with.exp
eval-okay-delayed-with.nix
eval-okay-dynamic-attrs-2.exp
eval-okay-dynamic-attrs-2.nix
eval-okay-dynamic-attrs-bare.exp
eval-okay-dynamic-attrs-bare.nix
eval-okay-dynamic-attrs.exp
eval-okay-dynamic-attrs.nix
eval-okay-elem.exp
eval-okay-elem.nix
eval-okay-empty-args.exp
eval-okay-empty-args.nix
eval-okay-eq-derivations.exp
eval-okay-eq-derivations.nix
eval-okay-eq.exp
eval-okay-eq.nix
eval-okay-filter.exp
eval-okay-filter.nix
eval-okay-flake-ref-to-string.exp
eval-okay-flake-ref-to-string.nix
eval-okay-flatten.exp
eval-okay-flatten.nix
eval-okay-float.exp
eval-okay-float.nix
eval-okay-floor-ceil.exp
eval-okay-floor-ceil.nix
eval-okay-foldlStrict-lazy-elements.exp
eval-okay-foldlStrict-lazy-elements.nix
eval-okay-foldlStrict-lazy-initial-accumulator.exp
eval-okay-foldlStrict-lazy-initial-accumulator.nix
eval-okay-foldlStrict.exp
eval-okay-foldlStrict.nix
eval-okay-fromjson-escapes.exp
eval-okay-fromjson-escapes.nix
eval-okay-fromjson.exp
eval-okay-fromjson.nix
eval-okay-fromTOML-timestamps.exp
eval-okay-fromTOML-timestamps.flags
eval-okay-fromTOML-timestamps.nix
eval-okay-fromTOML.exp
eval-okay-fromTOML.nix
eval-okay-functionargs.exp
eval-okay-functionargs.exp.xml
eval-okay-functionargs.nix
eval-okay-getattrpos-functionargs.exp
eval-okay-getattrpos-functionargs.nix
eval-okay-getattrpos-undefined.exp
eval-okay-getattrpos-undefined.nix
eval-okay-getattrpos.exp
eval-okay-getattrpos.nix
eval-okay-getenv.exp
eval-okay-getenv.nix
eval-okay-groupBy.exp
eval-okay-groupBy.nix
eval-okay-hash.exp
eval-okay-hashfile.exp
eval-okay-hashfile.nix
eval-okay-hashstring.exp
eval-okay-hashstring.nix
eval-okay-if.exp
eval-okay-if.nix
eval-okay-import.exp
eval-okay-import.nix
eval-okay-ind-string.exp
eval-okay-ind-string.nix
eval-okay-inherit-attr-pos.exp report inherit attr errors at the duplicate name 2024-03-18 16:12:45 +01:00
eval-okay-inherit-attr-pos.nix report inherit attr errors at the duplicate name 2024-03-18 16:12:45 +01:00
eval-okay-inherit-from.err.exp evaluate inherit (from) exprs only once per directive 2024-03-10 03:18:32 -06:00
eval-okay-inherit-from.exp evaluate inherit (from) exprs only once per directive 2024-03-10 03:18:32 -06:00
eval-okay-inherit-from.nix evaluate inherit (from) exprs only once per directive 2024-03-10 03:18:32 -06:00
eval-okay-intersectAttrs.exp
eval-okay-intersectAttrs.nix
eval-okay-let.exp
eval-okay-let.nix
eval-okay-list.exp
eval-okay-list.nix
eval-okay-listtoattrs.exp
eval-okay-listtoattrs.nix
eval-okay-logic.exp
eval-okay-logic.nix
eval-okay-map.exp
eval-okay-map.nix
eval-okay-mapattrs.exp
eval-okay-mapattrs.nix
eval-okay-merge-dynamic-attrs.exp
eval-okay-merge-dynamic-attrs.nix
eval-okay-nested-with.exp
eval-okay-nested-with.nix
eval-okay-new-let.exp
eval-okay-new-let.nix
eval-okay-null-dynamic-attrs.exp
eval-okay-null-dynamic-attrs.nix
eval-okay-overrides.exp
eval-okay-overrides.nix
eval-okay-parse-flake-ref.exp
eval-okay-parse-flake-ref.nix
eval-okay-partition.exp
eval-okay-partition.nix
eval-okay-path-string-interpolation.exp
eval-okay-path-string-interpolation.nix
eval-okay-path.exp
eval-okay-path.nix
eval-okay-pathexists.exp
eval-okay-pathexists.nix
eval-okay-patterns.exp
eval-okay-patterns.nix
eval-okay-print.err.exp
eval-okay-print.exp
eval-okay-print.nix
eval-okay-readDir.exp
eval-okay-readDir.nix
eval-okay-readfile.exp
eval-okay-readfile.nix
eval-okay-readFileType.exp
eval-okay-readFileType.nix
eval-okay-redefine-builtin.exp
eval-okay-redefine-builtin.nix
eval-okay-regex-match.exp
eval-okay-regex-match.nix
eval-okay-regex-split.exp
eval-okay-regex-split.nix
eval-okay-regression-20220122.exp
eval-okay-regression-20220122.nix
eval-okay-regression-20220125.exp
eval-okay-regression-20220125.nix
eval-okay-remove.exp
eval-okay-remove.nix
eval-okay-repeated-empty-attrs.exp
eval-okay-repeated-empty-attrs.nix
eval-okay-repeated-empty-list.exp
eval-okay-repeated-empty-list.nix
eval-okay-replacestrings.exp
eval-okay-replacestrings.nix
eval-okay-scope-1.exp
eval-okay-scope-1.nix
eval-okay-scope-2.exp
eval-okay-scope-2.nix
eval-okay-scope-3.exp
eval-okay-scope-3.nix
eval-okay-scope-4.exp
eval-okay-scope-4.nix
eval-okay-scope-6.exp
eval-okay-scope-6.nix
eval-okay-scope-7.exp
eval-okay-scope-7.nix
eval-okay-search-path.exp
eval-okay-search-path.flags
eval-okay-search-path.nix
eval-okay-seq.exp
eval-okay-seq.nix
eval-okay-sort.exp
eval-okay-sort.nix
eval-okay-splitversion.exp
eval-okay-splitversion.nix
eval-okay-string.exp
eval-okay-string.nix
eval-okay-strings-as-attrs-names.exp
eval-okay-strings-as-attrs-names.nix
eval-okay-substring-context.exp
eval-okay-substring-context.nix
eval-okay-substring.exp
eval-okay-substring.nix
eval-okay-symlink-resolution.exp
eval-okay-symlink-resolution.nix
eval-okay-tail-call-1.exp-disabled
eval-okay-tail-call-1.nix
eval-okay-tojson.exp
eval-okay-tojson.nix
eval-okay-toxml.exp
eval-okay-toxml.nix
eval-okay-toxml2.exp
eval-okay-toxml2.nix
eval-okay-tryeval.exp
eval-okay-tryeval.nix
eval-okay-types.exp
eval-okay-types.nix
eval-okay-unsafeGetAttrPos.exp use byte indexed locations for PosIdx 2024-03-18 16:12:46 +01:00
eval-okay-unsafeGetAttrPos.imported-nix use byte indexed locations for PosIdx 2024-03-18 16:12:46 +01:00
eval-okay-unsafeGetAttrPos.nix use byte indexed locations for PosIdx 2024-03-18 16:12:46 +01:00
eval-okay-versions.exp
eval-okay-versions.nix
eval-okay-with.exp
eval-okay-with.nix
eval-okay-xml.exp.xml
eval-okay-xml.nix
eval-okay-zipAttrsWith.exp
eval-okay-zipAttrsWith.nix
framework.sh
imported.nix
imported2.nix
lib.nix
parse-fail-dup-attrs-1.err.exp keep copies of parser inputs that are in-memory only 2024-03-18 07:56:23 -06:00
parse-fail-dup-attrs-1.nix
parse-fail-dup-attrs-2.err.exp report inherit attr errors at the duplicate name 2024-03-18 16:12:45 +01:00
parse-fail-dup-attrs-2.nix
parse-fail-dup-attrs-3.err.exp report inherit attr errors at the duplicate name 2024-03-18 16:12:45 +01:00
parse-fail-dup-attrs-3.nix
parse-fail-dup-attrs-4.err.exp keep copies of parser inputs that are in-memory only 2024-03-18 07:56:23 -06:00
parse-fail-dup-attrs-4.nix
parse-fail-dup-attrs-7.err.exp report inherit attr errors at the duplicate name 2024-03-18 16:12:45 +01:00
parse-fail-dup-attrs-7.nix
parse-fail-dup-formals.err.exp
parse-fail-dup-formals.nix
parse-fail-eof-in-string.err.exp diagnose "unexpected EOF" at EOF 2024-03-18 16:12:46 +01:00
parse-fail-eof-in-string.nix
parse-fail-eof-pos.err.exp diagnose "unexpected EOF" at EOF 2024-03-18 16:12:46 +01:00
parse-fail-eof-pos.nix diagnose "unexpected EOF" at EOF 2024-03-18 16:12:46 +01:00
parse-fail-mixed-nested-attrs1.err.exp
parse-fail-mixed-nested-attrs1.nix
parse-fail-mixed-nested-attrs2.err.exp
parse-fail-mixed-nested-attrs2.nix
parse-fail-patterns-1.err.exp
parse-fail-patterns-1.nix
parse-fail-regression-20060610.err.exp report inherit attr errors at the duplicate name 2024-03-18 16:12:45 +01:00
parse-fail-regression-20060610.nix
parse-fail-undef-var-2.err.exp keep copies of parser inputs that are in-memory only 2024-03-18 07:56:23 -06:00
parse-fail-undef-var-2.nix
parse-fail-undef-var.err.exp
parse-fail-undef-var.nix
parse-fail-utf8.err.exp keep copies of parser inputs that are in-memory only 2024-03-18 07:56:23 -06:00
parse-fail-utf8.nix
parse-okay-1.exp
parse-okay-1.nix
parse-okay-crlf.exp
parse-okay-crlf.nix
parse-okay-dup-attrs-5.exp
parse-okay-dup-attrs-5.nix
parse-okay-dup-attrs-6.exp
parse-okay-dup-attrs-6.nix
parse-okay-inherits.exp group inherit by source during Expr::show 2024-03-10 03:18:32 -06:00
parse-okay-inherits.nix add test for inherit expr printing 2024-03-10 03:18:32 -06:00
parse-okay-mixed-nested-attrs-1.exp
parse-okay-mixed-nested-attrs-1.nix
parse-okay-mixed-nested-attrs-2.exp
parse-okay-mixed-nested-attrs-2.nix
parse-okay-mixed-nested-attrs-3.exp
parse-okay-mixed-nested-attrs-3.nix
parse-okay-regression-751.exp
parse-okay-regression-751.nix
parse-okay-regression-20041027.exp
parse-okay-regression-20041027.nix
parse-okay-subversion.exp normalize formal order on ExprLambda::show 2024-03-18 07:56:34 -06:00
parse-okay-subversion.nix
parse-okay-url.exp
parse-okay-url.nix