Annotate

spec/util_jsonschema_spec.lua @ 12579:ca6a43fe0231 0.12

util.jsonschema: Fix validation to not assume presence of "type" field MattJ reported a curious issue where validation did not work as expected. Primarily that the "type" field was expected to be mandatory, and thus leaving it out would result in no checks being performed. This was likely caused by misreading during initial development. Spent some time testing against https://github.com/json-schema-org/JSON-Schema-Test-Suite.git and discovered a multitude of issues, far too many to bother splitting into separate commits. More than half of them fail. Many because of features not implemented, which have been marked NYI. For example, some require deep comparisons e.g. when objects or arrays are present in enums fields. Some because of quirks with how Lua differs from JavaScript, e.g. no distinct array or object types. Tests involving fractional floating point numbers. We're definitely not going to follow references to remote resources. Or deal with UTF-16 sillyness. One test asserted that 1.0 is an integer, where Lua 5.3+ will disagree.
author Kim Alvefur <zash@zash.se>
date Fri, 08 Jul 2022 14:38:23 +0200
child 12756:cd7da871ce10
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
12579
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
1 local js = require "util.jsonschema";
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
2 local json = require "util.json";
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
3 local lfs = require "lfs";
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
4
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
5 -- https://github.com/json-schema-org/JSON-Schema-Test-Suite.git 2.0.0-550-g88d6948
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
6 local test_suite_dir = "spec/JSON-Schema-Test-Suite/tests/draft2020-12"
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
7 if lfs.attributes(test_suite_dir, "mode") ~= "directory" then return end
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
8
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
9 -- Tests to skip and short reason why (NYI = not yet implemented)
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
10 local skip = {
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
11 ["ref.json:0:3"] = "NYI additionalProperties";
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
12 ["ref.json:3:2"] = "FIXME investigate, util.jsonpath issue?",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
13 ["ref.json:6:1"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
14 ["required.json:0:2"] = "distinguishing objects from arrays",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
15 ["additionalProperties.json:0:2"] = "distinguishing objects from arrays",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
16 ["additionalProperties.json:0:5"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
17 ["additionalProperties.json:1:0"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
18 ["anchor.json"] = "$anchor NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
19 ["const.json:1"] = "deepcompare",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
20 ["const.json:13:2"] = "IEEE 754 equality",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
21 ["const.json:2"] = "deepcompare",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
22 ["const.json:8"] = "deepcompare",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
23 ["const.json:9"] = "deepcompare",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
24 ["contains.json:0:5"] = "distinguishing objects from arrays",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
25 ["defs.json"] = "need built-in meta-schema",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
26 ["dependentRequired.json"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
27 ["dependentSchemas.json"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
28 ["dynamicRef.json"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
29 ["enum.json:1:3"] = "deepcompare",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
30 ["id.json"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
31 ["maxContains.json"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
32 ["maxLength.json:0:4"] = "UTF-16",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
33 ["maxProperties.json"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
34 ["minContains.json"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
35 ["minLength.json:0:4"] = "UTF-16",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
36 ["minProperties.json"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
37 ["multipleOf.json:1"] = "multiples of IEEE 754 fractions",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
38 ["multipleOf.json:2"] = "multiples of IEEE 754 fractions",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
39 ["pattern.json"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
40 ["patternProperties.json"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
41 ["properties.json:1:2"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
42 ["properties.json:1:3"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
43 ["ref.json:14"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
44 ["ref.json:15"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
45 ["ref.json:16"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
46 ["ref.json:17"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
47 ["ref.json:18"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
48 ["ref.json:13"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
49 ["ref.json:19"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
50 ["ref.json:11"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
51 ["ref.json:12:1"] = "FIXME",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
52 ["refRemote.json"] = "DEFINITELY NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
53 ["type.json:3:4"] = "distinguishing objects from arrays",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
54 ["type.json:3:6"] = "null is weird",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
55 ["type.json:4:3"] = "distinguishing objects from arrays",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
56 ["type.json:4:6"] = "null is weird",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
57 ["type.json:9:4"] = "null is weird",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
58 ["type.json:9:6"] = "null is weird",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
59 ["unevaluatedItems.json"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
60 ["unevaluatedProperties.json"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
61 ["uniqueItems.json:0:11"] = "deepcompare",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
62 ["uniqueItems.json:0:13"] = "deepcompare",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
63 ["uniqueItems.json:0:14"] = "deepcompare",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
64 ["uniqueItems.json:0:22"] = "deepcompare",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
65 ["uniqueItems.json:0:24"] = "deepcompare",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
66 ["uniqueItems.json:0:9"] = "deepcompare",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
67 ["unknownKeyword.json"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
68 ["vocabulary.json"] = "NYI",
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
69 };
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
70
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
71 local function label(s, i)
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
72 return string.format("%s:%d", s, i-1);
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
73 end
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
74
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
75 describe("util.jsonschema.validate", function()
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
76 for test_case_file in lfs.dir(test_suite_dir) do
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
77 -- print(skip[test_case_file] and "do " or "skip", test_case_file)
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
78 if test_case_file:sub(-5) == ".json" and not skip[test_case_file] then
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
79 describe(test_case_file, function()
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
80 local test_cases;
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
81 setup(function()
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
82 local f = assert(io.open(test_suite_dir .. "/" .. test_case_file));
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
83 local rawdata = assert(f:read("*a"), "failed to read " .. test_case_file)
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
84 test_cases = assert(json.decode(rawdata), "failed to parse " .. test_case_file)
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
85 end)
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
86 describe("tests", function()
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
87 for i, schema_test in ipairs(test_cases) do
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
88 local generic_label = label(test_case_file, i);
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
89 describe(schema_test.description or generic_label, function()
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
90 for j, test in ipairs(schema_test.tests) do
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
91 local specific_label = label(generic_label, j);
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
92 ((skip[generic_label] or skip[specific_label]) and pending or it)(test.description, function()
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
93 assert.equal(test.valid, js.validate(schema_test.schema, test.data), specific_label .. " " .. test.description);
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
94 end)
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
95 end
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
96 end)
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
97 end
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
98 end)
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
99 end)
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
100 end
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
101 end
ca6a43fe0231 util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff changeset
102 end);