Software /
code /
prosody
Annotate
spec/util_jsonschema_spec.lua @ 12579:ca6a43fe0231 0.12
util.jsonschema: Fix validation to not assume presence of "type" field
MattJ reported a curious issue where validation did not work as
expected. Primarily that the "type" field was expected to be mandatory,
and thus leaving it out would result in no checks being performed.
This was likely caused by misreading during initial development.
Spent some time testing against
https://github.com/json-schema-org/JSON-Schema-Test-Suite.git and
discovered a multitude of issues, far too many to bother splitting into
separate commits.
More than half of them fail. Many because of features not implemented,
which have been marked NYI. For example, some require deep comparisons
e.g. when objects or arrays are present in enums fields.
Some because of quirks with how Lua differs from JavaScript, e.g. no
distinct array or object types. Tests involving fractional floating
point numbers. We're definitely not going to follow references to remote
resources. Or deal with UTF-16 sillyness. One test asserted that 1.0 is
an integer, where Lua 5.3+ will disagree.
author | Kim Alvefur <zash@zash.se> |
---|---|
date | Fri, 08 Jul 2022 14:38:23 +0200 |
child | 12756:cd7da871ce10 |
rev | line source |
---|---|
12579
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
1 local js = require "util.jsonschema"; |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
2 local json = require "util.json"; |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
3 local lfs = require "lfs"; |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
4 |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
5 -- https://github.com/json-schema-org/JSON-Schema-Test-Suite.git 2.0.0-550-g88d6948 |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
6 local test_suite_dir = "spec/JSON-Schema-Test-Suite/tests/draft2020-12" |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
7 if lfs.attributes(test_suite_dir, "mode") ~= "directory" then return end |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
8 |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
9 -- Tests to skip and short reason why (NYI = not yet implemented) |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
10 local skip = { |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
11 ["ref.json:0:3"] = "NYI additionalProperties"; |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
12 ["ref.json:3:2"] = "FIXME investigate, util.jsonpath issue?", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
13 ["ref.json:6:1"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
14 ["required.json:0:2"] = "distinguishing objects from arrays", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
15 ["additionalProperties.json:0:2"] = "distinguishing objects from arrays", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
16 ["additionalProperties.json:0:5"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
17 ["additionalProperties.json:1:0"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
18 ["anchor.json"] = "$anchor NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
19 ["const.json:1"] = "deepcompare", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
20 ["const.json:13:2"] = "IEEE 754 equality", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
21 ["const.json:2"] = "deepcompare", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
22 ["const.json:8"] = "deepcompare", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
23 ["const.json:9"] = "deepcompare", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
24 ["contains.json:0:5"] = "distinguishing objects from arrays", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
25 ["defs.json"] = "need built-in meta-schema", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
26 ["dependentRequired.json"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
27 ["dependentSchemas.json"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
28 ["dynamicRef.json"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
29 ["enum.json:1:3"] = "deepcompare", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
30 ["id.json"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
31 ["maxContains.json"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
32 ["maxLength.json:0:4"] = "UTF-16", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
33 ["maxProperties.json"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
34 ["minContains.json"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
35 ["minLength.json:0:4"] = "UTF-16", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
36 ["minProperties.json"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
37 ["multipleOf.json:1"] = "multiples of IEEE 754 fractions", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
38 ["multipleOf.json:2"] = "multiples of IEEE 754 fractions", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
39 ["pattern.json"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
40 ["patternProperties.json"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
41 ["properties.json:1:2"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
42 ["properties.json:1:3"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
43 ["ref.json:14"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
44 ["ref.json:15"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
45 ["ref.json:16"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
46 ["ref.json:17"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
47 ["ref.json:18"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
48 ["ref.json:13"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
49 ["ref.json:19"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
50 ["ref.json:11"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
51 ["ref.json:12:1"] = "FIXME", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
52 ["refRemote.json"] = "DEFINITELY NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
53 ["type.json:3:4"] = "distinguishing objects from arrays", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
54 ["type.json:3:6"] = "null is weird", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
55 ["type.json:4:3"] = "distinguishing objects from arrays", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
56 ["type.json:4:6"] = "null is weird", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
57 ["type.json:9:4"] = "null is weird", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
58 ["type.json:9:6"] = "null is weird", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
59 ["unevaluatedItems.json"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
60 ["unevaluatedProperties.json"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
61 ["uniqueItems.json:0:11"] = "deepcompare", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
62 ["uniqueItems.json:0:13"] = "deepcompare", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
63 ["uniqueItems.json:0:14"] = "deepcompare", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
64 ["uniqueItems.json:0:22"] = "deepcompare", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
65 ["uniqueItems.json:0:24"] = "deepcompare", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
66 ["uniqueItems.json:0:9"] = "deepcompare", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
67 ["unknownKeyword.json"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
68 ["vocabulary.json"] = "NYI", |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
69 }; |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
70 |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
71 local function label(s, i) |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
72 return string.format("%s:%d", s, i-1); |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
73 end |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
74 |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
75 describe("util.jsonschema.validate", function() |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
76 for test_case_file in lfs.dir(test_suite_dir) do |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
77 -- print(skip[test_case_file] and "do " or "skip", test_case_file) |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
78 if test_case_file:sub(-5) == ".json" and not skip[test_case_file] then |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
79 describe(test_case_file, function() |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
80 local test_cases; |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
81 setup(function() |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
82 local f = assert(io.open(test_suite_dir .. "/" .. test_case_file)); |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
83 local rawdata = assert(f:read("*a"), "failed to read " .. test_case_file) |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
84 test_cases = assert(json.decode(rawdata), "failed to parse " .. test_case_file) |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
85 end) |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
86 describe("tests", function() |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
87 for i, schema_test in ipairs(test_cases) do |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
88 local generic_label = label(test_case_file, i); |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
89 describe(schema_test.description or generic_label, function() |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
90 for j, test in ipairs(schema_test.tests) do |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
91 local specific_label = label(generic_label, j); |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
92 ((skip[generic_label] or skip[specific_label]) and pending or it)(test.description, function() |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
93 assert.equal(test.valid, js.validate(schema_test.schema, test.data), specific_label .. " " .. test.description); |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
94 end) |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
95 end |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
96 end) |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
97 end |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
98 end) |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
99 end) |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
100 end |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
101 end |
ca6a43fe0231
util.jsonschema: Fix validation to not assume presence of "type" field
Kim Alvefur <zash@zash.se>
parents:
diff
changeset
|
102 end); |