Annotate

util/http.lua @ 13801:a5d5fefb8b68 13.0

mod_tls: Enable Prosody's certificate checking for incoming s2s connections (fixes #1916) (thanks Damian, Zash) Various options in Prosody allow control over the behaviour of the certificate verification process For example, some deployments choose to allow falling back to traditional "dialback" authentication (XEP-0220), while others verify via DANE, hard-coded fingerprints, or other custom plugins. Implementing this flexibility requires us to override OpenSSL's default certificate verification, to allow Prosody to verify the certificate itself, apply custom policies and make decisions based on the outcome. To enable our custom logic, we have to suppress OpenSSL's default behaviour of aborting the connection with a TLS alert message. With LuaSec, this can be achieved by using the verifyext "lsec_continue" flag. We also need to use the lsec_ignore_purpose flag, because XMPP s2s uses server certificates as "client" certificates (for mutual TLS verification in outgoing s2s connections). Commit 99d2100d2918 moved these settings out of the defaults and into mod_s2s, because we only really need these changes for s2s, and they should be opt-in, rather than automatically applied to all TLS services we offer. That commit was incomplete, because it only added the flags for incoming direct TLS connections. StartTLS connections are handled by mod_tls, which was not applying the lsec_* flags. It previously worked because they were already in the defaults. This resulted in incoming s2s connections with "invalid" certificates being aborted early by OpenSSL, even if settings such as `s2s_secure_auth = false` or DANE were present in the config. Outgoing s2s connections inherit verify "none" from the defaults, which means OpenSSL will receive the cert but will not terminate the connection when it is deemed invalid. This means we don't need lsec_continue there, and we also don't need lsec_ignore_purpose (because the remote peer is a "server"). Wondering why we can't just use verify "none" for incoming s2s? It's because in that mode, OpenSSL won't request a certificate from the peer for incoming connections. Setting verify "peer" is how you ask OpenSSL to request a certificate from the client, but also what triggers its built-in verification.
author Matthew Wild <mwild1@gmail.com>
date Tue, 01 Apr 2025 17:26:56 +0100
parent 13124:f15e23840780
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
5299
cc9d460aa779 util.http: New module for HTTP helper functions
Florian Zeitz <florob@babelmonkeys.de>
parents:
diff changeset
1 -- Prosody IM
cc9d460aa779 util.http: New module for HTTP helper functions
Florian Zeitz <florob@babelmonkeys.de>
parents:
diff changeset
2 -- Copyright (C) 2013 Florian Zeitz
cc9d460aa779 util.http: New module for HTTP helper functions
Florian Zeitz <florob@babelmonkeys.de>
parents:
diff changeset
3 --
cc9d460aa779 util.http: New module for HTTP helper functions
Florian Zeitz <florob@babelmonkeys.de>
parents:
diff changeset
4 -- This project is MIT/X11 licensed. Please see the
cc9d460aa779 util.http: New module for HTTP helper functions
Florian Zeitz <florob@babelmonkeys.de>
parents:
diff changeset
5 -- COPYING file in the source package for more information.
cc9d460aa779 util.http: New module for HTTP helper functions
Florian Zeitz <florob@babelmonkeys.de>
parents:
diff changeset
6 --
cc9d460aa779 util.http: New module for HTTP helper functions
Florian Zeitz <florob@babelmonkeys.de>
parents:
diff changeset
7
5471
34bfd26525f5 util.http: Refactor and import all necessary functions
Matthew Wild <mwild1@gmail.com>
parents: 5458
diff changeset
8 local format, char = string.format, string.char;
9759
1af5106a2c34 util.http: Pre-generate urlencoding mappings (optimization)
Kim Alvefur <zash@zash.se>
parents: 9504
diff changeset
9 local pairs, ipairs = pairs, ipairs;
5471
34bfd26525f5 util.http: Refactor and import all necessary functions
Matthew Wild <mwild1@gmail.com>
parents: 5458
diff changeset
10 local t_insert, t_concat = table.insert, table.concat;
5299
cc9d460aa779 util.http: New module for HTTP helper functions
Florian Zeitz <florob@babelmonkeys.de>
parents:
diff changeset
11
9759
1af5106a2c34 util.http: Pre-generate urlencoding mappings (optimization)
Kim Alvefur <zash@zash.se>
parents: 9504
diff changeset
12 local url_codes = {};
1af5106a2c34 util.http: Pre-generate urlencoding mappings (optimization)
Kim Alvefur <zash@zash.se>
parents: 9504
diff changeset
13 for i = 0, 255 do
1af5106a2c34 util.http: Pre-generate urlencoding mappings (optimization)
Kim Alvefur <zash@zash.se>
parents: 9504
diff changeset
14 local c = char(i);
1af5106a2c34 util.http: Pre-generate urlencoding mappings (optimization)
Kim Alvefur <zash@zash.se>
parents: 9504
diff changeset
15 local u = format("%%%02x", i);
1af5106a2c34 util.http: Pre-generate urlencoding mappings (optimization)
Kim Alvefur <zash@zash.se>
parents: 9504
diff changeset
16 url_codes[c] = u;
1af5106a2c34 util.http: Pre-generate urlencoding mappings (optimization)
Kim Alvefur <zash@zash.se>
parents: 9504
diff changeset
17 url_codes[u] = c;
9785
ff88b03c343f util.http: Fix decoding of uppercase URL encoded chars
Kim Alvefur <zash@zash.se>
parents: 9759
diff changeset
18 url_codes[u:upper()] = c;
9759
1af5106a2c34 util.http: Pre-generate urlencoding mappings (optimization)
Kim Alvefur <zash@zash.se>
parents: 9504
diff changeset
19 end
5471
34bfd26525f5 util.http: Refactor and import all necessary functions
Matthew Wild <mwild1@gmail.com>
parents: 5458
diff changeset
20 local function urlencode(s)
9759
1af5106a2c34 util.http: Pre-generate urlencoding mappings (optimization)
Kim Alvefur <zash@zash.se>
parents: 9504
diff changeset
21 return s and (s:gsub("[^a-zA-Z0-9.~_-]", url_codes));
5458
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
22 end
5471
34bfd26525f5 util.http: Refactor and import all necessary functions
Matthew Wild <mwild1@gmail.com>
parents: 5458
diff changeset
23 local function urldecode(s)
9759
1af5106a2c34 util.http: Pre-generate urlencoding mappings (optimization)
Kim Alvefur <zash@zash.se>
parents: 9504
diff changeset
24 return s and (s:gsub("%%%x%x", url_codes));
5458
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
25 end
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
26
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
27 local function _formencodepart(s)
9759
1af5106a2c34 util.http: Pre-generate urlencoding mappings (optimization)
Kim Alvefur <zash@zash.se>
parents: 9504
diff changeset
28 return s and (urlencode(s):gsub("%%20", "+"));
5458
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
29 end
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
30
5471
34bfd26525f5 util.http: Refactor and import all necessary functions
Matthew Wild <mwild1@gmail.com>
parents: 5458
diff changeset
31 local function formencode(form)
5458
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
32 local result = {};
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
33 if form[1] then -- Array of ordered { name, value }
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
34 for _, field in ipairs(form) do
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
35 t_insert(result, _formencodepart(field.name).."=".._formencodepart(field.value));
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
36 end
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
37 else -- Unordered map of name -> value
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
38 for name, value in pairs(form) do
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
39 t_insert(result, _formencodepart(name).."=".._formencodepart(value));
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
40 end
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
41 end
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
42 return t_concat(result, "&");
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
43 end
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
44
5471
34bfd26525f5 util.http: Refactor and import all necessary functions
Matthew Wild <mwild1@gmail.com>
parents: 5458
diff changeset
45 local function formdecode(s)
5458
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
46 if not s:match("=") then return urldecode(s); end
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
47 local r = {};
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
48 for k, v in s:gmatch("([^=&]*)=([^&]*)") do
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
49 k, v = k:gsub("%+", "%%20"), v:gsub("%+", "%%20");
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
50 k, v = urldecode(k), urldecode(v);
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
51 t_insert(r, { name = k, value = v });
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
52 r[k] = v;
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
53 end
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
54 return r;
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
55 end
84162b81c863 net.http, util.http: Move definitions of urlencode/decode and formencode/decode to util.http (possible to use them without unnecessary network-related dependencies)
Matthew Wild <mwild1@gmail.com>
parents: 5299
diff changeset
56
5471
34bfd26525f5 util.http: Refactor and import all necessary functions
Matthew Wild <mwild1@gmail.com>
parents: 5458
diff changeset
57 local function contains_token(field, token)
5299
cc9d460aa779 util.http: New module for HTTP helper functions
Florian Zeitz <florob@babelmonkeys.de>
parents:
diff changeset
58 field = ","..field:gsub("[ \t]", ""):lower()..",";
cc9d460aa779 util.http: New module for HTTP helper functions
Florian Zeitz <florob@babelmonkeys.de>
parents:
diff changeset
59 return field:find(","..token:lower()..",", 1, true) ~= nil;
cc9d460aa779 util.http: New module for HTTP helper functions
Florian Zeitz <florob@babelmonkeys.de>
parents:
diff changeset
60 end
cc9d460aa779 util.http: New module for HTTP helper functions
Florian Zeitz <florob@babelmonkeys.de>
parents:
diff changeset
61
9504
cfbea3064aa9 mod_http: Move normalize_path to util.http
Kim Alvefur <zash@zash.se>
parents: 5471
diff changeset
62 local function normalize_path(path, is_dir)
cfbea3064aa9 mod_http: Move normalize_path to util.http
Kim Alvefur <zash@zash.se>
parents: 5471
diff changeset
63 if is_dir then
cfbea3064aa9 mod_http: Move normalize_path to util.http
Kim Alvefur <zash@zash.se>
parents: 5471
diff changeset
64 if path:sub(-1,-1) ~= "/" then path = path.."/"; end
cfbea3064aa9 mod_http: Move normalize_path to util.http
Kim Alvefur <zash@zash.se>
parents: 5471
diff changeset
65 else
cfbea3064aa9 mod_http: Move normalize_path to util.http
Kim Alvefur <zash@zash.se>
parents: 5471
diff changeset
66 if path:sub(-1,-1) == "/" then path = path:sub(1, -2); end
cfbea3064aa9 mod_http: Move normalize_path to util.http
Kim Alvefur <zash@zash.se>
parents: 5471
diff changeset
67 end
cfbea3064aa9 mod_http: Move normalize_path to util.http
Kim Alvefur <zash@zash.se>
parents: 5471
diff changeset
68 if path:sub(1,1) ~= "/" then path = "/"..path; end
cfbea3064aa9 mod_http: Move normalize_path to util.http
Kim Alvefur <zash@zash.se>
parents: 5471
diff changeset
69 return path;
cfbea3064aa9 mod_http: Move normalize_path to util.http
Kim Alvefur <zash@zash.se>
parents: 5471
diff changeset
70 end
cfbea3064aa9 mod_http: Move normalize_path to util.http
Kim Alvefur <zash@zash.se>
parents: 5471
diff changeset
71
13124
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
72 --- Parse the RFC 7239 Forwarded header into array of key-value pairs.
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
73 local function parse_forwarded(forwarded)
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
74 if type(forwarded) ~= "string" then
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
75 return nil;
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
76 end
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
77
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
78 local fwd = {}; -- array
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
79 local cur = {}; -- map, to which we add the next key-value pair
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
80 for key, quoted, value, delim in forwarded:gmatch("(%w+)%s*=%s*(\"?)([^,;\"]+)%2%s*(.?)") do
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
81 -- FIXME quoted quotes like "foo\"bar"
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
82 -- unlikely when only dealing with IP addresses
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
83 if quoted == '"' then
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
84 value = value:gsub("\\(.)", "%1");
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
85 end
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
86
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
87 cur[key:lower()] = value;
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
88 if delim == "" or delim == "," then
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
89 t_insert(fwd, cur)
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
90 if delim == "" then
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
91 -- end of the string
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
92 break;
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
93 end
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
94 cur = {};
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
95 elseif delim ~= ";" then
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
96 -- misparsed
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
97 return false;
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
98 end
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
99 end
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
100
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
101 return fwd;
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
102 end
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
103
5471
34bfd26525f5 util.http: Refactor and import all necessary functions
Matthew Wild <mwild1@gmail.com>
parents: 5458
diff changeset
104 return {
34bfd26525f5 util.http: Refactor and import all necessary functions
Matthew Wild <mwild1@gmail.com>
parents: 5458
diff changeset
105 urlencode = urlencode, urldecode = urldecode;
34bfd26525f5 util.http: Refactor and import all necessary functions
Matthew Wild <mwild1@gmail.com>
parents: 5458
diff changeset
106 formencode = formencode, formdecode = formdecode;
34bfd26525f5 util.http: Refactor and import all necessary functions
Matthew Wild <mwild1@gmail.com>
parents: 5458
diff changeset
107 contains_token = contains_token;
9504
cfbea3064aa9 mod_http: Move normalize_path to util.http
Kim Alvefur <zash@zash.se>
parents: 5471
diff changeset
108 normalize_path = normalize_path;
13124
f15e23840780 util.http: Implement parser for RFC 7239 Forwarded header
Kim Alvefur <zash@zash.se>
parents: 9785
diff changeset
109 parse_forwarded = parse_forwarded;
5471
34bfd26525f5 util.http: Refactor and import all necessary functions
Matthew Wild <mwild1@gmail.com>
parents: 5458
diff changeset
110 };