prosody-modules
1893ae742f66
mod_firewall/definitions.lib.lua

Find changesets by keywords by author, files, the commit message, revision number or hash, or revset expression.

Software / code / prosody-modules

Annotate

mod_firewall/definitions.lib.lua @ 5615:1893ae742f66

mod_http_oauth2: Show errors on device flow user code entry page If the user enters the code incorrectly, having to click back to try again is no fun. Instead, show the error and the code entry form again.
author Kim Alvefur <zash@zash.se>
date Wed, 19 Jul 2023 13:05:47 +0200
parent 5233:7eec2bc67c26
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2079
edec9de0220a mod_firewall: Silence warnings about unused arguments [luacheck]
Kim Alvefur <zash@zash.se>
parents: 1863
diff changeset
1
edec9de0220a mod_firewall: Silence warnings about unused arguments [luacheck]
Kim Alvefur <zash@zash.se>
parents: 1863
diff changeset
2 -- Name arguments are unused here
edec9de0220a mod_firewall: Silence warnings about unused arguments [luacheck]
Kim Alvefur <zash@zash.se>
parents: 1863
diff changeset
3 -- luacheck: ignore 212
999
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
4
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
5 local definition_handlers = {};
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
6
2520
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
7 local http = require "net.http";
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
8 local timer = require "util.timer";
1863
92602cfac751 mod_firewall: Fix missing import of util.set (used to be global)
Kim Alvefur <zash@zash.se>
parents: 999
diff changeset
9 local set = require"util.set";
999
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
10 local new_throttle = require "util.throttle".create;
2586
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
11 local hashes = require "util.hashes";
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
12 local jid = require "util.jid";
4016
b872f111b7af mod_firewall: Add option to ignore missing list files
Matthew Wild <mwild1@gmail.com>
parents: 3240
diff changeset
13 local lfs = require "lfs";
2128
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
14
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
15 local multirate_cache_size = module:get_option_number("firewall_multirate_cache_limit", 1000);
999
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
16
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
17 function definition_handlers.ZONE(zone_name, zone_members)
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
18 local zone_member_list = {};
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
19 for member in zone_members:gmatch("[^, ]+") do
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
20 zone_member_list[#zone_member_list+1] = member;
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
21 end
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
22 return set.new(zone_member_list)._items;
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
23 end
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
24
2128
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
25 -- Helper function used by RATE handler
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
26 local function evict_only_unthrottled(name, throttle)
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
27 throttle:update();
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
28 -- Check whether the throttle is at max balance (i.e. totally safe to forget about it)
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
29 if throttle.balance < throttle.max then
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
30 -- Not safe to forget
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
31 return false;
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
32 end
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
33 end
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
34
999
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
35 function definition_handlers.RATE(name, line)
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
36 local rate = assert(tonumber(line:match("([%d.]+)")), "Unable to parse rate");
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
37 local burst = tonumber(line:match("%(%s*burst%s+([%d.]+)%s*%)")) or 1;
2128
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
38 local max_throttles = tonumber(line:match("%(%s*entries%s+([%d]+)%s*%)")) or multirate_cache_size;
2370
5fe483b73fd2 mod_firewall: Rate limiting: Document 'entries' and add option to allow overflowing when full
Matthew Wild <mwild1@gmail.com>
parents: 2131
diff changeset
39 local deny_when_full = not line:match("%(allow overflow%)");
2128
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
40 return {
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
41 single = function ()
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
42 return new_throttle(rate*burst, burst);
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
43 end;
2859
22e11645a895 mod_firewall: Trim trailing whitespace [luacheck]
Kim Alvefur <zash@zash.se>
parents: 2587
diff changeset
44
2128
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
45 multi = function ()
2370
5fe483b73fd2 mod_firewall: Rate limiting: Document 'entries' and add option to allow overflowing when full
Matthew Wild <mwild1@gmail.com>
parents: 2131
diff changeset
46 local cache = require "util.cache".new(max_throttles, deny_when_full and evict_only_unthrottled or nil);
2128
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
47 return {
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
48 poll_on = function (_, key, amount)
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
49 assert(key, "no key");
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
50 local throttle = cache:get(key);
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
51 if not throttle then
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
52 throttle = new_throttle(rate*burst, burst);
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
53 if not cache:set(key, throttle) then
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
54 module:log("warn", "Multirate '%s' has hit its maximum number of active throttles (%d), denying new events", name, max_throttles);
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
55 return false;
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
56 end
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
57 end
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
58 return throttle:poll(amount);
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
59 end;
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
60 }
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
61 end;
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental)
Matthew Wild <mwild1@gmail.com>
parents: 2079
diff changeset
62 };
999
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
63 end
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
64
2520
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
65 local list_backends = {
2898
9fd61234b6f0 mod_firewall/definitions: Comments on LIST backends
Kim Alvefur <zash@zash.se>
parents: 2859
diff changeset
66 -- %LIST name: memory (limit: number)
2520
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
67 memory = {
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
68 init = function (self, type, opts)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
69 if opts.limit then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
70 local have_cache_lib, cache_lib = pcall(require, "util.cache");
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
71 if not have_cache_lib then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
72 error("In-memory lists with a size limit require Prosody 0.10");
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
73 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
74 self.cache = cache_lib.new((assert(tonumber(opts.limit), "Invalid list limit")));
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
75 if not self.cache.table then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
76 error("In-memory lists with a size limit require a newer version of Prosody 0.10");
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
77 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
78 self.items = self.cache:table();
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
79 else
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
80 self.items = {};
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
81 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
82 end;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
83 add = function (self, item)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
84 self.items[item] = true;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
85 end;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
86 remove = function (self, item)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
87 self.items[item] = nil;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
88 end;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
89 contains = function (self, item)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
90 return self.items[item] == true;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
91 end;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
92 };
2898
9fd61234b6f0 mod_firewall/definitions: Comments on LIST backends
Kim Alvefur <zash@zash.se>
parents: 2859
diff changeset
93
9fd61234b6f0 mod_firewall/definitions: Comments on LIST backends
Kim Alvefur <zash@zash.se>
parents: 2859
diff changeset
94 -- %LIST name: http://example.com/ (ttl: number, pattern: pat, hash: sha1)
2520
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
95 http = {
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
96 init = function (self, url, opts)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
97 local poll_interval = assert(tonumber(opts.ttl or "3600"), "invalid ttl for <"..url.."> (expected number of seconds)");
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
98 local pattern = opts.pattern or "([^\r\n]+)\r?\n";
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
99 assert(pcall(string.match, "", pattern), "invalid pattern for <"..url..">");
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
100 if opts.hash then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
101 assert(opts.hash:match("^%w+$") and type(hashes[opts.hash]) == "function", "invalid hash function: "..opts.hash);
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
102 self.hash_function = hashes[opts.hash];
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
103 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
104 local etag;
2522
72cbec103709 mod_firewall: Improve HTTP polling logic
Matthew Wild <mwild1@gmail.com>
parents: 2520
diff changeset
105 local failure_count = 0;
72cbec103709 mod_firewall: Improve HTTP polling logic
Matthew Wild <mwild1@gmail.com>
parents: 2520
diff changeset
106 local retry_intervals = { 60, 120, 300 };
4127
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported
Matthew Wild <mwild1@gmail.com>
parents: 4017
diff changeset
107 -- By default only check the certificate if net.http supports SNI
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported
Matthew Wild <mwild1@gmail.com>
parents: 4017
diff changeset
108 local sni_supported = http.feature and http.features.sni;
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported
Matthew Wild <mwild1@gmail.com>
parents: 4017
diff changeset
109 local insecure = false;
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported
Matthew Wild <mwild1@gmail.com>
parents: 4017
diff changeset
110 if opts.checkcert == "never" then
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported
Matthew Wild <mwild1@gmail.com>
parents: 4017
diff changeset
111 insecure = true;
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported
Matthew Wild <mwild1@gmail.com>
parents: 4017
diff changeset
112 elseif (opts.checkcert == nil or opts.checkcert == "when-sni") and not sni_supported then
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported
Matthew Wild <mwild1@gmail.com>
parents: 4017
diff changeset
113 insecure = false;
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported
Matthew Wild <mwild1@gmail.com>
parents: 4017
diff changeset
114 end
2520
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
115 local function update_list()
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
116 http.request(url, {
4127
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported
Matthew Wild <mwild1@gmail.com>
parents: 4017
diff changeset
117 insecure = insecure;
2520
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
118 headers = {
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
119 ["If-None-Match"] = etag;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
120 };
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
121 }, function (body, code, response)
2522
72cbec103709 mod_firewall: Improve HTTP polling logic
Matthew Wild <mwild1@gmail.com>
parents: 2520
diff changeset
122 local next_poll = poll_interval;
2520
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
123 if code == 200 and body then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
124 etag = response.headers.etag;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
125 local items = {};
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
126 for entry in body:gmatch(pattern) do
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
127 items[entry] = true;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
128 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
129 self.items = items;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
130 module:log("debug", "Fetched updated list from <%s>", url);
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
131 elseif code == 304 then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
132 module:log("debug", "List at <%s> is unchanged", url);
2522
72cbec103709 mod_firewall: Improve HTTP polling logic
Matthew Wild <mwild1@gmail.com>
parents: 2520
diff changeset
133 elseif code == 0 or (code >= 400 and code <=599) then
2520
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
134 module:log("warn", "Failed to fetch list from <%s>: %d %s", url, code, tostring(body));
2522
72cbec103709 mod_firewall: Improve HTTP polling logic
Matthew Wild <mwild1@gmail.com>
parents: 2520
diff changeset
135 failure_count = failure_count + 1;
72cbec103709 mod_firewall: Improve HTTP polling logic
Matthew Wild <mwild1@gmail.com>
parents: 2520
diff changeset
136 next_poll = retry_intervals[failure_count] or retry_intervals[#retry_intervals];
2520
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
137 end
2522
72cbec103709 mod_firewall: Improve HTTP polling logic
Matthew Wild <mwild1@gmail.com>
parents: 2520
diff changeset
138 if next_poll > 0 then
72cbec103709 mod_firewall: Improve HTTP polling logic
Matthew Wild <mwild1@gmail.com>
parents: 2520
diff changeset
139 timer.add_task(next_poll+math.random(0, 60), update_list);
2520
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
140 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
141 end);
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
142 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
143 update_list();
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
144 end;
2530
84e103fd8039 mod_firewall: Add dummy add/remove methods for HTTP lists
Matthew Wild <mwild1@gmail.com>
parents: 2528
diff changeset
145 add = function ()
84e103fd8039 mod_firewall: Add dummy add/remove methods for HTTP lists
Matthew Wild <mwild1@gmail.com>
parents: 2528
diff changeset
146 end;
84e103fd8039 mod_firewall: Add dummy add/remove methods for HTTP lists
Matthew Wild <mwild1@gmail.com>
parents: 2528
diff changeset
147 remove = function ()
84e103fd8039 mod_firewall: Add dummy add/remove methods for HTTP lists
Matthew Wild <mwild1@gmail.com>
parents: 2528
diff changeset
148 end;
2520
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
149 contains = function (self, item)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
150 if self.hash_function then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
151 item = self.hash_function(item);
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
152 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
153 return self.items and self.items[item] == true;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
154 end;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
155 };
2898
9fd61234b6f0 mod_firewall/definitions: Comments on LIST backends
Kim Alvefur <zash@zash.se>
parents: 2859
diff changeset
156
9fd61234b6f0 mod_firewall/definitions: Comments on LIST backends
Kim Alvefur <zash@zash.se>
parents: 2859
diff changeset
157 -- %LIST: file:/path/to/file
2532
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
158 file = {
2535
b85d88737a32 mod_firewall: Fix file backend init code
Matthew Wild <mwild1@gmail.com>
parents: 2532
diff changeset
159 init = function (self, file_spec, opts)
4017
015452258952 mod_firewall: Ensure file lists are always initialized empty
Matthew Wild <mwild1@gmail.com>
parents: 4016
diff changeset
160 local n, items = 0, {};
015452258952 mod_firewall: Ensure file lists are always initialized empty
Matthew Wild <mwild1@gmail.com>
parents: 4016
diff changeset
161 self.items = items;
2535
b85d88737a32 mod_firewall: Fix file backend init code
Matthew Wild <mwild1@gmail.com>
parents: 2532
diff changeset
162 local filename = file_spec:gsub("^file:", "");
4016
b872f111b7af mod_firewall: Add option to ignore missing list files
Matthew Wild <mwild1@gmail.com>
parents: 3240
diff changeset
163 if opts.missing == "ignore" and not lfs.attributes(filename, "mode") then
b872f111b7af mod_firewall: Add option to ignore missing list files
Matthew Wild <mwild1@gmail.com>
parents: 3240
diff changeset
164 module:log("debug", "Ignoring missing list file: %s", filename);
b872f111b7af mod_firewall: Add option to ignore missing list files
Matthew Wild <mwild1@gmail.com>
parents: 3240
diff changeset
165 return;
b872f111b7af mod_firewall: Add option to ignore missing list files
Matthew Wild <mwild1@gmail.com>
parents: 3240
diff changeset
166 end
2532
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
167 local file, err = io.open(filename);
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
168 if not file then
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
169 module:log("warn", "Failed to open list from %s: %s", filename, err);
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
170 return;
3240
c30f2cfe9f15 mod_firewall: Assume empty list if file could not be loaded
Matthew Wild <mwild1@gmail.com>
parents: 2898
diff changeset
171 else
c30f2cfe9f15 mod_firewall: Assume empty list if file could not be loaded
Matthew Wild <mwild1@gmail.com>
parents: 2898
diff changeset
172 for line in file:lines() do
c30f2cfe9f15 mod_firewall: Assume empty list if file could not be loaded
Matthew Wild <mwild1@gmail.com>
parents: 2898
diff changeset
173 if not items[line] then
c30f2cfe9f15 mod_firewall: Assume empty list if file could not be loaded
Matthew Wild <mwild1@gmail.com>
parents: 2898
diff changeset
174 n = n + 1;
c30f2cfe9f15 mod_firewall: Assume empty list if file could not be loaded
Matthew Wild <mwild1@gmail.com>
parents: 2898
diff changeset
175 items[line] = true;
c30f2cfe9f15 mod_firewall: Assume empty list if file could not be loaded
Matthew Wild <mwild1@gmail.com>
parents: 2898
diff changeset
176 end
2536
22a271641c29 mod_firewall: Improve debug logging for LIST file backend
Matthew Wild <mwild1@gmail.com>
parents: 2535
diff changeset
177 end
2532
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
178 end
2536
22a271641c29 mod_firewall: Improve debug logging for LIST file backend
Matthew Wild <mwild1@gmail.com>
parents: 2535
diff changeset
179 module:log("debug", "Loaded %d items from %s", n, filename);
2532
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
180 end;
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
181 add = function (self, item)
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
182 self.items[item] = true;
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
183 end;
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
184 remove = function (self, item)
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
185 self.items[item] = nil;
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
186 end;
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
187 contains = function (self, item)
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
188 return self.items and self.items[item] == true;
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
189 end;
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm)
Matthew Wild <mwild1@gmail.com>
parents: 2530
diff changeset
190 };
4512
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
191
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
192 -- %LIST: pubsub:pubsub.example.com/node
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
193 -- TODO or the actual URI scheme? Bit overkill maybe?
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
194 -- TODO Publish items back to the service?
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
195 -- Step 1: Receiving pubsub events and storing them in the list
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
196 -- We'll start by using only the item id.
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
197 -- TODO Invent some custom schema for this? Needed for just a set of strings?
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
198 pubsubitemid = {
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
199 init = function(self, pubsub_spec, opts)
5232
0f943619e01a mod_firewall: Fix parsing of pubsubitemid list specification
Matthew Wild <mwild1@gmail.com>
parents: 4512
diff changeset
200 local service_addr, node = pubsub_spec:match("^pubsubitemid:([^/]*)/(.*)");
5233
7eec2bc67c26 mod_firewall: Warn about invalid pubsubitemid list specification
Matthew Wild <mwild1@gmail.com>
parents: 5232
diff changeset
201 if not service_addr then
7eec2bc67c26 mod_firewall: Warn about invalid pubsubitemid list specification
Matthew Wild <mwild1@gmail.com>
parents: 5232
diff changeset
202 module:log("warn", "Invalid list specification (expected 'pubsubitemid:<service>/<node>', got: '%s')", pubsub_spec);
7eec2bc67c26 mod_firewall: Warn about invalid pubsubitemid list specification
Matthew Wild <mwild1@gmail.com>
parents: 5232
diff changeset
203 return;
7eec2bc67c26 mod_firewall: Warn about invalid pubsubitemid list specification
Matthew Wild <mwild1@gmail.com>
parents: 5232
diff changeset
204 end
4512
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
205 module:depends("pubsub_subscription");
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
206 module:add_item("pubsub-subscription", {
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
207 service = service_addr;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
208 node = node;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
209 on_subscribed = function ()
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
210 self.items = {};
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
211 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
212 on_item = function (event)
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
213 self:add(event.item.attr.id);
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
214 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
215 on_retract = function (event)
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
216 self:remove(event.item.attr.id);
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
217 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
218 on_purge = function ()
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
219 self.items = {};
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
220 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
221 on_unsubscribed = function ()
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
222 self.items = nil;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
223 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
224 on_delete= function ()
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
225 self.items = nil;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
226 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
227 });
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
228 -- TODO Initial fetch? Or should mod_pubsub_subscription do this?
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
229 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
230 add = function (self, item)
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
231 if self.items then
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
232 self.items[item] = true;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
233 end
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
234 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
235 remove = function (self, item)
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
236 if self.items then
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
237 self.items[item] = nil;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
238 end
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
239 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
240 contains = function (self, item)
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
241 return self.items and self.items[item] == true;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
242 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub
Kim Alvefur <zash@zash.se>
parents: 4127
diff changeset
243 };
2520
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
244 };
2523
a3a18d09ae8a mod_firewall: Also handle HTTPS for lists
Matthew Wild <mwild1@gmail.com>
parents: 2522
diff changeset
245 list_backends.https = list_backends.http;
2520
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
246
2586
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
247 local normalize_functions = {
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
248 upper = string.upper, lower = string.lower;
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
249 md5 = hashes.md5, sha1 = hashes.sha1, sha256 = hashes.sha256;
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
250 prep = jid.prep, bare = jid.bare;
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
251 };
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
252
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
253 local function wrap_list_method(list_method, filter)
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
254 return function (self, item)
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
255 return list_method(self, filter(item));
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
256 end
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
257 end
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
258
2520
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
259 local function create_list(list_backend, list_def, opts)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
260 if not list_backends[list_backend] then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
261 error("Unknown list type '"..list_backend.."'", 0);
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
262 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
263 local list = setmetatable({}, { __index = list_backends[list_backend] });
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
264 if list.init then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
265 list:init(list_def, opts);
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
266 end
2586
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
267 if opts.filter then
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
268 local filters = {};
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
269 for func_name in opts.filter:gmatch("[%w_]+") do
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
270 if func_name == "log" then
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
271 table.insert(filters, function (s)
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
272 --print("&&&&&", s);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
273 module:log("debug", "Checking list <%s> for: %s", list_def, s);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
274 return s;
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
275 end);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
276 else
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
277 assert(normalize_functions[func_name], "Unknown list filter: "..func_name);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
278 table.insert(filters, normalize_functions[func_name]);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
279 end
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
280 end
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
281
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
282 local filter;
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
283 local n = #filters;
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
284 if n == 1 then
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
285 filter = filters[1];
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
286 else
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
287 function filter(s)
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
288 for i = 1, n do
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
289 s = filters[i](s or "");
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
290 end
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
291 return s;
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
292 end
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
293 end
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
294
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
295 list.add = wrap_list_method(list.add, filter);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
296 list.remove = wrap_list_method(list.remove, filter);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
297 list.contains = wrap_list_method(list.contains, filter);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists
Matthew Wild <mwild1@gmail.com>
parents: 2536
diff changeset
298 end
2520
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
299 return list;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
300 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
301
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
302 --[[
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
303 %LIST spammers: memory (source: /etc/spammers.txt)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
304
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
305 %LIST spammers: memory (source: /etc/spammers.txt)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
306
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
307
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
308 %LIST spammers: http://example.com/blacklist.txt
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
309 ]]
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
310
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
311 function definition_handlers.LIST(list_name, list_definition)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
312 local list_backend = list_definition:match("^%w+");
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
313 local opts = {};
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
314 local opt_string = list_definition:match("^%S+%s+%((.+)%)");
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
315 if opt_string then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
316 for opt_k, opt_v in opt_string:gmatch("(%w+): ?([^,]+)") do
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
317 opts[opt_k] = opt_v;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
318 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
319 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
320 return create_list(list_backend, list_definition:match("^%S+"), opts);
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
321 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP
Matthew Wild <mwild1@gmail.com>
parents: 2370
diff changeset
322
2528
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list
Matthew Wild <mwild1@gmail.com>
parents: 2523
diff changeset
323 function definition_handlers.PATTERN(name, pattern)
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list
Matthew Wild <mwild1@gmail.com>
parents: 2523
diff changeset
324 local ok, err = pcall(string.match, "", pattern);
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list
Matthew Wild <mwild1@gmail.com>
parents: 2523
diff changeset
325 if not ok then
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list
Matthew Wild <mwild1@gmail.com>
parents: 2523
diff changeset
326 error("Invalid pattern '"..name.."': "..err);
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list
Matthew Wild <mwild1@gmail.com>
parents: 2523
diff changeset
327 end
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list
Matthew Wild <mwild1@gmail.com>
parents: 2523
diff changeset
328 return pattern;
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list
Matthew Wild <mwild1@gmail.com>
parents: 2523
diff changeset
329 end
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list
Matthew Wild <mwild1@gmail.com>
parents: 2523
diff changeset
330
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list
Matthew Wild <mwild1@gmail.com>
parents: 2523
diff changeset
331 function definition_handlers.SEARCH(name, pattern)
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list
Matthew Wild <mwild1@gmail.com>
parents: 2523
diff changeset
332 return pattern;
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list
Matthew Wild <mwild1@gmail.com>
parents: 2523
diff changeset
333 end
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list
Matthew Wild <mwild1@gmail.com>
parents: 2523
diff changeset
334
999
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with %
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
335 return definition_handlers;