prosody
53e0ae770917
teal-src/plugins/mod_cron.tl

Find changesets by keywords by author, files, the commit message, revision number or hash, or revset expression.

Software / code / prosody

Annotate

teal-src/plugins/mod_cron.tl @ 12180:53e0ae770917

util.xml: Do not allow doctypes, comments or processing instructions Yes. This is as bad as it sounds. CVE pending. In Prosody itself, this only affects mod_websocket, which uses util.xml to parse the <open/> frame, thus allowing unauthenticated remote DoS using Billion Laughs. However, third-party modules using util.xml may also be affected by this. This commit installs handlers which disallow the use of doctype declarations and processing instructions without any escape hatch. It, by default, also introduces such a handler for comments, however, there is a way to enable comments nontheless. This is because util.xml is used to parse human-facing data, where comments are generally a desirable feature, and also because comments are generally harmless.
author Jonas Schäfer <jonas@wielicki.name>
date Mon, 10 Jan 2022 18:23:54 +0100
parent 12002:cbed7d8d8f35
child 12186:7f25ac9d8f0d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
11986
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
1 module:set_global();
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
2
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
3 local async = require "util.async";
12000
00c57684cf20 mod_cron: Follow convention of imports at the top
Kim Alvefur <zash@zash.se>
parents: 11995
diff changeset
4 local datetime = require "util.datetime";
11986
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
5
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
6 local record map_store<K,V>
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
7 -- TODO move to somewhere sensible
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
8 get : function (map_store<K,V>, string, K) : V
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
9 set : function (map_store<K,V>, string, K, V)
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
10 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
11
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
12 local enum frequency
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
13 "hourly"
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
14 "daily"
12002
cbed7d8d8f35 mod_cron: Add a 'weekly' job frequency
Kim Alvefur <zash@zash.se>
parents: 12001
diff changeset
15 "weekly"
11986
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
16 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
17
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
18 local record task_spec
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
19 id : string -- unique id
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
20 name : string -- name or short description
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
21 when : frequency
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
22 last : integer
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
23 run : function (task_spec, integer)
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
24 save : function (task_spec, integer)
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
25 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
26
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
27 local record task_event
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
28 source : module
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
29 item : task_spec
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
30 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
31
12002
cbed7d8d8f35 mod_cron: Add a 'weekly' job frequency
Kim Alvefur <zash@zash.se>
parents: 12001
diff changeset
32 local periods : { frequency : integer } = { hourly = 3600, daily = 86400, weekly = 7*86400 }
11986
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
33
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
34 local active_hosts : { string : boolean } = { }
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
35
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
36 function module.add_host(host_module : moduleapi)
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
37
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
38 local last_run_times = host_module:open_store("cron", "map") as map_store<string,integer>;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
39 active_hosts[host_module.host] = true;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
40
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
41 local function save_task(task : task_spec, started_at : integer)
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
42 last_run_times:set(nil, task.id, started_at);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
43 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
44
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
45 local function task_added(event : task_event) : boolean
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
46 local task = event.item;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
47 if task.name == nil then
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
48 task.name = task.when;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
49 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
50 if task.id == nil then
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
51 task.id = event.source.name .. "/" .. task.name:gsub("%W", "_"):lower();
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
52 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
53 if task.last == nil then
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
54 task.last = last_run_times:get(nil, task.id);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
55 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
56 task.save = save_task;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
57 module:log("debug", "%s task %s added, last run %s", task.when, task.id,
12000
00c57684cf20 mod_cron: Follow convention of imports at the top
Kim Alvefur <zash@zash.se>
parents: 11995
diff changeset
58 task.last and datetime.datetime(task.last) or "never");
12001
5a8c6f9a4583 mod_cron: Initialize timestamp of new tasks to start of period
Kim Alvefur <zash@zash.se>
parents: 12000
diff changeset
59 if task.last == nil then
5a8c6f9a4583 mod_cron: Initialize timestamp of new tasks to start of period
Kim Alvefur <zash@zash.se>
parents: 12000
diff changeset
60 -- initialize new tasks so e.g. daily tasks run at ~midnight UTC for now
11995
bbd3ac65640d mod_cron: Initialize daily tasks so they run around midnight UTC
Kim Alvefur <zash@zash.se>
parents: 11986
diff changeset
61 local now = os.time();
12001
5a8c6f9a4583 mod_cron: Initialize timestamp of new tasks to start of period
Kim Alvefur <zash@zash.se>
parents: 12000
diff changeset
62 task.last = now - now % periods[task.when];
11995
bbd3ac65640d mod_cron: Initialize daily tasks so they run around midnight UTC
Kim Alvefur <zash@zash.se>
parents: 11986
diff changeset
63 end
11986
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
64 return true;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
65 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
66
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
67 local function task_removed(event : task_event) : boolean
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
68 local task = event.item;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
69 host_module:log("debug", "Task %s removed", task.id);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
70 return true;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
71 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
72
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
73 host_module:handle_items("task", task_added, task_removed, true);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
74
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
75 function host_module.unload()
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
76 active_hosts[host_module.host]=nil;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
77 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
78 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
79
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
80 local function should_run(when : frequency, last : integer) : boolean
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
81 return not last or last + periods[when] <= os.time();
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
82 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
83
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
84 local function run_task(task : task_spec)
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
85 local started_at = os.time();
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
86 task:run(started_at);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
87 task:save(started_at);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
88 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
89
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
90 local task_runner = async.runner(run_task);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
91 module:add_timer(1, function() : integer
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
92 module:log("info", "Running periodic tasks");
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
93 local delay = 3600;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
94 for host in pairs(active_hosts) do
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
95 module:log("debug", "Running periodic tasks for host %s", host);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
96 for _, task in ipairs(module:context(host):get_host_items("task") as { task_spec } ) do
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
97 module:log("debug", "Considering %s task %s (%s)", task.when, task.id, task.run);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
98 if should_run(task.when, task.last) then task_runner:run(task); end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
99 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
100 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
101 module:log("debug", "Wait %ds", delay);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
102 return delay;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
103 end);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
104
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
105 -- TODO measure load, pick a good time to do stuff