prosody
7f25ac9d8f0d
teal-src/plugins/mod_cron.tl

Find changesets by keywords by author, files, the commit message, revision number or hash, or revset expression.

Software / code / prosody

Annotate

teal-src/plugins/mod_cron.tl @ 12186:7f25ac9d8f0d

mod_cron: Allow for a small amount of timer drift If the timer activates a bit early then a task might be just a few seconds short of being allowed to run. This would run such a task rather than wait another hour. The value 0.5% chosen so that a weekly task does not run an entire hour earlier than last time.
author Kim Alvefur <zash@zash.se>
date Sat, 15 Jan 2022 09:09:24 +0100
parent 12002:cbed7d8d8f35
child 12489:8b42575738f0
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
11986
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
1 module:set_global();
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
2
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
3 local async = require "util.async";
12000
00c57684cf20 mod_cron: Follow convention of imports at the top
Kim Alvefur <zash@zash.se>
parents: 11995
diff changeset
4 local datetime = require "util.datetime";
11986
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
5
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
6 local record map_store<K,V>
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
7 -- TODO move to somewhere sensible
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
8 get : function (map_store<K,V>, string, K) : V
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
9 set : function (map_store<K,V>, string, K, V)
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
10 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
11
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
12 local enum frequency
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
13 "hourly"
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
14 "daily"
12002
cbed7d8d8f35 mod_cron: Add a 'weekly' job frequency
Kim Alvefur <zash@zash.se>
parents: 12001
diff changeset
15 "weekly"
11986
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
16 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
17
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
18 local record task_spec
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
19 id : string -- unique id
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
20 name : string -- name or short description
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
21 when : frequency
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
22 last : integer
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
23 run : function (task_spec, integer)
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
24 save : function (task_spec, integer)
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
25 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
26
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
27 local record task_event
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
28 source : module
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
29 item : task_spec
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
30 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
31
12002
cbed7d8d8f35 mod_cron: Add a 'weekly' job frequency
Kim Alvefur <zash@zash.se>
parents: 12001
diff changeset
32 local periods : { frequency : integer } = { hourly = 3600, daily = 86400, weekly = 7*86400 }
11986
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
33
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
34 local active_hosts : { string : boolean } = { }
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
35
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
36 function module.add_host(host_module : moduleapi)
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
37
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
38 local last_run_times = host_module:open_store("cron", "map") as map_store<string,integer>;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
39 active_hosts[host_module.host] = true;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
40
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
41 local function save_task(task : task_spec, started_at : integer)
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
42 last_run_times:set(nil, task.id, started_at);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
43 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
44
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
45 local function task_added(event : task_event) : boolean
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
46 local task = event.item;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
47 if task.name == nil then
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
48 task.name = task.when;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
49 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
50 if task.id == nil then
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
51 task.id = event.source.name .. "/" .. task.name:gsub("%W", "_"):lower();
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
52 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
53 if task.last == nil then
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
54 task.last = last_run_times:get(nil, task.id);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
55 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
56 task.save = save_task;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
57 module:log("debug", "%s task %s added, last run %s", task.when, task.id,
12000
00c57684cf20 mod_cron: Follow convention of imports at the top
Kim Alvefur <zash@zash.se>
parents: 11995
diff changeset
58 task.last and datetime.datetime(task.last) or "never");
12001
5a8c6f9a4583 mod_cron: Initialize timestamp of new tasks to start of period
Kim Alvefur <zash@zash.se>
parents: 12000
diff changeset
59 if task.last == nil then
5a8c6f9a4583 mod_cron: Initialize timestamp of new tasks to start of period
Kim Alvefur <zash@zash.se>
parents: 12000
diff changeset
60 -- initialize new tasks so e.g. daily tasks run at ~midnight UTC for now
11995
bbd3ac65640d mod_cron: Initialize daily tasks so they run around midnight UTC
Kim Alvefur <zash@zash.se>
parents: 11986
diff changeset
61 local now = os.time();
12001
5a8c6f9a4583 mod_cron: Initialize timestamp of new tasks to start of period
Kim Alvefur <zash@zash.se>
parents: 12000
diff changeset
62 task.last = now - now % periods[task.when];
11995
bbd3ac65640d mod_cron: Initialize daily tasks so they run around midnight UTC
Kim Alvefur <zash@zash.se>
parents: 11986
diff changeset
63 end
11986
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
64 return true;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
65 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
66
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
67 local function task_removed(event : task_event) : boolean
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
68 local task = event.item;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
69 host_module:log("debug", "Task %s removed", task.id);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
70 return true;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
71 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
72
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
73 host_module:handle_items("task", task_added, task_removed, true);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
74
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
75 function host_module.unload()
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
76 active_hosts[host_module.host]=nil;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
77 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
78 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
79
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
80 local function should_run(when : frequency, last : integer) : boolean
12186
7f25ac9d8f0d mod_cron: Allow for a small amount of timer drift
Kim Alvefur <zash@zash.se>
parents: 12002
diff changeset
81 return not last or last + periods[when]*0.995 <= os.time();
11986
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
82 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
83
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
84 local function run_task(task : task_spec)
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
85 local started_at = os.time();
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
86 task:run(started_at);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
87 task:save(started_at);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
88 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
89
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
90 local task_runner = async.runner(run_task);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
91 module:add_timer(1, function() : integer
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
92 module:log("info", "Running periodic tasks");
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
93 local delay = 3600;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
94 for host in pairs(active_hosts) do
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
95 module:log("debug", "Running periodic tasks for host %s", host);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
96 for _, task in ipairs(module:context(host):get_host_items("task") as { task_spec } ) do
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
97 module:log("debug", "Considering %s task %s (%s)", task.when, task.id, task.run);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
98 if should_run(task.when, task.last) then task_runner:run(task); end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
99 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
100 end
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
101 module:log("debug", "Wait %ds", delay);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
102 return delay;
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
103 end);
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
104
3d5135e8a2a7 mod_cron: Initial commit of periodic task runner
Kim Alvefur <zash@zash.se>
parents:
diff changeset
105 -- TODO measure load, pick a good time to do stuff