prosody
f69b3e39e0c1
util/async.lua

Find changesets by keywords by author, files, the commit message, revision number or hash, or revset expression.

Software / code / prosody

Annotate

util/async.lua @ 8918:f69b3e39e0c1

mod_bosh: Add extra debug logging to help with #1134
author Kim Alvefur <zash@zash.se>
date Tue, 15 May 2018 01:22:35 +0200
parent 8788:7a9b680a79fb
child 8927:ed0891383e78
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
8788
7a9b680a79fb util.async: Move runner id into log tag
Kim Alvefur <zash@zash.se>
parents: 8766
diff changeset
1 local logger = require "util.logger";
7a9b680a79fb util.async: Move runner id into log tag
Kim Alvefur <zash@zash.se>
parents: 8766
diff changeset
2 local log = logger.init("util.async");
8600
96f20cf92b84 util.async: Add per-runner ids and add runner:log() method
Matthew Wild <mwild1@gmail.com>
parents: 8408
diff changeset
3 local new_id = require "util.id".short;
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
4
8407
f652e1ea2f69 util.async: Factor out thread check into a function
Kim Alvefur <zash@zash.se>
parents: 8237
diff changeset
5 local function checkthread()
8408
b751bee6a829 util.async: Fix thread check to work correctly in Lua 5.2
Kim Alvefur <zash@zash.se>
parents: 8407
diff changeset
6 local thread, main = coroutine.running();
b751bee6a829 util.async: Fix thread check to work correctly in Lua 5.2
Kim Alvefur <zash@zash.se>
parents: 8407
diff changeset
7 if not thread or main then
8407
f652e1ea2f69 util.async: Factor out thread check into a function
Kim Alvefur <zash@zash.se>
parents: 8237
diff changeset
8 error("Not running in an async context, see https://prosody.im/doc/developers/util/async");
f652e1ea2f69 util.async: Factor out thread check into a function
Kim Alvefur <zash@zash.se>
parents: 8237
diff changeset
9 end
f652e1ea2f69 util.async: Factor out thread check into a function
Kim Alvefur <zash@zash.se>
parents: 8237
diff changeset
10 return thread;
f652e1ea2f69 util.async: Factor out thread check into a function
Kim Alvefur <zash@zash.se>
parents: 8237
diff changeset
11 end
f652e1ea2f69 util.async: Factor out thread check into a function
Kim Alvefur <zash@zash.se>
parents: 8237
diff changeset
12
8627
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
13 local function runner_from_thread(thread)
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
14 local level = 0;
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
15 -- Find the 'level' of the top-most function (0 == current level, 1 == caller, ...)
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
16 while debug.getinfo(thread, level, "") do level = level + 1; end
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
17 local name, runner = debug.getlocal(thread, level-1, 1);
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
18 if name ~= "self" or type(runner) ~= "table" or runner.thread ~= thread then
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
19 return nil;
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
20 end
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
21 return runner;
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
22 end
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
23
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
24 local function call_watcher(runner, watcher_name, ...)
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
25 local watcher = runner.watchers[watcher_name];
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
26 if not watcher then
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
27 return false;
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
28 end
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
29 runner:log("debug", "Calling '%s' watcher", watcher_name);
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
30 local ok, err = pcall(watcher, runner, ...); -- COMPAT: Switch to xpcall after Lua 5.1
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
31 if not ok then
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
32 runner:log("error", "Error in '%s' watcher: %s", watcher_name, err);
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
33 return nil, err;
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
34 end
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
35 return true;
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
36 end
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
37
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
38 local function runner_continue(thread)
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
39 -- ASSUMPTION: runner is in 'waiting' state (but we don't have the runner to know for sure)
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
40 if coroutine.status(thread) ~= "suspended" then -- This should suffice
8629
ddb04cacb8b1 util.async: Bump log warnings to error level
Matthew Wild <mwild1@gmail.com>
parents: 8627
diff changeset
41 log("error", "unexpected async state: thread not suspended");
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
42 return false;
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
43 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
44 local ok, state, runner = coroutine.resume(thread);
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
45 if not ok then
8609
9f6ab206d741 util.async: Ensure runner is left in correct state after out-of-main-loop error (+tests)
Matthew Wild <mwild1@gmail.com>
parents: 8604
diff changeset
46 local err = state;
7436
649b89b2c840 util.async: Add some more comments for clarity
Matthew Wild <mwild1@gmail.com>
parents: 7359
diff changeset
47 -- Running the coroutine failed, which means we have to find the runner manually,
649b89b2c840 util.async: Add some more comments for clarity
Matthew Wild <mwild1@gmail.com>
parents: 7359
diff changeset
48 -- in order to inform the error handler
8627
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
49 runner = runner_from_thread(thread);
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
50 if not runner then
8629
ddb04cacb8b1 util.async: Bump log warnings to error level
Matthew Wild <mwild1@gmail.com>
parents: 8627
diff changeset
51 log("error", "unexpected async state: unable to locate runner during error handling");
8613
dbb4788db8e3 util.async: Convert asserts to a return false (same as other unexpected behaviour)
Matthew Wild <mwild1@gmail.com>
parents: 8611
diff changeset
52 return false;
dbb4788db8e3 util.async: Convert asserts to a return false (same as other unexpected behaviour)
Matthew Wild <mwild1@gmail.com>
parents: 8611
diff changeset
53 end
8627
24b59a62acce util.async: Split runner_continue into smaller functions for easier testing and safety
Matthew Wild <mwild1@gmail.com>
parents: 8625
diff changeset
54 call_watcher(runner, "error", debug.traceback(thread, err));
8609
9f6ab206d741 util.async: Ensure runner is left in correct state after out-of-main-loop error (+tests)
Matthew Wild <mwild1@gmail.com>
parents: 8604
diff changeset
55 runner.state, runner.thread = "ready", nil;
8616
a15c891c6232 util.async: ensure change in e77b37de482e applies after out-of-loop resume also
Matthew Wild <mwild1@gmail.com>
parents: 8615
diff changeset
56 return runner:run();
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
57 elseif state == "ready" then
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
58 -- If state is 'ready', it is our responsibility to update runner.state from 'waiting'.
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
59 -- We also have to :run(), because the queue might have further items that will not be
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
60 -- processed otherwise. FIXME: It's probably best to do this in a nexttick (0 timer).
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
61 runner.state = "ready";
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
62 runner:run();
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
63 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
64 return true;
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
65 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
66
5790
959163e4d631 util.async: Make functions local
Matthew Wild <mwild1@gmail.com>
parents: 5788
diff changeset
67 local function waiter(num)
8407
f652e1ea2f69 util.async: Factor out thread check into a function
Kim Alvefur <zash@zash.se>
parents: 8237
diff changeset
68 local thread = checkthread();
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
69 num = num or 1;
5792
aac4c6147647 util.async: waiter: Remove restriction about wait() being called before done()
Matthew Wild <mwild1@gmail.com>
parents: 5791
diff changeset
70 local waiting;
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
71 return function ()
5792
aac4c6147647 util.async: waiter: Remove restriction about wait() being called before done()
Matthew Wild <mwild1@gmail.com>
parents: 5791
diff changeset
72 if num == 0 then return; end -- already done
aac4c6147647 util.async: waiter: Remove restriction about wait() being called before done()
Matthew Wild <mwild1@gmail.com>
parents: 5791
diff changeset
73 waiting = true;
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
74 coroutine.yield("wait");
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
75 end, function ()
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
76 num = num - 1;
5792
aac4c6147647 util.async: waiter: Remove restriction about wait() being called before done()
Matthew Wild <mwild1@gmail.com>
parents: 5791
diff changeset
77 if num == 0 and waiting then
aac4c6147647 util.async: waiter: Remove restriction about wait() being called before done()
Matthew Wild <mwild1@gmail.com>
parents: 5791
diff changeset
78 runner_continue(thread);
5793
e8c79796ead9 util.async: waiter: Throw error if done() called too many times
Kim Alvefur <zash@zash.se>
parents: 5792
diff changeset
79 elseif num < 0 then
e8c79796ead9 util.async: waiter: Throw error if done() called too many times
Kim Alvefur <zash@zash.se>
parents: 5792
diff changeset
80 error("done() called too many times");
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
81 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
82 end;
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
83 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
84
5797
a493b79cfad0 util.async: Make guarder() local
Matthew Wild <mwild1@gmail.com>
parents: 5796
diff changeset
85 local function guarder()
5796
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
86 local guards = {};
8655
ba6a6a04b46c util.async: Allow nil as a guard key
Matthew Wild <mwild1@gmail.com>
parents: 8651
diff changeset
87 local default_id = {};
5796
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
88 return function (id, func)
8655
ba6a6a04b46c util.async: Allow nil as a guard key
Matthew Wild <mwild1@gmail.com>
parents: 8651
diff changeset
89 id = id or default_id;
8407
f652e1ea2f69 util.async: Factor out thread check into a function
Kim Alvefur <zash@zash.se>
parents: 8237
diff changeset
90 local thread = checkthread();
5796
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
91 local guard = guards[id];
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
92 if not guard then
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
93 guard = {};
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
94 guards[id] = guard;
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
95 log("debug", "New guard!");
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
96 else
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
97 table.insert(guard, thread);
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
98 log("debug", "Guarded. %d threads waiting.", #guard)
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
99 coroutine.yield("wait");
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
100 end
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
101 local function exit()
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
102 local next_waiting = table.remove(guard, 1);
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
103 if next_waiting then
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
104 log("debug", "guard: Executing next waiting thread (%d left)", #guard)
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
105 runner_continue(next_waiting);
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
106 else
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
107 log("debug", "Guard off duty.")
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
108 guards[id] = nil;
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
109 end
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
110 end
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
111 if func then
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
112 func();
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
113 exit();
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
114 return;
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
115 end
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
116 return exit;
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
117 end;
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
118 end
0b66cb959161 util.async: Add guarder method, to create guards to ensure only a single runner can pass through a section of code at a time
Matthew Wild <mwild1@gmail.com>
parents: 5794
diff changeset
119
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
120 local runner_mt = {};
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
121 runner_mt.__index = runner_mt;
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
122
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
123 local function runner_create_thread(func, self)
7725
f928695a2af1 util.async: Add annotation to ignore warning [luacheck]
Kim Alvefur <zash@zash.se>
parents: 7724
diff changeset
124 local thread = coroutine.create(function (self) -- luacheck: ignore 432/self
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
125 while true do
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
126 func(coroutine.yield("ready", self));
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
127 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
128 end);
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
129 assert(coroutine.resume(thread, self)); -- Start it up, it will return instantly to wait for the first input
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
130 return thread;
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
131 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
132
8681
0c077800cd70 util.async: Make parameters to async.runner() optional
Matthew Wild <mwild1@gmail.com>
parents: 8669
diff changeset
133 local function default_error_watcher(runner, err)
0c077800cd70 util.async: Make parameters to async.runner() optional
Matthew Wild <mwild1@gmail.com>
parents: 8669
diff changeset
134 runner:log("error", "Encountered error: %s", err);
0c077800cd70 util.async: Make parameters to async.runner() optional
Matthew Wild <mwild1@gmail.com>
parents: 8669
diff changeset
135 error(err);
0c077800cd70 util.async: Make parameters to async.runner() optional
Matthew Wild <mwild1@gmail.com>
parents: 8669
diff changeset
136 end
0c077800cd70 util.async: Make parameters to async.runner() optional
Matthew Wild <mwild1@gmail.com>
parents: 8669
diff changeset
137 local function default_func(f) f(); end
5790
959163e4d631 util.async: Make functions local
Matthew Wild <mwild1@gmail.com>
parents: 5788
diff changeset
138 local function runner(func, watchers, data)
8788
7a9b680a79fb util.async: Move runner id into log tag
Kim Alvefur <zash@zash.se>
parents: 8766
diff changeset
139 local id = new_id();
7a9b680a79fb util.async: Move runner id into log tag
Kim Alvefur <zash@zash.se>
parents: 8766
diff changeset
140 local _log = logger.init("runner" .. id);
8681
0c077800cd70 util.async: Make parameters to async.runner() optional
Matthew Wild <mwild1@gmail.com>
parents: 8669
diff changeset
141 return setmetatable({ func = func or default_func, thread = false, state = "ready", notified_state = "ready",
8788
7a9b680a79fb util.async: Move runner id into log tag
Kim Alvefur <zash@zash.se>
parents: 8766
diff changeset
142 queue = {}, watchers = watchers or { error = default_error_watcher }, data = data, id = id, _log = _log; }
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
143 , runner_mt);
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
144 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
145
7436
649b89b2c840 util.async: Add some more comments for clarity
Matthew Wild <mwild1@gmail.com>
parents: 7359
diff changeset
146 -- Add a task item for the runner to process
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
147 function runner_mt:run(input)
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
148 if input ~= nil then
8602
9901deadc068 util.async: Fix order of statements so queue count makes more sense
Matthew Wild <mwild1@gmail.com>
parents: 8601
diff changeset
149 table.insert(self.queue, input);
8600
96f20cf92b84 util.async: Add per-runner ids and add runner:log() method
Matthew Wild <mwild1@gmail.com>
parents: 8408
diff changeset
150 self:log("debug", "queued new work item, %d items queued", #self.queue);
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
151 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
152 if self.state ~= "ready" then
7436
649b89b2c840 util.async: Add some more comments for clarity
Matthew Wild <mwild1@gmail.com>
parents: 7359
diff changeset
153 -- The runner is busy. Indicate that the task item has been
649b89b2c840 util.async: Add some more comments for clarity
Matthew Wild <mwild1@gmail.com>
parents: 7359
diff changeset
154 -- queued, and return information about the current runner state
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
155 return true, self.state, #self.queue;
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
156 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
157
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
158 local q, thread = self.queue, self.thread;
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
159 if not thread or coroutine.status(thread) == "dead" then
8600
96f20cf92b84 util.async: Add per-runner ids and add runner:log() method
Matthew Wild <mwild1@gmail.com>
parents: 8408
diff changeset
160 self:log("debug", "creating new coroutine");
7436
649b89b2c840 util.async: Add some more comments for clarity
Matthew Wild <mwild1@gmail.com>
parents: 7359
diff changeset
161 -- Create a new coroutine for this runner
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
162 thread = runner_create_thread(self.func, self);
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
163 self.thread = thread;
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
164 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
165
7436
649b89b2c840 util.async: Add some more comments for clarity
Matthew Wild <mwild1@gmail.com>
parents: 7359
diff changeset
166 -- Process task item(s) while the queue is not empty, and we're not blocked
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
167 local n, state, err = #q, self.state, nil;
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
168 self.state = "running";
8600
96f20cf92b84 util.async: Add per-runner ids and add runner:log() method
Matthew Wild <mwild1@gmail.com>
parents: 8408
diff changeset
169 self:log("debug", "running main loop");
8603
dc5f3302a642 util.async: Bugfix, don't continue main loop while there is a pending error
Matthew Wild <mwild1@gmail.com>
parents: 8602
diff changeset
170 while n > 0 and state == "ready" and not err do
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
171 local consumed;
7436
649b89b2c840 util.async: Add some more comments for clarity
Matthew Wild <mwild1@gmail.com>
parents: 7359
diff changeset
172 -- Loop through queue items, and attempt to run them
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
173 for i = 1,n do
7724
20a69ef5570c util.async: Rename variable to avoid name clash [luacheck]
Kim Alvefur <zash@zash.se>
parents: 7436
diff changeset
174 local queued_input = q[i];
20a69ef5570c util.async: Rename variable to avoid name clash [luacheck]
Kim Alvefur <zash@zash.se>
parents: 7436
diff changeset
175 local ok, new_state = coroutine.resume(thread, queued_input);
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
176 if not ok then
7436
649b89b2c840 util.async: Add some more comments for clarity
Matthew Wild <mwild1@gmail.com>
parents: 7359
diff changeset
177 -- There was an error running the coroutine, save the error, mark runner as ready to begin again
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
178 consumed, state, err = i, "ready", debug.traceback(thread, new_state);
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
179 self.thread = nil;
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
180 break;
5791
2c98061b6b1e util.async: runner: Fix check for new state to recognise transition to 'waiting'
Matthew Wild <mwild1@gmail.com>
parents: 5790
diff changeset
181 elseif new_state == "wait" then
7436
649b89b2c840 util.async: Add some more comments for clarity
Matthew Wild <mwild1@gmail.com>
parents: 7359
diff changeset
182 -- Runner is blocked on waiting for a task item to complete
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
183 consumed, state = i, "waiting";
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
184 break;
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
185 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
186 end
7436
649b89b2c840 util.async: Add some more comments for clarity
Matthew Wild <mwild1@gmail.com>
parents: 7359
diff changeset
187 -- Loop ended - either queue empty because all tasks passed without blocking (consumed == nil)
649b89b2c840 util.async: Add some more comments for clarity
Matthew Wild <mwild1@gmail.com>
parents: 7359
diff changeset
188 -- or runner is blocked/errored, and consumed will contain the number of tasks processed so far
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
189 if not consumed then consumed = n; end
7436
649b89b2c840 util.async: Add some more comments for clarity
Matthew Wild <mwild1@gmail.com>
parents: 7359
diff changeset
190 -- Remove consumed items from the queue array
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
191 if q[n+1] ~= nil then
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
192 n = #q;
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
193 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
194 for i = 1, n do
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
195 q[i] = q[consumed+i];
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
196 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
197 n = #q;
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
198 end
7436
649b89b2c840 util.async: Add some more comments for clarity
Matthew Wild <mwild1@gmail.com>
parents: 7359
diff changeset
199 -- Runner processed all items it can, so save current runner state
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
200 self.state = state;
5794
66c3ad5d29ad util.async: Fix logic bug that prevented error watcher being called for runners
Matthew Wild <mwild1@gmail.com>
parents: 5793
diff changeset
201 if err or state ~= self.notified_state then
8604
1c8c7fd259c8 util.async: Log the non-error state as well when there is an error being processed
Matthew Wild <mwild1@gmail.com>
parents: 8603
diff changeset
202 self:log("debug", "changed state from %s to %s", self.notified_state, err and ("error ("..state..")") or state);
5794
66c3ad5d29ad util.async: Fix logic bug that prevented error watcher being called for runners
Matthew Wild <mwild1@gmail.com>
parents: 5793
diff changeset
203 if err then
66c3ad5d29ad util.async: Fix logic bug that prevented error watcher being called for runners
Matthew Wild <mwild1@gmail.com>
parents: 5793
diff changeset
204 state = "error"
66c3ad5d29ad util.async: Fix logic bug that prevented error watcher being called for runners
Matthew Wild <mwild1@gmail.com>
parents: 5793
diff changeset
205 else
66c3ad5d29ad util.async: Fix logic bug that prevented error watcher being called for runners
Matthew Wild <mwild1@gmail.com>
parents: 5793
diff changeset
206 self.notified_state = state;
66c3ad5d29ad util.async: Fix logic bug that prevented error watcher being called for runners
Matthew Wild <mwild1@gmail.com>
parents: 5793
diff changeset
207 end
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
208 local handler = self.watchers[state];
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
209 if handler then handler(self, err); end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
210 end
8615
e77b37de482e util.async: Behaviour change: continue to process queued items after errors
Matthew Wild <mwild1@gmail.com>
parents: 8613
diff changeset
211 if n > 0 then
e77b37de482e util.async: Behaviour change: continue to process queued items after errors
Matthew Wild <mwild1@gmail.com>
parents: 8613
diff changeset
212 return self:run();
e77b37de482e util.async: Behaviour change: continue to process queued items after errors
Matthew Wild <mwild1@gmail.com>
parents: 8613
diff changeset
213 end
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
214 return true, state, n;
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
215 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
216
7436
649b89b2c840 util.async: Add some more comments for clarity
Matthew Wild <mwild1@gmail.com>
parents: 7359
diff changeset
217 -- Add a task item to the queue without invoking the runner, even if it is idle
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
218 function runner_mt:enqueue(input)
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
219 table.insert(self.queue, input);
8602
9901deadc068 util.async: Fix order of statements so queue count makes more sense
Matthew Wild <mwild1@gmail.com>
parents: 8601
diff changeset
220 self:log("debug", "queued new work item, %d items queued", #self.queue);
8766
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
221 return self;
5788
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
222 end
3556f338caa3 util.async: New library to provide support around coroutine-based non-blocking functions
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
223
8600
96f20cf92b84 util.async: Add per-runner ids and add runner:log() method
Matthew Wild <mwild1@gmail.com>
parents: 8408
diff changeset
224 function runner_mt:log(level, fmt, ...)
8788
7a9b680a79fb util.async: Move runner id into log tag
Kim Alvefur <zash@zash.se>
parents: 8766
diff changeset
225 return self._log(level, fmt, ...);
8600
96f20cf92b84 util.async: Add per-runner ids and add runner:log() method
Matthew Wild <mwild1@gmail.com>
parents: 8408
diff changeset
226 end
96f20cf92b84 util.async: Add per-runner ids and add runner:log() method
Matthew Wild <mwild1@gmail.com>
parents: 8408
diff changeset
227
8766
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
228 function runner_mt:onready(f)
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
229 self.watchers.ready = f;
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
230 return self;
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
231 end
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
232
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
233 function runner_mt:onwaiting(f)
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
234 self.watchers.waiting = f;
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
235 return self;
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
236 end
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
237
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
238 function runner_mt:onerror(f)
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
239 self.watchers.error = f;
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
240 return self;
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
241 end
01264bc60c2b util.async: Add helper methods for setting watchers
Matthew Wild <mwild1@gmail.com>
parents: 8685
diff changeset
242
8648
ca710a71d730 util.async: Add ready() to check whether running in async context
Matthew Wild <mwild1@gmail.com>
parents: 8629
diff changeset
243 local function ready()
ca710a71d730 util.async: Add ready() to check whether running in async context
Matthew Wild <mwild1@gmail.com>
parents: 8629
diff changeset
244 return pcall(checkthread);
ca710a71d730 util.async: Add ready() to check whether running in async context
Matthew Wild <mwild1@gmail.com>
parents: 8629
diff changeset
245 end
ca710a71d730 util.async: Add ready() to check whether running in async context
Matthew Wild <mwild1@gmail.com>
parents: 8629
diff changeset
246
8649
9246f64d6f1e util.async: Add once() to create temporary runners
Matthew Wild <mwild1@gmail.com>
parents: 8648
diff changeset
247 return {
9246f64d6f1e util.async: Add once() to create temporary runners
Matthew Wild <mwild1@gmail.com>
parents: 8648
diff changeset
248 ready = ready;
9246f64d6f1e util.async: Add once() to create temporary runners
Matthew Wild <mwild1@gmail.com>
parents: 8648
diff changeset
249 waiter = waiter;
9246f64d6f1e util.async: Add once() to create temporary runners
Matthew Wild <mwild1@gmail.com>
parents: 8648
diff changeset
250 guarder = guarder;
8651
1b7c5933b215 util.async: Add sleep() method
Matthew Wild <mwild1@gmail.com>
parents: 8649
diff changeset
251 runner = runner;
8649
9246f64d6f1e util.async: Add once() to create temporary runners
Matthew Wild <mwild1@gmail.com>
parents: 8648
diff changeset
252 };