Software `/` code `/` prosody

Annotate

plugins/mod_csi_simple.lua @ 11913:75d69e4c54a2

mod_csi_simple: Unlock writes after event, to allow things to be queued E.g. mod_smacks could queue an <r>, which would be more likely to be included in the same write and TCP segment as the previously buffered data, reducing syscalls and network packets needing to be sent.

author	Kim Alvefur <zash@zash.se>
date	Tue, 16 Nov 2021 13:37:18 +0100
parent	11834:f54d9abc4e14
child	11914:8f7946ce7d66

rev	line source
10724 7835b9f14cb8 mod_csi_simple: Allow configuring extra tags indicating importance Kim Alvefur <zash@zash.se> parents: 10414 diff changeset	1 -- Copyright (C) 2016-2020 Kim Alvefur
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	2 --
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	3 -- This project is MIT/X11 licensed. Please see the
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	4 -- COPYING file in the source package for more information.
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	5 --
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	6
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	7 module:depends"csi"
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	8
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	9 local jid = require "util.jid";
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	10 local st = require "util.stanza";
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	11 local dt = require "util.datetime";
9912 601f9781a605 mod_csi_simple: Count buffered items and flush when it reaches configured limit Kim Alvefur <zash@zash.se> parents: 9911 diff changeset	12 local filters = require "util.filters";
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	13
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	14 local queue_size = module:get_option_number("csi_queue_size", 256);
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	15
10724 7835b9f14cb8 mod_csi_simple: Allow configuring extra tags indicating importance Kim Alvefur <zash@zash.se> parents: 10414 diff changeset	16 local important_payloads = module:get_option_set("csi_important_payloads", { });
7835b9f14cb8 mod_csi_simple: Allow configuring extra tags indicating importance Kim Alvefur <zash@zash.se> parents: 10414 diff changeset	17
10806 24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	18 function is_important(stanza) --> boolean, reason: string
10832 7395f6e68dba mod_csi_simple: Report whitespace keepalives Kim Alvefur <zash@zash.se> parents: 10831 diff changeset	19 if stanza == " " then
7395f6e68dba mod_csi_simple: Report whitespace keepalives Kim Alvefur <zash@zash.se> parents: 10831 diff changeset	20 return true, "whitespace keepalive";
7395f6e68dba mod_csi_simple: Report whitespace keepalives Kim Alvefur <zash@zash.se> parents: 10831 diff changeset	21 elseif type(stanza) == "string" then
10831 7dd7cdb43181 mod_csi_simple: Identify raw string data in logging and stats Kim Alvefur <zash@zash.se> parents: 10830 diff changeset	22 return true, "raw data";
7dd7cdb43181 mod_csi_simple: Identify raw string data in logging and stats Kim Alvefur <zash@zash.se> parents: 10830 diff changeset	23 elseif not st.is_stanza(stanza) then
10833 ac691f305ea7 mod_csi_simple: Report whatever's not a stirng and not a stanza Kim Alvefur <zash@zash.se> parents: 10832 diff changeset	24 -- This should probably never happen
ac691f305ea7 mod_csi_simple: Report whatever's not a stirng and not a stanza Kim Alvefur <zash@zash.se> parents: 10832 diff changeset	25 return true, type(stanza);
9632 fdefc43bffff mod_csi_simple: Consider non-stanza objects important Kim Alvefur <zash@zash.se> parents: 9631 diff changeset	26 end
10770 b4cbe72966c9 mod_csi_simple: Consider nonzas important Kim Alvefur <zash@zash.se> parents: 10769 diff changeset	27 if stanza.attr.xmlns ~= nil then
b4cbe72966c9 mod_csi_simple: Consider nonzas important Kim Alvefur <zash@zash.se> parents: 10769 diff changeset	28 -- stream errors, stream management etc
10807 b92afa0a4119 mod_csi_simple: Add short reasons to report Kim Alvefur <zash@zash.se> parents: 10806 diff changeset	29 return true, "nonza";
10770 b4cbe72966c9 mod_csi_simple: Consider nonzas important Kim Alvefur <zash@zash.se> parents: 10769 diff changeset	30 end
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	31 local st_name = stanza.name;
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	32 if not st_name then return false; end
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	33 local st_type = stanza.attr.type;
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	34 if st_name == "presence" then
10824 c8430ee33967 mod_csi_simple: Fix treating presence errors as presence updates Kim Alvefur <zash@zash.se> parents: 10823 diff changeset	35 if st_type == nil or st_type == "unavailable" or st_type == "error" then
10807 b92afa0a4119 mod_csi_simple: Add short reasons to report Kim Alvefur <zash@zash.se> parents: 10806 diff changeset	36 return false, "presence update";
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	37 end
10801 2b97aac0ea3c mod_csi_simple: Don't consider presence errors as important Kim Alvefur <zash@zash.se> parents: 10772 diff changeset	38 -- TODO Some MUC awareness, e.g. check for the 'this relates to you' status code
10807 b92afa0a4119 mod_csi_simple: Add short reasons to report Kim Alvefur <zash@zash.se> parents: 10806 diff changeset	39 return true, "subscription request";
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	40 elseif st_name == "message" then
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	41 if st_type == "headline" then
10807 b92afa0a4119 mod_csi_simple: Add short reasons to report Kim Alvefur <zash@zash.se> parents: 10806 diff changeset	42 -- Headline messages are ephemeral by definition
b92afa0a4119 mod_csi_simple: Add short reasons to report Kim Alvefur <zash@zash.se> parents: 10806 diff changeset	43 return false, "headline";
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	44 end
10801 2b97aac0ea3c mod_csi_simple: Don't consider presence errors as important Kim Alvefur <zash@zash.se> parents: 10772 diff changeset	45 if st_type == "error" then
10807 b92afa0a4119 mod_csi_simple: Add short reasons to report Kim Alvefur <zash@zash.se> parents: 10806 diff changeset	46 return true, "delivery failure";
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	47 end
9768 ab12fd48e124 mod_csi_simple: Consider messages forwarded from another of the users clients as important (fixes part of #1250) Kim Alvefur <zash@zash.se> parents: 9767 diff changeset	48 if stanza:get_child("sent", "urn:xmpp:carbons:2") then
10807 b92afa0a4119 mod_csi_simple: Add short reasons to report Kim Alvefur <zash@zash.se> parents: 10806 diff changeset	49 return true, "carbon";
9768 ab12fd48e124 mod_csi_simple: Consider messages forwarded from another of the users clients as important (fixes part of #1250) Kim Alvefur <zash@zash.se> parents: 9767 diff changeset	50 end
9769 150e9574c149 mod_csi_simple: Unpack Carbons-forwarded messages (fixes part of #1250) Kim Alvefur <zash@zash.se> parents: 9768 diff changeset	51 local forwarded = stanza:find("{urn:xmpp:carbons:2}received/{urn:xmpp:forward:0}/{jabber:client}message");
150e9574c149 mod_csi_simple: Unpack Carbons-forwarded messages (fixes part of #1250) Kim Alvefur <zash@zash.se> parents: 9768 diff changeset	52 if forwarded then
150e9574c149 mod_csi_simple: Unpack Carbons-forwarded messages (fixes part of #1250) Kim Alvefur <zash@zash.se> parents: 9768 diff changeset	53 stanza = forwarded;
150e9574c149 mod_csi_simple: Unpack Carbons-forwarded messages (fixes part of #1250) Kim Alvefur <zash@zash.se> parents: 9768 diff changeset	54 end
9767 57ceffb13963 mod_csi_simple: Tweak check for <body> Kim Alvefur <zash@zash.se> parents: 9651 diff changeset	55 if stanza:get_child("body") then
10807 b92afa0a4119 mod_csi_simple: Add short reasons to report Kim Alvefur <zash@zash.se> parents: 10806 diff changeset	56 return true, "body";
9767 57ceffb13963 mod_csi_simple: Tweak check for <body> Kim Alvefur <zash@zash.se> parents: 9651 diff changeset	57 end
9770 76cb409db537 mod_csi_simple: Consider messages with subject (eg MUC joins) (fixes part of #1250) Kim Alvefur <zash@zash.se> parents: 9769 diff changeset	58 if stanza:get_child("subject") then
10807 b92afa0a4119 mod_csi_simple: Add short reasons to report Kim Alvefur <zash@zash.se> parents: 10806 diff changeset	59 -- Last step of a MUC join
b92afa0a4119 mod_csi_simple: Add short reasons to report Kim Alvefur <zash@zash.se> parents: 10806 diff changeset	60 return true, "subject";
9770 76cb409db537 mod_csi_simple: Consider messages with subject (eg MUC joins) (fixes part of #1250) Kim Alvefur <zash@zash.se> parents: 9769 diff changeset	61 end
9771 bf92f37de137 mod_csi_simple: Consider messages encrypted payload as important (fixes part of #1250) Kim Alvefur <zash@zash.se> parents: 9770 diff changeset	62 if stanza:get_child("encryption", "urn:xmpp:eme:0") then
10807 b92afa0a4119 mod_csi_simple: Add short reasons to report Kim Alvefur <zash@zash.se> parents: 10806 diff changeset	63 -- Since we can't know what an encrypted message contains, we assume it's important
b92afa0a4119 mod_csi_simple: Add short reasons to report Kim Alvefur <zash@zash.se> parents: 10806 diff changeset	64 -- XXX Experimental XEP
b92afa0a4119 mod_csi_simple: Add short reasons to report Kim Alvefur <zash@zash.se> parents: 10806 diff changeset	65 return true, "encrypted";
9771 bf92f37de137 mod_csi_simple: Consider messages encrypted payload as important (fixes part of #1250) Kim Alvefur <zash@zash.se> parents: 9770 diff changeset	66 end
10733 89e0f5cb60a1 mod_csi_simple: Consider MUC invites important Kim Alvefur <zash@zash.se> parents: 10724 diff changeset	67 if stanza:get_child("x", "jabber:x:conference") or stanza:find("{http://jabber.org/protocol/muc#user}x/invite") then
10807 b92afa0a4119 mod_csi_simple: Add short reasons to report Kim Alvefur <zash@zash.se> parents: 10806 diff changeset	68 return true, "invite";
10733 89e0f5cb60a1 mod_csi_simple: Consider MUC invites important Kim Alvefur <zash@zash.se> parents: 10724 diff changeset	69 end
10822 af42448a8e98 mod_csi_simple: Fix unintentional order of rules from merge Kim Alvefur <zash@zash.se> parents: 10817 diff changeset	70 if stanza:get_child(nil, "urn:xmpp:jingle-message:0") then
11260 08b397c21805 mod_csi_simple,mod_carbons,mod_mam: Update comment about XEP-0353 Kim Alvefur <zash@zash.se> parents: 10833 diff changeset	71 -- XXX Experimental XEP
10822 af42448a8e98 mod_csi_simple: Fix unintentional order of rules from merge Kim Alvefur <zash@zash.se> parents: 10817 diff changeset	72 return true, "jingle call";
af42448a8e98 mod_csi_simple: Fix unintentional order of rules from merge Kim Alvefur <zash@zash.se> parents: 10817 diff changeset	73 end
10724 7835b9f14cb8 mod_csi_simple: Allow configuring extra tags indicating importance Kim Alvefur <zash@zash.se> parents: 10414 diff changeset	74 for important in important_payloads do
7835b9f14cb8 mod_csi_simple: Allow configuring extra tags indicating importance Kim Alvefur <zash@zash.se> parents: 10414 diff changeset	75 if stanza:find(important) then
7835b9f14cb8 mod_csi_simple: Allow configuring extra tags indicating importance Kim Alvefur <zash@zash.se> parents: 10414 diff changeset	76 return true;
7835b9f14cb8 mod_csi_simple: Allow configuring extra tags indicating importance Kim Alvefur <zash@zash.se> parents: 10414 diff changeset	77 end
9771 bf92f37de137 mod_csi_simple: Consider messages encrypted payload as important (fixes part of #1250) Kim Alvefur <zash@zash.se> parents: 9770 diff changeset	78 end
9767 57ceffb13963 mod_csi_simple: Tweak check for <body> Kim Alvefur <zash@zash.se> parents: 9651 diff changeset	79 return false;
10772 31e702c5f475 mod_csi_simple: Explicitly mention iq stanzas Kim Alvefur <zash@zash.se> parents: 10771 diff changeset	80 elseif st_name == "iq" then
31e702c5f475 mod_csi_simple: Explicitly mention iq stanzas Kim Alvefur <zash@zash.se> parents: 10771 diff changeset	81 return true;
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	82 end
10806 24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	83 end
24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	84
24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	85 module:hook("csi-is-stanza-important", function (event)
24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	86 local important, why = is_important(event.stanza);
24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	87 event.reason = why;
24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	88 return important;
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	89 end, -1);
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	90
10806 24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	91 local function should_flush(stanza, session, ctr) --> boolean, reason: string
24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	92 if ctr >= queue_size then
24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	93 return true, "queue size limit reached";
24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	94 end
24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	95 local event = { stanza = stanza, session = session };
24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	96 local ret = module:fire_event("csi-is-stanza-important", event)
24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	97 return ret, event.reason;
24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	98 end
24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	99
9911 ed011935c22d mod_csi_simple: Break out stanza timestamping into a function for future reuse Kim Alvefur <zash@zash.se> parents: 9909 diff changeset	100 local function with_timestamp(stanza, from)
ed011935c22d mod_csi_simple: Break out stanza timestamping into a function for future reuse Kim Alvefur <zash@zash.se> parents: 9909 diff changeset	101 if st.is_stanza(stanza) and stanza.attr.xmlns == nil and stanza.name ~= "iq" then
ed011935c22d mod_csi_simple: Break out stanza timestamping into a function for future reuse Kim Alvefur <zash@zash.se> parents: 9909 diff changeset	102 stanza = st.clone(stanza);
ed011935c22d mod_csi_simple: Break out stanza timestamping into a function for future reuse Kim Alvefur <zash@zash.se> parents: 9909 diff changeset	103 stanza:add_direct_child(st.stanza("delay", {xmlns = "urn:xmpp:delay", from = from, stamp = dt.datetime()}));
ed011935c22d mod_csi_simple: Break out stanza timestamping into a function for future reuse Kim Alvefur <zash@zash.se> parents: 9909 diff changeset	104 end
ed011935c22d mod_csi_simple: Break out stanza timestamping into a function for future reuse Kim Alvefur <zash@zash.se> parents: 9909 diff changeset	105 return stanza;
ed011935c22d mod_csi_simple: Break out stanza timestamping into a function for future reuse Kim Alvefur <zash@zash.se> parents: 9909 diff changeset	106 end
ed011935c22d mod_csi_simple: Break out stanza timestamping into a function for future reuse Kim Alvefur <zash@zash.se> parents: 9909 diff changeset	107
11834 f54d9abc4e14 mod_csi_simple: Provide custom set of timing buckets Kim Alvefur <zash@zash.se> parents: 11573 diff changeset	108 local measure_buffer_hold = module:measure("buffer_hold", "times",
f54d9abc4e14 mod_csi_simple: Provide custom set of timing buckets Kim Alvefur <zash@zash.se> parents: 11573 diff changeset	109 { buckets = { 0.1; 1; 5; 10; 15; 30; 60; 120; 180; 300; 600; 900 } });
10829 67a09706e56e mod_csi_simple: Record stats of how long buffers are held Kim Alvefur <zash@zash.se> parents: 10828 diff changeset	110
11573 cb5748f94840 mod_csi_simple: convert to use new metric interface for flush reasons Jonas Schäfer <jonas@wielicki.name> parents: 11425 diff changeset	111 local flush_reasons = module:metric(
cb5748f94840 mod_csi_simple: convert to use new metric interface for flush reasons Jonas Schäfer <jonas@wielicki.name> parents: 11425 diff changeset	112 "counter", "flushes", "",
cb5748f94840 mod_csi_simple: convert to use new metric interface for flush reasons Jonas Schäfer <jonas@wielicki.name> parents: 11425 diff changeset	113 "CSI queue flushes",
cb5748f94840 mod_csi_simple: convert to use new metric interface for flush reasons Jonas Schäfer <jonas@wielicki.name> parents: 11425 diff changeset	114 { "reason" }
cb5748f94840 mod_csi_simple: convert to use new metric interface for flush reasons Jonas Schäfer <jonas@wielicki.name> parents: 11425 diff changeset	115 );
10830 8889d5037aca mod_csi_simple: Collect stats on flush reasons Kim Alvefur <zash@zash.se> parents: 10829 diff changeset	116
9912 601f9781a605 mod_csi_simple: Count buffered items and flush when it reaches configured limit Kim Alvefur <zash@zash.se> parents: 9911 diff changeset	117 local function manage_buffer(stanza, session)
601f9781a605 mod_csi_simple: Count buffered items and flush when it reaches configured limit Kim Alvefur <zash@zash.se> parents: 9911 diff changeset	118 local ctr = session.csi_counter or 0;
10806 24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	119 local flush, why = should_flush(stanza, session, ctr);
24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	120 if flush then
10829 67a09706e56e mod_csi_simple: Record stats of how long buffers are held Kim Alvefur <zash@zash.se> parents: 10828 diff changeset	121 if session.csi_measure_buffer_hold then
67a09706e56e mod_csi_simple: Record stats of how long buffers are held Kim Alvefur <zash@zash.se> parents: 10828 diff changeset	122 session.csi_measure_buffer_hold();
67a09706e56e mod_csi_simple: Record stats of how long buffers are held Kim Alvefur <zash@zash.se> parents: 10828 diff changeset	123 session.csi_measure_buffer_hold = nil;
67a09706e56e mod_csi_simple: Record stats of how long buffers are held Kim Alvefur <zash@zash.se> parents: 10828 diff changeset	124 end
11573 cb5748f94840 mod_csi_simple: convert to use new metric interface for flush reasons Jonas Schäfer <jonas@wielicki.name> parents: 11425 diff changeset	125 flush_reasons:with_labels(why or "important"):add(1);
10806 24e2b571d29a mod_csi_simple: Refactor to allow logging reason for buffer flush Kim Alvefur <zash@zash.se> parents: 10801 diff changeset	126 session.log("debug", "Flushing buffer (%s; queue size is %d)", why or "important", session.csi_counter);
11379 5c820553ef82 mod_csi_simple: Set session state to 'flushing' while doing so Kim Alvefur <zash@zash.se> parents: 11260 diff changeset	127 session.state = "flushing";
11380 9a1758c5aaa4 mod_csi_simple: Fire event when flushing queue Kim Alvefur <zash@zash.se> parents: 11379 diff changeset	128 module:fire_event("csi-flushing", { session = session });
11913 75d69e4c54a2 mod_csi_simple: Unlock writes after event, to allow things to be queued Kim Alvefur <zash@zash.se> parents: 11834 diff changeset	129 session.conn:resume_writes();
9912 601f9781a605 mod_csi_simple: Count buffered items and flush when it reaches configured limit Kim Alvefur <zash@zash.se> parents: 9911 diff changeset	130 else
10808 0d365c0ee9fe mod_csi_simple: Log reasons for not flushing Kim Alvefur <zash@zash.se> parents: 10807 diff changeset	131 session.log("debug", "Holding buffer (%s; queue size is %d)", why or "unimportant", session.csi_counter);
9912 601f9781a605 mod_csi_simple: Count buffered items and flush when it reaches configured limit Kim Alvefur <zash@zash.se> parents: 9911 diff changeset	132 stanza = with_timestamp(stanza, jid.join(session.username, session.host))
601f9781a605 mod_csi_simple: Count buffered items and flush when it reaches configured limit Kim Alvefur <zash@zash.se> parents: 9911 diff changeset	133 end
601f9781a605 mod_csi_simple: Count buffered items and flush when it reaches configured limit Kim Alvefur <zash@zash.se> parents: 9911 diff changeset	134 session.csi_counter = ctr + 1;
601f9781a605 mod_csi_simple: Count buffered items and flush when it reaches configured limit Kim Alvefur <zash@zash.se> parents: 9911 diff changeset	135 return stanza;
601f9781a605 mod_csi_simple: Count buffered items and flush when it reaches configured limit Kim Alvefur <zash@zash.se> parents: 9911 diff changeset	136 end
601f9781a605 mod_csi_simple: Count buffered items and flush when it reaches configured limit Kim Alvefur <zash@zash.se> parents: 9911 diff changeset	137
9913 7d78b24d8449 mod_csi_simple: Trigger buffer flush on seeing incoming data Kim Alvefur <zash@zash.se> parents: 9912 diff changeset	138 local function flush_buffer(data, session)
10828 c12ed21f877e mod_csi_simple: Change debug message of client-triggered flush for coherence Kim Alvefur <zash@zash.se> parents: 10827 diff changeset	139 session.log("debug", "Flushing buffer (%s; queue size is %d)", "client activity", session.csi_counter);
11573 cb5748f94840 mod_csi_simple: convert to use new metric interface for flush reasons Jonas Schäfer <jonas@wielicki.name> parents: 11425 diff changeset	140 flush_reasons:with_labels("client activity"):add(1);
10829 67a09706e56e mod_csi_simple: Record stats of how long buffers are held Kim Alvefur <zash@zash.se> parents: 10828 diff changeset	141 if session.csi_measure_buffer_hold then
67a09706e56e mod_csi_simple: Record stats of how long buffers are held Kim Alvefur <zash@zash.se> parents: 10828 diff changeset	142 session.csi_measure_buffer_hold();
67a09706e56e mod_csi_simple: Record stats of how long buffers are held Kim Alvefur <zash@zash.se> parents: 10828 diff changeset	143 session.csi_measure_buffer_hold = nil;
67a09706e56e mod_csi_simple: Record stats of how long buffers are held Kim Alvefur <zash@zash.se> parents: 10828 diff changeset	144 end
9913 7d78b24d8449 mod_csi_simple: Trigger buffer flush on seeing incoming data Kim Alvefur <zash@zash.se> parents: 9912 diff changeset	145 session.conn:resume_writes();
7d78b24d8449 mod_csi_simple: Trigger buffer flush on seeing incoming data Kim Alvefur <zash@zash.se> parents: 9912 diff changeset	146 return data;
7d78b24d8449 mod_csi_simple: Trigger buffer flush on seeing incoming data Kim Alvefur <zash@zash.se> parents: 9912 diff changeset	147 end
7d78b24d8449 mod_csi_simple: Trigger buffer flush on seeing incoming data Kim Alvefur <zash@zash.se> parents: 9912 diff changeset	148
9918 6e9dcec259d0 mod_csi_simple: Separate out functions to enable/disable optimizations Kim Alvefur <zash@zash.se> parents: 9917 diff changeset	149 function enable_optimizations(session)
10277 45a58127a3e5 mod_csi_simple: Remove duplicated check for connection Kim Alvefur <zash@zash.se> parents: 10025 diff changeset	150 if session.conn and session.conn.pause_writes then
9909 3229be01a08a mod_csi_simple: Use write locks in net.server if available Kim Alvefur <zash@zash.se> parents: 9771 diff changeset	151 session.conn:pause_writes();
10829 67a09706e56e mod_csi_simple: Record stats of how long buffers are held Kim Alvefur <zash@zash.se> parents: 10828 diff changeset	152 session.csi_measure_buffer_hold = measure_buffer_hold();
10827 d8e83d94a99a mod_csi_simple: Reset queue counter to zero when enabling Kim Alvefur <zash@zash.se> parents: 10826 diff changeset	153 session.csi_counter = 0;
9912 601f9781a605 mod_csi_simple: Count buffered items and flush when it reaches configured limit Kim Alvefur <zash@zash.se> parents: 9911 diff changeset	154 filters.add_filter(session, "stanzas/out", manage_buffer);
9913 7d78b24d8449 mod_csi_simple: Trigger buffer flush on seeing incoming data Kim Alvefur <zash@zash.se> parents: 9912 diff changeset	155 filters.add_filter(session, "bytes/in", flush_buffer);
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	156 else
9917 45b5528b128a mod_csi_simple: Remove old "pump" queue/buffer method, handled in net.server now Kim Alvefur <zash@zash.se> parents: 9914 diff changeset	157 session.log("warn", "Session connection does not support write pausing");
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	158 end
9918 6e9dcec259d0 mod_csi_simple: Separate out functions to enable/disable optimizations Kim Alvefur <zash@zash.se> parents: 9917 diff changeset	159 end
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	160
9922 06bf5ccd859f mod_csi_simple: Fix type in function name Matthew Wild <mwild1@gmail.com> parents: 9921 diff changeset	161 function disable_optimizations(session)
10303 c434bff22b14 mod_csi_simple: Always remove session filters when disabling CSI Kim Alvefur <zash@zash.se> parents: 10302 diff changeset	162 filters.remove_filter(session, "stanzas/out", manage_buffer);
c434bff22b14 mod_csi_simple: Always remove session filters when disabling CSI Kim Alvefur <zash@zash.se> parents: 10302 diff changeset	163 filters.remove_filter(session, "bytes/in", flush_buffer);
10826 4f7226d5ee30 mod_csi_simple: Forget queue counter when disabling optimizations Kim Alvefur <zash@zash.se> parents: 10825 diff changeset	164 session.csi_counter = nil;
10829 67a09706e56e mod_csi_simple: Record stats of how long buffers are held Kim Alvefur <zash@zash.se> parents: 10828 diff changeset	165 if session.csi_measure_buffer_hold then
67a09706e56e mod_csi_simple: Record stats of how long buffers are held Kim Alvefur <zash@zash.se> parents: 10828 diff changeset	166 session.csi_measure_buffer_hold();
67a09706e56e mod_csi_simple: Record stats of how long buffers are held Kim Alvefur <zash@zash.se> parents: 10828 diff changeset	167 session.csi_measure_buffer_hold = nil;
67a09706e56e mod_csi_simple: Record stats of how long buffers are held Kim Alvefur <zash@zash.se> parents: 10828 diff changeset	168 end
10277 45a58127a3e5 mod_csi_simple: Remove duplicated check for connection Kim Alvefur <zash@zash.se> parents: 10025 diff changeset	169 if session.conn and session.conn.resume_writes then
9909 3229be01a08a mod_csi_simple: Use write locks in net.server if available Kim Alvefur <zash@zash.se> parents: 9771 diff changeset	170 session.conn:resume_writes();
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	171 end
9918 6e9dcec259d0 mod_csi_simple: Separate out functions to enable/disable optimizations Kim Alvefur <zash@zash.se> parents: 9917 diff changeset	172 end
6e9dcec259d0 mod_csi_simple: Separate out functions to enable/disable optimizations Kim Alvefur <zash@zash.se> parents: 9917 diff changeset	173
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	174 module:hook("csi-client-inactive", function (event)
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	175 local session = event.origin;
9918 6e9dcec259d0 mod_csi_simple: Separate out functions to enable/disable optimizations Kim Alvefur <zash@zash.se> parents: 9917 diff changeset	176 enable_optimizations(session);
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	177 end);
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	178
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	179 module:hook("csi-client-active", function (event)
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	180 local session = event.origin;
9922 06bf5ccd859f mod_csi_simple: Fix type in function name Matthew Wild <mwild1@gmail.com> parents: 9921 diff changeset	181 disable_optimizations(session);
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	182 end);
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	183
10025 4498f601516d mod_csi_simple: Disable optimizations on disconnect (fixes #1358) Kim Alvefur <zash@zash.se> parents: 9923 diff changeset	184 module:hook("pre-resource-unbind", function (event)
4498f601516d mod_csi_simple: Disable optimizations on disconnect (fixes #1358) Kim Alvefur <zash@zash.se> parents: 9923 diff changeset	185 local session = event.session;
4498f601516d mod_csi_simple: Disable optimizations on disconnect (fixes #1358) Kim Alvefur <zash@zash.se> parents: 9923 diff changeset	186 disable_optimizations(session);
10414 51ebfdeccad7 mod_csi_simple: Make sure to disable optimizations before mod_smacks (thanks pep.) Kim Alvefur <zash@zash.se> parents: 10303 diff changeset	187 end, 1);
9912 601f9781a605 mod_csi_simple: Count buffered items and flush when it reaches configured limit Kim Alvefur <zash@zash.se> parents: 9911 diff changeset	188
601f9781a605 mod_csi_simple: Count buffered items and flush when it reaches configured limit Kim Alvefur <zash@zash.se> parents: 9911 diff changeset	189 module:hook("c2s-ondrain", function (event)
601f9781a605 mod_csi_simple: Count buffered items and flush when it reaches configured limit Kim Alvefur <zash@zash.se> parents: 9911 diff changeset	190 local session = event.session;
11379 5c820553ef82 mod_csi_simple: Set session state to 'flushing' while doing so Kim Alvefur <zash@zash.se> parents: 11260 diff changeset	191 if (session.state == "flushing" or session.state == "inactive") and session.conn and session.conn.pause_writes then
5c820553ef82 mod_csi_simple: Set session state to 'flushing' while doing so Kim Alvefur <zash@zash.se> parents: 11260 diff changeset	192 session.state = "inactive";
9923 e83dfcdeab59 mod_csi_simple: Include queue size in debug messages Kim Alvefur <zash@zash.se> parents: 9922 diff changeset	193 session.conn:pause_writes();
10829 67a09706e56e mod_csi_simple: Record stats of how long buffers are held Kim Alvefur <zash@zash.se> parents: 10828 diff changeset	194 session.csi_measure_buffer_hold = measure_buffer_hold();
9923 e83dfcdeab59 mod_csi_simple: Include queue size in debug messages Kim Alvefur <zash@zash.se> parents: 9922 diff changeset	195 session.log("debug", "Buffer flushed, resuming inactive mode (queue size was %d)", session.csi_counter);
9912 601f9781a605 mod_csi_simple: Count buffered items and flush when it reaches configured limit Kim Alvefur <zash@zash.se> parents: 9911 diff changeset	196 session.csi_counter = 0;
9589 aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	197 end
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	198 end);
aeb054ee88c5 mod_csi_simple: Import modified version of mod_csi_pump from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	199
9919 d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	200 function module.load()
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	201 for _, user_session in pairs(prosody.hosts[module.host].sessions) do
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	202 for _, session in pairs(user_session.sessions) do
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	203 if session.state == "inactive" then
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	204 enable_optimizations(session);
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	205 end
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	206 end
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	207 end
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	208 end
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	209
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	210 function module.unload()
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	211 for _, user_session in pairs(prosody.hosts[module.host].sessions) do
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	212 for _, session in pairs(user_session.sessions) do
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	213 if session.state == "inactive" then
9922 06bf5ccd859f mod_csi_simple: Fix type in function name Matthew Wild <mwild1@gmail.com> parents: 9921 diff changeset	214 disable_optimizations(session);
9919 d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	215 end
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	216 end
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	217 end
d602a495409b mod_csi_simple: Disable optimizations on unload and re-enable on load Kim Alvefur <zash@zash.se> parents: 9918 diff changeset	218 end
11401 228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	219
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	220 function module.command(arg)
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	221 if arg[1] ~= "test" then
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	222 print("Usage: "..module.name.." test < test-stream.xml")
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	223 print("");
11425 fc7706fe115d mod_csi_simple: s/algoritm/algorithm/ [codespell] Kim Alvefur <zash@zash.se> parents: 11401 diff changeset	224 print("Provide a series of stanzas to test against importance algorithm");
11401 228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	225 return 1;
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	226 end
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	227 -- luacheck: ignore 212/self
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	228 local xmppstream = require "util.xmppstream";
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	229 local input_session = { notopen = true }
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	230 local stream_callbacks = { stream_ns = "jabber:client", default_ns = "jabber:client" };
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	231 function stream_callbacks:handlestanza(stanza)
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	232 local important, because = is_important(stanza);
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	233 print("--");
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	234 print(stanza:indent(nil, " "));
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	235 -- :pretty_print() maybe?
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	236 if important then
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	237 print((because or "unspecified reason").. " -> important");
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	238 else
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	239 print((because or "unspecified reason").. " -> unimportant");
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	240 end
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	241 end
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	242 local input_stream = xmppstream.new(input_session, stream_callbacks);
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	243 input_stream:reset();
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	244 input_stream:feed(st.stanza("stream", { xmlns = "jabber:client" }):top_tag());
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	245 input_session.notopen = nil;
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	246
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	247 for line in io.lines() do
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	248 input_stream:feed(line);
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	249 end
228bd43fbc3d mod_csi_simple: Add command to test importance algorithm on stream of stanzas Kim Alvefur <zash@zash.se> parents: 11380 diff changeset	250 end

prosody

75d69e4c54a2

plugins/mod_csi_simple.lua

Software / code / prosody

Annotate

plugins/mod_csi_simple.lua @ 11913:75d69e4c54a2

Software `/` code `/` prosody