Software `/` code `/` prosody

Annotate

plugins/mod_mam/mod_mam.lua @ 13014:06453c564141

util.startup: Add prosody.started promise to easily execute code after startup To avoid a race where server-started fires before the promise function body is run (on next tick), I moved server-started to fire on the next tick, which seems sensible anyway. Errors are logged, I'm not sure if we ought to be doing something more here. I'm sure we'll find out.

author	Matthew Wild <mwild1@gmail.com>
date	Sat, 01 Apr 2023 11:56:38 +0100
parent	12977:74b9e05af71e
child	13055:e732f9dfdfc8

rev	line source
7851 80ee0d9cd56f mod_mam: Normalize copyright headers Kim Alvefur <zash@zash.se> parents: 7850 diff changeset	1 -- Prosody IM
80ee0d9cd56f mod_mam: Normalize copyright headers Kim Alvefur <zash@zash.se> parents: 7850 diff changeset	2 -- Copyright (C) 2008-2017 Matthew Wild
80ee0d9cd56f mod_mam: Normalize copyright headers Kim Alvefur <zash@zash.se> parents: 7850 diff changeset	3 -- Copyright (C) 2008-2017 Waqas Hussain
11272 44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	4 -- Copyright (C) 2011-2021 Kim Alvefur
7851 80ee0d9cd56f mod_mam: Normalize copyright headers Kim Alvefur <zash@zash.se> parents: 7850 diff changeset	5 --
80ee0d9cd56f mod_mam: Normalize copyright headers Kim Alvefur <zash@zash.se> parents: 7850 diff changeset	6 -- This project is MIT/X11 licensed. Please see the
80ee0d9cd56f mod_mam: Normalize copyright headers Kim Alvefur <zash@zash.se> parents: 7850 diff changeset	7 -- COPYING file in the source package for more information.
80ee0d9cd56f mod_mam: Normalize copyright headers Kim Alvefur <zash@zash.se> parents: 7850 diff changeset	8 --
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	9 -- XEP-0313: Message Archive Management for Prosody
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	10 --
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	11
7903 036c771b842a mod_mam: Update namespace to XEP-0313 v0.6 Kim Alvefur <zash@zash.se> parents: 7902 diff changeset	12 local xmlns_mam = "urn:xmpp:mam:2";
11268 e3f6f0b39e7b mod_mam: Advertise extended MAM 0.7.x behind a feature flag Kim Alvefur <zash@zash.se> parents: 11260 diff changeset	13 local xmlns_mam_ext = "urn:xmpp:mam:2#extended";
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	14 local xmlns_delay = "urn:xmpp:delay";
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	15 local xmlns_forward = "urn:xmpp:forward:0";
7890 9e4de27e8e08 mod_mam: Move stanza ID namespace to a common variable Kim Alvefur <zash@zash.se> parents: 7882 diff changeset	16 local xmlns_st_id = "urn:xmpp:sid:0";
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	17
12977 74b9e05af71e plugins: Prefix module imports with prosody namespace Kim Alvefur <zash@zash.se> parents: 12860 diff changeset	18 local um = require "prosody.core.usermanager";
74b9e05af71e plugins: Prefix module imports with prosody namespace Kim Alvefur <zash@zash.se> parents: 12860 diff changeset	19 local st = require "prosody.util.stanza";
74b9e05af71e plugins: Prefix module imports with prosody namespace Kim Alvefur <zash@zash.se> parents: 12860 diff changeset	20 local rsm = require "prosody.util.rsm";
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	21 local get_prefs = module:require"mamprefs".get;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	22 local set_prefs = module:require"mamprefs".set;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	23 local prefs_to_stanza = module:require"mamprefsxml".tostanza;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	24 local prefs_from_stanza = module:require"mamprefsxml".fromstanza;
12977 74b9e05af71e plugins: Prefix module imports with prosody namespace Kim Alvefur <zash@zash.se> parents: 12860 diff changeset	25 local jid_bare = require "prosody.util.jid".bare;
74b9e05af71e plugins: Prefix module imports with prosody namespace Kim Alvefur <zash@zash.se> parents: 12860 diff changeset	26 local jid_split = require "prosody.util.jid".split;
74b9e05af71e plugins: Prefix module imports with prosody namespace Kim Alvefur <zash@zash.se> parents: 12860 diff changeset	27 local jid_resource = require "prosody.util.jid".resource;
74b9e05af71e plugins: Prefix module imports with prosody namespace Kim Alvefur <zash@zash.se> parents: 12860 diff changeset	28 local jid_prepped_split = require "prosody.util.jid".prepped_split;
74b9e05af71e plugins: Prefix module imports with prosody namespace Kim Alvefur <zash@zash.se> parents: 12860 diff changeset	29 local dataform = require "prosody.util.dataforms".new;
74b9e05af71e plugins: Prefix module imports with prosody namespace Kim Alvefur <zash@zash.se> parents: 12860 diff changeset	30 local get_form_type = require "prosody.util.dataforms".get_type;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	31 local host = module.host;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	32
12977 74b9e05af71e plugins: Prefix module imports with prosody namespace Kim Alvefur <zash@zash.se> parents: 12860 diff changeset	33 local rm_load_roster = require "prosody.core.rostermanager".load_roster;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	34
7838 e5d5e5946af5 mod_mam: Use is_stanza from util.stanza Kim Alvefur <zash@zash.se> parents: 7837 diff changeset	35 local is_stanza = st.is_stanza;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	36 local tostring = tostring;
12977 74b9e05af71e plugins: Prefix module imports with prosody namespace Kim Alvefur <zash@zash.se> parents: 12860 diff changeset	37 local time_now = require "prosody.util.time".now;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	38 local m_min = math.min;
12977 74b9e05af71e plugins: Prefix module imports with prosody namespace Kim Alvefur <zash@zash.se> parents: 12860 diff changeset	39 local timestamp, datestamp = import("prosody.util.datetime", "datetime", "date");
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	40 local default_max_items, max_max_items = 20, module:get_option_number("max_archive_query_results", 50);
7849 93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	41 local strip_tags = module:get_option_set("dont_archive_namespaces", { "http://jabber.org/protocol/chatstates" });
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	42
7850 10d91860172f mod_mam: Change store name to "archive" but make it configurable for people who have data in "archive2" Kim Alvefur <zash@zash.se> parents: 7849 diff changeset	43 local archive_store = module:get_option_string("archive_store", "archive");
8047 eb9784561387 mod_mam: Remove extraneous assert when opening the store Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> parents: 8000 diff changeset	44 local archive = module:open_store(archive_store, "archive");
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	45
9886 710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	46 local cleanup_after = module:get_option_string("archive_expires_after", "1w");
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	47 local archive_item_limit = module:get_option_number("storage_archive_item_limit", archive.caps and archive.caps.quota or 1000);
10800 62794e065e33 MAM: Remove 1% of contents when reaching limits, fix #1545 Kim Alvefur <zash@zash.se> parents: 10773 diff changeset	48 local archive_truncate = math.floor(archive_item_limit * 0.99);
62794e065e33 MAM: Remove 1% of contents when reaching limits, fix #1545 Kim Alvefur <zash@zash.se> parents: 10773 diff changeset	49
9553 9a5485550bfd mod_mam: Ignore case of null storage driver Kim Alvefur <zash@zash.se> parents: 8905 diff changeset	50 if not archive.find then
9554 cba6e6168d26 mod_mam: Upgrade case of invalid archive store driver to hard error Kim Alvefur <zash@zash.se> parents: 9553 diff changeset	51 error("mod_"..(archive._provided_by or archive.name and "storage_"..archive.name).." does not support archiving\n"
cba6e6168d26 mod_mam: Upgrade case of invalid archive store driver to hard error Kim Alvefur <zash@zash.se> parents: 9553 diff changeset	52 .."See https://prosody.im/doc/storage and https://prosody.im/doc/archiving for more information");
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	53 end
8577 75d5eee6fcdf mod_mam: Add an option for whether to include 'total' counts by default in queries Kim Alvefur <zash@zash.se> parents: 8576 diff changeset	54 local use_total = module:get_option_boolean("mam_include_total", true);
8132 6ddddfe05a74 mod_mam: Don't ask the storage backend to count all items when expiry is disabled Kim Alvefur <zash@zash.se> parents: 8047 diff changeset	55
12556 706375b75475 mod_mam: Silence luacheck (yay warnings in CI but not locally) Kim Alvefur <zash@zash.se> parents: 12555 diff changeset	56 function schedule_cleanup(_username, _date) -- luacheck: ignore 212
12555 519e6403f455 mod_mam: Clarify comment (thanks chili-b) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	57 -- Called to make a note of which users have messages on which days, which in
519e6403f455 mod_mam: Clarify comment (thanks chili-b) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	58 -- turn is used to optimize the message expiry routine.
519e6403f455 mod_mam: Clarify comment (thanks chili-b) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	59 --
519e6403f455 mod_mam: Clarify comment (thanks chili-b) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	60 -- This noop is conditionally replaced later depending on retention settings
519e6403f455 mod_mam: Clarify comment (thanks chili-b) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	61 -- and storage backend capabilities.
8230 154852646095 mod_mam: Use a FIFO queue for scheduling archive expiry Kim Alvefur <zash@zash.se> parents: 8207 diff changeset	62 end
154852646095 mod_mam: Use a FIFO queue for scheduling archive expiry Kim Alvefur <zash@zash.se> parents: 8207 diff changeset	63
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	64 -- Handle prefs.
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	65 module:hook("iq/self/"..xmlns_mam..":prefs", function(event)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	66 local origin, stanza = event.origin, event.stanza;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	67 local user = origin.username;
8252 63e505578d4f mod_mam: Also return the preferences on set Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> parents: 8250 diff changeset	68 if stanza.attr.type == "set" then
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	69 local new_prefs = stanza:get_child("prefs", xmlns_mam);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	70 local prefs = prefs_from_stanza(new_prefs);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	71 local ok, err = set_prefs(user, prefs);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	72 if not ok then
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	73 origin.send(st.error_reply(stanza, "cancel", "internal-server-error", "Error storing preferences: "..tostring(err)));
8252 63e505578d4f mod_mam: Also return the preferences on set Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> parents: 8250 diff changeset	74 return true;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	75 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	76 end
8538 3eb4cafb3b64 mod_mam: Implement option to enable MAM implicitly when client support is detected (#867) Kim Alvefur <zash@zash.se> parents: 8252 diff changeset	77 local prefs = prefs_to_stanza(get_prefs(user, true));
8252 63e505578d4f mod_mam: Also return the preferences on set Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> parents: 8250 diff changeset	78 local reply = st.reply(stanza):add_child(prefs);
63e505578d4f mod_mam: Also return the preferences on set Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> parents: 8250 diff changeset	79 origin.send(reply);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	80 return true;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	81 end);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	82
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	83 local query_form = dataform {
11875 210a785dfa8a mod_mam: Use util.dataforms timestamp validation Kim Alvefur <zash@zash.se> parents: 11819 diff changeset	84 { name = "FORM_TYPE"; type = "hidden"; value = xmlns_mam };
210a785dfa8a mod_mam: Use util.dataforms timestamp validation Kim Alvefur <zash@zash.se> parents: 11819 diff changeset	85 { name = "with"; type = "jid-single" };
210a785dfa8a mod_mam: Use util.dataforms timestamp validation Kim Alvefur <zash@zash.se> parents: 11819 diff changeset	86 { name = "start"; type = "text-single"; datatype = "xs:dateTime" };
210a785dfa8a mod_mam: Use util.dataforms timestamp validation Kim Alvefur <zash@zash.se> parents: 11819 diff changeset	87 { name = "end"; type = "text-single"; datatype = "xs:dateTime" };
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	88 };
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	89
11272 44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	90 if archive.caps and archive.caps.full_id_range then
44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	91 table.insert(query_form, { name = "before-id"; type = "text-single"; });
44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	92 table.insert(query_form, { name = "after-id"; type = "text-single"; });
44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	93 end
44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	94
11280 e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	95 if archive.caps and archive.caps.ids then
e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	96 table.insert(query_form, { name = "ids"; type = "list-multi"; });
e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	97 end
e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	98
e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	99
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	100 -- Serve form
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	101 module:hook("iq-get/self/"..xmlns_mam..":query", function(event)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	102 local origin, stanza = event.origin, event.stanza;
8538 3eb4cafb3b64 mod_mam: Implement option to enable MAM implicitly when client support is detected (#867) Kim Alvefur <zash@zash.se> parents: 8252 diff changeset	103 get_prefs(origin.username, true);
7904 c011cecad576 mod_mam: Add missing wrapping <query> element when returning the query form Kim Alvefur <zash@zash.se> parents: 7903 diff changeset	104 origin.send(st.reply(stanza):query(xmlns_mam):add_child(query_form:form()));
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	105 return true;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	106 end);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	107
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	108 -- Handle archive queries
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	109 module:hook("iq-set/self/"..xmlns_mam..":query", function(event)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	110 local origin, stanza = event.origin, event.stanza;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	111 local query = stanza.tags[1];
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	112 local qid = query.attr.queryid;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	113
10298 906ea5e1ec75 mod_mam: Add flag to session when it performs a MAM query Matthew Wild <mwild1@gmail.com> parents: 10028 diff changeset	114 origin.mam_requested = true;
906ea5e1ec75 mod_mam: Add flag to session when it performs a MAM query Matthew Wild <mwild1@gmail.com> parents: 10028 diff changeset	115
8538 3eb4cafb3b64 mod_mam: Implement option to enable MAM implicitly when client support is detected (#867) Kim Alvefur <zash@zash.se> parents: 8252 diff changeset	116 get_prefs(origin.username, true);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	117
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	118 -- Search query parameters
11280 e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	119 local qwith, qstart, qend, qbefore, qafter, qids;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	120 local form = query:get_child("x", "jabber:x:data");
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	121 if form then
10559 cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	122 local form_type, err = get_form_type(form);
cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	123 if not form_type then
cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	124 origin.send(st.error_reply(stanza, "modify", "bad-request", "Invalid dataform: "..err));
cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	125 return true;
cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	126 elseif form_type ~= xmlns_mam then
cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	127 origin.send(st.error_reply(stanza, "modify", "bad-request", "Unexpected FORM_TYPE, expected '"..xmlns_mam.."'"));
cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	128 return true;
cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	129 end
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	130 form, err = query_form:data(form);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	131 if err then
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	132 origin.send(st.error_reply(stanza, "modify", "bad-request", select(2, next(err))));
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	133 return true;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	134 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	135 qwith, qstart, qend = form["with"], form["start"], form["end"];
11272 44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	136 qbefore, qafter = form["before-id"], form["after-id"];
11280 e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	137 qids = form["ids"];
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	138 qwith = qwith and jid_bare(qwith); -- dataforms does jidprep
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	139 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	140
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	141 -- RSM stuff
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	142 local qset = rsm.get(query);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	143 local qmax = m_min(qset and qset.max or default_max_items, max_max_items);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	144 local reverse = qset and qset.before or false;
12857 cc86d77481fc mod_mam,mod_muc_mam: Minimize differences (reorder, copy some comments) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	145
11272 44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	146 local before, after = qset and qset.before or qbefore, qset and qset.after or qafter;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	147 if type(before) ~= "string" then before = nil; end
11819 f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	148
12857 cc86d77481fc mod_mam,mod_muc_mam: Minimize differences (reorder, copy some comments) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	149 -- A reverse query needs to be flipped
cc86d77481fc mod_mam,mod_muc_mam: Minimize differences (reorder, copy some comments) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	150 local flip = reverse;
cc86d77481fc mod_mam,mod_muc_mam: Minimize differences (reorder, copy some comments) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	151 -- A flip-page query needs to be the opposite of that.
cc86d77481fc mod_mam,mod_muc_mam: Minimize differences (reorder, copy some comments) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	152 if query:get_child("flip-page") then flip = not flip end
11819 f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	153
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	154 module:log("debug", "Archive query by %s id=%s with=%s when=%s...%s rsm=%q",
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	155 origin.username,
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	156 qid or stanza.attr.id,
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	157 qwith or "*",
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	158 qstart and timestamp(qstart) or "",
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	159 qend and timestamp(qend) or "",
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	160 qset);
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	161
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	162 -- Load all the data!
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	163 local data, err = archive:find(origin.username, {
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	164 start = qstart; ["end"] = qend; -- Time range
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	165 with = qwith;
8904 faca839ddbbb mod_mam: Handle edge-case of max=0 so that complete attr is set (fixes #1128) Kim Alvefur <zash@zash.se> parents: 8580 diff changeset	166 limit = qmax == 0 and 0 or qmax + 1;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	167 before = before; after = after;
11280 e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	168 ids = qids;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	169 reverse = reverse;
8172 66e32c34250b mod_mam: Request a total count if no items are requested Kim Alvefur <zash@zash.se> parents: 8149 diff changeset	170 total = use_total or qmax == 0;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	171 });
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	172
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	173 if not data then
10076 ee85cea08127 mod_mam: Log query failure reason Kim Alvefur <zash@zash.se> parents: 10075 diff changeset	174 module:log("debug", "Archive query id=%s failed: %s", qid or stanza.attr.id, err);
10021 4715c5d1eb69 mod_mam: Propagate item-not-found to client (fixes #1325) Kim Alvefur <zash@zash.se> parents: 9894 diff changeset	175 if err == "item-not-found" then
4715c5d1eb69 mod_mam: Propagate item-not-found to client (fixes #1325) Kim Alvefur <zash@zash.se> parents: 9894 diff changeset	176 origin.send(st.error_reply(stanza, "modify", "item-not-found"));
4715c5d1eb69 mod_mam: Propagate item-not-found to client (fixes #1325) Kim Alvefur <zash@zash.se> parents: 9894 diff changeset	177 else
4715c5d1eb69 mod_mam: Propagate item-not-found to client (fixes #1325) Kim Alvefur <zash@zash.se> parents: 9894 diff changeset	178 origin.send(st.error_reply(stanza, "cancel", "internal-server-error"));
4715c5d1eb69 mod_mam: Propagate item-not-found to client (fixes #1325) Kim Alvefur <zash@zash.se> parents: 9894 diff changeset	179 end
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	180 return true;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	181 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	182 local total = tonumber(err);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	183
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	184 local msg_reply_attr = { to = stanza.attr.from, from = stanza.attr.to };
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	185
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	186 local results = {};
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	187
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	188 -- Wrap it in stuff and deliver
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	189 local first, last;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	190 local count = 0;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	191 local complete = "true";
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	192 for id, item, when in data do
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	193 count = count + 1;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	194 if count > qmax then
8905 65d1a80d3565 mod_mam: Add coment on how 'complete' works Kim Alvefur <zash@zash.se> parents: 8904 diff changeset	195 -- We requested qmax+1 items. If that many items are retrieved then
65d1a80d3565 mod_mam: Add coment on how 'complete' works Kim Alvefur <zash@zash.se> parents: 8904 diff changeset	196 -- there are more results to page through, so:
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	197 complete = nil;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	198 break;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	199 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	200 local fwd_st = st.message(msg_reply_attr)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	201 :tag("result", { xmlns = xmlns_mam, queryid = qid, id = id })
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	202 :tag("forwarded", { xmlns = xmlns_forward })
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	203 :tag("delay", { xmlns = xmlns_delay, stamp = timestamp(when) }):up();
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	204
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	205 if not is_stanza(item) then
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	206 item = st.deserialize(item);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	207 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	208 item.attr.xmlns = "jabber:client";
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	209 fwd_st:add_child(item);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	210
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	211 if not first then first = id; end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	212 last = id;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	213
11269 342ac5d806fb mod_mam: Add support for page flipping Kim Alvefur <zash@zash.se> parents: 11268 diff changeset	214 if flip then
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	215 results[count] = fwd_st;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	216 else
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	217 origin.send(fwd_st);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	218 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	219 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	220
11269 342ac5d806fb mod_mam: Add support for page flipping Kim Alvefur <zash@zash.se> parents: 11268 diff changeset	221 if flip then
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	222 for i = #results, 1, -1 do
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	223 origin.send(results[i]);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	224 end
11269 342ac5d806fb mod_mam: Add support for page flipping Kim Alvefur <zash@zash.se> parents: 11268 diff changeset	225 end
342ac5d806fb mod_mam: Add support for page flipping Kim Alvefur <zash@zash.se> parents: 11268 diff changeset	226 if reverse then
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	227 first, last = last, first;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	228 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	229
7839 f3e1925f29c2 mod_mam: Update to XEP-0313 v0.5.1 Kim Alvefur <zash@zash.se> parents: 7838 diff changeset	230 origin.send(st.reply(stanza)
11337 f89c8e6beaa6 mod_mam: Remove obsolete 'queryid' attribute from iq-result (thanks paul) Kim Alvefur <zash@zash.se> parents: 11281 diff changeset	231 :tag("fin", { xmlns = xmlns_mam, complete = complete })
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	232 :add_child(rsm.generate {
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	233 first = first, last = last, count = total }));
10078 44371850c6b0 mod_mam: Move final log message to end of query procedure Kim Alvefur <zash@zash.se> parents: 10077 diff changeset	234
44371850c6b0 mod_mam: Move final log message to end of query procedure Kim Alvefur <zash@zash.se> parents: 10077 diff changeset	235 -- That's all folks!
10079 a36c731ed540 mod_mam: Report correct count of results for forward queries Kim Alvefur <zash@zash.se> parents: 10078 diff changeset	236 module:log("debug", "Archive query id=%s completed, %d items returned", qid or stanza.attr.id, complete and count or count - 1);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	237 return true;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	238 end);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	239
11270 eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	240 module:hook("iq-get/self/"..xmlns_mam..":metadata", function (event)
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	241 local origin, stanza = event.origin, event.stanza;
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	242
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	243 local reply = st.reply(stanza):tag("metadata", { xmlns = xmlns_mam });
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	244
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	245 do
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	246 local first = archive:find(origin.username, { limit = 1 });
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	247 if not first then
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	248 origin.send(st.error_reply(stanza, "cancel", "internal-server-error"));
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	249 return true;
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	250 end
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	251
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	252 local id, _, when = first();
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	253 if id then
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	254 reply:tag("start", { id = id, timestamp = timestamp(when) }):up();
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	255 end
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	256 end
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	257
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	258 do
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	259 local last = archive:find(origin.username, { limit = 1, reverse = true });
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	260 if not last then
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	261 origin.send(st.error_reply(stanza, "cancel", "internal-server-error"));
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	262 return true;
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	263 end
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	264
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	265 local id, _, when = last();
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	266 if id then
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	267 reply:tag("end", { id = id, timestamp = timestamp(when) }):up();
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	268 end
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	269 end
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	270
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	271 origin.send(reply);
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	272 return true;
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	273 end);
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	274
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	275 local function has_in_roster(user, who)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	276 local roster = rm_load_roster(user, host);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	277 module:log("debug", "%s has %s in roster? %s", user, who, roster[who] and "yes" or "no");
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	278 return roster[who];
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	279 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	280
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	281 local function shall_store(user, who)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	282 -- TODO Cache this?
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	283 if not um.user_exists(user, host) then
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	284 module:log("debug", "%s@%s does not exist", user, host)
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	285 return false;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	286 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	287 local prefs = get_prefs(user);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	288 local rule = prefs[who];
10111 0f335815244f plugins: Remove tostring call from logging Kim Alvefur <zash@zash.se> parents: 10079 diff changeset	289 module:log("debug", "%s's rule for %s is %s", user, who, rule);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	290 if rule ~= nil then
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	291 return rule;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	292 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	293 -- Below could be done by a metatable
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	294 local default = prefs[false];
10111 0f335815244f plugins: Remove tostring call from logging Kim Alvefur <zash@zash.se> parents: 10079 diff changeset	295 module:log("debug", "%s's default rule is %s", user, default);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	296 if default == "roster" then
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	297 return has_in_roster(user, who);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	298 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	299 return default;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	300 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	301
8207 8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	302 local function strip_stanza_id(stanza, user)
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	303 if stanza:get_child("stanza-id", xmlns_st_id) then
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	304 stanza = st.clone(stanza);
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	305 stanza:maptags(function (tag)
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	306 if tag.name == "stanza-id" and tag.attr.xmlns == xmlns_st_id then
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	307 local by_user, by_host, res = jid_prepped_split(tag.attr.by);
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	308 if not res and by_host == host and by_user == user then
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	309 return nil;
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	310 end
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	311 end
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	312 return tag;
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	313 end);
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	314 end
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	315 return stanza;
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	316 end
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	317
12306 81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	318 local function should_store(stanza) --> boolean, reason: string
10736 c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	319 local st_type = stanza.attr.type or "normal";
10751 4db4bd8a7822 mod_mam: Don't store any groupchat messages Kim Alvefur <zash@zash.se> parents: 10746 diff changeset	320 -- FIXME pass direction of stanza and use that along with bare/full JID addressing
4db4bd8a7822 mod_mam: Don't store any groupchat messages Kim Alvefur <zash@zash.se> parents: 10746 diff changeset	321 -- for more accurate MUC / type=groupchat check
10734 136c41a3d03c mod_mam: Factor out "should we store this" into a function Kim Alvefur <zash@zash.se> parents: 10683 diff changeset	322
10736 c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	323 if st_type == "headline" then
c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	324 -- Headline messages are ephemeral by definition
c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	325 return false, "headline";
c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	326 end
12306 81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	327 if st_type == "error" then
12030 9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	328 -- Errors not sent sent from a local client
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	329 -- Why would a client send an error anyway?
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	330 if jid_resource(stanza.attr.to) then
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	331 -- Store delivery failure notifications so you know if your own messages
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	332 -- were not delivered.
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	333 return true, "bounce";
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	334 else
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	335 -- Skip errors for messages that come from your account, such as PEP
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	336 -- notifications.
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	337 return false, "bounce";
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	338 end
10745 79b29f35fac1 mod_mam: Save delivery failures (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 10744 diff changeset	339 end
10751 4db4bd8a7822 mod_mam: Don't store any groupchat messages Kim Alvefur <zash@zash.se> parents: 10746 diff changeset	340 if st_type == "groupchat" then
10736 c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	341 -- MUC messages always go to the full JID, usually archived by the MUC
c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	342 return false, "groupchat";
c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	343 end
10743 3967cf10de1b mod_mam: Respect no-store hint (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 10742 diff changeset	344 if stanza:get_child("no-store", "urn:xmpp:hints")
3967cf10de1b mod_mam: Respect no-store hint (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 10742 diff changeset	345 or stanza:get_child("no-permanent-store", "urn:xmpp:hints") then
10746 2e31d67b9a29 mod_mam: Make note of Experimental (or Deferred) XEPs Kim Alvefur <zash@zash.se> parents: 10745 diff changeset	346 -- XXX Experimental XEP
10737 b2ede421adeb mod_mam: Rework hints handling Kim Alvefur <zash@zash.se> parents: 10736 diff changeset	347 return false, "hint";
b2ede421adeb mod_mam: Rework hints handling Kim Alvefur <zash@zash.se> parents: 10736 diff changeset	348 end
b2ede421adeb mod_mam: Rework hints handling Kim Alvefur <zash@zash.se> parents: 10736 diff changeset	349 if stanza:get_child("store", "urn:xmpp:hints") then
b2ede421adeb mod_mam: Rework hints handling Kim Alvefur <zash@zash.se> parents: 10736 diff changeset	350 return true, "hint";
b2ede421adeb mod_mam: Rework hints handling Kim Alvefur <zash@zash.se> parents: 10736 diff changeset	351 end
10736 c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	352 if stanza:get_child("body") then
c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	353 return true, "body";
c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	354 end
10738 19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	355 if stanza:get_child("subject") then
10742 70b7c8f3d32d mod_mam: Fix typo in comment Kim Alvefur <zash@zash.se> parents: 10741 diff changeset	356 -- XXX Who would send a message with a subject but without a body?
10738 19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	357 return true, "subject";
19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	358 end
19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	359 if stanza:get_child("encryption", "urn:xmpp:eme:0") then
19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	360 -- Since we can't know what an encrypted message contains, we assume it's important
10746 2e31d67b9a29 mod_mam: Make note of Experimental (or Deferred) XEPs Kim Alvefur <zash@zash.se> parents: 10745 diff changeset	361 -- XXX Experimental XEP
10738 19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	362 return true, "encrypted";
19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	363 end
10739 50f0a4d83731 mod_mam: Store XEP-0184 receipts and requests Kim Alvefur <zash@zash.se> parents: 10738 diff changeset	364 if stanza:get_child(nil, "urn:xmpp:receipts") then
50f0a4d83731 mod_mam: Store XEP-0184 receipts and requests Kim Alvefur <zash@zash.se> parents: 10738 diff changeset	365 -- If it's important enough to ask for a receipt then it's important enough to archive
50f0a4d83731 mod_mam: Store XEP-0184 receipts and requests Kim Alvefur <zash@zash.se> parents: 10738 diff changeset	366 -- and the same applies to the receipt
50f0a4d83731 mod_mam: Store XEP-0184 receipts and requests Kim Alvefur <zash@zash.se> parents: 10738 diff changeset	367 return true, "receipt";
50f0a4d83731 mod_mam: Store XEP-0184 receipts and requests Kim Alvefur <zash@zash.se> parents: 10738 diff changeset	368 end
10744 16002abe61b1 mod_mam: Keep chat markers (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 10743 diff changeset	369 if stanza:get_child(nil, "urn:xmpp:chat-markers:0") then
10746 2e31d67b9a29 mod_mam: Make note of Experimental (or Deferred) XEPs Kim Alvefur <zash@zash.se> parents: 10745 diff changeset	370 -- XXX Experimental XEP
10744 16002abe61b1 mod_mam: Keep chat markers (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 10743 diff changeset	371 return true, "marker";
16002abe61b1 mod_mam: Keep chat markers (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 10743 diff changeset	372 end
10738 19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	373 if stanza:get_child("x", "jabber:x:conference")
19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	374 or stanza:find("{http://jabber.org/protocol/muc#user}x/invite") then
19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	375 return true, "invite";
19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	376 end
12234 1c47162dd965 plugins: Update for namespace bump in XEP-0353 v0.4.0 Kim Alvefur <zash@zash.se> parents: 12030 diff changeset	377 if stanza:get_child(nil, "urn:xmpp:jingle-message:0") or stanza:get_child(nil, "urn:xmpp:jingle-message:1") then
11260 08b397c21805 mod_csi_simple,mod_carbons,mod_mam: Update comment about XEP-0353 Kim Alvefur <zash@zash.se> parents: 10821 diff changeset	378 -- XXX Experimental XEP
10821 30fc1ed5647a mod_mam: Archive XEP-0353: Jingle Message Initiation Kim Alvefur <zash@zash.se> parents: 10800 diff changeset	379 return true, "jingle call";
30fc1ed5647a mod_mam: Archive XEP-0353: Jingle Message Initiation Kim Alvefur <zash@zash.se> parents: 10800 diff changeset	380 end
10734 136c41a3d03c mod_mam: Factor out "should we store this" into a function Kim Alvefur <zash@zash.se> parents: 10683 diff changeset	381
10741 27f1fcd85ccd mod_mam: Prefer not archiving if no interesting payloads are found Kim Alvefur <zash@zash.se> parents: 10740 diff changeset	382 -- The IM-NG thing to do here would be to return `not st_to_full`
27f1fcd85ccd mod_mam: Prefer not archiving if no interesting payloads are found Kim Alvefur <zash@zash.se> parents: 10740 diff changeset	383 -- One day ...
27f1fcd85ccd mod_mam: Prefer not archiving if no interesting payloads are found Kim Alvefur <zash@zash.se> parents: 10740 diff changeset	384 return false, "default";
10734 136c41a3d03c mod_mam: Factor out "should we store this" into a function Kim Alvefur <zash@zash.se> parents: 10683 diff changeset	385 end
136c41a3d03c mod_mam: Factor out "should we store this" into a function Kim Alvefur <zash@zash.se> parents: 10683 diff changeset	386
12306 81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	387 module:hook("archive-should-store", function (event)
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	388 local should, why = should_store(event.stanza);
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	389 event.reason = why;
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	390 return should;
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	391 end, -1)
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	392
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	393 -- Handle messages
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	394 local function message_handler(event, c2s)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	395 local origin, stanza = event.origin, event.stanza;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	396 local log = c2s and origin.log or module._log;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	397 local orig_from = stanza.attr.from;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	398 local orig_to = stanza.attr.to or orig_from;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	399 -- Stanza without 'to' are treated as if it was to their own bare jid
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	400
11727 f3aee8a825cc Fix various spelling errors (thanks codespell) Kim Alvefur <zash@zash.se> parents: 11337 diff changeset	401 -- Whose storage do we put it in?
7844 316f5166eedb mod_mam: Find out which party is the user and which is the 'with' earlier Kim Alvefur <zash@zash.se> parents: 7843 diff changeset	402 local store_user = c2s and origin.username or jid_split(orig_to);
316f5166eedb mod_mam: Find out which party is the user and which is the 'with' earlier Kim Alvefur <zash@zash.se> parents: 7843 diff changeset	403 -- And who are they chatting with?
316f5166eedb mod_mam: Find out which party is the user and which is the 'with' earlier Kim Alvefur <zash@zash.se> parents: 7843 diff changeset	404 local with = jid_bare(c2s and orig_to or orig_from);
316f5166eedb mod_mam: Find out which party is the user and which is the 'with' earlier Kim Alvefur <zash@zash.se> parents: 7843 diff changeset	405
7845 eeb22f912577 mod_mam: Filter out spoofed XEP-0359 tags Kim Alvefur <zash@zash.se> parents: 7844 diff changeset	406 -- Filter out <stanza-id> that claim to be from us
8207 8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	407 event.stanza = strip_stanza_id(stanza, store_user);
7845 eeb22f912577 mod_mam: Filter out spoofed XEP-0359 tags Kim Alvefur <zash@zash.se> parents: 7844 diff changeset	408
12306 81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	409 local event_payload = { stanza = stanza; session = origin };
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	410 local should = module:fire_event("archive-should-store", event_payload);
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	411 local why = event_payload.reason;
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	412
10734 136c41a3d03c mod_mam: Factor out "should we store this" into a function Kim Alvefur <zash@zash.se> parents: 10683 diff changeset	413 if not should then
12306 81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	414 log("debug", "Not archiving stanza: %s (%s)", stanza:top_tag(), event_payload.reason);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	415 return;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	416 end
7842 9332b43931f5 mod_mam: Add some comments explaining archive expiry Kim Alvefur <zash@zash.se> parents: 7841 diff changeset	417
8193 bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	418 local clone_for_storage;
7849 93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	419 if not strip_tags:empty() then
8193 bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	420 clone_for_storage = st.clone(stanza);
bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	421 clone_for_storage:maptags(function (tag)
7849 93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	422 if strip_tags:contains(tag.attr.xmlns) then
93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	423 return nil;
93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	424 else
93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	425 return tag;
93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	426 end
93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	427 end);
8193 bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	428 if #clone_for_storage.tags == 0 then
8250 9ea5ea53744b mod_mam: Log a message when not archiving because it only had ignored tags Kim Alvefur <zash@zash.se> parents: 8231 diff changeset	429 log("debug", "Not archiving stanza: %s (empty when stripped)", stanza:top_tag());
7849 93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	430 return;
93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	431 end
8193 bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	432 else
bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	433 clone_for_storage = stanza;
7849 93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	434 end
93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	435
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	436 -- Check with the users preferences
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	437 if shall_store(store_user, with) then
10735 f2838ffcc499 mod_mam: Log 'why' a stanza is archived Kim Alvefur <zash@zash.se> parents: 10734 diff changeset	438 log("debug", "Archiving stanza: %s (%s)", stanza:top_tag(), why);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	439
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	440 -- And stash it
9886 710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	441 local time = time_now();
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	442 local ok, err = archive:append(store_user, nil, clone_for_storage, time, with);
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	443 if not ok and err == "quota-limit" then
9894 8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	444 if type(cleanup_after) == "number" then
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	445 module:log("debug", "User '%s' over quota, cleaning archive", store_user);
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	446 local cleaned = archive:delete(store_user, {
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	447 ["end"] = (os.time() - cleanup_after);
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	448 });
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	449 if cleaned then
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	450 ok, err = archive:append(store_user, nil, clone_for_storage, time, with);
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	451 end
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	452 end
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	453 if not ok and (archive.caps and archive.caps.truncate) then
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	454 module:log("debug", "User '%s' over quota, truncating archive", store_user);
9886 710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	455 local truncated = archive:delete(store_user, {
10800 62794e065e33 MAM: Remove 1% of contents when reaching limits, fix #1545 Kim Alvefur <zash@zash.se> parents: 10773 diff changeset	456 truncate = archive_truncate;
9886 710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	457 });
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	458 if truncated then
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	459 ok, err = archive:append(store_user, nil, clone_for_storage, time, with);
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	460 end
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	461 end
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	462 end
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	463 if ok then
8193 bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	464 local clone_for_other_handlers = st.clone(stanza);
7908 dbdaa8487ecd mod_mam: Fix to treat first return value from archive:append as assigned ID Kim Alvefur <zash@zash.se> parents: 7906 diff changeset	465 local id = ok;
8193 bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	466 clone_for_other_handlers:tag("stanza-id", { xmlns = xmlns_st_id, by = store_user.."@"..host, id = id }):up();
bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	467 event.stanza = clone_for_other_handlers;
8230 154852646095 mod_mam: Use a FIFO queue for scheduling archive expiry Kim Alvefur <zash@zash.se> parents: 8207 diff changeset	468 schedule_cleanup(store_user);
8193 bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	469 module:fire_event("archive-message-added", { origin = origin, stanza = clone_for_storage, for_user = store_user, id = id });
10523 86422db90e02 mod_mam: Log error when unable to store stanza (fix #1478) Kim Alvefur <zash@zash.se> parents: 10298 diff changeset	470 else
86422db90e02 mod_mam: Log error when unable to store stanza (fix #1478) Kim Alvefur <zash@zash.se> parents: 10298 diff changeset	471 log("error", "Could not archive stanza: %s", err);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	472 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	473 else
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	474 log("debug", "Not archiving stanza: %s (prefs)", stanza:top_tag());
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	475 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	476 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	477
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	478 local function c2s_message_handler(event)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	479 return message_handler(event, true);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	480 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	481
8207 8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	482 -- Filter out <stanza-id> before the message leaves the server to prevent privacy leak.
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	483 local function strip_stanza_id_after_other_events(event)
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	484 event.stanza = strip_stanza_id(event.stanza, event.origin.username);
7902 92b70a921acb mod_mam: Strip stanza-id tags after carbons Kim Alvefur <zash@zash.se> parents: 7901 diff changeset	485 end
92b70a921acb mod_mam: Strip stanza-id tags after carbons Kim Alvefur <zash@zash.se> parents: 7901 diff changeset	486
8207 8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	487 module:hook("pre-message/bare", strip_stanza_id_after_other_events, -1);
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	488 module:hook("pre-message/full", strip_stanza_id_after_other_events, -1);
7902 92b70a921acb mod_mam: Strip stanza-id tags after carbons Kim Alvefur <zash@zash.se> parents: 7901 diff changeset	489
11758 c35b81575d5d mod_mam: Explain behavior with absent mod_offline in a comment Kim Alvefur <zash@zash.se> parents: 11757 diff changeset	490 -- Catch messages not stored by mod_offline and mark them as stored if they
c35b81575d5d mod_mam: Explain behavior with absent mod_offline in a comment Kim Alvefur <zash@zash.se> parents: 11757 diff changeset	491 -- have been archived. This would generally only happen if mod_offline is
c35b81575d5d mod_mam: Explain behavior with absent mod_offline in a comment Kim Alvefur <zash@zash.se> parents: 11757 diff changeset	492 -- disabled. Otherwise the message would generate a delivery failure report,
c35b81575d5d mod_mam: Explain behavior with absent mod_offline in a comment Kim Alvefur <zash@zash.se> parents: 11757 diff changeset	493 -- which would not be accurate because it has been archived.
11757 8141645e3865 mod_mam: "Handle" messages that have been archived in the absense of mod_offline Kim Alvefur <zash@zash.se> parents: 11727 diff changeset	494 module:hook("message/offline/handle", function(event)
8141645e3865 mod_mam: "Handle" messages that have been archived in the absense of mod_offline Kim Alvefur <zash@zash.se> parents: 11727 diff changeset	495 local stanza = event.stanza;
11759 9925be5d3b8b mod_mam: Only check for locally generated stanza-ids Kim Alvefur <zash@zash.se> parents: 11758 diff changeset	496 local user = event.username .. "@" .. host;
11795 41af102c7190 mod_mam: Reduce line count using new util.stanza attr method Kim Alvefur <zash@zash.se> parents: 11760 diff changeset	497 if stanza:get_child_with_attr("stanza-id", xmlns_st_id, "by", user) then
41af102c7190 mod_mam: Reduce line count using new util.stanza attr method Kim Alvefur <zash@zash.se> parents: 11760 diff changeset	498 return true;
11757 8141645e3865 mod_mam: "Handle" messages that have been archived in the absense of mod_offline Kim Alvefur <zash@zash.se> parents: 11727 diff changeset	499 end
8141645e3865 mod_mam: "Handle" messages that have been archived in the absense of mod_offline Kim Alvefur <zash@zash.se> parents: 11727 diff changeset	500 end, -2);
8141645e3865 mod_mam: "Handle" messages that have been archived in the absense of mod_offline Kim Alvefur <zash@zash.se> parents: 11727 diff changeset	501
11760 d66738eeb875 mod_mam: Suppress offline message broadcast for MAM clients Kim Alvefur <zash@zash.se> parents: 11759 diff changeset	502 -- Don't broadcast offline messages to clients that have queried the archive.
d66738eeb875 mod_mam: Suppress offline message broadcast for MAM clients Kim Alvefur <zash@zash.se> parents: 11759 diff changeset	503 module:hook("message/offline/broadcast", function (event)
d66738eeb875 mod_mam: Suppress offline message broadcast for MAM clients Kim Alvefur <zash@zash.se> parents: 11759 diff changeset	504 if event.origin.mam_requested then
d66738eeb875 mod_mam: Suppress offline message broadcast for MAM clients Kim Alvefur <zash@zash.se> parents: 11759 diff changeset	505 return true;
d66738eeb875 mod_mam: Suppress offline message broadcast for MAM clients Kim Alvefur <zash@zash.se> parents: 11759 diff changeset	506 end
d66738eeb875 mod_mam: Suppress offline message broadcast for MAM clients Kim Alvefur <zash@zash.se> parents: 11759 diff changeset	507 end);
d66738eeb875 mod_mam: Suppress offline message broadcast for MAM clients Kim Alvefur <zash@zash.se> parents: 11759 diff changeset	508
8575 5040c8ce32dd Backed out changeset 97a094fdf101, interferes with 6ddddfe05a74 Kim Alvefur <zash@zash.se> parents: 8538 diff changeset	509 if cleanup_after ~= "never" then
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	510 local cleanup_storage = module:open_store("archive_cleanup");
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	511 local cleanup_map = module:open_store("archive_cleanup", "map");
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	512
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	513 local day = 86400;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	514 local multipliers = { d = day, w = day * 7, m = 31 * day, y = 365.2425 * day };
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	515 local n, m = cleanup_after:lower():match("(%d+)%s*([dwmy]?)");
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	516 if not n then
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	517 module:log("error", "Could not parse archive_expires_after string %q", cleanup_after);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	518 return false;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	519 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	520
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	521 cleanup_after = tonumber(n) * ( multipliers[m] or 1 );
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	522
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	523 module:log("debug", "archive_expires_after = %d -- in seconds", cleanup_after);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	524
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	525 if not archive.delete then
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	526 module:log("error", "archive_expires_after set but mod_%s does not support deleting", archive._provided_by);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	527 return false;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	528 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	529
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	530 -- For each day, store a set of users that have new messages. To expire
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	531 -- messages, we collect the union of sets of users from dates that fall
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	532 -- outside the cleanup range.
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	533
11968 6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	534 if not (archive.caps and archive.caps.wildcard_delete) then
12977 74b9e05af71e plugins: Prefix module imports with prosody namespace Kim Alvefur <zash@zash.se> parents: 12860 diff changeset	535 local last_date = require "prosody.util.cache".new(module:get_option_number("archive_cleanup_date_cache_size", 1000));
11968 6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	536 function schedule_cleanup(username, date)
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	537 date = date or datestamp();
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	538 if last_date:get(username) == date then return end
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	539 local ok = cleanup_map:set(date, username, true);
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	540 if ok then
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	541 last_date:set(username, date);
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	542 end
10028 79ba2d709e72 mod_mam: Cache last date that archive owner has messages to reduce writes (fixes #1368) Kim Alvefur <zash@zash.se> parents: 9879 diff changeset	543 end
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	544 end
11968 6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	545
9757 03ed7f10d8da mod_mam: Measure how long it takes to run the message expiry job job Kim Alvefur <zash@zash.se> parents: 9752 diff changeset	546 local cleanup_time = module:measure("cleanup", "times");
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	547
12977 74b9e05af71e plugins: Prefix module imports with prosody namespace Kim Alvefur <zash@zash.se> parents: 12860 diff changeset	548 local async = require "prosody.util.async";
11988 18c0ca5fcbb8 mod_mam: Switch to new cron API Kim Alvefur <zash@zash.se> parents: 11968 diff changeset	549 module:daily("Remove expired messages", function ()
9757 03ed7f10d8da mod_mam: Measure how long it takes to run the message expiry job job Kim Alvefur <zash@zash.se> parents: 9752 diff changeset	550 local cleanup_done = cleanup_time();
11968 6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	551
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	552 if archive.caps and archive.caps.wildcard_delete then
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	553 local ok, err = archive:delete(true, { ["end"] = os.time() - cleanup_after })
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	554 if ok then
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	555 local sum = tonumber(ok);
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	556 if sum then
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	557 module:log("info", "Deleted %d expired messages", sum);
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	558 else
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	559 -- driver did not tell
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	560 module:log("info", "Deleted all expired messages");
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	561 end
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	562 else
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	563 module:log("error", "Could not delete messages: %s", err);
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	564 end
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	565 cleanup_done();
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	566 return;
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	567 end
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	568
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	569 local users = {};
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	570 local cut_off = datestamp(os.time() - cleanup_after);
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	571 for date in cleanup_storage:users() do
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	572 if date <= cut_off then
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	573 module:log("debug", "Messages from %q should be expired", date);
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	574 local messages_this_day = cleanup_storage:get(date);
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	575 if messages_this_day then
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	576 for user in pairs(messages_this_day) do
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	577 users[user] = true;
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	578 end
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	579 if date < cut_off then
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	580 -- Messages from the same day as the cut-off might not have expired yet,
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	581 -- but all earlier will have, so clear storage for those days.
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	582 cleanup_storage:set(date, nil);
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	583 end
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	584 end
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	585 end
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	586 end
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	587 local sum, num_users = 0, 0;
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	588 for user in pairs(users) do
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	589 local ok, err = archive:delete(user, { ["end"] = os.time() - cleanup_after; })
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	590 if ok then
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	591 num_users = num_users + 1;
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	592 sum = sum + (tonumber(ok) or 0);
10524 7c29a6e652d2 mod_mam: Log error when unable to delete old messages (fix #1479) [luacheck] Kim Alvefur <zash@zash.se> parents: 10523 diff changeset	593 else
10525 9cf7d9761ca2 mod_mam: Schedule cleanup again if unable to delete messages Kim Alvefur <zash@zash.se> parents: 10524 diff changeset	594 cleanup_map:set(cut_off, user, true);
10524 7c29a6e652d2 mod_mam: Log error when unable to delete old messages (fix #1479) [luacheck] Kim Alvefur <zash@zash.se> parents: 10523 diff changeset	595 module:log("error", "Could not delete messages for user '%s': %s", user, err);
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	596 end
10682 62ef68f95b6f mod_mam,mod_muc_mam: Allow other work to be performed during archive cleanup (fixes #1504) Kim Alvefur <zash@zash.se> parents: 10567 diff changeset	597 local wait, done = async.waiter();
62ef68f95b6f mod_mam,mod_muc_mam: Allow other work to be performed during archive cleanup (fixes #1504) Kim Alvefur <zash@zash.se> parents: 10567 diff changeset	598 module:add_timer(0.01, done);
62ef68f95b6f mod_mam,mod_muc_mam: Allow other work to be performed during archive cleanup (fixes #1504) Kim Alvefur <zash@zash.se> parents: 10567 diff changeset	599 wait();
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	600 end
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	601 module:log("info", "Deleted %d expired messages for %d users", sum, num_users);
9757 03ed7f10d8da mod_mam: Measure how long it takes to run the message expiry job job Kim Alvefur <zash@zash.se> parents: 9752 diff changeset	602 cleanup_done();
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	603 end);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	604
8132 6ddddfe05a74 mod_mam: Don't ask the storage backend to count all items when expiry is disabled Kim Alvefur <zash@zash.se> parents: 8047 diff changeset	605 else
8576 07ff7b8b702b mod_mam: Log a debug message if archive expiry has been disabled Kim Alvefur <zash@zash.se> parents: 8575 diff changeset	606 module:log("debug", "Archive expiry disabled");
8132 6ddddfe05a74 mod_mam: Don't ask the storage backend to count all items when expiry is disabled Kim Alvefur <zash@zash.se> parents: 8047 diff changeset	607 -- Don't ask the backend to count the potentially unbounded number of items,
6ddddfe05a74 mod_mam: Don't ask the storage backend to count all items when expiry is disabled Kim Alvefur <zash@zash.se> parents: 8047 diff changeset	608 -- it'll get slow.
8577 75d5eee6fcdf mod_mam: Add an option for whether to include 'total' counts by default in queries Kim Alvefur <zash@zash.se> parents: 8576 diff changeset	609 use_total = module:get_option_boolean("mam_include_total", false);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	610 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	611
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	612 -- Stanzas sent by local clients
7840 92b6aa3ea2ce mod_mam: Decrease priority to zero Kim Alvefur <zash@zash.se> parents: 7839 diff changeset	613 module:hook("pre-message/bare", c2s_message_handler, 0);
92b6aa3ea2ce mod_mam: Decrease priority to zero Kim Alvefur <zash@zash.se> parents: 7839 diff changeset	614 module:hook("pre-message/full", c2s_message_handler, 0);
7905 daf83bd94906 mod_mam: Fix typo in comment Kim Alvefur <zash@zash.se> parents: 7904 diff changeset	615 -- Stanzas to local clients
7840 92b6aa3ea2ce mod_mam: Decrease priority to zero Kim Alvefur <zash@zash.se> parents: 7839 diff changeset	616 module:hook("message/bare", message_handler, 0);
92b6aa3ea2ce mod_mam: Decrease priority to zero Kim Alvefur <zash@zash.se> parents: 7839 diff changeset	617 module:hook("message/full", message_handler, 0);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	618
11281 142fb655b885 mod_mam: Advertise extended MAM when archive storage is capable Kim Alvefur <zash@zash.se> parents: 11280 diff changeset	619 local advertise_extended = archive.caps and archive.caps.full_id_range and archive.caps.ids;
11268 e3f6f0b39e7b mod_mam: Advertise extended MAM 0.7.x behind a feature flag Kim Alvefur <zash@zash.se> parents: 11260 diff changeset	620
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	621 module:hook("account-disco-info", function(event)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	622 (event.reply or event.stanza):tag("feature", {var=xmlns_mam}):up();
11268 e3f6f0b39e7b mod_mam: Advertise extended MAM 0.7.x behind a feature flag Kim Alvefur <zash@zash.se> parents: 11260 diff changeset	623 if advertise_extended then
e3f6f0b39e7b mod_mam: Advertise extended MAM 0.7.x behind a feature flag Kim Alvefur <zash@zash.se> parents: 11260 diff changeset	624 (event.reply or event.stanza):tag("feature", {var=xmlns_mam_ext}):up();
e3f6f0b39e7b mod_mam: Advertise extended MAM 0.7.x behind a feature flag Kim Alvefur <zash@zash.se> parents: 11260 diff changeset	625 end
7891 01d2a2af3146 mod_mam: Advertise Stanza ID support (XEP-0359) (thanks iNPUTmice) Kim Alvefur <zash@zash.se> parents: 7890 diff changeset	626 (event.reply or event.stanza):tag("feature", {var=xmlns_st_id}):up();
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	627 end);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	628

prosody

06453c564141

plugins/mod_mam/mod_mam.lua

Software / code / prosody

Annotate

plugins/mod_mam/mod_mam.lua @ 13014:06453c564141

Software `/` code `/` prosody