Software `/` code `/` prosody

Annotate

plugins/mod_mam/mod_mam.lua @ 12555:519e6403f455

mod_mam: Clarify comment (thanks chili-b) This was slightly inaccurate since 6e1af07921d1 because the conditions are more complicated now.

author	Kim Alvefur <zash@zash.se>
date	Tue, 14 Jun 2022 22:03:02 +0200
parent	12307:dcad3a207915
child	12556:706375b75475

rev	line source
7851 80ee0d9cd56f mod_mam: Normalize copyright headers Kim Alvefur <zash@zash.se> parents: 7850 diff changeset	1 -- Prosody IM
80ee0d9cd56f mod_mam: Normalize copyright headers Kim Alvefur <zash@zash.se> parents: 7850 diff changeset	2 -- Copyright (C) 2008-2017 Matthew Wild
80ee0d9cd56f mod_mam: Normalize copyright headers Kim Alvefur <zash@zash.se> parents: 7850 diff changeset	3 -- Copyright (C) 2008-2017 Waqas Hussain
11272 44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	4 -- Copyright (C) 2011-2021 Kim Alvefur
7851 80ee0d9cd56f mod_mam: Normalize copyright headers Kim Alvefur <zash@zash.se> parents: 7850 diff changeset	5 --
80ee0d9cd56f mod_mam: Normalize copyright headers Kim Alvefur <zash@zash.se> parents: 7850 diff changeset	6 -- This project is MIT/X11 licensed. Please see the
80ee0d9cd56f mod_mam: Normalize copyright headers Kim Alvefur <zash@zash.se> parents: 7850 diff changeset	7 -- COPYING file in the source package for more information.
80ee0d9cd56f mod_mam: Normalize copyright headers Kim Alvefur <zash@zash.se> parents: 7850 diff changeset	8 --
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	9 -- XEP-0313: Message Archive Management for Prosody
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	10 --
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	11
7903 036c771b842a mod_mam: Update namespace to XEP-0313 v0.6 Kim Alvefur <zash@zash.se> parents: 7902 diff changeset	12 local xmlns_mam = "urn:xmpp:mam:2";
11268 e3f6f0b39e7b mod_mam: Advertise extended MAM 0.7.x behind a feature flag Kim Alvefur <zash@zash.se> parents: 11260 diff changeset	13 local xmlns_mam_ext = "urn:xmpp:mam:2#extended";
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	14 local xmlns_delay = "urn:xmpp:delay";
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	15 local xmlns_forward = "urn:xmpp:forward:0";
7890 9e4de27e8e08 mod_mam: Move stanza ID namespace to a common variable Kim Alvefur <zash@zash.se> parents: 7882 diff changeset	16 local xmlns_st_id = "urn:xmpp:sid:0";
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	17
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	18 local um = require "core.usermanager";
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	19 local st = require "util.stanza";
7882 1017a4f8929d mod_mam: Correct import of util.rsm Kim Alvefur <zash@zash.se> parents: 7856 diff changeset	20 local rsm = require "util.rsm";
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	21 local get_prefs = module:require"mamprefs".get;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	22 local set_prefs = module:require"mamprefs".set;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	23 local prefs_to_stanza = module:require"mamprefsxml".tostanza;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	24 local prefs_from_stanza = module:require"mamprefsxml".fromstanza;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	25 local jid_bare = require "util.jid".bare;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	26 local jid_split = require "util.jid".split;
12030 9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	27 local jid_resource = require "util.jid".resource;
7845 eeb22f912577 mod_mam: Filter out spoofed XEP-0359 tags Kim Alvefur <zash@zash.se> parents: 7844 diff changeset	28 local jid_prepped_split = require "util.jid".prepped_split;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	29 local dataform = require "util.dataforms".new;
10559 cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	30 local get_form_type = require "util.dataforms".get_type;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	31 local host = module.host;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	32
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	33 local rm_load_roster = require "core.rostermanager".load_roster;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	34
7838 e5d5e5946af5 mod_mam: Use is_stanza from util.stanza Kim Alvefur <zash@zash.se> parents: 7837 diff changeset	35 local is_stanza = st.is_stanza;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	36 local tostring = tostring;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	37 local time_now = os.time;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	38 local m_min = math.min;
11875 210a785dfa8a mod_mam: Use util.dataforms timestamp validation Kim Alvefur <zash@zash.se> parents: 11819 diff changeset	39 local timestamp, datestamp = import( "util.datetime", "datetime", "date");
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	40 local default_max_items, max_max_items = 20, module:get_option_number("max_archive_query_results", 50);
7849 93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	41 local strip_tags = module:get_option_set("dont_archive_namespaces", { "http://jabber.org/protocol/chatstates" });
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	42
7850 10d91860172f mod_mam: Change store name to "archive" but make it configurable for people who have data in "archive2" Kim Alvefur <zash@zash.se> parents: 7849 diff changeset	43 local archive_store = module:get_option_string("archive_store", "archive");
8047 eb9784561387 mod_mam: Remove extraneous assert when opening the store Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> parents: 8000 diff changeset	44 local archive = module:open_store(archive_store, "archive");
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	45
9886 710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	46 local cleanup_after = module:get_option_string("archive_expires_after", "1w");
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	47 local archive_item_limit = module:get_option_number("storage_archive_item_limit", archive.caps and archive.caps.quota or 1000);
10800 62794e065e33 MAM: Remove 1% of contents when reaching limits, fix #1545 Kim Alvefur <zash@zash.se> parents: 10773 diff changeset	48 local archive_truncate = math.floor(archive_item_limit * 0.99);
62794e065e33 MAM: Remove 1% of contents when reaching limits, fix #1545 Kim Alvefur <zash@zash.se> parents: 10773 diff changeset	49
9553 9a5485550bfd mod_mam: Ignore case of null storage driver Kim Alvefur <zash@zash.se> parents: 8905 diff changeset	50 if not archive.find then
9554 cba6e6168d26 mod_mam: Upgrade case of invalid archive store driver to hard error Kim Alvefur <zash@zash.se> parents: 9553 diff changeset	51 error("mod_"..(archive._provided_by or archive.name and "storage_"..archive.name).." does not support archiving\n"
cba6e6168d26 mod_mam: Upgrade case of invalid archive store driver to hard error Kim Alvefur <zash@zash.se> parents: 9553 diff changeset	52 .."See https://prosody.im/doc/storage and https://prosody.im/doc/archiving for more information");
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	53 end
8577 75d5eee6fcdf mod_mam: Add an option for whether to include 'total' counts by default in queries Kim Alvefur <zash@zash.se> parents: 8576 diff changeset	54 local use_total = module:get_option_boolean("mam_include_total", true);
8132 6ddddfe05a74 mod_mam: Don't ask the storage backend to count all items when expiry is disabled Kim Alvefur <zash@zash.se> parents: 8047 diff changeset	55
12555 519e6403f455 mod_mam: Clarify comment (thanks chili-b) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	56 function schedule_cleanup(_username, _date)
519e6403f455 mod_mam: Clarify comment (thanks chili-b) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	57 -- Called to make a note of which users have messages on which days, which in
519e6403f455 mod_mam: Clarify comment (thanks chili-b) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	58 -- turn is used to optimize the message expiry routine.
519e6403f455 mod_mam: Clarify comment (thanks chili-b) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	59 --
519e6403f455 mod_mam: Clarify comment (thanks chili-b) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	60 -- This noop is conditionally replaced later depending on retention settings
519e6403f455 mod_mam: Clarify comment (thanks chili-b) Kim Alvefur <zash@zash.se> parents: 12307 diff changeset	61 -- and storage backend capabilities.
8230 154852646095 mod_mam: Use a FIFO queue for scheduling archive expiry Kim Alvefur <zash@zash.se> parents: 8207 diff changeset	62 end
154852646095 mod_mam: Use a FIFO queue for scheduling archive expiry Kim Alvefur <zash@zash.se> parents: 8207 diff changeset	63
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	64 -- Handle prefs.
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	65 module:hook("iq/self/"..xmlns_mam..":prefs", function(event)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	66 local origin, stanza = event.origin, event.stanza;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	67 local user = origin.username;
8252 63e505578d4f mod_mam: Also return the preferences on set Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> parents: 8250 diff changeset	68 if stanza.attr.type == "set" then
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	69 local new_prefs = stanza:get_child("prefs", xmlns_mam);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	70 local prefs = prefs_from_stanza(new_prefs);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	71 local ok, err = set_prefs(user, prefs);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	72 if not ok then
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	73 origin.send(st.error_reply(stanza, "cancel", "internal-server-error", "Error storing preferences: "..tostring(err)));
8252 63e505578d4f mod_mam: Also return the preferences on set Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> parents: 8250 diff changeset	74 return true;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	75 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	76 end
8538 3eb4cafb3b64 mod_mam: Implement option to enable MAM implicitly when client support is detected (#867) Kim Alvefur <zash@zash.se> parents: 8252 diff changeset	77 local prefs = prefs_to_stanza(get_prefs(user, true));
8252 63e505578d4f mod_mam: Also return the preferences on set Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> parents: 8250 diff changeset	78 local reply = st.reply(stanza):add_child(prefs);
63e505578d4f mod_mam: Also return the preferences on set Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> parents: 8250 diff changeset	79 origin.send(reply);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	80 return true;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	81 end);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	82
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	83 local query_form = dataform {
11875 210a785dfa8a mod_mam: Use util.dataforms timestamp validation Kim Alvefur <zash@zash.se> parents: 11819 diff changeset	84 { name = "FORM_TYPE"; type = "hidden"; value = xmlns_mam };
210a785dfa8a mod_mam: Use util.dataforms timestamp validation Kim Alvefur <zash@zash.se> parents: 11819 diff changeset	85 { name = "with"; type = "jid-single" };
210a785dfa8a mod_mam: Use util.dataforms timestamp validation Kim Alvefur <zash@zash.se> parents: 11819 diff changeset	86 { name = "start"; type = "text-single"; datatype = "xs:dateTime" };
210a785dfa8a mod_mam: Use util.dataforms timestamp validation Kim Alvefur <zash@zash.se> parents: 11819 diff changeset	87 { name = "end"; type = "text-single"; datatype = "xs:dateTime" };
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	88 };
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	89
11272 44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	90 if archive.caps and archive.caps.full_id_range then
44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	91 table.insert(query_form, { name = "before-id"; type = "text-single"; });
44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	92 table.insert(query_form, { name = "after-id"; type = "text-single"; });
44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	93 end
44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	94
11280 e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	95 if archive.caps and archive.caps.ids then
e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	96 table.insert(query_form, { name = "ids"; type = "list-multi"; });
e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	97 end
e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	98
e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	99
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	100 -- Serve form
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	101 module:hook("iq-get/self/"..xmlns_mam..":query", function(event)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	102 local origin, stanza = event.origin, event.stanza;
8538 3eb4cafb3b64 mod_mam: Implement option to enable MAM implicitly when client support is detected (#867) Kim Alvefur <zash@zash.se> parents: 8252 diff changeset	103 get_prefs(origin.username, true);
7904 c011cecad576 mod_mam: Add missing wrapping <query> element when returning the query form Kim Alvefur <zash@zash.se> parents: 7903 diff changeset	104 origin.send(st.reply(stanza):query(xmlns_mam):add_child(query_form:form()));
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	105 return true;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	106 end);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	107
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	108 -- Handle archive queries
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	109 module:hook("iq-set/self/"..xmlns_mam..":query", function(event)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	110 local origin, stanza = event.origin, event.stanza;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	111 local query = stanza.tags[1];
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	112 local qid = query.attr.queryid;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	113
10298 906ea5e1ec75 mod_mam: Add flag to session when it performs a MAM query Matthew Wild <mwild1@gmail.com> parents: 10028 diff changeset	114 origin.mam_requested = true;
906ea5e1ec75 mod_mam: Add flag to session when it performs a MAM query Matthew Wild <mwild1@gmail.com> parents: 10028 diff changeset	115
8538 3eb4cafb3b64 mod_mam: Implement option to enable MAM implicitly when client support is detected (#867) Kim Alvefur <zash@zash.se> parents: 8252 diff changeset	116 get_prefs(origin.username, true);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	117
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	118 -- Search query parameters
11280 e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	119 local qwith, qstart, qend, qbefore, qafter, qids;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	120 local form = query:get_child("x", "jabber:x:data");
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	121 if form then
10559 cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	122 local form_type, err = get_form_type(form);
cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	123 if not form_type then
cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	124 origin.send(st.error_reply(stanza, "modify", "bad-request", "Invalid dataform: "..err));
cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	125 return true;
cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	126 elseif form_type ~= xmlns_mam then
cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	127 origin.send(st.error_reply(stanza, "modify", "bad-request", "Unexpected FORM_TYPE, expected '"..xmlns_mam.."'"));
cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	128 return true;
cfc05e46b979 mod_mam: More careful validation of MAM query form Kim Alvefur <zash@zash.se> parents: 10299 diff changeset	129 end
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	130 form, err = query_form:data(form);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	131 if err then
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	132 origin.send(st.error_reply(stanza, "modify", "bad-request", select(2, next(err))));
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	133 return true;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	134 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	135 qwith, qstart, qend = form["with"], form["start"], form["end"];
11272 44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	136 qbefore, qafter = form["before-id"], form["after-id"];
11280 e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	137 qids = form["ids"];
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	138 qwith = qwith and jid_bare(qwith); -- dataforms does jidprep
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	139 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	140
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	141 -- RSM stuff
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	142 local qset = rsm.get(query);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	143 local qmax = m_min(qset and qset.max or default_max_items, max_max_items);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	144 local reverse = qset and qset.before or false;
11272 44c9cb4094bb mod_mam: Add support for before-id and after-id fields Kim Alvefur <zash@zash.se> parents: 11270 diff changeset	145 local before, after = qset and qset.before or qbefore, qset and qset.after or qafter;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	146 if type(before) ~= "string" then before = nil; end
11819 f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	147
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	148
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	149 module:log("debug", "Archive query by %s id=%s with=%s when=%s...%s rsm=%q",
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	150 origin.username,
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	151 qid or stanza.attr.id,
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	152 qwith or "*",
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	153 qstart and timestamp(qstart) or "",
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	154 qend and timestamp(qend) or "",
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	155 qset);
f590e3c51eff mod_mam: Merge main and RSM-specific log message here too Kim Alvefur <zash@zash.se> parents: 11795 diff changeset	156
11269 342ac5d806fb mod_mam: Add support for page flipping Kim Alvefur <zash@zash.se> parents: 11268 diff changeset	157 -- A reverse query needs to be flipped
342ac5d806fb mod_mam: Add support for page flipping Kim Alvefur <zash@zash.se> parents: 11268 diff changeset	158 local flip = reverse;
342ac5d806fb mod_mam: Add support for page flipping Kim Alvefur <zash@zash.se> parents: 11268 diff changeset	159 -- A flip-page query needs to be the opposite of that.
342ac5d806fb mod_mam: Add support for page flipping Kim Alvefur <zash@zash.se> parents: 11268 diff changeset	160 if query:get_child("flip-page") then flip = not flip end
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	161
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	162 -- Load all the data!
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	163 local data, err = archive:find(origin.username, {
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	164 start = qstart; ["end"] = qend; -- Time range
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	165 with = qwith;
8904 faca839ddbbb mod_mam: Handle edge-case of max=0 so that complete attr is set (fixes #1128) Kim Alvefur <zash@zash.se> parents: 8580 diff changeset	166 limit = qmax == 0 and 0 or qmax + 1;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	167 before = before; after = after;
11280 e35e98541ca0 mod_mam: Allow querying by set of IDs Kim Alvefur <zash@zash.se> parents: 11272 diff changeset	168 ids = qids;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	169 reverse = reverse;
8172 66e32c34250b mod_mam: Request a total count if no items are requested Kim Alvefur <zash@zash.se> parents: 8149 diff changeset	170 total = use_total or qmax == 0;
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	171 });
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	172
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	173 if not data then
10076 ee85cea08127 mod_mam: Log query failure reason Kim Alvefur <zash@zash.se> parents: 10075 diff changeset	174 module:log("debug", "Archive query id=%s failed: %s", qid or stanza.attr.id, err);
10021 4715c5d1eb69 mod_mam: Propagate item-not-found to client (fixes #1325) Kim Alvefur <zash@zash.se> parents: 9894 diff changeset	175 if err == "item-not-found" then
4715c5d1eb69 mod_mam: Propagate item-not-found to client (fixes #1325) Kim Alvefur <zash@zash.se> parents: 9894 diff changeset	176 origin.send(st.error_reply(stanza, "modify", "item-not-found"));
4715c5d1eb69 mod_mam: Propagate item-not-found to client (fixes #1325) Kim Alvefur <zash@zash.se> parents: 9894 diff changeset	177 else
4715c5d1eb69 mod_mam: Propagate item-not-found to client (fixes #1325) Kim Alvefur <zash@zash.se> parents: 9894 diff changeset	178 origin.send(st.error_reply(stanza, "cancel", "internal-server-error"));
4715c5d1eb69 mod_mam: Propagate item-not-found to client (fixes #1325) Kim Alvefur <zash@zash.se> parents: 9894 diff changeset	179 end
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	180 return true;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	181 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	182 local total = tonumber(err);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	183
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	184 local msg_reply_attr = { to = stanza.attr.from, from = stanza.attr.to };
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	185
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	186 local results = {};
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	187
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	188 -- Wrap it in stuff and deliver
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	189 local first, last;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	190 local count = 0;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	191 local complete = "true";
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	192 for id, item, when in data do
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	193 count = count + 1;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	194 if count > qmax then
8905 65d1a80d3565 mod_mam: Add coment on how 'complete' works Kim Alvefur <zash@zash.se> parents: 8904 diff changeset	195 -- We requested qmax+1 items. If that many items are retrieved then
65d1a80d3565 mod_mam: Add coment on how 'complete' works Kim Alvefur <zash@zash.se> parents: 8904 diff changeset	196 -- there are more results to page through, so:
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	197 complete = nil;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	198 break;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	199 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	200 local fwd_st = st.message(msg_reply_attr)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	201 :tag("result", { xmlns = xmlns_mam, queryid = qid, id = id })
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	202 :tag("forwarded", { xmlns = xmlns_forward })
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	203 :tag("delay", { xmlns = xmlns_delay, stamp = timestamp(when) }):up();
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	204
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	205 if not is_stanza(item) then
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	206 item = st.deserialize(item);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	207 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	208 item.attr.xmlns = "jabber:client";
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	209 fwd_st:add_child(item);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	210
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	211 if not first then first = id; end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	212 last = id;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	213
11269 342ac5d806fb mod_mam: Add support for page flipping Kim Alvefur <zash@zash.se> parents: 11268 diff changeset	214 if flip then
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	215 results[count] = fwd_st;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	216 else
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	217 origin.send(fwd_st);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	218 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	219 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	220
11269 342ac5d806fb mod_mam: Add support for page flipping Kim Alvefur <zash@zash.se> parents: 11268 diff changeset	221 if flip then
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	222 for i = #results, 1, -1 do
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	223 origin.send(results[i]);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	224 end
11269 342ac5d806fb mod_mam: Add support for page flipping Kim Alvefur <zash@zash.se> parents: 11268 diff changeset	225 end
342ac5d806fb mod_mam: Add support for page flipping Kim Alvefur <zash@zash.se> parents: 11268 diff changeset	226 if reverse then
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	227 first, last = last, first;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	228 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	229
7839 f3e1925f29c2 mod_mam: Update to XEP-0313 v0.5.1 Kim Alvefur <zash@zash.se> parents: 7838 diff changeset	230 origin.send(st.reply(stanza)
11337 f89c8e6beaa6 mod_mam: Remove obsolete 'queryid' attribute from iq-result (thanks paul) Kim Alvefur <zash@zash.se> parents: 11281 diff changeset	231 :tag("fin", { xmlns = xmlns_mam, complete = complete })
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	232 :add_child(rsm.generate {
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	233 first = first, last = last, count = total }));
10078 44371850c6b0 mod_mam: Move final log message to end of query procedure Kim Alvefur <zash@zash.se> parents: 10077 diff changeset	234
44371850c6b0 mod_mam: Move final log message to end of query procedure Kim Alvefur <zash@zash.se> parents: 10077 diff changeset	235 -- That's all folks!
10079 a36c731ed540 mod_mam: Report correct count of results for forward queries Kim Alvefur <zash@zash.se> parents: 10078 diff changeset	236 module:log("debug", "Archive query id=%s completed, %d items returned", qid or stanza.attr.id, complete and count or count - 1);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	237 return true;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	238 end);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	239
11270 eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	240 module:hook("iq-get/self/"..xmlns_mam..":metadata", function (event)
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	241 local origin, stanza = event.origin, event.stanza;
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	242
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	243 local reply = st.reply(stanza):tag("metadata", { xmlns = xmlns_mam });
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	244
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	245 do
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	246 local first = archive:find(origin.username, { limit = 1 });
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	247 if not first then
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	248 origin.send(st.error_reply(stanza, "cancel", "internal-server-error"));
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	249 return true;
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	250 end
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	251
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	252 local id, _, when = first();
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	253 if id then
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	254 reply:tag("start", { id = id, timestamp = timestamp(when) }):up();
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	255 end
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	256 end
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	257
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	258 do
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	259 local last = archive:find(origin.username, { limit = 1, reverse = true });
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	260 if not last then
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	261 origin.send(st.error_reply(stanza, "cancel", "internal-server-error"));
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	262 return true;
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	263 end
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	264
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	265 local id, _, when = last();
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	266 if id then
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	267 reply:tag("end", { id = id, timestamp = timestamp(when) }):up();
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	268 end
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	269 end
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	270
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	271 origin.send(reply);
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	272 return true;
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	273 end);
eaaa8ca742a7 mod_mam: Implement extended MAM metadata query Kim Alvefur <zash@zash.se> parents: 11269 diff changeset	274
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	275 local function has_in_roster(user, who)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	276 local roster = rm_load_roster(user, host);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	277 module:log("debug", "%s has %s in roster? %s", user, who, roster[who] and "yes" or "no");
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	278 return roster[who];
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	279 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	280
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	281 local function shall_store(user, who)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	282 -- TODO Cache this?
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	283 if not um.user_exists(user, host) then
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	284 module:log("debug", "%s@%s does not exist", user, host)
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	285 return false;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	286 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	287 local prefs = get_prefs(user);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	288 local rule = prefs[who];
10111 0f335815244f plugins: Remove tostring call from logging Kim Alvefur <zash@zash.se> parents: 10079 diff changeset	289 module:log("debug", "%s's rule for %s is %s", user, who, rule);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	290 if rule ~= nil then
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	291 return rule;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	292 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	293 -- Below could be done by a metatable
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	294 local default = prefs[false];
10111 0f335815244f plugins: Remove tostring call from logging Kim Alvefur <zash@zash.se> parents: 10079 diff changeset	295 module:log("debug", "%s's default rule is %s", user, default);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	296 if default == "roster" then
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	297 return has_in_roster(user, who);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	298 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	299 return default;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	300 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	301
8207 8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	302 local function strip_stanza_id(stanza, user)
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	303 if stanza:get_child("stanza-id", xmlns_st_id) then
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	304 stanza = st.clone(stanza);
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	305 stanza:maptags(function (tag)
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	306 if tag.name == "stanza-id" and tag.attr.xmlns == xmlns_st_id then
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	307 local by_user, by_host, res = jid_prepped_split(tag.attr.by);
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	308 if not res and by_host == host and by_user == user then
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	309 return nil;
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	310 end
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	311 end
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	312 return tag;
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	313 end);
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	314 end
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	315 return stanza;
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	316 end
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	317
12306 81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	318 local function should_store(stanza) --> boolean, reason: string
10736 c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	319 local st_type = stanza.attr.type or "normal";
10751 4db4bd8a7822 mod_mam: Don't store any groupchat messages Kim Alvefur <zash@zash.se> parents: 10746 diff changeset	320 -- FIXME pass direction of stanza and use that along with bare/full JID addressing
4db4bd8a7822 mod_mam: Don't store any groupchat messages Kim Alvefur <zash@zash.se> parents: 10746 diff changeset	321 -- for more accurate MUC / type=groupchat check
10734 136c41a3d03c mod_mam: Factor out "should we store this" into a function Kim Alvefur <zash@zash.se> parents: 10683 diff changeset	322
10736 c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	323 if st_type == "headline" then
c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	324 -- Headline messages are ephemeral by definition
c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	325 return false, "headline";
c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	326 end
12306 81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	327 if st_type == "error" then
12030 9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	328 -- Errors not sent sent from a local client
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	329 -- Why would a client send an error anyway?
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	330 if jid_resource(stanza.attr.to) then
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	331 -- Store delivery failure notifications so you know if your own messages
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	332 -- were not delivered.
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	333 return true, "bounce";
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	334 else
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	335 -- Skip errors for messages that come from your account, such as PEP
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	336 -- notifications.
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	337 return false, "bounce";
9f8206e99b89 mod_mam: Avoid storing bounces for messages from the bare account (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 11988 diff changeset	338 end
10745 79b29f35fac1 mod_mam: Save delivery failures (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 10744 diff changeset	339 end
10751 4db4bd8a7822 mod_mam: Don't store any groupchat messages Kim Alvefur <zash@zash.se> parents: 10746 diff changeset	340 if st_type == "groupchat" then
10736 c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	341 -- MUC messages always go to the full JID, usually archived by the MUC
c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	342 return false, "groupchat";
c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	343 end
10743 3967cf10de1b mod_mam: Respect no-store hint (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 10742 diff changeset	344 if stanza:get_child("no-store", "urn:xmpp:hints")
3967cf10de1b mod_mam: Respect no-store hint (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 10742 diff changeset	345 or stanza:get_child("no-permanent-store", "urn:xmpp:hints") then
10746 2e31d67b9a29 mod_mam: Make note of Experimental (or Deferred) XEPs Kim Alvefur <zash@zash.se> parents: 10745 diff changeset	346 -- XXX Experimental XEP
10737 b2ede421adeb mod_mam: Rework hints handling Kim Alvefur <zash@zash.se> parents: 10736 diff changeset	347 return false, "hint";
b2ede421adeb mod_mam: Rework hints handling Kim Alvefur <zash@zash.se> parents: 10736 diff changeset	348 end
b2ede421adeb mod_mam: Rework hints handling Kim Alvefur <zash@zash.se> parents: 10736 diff changeset	349 if stanza:get_child("store", "urn:xmpp:hints") then
b2ede421adeb mod_mam: Rework hints handling Kim Alvefur <zash@zash.se> parents: 10736 diff changeset	350 return true, "hint";
b2ede421adeb mod_mam: Rework hints handling Kim Alvefur <zash@zash.se> parents: 10736 diff changeset	351 end
10736 c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	352 if stanza:get_child("body") then
c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	353 return true, "body";
c5a3576a5335 mod_mam: Invert check for type Kim Alvefur <zash@zash.se> parents: 10735 diff changeset	354 end
10738 19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	355 if stanza:get_child("subject") then
10742 70b7c8f3d32d mod_mam: Fix typo in comment Kim Alvefur <zash@zash.se> parents: 10741 diff changeset	356 -- XXX Who would send a message with a subject but without a body?
10738 19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	357 return true, "subject";
19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	358 end
19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	359 if stanza:get_child("encryption", "urn:xmpp:eme:0") then
19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	360 -- Since we can't know what an encrypted message contains, we assume it's important
10746 2e31d67b9a29 mod_mam: Make note of Experimental (or Deferred) XEPs Kim Alvefur <zash@zash.se> parents: 10745 diff changeset	361 -- XXX Experimental XEP
10738 19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	362 return true, "encrypted";
19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	363 end
10739 50f0a4d83731 mod_mam: Store XEP-0184 receipts and requests Kim Alvefur <zash@zash.se> parents: 10738 diff changeset	364 if stanza:get_child(nil, "urn:xmpp:receipts") then
50f0a4d83731 mod_mam: Store XEP-0184 receipts and requests Kim Alvefur <zash@zash.se> parents: 10738 diff changeset	365 -- If it's important enough to ask for a receipt then it's important enough to archive
50f0a4d83731 mod_mam: Store XEP-0184 receipts and requests Kim Alvefur <zash@zash.se> parents: 10738 diff changeset	366 -- and the same applies to the receipt
50f0a4d83731 mod_mam: Store XEP-0184 receipts and requests Kim Alvefur <zash@zash.se> parents: 10738 diff changeset	367 return true, "receipt";
50f0a4d83731 mod_mam: Store XEP-0184 receipts and requests Kim Alvefur <zash@zash.se> parents: 10738 diff changeset	368 end
10744 16002abe61b1 mod_mam: Keep chat markers (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 10743 diff changeset	369 if stanza:get_child(nil, "urn:xmpp:chat-markers:0") then
10746 2e31d67b9a29 mod_mam: Make note of Experimental (or Deferred) XEPs Kim Alvefur <zash@zash.se> parents: 10745 diff changeset	370 -- XXX Experimental XEP
10744 16002abe61b1 mod_mam: Keep chat markers (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 10743 diff changeset	371 return true, "marker";
16002abe61b1 mod_mam: Keep chat markers (thanks Ge0rG) Kim Alvefur <zash@zash.se> parents: 10743 diff changeset	372 end
10738 19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	373 if stanza:get_child("x", "jabber:x:conference")
19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	374 or stanza:find("{http://jabber.org/protocol/muc#user}x/invite") then
19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	375 return true, "invite";
19ffb2ebf114 mod_mam: Add more positive hints for storage Kim Alvefur <zash@zash.se> parents: 10737 diff changeset	376 end
12234 1c47162dd965 plugins: Update for namespace bump in XEP-0353 v0.4.0 Kim Alvefur <zash@zash.se> parents: 12030 diff changeset	377 if stanza:get_child(nil, "urn:xmpp:jingle-message:0") or stanza:get_child(nil, "urn:xmpp:jingle-message:1") then
11260 08b397c21805 mod_csi_simple,mod_carbons,mod_mam: Update comment about XEP-0353 Kim Alvefur <zash@zash.se> parents: 10821 diff changeset	378 -- XXX Experimental XEP
10821 30fc1ed5647a mod_mam: Archive XEP-0353: Jingle Message Initiation Kim Alvefur <zash@zash.se> parents: 10800 diff changeset	379 return true, "jingle call";
30fc1ed5647a mod_mam: Archive XEP-0353: Jingle Message Initiation Kim Alvefur <zash@zash.se> parents: 10800 diff changeset	380 end
10734 136c41a3d03c mod_mam: Factor out "should we store this" into a function Kim Alvefur <zash@zash.se> parents: 10683 diff changeset	381
10741 27f1fcd85ccd mod_mam: Prefer not archiving if no interesting payloads are found Kim Alvefur <zash@zash.se> parents: 10740 diff changeset	382 -- The IM-NG thing to do here would be to return `not st_to_full`
27f1fcd85ccd mod_mam: Prefer not archiving if no interesting payloads are found Kim Alvefur <zash@zash.se> parents: 10740 diff changeset	383 -- One day ...
27f1fcd85ccd mod_mam: Prefer not archiving if no interesting payloads are found Kim Alvefur <zash@zash.se> parents: 10740 diff changeset	384 return false, "default";
10734 136c41a3d03c mod_mam: Factor out "should we store this" into a function Kim Alvefur <zash@zash.se> parents: 10683 diff changeset	385 end
136c41a3d03c mod_mam: Factor out "should we store this" into a function Kim Alvefur <zash@zash.se> parents: 10683 diff changeset	386
12306 81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	387 module:hook("archive-should-store", function (event)
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	388 local should, why = should_store(event.stanza);
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	389 event.reason = why;
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	390 return should;
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	391 end, -1)
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	392
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	393 -- Handle messages
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	394 local function message_handler(event, c2s)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	395 local origin, stanza = event.origin, event.stanza;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	396 local log = c2s and origin.log or module._log;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	397 local orig_from = stanza.attr.from;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	398 local orig_to = stanza.attr.to or orig_from;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	399 -- Stanza without 'to' are treated as if it was to their own bare jid
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	400
11727 f3aee8a825cc Fix various spelling errors (thanks codespell) Kim Alvefur <zash@zash.se> parents: 11337 diff changeset	401 -- Whose storage do we put it in?
7844 316f5166eedb mod_mam: Find out which party is the user and which is the 'with' earlier Kim Alvefur <zash@zash.se> parents: 7843 diff changeset	402 local store_user = c2s and origin.username or jid_split(orig_to);
316f5166eedb mod_mam: Find out which party is the user and which is the 'with' earlier Kim Alvefur <zash@zash.se> parents: 7843 diff changeset	403 -- And who are they chatting with?
316f5166eedb mod_mam: Find out which party is the user and which is the 'with' earlier Kim Alvefur <zash@zash.se> parents: 7843 diff changeset	404 local with = jid_bare(c2s and orig_to or orig_from);
316f5166eedb mod_mam: Find out which party is the user and which is the 'with' earlier Kim Alvefur <zash@zash.se> parents: 7843 diff changeset	405
7845 eeb22f912577 mod_mam: Filter out spoofed XEP-0359 tags Kim Alvefur <zash@zash.se> parents: 7844 diff changeset	406 -- Filter out <stanza-id> that claim to be from us
8207 8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	407 event.stanza = strip_stanza_id(stanza, store_user);
7845 eeb22f912577 mod_mam: Filter out spoofed XEP-0359 tags Kim Alvefur <zash@zash.se> parents: 7844 diff changeset	408
12306 81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	409 local event_payload = { stanza = stanza; session = origin };
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	410 local should = module:fire_event("archive-should-store", event_payload);
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	411 local why = event_payload.reason;
81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	412
10734 136c41a3d03c mod_mam: Factor out "should we store this" into a function Kim Alvefur <zash@zash.se> parents: 10683 diff changeset	413 if not should then
12306 81fc7fc77e68 mod_mam: Allow plugging into archive decision Kim Alvefur <zash@zash.se> parents: 12234 diff changeset	414 log("debug", "Not archiving stanza: %s (%s)", stanza:top_tag(), event_payload.reason);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	415 return;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	416 end
7842 9332b43931f5 mod_mam: Add some comments explaining archive expiry Kim Alvefur <zash@zash.se> parents: 7841 diff changeset	417
8193 bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	418 local clone_for_storage;
7849 93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	419 if not strip_tags:empty() then
8193 bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	420 clone_for_storage = st.clone(stanza);
bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	421 clone_for_storage:maptags(function (tag)
7849 93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	422 if strip_tags:contains(tag.attr.xmlns) then
93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	423 return nil;
93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	424 else
93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	425 return tag;
93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	426 end
93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	427 end);
8193 bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	428 if #clone_for_storage.tags == 0 then
8250 9ea5ea53744b mod_mam: Log a message when not archiving because it only had ignored tags Kim Alvefur <zash@zash.se> parents: 8231 diff changeset	429 log("debug", "Not archiving stanza: %s (empty when stripped)", stanza:top_tag());
7849 93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	430 return;
93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	431 end
8193 bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	432 else
bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	433 clone_for_storage = stanza;
7849 93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	434 end
93a068ef4b2c mod_mam: Allow a set of namespaces to be stripped from stored stanzas, default to chat states (fixes #763) Kim Alvefur <zash@zash.se> parents: 7848 diff changeset	435
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	436 -- Check with the users preferences
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	437 if shall_store(store_user, with) then
10735 f2838ffcc499 mod_mam: Log 'why' a stanza is archived Kim Alvefur <zash@zash.se> parents: 10734 diff changeset	438 log("debug", "Archiving stanza: %s (%s)", stanza:top_tag(), why);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	439
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	440 -- And stash it
9886 710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	441 local time = time_now();
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	442 local ok, err = archive:append(store_user, nil, clone_for_storage, time, with);
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	443 if not ok and err == "quota-limit" then
9894 8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	444 if type(cleanup_after) == "number" then
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	445 module:log("debug", "User '%s' over quota, cleaning archive", store_user);
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	446 local cleaned = archive:delete(store_user, {
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	447 ["end"] = (os.time() - cleanup_after);
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	448 });
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	449 if cleaned then
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	450 ok, err = archive:append(store_user, nil, clone_for_storage, time, with);
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	451 end
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	452 end
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	453 if not ok and (archive.caps and archive.caps.truncate) then
8747ccf0008c mod_mam: On quota hit, separately delete by time and by item count Kim Alvefur <zash@zash.se> parents: 9886 diff changeset	454 module:log("debug", "User '%s' over quota, truncating archive", store_user);
9886 710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	455 local truncated = archive:delete(store_user, {
10800 62794e065e33 MAM: Remove 1% of contents when reaching limits, fix #1545 Kim Alvefur <zash@zash.se> parents: 10773 diff changeset	456 truncate = archive_truncate;
9886 710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	457 });
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	458 if truncated then
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	459 ok, err = archive:append(store_user, nil, clone_for_storage, time, with);
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	460 end
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	461 end
710a116341cd mod_mam: Trim archive when quota has been exceeded Kim Alvefur <zash@zash.se> parents: 9882 diff changeset	462 end
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	463 if ok then
8193 bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	464 local clone_for_other_handlers = st.clone(stanza);
7908 dbdaa8487ecd mod_mam: Fix to treat first return value from archive:append as assigned ID Kim Alvefur <zash@zash.se> parents: 7906 diff changeset	465 local id = ok;
8193 bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	466 clone_for_other_handlers:tag("stanza-id", { xmlns = xmlns_st_id, by = store_user.."@"..host, id = id }):up();
bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	467 event.stanza = clone_for_other_handlers;
8230 154852646095 mod_mam: Use a FIFO queue for scheduling archive expiry Kim Alvefur <zash@zash.se> parents: 8207 diff changeset	468 schedule_cleanup(store_user);
8193 bb0118e46c45 mod_mam: Clone stanzas before mutating (thanks waqas) (fixes #961) Kim Alvefur <zash@zash.se> parents: 8172 diff changeset	469 module:fire_event("archive-message-added", { origin = origin, stanza = clone_for_storage, for_user = store_user, id = id });
10523 86422db90e02 mod_mam: Log error when unable to store stanza (fix #1478) Kim Alvefur <zash@zash.se> parents: 10298 diff changeset	470 else
86422db90e02 mod_mam: Log error when unable to store stanza (fix #1478) Kim Alvefur <zash@zash.se> parents: 10298 diff changeset	471 log("error", "Could not archive stanza: %s", err);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	472 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	473 else
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	474 log("debug", "Not archiving stanza: %s (prefs)", stanza:top_tag());
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	475 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	476 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	477
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	478 local function c2s_message_handler(event)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	479 return message_handler(event, true);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	480 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	481
8207 8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	482 -- Filter out <stanza-id> before the message leaves the server to prevent privacy leak.
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	483 local function strip_stanza_id_after_other_events(event)
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	484 event.stanza = strip_stanza_id(event.stanza, event.origin.username);
7902 92b70a921acb mod_mam: Strip stanza-id tags after carbons Kim Alvefur <zash@zash.se> parents: 7901 diff changeset	485 end
92b70a921acb mod_mam: Strip stanza-id tags after carbons Kim Alvefur <zash@zash.se> parents: 7901 diff changeset	486
8207 8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	487 module:hook("pre-message/bare", strip_stanza_id_after_other_events, -1);
8ea0484871e8 mod_mam: Factor out <stanza-id> stripping so it can be reused in two places Kim Alvefur <zash@zash.se> parents: 8206 diff changeset	488 module:hook("pre-message/full", strip_stanza_id_after_other_events, -1);
7902 92b70a921acb mod_mam: Strip stanza-id tags after carbons Kim Alvefur <zash@zash.se> parents: 7901 diff changeset	489
11758 c35b81575d5d mod_mam: Explain behavior with absent mod_offline in a comment Kim Alvefur <zash@zash.se> parents: 11757 diff changeset	490 -- Catch messages not stored by mod_offline and mark them as stored if they
c35b81575d5d mod_mam: Explain behavior with absent mod_offline in a comment Kim Alvefur <zash@zash.se> parents: 11757 diff changeset	491 -- have been archived. This would generally only happen if mod_offline is
c35b81575d5d mod_mam: Explain behavior with absent mod_offline in a comment Kim Alvefur <zash@zash.se> parents: 11757 diff changeset	492 -- disabled. Otherwise the message would generate a delivery failure report,
c35b81575d5d mod_mam: Explain behavior with absent mod_offline in a comment Kim Alvefur <zash@zash.se> parents: 11757 diff changeset	493 -- which would not be accurate because it has been archived.
11757 8141645e3865 mod_mam: "Handle" messages that have been archived in the absense of mod_offline Kim Alvefur <zash@zash.se> parents: 11727 diff changeset	494 module:hook("message/offline/handle", function(event)
8141645e3865 mod_mam: "Handle" messages that have been archived in the absense of mod_offline Kim Alvefur <zash@zash.se> parents: 11727 diff changeset	495 local stanza = event.stanza;
11759 9925be5d3b8b mod_mam: Only check for locally generated stanza-ids Kim Alvefur <zash@zash.se> parents: 11758 diff changeset	496 local user = event.username .. "@" .. host;
11795 41af102c7190 mod_mam: Reduce line count using new util.stanza attr method Kim Alvefur <zash@zash.se> parents: 11760 diff changeset	497 if stanza:get_child_with_attr("stanza-id", xmlns_st_id, "by", user) then
41af102c7190 mod_mam: Reduce line count using new util.stanza attr method Kim Alvefur <zash@zash.se> parents: 11760 diff changeset	498 return true;
11757 8141645e3865 mod_mam: "Handle" messages that have been archived in the absense of mod_offline Kim Alvefur <zash@zash.se> parents: 11727 diff changeset	499 end
8141645e3865 mod_mam: "Handle" messages that have been archived in the absense of mod_offline Kim Alvefur <zash@zash.se> parents: 11727 diff changeset	500 end, -2);
8141645e3865 mod_mam: "Handle" messages that have been archived in the absense of mod_offline Kim Alvefur <zash@zash.se> parents: 11727 diff changeset	501
11760 d66738eeb875 mod_mam: Suppress offline message broadcast for MAM clients Kim Alvefur <zash@zash.se> parents: 11759 diff changeset	502 -- Don't broadcast offline messages to clients that have queried the archive.
d66738eeb875 mod_mam: Suppress offline message broadcast for MAM clients Kim Alvefur <zash@zash.se> parents: 11759 diff changeset	503 module:hook("message/offline/broadcast", function (event)
d66738eeb875 mod_mam: Suppress offline message broadcast for MAM clients Kim Alvefur <zash@zash.se> parents: 11759 diff changeset	504 if event.origin.mam_requested then
d66738eeb875 mod_mam: Suppress offline message broadcast for MAM clients Kim Alvefur <zash@zash.se> parents: 11759 diff changeset	505 return true;
d66738eeb875 mod_mam: Suppress offline message broadcast for MAM clients Kim Alvefur <zash@zash.se> parents: 11759 diff changeset	506 end
d66738eeb875 mod_mam: Suppress offline message broadcast for MAM clients Kim Alvefur <zash@zash.se> parents: 11759 diff changeset	507 end);
d66738eeb875 mod_mam: Suppress offline message broadcast for MAM clients Kim Alvefur <zash@zash.se> parents: 11759 diff changeset	508
8575 5040c8ce32dd Backed out changeset 97a094fdf101, interferes with 6ddddfe05a74 Kim Alvefur <zash@zash.se> parents: 8538 diff changeset	509 if cleanup_after ~= "never" then
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	510 local cleanup_storage = module:open_store("archive_cleanup");
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	511 local cleanup_map = module:open_store("archive_cleanup", "map");
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	512
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	513 local day = 86400;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	514 local multipliers = { d = day, w = day * 7, m = 31 * day, y = 365.2425 * day };
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	515 local n, m = cleanup_after:lower():match("(%d+)%s*([dwmy]?)");
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	516 if not n then
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	517 module:log("error", "Could not parse archive_expires_after string %q", cleanup_after);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	518 return false;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	519 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	520
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	521 cleanup_after = tonumber(n) * ( multipliers[m] or 1 );
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	522
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	523 module:log("debug", "archive_expires_after = %d -- in seconds", cleanup_after);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	524
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	525 if not archive.delete then
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	526 module:log("error", "archive_expires_after set but mod_%s does not support deleting", archive._provided_by);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	527 return false;
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	528 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	529
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	530 -- For each day, store a set of users that have new messages. To expire
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	531 -- messages, we collect the union of sets of users from dates that fall
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	532 -- outside the cleanup range.
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	533
11968 6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	534 if not (archive.caps and archive.caps.wildcard_delete) then
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	535 local last_date = require "util.cache".new(module:get_option_number("archive_cleanup_date_cache_size", 1000));
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	536 function schedule_cleanup(username, date)
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	537 date = date or datestamp();
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	538 if last_date:get(username) == date then return end
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	539 local ok = cleanup_map:set(date, username, true);
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	540 if ok then
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	541 last_date:set(username, date);
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	542 end
10028 79ba2d709e72 mod_mam: Cache last date that archive owner has messages to reduce writes (fixes #1368) Kim Alvefur <zash@zash.se> parents: 9879 diff changeset	543 end
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	544 end
11968 6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	545
9757 03ed7f10d8da mod_mam: Measure how long it takes to run the message expiry job job Kim Alvefur <zash@zash.se> parents: 9752 diff changeset	546 local cleanup_time = module:measure("cleanup", "times");
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	547
10682 62ef68f95b6f mod_mam,mod_muc_mam: Allow other work to be performed during archive cleanup (fixes #1504) Kim Alvefur <zash@zash.se> parents: 10567 diff changeset	548 local async = require "util.async";
11988 18c0ca5fcbb8 mod_mam: Switch to new cron API Kim Alvefur <zash@zash.se> parents: 11968 diff changeset	549 module:daily("Remove expired messages", function ()
9757 03ed7f10d8da mod_mam: Measure how long it takes to run the message expiry job job Kim Alvefur <zash@zash.se> parents: 9752 diff changeset	550 local cleanup_done = cleanup_time();
11968 6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	551
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	552 if archive.caps and archive.caps.wildcard_delete then
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	553 local ok, err = archive:delete(true, { ["end"] = os.time() - cleanup_after })
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	554 if ok then
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	555 local sum = tonumber(ok);
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	556 if sum then
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	557 module:log("info", "Deleted %d expired messages", sum);
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	558 else
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	559 -- driver did not tell
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	560 module:log("info", "Deleted all expired messages");
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	561 end
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	562 else
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	563 module:log("error", "Could not delete messages: %s", err);
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	564 end
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	565 cleanup_done();
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	566 return;
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	567 end
6e1af07921d1 mod_mam,mod_muc_mam: Simplify deletion when multi-user-deletion is supported Kim Alvefur <zash@zash.se> parents: 11875 diff changeset	568
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	569 local users = {};
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	570 local cut_off = datestamp(os.time() - cleanup_after);
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	571 for date in cleanup_storage:users() do
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	572 if date <= cut_off then
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	573 module:log("debug", "Messages from %q should be expired", date);
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	574 local messages_this_day = cleanup_storage:get(date);
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	575 if messages_this_day then
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	576 for user in pairs(messages_this_day) do
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	577 users[user] = true;
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	578 end
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	579 if date < cut_off then
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	580 -- Messages from the same day as the cut-off might not have expired yet,
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	581 -- but all earlier will have, so clear storage for those days.
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	582 cleanup_storage:set(date, nil);
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	583 end
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	584 end
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	585 end
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	586 end
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	587 local sum, num_users = 0, 0;
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	588 for user in pairs(users) do
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	589 local ok, err = archive:delete(user, { ["end"] = os.time() - cleanup_after; })
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	590 if ok then
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	591 num_users = num_users + 1;
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	592 sum = sum + (tonumber(ok) or 0);
10524 7c29a6e652d2 mod_mam: Log error when unable to delete old messages (fix #1479) [luacheck] Kim Alvefur <zash@zash.se> parents: 10523 diff changeset	593 else
10525 9cf7d9761ca2 mod_mam: Schedule cleanup again if unable to delete messages Kim Alvefur <zash@zash.se> parents: 10524 diff changeset	594 cleanup_map:set(cut_off, user, true);
10524 7c29a6e652d2 mod_mam: Log error when unable to delete old messages (fix #1479) [luacheck] Kim Alvefur <zash@zash.se> parents: 10523 diff changeset	595 module:log("error", "Could not delete messages for user '%s': %s", user, err);
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	596 end
10682 62ef68f95b6f mod_mam,mod_muc_mam: Allow other work to be performed during archive cleanup (fixes #1504) Kim Alvefur <zash@zash.se> parents: 10567 diff changeset	597 local wait, done = async.waiter();
62ef68f95b6f mod_mam,mod_muc_mam: Allow other work to be performed during archive cleanup (fixes #1504) Kim Alvefur <zash@zash.se> parents: 10567 diff changeset	598 module:add_timer(0.01, done);
62ef68f95b6f mod_mam,mod_muc_mam: Allow other work to be performed during archive cleanup (fixes #1504) Kim Alvefur <zash@zash.se> parents: 10567 diff changeset	599 wait();
9879 ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	600 end
ddc07fb8dcd4 mod_mam: Perform message expiry based on building an index by date (backport of 39ee70fbb009 from trunk) Kim Alvefur <zash@zash.se> parents: 9555 diff changeset	601 module:log("info", "Deleted %d expired messages for %d users", sum, num_users);
9757 03ed7f10d8da mod_mam: Measure how long it takes to run the message expiry job job Kim Alvefur <zash@zash.se> parents: 9752 diff changeset	602 cleanup_done();
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	603 end);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	604
8132 6ddddfe05a74 mod_mam: Don't ask the storage backend to count all items when expiry is disabled Kim Alvefur <zash@zash.se> parents: 8047 diff changeset	605 else
8576 07ff7b8b702b mod_mam: Log a debug message if archive expiry has been disabled Kim Alvefur <zash@zash.se> parents: 8575 diff changeset	606 module:log("debug", "Archive expiry disabled");
8132 6ddddfe05a74 mod_mam: Don't ask the storage backend to count all items when expiry is disabled Kim Alvefur <zash@zash.se> parents: 8047 diff changeset	607 -- Don't ask the backend to count the potentially unbounded number of items,
6ddddfe05a74 mod_mam: Don't ask the storage backend to count all items when expiry is disabled Kim Alvefur <zash@zash.se> parents: 8047 diff changeset	608 -- it'll get slow.
8577 75d5eee6fcdf mod_mam: Add an option for whether to include 'total' counts by default in queries Kim Alvefur <zash@zash.se> parents: 8576 diff changeset	609 use_total = module:get_option_boolean("mam_include_total", false);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	610 end
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	611
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	612 -- Stanzas sent by local clients
7840 92b6aa3ea2ce mod_mam: Decrease priority to zero Kim Alvefur <zash@zash.se> parents: 7839 diff changeset	613 module:hook("pre-message/bare", c2s_message_handler, 0);
92b6aa3ea2ce mod_mam: Decrease priority to zero Kim Alvefur <zash@zash.se> parents: 7839 diff changeset	614 module:hook("pre-message/full", c2s_message_handler, 0);
7905 daf83bd94906 mod_mam: Fix typo in comment Kim Alvefur <zash@zash.se> parents: 7904 diff changeset	615 -- Stanzas to local clients
7840 92b6aa3ea2ce mod_mam: Decrease priority to zero Kim Alvefur <zash@zash.se> parents: 7839 diff changeset	616 module:hook("message/bare", message_handler, 0);
92b6aa3ea2ce mod_mam: Decrease priority to zero Kim Alvefur <zash@zash.se> parents: 7839 diff changeset	617 module:hook("message/full", message_handler, 0);
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	618
11281 142fb655b885 mod_mam: Advertise extended MAM when archive storage is capable Kim Alvefur <zash@zash.se> parents: 11280 diff changeset	619 local advertise_extended = archive.caps and archive.caps.full_id_range and archive.caps.ids;
11268 e3f6f0b39e7b mod_mam: Advertise extended MAM 0.7.x behind a feature flag Kim Alvefur <zash@zash.se> parents: 11260 diff changeset	620
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	621 module:hook("account-disco-info", function(event)
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	622 (event.reply or event.stanza):tag("feature", {var=xmlns_mam}):up();
11268 e3f6f0b39e7b mod_mam: Advertise extended MAM 0.7.x behind a feature flag Kim Alvefur <zash@zash.se> parents: 11260 diff changeset	623 if advertise_extended then
e3f6f0b39e7b mod_mam: Advertise extended MAM 0.7.x behind a feature flag Kim Alvefur <zash@zash.se> parents: 11260 diff changeset	624 (event.reply or event.stanza):tag("feature", {var=xmlns_mam_ext}):up();
e3f6f0b39e7b mod_mam: Advertise extended MAM 0.7.x behind a feature flag Kim Alvefur <zash@zash.se> parents: 11260 diff changeset	625 end
7891 01d2a2af3146 mod_mam: Advertise Stanza ID support (XEP-0359) (thanks iNPUTmice) Kim Alvefur <zash@zash.se> parents: 7890 diff changeset	626 (event.reply or event.stanza):tag("feature", {var=xmlns_st_id}):up();
7836 30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	627 end);
30fac9154fd4 mod_mam: Import from prosody-modules Kim Alvefur <zash@zash.se> parents: diff changeset	628

prosody

519e6403f455

plugins/mod_mam/mod_mam.lua

Software / code / prosody

Annotate

plugins/mod_mam/mod_mam.lua @ 12555:519e6403f455

Software `/` code `/` prosody