Software `/` code `/` prosody

Annotate

plugins/mod_storage_internal.lua @ 11906:ba3344926e18

MUC: Add option to include form in registration query This was originally not done based on my interpretation of XEP-0045. Today's reading, however, revealed that it actually says the result > SHOULD contain **at least** a <username/> element (emphasis mine) I take this to mean that including a form **is** allowed (and I think this is sensible). Tigase already includes the form I believe. I've gated the new behaviour behind a (default off) option, because it hasn't been tested for compatibility with clients. My primary desire for it is in Snikket, where the clients will be tested to ensure compatibility with this. I don't anticipate that (m)any clients would break, so maybe after 0.12 we can experiment with enabling it by default and eventually remove the option.

author	Matthew Wild <mwild1@gmail.com>
date	Mon, 15 Nov 2021 16:11:03 +0000
parent	11765:1cac469b18d0
child	12977:74b9e05af71e

rev	line source
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	1 local cache = require "util.cache";
4085 7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	2 local datamanager = require "core.storagemanager".olddm;
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	3 local array = require "util.array";
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	4 local datetime = require "util.datetime";
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	5 local st = require "util.stanza";
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	6 local now = require "util.time".now;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	7 local id = require "util.id".medium;
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	8 local jid_join = require "util.jid".join;
11278 c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	9 local set = require "util.set";
4085 7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	10
7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	11 local host = module.host;
7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	12
9888 5669bb11ac7b mod_storage_internal: Increase default quota to 10 000 Kim Alvefur <zash@zash.se> parents: 9885 diff changeset	13 local archive_item_limit = module:get_option_number("storage_archive_item_limit", 10000);
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	14 local archive_item_count_cache = cache.new(module:get_option("storage_archive_item_limit_cache_size", 1000));
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	15
5122 b41c33dc7c36 mod_storage_: Don't explicitly set driver name, to ease copying/renaming modules. Waqas Hussain <waqas20@gmail.com>* parents: 5121 diff changeset	16 local driver = {};
4085 7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	17
5153 688aeac0012a mod_storage_internal, datamanager: Add support for iterating over users with data in a store Kim Alvefur <zash@zash.se> parents: 5133 diff changeset	18 function driver:open(store, typ)
8018 9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	19 local mt = self[typ or "keyval"]
9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	20 if not mt then
6283 7cf6d3a2c855 mod_storage_{none,internal,sql}: Return error for unsupported (everything but keyval) store types Kim Alvefur <zash@zash.se> parents: 5153 diff changeset	21 return nil, "unsupported-store";
7cf6d3a2c855 mod_storage_{none,internal,sql}: Return error for unsupported (everything but keyval) store types Kim Alvefur <zash@zash.se> parents: 5153 diff changeset	22 end
8018 9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	23 return setmetatable({ store = store, type = typ }, mt);
4085 7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	24 end
8018 9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	25
8020 342ce07836de mod_storage_internal: Ignore unused 'self' argument [luacheck] Kim Alvefur <zash@zash.se> parents: 8019 diff changeset	26 function driver:stores(username) -- luacheck: ignore 212/self
8019 925098aad268 mod_storage_internal: Reorder methods Kim Alvefur <zash@zash.se> parents: 8018 diff changeset	27 return datamanager.stores(username, host);
925098aad268 mod_storage_internal: Reorder methods Kim Alvefur <zash@zash.se> parents: 8018 diff changeset	28 end
925098aad268 mod_storage_internal: Reorder methods Kim Alvefur <zash@zash.se> parents: 8018 diff changeset	29
8020 342ce07836de mod_storage_internal: Ignore unused 'self' argument [luacheck] Kim Alvefur <zash@zash.se> parents: 8019 diff changeset	30 function driver:purge(user) -- luacheck: ignore 212/self
8019 925098aad268 mod_storage_internal: Reorder methods Kim Alvefur <zash@zash.se> parents: 8018 diff changeset	31 return datamanager.purge(user, host);
925098aad268 mod_storage_internal: Reorder methods Kim Alvefur <zash@zash.se> parents: 8018 diff changeset	32 end
925098aad268 mod_storage_internal: Reorder methods Kim Alvefur <zash@zash.se> parents: 8018 diff changeset	33
8018 9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	34 local keyval = { };
9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	35 driver.keyval = { __index = keyval };
9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	36
9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	37 function keyval:get(user)
4085 7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	38 return datamanager.load(user, host, self.store);
7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	39 end
7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	40
8018 9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	41 function keyval:set(user, data)
4085 7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	42 return datamanager.store(user, host, self.store, data);
7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	43 end
7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	44
8018 9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	45 function keyval:users()
5153 688aeac0012a mod_storage_internal, datamanager: Add support for iterating over users with data in a store Kim Alvefur <zash@zash.se> parents: 5133 diff changeset	46 return datamanager.users(host, self.store, self.type);
688aeac0012a mod_storage_internal, datamanager: Add support for iterating over users with data in a store Kim Alvefur <zash@zash.se> parents: 5133 diff changeset	47 end
688aeac0012a mod_storage_internal, datamanager: Add support for iterating over users with data in a store Kim Alvefur <zash@zash.se> parents: 5133 diff changeset	48
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	49 local archive = {};
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	50 driver.archive = { __index = archive };
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	51
9884 9751c17f5281 mod_storage_internal,_sql: Expose archive capabilities feature set Kim Alvefur <zash@zash.se> parents: 9883 diff changeset	52 archive.caps = {
9751c17f5281 mod_storage_internal,_sql: Expose archive capabilities feature set Kim Alvefur <zash@zash.se> parents: 9883 diff changeset	53 total = true;
9751c17f5281 mod_storage_internal,_sql: Expose archive capabilities feature set Kim Alvefur <zash@zash.se> parents: 9883 diff changeset	54 quota = archive_item_limit;
9751c17f5281 mod_storage_internal,_sql: Expose archive capabilities feature set Kim Alvefur <zash@zash.se> parents: 9883 diff changeset	55 truncate = true;
11275 b8fada57faf0 mod_storage_internal: Add support for full ID range query Kim Alvefur <zash@zash.se> parents: 10926 diff changeset	56 full_id_range = true;
11278 c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	57 ids = true;
9884 9751c17f5281 mod_storage_internal,_sql: Expose archive capabilities feature set Kim Alvefur <zash@zash.se> parents: 9883 diff changeset	58 };
9751c17f5281 mod_storage_internal,_sql: Expose archive capabilities feature set Kim Alvefur <zash@zash.se> parents: 9883 diff changeset	59
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	60 function archive:append(username, key, value, when, with)
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	61 when = when or now();
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	62 if not st.is_stanza(value) then
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	63 return nil, "unsupported-datatype";
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	64 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	65 value = st.preserialize(st.clone(value));
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	66 value.when = when;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	67 value.with = with;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	68 value.attr.stamp = datetime.datetime(when);
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	69
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	70 local cache_key = jid_join(username, host, self.store);
64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	71 local item_count = archive_item_count_cache:get(cache_key);
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	72
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	73 if key then
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	74 local items, err = datamanager.list_load(username, host, self.store);
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	75 if not items and err then return items, err; end
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	76
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	77 -- Check the quota
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	78 item_count = items and #items or 0;
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	79 archive_item_count_cache:set(cache_key, item_count);
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	80 if item_count >= archive_item_limit then
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	81 module:log("debug", "%s reached or over quota, not adding to store", username);
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	82 return nil, "quota-limit";
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	83 end
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	84
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	85 if items then
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	86 -- Filter out any item with the same key as the one being added
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	87 items = array(items);
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	88 items:filter(function (item)
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	89 return item.key ~= key;
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	90 end);
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	91
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	92 value.key = key;
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	93 items:push(value);
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	94 local ok, err = datamanager.list_store(username, host, self.store, items);
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	95 if not ok then return ok, err; end
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	96 archive_item_count_cache:set(cache_key, #items);
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	97 return key;
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	98 end
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	99 else
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	100 if not item_count then -- Item count not cached?
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	101 -- We need to load the list to get the number of items currently stored
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	102 local items, err = datamanager.list_load(username, host, self.store);
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	103 if not items and err then return items, err; end
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	104 item_count = items and #items or 0;
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	105 archive_item_count_cache:set(cache_key, item_count);
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	106 end
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	107 if item_count >= archive_item_limit then
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	108 module:log("debug", "%s reached or over quota, not adding to store", username);
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	109 return nil, "quota-limit";
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	110 end
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	111 key = id();
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	112 end
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	113
9895 6bd65bff03b5 mod_storage_internal: Include store name when reporting quota status Kim Alvefur <zash@zash.se> parents: 9888 diff changeset	114 module:log("debug", "%s has %d items out of %d limit in store %s", username, item_count, archive_item_limit, self.store);
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	115
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	116 value.key = key;
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	117
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	118 local ok, err = datamanager.list_append(username, host, self.store, value);
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	119 if not ok then return ok, err; end
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	120 archive_item_count_cache:set(cache_key, item_count+1);
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	121 return key;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	122 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	123
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	124 function archive:find(username, query)
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	125 local items, err = datamanager.list_load(username, host, self.store);
8173 3ff99d49082f mod_storage_internal: Return a noop iterator if archive is empty (fixes #920) Kim Alvefur <zash@zash.se> parents: 8143 diff changeset	126 if not items then
3ff99d49082f mod_storage_internal: Return a noop iterator if archive is empty (fixes #920) Kim Alvefur <zash@zash.se> parents: 8143 diff changeset	127 if err then
3ff99d49082f mod_storage_internal: Return a noop iterator if archive is empty (fixes #920) Kim Alvefur <zash@zash.se> parents: 8143 diff changeset	128 return items, err;
9997 7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	129 elseif query then
10024 4a0d990253a0 mod_storage_internal: Return appropriate error even with empty archive Kim Alvefur <zash@zash.se> parents: 10018 diff changeset	130 if query.before or query.after then
4a0d990253a0 mod_storage_internal: Return appropriate error even with empty archive Kim Alvefur <zash@zash.se> parents: 10018 diff changeset	131 return nil, "item-not-found";
4a0d990253a0 mod_storage_internal: Return appropriate error even with empty archive Kim Alvefur <zash@zash.se> parents: 10018 diff changeset	132 end
9997 7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	133 if query.total then
7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	134 return function () end, 0;
7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	135 end
8173 3ff99d49082f mod_storage_internal: Return a noop iterator if archive is empty (fixes #920) Kim Alvefur <zash@zash.se> parents: 8143 diff changeset	136 end
9997 7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	137 return function () end;
8173 3ff99d49082f mod_storage_internal: Return a noop iterator if archive is empty (fixes #920) Kim Alvefur <zash@zash.se> parents: 8143 diff changeset	138 end
9997 7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	139 local count = nil;
10926 c55bd98a54f8 mod_storage_internal, mod_storage_memory: Add support for query.before Matthew Wild <mwild1@gmail.com> parents: 10844 diff changeset	140 local i, last_key = 0;
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	141 if query then
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	142 items = array(items);
8089 4ba8cb75d925 mod_storage_internal: Support the 'key' archive query field Kim Alvefur <zash@zash.se> parents: 8023 diff changeset	143 if query.key then
4ba8cb75d925 mod_storage_internal: Support the 'key' archive query field Kim Alvefur <zash@zash.se> parents: 8023 diff changeset	144 items:filter(function (item)
4ba8cb75d925 mod_storage_internal: Support the 'key' archive query field Kim Alvefur <zash@zash.se> parents: 8023 diff changeset	145 return item.key == query.key;
4ba8cb75d925 mod_storage_internal: Support the 'key' archive query field Kim Alvefur <zash@zash.se> parents: 8023 diff changeset	146 end);
4ba8cb75d925 mod_storage_internal: Support the 'key' archive query field Kim Alvefur <zash@zash.se> parents: 8023 diff changeset	147 end
11278 c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	148 if query.ids then
c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	149 local ids = set.new(query.ids);
c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	150 items:filter(function (item)
c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	151 return ids:contains(item.key);
c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	152 end);
c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	153 end
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	154 if query.with then
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	155 items:filter(function (item)
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	156 return item.with == query.with;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	157 end);
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	158 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	159 if query.start then
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	160 items:filter(function (item)
10843 8fcd46ee9bf5 mod_storage_internal: Fix error in time limited queries on items without 'when' field, fixes #1557 Kim Alvefur <zash@zash.se> parents: 9105 diff changeset	161 local when = item.when or datetime.parse(item.attr.stamp);
8fcd46ee9bf5 mod_storage_internal: Fix error in time limited queries on items without 'when' field, fixes #1557 Kim Alvefur <zash@zash.se> parents: 9105 diff changeset	162 return when >= query.start;
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	163 end);
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	164 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	165 if query["end"] then
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	166 items:filter(function (item)
10843 8fcd46ee9bf5 mod_storage_internal: Fix error in time limited queries on items without 'when' field, fixes #1557 Kim Alvefur <zash@zash.se> parents: 9105 diff changeset	167 local when = item.when or datetime.parse(item.attr.stamp);
8fcd46ee9bf5 mod_storage_internal: Fix error in time limited queries on items without 'when' field, fixes #1557 Kim Alvefur <zash@zash.se> parents: 9105 diff changeset	168 return when <= query["end"];
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	169 end);
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	170 end
9997 7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	171 if query.total then
7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	172 count = #items;
7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	173 end
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	174 if query.reverse then
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	175 items:reverse();
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	176 if query.before then
10018 7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	177 local found = false;
9997 7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	178 for j = 1, #items do
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	179 if (items[j].key or tostring(j)) == query.before then
10018 7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	180 found = true;
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	181 i = j;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	182 break;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	183 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	184 end
10018 7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	185 if not found then
7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	186 return nil, "item-not-found";
7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	187 end
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	188 end
11275 b8fada57faf0 mod_storage_internal: Add support for full ID range query Kim Alvefur <zash@zash.se> parents: 10926 diff changeset	189 last_key = query.after;
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	190 elseif query.after then
10018 7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	191 local found = false;
9997 7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	192 for j = 1, #items do
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	193 if (items[j].key or tostring(j)) == query.after then
10018 7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	194 found = true;
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	195 i = j;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	196 break;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	197 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	198 end
10018 7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	199 if not found then
7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	200 return nil, "item-not-found";
7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	201 end
11275 b8fada57faf0 mod_storage_internal: Add support for full ID range query Kim Alvefur <zash@zash.se> parents: 10926 diff changeset	202 last_key = query.before;
b8fada57faf0 mod_storage_internal: Add support for full ID range query Kim Alvefur <zash@zash.se> parents: 10926 diff changeset	203 elseif query.before then
b8fada57faf0 mod_storage_internal: Add support for full ID range query Kim Alvefur <zash@zash.se> parents: 10926 diff changeset	204 last_key = query.before;
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	205 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	206 if query.limit and #items - i > query.limit then
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	207 items[i+query.limit+1] = nil;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	208 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	209 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	210 return function ()
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	211 i = i + 1;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	212 local item = items[i];
10926 c55bd98a54f8 mod_storage_internal, mod_storage_memory: Add support for query.before Matthew Wild <mwild1@gmail.com> parents: 10844 diff changeset	213 if not item or (last_key and item.key == last_key) then
c55bd98a54f8 mod_storage_internal, mod_storage_memory: Add support for query.before Matthew Wild <mwild1@gmail.com> parents: 10844 diff changeset	214 return;
c55bd98a54f8 mod_storage_internal, mod_storage_memory: Add support for query.before Matthew Wild <mwild1@gmail.com> parents: 10844 diff changeset	215 end
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	216 local key = item.key or tostring(i);
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	217 local when = item.when or datetime.parse(item.attr.stamp);
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	218 local with = item.with;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	219 item.key, item.when, item.with = nil, nil, nil;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	220 item.attr.stamp = nil;
11765 1cac469b18d0 mod_storage_internal: Stop storing XEP-0091 timestamp Kim Alvefur <zash@zash.se> parents: 11278 diff changeset	221 -- COMPAT Stored data may still contain legacy XEP-0091 timestamp
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	222 item.attr.stamp_legacy = nil;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	223 item = st.deserialize(item);
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	224 return key, item, when, with;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	225 end, count;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	226 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	227
10837 f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	228 function archive:get(username, wanted_key)
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	229 local iter, err = self:find(username, { key = wanted_key })
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	230 if not iter then return iter, err; end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	231 for key, stanza, when, with in iter do
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	232 if key == wanted_key then
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	233 return stanza, when, with;
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	234 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	235 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	236 return nil, "item-not-found";
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	237 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	238
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	239 function archive:set(username, key, new_value, new_when, new_with)
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	240 local items, err = datamanager.list_load(username, host, self.store);
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	241 if not items then
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	242 if err then
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	243 return items, err;
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	244 else
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	245 return nil, "item-not-found";
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	246 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	247 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	248
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	249 for i = 1, #items do
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	250 local old_item = items[i];
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	251 if old_item.key == key then
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	252 local item = st.preserialize(st.clone(new_value));
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	253
10842 5a6ba2f38e2b mod_storage_internal: Fix keeping old timestamp in archive map API Kim Alvefur <zash@zash.se> parents: 10837 diff changeset	254 local when = new_when or old_item.when or datetime.parse(old_item.attr.stamp);
10837 f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	255 item.key = key;
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	256 item.when = when;
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	257 item.with = new_with or old_item.with;
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	258 item.attr.stamp = datetime.datetime(when);
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	259 items[i] = item;
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	260 return datamanager.list_store(username, host, self.store, items);
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	261 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	262 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	263
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	264 return nil, "item-not-found";
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	265 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	266
8023 5a9d491cc714 mod_storage_internal: Add the dates method Kim Alvefur <zash@zash.se> parents: 8022 diff changeset	267 function archive:dates(username)
5a9d491cc714 mod_storage_internal: Add the dates method Kim Alvefur <zash@zash.se> parents: 8022 diff changeset	268 local items, err = datamanager.list_load(username, host, self.store);
5a9d491cc714 mod_storage_internal: Add the dates method Kim Alvefur <zash@zash.se> parents: 8022 diff changeset	269 if not items then return items, err; end
5a9d491cc714 mod_storage_internal: Add the dates method Kim Alvefur <zash@zash.se> parents: 8022 diff changeset	270 return array(items):pluck("when"):map(datetime.date):unique();
5a9d491cc714 mod_storage_internal: Add the dates method Kim Alvefur <zash@zash.se> parents: 8022 diff changeset	271 end
5a9d491cc714 mod_storage_internal: Add the dates method Kim Alvefur <zash@zash.se> parents: 8022 diff changeset	272
9903 2c5546cc5c70 mod_storage_internal: Implement a summary API returning message counts per contact Kim Alvefur <zash@zash.se> parents: 9895 diff changeset	273 function archive:summary(username, query)
2c5546cc5c70 mod_storage_internal: Implement a summary API returning message counts per contact Kim Alvefur <zash@zash.se> parents: 9895 diff changeset	274 local iter, err = self:find(username, query)
2c5546cc5c70 mod_storage_internal: Implement a summary API returning message counts per contact Kim Alvefur <zash@zash.se> parents: 9895 diff changeset	275 if not iter then return iter, err; end
10220 1e2b444acb72 mod_storage_: Tweak :summary API to allow future expansion with more fields Kim Alvefur <zash@zash.se>* parents: 10024 diff changeset	276 local counts = {};
10222 51f145094648 mod_storage_: Also include timestmap of first message in :summary API Kim Alvefur <zash@zash.se>* parents: 10221 diff changeset	277 local earliest = {};
10221 068692cb9e78 mod_storage_: Include timestamp of latest message in :summary API Kim Alvefur <zash@zash.se>* parents: 10220 diff changeset	278 local latest = {};
10223 d185c4961ee0 mod_storage_internal: Include last text message Kim Alvefur <zash@zash.se> parents: 10222 diff changeset	279 local body = {};
d185c4961ee0 mod_storage_internal: Include last text message Kim Alvefur <zash@zash.se> parents: 10222 diff changeset	280 for _, stanza, when, with in iter do
10220 1e2b444acb72 mod_storage_: Tweak :summary API to allow future expansion with more fields Kim Alvefur <zash@zash.se>* parents: 10024 diff changeset	281 counts[with] = (counts[with] or 0) + 1;
10222 51f145094648 mod_storage_: Also include timestmap of first message in :summary API Kim Alvefur <zash@zash.se>* parents: 10221 diff changeset	282 if earliest[with] == nil then
51f145094648 mod_storage_: Also include timestmap of first message in :summary API Kim Alvefur <zash@zash.se>* parents: 10221 diff changeset	283 earliest[with] = when;
51f145094648 mod_storage_: Also include timestmap of first message in :summary API Kim Alvefur <zash@zash.se>* parents: 10221 diff changeset	284 end
10221 068692cb9e78 mod_storage_: Include timestamp of latest message in :summary API Kim Alvefur <zash@zash.se>* parents: 10220 diff changeset	285 latest[with] = when;
10223 d185c4961ee0 mod_storage_internal: Include last text message Kim Alvefur <zash@zash.se> parents: 10222 diff changeset	286 body[with] = stanza:get_child_text("body") or body[with];
9903 2c5546cc5c70 mod_storage_internal: Implement a summary API returning message counts per contact Kim Alvefur <zash@zash.se> parents: 9895 diff changeset	287 end
10220 1e2b444acb72 mod_storage_: Tweak :summary API to allow future expansion with more fields Kim Alvefur <zash@zash.se>* parents: 10024 diff changeset	288 return {
1e2b444acb72 mod_storage_: Tweak :summary API to allow future expansion with more fields Kim Alvefur <zash@zash.se>* parents: 10024 diff changeset	289 counts = counts;
10222 51f145094648 mod_storage_: Also include timestmap of first message in :summary API Kim Alvefur <zash@zash.se>* parents: 10221 diff changeset	290 earliest = earliest;
10221 068692cb9e78 mod_storage_: Include timestamp of latest message in :summary API Kim Alvefur <zash@zash.se>* parents: 10220 diff changeset	291 latest = latest;
10223 d185c4961ee0 mod_storage_internal: Include last text message Kim Alvefur <zash@zash.se> parents: 10222 diff changeset	292 body = body;
10220 1e2b444acb72 mod_storage_: Tweak :summary API to allow future expansion with more fields Kim Alvefur <zash@zash.se>* parents: 10024 diff changeset	293 };
9903 2c5546cc5c70 mod_storage_internal: Implement a summary API returning message counts per contact Kim Alvefur <zash@zash.se> parents: 9895 diff changeset	294 end
2c5546cc5c70 mod_storage_internal: Implement a summary API returning message counts per contact Kim Alvefur <zash@zash.se> parents: 9895 diff changeset	295
10001 55f5588d71c6 mod_storage_internal: Add support for iterating over users in archive stores Kim Alvefur <zash@zash.se> parents: 9997 diff changeset	296 function archive:users()
55f5588d71c6 mod_storage_internal: Add support for iterating over users in archive stores Kim Alvefur <zash@zash.se> parents: 9997 diff changeset	297 return datamanager.users(host, self.store, "list");
55f5588d71c6 mod_storage_internal: Add support for iterating over users in archive stores Kim Alvefur <zash@zash.se> parents: 9997 diff changeset	298 end
55f5588d71c6 mod_storage_internal: Add support for iterating over users in archive stores Kim Alvefur <zash@zash.se> parents: 9997 diff changeset	299
8022 05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	300 function archive:delete(username, query)
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	301 local cache_key = jid_join(username, host, self.store);
8022 05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	302 if not query or next(query) == nil then
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	303 archive_item_count_cache:set(cache_key, nil);
8022 05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	304 return datamanager.list_store(username, host, self.store, nil);
05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	305 end
05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	306 local items, err = datamanager.list_load(username, host, self.store);
8143 c4c159953c72 mod_storage_internal: Handle case of empty item store when deleting (fixes #910) Kim Alvefur <zash@zash.se> parents: 8142 diff changeset	307 if not items then
c4c159953c72 mod_storage_internal: Handle case of empty item store when deleting (fixes #910) Kim Alvefur <zash@zash.se> parents: 8142 diff changeset	308 if err then
c4c159953c72 mod_storage_internal: Handle case of empty item store when deleting (fixes #910) Kim Alvefur <zash@zash.se> parents: 8142 diff changeset	309 return items, err;
c4c159953c72 mod_storage_internal: Handle case of empty item store when deleting (fixes #910) Kim Alvefur <zash@zash.se> parents: 8142 diff changeset	310 end
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	311 archive_item_count_cache:set(cache_key, 0);
8143 c4c159953c72 mod_storage_internal: Handle case of empty item store when deleting (fixes #910) Kim Alvefur <zash@zash.se> parents: 8142 diff changeset	312 -- Store is empty
c4c159953c72 mod_storage_internal: Handle case of empty item store when deleting (fixes #910) Kim Alvefur <zash@zash.se> parents: 8142 diff changeset	313 return 0;
c4c159953c72 mod_storage_internal: Handle case of empty item store when deleting (fixes #910) Kim Alvefur <zash@zash.se> parents: 8142 diff changeset	314 end
8022 05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	315 items = array(items);
8142 13e075549cb3 mod_storage_internal: Correctly calculate number of deleted items (fixes #912) Kim Alvefur <zash@zash.se> parents: 8089 diff changeset	316 local count_before = #items;
8310 1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	317 if query then
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	318 if query.key then
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	319 items:filter(function (item)
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	320 return item.key ~= query.key;
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	321 end);
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	322 end
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	323 if query.with then
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	324 items:filter(function (item)
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	325 return item.with ~= query.with;
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	326 end);
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	327 end
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	328 if query.start then
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	329 items:filter(function (item)
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	330 return item.when < query.start;
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	331 end);
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	332 end
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	333 if query["end"] then
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	334 items:filter(function (item)
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	335 return item.when > query["end"];
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	336 end);
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	337 end
9105 e735c9865f42 mod_storage_internal: Only apply truncate if there are more items than requested Kim Alvefur <zash@zash.se> parents: 8402 diff changeset	338 if query.truncate and #items > query.truncate then
8393 b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	339 if query.reverse then
b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	340 -- Before: { 1, 2, 3, 4, 5, }
b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	341 -- After: { 1, 2, 3 }
8394 4892c22403d5 mod_storage_internal: Optimize truncation Kim Alvefur <zash@zash.se> parents: 8393 diff changeset	342 for i = #items, query.truncate + 1, -1 do
4892c22403d5 mod_storage_internal: Optimize truncation Kim Alvefur <zash@zash.se> parents: 8393 diff changeset	343 items[i] = nil;
8393 b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	344 end
b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	345 else
b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	346 -- Before: { 1, 2, 3, 4, 5, }
b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	347 -- After: { 3, 4, 5 }
8394 4892c22403d5 mod_storage_internal: Optimize truncation Kim Alvefur <zash@zash.se> parents: 8393 diff changeset	348 local offset = #items - query.truncate;
4892c22403d5 mod_storage_internal: Optimize truncation Kim Alvefur <zash@zash.se> parents: 8393 diff changeset	349 for i = 1, #items do
4892c22403d5 mod_storage_internal: Optimize truncation Kim Alvefur <zash@zash.se> parents: 8393 diff changeset	350 items[i] = items[i+offset];
8393 b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	351 end
b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	352 end
b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	353 end
8310 1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	354 end
8142 13e075549cb3 mod_storage_internal: Correctly calculate number of deleted items (fixes #912) Kim Alvefur <zash@zash.se> parents: 8089 diff changeset	355 local count = count_before - #items;
8402 469afa02947b mod_storage_internal: Skip write if no items matched a deletion query Kim Alvefur <zash@zash.se> parents: 8394 diff changeset	356 if count == 0 then
469afa02947b mod_storage_internal: Skip write if no items matched a deletion query Kim Alvefur <zash@zash.se> parents: 8394 diff changeset	357 return 0; -- No changes, skip write
469afa02947b mod_storage_internal: Skip write if no items matched a deletion query Kim Alvefur <zash@zash.se> parents: 8394 diff changeset	358 end
8022 05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	359 local ok, err = datamanager.list_store(username, host, self.store, items);
05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	360 if not ok then return ok, err; end
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	361 archive_item_count_cache:set(cache_key, #items);
8022 05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	362 return count;
05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	363 end
05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	364
5121 b5a5643f8572 core.storagemanager, mod_storage_: "data-driver" -> "storage-provider", to allow using module:provides(). Waqas Hussain <waqas20@gmail.com>* parents: 5039 diff changeset	365 module:provides("storage", driver);

prosody

ba3344926e18

plugins/mod_storage_internal.lua

Software / code / prosody

Annotate

plugins/mod_storage_internal.lua @ 11906:ba3344926e18

Software `/` code `/` prosody