Software `/` code `/` prosody

Annotate

plugins/mod_storage_internal.lua @ 11961:542a9a503073

util.async: Add sleep() method with configurable scheduling backend No scheduler set by default, so it will error (we plan to initialize it in util.startup). We wanted to avoid a hard dependency on util.timer (which in turn depends on network backends, etc.), and we didn't add timer.sleep() because we didn't want to add a hard dependency on util.async for things that don't need it.

author	Matthew Wild <mwild1@gmail.com>
date	Mon, 29 Nov 2021 14:11:24 +0000
parent	11765:1cac469b18d0
child	12977:74b9e05af71e

rev	line source
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	1 local cache = require "util.cache";
4085 7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	2 local datamanager = require "core.storagemanager".olddm;
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	3 local array = require "util.array";
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	4 local datetime = require "util.datetime";
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	5 local st = require "util.stanza";
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	6 local now = require "util.time".now;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	7 local id = require "util.id".medium;
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	8 local jid_join = require "util.jid".join;
11278 c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	9 local set = require "util.set";
4085 7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	10
7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	11 local host = module.host;
7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	12
9888 5669bb11ac7b mod_storage_internal: Increase default quota to 10 000 Kim Alvefur <zash@zash.se> parents: 9885 diff changeset	13 local archive_item_limit = module:get_option_number("storage_archive_item_limit", 10000);
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	14 local archive_item_count_cache = cache.new(module:get_option("storage_archive_item_limit_cache_size", 1000));
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	15
5122 b41c33dc7c36 mod_storage_: Don't explicitly set driver name, to ease copying/renaming modules. Waqas Hussain <waqas20@gmail.com>* parents: 5121 diff changeset	16 local driver = {};
4085 7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	17
5153 688aeac0012a mod_storage_internal, datamanager: Add support for iterating over users with data in a store Kim Alvefur <zash@zash.se> parents: 5133 diff changeset	18 function driver:open(store, typ)
8018 9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	19 local mt = self[typ or "keyval"]
9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	20 if not mt then
6283 7cf6d3a2c855 mod_storage_{none,internal,sql}: Return error for unsupported (everything but keyval) store types Kim Alvefur <zash@zash.se> parents: 5153 diff changeset	21 return nil, "unsupported-store";
7cf6d3a2c855 mod_storage_{none,internal,sql}: Return error for unsupported (everything but keyval) store types Kim Alvefur <zash@zash.se> parents: 5153 diff changeset	22 end
8018 9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	23 return setmetatable({ store = store, type = typ }, mt);
4085 7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	24 end
8018 9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	25
8020 342ce07836de mod_storage_internal: Ignore unused 'self' argument [luacheck] Kim Alvefur <zash@zash.se> parents: 8019 diff changeset	26 function driver:stores(username) -- luacheck: ignore 212/self
8019 925098aad268 mod_storage_internal: Reorder methods Kim Alvefur <zash@zash.se> parents: 8018 diff changeset	27 return datamanager.stores(username, host);
925098aad268 mod_storage_internal: Reorder methods Kim Alvefur <zash@zash.se> parents: 8018 diff changeset	28 end
925098aad268 mod_storage_internal: Reorder methods Kim Alvefur <zash@zash.se> parents: 8018 diff changeset	29
8020 342ce07836de mod_storage_internal: Ignore unused 'self' argument [luacheck] Kim Alvefur <zash@zash.se> parents: 8019 diff changeset	30 function driver:purge(user) -- luacheck: ignore 212/self
8019 925098aad268 mod_storage_internal: Reorder methods Kim Alvefur <zash@zash.se> parents: 8018 diff changeset	31 return datamanager.purge(user, host);
925098aad268 mod_storage_internal: Reorder methods Kim Alvefur <zash@zash.se> parents: 8018 diff changeset	32 end
925098aad268 mod_storage_internal: Reorder methods Kim Alvefur <zash@zash.se> parents: 8018 diff changeset	33
8018 9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	34 local keyval = { };
9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	35 driver.keyval = { __index = keyval };
9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	36
9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	37 function keyval:get(user)
4085 7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	38 return datamanager.load(user, host, self.store);
7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	39 end
7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	40
8018 9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	41 function keyval:set(user, data)
4085 7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	42 return datamanager.store(user, host, self.store, data);
7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	43 end
7699cef04740 storagemanager, mod_storage_internal: Split out default driver to mod_storage_internal, and greatly simplify storagemanager's error handling and fallback code Matthew Wild <mwild1@gmail.com> parents: diff changeset	44
8018 9545d0a9401f mod_storage_internal: Separate driver from keyval implementation Kim Alvefur <zash@zash.se> parents: 6283 diff changeset	45 function keyval:users()
5153 688aeac0012a mod_storage_internal, datamanager: Add support for iterating over users with data in a store Kim Alvefur <zash@zash.se> parents: 5133 diff changeset	46 return datamanager.users(host, self.store, self.type);
688aeac0012a mod_storage_internal, datamanager: Add support for iterating over users with data in a store Kim Alvefur <zash@zash.se> parents: 5133 diff changeset	47 end
688aeac0012a mod_storage_internal, datamanager: Add support for iterating over users with data in a store Kim Alvefur <zash@zash.se> parents: 5133 diff changeset	48
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	49 local archive = {};
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	50 driver.archive = { __index = archive };
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	51
9884 9751c17f5281 mod_storage_internal,_sql: Expose archive capabilities feature set Kim Alvefur <zash@zash.se> parents: 9883 diff changeset	52 archive.caps = {
9751c17f5281 mod_storage_internal,_sql: Expose archive capabilities feature set Kim Alvefur <zash@zash.se> parents: 9883 diff changeset	53 total = true;
9751c17f5281 mod_storage_internal,_sql: Expose archive capabilities feature set Kim Alvefur <zash@zash.se> parents: 9883 diff changeset	54 quota = archive_item_limit;
9751c17f5281 mod_storage_internal,_sql: Expose archive capabilities feature set Kim Alvefur <zash@zash.se> parents: 9883 diff changeset	55 truncate = true;
11275 b8fada57faf0 mod_storage_internal: Add support for full ID range query Kim Alvefur <zash@zash.se> parents: 10926 diff changeset	56 full_id_range = true;
11278 c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	57 ids = true;
9884 9751c17f5281 mod_storage_internal,_sql: Expose archive capabilities feature set Kim Alvefur <zash@zash.se> parents: 9883 diff changeset	58 };
9751c17f5281 mod_storage_internal,_sql: Expose archive capabilities feature set Kim Alvefur <zash@zash.se> parents: 9883 diff changeset	59
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	60 function archive:append(username, key, value, when, with)
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	61 when = when or now();
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	62 if not st.is_stanza(value) then
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	63 return nil, "unsupported-datatype";
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	64 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	65 value = st.preserialize(st.clone(value));
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	66 value.when = when;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	67 value.with = with;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	68 value.attr.stamp = datetime.datetime(when);
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	69
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	70 local cache_key = jid_join(username, host, self.store);
64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	71 local item_count = archive_item_count_cache:get(cache_key);
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	72
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	73 if key then
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	74 local items, err = datamanager.list_load(username, host, self.store);
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	75 if not items and err then return items, err; end
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	76
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	77 -- Check the quota
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	78 item_count = items and #items or 0;
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	79 archive_item_count_cache:set(cache_key, item_count);
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	80 if item_count >= archive_item_limit then
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	81 module:log("debug", "%s reached or over quota, not adding to store", username);
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	82 return nil, "quota-limit";
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	83 end
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	84
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	85 if items then
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	86 -- Filter out any item with the same key as the one being added
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	87 items = array(items);
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	88 items:filter(function (item)
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	89 return item.key ~= key;
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	90 end);
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	91
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	92 value.key = key;
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	93 items:push(value);
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	94 local ok, err = datamanager.list_store(username, host, self.store, items);
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	95 if not ok then return ok, err; end
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	96 archive_item_count_cache:set(cache_key, #items);
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	97 return key;
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	98 end
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	99 else
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	100 if not item_count then -- Item count not cached?
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	101 -- We need to load the list to get the number of items currently stored
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	102 local items, err = datamanager.list_load(username, host, self.store);
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	103 if not items and err then return items, err; end
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	104 item_count = items and #items or 0;
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	105 archive_item_count_cache:set(cache_key, item_count);
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	106 end
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	107 if item_count >= archive_item_limit then
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	108 module:log("debug", "%s reached or over quota, not adding to store", username);
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	109 return nil, "quota-limit";
f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	110 end
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	111 key = id();
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	112 end
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	113
9895 6bd65bff03b5 mod_storage_internal: Include store name when reporting quota status Kim Alvefur <zash@zash.se> parents: 9888 diff changeset	114 module:log("debug", "%s has %d items out of %d limit in store %s", username, item_count, archive_item_limit, self.store);
9883 f76bd399267c mod_storage_internal,_sql: Add limit to number of items in an archive store (fixes #733) Matthew Wild <mwild1@gmail.com> parents: 9105 diff changeset	115
8309 5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	116 value.key = key;
5281c479955a mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql) Kim Alvefur <zash@zash.se> parents: 8173 diff changeset	117
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	118 local ok, err = datamanager.list_append(username, host, self.store, value);
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	119 if not ok then return ok, err; end
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	120 archive_item_count_cache:set(cache_key, item_count+1);
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	121 return key;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	122 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	123
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	124 function archive:find(username, query)
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	125 local items, err = datamanager.list_load(username, host, self.store);
8173 3ff99d49082f mod_storage_internal: Return a noop iterator if archive is empty (fixes #920) Kim Alvefur <zash@zash.se> parents: 8143 diff changeset	126 if not items then
3ff99d49082f mod_storage_internal: Return a noop iterator if archive is empty (fixes #920) Kim Alvefur <zash@zash.se> parents: 8143 diff changeset	127 if err then
3ff99d49082f mod_storage_internal: Return a noop iterator if archive is empty (fixes #920) Kim Alvefur <zash@zash.se> parents: 8143 diff changeset	128 return items, err;
9997 7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	129 elseif query then
10024 4a0d990253a0 mod_storage_internal: Return appropriate error even with empty archive Kim Alvefur <zash@zash.se> parents: 10018 diff changeset	130 if query.before or query.after then
4a0d990253a0 mod_storage_internal: Return appropriate error even with empty archive Kim Alvefur <zash@zash.se> parents: 10018 diff changeset	131 return nil, "item-not-found";
4a0d990253a0 mod_storage_internal: Return appropriate error even with empty archive Kim Alvefur <zash@zash.se> parents: 10018 diff changeset	132 end
9997 7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	133 if query.total then
7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	134 return function () end, 0;
7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	135 end
8173 3ff99d49082f mod_storage_internal: Return a noop iterator if archive is empty (fixes #920) Kim Alvefur <zash@zash.se> parents: 8143 diff changeset	136 end
9997 7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	137 return function () end;
8173 3ff99d49082f mod_storage_internal: Return a noop iterator if archive is empty (fixes #920) Kim Alvefur <zash@zash.se> parents: 8143 diff changeset	138 end
9997 7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	139 local count = nil;
10926 c55bd98a54f8 mod_storage_internal, mod_storage_memory: Add support for query.before Matthew Wild <mwild1@gmail.com> parents: 10844 diff changeset	140 local i, last_key = 0;
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	141 if query then
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	142 items = array(items);
8089 4ba8cb75d925 mod_storage_internal: Support the 'key' archive query field Kim Alvefur <zash@zash.se> parents: 8023 diff changeset	143 if query.key then
4ba8cb75d925 mod_storage_internal: Support the 'key' archive query field Kim Alvefur <zash@zash.se> parents: 8023 diff changeset	144 items:filter(function (item)
4ba8cb75d925 mod_storage_internal: Support the 'key' archive query field Kim Alvefur <zash@zash.se> parents: 8023 diff changeset	145 return item.key == query.key;
4ba8cb75d925 mod_storage_internal: Support the 'key' archive query field Kim Alvefur <zash@zash.se> parents: 8023 diff changeset	146 end);
4ba8cb75d925 mod_storage_internal: Support the 'key' archive query field Kim Alvefur <zash@zash.se> parents: 8023 diff changeset	147 end
11278 c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	148 if query.ids then
c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	149 local ids = set.new(query.ids);
c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	150 items:filter(function (item)
c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	151 return ids:contains(item.key);
c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	152 end);
c3907f05bed4 mod_storage_internal: Support query for set of IDs Kim Alvefur <zash@zash.se> parents: 11275 diff changeset	153 end
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	154 if query.with then
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	155 items:filter(function (item)
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	156 return item.with == query.with;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	157 end);
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	158 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	159 if query.start then
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	160 items:filter(function (item)
10843 8fcd46ee9bf5 mod_storage_internal: Fix error in time limited queries on items without 'when' field, fixes #1557 Kim Alvefur <zash@zash.se> parents: 9105 diff changeset	161 local when = item.when or datetime.parse(item.attr.stamp);
8fcd46ee9bf5 mod_storage_internal: Fix error in time limited queries on items without 'when' field, fixes #1557 Kim Alvefur <zash@zash.se> parents: 9105 diff changeset	162 return when >= query.start;
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	163 end);
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	164 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	165 if query["end"] then
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	166 items:filter(function (item)
10843 8fcd46ee9bf5 mod_storage_internal: Fix error in time limited queries on items without 'when' field, fixes #1557 Kim Alvefur <zash@zash.se> parents: 9105 diff changeset	167 local when = item.when or datetime.parse(item.attr.stamp);
8fcd46ee9bf5 mod_storage_internal: Fix error in time limited queries on items without 'when' field, fixes #1557 Kim Alvefur <zash@zash.se> parents: 9105 diff changeset	168 return when <= query["end"];
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	169 end);
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	170 end
9997 7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	171 if query.total then
7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	172 count = #items;
7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	173 end
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	174 if query.reverse then
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	175 items:reverse();
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	176 if query.before then
10018 7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	177 local found = false;
9997 7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	178 for j = 1, #items do
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	179 if (items[j].key or tostring(j)) == query.before then
10018 7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	180 found = true;
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	181 i = j;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	182 break;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	183 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	184 end
10018 7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	185 if not found then
7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	186 return nil, "item-not-found";
7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	187 end
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	188 end
11275 b8fada57faf0 mod_storage_internal: Add support for full ID range query Kim Alvefur <zash@zash.se> parents: 10926 diff changeset	189 last_key = query.after;
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	190 elseif query.after then
10018 7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	191 local found = false;
9997 7c4631d7b6fb mod_storage_internal,memory: Only return total count if requested Kim Alvefur <zash@zash.se> parents: 9903 diff changeset	192 for j = 1, #items do
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	193 if (items[j].key or tostring(j)) == query.after then
10018 7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	194 found = true;
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	195 i = j;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	196 break;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	197 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	198 end
10018 7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	199 if not found then
7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	200 return nil, "item-not-found";
7408b9473729 mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325) Kim Alvefur <zash@zash.se> parents: 10001 diff changeset	201 end
11275 b8fada57faf0 mod_storage_internal: Add support for full ID range query Kim Alvefur <zash@zash.se> parents: 10926 diff changeset	202 last_key = query.before;
b8fada57faf0 mod_storage_internal: Add support for full ID range query Kim Alvefur <zash@zash.se> parents: 10926 diff changeset	203 elseif query.before then
b8fada57faf0 mod_storage_internal: Add support for full ID range query Kim Alvefur <zash@zash.se> parents: 10926 diff changeset	204 last_key = query.before;
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	205 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	206 if query.limit and #items - i > query.limit then
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	207 items[i+query.limit+1] = nil;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	208 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	209 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	210 return function ()
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	211 i = i + 1;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	212 local item = items[i];
10926 c55bd98a54f8 mod_storage_internal, mod_storage_memory: Add support for query.before Matthew Wild <mwild1@gmail.com> parents: 10844 diff changeset	213 if not item or (last_key and item.key == last_key) then
c55bd98a54f8 mod_storage_internal, mod_storage_memory: Add support for query.before Matthew Wild <mwild1@gmail.com> parents: 10844 diff changeset	214 return;
c55bd98a54f8 mod_storage_internal, mod_storage_memory: Add support for query.before Matthew Wild <mwild1@gmail.com> parents: 10844 diff changeset	215 end
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	216 local key = item.key or tostring(i);
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	217 local when = item.when or datetime.parse(item.attr.stamp);
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	218 local with = item.with;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	219 item.key, item.when, item.with = nil, nil, nil;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	220 item.attr.stamp = nil;
11765 1cac469b18d0 mod_storage_internal: Stop storing XEP-0091 timestamp Kim Alvefur <zash@zash.se> parents: 11278 diff changeset	221 -- COMPAT Stored data may still contain legacy XEP-0091 timestamp
8021 83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	222 item.attr.stamp_legacy = nil;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	223 item = st.deserialize(item);
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	224 return key, item, when, with;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	225 end, count;
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	226 end
83f18982bcfd mod_storage_internal: Add basic archive store implementation Kim Alvefur <zash@zash.se> parents: 8020 diff changeset	227
10837 f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	228 function archive:get(username, wanted_key)
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	229 local iter, err = self:find(username, { key = wanted_key })
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	230 if not iter then return iter, err; end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	231 for key, stanza, when, with in iter do
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	232 if key == wanted_key then
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	233 return stanza, when, with;
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	234 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	235 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	236 return nil, "item-not-found";
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	237 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	238
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	239 function archive:set(username, key, new_value, new_when, new_with)
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	240 local items, err = datamanager.list_load(username, host, self.store);
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	241 if not items then
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	242 if err then
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	243 return items, err;
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	244 else
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	245 return nil, "item-not-found";
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	246 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	247 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	248
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	249 for i = 1, #items do
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	250 local old_item = items[i];
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	251 if old_item.key == key then
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	252 local item = st.preserialize(st.clone(new_value));
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	253
10842 5a6ba2f38e2b mod_storage_internal: Fix keeping old timestamp in archive map API Kim Alvefur <zash@zash.se> parents: 10837 diff changeset	254 local when = new_when or old_item.when or datetime.parse(old_item.attr.stamp);
10837 f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	255 item.key = key;
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	256 item.when = when;
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	257 item.with = new_with or old_item.with;
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	258 item.attr.stamp = datetime.datetime(when);
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	259 items[i] = item;
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	260 return datamanager.list_store(username, host, self.store, items);
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	261 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	262 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	263
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	264 return nil, "item-not-found";
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	265 end
f23363380599 mod_storage_internal: Implement key-value API Kim Alvefur <zash@zash.se> parents: 10223 diff changeset	266
8023 5a9d491cc714 mod_storage_internal: Add the dates method Kim Alvefur <zash@zash.se> parents: 8022 diff changeset	267 function archive:dates(username)
5a9d491cc714 mod_storage_internal: Add the dates method Kim Alvefur <zash@zash.se> parents: 8022 diff changeset	268 local items, err = datamanager.list_load(username, host, self.store);
5a9d491cc714 mod_storage_internal: Add the dates method Kim Alvefur <zash@zash.se> parents: 8022 diff changeset	269 if not items then return items, err; end
5a9d491cc714 mod_storage_internal: Add the dates method Kim Alvefur <zash@zash.se> parents: 8022 diff changeset	270 return array(items):pluck("when"):map(datetime.date):unique();
5a9d491cc714 mod_storage_internal: Add the dates method Kim Alvefur <zash@zash.se> parents: 8022 diff changeset	271 end
5a9d491cc714 mod_storage_internal: Add the dates method Kim Alvefur <zash@zash.se> parents: 8022 diff changeset	272
9903 2c5546cc5c70 mod_storage_internal: Implement a summary API returning message counts per contact Kim Alvefur <zash@zash.se> parents: 9895 diff changeset	273 function archive:summary(username, query)
2c5546cc5c70 mod_storage_internal: Implement a summary API returning message counts per contact Kim Alvefur <zash@zash.se> parents: 9895 diff changeset	274 local iter, err = self:find(username, query)
2c5546cc5c70 mod_storage_internal: Implement a summary API returning message counts per contact Kim Alvefur <zash@zash.se> parents: 9895 diff changeset	275 if not iter then return iter, err; end
10220 1e2b444acb72 mod_storage_: Tweak :summary API to allow future expansion with more fields Kim Alvefur <zash@zash.se>* parents: 10024 diff changeset	276 local counts = {};
10222 51f145094648 mod_storage_: Also include timestmap of first message in :summary API Kim Alvefur <zash@zash.se>* parents: 10221 diff changeset	277 local earliest = {};
10221 068692cb9e78 mod_storage_: Include timestamp of latest message in :summary API Kim Alvefur <zash@zash.se>* parents: 10220 diff changeset	278 local latest = {};
10223 d185c4961ee0 mod_storage_internal: Include last text message Kim Alvefur <zash@zash.se> parents: 10222 diff changeset	279 local body = {};
d185c4961ee0 mod_storage_internal: Include last text message Kim Alvefur <zash@zash.se> parents: 10222 diff changeset	280 for _, stanza, when, with in iter do
10220 1e2b444acb72 mod_storage_: Tweak :summary API to allow future expansion with more fields Kim Alvefur <zash@zash.se>* parents: 10024 diff changeset	281 counts[with] = (counts[with] or 0) + 1;
10222 51f145094648 mod_storage_: Also include timestmap of first message in :summary API Kim Alvefur <zash@zash.se>* parents: 10221 diff changeset	282 if earliest[with] == nil then
51f145094648 mod_storage_: Also include timestmap of first message in :summary API Kim Alvefur <zash@zash.se>* parents: 10221 diff changeset	283 earliest[with] = when;
51f145094648 mod_storage_: Also include timestmap of first message in :summary API Kim Alvefur <zash@zash.se>* parents: 10221 diff changeset	284 end
10221 068692cb9e78 mod_storage_: Include timestamp of latest message in :summary API Kim Alvefur <zash@zash.se>* parents: 10220 diff changeset	285 latest[with] = when;
10223 d185c4961ee0 mod_storage_internal: Include last text message Kim Alvefur <zash@zash.se> parents: 10222 diff changeset	286 body[with] = stanza:get_child_text("body") or body[with];
9903 2c5546cc5c70 mod_storage_internal: Implement a summary API returning message counts per contact Kim Alvefur <zash@zash.se> parents: 9895 diff changeset	287 end
10220 1e2b444acb72 mod_storage_: Tweak :summary API to allow future expansion with more fields Kim Alvefur <zash@zash.se>* parents: 10024 diff changeset	288 return {
1e2b444acb72 mod_storage_: Tweak :summary API to allow future expansion with more fields Kim Alvefur <zash@zash.se>* parents: 10024 diff changeset	289 counts = counts;
10222 51f145094648 mod_storage_: Also include timestmap of first message in :summary API Kim Alvefur <zash@zash.se>* parents: 10221 diff changeset	290 earliest = earliest;
10221 068692cb9e78 mod_storage_: Include timestamp of latest message in :summary API Kim Alvefur <zash@zash.se>* parents: 10220 diff changeset	291 latest = latest;
10223 d185c4961ee0 mod_storage_internal: Include last text message Kim Alvefur <zash@zash.se> parents: 10222 diff changeset	292 body = body;
10220 1e2b444acb72 mod_storage_: Tweak :summary API to allow future expansion with more fields Kim Alvefur <zash@zash.se>* parents: 10024 diff changeset	293 };
9903 2c5546cc5c70 mod_storage_internal: Implement a summary API returning message counts per contact Kim Alvefur <zash@zash.se> parents: 9895 diff changeset	294 end
2c5546cc5c70 mod_storage_internal: Implement a summary API returning message counts per contact Kim Alvefur <zash@zash.se> parents: 9895 diff changeset	295
10001 55f5588d71c6 mod_storage_internal: Add support for iterating over users in archive stores Kim Alvefur <zash@zash.se> parents: 9997 diff changeset	296 function archive:users()
55f5588d71c6 mod_storage_internal: Add support for iterating over users in archive stores Kim Alvefur <zash@zash.se> parents: 9997 diff changeset	297 return datamanager.users(host, self.store, "list");
55f5588d71c6 mod_storage_internal: Add support for iterating over users in archive stores Kim Alvefur <zash@zash.se> parents: 9997 diff changeset	298 end
55f5588d71c6 mod_storage_internal: Add support for iterating over users in archive stores Kim Alvefur <zash@zash.se> parents: 9997 diff changeset	299
8022 05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	300 function archive:delete(username, query)
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	301 local cache_key = jid_join(username, host, self.store);
8022 05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	302 if not query or next(query) == nil then
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	303 archive_item_count_cache:set(cache_key, nil);
8022 05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	304 return datamanager.list_store(username, host, self.store, nil);
05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	305 end
05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	306 local items, err = datamanager.list_load(username, host, self.store);
8143 c4c159953c72 mod_storage_internal: Handle case of empty item store when deleting (fixes #910) Kim Alvefur <zash@zash.se> parents: 8142 diff changeset	307 if not items then
c4c159953c72 mod_storage_internal: Handle case of empty item store when deleting (fixes #910) Kim Alvefur <zash@zash.se> parents: 8142 diff changeset	308 if err then
c4c159953c72 mod_storage_internal: Handle case of empty item store when deleting (fixes #910) Kim Alvefur <zash@zash.se> parents: 8142 diff changeset	309 return items, err;
c4c159953c72 mod_storage_internal: Handle case of empty item store when deleting (fixes #910) Kim Alvefur <zash@zash.se> parents: 8142 diff changeset	310 end
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	311 archive_item_count_cache:set(cache_key, 0);
8143 c4c159953c72 mod_storage_internal: Handle case of empty item store when deleting (fixes #910) Kim Alvefur <zash@zash.se> parents: 8142 diff changeset	312 -- Store is empty
c4c159953c72 mod_storage_internal: Handle case of empty item store when deleting (fixes #910) Kim Alvefur <zash@zash.se> parents: 8142 diff changeset	313 return 0;
c4c159953c72 mod_storage_internal: Handle case of empty item store when deleting (fixes #910) Kim Alvefur <zash@zash.se> parents: 8142 diff changeset	314 end
8022 05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	315 items = array(items);
8142 13e075549cb3 mod_storage_internal: Correctly calculate number of deleted items (fixes #912) Kim Alvefur <zash@zash.se> parents: 8089 diff changeset	316 local count_before = #items;
8310 1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	317 if query then
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	318 if query.key then
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	319 items:filter(function (item)
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	320 return item.key ~= query.key;
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	321 end);
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	322 end
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	323 if query.with then
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	324 items:filter(function (item)
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	325 return item.with ~= query.with;
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	326 end);
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	327 end
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	328 if query.start then
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	329 items:filter(function (item)
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	330 return item.when < query.start;
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	331 end);
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	332 end
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	333 if query["end"] then
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	334 items:filter(function (item)
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	335 return item.when > query["end"];
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	336 end);
1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	337 end
9105 e735c9865f42 mod_storage_internal: Only apply truncate if there are more items than requested Kim Alvefur <zash@zash.se> parents: 8402 diff changeset	338 if query.truncate and #items > query.truncate then
8393 b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	339 if query.reverse then
b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	340 -- Before: { 1, 2, 3, 4, 5, }
b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	341 -- After: { 1, 2, 3 }
8394 4892c22403d5 mod_storage_internal: Optimize truncation Kim Alvefur <zash@zash.se> parents: 8393 diff changeset	342 for i = #items, query.truncate + 1, -1 do
4892c22403d5 mod_storage_internal: Optimize truncation Kim Alvefur <zash@zash.se> parents: 8393 diff changeset	343 items[i] = nil;
8393 b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	344 end
b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	345 else
b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	346 -- Before: { 1, 2, 3, 4, 5, }
b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	347 -- After: { 3, 4, 5 }
8394 4892c22403d5 mod_storage_internal: Optimize truncation Kim Alvefur <zash@zash.se> parents: 8393 diff changeset	348 local offset = #items - query.truncate;
4892c22403d5 mod_storage_internal: Optimize truncation Kim Alvefur <zash@zash.se> parents: 8393 diff changeset	349 for i = 1, #items do
4892c22403d5 mod_storage_internal: Optimize truncation Kim Alvefur <zash@zash.se> parents: 8393 diff changeset	350 items[i] = items[i+offset];
8393 b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	351 end
b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	352 end
b6a7b83f8d87 mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store Kim Alvefur <zash@zash.se> parents: 8310 diff changeset	353 end
8310 1759491b53db mod_storage_internal: Add more extensive query support to archive:delete method Kim Alvefur <zash@zash.se> parents: 8309 diff changeset	354 end
8142 13e075549cb3 mod_storage_internal: Correctly calculate number of deleted items (fixes #912) Kim Alvefur <zash@zash.se> parents: 8089 diff changeset	355 local count = count_before - #items;
8402 469afa02947b mod_storage_internal: Skip write if no items matched a deletion query Kim Alvefur <zash@zash.se> parents: 8394 diff changeset	356 if count == 0 then
469afa02947b mod_storage_internal: Skip write if no items matched a deletion query Kim Alvefur <zash@zash.se> parents: 8394 diff changeset	357 return 0; -- No changes, skip write
469afa02947b mod_storage_internal: Skip write if no items matched a deletion query Kim Alvefur <zash@zash.se> parents: 8394 diff changeset	358 end
8022 05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	359 local ok, err = datamanager.list_store(username, host, self.store, items);
05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	360 if not ok then return ok, err; end
9885 64e16d1e91f6 mod_storage_internal,_sql: Key item count cache on both username and store Kim Alvefur <zash@zash.se> parents: 9884 diff changeset	361 archive_item_count_cache:set(cache_key, #items);
8022 05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	362 return count;
05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	363 end
05e201468f27 mod_storage_internal: Add support for removing archived items Kim Alvefur <zash@zash.se> parents: 8021 diff changeset	364
5121 b5a5643f8572 core.storagemanager, mod_storage_: "data-driver" -> "storage-provider", to allow using module:provides(). Waqas Hussain <waqas20@gmail.com>* parents: 5039 diff changeset	365 module:provides("storage", driver);

prosody

542a9a503073

plugins/mod_storage_internal.lua

Software / code / prosody

Annotate

plugins/mod_storage_internal.lua @ 11961:542a9a503073

Software `/` code `/` prosody