Software `/` code `/` prosody

Annotate

util/serialization.lua @ 9751:39ee70fbb009

mod_mam: Perform message expiry based on building an index by date For each day, store a set of users that have new messages. To expire messages, we collect the union of sets of users from dates that fall outside the cleanup range. The previous algoritm did not work well with many users, especially with the default settings.

author	Kim Alvefur <zash@zash.se>
date	Thu, 03 Jan 2019 17:25:43 +0100
parent	9692:affcbccc1dff
child	9863:aebfd1601ae2

rev	line source
1523 841d61be198f Remove version number from copyright headers Matthew Wild <mwild1@gmail.com> parents: 1135 diff changeset	1 -- Prosody IM
2923 b7049746bd29 Update copyright headers for 2010 Matthew Wild <mwild1@gmail.com> parents: 2222 diff changeset	2 -- Copyright (C) 2008-2010 Matthew Wild
b7049746bd29 Update copyright headers for 2010 Matthew Wild <mwild1@gmail.com> parents: 2222 diff changeset	3 -- Copyright (C) 2008-2010 Waqas Hussain
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	4 -- Copyright (C) 2018 Kim Alvefur
5776 bd0ff8ae98a8 Remove all trailing whitespace Florian Zeitz <florob@babelmonkeys.de> parents: 5021 diff changeset	5 --
758 b1885732e979 GPL->MIT! Matthew Wild <mwild1@gmail.com> parents: 615 diff changeset	6 -- This project is MIT/X11 licensed. Please see the
b1885732e979 GPL->MIT! Matthew Wild <mwild1@gmail.com> parents: 615 diff changeset	7 -- COPYING file in the source package for more information.
546 1e65f64dfabf Added module util.serialization Waqas Hussain <waqas20@gmail.com> parents: diff changeset	8 --
1e65f64dfabf Added module util.serialization Waqas Hussain <waqas20@gmail.com> parents: diff changeset	9
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	10 local getmetatable = getmetatable;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	11 local next, type = next, type;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	12 local s_format = string.format;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	13 local s_gsub = string.gsub;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	14 local s_rep = string.rep;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	15 local s_char = string.char;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	16 local s_match = string.match;
546 1e65f64dfabf Added module util.serialization Waqas Hussain <waqas20@gmail.com> parents: diff changeset	17 local t_concat = table.concat;
1e65f64dfabf Added module util.serialization Waqas Hussain <waqas20@gmail.com> parents: diff changeset	18
3736 73399dd525e8 util.serialization: Implemented deserialize(). Waqas Hussain <waqas20@gmail.com> parents: 2923 diff changeset	19 local pcall = pcall;
5021 85b2689dbcfe Eliminate direct setfenv usage Florian Zeitz <florob@babelmonkeys.de> parents: 3745 diff changeset	20 local envload = require"util.envload".envload;
85b2689dbcfe Eliminate direct setfenv usage Florian Zeitz <florob@babelmonkeys.de> parents: 3745 diff changeset	21
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	22 local pos_inf, neg_inf = math.huge, -math.huge;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	23 local m_type = math.type or function (n)
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	24 return n % 1 == 0 and n <= 9007199254740992 and n >= -9007199254740992 and "integer" or "float";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	25 end;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	26
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	27 local char_to_hex = {};
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	28 for i = 0,255 do
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	29 char_to_hex[s_char(i)] = s_format("%02x", i);
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	30 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	31
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	32 local function to_hex(s)
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	33 return (s_gsub(s, ".", char_to_hex));
546 1e65f64dfabf Added module util.serialization Waqas Hussain <waqas20@gmail.com> parents: diff changeset	34 end
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	35
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	36 local function fatal_error(obj, why)
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	37 error("Can't serialize "..type(obj) .. (why and ": ".. why or ""));
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	38 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	39
9569 0bc399953c27 util.serialization: Rename non-fatal fallback handler for clarity Kim Alvefur <zash@zash.se> parents: 9568 diff changeset	40 local function nonfatal_fallback(x, why)
9570 ce403b6470f8 util.serialization: Encode non-fatal error in way that can be restored Kim Alvefur <zash@zash.se> parents: 9569 diff changeset	41 return s_format("{__type=%q,__error=%q}", type(x), why or "fail");
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	42 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	43
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	44 local string_escapes = {
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	45 ['\a'] = [[\a]]; ['\b'] = [[\b]];
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	46 ['\f'] = [[\f]]; ['\n'] = [[\n]];
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	47 ['\r'] = [[\r]]; ['\t'] = [[\t]];
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	48 ['\v'] = [[\v]]; ['\\'] = [[\\]];
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	49 ['\"'] = [[\"]]; ['\''] = [[\']];
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	50 }
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	51
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	52 for i = 0, 255 do
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	53 local c = s_char(i);
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	54 if not string_escapes[c] then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	55 string_escapes[c] = s_format("\\%03d", i);
546 1e65f64dfabf Added module util.serialization Waqas Hussain <waqas20@gmail.com> parents: diff changeset	56 end
1e65f64dfabf Added module util.serialization Waqas Hussain <waqas20@gmail.com> parents: diff changeset	57 end
1e65f64dfabf Added module util.serialization Waqas Hussain <waqas20@gmail.com> parents: diff changeset	58
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	59 local default_keywords = {
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	60 ["do"] = true; ["and"] = true; ["else"] = true; ["break"] = true;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	61 ["if"] = true; ["end"] = true; ["goto"] = true; ["false"] = true;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	62 ["in"] = true; ["for"] = true; ["then"] = true; ["local"] = true;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	63 ["or"] = true; ["nil"] = true; ["true"] = true; ["until"] = true;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	64 ["elseif"] = true; ["function"] = true; ["not"] = true;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	65 ["repeat"] = true; ["return"] = true; ["while"] = true;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	66 };
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	67
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	68 local function new(opt)
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	69 if type(opt) ~= "table" then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	70 opt = { preset = opt };
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	71 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	72
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	73 local types = {
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	74 table = true;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	75 string = true;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	76 number = true;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	77 boolean = true;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	78 ["nil"] = true;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	79 };
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	80
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	81 -- presets
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	82 if opt.preset == "debug" then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	83 opt.preset = "oneline";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	84 opt.freeze = true;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	85 opt.fatal = false;
9569 0bc399953c27 util.serialization: Rename non-fatal fallback handler for clarity Kim Alvefur <zash@zash.se> parents: 9568 diff changeset	86 opt.fallback = nonfatal_fallback;
9568 69f589c888e7 util.serialization: Disable use of unquoted table keys by default Kim Alvefur <zash@zash.se> parents: 9567 diff changeset	87 opt.unquoted = true;
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	88 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	89 if opt.preset == "oneline" then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	90 opt.indentwith = opt.indentwith or "";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	91 opt.itemstart = opt.itemstart or " ";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	92 opt.itemlast = opt.itemlast or "";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	93 opt.tend = opt.tend or " }";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	94 elseif opt.preset == "compact" then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	95 opt.indentwith = opt.indentwith or "";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	96 opt.itemstart = opt.itemstart or "";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	97 opt.itemlast = opt.itemlast or "";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	98 opt.equals = opt.equals or "=";
9568 69f589c888e7 util.serialization: Disable use of unquoted table keys by default Kim Alvefur <zash@zash.se> parents: 9567 diff changeset	99 opt.unquoted = true;
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	100 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	101
9569 0bc399953c27 util.serialization: Rename non-fatal fallback handler for clarity Kim Alvefur <zash@zash.se> parents: 9568 diff changeset	102 local fallback = opt.fallback or opt.fatal == false and nonfatal_fallback or fatal_error;
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	103
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	104 local function ser(v)
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	105 return (types[type(v)] or fallback)(v);
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	106 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	107
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	108 local keywords = opt.keywords or default_keywords;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	109
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	110 -- indented
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	111 local indentwith = opt.indentwith or "\t";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	112 local itemstart = opt.itemstart or "\n";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	113 local itemsep = opt.itemsep or ";";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	114 local itemlast = opt.itemlast or ";\n";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	115 local tstart = opt.tstart or "{";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	116 local tend = opt.tend or "}";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	117 local kstart = opt.kstart or "[";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	118 local kend = opt.kend or "]";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	119 local equals = opt.equals or " = ";
9568 69f589c888e7 util.serialization: Disable use of unquoted table keys by default Kim Alvefur <zash@zash.se> parents: 9567 diff changeset	120 local unquoted = opt.unquoted == true and "^[%a_][%w_]*$" or opt.unquoted;
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	121 local hex = opt.hex;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	122 local freeze = opt.freeze;
9564 ed0090f8b709 util.serialization: Make maximum table depth configurable Kim Alvefur <zash@zash.se> parents: 9486 diff changeset	123 local maxdepth = opt.maxdepth or 127;
9567 dbfa286cfa88 util.serialization: Add option for allowing multiple references to the same table (but not cycles) Kim Alvefur <zash@zash.se> parents: 9565 diff changeset	124 local multirefs = opt.multiref;
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	125
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	126 -- serialize one table, recursively
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	127 -- t - table being serialized
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	128 -- o - array where tokens are added, concatenate to get final result
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	129 -- - also used to detect cycles
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	130 -- l - position in o of where to insert next token
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	131 -- d - depth, used for indentation
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	132 local function serialize_table(t, o, l, d)
9565 9a1e2f5f674f util.serialization: Separate errors for multiple table references and max depth Kim Alvefur <zash@zash.se> parents: 9564 diff changeset	133 if o[t] then
9a1e2f5f674f util.serialization: Separate errors for multiple table references and max depth Kim Alvefur <zash@zash.se> parents: 9564 diff changeset	134 o[l], l = fallback(t, "table has multiple references"), l + 1;
9a1e2f5f674f util.serialization: Separate errors for multiple table references and max depth Kim Alvefur <zash@zash.se> parents: 9564 diff changeset	135 return l;
9a1e2f5f674f util.serialization: Separate errors for multiple table references and max depth Kim Alvefur <zash@zash.se> parents: 9564 diff changeset	136 elseif d > maxdepth then
9a1e2f5f674f util.serialization: Separate errors for multiple table references and max depth Kim Alvefur <zash@zash.se> parents: 9564 diff changeset	137 o[l], l = fallback(t, "max table depth reached"), l + 1;
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	138 return l;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	139 end
546 1e65f64dfabf Added module util.serialization Waqas Hussain <waqas20@gmail.com> parents: diff changeset	140
9567 dbfa286cfa88 util.serialization: Add option for allowing multiple references to the same table (but not cycles) Kim Alvefur <zash@zash.se> parents: 9565 diff changeset	141 -- Keep track of table loops
dbfa286cfa88 util.serialization: Add option for allowing multiple references to the same table (but not cycles) Kim Alvefur <zash@zash.se> parents: 9565 diff changeset	142 local ot = t; -- reference pre-freeze
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	143 o[t] = true;
9567 dbfa286cfa88 util.serialization: Add option for allowing multiple references to the same table (but not cycles) Kim Alvefur <zash@zash.se> parents: 9565 diff changeset	144 o[ot] = true;
9485 c667887d78ad util.serialization: Simpler metatable pre-processing Kim Alvefur <zash@zash.se> parents: 9484 diff changeset	145
c667887d78ad util.serialization: Simpler metatable pre-processing Kim Alvefur <zash@zash.se> parents: 9484 diff changeset	146 if freeze == true then
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	147 -- opportunity to do pre-serialization
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	148 local mt = getmetatable(t);
9485 c667887d78ad util.serialization: Simpler metatable pre-processing Kim Alvefur <zash@zash.se> parents: 9484 diff changeset	149 if type(mt) == "table" then
c667887d78ad util.serialization: Simpler metatable pre-processing Kim Alvefur <zash@zash.se> parents: 9484 diff changeset	150 local tag = mt.__name;
c667887d78ad util.serialization: Simpler metatable pre-processing Kim Alvefur <zash@zash.se> parents: 9484 diff changeset	151 local fr = mt.__freeze;
c667887d78ad util.serialization: Simpler metatable pre-processing Kim Alvefur <zash@zash.se> parents: 9484 diff changeset	152
c667887d78ad util.serialization: Simpler metatable pre-processing Kim Alvefur <zash@zash.se> parents: 9484 diff changeset	153 if type(fr) == "function" then
c667887d78ad util.serialization: Simpler metatable pre-processing Kim Alvefur <zash@zash.se> parents: 9484 diff changeset	154 t = fr(t);
c667887d78ad util.serialization: Simpler metatable pre-processing Kim Alvefur <zash@zash.se> parents: 9484 diff changeset	155 if type(tag) == "string" then
c667887d78ad util.serialization: Simpler metatable pre-processing Kim Alvefur <zash@zash.se> parents: 9484 diff changeset	156 o[l], l = tag, l + 1;
c667887d78ad util.serialization: Simpler metatable pre-processing Kim Alvefur <zash@zash.se> parents: 9484 diff changeset	157 end
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	158 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	159 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	160 end
9485 c667887d78ad util.serialization: Simpler metatable pre-processing Kim Alvefur <zash@zash.se> parents: 9484 diff changeset	161
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	162 o[l], l = tstart, l + 1;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	163 local indent = s_rep(indentwith, d);
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	164 local numkey = 1;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	165 local ktyp, vtyp;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	166 for k,v in next,t do
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	167 o[l], l = itemstart, l + 1;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	168 o[l], l = indent, l + 1;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	169 ktyp, vtyp = type(k), type(v);
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	170 if k == numkey then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	171 -- next index in array part
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	172 -- assuming that these are found in order
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	173 numkey = numkey + 1;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	174 elseif unquoted and ktyp == "string" and
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	175 not keywords[k] and s_match(k, unquoted) then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	176 -- unquoted keys
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	177 o[l], l = k, l + 1;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	178 o[l], l = equals, l + 1;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	179 else
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	180 -- quoted keys
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	181 o[l], l = kstart, l + 1;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	182 if ktyp == "table" then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	183 l = serialize_table(k, o, l, d+1);
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	184 else
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	185 o[l], l = ser(k), l + 1;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	186 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	187 -- =
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	188 o[l], o[l+1], l = kend, equals, l + 2;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	189 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	190
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	191 -- the value
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	192 if vtyp == "table" then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	193 l = serialize_table(v, o, l, d+1);
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	194 else
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	195 o[l], l = ser(v), l + 1;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	196 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	197 -- last item?
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	198 if next(t, k) ~= nil then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	199 o[l], l = itemsep, l + 1;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	200 else
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	201 o[l], l = itemlast, l + 1;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	202 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	203 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	204 if next(t) ~= nil then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	205 o[l], l = s_rep(indentwith, d-1), l + 1;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	206 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	207 o[l], l = tend, l +1;
9567 dbfa286cfa88 util.serialization: Add option for allowing multiple references to the same table (but not cycles) Kim Alvefur <zash@zash.se> parents: 9565 diff changeset	208
dbfa286cfa88 util.serialization: Add option for allowing multiple references to the same table (but not cycles) Kim Alvefur <zash@zash.se> parents: 9565 diff changeset	209 if multirefs then
dbfa286cfa88 util.serialization: Add option for allowing multiple references to the same table (but not cycles) Kim Alvefur <zash@zash.se> parents: 9565 diff changeset	210 o[t] = nil;
dbfa286cfa88 util.serialization: Add option for allowing multiple references to the same table (but not cycles) Kim Alvefur <zash@zash.se> parents: 9565 diff changeset	211 o[ot] = nil;
dbfa286cfa88 util.serialization: Add option for allowing multiple references to the same table (but not cycles) Kim Alvefur <zash@zash.se> parents: 9565 diff changeset	212 end
dbfa286cfa88 util.serialization: Add option for allowing multiple references to the same table (but not cycles) Kim Alvefur <zash@zash.se> parents: 9565 diff changeset	213
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	214 return l;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	215 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	216
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	217 function types.table(t)
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	218 local o = {};
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	219 serialize_table(t, o, 1, 1);
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	220 return t_concat(o);
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	221 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	222
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	223 local function serialize_string(s)
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	224 return '"' .. s_gsub(s, "[%z\1-\31\"\'\\\127-\255]", string_escapes) .. '"';
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	225 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	226
9483 903e0cfd4cc9 util.serialization: Make check of prefix for optional hex encoding stricter Kim Alvefur <zash@zash.se> parents: 9480 diff changeset	227 if type(hex) == "string" then
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	228 function types.string(s)
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	229 local esc = serialize_string(s);
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	230 if #esc > (#s*2+2+#hex) then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	231 return hex .. '"' .. to_hex(s) .. '"';
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	232 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	233 return esc;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	234 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	235 else
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	236 types.string = serialize_string;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	237 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	238
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	239 function types.number(t)
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	240 if m_type(t) == "integer" then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	241 return s_format("%d", t);
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	242 elseif t == pos_inf then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	243 return "(1/0)";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	244 elseif t == neg_inf then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	245 return "(-1/0)";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	246 elseif t ~= t then
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	247 return "(0/0)";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	248 end
9486 20aad0108999 util.serialization: Remove encoding of very large or very small numbers in scientific notation Kim Alvefur <zash@zash.se> parents: 9485 diff changeset	249 return s_format("%.18g", t);
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	250 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	251
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	252 -- Are these faster than tostring?
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	253 types["nil"] = function()
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	254 return "nil";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	255 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	256
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	257 function types.boolean(t)
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	258 return t and "true" or "false";
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	259 end
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	260
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	261 return ser;
546 1e65f64dfabf Added module util.serialization Waqas Hussain <waqas20@gmail.com> parents: diff changeset	262 end
1e65f64dfabf Added module util.serialization Waqas Hussain <waqas20@gmail.com> parents: diff changeset	263
6777 5de6b93d0190 util.: Remove use of module() function, make all module functions local and return them in a table at the end Kim Alvefur <zash@zash.se>* parents: 6670 diff changeset	264 local function deserialize(str)
3736 73399dd525e8 util.serialization: Implemented deserialize(). Waqas Hussain <waqas20@gmail.com> parents: 2923 diff changeset	265 if type(str) ~= "string" then return nil; end
73399dd525e8 util.serialization: Implemented deserialize(). Waqas Hussain <waqas20@gmail.com> parents: 2923 diff changeset	266 str = "return "..str;
9484 1d1541630c20 util.serialization: Use '=' prefix for chunk source description Kim Alvefur <zash@zash.se> parents: 9483 diff changeset	267 local f, err = envload(str, "=serialized data", {});
3736 73399dd525e8 util.serialization: Implemented deserialize(). Waqas Hussain <waqas20@gmail.com> parents: 2923 diff changeset	268 if not f then return nil, err; end
73399dd525e8 util.serialization: Implemented deserialize(). Waqas Hussain <waqas20@gmail.com> parents: 2923 diff changeset	269 local success, ret = pcall(f);
73399dd525e8 util.serialization: Implemented deserialize(). Waqas Hussain <waqas20@gmail.com> parents: 2923 diff changeset	270 if not success then return nil, ret; end
73399dd525e8 util.serialization: Implemented deserialize(). Waqas Hussain <waqas20@gmail.com> parents: 2923 diff changeset	271 return ret;
546 1e65f64dfabf Added module util.serialization Waqas Hussain <waqas20@gmail.com> parents: diff changeset	272 end
1e65f64dfabf Added module util.serialization Waqas Hussain <waqas20@gmail.com> parents: diff changeset	273
6777 5de6b93d0190 util.: Remove use of module() function, make all module functions local and return them in a table at the end Kim Alvefur <zash@zash.se>* parents: 6670 diff changeset	274 return {
9008 ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	275 new = new;
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	276 serialize = function (x, opt)
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	277 return new(opt)(x);
ae3c52419ec1 util.serialization: Rewritte for performance and flexibility Kim Alvefur <zash@zash.se> parents: 8555 diff changeset	278 end;
6777 5de6b93d0190 util.: Remove use of module() function, make all module functions local and return them in a table at the end Kim Alvefur <zash@zash.se>* parents: 6670 diff changeset	279 deserialize = deserialize;
5de6b93d0190 util.: Remove use of module() function, make all module functions local and return them in a table at the end Kim Alvefur <zash@zash.se>* parents: 6670 diff changeset	280 };

prosody

39ee70fbb009

util/serialization.lua

Software / code / prosody

Annotate

util/serialization.lua @ 9751:39ee70fbb009

Software `/` code `/` prosody