prosody-modules
fb4a50bf60f1
mod_pubsub_feeds/feeds.lib.lua

Find changesets by keywords by author, files, the commit message, revision number or hash, or revset expression.

Software / code / prosody-modules

Annotate

mod_pubsub_feeds/feeds.lib.lua @ 4542:fb4a50bf60f1

mod_prometheus: Invoke stats collection if in 'manual' mode Since 10d13e0554f9 a special value for statistics_interval "manual" exists, where a module is expected to invoke processing in connection to collection of stats. This makes internal collection and exporting to Prometheus happens at the same time with no chance of timers getting out of sync.
author Kim Alvefur <zash@zash.se>
date Tue, 13 Apr 2021 23:53:53 +0200
parent 4428:b328ca621ba6
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2132
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
1 local st = require "util.stanza";
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
2 -- RSS->Atom translator
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
3 -- http://code.matthewwild.co.uk/lua-feeds/
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
4
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
5 -- Helpers to translate item child elements
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
6 local rss2atom = {};
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
7 function rss2atom.title(atom_entry, tag)
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
8 atom_entry:tag("title"):text(tag:get_text()):up();
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
9 end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
10
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
11 function rss2atom.link(atom_entry, tag)
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
12 atom_entry:tag("link", { href = tag:get_text() }):up();
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
13 end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
14
4428
b328ca621ba6 mod_pubsub_feeds: Translate RSS <enclosure> to Atom for podcasts
Kim Alvefur <zash@zash.se>
parents: 4427
diff changeset
15 function rss2atom.enclosure(atom_entry, tag)
b328ca621ba6 mod_pubsub_feeds: Translate RSS <enclosure> to Atom for podcasts
Kim Alvefur <zash@zash.se>
parents: 4427
diff changeset
16 atom_entry:tag("link", { rel = "enclosure", href = tag.attr.url, type = tag.attr.type, length = tag.attr.length }):up();
b328ca621ba6 mod_pubsub_feeds: Translate RSS <enclosure> to Atom for podcasts
Kim Alvefur <zash@zash.se>
parents: 4427
diff changeset
17 end
b328ca621ba6 mod_pubsub_feeds: Translate RSS <enclosure> to Atom for podcasts
Kim Alvefur <zash@zash.se>
parents: 4427
diff changeset
18
2132
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
19 function rss2atom.author(atom_entry, tag)
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
20 atom_entry:tag("author")
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
21 :tag("email"):text(tag:get_text()):up()
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
22 :up();
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
23 end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
24
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
25 function rss2atom.guid(atom_entry, tag)
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
26 atom_entry:tag("id"):text(tag:get_text()):up();
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
27 end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
28
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
29 function rss2atom.category(atom_entry, tag)
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
30 atom_entry:tag("category", { term = tag:get_text(), scheme = tag.attr.domain }):up();
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
31 end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
32
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
33 function rss2atom.description(atom_entry, tag)
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
34 atom_entry:tag("summary"):text(tag:get_text()):up();
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
35 end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
36
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
37 local months = {
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
38 jan = "01", feb = "02", mar = "03", apr = "04", may = "05", jun = "06";
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
39 jul = "07", aug = "08", sep = "09", oct = "10", nov = "11", dec = "12";
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
40 };
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
41
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
42 function rss2atom.pubDate(atom_entry, tag)
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
43 local pubdate = tag:get_text():gsub("^%a+,", ""):gsub("^%s*", "");
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
44 local date, month, year, hour, minute, second, zone =
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
45 pubdate:match("^(%d%d?) (%a+) (%d+) (%d+):(%d+):?(%d*) ?(.*)$");
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
46 if not date then return; end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
47 if #date == 1 then
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
48 date = "0"..date;
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
49 end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
50 month = months[month:sub(1,3):lower()];
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
51 if #year == 2 then -- GAH!
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
52 if tonumber(year) > 80 then
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
53 year = "19"..year;
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
54 else
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
55 year = "20"..year;
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
56 end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
57 end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
58 if zone == "UT" or zone == "GMT" then zone = "Z"; end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
59 if #second == 0 then
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
60 second = "00";
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
61 end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
62 local date_string = string.format("%s-%s-%sT%s:%s:%s%s", year, month, date, hour, minute, second, zone);
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
63 atom_entry:tag("published"):text(date_string):up();
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
64 end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
65
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
66 -- Translate a single item to atom
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
67 local function translate_rss(rss_feed)
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
68 local feed = st.stanza("feed", { xmlns = "http://www.w3.org/2005/Atom" });
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
69 local channel = rss_feed:get_child("channel");
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
70 -- TODO channel properties
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
71 for item in channel:childtags("item") do
2378
649f733aa3dc mod_pubsub_feeds/feeds.lib: Fix converting RSS items to more than one Atom entry (thanks walduhu)
Kim Alvefur <zash@zash.se>
parents: 2133
diff changeset
72 feed:tag("entry");
2133
85762420a2c0 mod_pubsub_feeds: Use correct loop variable
Kim Alvefur <zash@zash.se>
parents: 2132
diff changeset
73 for tag in item:childtags() do
2132
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
74 local translator = rss2atom[tag.name];
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
75 if translator then
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
76 translator(feed, tag);
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
77 end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
78 end
4427
c402b273f2e3 mod_pubsub_feeds: Copy Atom tags from RSS as-is
Kim Alvefur <zash@zash.se>
parents: 2378
diff changeset
79 -- Preserve Atom-namespaced items
c402b273f2e3 mod_pubsub_feeds: Copy Atom tags from RSS as-is
Kim Alvefur <zash@zash.se>
parents: 2378
diff changeset
80 for atomtag in item:childtags(nil, "http://www.w3.org/2005/Atom") do
c402b273f2e3 mod_pubsub_feeds: Copy Atom tags from RSS as-is
Kim Alvefur <zash@zash.se>
parents: 2378
diff changeset
81 feed:add_child(st.clone(atomtag));
c402b273f2e3 mod_pubsub_feeds: Copy Atom tags from RSS as-is
Kim Alvefur <zash@zash.se>
parents: 2378
diff changeset
82 end
2378
649f733aa3dc mod_pubsub_feeds/feeds.lib: Fix converting RSS items to more than one Atom entry (thanks walduhu)
Kim Alvefur <zash@zash.se>
parents: 2133
diff changeset
83 feed:reset();
2132
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
84 end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
85 return feed;
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
86 end
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
87
b149ea428b81 mod_pubsub_feeds: Switch to use util.xml for parsing feeds and include RSS to Atom translation code from lua-feeds
Kim Alvefur <zash@zash.se>
parents:
diff changeset
88 return { translate_rss = translate_rss }