Software /
code /
prosody
File
configure @ 11523:5f15ab7c6ae5
Statistics: Rewrite statistics backends to use OpenMetrics
The metric subsystem of Prosody has had some shortcomings from
the perspective of the current state-of-the-art in metric
observability.
The OpenMetrics standard [0] is a formalization of the data
model (and serialization format) of the well-known and
widely-used Prometheus [1] software stack.
The previous stats subsystem of Prosody did not map well to that
format (see e.g. [2] and [3]); the key reason is that it was
trying to do too much math on its own ([2]) while lacking
first-class support for "families" of metrics ([3]) and
structured metric metadata (despite the `extra` argument to
metrics, there was no standard way of representing common things
like "tags" or "labels").
Even though OpenMetrics has grown from the Prometheus world of
monitoring, it maps well to other popular monitoring stacks
such as:
- InfluxDB (labels can be mapped to tags and fields as necessary)
- Carbon/Graphite (labels can be attached to the metric name with
dot-separation)
- StatsD (see graphite when assuming that graphite is used as
backend, which is the default)
The util.statsd module has been ported to use the OpenMetrics
model as a proof of concept. An implementation which exposes
the util.statistics backend data as Prometheus metrics is
ready for publishing in prosody-modules (most likely as
mod_openmetrics_prometheus to avoid breaking existing 0.11
deployments).
At the same time, the previous measure()-based API had one major
advantage: It is really simple and easy to use without requiring
lots of knowledge about OpenMetrics or similar concepts. For that
reason as well as compatibility with existing code, it is preserved
and may even be extended in the future.
However, code relying on the `stats-updated` event as well as
`get_stats` from `statsmanager` will break because the data
model has changed completely; in case of `stats-updated`, the
code will simply not run (as the event was renamed in order
to avoid conflicts); the `get_stats` function has been removed
completely (so it will cause a traceback when it is attempted
to be used).
Note that the measure_*_event methods have been removed from
the module API. I was unable to find any uses or documentation
and thus deemed they should not be ported. Re-implementation is
possible when necessary.
[0]: https://openmetrics.io/
[1]: https://prometheus.io/
[2]: #959
[3]: #960
author | Jonas Schäfer <jonas@wielicki.name> |
---|---|
date | Sun, 18 Apr 2021 11:47:41 +0200 |
parent | 11009:74ef9f2334f3 |
child | 11528:d3b0049e50f5 |
line wrap: on
line source
#!/bin/sh # Defaults APP_NAME="Prosody" APP_DIRNAME="prosody" PREFIX="/usr/local" SYSCONFDIR="$PREFIX/etc/$APP_DIRNAME" LIBDIR="$PREFIX/lib" DATADIR="$PREFIX/var/lib/$APP_DIRNAME" LUA_SUFFIX="" LUA_DIR="/usr" LUA_BINDIR="/usr/bin" LUA_INCDIR="/usr/include" LUA_LIBDIR="/usr/lib" IDN_LIB="idn" ICU_FLAGS="-licui18n -licudata -licuuc" OPENSSL_LIB="crypto" CC="gcc" LD="gcc" RUNWITH="lua" EXCERTS="yes" PRNG= PRNGLIBS= CFLAGS="-fPIC -std=c99" CFLAGS="$CFLAGS -Wall -pedantic -Wextra -Wshadow -Wformat=2" LDFLAGS="-shared" IDN_LIBRARY="idn" # Help show_help() { cat <<EOF Configure $APP_NAME prior to building. --help This help. --ostype=OS Use one of the OS presets. May be one of: debian, macosx, linux, freebsd, openbsd, netbsd --prefix=DIR Prefix where $APP_NAME should be installed. Default is $PREFIX --sysconfdir=DIR Location where the config file should be installed. Default is \$PREFIX/etc/$APP_DIRNAME --libdir=DIR Location where the server files should be stored. Default is \$PREFIX/lib --datadir=DIR Location where the server data should be stored. Default is \$PREFIX/var/lib/$APP_DIRNAME --lua-version=VERSION Use specific Lua version: 5.1, 5.2, or 5.3 Default is auto-detected. --lua-suffix=SUFFIX Versioning suffix to use in Lua filenames. Default is "$LUA_SUFFIX" (lua$LUA_SUFFIX...) --with-lua=PREFIX Use Lua from given prefix. Default is auto-detected (the parent directory of \$LUA_BINDIR). --with-lua-bin=DIR You can also specify Lua's bin dir. Default is the directory of the auto-detected Lua interpreter, or \$LUA_DIR/bin if --with-lua is used. --runwith=BINARY What Lua binary to set as runtime environment. Default is $RUNWITH --with-lua-include=DIR You can also specify Lua's includes dir. Default is \$LUA_DIR/include --with-lua-lib=DIR You can also specify Lua's libraries dir. Default is \$LUA_DIR/lib --with-idn=LIB The name of the IDN library to link with. Default is $IDN_LIB --idn-library=(idn|icu) Select library to use for IDNA functionality. idn: use GNU libidn (default) icu: use ICU from IBM --with-ssl=LIB The name of the SSL to link with. Default is $OPENSSL_LIB --with-random=METHOD CSPRNG backend to use. One of getrandom: Linux kernel arc4random: OpenBSD kernel openssl: OpenSSL RAND method Default is to use /dev/urandom --cflags=FLAGS Flags to pass to the compiler Default is $CFLAGS --add-cflags=FLAGS Adds additional CFLAGS, preserving defaults. Can be repeated. --ldflags=FLAGS Flags to pass to the linker Default is $LDFLAGS --add-ldflags=FLAGS Adds additional linker flags, preserving defaults. Can be repeated. --c-compiler=CC The C compiler to use when building modules. Default is $CC --compiler-wrapper=WRAPPER Adds a prefix to compiler and linker calls, usable for eg distcc or ccache. --linker=CC The linker to use when building modules. Default is $LD --no-example-certs Disables generation of example certificates. EOF } # Helper functions find_program() { prog=$(command -v "$1" 2>/dev/null) if [ -n "$prog" ] then dirname "$prog" fi } die() { echo "$*" echo echo "configure failed." echo exit 1 } # shellcheck disable=SC2039 case $(echo -n x) in -n*) echo_n_flag='';; *) echo_n_flag='-n';; esac echo_n() { echo $echo_n_flag "$*" } # ---------------------------------------------------------------------------- # MAIN PROGRAM # ---------------------------------------------------------------------------- # Parse options while [ -n "$1" ] do value=$(echo "$1" | sed 's/[^=]*.\(.*\)/\1/') key=$(echo "$1" | sed 's/=.*//') # shellcheck disable=SC2088 if echo "$value" | grep "~" >/dev/null 2>/dev/null then echo echo '*WARNING*: the "~" sign is not expanded in flags.' # shellcheck disable=SC2016 echo 'If you mean the home directory, use $HOME instead.' echo fi case "$key" in --help) show_help exit 0 ;; --prefix) [ -n "$value" ] || die "Missing value in flag $key." PREFIX="$value" PREFIX_SET=yes ;; --sysconfdir) [ -n "$value" ] || die "Missing value in flag $key." SYSCONFDIR="$value" SYSCONFDIR_SET=yes ;; --ostype) OSPRESET="$value" OSPRESET_SET="yes" ;; --libdir) LIBDIR="$value" LIBDIR_SET=yes ;; --datadir) DATADIR="$value" DATADIR_SET=yes ;; --lua-suffix) [ -n "$value" ] || die "Missing value in flag $key." LUA_SUFFIX="$value" LUA_SUFFIX_SET=yes ;; --lua-version|--with-lua-version) [ -n "$value" ] || die "Missing value in flag $key." LUA_VERSION="$value" [ "$LUA_VERSION" = "5.1" ] || [ "$LUA_VERSION" = "5.2" ] || [ "$LUA_VERSION" = "5.3" ] || [ "$LUA_VERSION" = "5.4" ] || die "Invalid Lua version in flag $key." LUA_VERSION_SET=yes ;; --with-lua) [ -n "$value" ] || die "Missing value in flag $key." LUA_DIR="$value" LUA_DIR_SET=yes ;; --with-lua-bin) [ -n "$value" ] || die "Missing value in flag $key." LUA_BINDIR="$value" LUA_BINDIR_SET=yes ;; --with-lua-include) [ -n "$value" ] || die "Missing value in flag $key." LUA_INCDIR="$value" LUA_INCDIR_SET=yes ;; --with-lua-lib) [ -n "$value" ] || die "Missing value in flag $key." LUA_LIBDIR="$value" LUA_LIBDIR_SET=yes ;; --with-idn) IDN_LIB="$value" ;; --idn-library) IDN_LIBRARY="$value" ;; --with-ssl) OPENSSL_LIB="$value" ;; --with-random) case "$value" in getrandom) PRNG=GETRANDOM ;; openssl) PRNG=OPENSSL ;; arc4random) PRNG=ARC4RANDOM ;; esac ;; --cflags) CFLAGS="$value" ;; --add-cflags) CFLAGS="$CFLAGS $value" ;; --ldflags) LDFLAGS="$value" ;; --add-ldflags) LDFLAGS="$LDFLAGS $value" ;; --c-compiler) CC="$value" ;; --linker) LD="$value" ;; --runwith) RUNWITH="$value" RUNWITH_SET=yes ;; --no-example-certs) EXCERTS= ;; --compiler-wrapper) CC="$value $CC" LD="$value $LD" ;; *) die "Error: Unknown flag: $1" ;; esac shift done if [ "$OSPRESET_SET" = "yes" ]; then # TODO make this a switch? if [ "$OSPRESET" = "debian" ]; then CFLAGS="$CFLAGS -ggdb" fi if [ "$OSPRESET" = "macosx" ]; then if [ "$LUA_INCDIR_SET" != "yes" ]; then LUA_INCDIR=/usr/local/include; LUA_INCDIR_SET=yes fi if [ "$LUA_LIBDIR_SET" != "yes" ]; then LUA_LIBDIR=/usr/local/lib LUA_LIBDIR_SET=yes fi CFLAGS="$CFLAGS -mmacosx-version-min=10.3" LDFLAGS="-bundle -undefined dynamic_lookup" fi if [ "$OSPRESET" = "linux" ]; then CFLAGS="$CFLAGS -ggdb" fi if [ "$OSPRESET" = "freebsd" ] || [ "$OSPRESET" = "openbsd" ]; then LUA_INCDIR="/usr/local/include/lua51" LUA_INCDIR_SET=yes CFLAGS="-Wall -fPIC -I/usr/local/include" LDFLAGS="-I/usr/local/include -L/usr/local/lib -shared" LUA_SUFFIX="51" LUA_SUFFIX_SET=yes LUA_DIR=/usr/local LUA_DIR_SET=yes CC=cc LD=ld fi if [ "$OSPRESET" = "openbsd" ]; then LUA_INCDIR="/usr/local/include"; LUA_INCDIR_SET="yes" fi if [ "$OSPRESET" = "netbsd" ]; then LUA_INCDIR="/usr/pkg/include/lua-5.1" LUA_INCDIR_SET=yes LUA_LIBDIR="/usr/pkg/lib/lua/5.1" LUA_LIBDIR_SET=yes CFLAGS="-Wall -fPIC -I/usr/pkg/include" LDFLAGS="-L/usr/pkg/lib -Wl,-rpath,/usr/pkg/lib -shared" fi if [ "$OSPRESET" = "pkg-config" ]; then if [ "$LUA_SUFFIX_SET" != "yes" ]; then LUA_SUFFIX="5.1"; LUA_SUFFIX_SET=yes fi LUA_CF="$(pkg-config --cflags-only-I lua$LUA_SUFFIX)" LUA_CF="${LUA_CF#*-I}" LUA_CF="${LUA_CF%% *}" if [ "$LUA_CF" != "" ]; then LUA_INCDIR="$LUA_CF" LUA_INCDIR_SET=yes fi CFLAGS="$CFLAGS" fi fi if [ "$PREFIX_SET" = "yes" ] && [ ! "$SYSCONFDIR_SET" = "yes" ] then if [ "$PREFIX" = "/usr" ] then SYSCONFDIR=/etc/$APP_DIRNAME else SYSCONFDIR=$PREFIX/etc/$APP_DIRNAME fi fi if [ "$PREFIX_SET" = "yes" ] && [ ! "$DATADIR_SET" = "yes" ] then if [ "$PREFIX" = "/usr" ] then DATADIR=/var/lib/$APP_DIRNAME else DATADIR=$PREFIX/var/lib/$APP_DIRNAME fi fi if [ "$PREFIX_SET" = "yes" ] && [ ! "$LIBDIR_SET" = "yes" ] then LIBDIR=$PREFIX/lib fi detect_lua_version() { detected_lua=$("$1" -e 'print(_VERSION:match(" (5%.[1234])$"))' 2> /dev/null) if [ "$detected_lua" != "nil" ] then if [ "$LUA_VERSION_SET" != "yes" ] then echo "Lua version detected: $detected_lua" LUA_VERSION=$detected_lua return 0 elif [ "$LUA_VERSION" = "$detected_lua" ] then return 0 fi fi return 1 } search_interpreter() { suffix="$1" if [ "$LUA_BINDIR_SET" = "yes" ] then find_lua="$LUA_BINDIR" elif [ "$LUA_DIR_SET" = "yes" ] then LUA_BINDIR="$LUA_DIR/bin" if [ -f "$LUA_BINDIR/lua$suffix" ] then find_lua="$LUA_BINDIR" fi else find_lua=$(find_program lua"$suffix") fi if [ -n "$find_lua" ] && [ -x "$find_lua/lua$suffix" ] then if detect_lua_version "$find_lua/lua$suffix" then echo "Lua interpreter found: $find_lua/lua$suffix..." if [ "$LUA_BINDIR_SET" != "yes" ] then LUA_BINDIR="$find_lua" fi if [ "$LUA_DIR_SET" != "yes" ] then LUA_DIR=$(dirname "$find_lua") fi LUA_SUFFIX="$suffix" return 0 fi fi return 1 } lua_interp_found=no if [ "$LUA_SUFFIX_SET" != "yes" ] then if [ "$LUA_VERSION_SET" = "yes" ] && [ "$LUA_VERSION" = "5.1" ] then suffixes="5.1 51 -5.1 -51" elif [ "$LUA_VERSION_SET" = "yes" ] && [ "$LUA_VERSION" = "5.2" ] then suffixes="5.2 52 -5.2 -52" elif [ "$LUA_VERSION_SET" = "yes" ] && [ "$LUA_VERSION" = "5.3" ] then suffixes="5.3 53 -5.3 -53" elif [ "$LUA_VERSION_SET" = "yes" ] && [ "$LUA_VERSION" = "5.4" ] then suffixes="5.4 54 -5.4 -54" else suffixes="5.1 51 -5.1 -51" suffixes="$suffixes 5.2 52 -5.2 -52" suffixes="$suffixes 5.3 53 -5.3 -53" suffixes="$suffixes 5.4 54 -5.4 -54" fi for suffix in "" $suffixes do search_interpreter "$suffix" && { lua_interp_found=yes break } done else search_interpreter "$LUA_SUFFIX" && { lua_interp_found=yes } fi # See #1353 if [ "$LUA_DIR_SET" != "yes" ] && [ "$LUA_DIR" = "/" ] then LUA_DIR="/usr" fi if [ "$lua_interp_found" != "yes" ] && [ "$RUNWITH_SET" != "yes" ] then if [ "$LUA_VERSION_SET" ]; then interp="Lua $LUA_VERSION"; else interp="Lua"; fi if [ "$LUA_DIR_SET" ] || [ "$LUA_BINDIR_SET" ]; then where="$LUA_BINDIR"; else where="\$PATH"; fi echo "$interp interpreter not found in $where" die "You may want to use the flags --with-lua, --with-lua-bin and/or --lua-suffix. See --help." fi if [ "$LUA_VERSION_SET" = "yes" ] && [ "$RUNWITH_SET" != "yes" ] then echo_n "Checking if $LUA_BINDIR/lua$LUA_SUFFIX is Lua version $LUA_VERSION... " if detect_lua_version "$LUA_BINDIR/lua$LUA_SUFFIX" then echo "yes" else echo "no" die "You may want to use the flags --with-lua, --with-lua-bin and/or --lua-suffix. See --help." fi fi if [ "$LUA_INCDIR_SET" != "yes" ] then LUA_INCDIR="$LUA_DIR/include" fi if [ "$LUA_LIBDIR_SET" != "yes" ] then LUA_LIBDIR="$LUA_DIR/lib" fi lua_h="$LUA_INCDIR/lua.h" echo_n "Looking for lua.h at $lua_h..." if [ -f "$lua_h" ] then echo found else echo "not found" for postfix in "$LUA_VERSION" "$LUA_SUFFIX"; do if ! [ "$postfix" = "" ]; then v_dir="$LUA_INCDIR/lua/$postfix"; else v_dir="$LUA_INCDIR/lua"; fi lua_h="$v_dir/lua.h" echo_n "Looking for lua.h at $lua_h..." if [ -f "$lua_h" ] then LUA_INCDIR="$v_dir" echo found break; else echo "not found" d_dir="$LUA_INCDIR/lua$postfix" lua_h="$d_dir/lua.h" echo_n "Looking for lua.h at $lua_h..." if [ -f "$lua_h" ] then echo found LUA_INCDIR="$d_dir" break; else echo "not found" fi fi done if [ ! -f "$lua_h" ]; then echo "lua.h not found." echo die "You may want to use the flag --with-lua or --with-lua-include. See --help." fi fi if [ "$lua_interp_found" = "yes" ] then echo_n "Checking if Lua header version matches that of the interpreter... " header_version=$(sed -n 's/.*LUA_VERSION_NUM.*5.\(.\).*/5.\1/p' "$lua_h") if [ "$header_version" = "$LUA_VERSION" ] then echo "yes" else echo "no" echo "lua.h version mismatch (interpreter: $LUA_VERSION; lua.h: $header_version)." die "You may want to use the flag --with-lua or --with-lua-include. See --help." fi fi if [ "$IDN_LIBRARY" = "icu" ] then IDNA_LIBS="$ICU_FLAGS" IDNA_FLAGS="-DUSE_STRINGPREP_ICU" fi if [ "$IDN_LIBRARY" = "idn" ] then IDNA_LIBS="-l$IDN_LIB" fi if [ -f config.unix ]; then rm -f config.unix fi if [ "$RUNWITH_SET" != yes ]; then RUNWITH="lua$LUA_SUFFIX" fi OPENSSL_LIBS="-l$OPENSSL_LIB" if [ "$PRNG" = "OPENSSL" ]; then PRNGLIBS=$OPENSSL_LIBS elif [ "$PRNG" = "ARC4RANDOM" ] && [ "$(uname)" = "Linux" ]; then PRNGLIBS="-lbsd" fi # Write config echo "Writing configuration..." echo rm -f built cat <<EOF > config.unix # This file was automatically generated by the configure script. # Run "./configure --help" for details. LUA_VERSION=$LUA_VERSION PREFIX=$PREFIX SYSCONFDIR=$SYSCONFDIR LIBDIR=$LIBDIR DATADIR=$DATADIR LUA_SUFFIX=$LUA_SUFFIX LUA_DIR=$LUA_DIR LUA_DIR_SET=$LUA_DIR_SET LUA_INCDIR=$LUA_INCDIR LUA_LIBDIR=$LUA_LIBDIR LUA_BINDIR=$LUA_BINDIR IDN_LIB=$IDN_LIB IDNA_FLAGS=$IDNA_FLAGS IDNA_LIBS=$IDNA_LIBS OPENSSL_LIBS=$OPENSSL_LIBS CFLAGS=$CFLAGS LDFLAGS=$LDFLAGS CC=$CC LD=$LD RUNWITH=$RUNWITH EXCERTS=$EXCERTS RANDOM=$PRNG RANDOM_LIBS=$PRNGLIBS EOF echo "Installation prefix: $PREFIX" echo "$APP_NAME configuration directory: $SYSCONFDIR" echo "Using Lua from: $LUA_DIR" make clean > /dev/null 2> /dev/null echo echo "Done. You can now run 'make' to build." echo