Software `/` code `/` prosody-modules

---
labels:
- 'Stage-Alpha'
summary: RESTful XMPP API
rockspec:
  build:
    modules:
      mod_rest.jsonmap: jsonmap.lib.lua
    copy_directories:
    - example
    - res
---

# Introduction

This is yet another RESTful API for sending and receiving stanzas via
Prosody. It can be used to build bots and components implemented as HTTP
services. It is the spiritual successor to [mod_post_msg] and absorbs
use cases from [mod_http_rest] and [mod_component_http] and other such
modules floating around the Internet.

# Usage

## On VirtualHosts

```lua
VirtualHost "example.com"
modules_enabled = {"rest"}
```

## As a Component

``` {.lua}
Component "rest.example.net" "rest"
component_secret = "dmVyeSBzZWNyZXQgdG9rZW4K"
modules_enabled = {"http_oauth2"}
```

## OAuth2

[mod_http_oauth2] can be used to grant bearer tokens which are accepted
by mod_rest.  Tokens can be passed to `curl` like `--oauth2-bearer
dmVyeSBzZWNyZXQgdG9rZW4K` instead of using `--user`.

## Sending stanzas

The API endpoint becomes available at the path `/rest`, so the full URL
will be something like `https://your-prosody.example:5281/rest`.

To try it, simply `curl` an XML stanza payload:

``` {.sh}
curl https://prosody.example:5281/rest \
    --user username \
    -H 'Content-Type: application/xmpp+xml' \
    --data-binary '<message type="chat" to="user@example.org">
            <body>Hello!</body>
        </message>'
```

or a JSON payload:

``` {.sh}
curl https://prosody.example:5281/rest \
    --user username \
    -H 'Content-Type: application/json' \
    --data-binary '{
           "body" : "Hello!",
           "kind" : "message",
           "to" : "user@example.org",
           "type" : "chat"
        }'
```

The `Content-Type` header is important!

### Parameters in path

New alternative format with the parameters `kind`, `type`, and `to`
embedded in the path:

```
curl https://prosody.example:5281/rest/message/chat/john@example.com \
    --user username \
    -H 'Content-Type: text/plain' \
    --data-binary 'Hello John!'
```

### Replies

A POST containing an `<iq>` stanza automatically wait for the reply,
long-polling style.

``` {.sh}
curl https://prosody.example:5281/rest \
    --user username \
    -H 'Content-Type: application/xmpp+xml' \
    --data-binary '<iq type="get" to="example.net">
            <ping xmlns="urn:xmpp:ping"/>
        </iq>'
```

Replies to other kinds of stanzas that are generated by the same Prosody
instance *MAY* be returned in the HTTP response. Replies from other
entities (connected clients or remote servers) will not be returned, but
can be forwarded via the callback API described in the next section.

### Simple info queries

A subset of IQ stanzas can be sent as simple GET requests

```
curl https://prosody.example:5281/rest/version/example.com \
    --user username \
    -H 'Accept: application/json'
```

The supported queries are

-   `archive`
-   `disco`
-   `extdisco`
-   `items`
-   `lastactivity`
-   `oob`
-   `payload`
-   `ping`
-   `stats`
-   `version`

## Receiving stanzas

TL;DR: Set this webhook callback URL, get XML `POST`-ed there.

``` {.lua}
Component "rest.example.net" "rest"
rest_callback_url = "http://my-api.example:9999/stanzas"
```

The callback URL supports a few variables from the stanza being sent,
namely `{kind}` (e.g. message, presence, iq or meta) and ones
corresponding to stanza attributes: `{type}`, `{to}` and `{from}`.

The preferred format can be indicated via the Accept header in response
to an OPTIONS probe that mod_rest does on startup, or by configuring:

``` {.lua}
rest_callback_content_type = "application/json"
```

Example callback looks like:

``` {.xml}
POST /stanzas HTTP/1.1
Content-Type: application/xmpp+xml
Content-Length: 102

<message to="bot@rest.example.net" from="user@example.com" type="chat">
<body>Hello</body>
</message>
```

or as JSON:

``` {.json}
POST /stanzas HTTP/1.1
Content-Type: application/json
Content-Length: 133

{
   "body" : "Hello",
   "from" : "user@example.com",
   "kind" : "message",
   "to" : "bot@rest.example.net",
   "type" : "chat"
}
```

### Which stanzas

The set of stanzas routed to the callback is determined by these two
settings:

`rest_callback_stanzas`
:   The stanza kinds to handle, defaults to `{ "message", "presence", "iq" }`

`rest_callback_events`
:   For the selected stanza kinds, which events to handle.  When loaded
on a Component, this defaults to `{ "bare", "full", "host" }`, while on
a VirtualHost the default is `{ "host" }`.

Events correspond to which form of address was used in the `to`
attribute of the stanza.

bare
:   `localpart@hostpart`

full
:   `localpart@hostpart/resourcepart`

host
:   `hostpart`

The following example would handle only stanzas like `<message
to="anything@hello.example"/>`

```lua
Component "hello.example" "rest"
rest_callback_url = "http://hello.internal.example:9003/api"
rest_callback_stanzas = { "message" }
rest_callback_events = { "bare" }
```

### Replying

To accept the stanza without returning a reply, respond with HTTP status
code `202` or `204`.

HTTP status codes in the `4xx` and `5xx` range are mapped to an
appropriate stanza error.

For full control over the response, set the `Content-Type` header to
`application/xmpp+xml` and return an XMPP stanza as an XML snippet.

``` {.xml}
HTTP/1.1 200 Ok
Content-Type: application/xmpp+xml

<message type="chat">
<body>Yes, this is bot</body>
</message>
```

## Payload format

### JSON

``` {.json}
{
   "body" : "Hello!",
   "kind" : "message",
   "type" : "chat"
}
```

Further JSON object keys as follows:

#### Messages

`kind`
:   `"message"`

`type`
:   Commonly `"chat"` for 1-to-1 messages and `"groupchat"` for group
    chat messages. Others include `"normal"`, `"headline"` and
    `"error"`.

`body`
:   Human-readable message text.

`subject`
:   Message subject or MUC topic.

`html`
:   HTML.

`oob_url`
:   URL of an out-of-band resource, often used for images.

#### Presence

`kind`
:   `"presence"`

`type`
:   Empty for online or `"unavailable"` for offline.

`show`
:   [Online
    status](https://xmpp.org/rfcs/rfc6121.html#presence-syntax-children-show),
    `away`, `dnd` etc.

`status`
:   Human-readable status message.

#### Info-Queries

Only one type of payload can be included in an `iq`.

`kind`
:   `"iq"`

`type`
:   `"get"` or `"set"` for queries, `"response"` or `"error"` for
    replies.

`ping`
:   Send a ping. Get a pong. Maybe.

`disco`
:   Retrieve service discovery information about an entity.

`items`
:   Discover list of items (other services, groupchats etc).

### XML

``` {.xml}
<message type="" id="" to="" from="" xml:lang="">
...
</message>
```

An XML declaration (`<?xml?>`) **MUST NOT** be included.

The payload MUST contain one (1) `message`, `presence` or `iq` stanza.

The stanzas MUST NOT have an `xmlns` attribute, and the default/empty
namespace is treated as `jabber:client`.

# Examples

## Python / Flask

Simple echo bot that responds to messages as XML:

``` {.python}
from flask import Flask, Response, request
import xml.etree.ElementTree as ET

app = Flask("echobot")


@app.before_request
def parse():
    request.stanza = ET.fromstring(request.data)


@app.route("/", methods=["POST"])
def hello():
    if request.stanza.tag == "message":
        return Response(
            "<message><body>Yes this is bot</body></message>",
            content_type="application/xmpp+xml",
        )

    return Response(status=501)


if __name__ == "__main__":
    app.run()
```

And a JSON variant:

``` {.python}
from flask import Flask, Response, request, jsonify

app = Flask("echobot")


@app.route("/", methods=["POST"])
def hello():
    print(request.data)
    if request.is_json:
        data = request.get_json()
        if data["kind"] == "message":
            return jsonify({"body": "hello"})

    return Response(status=501)


if __name__ == "__main__":
    app.run()
```

Remember to set `rest_callback_content_type = "application/json"` for
this to work.

# JSON mapping

This section describes the JSON mapping. It can't represent any possible
stanza, for full flexibility use the XML mode.

## Stanza basics

`kind`
:   String representing the kind of stanza, one of `"message"`,
    `"presence"` or `"iq"`.

`type`
:   String with the type of stanza, appropriate values vary depending on
    `kind`, see [RFC 6121]. E.g.`"chat"` for *message* stanzas etc.

`to`
:   String containing the XMPP Address of the destination / recipient of
    the stanza.

`from`
:   String containing the XMPP Address of the sender the stanza.

`id`
:   String with a reasonably unique identifier for the stanza.

## Basic Payloads

### Messages

`body`
:   String, human readable text message.

`subject`
:   String, human readable summary equivalent to an email subject or the
    chat room topic in a `type:groupchat` message.

### Presence

`show`
:   String representing availability, e.g. `"away"`, `"dnd"`. No value
    means a normal online status. See [RFC 6121] for the full list.

`status`
:   String with a human readable text message describing availability.

## More payloads

### Messages

`state`
:   String with current chat state, e.g. `"active"` (default) and
    `"composing"` (typing).

`html`
:   String with HTML allowing rich formatting. **MUST** be contained in a
    `<body>` element.

`oob_url`
:   String with an URL of an external resource.

### Presence

`muc`
:   Object with [MUC][XEP-0045] related properties.

### IQ

`ping`
:   Boolean, a simple ping query. "Pongs" have only basic fields
    presents.

`version`
:   Map with `name`, `version` fields, and optionally an `os` field, to
    describe the software.

#### Service Discovery

`disco`

:   Boolean `true` in a `kind:iq` `type:get` for a service discovery
    query.

    Responses have a map containing an array of available features in
    the `features` key and an array of "identities" in the `identities`
    key. Each identity has a `category` and `type` field as well as an
    optional `name` field. See [XEP-0030] for further details.

`items`
:   Boolean `true` in a `kind:iq` `type:get` for a service discovery
    items list query. The response contain an array of items like
    `{"jid":"xmpp.address.here","name":"Description of item"}`.

`extensions`
:   Map of extended feature discovery (see [XEP-0128]) data with
    `FORM_DATA` fields as the keys pointing at maps with the rest of the
    data.

#### Ad-Hoc Commands

Used to execute arbitrary commands on supporting entities.

`command`

:   String representing the command `node` or Map with the following
    possible fields:

    `node`
    :   Required string with node from disco\#items query for the
        command to execute.

    `action`
    :   Optional enum string defaulting to `"execute"`. Multi-step
        commands may involve `"next"`, `"prev"`, `"complete"` or
        `"cancel"`.

    `actions`
    :   Set (map of strings to `true`) with available actions to proceed
        with in multi-step commands.

    `status`
    :   String describing the status of the command, normally
        `"executing"`.

    `sessionid`
    :   Random session ID issued by the responder to identify the
        session in multi-step commands.

    `note`
    :   Map with `"type"` and `"text"` fields that carry simple result
        information.

    `form`
    :   Data form with description of expected input and data types in
        the next step of multi-step commands. **TODO** document format.

    `data`
    :   Map with only the data for result dataforms. Fields may be
        strings or arrays of strings.

##### Example

Discovering commands:

``` {.json}
{
   "items" : {
      "node" : "http://jabber.org/protocol/commands"
   },
   "id" : "8iN9hwdAAcfTBchm",
   "kind" : "iq",
   "to" : "example.com",
   "type" : "get"
}
```

Response:

``` {.json}
{
   "from" : "example.com",
   "id" : "8iN9hwdAAcfTBchm",
   "items" : [
      {
         "jid" : "example.com",
         "name" : "Get uptime",
         "node" : "uptime"
      }
   ],
   "kind" : "iq",
   "type" : "result"
}
```

Execute the command:

``` {.json}
{
   "command" : {
      "node" : "uptime"
   },
   "id" : "Jv-87nRaP6Mnrp8l",
   "kind" : "iq",
   "to" : "example.com",
   "type" : "set"
}
```

Executed:

``` {.json}
{
   "command" : {
      "node" : "uptime",
      "note" : {
         "text" : "This server has been running for 0 days, 20 hours and 54 minutes (since Fri Feb  7 18:05:30 2020)",
         "type" : "info"
      },
      "sessionid" : "6380880a-93e9-4f13-8ee2-171927a40e67",
      "status" : "completed"
   },
   "from" : "example.com",
   "id" : "Jv-87nRaP6Mnrp8l",
   "kind" : "iq",
   "type" : "result"
}
```

# TODO

-   Describe multi-step commands with dataforms.
-   Versioned API, i.e. /v1/stanzas
-   Bind resource to webhook/callback

# Compatibility

Requires Prosody trunk / 0.12
author	Jonas Schäfer <jonas@wielicki.name>
date	Sat, 01 Apr 2023 12:10:56 +0200
parent	5179:5be04d1b16fb
child	5931:d194d1012fd3