prosody-modules
fe081789f7b5
mod_http_upload_external/README.md

Find changesets by keywords by author, files, the commit message, revision number or hash, or revset expression.

Software / code / prosody-modules

Comparison

mod_http_upload_external/README.md @ 6003:fe081789f7b5

All community modules: Unify file extention of Markdown files to .md
author Menel <menel@snikket.de>
date Tue, 22 Oct 2024 10:26:01 +0200
parent 5933:mod_http_upload_external/README.markdown@070b0db6c4a0
child 6093:c359259a494d
comparison
equal deleted inserted replaced
6002:5a65a632d5b9 6003:fe081789f7b5
1 ---
2 description: HTTP File Upload (external service)
3 labels:
4 - Stage-Alpha
5 ---
6
7 Introduction
8 ============
9
10 This module implements [XEP-0363], which lets clients upload files
11 over HTTP to an external web server.
12
13 This module generates URLs that are signed using a HMAC. Any web service that can authenticate
14 these URLs can be used.
15
16 Implementations
17 ---------------
18
19 * [PHP implementation](https://hg.prosody.im/prosody-modules/raw-file/tip/mod_http_upload_external/share.php)
20 * [Python3+Flask implementation](https://github.com/horazont/xmpp-http-upload)
21 * [Go implementation, Prosody Filer](https://github.com/ThomasLeister/prosody-filer)
22 * [Perl implementation for nginx](https://github.com/weiss/ngx_http_upload)
23 * [Rust implementation](https://gitlab.com/nyovaya/xmpp-http-upload)
24
25 To implement your own service compatible with this module, check out the implementation notes below
26 (and if you publish your implementation - let us know!).
27
28 Configuration
29 =============
30
31 The module can be added as a new Component definition:
32
33 ``` {.lua}
34 Component "upload.example.org" "http_upload_external"
35 http_upload_external_base_url = "https://your.example.com/upload/service"
36 http_upload_external_secret = "your shared secret"
37 ```
38
39 It should **not** be added to modules_enabled.
40
41
42 External URL
43 ------------
44
45 You need to provide the path to the external service. Ensure it ends with '/'.
46
47 For example, to use the PHP implementation linked above, you might set it to:
48
49 ``` {.lua}
50 http_upload_external_base_url = "https://your.example.com/path/to/share.php/"
51 ```
52
53 Secret
54 ------
55
56 Set a long and unpredictable string as your secret. This is so the upload service can verify that
57 the upload comes from mod_http_upload_external, and random strangers can't upload to your server.
58
59 ``` {.lua}
60 http_upload_external_secret = "this is a secret string!"
61 ```
62
63 You need to set exactly the same secret string in your external service.
64
65 Limits
66 ------
67
68 A maximum file size can be set by:
69
70 ``` {.lua}
71 http_upload_external_file_size_limit = 123 -- bytes
72 ```
73
74 Default is 100MB (100\*1024\*1024).
75
76 Access
77 ------
78
79 You may want to give upload access to additional entities such as components
80 by using the `http_upload_external_access` config option.
81
82 ``` {.lua}
83 http_upload_external_access = {"gateway.example.com"};
84 ```
85
86 Compatibility
87 =============
88
89 Works with Prosody 0.9.x and later.
90
91 Implementation
92 ==============
93
94 To implement your own external service that is compatible with this module, you need to expose a
95 simple API that allows the HTTP GET, HEAD and PUT methods on arbitrary URLs located on your service.
96
97 For example, if http_upload_external_base_url is set to `https://example.com/upload/` then your service
98 might receive the following requests:
99
100 Upload a new file:
101
102 ```
103 PUT https://example.com/upload/foo/bar.jpg?v=49e9309ff543ace93d25be90635ba8e9965c4f23fc885b2d86c947a5d59e55b2
104 ```
105
106 Recipient checks the file size and other headers:
107
108 ```
109 HEAD https://example.com/upload/foo/bar.jpg
110 ```
111
112 Recipient downloads the file:
113
114 ```
115 GET https://example.com/upload/foo/bar.jpg
116 ```
117
118 The only tricky logic is in validation of the PUT request. Firstly, don't overwrite existing files (return 409 Conflict).
119
120 Then you need to validate the auth token.
121
122 ### Validating the auth token
123
124
125 | Version | Supports |
126 |:--------|:--------------------------------------------------------------------------------------------------------|
127 | v | Validates only filename and size. Does not support file type restrictions by the XMPP server. |
128 | v2 | Validates the filename, size and MIME type. This allows the server to implement MIME type restrictions. |
129
130 It is probable that a future v3 will be specified that allows carrying information about the uploader identity, allowing
131 the implementation of per-user quotas and limits.
132
133 Implementations may implement one or more versions of the protocol simultaneously. The XMPP server generates the URLs and
134 ultimately selects which version will be used.
135
136 XMPP servers MUST only generate URLs with **one** of the versions listed here. However in case multiple parameters are
137 present, upload services MUST **only** use the token from the highest parameter version that they support.
138
139 #### Version 1 (v)
140
141 The token will be in the URL query parameter 'v'. If it is absent, fail with 403 Forbidden.
142
143 Calculate the expected auth token by reading the value of the Content-Length header of the PUT request. E.g. for a 1MB file
144 will have a Content-Length of '1048576'. Append this to the uploaded file name, separated by a space (0x20) character.
145
146 For the above example, you would end up with the following string: "foo/bar.jpg 1048576"
147
148 The auth token is a SHA256 HMAC of this string, using the configured secret as the key. E.g.
149
150 ```
151 calculated_auth_token = hmac_sha256("foo/bar.jpg 1048576", "secret string")
152 ```
153
154 If this is not equal to the 'v' parameter provided in the upload URL, reject the upload with 403 Forbidden.
155
156 **Security note:** When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string
157 comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such
158 as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go.
159
160 #### Version 2 (v2)
161
162 The token will be in the URL query parameter 'v2'. If it is absent, fail with 403 Forbidden.
163
164 | Input | Example |Read from |
165 |:----------------|:------------|:--------------------------------------------------------------------|
166 |`file_path` | foo/bar.jpg | The URL of the PUT request, with the service's base prefix removed. |
167 |`content_length` | 1048576 | Content-Length header |
168 |`content_type` | image/jpeg | Content-Type header |
169
170 The parameters should be joined into a single string, separated by NUL bytes (`\0`):
171
172 ```
173 signed_string = ( file_path + '\0' + content_length + '\0' + content_type )
174 ```
175
176 ```
177 signed_string = "foo/bar.jpg\01048576\0image/jpeg"
178 ```
179
180 The expected auth token is the SHA256 HMAC of this string, using the configured secret key as the key. E.g.:
181
182 ```
183 calculated_auth_token = hmac_sha256(signed_string, "secret string")
184 ```
185
186 If this is not equal to the 'v2' parameter provided in the upload URL, reject the upload with 403 Forbidden.
187
188 **Security note:** When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string
189 comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such
190 as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go.
191
192 ### Security considerations
193
194 #### HTTPS
195
196 All uploads and downloads should only be over HTTPS. The security of the served content is protected only
197 by the uniqueness present in the URLs themselves, and not using HTTPS may leak the URLs and contents to third-parties.
198
199 Implementations should consider including HSTS and HPKP headers, with consent of the administrator.
200
201 #### MIME types
202
203 If the upload Content-Type header matches any of the following MIME types, it MUST be preserved and included in the Content-Type
204 of any GET requests made to download the file:
205
206 - `image/*`
207 - `video/*`
208 - `audio/*`
209 - `text/plain`
210
211 It is recommended that other MIME types are preserved, but served with the addition of the following header:
212
213 ```
214 Content-Disposition: attachment
215 ```
216
217 This prevents the browser interpreting scripts and other resources that may potentially be malicious.
218
219 Some browsers may also benefit from explicitly telling them not to try guessing the type of a file:
220
221 ```
222 X-Content-Type-Options: nosniff
223 ```
224
225 #### Security headers
226
227 The following headers should be included to provide additional sandboxing of resources, considering the uploaded
228 content is not understood or trusted by the upload service:
229
230 ```
231 Content-Security-Policy: default-src 'none'
232 X-Content-Security-Policy: default-src 'none'
233 X-WebKit-CSP: default-src 'none'
234 ```