prosody-modules
e49660ba3161
mod_http_upload_external/README.markdown

Find changesets by keywords by author, files, the commit message, revision number or hash, or revset expression.

Software / code / prosody-modules

Annotate

mod_http_upload_external/README.markdown @ 3358:e49660ba3161

mod_http_upload_external: Improve implementation docs, including v2 details
author Matthew Wild <mwild1@gmail.com>
date Sun, 21 Oct 2018 14:54:19 +0100
parent 3189:57332ea0c1c7
child 3359:3d01ab6b1186
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2334
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
1 ---
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
2 description: HTTP File Upload (external service)
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
3 labels: 'Stage-Alpha'
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
4 ---
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
5
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
6 Introduction
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
7 ============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
8
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
9 This module implements [XEP-0363], which lets clients upload files
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
10 over HTTP to an external web server.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
11
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
12 This module generates URLs that are signed using a HMAC. Any web service that can authenticate
2823
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
13 these URLs can be used.
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
14
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
15 Implementations
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
16 ---------------
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
17
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
18 * [PHP implementation](https://hg.prosody.im/prosody-modules/raw-file/tip/mod_http_upload_external/share.php)
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
19 * [Python3+Flask implementation](https://github.com/horazont/xmpp-http-upload)
3168
73a610c3c7a9 mod_http_external: Link to prosody-filer (Go implementation)
Matthew Wild <mwild1@gmail.com>
parents: 2823
diff changeset
20 * [Go implementation, Prosody Filer](https://github.com/ThomasLeister/prosody-filer)
3189
57332ea0c1c7 mod_http_upload_external/README: Add Perl implementation by Holger to list
Kim Alvefur <zash@zash.se>
parents: 3168
diff changeset
21 * [Perl implementation for nginx](https://github.com/weiss/ngx_http_upload)
2823
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
22
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
23 To implement your own service compatible with this module, check out the implementation notes below
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
24 (and if you publish your implementation - let us know!).
2334
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
25
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
26 Configuration
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
27 =============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
28
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
29 Add `"http_upload_external"` to modules_enabled in your global section, or under the host(s) you wish
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
30 to use it on.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
31
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
32 External URL
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
33 ------------
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
34
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
35 You need to provide the path to the external service. Ensure it ends with '/'.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
36
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
37 For example, to use the PHP implementation linked above, you might set it to:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
38
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
39 ``` {.lua}
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
40 http_upload_external_base_url = "https://your.example.com/path/to/share.php/"
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
41 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
42
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
43 Secret
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
44 ------
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
45
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
46 Set a long and unpredictable string as your secret. This is so the upload service can verify that
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
47 the upload comes from mod_http_upload_external, and random strangers can't upload to your server.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
48
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
49 ``` {.lua}
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
50 http_upload_external_secret = "this is a secret string!"
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
51 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
52
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
53 You need to set exactly the same secret string in your external service.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
54
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
55 Limits
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
56 ------
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
57
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
58 A maximum file size can be set by:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
59
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
60 ``` {.lua}
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
61 http_upload_external_file_size_limit = 123 -- bytes
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
62 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
63
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
64 Default is 100MB (100\*1024\*1024).
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
65
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
66 Compatibility
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
67 =============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
68
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
69 Works with Prosody 0.9.x and later.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
70
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
71 Implementation
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
72 ==============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
73
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
74 To implement your own external service that is compatible with this module, you need to expose a
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
75 simple API that allows the HTTP GET, HEAD and PUT methods on arbitrary URLs located on your service.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
76
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
77 For example, if http_upload_external_base_url is set to `https://example.com/upload/` then your service
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
78 might receive the following requests:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
79
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
80 Upload a new file:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
81
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
82 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
83 PUT https://example.com/upload/foo/bar.jpg?v=49e9309ff543ace93d25be90635ba8e9965c4f23fc885b2d86c947a5d59e55b2
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
84 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
85
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
86 Recipient checks the file size and other headers:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
87
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
88 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
89 HEAD https://example.com/upload/foo/bar.jpg
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
90 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
91
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
92 Recipient downloads the file:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
93
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
94 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
95 GET https://example.com/upload/foo/bar.jpg
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
96 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
97
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
98 The only tricky logic is in validation of the PUT request. Firstly, don't overwrite existing files (return 409 Conflict).
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
99
3358
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
100 Then you need to validate the auth token.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
101
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
102 ### Validating the auth token
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
103
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
104
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
105 | Version | Supports |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
106 |:--------|:--------------------------------------------------------------------------------------------------------|
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
107 | v | Validates only filename and size. Does not support file type restrictions by the XMPP server. |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
108 | v2 | Validates the filename, size and MIME type. This allows the server to implement MIME type restrictions. |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
109
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
110 It is probable that a future v3 will be specified that allows carrying information about the uploader identity, allowing
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
111 the implementation of per-user quotas and limits.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
112
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
113 Implementations may implement one or more versions of the protocol simultaneously. The XMPP server generates the URLs and ultimately selects which version will be used.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
114
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
115 #### Version 1 (v)
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
116
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
117 The token will be in the URL query parameter 'v'. If it is absent, fail with 403 Forbidden.
2334
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
118
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
119 Calculate the expected auth token by reading the value of the Content-Length header of the PUT request. E.g. for a 1MB file
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
120 will have a Content-Length of '1048576'. Append this to the uploaded file name, separated by a space (0x20) character.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
121
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
122 For the above example, you would end up with the following string: "foo/bar.jpg 1048576"
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
123
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
124 The auth token is a SHA256 HMAC of this string, using the configured secret as the key. E.g.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
125
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
126 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
127 calculated_auth_token = hmac_sha256("foo/bar.jpg 1048576", "secret string")
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
128 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
129
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
130 If this is not equal to the 'v' parameter provided in the upload URL, reject the upload with 403 Forbidden.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
131
3358
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
132 **Security note:** When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
133 comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
134 as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
135
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
136 #### Version 2 (v2)
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
137
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
138 The token will be in the URL query parameter 'v2'. If it is absent, fail with 403 Forbidden.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
139
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
140 | Input | Example |Read from |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
141 |:--------------|:------------|:--------------------------------------------------------------------|
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
142 |`file_path` | foo/bar.jpg | The URL of the PUT request, with the service's base prefix removed. |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
143 |`content_size` | 1048576 | Content-Size header |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
144 |`content_type` | image/jpeg | Content-Type header |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
145
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
146 The parameters should be joined into a single string, separated by NUL bytes (`\0`):
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
147
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
148 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
149 signed_string = ( file_path + '\0' + content_size + '\0' + content_type )
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
150 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
151
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
152 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
153 signed_string = "foo/bar.jpg\01048576\0image/jpeg"
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
154 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
155
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
156 The expected auth token is the SHA256 HMAC of this string, using the configured secret key as the key. E.g.:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
157
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
158 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
159 calculated_auth_token = hmac_sha256(signed_string, "secret string")
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
160 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
161
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
162 If this is not equal to the 'v2' parameter provided in the upload URL, reject the upload with 403 Forbidden.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
163
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
164 **Security note:** When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
165 comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
166 as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
167
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
168 ### Security considerations
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
169
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
170 #### HTTPS
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
171
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
172 All uploads and downloads should only be over HTTPS. The security of the served content is protected only
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
173 by the uniqueness present in the URLs themselves, and not using HTTPS may leak the URLs and contents to third-parties.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
174
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
175 Implementations should consider including HSTS and HPKP headers, with consent of the administrator.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
176
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
177 #### MIME types
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
178
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
179 If the upload Content-Type header matches any of the following MIME types, it MUST be preserved and included in the Content-Type
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
180 of any GET requests made to download the file:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
181
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
182 - `image/*`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
183 - `video/*`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
184 - `audio/*`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
185 - `text/plain`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
186
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
187 It is recommended that other MIME types are preserved, but served with the addition of the following header:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
188
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
189 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
190 Content-Disposition: attachment
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
191 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
192
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
193 This prevents the browser interpreting scripts and other resources that may potentially be malicious.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
194
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
195 Some browsers may also benefit from explicitly telling them not to try guessing the type of a file:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
196
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
197 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
198 X-Content-Type-Options "nosniff"
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
199 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
200
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
201 #### Security headers
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
202
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
203 The following headers should be included to provide additional sandboxing of resources, considering the uploaded
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
204 content is not understood or trusted by the upload service:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
205
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
206 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
207 Content-Security-Policy: "default-src 'none'"
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
208 X-Content-Security-Policy: "default-src 'none'"
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
209 X-WebKit-CSP: "default-src 'none'"
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
210 ```