Software /
code /
prosody
Annotate
util/dbuffer.lua @ 11592:64cfa396bb84
net.server_epoll: Fix reporting of socket connect timeout
If the underlying TCP connection times out before the write timeout
kicks in, end up here with err="timeout", which the following code
treats as a minor issue.
Then, due to epoll apparently returning the EPOLLOUT (writable) event
too, we go on and try to write to the socket (commonly stream headers).
This fails because the socket is closed, which becomes the error
returned up the stack to the rest of Prosody.
This also trips the 'onconnect' signal, which has effects on various
things, such as the net.connect state machine. Probably undesirable
effects.
With this, we instead return "connection timeout", like server_event,
and destroy the connection handle properly. And then nothing else
happens because the connection has been destroyed.
author | Kim Alvefur <zash@zash.se> |
---|---|
date | Mon, 07 Jun 2021 17:37:14 +0200 |
parent | 11196:f243836c449a |
child | 11636:11e0a0a08da3 |
rev | line source |
---|---|
11104
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
1 local queue = require "util.queue"; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
2 |
11189
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
3 local s_byte, s_sub = string.byte, string.sub; |
11104
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
4 local dbuffer_methods = {}; |
11196
f243836c449a
util.dbuffer: Add __name to metatable
Matthew Wild <mwild1@gmail.com>
parents:
11190
diff
changeset
|
5 local dynamic_buffer_mt = { __name = "dbuffer", __index = dbuffer_methods }; |
11104
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
6 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
7 function dbuffer_methods:write(data) |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
8 if self.max_size and #data + self._length > self.max_size then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
9 return nil; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
10 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
11 local ok = self.items:push(data); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
12 if not ok then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
13 self:collapse(); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
14 ok = self.items:push(data); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
15 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
16 if not ok then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
17 return nil; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
18 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
19 self._length = self._length + #data; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
20 return true; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
21 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
22 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
23 function dbuffer_methods:read_chunk(requested_bytes) |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
24 local chunk, consumed = self.items:peek(), self.front_consumed; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
25 if not chunk then return; end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
26 local chunk_length = #chunk; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
27 local remaining_chunk_length = chunk_length - consumed; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
28 if not requested_bytes then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
29 requested_bytes = remaining_chunk_length; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
30 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
31 if remaining_chunk_length <= requested_bytes then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
32 self.front_consumed = 0; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
33 self._length = self._length - remaining_chunk_length; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
34 self.items:pop(); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
35 assert(#chunk:sub(consumed + 1, -1) == remaining_chunk_length); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
36 return chunk:sub(consumed + 1, -1), remaining_chunk_length; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
37 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
38 local end_pos = consumed + requested_bytes; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
39 self.front_consumed = end_pos; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
40 self._length = self._length - requested_bytes; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
41 assert(#chunk:sub(consumed + 1, end_pos) == requested_bytes); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
42 return chunk:sub(consumed + 1, end_pos), requested_bytes; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
43 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
44 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
45 function dbuffer_methods:read(requested_bytes) |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
46 local chunks; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
47 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
48 if requested_bytes and requested_bytes > self._length then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
49 return nil; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
50 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
51 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
52 local chunk, read_bytes = self:read_chunk(requested_bytes); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
53 if not requested_bytes then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
54 return chunk; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
55 elseif chunk then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
56 requested_bytes = requested_bytes - read_bytes; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
57 if requested_bytes == 0 then -- Already read everything we need |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
58 return chunk; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
59 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
60 chunks = {}; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
61 else |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
62 return nil; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
63 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
64 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
65 -- Need to keep reading more chunks |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
66 while chunk do |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
67 table.insert(chunks, chunk); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
68 if requested_bytes > 0 then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
69 chunk, read_bytes = self:read_chunk(requested_bytes); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
70 requested_bytes = requested_bytes - read_bytes; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
71 else |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
72 break; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
73 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
74 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
75 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
76 return table.concat(chunks); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
77 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
78 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
79 function dbuffer_methods:discard(requested_bytes) |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
80 if requested_bytes > self._length then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
81 return nil; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
82 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
83 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
84 local chunk, read_bytes = self:read_chunk(requested_bytes); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
85 if chunk then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
86 requested_bytes = requested_bytes - read_bytes; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
87 if requested_bytes == 0 then -- Already read everything we need |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
88 return true; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
89 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
90 else |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
91 return nil; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
92 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
93 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
94 while chunk do |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
95 if requested_bytes > 0 then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
96 chunk, read_bytes = self:read_chunk(requested_bytes); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
97 requested_bytes = requested_bytes - read_bytes; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
98 else |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
99 break; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
100 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
101 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
102 return true; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
103 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
104 |
11189
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
105 -- Normalize i, j into absolute offsets within the |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
106 -- front chunk (accounting for front_consumed), and |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
107 -- ensure there is enough data in the first chunk |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
108 -- to cover any subsequent :sub() or :byte() operation |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
109 function dbuffer_methods:_prep_sub(i, j) |
11104
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
110 if j == nil then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
111 j = -1; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
112 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
113 if j < 0 then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
114 j = self._length + (j+1); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
115 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
116 if i < 0 then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
117 i = self._length + (i+1); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
118 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
119 if i < 1 then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
120 i = 1; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
121 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
122 if j > self._length then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
123 j = self._length; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
124 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
125 if i > j then |
11189
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
126 return nil; |
11104
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
127 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
128 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
129 self:collapse(j); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
130 |
11189
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
131 if self.front_consumed > 0 then |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
132 i = i + self.front_consumed; |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
133 j = j + self.front_consumed; |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
134 end |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
135 |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
136 return i, j; |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
137 end |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
138 |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
139 function dbuffer_methods:sub(i, j) |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
140 i, j = self:_prep_sub(i, j); |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
141 if not i then |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
142 return ""; |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
143 end |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
144 return s_sub(self.items:peek(), i, j); |
11104
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
145 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
146 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
147 function dbuffer_methods:byte(i, j) |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
148 i = i or 1; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
149 j = j or i; |
11189
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
150 i, j = self:_prep_sub(i, j); |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
151 if not i then |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
152 return; |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
153 end |
0ff148362a3d
util.dbuffer: Optimize :sub() and :byte()
Matthew Wild <mwild1@gmail.com>
parents:
11156
diff
changeset
|
154 return s_byte(self.items:peek(), i, j); |
11104
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
155 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
156 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
157 function dbuffer_methods:length() |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
158 return self._length; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
159 end |
11156
a8ef69f7fc35
util.dbuffer: Expose length as :len() method, like strings
Kim Alvefur <zash@zash.se>
parents:
11104
diff
changeset
|
160 dbuffer_methods.len = dbuffer_methods.length; -- strings have :len() |
11104
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
161 dynamic_buffer_mt.__len = dbuffer_methods.length; -- support # operator |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
162 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
163 function dbuffer_methods:collapse(bytes) |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
164 bytes = bytes or self._length; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
165 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
166 local front_chunk = self.items:peek(); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
167 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
168 if not front_chunk or #front_chunk - self.front_consumed >= bytes then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
169 return; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
170 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
171 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
172 local front_chunks = { front_chunk:sub(self.front_consumed+1) }; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
173 local front_bytes = #front_chunks[1]; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
174 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
175 while front_bytes < bytes do |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
176 self.items:pop(); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
177 local chunk = self.items:peek(); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
178 front_bytes = front_bytes + #chunk; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
179 table.insert(front_chunks, chunk); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
180 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
181 self.items:replace(table.concat(front_chunks)); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
182 self.front_consumed = 0; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
183 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
184 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
185 local function new(max_size, max_chunks) |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
186 if max_size and max_size <= 0 then |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
187 return nil; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
188 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
189 return setmetatable({ |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
190 front_consumed = 0; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
191 _length = 0; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
192 max_size = max_size; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
193 items = queue.new(max_chunks or 32); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
194 }, dynamic_buffer_mt); |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
195 end |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
196 |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
197 return { |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
198 new = new; |
6632acc96cf6
util.dbuffer: Fix :sub() not working with partially-consumed chunks (thanks Zash for test case)
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
199 }; |