GNU bug report logs -
#19939
http client: Chunks shouldn't be read at once
Previous Next
Reported by: ludo <at> gnu.org (Ludovic Courtès)
Date: Tue, 24 Feb 2015 21:00:02 UTC
Severity: normal
Done: ludo <at> gnu.org (Ludovic Courtès)
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 19939 in the body.
You can then email your comments to 19939 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-guile <at> gnu.org
:
bug#19939
; Package
guile
.
(Tue, 24 Feb 2015 21:00:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
ludo <at> gnu.org (Ludovic Courtès)
:
New bug report received and forwarded. Copy sent to
bug-guile <at> gnu.org
.
(Tue, 24 Feb 2015 21:00:03 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
As of 2.0.11, the chunked input port of the HTTP client reads whole
chunks at once:
--8<---------------cut here---------------start------------->8---
(define (read-chunk port)
(let ((size (read-chunk-header port)))
(read-chunk-body port size)))
(define (read-chunk-body port size)
(let ((bv (get-bytevector-n port size)))
(get-u8 port) ; CR
(get-u8 port) ; LF
bv))
(define* (make-chunked-input-port port #:key (keep-alive? #f))
"Returns a new port which translates HTTP chunked transfer encoded
data from PORT into a non-encoded format. Returns eof when it has
read the final chunk from PORT. This does not necessarily mean
that there is no more data on PORT. When the returned port is
closed it will also close PORT, unless the KEEP-ALIVE? is true."
(define (next-chunk)
(read-chunk port))
[...]
(define (read! bv idx to-read)
[...]
(set! buffer (next-chunk))
[...]
(make-custom-binary-input-port "chunked input port" read! #f #f close))
--8<---------------cut here---------------end--------------->8---
This is undesirable because:
1. the HTTP server can produce arbitrarily large chunks, leading to
large memory use in the client (nginx does indeed produce very
large chunks in some cases);
2. it adds an extra level of buffering that the caller of ‘http-get’
does not control (a read of 1 byte from the HTTP body port leads
to an actual read of a whole chunk);
3. it introduces extra copying and allocations.
Ludo’.
Reply sent
to
ludo <at> gnu.org (Ludovic Courtès)
:
You have taken responsibility.
(Sun, 01 Mar 2015 22:45:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
ludo <at> gnu.org (Ludovic Courtès)
:
bug acknowledged by developer.
(Sun, 01 Mar 2015 22:45:03 GMT)
Full text and
rfc822 format available.
Message #10 received at 19939-done <at> debbugs.gnu.org (full text, mbox):
Fixed in commit 00d3ecf, which will be in 2.0.12.
Ludo’.
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Mon, 30 Mar 2015 11:24:04 GMT)
Full text and
rfc822 format available.
This bug report was last modified 9 years and 27 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.