GNU bug report logs - #39800
(web client) gracelessly handles premature TLS connection termination

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 39800 in the body.
You can then email your comments to 39800 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "franco.rcr <at> gmail.com" <franco.rcr <at> gmail.com>
To: bug-guile <at> gnu.org
Subject: gnutls guile bug receiving https data
Date: Wed, 26 Feb 2020 16:20:09 +0100

Hello,
I installed gnutls for guile and checked the gnutls module with this 
simple code:


;;Guile version 3.0 and gnutls  from git

,show version
GNU Guile 3.0.0.15-ff14b7

(gnutls-version)
$6 = "3.6.12"

;;Now, submitting this simple https request, you get an exception
(http-request "https://www.google.com")
ice-9/boot-9.scm:1669:16: In procedure raise-exception:
Throw to key `gnutls-error' with args `(#<gnutls-error-enum La 
connessione TLS non è stata terminata in modo corretto.> 
read_from_session_record_port)'.

;;instead, without https there are no errors
(http-request "http://www.google.com") ;;works fine.


The error happens only on https://www.google.com and does not throw with 
a lot of other https web sites.
Furthermore the error is not throw if the method is HEAD, so it is 
related to the data part of the https answer.
I tried to enter in the internal implementation of the http web client 
but after some tests I decided to do some simple tests at application level.
I rewrote the get-bytevector-all, with a loop that reads one byte per 
time and the error was thrown anyway.
I catched the error and I've got the complete answer from the google web 
server.
In the following there is my applicative solution, where I rewrote the 
get-bytevector-all by adding the error checking and specifiyng to 
http-request that the data has to be returned as a port (#:streaming? #t).


;;A macro to catch errors
(define-syntax my-noerr
  (syntax-rules ()
    ((_ __error-return exp ...)
     (let
         ((__st #f))
       (catch #t
         (lambda() exp ...)
         (lambda (k . p) __error-return))))))

;;The rewriting of get-bytevector-all
(defun get-bytevector-all (port)
  (u8-list->bytevector (let loop ((port port))
     (let ((v (my-noerr (eof-object) (get-u8 port))))
       (if (eof-object? v)
       #nil
       (cons v (loop port)))))))

;;the piece of code that now gives the correct result
  (let-values (((a b)(http-request "https://www.google.com" 
#:streaming? #t)))
    (bytevector->string (get-bytevector-all b) "ISO-8859-1"))

As conclusion I can say that web modules read correctly the http answers 
and, with some (one for me, the google web site) https sites there is a 
misinterpretation of EOF in the layer between http and https.

Franco.

Message #8 received at 39800 <at> debbugs.gnu.org (full text, mbox):

From: Andy Wingo <wingo <at> igalia.com>
To: "franco.rcr\@gmail.com" <franco.rcr <at> gmail.com>
Cc: ludo <at> gnu.org, 39800 <at> debbugs.gnu.org
Subject: Re: bug#39800: gnutls guile bug receiving https data
Date: Tue, 03 Mar 2020 21:51:47 +0100

Thanks very much for the report!

I think this one may be a good one for Ludovic, if I may be so bold.
Apologies for the top-post but I couldn't clip your excellent report.

Cheers,

Andy

"franco.rcr <at> gmail.com" <franco.rcr <at> gmail.com> writes:

> Hello,
> I installed gnutls for guile and checked the gnutls module with this
> simple code:
>
>
> ;;Guile version 3.0 and gnutls  from git
>
> ,show version
> GNU Guile 3.0.0.15-ff14b7
>
> (gnutls-version)
> $6 = "3.6.12"
>
> ;;Now, submitting this simple https request, you get an exception
> (http-request "https://www.google.com")
> ice-9/boot-9.scm:1669:16: In procedure raise-exception:
> Throw to key `gnutls-error' with args `(#<gnutls-error-enum La
> connessione TLS non è stata terminata in modo corretto.>
> read_from_session_record_port)'.
>
> ;;instead, without https there are no errors
> (http-request "http://www.google.com") ;;works fine.
>
>
> The error happens only on https://www.google.com and does not throw with
> a lot of other https web sites.
> Furthermore the error is not throw if the method is HEAD, so it is
> related to the data part of the https answer.
> I tried to enter in the internal implementation of the http web client
> but after some tests I decided to do some simple tests at application
> level.
> I rewrote the get-bytevector-all, with a loop that reads one byte per
> time and the error was thrown anyway.
> I catched the error and I've got the complete answer from the google web
> server.
>
> In the following there is my applicative solution, where I rewrote the
> get-bytevector-all by adding the error checking and specifiyng to
> http-request that the data has to be returned as a port (#:streaming?
> #t).
>
>
> ;;A macro to catch errors
> (define-syntax my-noerr
>   (syntax-rules ()
>     ((_ __error-return exp ...)
>      (let
>          ((__st #f))
>        (catch #t
>          (lambda() exp ...)
>          (lambda (k . p) __error-return))))))
>
> ;;The rewriting of get-bytevector-all
> (defun get-bytevector-all (port)
>   (u8-list->bytevector (let loop ((port port))
>      (let ((v (my-noerr (eof-object) (get-u8 port))))
>        (if (eof-object? v)
>        #nil
>        (cons v (loop port)))))))
>
> ;;the piece of code that now gives the correct result
>   (let-values (((a b)(http-request "https://www.google.com" #:streaming?
> #t)))
>     (bytevector->string (get-bytevector-all b) "ISO-8859-1"))
>
> As conclusion I can say that web modules read correctly the http answers
> and, with some (one for me, the google web site) https sites there is a
> misinterpretation of EOF in the layer between http and https.
>
> Franco.

Message #13 received at 39800-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: "franco.rcr\@gmail.com" <franco.rcr <at> gmail.com>
Cc: 39800-done <at> debbugs.gnu.org
Subject: Re: bug#39800: gnutls guile bug receiving https data
Date: Fri, 06 Mar 2020 23:46:35 +0100

Hi,

"franco.rcr <at> gmail.com" <franco.rcr <at> gmail.com> skribis:

> ;;Now, submitting this simple https request, you get an exception
> (http-request "https://www.google.com")
> ice-9/boot-9.scm:1669:16: In procedure raise-exception:
> Throw to key `gnutls-error' with args `(#<gnutls-error-enum La
> connessione TLS non è stata terminata in modo corretto.>
> read_from_session_record_port)'.

That happens when passing #:keep-alive? #f (the default), specifically
when reading the response body:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (http-request "https://www.google.com" #:keep-alive? #f #:streaming? #t)
$1 = #<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers: ((date . #<date nanosecond: 0 second: 56 minute: 45 hour: 21 day: 6 month: 3 year: 2020 zone-offset: 0>) (expires . #<date nanosecond: 0 second: 0 minute: 0 hour: 0 day: 1 month: 1 year: 1970 zone-offset: 0>) (cache-control private (max-age . 0)) (content-type text/html (charset . "ISO-8859-1")) (p3p . "CP=\"This is not a P3P policy! See g.co/p3phelp for more info.\"") (server . "gws") (x-xss-protection . "0") (x-frame-options . "SAMEORIGIN") (set-cookie . "1P_JAR=2020-03-06-21; expires=Sun, 05-Apr-2020 21:45:56 GMT; path=/; domain=.google.com; Secure") (set-cookie . "NID=199=yXgE_KAGvxJbZAIGEXLt8CsEe3pre-RRLm1Jqap3b3iJRqZZq_PJ9wCT798mfDZ2TC5_3mKnM5KABSh8CguI64SsNoWHIc9EsW2osFltsIJnMswXhrtjQFDpfm_fUb6RDrWrqKHkOuvkG7Izp5im1Ys1TzGdztrFmOQV4FOraJk; expires=Sat, 05-Sep-2020 21:45:56 GMT; path=/; domain=.google.com; HttpOnly") (alt-svc . "quic=\":443\"; ma=2592000; v=\"46,43\",h3-Q050=\":443\"; ma=2592000,h3-Q049=\":443\"; ma=2592000,h3-Q048=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000") (accept-ranges none) (vary accept-encoding) (connection close)) port: #<input-output: file 7f3ae7dd3540>>
$2 = #<input-output: file 7f3ae7dd3540>
scheme@(guile-user)> (define bv (get-bytevector-all $2))
ice-9/boot-9.scm:1669:16: In procedure raise-exception:
Throw to key `gnutls-error' with args `(#<gnutls-error-enum The TLS connection was non-properly terminated.> read_from_session_record_port)'.
--8<---------------cut here---------------end--------------->8---

The reason for this is that google.com closes the connection right away,
without sending a proper TLS “bye” message as is conventionally done.

Fixed in commit 076276c4f580368b4106316a77752d69c8f1494a, which will be
in 3.0.1.

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.