GNU bug report logs - #19439
HTTP client rejects invalid ETag headers

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 19439 in the body.
You can then email your comments to 19439 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: rekado <rekado <at> elephly.net>
To: bug-guix <at> gnu.org
Subject: guix download fails with "bad qstring header component"
Date: Thu, 25 Dec 2014 11:54:21 +0100

In an attempt to build the OpenJDK tools with icepick, I'm getting an
error when downloading the icepick tarball:

~~~~~~~~~~~~~
$ ./pre-inst-env guix download http://icedtea.wildebeest.org/hg/icepick/archive/15425b469aea.tar.gz
starting download of `/tmp/guix-file.zrKNcT' from `http://icedtea.wildebeest.org/hg/icepick/archive/15425b469aea.tar.gz'...
ERROR: Bad qstring header component: 1200667312.0

failed to download "/tmp/guix-file.zrKNcT" from "http://icedtea.wildebeest.org/hg/icepick/archive/15425b469aea.tar.gz"
guix download: error: http://icedtea.wildebeest.org/hg/icepick/archive/15425b469aea.tar.gz: download failed
~~~~~~~~~~~~~

Message #8 received at 19439 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: rekado <rekado <at> elephly.net>
Cc: 19439 <at> debbugs.gnu.org
Subject: Re: bug#19439: guix download fails with "bad qstring header component"
Date: Fri, 26 Dec 2014 23:27:19 +0100

rekado <rekado <at> elephly.net> skribis:

> $ ./pre-inst-env guix download http://icedtea.wildebeest.org/hg/icepick/archive/15425b469aea.tar.gz
> starting download of `/tmp/guix-file.zrKNcT' from `http://icedtea.wildebeest.org/hg/icepick/archive/15425b469aea.tar.gz'...
> ERROR: Bad qstring header component: 1200667312.0

Indeed.  The HTTP headers here look like this:

--8<---------------cut here---------------start------------->8---
Date: Fri, 26 Dec 2014 22:10:49 GMT
Server: Apache/2.2.22 (Debian)
ETag: 1200667312.0
--8<---------------cut here---------------end--------------->8---

For details, this can be reproduced like this:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(web client)
scheme@(guile-user)> (http-get "http://icedtea.wildebeest.org/hg/icepick/archive/15425b469aea.tar.gz")
web/http.scm:854:12: In procedure parse-entity-tag:
web/http.scm:854:12: Bad qstring header component: 1200667312.0


Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,bt
In web/client.scm:
    228:8  4 (request "http://icedtea.wildebeest.org/hg/icepick/archive/15425b469aea.tar.gz" #:body #f #:port #<input-output: s…> …)
In web/response.scm:
    199:6  3 (read-response #<input-output: socket 13>)
In web/http.scm:
   218:33  2 (lp ((server . "Apache/2.2.22 (Debian)") (date . #<date nanosecond: 0 second: 3 minute: 14 hour: 22 day: 26 mont…>)))
   188:11  1 (read-header #<input-output: socket 13>)
   854:12  0 (parse-entity-tag "1200667312.0")
--8<---------------cut here---------------end--------------->8---

And indeed, Section 14.19 of
<http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html> says that the
‘ETag’ header has type ‘entity-tag’.

‘parse-entity-tag’ is written like this:

--8<---------------cut here---------------start------------->8---
(define (parse-entity-tag val)
  (if (string-prefix? "W/" val)
      (cons (parse-qstring val 2) #f)
      (cons (parse-qstring val) #t)))
--8<---------------cut here---------------end--------------->8---

Section 3.11 at <http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html>
confirms the above definition for ‘entity-tag’.

Lastly, ‘quoted-string’ in
<http://www.w3.org/Protocols/rfc2616/rfc2616-sec2.html> is defined as a
string surrounded by double quotes, basically.

So, in short, I think Apache httpd is returning an invalid ETag header,
and Guile’s HTTP client is right in rejecting it.

Having said that doesn’t help much though.  I’m not sure how frequent
this is, and whether/how this could be worked around.  Ideas?

Thanks,
Ludo’.

Message #11 received at 19439 <at> debbugs.gnu.org (full text, mbox):

From: Eric Bavier <ericbavier <at> gmail.com>
To: 19439 <at> debbugs.gnu.org
Subject: Re: bug#19439: guix download fails with "bad qstring header component"
Date: Thu, 29 Jan 2015 10:16:54 -0600

I've encountered the same issue while attempting to upgrade the
python-pillow package.  For the newer versions 2.6.2 and 2.7.0, the
following errors are reported when downloading the source::

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
starting download of
`/gnu/store/i40q1mk9dnkg2mfyvfhjyq8lcywmgvh1-Pillow-2.7.0.tar.gz' from
`https://pypi.python.org/packages/source/P/Pillow/Pillow-2.7.0.tar.gz'...
ERROR: Bad qstring header component: 3fdc9f16a1b60366ae8158b424655d10

failed to download
"/gnu/store/i40q1mk9dnkg2mfyvfhjyq8lcywmgvh1-Pillow-2.7.0.tar.gz" from
"https://pypi.python.org/packages/source/P/Pillow/Pillow-2.7.0.tar.gz"
builder for
`/gnu/store/bij5w1kppj9wiqgarmsa3nc8z73i1v6l-Pillow-2.7.0.tar.gz.drv'
failed to produce output path
`/gnu/store/i40q1mk9dnkg2mfyvfhjyq8lcywmgvh1-Pillow-2.7.0.tar.gz'
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I don't have any ideas on how to work around this issue in general.  In
this case, we might be able to fetch the source from github rather than
pypi.  But this should add to the question of how frequent this problem
is.

-- 
Eric Bavier

Message #14 received at 19439 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Eric Bavier <ericbavier <at> gmail.com>
Cc: 19439 <at> debbugs.gnu.org
Subject: Re: bug#19439: guix download fails with "bad qstring header component"
Date: Tue, 03 Feb 2015 23:32:21 +0100

Eric Bavier writes:

> I've encountered the same issue while attempting to upgrade the
> python-pillow package.  For the newer versions 2.6.2 and 2.7.0, the
> following errors are reported when downloading the source::

[...]

> I don't have any ideas on how to work around this issue in general.  In
> this case, we might be able to fetch the source from github rather than
> pypi.  But this should add to the question of how frequent this problem
> is.

FWIW, pypi.python.org has been fixed after reporting the issue to the
webmasters.

Would it be desirable for the HTTP client to be a little more tolerant
in general?

~~ Ricardo

Message #17 received at 19439 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Andy Wingo <wingo <at> igalia.com>
Cc: guix-devel <at> gnu.org, 19439 <at> debbugs.gnu.org,
 Steve Sprang <steve.sprang <at> gmail.com>, guile-devel <at> gnu.org
Subject: Re: [PATCH] gnu: Add erlang.
Date: Wed, 06 Jan 2016 17:07:51 +0100

On Mon 04 Jan 2016 06:20, Steve Sprang <steve.sprang <at> gmail.com> writes:

> From http://www.erlang.org/download/otp_src_18.2.1.tar.gz...
> ERROR: Bad qstring header component: kMSMAn68110840

Andy Wingo <wingo <at> igalia.com> skribis:

> The Etag value is invalid:
>
>   https://tools.ietf.org/html/rfc7232#section-2.3
>
> We could relax Guile's etag parser to assume that an etag not starting
> with either W/ or " is a strong etag without quotes.  To do that you
> would patch guile's http.scm to say:
>
> (define (parse-entity-tag val)
>   (cond
>    ((string-prefix? "W/" val) (cons (parse-qstring val 2) #f))
>    ((string-prefix? "\"" val) (cons (parse-qstring val) #t))
>    (else (cons val #t))))
>
> Considering that this error has come up a few times and that the less
> strict parser doesn't change the Guile programming interface or endanger
> the web in any significant way, I think changing Guile's HTTP parser
> would be OK.
>
> Thoughts?

Indeed, this has been reported a few times:

  http://bugs.gnu.org/19439

I think we should implement the proposed change.  Could you look
into it?

Thanks,
Ludo’.

Message #24 received at 19439 <at> debbugs.gnu.org (full text, mbox):

From: Andy Wingo <wingo <at> igalia.com>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: guix-devel <at> gnu.org, 19439 <at> debbugs.gnu.org,
 Steve Sprang <steve.sprang <at> gmail.com>, guile-devel <at> gnu.org
Subject: Re: [PATCH] gnu: Add erlang.
Date: Thu, 07 Jan 2016 10:56:29 +0100

On Wed 06 Jan 2016 17:07, ludo <at> gnu.org (Ludovic Courtès) writes:

> On Mon 04 Jan 2016 06:20, Steve Sprang <steve.sprang <at> gmail.com> writes:
>
>> From http://www.erlang.org/download/otp_src_18.2.1.tar.gz...
>> ERROR: Bad qstring header component: kMSMAn68110840
>
> I think we should implement the proposed change.  Could you look
> into it?

Done on master in 6d7c09c8a9900794a855b9c69c57c3d1736506ed.  I'd
cherry-pick back to 2.0 but I don't have a checkout on this machine;
please feel free :)

Cheers,

Andy

Message #29 received at 19439-done <at> debbugs.gnu.org (full text, mbox):

From: Andy Wingo <wingo <at> pobox.com>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: guix-devel <at> gnu.org, 19439-done <at> debbugs.gnu.org, guile-devel <at> gnu.org,
 Steve Sprang <steve.sprang <at> gmail.com>
Subject: Re: bug#19439: [PATCH] gnu: Add erlang.
Date: Wed, 22 Jun 2016 21:42:38 +0200

Backported to 2.0 in c5dac3595f42afbd76ab9110d7473bf4d16b0520 by
Ludovic; closing bug.

On Thu 07 Jan 2016 10:56, Andy Wingo <wingo <at> igalia.com> writes:

> On Wed 06 Jan 2016 17:07, ludo <at> gnu.org (Ludovic Courtès) writes:
>
>> On Mon 04 Jan 2016 06:20, Steve Sprang <steve.sprang <at> gmail.com> writes:
>>
>>> From http://www.erlang.org/download/otp_src_18.2.1.tar.gz...
>>> ERROR: Bad qstring header component: kMSMAn68110840
>>
>> I think we should implement the proposed change.  Could you look
>> into it?
>
> Done on master in 6d7c09c8a9900794a855b9c69c57c3d1736506ed.  I'd
> cherry-pick back to 2.0 but I don't have a checkout on this machine;
> please feel free :)
>
> Cheers,
>
> Andy

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.