GNU bug report logs - #14128
web/http.scm: bad-header date check (UTC?)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 14128 in the body.
You can then email your comments to 14128 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Aleix Conchillo Flaqué <aconchillo <at> gmail.com>
To: bug-guile <at> gnu.org
Subject: web/http.scm: bad-header date check (UTC?)
Date: Wed, 3 Apr 2013 00:59:03 -0700

[Message part 1 (text/plain, inline)]

Hi,

I was doing a test using gnutls and connecting to twitter api
services. Example attached.

I successfully passed the TLS/SSL part, but then I got this HTTP bad
header issue. I am running Guile 2.0.5 from Debian/unstable but
current stable-2.0 branch also suffers from the safe problem (I
think).

=============
184: 1 [read-header #<input-output: gnutls-session-port 2e94c30>]
 754: 0 [parse-asctime-date "Wed, 03 Apr 2013 07:29:49 UTC"]

web/http.scm:754:6: In procedure parse-asctime-date:
web/http.scm:754:6: Throw to key `bad-header' with args `(date "Wed,
03 Apr 2013 07:29:49 UTC")'.
=============

Basically, the problem is that web/http.scm only accepts a GMT suffix
and the twitter server sends UTC. I think the RFC 822 standard accepts
UT instead of UTC, so it would be wrong as well.

Really, not sure what's the right way to fix this considering that
even the twitter server does not send a correct day (as far as I
understood the RFC).

Thanks,

Aleix

[twitter-user-timeline.scm (application/octet-stream, attachment)]

Message #8 received at 14128 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Hartwig <mandyke <at> gmail.com>
To: Aleix Conchillo Flaqué <aconchillo <at> gmail.com>
Cc: 14128 <at> debbugs.gnu.org
Subject: Re: bug#14128: web/http.scm: bad-header date check (UTC?)
Date: Wed, 3 Apr 2013 17:47:01 +0800

On 3 April 2013 15:59, Aleix Conchillo Flaqué <aconchillo <at> gmail.com> wrote:
> Hi,
>
> I was doing a test using gnutls and connecting to twitter api
> services. Example attached.
>
> I successfully passed the TLS/SSL part, but then I got this HTTP bad
> header issue. I am running Guile 2.0.5 from Debian/unstable but
> current stable-2.0 branch also suffers from the safe problem (I
> think).
>
> =============
> 184: 1 [read-header #<input-output: gnutls-session-port 2e94c30>]
>  754: 0 [parse-asctime-date "Wed, 03 Apr 2013 07:29:49 UTC"]
>
> web/http.scm:754:6: In procedure parse-asctime-date:
> web/http.scm:754:6: Throw to key `bad-header' with args `(date "Wed,
> 03 Apr 2013 07:29:49 UTC")'.
> =============
>
> Basically, the problem is that web/http.scm only accepts a GMT suffix
> and the twitter server sends UTC. I think the RFC 822 standard accepts
> UT instead of UTC, so it would be wrong as well.

RFC 2616 _requires_ http date values have a suffix of " GMT", other
time zone values are not valid.  RFC 822 has no relevance.  Some times
you see these “API” or “web services” breaking HTTP spec. in various
ways and claiming “well we are not HTTP just very-HTTP-like” but that
is not useful for people trying to interact.

Turns out there are a few broken http servers around.  2616 recommends
to convert non-GMT date values to GMT where such conversion is
unambiguous, and in Guile we have done this a bit.  Apparently we are
supposed to do this a bit more and accomodate yet another
non-compliant service?

Erk.  What is the point of defining protocols and standards then?

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Thien-Thi Nguyen <ttn <at> gnuvola.org>
To: Daniel Hartwig <mandyke <at> gmail.com>
Cc: bug-guile <at> gnu.org,
	Aleix Conchillo Flaqué <aconchillo <at> gmail.com>
Subject: Re: bug#14128: web/http.scm: bad-header date check (UTC?)
Date: Wed, 03 Apr 2013 12:33:44 +0200

[Message part 1 (text/plain, inline)]

() Daniel Hartwig <mandyke <at> gmail.com>
() Wed, 3 Apr 2013 17:47:01 +0800

   Apparently we are supposed to do this a bit more and accomodate yet
   another non-compliant service?

Maybe that stuff should be exposed to the user.  Do a best effort
conversion and if not successful, return a pair ‘(raw-string . STRING)’
or whatever -- it's enough that it has a different type and that the
type is documented.  This way, you avoid carrying forward lots of cruft.
Do it now before it's too late!

   Erk.  What is the point of defining protocols and standards then?

One purpose is to challenge programmers to learn how and when to say "no".

-- 
Thien-Thi Nguyen
GPG key: 4C807502

[Message part 2 (application/pgp-signature, inline)]

Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Daniel Hartwig <mandyke <at> gmail.com>
To: Thien-Thi Nguyen <ttn <at> gnuvola.org>
Cc: bug-guile <at> gnu.org,
	Aleix Conchillo Flaqué <aconchillo <at> gmail.com>
Subject: Re: bug#14128: web/http.scm: bad-header date check (UTC?)
Date: Wed, 3 Apr 2013 18:47:22 +0800

On 3 April 2013 18:33, Thien-Thi Nguyen <ttn <at> gnuvola.org> wrote:
> () Daniel Hartwig <mandyke <at> gmail.com>
> () Wed, 3 Apr 2013 17:47:01 +0800
>
>    Apparently we are supposed to do this a bit more and accomodate yet
>    another non-compliant service?
>
> Maybe that stuff should be exposed to the user.  Do a best effort
> conversion and if not successful, return a pair ‘(raw-string . STRING)’
> or whatever -- it's enough that it has a different type and that the
> type is documented. This way, you avoid carrying forward lots of cruft.
> Do it now before it's too late!

Interesting.  Though this does gradually erode the type barrier
erected by the web module.  I am reluctant to cede this territory.
Instead of this cruft accumulating in a few places (the web modules),
it becomes gratutiously spread around and duplicated in other
programs.

It was previously suggested to implement a permissive flag that, while
not passing unparsed data to the users, will at least not raise errors
and stop.

>
>
>    Erk.  What is the point of defining protocols and standards then?
>
> One purpose is to challenge programmers to learn how and when to say "no".
>

:-)

Message #17 received at 14128 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Daniel Hartwig <mandyke <at> gmail.com>
Cc: Aleix Conchillo Flaqué <aconchillo <at> gmail.com>,
	14128 <at> debbugs.gnu.org
Subject: Re: bug#14128: web/http.scm: bad-header date check (UTC?)
Date: Wed, 03 Apr 2013 14:34:37 +0200

Daniel Hartwig <mandyke <at> gmail.com> skribis:

> RFC 2616 _requires_ http date values have a suffix of " GMT"

What about adding an exception for “UTC”?  :-)

It’s the same timezone, only with a different name, so it shouldn’t cost
us much.

WDYT?

Ludo’.

Message #20 received at 14128 <at> debbugs.gnu.org (full text, mbox):

From: Aleix Conchillo Flaqué <aconchillo <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 14128 <at> debbugs.gnu.org, Daniel Hartwig <mandyke <at> gmail.com>
Subject: Re: bug#14128: web/http.scm: bad-header date check (UTC?)
Date: Wed, 3 Apr 2013 07:57:48 -0700

On Wed, Apr 3, 2013 at 5:34 AM, Ludovic Courtès <ludo <at> gnu.org> wrote:
> Daniel Hartwig <mandyke <at> gmail.com> skribis:
>
>> RFC 2616 _requires_ http date values have a suffix of " GMT"
>
> What about adding an exception for “UTC”?  :-)
>
> It’s the same timezone, only with a different name, so it shouldn’t cost
> us much.
>
> WDYT?
>

This is what I did locally so I could continue testing. But
unfortunately, it's is very likely that there are more broken servers
around. The permissive flag seems like an "all-or-nothing", so I think
Thien-Thi's suggestion sounds pretty reasonable: for all headers
checks, give a default but let user specify their owns (if I
understood that correctly). In my case I would just provide a #t for
the date check.

Best,

Aleix

Message #23 received at 14128 <at> debbugs.gnu.org (full text, mbox):

From: Aleix Conchillo Flaqué <aconchillo <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 14128 <at> debbugs.gnu.org, Daniel Hartwig <mandyke <at> gmail.com>
Subject: Re: bug#14128: web/http.scm: bad-header date check (UTC?)
Date: Wed, 3 Apr 2013 08:02:06 -0700

On Wed, Apr 3, 2013 at 7:57 AM, Aleix Conchillo Flaqué
<aconchillo <at> gmail.com> wrote:
>
> This is what I did locally so I could continue testing. But
> unfortunately, it's is very likely that there are more broken servers
> around. The permissive flag seems like an "all-or-nothing", so I think
> Thien-Thi's suggestion sounds pretty reasonable: for all headers
> checks, give a default but let user specify their owns (if I
> understood that correctly). In my case I would just provide a #t for
> the date check.
>

I have to add, that it might be also true that you end up with all
checks returning true, to make every server happy.

So, all in all, I am a bit confused, may be the permissive flag is not
that bad. The user could still read the headers after that, right? And
decide whether they are good or bad.

Aleix

Message #26 received at 14128 <at> debbugs.gnu.org (full text, mbox):

From: Andy Wingo <wingo <at> pobox.com>
To: Aleix Conchillo Flaqué <aconchillo <at> gmail.com>
Cc: 14128 <at> debbugs.gnu.org
Subject: Re: bug#14128: web/http.scm: bad-header date check (UTC?)
Date: Thu, 04 Apr 2013 23:01:29 +0200

On Wed 03 Apr 2013 09:59, Aleix Conchillo Flaqué <aconchillo <at> gmail.com> writes:

> =============
> 184: 1 [read-header #<input-output: gnutls-session-port 2e94c30>]
>  754: 0 [parse-asctime-date "Wed, 03 Apr 2013 07:29:49 UTC"]
>
> web/http.scm:754:6: In procedure parse-asctime-date:
> web/http.scm:754:6: Throw to key `bad-header' with args `(date "Wed,
> 03 Apr 2013 07:29:49 UTC")'.
> =============

Just reported this:
https://twitter.com/andywingo/status/319916865642168320

Let's see if they fix it before doing anything on our side.

Andy
-- 
http://wingolog.org/

Message #29 received at 14128 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Hartwig <mandyke <at> gmail.com>
To: Thien-Thi Nguyen <ttn <at> gnuvola.org>
Cc: aconchillo <at> gmail.com, 14128 <at> debbugs.gnu.org
Subject: Re: bug#14128: web/http.scm: bad-header date check (UTC?)
Date: Fri, 5 Apr 2013 06:45:15 +0800

On 3 April 2013 20:32, Thien-Thi Nguyen <ttn <at> gnuvola.org> wrote:
> () Daniel Hartwig <mandyke <at> gmail.com>
> () Wed, 3 Apr 2013 18:47:22 +0800
>
>    Interesting.  Though this does gradually erode the type barrier
>    erected by the web module.  I am reluctant to cede this territory.
>
> I suppose playing whack-a-mole is fun sometimes, for a while.
>
>    Instead of this cruft accumulating in a few places (the web modules),
>    it becomes gratutiously spread around and duplicated in other
>    programs.
>
> Well more precisely, in those programs which talk to such servers, and
> actually care about that particular data.  (Many programs do not.)
>
>    It was previously suggested to implement a permissive flag that,
>    while not passing unparsed data to the users, will at least not raise
>    errors and stop.
>
> In this direction, you might as well go higher-order.  Client code
> specifies the conversion procedure, and there is a reasonable default
> (also exported, so that the user can compose it w/ their own bad-input
> detection/handling).  This is more Schemely, in my mind.

That is ‘declare-header!’.

Message #34 received at 14128-done <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: Aleix Conchillo Flaqué <aconchillo <at> gmail.com>
Cc: 14128-done <at> debbugs.gnu.org
Subject: Re: bug#14128: web/http.scm: bad-header date check (UTC?)
Date: Tue, 19 Nov 2013 03:35:23 -0500

Andy Wingo agreed on IRC to accepting UTC, since we've since discovered
that not only various web servers, but also some transparent web proxies
put UTC in the date header, which was causing problems for many people.

So, I've pushed a change to stable-2.0 to accept UTC.

http://git.savannah.gnu.org/gitweb/?p=guile.git;a=commit;h=8904b7a9362ede0de324e55701d8ea1d018f8738

I'm closing this bug now.  Thanks for the report!

      Mark

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.