GNU bug report logs - #8357
Improve yesterday and tomorrow calculations for common cases

Previous Next

To reply to this bug, email your comments to 8357 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Heikki Orsila <shd <at> modeemi.fi>
To: bug-coreutils <at> gnu.org
Subject: coreutils bug in date
Date: Mon, 28 Mar 2011 00:43:54 +0300

Hello, I found a case where GNU date works pretty oddly.
At 2011-03-27 Finland moved to 'summer time'. After midnight at 2011-03-28
our web site's cron script did (date -I -d yesterday) which yielded
unexpected results:

$ date
Mon Mar 28 00:36:07 EEST 2011
$ date -I -d yesterday
2011-03-26
$ date -I -d '24 hours ago'
2011-03-26

Our scripts produced bad results due to this "feature/bug". Maybe our fault, 
but this is definitely unexpected behaviour.

-- 
Heikki Orsila
heikki.orsila <at> iki.fi
http://www.iki.fi/shd

Message #8 received at 8357 <at> debbugs.gnu.org (full text, mbox):

From: Bob Proulx <bob <at> proulx.com>
To: Heikki Orsila <shd <at> modeemi.fi>
Cc: 8357 <at> debbugs.gnu.org
Subject: Re: bug#8357: coreutils bug in date
Date: Sun, 27 Mar 2011 18:04:37 -0600

tags 8357 + moreinfo notabug
thanks

Heikki Orsila wrote:
> Hello, I found a case where GNU date works pretty oddly.
> At 2011-03-27 Finland moved to 'summer time'. After midnight at 2011-03-28
> our web site's cron script did (date -I -d yesterday) which yielded
> unexpected results:
> 
> $ date
> Mon Mar 28 00:36:07 EEST 2011
> $ date -I -d yesterday
> 2011-03-26
> $ date -I -d '24 hours ago'
> 2011-03-26
> 
> Our scripts produced bad results due to this "feature/bug". Maybe our fault, 
> but this is definitely unexpected behaviour.

Thank you for your bug report.  However I this is not a bug in date
but a misunderstanding of how relative times and DST interact.

Note that 'date -I' is a deprecated option.  Please consider updating
the syntax.  In a future release -I may be removed.

First, let's understand when the time changes.

  $ zdump -v Europe/Helsinki | grep 2011
  Europe/Helsinki  Sun Mar 27 00:59:59 2011 UTC = Sun Mar 27 02:59:59 2011 EET isdst=0 gmtoff=7200
  Europe/Helsinki  Sun Mar 27 01:00:00 2011 UTC = Sun Mar 27 04:00:00 2011 EEST isdst=1 gmtoff=10800
  Europe/Helsinki  Sun Oct 30 00:59:59 2011 UTC = Sun Oct 30 03:59:59 2011 EEST isdst=1 gmtoff=10800
  Europe/Helsinki  Sun Oct 30 01:00:00 2011 UTC = Sun Oct 30 03:00:00 2011 EET isdst=0 gmtoff=7200

Sun Mar 27 02:59:59 2011 EET was the last second in EET and Sun Mar 27
04:00:00 2011 EEST was the very next second in your timezone.  There
is no 03:00:00 through 03:59:59.  The time skips (by law and politics)
directly to 04:00:00 in that timezone.  The day is defined to be one
hour shorter that day.

The documentation for date says:

     The unit of time displacement may be selected by the string `year'
  or `month' for moving by whole years or months.  These are fuzzy units,
  as years and months are not all of equal duration.  More precise units
  are `fortnight' which is worth 14 days, `week' worth 7 days, `day'
  worth 24 hours, `hour' worth 60 minutes, `minute' or `min' worth 60
  seconds, and `second' or `sec' worth one second.  An `s' suffix on
  these units is accepted and ignored.

     The string `tomorrow' is worth one day in the future (equivalent to
  `day'), the string `yesterday' is worth one day in the past (equivalent
  to `day ago').

So 'yesterday' or '24 hours ago' are equivalent.  These are exactly 24
hours ago or 86400 seconds.  This is a critical point in your problem.

  $ TZ=Europe/Helsinki date -d 'Mon Mar 28 00:36:07 EEST 2011'
  Mon Mar 28 00:36:07 EEST 2011

  $ TZ=Europe/Helsinki date -d 'Mon Mar 28 00:36:07 EEST 2011 24 hours ago'
  Sat Mar 26 23:36:07 EET 2011

This recreates the same results you posted.  The reasoning is a little
more obvious if I list out the hours explicitly.

  Mon Mar 28 00:36:07 EEST 2011   -0 hours
  Sun Mar 27 23:36:07 EEST 2011   -1 hours
  Sun Mar 27 22:36:07 EEST 2011   -2 hours
  Sun Mar 27 21:36:07 EEST 2011   -3 hours
  Sun Mar 27 20:36:07 EEST 2011   -4 hours
  Sun Mar 27 19:36:07 EEST 2011   -5 hours
  Sun Mar 27 18:36:07 EEST 2011   -6 hours
  Sun Mar 27 17:36:07 EEST 2011   -7 hours
  Sun Mar 27 16:36:07 EEST 2011   -8 hours
  Sun Mar 27 15:36:07 EEST 2011   -9 hours
  Sun Mar 27 14:36:07 EEST 2011  -10 hours
  Sun Mar 27 13:36:07 EEST 2011  -11 hours
  Sun Mar 27 12:36:07 EEST 2011  -12 hours
  Sun Mar 27 11:36:07 EEST 2011  -13 hours
  Sun Mar 27 10:36:07 EEST 2011  -14 hours
  Sun Mar 27 09:36:07 EEST 2011  -15 hours
  Sun Mar 27 08:36:07 EEST 2011  -16 hours
  Sun Mar 27 07:36:07 EEST 2011  -17 hours
  Sun Mar 27 06:36:07 EEST 2011  -18 hours
  Sun Mar 27 05:36:07 EEST 2011  -19 hours
  Sun Mar 27 04:36:07 EEST 2011  -20 hours  <-- just after change
  Sun Mar 27 02:36:07 EET 2011   -21 hours  <-- just before change
  Sun Mar 27 01:36:07 EET 2011   -22 hours
  Sun Mar 27 00:36:07 EET 2011   -23 hours
  Sat Mar 26 23:36:07 EET 2011   -24 hours

Observe that there is no 03:00:00-03:59:59 and March 27 is only 23
hours long that day.  It is shorter than 24 hours.  Using a 24 hour
calculation near midnight skips over that day completely.

With this table it should be obvious that date is behaving correctly
and the day that is shorter than 24 hours due to the conversion to DST
has caused -24 hours to skip completely over it.

Please see this FAQ entry for detailed explanation and hints on how to
avoid this problem.

  http://www.gnu.org/software/coreutils/faq/#The-date-command-is-not-working-right_002e

Instead of doing calculations around DST I suggest using calculations
around noon.  I am not aware of any location that changes DST at noon
which makes it a much safer time to do date calculations.  The FAQ
entry provides examples.  Or if possible use UTC which also avoids the
problem by never switching to DST.

Bob

Message #13 received at 8357 <at> debbugs.gnu.org (full text, mbox):

From: Heikki Orsila <shd <at> modeemi.fi>
To: Bob Proulx <bob <at> proulx.com>
Cc: 8357 <at> debbugs.gnu.org
Subject: Re: bug#8357: coreutils bug in date
Date: Mon, 28 Mar 2011 09:27:28 +0300

On Sun, Mar 27, 2011 at 06:04:37PM -0600, Bob Proulx wrote:
> Thank you for your bug report.  However I this is not a bug in date
> but a misunderstanding of how relative times and DST interact.
> ...
>      The string `tomorrow' is worth one day in the future (equivalent to
>   `day'), the string `yesterday' is worth one day in the past (equivalent
>   to `day ago').
> 
> So 'yesterday' or '24 hours ago' are equivalent.  These are exactly 24
> hours ago or 86400 seconds.  This is a critical point in your problem.

I see the problem, but IMO, the definition of yesterday should be changed
or there should be a new definition.

What about defining new units: 'calday', 'calyesterday', 'caltomorrow'? 
For example, $(date -d '1 calday ago' '+%F')? These would count actual 
calendar days rather than relative time (in units of 24 hours)?

Yes I know there is a work around in the doc you provided (thanks for that).

Heikki Orsila

Message #16 received at 8357 <at> debbugs.gnu.org (full text, mbox):

From: Bob Proulx <bob <at> proulx.com>
To: Heikki Orsila <shd <at> modeemi.fi>
Cc: 8357 <at> debbugs.gnu.org
Subject: Re: bug#8357: coreutils bug in date
Date: Tue, 29 Mar 2011 14:38:08 -0600

Heikki Orsila wrote:
> Bob Proulx wrote:
> > So 'yesterday' or '24 hours ago' are equivalent.  These are exactly 24
> > hours ago or 86400 seconds.  This is a critical point in your problem.
> 
> I see the problem, but IMO, the definition of yesterday should be changed
> or there should be a new definition.
> 
> What about defining new units: 'calday', 'calyesterday', 'caltomorrow'? 
> For example, $(date -d '1 calday ago' '+%F')? These would count actual 
> calendar days rather than relative time (in units of 24 hours)?

Something like that could be done.  I think there would be support for
that direction of development.

That is the problem when people put in features such as the relative
date calculations.  A simple but only partially implemented feature
sounds great but later causes an endless stream of problems.  They add
a partially nice feature but it isn't 100% perfect and then ten years
go by and people start to use it and then find the rough spots.  But
by then it is now a very long standing behavior and can't be changed.
Your suggestion to use different names avoids breaking the previous
behavior.  But still someone would need to actually do the work and
the code is not simple.  Working with human produced calender dates
has many traps and pitfalls.

Bob

Message #19 received at 8357 <at> debbugs.gnu.org (full text, mbox):

From: Bob Proulx <bob <at> proulx.com>
To: Heikki Orsila <shd <at> modeemi.fi>
Cc: 8357 <at> debbugs.gnu.org
Subject: Re: bug#8357: coreutils bug in date
Date: Sat, 9 Apr 2011 21:49:29 -0600

retitle 8357 Improve yesterday and tomorrow calculations for common cases
severity 8357 wishlist
thanks

Bob Proulx wrote:
> Heikki Orsila wrote:
> > Bob Proulx wrote:
> > > So 'yesterday' or '24 hours ago' are equivalent.  These are exactly 24
> > > hours ago or 86400 seconds.  This is a critical point in your problem.
> > 
> > I see the problem, but IMO, the definition of yesterday should be changed
> > or there should be a new definition.
> > 
> > What about defining new units: 'calday', 'calyesterday', 'caltomorrow'? 
> > For example, $(date -d '1 calday ago' '+%F')? These would count actual 
> > calendar days rather than relative time (in units of 24 hours)?
> 
> Something like that could be done.  I think there would be support for
> that direction of development.

There hasn't been any further discussion about the topic.  I don't
think this is going to be changed immediately.  It would need to be
discussed and an acceptable solution proposed and also someone would
need to actually do the work.  And so am reclassifying this ticket as
a wishlist to improve the date handling for the common cases of
yesterday and tomorrow just to document the issue.

Bob

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.