GNU bug report logs - #30795
date: change default time from 12am for better DST calc

Previous Next

To reply to this bug, email your comments to 30795 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jared Chagnon <jchagnon <at> enservio.com>
To: "bug-coreutils <at> gnu.org" <bug-coreutils <at> gnu.org>
Subject: Re: Issue with date command and EDT
Date: Tue, 13 Mar 2018 15:20:54 +0000

[Message part 1 (text/plain, inline)]

Forgot to include this:

OS Release:
Red Hat Enterprise Linux Server release 6.9 (Santiago)

Packages installed:
tzdata-2018c-1.el6.noarch
coreutils-8.4-46.0.1.el6.x86_64
coreutils-libs-8.4-46.0.1.el6.x86_64

Jared Chagnon
Systems Engineering Manager
P:  781.292.6033
M:  508.566.0081
<mailto:jchagnon <at> enservio.com>jchagnon <at> enservio.com<mailto:jchagnon <at> enservio.com>
www.enservio.com<http://www.enservio.com/>

[cid:A7AEE8A5-9D9D-44B8-B77C-BE36F5D6A2C5 <at> enservio.lan]

On Mar 13, 2018, at 9:10 AM, Jared Chagnon <jchagnon <at> enservio.com<mailto:jchagnon <at> enservio.com>> wrote:

Please see the issue below:

I setup a test script:

#/bin/bash

echo "`date`"
echo "`date +%z`"

currentdate=`date +%Y%m%d_%H%M%S`
echo "current date: $currentdate"

olddate=`date "+%Y%m%d_%H%M%S" --date='4 days ago'`
echo "old date 4 days ago: $olddate”

————— Output —————

./datetest.sh
Tue Mar 13 09:08:34 EDT 2018
-0400
current date: 20180313_090834
old date 4 days ago: 20180309_080834

————— Notes ——————

My local time zone is America\New_York.  Please note the date modification is not using EDT and resulting in EST.

Jared Chagnon
Systems Engineering Manager
P:  781.292.6033
M:  508.566.0081
<mailto:jchagnon <at> enservio.com>jchagnon <at> enservio.com<mailto:jchagnon <at> enservio.com>
www.enservio.com<http://www.enservio.com/>

[cid:A7AEE8A5-9D9D-44B8-B77C-BE36F5D6A2C5 <at> enservio.lan]

[Message part 2 (text/html, inline)]

[image001.png (image/png, inline)]

Message #8 received at 30795 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Jared Chagnon <jchagnon <at> enservio.com>, 30795 <at> debbugs.gnu.org
Subject: Re: bug#30795: Issue with date command and EDT
Date: Tue, 13 Mar 2018 19:42:59 -0600

Hello,

On 2018-03-13 09:20 AM, Jared Chagnon wrote:
> I setup a test script:
> 
> #/bin/bash
> 
> echo "`date`"
> echo "`date +%z`"
> 
> currentdate=`date +%Y%m%d_%H%M%S`
> echo "current date: $currentdate"
> 
> olddate=`date "+%Y%m%d_%H%M%S" --date='4 days ago'`
> echo "old date 4 days ago: $olddate”

An unrelated suggestion:
There's no need to use backticks to execute date and return the value
and then echo it. Simply running 'date' will print the date to STDOUT.
e.g.

  date
  date +%z
  printf "current date: "
  date +%Y%m%d_%H%M%S

or even:

  date +"Current Date: %Y%m%d_%H%M%S"

> ————— Output —————
> 
> ./datetest.sh
> Tue Mar 13 09:08:34 EDT 2018
> -0400
> current date: 20180313_090834
> old date 4 days ago: 20180309_080834
> 
> ————— Notes ——————
> 
> My local time zone is America\New_York.  Please note the date modification is not using EDT and resulting in EST.

First,
lets verify our baseline assumptions about New York time zones
are the same:

EST = Eastern Standard Time, used in autumn/winter = UTC-05:00.
EDT = Eastern Daylight Time, used in spring/summer = UTC-04:00.

In 2018, daylight saving time (DST) started on March 11th, 2:am.
"started" here means switching from EST to EDT.

That is:
  2018-03-09 is EST (still winter, DST not used).
  2018-03-13 is EDT (summer time, DST used).

This can be verified with:

  $ TZ="America/New_York" date +%F-%Z -d '2018-03-09'
  2018-03-09-EST
  $ TZ="America/New_York" date +%F-%Z -d '2018-03-13'
  2018-03-13-EDT


Second,
in your example, the current date is 2018-03-13 (and implied: EDT),
and subtracting 4 days gives 2018-03-09.
Due to your time zone, 'date' knows that your timezone does use DST.
This date adjustment also crossed the DST change date (march 11th),
so date correctly adjusted the clock one hour backwards (from
09:08:34 to 08:08:34.


Third,
in newer version we've added a "--debug" option that helps
diagnosing such issues:

  $ TZ="America/New_York" \
     date --debug +%Y%m%d_%H%M%S_%Z --date="4 days ago"
  date: parsed relative part: -4 day(s)
  date: input timezone: TZ="America/New_York" environment value
  date: using current time as starting value: '21:13:23'
  date: using current date as starting value: '(Y-M-D) 2018-03-13'
  date: starting date/time: '(Y-M-D) 2018-03-13 21:13:23'
  date: warning: when adding relative days, it is recommended to \
            specify noon
  date: after date adjustment (+0 years, +0 months, -4 days),
  date:     new date/time = '(Y-M-D) 2018-03-09 20:13:23'
->date: warning: daylight saving time changed after date adjustment <-
  date: '(Y-M-D) 2018-03-09 20:13:23' = 1520644403 epoch-seconds
  date: timezone: TZ="America/New_York" environment value
  date: final: 1520644403.136862048 (epoch-seconds)
  date: final: (Y-M-D) 2018-03-10 01:13:23 (UTC)
  date: final: (Y-M-D) 2018-03-09 20:13:23 (UTC-05)
  20180309_201323_EST



Fourth,
There is a subtle issue of how 'date' knows when to do DST adjustments
(and when not to).

When using 'date' without specifying the current date/time, it
gets the date,time and time zone from your operating system.

In your case it was "America/New_York". 'date' then checked the timezone
database and so it knows that NY does use daylight saving time,
and it knows that currently it is EDT, will subtract an hour when 
switching to EST.

When giving 'date' an explicit date without time zone, it doesn't
know if DST should be used or not - and assumes it should not be used.
Therefore it will not subtract an hour.

Observe the following:

  $ date +%F
  2018-03-13

  # Without timezone - hour stays the same:
  $ date -d '2018-03-13 09:00:00 4 days ago'
  Fri Mar  9 09:00:00 EST 2018

  # With timezone that indicates DST - hour is adjusted:
  $ date -d '2018-03-13 09:00:00 EDT 4 days ago'
  Fri Mar  9 08:00:00 EST 2018

These issues can be troubleshooted using the "--debug" option.



Fifth,
There is also one additional nuance:
If you don't specify the hour, 'date' assumes midnight.
If DST is active, it will subtract an hour resulting in 11pm
in the previous day:

  $ date -d '2018-03-13 4 days ago'
  Fri Mar  9 00:00:00 EST 2018

  $ date -d '2018-03-13 EDT 4 days ago'
  Thu Mar  8 23:00:00 EST 2018

This will be very confusing if one prints only the date, eg:

  $ date -d '2018-03-13 4 days ago' +%F
  2018-03-09

  $ date -d '2018-03-13 EDT 4 days ago' +%F
  2018-03-08

Therefore it is always recommended to use noon (12pm)
as explicit time when adjusting days (and when using --debug
there will be a warning to that effect).








If this explanation solves the issue - great.
If not, please provide further details and examples of the commands and
output you expect.

regards,
 - assaf

Message #11 received at 30795 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Assaf Gordon <assafgordon <at> gmail.com>,
 Jared Chagnon <jchagnon <at> enservio.com>, 30795 <at> debbugs.gnu.org
Subject: Re: bug#30795: Issue with date command and EDT
Date: Wed, 14 Mar 2018 17:22:04 -0700

On 03/13/2018 06:42 PM, Assaf Gordon wrote:
> Therefore it is always recommended to use noon (12pm)
> as explicit time when adjusting days

Maybe "date" should default to 12:00 instead of to 00:00 when the time 
is not specified? That would avoid this sort of problem, typically.

Message #14 received at 30795 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Jared Chagnon <jchagnon <at> enservio.com>, 30795 <at> debbugs.gnu.org
Subject: Re: bug#30795: Issue with date command and EDT
Date: Thu, 15 Mar 2018 01:15:28 -0600

[Message part 1 (text/plain, inline)]

Hello,

On Wed, Mar 14, 2018 at 05:22:04PM -0700, Paul Eggert wrote:
> On 03/13/2018 06:42 PM, Assaf Gordon wrote:
> >Therefore it is always recommended to use noon (12pm)
> >as explicit time when adjusting days
> 
> Maybe "date" should default to 12:00 instead of to 00:00 when the time is
> not specified? That would avoid this sort of problem, typically.

Technically it's an easy fix (patch attached),
but it changes a long-standing behavior.

I wonder if it will break some existing scripts that might rely
on being 'midnight'? (even implicitly, because the user isn't aware of
this nuance).

For example, currently '2018-03-15 + 14 hours' is 2pm on March 15th.
With this change, it'll result in 2am on March 16th.

What do you think?

-assaf

[0001-parse-datetime-default-to-noon-for-dates-without-tim.patch (text/x-diff, attachment)]

Message #17 received at 30795 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Assaf Gordon <assafgordon <at> gmail.com>, Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Jared Chagnon <jchagnon <at> enservio.com>, 30795 <at> debbugs.gnu.org
Subject: Re: bug#30795: Issue with date command and EDT
Date: Thu, 15 Mar 2018 04:18:03 -0700

On 15/03/18 00:15, Assaf Gordon wrote:
> Hello,
> 
> On Wed, Mar 14, 2018 at 05:22:04PM -0700, Paul Eggert wrote:
>> On 03/13/2018 06:42 PM, Assaf Gordon wrote:
>>> Therefore it is always recommended to use noon (12pm)
>>> as explicit time when adjusting days
>>
>> Maybe "date" should default to 12:00 instead of to 00:00 when the time is
>> not specified? That would avoid this sort of problem, typically.
> 
> Technically it's an easy fix (patch attached),
> but it changes a long-standing behavior.
> 
> I wonder if it will break some existing scripts that might rely
> on being 'midnight'? (even implicitly, because the user isn't aware of
> this nuance).
> 
> For example, currently '2018-03-15 + 14 hours' is 2pm on March 15th.
> With this change, it'll result in 2am on March 16th.
> 
> What do you think?

Yes picking 12:00 always is too simplistic I think.
The base should be dependent on the relative unit.
I proposed a solution previously at: http://bugs.gnu.org/18159#8

See also https://bugs.gnu.org/11101

cheers,
Pádraig

Message #20 received at 30795 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Assaf Gordon <assafgordon <at> gmail.com>
Cc: Jared Chagnon <jchagnon <at> enservio.com>, 30795 <at> debbugs.gnu.org
Subject: Re: bug#30795: Issue with date command and EDT
Date: Thu, 15 Mar 2018 15:33:10 -0700

On 03/15/2018 12:15 AM, Assaf Gordon wrote:
> Technically it's an easy fix (patch attached),
> but it changes a long-standing behavior.

Yes, that's a problem. Perhaps we should take the last unit requested in 
the output format, divide that by two, and add that to the time instead. 
If the output format is %Y-%m-%d the output unit is one day, so we'd add 
12 hours. With the default output format we'd add only 0.5 s, which 
would be a no-op.

This isn't as fancy as what Pádraig is proposing, but it should be easy 
to implement.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.