GNU bug report logs - #20699
subdir-objects with source from sibling directory breaks distcheck

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 20699 in the body.
You can then email your comments to 20699 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Hans-Bernhard Bröker <HBBroeker <at> t-online.de>
To: bug-automake <at> gnu.org
Subject: subdir-objects with source from sibling directory breaks distcheck
Date: Sat, 30 May 2015 14:12:38 +0200

[Message part 1 (text/plain, inline)]

Hello everyone,

I'm just about to turn on option subdir-objects in project gnuplot, but 
ran in to a snag.  Make distcheck now fails in the distclean phase 
because distclean in one subdirectory removes dependency files that 
make, in another one, still needs.

The trigger was this in docs/Makefile.am:

doc2wxhtml_SOURCES = windows/doc2html.c ../src/version.c
doc2wxhtml_CPPFLAGS = -DALL_TERM_DOC -DWXHELP -I../src $(AM_CPPFLAGS)

The reference to windows/doc2html.c was what had me turn on 
subdir-objects.  The reference to a source file from a neighboring 
../src directory triggers the breakage.  All is well in typical 
day-to-day operation, but since the distclean phase of a make distcheck 
kills src/.deps before visiting docs, make in docs now dies:

Making distclean in docs
Makefile:592: ../src/.deps/doc2wxhtml-version.Po: No such file or directory

Tool versions: automake-1.14.1 and 1.15, autoconf-2.69, on current Cygwin.

I've attached a copy of amhello modified to trigger the bug.

[amhello-1.0.tar.gz (application/x-gzip, attachment)]

Message #8 received at 20699 <at> debbugs.gnu.org (full text, mbox):

From: Hans-Bernhard Bröker <HBBroeker <at> t-online.de>
To: 20699 <at> debbugs.gnu.org
Subject: Re: bug#20699: Acknowledgement (subdir-objects with source from
 sibling directory breaks distcheck)
Date: Sun, 21 Jun 2015 23:14:32 +0200

It's actually even a little worse:

Any dependency on sources in another directory causes a simple "make 
clean" in one directory to erase _all_ object files on that other one, 
i.e. if docs/Makefile.am has

noinst_PROGRAMS = a

a_SOURCES = sub/a.c ../src/main.c
a_CPPFLAGS = -DFOO=BAR $(AM_CPPFLAGS)

then "make -C docs clean" does the following:

make: Entering directory '/cygdrive/c/prg/gp/am_bug/amhello-1.0/docs'
test -z "a.exe" || rm -f a.exe
rm -f *.o
rm -f ../src/*.o
rm -f sub/*.o
make: Leaving directory '/cygdrive/c/prg/gp/am_bug/amhello-1.0/docs'

But what logic would a "clean" in one sub project erase all the work 
done in another?

With consequences like that, I think the (all but forced, now) option 
"subdir-objects" needs to be reconsidered.

Message #11 received at 20699 <at> debbugs.gnu.org (full text, mbox):

From: Peter Rosin <peda <at> lysator.liu.se>
To: Hans-Bernhard Bröker <HBBroeker <at> t-online.de>, 
 20699 <at> debbugs.gnu.org
Subject: Re: bug#20699: Acknowledgement (subdir-objects with source from
 sibling directory breaks distcheck)
Date: Tue, 23 Jun 2015 00:05:18 +0200

On 2015-06-21 23:14, Hans-Bernhard Bröker wrote:
> It's actually even a little worse:
> 
> Any dependency on sources in another directory causes a simple "make clean" in one directory to erase _all_ object files on that other one, i.e. if docs/Makefile.am has
> 
> noinst_PROGRAMS = a
> 
> a_SOURCES = sub/a.c ../src/main.c
> a_CPPFLAGS = -DFOO=BAR $(AM_CPPFLAGS)
> 
> then "make -C docs clean" does the following:
> 
> make: Entering directory '/cygdrive/c/prg/gp/am_bug/amhello-1.0/docs'
> test -z "a.exe" || rm -f a.exe
> rm -f *.o
> rm -f ../src/*.o
> rm -f sub/*.o
> make: Leaving directory '/cygdrive/c/prg/gp/am_bug/amhello-1.0/docs'
> 
> But what logic would a "clean" in one sub project erase all the work done in another?
> 
> With consequences like that, I think the (all but forced, now) option "subdir-objects" needs to be reconsidered.

I think that in Automake it is implicit that a makefile controls the
directory of the makefile itself and possibly subdirectories of that
directory. Sibling directories of this main directory are supposed to be
controlled by a makefile in that sibling directory (or by a makefile in
a parent directory). Notice how the option is named *subdir*-objects
(and not *sibling*-objects or something such) and ../foo is hardly a
subdir.

You are perhaps better off converting the project to use a single
top level makefile instead of using a recursive build.

Cheers,
Peter

Message #14 received at 20699 <at> debbugs.gnu.org (full text, mbox):

From: Hans-Bernhard Bröker <HBBroeker <at> t-online.de>
To: Peter Rosin <peda <at> lysator.liu.se>, 20699 <at> debbugs.gnu.org
Subject: Re: bug#20699: Acknowledgement (subdir-objects with source from
 sibling directory breaks distcheck)
Date: Tue, 23 Jun 2015 01:03:20 +0200

Am 23.06.2015 um 00:05 schrieb Peter Rosin:
> On 2015-06-21 23:14, Hans-Bernhard Bröker wrote:
>> It's actually even a little worse:
>>
>> Any dependency on sources in another directory causes a simple "make clean" in one directory to erase _all_ object files on that other one, i.e. if docs/Makefile.am has
[...]
>> With consequences like that, I think the (all but forced, now) option "subdir-objects" needs to be reconsidered.

> Sibling directories of this main directory are supposed to be
> controlled by a makefile in that sibling directory

They are. In the project this occurred in, ./src (and subdirs therein) 
holds the the main program, while ./docs is contains the master 
documentation and tools to combine it with documentation fragments from 
multiple source code files, and convert that to multiple output formats.

Now one of the documentation builders wants to refer to the compiled 
version information, which is with the main source.

> (or by a makefile in
> a parent directory). Notice how the option is named *subdir*-objects
> (and not *sibling*-objects or something such) and ../foo is hardly a
> subdir.

Agreed, it's not.  But then why does option subdir-objects even affect 
the way it's treated?

Without it, the problematic rm ../src/*.o isn't generated, and the tool 
complains that the subdir-objects is about to be forced on me soon. 
That doesn't bode well.

> You are perhaps better off converting the project to use a single
> top level makefile instead of using a recursive build.

That would be rather a long stretch.  There's a lot more recursion than 
this, some of it conditional, and it's worked reasonably well now for 
decades.  Converting all that into a single top-level makefile would be 
quite a painful workaround.

Message #17 received at 20699 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: HBBroeker <at> t-online.de, 20699 <at> debbugs.gnu.org
Subject: Re: bug#37499: Wrong location of deps directory when using AC_SUBST
Date: Sat, 28 Nov 2020 19:46:22 -0700

Hi Hans-Bernhard - you wrote recently, in connection with bug#37499:

    This may actually be related to my ancient bug entry #20699.

Comparing this bug (https://bugs.gnu.org/20699) with 37499, yeah,
perhaps they are related, but I tend to think not, since the OP in 37499
is not using subdir-objects, which seems to be the key point here.

Looking at 20699, as usual, I'm confused. Sorry. It seems like Automake
is just following instructions. The docs/Makefile.am declares a
dependency on ../src/main.c, so it removes ../src/*.o. How could it
guess otherwise? I feel like I'm missing some basic point.

And, even if it was refined to remove only ../src/main.o (and using *.o
is surely intentional, so I'd be disinclined to make such a change even
if I could figure out how), I gather that wouldn't solve your problem.

It does not seem feasible or desirable to somehow compute that
./src/whatever is not technically a "subdirectory" and therefore have
subdir-objects do nothing with those files. And only for the clean
targets? I can't wrap my head around this.

Let me step back and ask more generally: Is there some behavior in
current Automake (1.16.3) which is affecting current Gnuplot adversely?
If there is a practical problem that can be fixed, I want to fix it, but
making sweeping changes to how subdir-objects behaves doesn't sound like
a good idea.

As for your remark:

  the tool complains that the subdir-objects is about to be forced on me
  soon. That doesn't bode well.

Well, I agree, but:

1) what tool? Automake? And "forced" how?

2) as far as I'm concerned, Automake is not going to "force" anything
new on anyone in the future. I deplore changes that gratuitously break
code that has worked perfectly well for many years, and don't want to
make any such changes. So if current automake works well enough for you,
it shouldn't break in the future. At least until someone other than me
starts working on it.

Best regards,
Karl

Message #20 received at 20699 <at> debbugs.gnu.org (full text, mbox):

From: Hans-Bernhard Bröker <HBBroeker <at> t-online.de>
To: Karl Berry <karl <at> freefriends.org>, 20699 <at> debbugs.gnu.org
Subject: Re: bug#20699: bug#37499: Wrong location of deps directory when using
 AC_SUBST
Date: Wed, 2 Dec 2020 02:10:40 +0100

Am 29.11.2020 um 03:46 schrieb Karl Berry:

> Looking at 20699, as usual, I'm confused. Sorry. It seems like Automake
> is just following instructions. The docs/Makefile.am declares a
> dependency on ../src/main.c, so it removes ../src/*.o. How could it
> guess otherwise? 

Guessing shouldn't be involved.  Either source files like this, which 
lie outside the subtree of the current Makefile.am, should either be 
documented as frowned upon, or they must be treated correctly.  To be 
treated correctly, IMHO they must avoid breaking the distcheck target.

If that is impossible, that may mean the subdir-objects option is not 
ready to be made the default, much less a non-option.

> It does not seem feasible or desirable to somehow compute that
> ./src/whatever is not technically a "subdirectory" and therefore have
> subdir-objects do nothing with those files. And only for the clean
> targets? I can't wrap my head around this.

Maybe sources from directories that are not, in fact, subdirs should 
simply be exempt from subdir-objects handling.  Or if they do get that 
handling, then their object files would have to be put into cleverly 
named artificial subdirs of the individual build dir.  From there they 
can be cleaned without affecting other SUBDIRS of the main project.

> Let me step back and ask more generally: Is there some behavior in
> current Automake (1.16.3) which is affecting current Gnuplot adversely?

Not any more.  We had to work around it, and mangaed to do so by 
changing the method of version number provisioning from a generated 
source file to AC_SUBST()ed preprocessor definitions.

>    the tool complains that the subdir-objects is about to be forced on me
>    soon. That doesn't bode well.
> 
> Well, I agree, but:
> 
> 1) what tool? Automake? And "forced" how?

If I try to turn off subdir-objects right now, while there are some 
other, genuine subdir source dependencies in some Makefile.am, Automake 
1.16.1 tells me this:

automake-1.16: warning: possible forward-incompatibility.
automake-1.16: At least a source file is in a subdirectory, but the 
'subdir-objects'
automake-1.16: automake option hasn't been enabled.  For now, the 
corresponding output
automake-1.16: object file(s) will be placed in the top-level directory. 
 However,
automake-1.16: this behaviour will change in future Automake versions: 
they will
automake-1.16: unconditionally cause object files to be placed in the 
same subdirectory
automake-1.16: of the corresponding sources.
automake-1.16: You are advised to start using 'subdir-objects' option 
throughout your
automake-1.16: project, to avoid future incompatibilities.

That says "will change" rather than "may change", and it says 
"unconditionally".  To me, that did read "forced on me, soon."  I just 
over-estimated the "soon" aspect by a couple of years.

Message #23 received at 20699 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: HBBroeker <at> t-online.de
Cc: 20699 <at> debbugs.gnu.org
Subject: Re: bug#20699: bug#37499: Wrong location of deps directory when using
 AC_SUBST
Date: Thu, 3 Dec 2020 15:03:59 -0700

[Message part 1 (text/plain, inline)]

    Either source files like this, which lie outside the subtree of the
    current Makefile.am, should either be documented as frowned upon, or
    they must be treated correctly.  To be treated correctly, IMHO they
    must avoid breaking the distcheck target.

I agree. I added a caveat to the manual about the current behavior.

    If that is impossible, that may mean the subdir-objects option is not 
    ready to be made the default, much less a non-option.

I agree. I made the warning less forceful ("may", deleted "unconditionally").

    Maybe sources from directories that are not, in fact, subdirs should 
    simply be exempt from subdir-objects handling.  

That does seem like it would have been a cleaner implementation. But
since it was not done that way originally, I do not want to change the
existing behavior at this late date. Experience teaches that any change
of longstanding behavior will likely cause problems in some existing
projects, even while improving others.

    > affecting current Gnuplot adversely?
    Not any more.  

I'm glad to know you were able to work around it. I'm sorry it was necessary.

    That says "will change" rather than "may change", and it says 
    "unconditionally".  To me, that did read "forced on me, soon." 

You're right that that was the implication. Looking at the comments, the
change was/is intended for Automake 2.0, though. It is not surprising it
did not happen quickly.

At this point, Jim and I are basically the only people making changes to
Automake, and neither I nor (as far as I understand it) he have time or
inclination to do substantial development.  We just want to fix reported
bugs, to keep Automake alive.

Besides changing the warning text, I left a bug reference and comment in
the source about this change to unconditional subdir-objects being
problematic, for the benefit of future developments. (Personally, I
would never make such a change "unconditionally"; there should always be
a way to preserve existing behavior.)

I can't think of more to do here at present, so I'm tentatively closing
this bug. Feel free to send more if desired.

Thanks for all the info and patiently explaining the problem about
sixteen times :).

Karl

[subobj.patch (application/octet-stream, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.