GNU bug report logs - #18301
texi-vers.am: Problem with parallel builds due to vti.tmp

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 18301 in the body.
You can then email your comments to 18301 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Friedrich Beckmann <friedrich.beckmann <at> gmx.de>
To: bug-automake <at> gnu.org
Subject: texi-vers.am: Problem with parallel builds due to vti.tmp
Date: Wed, 20 Aug 2014 11:17:59 +0200

[Message part 1 (text/plain, inline)]

Package: automake
Version: 1.11.6-1

I encounter a problem with automake during parallel builds for the debian pspp package. When I run the build
process non-parallel then it works. The problem only occurs sometimes when I do a parallel build. The build
is a debian package build. Here are the relevant sections from the build.log and Makefile.in: 

======== snippet from pspp_0.8.3-3_amd64.build ============
make[4]: Entering directory `/home/fritz/pspp/new/pspp-0.8.3'
make[4]: »doc/.dirstamp« ist bereits aktualisiert.
make[4]: Leaving directory `/home/fritz/pspp/new/pspp-0.8.3'
Updating ./doc/version.texi
Updating ./doc/version.texi
cp: Aufruf von stat für „vti.tmp“ nicht möglich: Datei oder Verzeichnis nicht gefunden
make[3]: *** [doc/stamp-vti] Fehler 1
make[3]: Leaving directory `/home/fritz/pspp/new/pspp-0.8.3'
make[2]: *** [pdf-recursive] Fehler 1
make[2]: *** Warte auf noch nicht beendete Prozesse…
================================================================

The build log shows „Updating ./doc/version.texi“ two times. This message is generated when
the stamp-vti target is build. So obviously the parallel build results in two parallel processes to build
stamp-vti. 

================ Relevant section from Makefile.in ======================
$(srcdir)/doc/version.texi:  $(srcdir)/doc/stamp-vti
$(srcdir)/doc/stamp-vti: doc/pspp.texinfo $(top_srcdir)/configure
        test -f doc/$(am__dirstamp) || $(MAKE) $(AM_MAKEFLAGS) doc/$(am__dirstamp)
        @(dir=.; test -f ./doc/pspp.texinfo || dir=$(srcdir); \
        set `$(SHELL) $(top_srcdir)/build-aux/mdate-sh $$dir/doc/pspp.texinfo`; \
        echo "@set UPDATED $$1 $$2 $$3"; \
        echo "@set UPDATED-MONTH $$2 $$3"; \
        echo "@set EDITION $(VERSION)"; \
        echo "@set VERSION $(VERSION)") > vti.tmp
        @cmp -s vti.tmp $(srcdir)/doc/version.texi \
          || (echo "Updating $(srcdir)/doc/version.texi"; \
              cp vti.tmp $(srcdir)/doc/version.texi)
        -@rm -f vti.tmp
================================================================

I think the problem is due to the file „vti.tmp“ which is generated and removed during the build. When
one process has already deleted the file and the other process wants to delete it, then the error message
in the build log: "cp: Aufruf von stat für „vti.tmp“ nicht möglich: Datei oder Verzeichnis nicht gefunden“
will show up. This means that the file vti.tmp is not found for the copy action.

A possible solution could be based on unique temporary files, instead of vti.tmp.  The relevant automake file
is http://git.savannah.gnu.org/cgit/automake.git/tree/lib/am/texi-vers.am. I think the relevant part of the file
did not change since version 1.11.6. Attached is the full build.log. 
The bug report in the pspp project is here: https://savannah.gnu.org/bugs/?43029

Friedrich

[pspp_0.8.3-3_amd64.build (application/octet-stream, attachment)]

[Message part 3 (text/plain, inline)]

Message #8 received at 18301 <at> debbugs.gnu.org (full text, mbox):

From: Friedrich Beckmann <friedrich.beckmann <at> gmx.de>
To: 18301 <at> debbugs.gnu.org
Subject: bug#18301: Acknowledgement (texi-vers.am: Problem with parallel
 builds due to vti.tmp)
Date: Wed, 20 Aug 2014 19:31:58 +0200

[Message part 1 (text/plain, inline)]

Tags: patch
Severity: normal

Please find attached a patch to make texi-vers.am compatible with parallel builds. 
I tried this patch with my build for pspp here and it works. During execution the existance
of vti.tmp is checked. Only when vti.tmp disappears, the processing continues. When there is
a stale vti.tmp file the processing will wait forever. When the make process is the interrupted, e.g.
with ctrl-c, the vti.tmp file is removed. Then the next make attempt should work. 

Friedrich

[texi-vers.patch (application/octet-stream, attachment)]

Message #13 received at 18301 <at> debbugs.gnu.org (full text, mbox):

From: Friedrich Beckmann <friedrich.beckmann <at> gmx.de>
To: 18301 <at> debbugs.gnu.org
Subject: Updated the patch.
Date: Thu, 21 Aug 2014 00:18:59 +0200

[Message part 1 (text/plain, inline)]

I updated the patch to have a more secure semaphore behavior along the line described
here: http://www.gnu.org/software/automake/manual/html_node/Multiple-Outputs.html

[Message part 2 (text/html, inline)]

[texi-vers-v2.patch (application/octet-stream, attachment)]

[Message part 4 (text/html, inline)]

Message #18 received at 18301-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: 18301-done <at> debbugs.gnu.org
Subject: Re:  texi-vers.am: Problem with parallel builds due to vti.tmp
Date: Sat, 23 Aug 2014 07:58:37 -0700

[Message part 1 (text/plain, inline)]

Thanks, I installed the attached patch, which has the more-usual 
approach of a uniquely-named temporary.

[0001-build-fix-race-in-parallel-builds.patch (text/plain, attachment)]

Message #21 received at 18301 <at> debbugs.gnu.org (full text, mbox):

From: Friedrich Beckmann <friedrich.beckmann <at> gmx.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 18301 <at> debbugs.gnu.org
Subject: Re: bug#18301: closed (Re: texi-vers.am: Problem with parallel builds
 due to vti.tmp)
Date: Sat, 23 Aug 2014 20:33:21 +0200

Dear Paul,

thank for looking at the problem, looking at the patch and introducing a nice solution! 
I tried your patch here in my build environment and I do not see any problems. 

Thank you! 

Friedrich 

Message #24 received at 18301 <at> debbugs.gnu.org (full text, mbox):

From: Friedrich Beckmann <friedrich.beckmann <at> gmx.de>
To: 18301 <at> debbugs.gnu.org
Subject: Re: bug#18301: closed (Re: texi-vers.am: Problem with parallel builds
 due to vti.tmp)
Date: Sat, 23 Aug 2014 20:37:10 +0200

The patch is here in git: http://git.savannah.gnu.org/cgit/automake.git/commit/?id=74cdef7897da220bf062957ff9699815681441ef

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.