GNU bug report logs - #24060
Test failure: tests/publish.scm

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 24060 in the body.
You can then email your comments to 24060 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Chris Marusich <cmmarusich <at> gmail.com>
To: bug-guix <at> gnu.org
Subject: Test failure: tests/publish.scm
Date: Sun, 24 Jul 2016 02:31:53 -0700

[Message part 1 (text/plain, inline)]

Hi,

I've noticed that tests/publish.scm fails when run with the other tests
in parallel (see attached test-suite.log file).  Specifically, it fails
when I run "make -j check".  It passes when I run "make check".  It
passes when I run "make -j check TESTS=tests/publish.scm".  It also
passes when I run "make check TESTS=tests/publish.scm".  Here's the
failure message:

--8<---------------cut here---------------start------------->8---
test-name: /nar/gzip/*
location: /home/marusich/guix/tests/publish.scm:163
source:
+ (test-equal
+   "/nar/gzip/*"
+   "bar"
+   (call-with-temporary-output-file
+     (lambda (temp port)
+       (let ((nar (http-get-port
+                    (publish-uri
+                      (string-append "/nar/gzip/" (basename %item))))))
+         (call-with-gzip-input-port
+           nar
+           (cut restore-file <> temp)))
+       (call-with-input-file temp read-string))))
GET /nar/gzip/z8qsg9bmmjzgfi0bbgls1wxd0wmhhm2i-item
expected-value: bar
actual-value: #f
actual-error:
+ (srfi-34
+   #<condition &nar-error [file: "/tmp/guix-file.7fzEA2" port: #<closed: r6rs-custom-binary-input-port 0>] 27616f0>)
result: FAIL
--8<---------------cut here---------------end--------------->8---

I'm using GuixSD with Guix v0.10.0.  I observed this failure using
commit 8d63dddac11882ef85283bda259e0142b65ae28f.  I built guix with
dependencies provided via "guix environment guix"; however, since there
are two versions of guix defined gnu/packages/package-management.scm,
I'm not sure which one is being used to determine dependencies.

-- 
Chris

[test-suite.log (application/octet-stream, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 24060 <at> debbugs.gnu.org (full text, mbox):

From: Chris Marusich <cmmarusich <at> gmail.com>
To: 24060 <at> debbugs.gnu.org
Subject: Re: bug#24060: Test failure: tests/publish.scm
Date: Sun, 24 Jul 2016 16:01:42 -0700

[Message part 1 (text/plain, inline)]

Hi,

I've performed a git bisect.  The commit which introduced this behavior
is 4a1fc562ae5eedf40f6ae4eabe30580b0983b8f6.

-- 
Chris

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 24060 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Chris Marusich <cmmarusich <at> gmail.com>
Cc: 24060 <at> debbugs.gnu.org
Subject: Re: bug#24060: Test failure: tests/publish.scm
Date: Tue, 26 Jul 2016 23:08:35 +0200

Hello!

Chris Marusich <cmmarusich <at> gmail.com> skribis:

> test-name: /nar/gzip/*
> location: /home/marusich/guix/tests/publish.scm:163
> source:
> + (test-equal
> +   "/nar/gzip/*"
> +   "bar"
> +   (call-with-temporary-output-file
> +     (lambda (temp port)
> +       (let ((nar (http-get-port
> +                    (publish-uri
> +                      (string-append "/nar/gzip/" (basename %item))))))
> +         (call-with-gzip-input-port
> +           nar
> +           (cut restore-file <> temp)))
> +       (call-with-input-file temp read-string))))
> GET /nar/gzip/z8qsg9bmmjzgfi0bbgls1wxd0wmhhm2i-item
> expected-value: bar
> actual-value: #f
> actual-error:
> + (srfi-34
> +   #<condition &nar-error [file: "/tmp/guix-file.7fzEA2" port: #<closed: r6rs-custom-binary-input-port 0>] 27616f0>)
> result: FAIL

I fail to reproduce it with ‘make -j’ on my 4-core laptop.  Is it easy
to reproduce for you?

Does the zlib test always pass, as per:

  while make check TESTS=tests/zlib.scm ; do : ; done

?

Thanks,
Ludo’.

Message #16 received at 24060-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Chris Marusich <cmmarusich <at> gmail.com>
Cc: 24060-done <at> debbugs.gnu.org
Subject: Re: bug#24060: Test failure: tests/publish.scm
Date: Wed, 27 Jul 2016 12:49:25 +0200

Hi,

I managed to reproduce it with more ‘make -j’ runs (it would fail
roughly once every 5 runs).

This is fixed by 37402ecb4379d7199dd4b3386488261938edc780 and the
problem is evidenced by 688ec13c459602d475bccd3638a6802dc0a6ce23.

Thank you!

Ludo’.

Message #19 received at 24060 <at> debbugs.gnu.org (full text, mbox):

From: Chris Marusich <cmmarusich <at> gmail.com>
To: 24060 <at> debbugs.gnu.org
Subject: Test (tests/publish.scm) still fails after fix
Date: Thu, 28 Jul 2016 09:07:58 -0700

[Message part 1 (text/plain, inline)]

Hi,

I tried running the tests against commit
37402ecb4379d7199dd4b3386488261938edc780, but this time the test fails
for a different reason.  Does it succeed for you?

--8<---------------cut here---------------start------------->8---
$ make check 'TESTS=tests/publish.scm'
make  check-recursive
make[1]: Entering directory '/home/marusich/guix'
Making check in po/guix
make[2]: Entering directory '/home/marusich/guix/po/guix'
make[2]: Leaving directory '/home/marusich/guix/po/guix'
Making check in po/packages
make[2]: Entering directory '/home/marusich/guix/po/packages'
make[2]: Leaving directory '/home/marusich/guix/po/packages'
make[2]: Entering directory '/home/marusich/guix'
Compiling Scheme modules...
make  check-TESTS check-local
make[3]: Entering directory '/home/marusich/guix'
make[4]: Entering directory '/home/marusich/guix'
Makefile:4237: recipe for target 'tests/publish.log' failed
make[4]: *** [tests/publish.log] Error 134
make[4]: Leaving directory '/home/marusich/guix'
Makefile:4216: recipe for target 'check-TESTS' failed
make[3]: *** [check-TESTS] Error 2
make[3]: Leaving directory '/home/marusich/guix'
Makefile:4459: recipe for target 'check-am' failed
make[2]: *** [check-am] Error 2
make[2]: Leaving directory '/home/marusich/guix'
Makefile:3996: recipe for target 'check-recursive' failed
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory '/home/marusich/guix'
Makefile:4461: recipe for target 'check' failed
make: *** [check] Error 2
--8<---------------cut here---------------end--------------->8---

No test-suite.log was emitted.  However, tests/publish.log contains the
following failure:

--8<---------------cut here---------------start------------->8---
test-name: /*.narinfo with compression
location: /home/marusich/guix/tests/publish.scm:184
source:
+ (test-equal
+   "/*.narinfo with compression"
+   `(("StorePath" unquote %item)
+     ("URL"
+      unquote
+      (string-append "nar/gzip/" (basename %item)))
+     ("Compression" . "gzip"))
+   (let ((thread
+           (call-with-new-thread
+             (lambda () (guix-publish "--port=6799" "-C5")))))
+     (wait-until-ready 6799)
+     (let* ((url (string-append
+                   "http://localhost:6799/"
+                   (store-path-hash-part %item)
+                   ".narinfo"))
+            (body (http-get-port url)))
+       (filter
+         (lambda (item)
+           (match item
+                  (("Compression" . _) #t)
+                  (("StorePath" . _) #t)
+                  (("URL" . _) #t)
+                  (_ #f)))
+         (recutils->alist body)))))
publishing /home/marusich/guix/test-tmp/store on 0.0.0.0, port 6799
GET /z8qsg9bmmjzgfi0bbgls1wxd0wmhhm2i.narinfo
guile: ports.c:2527: scm_i_port_iconv_descriptors: Assertion `pti->encoding_mode == SCM_PORT_ENCODING_MODE_ICONV' failed.
--8<---------------cut here---------------end--------------->8---

It looks like there may still be a problem here.

-- 
Chris

[signature.asc (application/pgp-signature, inline)]

Message #22 received at 24060 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Chris Marusich <cmmarusich <at> gmail.com>
Cc: 24060 <at> debbugs.gnu.org
Subject: Re: bug#24060: Test (tests/publish.scm) still fails after fix
Date: Thu, 28 Jul 2016 23:20:00 +0200

[Message part 1 (text/plain, inline)]

Hi,

Chris Marusich <cmmarusich <at> gmail.com> skribis:

> I tried running the tests against commit
> 37402ecb4379d7199dd4b3386488261938edc780, but this time the test fails
> for a different reason.  Does it succeed for you?

Yes.

Is this new failure systematic, or does it occur randomly?

> publishing /home/marusich/guix/test-tmp/store on 0.0.0.0, port 6799
> GET /z8qsg9bmmjzgfi0bbgls1wxd0wmhhm2i.narinfo
> guile: ports.c:2527: scm_i_port_iconv_descriptors: Assertion `pti->encoding_mode == SCM_PORT_ENCODING_MODE_ICONV' failed.

What version of Guile do you use?

I wonder if this could be a remnant of <http://bugs.gnu.org/19610>.
Does commenting out the “/nar/gzip/*” test hide the problem?

If so, does the patch below help?

Thanks,
Ludo’.

[Message part 2 (text/x-patch, inline)]

$ git diff HEAD
diff --git a/tests/publish.scm b/tests/publish.scm
index 4dc8075..fda0b7c 100644
--- a/tests/publish.scm
+++ b/tests/publish.scm
@@ -172,11 +172,13 @@ References: ~%"
   "bar"
   (call-with-temporary-output-file
    (lambda (temp port)
-     (let ((nar (http-get-port
-                 (publish-uri
-                  (string-append "/nar/gzip/" (basename %item))))))
-       (call-with-gzip-input-port nar
-         (cut restore-file <> temp)))
+     (let* ((nar  (http-get-port
+                   (publish-uri
+                    (string-append "/nar/gzip/" (basename %item)))))
+            (data (call-with-gzip-input-port nar
+                    get-bytevector-all))
+            (input (open-bytevector-input-port data)))
+       (restore-file input temp))
      (call-with-input-file temp read-string))))
 
 (unless (zlib-available?)

Message #25 received at 24060 <at> debbugs.gnu.org (full text, mbox):

From: Chris Marusich <cmmarusich <at> gmail.com>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 24060 <at> debbugs.gnu.org
Subject: Re: bug#24060: Test (tests/publish.scm) still fails after fix
Date: Sat, 30 Jul 2016 00:59:49 -0700

[Message part 1 (text/plain, inline)]

ludo <at> gnu.org (Ludovic Courtès) writes:

> Is this new failure systematic, or does it occur randomly?

I've tried running the test over 100 times, and it failed every time.
I've also tested it in a QEMU guest running GuixSD, and it still fails.
I will see if I can reproduce it on a totally different physical
computer and get back to you.

> What version of Guile do you use?

The version is "guile (GNU Guile) 2.0.11".  This version comes from
running "guix environment guix".

I just now tried the same tests using guile-next (which is version
2.1.2), and they still failed.

> I wonder if this could be a remnant of <http://bugs.gnu.org/19610>.
> Does commenting out the “/nar/gzip/*” test hide the problem?

Commenting out that test-equals section did not hide the problem.

> If so, does the patch below help?

I tried the patch anyway, and unfortunately the test still failed.

-- 
Chris

[signature.asc (application/pgp-signature, inline)]

Message #28 received at 24060 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Chris Marusich <cmmarusich <at> gmail.com>
Cc: 24060 <at> debbugs.gnu.org
Subject: Re: bug#24060: Test (tests/publish.scm) still fails after fix
Date: Sat, 30 Jul 2016 11:16:51 +0200

Chris Marusich <cmmarusich <at> gmail.com> skribis:

> ludo <at> gnu.org (Ludovic Courtès) writes:
>
>> Is this new failure systematic, or does it occur randomly?
>
> I've tried running the test over 100 times, and it failed every time.
> I've also tested it in a QEMU guest running GuixSD, and it still fails.
> I will see if I can reproduce it on a totally different physical
> computer and get back to you.

Oh.  Weird because ‘guix build guix’ doesn’t have this problem AFAICS.

>> I wonder if this could be a remnant of <http://bugs.gnu.org/19610>.
>> Does commenting out the “/nar/gzip/*” test hide the problem?
>
> Commenting out that test-equals section did not hide the problem.

Could you also comment out "/*.narinfo with compression" and see?
We need to find out which of the tests triggers the Guile crash; it’s
most likely one of the recently-added tests.

Thanks,
Ludo’.

Message #31 received at 24060 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Chris Marusich <cmmarusich <at> gmail.com>
Cc: 24060 <at> debbugs.gnu.org
Subject: Re: bug#24060: Test (tests/publish.scm) still fails after fix
Date: Tue, 02 Aug 2016 18:10:45 +0200

Chris Marusich <cmmarusich <at> gmail.com> skribis:

> ludo <at> gnu.org (Ludovic Courtès) writes:
>
>> Is this new failure systematic, or does it occur randomly?
>
> I've tried running the test over 100 times, and it failed every time.
> I've also tested it in a QEMU guest running GuixSD, and it still fails.
> I will see if I can reproduce it on a totally different physical
> computer and get back to you.
>
>> What version of Guile do you use?
>
> The version is "guile (GNU Guile) 2.0.11".  This version comes from
> running "guix environment guix".
>
> I just now tried the same tests using guile-next (which is version
> 2.1.2), and they still failed.
>
>> I wonder if this could be a remnant of <http://bugs.gnu.org/19610>.
>> Does commenting out the “/nar/gzip/*” test hide the problem?
>
> Commenting out that test-equals section did not hide the problem.

It is reproducible on our current master (post-core-updates merge, using
glibc 2.23).  I’ve debugged a bit and found that commit
2c7b48c2fbccc0b2fc09ae055ce5b32f3fe6b441 fixes it for me.

Could you try again with current master?

For good measure, I also pushed a5c376034f0c465c00e88283dae6d59ac49612a9
to protect us from thread-safety issues that could in theory arise.

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.