GNU bug report logs - #17976
24.3; url-retrieve-synchronously doesn't fallback to IPv4

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 17976 in the body.
You can then email your comments to 17976 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.3; url-retrieve-synchronously doesn't fallback to IPv4
Date: Wed, 09 Jul 2014 14:38:17 +0200

My machine has both IPv6 and IPv4.  I'm accessing a server that has both
IPv6 and IPv4, but whose web server listens on IPv4 only.  The web server
is doing the right thing, sending RST upon IPv6 connection attempts.  Wget
and telnet do the right thing, they fallback to IPv4:

  $ telnet moule.informatique.univ-paris-diderot.fr 8080
  Trying 2001:660:3301:8070::40...
  Trying 194.254.199.40...
  Connected to moule.informatique.univ-paris-diderot.fr.
  Escape character is '^]'.

On the other hand, url-retrieve-synchronously returns an empty string.
Tcpdump shows that it's never even trying IPv4:

  (with-current-buffer
    (url-retrieve-synchronously
     "http://moule.informatique.univ-paris-diderot.fr:8080")
    (buffer-substring (point-min) (point-max)))
  ""

  (with-current-buffer
    (url-retrieve-synchronously "http://194.254.199.40:8080")
    (buffer-substring (point-min) (point-max)))
  "HTTP/1.1 200 OK
  ..."

Message #8 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr>
Cc: 17976 <at> debbugs.gnu.org
Subject: Re: bug#17976: 24.3;
 url-retrieve-synchronously doesn't fallback to IPv4
Date: Wed, 09 Jul 2014 15:06:37 +0200

Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr> writes:

> My machine has both IPv6 and IPv4.  I'm accessing a server that has both
> IPv6 and IPv4, but whose web server listens on IPv4 only.  The web server
> is doing the right thing, sending RST upon IPv6 connection attempts.  Wget
> and telnet do the right thing, they fallback to IPv4:
>
>   $ telnet moule.informatique.univ-paris-diderot.fr 8080
>   Trying 2001:660:3301:8070::40...
>   Trying 194.254.199.40...
>   Connected to moule.informatique.univ-paris-diderot.fr.
>   Escape character is '^]'.
>
> On the other hand, url-retrieve-synchronously returns an empty string.
> Tcpdump shows that it's never even trying IPv4:
>
>   (with-current-buffer
>     (url-retrieve-synchronously
>      "http://moule.informatique.univ-paris-diderot.fr:8080")
>     (buffer-substring (point-min) (point-max)))
>   ""

I cannot reproduce that here.  Emacs is properly falling back to IPv4
and I get the expected contents from this command.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Message #11 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr>
To: Andreas Schwab <schwab <at> suse.de>
Cc: 17976 <at> debbugs.gnu.org
Subject: Re: bug#17976: 24.3;
 url-retrieve-synchronously doesn't fallback to IPv4
Date: Wed, 09 Jul 2014 22:25:41 +0200

Andreas Schwab wrote:

> I cannot reproduce that here.

Strange.  I've just reproduced this with a recent Emacs-24 (dated 5 July 2014).

> Emacs is properly falling back to IPv4

Have you confirmed that you're getting an RST and falling back to IPv4?

-- Juliusz

Message #14 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr>
Cc: 17976 <at> debbugs.gnu.org
Subject: Re: bug#17976: 24.3;
 url-retrieve-synchronously doesn't fallback to IPv4
Date: Thu, 10 Jul 2014 09:17:16 +0200

Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr> writes:

> Have you confirmed that you're getting an RST and falling back to IPv4?

Yes, sure.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Message #17 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr>
To: Andreas Schwab <schwab <at> suse.de>
Cc: 17976 <at> debbugs.gnu.org
Subject: Re: bug#17976: 24.3;
 url-retrieve-synchronously doesn't fallback to IPv4
Date: Thu, 10 Jul 2014 22:28:37 +0200

The following patch works around the issue for me, so it could be
a timing-dependent problem:

--- url-gw.el.orig      2014-07-10 22:26:05.087977204 +0200
+++ url-gw.el   2014-07-10 22:25:32.972959153 +0200
@@ -240,8 +240,7 @@
                           name buffer host service
                           :type gw-method
                           ;; Use non-blocking socket if we can.
-                          :nowait (featurep 'make-network-process
-                                            '(:nowait t))))
+                          :nowait nil))
                         (`socks
                          (socks-open-network-stream name buffer host service))
                         (`telnet

Message #20 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr>
To: Andreas Schwab <schwab <at> suse.de>
Cc: 17976 <at> debbugs.gnu.org
Subject: Re: bug#17976: 24.3;
 url-retrieve-synchronously doesn't fallback to IPv4
Date: Thu, 10 Jul 2014 22:54:14 +0200

Here's a trace (edited to remove irrelevant stuff).  I'm only guessing,
but to my untrained eyes it looks like the code around process.c:4920.

-- Juliusz

socket(PF_INET6, SOCK_STREAM, IPPROTO_TCP) = 13
fcntl(13, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
connect(13, {sa_family=AF_INET6, sin6_port=htons(8080), inet_pton(AF_INET6, "2001:660:3301:8070::40", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EINPROGRESS (Operation now in progress)
getsockname(13, {sa_family=AF_INET6, sin6_port=htons(33826), inet_pton(AF_INET6, "2a01:e34:ec22:84a0::", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
fcntl(13, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
pselect6(14, [12], [13], NULL, {1957, 16494001}, {NULL, 8}) = 1 (out [13], left {1957, 10004292})
getsockopt(13, SOL_SOCKET, SO_ERROR, [111], [4]) = 0
close(13)                               = 0

Message #23 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr>
Cc: 17976 <at> debbugs.gnu.org
Subject: Re: bug#17976: 24.3;
 url-retrieve-synchronously doesn't fallback to IPv4
Date: Fri, 11 Jul 2014 14:21:56 +0200

Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr> writes:

> Andreas Schwab wrote:
>
>> I cannot reproduce that here.
>
> Strange.  I've just reproduced this with a recent Emacs-24 (dated 5 July 2014).

I can as well, after disabling the proxy services.  Sorry.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #30 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr>
Cc: 17976 <at> debbugs.gnu.org
Subject: Re: bug#17976: 24.3;
 url-retrieve-synchronously doesn't fallback to IPv4
Date: Fri, 25 Dec 2015 21:33:56 +0100

Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr> writes:

> My machine has both IPv6 and IPv4.  I'm accessing a server that has both
> IPv6 and IPv4, but whose web server listens on IPv4 only.  The web server
> is doing the right thing, sending RST upon IPv6 connection attempts.  Wget
> and telnet do the right thing, they fallback to IPv4:
>
>   $ telnet moule.informatique.univ-paris-diderot.fr 8080
>   Trying 2001:660:3301:8070::40...
>   Trying 194.254.199.40...
>   Connected to moule.informatique.univ-paris-diderot.fr.
>   Escape character is '^]'.

Perhaps this should be a general network connection thing?

I know nothing about IPv6, though.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #33 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 17976 <at> debbugs.gnu.org, jch <at> pps.univ-paris-diderot.fr
Subject: Re: bug#17976: 24.3;
 url-retrieve-synchronously doesn't fallback to IPv4
Date: Fri, 25 Dec 2015 22:43:24 +0200

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Date: Fri, 25 Dec 2015 21:33:56 +0100
> Cc: 17976 <at> debbugs.gnu.org
> 
> Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr> writes:
> 
> > My machine has both IPv6 and IPv4.  I'm accessing a server that has both
> > IPv6 and IPv4, but whose web server listens on IPv4 only.  The web server
> > is doing the right thing, sending RST upon IPv6 connection attempts.  Wget
> > and telnet do the right thing, they fallback to IPv4:
> >
> >   $ telnet moule.informatique.univ-paris-diderot.fr 8080
> >   Trying 2001:660:3301:8070::40...
> >   Trying 194.254.199.40...
> >   Connected to moule.informatique.univ-paris-diderot.fr.
> >   Escape character is '^]'.
> 
> Perhaps this should be a general network connection thing?

If by "general network connection" you mean on the C level in
process.c, then no, I don't think so.  Protocol-specific code is not
there, it's above that in Lisp.  IMO, url*.el is where this should be
handled.

Message #36 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 17976 <at> debbugs.gnu.org, jch <at> pps.univ-paris-diderot.fr
Subject: Re: bug#17976: 24.3;
 url-retrieve-synchronously doesn't fallback to IPv4
Date: Fri, 25 Dec 2015 21:48:57 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

> If by "general network connection" you mean on the C level in
> process.c, then no, I don't think so.  Protocol-specific code is not
> there, it's above that in Lisp.  IMO, url*.el is where this should be
> handled.

I meant in `open-network-stream', so between the C layer and url*.el.

This is probably as relevant for, say, IMAP as it is for HTTP, I would
guess?

But I've never used IPv6.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #39 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 17976 <at> debbugs.gnu.org, Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr>
Subject: Re: bug#17976: 24.3;
 url-retrieve-synchronously doesn't fallback to IPv4
Date: Fri, 25 Dec 2015 22:00:42 +0100

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr> writes:
>
>> My machine has both IPv6 and IPv4.  I'm accessing a server that has both
>> IPv6 and IPv4, but whose web server listens on IPv4 only.  The web server
>> is doing the right thing, sending RST upon IPv6 connection attempts.  Wget
>> and telnet do the right thing, they fallback to IPv4:
>>
>>   $ telnet moule.informatique.univ-paris-diderot.fr 8080
>>   Trying 2001:660:3301:8070::40...
>>   Trying 194.254.199.40...
>>   Connected to moule.informatique.univ-paris-diderot.fr.
>>   Escape character is '^]'.
>
> Perhaps this should be a general network connection thing?

It needs to be fixed in open-network-stream.  Currently it doesn't
provide a way to fall back to a different address of a given host.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #42 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 17976 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 jch <at> pps.univ-paris-diderot.fr
Subject: Re: bug#17976: 24.3;
 url-retrieve-synchronously doesn't fallback to IPv4
Date: Fri, 25 Dec 2015 22:14:37 +0100

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> But I've never used IPv6.

This is independent of IPv6.  A hostname could also resolve to multiple
IPv4 addresses where only one of them is (currently) reachable.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #45 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Andreas Schwab <schwab <at> linux-m68k.org>
Cc: 17976 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 jch <at> pps.univ-paris-diderot.fr
Subject: Re: bug#17976: 24.3;
 url-retrieve-synchronously doesn't fallback to IPv4
Date: Fri, 25 Dec 2015 22:23:56 +0100

Andreas Schwab <schwab <at> linux-m68k.org> writes:

> This is independent of IPv6.  A hostname could also resolve to multiple
> IPv4 addresses where only one of them is (currently) reachable.

Yeah, that's true.  Emacs should reconnect anyway.

But how does this work in practice?  How are we doing name resolution?
Do we have access to all the IP addresses?  Do we have access to the
IPv6 addresses without...  doing stuff?  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #48 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 17976 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 jch <at> pps.univ-paris-diderot.fr
Subject: Re: bug#17976: 24.3;
 url-retrieve-synchronously doesn't fallback to IPv4
Date: Fri, 25 Dec 2015 22:44:04 +0100

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Do we have access to all the IP addresses?

getaddrinfo gets them all.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #51 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 17976 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Andreas Schwab <schwab <at> linux-m68k.org>
Subject: Re: bug#17976: 24.3;
 url-retrieve-synchronously doesn't fallback to IPv4
Date: Sat, 26 Dec 2015 00:15:44 +0100

>> This is independent of IPv6.  A hostname could also resolve to multiple
>> IPv4 addresses where only one of them is (currently) reachable.

Please hear this man.

> But how does this work in practice?  How are we doing name resolution?
> Do we have access to all the IP addresses?  Do we have access to the
> IPv6 addresses without...  doing stuff?  :-)

If we don't, it's a bug that must be fixed.  The multihoming techniques
that are being developed at IETF right now rely on giving multiple
addresses to a single host (and implementing a routing infrastructure to
match), we really need to be trying them all.

-- Juliusz

Message #58 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr>
Cc: 17976 <at> debbugs.gnu.org, Lars Ingebrigtsen <larsi <at> gnus.org>,
 Eli Zaretskii <eliz <at> gnu.org>, Artur Malabarba <bruce.connor.am <at> gmail.com>,
 Andreas Schwab <schwab <at> suse.de>
Subject: Re: 24.3; url-retrieve-synchronously doesn't fallback to IPv4
Date: Sun, 12 Jun 2016 11:36:43 -0700

[Message part 1 (text/plain, inline)]

I reproduced this bug in a test Fedora 23 environment where attempting to 
connect to one address failed (connection refused). The attached patch to Emacs 
master causes Emacs to try addresses addresses until one succeeds. Does this fix 
your problem? I suspect the code should check for error codes other than 
ECONNREFUSED, but I don't know what the list of such codes should be.

[0001-Try-other-addresses-when-connecting-to-multihomed.patch (text/x-diff, attachment)]

Message #63 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: 17976 <at> debbugs.gnu.org
Cc: Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr>,
 Andreas Schwab <schwab <at> suse.de>, 23620 <at> debbugs.gnu.org,
 Artur Malabarba <bruce.connor.am <at> gmail.com>,
 Lars Ingebrigtsen <larsi <at> gnus.org>, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: 24.3; url-retrieve-synchronously doesn't fallback to IPv4
Date: Sun, 26 Jun 2016 23:58:29 +0200

[Message part 1 (text/plain, inline)]

No further comment on the proposed patch, and as it should be a win for 
non-WINDOWSNT I decided to install it on the Emacs master branch. I 
looked into how libcurl addresses the issue, and it doesn't care whether 
the errno value reported by getsockopt is ECONNREFUSED so I adjusted the 
Emacs patch accordingly (see attached) before installing. Please give it 
a try if you have the time.

This patch does not affect behavior if WINDOWSNT. If the bug does not 
occur on MS-Windows I hope we can close Bug#17976. Otherwise we should 
leave the bug open for the WINDOWSNT case.

[0001-Try-other-addresses-when-connecting-to-multihomed.patch (text/x-patch, attachment)]

Message #66 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Andreas Schwab <schwab <at> suse.de>, 23620 <at> debbugs.gnu.org,
 Artur Malabarba <bruce.connor.am <at> gmail.com>, 17976 <at> debbugs.gnu.org,
 Lars Ingebrigtsen <larsi <at> gnus.org>, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: 24.3; url-retrieve-synchronously doesn't fallback to IPv4
Date: Mon, 27 Jun 2016 01:20:52 +0200

Hi Paul,

I'm the original submitter of #17976.

I can confirm that I still see the bug with Emacs 25, and I don't see the
bug with current trunk (commit g2989ad9).

I'm not sure if it's possible, but I'd appreciate it if this patch could
be backported to Emacs 25 (which is what I run usually).

Thanks for your help,

-- Juliusz Chroboczek

Message #73 received at 17976 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, Andreas Schwab <schwab <at> suse.de>,
 23620 <at> debbugs.gnu.org, Artur Malabarba <bruce.connor.am <at> gmail.com>,
 17976 <at> debbugs.gnu.org, Lars Ingebrigtsen <larsi <at> gnus.org>,
 Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#17976: 24.3;
 url-retrieve-synchronously doesn't fallback to IPv4
Date: Sat, 21 Oct 2017 19:55:34 -0400

close 17976
quit

Juliusz Chroboczek <jch <at> pps.univ-paris-diderot.fr> writes:

> I can confirm that I still see the bug with Emacs 25, and I don't see the
> bug with current trunk (commit g2989ad9).
>
> I'm not sure if it's possible, but I'd appreciate it if this patch could
> be backported to Emacs 25 (which is what I run usually).

I think it's clear enough by now that there's not going to be any more
Emacs 25 releases (unless there's another emergency, in which case this
patch still won't go in).

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.