Received: (at 67245) by debbugs.gnu.org; 13 May 2024 12:44:37 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon May 13 08:44:36 2024 Received: from localhost ([127.0.0.1]:60768 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1s6V2i-0001Ck-Fh for submit <at> debbugs.gnu.org; Mon, 13 May 2024 08:44:36 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37206) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1s6V2g-0001Ce-Co for 67245 <at> debbugs.gnu.org; Mon, 13 May 2024 08:44:35 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1s6V2Y-00061d-Gv; Mon, 13 May 2024 08:44:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=2VGotPU8kBTI0EM7RpRUtSmAxu6IfrLuiOIXuDhyPLU=; b=d95gHOCAkXdHMIyXa4cS GwD8spNbYK1kCULFvSJjIYWJLe5fpn2x1SZlhJ9uOX7azV6P5KCUulHwrXXs7uU7/UBCJyTvGJVz2 Riwf1LPTVOu0I633l8yUrAHh5npUE11sBKUKjDuroXu1T0jjRpZAFllJ1U77h7q3RamvTOQW6so5k +05Wt8h0qTFbGi2TP8ZdQ75zKvRAqgj8GzJSBO7RtK4SrwZAdJz6RerbDLf3yfcRNdpF/PE6NyOTf i+KGr8j4Nira3jgcWEjajekBhu65x5CQGmnueMnwfv/rd7cO4jmo4D6UQf0848c0ZxZKW6pLlpe88 6Si4sPNOv4nPsg==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> To: Christopher Baines <mail@HIDDEN> Subject: Re: [bug#67245] [PATCH v2] store: Add with-store/non-blocking. In-Reply-To: <e692ef2de80732b3ed87e95a489f053616f4130a.1715446400.git.mail@HIDDEN> (Christopher Baines's message of "Sat, 11 May 2024 17:53:20 +0100") References: <460cdfa67b473ea2f1593668b2d9d0fd159378d0.1700244314.git.mail@HIDDEN> <e692ef2de80732b3ed87e95a489f053616f4130a.1715446400.git.mail@HIDDEN> Date: Mon, 13 May 2024 14:44:18 +0200 Message-ID: <87seylankt.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 67245 Cc: Josselin Poiret <dev@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, 67245 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Hi, Christopher Baines <mail@HIDDEN> skribis: > For some applications, it's important to establish a non-blocking connect= ion > rather than just making the socket non-blocking after the connection is > established. This is because there is I/O on the socket that will block d= uring > the handshake. > > I've noticed this blocking during the handshake causing issues in the bui= ld > coordinator for example. > > This commit adds a new with-store variant to avoid changing the behaviour= of > with-store/open-connection to ensure that this change can't break anything > that depends on the blocking nature of the socket. > > * guix/store.scm (open-unix-domain-socket, open-inet-socket): Take > #:non-blocking? and use SOCK_NONBLOCK when calling socket if appropriate. > (connect-to-daemon, open-connection, call-with-store): Take #:non-blockin= g? > and pass it on. > (with-store/non-blocking): New syntax rule. > > Change-Id: I8225762b78448bc1f7b698c8de5d736e13f577bf [...] > +(define* (open-unix-domain-socket file #:key non-blocking?) > "Connect to the Unix-domain socket at FILE and return it. Raise a > -'&store-connection-error' upon error." > +'&store-connection-error' upon error. If NON-BLOCKING?, make the socket > +non-blocking." > (let ((s (with-fluids ((%default-port-encoding #f)) > ;; This trick allows use of the `scm_c_read' optimization. > - (socket PF_UNIX (logior SOCK_STREAM SOCK_CLOEXEC) 0))) > + (socket PF_UNIX > + (logior SOCK_STREAM SOCK_CLOEXEC SOCK_NONBLOCK) > + 0))) Make sure SOCK_NONBLOCK is added only when =E2=80=98non-blocking?=E2=80=99 = is true. > +(define-syntax-rule (with-store/non-blocking store exp ...) > + "Bind STORE to an non-blocking open connection to the store and evalua= te > +EXPs; automatically close the store when the dynamic extent of EXP is le= ft." > + (call-with-store (lambda (store) exp ...) #:non-blocking? #t)) I think we=E2=80=99ll need an entry in =E2=80=98.dir-locals.el=E2=80=99 and= one in (guix read-print) for proper formatting. OK for me with these changes! Thanks, Ludo=E2=80=99.
guix-patches@HIDDEN:bug#67245; Package guix-patches.
Full text available.
Received: (at 67245) by debbugs.gnu.org; 12 May 2024 17:39:23 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun May 12 13:39:23 2024
Received: from localhost ([127.0.0.1]:55565 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1s6DAR-0006xK-4l
for submit <at> debbugs.gnu.org; Sun, 12 May 2024 13:39:23 -0400
Received: from mira.cbaines.net ([212.71.252.8]:43450)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1s6DAM-0006x9-A9
for 67245 <at> debbugs.gnu.org; Sun, 12 May 2024 13:39:21 -0400
Received: from localhost (unknown [45.67.83.168])
by mira.cbaines.net (Postfix) with ESMTPSA id E231A27BBE2;
Sun, 12 May 2024 18:38:46 +0100 (BST)
Received: from felis (localhost [127.0.0.1])
by localhost (OpenSMTPD) with ESMTP id d3c9ae88;
Sun, 12 May 2024 17:38:44 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: [bug#67245] [PATCH] store: Use a non-blocking socket for store
connections.
In-Reply-To: <87fs0nndy9.fsf@HIDDEN> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
=?utf-8?Q?s?= message of "Thu, 30 Nov 2023 22:11:58 +0100")
References: <460cdfa67b473ea2f1593668b2d9d0fd159378d0.1700244314.git.mail@HIDDEN>
<87fs0s17op.fsf@HIDDEN> <875y1nmrwf.fsf@HIDDEN>
<87fs0nndy9.fsf@HIDDEN>
User-Agent: mu4e 1.12.2; emacs 29.3
Date: Sun, 12 May 2024 18:38:41 +0100
Message-ID: <87bk5b7wwu.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
micalg=pgp-sha512; protocol="application/pgp-signature"
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67245
Cc: 67245 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
>>>> +++ b/guix/store.scm
>>>> @@ -460,7 +460,9 @@ (define (open-unix-domain-socket file)
>>>> '&store-connection-error' upon error."
>>>> (let ((s (with-fluids ((%default-port-encoding #f))
>>>> ;; This trick allows use of the `scm_c_read' optimizatio=
n.
>>>> - (socket PF_UNIX (logior SOCK_STREAM SOCK_CLOEXEC) 0)))
>>>> + (socket PF_UNIX
>>>> + (logior SOCK_STREAM SOCK_CLOEXEC SOCK_NONBLOCK)
>>>> + 0)))
>>>
>>> We cannot do this here because callers have to be prepared to deal with
>>> non-blocking sockets, and that=E2=80=99s not the case in Guix itself.
>>
>> I can see potential problems for programs outside of Guix which use
>> suspendable ports, but given Guix doesn't use suspendable ports, this
>> won't change behaviour, right?
>>
>> Obviously Guile will be working a bit differently, using poll when it
>> needs to wait for I/O, but at the scheme level within Guix, things
>> should be no different.
>
> Hmm yes, I think you=E2=80=99re right.
>
> One issue is if we hand over the file descriptor to something that=E2=80=
=99s not
> Guile. Off the top of my head, this happens with inferiors and in the
> =E2=80=98build=E2=80=99 procedure of =E2=80=98build-self.scm=E2=80=99 (we=
ll, the process that receives
> that file descriptor is Guile, but if it=E2=80=99s older than 3.0 (?), th=
en it
> may behave differently.)
>
> So it should be safe indeed, but adds a bit of overhead (hopping via
> =E2=80=98current-{read,write}-waiter=E2=80=99) and needs good testing.
>
> Laziness gives an incentive for the status quo, but I=E2=80=99m not oppos=
ed to
> the change if we get more confidence (test suite passing, tests with
> inferiors and =E2=80=98time-machine=E2=80=99, and some more auditing.)
Maybe we can just move the with-store/non-blocking in to Guix, as that
will solve the immediate issue.
I've sent a new patch for that.
--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmZA/qFfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9Xfg6xAAs1SQRJtgIZUhwaIey4Fv1Moc4A0j8Rto
0QPVUBa3bqiXeAxyFkXTxSW4ke8jILjzhTcnmDk16lijmP+FPYN57JIVu0lLQDQf
PkggqF8eJ0Te8born4s4v+5O601xKVH3zeZJl3BETIETVC+F4Rd7SbpmDxlAq10C
PUaaoPRGphTdbLmgXc6srN7+tmSQFt4SiDMmv9vKNs1Wll47slEBRdacq7V8Gc+8
b5cdYXpOTloGhVFMf6ZJLnlm2kViy9LFKHKTciWej5xgutzwjUht2oQxGUeoYHV7
huFVHNAizYjM4k0uoGF8qi9jutev0PTvIWZ5b3GXih6iFtoZcFbyuqN3siqxYlnJ
1rKat5VYhqED8cHgg+73ywz+1n6B5gBXT+2tgdI98uAZ1vaD2M2f7taSIfHtjMio
qQ035phm1MrGu3F8udtVoO4YCCHEq3Dj8MxSRd9m+q0c6KYwerkq2c5i8JFyGuT2
OPU5xSyjrn+A6ZWzxKHMIyisTD7X2tYmHE2W8cd8fzt7Fr4rimcHaLq69obmhhrb
V9cZOqL4Ohtgd03BwLipq+GLEzSCFkbEFgqPJkDY06uXr3ZvPP2q+wLuTLQWj7IE
mShUN8qKTjOVgZfwU6jKJofghAIBWUeBNRhYXKxsWsf0kUxWBkKd67shQPluyalI
vwbaCfLqfKA=
=/9YT
-----END PGP SIGNATURE-----
--=-=-=--
guix-patches@HIDDEN:bug#67245; Package guix-patches.
Full text available.
Received: (at 67245) by debbugs.gnu.org; 11 May 2024 16:53:55 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat May 11 12:53:55 2024
Received: from localhost ([127.0.0.1]:49261 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1s5pyt-0000nY-3G
for submit <at> debbugs.gnu.org; Sat, 11 May 2024 12:53:55 -0400
Received: from mira.cbaines.net ([212.71.252.8]:43448)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1s5pyr-0000nS-G6
for 67245 <at> debbugs.gnu.org; Sat, 11 May 2024 12:53:54 -0400
Received: from localhost (unknown [89.207.171.88])
by mira.cbaines.net (Postfix) with ESMTPSA id 1231727BBE2
for <67245 <at> debbugs.gnu.org>; Sat, 11 May 2024 17:53:23 +0100 (BST)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id b0e00431
for <67245 <at> debbugs.gnu.org>; Sat, 11 May 2024 16:53:21 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
To: 67245 <at> debbugs.gnu.org
Subject: [PATCH v2] store: Add with-store/non-blocking.
Date: Sat, 11 May 2024 17:53:20 +0100
Message-ID: <e692ef2de80732b3ed87e95a489f053616f4130a.1715446400.git.mail@HIDDEN>
X-Mailer: git-send-email 2.41.0
MIME-Version: 1.0
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67245
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
For some applications, it's important to establish a non-blocking connection
rather than just making the socket non-blocking after the connection is
established. This is because there is I/O on the socket that will block during
the handshake.
I've noticed this blocking during the handshake causing issues in the build
coordinator for example.
This commit adds a new with-store variant to avoid changing the behaviour of
with-store/open-connection to ensure that this change can't break anything
that depends on the blocking nature of the socket.
* guix/store.scm (open-unix-domain-socket, open-inet-socket): Take
#:non-blocking? and use SOCK_NONBLOCK when calling socket if appropriate.
(connect-to-daemon, open-connection, call-with-store): Take #:non-blocking?
and pass it on.
(with-store/non-blocking): New syntax rule.
Change-Id: I8225762b78448bc1f7b698c8de5d736e13f577bf
---
guix/store.scm | 53 +++++++++++++++++++++++++++++++++++---------------
1 file changed, 37 insertions(+), 16 deletions(-)
diff --git a/guix/store.scm b/guix/store.scm
index a238cb627a..3e8202a43a 100644
--- a/guix/store.scm
+++ b/guix/store.scm
@@ -106,6 +106,7 @@ (define-module (guix store)
port->connection
close-connection
with-store
+ with-store/non-blocking
set-build-options
set-build-options*
valid-path?
@@ -462,12 +463,15 @@ (define-syntax-rule (system-error-to-connection-error file exp ...)
(file file)
(errno errno))))))))
-(define (open-unix-domain-socket file)
+(define* (open-unix-domain-socket file #:key non-blocking?)
"Connect to the Unix-domain socket at FILE and return it. Raise a
-'&store-connection-error' upon error."
+'&store-connection-error' upon error. If NON-BLOCKING?, make the socket
+non-blocking."
(let ((s (with-fluids ((%default-port-encoding #f))
;; This trick allows use of the `scm_c_read' optimization.
- (socket PF_UNIX (logior SOCK_STREAM SOCK_CLOEXEC) 0)))
+ (socket PF_UNIX
+ (logior SOCK_STREAM SOCK_CLOEXEC SOCK_NONBLOCK)
+ 0)))
(a (make-socket-address PF_UNIX file)))
(system-error-to-connection-error file
@@ -478,9 +482,10 @@ (define %default-guix-port
;; Default port when connecting to a daemon over TCP/IP.
44146)
-(define (open-inet-socket host port)
+(define* (open-inet-socket host port #:key non-blocking?)
"Connect to the Unix-domain socket at HOST:PORT and return it. Raise a
-'&store-connection-error' upon error."
+'&store-connection-error' upon error. If NON-BLOCKING?, make the socket
+non-blocking."
(define addresses
(getaddrinfo host
(if (number? port) (number->string port) port)
@@ -495,7 +500,10 @@ (define (open-inet-socket host port)
((ai rest ...)
(let ((s (socket (addrinfo:fam ai)
;; TCP/IP only
- (logior SOCK_STREAM SOCK_CLOEXEC) IPPROTO_IP)))
+ (if non-blocking?
+ (logior SOCK_STREAM SOCK_CLOEXEC SOCK_NONBLOCK)
+ (logior SOCK_STREAM SOCK_CLOEXEC))
+ IPPROTO_IP)))
(catch 'system-error
(lambda ()
@@ -514,9 +522,10 @@ (define (open-inet-socket host port)
(errno (system-error-errno args)))))
(loop rest)))))))))
-(define (connect-to-daemon uri)
+(define* (connect-to-daemon uri #:key non-blocking?)
"Connect to the daemon at URI, a string that may be an actual URI or a file
-name, and return an input/output port.
+name, and return an input/output port. If NON-BLOCKING?, use a non-blocking
+socket when using the file, unix or guix URI schemes.
This is a low-level procedure that does not perform the initial handshake with
the daemon. Use 'open-connection' for that."
@@ -533,11 +542,13 @@ (define (connect-to-daemon uri)
(match (uri-scheme uri)
((or #f 'file 'unix)
(lambda (_)
- (open-unix-domain-socket (uri-path uri))))
+ (open-unix-domain-socket (uri-path uri)
+ #:non-blocking? non-blocking?)))
('guix
(lambda (_)
(open-inet-socket (uri-host uri)
- (or (uri-port uri) %default-guix-port))))
+ (or (uri-port uri) %default-guix-port)
+ #:non-blocking? non-blocking?)))
((? symbol? scheme)
;; Try to dynamically load a module for SCHEME.
;; XXX: Errors are swallowed.
@@ -557,7 +568,8 @@ (define (connect-to-daemon uri)
(connect uri))
(define* (open-connection #:optional (uri (%daemon-socket-uri))
- #:key port (reserve-space? #t) cpu-affinity)
+ #:key port (reserve-space? #t) cpu-affinity
+ non-blocking?)
"Connect to the daemon at URI (a string), or, if PORT is not #f, use it as
the I/O port over which to communicate to a build daemon.
@@ -565,7 +577,9 @@ (define* (open-connection #:optional (uri (%daemon-socket-uri))
space on the file system so that the garbage collector can still operate,
should the disk become full. When CPU-AFFINITY is true, it must be an integer
corresponding to an OS-level CPU number to which the daemon's worker process
-for this connection will be pinned. Return a server object."
+for this connection will be pinned. If NON-BLOCKING?, use a non-blocking
+socket when using the file, unix or guix URI schemes. Return a server
+object."
(define (handshake-error)
(raise (condition
(&store-connection-error (file (or port uri))
@@ -577,7 +591,8 @@ (define* (open-connection #:optional (uri (%daemon-socket-uri))
;; really a connection error.
(handshake-error)))
(let*-values (((port)
- (or port (connect-to-daemon uri)))
+ (or port (connect-to-daemon
+ uri #:non-blocking? non-blocking?)))
((output flush)
(buffering-output-port port
(make-bytevector 8192))))
@@ -657,9 +672,10 @@ (define (close-connection server)
"Close the connection to SERVER."
(close (store-connection-socket server)))
-(define (call-with-store proc)
- "Call PROC with an open store connection."
- (let ((store (open-connection)))
+(define* (call-with-store proc #:key non-blocking?)
+ "Call PROC with an open store connection. Pass NON-BLOCKING? to
+open-connection."
+ (let ((store (open-connection #:non-blocking? non-blocking?)))
(define (thunk)
(parameterize ((current-store-protocol-version
(store-connection-version store)))
@@ -678,6 +694,11 @@ (define-syntax-rule (with-store store exp ...)
automatically close the store when the dynamic extent of EXP is left."
(call-with-store (lambda (store) exp ...)))
+(define-syntax-rule (with-store/non-blocking store exp ...)
+ "Bind STORE to an non-blocking open connection to the store and evaluate
+EXPs; automatically close the store when the dynamic extent of EXP is left."
+ (call-with-store (lambda (store) exp ...) #:non-blocking? #t))
+
(define current-store-protocol-version
;; Protocol version of the store currently used. XXX: This is a hack to
;; communicate the protocol version to the build output port. It's a hack
base-commit: 9288654773a110156e0bb6fc703a9c24f5bfc527
--
2.41.0
guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:bug#67245; Package guix-patches.
Full text available.
Received: (at 67245) by debbugs.gnu.org; 30 Nov 2023 21:12:18 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Nov 30 16:12:18 2023
Received: from localhost ([127.0.0.1]:54580 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1r8oKX-0002kQ-Ma
for submit <at> debbugs.gnu.org; Thu, 30 Nov 2023 16:12:18 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:38326)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1r8oKV-0002k7-6P
for 67245 <at> debbugs.gnu.org; Thu, 30 Nov 2023 16:12:16 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1r8oKH-00054J-Hg; Thu, 30 Nov 2023 16:12:01 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=PKvLPR2496liOOQRwxJvgMWip5X5grNXFdZ+HJBYBsQ=; b=NUqzW8asuniYuolnnjwG
3MEEv48gWh9wy2ESNXzFajjKu3F9J7WQSjaIVhrRdIsdRlknSEf9VracNOBf+SeL20gP86nxv7uyC
3bEi/Na5gZ+vc1HbYUF4AZWoZlGlm7OVtLRAjWRVo0BXPlRP8zG6pa8wGIW1NIwccUTAqYyvi9NU3
yBkWOQY7BL2KIKt0Q8U9zO3Mn1xg41VAwJ0iezsrZjq4qwsDR2dLq+ZVNSHcMhmSiH4Marz5BR3d7
quygTJSCLLZdu6XVvkZIwidklyMoznud/rDlSCPcvGyVU/durZZy3tffq1dU8LptA9NZx0tlTaM4f
RHsuG94FR5rfgw==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Christopher Baines <mail@HIDDEN>
Subject: Re: [bug#67245] [PATCH] store: Use a non-blocking socket for store
connections.
In-Reply-To: <875y1nmrwf.fsf@HIDDEN> (Christopher Baines's message of
"Mon, 27 Nov 2023 09:48:09 +0000")
References: <460cdfa67b473ea2f1593668b2d9d0fd159378d0.1700244314.git.mail@HIDDEN>
<87fs0s17op.fsf@HIDDEN> <875y1nmrwf.fsf@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: =?utf-8?Q?D=C3=A9cadi?= 10 Frimaire an 232 de la
=?utf-8?Q?R=C3=A9volution=2C?= jour de la Pioche
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Thu, 30 Nov 2023 22:11:58 +0100
Message-ID: <87fs0nndy9.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67245
Cc: 67245 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Hi Chris,
Christopher Baines <mail@HIDDEN> skribis:
> Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
>> I feel we should really discuss on Guix + Fibers since we=E2=80=99ve app=
arently
>> been going through the exact same set of issues. :-)
>>
>> (The other thing that comes to mind is the resource pool!)
>
> I'm mostly ignoring these issues then coping the code once you write it
> :)
Heh, so we=E2=80=99re already in sync maybe, not bad. :-)
>>> +++ b/guix/store.scm
>>> @@ -460,7 +460,9 @@ (define (open-unix-domain-socket file)
>>> '&store-connection-error' upon error."
>>> (let ((s (with-fluids ((%default-port-encoding #f))
>>> ;; This trick allows use of the `scm_c_read' optimization.
>>> - (socket PF_UNIX (logior SOCK_STREAM SOCK_CLOEXEC) 0)))
>>> + (socket PF_UNIX
>>> + (logior SOCK_STREAM SOCK_CLOEXEC SOCK_NONBLOCK)
>>> + 0)))
>>
>> We cannot do this here because callers have to be prepared to deal with
>> non-blocking sockets, and that=E2=80=99s not the case in Guix itself.
>
> I can see potential problems for programs outside of Guix which use
> suspendable ports, but given Guix doesn't use suspendable ports, this
> won't change behaviour, right?
>
> Obviously Guile will be working a bit differently, using poll when it
> needs to wait for I/O, but at the scheme level within Guix, things
> should be no different.
Hmm yes, I think you=E2=80=99re right.
One issue is if we hand over the file descriptor to something that=E2=80=99=
s not
Guile. Off the top of my head, this happens with inferiors and in the
=E2=80=98build=E2=80=99 procedure of =E2=80=98build-self.scm=E2=80=99 (well=
, the process that receives
that file descriptor is Guile, but if it=E2=80=99s older than 3.0 (?), then=
it
may behave differently.)
So it should be safe indeed, but adds a bit of overhead (hopping via
=E2=80=98current-{read,write}-waiter=E2=80=99) and needs good testing.
Laziness gives an incentive for the status quo, but I=E2=80=99m not opposed=
to
the change if we get more confidence (test suite passing, tests with
inferiors and =E2=80=98time-machine=E2=80=99, and some more auditing.)
Ludo=E2=80=99.
guix-patches@HIDDEN:bug#67245; Package guix-patches.
Full text available.Received: (at 67245) by debbugs.gnu.org; 27 Nov 2023 10:07:11 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Nov 27 05:07:11 2023 Received: from localhost ([127.0.0.1]:43353 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1r7YWF-00089a-6h for submit <at> debbugs.gnu.org; Mon, 27 Nov 2023 05:07:11 -0500 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:55759) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1r7YWB-00089M-0v for 67245 <at> debbugs.gnu.org; Mon, 27 Nov 2023 05:07:09 -0500 Received: from localhost (unknown [217.155.61.229]) by mira.cbaines.net (Postfix) with ESMTPSA id 236BE27BBE2; Mon, 27 Nov 2023 10:06:59 +0000 (GMT) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 85636a02; Mon, 27 Nov 2023 10:06:58 +0000 (UTC) References: <460cdfa67b473ea2f1593668b2d9d0fd159378d0.1700244314.git.mail@HIDDEN> <87fs0s17op.fsf@HIDDEN> User-agent: mu4e 1.10.7; emacs 29.1 From: Christopher Baines <mail@HIDDEN> To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Subject: Re: [bug#67245] [PATCH] store: Use a non-blocking socket for store connections. Date: Mon, 27 Nov 2023 09:48:09 +0000 In-reply-to: <87fs0s17op.fsf@HIDDEN> Message-ID: <875y1nmrwf.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 67245 Cc: 67245 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s <ludo@HIDDEN> writes: > Hi Christopher, > > Christopher Baines <mail@HIDDEN> skribis: > >> For some applications, it's important to do this here rather than just m= aking >> the socket non-blocking after the connection is established because ther= e can >> be I/O on the socket that will block during the handshake. >> >> I've noticed this blocking during the handshake causing issues in the bu= ild >> coordinator for example. >> >> * guix/store.scm (open-unix-domain-socket, open-inet-socket): Pass >> SOCK_NONBLOCK when calling socket. >> >> Change-Id: I8225762b78448bc1f7b698c8de5d736e13f577bf > > I feel we should really discuss on Guix + Fibers since we=E2=80=99ve appa= rently > been going through the exact same set of issues. :-) > > (The other thing that comes to mind is the resource pool!) I'm mostly ignoring these issues then coping the code once you write it :) >> +++ b/guix/store.scm >> @@ -460,7 +460,9 @@ (define (open-unix-domain-socket file) >> '&store-connection-error' upon error." >> (let ((s (with-fluids ((%default-port-encoding #f)) >> ;; This trick allows use of the `scm_c_read' optimization. >> - (socket PF_UNIX (logior SOCK_STREAM SOCK_CLOEXEC) 0))) >> + (socket PF_UNIX >> + (logior SOCK_STREAM SOCK_CLOEXEC SOCK_NONBLOCK) >> + 0))) > > We cannot do this here because callers have to be prepared to deal with > non-blocking sockets, and that=E2=80=99s not the case in Guix itself. I can see potential problems for programs outside of Guix which use suspendable ports, but given Guix doesn't use suspendable ports, this won't change behaviour, right? Obviously Guile will be working a bit differently, using poll when it needs to wait for I/O, but at the scheme level within Guix, things should be no different. I tried guix weather with this change, and things seemed fine. Is there a specific bit of Guix you're concerned about? --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmVkakBfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XcoJhAArMwV2vXM3qOOR01PaYfTHmQb/NcB3XiK VXiusYS9WB15qLYF8v3qxhXtvhy/4ZBmmTCe9l/adwjqGqoHAeBqSRHNODvCZG3+ KXEclNOe1/7zWFlIkzP3bjHBQr9qOHdEq2NLl+Btyzn/9c8WGYeR0SWCgY4rjDtX fUicq8czkuAURryhErg1wyDXgHdNwt4mez7sH7Tx3biBSC71PRmc/Cjbn5n3bhM9 cqJ8BWpel198VsDAJpCF+s6PvmfHRMCvuFie0IVl6O3wuvGkMoO6VofMg8i0mlsE sBh97Cio+ShZRhtqjYP6L50cNnQNwH3WT7QfhG0qMOPJJCje21MRckmyfWiot4pp BQmYVmyBxzOesKsfWvexuCeQN7BF9XAkFgf+vu2xfCmjxTNdCrqB/6YswVqa4/Z2 oIB0hiWEMhQxC0bCDU2dIExi1LkrXlAWQJPxPYxwaz3VnT3fUy7zHW5Ql2iXFd/r 2Sr8Iy7MvGVpmwyaQA3LgLcDji/h1vxQpzqniZDFaqFPMgdSEcxCG3wKGzZIsPAJ MtyA5FPYxPHt042bWrGMNdyiVMpBpSZvzfgexCpJi96jikfD9tkWOyEjxgUnk7Jo 4QA03acFYUPxFkIiey6mJOFvzVzeTogFMqe6sV7/Bvz2fWv/dSczx8MbWNaEeoRP qEmCKTgUzvE= =iswm -----END PGP SIGNATURE----- --=-=-=--
guix-patches@HIDDEN:bug#67245; Package guix-patches.
Full text available.
Received: (at 67245) by debbugs.gnu.org; 26 Nov 2023 22:17:20 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Nov 26 17:17:20 2023
Received: from localhost ([127.0.0.1]:42813 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1r7NRH-00057G-QP
for submit <at> debbugs.gnu.org; Sun, 26 Nov 2023 17:17:20 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:45842)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1r7NRF-000571-DJ
for 67245 <at> debbugs.gnu.org; Sun, 26 Nov 2023 17:17:18 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
id 1r7NR3-0005kp-60; Sun, 26 Nov 2023 17:17:05 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To:
From; bh=EIoodbFrlykBpqlgECvTW/T9LgKMfswp2BhPBjiMboc=; b=JNivdy7H6ADrs3dMAva3
eJSTy8Shl8gB1uzlDYJuhlITrsooCZC4SQZ16/VdF+vHF1jAwdaScySz79wyT0lza3KZEJpVXGhDP
E80s6C+0ZWYGuT1akgUDbx0sdJSt6EisG7lWthGoJ0Hznwi00tW4fBB9qfefcRIuDUUPM6mUA0GQ9
5lHdtKF82L/z2YSAfMlDgCXtBisLvCraBYTlC0EacCGy0Y5ZnOn9SeCQuSqeg38Vgj7kKemelwvTA
/9/Mg+OqmK//IGemsb/V2JSRcOgeF8ZhW/sYWs+L66lA5ksVLHotnbtFgrxcRCj/kzmSaSKZRAVdI
dJOyKDAWrFVlmQ==;
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Christopher Baines <mail@HIDDEN>
Subject: Re: [bug#67245] [PATCH] store: Use a non-blocking socket for store
connections.
In-Reply-To: <460cdfa67b473ea2f1593668b2d9d0fd159378d0.1700244314.git.mail@HIDDEN>
(Christopher Baines's message of "Fri, 17 Nov 2023 18:05:14 +0000")
References: <460cdfa67b473ea2f1593668b2d9d0fd159378d0.1700244314.git.mail@HIDDEN>
Date: Sun, 26 Nov 2023 23:16:54 +0100
Message-ID: <87fs0s17op.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67245
Cc: Josselin Poiret <dev@HIDDEN>,
Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
Tobias Geerinckx-Rice <me@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>,
67245 <at> debbugs.gnu.org, Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Hi Christopher,
Christopher Baines <mail@HIDDEN> skribis:
> For some applications, it's important to do this here rather than just ma=
king
> the socket non-blocking after the connection is established because there=
can
> be I/O on the socket that will block during the handshake.
>
> I've noticed this blocking during the handshake causing issues in the bui=
ld
> coordinator for example.
>
> * guix/store.scm (open-unix-domain-socket, open-inet-socket): Pass
> SOCK_NONBLOCK when calling socket.
>
> Change-Id: I8225762b78448bc1f7b698c8de5d736e13f577bf
I feel we should really discuss on Guix + Fibers since we=E2=80=99ve appare=
ntly
been going through the exact same set of issues. :-)
(The other thing that comes to mind is the resource pool!)
> +++ b/guix/store.scm
> @@ -460,7 +460,9 @@ (define (open-unix-domain-socket file)
> '&store-connection-error' upon error."
> (let ((s (with-fluids ((%default-port-encoding #f))
> ;; This trick allows use of the `scm_c_read' optimization.
> - (socket PF_UNIX (logior SOCK_STREAM SOCK_CLOEXEC) 0)))
> + (socket PF_UNIX
> + (logior SOCK_STREAM SOCK_CLOEXEC SOCK_NONBLOCK)
> + 0)))
We cannot do this here because callers have to be prepared to deal with
non-blocking sockets, and that=E2=80=99s not the case in Guix itself.
In Cuirass, I have this:
--8<---------------cut here---------------start------------->8---
(define (non-blocking-port port)
"Make PORT non-blocking and return it."
(let ((flags (fcntl port F_GETFL)))
(when (zero? (logand O_NONBLOCK flags))
(fcntl port F_SETFL (logior O_NONBLOCK flags)))
port))
(define (ensure-non-blocking-store-connection store)
"Mark the file descriptor that backs STORE, a <store-connection>, as
O_NONBLOCK."
(match (store-connection-socket store)
((? file-port? port)
(non-blocking-port port))
(_ #f)))
(define-syntax-rule (with-store/non-blocking store exp ...)
"Like 'with-store', bind STORE to a connection to the store, but ensure t=
hat
said connection is non-blocking (O_NONBLOCK). Evaluate EXP... in that
context."
(with-store store
(ensure-non-blocking-store-connection store)
(let ()
exp ...)))
--8<---------------cut here---------------end--------------->8---
Then =E2=80=98with-store/non-blocking=E2=80=99 is used in fiberized context=
where I know
this is fine.
I think it=E2=80=99ll have to remain this way until Guix itself is fiberize=
d or
something.
Does that make sense?
Ludo=E2=80=99.
guix-patches@HIDDEN:bug#67245; Package guix-patches.
Full text available.
Received: (at submit) by debbugs.gnu.org; 17 Nov 2023 18:05:36 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Nov 17 13:05:36 2023
Received: from localhost ([127.0.0.1]:47108 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1r43Dj-000851-T2
for submit <at> debbugs.gnu.org; Fri, 17 Nov 2023 13:05:36 -0500
Received: from lists.gnu.org ([2001:470:142::17]:39432)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mail@HIDDEN>) id 1r43Dh-00084l-Kv
for submit <at> debbugs.gnu.org; Fri, 17 Nov 2023 13:05:34 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <mail@HIDDEN>) id 1r43Db-0006kN-O9
for guix-patches@HIDDEN; Fri, 17 Nov 2023 13:05:27 -0500
Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27])
by eggs.gnu.org with esmtp (Exim 4.90_1)
(envelope-from <mail@HIDDEN>) id 1r43DV-0000ZB-3w
for guix-patches@HIDDEN; Fri, 17 Nov 2023 13:05:27 -0500
Received: from localhost (unknown [217.155.61.229])
by mira.cbaines.net (Postfix) with ESMTPSA id B1DA227BBE2
for <guix-patches@HIDDEN>; Fri, 17 Nov 2023 18:05:15 +0000 (GMT)
Received: from localhost (localhost [local])
by localhost (OpenSMTPD) with ESMTPA id fbe70812
for <guix-patches@HIDDEN>; Fri, 17 Nov 2023 18:05:14 +0000 (UTC)
From: Christopher Baines <mail@HIDDEN>
To: guix-patches@HIDDEN
Subject: [PATCH] store: Use a non-blocking socket for store connections.
Date: Fri, 17 Nov 2023 18:05:14 +0000
Message-ID: <460cdfa67b473ea2f1593668b2d9d0fd159378d0.1700244314.git.mail@HIDDEN>
X-Mailer: git-send-email 2.41.0
MIME-Version: 1.0
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Ricardo Wurmus <rekado@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Transfer-Encoding: 8bit
Received-SPF: pass client-ip=2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27;
envelope-from=mail@HIDDEN; helo=mira.cbaines.net
X-Spam_score_int: -18
X-Spam_score: -1.9
X-Spam_bar: -
X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001,
SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01,
UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 0.9 (/)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.1 (/)
For some applications, it's important to do this here rather than just making
the socket non-blocking after the connection is established because there can
be I/O on the socket that will block during the handshake.
I've noticed this blocking during the handshake causing issues in the build
coordinator for example.
* guix/store.scm (open-unix-domain-socket, open-inet-socket): Pass
SOCK_NONBLOCK when calling socket.
Change-Id: I8225762b78448bc1f7b698c8de5d736e13f577bf
---
guix/store.scm | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/guix/store.scm b/guix/store.scm
index f8e77b2cd9..216be98c05 100644
--- a/guix/store.scm
+++ b/guix/store.scm
@@ -460,7 +460,9 @@ (define (open-unix-domain-socket file)
'&store-connection-error' upon error."
(let ((s (with-fluids ((%default-port-encoding #f))
;; This trick allows use of the `scm_c_read' optimization.
- (socket PF_UNIX (logior SOCK_STREAM SOCK_CLOEXEC) 0)))
+ (socket PF_UNIX
+ (logior SOCK_STREAM SOCK_CLOEXEC SOCK_NONBLOCK)
+ 0)))
(a (make-socket-address PF_UNIX file)))
(system-error-to-connection-error file
@@ -488,7 +490,8 @@ (define (open-inet-socket host port)
((ai rest ...)
(let ((s (socket (addrinfo:fam ai)
;; TCP/IP only
- (logior SOCK_STREAM SOCK_CLOEXEC) IPPROTO_IP)))
+ (logior SOCK_STREAM SOCK_CLOEXEC SOCK_NONBLOCK)
+ IPPROTO_IP)))
(catch 'system-error
(lambda ()
base-commit: e35b7c5386c1bfacf47ed31bac9b503373dd26fc
--
2.41.0
Christopher Baines <mail@HIDDEN>:guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN.
Full text available.guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, rekado@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:bug#67245; Package guix-patches.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.