X-Loop: help-debbugs@HIDDEN Subject: [bug#47174] [PATCH 0/2] substitute: Handle closing connections to substitute servers. Resent-From: Christopher Baines <mail@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Mon, 15 Mar 2021 19:22:01 +0000 Resent-Message-ID: <handler.47174.B.161583611726811 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: report 47174 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 47174 <at> debbugs.gnu.org X-Debbugs-Original-To: guix-patches@HIDDEN Received: via spool by submit <at> debbugs.gnu.org id=B.161583611726811 (code B ref -1); Mon, 15 Mar 2021 19:22:01 +0000 Received: (at submit) by debbugs.gnu.org; 15 Mar 2021 19:21:57 +0000 Received: from localhost ([127.0.0.1]:37209 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1lLsmq-0006yN-S4 for submit <at> debbugs.gnu.org; Mon, 15 Mar 2021 15:21:57 -0400 Received: from lists.gnu.org ([209.51.188.17]:50388) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1lLsmo-0006yF-FR for submit <at> debbugs.gnu.org; Mon, 15 Mar 2021 15:21:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58256) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <mail@HIDDEN>) id 1lLsml-0000NJ-Jg for guix-patches@HIDDEN; Mon, 15 Mar 2021 15:21:54 -0400 Received: from mira.cbaines.net ([212.71.252.8]:53620) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from <mail@HIDDEN>) id 1lLsme-0008W0-Lq for guix-patches@HIDDEN; Mon, 15 Mar 2021 15:21:51 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id 2E6D727BC52 for <guix-patches@HIDDEN>; Mon, 15 Mar 2021 19:21:43 +0000 (GMT) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 75122fc1 for <guix-patches@HIDDEN>; Mon, 15 Mar 2021 19:21:42 +0000 (UTC) User-agent: mu4e 1.4.15; emacs 27.1 From: Christopher Baines <mail@HIDDEN> Date: Mon, 15 Mar 2021 19:21:41 +0000 Message-ID: <87y2eodxyy.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=212.71.252.8; envelope-from=mail@HIDDEN; helo=mira.cbaines.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -2.4 (--) --=-=-= Content-Type: text/plain Christopher Baines (2): guix: Alter http-fetch to return the response. substitute: Handle closing connections to substitute servers. guix/build/download-nar.scm | 5 +++-- guix/build/download.scm | 9 ++++++--- guix/http-client.scm | 12 ++++++------ guix/scripts/substitute.scm | 31 ++++++++++++++++++++++++------- 4 files changed, 39 insertions(+), 18 deletions(-) --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmBPs8VfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9Xef2A/+Lm+zmZgKgVlBPQz+dN6k5H9MnTXQNU3m iPPZ0BeaY9d8ZTg1zSnwLeF9kRQsm3Z/VlU0wmwv56AO3RS2U6q8iXzk+V6kmui/ 0a+c84k+TsIkFcyEJmQlGXXvcoGZpGi2BHMo4fdAsKdKkTW3nIk+fHfc0fJ31wuO skHZF5Q6KRIGyURtym9w2NB4fBjoesjJqsZ7Q55OXgViGITfDqCGprdlF1cxdGvt bNNCoNj26bOzlT7D10rW9IFZWJrG+6CGxfM0+d2LaW30viO+P8RPxLs5UfkcGsTE 41/1VCXUoCBAR2/GRoPGiZsYNIznLuV+/hqD+vNLh1nIctybX0cld74uttSROImp 4RiJy4YBdjnYu4e//rzui0jIU7QfX/iVSCYtgt6ymmsNVNfklzU0CEgrBHln8Dha RRWVGY2R4SWN57K2p5CSjDQiQzH37IP+N1WRKeebCR8Brub7mcA3d5kFprvz/BS3 2AvrNUkVnxBw2D4Vk30QB2OMDofIfxLBJmNsZbPsG87BUjaspOB8kPvzSl00RnBr WFICIPwEa5vAFv/N6fOQrkNCH2eNbK3R/2LEcLeNWTRq7UiXu1zL7kNtfjyZ4sXZ C/zE+xMXGkCcLD9QeubMCuW3usRJAQgdNra6pl71FCE1VxyjC30/JBOTjBlXmHPG ICJyXcsD3QQ= =SskM -----END PGP SIGNATURE----- --=-=-=--
Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) Content-Type: text/plain; charset=utf-8 X-Loop: help-debbugs@HIDDEN From: help-debbugs@HIDDEN (GNU bug Tracking System) To: Christopher Baines <mail@HIDDEN> Subject: bug#47174: Acknowledgement ([PATCH 0/2] substitute: Handle closing connections to substitute servers.) Message-ID: <handler.47174.B.161583611726811.ack <at> debbugs.gnu.org> References: <87y2eodxyy.fsf@HIDDEN> X-Gnu-PR-Message: ack 47174 X-Gnu-PR-Package: guix-patches X-Gnu-PR-Keywords: patch Reply-To: 47174 <at> debbugs.gnu.org Date: Mon, 15 Mar 2021 19:22:02 +0000 Thank you for filing a new bug report with debbugs.gnu.org. This is an automatically generated reply to let you know your message has been received. Your message is being forwarded to the package maintainers and other interested parties for their attention; they will reply in due course. Your message has been sent to the package maintainer(s): guix-patches@HIDDEN If you wish to submit further information on this problem, please send it to 47174 <at> debbugs.gnu.org. Please do not send mail to help-debbugs@HIDDEN unless you wish to report a problem with the Bug-tracking system. --=20 47174: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D47174 GNU Bug Tracking System Contact help-debbugs@HIDDEN with problems
X-Loop: help-debbugs@HIDDEN Subject: [bug#47174] [PATCH 2/2] substitute: Handle closing connections to substitute servers. Resent-From: Christopher Baines <mail@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Mon, 15 Mar 2021 19:25:02 +0000 Resent-Message-ID: <handler.47174.B47174.161583629227114 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 47174 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 47174 <at> debbugs.gnu.org Received: via spool by 47174-submit <at> debbugs.gnu.org id=B47174.161583629227114 (code B ref 47174); Mon, 15 Mar 2021 19:25:02 +0000 Received: (at 47174) by debbugs.gnu.org; 15 Mar 2021 19:24:52 +0000 Received: from localhost ([127.0.0.1]:37215 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1lLspg-00073F-Bn for submit <at> debbugs.gnu.org; Mon, 15 Mar 2021 15:24:52 -0400 Received: from mira.cbaines.net ([212.71.252.8]:35640) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1lLspe-000731-CP for 47174 <at> debbugs.gnu.org; Mon, 15 Mar 2021 15:24:50 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id 6C29B27BC54 for <47174 <at> debbugs.gnu.org>; Mon, 15 Mar 2021 19:24:49 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 1dfa5663 for <47174 <at> debbugs.gnu.org>; Mon, 15 Mar 2021 19:24:49 +0000 (UTC) From: Christopher Baines <mail@HIDDEN> Date: Mon, 15 Mar 2021 19:24:49 +0000 Message-Id: <20210315192449.16248-2-mail@HIDDEN> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210315192449.16248-1-mail@HIDDEN> References: <20210315192449.16248-1-mail@HIDDEN> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) When reusing a HTTP connection to fetch multiple nars, and the remote server signals that the connection should be closed. * guix/scripts/substitute.scm (process-substitution): Close connections to substitute servers when a Connection: close header is specified in the response. --- guix/scripts/substitute.scm | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index cb79ea6927..deb6fbdaa2 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -406,7 +406,9 @@ the current output port." (case (uri-scheme uri) ((file) (let ((port (open-file (uri-path uri) "r0b"))) - (values port (stat:size (stat port))))) + (values port + (stat:size (stat port)) + (const #t)))) ; no cleanup to do ((http https) (guard (c ((http-get-error? c) (leave (G_ "download from '~a' failed: ~a, ~s~%") @@ -434,7 +436,12 @@ the current output port." #:buffered? #f #:verify-certificate? #f))) (values raw - (response-content-length response)))))))) + (response-content-length response) + (match (assq 'connection (response-headers response)) + (('connection 'close) + (lambda () + (close-port (response-port response)))) + (_ (const #t)))))))))) (else (leave (G_ "unsupported substitute URI scheme: ~a~%") (uri->string uri))))) @@ -449,7 +456,7 @@ the current output port." (format (current-error-port) (G_ "Downloading ~a...~%") (uri->string uri))) - (let*-values (((raw download-size) + (let*-values (((raw download-size post-fetch-cleanup) ;; 'guix publish' without '--cache' doesn't specify a ;; Content-Length, so DOWNLOAD-SIZE is #f in this case. (fetch uri)) @@ -493,6 +500,10 @@ the current output port." ;; Wait for the reporter to finish. (every (compose zero? cdr waitpid) pids) + ;; Do post-fetch cleanup, maybe closing the HTTP connection if HTTP is + ;; being used, and the connection should be closed + (post-fetch-cleanup) + ;; Skip a line after what 'progress-reporter/file' printed, and another ;; one to visually separate substitutions. (display "\n\n" (current-error-port)) -- 2.30.1
X-Loop: help-debbugs@HIDDEN Subject: [bug#47174] [PATCH 1/2] guix: Alter http-fetch to return the response. References: <87y2eodxyy.fsf@HIDDEN> In-Reply-To: <87y2eodxyy.fsf@HIDDEN> Resent-From: Christopher Baines <mail@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Mon, 15 Mar 2021 19:25:02 +0000 Resent-Message-ID: <handler.47174.B47174.161583629327121 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 47174 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 47174 <at> debbugs.gnu.org Received: via spool by 47174-submit <at> debbugs.gnu.org id=B47174.161583629327121 (code B ref 47174); Mon, 15 Mar 2021 19:25:02 +0000 Received: (at 47174) by debbugs.gnu.org; 15 Mar 2021 19:24:53 +0000 Received: from localhost ([127.0.0.1]:37217 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1lLspg-00073H-Mh for submit <at> debbugs.gnu.org; Mon, 15 Mar 2021 15:24:53 -0400 Received: from mira.cbaines.net ([212.71.252.8]:35638) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1lLspe-000730-CM for 47174 <at> debbugs.gnu.org; Mon, 15 Mar 2021 15:24:50 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id 5FFE527BC52 for <47174 <at> debbugs.gnu.org>; Mon, 15 Mar 2021 19:24:49 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 55fff41d for <47174 <at> debbugs.gnu.org>; Mon, 15 Mar 2021 19:24:49 +0000 (UTC) From: Christopher Baines <mail@HIDDEN> Date: Mon, 15 Mar 2021 19:24:48 +0000 Message-Id: <20210315192449.16248-1-mail@HIDDEN> X-Mailer: git-send-email 2.30.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Rather than just the port and response-content-length. I'm looking at using the response headers within the substitute script to work out when to close the connection. * guix/http-client.scm (http-fetch): Return the response as the second value, rather than the response-content-length. * guix/build/download-nar.scm (download-nar): Adapt accordingly. * guix/build/download.scm (url-fetch): Adapt accordingly. * guix/scripts/substitute.scm (process-substitution): Adapt accordingly. --- guix/build/download-nar.scm | 5 +++-- guix/build/download.scm | 9 ++++++--- guix/http-client.scm | 12 ++++++------ guix/scripts/substitute.scm | 16 +++++++++++----- 4 files changed, 26 insertions(+), 16 deletions(-) diff --git a/guix/build/download-nar.scm b/guix/build/download-nar.scm index 867f3c10bb..fbb5d37c0a 100644 --- a/guix/build/download-nar.scm +++ b/guix/build/download-nar.scm @@ -23,6 +23,7 @@ #:autoload (zlib) (call-with-gzip-input-port) #:use-module (guix progress) #:use-module (web uri) + #:use-module (web response) #:use-module (srfi srfi-11) #:use-module (srfi srfi-26) #:use-module (ice-9 format) @@ -101,7 +102,7 @@ success, #f otherwise." ((url rest ...) (format #t "Trying content-addressed mirror at ~a...~%" (uri-host (string->uri url))) - (let-values (((port size) + (let-values (((port resp) (catch #t (lambda () (http-fetch (string->uri url))) @@ -109,7 +110,7 @@ success, #f otherwise." (values #f #f))))) (if (not port) (loop rest) - (begin + (let ((size (response-content-length resp))) (if size (format #t "Downloading from ~a (~,2h MiB)...~%" url (/ size (expt 2 20.))) diff --git a/guix/build/download.scm b/guix/build/download.scm index f24a1e20df..437184b9cb 100644 --- a/guix/build/download.scm +++ b/guix/build/download.scm @@ -21,6 +21,7 @@ (define-module (guix build download) #:use-module (web uri) #:use-module (web http) + #:use-module (web response) #:use-module ((web client) #:hide (open-socket-for-uri)) #:use-module (web response) #:use-module (guix base64) @@ -647,7 +648,7 @@ otherwise simply ignore them." (case (uri-scheme uri) ((http https) (false-if-exception* - (let-values (((port size) + (let-values (((port resp) (http-fetch uri #:verify-certificate? verify-certificate? #:timeout timeout))) @@ -657,9 +658,11 @@ otherwise simply ignore them." #:buffer-size %http-receive-buffer-size #:reporter (if print-build-trace? (progress-reporter/trace - file (uri->string uri) size) + file (uri->string uri) + (response-content-length resp)) (progress-reporter/file - (uri-abbreviation uri) size))) + (uri-abbreviation uri) + (response-content-length resp)))) (newline))) file))) ((ftp) diff --git a/guix/http-client.scm b/guix/http-client.scm index 2d7458a56e..47076d41f6 100644 --- a/guix/http-client.scm +++ b/guix/http-client.scm @@ -80,11 +80,11 @@ (verify-certificate? #t) (headers '((user-agent . "GNU Guile"))) timeout) - "Return an input port containing the data at URI, and the expected number of -bytes available or #f. If TEXT? is true, the data at URI is considered to be -textual. Follow any HTTP redirection. When BUFFERED? is #f, return an -unbuffered port, suitable for use in `filtered-port'. HEADERS is an alist of -extra HTTP headers. + "Return an input port containing the data at URI, and the HTTP response from +the server. If TEXT? is true, the data at URI is considered to be textual. +Follow any HTTP redirection. When BUFFERED? is #f, return an unbuffered port, +suitable for use in `filtered-port'. HEADERS is an alist of extra HTTP +headers. When KEEP-ALIVE? is true, the connection is marked as 'keep-alive' and PORT is not closed upon completion. @@ -120,7 +120,7 @@ Raise an '&http-get-error' condition if downloading fails." (response-code resp))) (case code ((200) - (values data (response-content-length resp))) + (values data resp)) ((301 ; moved permanently 302 ; found (redirection) 303 ; see other diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index 6892aa999b..cb79ea6927 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -60,6 +60,7 @@ #:use-module (srfi srfi-34) #:use-module (srfi srfi-35) #:use-module (web uri) + #:use-module (web response) #:use-module (guix http-client) #:export (%allow-unauthenticated-substitutes? %error-to-file-descriptor-4? @@ -424,11 +425,16 @@ the current output port." (call-with-connection-error-handling uri (lambda () - (http-fetch uri #:text? #f - #:open-connection open-connection-for-uri/cached - #:keep-alive? #t - #:buffered? #f - #:verify-certificate? #f)))))) + (let-values (((raw response) + (http-fetch + uri + #:text? #f + #:open-connection open-connection-for-uri/cached + #:keep-alive? #t + #:buffered? #f + #:verify-certificate? #f))) + (values raw + (response-content-length response)))))))) (else (leave (G_ "unsupported substitute URI scheme: ~a~%") (uri->string uri))))) -- 2.30.1
X-Loop: help-debbugs@HIDDEN Subject: [bug#47174] [PATCH 0/2] substitute: Handle closing connections to substitute servers. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Mon, 15 Mar 2021 20:37:02 +0000 Resent-Message-ID: <handler.47174.B47174.16158405711297 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 47174 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Christopher Baines <mail@HIDDEN> Cc: 47174 <at> debbugs.gnu.org Received: via spool by 47174-submit <at> debbugs.gnu.org id=B47174.16158405711297 (code B ref 47174); Mon, 15 Mar 2021 20:37:02 +0000 Received: (at 47174) by debbugs.gnu.org; 15 Mar 2021 20:36:11 +0000 Received: from localhost ([127.0.0.1]:37306 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1lLtwh-0000Kr-BG for submit <at> debbugs.gnu.org; Mon, 15 Mar 2021 16:36:11 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35428) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1lLtwf-0000Ke-2s for 47174 <at> debbugs.gnu.org; Mon, 15 Mar 2021 16:36:09 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:51905) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1lLtwZ-0000FY-64; Mon, 15 Mar 2021 16:36:03 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=46674 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from <ludo@HIDDEN>) id 1lLtwY-0007JT-Bt; Mon, 15 Mar 2021 16:36:02 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> References: <20210315192449.16248-1-mail@HIDDEN> <20210315192449.16248-2-mail@HIDDEN> Date: Mon, 15 Mar 2021 21:36:01 +0100 In-Reply-To: <20210315192449.16248-2-mail@HIDDEN> (Christopher Baines's message of "Mon, 15 Mar 2021 19:24:49 +0000") Message-ID: <874khccfym.fsf_-_@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Christopher Baines <mail@HIDDEN> skribis: > When reusing a HTTP connection to fetch multiple nars, and the remote ser= ver > signals that the connection should be closed. > > * guix/scripts/substitute.scm (process-substitution): Close connections to > substitute servers when a Connection: close header is specified in the > response. In the context of <https://issues.guix.gnu.org/47157>, honoring =E2=80=9CConnection: close=E2=80=9D isn=E2=80=99t enough. We need to handl= e the case where the server didn=E2=80=99t express the intent to close the connection but eventu= ally closed it after some time. Does that make sense? Ludo=E2=80=99.
X-Loop: help-debbugs@HIDDEN Subject: [bug#47174] [PATCH 0/2] substitute: Handle closing connections to substitute servers. Resent-From: Christopher Baines <mail@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Mon, 15 Mar 2021 20:43:02 +0000 Resent-Message-ID: <handler.47174.B47174.16158409581930 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 47174 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Cc: 47174 <at> debbugs.gnu.org Received: via spool by 47174-submit <at> debbugs.gnu.org id=B47174.16158409581930 (code B ref 47174); Mon, 15 Mar 2021 20:43:02 +0000 Received: (at 47174) by debbugs.gnu.org; 15 Mar 2021 20:42:38 +0000 Received: from localhost ([127.0.0.1]:37322 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1lLu2v-0000V4-VT for submit <at> debbugs.gnu.org; Mon, 15 Mar 2021 16:42:38 -0400 Received: from mira.cbaines.net ([212.71.252.8]:36892) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1lLu2t-0000Uv-Nw for 47174 <at> debbugs.gnu.org; Mon, 15 Mar 2021 16:42:36 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id 2097327BC52; Mon, 15 Mar 2021 20:42:35 +0000 (GMT) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 71a5997a; Mon, 15 Mar 2021 20:42:34 +0000 (UTC) References: <20210315192449.16248-1-mail@HIDDEN> <20210315192449.16248-2-mail@HIDDEN> <874khccfym.fsf_-_@HIDDEN> User-agent: mu4e 1.4.15; emacs 27.1 From: Christopher Baines <mail@HIDDEN> In-reply-to: <874khccfym.fsf_-_@HIDDEN> Date: Mon, 15 Mar 2021 20:42:34 +0000 Message-ID: <87pn00du85.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s <ludo@HIDDEN> writes: > Christopher Baines <mail@HIDDEN> skribis: > >> When reusing a HTTP connection to fetch multiple nars, and the remote se= rver >> signals that the connection should be closed. >> >> * guix/scripts/substitute.scm (process-substitution): Close connections = to >> substitute servers when a Connection: close header is specified in the >> response. > > In the context of <https://issues.guix.gnu.org/47157>, honoring > =E2=80=9CConnection: close=E2=80=9D isn=E2=80=99t enough. We need to han= dle the case where the > server didn=E2=80=99t express the intent to close the connection but even= tually > closed it after some time. > > Does that make sense? Yeah, of course, this was something I was thinking about in addition to the changes in [1]. 1: https://issues.guix.gnu.org/47160 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmBPxrpfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9Xd0Jg//cyAQJF65Ezu0jNXzIoiUOwNZ5xc2mfsO Rnbn/G3E+LiZnjzEBXRLLXobWuVON8jIwKcrmroVB+BIJbbEzTLCSu4hahmT/0SE osbOr0hC9e0a9XcVTRxj0w9raRfyAhtvOl7IANH/Nl+SfM9ZXU1BOve5VRGPnBwZ 5QCLfh/m0naWoSErs8KTtz4zfFPJPS8K/3F7euiH6wzbgv27LK7e9z1Le4MnmbAJ p47/Qt9i/mBPMYXkCs1uN0ciylC59u12lUPPLAyUk/5J+VBRyGCdy1rxRTVfogRv k7PtVOPvkjSwwYWIRF1YAU/EGlM3IurHj/EnbBLAKM8Nuaa9V27puwzs44TCWCz5 weArBWRuusiYQuCXHoJSb2x14MXWADZacTslEnbVqCqmnsq8osqq8COvySJHg+HU RSKWCXU+7g/ePqEjIfr4Wt/slBrawvl4eP6IYf4Gv8xAV5XeIQNdGBe8ZeqcBK6O yB/R4rd9bnikZwHWTWiximsNdXoSoHCGthi5D+1jFrHOjM/u3FAD/7KOLUjaRcL8 12PVpMFfz2XY0T2obZa/DtZxLKadjAxJZ03L6uyLbC+diNMlyj/hiy3iJ4SRonRp 8A0siXnnPWcHb4yYbufKFs8Fhj9kA3oQsAPnwvkdZGq6bQkD1gZ80msDS7r7jRBn EOtl2pIqcQY= =PQKS -----END PGP SIGNATURE----- --=-=-=--
X-Loop: help-debbugs@HIDDEN Subject: [bug#47174] [PATCH v2 2/2] substitute: Handle closing connections to substitute servers. Resent-From: Christopher Baines <mail@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sun, 16 May 2021 22:12:02 +0000 Resent-Message-ID: <handler.47174.B47174.16212030848677 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 47174 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 47174 <at> debbugs.gnu.org Received: via spool by 47174-submit <at> debbugs.gnu.org id=B47174.16212030848677 (code B ref 47174); Sun, 16 May 2021 22:12:02 +0000 Received: (at 47174) by debbugs.gnu.org; 16 May 2021 22:11:24 +0000 Received: from localhost ([127.0.0.1]:51628 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1liOyq-0002Fs-2w for submit <at> debbugs.gnu.org; Sun, 16 May 2021 18:11:24 -0400 Received: from mira.cbaines.net ([212.71.252.8]:38896) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1liOyo-0002Fe-Ih for 47174 <at> debbugs.gnu.org; Sun, 16 May 2021 18:11:22 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id C995427BC81 for <47174 <at> debbugs.gnu.org>; Sun, 16 May 2021 23:11:21 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 29660e40 for <47174 <at> debbugs.gnu.org>; Sun, 16 May 2021 22:11:21 +0000 (UTC) From: Christopher Baines <mail@HIDDEN> Date: Sun, 16 May 2021 23:11:21 +0100 Message-Id: <20210516221121.16705-2-mail@HIDDEN> X-Mailer: git-send-email 2.30.1 In-Reply-To: <20210516221121.16705-1-mail@HIDDEN> References: <20210516221121.16705-1-mail@HIDDEN> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) When reusing a HTTP connection to fetch multiple nars, and the remote server signals that the connection should be closed. * guix/scripts/substitute.scm (process-substitution): Close connections to substitute servers when a Connection: close header is specified in the response. --- guix/scripts/substitute.scm | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index 96f425eaa0..208b8f1273 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -464,7 +464,9 @@ PORT." (case (uri-scheme uri) ((file) (let ((port (open-file (uri-path uri) "r0b"))) - (values port (stat:size (stat port))))) + (values port + (stat:size (stat port)) + (const #t)))) ; no cleanup to do ((http https) (guard (c ((http-get-error? c) (leave (G_ "download from '~a' failed: ~a, ~s~%") @@ -487,7 +489,12 @@ PORT." #:keep-alive? #t #:buffered? #f))) (values raw - (response-content-length response))))))) + (response-content-length response) + (match (assq 'connection (response-headers response)) + (('connection 'close) + (lambda () + (close-port port))) + (_ (const #t))))))))) (else (leave (G_ "unsupported substitute URI scheme: ~a~%") (uri->string uri))))) @@ -504,7 +511,7 @@ PORT." (format (current-error-port) (G_ "Downloading ~a...~%") (uri->string uri))) - (let*-values (((raw download-size) + (let*-values (((raw download-size post-fetch-cleanup) ;; 'guix publish' without '--cache' doesn't specify a ;; Content-Length, so DOWNLOAD-SIZE is #f in this case. (fetch uri)) @@ -565,6 +572,10 @@ PORT." ;; Wait for the reporter to finish. (every (compose zero? cdr waitpid) pids) + ;; Do post-fetch cleanup, maybe closing the HTTP connection if HTTP is + ;; being used, and the connection should be closed + (post-fetch-cleanup) + ;; Skip a line after what 'progress-reporter/file' printed, and another ;; one to visually separate substitutions. When PRINT-BUILD-TRACE? is ;; true, leave it up to (guix status) to prettify things. -- 2.30.1
X-Loop: help-debbugs@HIDDEN Subject: [bug#47174] [PATCH v2 1/2] guix: Alter http-fetch to return the response. References: <87y2eodxyy.fsf@HIDDEN> In-Reply-To: <87y2eodxyy.fsf@HIDDEN> Resent-From: Christopher Baines <mail@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sun, 16 May 2021 22:12:02 +0000 Resent-Message-ID: <handler.47174.B47174.16212030848685 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 47174 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 47174 <at> debbugs.gnu.org Received: via spool by 47174-submit <at> debbugs.gnu.org id=B47174.16212030848685 (code B ref 47174); Sun, 16 May 2021 22:12:02 +0000 Received: (at 47174) by debbugs.gnu.org; 16 May 2021 22:11:24 +0000 Received: from localhost ([127.0.0.1]:51630 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1liOyq-0002Fu-BD for submit <at> debbugs.gnu.org; Sun, 16 May 2021 18:11:24 -0400 Received: from mira.cbaines.net ([212.71.252.8]:38894) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <mail@HIDDEN>) id 1liOyo-0002Ff-Ih for 47174 <at> debbugs.gnu.org; Sun, 16 May 2021 18:11:23 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id C0B0627BC78 for <47174 <at> debbugs.gnu.org>; Sun, 16 May 2021 23:11:21 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 309302e4 for <47174 <at> debbugs.gnu.org>; Sun, 16 May 2021 22:11:21 +0000 (UTC) From: Christopher Baines <mail@HIDDEN> Date: Sun, 16 May 2021 23:11:20 +0100 Message-Id: <20210516221121.16705-1-mail@HIDDEN> X-Mailer: git-send-email 2.30.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Rather than just the port and response-content-length. I'm looking at using the response headers within the substitute script to work out when to close the connection. * guix/http-client.scm (http-fetch): Return the response as the second value, rather than the response-content-length. * guix/build/download-nar.scm (download-nar): Adapt accordingly. * guix/build/download.scm (url-fetch): Adapt accordingly. * guix/scripts/substitute.scm (process-substitution): Adapt accordingly. --- guix/build/download-nar.scm | 5 +++-- guix/build/download.scm | 9 ++++++--- guix/http-client.scm | 12 ++++++------ guix/scripts/substitute.scm | 12 ++++++++---- 4 files changed, 23 insertions(+), 15 deletions(-) diff --git a/guix/build/download-nar.scm b/guix/build/download-nar.scm index 867f3c10bb..fbb5d37c0a 100644 --- a/guix/build/download-nar.scm +++ b/guix/build/download-nar.scm @@ -23,6 +23,7 @@ #:autoload (zlib) (call-with-gzip-input-port) #:use-module (guix progress) #:use-module (web uri) + #:use-module (web response) #:use-module (srfi srfi-11) #:use-module (srfi srfi-26) #:use-module (ice-9 format) @@ -101,7 +102,7 @@ success, #f otherwise." ((url rest ...) (format #t "Trying content-addressed mirror at ~a...~%" (uri-host (string->uri url))) - (let-values (((port size) + (let-values (((port resp) (catch #t (lambda () (http-fetch (string->uri url))) @@ -109,7 +110,7 @@ success, #f otherwise." (values #f #f))))) (if (not port) (loop rest) - (begin + (let ((size (response-content-length resp))) (if size (format #t "Downloading from ~a (~,2h MiB)...~%" url (/ size (expt 2 20.))) diff --git a/guix/build/download.scm b/guix/build/download.scm index b14db42352..d2006cc1fd 100644 --- a/guix/build/download.scm +++ b/guix/build/download.scm @@ -22,6 +22,7 @@ (define-module (guix build download) #:use-module (web uri) #:use-module (web http) + #:use-module (web response) #:use-module ((web client) #:hide (open-socket-for-uri)) #:use-module (web response) #:use-module (guix base64) @@ -706,7 +707,7 @@ otherwise simply ignore them." (case (uri-scheme uri) ((http https) (false-if-exception* - (let-values (((port size) + (let-values (((port resp) (http-fetch uri #:verify-certificate? verify-certificate? #:timeout timeout))) @@ -716,9 +717,11 @@ otherwise simply ignore them." #:buffer-size %http-receive-buffer-size #:reporter (if print-build-trace? (progress-reporter/trace - file (uri->string uri) size) + file (uri->string uri) + (response-content-length resp)) (progress-reporter/file - (uri-abbreviation uri) size))) + (uri-abbreviation uri) + (response-content-length resp)))) (newline))) file))) ((ftp) diff --git a/guix/http-client.scm b/guix/http-client.scm index 10bc278023..189535079b 100644 --- a/guix/http-client.scm +++ b/guix/http-client.scm @@ -81,11 +81,11 @@ (headers '((user-agent . "GNU Guile"))) (log-port (current-error-port)) timeout) - "Return an input port containing the data at URI, and the expected number of -bytes available or #f. If TEXT? is true, the data at URI is considered to be -textual. Follow any HTTP redirection. When BUFFERED? is #f, return an -unbuffered port, suitable for use in `filtered-port'. HEADERS is an alist of -extra HTTP headers. + "Return an input port containing the data at URI, and the HTTP response from +the server. If TEXT? is true, the data at URI is considered to be textual. +Follow any HTTP redirection. When BUFFERED? is #f, return an unbuffered port, +suitable for use in `filtered-port'. HEADERS is an alist of extra HTTP +headers. When KEEP-ALIVE? is true, the connection is marked as 'keep-alive' and PORT is not closed upon completion. @@ -123,7 +123,7 @@ Raise an '&http-get-error' condition if downloading fails." (response-code resp))) (case code ((200) - (values data (response-content-length resp))) + (values data resp)) ((301 ; moved permanently 302 ; found (redirection) 303 ; see other diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index 8e4eae00b3..96f425eaa0 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -61,6 +61,7 @@ #:use-module (srfi srfi-34) #:use-module (srfi srfi-35) #:use-module (web uri) + #:use-module (web response) #:use-module (guix http-client) #:export (%allow-unauthenticated-substitutes? %reply-file-descriptor @@ -480,10 +481,13 @@ PORT." (uri->string uri)) (warning (G_ "try `--no-substitutes' if the problem persists~%"))) (with-cached-connection uri port - (http-fetch uri #:text? #f - #:port port - #:keep-alive? #t - #:buffered? #f))))) + (let-values (((raw response) + (http-fetch uri #:text? #f + #:port port + #:keep-alive? #t + #:buffered? #f))) + (values raw + (response-content-length response))))))) (else (leave (G_ "unsupported substitute URI scheme: ~a~%") (uri->string uri))))) -- 2.30.1
X-Loop: help-debbugs@HIDDEN Subject: [bug#47174] [PATCH v2 1/2] guix: Alter http-fetch to return the response. Resent-From: Mathieu Othacehe <othacehe@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Mon, 17 May 2021 14:45:01 +0000 Resent-Message-ID: <handler.47174.B47174.162126266822120 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 47174 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Christopher Baines <mail@HIDDEN> Cc: 47174 <at> debbugs.gnu.org Received: via spool by 47174-submit <at> debbugs.gnu.org id=B47174.162126266822120 (code B ref 47174); Mon, 17 May 2021 14:45:01 +0000 Received: (at 47174) by debbugs.gnu.org; 17 May 2021 14:44:28 +0000 Received: from localhost ([127.0.0.1]:53382 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1lieTr-0005kg-M4 for submit <at> debbugs.gnu.org; Mon, 17 May 2021 10:44:27 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38988) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <othacehe@HIDDEN>) id 1lieTo-0005kO-UB for 47174 <at> debbugs.gnu.org; Mon, 17 May 2021 10:44:26 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:44900) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from <othacehe@HIDDEN>) id 1lieTj-0000ET-7e; Mon, 17 May 2021 10:44:19 -0400 Received: from [109.190.253.11] (port=55366 helo=meije) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <othacehe@HIDDEN>) id 1lieTi-0001Ww-Dn; Mon, 17 May 2021 10:44:19 -0400 From: Mathieu Othacehe <othacehe@HIDDEN> References: <87y2eodxyy.fsf@HIDDEN> <20210516221121.16705-1-mail@HIDDEN> Date: Mon, 17 May 2021 16:44:15 +0200 In-Reply-To: <20210516221121.16705-1-mail@HIDDEN> (Christopher Baines's message of "Sun, 16 May 2021 23:11:20 +0100") Message-ID: <87o8d9a08w.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Hello Chis, > * guix/http-client.scm (http-fetch): Return the response as the second value, > rather than the response-content-length. I think there is a missing adaptation in the call-with-nar procedure of the (guix scripts challenge) module. Otherwise, looks fine! Thanks, Mathieu
X-Loop: help-debbugs@HIDDEN Subject: [bug#47174] [PATCH v2 2/2] substitute: Handle closing connections to substitute servers. Resent-From: Mathieu Othacehe <othacehe@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Mon, 17 May 2021 14:47:02 +0000 Resent-Message-ID: <handler.47174.B47174.162126279827377 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 47174 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Christopher Baines <mail@HIDDEN> Cc: 47174 <at> debbugs.gnu.org Received: via spool by 47174-submit <at> debbugs.gnu.org id=B47174.162126279827377 (code B ref 47174); Mon, 17 May 2021 14:47:02 +0000 Received: (at 47174) by debbugs.gnu.org; 17 May 2021 14:46:38 +0000 Received: from localhost ([127.0.0.1]:53386 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1lieVy-00077G-3d for submit <at> debbugs.gnu.org; Mon, 17 May 2021 10:46:38 -0400 Received: from eggs.gnu.org ([209.51.188.92]:39606) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <othacehe@HIDDEN>) id 1lieVv-00072R-UB for 47174 <at> debbugs.gnu.org; Mon, 17 May 2021 10:46:36 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:45020) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from <othacehe@HIDDEN>) id 1lieVq-0001kn-Jb; Mon, 17 May 2021 10:46:30 -0400 Received: from [109.190.253.11] (port=55380 helo=meije) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <othacehe@HIDDEN>) id 1lieVp-0001sr-Qs; Mon, 17 May 2021 10:46:30 -0400 From: Mathieu Othacehe <othacehe@HIDDEN> References: <20210516221121.16705-1-mail@HIDDEN> <20210516221121.16705-2-mail@HIDDEN> Date: Mon, 17 May 2021 16:46:27 +0200 In-Reply-To: <20210516221121.16705-2-mail@HIDDEN> (Christopher Baines's message of "Sun, 16 May 2021 23:11:21 +0100") Message-ID: <87k0nxa058.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > + (match (assq 'connection (response-headers response)) > + (('connection 'close) > + (lambda () > + (close-port port))) You could maybe factorize it in a close-connection? procedure. Out of curiosity, when does the remote server asks for connection closing? Thanks, Mathieu
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.