GNU bug report logs - #70179
[PATCH 0/3] Use system nss-certs in Python.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 70179 in the body.
You can then email your comments to 70179 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: guix-patches <at> gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH 0/3] Use system nss-certs in Python.
Date: Thu,  4 Apr 2024 08:55:05 +0300

It turns out that the Python ecosystem bundles a version of nss-certs.
This patch series should change it so that it uses the system nss-certs
instead.

Efraim Flashner (3):
  gnu: python-certifi: Use system SSL certificates.
  gnu: python-pip: Use system SSL certificates.
  gnu: python: Use system SSL certificates.

 gnu/packages/python-build.scm  | 34 +++++++++++++++++
 gnu/packages/python-crypto.scm | 34 +++++++++++++++++
 gnu/packages/python.scm        | 67 ++++++++++++++++++++++++++++++++++
 3 files changed, 135 insertions(+)


base-commit: 188d18fc47f0d38edfe06e3e5834fa8587bd300b
-- 
Efraim Flashner   <efraim <at> flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Message #8 received at 70179 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: 70179 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH v2 1/3] gnu: python-certifi: Use system SSL certificates.
Date: Thu,  4 Apr 2024 08:56:44 +0300

* gnu/packages/python-crypto.scm (python-certifi)[replacement]: New
field.
(python-certifi/fixed): Provide a python-certifi which only offers to
use the system's SSL certificates.

Change-Id: Ie1871be42988dff3cccfe24bca626149fee0f371
---
 gnu/packages/python-crypto.scm | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/gnu/packages/python-crypto.scm b/gnu/packages/python-crypto.scm
index 3e1472a6c9..05b6c82fd9 100644
--- a/gnu/packages/python-crypto.scm
+++ b/gnu/packages/python-crypto.scm
@@ -469,6 +469,7 @@ (define-public python-certifi
   (package
     (name "python-certifi")
     (version "2022.6.15")
+    (replacement python-certifi/fixed)
     (source (origin
               (method url-fetch)
               (uri (pypi-uri "certifi" version))
@@ -484,6 +485,39 @@ (define-public python-certifi
 is used by the Requests library to verify HTTPS requests.")
     (license license:asl2.0)))
 
+(define python-certifi/fixed
+  (package
+    (inherit python-certifi)
+    (source (origin
+              (inherit (package-source python-certifi))
+              (snippet
+               #~(begin
+                   (delete-file "certifi/cacert.pem")
+                   (delete-file "certifi/core.py")
+                   (with-output-to-file "certifi/core.py"
+                     (lambda _
+                       (display "\"\"\"
+certifi.py
+~~~~~~~~~~
+This module returns the installation location of SSL_CERT_FILE or
+/etc/ssl/certs/ca-certificates.crt, or its contents.
+\"\"\"
+import os
+
+_CA_CERTS = None
+
+try:
+    _CA_CERTS = os.environ [\"SSL_CERT_FILE\"]
+except:
+    _CA_CERTS = os.path.join(\"/etc\", \"ssl\", \"certs\", \"ca-certificates.crt\")
+
+def where() -> str:
+    return _CA_CERTS
+
+def contents() -> str:
+    with open(where(), \"r\", encoding=\"ascii\") as data:
+        return data.read()")))))))))
+
 (define-public python-cryptography-vectors
   (package
     (name "python-cryptography-vectors")
-- 
Efraim Flashner   <efraim <at> flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Message #11 received at 70179 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: 70179 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH v2 2/3] gnu: python-pip: Use system SSL certificates.
Date: Thu,  4 Apr 2024 08:56:45 +0300

* gnu/packages/python-build.scm (python-pip)[replacement]: New
field.
(python-pip/fixed): Provide a python-pip with a patched python-certifi
which only offers to use the system's SSL certificates.

Change-Id: Icea0769b881dc8d760562f0405fa8ea8167a4bd4
---
 gnu/packages/python-build.scm | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm
index 2ea457cdba..5b71d1502e 100644
--- a/gnu/packages/python-build.scm
+++ b/gnu/packages/python-build.scm
@@ -260,6 +260,7 @@ (define-public python-pip
   (package
     (name "python-pip")
     (version "23.1")
+    (replacement python-pip/fixed)
     (source
      (origin
        (method url-fetch)
@@ -277,6 +278,39 @@ (define-public python-pip
 Python Package Index (PyPI).")
     (license license:expat)))
 
+(define python-pip/fixed
+  (package
+    (inherit python-pip)
+    (source (origin
+              (inherit (package-source python-pip))
+              (snippet
+               #~(begin
+                   (delete-file "src/pip/_vendor/certifi/cacert.pem")
+                   (delete-file "src/pip/_vendor/certifi/core.py")
+                   (with-output-to-file "src/pip/_vendor/certifi/core.py"
+                     (lambda _
+                       (display "\"\"\"
+certifi.py
+~~~~~~~~~~
+This module returns the installation location of SSL_CERT_FILE or
+/etc/ssl/certs/ca-certificates.crt, or its contents.
+\"\"\"
+import os
+
+_CA_CERTS = None
+
+try:
+    _CA_CERTS = os.environ [\"SSL_CERT_FILE\"]
+except:
+    _CA_CERTS = os.path.join(\"/etc\", \"ssl\", \"certs\", \"ca-certificates.crt\")
+
+def where() -> str:
+    return _CA_CERTS
+
+def contents() -> str:
+    with open(where(), \"r\", encoding=\"ascii\") as data:
+        return data.read()")))))))))
+
 (define-public python-setuptools
   (package
     (name "python-setuptools")
-- 
Efraim Flashner   <efraim <at> flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Message #14 received at 70179 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: 70179 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH v2 3/3] gnu: python: Use system SSL certificates.
Date: Thu,  4 Apr 2024 08:56:46 +0300

* gnu/packages/python.scm (python)[replacement]: New field.
(python/fixed): Provide a python with a patched python-certifi which
only offers to use the system's SSL certificates.

Change-Id: Ic5bcfb6b32282a7e0628232b1dc4cd60f3f2da52
---
 gnu/packages/python.scm | 67 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 12a5148cb1..3ad4c5d5e7 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -96,6 +96,7 @@ (define-module (gnu packages python)
   #:use-module (guix gexp)
   #:use-module (guix packages)
   #:use-module (guix download)
+  #:use-module (guix search-paths)
   #:use-module (guix utils)
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system trivial)
@@ -424,6 +425,7 @@ (define-public python-3.10
     (inherit python-2)
     (name "python")
     (version "3.10.7")
+    (replacement python-3.10/fixed)
     (source (origin
               (method url-fetch)
               (uri (string-append "https://www.python.org/ftp/python/"
@@ -590,6 +592,7 @@ (define-public python-3.10
            inputs)))
     (native-search-paths
      (list (guix-pythonpath-search-path version)
+           $SSL_CERT_FILE
            ;; Used to locate tzdata by the zoneinfo module introduced in
            ;; Python 3.9.
            (search-path-specification
@@ -982,6 +985,70 @@ (define-public python-3.12
     (properties '((cpe-name . "python")))
     (license license:psfl)))
 
+(define python-3.10/fixed
+  (package
+    (inherit python-3.10)
+    (arguments
+     (substitute-keyword-arguments (package-arguments python-3.10)
+       ((#:phases phases)
+        #~(modify-phases #$phases
+            ;; Also remove the bundled CA certificates.
+            ;; TODO: Rename this phase when merging back into python.
+            (replace 'remove-windows-binaries
+              (lambda _
+                ;; Delete .exe from embedded .whl (zip) files
+                (for-each
+                 (lambda (whl)
+                   (let ((dir "whl-content")
+                         (circa-1980 (* 10 366 24 60 60)))
+                     (mkdir-p dir)
+                     (with-directory-excursion dir
+                       (let ((whl (string-append "../" whl)))
+                         (invoke "unzip" whl)
+                         (for-each delete-file
+                                   (find-files "." "\\.exe$"))
+                         (delete-file whl)
+
+                         ;; Search for cacert.pem, delete it, and rewrite the
+                         ;; file which directs python to look for it.
+                         (let ((cacert (find-files "." "cacert\\.pem")))
+                           (unless (null? cacert)
+                             (let ((certifi (dirname (car cacert))))
+                               (delete-file (string-append certifi "/cacert.pem"))
+                               (delete-file (string-append certifi "/core.py"))
+                               (with-output-to-file (string-append certifi "/core.py")
+                                 (lambda _
+                                   (display "\"\"\"
+certifi.py
+~~~~~~~~~~
+This module returns the installation location of SSL_CERT_FILE or
+/etc/ssl/certs/ca-certificates.crt, or its contents.
+\"\"\"
+import os
+
+_CA_CERTS = None
+
+try:
+    _CA_CERTS = os.environ [\"SSL_CERT_FILE\"]
+except:
+    _CA_CERTS = os.path.join(\"/etc\", \"ssl\", \"certs\", \"ca-certificates.crt\")
+
+def where() -> str:
+    return _CA_CERTS
+
+def contents() -> str:
+    with open(where(), \"r\", encoding=\"ascii\") as data:
+        return data.read()"))))))
+
+                         ;; Reset timestamps to prevent them from ending
+                         ;; up in the Zip archive.
+                         (ftw "." (lambda (file stat flag)
+                                    (utime file circa-1980 circa-1980)
+                                    #t))
+                         (apply invoke "zip" "-X" whl
+                                (find-files "." #:directories? #t))))
+                     (delete-file-recursively dir)))
+                 (find-files "Lib/ensurepip" "\\.whl$"))))))))))
 
 ;; Next 3.x version.
 (define-public python-next python-3.12)
-- 
Efraim Flashner   <efraim <at> flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Message #17 received at 70179 <at> debbugs.gnu.org (full text, mbox):

From: Lars-Dominik Braun <lars <at> 6xq.net>
To: Efraim Flashner <efraim <at> flashner.co.il>
Cc: Tanguy Le Carrour <tanguy <at> bioneland.org>,
 Munyoki Kilyungi <me <at> bonfacemunyoki.com>, 70179 <at> debbugs.gnu.org,
 jgart <jgart <at> dismail.de>, Marius Bakke <marius <at> gnu.org>,
 Sharlatan Hellseher <sharlatanus <at> gmail.com>
Subject: Re: [bug#70179] [PATCH 0/3] Use system nss-certs in Python.
Date: Fri, 5 Apr 2024 10:27:46 +0900

Hi Efraim,

> It turns out that the Python ecosystem bundles a version of nss-certs.
> This patch series should change it so that it uses the system nss-certs
> instead.

I would change the comment at the top of core.py so it mentions this is
a Guix-specific version of certifi.py, so it’s clear the package has
been altered. You probably don’t need `_CA_CERTS = None`, since the
try…except clause covers all cases.

Otherwise LGTM.

Lars

Message #22 received at 70179-done <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Lars-Dominik Braun <lars <at> 6xq.net>
Cc: Tanguy Le Carrour <tanguy <at> bioneland.org>,
 Munyoki Kilyungi <me <at> bonfacemunyoki.com>, 70179-done <at> debbugs.gnu.org,
 jgart <jgart <at> dismail.de>, Marius Bakke <marius <at> gnu.org>,
 Sharlatan Hellseher <sharlatanus <at> gmail.com>
Subject: Re: [bug#70179] [PATCH 0/3] Use system nss-certs in Python.
Date: Sun, 7 Apr 2024 15:06:29 +0300

[Message part 1 (text/plain, inline)]

On Fri, Apr 05, 2024 at 10:27:46AM +0900, Lars-Dominik Braun wrote:
> Hi Efraim,
> 
> > It turns out that the Python ecosystem bundles a version of nss-certs.
> > This patch series should change it so that it uses the system nss-certs
> > instead.
> 
> I would change the comment at the top of core.py so it mentions this is
> a Guix-specific version of certifi.py, so it’s clear the package has
> been altered. You probably don’t need `_CA_CERTS = None`, since the
> try…except clause covers all cases.
> 
> Otherwise LGTM.

I left the initial `_CA_CERTS = None` as a sort of initial declaration
of the variable, since I don't really know python that well and I didn't
think it was correct to declare it inside the try…except.

I added the line at the top of core.py saying it was Guix specific and I
also adjusted the commit message for python mentioning the
$SSL_CERT_FILE in the natives-search-paths.

Then I went to build my home-config and I realized what I'd done with
the native-search-paths in python-3.10 and I moved it to the replacement
python so it wouldn't cause a world rebuild.

Patches pushed!

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #25 received at 70179-done <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Efraim Flashner <efraim <at> flashner.co.il>,
 Lars-Dominik Braun <lars <at> 6xq.net>, 70179-done <at> debbugs.gnu.org,
 Marius Bakke <marius <at> gnu.org>, Munyoki Kilyungi <me <at> bonfacemunyoki.com>,
 Sharlatan Hellseher <sharlatanus <at> gmail.com>,
 Tanguy Le Carrour <tanguy <at> bioneland.org>, jgart <jgart <at> dismail.de>
Subject: Re: bug#70179: [PATCH 0/3] Use system nss-certs in Python.
Date: Sun, 7 Apr 2024 16:41:01 -0400

[Message part 1 (text/plain, inline)]

On Sun, Apr 07, 2024 at 03:06:29PM +0300, Efraim Flashner wrote:
> Patches pushed!

Thanks so much Efraim!

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.