1. Topics
  2. [PATCH] python3: add ${bindir}/python symlink

[PATCH] python3: add ${bindir}/python symlink

Markus Volk
 

Currently /usr/bin/python is provided by the python2 package. If python2
is not installed, the built image lacks a provider for it.
This results in failed scripts when using '/usr/bin/python' shebang.

This patch adds a /usr/bin/python symlink for python3 to fix this issue.
For images containing python2 and python3, the do_rootfs task will fail
because now both of them provide the same file.
To allow this corner case, a DISTRO_FEATURE 'python2' has been introduced=
which,
if set, will prevent python3 from creating the symlink.

The link is created with relative path because using an absolute path wou=
ld fail
for native and nativesdk.

Signed-off-by: Markus Volk <f_l_k@...>
---
meta/recipes-devtools/python/python3/python3-manifest.json | 1 +
meta/recipes-devtools/python/python3_3.11.0.bb | 4 ++++
2 files changed, 5 insertions(+)

diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json b=
/meta/recipes-devtools/python/python3/python3-manifest.json
index 64203cf0fc..7b6f509a45 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -203,6 +203,7 @@
"files": [
"${bindir}/python${PYTHON_MAJMIN}",
"${bindir}/python${PYTHON_MAJMIN}.real",
+ "${bindir}/python",
"${bindir}/python3",
"${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h",
"${libdir}/python${PYTHON_MAJMIN}/UserDict.py",
diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb b/meta/recipe=
s-devtools/python/python3_3.11.0.bb
index 92a1f69320..18b631f79d 100644
--- a/meta/recipes-devtools/python/python3_3.11.0.bb
+++ b/meta/recipes-devtools/python/python3_3.11.0.bb
@@ -144,6 +144,7 @@ do_install:prepend() {
=20
do_install:append:class-target() {
oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h
+ ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf .=
/python3 ${D}${bindir}/python', d)}
}
=20
do_install:append:class-native() {
@@ -156,6 +157,7 @@ do_install:append:class-native() {
# (these often end up too long for the #! parser in the kernel a=
s the
# buffer is 128 bytes long).
ln -s python3-native/python3 ${D}${bindir}/nativepython3
+ ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf .=
/python3-native/python3 ${D}${bindir}/nativepython', d)}
=20
# Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000=
of them
# and the overhead in each recipe-sysroot-native isn't worth it,=
particularly
@@ -213,6 +215,7 @@ do_install:append() {
}
=20
do_install:append:class-nativesdk () {
+ ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./pyt=
hon3 ${D}${bindir}/python', d)}
# Make sure we use /usr/bin/env python
for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do
sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
@@ -376,6 +379,7 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk =3D " =
${MLPREFIX}openssl ${MLPREFIX
=20
# For historical reasons PN is empty and provided by python3-modules
FILES:${PN} =3D ""
+RPROVIDES:${PN} =3D "${@bb.utils.contains('DISTRO_FEATURES', 'python2', =
'', '${bindir}/python', d)}"
RPROVIDES:${PN}-modules =3D "${PN}"
=20
FILES:${PN}-pydoc +=3D "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3=
"
--=20
2.34.1
Alexander Kanavin
 

I have to ask you to resend and drop the DISTRO_FEATURE idea. We do
not want to make the impression that python2 is supported in any way,
and as explained earlier this implies that all scripts that ask for
python would work with both 2 and 3. No, this is a clean break, and
anyone who still has 2.x stuff has to adjust it to ask for
#!/usr/bin/python2.

Alex

On Sun, 20 Nov 2022 at 18:53, Markus Volk <f_l_k@...> wrote:

Currently /usr/bin/python is provided by the python2 package. If python2
is not installed, the built image lacks a provider for it.
This results in failed scripts when using '/usr/bin/python' shebang.

This patch adds a /usr/bin/python symlink for python3 to fix this issue.
For images containing python2 and python3, the do_rootfs task will fail
because now both of them provide the same file.
To allow this corner case, a DISTRO_FEATURE 'python2' has been introduced which,
if set, will prevent python3 from creating the symlink.

The link is created with relative path because using an absolute path would fail
for native and nativesdk.

Signed-off-by: Markus Volk <f_l_k@...>
---
meta/recipes-devtools/python/python3/python3-manifest.json | 1 +
meta/recipes-devtools/python/python3_3.11.0.bb | 4 ++++
2 files changed, 5 insertions(+)

diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json b/meta/recipes-devtools/python/python3/python3-manifest.json
index 64203cf0fc..7b6f509a45 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -203,6 +203,7 @@
"files": [
"${bindir}/python${PYTHON_MAJMIN}",
"${bindir}/python${PYTHON_MAJMIN}.real",
+ "${bindir}/python",
"${bindir}/python3",
"${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h",
"${libdir}/python${PYTHON_MAJMIN}/UserDict.py",
diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb b/meta/recipes-devtools/python/python3_3.11.0.bb
index 92a1f69320..18b631f79d 100644
--- a/meta/recipes-devtools/python/python3_3.11.0.bb
+++ b/meta/recipes-devtools/python/python3_3.11.0.bb
@@ -144,6 +144,7 @@ do_install:prepend() {

do_install:append:class-target() {
oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h
+ ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)}
}

do_install:append:class-native() {
@@ -156,6 +157,7 @@ do_install:append:class-native() {
# (these often end up too long for the #! parser in the kernel as the
# buffer is 128 bytes long).
ln -s python3-native/python3 ${D}${bindir}/nativepython3
+ ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3-native/python3 ${D}${bindir}/nativepython', d)}

# Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 of them
# and the overhead in each recipe-sysroot-native isn't worth it, particularly
@@ -213,6 +215,7 @@ do_install:append() {
}

do_install:append:class-nativesdk () {
+ ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)}
# Make sure we use /usr/bin/env python
for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do
sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
@@ -376,6 +379,7 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk = " ${MLPREFIX}openssl ${MLPREFIX

# For historical reasons PN is empty and provided by python3-modules
FILES:${PN} = ""
+RPROVIDES:${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', '${bindir}/python', d)}"
RPROVIDES:${PN}-modules = "${PN}"

FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3"
--
2.34.1
Markus Volk
 

Am So, 20. Nov 2022 um 19:00:39 +0100 schrieb Alexander Kanavin <alex.kanavin@...>:
I have to ask you to resend and drop the DISTRO_FEATURE idea. We do not want to make the impression that python2 is supported in any way, and as explained earlier this implies that all scripts that ask for python would work with both 2 and 3. No, this is a clean break, and anyone who still has 2.x stuff has to adjust it to ask for #!/usr/bin/python2.

I fully agree 
Khem Raj
 



On Sun, Nov 20, 2022 at 9:52 AM Markus Volk <f_l_k@...> wrote:
Currently /usr/bin/python is provided by the python2 package. If python2
is not installed, the built image lacks a provider for it.
This results in failed scripts when using '/usr/bin/python' shebang.

This patch adds a /usr/bin/python symlink for python3 to fix this issue.
For images containing python2 and python3, the do_rootfs task will fail
because now both of them provide the same file.
To allow this corner case, a DISTRO_FEATURE 'python2' has been introduced which,
if set, will prevent python3 from creating the symlink.

The link is created with relative path because using an absolute path would fail
for native and nativesdk.

What happens if and when python4 comes ?
This might get us into same mess we had migrating python2 to 3



Signed-off-by: Markus Volk <f_l_k@...>
---
 meta/recipes-devtools/python/python3/python3-manifest.json | 1 +
 meta/recipes-devtools/python/python3_3.11.0.bb             | 4 ++++
 2 files changed, 5 insertions(+)

diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json b/meta/recipes-devtools/python/python3/python3-manifest.json
index 64203cf0fc..7b6f509a45 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -203,6 +203,7 @@
         "files": [
             "${bindir}/python${PYTHON_MAJMIN}",
             "${bindir}/python${PYTHON_MAJMIN}.real",
+            "${bindir}/python",
             "${bindir}/python3",
             "${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h",
             "${libdir}/python${PYTHON_MAJMIN}/UserDict.py",
diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb b/meta/recipes-devtools/python/python3_3.11.0.bb
index 92a1f69320..18b631f79d 100644
--- a/meta/recipes-devtools/python/python3_3.11.0.bb
+++ b/meta/recipes-devtools/python/python3_3.11.0.bb
@@ -144,6 +144,7 @@ do_install:prepend() {

 do_install:append:class-target() {
         oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h
+        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)}
 }

 do_install:append:class-native() {
@@ -156,6 +157,7 @@ do_install:append:class-native() {
         # (these often end up too long for the #! parser in the kernel as the
         # buffer is 128 bytes long).
         ln -s python3-native/python3 ${D}${bindir}/nativepython3
+        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3-native/python3 ${D}${bindir}/nativepython', d)}

         # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 of them
         # and the overhead in each recipe-sysroot-native isn't worth it, particularly
@@ -213,6 +215,7 @@ do_install:append() {
 }

 do_install:append:class-nativesdk () {
+    ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)}
     # Make sure we use /usr/bin/env python
     for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do
          sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
@@ -376,6 +379,7 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk = " ${MLPREFIX}openssl ${MLPREFIX

 # For historical reasons PN is empty and provided by python3-modules
 FILES:${PN} = ""
+RPROVIDES:${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', '${bindir}/python', d)}"
 RPROVIDES:${PN}-modules = "${PN}"

 FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3"
--
2.34.1
Alexander Kanavin
 

Python community has no plans for python4 whatsoever. Anything but gentle and steady evolution won’t be well received.


Alex

On Mon 21. Nov 2022 at 0.17, Khem Raj <raj.khem@...> wrote:


On Sun, Nov 20, 2022 at 9:52 AM Markus Volk <f_l_k@...> wrote:
Currently /usr/bin/python is provided by the python2 package. If python2
is not installed, the built image lacks a provider for it.
This results in failed scripts when using '/usr/bin/python' shebang.

This patch adds a /usr/bin/python symlink for python3 to fix this issue.
For images containing python2 and python3, the do_rootfs task will fail
because now both of them provide the same file.
To allow this corner case, a DISTRO_FEATURE 'python2' has been introduced which,
if set, will prevent python3 from creating the symlink.

The link is created with relative path because using an absolute path would fail
for native and nativesdk.

What happens if and when python4 comes ?
This might get us into same mess we had migrating python2 to 3



Signed-off-by: Markus Volk <f_l_k@...>
---
 meta/recipes-devtools/python/python3/python3-manifest.json | 1 +
 meta/recipes-devtools/python/python3_3.11.0.bb             | 4 ++++
 2 files changed, 5 insertions(+)

diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json b/meta/recipes-devtools/python/python3/python3-manifest.json
index 64203cf0fc..7b6f509a45 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -203,6 +203,7 @@
         "files": [
             "${bindir}/python${PYTHON_MAJMIN}",
             "${bindir}/python${PYTHON_MAJMIN}.real",
+            "${bindir}/python",
             "${bindir}/python3",
             "${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h",
             "${libdir}/python${PYTHON_MAJMIN}/UserDict.py",
diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb b/meta/recipes-devtools/python/python3_3.11.0.bb
index 92a1f69320..18b631f79d 100644
--- a/meta/recipes-devtools/python/python3_3.11.0.bb
+++ b/meta/recipes-devtools/python/python3_3.11.0.bb
@@ -144,6 +144,7 @@ do_install:prepend() {

 do_install:append:class-target() {
         oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h
+        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)}
 }

 do_install:append:class-native() {
@@ -156,6 +157,7 @@ do_install:append:class-native() {
         # (these often end up too long for the #! parser in the kernel as the
         # buffer is 128 bytes long).
         ln -s python3-native/python3 ${D}${bindir}/nativepython3
+        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3-native/python3 ${D}${bindir}/nativepython', d)}

         # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 of them
         # and the overhead in each recipe-sysroot-native isn't worth it, particularly
@@ -213,6 +215,7 @@ do_install:append() {
 }

 do_install:append:class-nativesdk () {
+    ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)}
     # Make sure we use /usr/bin/env python
     for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do
          sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
@@ -376,6 +379,7 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk = " ${MLPREFIX}openssl ${MLPREFIX

 # For historical reasons PN is empty and provided by python3-modules
 FILES:${PN} = ""
+RPROVIDES:${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', '${bindir}/python', d)}"
 RPROVIDES:${PN}-modules = "${PN}"

 FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3"
--
2.34.1
Khem Raj
 



On Sun, Nov 20, 2022 at 3:56 PM Alexander Kanavin <alex.kanavin@...> wrote:
Python community has no plans for python4 whatsoever. Anything but gentle and steady evolution won’t be well received.


Good news article although it does not say that python3 and python are analogous and python package does not explicitly install python binary or symlink either so it’s left up to how distros want to presume it. If we want to do that then it would be prudent to see what other distributions are doing in this regard. So we are not alone in case trouble arises 


Alex

On Mon 21. Nov 2022 at 0.17, Khem Raj <raj.khem@...> wrote:


On Sun, Nov 20, 2022 at 9:52 AM Markus Volk <f_l_k@...> wrote:
Currently /usr/bin/python is provided by the python2 package. If python2
is not installed, the built image lacks a provider for it.
This results in failed scripts when using '/usr/bin/python' shebang.

This patch adds a /usr/bin/python symlink for python3 to fix this issue.
For images containing python2 and python3, the do_rootfs task will fail
because now both of them provide the same file.
To allow this corner case, a DISTRO_FEATURE 'python2' has been introduced which,
if set, will prevent python3 from creating the symlink.

The link is created with relative path because using an absolute path would fail
for native and nativesdk.

What happens if and when python4 comes ?
This might get us into same mess we had migrating python2 to 3



Signed-off-by: Markus Volk <f_l_k@...>
---
 meta/recipes-devtools/python/python3/python3-manifest.json | 1 +
 meta/recipes-devtools/python/python3_3.11.0.bb             | 4 ++++
 2 files changed, 5 insertions(+)

diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json b/meta/recipes-devtools/python/python3/python3-manifest.json
index 64203cf0fc..7b6f509a45 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -203,6 +203,7 @@
         "files": [
             "${bindir}/python${PYTHON_MAJMIN}",
             "${bindir}/python${PYTHON_MAJMIN}.real",
+            "${bindir}/python",
             "${bindir}/python3",
             "${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h",
             "${libdir}/python${PYTHON_MAJMIN}/UserDict.py",
diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb b/meta/recipes-devtools/python/python3_3.11.0.bb
index 92a1f69320..18b631f79d 100644
--- a/meta/recipes-devtools/python/python3_3.11.0.bb
+++ b/meta/recipes-devtools/python/python3_3.11.0.bb
@@ -144,6 +144,7 @@ do_install:prepend() {

 do_install:append:class-target() {
         oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h
+        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)}
 }

 do_install:append:class-native() {
@@ -156,6 +157,7 @@ do_install:append:class-native() {
         # (these often end up too long for the #! parser in the kernel as the
         # buffer is 128 bytes long).
         ln -s python3-native/python3 ${D}${bindir}/nativepython3
+        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3-native/python3 ${D}${bindir}/nativepython', d)}

         # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 of them
         # and the overhead in each recipe-sysroot-native isn't worth it, particularly
@@ -213,6 +215,7 @@ do_install:append() {
 }

 do_install:append:class-nativesdk () {
+    ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)}
     # Make sure we use /usr/bin/env python
     for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do
          sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
@@ -376,6 +379,7 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk = " ${MLPREFIX}openssl ${MLPREFIX

 # For historical reasons PN is empty and provided by python3-modules
 FILES:${PN} = ""
+RPROVIDES:${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', '${bindir}/python', d)}"
 RPROVIDES:${PN}-modules = "${PN}"

 FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3"
--
2.34.1
Alexander Kanavin
 

On Mon, 21 Nov 2022 at 16:17, Khem Raj <raj.khem@...> wrote:
Python community has no plans for python4 whatsoever. Anything but gentle and steady evolution won’t be well received.

https://www.techrepublic.com/article/programming-languages-why-python-4-0-will-probably-never-arrive-according-to-its-creator/

Good news article although it does not say that python3 and python are analogous and python package does not explicitly install python binary or symlink either so it’s left up to how distros want to presume it. If we want to do that then it would be prudent to see what other distributions are doing in this regard. So we are not alone in case trouble arises
Fedora has done it 3 years ago:
https://pagure.io/fedora-docs/release-notes/pull-request/410#request_diff

Alex
Alexander Kanavin
 

And here's debian:
https://wiki.debian.org/Python/FAQ#Python_2_support

On my Debian systems, indeed, /usr/bin/python is absent.

Alex

On Mon, 21 Nov 2022 at 18:18, Alexander Kanavin via
lists.openembedded.org <alex.kanavin=gmail.com@...>
wrote:


On Mon, 21 Nov 2022 at 16:17, Khem Raj <raj.khem@...> wrote:
Python community has no plans for python4 whatsoever. Anything but gentle and steady evolution won’t be well received.

https://www.techrepublic.com/article/programming-languages-why-python-4-0-will-probably-never-arrive-according-to-its-creator/

Good news article although it does not say that python3 and python are analogous and python package does not explicitly install python binary or symlink either so it’s left up to how distros want to presume it. If we want to do that then it would be prudent to see what other distributions are doing in this regard. So we are not alone in case trouble arises
Fedora has done it 3 years ago:
https://pagure.io/fedora-docs/release-notes/pull-request/410#request_diff

Alex
Markus Volk
 

On Mon, Nov 21 2022 at 06:48:07 PM +0100, Alexander Kanavin <alex.kanavin@...> wrote:
On my Debian systems, indeed, /usr/bin/python is absent.

Debian has a package for this
Ross Burton
 

On 22 Nov 2022, at 04:42, Markus Volk via lists.openembedded.org <f_l_k=t-online.de@...> wrote:
On Mon, Nov 21 2022 at 06:48:07 PM +0100, Alexander Kanavin <alex.kanavin@...> wrote:
On my Debian systems, indeed, /usr/bin/python is absent.
Debian has a package for this
https://packages.debian.org/bookworm/python-is-python3
And that’s a perfectly good solution that I like.

Personally, I think people should forget that /usr/bin/python exists: the recommendation from Python is to call python2 or python3. I can quote from PEP-0394:

“””
Depending on a distribution or system configuration, python may or may not be installed. If python is installed its target interpreter may refer to python2 or python3.”

…

• Distributors may choose to set the behavior of the python command as follows:
• python2,
• python3,
• not provide python command,
• allow python to be configurable by an end user or a system administrator.
“””

We’ve picked option 3. As per Python upstream, that’s absolutely fine.

If you have a serious need that /usr/bin/python exists, and is a symlink to python3, then could you not make a simple recipe that RDEPENDS on python3 and ships just a /usr/bin/python -> python3 symlink? You can even put this in your layer to avoid having to debate it with the oe-core maintainers.

Ross
Alexander Kanavin
 

The serious need is that we need to patch all the scripts that ask for python to add a 3 to it. And there will be more of these going forward, not less. I’d rather just always have python available. Not a problem worth deliberating over to be honest.

Alex

On Tue 22. Nov 2022 at 18.41, Ross Burton <Ross.Burton@...> wrote:
On 22 Nov 2022, at 04:42, Markus Volk via lists.openembedded.org <f_l_k=t-online.de@...> wrote:
> On Mon, Nov 21 2022 at 06:48:07 PM +0100, Alexander Kanavin <alex.kanavin@...> wrote:
>> On my Debian systems, indeed, /usr/bin/python is absent.
>
> Debian has a package for this
> https://packages.debian.org/bookworm/python-is-python3

And that’s a perfectly good solution that I like.

Personally, I think people should forget that /usr/bin/python exists: the recommendation from Python is to call python2 or python3.  I can quote from PEP-0394:

“””
Depending on a distribution or system configuration, python may or may not be installed. If python is installed its target interpreter may refer to python2 or python3.”

…

    • Distributors may choose to set the behavior of the python command as follows:
        • python2,
        • python3,
        • not provide python command,
        • allow python to be configurable by an end user or a system administrator.
“””

We’ve picked option 3.  As per Python upstream, that’s absolutely fine.

If you have a serious need that /usr/bin/python exists, and is a symlink to python3, then could you not make a simple recipe that RDEPENDS on python3 and ships just a /usr/bin/python -> python3 symlink?  You can even put this in your layer to avoid having to debate it with the oe-core maintainers.

Ross
Markus Volk
 

I use a bbappend for python3 in meta-wayland for almost a year now because i had issues installing python scripts with /usr/bin/python shebang e.g. here in sway:

Issue has been that, if adding this script to do_install(), package_qa failed because of the missing /usr/bin/python provider.

Ran into this quite a few times in the past and then created the symlink instead of patching shebangs

I just wanted to mention, that having a symlink would be an alternative to patching shebangs and as i was asked to send a patch, i did. But I'm fine with whatever is done in oe-core.

Basically I also think the best solution would be, upstream would deprecate /usr/bin/python usage. Unfortunaltey it's not, and so there will always be people using it.

Am Di, 22. Nov 2022 um 17:41:22 +0000 schrieb Ross Burton <ross.burton@...>:

On 22 Nov 2022, at 04:42, Markus Volk via lists.openembedded.org <f_l_k=t-online.de@...> wrote:
On Mon, Nov 21 2022 at 06:48:07 PM +0100, Alexander Kanavin <alex.kanavin@...> wrote:
On my Debian systems, indeed, /usr/bin/python is absent.
Debian has a package for this https://packages.debian.org/bookworm/python-is-python3
And that’s a perfectly good solution that I like. Personally, I think people should forget that /usr/bin/python exists: the recommendation from Python is to call python2 or python3. I can quote from PEP-0394: “”” Depending on a distribution or system configuration, python may or may not be installed. If python is installed its target interpreter may refer to python2 or python3.” … • Distributors may choose to set the behavior of the python command as follows: • python2, • python3, • not provide python command, • allow python to be configurable by an end user or a system administrator. “”” We’ve picked option 3. As per Python upstream, that’s absolutely fine. If you have a serious need that /usr/bin/python exists, and is a symlink to python3, then could you not make a simple recipe that RDEPENDS on python3 and ships just a /usr/bin/python -> python3 symlink? You can even put this in your layer to avoid having to debate it with the oe-core maintainers. Ross
Richard Purdie
 

On Tue, 2022-11-22 at 19:03 +0100, Alexander Kanavin wrote:
The serious need is that we need to patch all the scripts that ask
for python to add a 3 to it. And there will be more of these going
forward, not less. I’d rather just always have python available. Not
a problem worth deliberating over to be honest.
I don't think it is that simple. 

a) If we take the change, we're no longer explicit in what we're using
which means users can be taken by surprise when a python2 script finds
python3 and could sometimes break.

b) Our current approach means that people have at least looked at and
decided something is python3 safe.

c) It means that anyone still using python2 components is left with no
way to make things work. Whilst nobody should be using python2 anymore,
I suspect some still are and this will be a huge problem to them.

d) If there ever is a python4, we'll have to redo a lot of the work
we're done to reach this point where things are fairly clean.


Rightly or wrongly, this patch will cause large amounts of pain for
some portion of our userbase and I'm not sure we have enough
justification to do that. That pain wouldn't likely be realised for
some time either :/.

Cheers,

Richard
Alexander Kanavin
 

On Tue, 22 Nov 2022 at 19:14, Richard Purdie
<richard.purdie@...> wrote:
Rightly or wrongly, this patch will cause large amounts of pain for
some portion of our userbase and I'm not sure we have enough
justification to do that. That pain wouldn't likely be realised for
some time either :/.
I have to point out that meta-python2 hasn't even received a
compatibility update for langsdale:
https://git.openembedded.org/meta-python2

While this might be 'too soon' to conclude that python2 is truly dead,
maybe a year (or two, or three) from now it won't be. Fedora has
already made the switch, Debian will follow, and honestly, I just
can't muster any sympathy for python2 users anymore. You can't push
back paying off technical debt forever and expect others to
accommodate you.

Let me propose this: a PACKAGECONFIG for the python recipe that adds
and installs the symlink in a dedicated package. We can keep it off
for now, but somewhere down the line we could revisit that against
established practice and what PEPs say then.

Alex
Alexandre Belloni
 

On 22/11/2022 19:32:17+0100, Alexander Kanavin wrote:
On Tue, 22 Nov 2022 at 19:14, Richard Purdie
<richard.purdie@...> wrote:
Rightly or wrongly, this patch will cause large amounts of pain for
some portion of our userbase and I'm not sure we have enough
justification to do that. That pain wouldn't likely be realised for
some time either :/.
I have to point out that meta-python2 hasn't even received a
compatibility update for langsdale:
https://git.openembedded.org/meta-python2

While this might be 'too soon' to conclude that python2 is truly dead,
maybe a year (or two, or three) from now it won't be. Fedora has
already made the switch, Debian will follow, and honestly, I just
can't muster any sympathy for python2 users anymore. You can't push
back paying off technical debt forever and expect others to
accommodate you.

Let me propose this: a PACKAGECONFIG for the python recipe that adds
and installs the symlink in a dedicated package. We can keep it off
for now, but somewhere down the line we could revisit that against
established practice and what PEPs say then.
I actually like the idea of having a python-is-python3 package so that
affected recipes could simply add it to their dependencies. That would
make it explicit that python is not python2.

Alex



--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
Peter Kjellerstedt
 

-----Original Message-----
From: openembedded-core@... <openembedded-
core@...> On Behalf Of Alexandre Belloni via
lists.openembedded.org
Sent: den 22 november 2022 23:29
To: Alexander Kanavin <alex.kanavin@...>
Cc: Richard Purdie <richard.purdie@...>; Ross Burton
<Ross.Burton@...>; Khem Raj <raj.khem@...>; Markus Volk
<f_l_k@...>; openembedded-core@...
Subject: Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

On 22/11/2022 19:32:17+0100, Alexander Kanavin wrote:
On Tue, 22 Nov 2022 at 19:14, Richard Purdie
<richard.purdie@...> wrote:
Rightly or wrongly, this patch will cause large amounts of pain for
some portion of our userbase and I'm not sure we have enough
justification to do that. That pain wouldn't likely be realised for
some time either :/.
I have to point out that meta-python2 hasn't even received a
compatibility update for langsdale:
https://git.openembedded.org/meta-python2

While this might be 'too soon' to conclude that python2 is truly dead,
maybe a year (or two, or three) from now it won't be. Fedora has
already made the switch, Debian will follow, and honestly, I just
can't muster any sympathy for python2 users anymore. You can't push
back paying off technical debt forever and expect others to
accommodate you.

Let me propose this: a PACKAGECONFIG for the python recipe that adds
and installs the symlink in a dedicated package. We can keep it off
for now, but somewhere down the line we could revisit that against
established practice and what PEPs say then.
I actually like the idea of having a python-is-python3 package so that
affected recipes could simply add it to their dependencies. That would
make it explicit that python is not python2.
Please not that in the Debian case, nothing depends on python-is-python3.
It is only provided so that users can manually install it if they have
a Python script not provided by Debian that requires it.

And note that Debian also provides a python-is-python2 package, and
obviously things would eventually fail if packages actually were to
depend on either as this would open up for conflicts.

Thus all Python scripts in packages provided by Debian are required to
explicitly specify either python2 or python3 on the shebang line.

Alex
//Peter
Ross Burton
 

On 22 Nov 2022, at 18:03, Alexander Kanavin <alex.kanavin@...> wrote:

The serious need is that we need to patch all the scripts that ask for python to add a 3 to it. And there will be more of these going forward, not less. I’d rather just always have python available. Not a problem worth deliberating over to be honest.
But as per PEP 0384, /usr/bin/python could be python2, or not there at all. Software should use python3.

Ross
Ross Burton
 

On 22 Nov 2022, at 18:32, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@...> wrote:
While this might be 'too soon' to conclude that python2 is truly dead,
maybe a year (or two, or three) from now it won't be. Fedora has
already made the switch, Debian will follow, and honestly, I just
can't muster any sympathy for python2 users anymore. You can't push
back paying off technical debt forever and expect others to
accommodate you.
I absolutely agree with you here, which is why the one true binary should be /usr/bin/python3.

Ross
Alexander Kanavin
 

On Wed, 23 Nov 2022 at 10:59, Ross Burton <Ross.Burton@...> wrote:
The serious need is that we need to patch all the scripts that ask for python to add a 3 to it. And there will be more of these going forward, not less. I’d rather just always have python available. Not a problem worth deliberating over to be honest.
But as per PEP 0384, /usr/bin/python could be python2, or not there at all. Software should use python3.
But would anyone pay attention to the PEP when writing new scripts? If
/usr/bin/python is available by default, as it is already on Fedora,
and probably will be on Debian (in testing/sid there is no choice
between 2 and 3 anymore - 2 is gone, and that clears the road to have
python->python3 installed by default), then it's not unreasonable to
have the script use that. I wouldn't want to know or care about
python's historical baggage. That's why I'm saying we're going to see
more upstreams that are just having #!/usr/bin/python and don't give a
damn, even when you point it out to them.

Alex
Khem Raj
 

On Wed, Nov 23, 2022 at 2:55 AM Alexander Kanavin
<alex.kanavin@...> wrote:

On Wed, 23 Nov 2022 at 10:59, Ross Burton <Ross.Burton@...> wrote:
The serious need is that we need to patch all the scripts that ask for python to add a 3 to it. And there will be more of these going forward, not less. I’d rather just always have python available. Not a problem worth deliberating over to be honest.
But as per PEP 0384, /usr/bin/python could be python2, or not there at all. Software should use python3.
But would anyone pay attention to the PEP when writing new scripts?
they should be pointed to PEP and I hope python tutors teach it from day 1.

If
/usr/bin/python is available by default, as it is already on Fedora,
There perhaps is a reason for that in Fedora, but it is in conflict
with what PEP0384 is saying

and probably will be on Debian (in testing/sid there is no choice
between 2 and 3 anymore - 2 is gone, and that clears the road to have
python->python3 installed by default), then it's not unreasonable to
have the script use that. I wouldn't want to know or care about
python's historical baggage. That's why I'm saying we're going to see
more upstreams that are just having #!/usr/bin/python and don't give a
damn, even when you point it out to them.
If you want to ignore the baggage then know that there is no
/usr/bin/python anymore.

Alex
1 - 20 of 21