[PATCH] bitbake: git fetcher: use urllib quote ...


Thilo C.
 

to use the path url-compatible. This needs to happen before the shell quotation happens.

Without this commit, spaces in the clone URL will be used as " " and not as "%20" which will fail.
This commit changes the " " in the URL to "%20" when it is a http or https url.

Signed-off-by: Thilo Cestonaro <thilo.cestonaro@...>
---
 lib/bb/fetch2/git.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
index 578edc59..dc7f848d 100644
--- a/lib/bb/fetch2/git.py
+++ b/lib/bb/fetch2/git.py
@@ -66,6 +66,7 @@ import re
 import shlex
 import subprocess
 import tempfile
+import urllib
 import bb
 import bb.progress
 from contextlib import contextmanager
@@ -697,7 +698,12 @@ class Git(FetchMethod):
             username = ud.user + '@'
         else:
             username = ""
-        return "%s://%s%s%s" % (ud.proto, username, ud.host, ud.path)
+
+        path = ud.path
+        if ud.proto in [ 'http', 'https' ]:
+            path = urllib.parse.quote(ud.path)
+
+        return "%s://%s%s%s" % (ud.proto, username, ud.host, path)
 
     def _revision_key(self, ud, d, name):
         """
--
2.37.2


Michael Opdenacker
 

Hi Thilo,


Thanks for the patch!


On 22.03.23 at 09:37, Thilo C. via lists.openembedded.org wrote:
to use the path url-compatible. This needs to happen before the shell quotation happens.

Without this commit, spaces in the clone URL will be used as " " and not as "%20" which will fail.
This commit changes the " " in the URL to "%20" when it is a http or https url.

Signed-off-by: Thilo Cestonaro <thilo.cestonaro@...>

Oops, are you sure it applies to the latest "master" branch? I tried and it didn't, though I don't see why...


Another issue with your patch is the commit author name, which will be "Thilo C. via lists.openembedded.org <thilo.cestonaro=thalesgroup.com@...>" instead of "Thilo Cestonaro <thilo.cestonaro@...>".


See https://www.openembedded.org/wiki/How_to_submit_a_patch_to_OpenEmbedded#Fixing_your_From_identity for details and a solution.


Don't hesitate to try to send the patch again to me privately before resending to the mailing list...


Thanks again
Cheers
Michael.

--
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


Alexandre Belloni
 

Hello,

This doesn't apply on master, can you rebase?

Thanks

On 22/03/2023 08:37:15+0000, Thilo C. via lists.openembedded.org wrote:
to use the path url-compatible. This needs to happen before the shell quotation happens.

Without this commit, spaces in the clone URL will be used as " " and not as "%20" which will fail.
This commit changes the " " in the URL to "%20" when it is a http or https url.

Signed-off-by: Thilo Cestonaro <thilo.cestonaro@...>
---
lib/bb/fetch2/git.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
index 578edc59..dc7f848d 100644
--- a/lib/bb/fetch2/git.py
+++ b/lib/bb/fetch2/git.py
@@ -66,6 +66,7 @@ import re
import shlex
import subprocess
import tempfile
+import urllib
import bb
import bb.progress
from contextlib import contextmanager
@@ -697,7 +698,12 @@ class Git(FetchMethod):
username = ud.user + '@'
else:
username = ""
- return "%s://%s%s%s" % (ud.proto, username, ud.host, ud.path)
+
+ path = ud.path
+ if ud.proto in [ 'http', 'https' ]:
+ path = urllib.parse.quote(ud.path)
+
+ return "%s://%s%s%s" % (ud.proto, username, ud.host, path)

def _revision_key(self, ud, d, name):
"""
--
2.37.2


--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


Thilo C.
 

Hi Michael

Sorry for the late answer.
I'm currently trying to be able to directly send the patches via Git but our IT does not allow it.

So here is another try with Outlook. And hopefully it does not change anything, like last time. As I think it was Outlooks fault that the patch did not apply.

This time I gzipped it. Hopefully this is ok for you!

Cheers
Thilo

________________________________________
Von: Michael Opdenacker <michael.opdenacker@...>
Gesendet: Mittwoch, 22. März 2023 11:40
An: CESTONARO Thilo
Cc: bitbake-devel@...
Betreff: Re: [bitbake-devel] [PATCH] bitbake: git fetcher: use urllib quote ...

Hi Thilo,


Thanks for the patch!


On 22.03.23 at 09:37, Thilo C. via lists.openembedded.org wrote:
to use the path url-compatible. This needs to happen before the shell
quotation happens.

Without this commit, spaces in the clone URL will be used as " " and
not as "%20" which will fail.
This commit changes the " " in the URL to "%20" when it is a http or
https url.

Signed-off-by: Thilo Cestonaro <thilo.cestonaro@...>

Oops, are you sure it applies to the latest "master" branch? I tried and
it didn't, though I don't see why...


Another issue with your patch is the commit author name, which will be
"Thilo C. via lists.openembedded.org
<thilo.cestonaro=thalesgroup.com@...>" instead of
"Thilo Cestonaro <thilo.cestonaro@...>".


See
https://www.openembedded.org/wiki/How_to_submit_a_patch_to_OpenEmbedded#Fixing_your_From_identity
for details and a solution.


Don't hesitate to try to send the patch again to me privately before
resending to the mailing list...


Thanks again
Cheers
Michael.

--
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


Richard Purdie
 

On Wed, 2023-03-22 at 08:37 +0000, Thilo C. via lists.openembedded.org
wrote:
to use the path url-compatible. This needs to happen before the shell
quotation happens.

Without this commit, spaces in the clone URL will be used as " " and
not as "%20" which will fail.
This commit changes the " " in the URL to "%20" when it is a http or
https url.

Signed-off-by: Thilo Cestonaro <thilo.cestonaro@...>
---
 lib/bb/fetch2/git.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
index 578edc59..dc7f848d 100644
--- a/lib/bb/fetch2/git.py
+++ b/lib/bb/fetch2/git.py
@@ -66,6 +66,7 @@ import re
 import shlex
 import subprocess
 import tempfile
+import urllib
 import bb
 import bb.progress
 from contextlib import contextmanager
@@ -697,7 +698,12 @@ class Git(FetchMethod):
             username = ud.user + '@'
         else:
             username = ""
-        return "%s://%s%s%s" % (ud.proto, username, ud.host,
ud.path)
+
+        path = ud.path
+        if ud.proto in [ 'http', 'https' ]:
+            path = urllib.parse.quote(ud.path)
+
+        return "%s://%s%s%s" % (ud.proto, username, ud.host, path)
 
     def _revision_key(self, ud, d, name):
         """
I'd like to ensure that we add test coverage to "bitbake-selftest" to
cover this situation (see lib/bb/tests/fetch.py).

We tend not to accept fixes like this unless we have test coverage to
prevent future regressions.

Cheers,

Richard