[PATCH 4/5] devtool: add support for multiple git url inside a cargo based recipe


Frederic Martinsons
 

From: Frederic Martinsons <frederic.martinsons@...>

Without that, the possible git urls that are in SRC_URI of a recipe
are sucked up during a devtool modify on the recipe.

Let's treat these dependency as local ones by using a new
dedicated variable to be set in the recipe and which will
contains the url of such dependencies.

Signed-off-by: Frederic Martinsons <frederic.martinsons@...>
---
meta/classes-recipe/cargo_common.bbclass | 7 +++++++
meta/classes/externalsrc.bbclass | 3 +++
meta/recipes-extended/rust-example/zvariant-crates.inc | 5 +++++
3 files changed, 15 insertions(+)

diff --git a/meta/classes-recipe/cargo_common.bbclass b/meta/classes-recipe/cargo_common.bbclass
index 82ab25b59c..358c7d7ed2 100644
--- a/meta/classes-recipe/cargo_common.bbclass
+++ b/meta/classes-recipe/cargo_common.bbclass
@@ -33,6 +33,13 @@ CARGO_DISABLE_BITBAKE_VENDORING ?= "0"
# Used by libstd-rs to point to the vendor dir included in rustc src
CARGO_VENDORING_DIRECTORY ?= "${CARGO_HOME}/bitbake"

+# Contains a list of crate that are fetch on a git
+# repository instead of from crates.io
+# This is useful when devtool'ing a rust recipe
+# and still patch the path of these dependencies
+# to get these dependencies locally
+CARGO_GIT_DEPENDENCIES ?= ""
+
CARGO_RUST_TARGET_CCLD ?= "${RUST_TARGET_CCLD}"
cargo_common_do_configure () {
mkdir -p ${CARGO_HOME}/bitbake
diff --git a/meta/classes/externalsrc.bbclass b/meta/classes/externalsrc.bbclass
index 26c5803ee6..b8ffeb8614 100644
--- a/meta/classes/externalsrc.bbclass
+++ b/meta/classes/externalsrc.bbclass
@@ -65,6 +65,7 @@ python () {

local_srcuri = []
fetch = bb.fetch2.Fetch((d.getVar('SRC_URI') or '').split(), d)
+ cargo_git_deps = (d.getVar('CARGO_GIT_DEPENDENCIES') or '').split()
for url in fetch.urls:
url_data = fetch.ud[url]
parm = url_data.parm
@@ -72,6 +73,8 @@ python () {
url_data.type == 'npmsw' or url_data.type == 'crate' or
'type' in parm and parm['type'] == 'kmeta'):
local_srcuri.append(url)
+ if url in cargo_git_deps:
+ local_srcuri.append(url)

d.setVar('SRC_URI', ' '.join(local_srcuri))

diff --git a/meta/recipes-extended/rust-example/zvariant-crates.inc b/meta/recipes-extended/rust-example/zvariant-crates.inc
index 297a784661..074ac1934e 100644
--- a/meta/recipes-extended/rust-example/zvariant-crates.inc
+++ b/meta/recipes-extended/rust-example/zvariant-crates.inc
@@ -125,11 +125,16 @@ SRC_URI += " \
crate://crates.io/winnow/0.4.0 \
crate://crates.io/zvariant_derive/3.12.0 \
crate://crates.io/zvariant_utils/1.0.0 \
+"
+
+CARGO_GIT_DEPENDENCIES = " \
git://github.com/gtk-rs/glib;protocol=https;nobranch=1;name=glib;destsuffix=glib \
git://github.com/gtk-rs/sys;protocol=https;nobranch=1;name=glib-sys;destsuffix=glib-sys;subpath=glib-sys \
git://github.com/gtk-rs/sys;protocol=https;nobranch=1;name=gobject-sys;destsuffix=gobject-sys;subpath=gobject-sys \
"

+SRC_URI += "${CARGO_GIT_DEPENDENCIES}"
+
SRCREV_FORMAT .= "_glib"
SRCREV_glib = "c9ee583cea07830c099cdcccd33eda9ef705ea93"

--
2.34.1


Alexander Kanavin
 

On Sat, 25 Mar 2023 at 10:01, Frederic Martinsons
<frederic.martinsons@...> wrote:
+ if url in cargo_git_deps:
+ local_srcuri.append(url)

d.setVar('SRC_URI', ' '.join(local_srcuri))
What actually happens without this patch, and what does the patch
change? The commit message isn't making it clear.

Adding special cases based on custom variables is also not great; why
would git:// entries in rust recipes be treated differently than
git:// entries in other recipes?

Alex


Alexander Kanavin
 

On Sat, 25 Mar 2023 at 10:01, Frederic Martinsons
<frederic.martinsons@...> wrote:
@@ -72,6 +73,8 @@ python () {
url_data.type == 'npmsw' or url_data.type == 'crate' or
'type' in parm and parm['type'] == 'kmeta'):
local_srcuri.append(url)
+ if url in cargo_git_deps:
+ local_srcuri.append(url)
Come to think of it, it's better to use the same type parameter with a
special value as in kernel recipes:

git://.....;type=cargo-dep;...

This would avoid the ugly special variable.

Alex


Frederic Martinsons
 

Sorry for making it not clear.

Without this patch, git url that are in the original SRC_URI of the recipe are simply
remove from the final SRC_URI under devtool and so it prevents
cargo_common_do_patch_paths of cargo_common.bbclass to patch these
component to fetch them locally.



Le sam. 25 mars 2023, 19:55, Alexander Kanavin <alex.kanavin@...> a écrit :
On Sat, 25 Mar 2023 at 10:01, Frederic Martinsons
<frederic.martinsons@...> wrote:
> +            if url in cargo_git_deps:
> +                local_srcuri.append(url)
>
>          d.setVar('SRC_URI', ' '.join(local_srcuri))

What actually happens without this patch, and what does the patch
change? The commit message isn't making it clear.

Adding special cases based on custom variables is also not great; why
would git:// entries in rust recipes be treated differently than
git:// entries in other recipes?

Alex


Frederic Martinsons
 

Ah, good idea, I'll try that and provide a V2 of this series if it fits.


On Sat, 25 Mar 2023 at 20:02, Alexander Kanavin <alex.kanavin@...> wrote:
On Sat, 25 Mar 2023 at 10:01, Frederic Martinsons
<frederic.martinsons@...> wrote:
> @@ -72,6 +73,8 @@ python () {
>                      url_data.type == 'npmsw' or url_data.type == 'crate' or
>                      'type' in parm and parm['type'] == 'kmeta'):
>                  local_srcuri.append(url)
> +            if url in cargo_git_deps:
> +                local_srcuri.append(url)

Come to think of it, it's better to use the same type parameter with a
special value as in kernel recipes:

git://.....;type=cargo-dep;...

This would avoid the ugly special variable.

Alex


Frederic Martinsons
 

I submit a V2 series where I made modifications you suggested, I choose to use a more
generic type (git-dependency instead of cargo-dep) since I foresee that could be used also

For the moment, I let the example recipe in oe-core because I don't know where it should
go (or even if I should keep it or remove it)

On Sun, 26 Mar 2023 at 07:15, Frédéric Martinsons <frederic.martinsons@...> wrote:
Ah, good idea, I'll try that and provide a V2 of this series if it fits.

On Sat, 25 Mar 2023 at 20:02, Alexander Kanavin <alex.kanavin@...> wrote:
On Sat, 25 Mar 2023 at 10:01, Frederic Martinsons
<frederic.martinsons@...> wrote:
> @@ -72,6 +73,8 @@ python () {
>                      url_data.type == 'npmsw' or url_data.type == 'crate' or
>                      'type' in parm and parm['type'] == 'kmeta'):
>                  local_srcuri.append(url)
> +            if url in cargo_git_deps:
> +                local_srcuri.append(url)

Come to think of it, it's better to use the same type parameter with a
special value as in kernel recipes:

git://.....;type=cargo-dep;...

This would avoid the ugly special variable.

Alex


Alexander Kanavin
 

I'm fine with v2, except for the zvariant recipe, that should probably
go to meta-rust (which itself should go to meta-oe, with cleanups).

Alex

On Sun, 26 Mar 2023 at 16:58, Frédéric Martinsons
<frederic.martinsons@...> wrote:

I submit a V2 series where I made modifications you suggested, I choose to use a more
generic type (git-dependency instead of cargo-dep) since I foresee that could be used also
for npm based recipe (see https://bugzilla.yoctoproject.org/show_bug.cgi?id=11015)

For the moment, I let the example recipe in oe-core because I don't know where it should
go (or even if I should keep it or remove it)

On Sun, 26 Mar 2023 at 07:15, Frédéric Martinsons <frederic.martinsons@...> wrote:

Ah, good idea, I'll try that and provide a V2 of this series if it fits.

On Sat, 25 Mar 2023 at 20:02, Alexander Kanavin <alex.kanavin@...> wrote:

On Sat, 25 Mar 2023 at 10:01, Frederic Martinsons
<frederic.martinsons@...> wrote:
@@ -72,6 +73,8 @@ python () {
url_data.type == 'npmsw' or url_data.type == 'crate' or
'type' in parm and parm['type'] == 'kmeta'):
local_srcuri.append(url)
+ if url in cargo_git_deps:
+ local_srcuri.append(url)
Come to think of it, it's better to use the same type parameter with a
special value as in kernel recipes:

git://.....;type=cargo-dep;...

This would avoid the ugly special variable.

Alex


Frederic Martinsons
 

Ok I'll provide a V3 where the zvariant recipe is removed.


On Sun, 26 Mar 2023 at 20:01, Alexander Kanavin <alex.kanavin@...> wrote:
I'm fine with v2, except for the zvariant recipe, that should probably
go to meta-rust (which itself should go to meta-oe, with cleanups).

Alex

On Sun, 26 Mar 2023 at 16:58, Frédéric Martinsons
<frederic.martinsons@...> wrote:
>
> I submit a V2 series where I made modifications you suggested, I choose to use a more
> generic type (git-dependency instead of cargo-dep) since I foresee that could be used also
> for npm based recipe (see https://bugzilla.yoctoproject.org/show_bug.cgi?id=11015)
>
> For the moment, I let the example recipe in oe-core because I don't know where it should
> go (or even if I should keep it or remove it)
>
> On Sun, 26 Mar 2023 at 07:15, Frédéric Martinsons <frederic.martinsons@...> wrote:
>>
>> Ah, good idea, I'll try that and provide a V2 of this series if it fits.
>>
>> On Sat, 25 Mar 2023 at 20:02, Alexander Kanavin <alex.kanavin@...> wrote:
>>>
>>> On Sat, 25 Mar 2023 at 10:01, Frederic Martinsons
>>> <frederic.martinsons@...> wrote:
>>> > @@ -72,6 +73,8 @@ python () {
>>> >                      url_data.type == 'npmsw' or url_data.type == 'crate' or
>>> >                      'type' in parm and parm['type'] == 'kmeta'):
>>> >                  local_srcuri.append(url)
>>> > +            if url in cargo_git_deps:
>>> > +                local_srcuri.append(url)
>>>
>>> Come to think of it, it's better to use the same type parameter with a
>>> special value as in kernel recipes:
>>>
>>> git://.....;type=cargo-dep;...
>>>
>>> This would avoid the ugly special variable.
>>>
>>> Alex