Skip to content

Adding possibility to 'cast' or copy to xt::xarray etc #267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 10, 2021
Merged

Adding possibility to 'cast' or copy to xt::xarray etc #267

merged 6 commits into from
Jun 10, 2021

Conversation

tdegeus
Copy link
Member

@tdegeus tdegeus commented Jun 3, 2021

No description provided.

@tdegeus tdegeus changed the title [WIP] Trying something quick and dirty Adding possibility to 'cast' or copy to xt::xarray etc Jun 4, 2021
Copy link
Member

@JohanMabille JohanMabille left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure it would make sense to implement the cast operator for strided_views (which is used for creating a view on an existing expression).

It would make sense to implement it for the xarray_adaptor, though. It would be a non-owning adpator on the underlying buffer.

template <class T>
struct xtensor_check_buffer
{
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can omit the definition of the generic case. A simple declaratoin should be enough.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought so too, but my compiler complained:

xtensor_type_caster_base.hpp:50:16: error: no template named 'xtensor_check_buffer'; did you mean 'xtensor_get_buffer'?
        struct xtensor_check_buffer<xt::xarray<T, L>>
               ^~~~~~~~~~~~~~~~~~~~
               xtensor_get_buffer

std::vector<size_t> shape(buf.ndim());
std::copy(buf.shape(), buf.shape() + buf.ndim(), shape.begin());
value = Type(shape);
std::copy(buf.data(), buf.data() + buf.size(), value.begin());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should take into consideration the strides of the numpy array. There is no guarantee that the element of the arrayare contiguous in memory (the array could actually be a view).

The strides number are numbers of bytes instead of numbers of elements, so we have to handle the conversion. Or we can use the pystride adaptor class.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I did not know about the pystride adaptor. So for the moment I use pybind11's way to ensure that data is contiguous, by using https://pybind11.readthedocs.io/en/stable/advanced/pycpp/numpy.html#arrays. Personally I have no big feelings about one or the other, but at the same time I am not entirely sure about the difference in efficiency.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah indeed c_style and f_style involve contiguity, I never paid attention to that, thanks for the pointer!

@tdegeus
Copy link
Member Author

tdegeus commented Jun 5, 2021

Essentially this should be good to go. However, I had to introduce on additional check to check on rank for xtensor. In particular, I want to do this:

static auto get(pybind11::handle src)
{
    auto buf = pybind11::array_t<T, pybind11::array::c_style | pybind11::array::forcecast>::ensure(src);
    if (buf.ndim() != 2) {
        return false;
    }
    return buf;
}

however my compiler gets (rightfully) confused:

error: 'auto' in return type deduced as 'pybind11::array_t<int, 17>' here but deduced as 'bool' in earlier return statement
                return buf;
                ^

So I worked around with the extra check. I would like to get rid of it. But I don't know what I can return instead of false that allows me to later do

if (!buf) { 
    return false;
}

Alternatively on could try to get rid of the check statically. But this I don't know how to do easily.

Copy link
Member

@JohanMabille JohanMabille left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me once the conflicts are solved. I've suggested some renaming that I find more explicit.

{
return array_t<T, array::c_style | array::forcecast>::ensure(src);
}
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpicking: what about the following renaming?
xtensor_get_buffer => pybind_array_getter_impl
get => run

{
return xtensor_get_buffer<T, L>::get(src);
}
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpicking: in line with the previous renaming proposal:
xtensor_chek_buffer => pybind_array_getter
get => run

{
return true;
}
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpicking:
xtensor_verify => pybind_array_dim_checker
get => run

{
using T = typename Type::value_type;

if (!convert && !array_t<T>::check_(src)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpicking: opening curly brace goes on a new line.

@tdegeus
Copy link
Member Author

tdegeus commented Jun 10, 2021

Thanks @JohanMabille ! Good to go.

@JohanMabille JohanMabille merged commit 06d86c9 into xtensor-stack:master Jun 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants