Skip to content

Bogus C-cast and illegal assumptions of Object-layout causes segfault in convhull.cpp #26952

@kallaballa

Description

@kallaballa

System Information

OpenCV version: 7a2b048 (Current HEAD)
Platform: OpenSUSE Tumbleweed (20250216) linux-6.13.2-1-default x86_64
GCC: gcc (SUSE Linux) 14.2.1 20241007 [revision 4af44f2cf7d281f3e4f3957efce10e8b2ccb2ad3]

Detailed description

At

Point2f** pointerf = (Point2f**)pointer;

The code assumes that a C-Cast will:

  1. Correctly convert between int and float
  2. The size of int and float

And additionally it makes illegal assumption of the memory layout of objects:

Point* data0 = points.ptr<Point>();

Which spuriously leads to a segfault at

std::sort(pointerf, pointerf + total, CHullCmpPoints<float>());

The pointer handling is none-optimal in other respects aswell. e.g. not using memcpy when possible and the general use of a pointer-of-pointer indirection while not necessary.

Steps to reproduce

https://docs.opencv.org/3.4/d7/d1d/tutorial_hull.html

Issue submission checklist

  • I report the issue, it's not a question
  • I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
  • I updated to the latest OpenCV version and the issue is still there
  • There is reproducer code and related data files (videos, images, onnx, etc)

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions