Prevent out-of-bounds read when sorting segment sides
GCC 6 `std::sort` sometimes compares an element to itself. For a normal implementation of comparison, this is useless, but not harmful. The render comparison predicate relies on accessing A[B[a][b]] when comparing `a` and `b`. Array `B` has `-1` in positions where `a == b`, which causes an access to `A[-1]`, which is undefined behavior. This crashes when using _GLIBCXX_DEBUG: Error: attempt to subscript container with out-of-bounds index -1, but container only holds 8 elements. Objects involved in the operation: sequence "this" @ 0x0x335adf0 { type = std::__debug::array<int, 8ul>::_Array_check_subscript<8ul>; } Since this is undefined behavior, non-debug builds might also misbehave. Current data layouts make it likely that the failure would not have externally observable consequences. Prevent the invalid access by short-circuiting the result if `a == b`.
This commit is contained in:
parent
dfe827e793
commit
0fd80b12df
|
@ -958,6 +958,8 @@ static bool compare_children(const vcsegptridx_t seg, sidenum_fast_t s0, sidenum
|
|||
{
|
||||
Assert(s0 != side_none && s1 != side_none);
|
||||
|
||||
if (s0 == s1)
|
||||
return false;
|
||||
if (Side_opposite[s0] == s1)
|
||||
return false;
|
||||
//find normals of adjoining sides
|
||||
|
|
Loading…
Reference in a new issue