Skip to content

Conversation

joaosaffran
Copy link
Contributor

This patch adds 2 small validation to DirectX backend. First, it checks if registers in descriptor tables are not overflowing, meaning they don't try to bind registers over the maximum allowed value; second, it checks if samplers are being mixed with other resource types.
Closes: #153057, #153058

@llvmbot
Copy link
Member

llvmbot commented Aug 12, 2025

@llvm/pr-subscribers-hlsl

@llvm/pr-subscribers-backend-directx

Author: None (joaosaffran)

Changes

This patch adds 2 small validation to DirectX backend. First, it checks if registers in descriptor tables are not overflowing, meaning they don't try to bind registers over the maximum allowed value; second, it checks if samplers are being mixed with other resource types.
Closes: #153057, #153058


Full diff: https://github.com/llvm/llvm-project/pull/153276.diff

4 Files Affected:

  • (modified) llvm/lib/Target/DirectX/DXILPostOptimizationValidation.cpp (+96-7)
  • (added) llvm/test/CodeGen/DirectX/rootsignature-validation-fail-appending-overflow.ll (+18)
  • (added) llvm/test/CodeGen/DirectX/rootsignature-validation-fail-register-overflow.ll (+17)
  • (added) llvm/test/CodeGen/DirectX/rootsignature-validation-fail-sampler-mix.ll (+18)
diff --git a/llvm/lib/Target/DirectX/DXILPostOptimizationValidation.cpp b/llvm/lib/Target/DirectX/DXILPostOptimizationValidation.cpp
index abcefe287d9bc..7067d6a2960e8 100644
--- a/llvm/lib/Target/DirectX/DXILPostOptimizationValidation.cpp
+++ b/llvm/lib/Target/DirectX/DXILPostOptimizationValidation.cpp
@@ -155,6 +155,31 @@ reportRegNotBound(Module &M, ResourceClass Class,
   M.getContext().diagnose(DiagnosticInfoGeneric(Message));
 }
 
+static void
+reportDescriptorTableMixingTypes(Module &M, uint32_t Location,
+                                 dxbc::DescriptorRangeType RangeType) {
+  SmallString<128> Message;
+  raw_svector_ostream OS(Message);
+  OS << "Samplers cannot be mixed with other "
+     << "resource types in a descriptor table, "
+     << getResourceClassName(toResourceClass(RangeType))
+     << "(location=" << Location << ")";
+
+  M.getContext().diagnose(DiagnosticInfoGeneric(Message));
+}
+
+static void reportOverlowingRange(Module &M, const dxbc::RTS0::v2::DescriptorRange &Range) {
+  SmallString<128> Message;
+  raw_svector_ostream OS(Message);
+  OS << "Cannot append range with implicit lower " 
+      << "bound after an unbounded range "
+      << getResourceClassName(toResourceClass(static_cast<dxbc::DescriptorRangeType>(Range.RangeType)))
+      << "(register=" << Range.BaseShaderRegister << ", space=" << 
+      Range.RegisterSpace
+      << ") exceeds maximum allowed value.";
+  M.getContext().diagnose(DiagnosticInfoGeneric(Message));
+}
+
 static void reportInvalidHandleTy(
     Module &M, const llvm::ArrayRef<dxil::ResourceInfo::ResourceBinding> &RDs,
     const iterator_range<SmallVectorImpl<dxil::ResourceInfo>::iterator>
@@ -236,10 +261,73 @@ getRootDescriptorsBindingInfo(const mcdxbc::RootSignatureDesc &RSD,
   return RDs;
 }
 
-static void validateRootSignature(Module &M,
-                                  const mcdxbc::RootSignatureDesc &RSD,
-                                  dxil::ModuleMetadataInfo &MMI,
-                                  DXILResourceMap &DRM) {
+
+
+static void validateDescriptorTables(Module &M,
+                                     const mcdxbc::RootSignatureDesc &RSD,
+                                     dxil::ModuleMetadataInfo &MMI,
+                                     DXILResourceMap &DRM) {
+  for (const mcdxbc::RootParameterInfo &ParamInfo : RSD.ParametersContainer) {
+    if (static_cast<dxbc::RootParameterType>(ParamInfo.Header.ParameterType) !=
+        dxbc::RootParameterType::DescriptorTable)
+      continue;
+
+    mcdxbc::DescriptorTable Table =
+        RSD.ParametersContainer.getDescriptorTable(ParamInfo.Location);
+
+    bool HasSampler = false;
+    bool HasOtherRangeType = false;
+    dxbc::DescriptorRangeType OtherRangeType;
+    uint32_t OtherRangeTypeLocation = 0;
+
+    uint64_t AppendingOffset = 0;
+
+
+    for (const dxbc::RTS0::v2::DescriptorRange &Range : Table.Ranges) {
+      dxbc::DescriptorRangeType RangeType =
+          static_cast<dxbc::DescriptorRangeType>(Range.RangeType);
+      
+      uint64_t Offset = AppendingOffset;
+      if(Range.OffsetInDescriptorsFromTableStart != ~0U)
+        Offset = Range.OffsetInDescriptorsFromTableStart;
+      
+      if(Offset > ~0U)
+        reportOverlowingRange(M, Range);
+      if(Range.NumDescriptors == ~0U) {
+        AppendingOffset = (uint64_t)~0U + (uint64_t)1ULL;
+      } else { 
+        uint64_t UpperBound = (uint64_t)Range.BaseShaderRegister + (uint64_t)Range.NumDescriptors - (uint64_t)1U;
+        if(UpperBound > ~0U)
+          reportOverlowingRange(M, Range);
+
+        uint64_t AppendingUpperBound = (uint64_t)Offset + (uint64_t)Range.NumDescriptors - (uint64_t)1U;
+        if(AppendingUpperBound > ~0U)
+          reportOverlowingRange(M, Range);
+        AppendingOffset = Offset + Range.NumDescriptors;
+      }
+      
+      if (RangeType == dxbc::DescriptorRangeType::Sampler) {
+        HasSampler = true;
+      } else {
+        HasOtherRangeType = true;
+        OtherRangeType = RangeType;
+        OtherRangeTypeLocation = ParamInfo.Location;
+      }
+    }
+
+    // Samplers cannot be mixed with other resources in a descriptor table.
+    if (HasSampler && HasOtherRangeType) {
+      reportDescriptorTableMixingTypes(M, OtherRangeTypeLocation,
+                                       OtherRangeType);
+      continue;
+    }
+  }
+}
+
+static void validateRootSignatureBindings(Module &M,
+                                          const mcdxbc::RootSignatureDesc &RSD,
+                                          dxil::ModuleMetadataInfo &MMI,
+                                          DXILResourceMap &DRM) {
 
   hlsl::BindingInfoBuilder Builder;
   dxbc::ShaderVisibility Visibility = tripleToVisibility(MMI.ShaderProfile);
@@ -309,7 +397,6 @@ static void validateRootSignature(Module &M,
         reportOverlappingRegisters(M, ReportedBinding, Overlaping);
       });
   // Next checks require that the root signature definition is valid.
-  // Next checks require that the root signature definition is valid.
   if (!HasOverlap) {
     SmallVector<ResourceInfo::ResourceBinding> RDs =
         getRootDescriptorsBindingInfo(RSD, Visibility);
@@ -351,8 +438,10 @@ static void reportErrors(Module &M, DXILResourceMap &DRM,
   assert(!DRBI.hasImplicitBinding() && "implicit bindings should be handled in "
                                        "DXILResourceImplicitBinding pass");
 
-  if (mcdxbc::RootSignatureDesc *RSD = getRootSignature(RSBI, MMI))
-    validateRootSignature(M, *RSD, MMI, DRM);
+  if (mcdxbc::RootSignatureDesc *RSD = getRootSignature(RSBI, MMI)) {
+    validateRootSignatureBindings(M, *RSD, MMI, DRM);
+    validateDescriptorTables(M, *RSD, MMI, DRM);
+  }
 }
 
 PreservedAnalyses
diff --git a/llvm/test/CodeGen/DirectX/rootsignature-validation-fail-appending-overflow.ll b/llvm/test/CodeGen/DirectX/rootsignature-validation-fail-appending-overflow.ll
new file mode 100644
index 0000000000000..865312a43b5a8
--- /dev/null
+++ b/llvm/test/CodeGen/DirectX/rootsignature-validation-fail-appending-overflow.ll
@@ -0,0 +1,18 @@
+; RUN: not opt -S -passes='dxil-post-optimization-validation' -mtriple=dxil-pc-shadermodel6.6-compute %s 2>&1 | FileCheck %s
+; CHECK: error: Cannot append range with implicit lower bound after an unbounded range UAV(register=3, space=4) exceeds maximum allowed value.
+
+@TB.str = private unnamed_addr constant [3 x i8] c"TB\00", align 1
+
+define void @CSMain() "hlsl.shader"="compute" {
+entry:
+  ret void
+}
+
+!dx.rootsignatures = !{!0}
+
+!0 = !{ptr @CSMain, !1, i32 2}
+!1 = !{!3}
+!3 = !{!"DescriptorTable", i32 0, !4, !5}
+!4 = !{!"UAV", i32 -1, i32 1, i32 2, i32 -1, i32 0}
+!5 = !{!"UAV", i32 1, i32 3, i32 4, i32 -1, i32 0}
+
diff --git a/llvm/test/CodeGen/DirectX/rootsignature-validation-fail-register-overflow.ll b/llvm/test/CodeGen/DirectX/rootsignature-validation-fail-register-overflow.ll
new file mode 100644
index 0000000000000..ef887baf80728
--- /dev/null
+++ b/llvm/test/CodeGen/DirectX/rootsignature-validation-fail-register-overflow.ll
@@ -0,0 +1,17 @@
+; RUN: not opt -S -passes='dxil-post-optimization-validation' -mtriple=dxil-pc-shadermodel6.6-compute %s 2>&1 | FileCheck %s
+; CHECK: error: Cannot append range with implicit lower bound after an unbounded range UAV(register=4294967295, space=0) exceeds maximum allowed value. 
+
+@TB.str = private unnamed_addr constant [3 x i8] c"TB\00", align 1
+
+define void @CSMain() "hlsl.shader"="compute" {
+entry:
+  ret void
+}
+
+!dx.rootsignatures = !{!0}
+
+!0 = !{ptr @CSMain, !1, i32 2}
+!1 = !{!3}
+!3 = !{!"DescriptorTable", i32 0, !4}
+!4 = !{!"UAV", i32 100, i32 4294967295, i32 0, i32 -1, i32 0}
+
diff --git a/llvm/test/CodeGen/DirectX/rootsignature-validation-fail-sampler-mix.ll b/llvm/test/CodeGen/DirectX/rootsignature-validation-fail-sampler-mix.ll
new file mode 100644
index 0000000000000..d4caeb2675d82
--- /dev/null
+++ b/llvm/test/CodeGen/DirectX/rootsignature-validation-fail-sampler-mix.ll
@@ -0,0 +1,18 @@
+; RUN: not opt -S -passes='dxil-post-optimization-validation' -mtriple=dxil-pc-shadermodel6.6-compute %s 2>&1 | FileCheck %s
+; CHECK: error: Samplers cannot be mixed with other resource types in a descriptor table, UAV(location=0)
+
+@TB.str = private unnamed_addr constant [3 x i8] c"TB\00", align 1
+
+define void @CSMain() "hlsl.shader"="compute" {
+entry:
+  ret void
+}
+
+!dx.rootsignatures = !{!0}
+
+!0 = !{ptr @CSMain, !1, i32 2}
+!1 = !{!3}
+!3 = !{!"DescriptorTable", i32 0, !4, !5}
+!4 = !{!"UAV", i32 1, i32 0, i32 0, i32 -1, i32 0}
+!5 = !{!"Sampler", i32 2, i32 0, i32 0, i32 -1, i32 0}
+

Copy link

⚠️ We detected that you are using a GitHub private e-mail address to contribute to the repo.
Please turn off Keep my email addresses private setting in your account.
See LLVM Developer Policy and LLVM Discourse for more information.

Copy link

github-actions bot commented Aug 12, 2025

✅ With the latest revision this PR passed the C/C++ code formatter.

@@ -0,0 +1,17 @@
; RUN: not opt -S -passes='dxil-post-optimization-validation' -mtriple=dxil-pc-shadermodel6.6-compute %s 2>&1 | FileCheck %s
; CHECK: error: Cannot append range with implicit lower bound after an unbounded range UAV(register=4294967295, space=0) exceeds maximum allowed value.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error message is incorrect, this case does have an explicit binding

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is the same error message as DXC. It has nothing to do with implicit binding. It means that you cannot bind if the lower bound is, implicit, exceeding the maximum allowed value. This happens when you append a range after an unbounded range.

Thant being said, I change the error message to fix this confusion.

Comment on lines 266 to 267
static void validateDescriptorTables(Module &M,
const mcdxbc::RootSignatureDesc &RSD) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
static void validateDescriptorTables(Module &M,
const mcdxbc::RootSignatureDesc &RSD) {
static void validateDescriptorTable(Module &M,
const mcdxbc::DescriptorTable &Table) {

Does it make more sense to call validateDescriptorTable when we encounter a descriptor table while iterating in validateRootSignature?

Comment on lines 317 to 322
// Samplers cannot be mixed with other resources in a descriptor table.
if (HasSampler && HasOtherRangeType) {
reportDescriptorTableMixingTypes(M, OtherRangeTypeLocation,
OtherRangeType);
continue;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Samplers cannot be mixed with other resources in a descriptor table.
if (HasSampler && HasOtherRangeType) {
reportDescriptorTableMixingTypes(M, OtherRangeTypeLocation,
OtherRangeType);
continue;
}
// Samplers cannot be mixed with other resources in a descriptor table.
if (HasSampler && HasOtherRangeType)
reportDescriptorTableMixingTypes(M, OtherRangeTypeLocation,
OtherRangeType);

dxbc::DescriptorRangeType RangeType =
static_cast<dxbc::DescriptorRangeType>(Range.RangeType);

uint64_t Offset = AppendingOffset;
Copy link
Contributor

@inbelic inbelic Aug 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am having a really hard time trying to figure what it happening here to validate it is correct. Can you add some comments to describe how this checks for an overflow

@llvmbot llvmbot added the HLSL HLSL Language Support label Aug 12, 2025
@inbelic
Copy link
Contributor

inbelic commented Aug 18, 2025

Given we want to use these for the frontend as well. Would it be possible to do these validations with an HLSLBinding instead? Both the front and backend will create the bindings I believe and could be a common interface?

Comment on lines 549 to 554
if (verifyOffsetOverflowing(AppendingRegister,
Range.OffsetInDescriptorsFromTableStart,
Range.BaseShaderRegister, Range.RegisterSpace,
Range.NumDescriptors))
return make_error<TableRegisterOverflowError>(
RangeType, Range.BaseShaderRegister, Range.RegisterSpace);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it makes sense to group all these verifications into the same function and same error message. DXC provided a unique error message for each scenario.

I also think if we split this into 3 validation checks, the functions would be much easier to follow:

  1. if Offset == append && AppendingRegister == unbound -> implict bound error
  2. if NumDescriptors != unbound && BaseRegister + NumDescriptors - 1 >= ~0u -> overflow for shader range error
  3. Let Offset = (Offset == append) ? AppendingRegister : Offset. Then if Offset + NumDescriptors - 1 >= ~0u -> overflow for descriptor range error

I would also personally return a std::optional<uint64_t> rather than updating AppendingRegister as a reference.

@joaosaffran joaosaffran requested a review from inbelic August 25, 2025 21:57
RangeType, Range.BaseShaderRegister, Range.RegisterSpace);

if (verifyRegisterOverflow(AppendingRegister, Range.NumDescriptors))
return make_error<TableRegisterOverflowError>(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These provide the same error message for the different cases. I think we should be at least on par, if not better than DXC here. DXC for reference.

uint32_t NumDescriptors) {
if (NumDescriptors == ~0U)
return (uint64_t)~0U + (uint64_t)1ULL;
return AppendingRegisterRegister + NumDescriptors;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also dependent on the current offset.

Suggested change
return AppendingRegisterRegister + NumDescriptors;
return Offset == append ? AppendingRegisterRegister + NumDescriptors : Offset + NumDescriptors;

@joaosaffran joaosaffran requested a review from inbelic August 26, 2025 02:43
Copy link
Contributor

@inbelic inbelic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM assuming the toResourceClass stuff will be removed when we rebase onto main

Comment on lines 203 to 204
if (NumDescriptors == ~0U)
return (uint64_t)~0U + (uint64_t)1ULL;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I don't think this is required anymore

if (verifyOffsetOverflow(Range.OffsetInDescriptorsFromTableStart,
AppendingRegister))
return make_error<TableRegisterOverflowError>(
uint64_t StartSlot = AppendingRegister;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: StartSlot can just be replaced in all uses by AppendingRegister. Or if you think StartSlot is a better name then we can just use that.

@@ -40,6 +41,12 @@ LLVM_ABI bool verifyMaxAnisotropy(uint32_t MaxAnisotropy);
LLVM_ABI bool verifyComparisonFunc(uint32_t ComparisonFunc);
LLVM_ABI bool verifyBorderColor(uint32_t BorderColor);
LLVM_ABI bool verifyLOD(float LOD);
LLVM_ABI bool verifyRegisterOverflow(uint64_t Register,
uint32_t NumDescriptors);
LLVM_ABI uint64_t updateAppendingRegister(uint64_t AppendingRegisterRegister,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function contain important logic to make sure the previous 2 validations work properly, so I added it in here, so it can be reused in the frontend.

Comment on lines +217 to +248
inline dxil::ResourceClass
toResourceClass(dxbc::DescriptorRangeType RangeType) {
using namespace dxbc;
switch (RangeType) {
case DescriptorRangeType::SRV:
return dxil::ResourceClass::SRV;
case DescriptorRangeType::UAV:
return dxil::ResourceClass::UAV;
case DescriptorRangeType::CBV:
return dxil::ResourceClass::CBuffer;
case DescriptorRangeType::Sampler:
return dxil::ResourceClass::Sampler;
}
llvm_unreachable("Unknown DescriptorRangeType");
}

inline dxil::ResourceClass toResourceClass(dxbc::RootParameterType Type) {
using namespace dxbc;
switch (Type) {
case RootParameterType::Constants32Bit:
return dxil::ResourceClass::CBuffer;
case RootParameterType::SRV:
return dxil::ResourceClass::SRV;
case RootParameterType::UAV:
return dxil::ResourceClass::UAV;
case RootParameterType::CBV:
return dxil::ResourceClass::CBuffer;
case dxbc::RootParameterType::DescriptorTable:
llvm_unreachable("DescriptorTable is not convertible to ResourceClass");
}
llvm_unreachable("Unknown RootParameterType");
}
Copy link
Contributor Author

@joaosaffran joaosaffran Aug 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those are being moved to here, to be used in Frontend/HLSL/RootSignatureMetadata.h and DirectX/DxilPostOptimizationValidation.cpp

Comment on lines +533 to +538
if (RangeType == dxbc::DescriptorRangeType::Sampler) {
HasSampler = true;
} else {
HasOtherRangeType = true;
OtherRangeType = RangeType;
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using curly brackets so it follows llvm coding style recommendations.

Comment on lines -15 to +16
!5 = !{ !"DescriptorTable", i32 0, !6, !8, !9, !10, !11, !12, !13, !14, !15, !16, !17, !18, !19, !20 }
!3 = !{ !5, !21 } ; list of root signature elements
!5 = !{ !"DescriptorTable", i32 0, !10, !11, !12, !13, !14, !15, !16, !17, !18, !19, !20 }
!21 = !{ !"DescriptorTable", i32 0,!6, !8, !9 }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This refactor is necessary, because this test was mixing samplers and non samplers in a single descriptor table, which is not allowed

Comment on lines -70 to -86
;DXC-NEXT: - RangeType: 3
;DXC-NEXT: NumDescriptors: 1
;DXC-NEXT: BaseShaderRegister: 0
;DXC-NEXT: RegisterSpace: 1
;DXC-NEXT: OffsetInDescriptorsFromTableStart: 4294967295
;DXC-NEXT: - RangeType: 3
;DXC-NEXT: NumDescriptors: 1
;DXC-NEXT: BaseShaderRegister: 0
;DXC-NEXT: RegisterSpace: 3
;DXC-NEXT: OffsetInDescriptorsFromTableStart: 4294967295
;DXC-NEXT: DESCRIPTORS_VOLATILE: true
;DXC-NEXT: - RangeType: 3
;DXC-NEXT: NumDescriptors: 1
;DXC-NEXT: BaseShaderRegister: 0
;DXC-NEXT: RegisterSpace: 4
;DXC-NEXT: OffsetInDescriptorsFromTableStart: 4294967295
;DXC-NEXT: DESCRIPTORS_STATIC_KEEPING_BUFFER_BOUNDS_CHECKS: true
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those are all samplers, they are being moved to bellow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend:DirectX HLSL HLSL Language Support
Projects
None yet
3 participants