Skip to content

Commit 3b0450a

Browse files
aeubanksaeubanks
committed
Add IR constructs for preallocated (inalloca replacement)
Add llvm.call.preallocated.{setup,arg} instrinsics. Add "preallocated" operand bundle which takes a token produced by llvm.call.preallocated.setup. Add "preallocated" parameter attribute, which is like byval but without the copy. Verifier changes for these IR constructs. See https://github.com/rnk/llvm-project/blob/call-setup-docs/llvm/docs/CallSetup.md Subscribers: hiraditya, jdoerfert, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D74651
1 parent 9ea5cc8 commit 3b0450a

23 files changed

+564
-37
lines changed

‎llvm/docs/LangRef.rst

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1056,6 +1056,26 @@ Currently, only the following parameter attributes are defined:
10561056
form and the known alignment of the pointer specified to the call
10571057
site. If the alignment is not specified, then the code generator
10581058
makes a target-specific assumption.
1059+
``preallocated(<ty>)``
1060+
This indicates that the pointer parameter should really be passed by
1061+
value to the function, and that the pointer parameter's pointee has
1062+
already been initialized before the call instruction. This attribute
1063+
is only valid on LLVM pointer arguments. The argument must be the value
1064+
returned by the appropriate
1065+
:ref:`llvm.call.preallocated.arg<int_call_preallocated_arg>`, although is
1066+
ignored during codegen.
1067+
1068+
Any function call with a ``preallocated`` attribute in any parameter
1069+
must have a ``"preallocated"`` operand bundle.
1070+
1071+
The preallocated attribute requires a type argument, which must be
1072+
the same as the pointee type of the argument.
1073+
1074+
The preallocated attribute also supports specifying an alignment with the
1075+
align attribute. It indicates the alignment of the stack slot to
1076+
form and the known alignment of the pointer specified to the call
1077+
site. If the alignment is not specified, then the code generator
1078+
makes a target-specific assumption.
10591079

10601080
.. _attr_inalloca:
10611081

@@ -1953,6 +1973,12 @@ attributes are supported:
19531973

19541974
<vector_redirection>:= optional, custom name of the vector function
19551975

1976+
``preallocated(<ty>)``
1977+
This attribute is required on calls to ``llvm.call.preallocated.arg``
1978+
and cannot be used on any other call. See
1979+
:ref:`llvm.call.preallocated.arg<int_call_preallocated_arg>` for more
1980+
details.
1981+
19561982
.. _glattrs:
19571983

19581984
Global Attributes
@@ -2165,6 +2191,33 @@ benefits:
21652191
simplifies and improves heuristics, e.g., for use "use-sensitive"
21662192
optimizations.
21672193

2194+
.. _ob_preallocated:
2195+
2196+
Preallocated Operand Bundles
2197+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2198+
2199+
Preallocated operand bundles are characterized by the ``"preallocated"``
2200+
operand bundle tag. These operand bundles allow separation of the allocation
2201+
of the call argument memory from the call site. This is necessary to pass
2202+
non-trivially copyable objects by value in a way that is compatible with MSVC
2203+
on some targets. There can be at most one ``"preallocated"`` operand bundle
2204+
attached to a call site and it must have exactly one bundle operand, which is
2205+
a token generated by ``@llvm.call.preallocated.setup``. A call with this
2206+
operand bundle should not adjust the stack before entering the function, as
2207+
that will have been done by one of the ``@llvm.call.preallocated.*`` intrinsics.
2208+
2209+
.. code-block:: llvm
2210+
2211+
%foo = type { i64, i32 }
2212+
2213+
...
2214+
2215+
%t = call token @llvm.call.preallocated.setup(i32 1)
2216+
%a = call i8* @llvm.call.preallocated.arg(token %t, i32 0) preallocated(%foo)
2217+
%b = bitcast i8* %a to %foo*
2218+
; initialize %b
2219+
call void @bar(i32 42, %foo* preallocated(%foo) %b) ["preallocated"(token %t)]
2220+
21682221
.. _moduleasm:
21692222

21702223
Module-Level Inline Assembly
@@ -11874,6 +11927,90 @@ call a helper function, read from an alternate memory space, or perform
1187411927
other operations necessary to locate the TLS area. Not all targets support
1187511928
this intrinsic.
1187611929

11930+
'``llvm.call.preallocated.setup``' Intrinsic
11931+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
11932+
11933+
Syntax:
11934+
"""""""
11935+
11936+
::
11937+
11938+
declare token @llvm.call.preallocated.setup(i32 %num_args)
11939+
11940+
Overview:
11941+
"""""""""
11942+
11943+
The '``llvm.call.preallocated.setup``' intrinsic returns a token which can
11944+
be used with a call's ``"preallocated"`` operand bundle to indicate that
11945+
certain arguments are allocated and initialized before the call.
11946+
11947+
Semantics:
11948+
""""""""""
11949+
11950+
The '``llvm.call.preallocated.setup``' intrinsic returns a token which is
11951+
associated with at most one call. The token can be passed to
11952+
'``@llvm.call.preallocated.arg``' to get a pointer to get that
11953+
corresponding argument. The token must be the parameter to a
11954+
``"preallocated"`` operand bundle for the corresponding call.
11955+
11956+
Nested calls to '``llvm.call.preallocated.setup``' are allowed, but must
11957+
be properly nested. e.g.
11958+
11959+
:: code-block:: llvm
11960+
11961+
%t1 = call token @llvm.call.preallocated.setup(i32 0)
11962+
%t2 = call token @llvm.call.preallocated.setup(i32 0)
11963+
call void foo() ["preallocated"(token %t2)]
11964+
call void foo() ["preallocated"(token %t1)]
11965+
11966+
is allowed, but not
11967+
11968+
:: code-block:: llvm
11969+
11970+
%t1 = call token @llvm.call.preallocated.setup(i32 0)
11971+
%t2 = call token @llvm.call.preallocated.setup(i32 0)
11972+
call void foo() ["preallocated"(token %t1)]
11973+
call void foo() ["preallocated"(token %t2)]
11974+
11975+
.. _int_call_preallocated_arg:
11976+
11977+
'``llvm.call.preallocated.arg``' Intrinsic
11978+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
11979+
11980+
Syntax:
11981+
"""""""
11982+
11983+
::
11984+
11985+
declare i8* @llvm.call.preallocated.arg(token %setup_token, i32 %arg_index)
11986+
11987+
Overview:
11988+
"""""""""
11989+
11990+
The '``llvm.call.preallocated.arg``' intrinsic returns a pointer to the
11991+
corresponding preallocated argument for the preallocated call.
11992+
11993+
Semantics:
11994+
""""""""""
11995+
11996+
The '``llvm.call.preallocated.arg``' intrinsic returns a pointer to the
11997+
``%arg_index``th argument with the ``preallocated`` attribute for
11998+
the call associated with the ``%setup_token``, which must be from
11999+
'``llvm.call.preallocated.setup``'.
12000+
12001+
A call to '``llvm.call.preallocated.arg``' must have a call site
12002+
``preallocated`` attribute. The type of the ``preallocated`` attribute must
12003+
match the type used by the ``preallocated`` attribute of the corresponding
12004+
argument at the preallocated call. The type is used in the case that an
12005+
``llvm.call.preallocated.setup`` does not have a corresponding call (e.g. due
12006+
to DCE), where otherwise we cannot know how large the arguments are.
12007+
12008+
It is undefined behavior if this is called with a token from an
12009+
'``llvm.call.preallocated.setup``' if another
12010+
'``llvm.call.preallocated.setup``' has already been called or if the
12011+
preallocated call corresponding to the '``llvm.call.preallocated.setup``'
12012+
has already been called.
12013+
1187712014
Standard C Library Intrinsics
1187812015
-----------------------------
1187912016

‎llvm/include/llvm/Bitcode/LLVMBitCodes.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -633,6 +633,7 @@ enum AttributeKindCodes {
633633
ATTR_KIND_NOFREE = 62,
634634
ATTR_KIND_NOSYNC = 63,
635635
ATTR_KIND_SANITIZE_MEMTAG = 64,
636+
ATTR_KIND_PREALLOCATED = 65,
636637
};
637638

638639
enum ComdatSelectionKindCodes {

‎llvm/include/llvm/IR/Attributes.h

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,7 @@ class Attribute {
108108
unsigned ElemSizeArg,
109109
const Optional<unsigned> &NumElemsArg);
110110
static Attribute getWithByValType(LLVMContext &Context, Type *Ty);
111+
static Attribute getWithPreallocatedType(LLVMContext &Context, Type *Ty);
111112

112113
static Attribute::AttrKind getAttrKindFromName(StringRef AttrName);
113114

@@ -302,6 +303,7 @@ class AttributeSet {
302303
uint64_t getDereferenceableBytes() const;
303304
uint64_t getDereferenceableOrNullBytes() const;
304305
Type *getByValType() const;
306+
Type *getPreallocatedType() const;
305307
std::pair<unsigned, Optional<unsigned>> getAllocSizeArgs() const;
306308
std::string getAsString(bool InAttrGrp = false) const;
307309

@@ -724,6 +726,7 @@ class AttrBuilder {
724726
uint64_t DerefOrNullBytes = 0;
725727
uint64_t AllocSizeArgs = 0;
726728
Type *ByValType = nullptr;
729+
Type *PreallocatedType = nullptr;
727730

728731
public:
729732
AttrBuilder() = default;
@@ -802,6 +805,9 @@ class AttrBuilder {
802805
/// Retrieve the byval type.
803806
Type *getByValType() const { return ByValType; }
804807

808+
/// Retrieve the preallocated type.
809+
Type *getPreallocatedType() const { return PreallocatedType; }
810+
805811
/// Retrieve the allocsize args, if the allocsize attribute exists. If it
806812
/// doesn't exist, pair(0, 0) is returned.
807813
std::pair<unsigned, Optional<unsigned>> getAllocSizeArgs() const;
@@ -845,6 +851,9 @@ class AttrBuilder {
845851
/// This turns a byval type into the form used internally in Attribute.
846852
AttrBuilder &addByValAttr(Type *Ty);
847853

854+
/// This turns a preallocated type into the form used internally in Attribute.
855+
AttrBuilder &addPreallocatedAttr(Type *Ty);
856+
848857
/// Add an allocsize attribute, using the representation returned by
849858
/// Attribute.getIntValue().
850859
AttrBuilder &addAllocSizeAttrFromRawRepr(uint64_t RawAllocSizeRepr);

‎llvm/include/llvm/IR/Attributes.td

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,9 @@ def OptimizeForSize : EnumAttr<"optsize">;
133133
/// Function must not be optimized.
134134
def OptimizeNone : EnumAttr<"optnone">;
135135

136+
/// Similar to byval but without a copy.
137+
def Preallocated : TypeAttr<"preallocated">;
138+
136139
/// Function does not access memory.
137140
def ReadNone : EnumAttr<"readnone">;
138141

‎llvm/include/llvm/IR/Intrinsics.td

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -508,6 +508,9 @@ def int_instrprof_value_profile : Intrinsic<[],
508508
llvm_i32_ty],
509509
[]>;
510510

511+
def int_call_preallocated_setup : Intrinsic<[llvm_token_ty], [llvm_i32_ty]>;
512+
def int_call_preallocated_arg : Intrinsic<[llvm_ptr_ty], [llvm_token_ty, llvm_i32_ty]>;
513+
511514
//===------------------- Standard C Library Intrinsics --------------------===//
512515
//
513516

‎llvm/include/llvm/IR/LLVMContext.h

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -83,12 +83,14 @@ class LLVMContext {
8383
/// Known operand bundle tag IDs, which always have the same value. All
8484
/// operand bundle tags that LLVM has special knowledge of are listed here.
8585
/// Additionally, this scheme allows LLVM to efficiently check for specific
86-
/// operand bundle tags without comparing strings.
86+
/// operand bundle tags without comparing strings. Keep this in sync with
87+
/// LLVMContext::LLVMContext().
8788
enum : unsigned {
8889
OB_deopt = 0, // "deopt"
8990
OB_funclet = 1, // "funclet"
9091
OB_gc_transition = 2, // "gc-transition"
9192
OB_cfguardtarget = 3, // "cfguardtarget"
93+
OB_preallocated = 4, // "preallocated"
9294
};
9395

9496
/// getMDKindID - Return a unique non-zero ID for the specified metadata kind.

‎llvm/lib/AsmParser/LLLexer.cpp

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -667,6 +667,7 @@ lltok::Kind LLLexer::LexIdentifier() {
667667
KEYWORD(optforfuzzing);
668668
KEYWORD(optnone);
669669
KEYWORD(optsize);
670+
KEYWORD(preallocated);
670671
KEYWORD(readnone);
671672
KEYWORD(readonly);
672673
KEYWORD(returned);

‎llvm/lib/AsmParser/LLParser.cpp

Lines changed: 38 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1345,6 +1345,13 @@ bool LLParser::ParseFnAttributeValuePairs(AttrBuilder &B,
13451345
case lltok::kw_uwtable: B.addAttribute(Attribute::UWTable); break;
13461346
case lltok::kw_willreturn: B.addAttribute(Attribute::WillReturn); break;
13471347
case lltok::kw_writeonly: B.addAttribute(Attribute::WriteOnly); break;
1348+
case lltok::kw_preallocated: {
1349+
Type *Ty;
1350+
if (ParsePreallocated(Ty))
1351+
return true;
1352+
B.addPreallocatedAttr(Ty);
1353+
break;
1354+
}
13481355

13491356
// Error handling.
13501357
case lltok::kw_inreg:
@@ -1373,7 +1380,9 @@ bool LLParser::ParseFnAttributeValuePairs(AttrBuilder &B,
13731380
break;
13741381
}
13751382

1376-
Lex.Lex();
1383+
// ParsePreallocated() consumes token
1384+
if (Token != lltok::kw_preallocated)
1385+
Lex.Lex();
13771386
}
13781387
}
13791388

@@ -1637,6 +1646,13 @@ bool LLParser::ParseOptionalParamAttrs(AttrBuilder &B) {
16371646
B.addByValAttr(Ty);
16381647
continue;
16391648
}
1649+
case lltok::kw_preallocated: {
1650+
Type *Ty;
1651+
if (ParsePreallocated(Ty))
1652+
return true;
1653+
B.addPreallocatedAttr(Ty);
1654+
continue;
1655+
}
16401656
case lltok::kw_dereferenceable: {
16411657
uint64_t Bytes;
16421658
if (ParseOptionalDerefAttrBytes(lltok::kw_dereferenceable, Bytes))
@@ -1804,10 +1820,15 @@ bool LLParser::ParseOptionalReturnAttrs(AttrBuilder &B) {
18041820
case lltok::kw_uwtable:
18051821
HaveError |= Error(Lex.getLoc(), "invalid use of function-only attribute");
18061822
break;
1807-
18081823
case lltok::kw_readnone:
18091824
case lltok::kw_readonly:
18101825
HaveError |= Error(Lex.getLoc(), "invalid use of attribute on return type");
1826+
break;
1827+
case lltok::kw_preallocated:
1828+
HaveError |=
1829+
Error(Lex.getLoc(),
1830+
"invalid use of parameter-only/call site-only attribute");
1831+
break;
18111832
}
18121833

18131834
Lex.Lex();
@@ -2519,6 +2540,21 @@ bool LLParser::ParseByValWithOptionalType(Type *&Result) {
25192540
return false;
25202541
}
25212542

2543+
/// ParsePreallocated
2544+
/// ::= preallocated(<ty>)
2545+
bool LLParser::ParsePreallocated(Type *&Result) {
2546+
Result = nullptr;
2547+
if (!EatIfPresent(lltok::kw_preallocated))
2548+
return true;
2549+
if (!EatIfPresent(lltok::lparen))
2550+
return Error(Lex.getLoc(), "expected '('");
2551+
if (ParseType(Result))
2552+
return true;
2553+
if (!EatIfPresent(lltok::rparen))
2554+
return Error(Lex.getLoc(), "expected ')'");
2555+
return false;
2556+
}
2557+
25222558
/// ParseOptionalOperandBundles
25232559
/// ::= /*empty*/
25242560
/// ::= '[' OperandBundle [, OperandBundle ]* ']'

‎llvm/lib/AsmParser/LLParser.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -338,6 +338,7 @@ namespace llvm {
338338
std::vector<unsigned> &FwdRefAttrGrps,
339339
bool inAttrGrp, LocTy &BuiltinLoc);
340340
bool ParseByValWithOptionalType(Type *&Result);
341+
bool ParsePreallocated(Type *&Result);
341342

342343
// Module Summary Index Parsing.
343344
bool SkipModuleSummaryEntry();

‎llvm/lib/AsmParser/LLToken.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -213,6 +213,7 @@ enum Kind {
213213
kw_optforfuzzing,
214214
kw_optnone,
215215
kw_optsize,
216+
kw_preallocated,
216217
kw_readnone,
217218
kw_readonly,
218219
kw_returned,

0 commit comments

Comments
 (0)