-
Notifications
You must be signed in to change notification settings - Fork 14.9k
[MLIR][WASM] Introduce an importer for Wasm binaries #152131
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[MLIR][WASM] Introduce an importer for Wasm binaries #152131
Conversation
This commit contains basic parsing infrastructure + base code to parse wasm binary file type section. --------- Co-authored-by: Ferdinand Lemaire <ferdinand.lemaire@woven-planet.global> Co-authored-by: Jessica Paquette <jessica.paquette@woven-planet.global>
--------- Co-authored-by: Ferdinand Lemaire <ferdinand.lemaire@woven-planet.global> Co-authored-by: Jessica Paquette <jessica.paquette@woven-planet.global>
--------- Co-authored-by: Ferdinand Lemaire <ferdinand.lemaire@woven-planet.global> Co-authored-by: Jessica Paquette <jessica.paquette@woven-planet.global>
-- Co-authored-by: Luc Forget <luc.forget@woven-planet.global> Co-authored-by: Jessica Paquette <jessica.paquette@woven-planet.global>
--------- Co-authored-by: Ferdinand Lemaire <ferdinand.lemaire@woven-planet.global> Co-authored-by: Jessica Paquette <jessica.paquette@woven-planet.global>
@llvm/pr-subscribers-mlir Author: Ferdinand Lemaire (flemairen6) ChangesThis is a follow-up to #149233 First step in introducing the wasm-import target to mlir-translate. Patch is 60.56 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/152131.diff 22 Files Affected:
diff --git a/mlir/include/mlir/InitAllTranslations.h b/mlir/include/mlir/InitAllTranslations.h
index 1ab80fb27fa9a..622024db5a8a2 100644
--- a/mlir/include/mlir/InitAllTranslations.h
+++ b/mlir/include/mlir/InitAllTranslations.h
@@ -20,6 +20,7 @@ namespace mlir {
void registerFromLLVMIRTranslation();
void registerFromSPIRVTranslation();
+void registerFromWasmTranslation();
void registerToCppTranslation();
void registerToLLVMIRTranslation();
void registerToSPIRVTranslation();
@@ -36,6 +37,7 @@ inline void registerAllTranslations() {
registerFromLLVMIRTranslation();
registerFromSPIRVTranslation();
registerIRDLToCppTranslation();
+ registerFromWasmTranslation();
registerToCppTranslation();
registerToLLVMIRTranslation();
registerToSPIRVTranslation();
diff --git a/mlir/include/mlir/Target/Wasm/WasmBinaryEncoding.h b/mlir/include/mlir/Target/Wasm/WasmBinaryEncoding.h
new file mode 100644
index 0000000000000..a5b124eecbe67
--- /dev/null
+++ b/mlir/include/mlir/Target/Wasm/WasmBinaryEncoding.h
@@ -0,0 +1,73 @@
+//===- WasmBinaryEncoding.h - Byte encodings for Wasm binary format ===----===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+// Define encodings for WebAssembly instructions, types, etc from the
+// WebAssembly binary format.
+//
+// Each encoding is defined in the WebAssembly binary specification.
+//
+//===----------------------------------------------------------------------===//
+#ifndef MLIR_TARGET_WASMBINARYENCODING
+#define MLIR_TARGET_WASMBINARYENCODING
+
+#include <cstddef>
+namespace mlir {
+struct WasmBinaryEncoding {
+ /// Byte encodings for WASM instructions.
+ struct OpCode {
+ // Locals, globals, constants.
+ static constexpr std::byte constI32{0x41};
+ static constexpr std::byte constI64{0x42};
+ static constexpr std::byte constFP32{0x43};
+ static constexpr std::byte constFP64{0x44};
+ };
+
+ /// Byte encodings of types in WASM binaries
+ struct Type {
+ static constexpr std::byte emptyBlockType{0x40};
+ static constexpr std::byte funcType{0x60};
+ static constexpr std::byte externRef{0x6F};
+ static constexpr std::byte funcRef{0x70};
+ static constexpr std::byte v128{0x7B};
+ static constexpr std::byte f64{0x7C};
+ static constexpr std::byte f32{0x7D};
+ static constexpr std::byte i64{0x7E};
+ static constexpr std::byte i32{0x7F};
+ };
+
+ /// Byte encodings of WASM imports.
+ struct Import {
+ static constexpr std::byte typeID{0x00};
+ static constexpr std::byte tableType{0x01};
+ static constexpr std::byte memType{0x02};
+ static constexpr std::byte globalType{0x03};
+ };
+
+ /// Byte encodings for WASM limits.
+ struct LimitHeader {
+ static constexpr std::byte lowLimitOnly{0x00};
+ static constexpr std::byte bothLimits{0x01};
+ };
+
+ /// Byte encodings describing the mutability of globals.
+ struct GlobalMutability {
+ static constexpr std::byte isConst{0x00};
+ static constexpr std::byte isMutable{0x01};
+ };
+
+ /// Byte encodings describing WASM exports.
+ struct Export {
+ static constexpr std::byte function{0x00};
+ static constexpr std::byte table{0x01};
+ static constexpr std::byte memory{0x02};
+ static constexpr std::byte global{0x03};
+ };
+
+ static constexpr std::byte endByte{0x0B};
+};
+} // namespace mlir
+
+#endif
diff --git a/mlir/include/mlir/Target/Wasm/WasmImporter.h b/mlir/include/mlir/Target/Wasm/WasmImporter.h
new file mode 100644
index 0000000000000..188c962dbb6da
--- /dev/null
+++ b/mlir/include/mlir/Target/Wasm/WasmImporter.h
@@ -0,0 +1,33 @@
+//===- WasmImporter.h - Helpers to create WebAssembly emitter ---*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines helpers to import WebAssembly code using the WebAssembly
+// dialect.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef MLIR_TARGET_WASM_WASMIMPORTER_H
+#define MLIR_TARGET_WASM_WASMIMPORTER_H
+
+#include "mlir/IR/BuiltinOps.h"
+#include "mlir/IR/MLIRContext.h"
+#include "mlir/IR/OwningOpRef.h"
+#include "llvm/Support/SourceMgr.h"
+
+namespace mlir::wasm {
+
+/// Translates the given operation to C++ code. The operation or operations in
+/// the region of 'op' need almost all be in EmitC dialect. The parameter
+/// 'declareVariablesAtTop' enforces that all variables for op results and block
+/// arguments are declared at the beginning of the function.
+/// If parameter 'fileId' is non-empty, then body of `emitc.file` ops
+/// with matching id are emitted.
+OwningOpRef<ModuleOp> importWebAssemblyToModule(llvm::SourceMgr &source, MLIRContext* context);
+} // namespace mlir::wasm
+
+#endif // MLIR_TARGET_WASM_WASMIMPORTER_H
diff --git a/mlir/lib/Target/CMakeLists.txt b/mlir/lib/Target/CMakeLists.txt
index 6eb0abc214d38..f0c3ac4d511c1 100644
--- a/mlir/lib/Target/CMakeLists.txt
+++ b/mlir/lib/Target/CMakeLists.txt
@@ -4,3 +4,4 @@ add_subdirectory(SPIRV)
add_subdirectory(LLVMIR)
add_subdirectory(LLVM)
add_subdirectory(SMTLIB)
+add_subdirectory(Wasm)
diff --git a/mlir/lib/Target/Wasm/CMakeLists.txt b/mlir/lib/Target/Wasm/CMakeLists.txt
new file mode 100644
index 0000000000000..890fc0ecfbeb6
--- /dev/null
+++ b/mlir/lib/Target/Wasm/CMakeLists.txt
@@ -0,0 +1,13 @@
+add_mlir_translation_library(MLIRTargetWasmImport
+ TranslateRegistration.cpp
+ TranslateFromWasm.cpp
+
+ ADDITIONAL_HEADER_DIRS
+ ${MLIR_MAIN_INCLUDE_DIR}/Target/Wasm
+
+ LINK_LIBS PUBLIC
+ MLIRWasmSSADialect
+ MLIRIR
+ MLIRSupport
+ MLIRTranslateLib
+)
diff --git a/mlir/lib/Target/Wasm/TranslateFromWasm.cpp b/mlir/lib/Target/Wasm/TranslateFromWasm.cpp
new file mode 100644
index 0000000000000..7fe59a49ba21f
--- /dev/null
+++ b/mlir/lib/Target/Wasm/TranslateFromWasm.cpp
@@ -0,0 +1,1243 @@
+//===- TranslateFromWasm.cpp - Translating to C++ calls -------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+#include "mlir/Dialect/WasmSSA/IR/WasmSSA.h"
+#include "mlir/IR/Attributes.h"
+#include "mlir/IR/Builders.h"
+#include "mlir/IR/BuiltinAttributeInterfaces.h"
+#include "mlir/IR/BuiltinTypes.h"
+#include "mlir/IR/Location.h"
+#include "mlir/Target/Wasm/WasmBinaryEncoding.h"
+#include "mlir/Target/Wasm/WasmImporter.h"
+#include "llvm/ADT/Statistic.h"
+#include "llvm/Support/Debug.h"
+#include "llvm/Support/FormatVariadic.h"
+#include "llvm/Support/LEB128.h"
+
+#include <cstdint>
+#include <variant>
+
+#define DEBUG_TYPE "wasm-translate"
+
+// Statistics.
+STATISTIC(numFunctionSectionItems, "Parsed functions");
+STATISTIC(numGlobalSectionItems, "Parsed globals");
+STATISTIC(numMemorySectionItems, "Parsed memories");
+STATISTIC(numTableSectionItems, "Parsed tables");
+
+static_assert(CHAR_BIT == 8, "This code expects std::byte to be exactly 8 bits");
+
+using namespace mlir;
+using namespace mlir::wasm;
+using namespace mlir::wasmssa;
+
+namespace {
+using section_id_t = uint8_t;
+enum struct WasmSectionType : section_id_t {
+ CUSTOM = 0,
+ TYPE = 1,
+ IMPORT = 2,
+ FUNCTION = 3,
+ TABLE = 4,
+ MEMORY = 5,
+ GLOBAL = 6,
+ EXPORT = 7,
+ START = 8,
+ ELEMENT = 9,
+ CODE = 10,
+ DATA = 11,
+ DATACOUNT = 12
+};
+
+constexpr section_id_t highestWasmSectionID{
+ static_cast<section_id_t>(WasmSectionType::DATACOUNT)};
+
+#define APPLY_WASM_SEC_TRANSFORM \
+ WASM_SEC_TRANSFORM(CUSTOM) \
+ WASM_SEC_TRANSFORM(TYPE) \
+ WASM_SEC_TRANSFORM(IMPORT) \
+ WASM_SEC_TRANSFORM(FUNCTION) \
+ WASM_SEC_TRANSFORM(TABLE) \
+ WASM_SEC_TRANSFORM(MEMORY) \
+ WASM_SEC_TRANSFORM(GLOBAL) \
+ WASM_SEC_TRANSFORM(EXPORT) \
+ WASM_SEC_TRANSFORM(START) \
+ WASM_SEC_TRANSFORM(ELEMENT) \
+ WASM_SEC_TRANSFORM(CODE) \
+ WASM_SEC_TRANSFORM(DATA) \
+ WASM_SEC_TRANSFORM(DATACOUNT)
+
+template <WasmSectionType>
+constexpr const char *wasmSectionName = "";
+
+#define WASM_SEC_TRANSFORM(section) \
+ template <> \
+ constexpr const char *wasmSectionName<WasmSectionType::section> = #section;
+APPLY_WASM_SEC_TRANSFORM
+#undef WASM_SEC_TRANSFORM
+
+constexpr bool sectionShouldBeUnique(WasmSectionType secType) {
+ return secType != WasmSectionType::CUSTOM;
+}
+
+template <std::byte... Bytes>
+struct ByteSequence{};
+
+template <std::byte... Bytes1, std::byte... Bytes2>
+constexpr ByteSequence<Bytes1..., Bytes2...>
+operator+(ByteSequence<Bytes1...>, ByteSequence<Bytes2...>) {
+ return {};
+}
+
+/// Template class for representing a byte sequence of only one byte
+template<std::byte Byte>
+struct UniqueByte : ByteSequence<Byte> {};
+
+template <typename T, T... Values>
+constexpr ByteSequence<std::byte{Values}...>
+byteSeqFromIntSeq(std::integer_sequence<T, Values...>) {
+ return {};
+}
+
+constexpr auto allOpCodes =
+ byteSeqFromIntSeq(std::make_integer_sequence<int, 256>());
+
+constexpr ByteSequence<
+ WasmBinaryEncoding::Type::i32, WasmBinaryEncoding::Type::i64,
+ WasmBinaryEncoding::Type::f32, WasmBinaryEncoding::Type::f64,
+ WasmBinaryEncoding::Type::v128>
+ valueTypesEncodings{};
+
+template<std::byte... allowedFlags>
+constexpr bool isValueOneOf(std::byte value, ByteSequence<allowedFlags...> = {}) {
+ return ((value == allowedFlags) | ... | false);
+}
+
+template<std::byte... flags>
+constexpr bool isNotIn(std::byte value, ByteSequence<flags...> = {}) {
+ return !isValueOneOf<flags...>(value);
+}
+
+struct GlobalTypeRecord {
+ Type type;
+ bool isMutable;
+};
+
+struct TypeIdxRecord {
+ size_t id;
+};
+
+struct SymbolRefContainer {
+ FlatSymbolRefAttr symbol;
+};
+
+struct GlobalSymbolRefContainer : SymbolRefContainer {
+ Type globalType;
+};
+
+struct FunctionSymbolRefContainer : SymbolRefContainer {
+ FunctionType functionType;
+};
+
+using ImportDesc = std::variant<TypeIdxRecord, TableType, LimitType, GlobalTypeRecord>;
+
+using parsed_inst_t = FailureOr<SmallVector<Value>>;
+
+struct WasmModuleSymbolTables {
+ SmallVector<FunctionSymbolRefContainer> funcSymbols;
+ SmallVector<GlobalSymbolRefContainer> globalSymbols;
+ SmallVector<SymbolRefContainer> memSymbols;
+ SmallVector<SymbolRefContainer> tableSymbols;
+ SmallVector<FunctionType> moduleFuncTypes;
+
+ std::string getNewSymbolName(StringRef prefix, size_t id) const {
+ return (prefix + Twine{id}).str();
+ }
+
+ std::string getNewFuncSymbolName() const {
+ auto id = funcSymbols.size();
+ return getNewSymbolName("func_", id);
+ }
+
+ std::string getNewGlobalSymbolName() const {
+ auto id = globalSymbols.size();
+ return getNewSymbolName("global_", id);
+ }
+
+ std::string getNewMemorySymbolName() const {
+ auto id = memSymbols.size();
+ return getNewSymbolName("mem_", id);
+ }
+
+ std::string getNewTableSymbolName() const {
+ auto id = tableSymbols.size();
+ return getNewSymbolName("table_", id);
+ }
+};
+
+class ParserHead;
+
+/// Wrapper around SmallVector to only allow access as push and pop on the
+/// stack. Makes sure that there are no "free accesses" on the stack to preserve
+/// its state.
+class ValueStack {
+private:
+ struct LabelLevel {
+ size_t stackIdx;
+ LabelLevelOpInterface levelOp;
+ };
+public:
+ bool empty() const { return values.empty(); }
+
+ size_t size() const { return values.size(); }
+
+ /// Pops values from the stack because they are being used in an operation.
+ /// @param operandTypes The list of expected types of the operation, used
+ /// to know how many values to pop and check if the types match the
+ /// expectation.
+ /// @param opLoc Location of the caller, used to report accurately the
+ /// location
+ /// if an error occurs.
+ /// @return Failure or the vector of popped values.
+ FailureOr<SmallVector<Value>> popOperands(TypeRange operandTypes,
+ Location *opLoc);
+
+ /// Push the results of an operation to the stack so they can be used in a
+ /// following operation.
+ /// @param results The list of results of the operation
+ /// @param opLoc Location of the caller, used to report accurately the
+ /// location
+ /// if an error occurs.
+ LogicalResult pushResults(ValueRange results, Location *opLoc);
+
+
+#if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
+ /// A simple dump function for debugging.
+ /// Writes output to llvm::dbgs().
+ LLVM_DUMP_METHOD void dump() const;
+#endif
+
+private:
+ SmallVector<Value> values;
+};
+
+using local_val_t = TypedValue<wasmssa::LocalRefType>;
+
+class ExpressionParser {
+public:
+ using locals_t = SmallVector<local_val_t>;
+ ExpressionParser(ParserHead &parser, WasmModuleSymbolTables const &symbols,
+ ArrayRef<local_val_t> initLocal)
+ : parser{parser}, symbols{symbols}, locals{initLocal} {}
+
+private:
+ template <std::byte opCode>
+ inline parsed_inst_t parseSpecificInstruction(OpBuilder &builder);
+
+ template <typename valueT>
+ parsed_inst_t
+ parseConstInst(OpBuilder &builder,
+ std::enable_if_t<std::is_arithmetic_v<valueT>> * = nullptr);
+
+
+ /// This function generates a dispatch tree to associate an opcode with a
+ /// parser. Parsers are registered by specialising the
+ /// `parseSpecificInstruction` function for the op code to handle.
+ ///
+ /// The dispatcher is generated by recursively creating all possible patterns
+ /// for an opcode and calling the relevant parser on the leaf.
+ ///
+ /// @tparam patternBitSize is the first bit for which the pattern is not fixed
+ ///
+ /// @tparam highBitPattern is the fixed pattern that this instance handles for
+ /// the 8-patternBitSize bits
+ template <size_t patternBitSize = 0, std::byte highBitPattern = std::byte{0}>
+ inline parsed_inst_t dispatchToInstParser(std::byte opCode,
+ OpBuilder &builder) {
+ static_assert(patternBitSize <= 8,
+ "PatternBitSize is outside of range of opcode space! "
+ "(expected at most 8 bits)");
+ if constexpr (patternBitSize < 8) {
+ constexpr std::byte bitSelect{1 << (7 - patternBitSize)};
+ constexpr std::byte nextHighBitPatternStem = highBitPattern << 1;
+ constexpr size_t nextPatternBitSize = patternBitSize + 1;
+ if ((opCode & bitSelect) != std::byte{0})
+ return dispatchToInstParser < nextPatternBitSize,
+ nextHighBitPatternStem | std::byte{1} > (opCode, builder);
+ return dispatchToInstParser<nextPatternBitSize, nextHighBitPatternStem>(
+ opCode, builder);
+ } else {
+ return parseSpecificInstruction<highBitPattern>(builder);
+ }
+ }
+
+ struct ParseResultWithInfo {
+ SmallVector<Value> opResults;
+ std::byte endingByte;
+ };
+
+public:
+ template<std::byte ParseEndByte = WasmBinaryEncoding::endByte>
+ parsed_inst_t parse(OpBuilder &builder,
+ UniqueByte<ParseEndByte> = {});
+
+ template <std::byte... ExpressionParseEnd>
+ FailureOr<ParseResultWithInfo>
+ parse(OpBuilder &builder,
+ ByteSequence<ExpressionParseEnd...> parsingEndFilters);
+
+ FailureOr<SmallVector<Value>>
+ popOperands(TypeRange operandTypes) {
+ return valueStack.popOperands(operandTypes, ¤tOpLoc.value());
+ }
+
+ LogicalResult pushResults(ValueRange results) {
+ return valueStack.pushResults(results, ¤tOpLoc.value());
+ }
+private:
+ std::optional<Location> currentOpLoc;
+ ParserHead &parser;
+ WasmModuleSymbolTables const &symbols;
+ locals_t locals;
+ ValueStack valueStack;
+ };
+
+class ParserHead {
+public:
+ ParserHead(StringRef src, StringAttr name) : head{src}, locName{name} {}
+ ParserHead(ParserHead &&) = default;
+private:
+ ParserHead(ParserHead const &other) = default;
+
+public:
+ auto getLocation() const {
+ return FileLineColLoc::get(locName, 0, anchorOffset + offset);
+ }
+
+ FailureOr<StringRef> consumeNBytes(size_t nBytes) {
+ LLVM_DEBUG(llvm::dbgs() << "Consume " << nBytes << " bytes\n");
+ LLVM_DEBUG(llvm::dbgs() << " Bytes remaining: " << size() << "\n");
+ LLVM_DEBUG(llvm::dbgs() << " Current offset: " << offset << "\n");
+ if (nBytes > size())
+ return emitError(getLocation(), "trying to extract ")
+ << nBytes << "bytes when only " << size() << "are avilables";
+
+ StringRef res = head.slice(offset, offset + nBytes);
+ offset += nBytes;
+ LLVM_DEBUG(llvm::dbgs()
+ << " Updated offset (+" << nBytes << "): " << offset << "\n");
+ return res;
+ }
+
+ FailureOr<std::byte> consumeByte() {
+ auto res = consumeNBytes(1);
+ if (failed(res))
+ return failure();
+ return std::byte{*res->bytes_begin()};
+ }
+
+ template <typename T>
+ FailureOr<T> parseLiteral();
+
+ FailureOr<uint32_t> parseVectorSize();
+
+private:
+ // TODO: This is equivalent to parseLiteral<uint32_t> and could be removed
+ // if parseLiteral specialization were moved here, but default GCC on Ubuntu
+ // 22.04 has bug with template specialization in class declaration
+ inline FailureOr<uint32_t> parseUI32();
+ inline FailureOr<int64_t> parseI64();
+
+public:
+ FailureOr<StringRef> parseName() {
+ FailureOr<uint32_t> size = parseVectorSize();
+ if (failed(size))
+ return failure();
+
+ return consumeNBytes(*size);
+ }
+
+ FailureOr<WasmSectionType> parseWasmSectionType() {
+ FailureOr<std::byte> id = consumeByte();
+ if (failed(id))
+ return failure();
+ if (std::to_integer<unsigned>(*id) > highestWasmSectionID)
+ return emitError(getLocation(), "invalid section ID: ")
+ << static_cast<int>(*id);
+ return static_cast<WasmSectionType>(*id);
+ }
+
+ FailureOr<LimitType> parseLimit(MLIRContext *ctx) {
+ using WasmLimits = WasmBinaryEncoding::LimitHeader;
+ FileLineColLoc limitLocation = getLocation();
+ FailureOr<std::byte> limitHeader = consumeByte();
+ if (failed(limitHeader))
+ return failure();
+
+ if (isNotIn<WasmLimits::bothLimits, WasmLimits::lowLimitOnly>(*limitHeader))
+ return emitError(limitLocation, "invalid limit header: ")
+ << static_cast<int>(*limitHeader);
+ FailureOr<uint32_t> minParse = parseUI32();
+ if (failed(minParse))
+ return failure();
+ std::optional<uint32_t> max{std::nullopt};
+ if (*limitHeader == WasmLimits::bothLimits) {
+ FailureOr<uint32_t> maxParse = parseUI32();
+ if (failed(maxParse))
+ return failure();
+ max = *maxParse;
+ }
+ return LimitType::get(ctx, *minParse, max);
+ }
+
+ FailureOr<Type> parseValueType(MLIRContext *ctx) {
+ FileLineColLoc typeLoc = getLocation();
+ FailureOr<std::byte> typeEncoding = consumeByte();
+ if (failed(typeEncoding))
+ return failure();
+ switch (*typeEncoding) {
+ case WasmBinaryEncoding::Type::i32:
+ return IntegerType::get(ctx, 32);
+ case WasmBinaryEncoding::Type::i64:
+ return IntegerType::get(ctx, 64);
+ case WasmBinaryEncoding::Type::f32:
+ return Float32Type::get(ctx);
+ case WasmBinaryEncoding::Type::f64:
+ return Float64Type::get(ctx);
+ case WasmBinaryEncoding::Type::v128:
+ return IntegerType::get(ctx, 128);
+ case WasmBinaryEncoding::Type::func...
[truncated]
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
30ec2ac
to
2cb8732
Compare
When you can't add review, just @ them in comments after opening up the PR if needed. |
c68ffbb
to
f2a38c9
Compare
Also contains non functional changes to use the `LDBG` macro.
f2a38c9
to
6f6c527
Compare
This is due to the tests in the wasm importer requiring it.
55a4acd
to
819e297
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR introduces an importer for WebAssembly (Wasm) binaries that translates them to MLIR using the WasmSSA dialect. It provides the foundation for a wasm-import
target in mlir-translate, focusing on establishing the core parsing infrastructure with initial support for basic WebAssembly language features.
- Implements a binary WebAssembly parser that can handle sections, imports, functions, tables, and memory declarations
- Creates translation registration and core parsing infrastructure for WebAssembly binary format
- Adds comprehensive test coverage for various WebAssembly constructs including imports, memory configurations, and error handling
Reviewed Changes
Copilot reviewed 23 out of 23 changed files in this pull request and generated 7 comments.
Show a summary per file
File | Description |
---|---|
mlir/lib/Target/Wasm/TranslateFromWasm.cpp | Main implementation of the WebAssembly binary parser and translator |
mlir/lib/Target/Wasm/TranslateRegistration.cpp | Registration of the wasm-import translation target |
mlir/include/mlir/Target/Wasm/WasmImporter.h | Public API header for WebAssembly import functionality |
mlir/include/mlir/Target/Wasm/WasmBinaryEncoding.h | Constants and encodings for WebAssembly binary format |
mlir/test/Target/Wasm/*.mlir | Test files for various WebAssembly import scenarios |
mlir/test/Target/Wasm/inputs/*.yaml.wasm | Test input files in YAML format for WebAssembly binaries |
mlir/test/Target/Wasm/*.yaml | Error condition test cases |
mlir/test/CMakeLists.txt | Addition of yaml2obj dependency for tests |
mlir/lib/Target/CMakeLists.txt | Build system integration for Wasm target |
mlir/include/mlir/InitAllTranslations.h | Registration of the new translation in the global registry |
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/169/builds/14060 Here is the relevant piece of the build log for the reference
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/172/builds/14509 Here is the relevant piece of the build log for the reference
|
I fixed the leak here: #153794 |
@flemairen6 the second breakage is fixed here: #153795 I also see a lot of warnings when building right now, is it because you implemented the structure and will use the code later?
|
@joker-eph : I can't directly reply to your comments, so writing here.
Yes, it makes sense. Thank you very much for fixing it.
I don't think we had specific reasons to use statistic, and for what we do with them at the moment, debug output is as good. For the Thanks again for the fixes. |
@flemairen6 @joker-eph I've landed f4bc315 to fix the warnings from this PR for now. I used |
llvm#152131 added a few tests that depend on mlir/test/Target/Wasm/inputs/*, e.g. mlir/test/Target/Wasm/import.mlir reads inputs/import.yaml.wasm. These inputs should be included as data dependency.
#152131 added a few tests that depend on `mlir/test/Target/Wasm/inputs/*`, e.g. `mlir/test/Target/Wasm/import.mlir` reads `inputs/import.yaml.wasm`. These inputs should be included as data dependency.
…get. (#153854) llvm/llvm-project#152131 added a few tests that depend on `mlir/test/Target/Wasm/inputs/*`, e.g. `mlir/test/Target/Wasm/import.mlir` reads `inputs/import.yaml.wasm`. These inputs should be included as data dependency.
llvm#152131 uses yaml2obj, which is not listed as a dependency of the lit tests in bazel. This is causing LLVM CI failures, e.g [1]. [1]: https://buildkite.com/llvm-project/upstream-bazel/builds/146788/steps/canvas?sid=0198af37-f624-470f-aac1-d9e0b42fab56
#152131 uses yaml2obj, which is not listed as a dependency of the lit tests in bazel. This is causing LLVM CI failures, e.g [1]. [1]: https://buildkite.com/llvm-project/upstream-bazel/builds/146788/steps/canvas?sid=0198af37-f624-470f-aac1-d9e0b42fab56
…875) llvm/llvm-project#152131 uses yaml2obj, which is not listed as a dependency of the lit tests in bazel. This is causing LLVM CI failures, e.g [1]. [1]: https://buildkite.com/llvm-project/upstream-bazel/builds/146788/steps/canvas?sid=0198af37-f624-470f-aac1-d9e0b42fab56
…154053) This is the continuation of #152131 This PR adds support for parsing the global initializer and function body, and support for decoding scalar numerical instructions and variable related instructions. --------- Co-authored-by: Ferdinand Lemaire <ferdinand.lemaire@woven-planet.global> Co-authored-by: Jessica Paquette <jessica.paquette@woven-planet.global> Co-authored-by: Luc Forget <luc.forget@woven.toyota>
…lvm#154053) This is the continuation of llvm#152131 This PR adds support for parsing the global initializer and function body, and support for decoding scalar numerical instructions and variable related instructions. --------- Co-authored-by: Ferdinand Lemaire <ferdinand.lemaire@woven-planet.global> Co-authored-by: Jessica Paquette <jessica.paquette@woven-planet.global> Co-authored-by: Luc Forget <luc.forget@woven.toyota>
This is a follow-up to #149233
First step in introducing the wasm-import target to mlir-translate.
This is the first PR to introduce the pass, with this PR, there is very little support for the actual WebAssembly language, it's mostly there to introduce the skeleton of the importer. A follow-up will come with support for a wider range of operators. It was split to make it easier to review, since it's a good chunk of work.