Added support for aarch64 and ppc64le binaries #161
Conversation
|
The checks are failing as expected due to the issue with missing ppc64le/aarch64 binaries: [INFO] ------------------------------------------------------------------------ This can be easly fixed in pom.xml or yaml running the pom, but I would rather discusse the "secondary arches" with you first. |
…ed arch64 for all three linuxes
and thus giving up hopes for simple fix for failing aarch64 and ppc64le tests
927170e to
72fde46
Compare
|
Hello @judovana, There are 2 major problem with the test suite and possibly with the Java API as such: 4096 is not the page size on all systemsThe default expectation of LMDB Java is that the page size on the system is 4096. That does not hold. Ultimately my aarch64 and ppc64le (Power 9) systems running RHEL 8 report: LMDB Java reads that as 1/2, i.e. gives LMDB native API uses pages, not bytesThe LMDB Java code looks as if So, in our case on these aarch64 and ppc64le systems, their diff --git a/src/main/java/org/lmdbjava/Env.java b/src/main/java/org/lmdbjava/Env.java
index 7634a34..c9c3397 100644
--- a/src/main/java/org/lmdbjava/Env.java
+++ b/src/main/java/org/lmdbjava/Env.java
@@ -486,7 +486,7 @@ public final class Env<T> implements AutoCloseable {
checkRc(LIB.mdb_env_create(envPtr));
final Pointer ptr = envPtr.getValue();
try {
- checkRc(LIB.mdb_env_set_mapsize(ptr, mapSize));
+ checkRc(LIB.mdb_env_set_mapsize(ptr, 8 * mapSize));
checkRc(LIB.mdb_env_set_maxdbs(ptr, maxDbs));
checkRc(LIB.mdb_env_set_maxreaders(ptr, maxReaders));
final int flagsMask = mask(flags);
diff --git a/src/test/java/org/lmdbjava/DbiTest.java b/src/test/java/org/lmdbjava/DbiTest.java
index b9bf453..c35c440 100644
--- a/src/test/java/org/lmdbjava/DbiTest.java
+++ b/src/test/java/org/lmdbjava/DbiTest.java
@@ -457,7 +457,6 @@ public final class DbiTest {
assertThat(stat.entries, is(3L));
assertThat(stat.leafPages, is(1L));
assertThat(stat.overflowPages, is(0L));
- assertThat(stat.pageSize, is(4_096));
}
@Test(expected = MapFullException.class)
diff --git a/src/test/java/org/lmdbjava/EnvTest.java b/src/test/java/org/lmdbjava/EnvTest.java
index 3e9ece9..811b18c 100644
--- a/src/test/java/org/lmdbjava/EnvTest.java
+++ b/src/test/java/org/lmdbjava/EnvTest.java
@@ -26,6 +26,7 @@ import static org.hamcrest.CoreMatchers.containsString;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.CoreMatchers.notNullValue;
import static org.hamcrest.MatcherAssert.assertThat;
+import static org.hamcrest.Matchers.greaterThanOrEqualTo;
import static org.hamcrest.Matchers.greaterThan;
import static org.lmdbjava.CopyFlags.MDB_CP_COMPACT;
import static org.lmdbjava.DbiFlags.MDB_CREATE;
@@ -68,7 +69,8 @@ public final class EnvTest {
.setMapSize(MEBIBYTES.toBytes(1))
.open(path, MDB_NOSUBDIR)) {
final EnvInfo info = env.info();
- assertThat(info.mapSize, is(MEBIBYTES.toBytes(1)));
+ // `is' does not make sense, one API uses bytes, the other uses pages...
+ assertThat(info.mapSize, greaterThanOrEqualTo(MEBIBYTES.toBytes(1)));
}
}
@@ -277,7 +279,7 @@ public final class EnvTest {
assertThat(info.lastPageNumber, is(1L));
assertThat(info.lastTransactionId, is(0L));
assertThat(info.mapAddress, is(0L));
- assertThat(info.mapSize, is(123_456L));
+ assertThat(info.mapSize, is(8 * 123_456L));
assertThat(info.maxReaders, is(4));
assertThat(info.numReaders, is(0));
assertThat(info.toString(), containsString("maxReaders="));
@@ -355,7 +357,7 @@ public final class EnvTest {
}
assertThat(mapFullExThrown, is(true));
- env.setMapSize(500_000);
+ env.setMapSize(8 * 500_000);
try (Txn<ByteBuffer> roTxn = env.txnRead()) {
assertThat(db.get(roTxn, bb(1)).getInt(), is(42));
@@ -390,7 +392,6 @@ public final class EnvTest {
assertThat(stat.entries, is(0L));
assertThat(stat.leafPages, is(0L));
assertThat(stat.overflowPages, is(0L));
- assertThat(stat.pageSize, is(4_096));
assertThat(stat.toString(), containsString("pageSize="));
}
}FixI don't know how to solve it elegantly at the moment. I propose dropping asserts for hardcoded It would help to get a comment from maintainers. I can implement the fix if you hint what would be a good direction. |
|
Perhaps @bambulin might know about the API discrepancy. Kozliku, did you use bytes or pages for setting |
|
@Karm we don't set the flag directly - the Java API offers |
|
@Karm can the magical 8 be please extracted to constant? If lmdb is able to return what size it expects, it sounds as good fix. I do not htink removal od asserts is god idea. It can hide an bug later hard to find. Until the fix is presented, the constant hsould be at least filed by if arch depndence, |
|
This PR can be closed as it was surpassed by #165 All tests pass on all secondary archs now (and amd64 is not broken either :) ) |
This PR is using new libraries created in lmdbjava/native#7 .
The usage for this "secondary architecture" is now somethng like:
Whole concept seems to be working, with two glitches:
I have to be doing something wrong, or lmdbjava must be doing somethng I faild to discover, as lmdb itself seems to work fine on aarch64/ppc64le. Both based on linux ditributions usages and on unittests.
ERROR] Failures:
[ERROR] DbiTest.stats:460
Expected: is <4096>
but: was <32768>
[ERROR] EnvTest.stats:393
Expected: is <4096>
but: was <32768>
[ERROR] Errors:
[ERROR] CursorIterableTest.before:132 » MapFull Environment mapsize reached (-30792)
... Skipped 11 identical lines...
[ERROR] CursorIterableTest.before:132 » MapFull Environment mapsize reached (-30792)
[ERROR] EnvTest.setMapSize:341 » MapFull Environment mapsize reached (-30792)
[ERROR] TxnTest.largeKeysRejected » Unexpected exception, expected<org.lmdbjava.Dbi$B...
[ERROR] TxnTest.rangeSearch:103 » MapFull Environment mapsize reached (-30792)
[ERROR] TxnTest.readOnlyTxnAllowedInReadOnlyEnv:138 » MapFull Environment mapsize reac...
[ERROR] TxnTest.readWriteTxnDeniedInReadOnlyEnv » Unexpected exception, expected<org....
[ERROR] TxnTest.testGetId:181 » MapFull Environment mapsize reached (-30792)
[ERROR] TxnTest.zeroByteKeysRejected » Unexpected exception, expected<org.lmdbjava.Db...
[INFO]
[ERROR] Tests run: 187, Failures: 2, Errors: 31, Skipped: 0
Before i spam OpenLDAP, do you have any idea what may be going on?