Skip to content

[DE-771] Issue with running queries in stream transaction  #298

Open
@jurisitsm

Description

@jurisitsm

Hi!
An application my team is building, uses arango database to process large chunks incoming data periodically.
For each individual batch of incoming data there is a large number of different db operations that have to executed (insertion, deletion, queries, updates etc.)
The current implementation has some performance issues, therefore we decided to rewrite the code responsible for executing these db operations to use arangodb's stream transaction.
The problem is, it seems that querying data, that has been added in the same transaction that it is being queried in, does not always return the legitimate results. I am currently experiencing this with edge collections.

We start the transaction the following way:

 var transaction = db.beginStreamTransaction(
        new StreamTransactionOptions().writeCollections(collections));    //array with the names of collection used
    try {
      //process data
      db.commitStreamTransaction(transaction.getId());
    } catch (Exception e) {
      db.abortStreamTransaction(transaction.getId());
    }

During this processing, if I try to query nodes that have been added in this same transaction, I can successfully achieve it like so:

  private static final String GET_NODES_QUERY = """
      FOR nodeEntity IN nodeEntities
          RETURN nodeEntity
      """;

    public List<NodeEntity> getNodeEntitiesInTransaction(String transactionId) {
    return arangoOperations.query(GET_NODES_QUERY, createQueryOptions(transactionId), NodeEntity.class).asListRemaining();
  }
  
    private AqlQueryOptions createQueryOptions(String transactionId) {
    var options = new AqlQueryOptions();
    options.streamTransactionId(transactionId);
    return options;
  }

But if I try to do the exact same logic, but instead of a collection annotated Document, it is one annotated Edge, the query always returns an empty response, even though I know, several instances of the given edge class have already been added. Apart from debugging I also know this, because after the transaction with given id is committed, the instances of the edge classes become immediately available.
I would appreciate help with this issue.
Additionally, I have another question, that is, whether the query method of the ArangoOperations class can only execute non-modifying queries, or any AQL statement provided as string (deletion for instance).

Thank you in advance.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions