Aggregation node missing start of batch

Symptoms

A batch in an Aggregation-type node has a start time that is later than expected. The batch is missing parts that should have been included in the aggregated batch.

There is also a Warning entry like the following, in the SIMCA-online server log. The times listed show the missing parts of the aggregated batch:

[2022-10-20 08:45:36.502+02:00] [TID:13560] [Warning] [BatchAggregator::AggregateBlock()]

Node: BakersYeastAgg

Duplicate batch ID: hb (2022-10-20 08:09:33 (1666246173) - 2022-10-20 08:27:00 (1666247220)). Batch not saved.

Cause

Batch Context Generator incorrectly creates batches that should not have been created because they are longer than 1 Maximum Batch Length ago during the initial current scan of data close to now.

This can happen when the Aggregation node is filled with data either

manually with Find batches when looking for recent batches close to now, or
When the node is first used by a project configuration for real-time execution

Recent batches close to now means batches that lived within the last 1 Maximum Batch Length (as configured in the node).

For example: The node's Maximum Batch Length is 24 hours. There are two parts of a batch: A1 at 50 h ago, and another A2 at 45 hours ago. We expect these to be aggregated to one batch, A.

If you do Find batches from 48 hours ago to 25 hours ago that is NOT close to now. The batches A1 and A2 will be correctly aggregated to A.
After clearing the node, if you do Find batches from 48 to 23 hours ago, that IS close to now because you are closer than 1 Maximum Batch Length ago. Now batch A2 will be aggregated to A, but A1 will be skipped because of the problem described in this article.

Workaround

Clear the node.
Do a manual Find batches for a time ending just before 1 Maximum Batch Length ago. The start time does not matter.

This ensures that batches are picked up correctly and after this you can do Find batches as you like.

More information

This problem does not happen when Batch Context Generator node is being used continuously. It can happen the first time the node is used, or after a break longer than 2 * the configured Maximum Batch Length for the node.

Internal bug number: 43521

Status

This is fixed in SIMCA-online 17.0.2.