I am running into an issue where the call to consumePartitions returns a non-empty List of PartitionsDetails where each element in the list is null. Every time I call consumePartitions it returns the same list and because the list has null elements I cannot call onFinish without getting a null pointer exception.
To reproduce the issue set the ConsumerConfiguration maxWorkingSetSize to 1000. Add two partitions to the PartitionFileSet and call consumePartitions(1) once to consume the first partition. Truncate the PartitionFileSet and call consumePartitions(1) again to retrieve the list with null PartitionDetails. Calling DropPartition on each of the partitions in the PartitionFileSet will cause the same behavior as preforming the truncate. Adding a new partition will not show up when consumePartitions(1) is called.