Unify method signatures for synchronous and asynchronous commit

[patchwork01]

Background

For asynchronous commits to the state store, updates are submitted to TransactionLogStateStore.addTransaction. For synchronous commits, updates are done against a number of methods on the interfaces FileReferenceStoreUpdates and PartitionStore.

Description

We'd like to simplify how we make updates to the state store, whether they're done synchronously or asynchronously.

We'd like this to make it easier to move the job tracker updates from where they are now during synchronous updates, to being derived from the transaction log.

We'd like to remove the old methods for performing state store updates, and just keep addTransaction.

Analysis

This will be easier after the S3 state store implementation is removed:

• Delete S3 state store implementation #4261

We can split out separate issues for each method signature if necessary. Here are some possible changes:

• Replace simple in memory state store implementation with in memory transaction log #4271
• Move addTransaction method to StateStore interface #4270
• State store wrapper for testing to preserve existing update methods #4279
• Apply state store wrapper for updates to all tests #4280
• Simplify production code to add a transaction sync/asynchronously #4281
• Simplify production code to perform ingest job tracker updates based on a state store change sync/asynchronously #4282
• Simplify production code to perform compaction job tracker updates based on a state store change sync/asynchronously #4323