Skip to content

New Presplit utility for Fate table#6471

Open
ArbaazKhan1 wants to merge 2 commits into
apache:mainfrom
ArbaazKhan1:accumulo-4979
Open

New Presplit utility for Fate table#6471
ArbaazKhan1 wants to merge 2 commits into
apache:mainfrom
ArbaazKhan1:accumulo-4979

Conversation

@ArbaazKhan1

Copy link
Copy Markdown
Contributor

closes issue #4979

Added support for pre-splitting Fate tables via accumulo admin command.

  • --numSplits option generates split points using the same partitioning logic as in FateManager.getDesiredPartitions. The logic ensures that pre-split boundires align with how the manger expects transaction IDs to be distributed. The split utility produces raw UUID row strings which should allow it be also be used by scan_ref.

  • --splitsFile flag writes the generated splits to a file.

Comment on lines +164 to +165
@Parameter(names = {"-n", "--numSplits"},
description = "Generate N split points for the fate table and print to stdout. N should always be greater than 1.")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
@Parameter(names = {"-n", "--numSplits"},
description = "Generate N split points for the fate table and print to stdout. N should always be greater than 1.")
@Parameter(names = {"-n", "--num-splits"},
description = "Generate N split points for the fate table and print to stdout. N must be >= 1.")

Comment on lines +294 to +295
try (PrintWriter writer = new PrintWriter(new BufferedWriter(
new OutputStreamWriter(Files.newOutputStream(Path.of(options.splitsFile)), UTF_8)))) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I usually try not to nest constructors in a try-with-resources block. Usually things work out okay but putting each object constructed on its own line makes it more certain that each object will be cleaned up properly in the try-with-resources block as opposed to relying on the closure of the outmost object to cascade down and close the rest.


@Parameter(names = {"-n", "--numSplits"},
description = "Generate N split points for the fate table and print to stdout. N should always be greater than 1.")
int numSplits = 1;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
int numSplits = 1;
int numSplits = -1;

I think we need to default to -1 here so that split generation is opt-in. If we leave it as 1, every fate invocation takes the numSplits > 0 branch and returns before normal commands like --print, --cancel, etc. can run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create utility to presplit the fate table

2 participants