documentation: 'compact' command; explain how to get max compact

[jumper444]

Small documentation suggestion. The 'compact' command includes --threshold switch defined as:

"...A segment is compacted if the amount of saved space is above the percentage value given by the --threshold option. If omitted, a threshold of 10% is used...."
That is the entire discussion of the switch.

I'm upgrading from a years older borg version (without compact command) and thought:

"well...i'm upgrading big....so, for one time, I want to get the MAX compact possible".

Documentation doesn't say anything about that (although presumptions can be made, but that is an uncomfortable word on backup commands).

I thought: "it is integer percent so I guess my choice would be 0 or 1 as 'max compact' [but it doesnt say]. In a perfect world it seems 0 would be best, but often on computers there are zero bound issues or things break when using zero. Documentation doesnt say the range for 'threshold' [0-99]? [1-100]? [1-99]? or tradeoffs on the number choices (does the time increase linearly with lower numbers or exponential?)"

"...Hmm...well the documentation doesnt SAY it wont take zero; It also doesn't tell me how to get max compaction. I guess I will just try zero and see what happens"

[as i type this seems to be working but VERY slow; would 1 have been better choice getting almost the same result but with dramatic speed up? aka linear vs exponential with lower numbers? I dont know(I cant run the command twice to compare unless i duplicate my entire repo first and then test run 0 and 1 on each separately; would rather the instructions mention)]

SUGGESTION: Add one or two sentences to 'compact' command which explains how to do a max compact and also mention any negatives. (Was 10 chosen as default due to an unmentioned tradeoff? Does 1 or 0 take non-linear time vs 10?)

[ThomasWaldmann]

Yeah, maximum compaction is with --threshold 0 (or 1, if 0 is not accepted).

But be aware that this might cause a lot of I/O without winning much, that's the reason why the default is at 10 (percent).

borg's segment files default to 500MiB size, so if you use 0% threshold it might free only a few bytes (in the worst case), but would read 500MiB from the non-compact segment file, write 500MiB to a new compact segment file and then delete the old segment file.

[jumper444]

I dont know how to put in a PR for docs, but I think the COMPACT command page should mention:

1. the lower bound for threshold switch is 0;
2. it will cause a complete compaction;
3. but that could result in a very large amount of reading and rewriting as you have said;
4. if user wants almost complete compaction but not 500M read/writes to free (for example) 700 bytes then using threshold of 1 is the best answer (unless Borg ever offers fractional thresholds like 0.1, 0.2, etc) (does it? does threshold have to be an integer?)

(I personally think 1-2% is a good number since 5M-10M is a not an unreasonable amount of storage to reclaim, but I'm ok with a few single meg lying around. 10% creating a 50M limit (but keeping 49M free spaces) seems a bit high to me, but as you said it is only a default and the user can choose. My point is that their choice is benefitted by knowing points 1,2,3, and 4.)

[ThomasWaldmann]

#9114