perf(vindex): parallelize Parquet reads and multipart index uploads

[jerry-024]

Purpose

Speed up vector-index builds by removing two object-storage bottlenecks:

1. full-file scans were unnecessarily excluded from parallel Parquet row-group reads;
2. serialized indexes uploaded multipart chunks one at a time.

Changes

• Treat full-coverage row_ranges without a deletion vector as an unfiltered scan, allowing the existing parallel row-group path to run. Partial ranges, deletion vectors, and empty ranges keep their previous semantics.
• Keep oversized row groups charged against the full configured Parquet read budget, preserving max_inflight_bytes as a strict projected-byte admission limit, and warn once when a row group exceeds it.
• Upload vector-index multipart chunks with four parts in flight. Other writers retain their existing serial behavior.
• Add opt-in aggregate diagnostics for vector-index build phases and Parquet row-group reads.
• Add a production-path IVF-PQ build benchmark example.

Reproduce the IVF-PQ build benchmark

The benchmark is in ivfpq_build_benchmark.rs. It builds an IVF-PQ index with dimension 768, cosine distance, nlist=4096, and pq.m=192 on an existing Paimon table:

PAIMON_CATALOG_OPTIONS='<catalog-options-json>' \
PAIMON_LOG_VECTOR_INDEX_BUILD_TIMING=1 \
cargo run --release -p paimon --example ivfpq_build_benchmark -- \
  <database> <table> <vector-column> --drop-existing

Benchmark

Cohere 10M × 768 dimensions, cosine IVF-PQ (nlist=4096, pq.m=192, 8 bits), 10 Parquet files / 100 row groups, direct OSS, and no local cache.

Parallel Parquet reads

Both baseline and this PR use an explicit 512 MiB Parquet read budget, so the comparison isolates the changes in this PR while retaining strict byte accounting.

metric	baseline	this PR	change
end-to-end build, 3-run median	546.153s	527.113s	−3.49%
source batch wait	105.346s	59.361s	−43.65%
peak in-flight row groups	1	2	+1
peak RSS	4.24 GiB	4.37 GiB	+137 MB

The 512 MiB budget is a workload-specific benchmark setting and must be configured explicitly. The default remains 256 MiB.

Concurrent multipart upload

With all other benchmark inputs held constant, four in-flight multipart uploads reduced the serialize/upload phase from 30.3s to 4.6s (−84.8%). The vector-index writer alone uses this concurrency; other writers keep their existing behavior.

Tests

• Full, stitched, partial, empty, and deletion-vector row selections, including _ROW_ID parity.
• Shared Parquet budget accounting and peak in-flight diagnostics.
• Oversized row groups consume the full budget and emit a one-time warning; ordinary and tiny row-group accounting remains strict.
• Vector-index build timing aggregation and existing builder tests.

API and format

No public API or file/index format changes. The concurrent writer helper is crate-private.