[wip][core][spark][flink] Support TRUNCATE TABLE on format tables

[sundapeng]

Stacked on #9296. Its commit shows up here too until it merges. Review only the last commit.

Purpose

TRUNCATE TABLE did not work on a Format Table, and the two statements failed differently.

TRUNCATE TABLE t was rejected in the planner: PaimonFormatTable does not implement
TruncatableTable, so DataSourceV2Strategy failed evaluating asTruncatable with
Table does not support truncates: <table>.

TRUNCATE TABLE t PARTITION (...) got further, into PaimonPartitionManagement, which serves only
FileStoreTable and failed there with Only FileStoreTable supports partitions. — a message about
an internal type, on a table that has partitions and lists them under SHOW PARTITIONS.

Underneath both, FormatTableCommit.truncateTable() and truncatePartitions() threw an empty
UnsupportedOperationException, and Flink's FlinkFormatTableSink did not implement
SupportsTruncate either.

The workaround was INSERT OVERWRITE ... PARTITION (...) SELECT ... WHERE false, which does not
extend to the whole table: without a static partition, an overwrite deletes only the directories it
actually wrote a file into, so an empty result clears nothing.

Approach

FormatTableCommit deletes the data files, reusing deletePreviousDataFile, the primitive a static
INSERT OVERWRITE already clears partition directories with. Only data files go: the partition
directories stay, and so do their catalog registrations, so SHOW PARTITIONS returns what it
returned before (SPARK-34418). Staging trees of concurrent writers are left alone, on the same
judgement FormatTableScan reads with.

What the emptied partitions hold is reported to the catalog as an exact zero, dated to the
truncation and carried with replaceStatistics, so a truncated partition stops describing files
that are gone. The two operations differ in what they claim: an overwrite reports only the files it
removed itself, so that concurrent writers do not each claim the whole subtree, while truncation
states that the partition holds nothing whoever deleted the files — a partition that was already
empty reports zero as well.

Which partitions the table has is answered by whatever the table reads its partitions from. For a
catalog-managed table that is the catalog, so truncating the whole table empties the registered
partitions and leaves a directory still awaiting MSCK REPAIR TABLE alone — matching what
TruncateTableCommand does for a v1 partitioned table. Under filesystem partition discovery the
directory is the answer, and truncating clears the partition levels below the table root.

On the Spark side PaimonFormatTable implements TruncatableTable and overrides
SupportsAtomicPartitionManagement.truncatePartition(s) rather than taking the statement over in
PaimonStrategy: TruncatePartitionExec already expands a partial spec and refreshes the cache,
and managed tables reach truncation through the same interface. Truncating a named partition the
table does not have is an error, reported per entry point as Spark's TruncateTableSuiteBase
expects.

Flink has no TRUNCATE TABLE ... PARTITION, so FlinkFormatTableSink implements only whole-table
truncation.

Tests

• FormatTableCommitTest: whole-table truncate under filesystem discovery and on a catalog-managed
  table (registered partitions emptied, an unregistered directory untouched), an unpartitioned
  table, named partitions, a leading-prefix spec, and the value-only default partition; staging
  trees survive throughout.
• FormatTableCommitStatisticsTest: named partitions report an exact zero as a total, a partition
  that was already empty reports zero too, a prefix reports the partitions underneath it, and
  truncating the table reports every registered partition and no unregistered one.
• FormatTablePartitionManagementTest, CatalogManagedPartitionTest: registrations survive,
  unregistered partitions are refused, a .. value is rejected before anything is deleted, and an
  end-to-end truncate through the REST catalog.
• FormatTableTestBase: TRUNCATE TABLE, TRUNCATE TABLE ... PARTITION with a full and a partial
  spec, and a partition the table does not have.
• FormatTableITCase: TRUNCATE TABLE on a partitioned and an unpartitioned Format Table in Flink.

API and Format

No format change. PaimonFormatTable gains TruncatableTable and FlinkFormatTableSink gains
SupportsTruncate — both engine-side interfaces already implemented for managed tables.
format-table.implementation = engine is unchanged and still does not support truncation.

Documentation

docs/docs/spark/sql-write.md and docs/docs/flink/sql-write.mdx.