legacy-timestamp-mapping.enabled |
false |
Boolean |
If true, map Paimon TIMESTAMP to Spark TIMESTAMP instead of TIMESTAMP_NTZ. |
read.allow.fullScan |
true |
Boolean |
Whether to allow full scan when reading a partitioned table. |
read.changelog |
false |
Boolean |
Whether to read row in the form of changelog (add rowkind column in row to represent its change type). |
read.stream.maxBytesPerTrigger |
(none) |
Long |
The maximum number of bytes returned in a single batch. |
read.stream.maxFilesPerTrigger |
(none) |
Integer |
The maximum number of files returned in a single batch. |
read.stream.maxRowsPerTrigger |
(none) |
Long |
The maximum number of rows returned in a single batch. |
read.stream.maxTriggerDelayMs |
(none) |
Long |
The maximum delay between two adjacent batches, which used to create MinRowsReadLimit with read.stream.minRowsPerTrigger together. |
read.stream.minRowsPerTrigger |
(none) |
Long |
The minimum number of rows returned in a single batch, which used to create MinRowsReadLimit with read.stream.maxTriggerDelayMs together. |
requiredSparkConfsCheck.enabled |
true |
Boolean |
Whether to verify SparkSession is initialized with required configurations. |
source.split.target-size-with-column-pruning |
false |
Boolean |
Whether to adjust the target split size based on pruned (projected) columns. If enabled, split size estimation uses only the columns actually being read. |
vector-search.lateral-join.parallelism |
16 |
Integer |
Parallelism used to repartition a single-partition LIMIT input before executing a lateral vector search. |
write.data-evolution.update-conflict-retry.max-attempts |
20 |
Integer |
Maximum attempts for Spark V1 UPDATE on data-evolution tables when concurrent partial-column updates conflict on the same row-id range and update columns. Values less than 2 disable retry. |
write.data-evolution.update-conflict-retry.wait-ms |
10 |
Long |
Wait time in milliseconds between retry attempts for Spark V1 UPDATE on data-evolution tables after row-id range update conflicts. |
write.merge-schema |
false |
Boolean |
If true, evolve the table schema to accept new columns from the incoming data. Existing column types are preserved and incoming values are cast to them; to also widen existing types, enable 'write.merge-schema.type-widening'. |
write.merge-schema.explicit-cast |
false |
Boolean |
Only effective when 'write.merge-schema.type-widening' is true. If true, also allow lossy type changes between compatible types (e.g. BIGINT -> INT, STRING -> DATE). |
write.merge-schema.type-widening |
false |
Boolean |
Only effective when 'write.merge-schema' is true. If true, widen an existing column type when the incoming data has a wider compatible type (e.g. INT -> BIGINT, DECIMAL precision increase). Lossy changes are still rejected unless 'write.merge-schema.explicit-cast' is also true. |
write.use-v2-write |
false |
Boolean |
If true, v2 write will be used. Currently, only HASH_FIXED and BUCKET_UNAWARE bucket modes are supported. Will fall back to v1 write for other bucket modes. Currently, Spark V2 write does not support TableCapability.STREAMING_WRITE. |