apache / hudi

26 Pull requests merged by 13 people

• [HUDI-1884] MergeInto Support Partial Update For COW

  #3154 merged Jul 17, 2021

• [HUDI-1447] DeltaStreamer kafka source supports consuming from specified timestamp

  #2438 merged Jul 17, 2021

• Revert "[HUDI-2087] Support Append only in Flink stream (#3252)"

  #3287 merged Jul 16, 2021

• [HUDI-2185] Remove the default parallelism of index bootstrap and buc…

  #3286 merged Jul 16, 2021

• [HUDI-1633] Make callback return HoodieWriteStat

  #2445 merged Jul 16, 2021

• [MINOR] Allow users to choose ORC as base file format in Spark SQL

  #3279 merged Jul 16, 2021

• [MINOR] Refactor hive sync tool to reduce duplicate code

  #3276 merged Jul 15, 2021

• [HUDI-2153] Fix BucketAssignFunction Context NullPointerException

  #3273 merged Jul 15, 2021

• [HUDI-2029] Implement compression for DiskBasedMap in Spillable Map

  #3128 merged Jul 15, 2021

• [HUDI-2149] Ensure and Audit docs for every configuration class in th…

  #3272 merged Jul 14, 2021

• [Minor] Correct the logs of enable/not-enable async cleaner service.

  #3271 merged Jul 14, 2021

• [HUDI-1828] Update unit tests to support ORC as the base file format

  #3237 merged Jul 14, 2021

• [HUDI-2180] Fix Compile Error For Spark3

  #3274 merged Jul 14, 2021

• [HUDI-2165] Support Transformer for HoodieFlinkStreamer

  #3270 merged Jul 14, 2021

• [HUDI-2171] Add parallelism conf for bootstrap operator

  #3268 merged Jul 14, 2021

• [HUDI-1969] Support reading logs for MOR Hive rt table

  #3033 merged Jul 14, 2021

• [HUDI-2045] Support Read Hoodie As DataSource Table For Flink And DeltaStreamer

  #3120 merged Jul 13, 2021

• [HUDI-2168] Fix for AccessControlException for anonymous user

  #3264 merged Jul 13, 2021

• [MINOR] Fix EXTERNAL_RECORD_AND_SCHEMA_TRANSFORMATION config

  #3250 merged Jul 13, 2021

• [HUDI-2144]Bug-Fix:Offline clustering(HoodieClusteringJob) will cause insert action losing data

  #3240 merged Jul 13, 2021

• [HUDI-2107] Support Read Log Only MOR Table For Spark

  #3193 merged Jul 12, 2021

• [HUDI-1483] Support async clustering for deltastreamer and Spark streaming

  #3142 merged Jul 11, 2021

• [HUDI-2142] Support setting bucket assign parallelism for flink write…

  #3239 merged Jul 10, 2021

• [HUDI-2143] Tweak the default compaction target IO to 500GB when flin…

  #3238 merged Jul 10, 2021

• [HUDI-2087] Support Append only in Flink stream

  #3252 merged Jul 10, 2021

• [MINOR] Fix some wrong assert reasons

  #3248 merged Jul 10, 2021

13 Pull requests opened by 9 people

• [HUDI-1548] Add documentation for schema evolution

  #3257 opened Jul 12, 2021

• [HUDI-2145] Create new bucket when NewFileAssignState filled

  #3258 opened Jul 12, 2021

• [HUDI-2164] Let users build cluster plan and execute this plan at once using HoodieClusteringJob for async clustering

  #3259 opened Jul 12, 2021

• [HUDI-1241][WIP] Generate configuration doc automatically

  #3260 opened Jul 12, 2021

• [HUDI-2169] Remove keyby when write.operation is Insert

  #3265 opened Jul 13, 2021

• [HUDI-2170] Always choose the latest record for HoodieRecordPayload

  #3267 opened Jul 13, 2021

• HUDI-1827 Add ORC Support in Bootstrap Op

  #3275 opened Jul 14, 2021

• [HUDI-2182] Support Compaction Command For Spark Sql

  #3277 opened Jul 15, 2021

• [HUDI-2181] Fix the incorrect doc descriptions for FlinkCreateHandle

  #3281 opened Jul 15, 2021

• [HUDI-2176, 2178, 2179] Virtual keys support to COW table

  #3282 opened Jul 15, 2021

• [HUDI-2184] Support setting hive sync partition extractor class based on flink co…

  #3284 opened Jul 16, 2021

• [HUDI-1771] Propagate CDC format for hoodie

  #3285 opened Jul 16, 2021

• [WIP] test 1771

  #3288 opened Jul 16, 2021

4 Issues closed by 4 people

• Spark Structured Streaming Pipeline on Hudi Source Table Failing

  #3283 closed Jul 16, 2021

• [SUPPORT] NoClassDefFoundError when run HoodieJavaWriteClientExample

  #3241 closed Jul 15, 2021

• [SUPPORT] Deltastreamer Error when reading AVRO Schema with UNION Types

  #3113 closed Jul 14, 2021

• [SUPPORT] No successful commits under path

  #3262 closed Jul 13, 2021

2 Issues opened by 2 people

• [SUPPORT] Use structedstreaming to consume kafka to write to hudi error

  #3280 opened Jul 15, 2021

• [SUPPORT] Upsert data with an identical record key and pre-combine field

  #3266 opened Jul 13, 2021

21 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [HUDI-2161] Adding support to disable meta columns with bulk insert operation

  #3247 commented on Jul 17, 2021 • 51 new comments

• [HUDI-1951] Add bucket hash index, compatible with the hive bucket

  #3173 commented on Jul 16, 2021 • 36 new comments

• [WIP][HUDI-1138] Add timeline-server-based marker file strategy for improving marker-related latency

  #3233 commented on Jul 16, 2021 • 21 new comments

• [HUDI-1860] Add INSERT_OVERWRITE and INSERT_OVERWRITE_TABLE support to DeltaStreamer

  #3184 commented on Jul 17, 2021 • 16 new comments

• [HUDI-2139] MergeInto MOR Table May Result InCorrect Result

  #3230 commented on Jul 16, 2021 • 7 new comments

• [HUDI-1425] Performance loss with the additional hoodieRecords.isEmpty() in HoodieSparkSqlWriter#write

  #2296 commented on Jul 16, 2021 • 4 new comments

• [HUDI-2117] Unpersist the input rdd after the commit is completed to …

  #3207 commented on Jul 14, 2021 • 4 new comments

• [WIP HUDI-53] Adding Record Level Index based on hoodie backed table

  #2487 commented on Jul 14, 2021 • 3 new comments

• [HUDI-1848] Adding support for HMS for running DDL queries in hive-sy…

  #2879 commented on Jul 15, 2021 • 3 new comments

• [HUDI-2063] Add Doc For Spark Sql Integrates With Hudi

  #3140 commented on Jul 14, 2021 • 3 new comments

• [HUDI-1160] Support update partial fields for CoW table

  #2666 commented on Jul 17, 2021 • 2 new comments

• [HUDI-2090] When hudi metadata is enabled, use different users to quer…

  #3183 commented on Jul 11, 2021 • 2 new comments

• [HUDI-2141] Integration flink metric in flink stream

  #3235 commented on Jul 12, 2021 • 2 new comments

• execute insert into ... error:Unexpected exception. This is a bug. Please consider filing an issue.

  #3249 commented on Jul 10, 2021 • 1 new comment

• [SUPPORT] Failed to push metrics to Prometheus pushgateway

  #2960 commented on Jul 16, 2021 • 1 new comment

• [SUPPORT] Sync to Hive using Metastore

  #2688 commented on Jul 16, 2021 • 1 new comment

• [WIP] [DO NOT MERGE] sample code for measuring hfile performance with column ranges

  #2635 commented on Jul 14, 2021 • 1 new comment

• [HUDI-2098] Support hdfs file lock

  #3189 commented on Jul 14, 2021 • 1 new comment

• [HUDI-1129] Improving schema evolution support in hudi

  #2927 commented on Jul 14, 2021 • 0 new comments

• [HUDI-2086] Redo the logical of mor_incremental_view for hive

  #3203 commented on Jul 15, 2021 • 0 new comments

• [HUDI-2137][WIP] Make hudi examples runnable locally

  #3244 commented on Jul 17, 2021 • 0 new comments