mypinkhacker/Apache
43.3
Weak · 17 November 2026
525.7k
lines of production code
Scala
with Java, Python
1
measurement over time
What this system is
Behavioural changes
1 commit (0 fixes) modifying examples/src/main/r/streaming
A change to existing behaviour in examples/src/main/r/streaming — 1 commit, 1 file: “\[SPARK-20015\]\[SPARKR\]\[SS\]\[DOC\]\[EXAMPLE\] Document R Structured Streaming (experimental) in R vignettes and R & SS programming guide, R example”.
examples/src/main/r/streaming · medium confidence · unverified
1 commit (0 fixes) modifying examples/src/main/resources/partitioned\_users.orc
A change to existing behaviour in examples/src/main/resources/partitioned\_users.orc — 1 commit, 3 files: “\[SPARK-27627\]\[SQL\] Make option "pathGlobFilter" as a general option for all file sources”.
_examples/src/main/resources/partitioned\users.orc · medium confidence · unverified
1 commit (0 fixes) modifying examples/src/main/scripts
A change to existing behaviour in examples/src/main/scripts — 1 commit, 1 file: “\[SPARK-27897\]\[EXAMPLES\] Move the get Gpu resources script to a scripts directory”.
examples/src/main/scripts · low confidence · unverified
1 commit (0 fixes) modifying graphx/src/main/java
A change to existing behaviour in graphx/src/main/java — 1 commit, 2 files: “\[SPARK-10576\] \[BUILD\] Move .java files out of src/main/scala”.
graphx/src/main/java · medium confidence · unverified
1 commit (0 fixes) modifying python/test\_support/sql/orc\_partitioned
A change to existing behaviour in python/test\_support/sql/orc\_partitioned — 1 commit, 5 files: “\[SPARK-9100\] \[SQL\] Adds DataFrame reader/writer shortcut methods for ORC”.
_python/test\_support/sql/orc\partitioned · medium confidence · unverified
1 commit (0 fixes) modifying python/test\_support/sql/parquet\_partitioned
A change to existing behaviour in python/test\_support/sql/parquet\_partitioned — 1 commit, 13 files: “\[SPARK-8060\] Improve DataFrame Python test coverage and documentation”.
_python/test\_support/sql/parquet\partitioned · medium confidence · unverified
1 commit (0 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/execution
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/execution — 1 commit, 1 file: “\[SPARK-27322\]\[SQL\] DataSourceV2 table relation”.
sql/catalyst/src/main/scala/org/apache/spark/sql/execution · low confidence · unverified
1 commit (0 fixes) modifying third\_party/asm-3.2, third\_party/hadoop-0.20.0/c++/Linux-amd64-64, third\_party/hadoop-0.20.0/c++/Linux-i386-32
A change to existing behaviour in third\_party/asm-3.2, third\_party/hadoop-0.20.0/c++/Linux-amd64-64, third\_party/hadoop-0.20.0/c++/Linux-i386-32 — 1 commit, 26 files: “Initial commit”.
_third\_party/asm-3.2, third\_party/hadoop-0.20.0/c++/Linux-amd64-64, third\party/hadoop-0.20.0/c++/Linux-i386-32 · medium confidence · unverified
10 commits (0 fixes) modifying third\_party
A change to existing behaviour in third\_party — 10 commits, 176 files: “multi-tracker branch now compiles and runs; but it crashes right before the end. The same problem is seen also in the master branch (in the ChainedStreaming implementation)”; “New nexus.jar”; “Added mesos.jar. Still not working. Major changes required”; and 7 more.
_third\party · medium confidence · unverified
10 commits (1 fix) modifying common/network-common/src/main/java/org/apache/spark/network
A change to existing behaviour in common/network-common/src/main/java/org/apache/spark/network — 10 commits (1 fix), 1 file: “\[SPARK-25692\]\[CORE\] Remove static initialization of worker eventLoop handling chunk fetch requests within TransportContext. This fixes ChunkFetchIntegrationSuite as well”; “\[SPARK-19534\]\[TESTS\] Convert Java tests to use lambdas, Java 8 features”; “\[SPARK-14011\]\[CORE\]\[SQL\] Enable \LineLength\ Java checkstyle rule”; and 7 more.
common/network-common/src/main/java/org/apache/spark/network · medium confidence · unverified
10 commits (2 fixes) modifying python/pyspark/ml/linalg
A change to existing behaviour in python/pyspark/ml/linalg — 10 commits (2 fixs), 1 file: “\[SPARK-24740\]\[PYTHON\]\[ML\] Make PySpark's tests compatible with NumPy 1.14+”; “\[SPARK-16965\]\[MLLIB\]\[PYSPARK\] Fix bound checking for SparseVector”; “\[SPARK-26638\]\[PYSPARK\]\[ML\] Pyspark vector classes always return error for unary negation”; and 7 more.
python/pyspark/ml/linalg · medium confidence · unverified
10 commits (4 fixes) modifying common/unsafe/src/main/java/org/apache/spark/unsafe/array
A change to existing behaviour in common/unsafe/src/main/java/org/apache/spark/unsafe/array — 10 commits (4 fixs), 2 files: “Revert \[SPARK-10399\] \[SPARK-23879\] \[SPARK-23762\] \[SPARK-25317\]”; “\[SPARK-10399\]\[CORE\]\[SQL\] Introduce multiple MemoryBlocks to choose several types of memory block”; “\[SPARK-22033\]\[CORE\] BufferHolder, other size checks should account for the specific VM array size limitations”; and 7 more.
common/unsafe/src/main/java/org/apache/spark/unsafe/array · medium confidence · unverified
10 commits (4 fixes) modifying examples/src/main/java/org/apache/spark/examples/sql/streaming
A change to existing behaviour in examples/src/main/java/org/apache/spark/examples/sql/streaming — 10 commits (4 fixs), 4 files: “\[SPARK-20523\]\[BUILD\] Clean up build warnings for 2.2.0 release”; “\[SPARK-16114\]\[SQL\] structured streaming network word count examples”; “\[MINOR\]\[BUILD\] Fix Java CheckStyle Error”; and 7 more.
examples/src/main/java/org/apache/spark/examples/sql/streaming · medium confidence · unverified
102 commits (8 fixes) modifying sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions
A change to existing behaviour in sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions — 102 commits (8 fixs), 9 files: “\[SPARK-9358\]\[SQL\] Code generation for UnsafeRow joiner”; “\[SPARK-10917\] \[SQL\] improve performance of complex type in columnar cache”; “\[SPARK-11313\]\[SQL\] implement cogroup on DataSets (support 2 datasets)”; and 99 more (refs \#7456).
sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions · medium confidence · unverified
103 commits (20 fixes) modifying launcher/src/main
A change to existing behaviour in launcher/src/main — 103 commits (20 fixs), 20 files: “\[SPARK-14702\] Make environment of SparkLauncher launched process more configurable”; “\[SPARK-8673\] \[LAUNCHER\] API and infrastructure for communicating with child apps”; “\[SPARK-12384\] Enables spark-clients to set the min(-Xms) and max(\*.memory config) j…”; and 98 more (refs \#13286, \#13280).
launcher/src/main · medium confidence · unverified
11 commits (0 fixes) modifying spark-examples\_2.12
A change to existing behaviour in spark-examples\_2.12 — 11 commits, 1 file: “\[SPARK-15031\]\[EXAMPLE\] Use SparkSession in examples”; “\[java8API\] SPARK-964 Investigate the potential for using JDK 8 lambda expressions for the Java/Scala APIs”; “SPARK-1565, update examples to be used with spark-submit script”; and 8 more (refs \#318).
_spark-examples\2.12 · medium confidence · unverified
11 commits (0 fixes) modifying sql/catalyst/src/main/java/org/apache/spark/sql/types
A change to existing behaviour in sql/catalyst/src/main/java/org/apache/spark/sql/types — 11 commits, 2 files: “\[SPARK-8753\]\[SQL\] Create an IntervalType data type”; “\[SQL\]\[minor\] Put DataTypes.java in java dir”; “\[SPARK-17864\]\[SQL\] Mark data type APIs as stable (not DeveloperApi)”; and 8 more.
sql/catalyst/src/main/java/org/apache/spark/sql/types · medium confidence · unverified
11 commits (2 fixes) modifying sql/catalyst/src/main/java/org/apache/spark/sql/streaming
A change to existing behaviour in sql/catalyst/src/main/java/org/apache/spark/sql/streaming — 11 commits (2 fixs), 2 files: “\[SPARK-20523\]\[BUILD\] Clean up build warnings for 2.2.0 release”; “\[SPARK-17864\]\[SQL\] Mark data type APIs as stable (not DeveloperApi)”; “\[SPARK-26026\]\[BUILD\] Published Scaladoc jars missing from Maven Central”; and 8 more.
sql/catalyst/src/main/java/org/apache/spark/sql/streaming · medium confidence · unverified
11 commits (3 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/errors
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/errors — 11 commits (3 fixs), 1 file: “\[SPARK-14255\]\[SQL\] Streaming Aggregation”; “Unified package definition format in Spark SQL”; “\[SPARK-12855\]\[SQL\] Remove parser dialect developer API”; and 7 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/errors · medium confidence · unverified
115 commits (19 fixes) modifying python
A change to existing behaviour in python — 115 commits (19 fixs), 13 files: “\[SPARK-4897\] \[PySpark\] Python 3 support”; “\[SPARK-5554\] \[SQL\] \[PySpark\] add more tests for DataFrame Python API”; “\[SPARK-5469\] restructure pyspark.sql into multiple files”; and 112 more.
python · medium confidence · unverified
119 commits (18 fixes) modifying graphx/src/main/scala/org/apache/spark/graphx
A change to existing behaviour in graphx/src/main/scala/org/apache/spark/graphx — 119 commits (18 fixs), 12 files: “\[HOTFIX\] \[SPARK-3400\] Revert 9b225ac "fix GraphX EdgeRDD zipPartitions"”; “SPARK-1488. Resolve scalac feature warnings during build”; “Make EdgeDirection val instead of case object for Java compat”; and 116 more (refs \#567, \#557).
graphx/src/main/scala/org/apache/spark/graphx · medium confidence · unverified
12 commits (1 fix) modifying spark-network-shuffle\_2.12
A change to existing behaviour in spark-network-shuffle\_2.12 — 12 commits (1 fix), 13 files: “\[SPARK-19529\] TransportClientFactory.createClient() shouldn't call awaitUninterruptibly()”; “\[SPARK-12583\]\[MESOS\] Mesos shuffle service: Don't delete shuffle files before application has stopped”; “\[SPARK-17332\]\[CORE\] Make Java Loggers static members”; and 9 more.
_spark-network-shuffle\2.12 · medium confidence · unverified
12 commits (2 fixes) modifying R/pkg/inst/profile
A change to existing behaviour in R/pkg/inst/profile — 12 commits (2 fixs), 2 files: “\[MINOR\]\[R\] Fix indents of sparkR welcome message to be consistent with pyspark and spark-shell”; “\[SPARK-5654\] Integrate SparkR”; “\[SPARK-8063\] \[SPARKR\] Spark master URL conflict between MASTER env variable and --master command line option”; and 9 more.
R/pkg/inst/profile · medium confidence · unverified
12 commits (2 fixes) modifying spark-catalyst\_2.12
A change to existing behaviour in spark-catalyst\_2.12 — 12 commits (2 fixs), 4 files: “\[SPARK-16525\] \[SQL\] Enable Row Based HashMap in HashAggregateExec”; “Revert \[SPARK-10399\] \[SPARK-23879\] \[SPARK-23762\] \[SPARK-25317\]”; “\[MINOR\] Fix newly added java-lint errors”; and 9 more.
_spark-catalyst\2.12 · medium confidence · unverified
12 commits (2 fixes) modifying spark-network-common\_2.12
A change to existing behaviour in spark-network-common\_2.12 — 12 commits (2 fixs), 10 files: “\[SPARK-20027\]\[DOCS\] Compilation fix in java docs”; “\[SPARK-14011\]\[CORE\]\[SQL\] Enable \LineLength\ Java checkstyle rule”; “\[SPARK-25535\]\[CORE\] Work around bad error handling in commons-crypto”; and 9 more.
_spark-network-common\2.12 · medium confidence · unverified
122 commits (15 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical — 122 commits (15 fixs), 9 files: “\[SPARK-20392\]\[SQL\] Set barrier to prevent re-entering a tree”; “\[DOCS\] Docs-only improvements”; “\[SPARK-12441\]\[SQL\] Fixing missingInput in Generate/MapPartitions/AppendColumns/MapGroups/CoGroup”; and 117 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical · medium confidence · unverified
128 commits (31 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/types
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/types — 128 commits (31 fixs), 10 files: “\[SPARK-8848\] \[SQL\] Refactors Parquet write path to follow parquet-format”; “\[MINOR\] Remove unnecessary gets when getting a value from map”; “\[SPARK-24586\]\[SQL\] Upcast should not allow casting from string to other types”; and 125 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/types · medium confidence · unverified
13 commits (4 fixes) modifying spark-core\_2.12
A change to existing behaviour in spark-core\_2.12 — 13 commits (4 fixs), 3 files: “Revert \[SPARK-10399\] \[SPARK-23879\] \[SPARK-23762\] \[SPARK-25317\]”; “\[SPARK-12885\]\[MINOR\] Rename 3 fields in ShuffleWriteMetrics”; “\[SPARK-15391\] \[SQL\] manage the temporary memory of timsort”; and 10 more.
_spark-core\2.12 · medium confidence · unverified
14 commits (1 fix) modifying common/network-common/src/main/java/org/apache/spark/network/sasl
A change to existing behaviour in common/network-common/src/main/java/org/apache/spark/network/sasl — 14 commits (1 fix), 8 files: “\[SPARK-24801\]\[CORE\] Avoid memory waste by empty byte\[\] arrays in SaslEncryption$EncryptedMessage”; “\[SPARK-6237\]\[NETWORK\] Network-layer changes to allow stream upload”; “\[SPARK-13823\]\[CORE\]\[STREAMING\]\[SQL\] Always specify Charset in String \<-\> byte\[\] conversions (and remaining Coverity items)”; and 11 more.
common/network-common/src/main/java/org/apache/spark/network/sasl · medium confidence · unverified
14 commits (2 fixes) modifying python/pyspark/mllib/stat
A change to existing behaviour in python/pyspark/mllib/stat — 14 commits (2 fixs), 5 files: “\[SPARK-14812\]\[ML\]\[MLLIB\]\[PYTHON\] Experimental, DeveloperApi annotation audit for ML”; “\[SPARK-24740\]\[PYTHON\]\[ML\] Make PySpark's tests compatible with NumPy 1.14+”; “\[SPARK-4897\] \[PySpark\] Python 3 support”; and 11 more.
python/pyspark/mllib/stat · medium confidence · unverified
14 commits (3 fixes) modifying common/kvstore/src/main
A change to existing behaviour in common/kvstore/src/main — 14 commits (3 fixs), 12 files: “\[SPARK-20643\]\[CORE\] Add listener implementation to collect app state”; “\[SPARK-25408\] Move to more ideomatic Java8”; “\[SPARK-20657\]\[CORE\] Speed up rendering of the stages page”; and 11 more.
common/kvstore/src/main · medium confidence · unverified
14 commits (3 fixes) modifying examples/src/main/r/ml
A change to existing behaviour in examples/src/main/r/ml — 14 commits (3 fixs), 20 files: “\[SPARK-19827\]\[R\] spark.ml R API for PIC”; “\[SPARK-20258\]\[DOC\]\[SPARKR\] Fix SparkR logistic regression example in programming guide (did not converge)”; “\[SPARK-19158\]\[SPARKR\]\[EXAMPLES\] Fix ml.R example fails due to lack of e1071 package”; and 11 more.
examples/src/main/r/ml · medium confidence · unverified
14 commits (5 fixes) modifying examples/src/main/scala/org/apache/spark/examples/streaming/clickstream
A change to existing behaviour in examples/src/main/scala/org/apache/spark/examples/streaming/clickstream — 14 commits (5 fixs), 2 files: “\[SPARK-7929\] Remove Bagel examples & whitespace fix for examples”; “\[SPARK-12692\]\[BUILD\]\[STREAMING\] Scala style: Fix the style violation (Space before "," or ":")”; “\[MINOR\]\[DOCS\] Use multi-line JavaDoc comments in Scala code”; and 11 more.
examples/src/main/scala/org/apache/spark/examples/streaming/clickstream · medium confidence · unverified
145 commits (10 fixes) modifying sql/catalyst/src/main/antlr4
A change to existing behaviour in sql/catalyst/src/main/antlr4 — 145 commits (10 fixs), 1 file: “\[SPARK-17013\]\[SQL\] Parse negative numeric literals”; “\[SPARK-15431\]\[SQL\] Support LIST FILE(s)\|JAR(s) command natively”; “\[SPARK-17910\]\[SQL\] Allow users to update the comment of a column”; and 141 more.
sql/catalyst/src/main/antlr4 · medium confidence · unverified
145 commits (12 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser — 145 commits (12 fixs), 4 files: “\[DOCS\] Docs-only improvements”; “\[SPARK-23908\]\[SQL\] Add transform function”; “\[SPARK-21164\]\[SQL\] Remove isTableSample from Sample and isGenerated from Alias and AttributeReference”; and 141 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser · medium confidence · unverified
150 commits (14 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen — 150 commits (14 fixs), 9 files: “\[MINOR\]\[DOCS\] Fix all typos in markdown files of \doc\ and similar patterns in other comments”; “\[HOT-FIX\] Recover some deprecations for 2.10 compatibility”; “\[SPARK-14393\]\[SQL\] values generated by non-deterministic functions shouldn't change after coalesce or union”; and 147 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen · medium confidence · unverified
152 commits (17 fixes) modifying pyspark
A change to existing behaviour in pyspark — 152 commits (17 fixs), 3 files: “\[SPARK-14812\]\[ML\]\[MLLIB\]\[PYTHON\] Experimental, DeveloperApi annotation audit for ML”; “\[SPARK-23643\]\[CORE\]\[SQL\]\[ML\] Shrinking the buffer in hashSeed up to size of the seed parameter”; “\[SPARK-21088\]\[ML\] CrossValidator, TrainValidationSplit support collect all models when fitting: Python API”; and 149 more.
pyspark · medium confidence · unverified
156 commits (18 fixes) modifying dev/deps
A change to existing behaviour in dev/deps — 156 commits (18 fixs), 2 files: “\[SPARK-22128\]\[CORE\] Update paranamer to 2.8 to avoid BytecodeReadingParanamer ArrayIndexOutOfBoundsException with Scala 2.12 + Java 8 lambda”; “\[SPARK-20978\]\[SQL\] Bump up Univocity version to 2.5.4”; “\[SPARK-16751\] Upgrade derby to 10.12.1.1”; and 148 more.
dev/deps · medium confidence · unverified
16 commits (0 fixes) modifying common/network-common/src/main/java/org/apache/spark/network/protocol
A change to existing behaviour in common/network-common/src/main/java/org/apache/spark/network/protocol — 16 commits, 17 files: “\[SPARK-24938\]\[CORE\] Prevent Netty from using onheap memory for headers without regard for configuration”; “\[SPARK-6237\]\[NETWORK\] Network-layer changes to allow stream upload”; “\[SPARK-13823\]\[CORE\]\[STREAMING\]\[SQL\] Always specify Charset in String \<-\> byte\[\] conversions (and remaining Coverity items)”; and 13 more.
common/network-common/src/main/java/org/apache/spark/network/protocol · medium confidence · unverified
167 commits (15 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog — 167 commits (15 fixs), 7 files: “\[SPARK-15034\]\[SPARK-15035\]\[SPARK-15036\]\[SQL\] Use spark.sql.warehouse.dir as the warehouse location”; “\[MINOR\]\[SQL\] Add a debug log when a SQL text is used for a view”; “\[SPARK-20349\]\[SQL\] ListFunctions returns duplicate functions after using persistent functions”; and 164 more (refs \#2).
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog · medium confidence · unverified
1688 commits (228 fixes) modifying sql/hive
A change to existing behaviour in sql/hive — 1688 commits (228 fixs), 9404 files: “\[SPARK-19219\]\[SQL\] Fix Parquet log output defaults”; “\[SPARK-21786\]\[SQL\] The 'spark.sql.parquet.compression.codec' and 'spark.sql.orc.compression.codec' configuration doesn't take effect on hive table writing”; “\[SPARK-17013\]\[SQL\] Parse negative numeric literals”; and 1671 more (refs \#12178, \#2, \#1071, \#17311, \#6882, …).
sql/hive · medium confidence · unverified
1798 commits (228 fixes) modifying mllib
A change to existing behaviour in mllib — 1798 commits (228 fixs), 544 files: “\[SPARK-25765\]\[ML\] Add training cost to BisectingKMeans summary”; “\[SPARK-25908\]\[CORE\]\[SQL\] Remove old deprecated items in Spark 3”; “\[MINOR\]\[MLLIB\] fix mllib compile warnings”; and 1793 more (refs \#12178, \#566, \#528, \#12625, \#8271, …).
mllib · medium confidence · unverified
18 commits (1 fix) modifying common/network-yarn/src/main/java/org/apache/spark/network/yarn
A change to existing behaviour in common/network-yarn/src/main/java/org/apache/spark/network/yarn — 18 commits (1 fix), 1 file: “\[SPARK-20523\]\[BUILD\] Clean up build warnings for 2.2.0 release”; “\[SPARK-21494\]\[NETWORK\] Use correct app id when authenticating to external service”; “\[SPARK-17321\]\[YARN\] Avoid writing shuffle metadata to disk if NM recovery is disabled”; and 15 more.
common/network-yarn/src/main/java/org/apache/spark/network/yarn · medium confidence · unverified
18 commits (2 fixes) modifying examples/src/main/java/org/apache/spark/examples/sql
A change to existing behaviour in examples/src/main/java/org/apache/spark/examples/sql — 18 commits (2 fixs), 4 files: “\[SPARK-19550\]\[BUILD\]\[CORE\]\[WIP\] Remove Java 7 support”; “\[SPARK-16734\]\[EXAMPLES\]\[SQL\] Revise examples of all language bindings”; “\[SPARK-16303\]\[DOCS\]\[EXAMPLES\] Minor Scala/Java example update”; and 15 more.
examples/src/main/java/org/apache/spark/examples/sql · medium confidence · unverified
184 commits (37 fixes) modifying bin
A change to existing behaviour in bin — 184 commits (37 fixs), 27 files: “SPARK-1286: Make usage of spark-env.sh idempotent”; “SPARK-1404: Always upgrade spark-env.sh vars to environment vars”; “SPARK-1099: Introduce local\[\*\] mode to infer number of cores”; and 178 more.
bin · medium confidence · unverified
19 commits (3 fixes) modifying R/pkg/inst/worker
A change to existing behaviour in R/pkg/inst/worker — 19 commits (3 fixs), 2 files: “\[SPARK-22063\]\[R\] Fixes lint check failures in R by latest commit sha1 ID of lint-r”; “\[SPARK-21093\]\[R\] Terminate R's worker processes in the parent of R's daemon to prevent a leak”; “\[SPARK-17919\] Make timeout to RBackend configurable in SparkR”; and 15 more.
R/pkg/inst/worker · medium confidence · unverified
19 commits (3 fixes) modifying examples/src/main/r
A change to existing behaviour in examples/src/main/r — 19 commits (3 fixs), 3 files: “\[SPARK-12146\]\[SPARKR\] SparkR jsonFile should support multiple input files”; “\[SPARK-16734\]\[EXAMPLES\]\[SQL\] Revise examples of all language bindings”; “\[SPARK-8124\] \[SPARKR\] Created more examples on SparkR DataFrames”; and 16 more.
examples/src/main/r · medium confidence · unverified
190 commits (36 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util — 190 commits (36 fixs), 23 files: “\[SPARK-15613\] \[SQL\] Fix incorrect days to millis conversion due to Daylight Saving Time”; “\[SQL\] address comments for to\_date/trunc”; “\[SPARK-9152\]\[SQL\] Implement code generation for Like and RLike”; and 187 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util · medium confidence · unverified
197 commits (9 fixes) modifying R/pkg
A change to existing behaviour in R/pkg — 197 commits (9 fixs), 7 files: “\[SPARK-25908\]\[CORE\]\[SQL\] Remove old deprecated items in Spark 3”; “\[SPARK-8495\] \[SPARKR\] Add a \.lintr\ file to validate the SparkR files and the \lint-r\ script”; “\[SPARK-6824\] Fill the docs for DataFrame API in SparkR”; and 193 more.
R/pkg · medium confidence · unverified
2 commits (0 fixes) modifying common/unsafe/src/main/java/org/apache/spark/unsafe/bitset
A change to existing behaviour in common/unsafe/src/main/java/org/apache/spark/unsafe/bitset — 2 commits, 1 file: “\[SPARK-14011\]\[CORE\]\[SQL\] Enable \LineLength\ Java checkstyle rule”; “\[SPARK-13548\]\[BUILD\] Move tags and unsafe modules into common”.
common/unsafe/src/main/java/org/apache/spark/unsafe/bitset · medium confidence · unverified
2 commits (0 fixes) modifying data/mllib/als
A change to existing behaviour in data/mllib/als — 2 commits, 3 files: “\[SPARK-5867\] \[SPARK-5892\] \[doc\] \[ml\] \[mllib\] Doc cleanups for 1.3 release”; “SPARK-2363. Clean MLlib's sample data files”.
data/mllib/als · medium confidence · unverified
2 commits (0 fixes) modifying external/kinesis-asl/src/main/java/org/apache/spark/streaming
A change to existing behaviour in external/kinesis-asl/src/main/java/org/apache/spark/streaming — 2 commits, 1 file: “\[SPARK-20168\]\[DSTREAM\] Add changes to use kinesis fetches from specific timestamp”; “\[MINOR\]\[BUILD\] Fix Java linter errors”.
external/kinesis-asl/src/main/java/org/apache/spark/streaming · medium confidence · unverified
2 commits (0 fixes) modifying external/kinesis-asl/src/main/python
A change to existing behaviour in external/kinesis-asl/src/main/python — 2 commits, 1 file: “\[SPARK-13595\]\[BUILD\] Move docker, extras modules into external”; “Fix typo in Python docstring kinesis example”.
external/kinesis-asl/src/main/python · medium confidence · unverified
2 commits (0 fixes) modifying sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2/expressions
A change to existing behaviour in sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2/expressions — 2 commits, 5 files: “\[SPARK-24252\]\[SQL\] Add TableCatalog API”; “\[SPARK-27181\]\[SQL\] Add public transform API”.
sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2/expressions · medium confidence · unverified
2 commits (0 fixes) modifying sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2/utils
A change to existing behaviour in sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2/utils — 2 commits, 1 file: “\[SPARK-27322\]\[SQL\] DataSourceV2 table relation”; “\[SPARK-27964\]\[SQL\] Move v2 catalog update methods to CatalogV2Util”.
sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2/utils · medium confidence · unverified
2 commits (0 fixes) modifying sql/catalyst/src/main/java/org/apache/spark/sql/vectorized
A change to existing behaviour in sql/catalyst/src/main/java/org/apache/spark/sql/vectorized — 2 commits, 5 files: “\[SPARK-27521\]\[SQL\] Move data source v2 to catalyst module”; “\[SPARK-27945\]\[SQL\] Minimal changes to support columnar processing”.
sql/catalyst/src/main/java/org/apache/spark/sql/vectorized · medium confidence · unverified
2 commits (0 fixes) modifying src/java
A change to existing behaviour in src/java — 2 commits, 3 files: “Replacing the native lzf compression code with the ning open-source compress-lzf library. (Apache 2.0 liscense)”; “Initial commit”.
src/java · medium confidence · unverified
2 commits (1 fix) modifying python/pyspark/sql/avro
A change to existing behaviour in python/pyspark/sql/avro — 2 commits (1 fix), 2 files: “\[SPARK-26856\]\[PYSPARK\] Python support for from\_avro and to\_avro APIs”; “\[SPARK-26856\]\[PYSPARK\]\[FOLLOWUP\] Fix UT failure due to wrong patterns for Kinesis assembly”.
python/pyspark/sql/avro · medium confidence · unverified
20 commits (2 fixes) modifying examples/src/main/python/streaming
A change to existing behaviour in examples/src/main/python/streaming — 20 commits (2 fixs), 7 files: “\[SPARK-4897\] \[PySpark\] Python 3 support”; “\[SPARK-3952\] \[Streaming\] \[PySpark\] add Python examples in Streaming Programming Guide”; “\[SPARK-9812\] \[STREAMING\] Fix Python 3 compatibility issue in PySpark Streaming and some docs”; and 17 more.
examples/src/main/python/streaming · medium confidence · unverified
20 commits (7 fixes) modifying spark-unsafe\_2.12
A change to existing behaviour in spark-unsafe\_2.12 — 20 commits (7 fixs), 3 files: “Revert \[SPARK-10399\] \[SPARK-23879\] \[SPARK-23762\] \[SPARK-25317\]”; “\[SPARK-19550\]\[BUILD\]\[CORE\]\[WIP\] Remove Java 7 support”; “\[SPARK-22349\] In on-heap mode, when allocating memory from pool,we should fill memory with \MEMORY\_DEBUG\_FILL\_CLEAN\_VALUE\”; and 17 more.
_spark-unsafe\2.12 · medium confidence · unverified
21 commits (3 fixes) modifying common/sketch
A change to existing behaviour in common/sketch — 21 commits (3 fixs), 12 files: “\[MINOR\]\[DOCS\] Fix all typos in markdown files of \doc\ and similar patterns in other comments”; “\[SPARK-12936\]\[SQL\] Initial bloom filter implementation”; “\[SPARK-25408\] Move to more ideomatic Java8”; and 18 more.
common/sketch · medium confidence · unverified
21 commits (6 fixes) modifying tools
A change to existing behaviour in tools — 21 commits (6 fixs), 1 file: “\[HOT-FIX\] Recover some deprecations for 2.10 compatibility”; “\[SPARK-8962\] Add Scalastyle rule to ban direct use of Class.forName; fix existing uses”; “\[SPARK-13920\]\[BUILD\] MIMA checks should apply to @Experimental and @DeveloperAPI APIs”; and 18 more.
tools · medium confidence · unverified
22 commits (2 fixes) modifying sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen
A change to existing behaviour in sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen — 22 commits (2 fixs), 4 files: “\[SPARK-26021\]\[SQL\] replace minus zero with zero in Platform.putDouble/Float”; “\[SPARK-11243\]\[SQL\] output UnsafeRow from columnar cache”; “\[SPARK-23864\]\[SQL\] Add unsafe object writing to UnsafeWriter”; and 19 more.
sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen · medium confidence · unverified
22 commits (3 fixes) modifying examples/src/main/scala/org/apache/spark/examples/graphx
A change to existing behaviour in examples/src/main/scala/org/apache/spark/examples/graphx — 22 commits (3 fixs), 9 files: “Minor - Fix trivial compilation warnings”; “\[GraphX\] Modify option name according to example doc in SynthBenchmark”; “\[SPARK-16345\]\[DOCUMENTATION\]\[EXAMPLES\]\[GRAPHX\] Extract graphx programming guide example snippets from source files instead of hard code them”; and 19 more (refs \#720).
examples/src/main/scala/org/apache/spark/examples/graphx · medium confidence · unverified
22 commits (3 fixes) modifying external/kinesis-asl/src/main/scala/org/apache/spark/streaming
A change to existing behaviour in external/kinesis-asl/src/main/scala/org/apache/spark/streaming — 22 commits (3 fixs), 6 files: “\[SPARK-20168\]\[DSTREAM\] Add changes to use kinesis fetches from specific timestamp”; “\[SPARK-13823\]\[CORE\]\[STREAMING\]\[SQL\] Always specify Charset in String \<-\> byte\[\] conversions (and remaining Coverity items)”; “\[SPARK-13595\]\[BUILD\] Move docker, extras modules into external”; and 19 more.
external/kinesis-asl/src/main/scala/org/apache/spark/streaming · medium confidence · unverified
22 commits (3 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv — 22 commits (3 fixs), 6 files: “\[SPARK-26151\]\[SQL\] Return partial results for bad CSV records”; “\[SPARK-27199\]\[SQL\] Replace TimeZone by ZoneId in TimestampFormatter API”; “\[SPARK-25493\]\[SQL\] Use auto-detection for CRLF in CSV datasource multiline mode”; and 19 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv · medium confidence · unverified
22 commits (4 fixes) modifying resource-managers/kubernetes/docker/src/main/dockerfiles/spark
A change to existing behaviour in resource-managers/kubernetes/docker/src/main/dockerfiles/spark — 22 commits (4 fixs), 2 files: “\[SPARK-22994\]\[K8S\] Use a single image for all Spark containers”; “\[SPARK-26015\]\[K8S\] Set a default UID for Spark on K8S Images”; “\[SPARK-24534\]\[K8S\] Bypass non spark-on-k8s commands”; and 19 more.
resource-managers/kubernetes/docker/src/main/dockerfiles/spark · medium confidence · unverified
22 commits (5 fixes) modifying common/unsafe/src/main/java/org/apache/spark/unsafe
A change to existing behaviour in common/unsafe/src/main/java/org/apache/spark/unsafe — 22 commits (5 fixs), 4 files: “\[SPARK-17127\] Make unaligned access in unsafe available for AArch64”; “Revert \[SPARK-10399\] \[SPARK-23879\] \[SPARK-23762\] \[SPARK-25317\]”; “\[SPARK-19550\]\[BUILD\]\[CORE\]\[WIP\] Remove Java 7 support”; and 19 more.
common/unsafe/src/main/java/org/apache/spark/unsafe · medium confidence · unverified
223 commits (30 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst — 223 commits (30 fixs), 8 files: “\[SPARK-22472\]\[SQL\] add null check for top-level primitive values”; “\[HOT-FIX\] Recover some deprecations for 2.10 compatibility”; “\[SPARK-23936\]\[SQL\] Implement map\_concat”; and 218 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst · medium confidence · unverified
23 commits (6 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation — 23 commits (6 fixs), 5 files: “\[SPARK-19271\]\[SQL\] Change non-cbo estimation of aggregate”; “\[SPARK-19020\]\[SQL\] Cardinality estimation of aggregate operator”; “Revert "Revert "\[SPARK-27351\]\[SQL\] Wrong outputRows estimation after AggregateEstimation wit…""”; and 20 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation · medium confidence · unverified
230 commits (36 fixes) modifying python/pyspark/mllib
A change to existing behaviour in python/pyspark/mllib — 230 commits (36 fixs), 12 files: “\[SPARK-25908\]\[CORE\]\[SQL\] Remove old deprecated items in Spark 3”; “\[SPARK-14900\]\[ML\]\[PYSPARK\] Add accuracy and deprecate precison,recall,f1”; “\[SPARK-14812\]\[ML\]\[MLLIB\]\[PYTHON\] Experimental, DeveloperApi annotation audit for ML”; and 225 more.
python/pyspark/mllib · medium confidence · unverified
233 commits (19 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/internal
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/internal — 233 commits (19 fixs), 3 files: “Revert "\[SPARK-24250\]\[SQL\] support accessing SQLConf inside tasks"”; “\[SPARK-27128\]\[SQL\] Migrate JSON to File Data Source V2”; “\[SPARK-22181\]\[SQL\] Adds ReplaceExceptWithFilter rule”; and 227 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/internal · medium confidence · unverified
24 commits (2 fixes) modifying examples/src/main/python/sql
A change to existing behaviour in examples/src/main/python/sql — 24 commits (2 fixs), 4 files: “\[MINOR\]\[DOCS\] Fixes two problems in the SQL programing guide page”; “\[SPARK-22221\]\[DOCS\] Adding User Documentation for Arrow”; “\[SPARK-16734\]\[EXAMPLES\]\[SQL\] Revise examples of all language bindings”; and 21 more.
examples/src/main/python/sql · medium confidence · unverified
24 commits (3 fixes) modifying sql
A change to existing behaviour in sql — 24 commits (3 fixs), 4 files: “\[SQL\]\[Doc\] Keep Spark SQL README.md up to date”; “\[SPARK-11807\] Remove support for Hadoop \< 2.2”; “\[MINOR\]\[SQL\]\[DOCS\] Update \sql/README.md\ and remove some unused imports in \sql\ module”; and 21 more.
sql · medium confidence · unverified
25 commits (6 fixes) modifying common/network-common/src/main/java/org/apache/spark/network/client
A change to existing behaviour in common/network-common/src/main/java/org/apache/spark/network/client — 25 commits (6 fixs), 10 files: “\[SPARK-27073\]\[CORE\] Fix a race condition when handling of IdleStateEvent”; “\[SPARK-27989\]\[CORE\] Added retries on the connection to the driver for k8s”; “\[SPARK-19550\]\[BUILD\]\[CORE\]\[WIP\] Remove Java 7 support”; and 22 more.
common/network-common/src/main/java/org/apache/spark/network/client · medium confidence · unverified
256 commits (58 fixes) modifying sql/hive-thriftserver
A change to existing behaviour in sql/hive-thriftserver — 256 commits (58 fixs), 51 files: “\[SPARK-15431\]\[SQL\] Support LIST FILE(s)\|JAR(s) command natively”; “\[SPARK-15431\]\[SQL\]\[BRANCH-2.0-TEST\] rework the clisuite test cases”; “\[MINOR\]\[DOCS\] Fix all typos in markdown files of \doc\ and similar patterns in other comments”; and 249 more (refs \#2, \#6493).
sql/hive-thriftserver · medium confidence · unverified
27 commits (0 fixes) modifying spark-parent\_2.12
A change to existing behaviour in spark-parent\_2.12 — 27 commits, 1 file: “refactor: Improve performance of the main algorithm on main.kt”; “style: Format code according to style guide on main.kt”; “chore: Update project dependencies on main.kt”; and 6 more.
_spark-parent\2.12 · medium confidence · unverified
27 commits (4 fixes) modifying external/kafka-0-10/src/main
A change to existing behaviour in external/kafka-0-10/src/main — 27 commits (4 fixs), 8 files: “\[SPARK-19550\]\[BUILD\]\[CORE\]\[WIP\] Remove Java 7 support”; “\[SPARK-24713\] AppMatser of spark streaming kafka OOM if there are hund…”; “\[SPARK-25233\]\[STREAMING\] Give the user the option of specifying a minimum message per partition per batch when using kafka direct API with backpressure”; and 24 more.
external/kafka-0-10/src/main · medium confidence · unverified
27 commits (5 fixes) modifying mllib-local/src/main/scala/org/apache/spark/ml/linalg
A change to existing behaviour in mllib-local/src/main/scala/org/apache/spark/ml/linalg — 27 commits (5 fixs), 3 files: “\[SPARK-16965\]\[MLLIB\]\[PYSPARK\] Fix bound checking for SparseVector”; “\[SPARK-14653\]\[ML\] Remove json4s from mllib-local”; “\[SPARK-22289\]\[ML\] Add JSON support for Matrix parameters (LR with coefficients bound)”; and 23 more.
mllib-local/src/main/scala/org/apache/spark/ml/linalg · medium confidence · unverified
29 commits (2 fixes) modifying examples/src/main/java/org/apache/spark/examples
A change to existing behaviour in examples/src/main/java/org/apache/spark/examples — 29 commits (2 fixs), 6 files: “\[SPARK-15031\]\[EXAMPLE\] Use SparkSession in examples”; “\[SPARK-16421\]\[EXAMPLES\]\[ML\] Improve ML Example Outputs”; “\[java8API\] SPARK-964 Investigate the potential for using JDK 8 lambda expressions for the Java/Scala APIs”; and 26 more (refs \#318).
examples/src/main/java/org/apache/spark/examples · medium confidence · unverified
29 commits (5 fixes) modifying common/unsafe/src/main/java/org/apache/spark/unsafe/types
A change to existing behaviour in common/unsafe/src/main/java/org/apache/spark/unsafe/types — 29 commits (5 fixs), 2 files: “Revert \[SPARK-10399\] \[SPARK-23879\] \[SPARK-23762\] \[SPARK-25317\]”; “\[SPARK-20027\]\[DOCS\] Compilation fix in java docs”; “\[SPARK-22130\]\[CORE\] UTF8String.trim() scans " " twice”; and 26 more.
common/unsafe/src/main/java/org/apache/spark/unsafe/types · medium confidence · unverified
3 commits (0 fixes) modifying common/network-yarn/src/main/java/org/apache/spark/network/yarn/util
A change to existing behaviour in common/network-yarn/src/main/java/org/apache/spark/network/yarn/util — 3 commits, 1 file: “\[SPARK-19991\]\[CORE\]\[YARN\] FileSegmentManagedBuffer performance improvement”; “\[SPARK-13529\]\[BUILD\] Move network/\* modules into common/network-\*”; “\[SPARK-18773\]\[CORE\] Make commons-crypto config translation consistent”.
common/network-yarn/src/main/java/org/apache/spark/network/yarn/util · medium confidence · unverified
3 commits (0 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalog
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalog — 3 commits, 3 files: “\[SPARK-24252\]\[SQL\] Add TableCatalog API”; “\[SPARK-26946\]\[SQL\] Identifiers for multi-catalog”; “\[SPARK-26946\]\[SQL\]\[FOLLOWUP\] Require lookup function”.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalog · medium confidence · unverified
3 commits (0 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/sql
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/sql — 3 commits, 1 file: “\[SPARK-27108\]\[SQL\] Add parsed SQL plans for create, CTAS”; “\[SPARK-27857\]\[SQL\] Move ALTER TABLE parsing into Catalyst”; “\[SPARK-27947\]\[SQL\] Enhance redactOptions to accept any Map type”.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/sql · medium confidence · unverified
3 commits (0 fixes) modifying sql/core/v2.3.5
A change to existing behaviour in sql/core/v2.3.5 — 3 commits, 4 files: “\[SPARK-27737\]\[SQL\] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile”; “\[SPARK-27105\]\[SQL\] Optimize away exponential complexity in ORC predicate conversion”; “\[SPARK-28108\]\[SQL\]\[test-hadoop3.2\] Simplify OrcFilters”.
sql/core/v2.3.5 · medium confidence · unverified
3 commits (0 fixes) modifying sql/hive-thriftserver/v1.2.1
A change to existing behaviour in sql/hive-thriftserver/v1.2.1 — 3 commits, 147 files: “\[SPARK-27354\]\[SQL\] Move incompatible code from the hive-thriftserver module to sql/hive-thriftserver/v1.2.1”; “\[SPARK-27749\]\[SQL\] hadoop-3.2 support hive-thriftserver”; “\[SPARK-28104\]\[SQL\] Implement Spark's own GetColumnsOperation”.
sql/hive-thriftserver/v1.2.1 · medium confidence · unverified
3 commits (0 fixes) modifying sql/hive-thriftserver/v2.3.5
A change to existing behaviour in sql/hive-thriftserver/v2.3.5 — 3 commits, 153 files: “\[SPARK-27749\]\[SQL\] hadoop-3.2 support hive-thriftserver”; “\[SPARK-27737\]\[SQL\]\[FOLLOW-UP\] Move sql/hive-thriftserver/v2.3.4 to sql/hive-thriftserver/v2.3.5”; “\[SPARK-28104\]\[SQL\] Implement Spark's own GetColumnsOperation”.
sql/hive-thriftserver/v2.3.5 · medium confidence · unverified
3 commits (0 fixes) modifying third\_party/hadoop-0.20.0
A change to existing behaviour in third\_party/hadoop-0.20.0 — 3 commits, 45 files: “multi-tracker branch now compiles and runs; but it crashes right before the end. The same problem is seen also in the master branch (in the ChainedStreaming implementation)”; “Removed Hadoop's SLF4J jars”; “Initial commit”.
_third\party/hadoop-0.20.0 · medium confidence · unverified
3 commits (1 fix) modifying src/native
A change to existing behaviour in src/native — 3 commits (1 fix), 2 files: “Fix native build to use build directory”; “Replacing the native lzf compression code with the ning open-source compress-lzf library. (Apache 2.0 liscense)”; “Initial commit”.
src/native · medium confidence · unverified
30 commits (3 fixes) modifying python/pyspark/mllib/linalg
A change to existing behaviour in python/pyspark/mllib/linalg — 30 commits (3 fixs), 2 files: “\[SPARK-11531\]\[ML\] SparseVector error Msg”; “\[SPARK-10631\] \[DOCUMENTATION, MLLIB, PYSPARK\] Added documentation for few APIs”; “\[SPARK-14812\]\[ML\]\[MLLIB\]\[PYTHON\] Experimental, DeveloperApi annotation audit for ML”; and 27 more.
python/pyspark/mllib/linalg · medium confidence · unverified
30 commits (5 fixes) modifying common/network-common/src/main/java/org/apache/spark/network/server
A change to existing behaviour in common/network-common/src/main/java/org/apache/spark/network/server — 30 commits (5 fixs), 10 files: “\[SPARK-20547\]\[REPL\] Throw RemoteClassLoadedError for transient errors in ExecutorClassLoader”; “\[SPARK-27073\]\[CORE\] Fix a race condition when handling of IdleStateEvent”; “\[SPARK-27868\]\[CORE\] Better default value and documentation for socket server backlog”; and 27 more.
common/network-common/src/main/java/org/apache/spark/network/server · medium confidence · unverified
32 commits (4 fixes) modifying examples/src/main/python
A change to existing behaviour in examples/src/main/python — 32 commits (4 fixs), 11 files: “\[SPARK-4897\] \[PySpark\] Python 3 support”; “\[SPARK-15031\]\[EXAMPLE\] Use SparkSession in examples”; “rmse was wrongly calculated”; and 29 more.
examples/src/main/python · medium confidence · unverified
327 commits (34 fixes) modifying python/pyspark/ml
A change to existing behaviour in python/pyspark/ml — 327 commits (34 fixs), 13 files: “\[SPARK-25765\]\[ML\] Add training cost to BisectingKMeans summary”; “\[SPARK-9768\] \[PYSPARK\] \[ML\] Add Python API and user guide for ml.feature.ElementwiseProduct”; “\[SPARK-15189\]\[PYSPARK\]\[DOCS\] Update ml.evaluation PyDoc”; and 324 more.
python/pyspark/ml · medium confidence · unverified
37 commits (6 fixes) modifying examples/src/main/scala/org/apache/spark/examples/streaming
A change to existing behaviour in examples/src/main/scala/org/apache/spark/examples/streaming — 37 commits (6 fixs), 9 files: “\[SPARK-7895\] \[STREAMING\] \[EXAMPLES\] Move Kafka examples from scala-2.10/src to src”; “Update RecoverableNetworkWordCount.scala”; “\[MINOR\] Typo fixes”; and 34 more (refs \#571).
examples/src/main/scala/org/apache/spark/examples/streaming · medium confidence · unverified
37 commits (7 fixes) modifying build
A change to existing behaviour in build — 37 commits (7 fixs), 4 files: “\[SPARK-11052\] Spaces in the build dir causes failures in the build/mv…”; “\[SPARK-19550\]\[BUILD\]\[CORE\]\[WIP\] Remove Java 7 support”; “\[SPARK-26144\]\[BUILD\] \build/mvn\ should detect \scala.version\ based on \scala.binary.version\”; and 33 more.
build · medium confidence · unverified
37 commits (7 fixes) modifying graphx/src/main/scala/org/apache/spark/graphx/util
A change to existing behaviour in graphx/src/main/scala/org/apache/spark/graphx/util — 37 commits (7 fixs), 5 files: “\[MINOR\]\[DOCS\] Fix all typos in markdown files of \doc\ and similar patterns in other comments”; “SPARK-1488. Resolve scalac feature warnings during build”; “Miscel doc update”; and 34 more (refs \#557, \#720).
graphx/src/main/scala/org/apache/spark/graphx/util · medium confidence · unverified
38 commits (4 fixes) modifying R/pkg/vignettes
A change to existing behaviour in R/pkg/vignettes — 38 commits (4 fixs), 1 file: “\[SPARK-23461\]\[R\] vignettes should include model predictions for some ML models”; “\[SPARK-21293\]\[SPARKR\]\[DOCS\] structured streaming doc update”; “\[SPARK-19386\]\[SPARKR\]\[FOLLOWUP\] fix error in vignettes”; and 35 more.
R/pkg/vignettes · medium confidence · unverified
38 commits (4 fixes) modifying examples/src/main/java/org/apache/spark/examples/streaming
A change to existing behaviour in examples/src/main/java/org/apache/spark/examples/streaming — 38 commits (4 fixs), 8 files: “\[SPARK-7895\] \[STREAMING\] \[EXAMPLES\] Move Kafka examples from scala-2.10/src to src”; “\[SPARK-11663\]\[STREAMING\] Add Java API for trackStateByKey”; “Revise formatting of previous commit f80e2629bb74bc62960c61ff313f7e7802d61319”; and 34 more (refs \#571).
examples/src/main/java/org/apache/spark/examples/streaming · medium confidence · unverified
381 commits (82 fixes) modifying dev
A change to existing behaviour in dev — 381 commits (82 fixs), 30 files: “\[SPARK-8495\] \[SPARKR\] Add a \.lintr\ file to validate the SparkR files and the \lint-r\ script”; “\[SPARK-7845\] \[BUILD\] Bumping default Hadoop version used in profile hadoop-1 to 1.2.1”; “\[SPARK-22063\]\[R\] Fixes lint check failures in R by latest commit sha1 ID of lint-r”; and 373 more (refs \#6967, \#592, \#569, \#565, \#13286, …).
dev · medium confidence · unverified
39 commits (4 fixes) modifying external/avro/src/main
A change to existing behaviour in external/avro/src/main — 39 commits (4 fixs), 12 files: “\[SPARK-24876\]\[SQL\] Avro: simplify schema serialization”; “\[SPARK-24887\]\[SQL\] Avro: use SerializableConfiguration in Spark utils to deduplicate code”; “\[SPARK-24881\]\[SQL\] New Avro option - compression”; and 35 more.
external/avro/src/main · medium confidence · unverified
398 commits (27 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis — 398 commits (27 fixs), 14 files: “\[SPARK-8269\] \[SQL\] string function: initcap”; “\[SPARK-20392\]\[SQL\] Set barrier to prevent re-entering a tree”; “\[DOCS\] Docs-only improvements”; and 391 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis · medium confidence · unverified
4 commits (0 fixes) modifying examples/src/main/scala/org/apache/spark/examples/sql
A change to existing behaviour in examples/src/main/scala/org/apache/spark/examples/sql — 4 commits, 2 files: “\[SPARK-21731\]\[BUILD\] Upgrade scalastyle to 0.9”; “\[SPARK-16046\]\[DOCS\] Aggregations in the Spark SQL programming guide”; “\[SPARK-26353\]\[SQL\] Add typed aggregate functions(max/min) to the example module”; and 1 more.
examples/src/main/scala/org/apache/spark/examples/sql · medium confidence · unverified
4 commits (0 fixes) modifying resource-managers/kubernetes/integration-tests/scripts
A change to existing behaviour in resource-managers/kubernetes/integration-tests/scripts — 4 commits, 1 file: “\[SPARK-25957\]\[K8S\] Make building alternate language binding docker images optional”; “\[SPARK-26343\]\[K8S\] Try to speed up running local k8s integration tests”; “\[SPARK-23010\]\[K8S\] Initial checkin of k8s integration tests”; and 1 more.
resource-managers/kubernetes/integration-tests/scripts · medium confidence · unverified
4 commits (0 fixes) modifying sql/catalyst/src/main/java/org/apache/spark/sql
A change to existing behaviour in sql/catalyst/src/main/java/org/apache/spark/sql — 4 commits, 1 file: “\[SPARK-17864\]\[SQL\] Mark data type APIs as stable (not DeveloperApi)”; “\[SPARK-5193\]\[SQL\] Remove Spark SQL Java-specific API”; “\[SPARK-26026\]\[BUILD\] Published Scaladoc jars missing from Maven Central”; and 1 more.
sql/catalyst/src/main/java/org/apache/spark/sql · medium confidence · unverified
4 commits (0 fixes) modifying sql/catalyst/src/main/java/org/apache/spark/sql/sources
A change to existing behaviour in sql/catalyst/src/main/java/org/apache/spark/sql/sources — 4 commits, 37 files: “\[SQL\]\[DOC\]\[MINOR\] update documents for Table and WriteBuilder”; “\[SPARK-24252\]\[SQL\] Add TableCatalog API”; “\[SPARK-27521\]\[SQL\] Move data source v2 to catalyst module”; and 1 more.
sql/catalyst/src/main/java/org/apache/spark/sql/sources · medium confidence · unverified
4 commits (0 fixes) modifying sql/catalyst/src/main/java/org/apache/spark/sql/util
A change to existing behaviour in sql/catalyst/src/main/java/org/apache/spark/sql/util — 4 commits, 1 file: “\[SPARK-27162\]\[SQL\] Add new method asCaseSensitiveMap in CaseInsensitiveStringMap”; “\[SPARK-27106\]\[SQL\] merge CaseInsensitiveStringMap and DataSourceOptions”; “\[SPARK-24252\]\[SQL\] Add v2 catalog plugin system”; and 1 more.
sql/catalyst/src/main/java/org/apache/spark/sql/util · medium confidence · unverified
4 commits (0 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/xml
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/xml — 4 commits, 1 file: “\[SPARK-17963\]\[SQL\]\[DOCUMENTATION\] Add examples (extend) in each expression and improve documentation”; “\[SPARK-16318\]\[SQL\] Implement all remaining xpath functions”; “\[SQL\]\[MINOR\] XPathDouble prettyPrint should say 'double' not 'float'”; and 1 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/xml · medium confidence · unverified
4 commits (0 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/util
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/util — 4 commits, 2 files: “\[SPARK-20460\]\[SQL\] Make it more consistent to handle column name duplication”; “\[SPARK-27521\]\[SQL\] Move data source v2 to catalyst module”; “\[SPARK-25565\]\[BUILD\] Add scalastyle rule to check add Locale.ROOT to .toLowerCase and .toUpperCase for internal calls”; and 1 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/util · medium confidence · unverified
4 commits (0 fixes) modifying sql/core/v1.2.1
A change to existing behaviour in sql/core/v1.2.1 — 4 commits, 4 files: “\[SPARK-27105\]\[SQL\] Optimize away exponential complexity in ORC predicate conversion”; “\[SPARK-28108\]\[SQL\]\[test-hadoop3.2\] Simplify OrcFilters”; “\[SPARK-27182\]\[SQL\] Move the conflict source code of the sql/core module to sql/core/v1.2.1”; and 1 more.
sql/core/v1.2.1 · medium confidence · unverified
4 commits (1 fix) modifying external/kinesis-asl/src/main/resources
A change to existing behaviour in external/kinesis-asl/src/main/resources — 4 commits (1 fix), 1 file: “\[SPARK-14134\]\[CORE\] Change the package name used for shading classes”; “\[SPARK-13595\]\[BUILD\] Move docker, extras modules into external”; “\[SPARK-27610\]\[YARN\] Shade netty native libraries”; and 1 more.
external/kinesis-asl/src/main/resources · medium confidence · unverified
4 commits (1 fix) modifying external/spark-ganglia-lgpl
A change to existing behaviour in external/spark-ganglia-lgpl — 4 commits (1 fix), 1 file: “\[SPARK-13238\]\[CORE\] Add ganglia dmax parameter”; “\[SPARK-13595\]\[BUILD\] Move docker, extras modules into external”; “\[SPARK-25655\]\[BUILD\] Add -Pspark-ganglia-lgpl to the scala style check”; and 1 more.
external/spark-ganglia-lgpl · medium confidence · unverified
4 commits (1 fix) modifying graphx/src/main/scala/org/apache/spark/graphx/util/collection
A change to existing behaviour in graphx/src/main/scala/org/apache/spark/graphx/util/collection — 4 commits (1 fix), 1 file: “SPARK-1786: Reopening PR 724”; “\[SPARK-6428\] Turn on explicit type checking for public methods”; “\[SPARK-3873\]\[GRAPHX\] Import order fixes”; and 1 more.
graphx/src/main/scala/org/apache/spark/graphx/util/collection · medium confidence · unverified
4 commits (1 fix) modifying spark-network-yarn\_2.12
A change to existing behaviour in spark-network-yarn\_2.12 — 4 commits (1 fix), 1 file: “\[SPARK-27773\]\[FOLLOW-UP\] Fix Checkstyle failure”; “\[SPARK-27773\]\[SHUFFLE\] add metrics for number of exceptions caught in ExternalShuffleBlockHandler”; “\[SPARK-25642\]\[YARN\] Adding two new metrics to record the number of registered connections as well as the number of active connections to YARN Shuffle Service”; and 1 more.
_spark-network-yarn\2.12 · medium confidence · unverified
402 commits (43 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer — 402 commits (43 fixs), 15 files: “\[SPARK-20392\]\[SQL\] Set barrier to prevent re-entering a tree”; “\[DOCS\] Docs-only improvements”; “\[SPARK-22181\]\[SQL\] Adds ReplaceExceptWithFilter rule”; and 397 more (refs \#8742).
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer · medium confidence · unverified
41 commits (7 fixes) modifying examples/src/main/python/mllib
A change to existing behaviour in examples/src/main/python/mllib — 41 commits (7 fixs), 45 files: “\[SPARK-4897\] \[PySpark\] Python 3 support”; “\[SPARK-13019\]\[DOCS\] Replace example code in mllib-statistics.md using include\_example”; “\[SPARK-11383\]\[DOCS\] Replaced example code in mllib-naive-bayes.md/mllib-isotonic-regression.md using include\_example”; and 38 more.
examples/src/main/python/mllib · medium confidence · unverified
4212 commits (691 fixes) modifying core
A change to existing behaviour in core — 4212 commits (691 fixs), 955 files: “\[SPARK-19062\] Utils.writeByteBuffer bug fix”; “Revert "\[SPARK-24250\]\[SQL\] support accessing SQLConf inside tasks"”; “\[SPARK-19219\]\[SQL\] Fix Parquet log output defaults”; and 4179 more (refs \#450, \#535, \#494, \#6457, \#8905, …).
core · medium confidence · unverified
44 commits (8 fixes) modifying common/network-common/src/main/java/org/apache/spark/network/util
A change to existing behaviour in common/network-common/src/main/java/org/apache/spark/network/util — 44 commits (8 fixs), 15 files: “\[SPARK-17314\]\[CORE\] Use Netty's DefaultThreadFactory to enable its fast ThreadLocal impl”; “\[SPARK-27868\]\[CORE\] Better default value and documentation for socket server backlog”; “\[SPARK-17379\]\[BUILD\] Upgrade netty-all to 4.0.41 final for bug fixes”; and 41 more.
common/network-common/src/main/java/org/apache/spark/network/util · medium confidence · unverified
4527 commits (564 fixes) modifying repository
A change to existing behaviour in repository — 4527 commits (564 fixs), 353 files: “Revert "\[SPARK-24250\]\[SQL\] support accessing SQLConf inside tasks"”; “\[SPARK-15613\] \[SQL\] Fix incorrect days to millis conversion due to Daylight Saving Time”; “\[SPARK-25765\]\[ML\] Add training cost to BisectingKMeans summary”; and 4493 more (refs \#6457, \#8905, \#12178, \#221, \#321, …).
repository · medium confidence · unverified
46 commits (3 fixes) modifying R
A change to existing behaviour in R — 46 commits (3 fixs), 14 files: “\[SPARK-8084\] \[SPARKR\] Make SparkR scripts fail on error”; “\[DOC\] Update some outdated links”; “\[SPARK-8603\]\[SPARKR\] Use shell() instead of system2() for SparkR on Windows”; and 43 more.
R · medium confidence · unverified
47 commits (6 fixes) modifying common/network-shuffle/src/main/java/org/apache/spark/network/shuffle
A change to existing behaviour in common/network-shuffle/src/main/java/org/apache/spark/network/shuffle — 47 commits (6 fixs), 14 files: “\[SPARK-15121\] Improve logging of external shuffle handler”; “\[SPARK-19550\]\[BUILD\]\[CORE\]\[WIP\] Remove Java 7 support”; “\[SPARK-21475\]\[Core\]Revert "\[SPARK-21475\]\[CORE\] Use NIO's Files API to replace FileInputStream/FileOutputStream in some critical paths"”; and 44 more (refs \#12944).
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle · medium confidence · unverified
4702 commits (646 fixes) modifying sql/core
A change to existing behaviour in sql/core — 4702 commits (646 fixs), 1329 files: “Revert "\[SPARK-24250\]\[SQL\] support accessing SQLConf inside tasks"”; “\[SPARK-19219\]\[SQL\] Fix Parquet log output defaults”; “\[SPARK-24804\] There are duplicate words in the test title in the DatasetSuite”; and 4664 more (refs \#12178, \#978, \#2, \#8494, \#12625, …).
sql/core · medium confidence · unverified
475 commits (62 fixes) modifying R/pkg/R
A change to existing behaviour in R/pkg/R — 475 commits (62 fixs), 34 files: “\[SPARK-25908\]\[CORE\]\[SQL\] Remove old deprecated items in Spark 3”; “\[SPARK-6824\] Fill the docs for DataFrame API in SparkR”; “\[SPARK-16508\]\[SPARKR\] Fix CRAN undocumented/duplicated arguments warnings”; and 470 more.
R/pkg/R · medium confidence · unverified
5 commits (0 fixes) modifying common/tags
A change to existing behaviour in common/tags — 5 commits, 13 files: “\[SPARK-15290\]\[BUILD\] Move annotations, like @Since / @DeveloperApi, into spark-tags”; “\[SPARK-26026\]\[BUILD\] Published Scaladoc jars missing from Maven Central”; “\[SPARK-26055\]\[CORE\] InterfaceStability annotations should be retained at runtime”; and 2 more.
common/tags · medium confidence · unverified
5 commits (0 fixes) modifying examples/src/main/python/sql/streaming
A change to existing behaviour in examples/src/main/python/sql/streaming — 5 commits, 3 files: “\[SPARK-16114\]\[SQL\] structured streaming network word count examples”; “\[MINOR\]\[DOC\] Use raw triple double quotes around docstrings where there are occurrences of backslashes”; “\[SPARK-23522\]\[PYTHON\] always use sys.exit over builtin exit”; and 2 more.
examples/src/main/python/sql/streaming · medium confidence · unverified
5 commits (0 fixes) modifying mllib-local/src/main/scala/org/apache/spark/ml/stat
A change to existing behaviour in mllib-local/src/main/scala/org/apache/spark/ml/stat — 5 commits, 2 files: “\[SPARK-15413\]\[ML\]\[MLLIB\] Change \toBreeze\ to \asBreeze\ in Vector and Matrix”; “\[SPARK-3359\]\[BUILD\]\[DOCS\] More changes to resolve javadoc 8 errors that will help unidoc/genjavadoc compatibility”; “\[SPARK-15411\]\[ML\] Add @since to ml.stat.MultivariateOnlineSummarizer.scala”; and 2 more.
mllib-local/src/main/scala/org/apache/spark/ml/stat · medium confidence · unverified
5 commits (0 fixes) modifying sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/xml
A change to existing behaviour in sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/xml — 5 commits, 1 file: “\[SPARK-16271\]\[SQL\] Implement Hive's UDFXPathUtil”; “\[MINOR\]\[BUILD\] Fix Java linter errors”; “\[SPARK-26508\]\[CORE\]\[SQL\] Address warning messages in Java reported at lgtm.com”; and 2 more.
sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/xml · medium confidence · unverified
5 commits (1 fix) modifying python/test\_support
A change to existing behaviour in python/test\_support — 5 commits (1 fix), 3 files: “\[SPARK-4897\] \[PySpark\] Python 3 support”; “Fix sys.path bug in PySpark SparkContext.addPyFile”; “License headers”; and 2 more.
_python/test\support · medium confidence · unverified
50 commits (8 fixes) modifying examples/src/main/java/org/apache/spark/examples/mllib
A change to existing behaviour in examples/src/main/java/org/apache/spark/examples/mllib — 50 commits (8 fixs), 38 files: “\[SPARK-13551\]\[MLLIB\] Fix wrong comment and remove meanless lines in mllib.JavaBisectingKMeansExample”; “\[SPARK-5900\]\[MLLIB\] make PIC and FPGrowth Java-friendly”; “\[MINOR\] Fix newly added java-lint errors”; and 47 more.
examples/src/main/java/org/apache/spark/examples/mllib · medium confidence · unverified
52 commits (4 fixes) modifying src/scala
A change to existing behaviour in src/scala — 52 commits (4 fixs), 37 files: “Renamed HdfsFile to HadoopFile”; “Imported changes from old repository (mostly Mosharaf's work, plus some fault tolerance code)”; “Added fork()/join() operations for SparkContext, as well as corresponding changes to MesosScheduler to support multiple ParallelOperations”; and 49 more (refs \#12, \#45).
src/scala · medium confidence · unverified
54 commits (6 fixes) modifying python/pyspark/streaming
A change to existing behaviour in python/pyspark/streaming — 54 commits (6 fixs), 6 files: “\[SPARK-4897\] \[PySpark\] Python 3 support”; “\[SPARK-10142\] \[STREAMING\] Made python checkpoint recovery handle non-local checkpoint paths and existing SparkContexts”; “\[SPARK-3952\] \[Streaming\] \[PySpark\] add Python examples in Streaming Programming Guide”; and 51 more.
python/pyspark/streaming · medium confidence · unverified
54 commits (8 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders — 54 commits (8 fixs), 3 files: “\[SPARK-16813\]\[SQL\] Remove private\[sql\] and private\[spark\] from catalyst package”; “Revert "\[SPARK-12692\]\[BUILD\]\[SQL\] Scala style: Fix the style violation (Space before "," or ":")"”; “\[SPARK-23595\]\[SQL\] ValidateExternalType should support interpreted execution”; and 51 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders · medium confidence · unverified
540 commits (88 fixes) modifying python/pyspark
A change to existing behaviour in python/pyspark — 540 commits (88 fixs), 28 files: “\[SPARK-25908\]\[CORE\]\[SQL\] Remove old deprecated items in Spark 3”; “\[SPARK-13687\]\[PYTHON\] Cleanup PySpark parallelize temporary files”; “Clean up setup code in PySpark checkpointing tests”; and 535 more (refs \#498, \#462, \#542).
python/pyspark · medium confidence · unverified
58 commits (8 fixes) modifying python/pyspark/ml/param
A change to existing behaviour in python/pyspark/ml/param — 58 commits (8 fixs), 3 files: “\[SPARK-21088\]\[ML\] CrossValidator, TrainValidationSplit support collect all models when fitting: Python API”; “\[SPARK-4897\] \[PySpark\] Python 3 support”; “\[SPARK-14104\]\[PYSPARK\]\[ML\] All Python param setters should use the \\_set\ method”; and 54 more.
python/pyspark/ml/param · medium confidence · unverified
588 commits (87 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions — 588 commits (87 fixs), 38 files: “\[SPARK-12421\]\[SQL\] Prevent Internal/External row from exposing state”; “\[DOCS\] Docs-only improvements”; “\[SPARK-18711\]\[SQL\] should disable subexpression elimination for LambdaVariable”; and 582 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions · medium confidence · unverified
59 commits (6 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json — 59 commits (6 fixs), 6 files: “\[SPARK-17764\]\[SQL\] Add \to\_json\ supporting to convert nested struct column to JSON string”; “\[SPARK-25040\]\[SQL\] Empty string for non string types should be disallowed”; “\[SPARK-19849\]\[SQL\] Support ArrayType in to\_json to produce JSON array”; and 56 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json · medium confidence · unverified
6 commits (0 fixes) modifying common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/protocol
A change to existing behaviour in common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/protocol — 6 commits, 1 file: “\[SPARK-14011\]\[CORE\]\[SQL\] Enable \LineLength\ Java checkstyle rule”; “\[SPARK-12583\]\[MESOS\] Mesos shuffle service: Don't delete shuffle files before application has stopped”; “\[SPARK-27665\]\[CORE\] Split fetch shuffle blocks protocol from OpenBlocks”; and 3 more.
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/protocol · medium confidence · unverified
6 commits (0 fixes) modifying external/kinesis-asl/src/main/scala/org/apache/spark/examples
A change to existing behaviour in external/kinesis-asl/src/main/scala/org/apache/spark/examples — 6 commits, 2 files: “\[SPARK-20168\]\[DSTREAM\] Add changes to use kinesis fetches from specific timestamp”; “\[SPARK-13595\]\[BUILD\] Move docker, extras modules into external”; “\[SPARK-20855\]\[Docs\]\[DStream\] Update the Spark kinesis docs to use the KinesisInputDStream builder instead of deprecated KinesisUtils”; and 3 more.
external/kinesis-asl/src/main/scala/org/apache/spark/examples · medium confidence · unverified
6 commits (0 fixes) modifying sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2
A change to existing behaviour in sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2 — 6 commits, 6 files: “\[SPARK-27162\]\[SQL\] Add new method asCaseSensitiveMap in CaseInsensitiveStringMap”; “\[SPARK-24252\]\[SQL\] Add TableCatalog API”; “\[SPARK-24252\]\[SQL\] Add v2 catalog plugin system”; and 3 more.
sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2 · medium confidence · unverified
6 commits (1 fix) modifying examples/src/main/scala/org/apache/spark/examples/sql/streaming
A change to existing behaviour in examples/src/main/scala/org/apache/spark/examples/sql/streaming — 6 commits (1 fix), 3 files: “\[SPARK-22403\]\[SS\] Add optional checkpointLocation argument to StructuredKafkaWordCount example”; “\[SPARK-16114\]\[SQL\] structured streaming network word count examples”; “\[SPARK-16886\]\[EXAMPLES\]\[DOC\] Fix some examples to be consistent and indentation in documentation”; and 3 more.
examples/src/main/scala/org/apache/spark/examples/sql/streaming · medium confidence · unverified
6 commits (1 fix) modifying external/kinesis-asl/src/main/java/org/apache/spark/examples
A change to existing behaviour in external/kinesis-asl/src/main/java/org/apache/spark/examples — 6 commits (1 fix), 1 file: “\[SPARK-14011\]\[CORE\]\[SQL\] Enable \LineLength\ Java checkstyle rule”; “\[SPARK-13595\]\[BUILD\] Move docker, extras modules into external”; “\[MINOR\]\[BUILD\] Fix lint-java breaks in Java”; and 3 more.
external/kinesis-asl/src/main/java/org/apache/spark/examples · medium confidence · unverified
6 commits (2 fixes) modifying common/unsafe/src/main/java/org/apache/spark/sql
A change to existing behaviour in common/unsafe/src/main/java/org/apache/spark/sql — 6 commits (2 fixs), 1 file: “Revert \[SPARK-10399\] \[SPARK-23879\] \[SPARK-23762\] \[SPARK-25317\]”; “\[SPARK-17495\]\[SQL\] Add more tests for hive hash”; “\[SPARK-10399\]\[CORE\]\[SQL\] Introduce multiple MemoryBlocks to choose several types of memory block”; and 3 more.
common/unsafe/src/main/java/org/apache/spark/sql · medium confidence · unverified
6 commits (2 fixes) modifying examples/src/main/scala/org/apache/spark/examples/pythonconverters
A change to existing behaviour in examples/src/main/scala/org/apache/spark/examples/pythonconverters — 6 commits (2 fixs), 1 file: “\[SPARK-3389\] Add Converter for ease of Parquet reading in PySpark”; “\[SPARK-7929\] Remove Bagel examples & whitespace fix for examples”; “\[SPARK-12084\]\[CORE\] Fix codes that uses ByteBuffer.array incorrectly”; and 3 more.
examples/src/main/scala/org/apache/spark/examples/pythonconverters · medium confidence · unverified
61 commits (7 fixes) modifying repl/src/main
A change to existing behaviour in repl/src/main — 61 commits (7 fixs), 4 files: “SPARK-1099: Introduce local\[\*\] mode to infer number of cores”; “\[SPARK-20547\]\[REPL\] Throw RemoteClassLoadedError for transient errors in ExecutorClassLoader”; “Makes Spark SIMR ready”; and 58 more (refs \#557).
repl/src/main · medium confidence · unverified
631 commits (136 fixes) modifying streaming
A change to existing behaviour in streaming — 631 commits (136 fixs), 142 files: “SPARK-1663. (Addendum) Fix signature of one version of JavaPairRDDStream.reduceByKeyAndWindow()”; “\[SPARK-7553\] \[STREAMING\] Added methods to maintain a singleton StreamingContext”; “\[SPARK-7217\] \[STREAMING\] Add configuration to control the default behavior of StreamingContext.stop() implicitly calling SparkContext.stop()”; and 627 more (refs \#365, \#567).
streaming · medium confidence · unverified
64 commits (4 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate — 64 commits (4 fixs), 18 files: “\[SPARK-12005\]\[SQL\] Work around VerifyError in HyperLogLogPlusPlus”; “\[MINOR\]\[DOCS\] Fix all typos in markdown files of \doc\ and similar patterns in other comments”; “\[SPARK-11720\]\[SQL\]\[ML\] Handle edge cases when count = 0 or 1 for Stats function”; and 61 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate · medium confidence · unverified
64 commits (4 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning — 64 commits (4 fixs), 4 files: “\[SPARK-21164\]\[SQL\] Remove isTableSample from Sample and isGenerated from Alias and AttributeReference”; “\[SPARK-13957\]\[SQL\] Support Group By Ordinal in SQL”; “\[SPARK-4322\]\[SQL\] Enables struct fields as sub expressions of grouping fields”; and 61 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning · medium confidence · unverified
688 commits (54 fixes) modifying (repo-wide)
A change to existing behaviour in (repo-wide) — 688 commits (54 fixs), 27 files: “\[SPARK-8495\] \[SPARKR\] Add a \.lintr\ file to validate the SparkR files and the \lint-r\ script”; “Use Hadoop 1.2.1 in application example”; “\[Minor\]ignore all config files in conf”; and 499 more (refs \#567, \#557, \#216).
(repo-wide) · low confidence · unverified
69 commits (5 fixes) modifying resource-managers/kubernetes/core/src/main
A change to existing behaviour in resource-managers/kubernetes/core/src/main — 69 commits (5 fixs), 35 files: “\[SPARK-24003\]\[CORE\] Add support to provide spark.executor.extraJavaOptions in terms of App Id and/or Executor Id's”; “\[SPARK-22845\]\[SCHEDULER\] Modify spark.kubernetes.allocation.batch.delay to take time instead of int”; “\[SPARK-24793\]\[K8S\] Enhance spark-submit for app management”; and 66 more.
resource-managers/kubernetes/core/src/main · medium confidence · unverified
69 commits (9 fixes) modifying examples/src/main/scala/org/apache/spark/examples
A change to existing behaviour in examples/src/main/scala/org/apache/spark/examples — 69 commits (9 fixs), 20 files: “Makes Spark SIMR ready”; “\[SPARK-26285\]\[CORE\] accumulator metrics sources for LongAccumulator and Doub…”; “\[SPARK-15031\]\[EXAMPLE\] Use SparkSession in examples”; and 66 more (refs \#567, \#540).
examples/src/main/scala/org/apache/spark/examples · medium confidence · unverified
7 commits (0 fixes) modifying common/network-shuffle/src/main/java/org/apache/spark/network/sasl
A change to existing behaviour in common/network-shuffle/src/main/java/org/apache/spark/network/sasl — 7 commits, 1 file: “\[SPARK-14011\]\[CORE\]\[SQL\] Enable \LineLength\ Java checkstyle rule”; “\[SPARK-21494\]\[NETWORK\] Use correct app id when authenticating to external service”; “\[SPARK-14355\]\[BUILD\] Fix typos in Exception/Testcase/Comments and static analysis results”; and 4 more.
common/network-shuffle/src/main/java/org/apache/spark/network/sasl · medium confidence · unverified
7 commits (0 fixes) modifying examples/src/main/resources
A change to existing behaviour in examples/src/main/resources — 7 commits, 10 files: “\[SPARK-3389\] Add Converter for ease of Parquet reading in PySpark”; “\[SPARK-16046\]\[DOCS\] Aggregations in the Spark SQL programming guide”; “\[SPARK-25656\]\[SQL\]\[DOC\]\[EXAMPLE\] Add a doc and examples about extra data source options”; and 4 more.
examples/src/main/resources · medium confidence · unverified
7 commits (0 fixes) modifying external
A change to existing behaviour in external — 7 commits, 11 files: “\[SPARK-13595\]\[BUILD\] Move docker, extras modules into external”; “\[SPARK-27794\]\[R\]\[DOCS\] Use https URL for CRAN repo”; “\[SPARK-23038\]\[TEST\] Update docker/spark-test (JDK/OS)”; and 4 more.
external · medium confidence · unverified
7 commits (0 fixes) modifying python/test\_support/sql
A change to existing behaviour in python/test\_support/sql — 7 commits, 7 files: “\[SPARK-13509\]\[SPARK-13507\]\[SQL\] Support for writing CSV with a single function call”; “\[SPARK-18352\]\[SQL\] Support parsing multiline json files”; “\[SPARK-10185\] \[SQL\] Feat sql comma separated paths”; and 4 more.
_python/test\support/sql · medium confidence · unverified
7 commits (1 fix) modifying examples/src/main/java/org/apache/spark/examples/sql/hive
A change to existing behaviour in examples/src/main/java/org/apache/spark/examples/sql/hive — 7 commits (1 fix), 1 file: “\[SPARK-17810\]\[SQL\] Default spark.sql.warehouse.dir is relative to local FS but can resolve as HDFS path”; “\[SPARK-16380\]\[EXAMPLES\] Update SQL examples and programming guide for Python language binding”; “\[SPARK-16303\]\[DOCS\]\[EXAMPLES\] Updated SQL programming guide and examples”; and 4 more.
examples/src/main/java/org/apache/spark/examples/sql/hive · medium confidence · unverified
7 commits (2 fixes) modifying common/network-common/src/main/java/org/apache/spark/network/buffer
A change to existing behaviour in common/network-common/src/main/java/org/apache/spark/network/buffer — 7 commits (2 fixs), 2 files: “\[SPARK-21475\]\[Core\]Revert "\[SPARK-21475\]\[CORE\] Use NIO's Files API to replace FileInputStream/FileOutputStream in some critical paths"”; “\[SPARK-25218\]\[CORE\] Fix potential resource leaks in TransportServer and SocketAuthHelper”; “\[SPARK-21475\]\[CORE\] Use NIO's Files API to replace FileInputStream/FileOutputStream in some critical paths”; and 4 more.
common/network-common/src/main/java/org/apache/spark/network/buffer · medium confidence · unverified
70 commits (13 fixes) modifying graphx/src/main/scala/org/apache/spark/graphx/lib
A change to existing behaviour in graphx/src/main/scala/org/apache/spark/graphx/lib — 70 commits (13 fixs), 9 files: “Miscel doc update”; “Updated doc for PageRank”; “\[SPARK-5814\]\[MLLIB\]\[GRAPHX\] Remove JBLAS from runtime”; and 67 more.
graphx/src/main/scala/org/apache/spark/graphx/lib · medium confidence · unverified
73 commits (10 fixes) modifying sbin
A change to existing behaviour in sbin — 73 commits (10 fixs), 20 files: “SPARK-1286: Make usage of spark-env.sh idempotent”; “\[SPARK-2410\]\[SQL\] Merging Hive Thrift/JDBC server”; “\[SPARK-13848\]\[SPARK-5185\] Update to Py4J 0.9.2 in order to fix classloading issue”; and 68 more.
sbin · medium confidence · unverified
73 commits (14 fixes) modifying examples/src/main/scala/org/apache/spark/examples/mllib
A change to existing behaviour in examples/src/main/scala/org/apache/spark/examples/mllib — 73 commits (14 fixs), 51 files: “\[HOT-FIX\] Recover some deprecations for 2.10 compatibility”; “\[SPARK-1545\] \[mllib\] Add Random Forests”; “\[SPARK-16694\]\[CORE\] Use for/foreach rather than map for Unit expressions whose side effects are required”; and 70 more.
examples/src/main/scala/org/apache/spark/examples/mllib · medium confidence · unverified
74 commits (8 fixes) modifying conf
A change to existing behaviour in conf — 74 commits (8 fixs), 6 files: “Further fixes to how Mesos is found and used”; “SPARK-1902 Silence stacktrace from logs when doing port failover to port n+1”; “MetricsServlet code refactor according to comments”; and 71 more.
conf · medium confidence · unverified
75 commits (12 fixes) modifying graphx/src/main/scala/org/apache/spark/graphx/impl
A change to existing behaviour in graphx/src/main/scala/org/apache/spark/graphx/impl — 75 commits (12 fixs), 11 files: “\[MINOR\]\[DOCS\] Fix all typos in markdown files of \doc\ and similar patterns in other comments”; “Make EdgeDirection val instead of case object for Java compat”; “Miscel doc update”; and 72 more.
graphx/src/main/scala/org/apache/spark/graphx/impl · medium confidence · unverified
753 commits (76 fixes) modifying python/pyspark/sql
A change to existing behaviour in python/pyspark/sql — 753 commits (76 fixs), 15 files: “\[SPARK-25908\]\[CORE\]\[SQL\] Remove old deprecated items in Spark 3”; “\[SPARK-18459\]\[SPARK-18460\]\[STRUCTUREDSTREAMING\] Rename triggerId to batchId and add triggerDetails to json in StreamingQueryStatus”; “\[SPARK-8269\] \[SQL\] string function: initcap”; and 745 more (refs \#9527, \#2).
python/pyspark/sql · medium confidence · unverified
77 commits (9 fixes) modifying resource-managers/mesos/src/main
A change to existing behaviour in resource-managers/mesos/src/main — 77 commits (9 fixs), 19 files: “\[SPARK-24003\]\[CORE\] Add support to provide spark.executor.extraJavaOptions in terms of App Id and/or Executor Id's”; “\[SPARK-11034\]\[LAUNCHER\]\[MESOS\] Launcher: add support for monitoring Mesos apps”; “\[SPARK-27015\]\[MESOS\] properly escape mesos scheduler arguments”; and 73 more.
resource-managers/mesos/src/main · medium confidence · unverified
78 commits (12 fixes) modifying dev/sparktestsupport
A change to existing behaviour in dev/sparktestsupport — 78 commits (12 fixs), 4 files: “\[SPARK-26036\]\[PYTHON\] Break large tests.py files into smaller files”; “\[SPARK-26033\]\[PYTHON\]\[TESTS\] Break large ml/tests.py file into smaller files”; “\[SPARK-17320\] add build\_profile\_flags entry to mesos build module”; and 74 more.
dev/sparktestsupport · medium confidence · unverified
78 commits (9 fixes) modifying examples/src/main/python/ml
A change to existing behaviour in examples/src/main/python/ml — 78 commits (9 fixs), 65 files: “\[SPARK-12247\]\[ML\]\[DOC\] Documentation for spark.ml's ALS and collaborative filtering in general”; “\[SPARK-13012\]\[DOCUMENTATION\] Replace example code in ml-guide.md using include\_example”; “\[SPARK-11551\]\[DOC\] Replace example code in ml-features.md using include\_example”; and 74 more (refs \#10002).
examples/src/main/python/ml · medium confidence · unverified
79 commits (9 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql — 79 commits (9 fixs), 4 files: “\[SPARK-6375\]\[SQL\] Fix formatting of error messages”; “\[SPARK-26318\]\[SQL\] Deprecate Row.merge”; “\[SQL\] Initial support for reporting location of error in sql string”; and 75 more.
sql/catalyst/src/main/scala/org/apache/spark/sql · medium confidence · unverified
8 commits (0 fixes) modifying external/kafka-0-10-token-provider/src/main
A change to existing behaviour in external/kafka-0-10-token-provider/src/main — 8 commits, 4 files: “\[SPARK-26772\]\[YARN\] Delete ServiceCredentialProvider and make HadoopDelegationTokenProvider a developer API”; “\[SPARK-27294\]\[SS\] Add multi-cluster Kafka delegation token”; “\[SPARK-26254\]\[CORE\] Extract Hive + Kafka dependencies from Core”; and 5 more.
external/kafka-0-10-token-provider/src/main · medium confidence · unverified
8 commits (2 fixes) modifying common/network-common/src/main/java/org/apache/spark/network/crypto
A change to existing behaviour in common/network-common/src/main/java/org/apache/spark/network/crypto — 8 commits (2 fixs), 6 files: “\[SPARK-20027\]\[DOCS\] Compilation fix in java docs”; “\[SPARK-19550\]\[BUILD\]\[CORE\]\[WIP\] Remove Java 7 support”; “\[SPARK-6237\]\[NETWORK\] Network-layer changes to allow stream upload”; and 5 more.
common/network-common/src/main/java/org/apache/spark/network/crypto · medium confidence · unverified
82 commits (11 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans — 82 commits (11 fixs), 2 files: “\[DOCS\] Docs-only improvements”; “\[SPARK-12441\]\[SQL\] Fixing missingInput in Generate/MapPartitions/AppendColumns/MapGroups/CoGroup”; “\[SPARK-21164\]\[SQL\] Remove isTableSample from Sample and isGenerated from Alias and AttributeReference”; and 79 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans · medium confidence · unverified
82 commits (15 fixes) modifying dev/create-release
A change to existing behaviour in dev/create-release — 82 commits (15 fixs), 10 files: “\[SPARK-25238\]\[PYTHON\] lint-python: Fix W605 warnings for pycodestyle 2.4”; “\[SPARK-11807\] Remove support for Hadoop \< 2.2”; “\[SPARK-22142\]\[BUILD\]\[STREAMING\] Move Flume support behind a profile, take 2”; and 79 more.
dev/create-release · medium confidence · unverified
83 commits (17 fixes) modifying external/kafka-0-10-sql/src/main
A change to existing behaviour in external/kafka-0-10-sql/src/main — 83 commits (17 fixs), 13 files: “\[SPARK-25501\]\[SS\] Add kafka delegation token support”; “\[MINOR\] Remove unnecessary gets when getting a value from map”; “\[SPARK-22956\]\[SS\] Bug fix for 2 streams union failover scenario”; and 80 more.
external/kafka-0-10-sql/src/main · medium confidence · unverified
9 commits (4 fixes) modifying common/unsafe/src/main/java/org/apache/spark/unsafe/memory
A change to existing behaviour in common/unsafe/src/main/java/org/apache/spark/unsafe/memory — 9 commits (4 fixs), 4 files: “Revert \[SPARK-10399\] \[SPARK-23879\] \[SPARK-23762\] \[SPARK-25317\]”; “\[SPARK-15391\] \[SQL\] manage the temporary memory of timsort”; “\[SPARK-16021\] Fill freed memory in test to help catch correctness bugs”; and 6 more.
common/unsafe/src/main/java/org/apache/spark/unsafe/memory · medium confidence · unverified
93 commits (16 fixes) modifying examples/src/main/java/org/apache/spark/examples/ml
A change to existing behaviour in examples/src/main/java/org/apache/spark/examples/ml — 93 commits (16 fixs), 66 files: “\[SPARK-12247\]\[ML\]\[DOC\] Documentation for spark.ml's ALS and collaborative filtering in general”; “\[SPARK-15296\]\[MLLIB\] Refactor All Java Tests that use SparkSession”; “\[SPARK-13012\]\[DOCUMENTATION\] Replace example code in ml-guide.md using include\_example”; and 88 more (refs \#10002).
examples/src/main/java/org/apache/spark/examples/ml · medium confidence · unverified
95 commits (12 fixes) modifying sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl
A change to existing behaviour in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl — 95 commits (12 fixs), 1 file: “\[SPARK-7286\]\[SQL\] Deprecate !== in favour of =!=”; “\[SPARK-10747\]\[SQL\] Support NULLS FIRST\|LAST clause in ORDER BY”; “\[SPARK-5040\]\[SQL\] Support expressing unresolved attributes using $"attribute name" notation in SQL DSL”; and 92 more.
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl · medium confidence · unverified
95 commits (13 fixes) modifying examples/src/main/scala/org/apache/spark/examples/ml
A change to existing behaviour in examples/src/main/scala/org/apache/spark/examples/ml — 95 commits (13 fixs), 65 files: “\[SPARK-13012\]\[DOCUMENTATION\] Replace example code in ml-guide.md using include\_example”; “\[SPARK-11551\]\[DOC\] Replace example code in ml-features.md using include\_example”; “\[SPARK-22735\]\[ML\]\[DOC\] Added VectorSizeHint docs and examples”; and 89 more (refs \#10002).
examples/src/main/scala/org/apache/spark/examples/ml · medium confidence · unverified
95 commits (6 fixes) modifying resource-managers/yarn/src/main
A change to existing behaviour in resource-managers/yarn/src/main — 95 commits (6 fixs), 23 files: “\[SPARK-24003\]\[CORE\] Add support to provide spark.executor.extraJavaOptions in terms of App Id and/or Executor Id's”; “\[SPARK-23296\]\[YARN\] Include stacktrace in YARN-app diagnostic”; “\[MINOR\]\[YARN\] Add disable yarn.nodemanager.vmem-check-enabled option to memLimitExceededLogMessage”; and 92 more.
resource-managers/yarn/src/main · medium confidence · unverified
970 commits (125 fixes) modifying project
A change to existing behaviour in project — 970 commits (125 fixs), 5 files: “\[SPARK-25765\]\[ML\] Add training cost to BisectingKMeans summary”; “\[SPARK-25908\]\[CORE\]\[SQL\] Remove old deprecated items in Spark 3”; “\[SPARK-4628\]\[BUILD\] Add a resolver to MiMaBuild.scala for mqttv3(1.0.1)”; and 961 more (refs \#596, \#12178, \#2, \#31, \#69, …).
project · medium confidence · unverified
Fixes
3 commits (2 fixes) fixing licenses-binary
A fix in licenses-binary — 3 commits (2 fixs), 51 files: “\[SPARK-27358\]\[UI\] Update jquery to 1.12.x to pick up security fixes”; “\[SPARK-26986\]\[ML\] Add JAXB reference impl to build for Java 9+”; “\[SPARK-24654\]\[BUILD\] Update, fix LICENSE and NOTICE, and specialize for source vs binary”.
licenses-binary · medium confidence · unverified
Test coverage
1 commit adding/updating tests in common/kvstore/src/test/resources; 1 commit adding/updating tests in common/network-shuffle/src/test/resources; 1 commit adding/updating tests in external/avro/src/test/java; 1 commit adding/updating tests in resource-managers/kubernetes/integration-tests/tests; 1 commit adding/updating tests in resource-managers/yarn/src/test/java; 1 commit adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/v2; 10 commits adding/updating tests in python/pyspark/ml/tests; 10 commits adding/updating tests in repl/src/test/resources; 10 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/csv; 100 commits adding/updating tests in R/pkg/tests/fulltests; 117 commits adding/updating tests in sql/hive/compatibility; 12 commits adding/updating tests in common/network-shuffle/src/test/java/org/apache/spark/network/sasl; 12 commits adding/updating tests in common/unsafe/src/test/java/org/apache/spark/unsafe; 12 commits adding/updating tests in python/pyspark/testing; 13 commits adding/updating tests in graphx/src/test/scala/org/apache/spark/graphx/impl; 13 commits adding/updating tests in python/pyspark/tests; 134 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser; 14 commits adding/updating tests in mllib-local/src/test/scala/org/apache/spark/ml/linalg; 14 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/util; 15 commits adding/updating tests in R/pkg/tests; 15 commits adding/updating tests in sql/catalyst/src/test/java/org/apache/spark/sql/catalyst; 17 commits adding/updating tests in resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s; 18 commits adding/updating tests in resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit; 19 commits adding/updating tests in common/unsafe/src/test/java/org/apache/spark/unsafe/types; 2 commits adding/updating tests in mllib-local/src/test/scala/org/apache/spark/ml/impl; 2 commits adding/updating tests in resource-managers/kubernetes/core/src/test/resources; 2 commits adding/updating tests in resource-managers/yarn/src/test/resources; 23 commits adding/updating tests in external/kinesis-asl/src/test/scala; 23 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/aggregate; 26 commits adding/updating tests in resource-managers/kubernetes/core/src/test/scala/org/apache/spark/scheduler; 27 commits adding/updating tests in external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010; 27 commits adding/updating tests in graphx/src/test/scala/org/apache/spark/graphx/lib; 294 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis; 3 commits adding/updating tests in dev/tests; 3 commits adding/updating tests in external/avro/src/test/resources; 3 commits adding/updating tests in external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/mocks; 3 commits adding/updating tests in external/kinesis-asl/src/test/resources; 3 commits adding/updating tests in python/pyspark/streaming/tests; 3 commits adding/updating tests in resource-managers/kubernetes/integration-tests/src/test/resources; 3 commits adding/updating tests in resource-managers/mesos/src/test/scala/org/apache/spark/deploy; 3 commits adding/updating tests in sql/catalyst/src/test/java/org/apache/spark/sql/catalog; 3 commits adding/updating tests in sql/catalyst/src/test/java/org/apache/spark/sql/streaming; 30 commits adding/updating tests in external/docker-integration-tests; 304 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer; 32 commits adding/updating tests in common/network-shuffle/src/test/java/org/apache/spark/network/shuffle; 32 commits adding/updating tests in resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest; 34 commits adding/updating tests in resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features; 38 commits adding/updating tests in resource-managers/mesos/src/test/scala/org/apache/spark/scheduler; 39 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql; 4 commits adding/updating tests in R/pkg/inst/tests; 4 commits adding/updating tests in external/kafka-0-10-sql/src/test/resources; 4 commits adding/updating tests in sql/catalyst/src/test/resources; 4 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalog; 4 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/json; 40 commits adding/updating tests in common/network-common/src/test; 41 commits adding/updating tests in external/avro/src/test/scala/org/apache/spark/sql/avro; 41 commits adding/updating tests in python/pyspark/sql/tests; 42 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation; 44 commits adding/updating tests in repl/src/test/scala; 46 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/trees; 48 commits adding/updating tests in launcher/src/test/java; 49 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen; 5 commits adding/updating tests in common/unsafe/src/test/java/org/apache/spark/unsafe/hash; 5 commits adding/updating tests in external/avro/src/test/scala/org/apache/spark/sql/execution; 5 commits adding/updating tests in graphx/src/test/scala/org/apache/spark/graphx/util; 5 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/xml; 501 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions; 55 commits adding/updating tests in graphx/src/test/scala/org/apache/spark/graphx; 56 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/types; 6 commits adding/updating tests in external/kafka-0-10-token-provider/src/test; 6 commits adding/updating tests in external/kafka-0-10/src/test/java; 6 commits adding/updating tests in mllib-local/src/test/scala/org/apache/spark/ml/util; 62 commits adding/updating tests in resource-managers/yarn/src/test/scala/org/apache/spark/deploy; 65 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst; 69 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders; 7 commits adding/updating tests in common/kvstore/src/test/java; 7 commits adding/updating tests in common/unsafe/src/test/scala; 7 commits adding/updating tests in external/kinesis-asl/src/test/java; 7 commits adding/updating tests in python/pyspark/mllib/tests; 7 commits adding/updating tests in resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/backend; 7 commits adding/updating tests in resource-managers/yarn/src/test/scala/org/apache/spark/scheduler; 7 commits adding/updating tests in src/test; 73 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util; 76 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans; 8 commits adding/updating tests in resource-managers/yarn/src/test/scala/org/apache/spark/network; 82 commits adding/updating tests in external/kafka-0-10-sql/src/test/scala; 9 commits adding/updating tests in graphx/src/test/resources; 9 commits adding/updating tests in launcher/src/test/resources; 9 commits adding/updating tests in resource-managers/kubernetes/integration-tests/dev; 96 commits adding/updating tests in sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog.
Dependencies
978 commits updating dependencies (42 manifests)
A dependency / build maintenance change in (dependencies) — 978 commits (148 fixs), 42 files: “\[SPARK-7895\] \[STREAMING\] \[EXAMPLES\] Move Kafka examples from scala-2.10/src to src”; “\[SPARK-19409\]\[BUILD\]\[TEST-MAVEN\] Fix ParquetAvroCompatibilitySuite failure due to test dependency on avro”; “Update to latest Scala Maven plugin and allow Zinc external compiler”; and 960 more (refs \#3837, \#221, \#211, \#841, \#527, …).
(dependencies) · high confidence · unverified
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Baseline
- First survey — no prior run to compare against. CAI 43.3.
Lenses
- Code Health 29.7
- Architecture 94.9
- Maturity 63.3
- Readiness 50.3
- Security 66.5
- Accessibility 46.7
- Performance 100.0
Changes since last survey
- 300 commits — 297 feature/other, 3 fixes
By area
- (root) — 270 commits
- core/src — 7 commits
- sql/core — 6 commits
- sql/catalyst — 5 commits
- mllib/src — 3 commits
- python/pyspark — 2 commits
- dev/.rat-excludes — 1 commit
- dev/run-tests.py — 1 commit
- dev/sparktestsupport — 1 commit
- resource-managers/kubernetes — 1 commit
- resource-managers/yarn — 1 commit
- sbin/start-slave.sh — 1 commit
- sql/hive-thriftserver — 1 commit
Notable commits
- fix: [SPARK-26985][CORE] Fix "access only some column of the all of columns " for big endian architecture BUG
- fix: [SPARK-28154][ML][FOLLOWUP] GMM fix double caching
- fix: [SPARK-28164] Fix usage description of start-slave.sh
- change: [MINOR][BUILD] Exclude pyspark-coverage-site/ dir from RAT
- change: [SPARK-11412][SQL] Support merge schema for ORC
- change: [SPARK-23179][SQL] Support option to throw exception if overflow occurs during Decimal arithmetic
- change: [SPARK-27369][CORE] Setup resources when Standalone Worker starts up
- change: [SPARK-27622][CORE] Avoiding the network when block manager fetches disk persisted RDD blocks from the same host
- change: [SPARK-27630][CORE] Properly handle task end events from completed stages
- change: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
- change: [SPARK-27823][CORE][DOC][FOLLOWUP] Update doc of config spark.driver.resourcesFile
- change: [SPARK-27871][SQL] LambdaVariable should use per-query unique IDs instead of globally unique IDs
- change: [SPARK-27893][SQL][PYTHON][FOLLOW-UP] Allow Scalar Pandas and Python UDFs can be tested with Scala test base
- change: [SPARK-27945][SQL] Minimal changes to support columnar processing
- change: [SPARK-27992][PYTHON] Allow Python to join with connection thread to propagate errors
- change: [SPARK-28005][YARN] Remove unnecessary log from SparkRackResolver
- change: [SPARK-28045][ML][PYTHON] add missing RankingEvaluator
- change: [SPARK-28056][.2][PYTHON][SQL] add docstring/doctest for SCALAR_ITER Pandas UDF
- change: [SPARK-28077][SQL] Support ANSI SQL OVERLAY function.
- change: [SPARK-28117][ML] LDA and BisectingKMeans cache the input dataset if necessary
- …and 280 more
Architecture
- 0 containers · 10 bounded contexts · 7 dependency edges (baseline)
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
mypinkhacker/Apache was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 17 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit 132743b6014c7380d207aade8a2c0c19c67721be — the exact code this score is about.
- Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.85+preprod-eba6548ac50f.