Commits

Breadcrumbs

History for

onmaster

User selector

All users

All time

Commit History

Commits on Apr 1, 2024

• [SPARK-47664][PYTHON][CONNECT] Validate the column name with cached schema

  Show description for 968cba2
  

  zhengruifeng

  authored and

  HyukjinKwon

  committed
  968cba2

  Copy full SHA for 968cba2

Commits on Mar 29, 2024

• [SPARK-47543][CONNECT][PYTHON][TESTS][FOLLOW-UP] Skip the test if pandas and PyArrow are unavailable

  Show description for 63529bf
  HyukjinKwon

  committed
  63529bf

  Copy full SHA for 63529bf

• [SPARK-47644][PYTHON][DOCS] Refine docstrings of try_*

  Show description for 5259dcc
  HyukjinKwon

  committed
  5259dcc

  Copy full SHA for 5259dcc

• [SPARK-47638][PS][CONNECT] Skip column name validation in PS

  Show description for a57f94b
  zhengruifeng

  committed
  a57f94b

  Copy full SHA for a57f94b

Commits on Mar 28, 2024

• [SPARK-47623][PYTHON][CONNECT][TESTS] Use `QuietTest` in parity tests

  Show description for 6cf5f95
  

  zhengruifeng

  authored and

  HyukjinKwon

  committed
  6cf5f95

  Copy full SHA for 6cf5f95

• [SPARK-47366][PYTHON] Add pyspark and dataframe parse_json aliases

  Show description for 7f1eadc
  

  gene-db

  and

  HyukjinKwon

  committed
  7f1eadc

  Copy full SHA for 7f1eadc

• [SPARK-47621][PYTHON][DOCS] Refine docstring of `try_sum`, `try_avg`, `avg`, `sum`, `mean`

  Show description for b594c4e
  

  HyukjinKwon

  authored and

  zhengruifeng

  committed
  b594c4e

  Copy full SHA for b594c4e

• [MINOR][PYTHON][TESTS] Remove redundant parity tests

  Show description for 8a6e5bb
  zhengruifeng

  committed
  8a6e5bb

  Copy full SHA for 8a6e5bb

• [SPARK-47619][PYTHON][DOCS] Refine docstring of `to_json/from_json`

  Show description for 67bd52c
  HyukjinKwon

  committed
  67bd52c

  Copy full SHA for 67bd52c

• [SPARK-47620][PYTHON][CONNECT] Add a helper function to sort columns

  Show description for 0eb96ae
  zhengruifeng

  committed
  0eb96ae

  Copy full SHA for 0eb96ae

• [SPARK-47543][CONNECT][PYTHON] Inferring `dict` as `MapType` from Pandas DataFrame to allow DataFrame creation

  Show description for 0318648
  

  itholic

  authored and

  HyukjinKwon

  committed
  0318648

  Copy full SHA for 0318648

Commits on Mar 27, 2024

• [SPARK-47107][SS][PYTHON] Implement partition reader for python streaming data source

  Show description for e3e7135
  

  chaoqin-li1123

  authored and

  HeartSaVioR

  committed
  e3e7135

  Copy full SHA for e3e7135

• [SPARK-47485][SQL][PYTHON][CONNECT] Create column with collations in dataframe API

  Show description for 1b3e5e7
  

  stefankandic

  authored and

  MaxGekk

  committed
  1b3e5e7

  Copy full SHA for 1b3e5e7

• [SPARK-47610][CORE] Always set `io.netty.tryReflectionSetAccessible=true`

  Show description for 5f392a2
  

  pan3793

  authored and

  LuciferYang

  committed
  5f392a2

  Copy full SHA for 5f392a2

• [SPARK-47562][CONNECT] Factor literal handling out of `plan.py`

  Show description for 9db527e
  zhengruifeng

  committed
  9db527e

  Copy full SHA for 9db527e

• [SPARK-47273][SS][PYTHON] implement Python data stream writer interface

  Show description for 5ac3918
  

  chaoqin-li1123

  authored and

  HeartSaVioR

  committed
  5ac3918

  Copy full SHA for 5ac3918

Commits on Mar 26, 2024

• [SPARK-47544][PYTHON] SparkSession builder method is incompatible with visual studio code intellisense

  Show description for 8e20f8e
  

  nija-at

  authored and

  dongjoon-hyun

  committed
  8e20f8e

  Copy full SHA for 8e20f8e

• [SPARK-47367][PYTHON][CONNECT][TESTS][FOLLOW-UP] Recover the test case for the number of partitions

  Show description for ded8cdf
  

  HyukjinKwon

  authored and

  dongjoon-hyun

  committed
  ded8cdf

  Copy full SHA for ded8cdf

• [SPARK-47560][PYTHON][CONNECT] Avoid RPC to validate column name with cached schema

  Show description for becbf8b
  zhengruifeng

  committed
  becbf8b

  Copy full SHA for becbf8b

• [SPARK-47367][PYTHON][CONNECT] Support Python data sources with Spark Connect

  Show description for a127df4
  

  allisonwang-db

  authored and

  HyukjinKwon

  committed
  a127df4

  Copy full SHA for a127df4

Commits on Mar 24, 2024

• [MINOR][PYTHON][DOCS] Fix a `pandas_udf` example

  Show description for 19e3de4
  

  zhengruifeng

  authored and

  HyukjinKwon

  committed
  19e3de4

  Copy full SHA for 19e3de4

• [SPARK-44708][PYTHON] Migrate test_reset_index assert_eq to use assertDataFrameEqual

  Show description for 2ef15ca
  

  gdhuper

  authored and

  HyukjinKwon

  committed
  2ef15ca

  Copy full SHA for 2ef15ca

• [SPARK-47497][SQL] Make `to_csv` support the output of `array/struct/map/binary` as pretty strings

  Show description for 18bb6a3
  

  panbingkun

  authored and

  dongjoon-hyun

  committed
  18bb6a3

  Copy full SHA for 18bb6a3

Commits on Mar 22, 2024

• [MINOR][PYTHON][DOCS] Fix typos in `PandasCogroupedOps.applyInArrow`

  Show description for 105f008
  

  zhengruifeng

  authored and

  HyukjinKwon

  committed
  105f008

  Copy full SHA for 105f008

• [SPARK-47499][PYTHON][CONNECT][TESTS] Enable `test_help_command` test

  Show description for ca44489
  

  zhengruifeng

  authored and

  dongjoon-hyun

  committed
  ca44489

  Copy full SHA for ca44489

• [SPARK-47500][PYTHON][CONNECT] Factor column name handling out of `plan.py`

  Show description for aea13fc
  

  zhengruifeng

  authored and

  HyukjinKwon

  committed
  aea13fc

  Copy full SHA for aea13fc

• Revert "[SPARK-46654][SQL][PYTHON] Make `to_csv` explicitly indicate that it does not support some types of data"

  Show description for 7b575d1
  cloud-fan

  committed
  7b575d1

  Copy full SHA for 7b575d1

Commits on Mar 21, 2024

• [SPARK-41888][PYTHON][CONNECT][TESTS] Enable doctest for `DataFrame.observe`

  Show description for 91df046
  

  zhengruifeng

  authored and

  dongjoon-hyun

  committed
  91df046

  Copy full SHA for 91df046

• Revert "[SPARK-47007][SQL][PYTHON][R][CONNECT] Add the `map_sort` function"

  Show description for bb3e275
  cloud-fan

  committed
  bb3e275

  Copy full SHA for bb3e275

Commits on Mar 20, 2024

• [SPARK-41625][PYTHON][CONNECT][TESTS] Enable doctest `pyspark.sql.connect.dataframe.DataFrame.isStreaming`

  Show description for df56390
  

  zhengruifeng

  authored and

  HyukjinKwon

  committed
  df56390

  Copy full SHA for df56390

• [SPARK-47480][PYTHON][CONNECT][TESTS] Enable doctest for `createDataFrame`

  Show description for a5910a2
  

  zhengruifeng

  authored and

  HyukjinKwon

  committed
  a5910a2

  Copy full SHA for a5910a2

• [SPARK-47007][SQL][PYTHON][R][CONNECT] Add the `map_sort` function

  Show description for 747846b
  

  2 peopleauthored and

  MaxGekk

  committed
  747846b

  Copy full SHA for 747846b

• [SPARK-47454][PYTHON][CONNECT][TESTS][FOLLOWUP] Further split `pyspark.sql.tests.test_dataframe`

  Show description for 46b46b7
  

  zhengruifeng

  authored and

  HyukjinKwon

  committed
  46b46b7

  Copy full SHA for 46b46b7

Commits on Mar 19, 2024

• [SPARK-47454][PYTHON][CONNECT][TESTS] Split `pyspark.sql.tests.test_dataframe`

  Show description for b6a8369
  

  zhengruifeng

  authored and

  HyukjinKwon

  committed
  b6a8369

  Copy full SHA for b6a8369

Commits on Mar 18, 2024

• [SPARK-47380][CONNECT] Ensure on the server side that the SparkSession is the same

  Show description for 51e8634
  

  nemanja-boric-databricks

  authored and

  hvanhovell

  committed
  51e8634

  Copy full SHA for 51e8634

Pagination

Previous

Next