pandas-dev/pandas

This system is the pandas library, a high-performance data analysis and manipulation tool for Python. It provides core data structures like DataFrames and Series, supporting a wide range of operations including I/O for formats like CSV and Excel, time-series processing, and statistical aggregations. The codebase emphasizes performance through C/Cython extensions, SIMD acceleration, and Numba integration, while also offering a robust extension API for custom data types and pluggable plotting backends.

64 changes.

This period centered on the release of pandas 2.0, introducing major features like Copy-on-Write, Arrow-backed string dtypes, and SAS file support. The project underwent significant architectural modernization, migrating the build system to Meson and reorganizing internal modules for better maintainability. Extensive efforts were also made to expand test coverage, refactor documentation builds, and enforce stricter dependency and API standards.

55 changes.

This period focused on restructuring the pandas codebase into modular packages and establishing a comprehensive, standardized test suite for extension arrays and core components. Key efforts included exposing public APIs for extensions and window indexers, refactoring internal modules like configuration and arithmetic operations, and significantly expanding test coverage across scalar types, indexing, and I/O engines.

50 changes.

This period focused on significantly expanding test coverage for core data structures, including nullable arrays, indexes, and string accessors, while consolidating scattered test suites into modular packages. Concurrently, the codebase underwent substantial architectural refactoring, introducing new modules for array algorithms, indexing logic, and CSV parser engines to improve maintainability and prepare for future features like Numba acceleration and PyArrow integration.

23 changes.

This period focused on exposing pandas' internal C components, such as the datetime parser and JSON engine, via public headers to support external extensions. Concurrently, the codebase integrated SIMD acceleration for statistical moments and updated vendored libraries to enhance performance and stability. The work was complemented by extensive test coverage for Copy-on-Write behavior, type hinting, and various scalar operations.

Introduces the \pandas.io.sas\ module with \read\_sas\ to read SAS data files in SAS7BDAT and XPORT formats. The implementation includes a Cython-based parser for performance, support for chunked iteration via \SASReader\, automatic encoding inference for SAS7BDAT files, and handling of various date/time formats and compression options.

pandas/io/sas · high confidence

The cheatsheet documentation now includes a README that provides direct access to the official Pandas Cheat Sheet in PDF and PowerPoint formats for English, Japanese, and Persian. It also links to alternative cheat sheets developed by DataCamp, available in PDF, Streamlit, and Google Colab formats.

doc/cheatsheet · high confidence

The website now includes the official pandas 3.0 release announcement and a corresponding release candidate post, detailing major breaking changes such as the default use of a dedicated string data type (replacing \object\ dtype) and the enforcement of Copy-on-Write behavior which removes the \SettingWithCopyWarning\. These blog posts provide users with specific migration guidance, including code examples for handling the new string inference and updating chained assignment patterns, alongside instructions for installing the release via PyPI and conda-forge.

web/pandas/community/blog · high confidence

A new script, doc/scripts/eval\_performance.py, has been added to generate performance comparison data for DataFrame.eval and DataFrame.query operations. The script benchmarks execution times across different DataFrame sizes using both the 'python' and 'numexpr' engines, and saves the resulting performance plots as PNG images to the documentation's static assets directory.

doc/scripts · high confidence

Introduces a new automation suite in \scripts/issue\_assignment\ that enforces issue assignment rules and manages pull request status. The system gates PRs by labeling and closing those linked to unassigned or already-assigned issues (unless the author is exempt), and automatically unassigns contributors from issues with no recent activity. It also replaces the previous stale-bot logic with a custom engine that marks PRs as stale based on author inactivity and auto-closes them after a set period, while intelligently managing labels like \Needs Issue Assignment\ and \Awaiting Review\.

_scripts/issue\assignment · high confidence

Users can now import all major pandas extension array types directly from the \pandas.arrays\ namespace. This new module exposes a unified set of classes including \ArrowExtensionArray\, \ArrowStringArray\, \BooleanArray\, \Categorical\, \DatetimeArray\, \FloatingArray\, \IntegerArray\, \IntervalArray\, \NumpyExtensionArray\, \PeriodArray\, \SparseArray\, \StringArray\, and \TimedeltaArray\, simplifying access to these specialized data structures.

pandas/arrays · high confidence

The documentation build process now includes custom validation rules for docstrings via monkey-patched numpydoc checks (GL04, PD01, SA05, EX04) to enforce pandas-specific style guidelines, such as preventing references to private classes and ensuring consistent terminology. Additionally, a new \signature\_overrides.py\ module manually defines constructor signatures for Cython-based offset classes (e.g., \BusinessDay\, \CustomBusinessDay\) to ensure their parameters appear correctly in the generated API reference, addressing limitations in how Cython exposes signatures to Sphinx.

doc/source · high confidence

The \pandas/io/excel\ package has been reorganized into a subdirectory with dedicated modules for each engine (\\_calamine.py\, \\_odfreader.py\, \\_odswriter.py\, \\_openpyxl.py\, \\_pyxlsb.py\, \\xlrd.py\). A new \CalamineReader\ engine is introduced to support reading xlsx, xls, xlsb, and ods files, offering a faster alternative to existing engines. The legacy \xlrd\ and \pyxlsb\ engines are now deprecated and emit \Pandas4Warning\ when used, recommending users switch to \calamine\. The \ODSWriter\ and \ODFReader\ modules provide full support for OpenDocument Spreadsheet files. The \\\init\\_.py\ file now explicitly registers the available writers (Openpyxl, XlsxWriter, ODS) and exports the core \read\_excel\, \ExcelFile\, and \ExcelWriter\ classes.

pandas/io/excel · high confidence

The new pandas.api.types module now provides a public toolkit API, exposing core dtype classes (CategoricalDtype, DatetimeTZDtype, IntervalDtype, PeriodDtype) and utility functions (infer\_dtype, union\_categoricals) that were previously internal or located in other modules.

pandas/api/types · high confidence

The \pandas/api/extensions\ module is now a public entry point, exposing key classes and functions for building custom pandas extensions. Users can now directly import \ExtensionArray\, \ExtensionDtype\, \ExtensionScalarOpsMixin\, and \no\_default\ from this location, along with accessor registration utilities (\register\_dataframe\_accessor\, \register\_index\_accessor\, \register\_series\_accessor\) and the \register\_extension\_dtype\ decorator. This centralizes the public interface for extension development, making it easier to access the necessary components for creating custom dtypes and accessors without relying on internal or less stable paths.

pandas/api/extensions · high confidence

Users can now import standard rolling window indexers directly from the public API at pandas.api.indexers. The new module exposes BaseIndexer, FixedForwardWindowIndexer, VariableOffsetWindowIndexer, and check\_array\_indexer, allowing custom window bound calculations and validation without relying on internal implementation paths.

pandas/api/indexers · high confidence

This entry marks the initial release of pandas 2.0, introducing a comprehensive set of new features and behavioral changes. Key additions include the Copy-on-Write (CoW) mechanism enabled by default for safer data handling, a new string dtype with Arrow-backed storage options, and the \pd.col\ accessor for column selection. The release also brings support for reading Iceberg tables, improved ExtensionArray operators, and the removal of deprecated APIs such as \ArrayManager\ and several legacy index types. Additionally, it enforces stricter type annotations and updates the build system to use Meson.

pandas · high confidence

This change introduces the core implementation for PyArrow-backed data in pandas by creating the \pandas/core/arrays/arrow\ module. It adds \ArrowExtensionArray\ as the primary ExtensionArray for handling PyArrow data types, enabling users to store and manipulate data using PyArrow's memory layout. The module also includes \ListAccessor\ and \StructAccessor\ to provide convenient accessors for list and struct data types, along with utility functions for converting PyArrow arrays to NumPy and custom extension types for Period and Interval data.

pandas/core/arrays/arrow · high confidence

Pandas now implements the DataFrame Interchange Protocol, allowing DataFrames to be exported to and imported from other libraries that support this standard. This includes new internal classes (PandasDataFrameXchg, PandasColumn, PandasBuffer) to handle the protocol's data exchange requirements, and a \from\_dataframe\ function that can construct a pandas DataFrame from any object implementing the protocol. The implementation supports various data types including integers, floats, booleans, strings, categoricals, and datetime types with timezones, and leverages DLPack for efficient buffer handling. Note that this protocol is deprecated in favor of the Arrow PyCapsule Interface.

pandas/core/interchange · high confidence

Adds a new static site generator script (\web/pandas\_web.py\) and a \web/README.md\ to build the pandas website. The generator processes markdown sources using Jinja2 templates and Markdown extensions, supports blog posts from local files and RSS feeds, and fetches maintainer information from the GitHub API (with fallback to a cached JSON file to handle API rate limits).

web · high confidence

A new \pandas.api.typing\ module has been added to provide public API classes and aliases specifically for type-hinting purposes. This module exposes intermediate result types such as \DataFrameGroupBy\, \SeriesGroupBy\, \Resampler\, \Rolling\, \Expanding\, and various reader classes (\JsonReader\, \SASReader\, \StataReader\), along with special types like \NAType\, \NaTType\, and \NoDefault\. It also includes a comprehensive set of type aliases from \pandas.\_typing\ (e.g., \Dtype\, \Axis\, \JoinHow\) to support static analysis. Additionally, the module handles the deprecation of \Groupby\ class names by exposing the corrected \GroupBy\ variants while maintaining backward compatibility for the old spellings via a warning.

pandas/api/typing · high confidence

The plotting module has been refactored to support third-party backends via a new entry-point system. Users can now select a plotting backend (e.g., 'matplotlib' or others) via the \plotting.backend\ option or the \backend\ argument in plot methods. The \pandas.plotting\ package now exposes a \PlotAccessor\ and core plotting functions (\hist\_series\, \hist\_frame\, etc.) that delegate to the selected backend, while specialized functions like \scatter\_matrix\ and \register\_matplotlib\_converters\ remain tied to Matplotlib.

pandas/plotting · high confidence

Pandas now includes a new internal module at pandas/core/\_numba that provides the infrastructure for a Numba-accelerated execution engine. This change introduces the core components—executor, extensions, and type definitions—required to compile user-defined functions and aggregations using Numba, enabling significant performance improvements for operations like apply, rolling, and groupby aggregations by leveraging JIT compilation.

_pandas/core/\numba · high confidence

Pandas introduces a new set of Numba-compiled kernels in \pandas/core/\_numba/kernels\ to power rolling and groupby operations. This change adds optimized implementations for sliding and grouped calculations of mean, sum, variance, and min/max, including support for the \skipna\ parameter in groupby aggregations. These kernels are designed to improve performance and numerical stability for windowed statistics by leveraging Numba's JIT compilation.

_pandas/core/\numba/kernels · high confidence

Added a new \doc/sphinxext\ directory containing custom Sphinx extensions to automate release documentation. The \contributors.py\ extension introduces a \.. contributors::\ directive that generates lists of code contributors and pull requests for a given revision range, while \announce.py\ provides the underlying logic to fetch this data from Git and GitHub. A \README.rst\ file documents the purpose of these tools.

doc/sphinxext · high confidence

The User Guide documentation has been significantly expanded and reorganized. A new '10 minutes to pandas' introductory guide has been added to help new users get started quickly. The documentation for the Nullable Boolean data type has been added, detailing indexing with NA values and Kleene logical operations. The Cookbook has been expanded with new idioms for conditional assignment, splitting frames, and building multi-column criteria. Additionally, various existing guides have been updated to reflect recent API changes, deprecations, and best practices, including clarifications on indexing, groupby operations, and I/O methods.

_doc/source/user\guide · high confidence

Added several new CSV and data files to the documentation examples: air quality datasets (\air\_quality\_long.csv\, \air\_quality\_no2.csv\, \air\_quality\_no2\_long.csv\, \air\_quality\_pm25\_long.csv\, \air\_quality\_parameters.csv\, \air\_quality\_stations.csv\) containing hourly pollutant measurements from stations in Antwerp, Paris, and London; and the classic \iris.data\ and \baseball.csv\ datasets for machine learning and statistical examples.

doc/data · high confidence

A new \pandas/core/util\ package has been introduced, consolidating internal utilities for data hashing and Numba integration. The \hashing.py\ module provides \hash\_pandas\_object\ and \hash\array\, enabling consistent hashing of Index, Series, and DataFrame objects with support for index inclusion and categorization. The \numba\.py\ module offers helper functions like \maybe\_use\_numba\ and \jit\_user\_function\ to manage Numba JIT compilation and argument preparation for user-defined functions, laying the groundwork for Numba-accelerated operations in groupby and other aggregations.

pandas/core/util · high confidence

The repository now includes a suite of new pre-commit scripts to enforce coding standards and consistency. These include \check\_for\_inconsistent\_pandas\_namespace.py\ to prevent mixing \pd.Series\ and \Series\ imports, \check\_test\_imports.py\ to ensure top-level pandas objects are accessed via \pd\, and \check\_test\_naming.py\ to enforce \test\ and \Test\ prefixes. Additional hooks validate that exceptions are documented (\pandas\_errors\_documented.py\), located correctly (\validate\_exception\_location.py\), and that test dependencies are installed in CI (\validate\_test\_dependencies.py\). Maintenance scripts help sort \whatsnew\ notes (\sort\_whatsnew\_note.py\), validate RST title capitalization (\validate\_rst\_title\_capitalization.py\), check for unwanted patterns (\validate\_unwanted\_patterns.py\), and sync dependency versions across configuration files (\validate\_min\_versions\_in\_sync.py\). A utility script \generate\_pip\_deps\_from\_conda.py\ converts conda environments to pip requirements, and \run\_vulture.py\ detects unused code.

scripts · high confidence

Users now have access to new public API submodules under pandas.api. The pandas.api.executors module exposes BaseExecutionEngine for custom function executor engines used with map and apply. The pandas.api.interchange module provides the DataFrame class and from\_dataframe function to support the DataFrame interchange protocol. Additionally, pandas.api.internals exports a low-level create\_dataframe\_from\_blocks helper function, allowing advanced users to construct DataFrames directly from block structures, though it requires strict adherence to internal assumptions regarding array shapes and types.

pandas/api · high confidence

The CSV parser's C implementation now exposes a public Application Binary Interface (ABI) through new header files (io.h, pd\_parser.h, tokenizer.h) in the include directory. This introduces a standardized C API (PandasParser\_CAPI) that allows external C/C++ extensions to directly invoke core parsing functions—such as tokenization, string-to-number conversion, and source management—without relying on internal, private symbols. For users, this stabilizes the interface for custom parsers or third-party libraries that need to integrate with pandas' high-performance C engine, ensuring compatibility across pandas versions.

_pandas/\libs/include/pandas/parser · high confidence

The \pandas/\_libs/include/pandas/datetime\ directory now contains public C headers (\date\_conversions.h\ and \pd\_datetime.h\) that expose the Pandas datetime C-API. This allows external C extensions to import and use core datetime conversion functions, such as \int64ToIso\ and \scaleNanosecToUnit\, through the \PandasDateTime\_CAPI\ capsule.

_pandas/\libs/include/pandas/datetime · high confidence

Indexing utilities and window-indexer classes have been reorganized from their previous locations into a new \pandas/core/indexers\ directory. This change introduces \pandas/core/indexers/\_\init\\_.py\ to expose core validation functions (such as \check\_array\_indexer\, \length\_of\_indexer\, and \validate\_indices\) and moves indexer implementations (like \BaseIndexer\, \FixedWindowIndexer\, and \VariableWindowIndexer\) into \objects.py\. Users relying on internal indexing logic should now import from \pandas.core.indexers\.

pandas/core/indexers · high confidence

The \pandas/core/tools\ package has been introduced, consolidating the implementation of core data conversion utilities into dedicated modules: \datetimes.py\ (for \to\_datetime\), \timedeltas.py\ (for \to\_timedelta\), \numeric.py\ (for \to\_numeric\), and \times.py\ (for \to\_time\). This change reorganizes the codebase by moving these functions out of the legacy \tslib\ and \tslibs\ modules into a structured \core/tools\ location, providing a clearer separation of concerns for datetime, timedelta, numeric, and time parsing logic while maintaining the existing public API.

pandas/core/tools · high confidence

The project has replaced its legacy build infrastructure with Meson for compiling C/C++/Cython extensions and Pixi for managing the development environment. This change introduces new configuration files including \meson.build\, \meson.options\, \pixi.toml\, and \pixi.lock\, alongside helper scripts \generate\_version.py\ and \generate\_pxi.py\ to handle versioning and Cython pre-processing. The \environment.yml\ has been updated to reflect the new dependency structure, and \pyproject.toml\ is now installed as a source file to support pytest configuration. Additionally, \.gitattributes\ and \.gitignore\ have been added to manage file types and exclude build artifacts, CI configurations, and documentation files from source distributions.

(repo-wide) · high confidence

The pandas reshape functionality (including concat, merge, pivot, melt, and tile) has been moved from the top-level pandas.tools namespace into the internal pandas.core.reshape package. This reorganization consolidates the implementation under the core package structure, while the public API remains accessible via the standard pandas namespace (e.g., pd.merge, pd.concat).

pandas/core/reshape · high confidence

The pandas codebase now includes a new \pandas/core/array\_algos\ package that centralizes low-level algorithms operating on NumPy arrays and ExtensionArrays. This module introduces dedicated implementations for masked reductions (sum, prod, min, max), masked accumulations (cumsum, cumprod, cummin, cummax), and datetimelike accumulations (cumsum, cummin, cummax). It also provides specialized logic for quantile calculations, element-wise replacement (including regex support), putmask operations, and optimized take/reindexing paths, while removing legacy helper functions to streamline the internal architecture.

_pandas/core/array\algos · high confidence

The CSV parsing logic in \pandas/io/parsers\ has been restructured from a monolithic implementation into distinct, modular components. The public API (\read\_csv\, \read\_table\, \read\fwf\) is now exposed via a new \\\init\\_.py\ that imports from \readers.py\, while the internal parsing engines are separated into dedicated wrapper classes: \CParserWrapper\ for the C engine, \PythonParser\ for the pure Python engine, and \ArrowParserWrapper\ for the PyArrow engine. This change centralizes engine-specific logic and keyword validation in \base\_parser.py\, improving maintainability and allowing each engine to handle its own specific parsing behaviors and error handling.

pandas/io/parsers · high confidence

The implementation of several DataFrame and Series methods has been moved from their original locations into the new pandas/core/methods package. This refactoring extracts the logic for describe, filter (including boolean mask support), select, nlargest/nsmallest, and to\_dict into dedicated modules (describe.py, filter.py, select.py, selectn.py, to\_dict.py). Users benefit from a more organized internal structure, while the public API for these methods remains unchanged.

pandas/core/methods · high confidence

The matplotlib plotting implementation has been reorganized from a monolithic structure into distinct modules (\boxplot\, \converter\, \core\, \groupby\, \hist\, \misc\, \style\, \timeseries\, \tools\). This change introduces a centralized \PLOT\CLASSES\ registry in \\\init\\_.py\ to map plot kinds to their specific handler classes, and extracts converter registration logic into a dedicated \converter\ module with context managers. The refactoring also adds a new \groupby\ module to handle data iteration for \by\-based plots and separates time-series specific logic into \timeseries.py\, improving code maintainability and separation of concerns without altering the public API.

_pandas/plotting/\matplotlib · high confidence

The arithmetic and comparison logic previously scattered across pandas core modules has been reorganized into a new \pandas/core/ops\ package. This change introduces dedicated modules for specific concerns: \array\_ops.py\ handles low-level array operations, \common.py\ provides boilerplate for method definitions, \invalid.py\ manages type-checking and error handling for comparisons, \mask\_ops.py\ implements Kleene logic for masked boolean arrays, \missing.py\ standardizes division-by-zero behavior, and \dispatch.py\ controls extension array dispatching. This modularization improves code maintainability and consistency for all arithmetic, comparison, and logical operations.

pandas/core/ops · high confidence

The windowing logic (Rolling, Expanding, and Exponential Moving Window) has been reorganized from a single monolithic file into a dedicated \pandas/core/window\ package. This change introduces separate modules for core classes (\rolling.py\, \expanding.py\, \ewm.py\), shared utilities (\common.py\), and Numba engine implementations (\numba\_.py\, \online.py\), improving code maintainability and separation of concerns without altering the public API.

pandas/core/window · high confidence

The \pandas/io/formats\ module has been restructured from a flat set of scripts into a formal Python package. This includes adding an \\_\init\\.py\ that explicitly manages public API exposure via \\\all\\_\, introducing a new \console.py\ module to handle terminal size detection and interactive session introspection, and creating a dedicated \\_color\_data.py\ file to house the CSS4 color map used by Excel export. This change improves modularity and encapsulation of the formatting subsystem without altering user-facing behavior.

pandas/io/formats · high confidence

The internal configuration module has been reorganized from a flat structure into a dedicated \pandas.\config\ package. This change introduces a new \\\init\\_.py\ that exposes the public configuration API (\get\_option\, \set\_option\, \reset\_option\, \describe\_option\, \option\_context\, and \options\) while ensuring that \dates\ and \display\ configurations are initialized early to support internal libraries. The core logic is now split into \config.py\ (handling option registration, validation, and deprecation), \dates.py\ (registering date parsing options), \display.py\ (handling display encoding), and \localization.py\ (providing locale utilities). This restructuring isolates configuration concerns and prepares the codebase for future configuration enhancements.

_pandas/\config · high confidence

The JSON I/O module has been reorganized from a flat structure into a dedicated sub-package under \pandas/io/json\. This change introduces new internal modules (\\_json.py\, \\_normalize.py\, \\_table\_schema.py\) to handle core serialization, normalization, and table schema building respectively, while the public API remains accessible via \pandas.io.json\ and top-level functions like \read\_json\, \to\_json\, and \json\_normalize\.

pandas/io/json · high confidence

The sparse array implementation has been reorganized into a new \pandas/core/arrays/sparse/\ package. This change introduces a dedicated \SparseAccessor\ and \SparseFrameAccessor\ to expose sparse-specific properties (like \density\, \fill\_value\, \sp\_values\) and conversion methods (\to\_coo\, \from\_coo\) on Series and DataFrames, replacing previous inline implementations. The core \SparseArray\ class and its supporting index structures (\IntIndex\, \BlockIndex\) are now centralized in \array.py\, with a new \scipy\_sparse.py\ module handling the conversion logic between pandas sparse Series and scipy sparse matrices. This restructuring improves modularity and prepares the codebase for further sparse-specific enhancements.

pandas/core/arrays/sparse · high confidence

The pandas C-extension build now includes the \ultrajson.h\ header from the vendored UltraJSON library. This header defines the core types, macros, and function pointers required for the JSON encoding and decoding components within pandas, ensuring the necessary interface is available for compilation.

_pandas/\libs/include/pandas/vendored/ujson · high confidence

The internal testing utilities previously located in a single \pandas/\testing/\\init\\_.py\ file have been reorganized into a dedicated package structure. The public API is now exposed via \pandas.\testing.\\init\\_\, which imports from new submodules: \asserters\ (containing \assert\_frame\_equal\, \assert\_series\_equal\, etc.), \\_warnings\ (containing \assert\_produces\_warning\), \\_io\ (containing file I/O helpers like \round\_trip\_pickle\), \compat\ (sharing helpers for DataFrame/Series tests), and \contexts\ (providing context managers like \decompress\_file\ and \set\_timezone\). This change improves maintainability and type-checking for the testing infrastructure without altering the public \pandas.testing\ interface.

_pandas/\testing · high confidence

A change to existing behaviour in doc/source/whatsnew — 6511 commits (3163 fixs), 111 files.

(repo-wide) · medium confidence · unverified

The project now includes explicit license texts for several third-party dependencies and bundled libraries in the LICENSES directory. This update adds the Bottleneck, dateutil, haven (MIT), klib, musl, Neri-Schneider, NumPy, packaging (Apache 2.0), PSF, pyperclip, pyupgrade, SAS7BDAT, StatsBase.jl, ultrajson, and xsimd licenses. This ensures compliance with the licensing terms of these included components.

LICENSES · high confidence

New header files have been added to the vendored NumPy datetime library to expose internal C-level structures and functions. Specifically, \np\_datetime.h\ now defines the \pandas\_timedeltastruct\ and various minimum/maximum datetime constants, along with declarations for branchless calendar algorithms and field extractors. Additionally, \np\_datetime\_strings.h\ exposes functions for parsing and generating ISO 8601 date-time strings, including support for timezone offsets and format requirements. These changes provide the necessary interface definitions for pandas' C extensions to interact with the underlying datetime logic.

_pandas/\libs/include/pandas/vendored/numpy · high confidence

The CI pipeline now includes a script to verify that the pandas package does not import a specific blocklist of heavy or unnecessary dependencies (such as matplotlib, scipy, and lxml), ensuring a leaner import footprint. Additionally, a UBSan suppression file has been added to handle undefined behavior warnings arising from Cython-generated C code, specifically addressing type mismatches in virtual dispatches that are inherent to Cython's implementation and cannot be fixed in the pandas source.

ci · high confidence

The error module now includes specialized warning messages for Copy-on-Write (CoW) chained assignment scenarios involving inplace methods. Users attempting to modify data via chained indexing with inplace operations (such as \df\[col\].method(value, inplace=True)\ or \df\[col\].update(other)\) will now receive clear instructions to use alternative patterns like \df.method({col: value}, inplace=True)\ or \df.update({col: other})\. This change ensures that users understand why the original operation fails to update the source data and how to correctly perform the intended modification.

pandas/errors · high confidence

The warning about an unusable numexpr version is now deferred until the first time numexpr would actually be used, rather than firing immediately at import time. Additionally, column names containing special characters (such as spaces or symbols) can now be used in DataFrame.query and eval by wrapping them in backticks, which are automatically cleaned and escaped during parsing.

pandas/core/computation · high confidence

The \pandas.tseries.frequencies.infer\_freq\ function now emits a deprecation warning when it returns a frequency string, signaling that future versions will return a \BaseOffset\ object instead. Users can opt into the new behavior immediately by setting the option \future.infer\_freq\_returns\_offset\ to \True\, or preserve the current string output by using the \.freqstr\ attribute on the result.

pandas/tseries · high confidence

The documentation templates have been updated to improve navigation and update footer information. The sidebar now collapses subpages in the API reference section while keeping other sections expanded, and maintains its scroll position when navigating between pages. The footer has been updated to remove the outdated 'Hosted by OVHCloud' attribution and now displays a copyright notice via NumFOCUS. Additionally, a new template for an announcement banner has been added, currently displaying information about the pandas 3.0 release.

_doc/\templates · high confidence

Pandas now requires NumPy version 2.0.2 or higher to run. If an older version of NumPy is detected, the library will raise an ImportError at startup, preventing silent failures or incorrect behavior due to API incompatibilities with newer NumPy versions.

pandas/compat/numpy · high confidence

The Qt-based clipboard backend now correctly restores the process locale after initializing the Qt application. Previously, creating the QApplication instance would permanently alter the process-wide locale (affecting settings like LC\_TIME and LC\_NUMERIC), which could cause unexpected behavior in other parts of the application. This change ensures that locale settings are preserved across clipboard operations.

pandas/io/clipboard · high confidence

The \pandas/\_libs/include/pandas\ directory now includes three new C header files: \moments.h\, \portable.h\, and \skiplist.h\. \moments.h\ defines a \Moments\ struct and a \moments\_reduce\ function for computing central moments, supporting the optimized rolling window calculations. \portable.h\ provides cross-platform compatibility macros, including ASCII character handling and platform-specific integer overflow checking (using Windows \intsafe.h\ or GCC builtins). \skiplist.h\ introduces a skiplist data structure with an arena-based allocator, designed to optimize memory allocation for rolling median, quantile, and rank operations.

_pandas/\libs/include/pandas · high confidence

Added new CSS files (\getting\_started.css\ and \pandas.css\) to customize the appearance of the documentation's getting-started tutorials and main index page. These styles define specific layouts for data introduction cards, callouts, and task lists, while also overriding the PyData Sphinx Theme to adjust color variables (such as the info color) and fix table width issues in the Styler user guide.

_doc/source/\static/css · high confidence

The documentation build process now uses a new \make.py\ script that replaces the previous build mechanism. This script introduces a \--single\ parameter to build individual documentation pages, enforces the C locale to ensure consistent formatting in examples, and automatically creates HTML redirects based on a new \redirects.csv\ file to handle moved or renamed API pages. Additionally, a \.gitignore\ file is added to exclude generated data files from version control.

doc · high confidence

The pandas website has been restructured with new pages for getting started, contributing, and trying pandas in the browser via an experimental JupyterLite shell. The site now features a version switcher supporting pandas 3.0 (stable) through 1.0, a dedicated blog section aggregating posts from multiple feeds, and updated sponsor and team information. Social media links have been updated to include Telegram and Mastodon, and the home page now highlights recommended books including the Pandas Cookbook 3 and Effective Pandas 2.

web/pandas · high confidence

The website now applies a dedicated syntax-highlighting theme (CodeHilite) to code blocks, providing distinct colors for keywords, strings, comments, and errors, along with improved line-height and padding. Additionally, new styles have been added for blockquotes, table of contents, and admonitions to enhance readability and visual consistency across documentation pages.

web/pandas/static/css · high confidence

The moments calculation functions (such as mean and central differences) in pandas are now implemented with SIMD-optimized C++ backends for x86\_64 (SSE2) and ARM64 (NEON) architectures, falling back to scalar implementations for other platforms. This change leverages the xsimd library to accelerate statistical computations, resulting in improved performance for operations relying on these moments.

_pandas/\libs/simd · high confidence

The C-based CSV parser in pandas/\_libs/src/parser has been refactored to use the fast\_float library for converting strings to floating-point numbers, replacing the previous bespoke parsing logic. This change improves parsing accuracy and performance, particularly for numbers with leading zeros and embedded NUL characters, and ensures correct IEEE 754 rounding behavior. The update also includes improvements to error handling for allocation failures and buffer management during parsing.

_pandas/\libs/src/parser · high confidence

The pandas website layout has been updated to use Bootstrap 5.0.1 for styling and Bootstrap Icons for iconography, replacing previous dependencies. The new layout includes a responsive navigation bar, a redesigned footer with links to Telegram, Mastodon, X (Twitter), GitHub, and Stack Overflow, and a theme-aware favicon that switches between light and dark modes based on the user's system preference. Additionally, analytics tracking has been switched to Plausible via the Scientific Python server.

_web/pandas/\templates · high confidence

The pandas compatibility module has been reorganized into dedicated submodules (\_constants, \_cpu, \_optional, pickle\_compat, pyarrow) to improve maintainability. The minimum supported version for PyArrow has been raised to 16.0.0, and the optional dependency list in \_optional.py has been updated to reflect current requirements (e.g., python-calamine 0.4.0, openpyxl 3.1.5, SQLAlchemy 2.0.42). A new \_cpu module detects available CPU counts by checking OS affinity masks and cgroup quotas to prevent oversubscription during parallel I/O operations. Pickle compatibility logic has been consolidated to handle deserialization of older pandas versions (up to 1.3.5), including specific fixes for Timestamp timezone handling and deprecated index types. Additionally, a workaround has been added for a PyArrow 21.0.0 bug on Windows affecting null value filling.

pandas/compat · high confidence

The C extension source files in pandas/\_libs/src/datetime have been reorganized and optimized. New dedicated modules (date\_conversions.c, pd\_datetime.c) now handle low-level conversions between Python datetime objects, NumPy datetime64 structures, and ISO 8601 strings. This refactoring includes performance improvements for serializing timezone-aware datetimes to JSON, fixes for memory leaks and crashes during JSON serialization, and corrections for dropping seconds in sub-minute UTC offsets or non-nanosecond timedelta indices.

_pandas/\libs/src/datetime · high confidence

The string accessor implementation has been restructured to delegate string operations to the underlying array's methods (e.g., \Series.str.upper()\ now calls \Series.array.\_str\_upper()\). This change introduces \ObjectStringArrayMixin\ for object-dtype arrays and prepares the accessor to support Arrow-backed string arrays, ensuring that string methods are handled by the specific array type rather than a centralized accessor logic.

pandas/core/strings · high confidence

The pandas.util module has been reorganized into a private package (pandas.util) with a new \_\init\\.py that uses \\getattr\\_ to lazily import symbols like hash\_array, hash\_pandas\_object, Appender, Substitution, and cache\_readonly only when accessed, avoiding circular import errors. The public-facing utilities previously exposed at the top level are now accessed via this lazy-loading mechanism, while internal helpers such as \_decorators, \_exceptions, \_doctools, \_print\_versions, \_test\_decorators, \_tester, and \_validators remain as private submodules within the package.

pandas/util · high confidence

The Cython implementation of rolling window functions in \pandas/\_libs/window/aggregations.pyx\ has been refactored to use Kahan summation for \roll\_sum\ and \roll\_mean\. This change mitigates floating-point precision errors that can occur when summing large numbers of values, ensuring more accurate results for users relying on rolling statistics. The module also includes new type stubs (\aggregations.pyi\) and a Meson build configuration to compile these extensions.

_pandas/\libs/window · high confidence

The moments calculation in pandas now uses a two-pass algorithm via xsimd for improved performance and numerical stability, particularly for higher-order moments. This change includes a specific build fix for Apple Clang versions prior to 17 to ensure compatibility with older Xcode releases.

_pandas/\libs/include/pandas/simd · high confidence

The pandas sparse module has been reorganized to expose the core sparse types, SparseArray and SparseDtype, via the pandas.core.sparse.api submodule. This change consolidates the public API for sparse data structures, making these specific components explicitly available for import from this location, while the underlying implementation remains in the arrays module.

pandas/core/sparse · high confidence

The vendored ujson module (now exposed as pandas.\_libs.\_ujson) has been refactored to eliminate crashes and data-loss bugs in DataFrame and Series JSON operations. The C implementation now correctly handles timezone-aware datetimes, non-nanosecond datetime64/timedelta units, and large unsigned integer scalars, preventing the segfaults and OverflowErrors that previously occurred with these types. Additionally, the JSON decoder has been hardened to properly report parse errors with position information instead of failing silently or crashing, ensuring that invalid JSON input raises a clear ValueError.

_pandas/\libs/src/vendored/ujson/python · high confidence

The Styler rendering engine now uses dedicated Jinja2 template files (html.tpl, latex.tpl, string.tpl, typst.tpl, and their sub-templates) to generate HTML, LaTeX, plain text, and Typst output. This change introduces structured templates that support conditional styling, caption handling, longtable environments, and visibility filtering, replacing the previous inline string generation logic with a modular, extensible template system.

pandas/io/formats/templates · high confidence

The vendored NumPy datetime source files in pandas/\_libs/src/vendored/numpy have been updated to fix several critical issues in date-time parsing and conversion. This includes correcting the handling of sub-picosecond datetime64 conversions, fixing undefined behavior and potential segfaults related to include order and pointer access, and resolving overflow issues in Period and timestamp conversions. Additionally, the ISO 8601 parser now correctly handles negative years (BC dates) and prevents reading from indeterminate pointers on negative lengths, ensuring more robust and accurate datetime operations for users.

_pandas/\libs/src/vendored/numpy · high confidence

The vendored khash library in pandas/\_libs/include/pandas/vendored/klib has been updated to include khash.h and khash\_python.h. This update introduces Python memory allocation tracing (via PyTraceMalloc\_Track) for the hash table allocators, ensuring memory usage is tracked by Python's memory profiler. It also adds support for complex number types (khcomplex64\_t and khcomplex128\_t) with dedicated hash and equality functions, and defines specialized hash maps and sets for float32, float64, complex64, and complex128 types. The library now handles NaN values consistently by treating all NaNs as equal for hashing purposes, and includes improvements to hash functions for floating-point numbers to reduce collisions.

_pandas/\libs/include/pandas/vendored/klib · high confidence

The vendored ujson library in pandas/\_libs/src/vendored/ujson/lib has been updated to address stability issues and improve code quality. This update includes fixes for segfaults caused by unchecked C-API returns in ujson\_dumps and resolves a buffer overflow issue when serializing deeply nested values or escaped strings. Additionally, the library source files have been cleaned up to address various warnings and static analysis findings, ensuring more robust JSON encoding and decoding behavior for users.

_pandas/\libs/src/vendored/ujson/lib · high confidence

The \pandas/util/version\ module has been updated to a newer version of the vendored \packaging\ library (changeset 24e5350b2ff3c5c7a36676c2af5f2cb39fd1baf8). This update brings the internal version parsing logic, including the \Version\ class, \InvalidVersion\ exception, and associated type aliases, in line with the upstream \pypa/packaging\ implementation. Users relying on pandas' internal version comparison or parsing behavior may see subtle changes in how edge-case version strings are handled or validated, consistent with the upstream \packaging\ library's evolution.

pandas/util/version · high confidence

1 commit adding/updating tests in pandas/tests/io/data/pickle, pandas/tests/io/data/spss; 6 commits adding/updating tests in pandas/tests/io/data/legacy\_pickle; Add ASV benchmarks for extension arrays and indexing engines; Add comprehensive test suite for MultiIndex; Add comprehensive test suite for pandas reshape operations; Add dedicated test suite for FloatingArray; Add index test suite and fixtures; Add regression tests for MultiIndex indexing behavior; Add test data fixtures for SQL and HTML parsing tests; Add test data fixtures for parser tests; Add tests for JSON ExtensionArray; Add tests for anti-join and cross-merge operations; Add tests for indexing utilities and regression coverage; Add tests for the Decimal extension array implementation; Add tests for the pandas.api namespace structure; Added Stata test data fixtures for formats 3, 5, and 6; Added XML test fixtures for I/O validation; Added base class Index tests; Added benchmarks for Series.isin performance; Added comprehensive test suite for BooleanArray; Added comprehensive test suite for Categorical arrays; Added comprehensive test suite for SparseArray and sparse accessors; Added comprehensive test suite for holiday calendar and observance logic; Added comprehensive test suite for the DataFrame Interchange Protocol; Added consistency tests for rolling, expanding, and EWM window moments; Added dedicated test suite for nullable integer arrays; Added expected HTML output fixtures for DataFrame formatting tests; Added test coverage for DataFrame constructors; Added test coverage for IntervalIndex indexing behavior; Added test coverage for dtype casting and promotion logic; Added test data for pandas cut function; Added test fixtures for JSON I/O; Added test fixtures for SAS file parsing edge cases; Added test infrastructure for custom Date extension arrays; Added test suite for IntervalArray and IntervalIndex; Added test suite for ODF (ODS) Excel engine; Added test suite for Series accessors; Added test suite for String dtype Index behavior; Added test suite for masked array arithmetic, indexing, and PyArrow compatibility; Added test suite for scripts and CI automation logic; Added tests for CSV parser usecols functionality; Added tests for Copy-on-Write behavior in Index types; Added tests for DatetimeArray constructors, reductions, and Arrow integration; Added tests for DatetimeIndex arithmetic, construction, and set operations; Added tests for DatetimeIndex, TimedeltaIndex, and PeriodIndex methods; Added tests for NumpyExtensionArray indexing and dtype behavior; Added tests for Period scalar arithmetic, frequency conversion, and construction edge cases; Added tests for PeriodIndex construction, formatting, and behavior; Added tests for PeriodIndex methods; Added tests for SAS file reading and byte-swap utilities; Added tests for StringDtype concatenation and array construction; Added tests for Timedelta unit conversion and rounding methods; Added tests for TimedeltaArray construction, reductions, and accumulators; Added tests for TimedeltaIndex methods; Added tests for Timestamp method behaviors; Added tests for UUID ExtensionArray plotting support; Added tests for common CSV parsing scenarios; Added tests for console encoding detection and CSS resolution; Added tests for generic DataFrame and Series behavior; Added tests for groupby transform with Numba engine and list/dict arguments; Added tests for numexpr blocked-version handling in eval/query; Added tests for object-dtype Index behavior; Added tests for preserving custom attributes during merge and concat operations; Added tests for scalar Interval arithmetic, construction, containment, and overlap behavior; Added tests for scalar NA, NaT, Timestamp, and Timedelta behavior; Added tests for the List extension array implementation; Added tests for the configuration and localization subsystems; Added tests for the web release preprocessor; Added tests for time series frequency inference and resolution logic; Added tests for tslibs internal components; CategoricalIndex test suite reorganization; Comprehensive test suite for CSV parsing engines; Comprehensive test suite for pandas arithmetic operations; Consolidated JSON I/O test suite; Consolidated test suite for DataFrame and Series apply, agg, and transform operations; Consolidated test suite for pandas Extension Arrays; Consolidated test suite for pandas conversion tools; Consolidated test suite for pandas reduction operations; Consolidated test suite for pd.concat and DataFrame.append; Expanded test coverage for Styler formatting, highlighting, and export capabilities; Expanded test coverage for pandas testing utilities; Expanded test coverage for rolling, expanding, and EWM window operations; Expanded test coverage for string accessor methods across multiple string dtypes; GroupBy aggregate tests reorganized into dedicated module; GroupBy method tests relocated to dedicated module; Initial test suite for pandas XML I/O; Initial test suite for pandas dtype system; New ASV benchmarks for IO operations; New ASV benchmarks for tslibs performance tracking; New base test suite for validating ExtensionArray implementations; New dedicated test suite for RangeIndex; New dedicated test suite for TimedeltaIndex; New dedicated test suite for numeric Index behavior; New test suite for CSV parser dtype handling; New test suite for Copy-on-Write behavior; New test suite for DatetimeIndex methods; New test suite for PeriodArray functionality; New test suite for pandas time-series offsets; New test suite for scalar Timedelta operations; New unit tests for pandas internal libraries; Organize Series method tests into dedicated files; Organized Series indexing test suite; Organized Timestamp scalar tests into dedicated submodules; Refactor IntervalIndex tests into modular files; Refactored PyTables test suite with modern fixtures; Reorganization of tseries test suite; Reorganize DataFrame method tests into dedicated files; Reorganized DataFrame indexing tests into a dedicated module; Resample test suite reorganization and expansion; Restructure and consolidate groupby test suite; Restructure and expand extension array test suite; Restructure base test suite into modular files; Restructured and expanded plotting test suite; Split DataFrame plotting tests into modular subdirectory; Tests for Series Arrow PyCapsule Interface; Tests for the pseudo-public internals API.

The project has switched its build backend from setuptools to Meson (via meson-python), requiring build tools like meson\>=1.2.3, meson-python\>=0.19.0, and Cython\>3.1.0. Minimum versions for core and optional dependencies have been raised across the board, including NumPy\>=2.0.2 (with specific requirements for Python 3.14+), pyarrow\>=16.0.0, SQLAlchemy\>=2.0.42, and matplotlib\>=3.10.5. The build system now also includes the xsimd library for SIMD detection, and Python 3.11 is the new minimum supported version.

(dependencies) · high confidence

The build system now uses fast\_float version 8.2.10 for string-to-number conversions, replacing previous bespoke parsers to improve performance. Additionally, the xsimd library (version 14.2.0) has been added as a new dependency to enable SIMD acceleration for numerical operations.

subprojects · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 68 → 62 (-6.4)
• Rubric changed (rubric-2026.09.15 → rubric-2026.09.18) — scores are not directly comparable.

• Code Health 97 → 78 (-18.9)
• Architecture 99 → 89 (-10.0)
• Maturity 72 → 73 (+0.2)
• Readiness 67 → 52 (-14.5)
• Security 73 → 73 (+0.0)
• Accessibility 63 → 63 (+0.0)
• Performance 100 (new)

• Documentation: no project overview (README.md)
• Off-boarding risk: anonymized user #1
• Off-boarding risk: anonymized user #2
• TodoComment (pandas/core/groupby/groupby.py)

• AppendableFrameTable.read (cognitive 39) (pandas/io/pytables.py)
• AppendableFrameTable.read (cyclomatic 21) (pandas/io/pytables.py)
• AppendableTable.delete (cognitive 19) (pandas/io/pytables.py)
• AppendableTable.write_data (cognitive 17) (pandas/io/pytables.py)
• Apply.compute_dict_like (cognitive 19) (pandas/core/apply.py)
• Apply.transform (cognitive 16) (pandas/core/apply.py)
• Apply.wrap_results_dict_like (cognitive 21) (pandas/core/apply.py)
• ArrowDtype.type (cyclomatic 23) (pandas/core/dtypes/dtypes.py)
• ArrowExtensionArray.__array_ufunc__ (cognitive 19) (pandas/core/arrays/arrow/array.py)
• ArrowExtensionArray.getitem (cognitive 25) (pandas/core/arrays/arrow/array.py)
• ArrowExtensionArray.getitem (cyclomatic 22) (pandas/core/arrays/arrow/array.py)
• ArrowExtensionArray.setitem (cognitive 24) (pandas/core/arrays/arrow/array.py)
• ArrowExtensionArray.setitem (cyclomatic 19) (pandas/core/arrays/arrow/array.py)
• ArrowExtensionArray._box_pa_array (cognitive 60) (pandas/core/arrays/arrow/array.py)
• ArrowExtensionArray._box_pa_array (cyclomatic 36) (pandas/core/arrays/arrow/array.py)
• ArrowExtensionArray._box_pa_scalar (cognitive 24) (pandas/core/arrays/arrow/array.py)
• ArrowExtensionArray._box_pa_scalar (cyclomatic 21) (pandas/core/arrays/arrow/array.py)
• ArrowExtensionArray._cast_pointwise_result (cognitive 36) (pandas/core/arrays/arrow/array.py)
• ArrowExtensionArray._cast_pointwise_result (cyclomatic 35) (pandas/core/arrays/arrow/array.py)
• ArrowExtensionArray._cmp_method (cognitive 30) (pandas/core/arrays/arrow/array.py)
• …and 952 more

• 73 commits — 45 feature/other, 28 fixes

• doc/source — 19 commits
• pandas/tests — 19 commits
• pandas/core — 15 commits
• pandas/_libs — 5 commits
• pandas/plotting — 4 commits
• pandas/io — 3 commits
• web/pandas — 2 commits
• (root) — 1 commit
• .github/actions — 1 commit
• .github/release_notes — 1 commit
• .github/workflows — 1 commit
• asv_bench/benchmarks — 1 commit
• pandas/_typing.py — 1 commit

• fix: BUG/CLN: Remove _agg_general and finalize more groupby methods (#69438)
• fix: BUG: Fix RangeIndex.symmetric_difference skips required sort when one side is empty (#68358)
• fix: BUG: Fix online ewm ignoring decay for single-column frames (#68278)
• fix: BUG: Fix sort_values and factorize with Arrow null type (GH#54908) (#69463)
• fix: BUG: MultiIndex reports the wrong problem for an over-long key (GH#45762) (#68922)
• fix: BUG: Timestamp rejected fold with positional date components (GH#52117) (#69134)
• fix: BUG: agg with an empty list or dict raises "No objects to concatenate" (GH#39609) (#69496)
• fix: BUG: allow arbitrary objects as to_replace in replace (GH#36522) (#69453)
• fix: BUG: don't trim Styler.to_latex output to render limits (GH#68310) (#69719)
• fix: BUG: hash_pandas_object on DataFrame with no columns and index=False (GH#24318) (#69489)
• fix: BUG: honor date_format for object-dtype datetimes in to_csv (GH#27306) (#69624)
• fix: BUG: honor display.precision for float index names (GH#25917) (#69640)
• fix: BUG: keep MultiIndex level names in sort_index with key (#69450)
• fix: BUG: keep left/right index dtype in Index.join on mismatched dtypes (GH#63371) (#69408)
• fix: BUG: merge on pyarrow keys with nulls fails on 32-bit (GH#57523) (#69784)
• fix: BUG: name the value_labels key in the non-unique Stata labels error (GH#54590) (#69507)
• fix: BUG: no extra leading space for object columns in to_string formatters (GH#26002) (#69627)
• fix: BUG: numexpr results use a nonstandard int64 type, breaking select_dtypes (GH#17945) (#69427)
• fix: BUG: raise TypeError, not an empty NotImplementedError, for unsupported groupby aggregations (#69717)
• fix: BUG: read_parquet/to_parquet fall back to fsspec when pyarrow can't build the filesystem (GH#58078) (#69494)
• …and 53 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

pandas-dev/pandas was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.