Pandas Cheat Sheet

Setup and Core Structures

Install pandas and create its primary data structures.

Install and Import

Install pandas and inspect the active version.

python

python -m pip install pandas

import pandas as pd

🟢 Use a virtual environment for each project

📌 Import pandas with the conventional pd alias

🔍 show_versions reports dependency details

🎯 Pin a compatible major version in applications

installimportversion

Series

Create a labeled one-dimensional array.

python

scores = pd.Series([88, 92, 79], name="score")

📌 A Series has values plus an Index

💡 Name Series used in joins or reports

🔍 Labels do not need to be integers

🎯 Set dtype explicitly when it matters

seriesindexdtype

DataFrame

Create a labeled two-dimensional table.

python

df = pd.DataFrame({
    "name": ["Ana", "Ben"],
    "score": [88, 92],
})

📌 Columns can have different dtypes

💡 Build from records for API-style data

🔍 The Index labels rows independently of position

🎯 Keep column names consistent at ingestion

dataframerecordsindex

Inspect Data

Understand shape, columns, types, and distributions.

Preview and Structure

Inspect rows and structural metadata.

python

df.head()
df.tail()
df.shape

⚡ head() returns five rows by default

📌 info() reveals null counts and dtypes

🔍 deep=True includes object-backed memory

🎯 Inspect data immediately after loading

inspectinfoshape

Summary Statistics

Summarize numeric and categorical columns.

python

df.describe()

📌 describe() defaults to numeric columns

💡 include="all" summarizes mixed data

🔍 value_counts can include missing values

🎯 Check distributions before choosing transformations

describestatisticsvalue-counts

Select Rows and Columns

Select data by label, position, or scalar location.

Columns and Slices

Select one or multiple columns and row slices.

python

names = df["name"]
subset = df[["name", "score"]]

📌 One column returns a Series

🔍 A column list returns a DataFrame

💡 select_dtypes avoids hard-coded column lists

⚠️ Plain slices apply to rows, not columns

columnsselectionslice

loc and iloc

Select by labels or integer positions.

python

df.loc["ana", "score"]
df.iloc[0, 1]

📌 loc uses labels and iloc uses positions

🔍 Label slices include the stop label

⚡ at and iat optimize scalar access

🎯 Use explicit indexers in reusable code

locilocselection

Filter and Query

Keep rows that match one or more conditions.

Boolean Filtering

Filter rows with vectorized conditions.

python

high = df[df["score"] >= 90]

📌 Wrap each combined condition in parentheses

⚡ Use & and | instead of and and or

💡 between and isin improve readability

🔍 String filters use the str accessor

filterbooleanisin

query

Filter with a readable expression string.

python

minimum = 90
high = df.query("score >= @minimum")

💡 Prefix Python variables with @

📌 Column names become expression variables

⚠️ Avoid untrusted input in query strings

🔍 Backticks escape columns with spaces

queryfilterexpressions

Assign and Index

Update values safely under pandas 3 Copy-on-Write.

Safe Assignment

Assign in one step with loc.

python

df.loc[df["score"] < 80, "status"] = "review"

📌 Chained assignment cannot update data in pandas 3

🎯 Use loc for conditional assignment

💡 assign supports readable method chains

🔍 Derived objects behave as copies under CoW

assignmentloccopy-on-write

Index and MultiIndex

Set meaningful row labels and hierarchical indexes.

python

df = df.set_index("id")
df = df.reset_index()

📌 Index values should identify rows predictably

💡 Sort a MultiIndex before label slicing

🔍 reset_index moves labels back to columns

🎯 Keep normal columns unless hierarchy adds value

indexmultiindexlabels

Missing and Duplicate Data

Detect, fill, remove, and deduplicate imperfect data.

Missing Values

Find and handle missing values by column.

python

df.isna().sum()
df = df.dropna(subset=["id"])

📌 Use isna() instead of comparing with NaN

💡 Fill values with domain-appropriate rules

🔍 dropna can target columns or thresholds

⚠️ Do not hide missingness without analysis

missing-datafillnadropna

Duplicates

Detect and remove repeated records.

python

df.duplicated().sum()
df = df.drop_duplicates()

📌 Define duplicates using business keys

💡 keep=False reveals every duplicate row

🔍 Sort before keeping the latest record

🎯 Inspect duplicates before deleting them

duplicatescleaningdrop-duplicates

Data Types

Inspect and convert columns to suitable pandas dtypes.

Convert Types

Convert columns with explicit error handling.

python

df["score"] = pd.to_numeric(df["score"], errors="coerce")

📌 Conversion errors can become missing values

💡 convert_dtypes selects nullable dtypes

🔍 Use utc=True for consistent timestamps

⚠️ Validate rows coerced to missing values

dtypesconversionnullable

pandas 3 String Dtype

Use the default dedicated string dtype safely.

python

names = pd.Series(["Ana", "Ben", None], dtype="str")

📌 pandas 3 infers str instead of object

🔍 Missing strings use NaN in the default str dtype

⚠️ Non-string assignment raises TypeError

🎯 Use is_string_dtype for compatibility checks

stringsdtypepandas-3

Text and Categories

Transform strings and optimize repeated labels.

Vectorized String Methods

Clean and extract text without Python loops.

python

df["name"] = df["name"].str.strip().str.title()

⚡ str methods operate on entire columns

📌 Missing values usually propagate safely

💡 extract uses capture groups

🎯 Prefer vectorized methods over apply for text

stringsregexcleaning

Categorical Data

Represent repeated finite labels efficiently.

python

df["size"] = df["size"].astype("category")

💡 Categories reduce memory for repeated labels

📌 Ordered categories support meaningful sorting

⚠️ Add a category before assigning a new label

🔍 cat accessor manages category metadata

categorydtypememory

Dates and Time

Parse, extract, group, and shift time-based data.

Datetime Operations

Parse timestamps and use the dt accessor.

python

df["created_at"] = pd.to_datetime(df["created_at"], utc=True)

📌 Parse timestamps before date operations

💡 Store shared timestamps in UTC

🔍 dt exposes vectorized date components

⚠️ Period conversion drops timezone information

datetimetimezoneresample

Timedeltas and Shifts

Calculate durations and compare adjacent rows.

python

df["duration"] = df["ended_at"] - df["started_at"]

📌 Sort before shift or diff calculations

💡 Timedelta arithmetic preserves units

🔍 shift aligns prior values by index

🎯 Name derived time units explicitly

timedeltashiftdiff

Operations and Sorting

Apply vectorized calculations and order results.

Vectorized Operations

Calculate with columns and align by labels.

python

df["total"] = df["price"] * df["quantity"]

⚡ Vectorized expressions avoid Python loops

📌 Arithmetic aligns objects by index labels

💡 where chooses values from a condition

🔍 rank supports several tie strategies

vectorizationwhererank

Sort and Reorder

Sort rows or control column order.

python

df = df.sort_values("score", ascending=False)

📌 Each sort key can have its own direction

💡 na_position controls missing placement

🔍 sort_index orders row or column labels

🎯 Assign or chain methods to keep the result

sortingordercolumns

Group and Aggregate

Split data into groups and calculate summaries.

GroupBy Aggregation

Calculate one or more summaries per group.

python

summary = df.groupby("team")["score"].mean()

📌 GroupBy follows split, apply, combine

💡 Named aggregation creates clear columns

🔍 as_index=False keeps group keys as columns

⚡ Prefer built-in aggregations over lambdas

groupbyaggregationsummary

Transform and Filter Groups

Return group-aligned values or remove whole groups.

python

df["team_avg"] = df.groupby("team")["score"].transform("mean")

📌 transform returns the original row shape

🔍 filter evaluates and keeps whole groups

⚡ Built-in transforms are usually faster

🎯 Use agg when the output should be summarized

groupbytransformfilter

Combine Data

Join related tables and concatenate compatible objects.

Merge and Join

Combine tables using matching key columns.

python

result = orders.merge(customers, on="customer_id", how="left")

📌 Choose join type based on required rows

💡 validate catches unexpected key cardinality

🔍 indicator shows where each row matched

⚠️ Duplicate keys can multiply rows

mergejoinkeys

Concatenate

Stack rows or align objects by columns.

python

all_rows = pd.concat([january, february], ignore_index=True)

📌 axis="index" stacks rows by default

💡 ignore_index creates a fresh row index

🔍 keys records each object source

⚠️ Column labels align and may create missing data

concatcombinealignment

Reshape Data

Convert between wide, long, nested, and summary forms.

Pivot and Melt

Reshape long data to wide form and back.

python

wide = df.pivot(index="date", columns="metric", values="value")

📌 pivot requires unique index-column pairs

💡 pivot_table aggregates duplicate pairs

🔍 melt converts wide columns into row values

🎯 Use long form for most grouped analysis

pivotmeltreshape

Explode and Normalize

Expand list values and flatten nested records.

python

rows = df.explode("tags", ignore_index=True)

📌 explode creates one row per list element

💡 json_normalize flattens nested records

🔍 Empty lists can produce missing values

🎯 Normalize only fields needed for analysis

explodejson-normalizereshape

Windows and Time Series

Calculate rolling, expanding, and indexed time metrics.

Rolling and Expanding

Calculate statistics across moving or cumulative windows.

python

df["moving_avg"] = df["value"].rolling(7, min_periods=1).mean()

📌 Sort time data before window operations

💡 Offset windows handle uneven timestamps

🔍 expanding uses all observations to date

🎯 Set min_periods for early-window behavior

rollingwindowtime-series

Import and Export

Read and write common tabular storage formats.

CSV, JSON, and Excel

Load and save common text and spreadsheet formats.

python

df = pd.read_csv("input.csv")
df.to_csv("output.csv", index=False)

📌 Select columns and dtypes during ingestion

💡 Parse dates while reading when practical

🔍 Excel support requires an engine package

⚠️ JSON orientation must match the consumer

csvjsonexcelio

Parquet, SQL, and Chunks

Use efficient storage and process large inputs.

python

df.to_parquet("data.parquet", index=False)
df = pd.read_parquet("data.parquet")

💡 Parquet preserves types and compresses columns

📌 Parameterize dynamic SQL outside this example

🔍 Chunking limits peak memory for text files

⚠️ Database writes need transaction planning

parquetsqlchunksio

Plotting and Performance

Visualize results and keep transformations efficient.

Quick Plots

Create exploratory plots from Series and DataFrames.

python

df.plot(x="date", y="value", kind="line")

🟢 Plotting requires Matplotlib by default

💡 Use plots for exploration, not final dashboards

🔍 Methods return a Matplotlib Axes object

🎯 Label axes and units clearly

plottingmatplotlibvisualization

Copy-on-Write and Performance

Write predictable pandas 3 transformations.

python

df.loc[df["score"] < 0, "score"] = 0

📌 Copy-on-Write is the only mode in pandas 3

⚠️ Chained assignment cannot modify the original

⚡ Vectorized operations usually beat row-wise apply

🎯 Measure memory and runtime before optimizing

copy-on-writeperformancepandas-3

Pandas

Table of Contents

Other Python Sheets

Setup and Core Structures

Inspect Data

Select Rows and Columns

Filter and Query

Assign and Index

Missing and Duplicate Data

Data Types

Text and Categories

Dates and Time

Operations and Sorting

Group and Aggregate

Combine Data

Reshape Data

Windows and Time Series

Import and Export

Plotting and Performance

More Python Cheat Sheets

Django

FastAPI

Flask