Lerner Consulting BlogReuven Lerner1 min readintro
np.nan in Pandas: Why missing values break comparisons
Summary
The post explains that np.nan is a special floating‑point value that propagates through arithmetic and never compares equal, even to itself. It shows how this breaks naïve equality checks in pandas and suggests using pandas' isna/notna utilities instead.
- np.nan is a float that propagates through arithmetic operations, yielding nan for any calculation involving it.
- Comparisons with np.nan always return False, including np.nan == np.nan.
- Pandas' standard equality checks (==) will treat rows with nan as unequal, which can break joins, filters, and deduplication.
- Use pandas.isna() or pandas.notna() to reliably detect missing values instead of == or != operators.
Anyone cleaning or analyzing data with pandas needs to handle nan correctly, or they’ll get silent bugs in filters, joins, and aggregations.
5/10
