<DataInsights />
  • 🏠 Home
  • πŸ“Š SQL
  • 🐍 Python
  • πŸ“ˆ Power BI
  • πŸ“— Excel
  • πŸ’Ό Career
  • 🎯 Interview Q&A
  • πŸ“ Case Study
  • πŸ“₯ Downloads
  • πŸš€ My Portfolio
<DataInsights />

Practical Data Analytics tutorials covering SQL, Python, Power BI, Excel and career guidance for aspiring analysts β€” 100% free.

Topics

  • SQL Tutorials
  • Python Guide
  • Power BI
  • Excel Tips
  • Career Guide

Quick Links

  • πŸ› οΈ All Tools
  • πŸ—“οΈ Archive
  • πŸ“¬ Contact
  • πŸ” Search
  • Portfolio
  • Kaggle
  • GitHub

Legal & Info

  • About
  • Contact
  • Privacy Policy
  • Disclaimer
  • Terms & Conditions
  • DMCA
  • Sitemap
Copyright © 2026 Data Insights by Jatin Kumar. All Rights Reserved.Built with ❀️ for Data Analysts
Home/Python/Statistics And Aggregation Functions in Pandas...

Statistics And Aggregation Functions in Pandas

A
August 3, 2026 Jatin Kumar 27 min read Python
Data Insights Masterclass β€” Part 9

Statistics & Aggregation Functions in Pandas: Complete Guide

Data ka asli insight tab milta hai jab aap numbers ko summarize, group, aur aggregate karein. Sikhiye 10 powerful statistical aur aggregation functions jo raw data ko actionable business intelligence mein convert karte hain.

πŸ“‘ Is Masterclass Guide Mein Aap Kya Sikhenge:

Data summarization aur statistical analysis ke 10 professional tools:

  • Central Tendency: mean(), median(), mode() β€” Data ka center kahan hai
  • Spread Measures: std(), var() β€” Data kitna spread/scattered hai
  • Aggregation Basics: sum(), count(), min(), max() β€” Quick column summaries
  • Group Intelligence: groupby() β€” Category-wise data summarization
  • Multi-Aggregation: agg() β€” Multiple statistics ek saath nikalna
  • Cross-Tabulation: pivot_table() β€” Excel-style summary tables banana

1. mean() β€” Average / Arithmetic Mean

πŸ” Kya Hai: mean() column ke sabhi numerical values ka arithmetic average calculate karta hai β€” sabhi values ka sum divide by total count. Yeh data ka "center point" batata hai aur sabse commonly used central tendency measure hai.

🎯 Kyu Use Hota Hai: Business decisions mein averages critical hain β€” average salary kitni hai, average order value kya hai, average delivery time kitna hai. Mean se baseline expectations set hoti hain aur comparisons hote hain β€” "Is employee ki salary average se zyada hai ya kam?".

πŸ’‘ Kab Use Hota Hai: KPI calculations (Average Order Value, Average Revenue Per User), missing value imputation, baseline comparisons, performance benchmarks, aur jab data approximately symmetric/normal distributed ho.

πŸ’» Real-World Code Examples:

Example 1: Employee dataset mein overall average salary aur department-wise average salary nikalna.

import pandas as pd
employees = pd.read_csv("employees.csv")
# Overall average salary
print(f"Average Salary: β‚Ή{employees['Salary'].mean():,.2f}")
# Department-wise average
dept_avg = employees.groupby("Department")["Salary"].mean().sort_values(ascending=False)
print(dept_avg)

Example 2: E-commerce mein Average Order Value (AOV) calculate karna β€” key business KPI.

# Overall AOV
aov = ecommerce["Revenue"].mean()
print(f"Average Order Value: β‚Ή{aov:,.2f}")
# Monthly AOV trend
ecommerce["Month"] = pd.to_datetime(ecommerce["OrderDate"]).dt.month
monthly_aov = ecommerce.groupby("Month")["Revenue"].mean().round(2)
print(monthly_aov)

πŸ“Š Expected Output:

# Average Salary: β‚Ή68,450.25

# Department-wise Average:
# Department   Salary
# IT           82500.00
# Finance      75200.50
# Marketing    62400.00
# HR           55800.75
# Operations   48200.00

# Average Order Value: β‚Ή2,450.75
# Monthly AOV:
# Month 1:  2320.50
# Month 12: 3890.40  # Festive season spike!

βœ… Best Practices:

  • Mean outliers se heavily influenced hota hai β€” ek CEO ki β‚Ή50 lakh salary puri company ki average salary distort kar degi. Skewed data mein median zyada reliable hai.
  • mean() by default NaN values skip karta hai (skipna=True). Agar NaN include karke mean chahiye (rarely needed) toh skipna=False pass karein.
  • Missing value imputation mein mean use karne se pehle check karein ki data normally distributed hai β€” skewed data ke liye median imputation better hai.

πŸ’¬ Crack the Interview:

Q1: Mean aur Median mein kab kaunsa use karein?
Ans: Symmetric/normal data mein mean use karein. Skewed data (income, house prices, transaction amounts) mein median use karein kyunki yeh outliers se affect nahi hota. Dono ka difference bada ho toh data skewed hai.

Q2: mean() entire DataFrame par call karne se kya hota hai?
Ans: df.mean() sabhi numerical columns ka mean ek Series mein return karta hai. Non-numeric columns automatically skip hote hain. df.mean(numeric_only=True) explicitly specify karna Pandas 2.0+ mein recommended hai.

Q3: Weighted mean kaise calculate karein Pandas mein?
Ans: Direct weighted mean function nahi hai. Manually karein: (df['Value'] * df['Weight']).sum() / df['Weight'].sum() ya numpy use karein: np.average(df['Value'], weights=df['Weight']).

2. median() β€” Middle Value (Robust Central Tendency)

πŸ” Kya Hai: median() data ko sorted order mein rakhne ke baad exact middle value return karta hai. Even count mein do middle values ka average hota hai. Yeh outlier-resistant central tendency measure hai jo skewed data mein mean se zyada accurate picture deta hai.

🎯 Kyu Use Hota Hai: Income data, property prices, transaction amounts β€” yeh sab highly right-skewed hote hain jahan kuch extreme values mean ko distort karte hain. Median "typical" value batata hai β€” "typical Indian household income" median se better represent hota hai, mean se nahi.

πŸ’‘ Kab Use Hota Hai: Skewed data analysis (salaries, prices, amounts), outlier-resistant imputation, real estate price analysis, median household income reports, aur jab bhi "typical" value chahiye na ki "average" value.

πŸ’» Real-World Code Examples:

Example 1: Salary data mein mean vs median compare karke skewness detect karna.

salary_mean   = employees["Salary"].mean()
salary_median = employees["Salary"].median()
print(f"Mean Salary:   β‚Ή{salary_mean:,.2f}")
print(f"Median Salary: β‚Ή{salary_median:,.2f}")
print(f"Difference:    β‚Ή{salary_mean - salary_median:,.2f}")
if salary_mean > salary_median * 1.1:
    print("⚠️ Data is RIGHT-SKEWED β€” use median for imputation!")

Example 2: Missing salary values ko median se impute karna (outlier-resistant approach).

# Group-wise median imputation (best practice)
print(f"Missing Salaries Before: {employees['Salary'].isna().sum()}")
employees["Salary"] = employees.groupby("Department")["Salary"].transform(
    lambda x: x.fillna(x.median())
)
print(f"Missing Salaries After:  {employees['Salary'].isna().sum()}")

πŸ“Š Expected Output:

# Mean Salary:   β‚Ή68,450.25
# Median Salary: β‚Ή62,000.00
# Difference:    β‚Ή6,450.25
# ⚠️ Data is RIGHT-SKEWED β€” use median for imputation!

# Missing Salaries Before: 280
# Missing Salaries After:  0    # All filled with
group median!

βœ… Best Practices:

  • Mean - Median ka difference bada hai toh data skewed hai. Mean > Median = right skewed (positive skew). Mean < Median = left skewed (negative skew).
  • Missing value imputation mein group-wise median (department-wise, category-wise) overall median se always better hai kyunki groups ka distribution different hota hai.
  • Median even count mein do middle values ka average deta hai β€” yeh existing value nahi bhi ho sakta dataset mein, lekin statistically correct representation hai.

πŸ’¬ Crack the Interview:

Q1: Median ka time complexity kya hai aur mean se slow kyun hai?
Ans: Mean O(n) hai β€” sirf sum aur divide. Median O(n log n) hai kyunki sorting required hai. Lekin Pandas internally optimized partial sort algorithm use karta hai (introselect) jo full sort se faster hai, isliye practical difference negligible hai.

Q2: Median imputation mean imputation se kab better hai?
Ans: Jab data mein outliers ya significant skew ho. Mean imputation outliers ko aur reinforce karti hai jabki median imputation data distribution ka actual center preserve karti hai. Salary, prices, transaction amounts β€” sab mein median imputation preferred hai.

Q3: quantile(0.5) aur median() mein kya difference hai?
Ans: Dono same value return karte hain β€” 50th percentile = median. Lekin quantile() zyada flexible hai kyunki koi bhi percentile nikal sakte hain: quantile(0.25) for Q1, quantile(0.75) for Q3. median() sirf 50th percentile ka shortcut hai.

3. mode() β€” Most Frequent Value

πŸ” Kya Hai: mode() column mein sabse frequently occurring value (most common value) return karta hai. Yeh categorical data ke liye primary central tendency measure hai. Agar multiple values same highest frequency rakhti hain toh sab return hoti hain (multi-modal).

🎯 Kyu Use Hota Hai: Categorical columns (Department, City, Gender) mein mean/median meaningless hai β€” mode batata hai ki sabse common category kaunsi hai. Missing categorical values impute karne ke liye mode sabse standard approach hai.

πŸ’‘ Kab Use Hota Hai: Categorical missing value imputation, most popular product/category finding, default value assignment, survey response analysis, aur market research mein dominant preference identify karne ke liye.

πŸ’» Real-World Code Examples:

Example 1: Employee dataset mein sabse common department, city aur rating find karna.

print("Most Common Department:", employees["Department"].mode()[0])
print("Most Common City:", employees["City"].mode()[0])
print("Most Common Rating:", employees["Rating"].mode()[0])

Example 2: Missing categorical values ko mode se impute karna (group-wise approach).

# Missing City values fill with overall mode
city_mode = employees["City"].mode()[0]
employees["City"] = employees["City"].fillna(city_mode)
# Group-wise mode imputation (Department-wise City mode)
employees["City"] = employees.groupby("Department")["City"].transform(
    lambda x: x.fillna(x.mode()[0] if not x.mode().empty else "Unknown")
)

πŸ“Š Expected Output:

# Most Common Department: IT       (3200 employees)
# Most Common City: Mumbai          (2850 employees)
# Most Common Rating: 4             (3500 employees)

βœ… Best Practices:

  • mode() DataFrame return karta hai (Series nahi) kyunki multi-modal possible hai. Hamesha [0] index lagayein first mode access karne ke liye.
  • Group-wise mode mein empty group handle karein β€” agar kisi group mein koi non-null value nahi toh mode() empty hoga aur error aayega. Lambda mein error handling zaroori hai.
  • High cardinality columns mein mode meaningless ho sakta hai β€” agar top mode ki frequency bhi low hai toh "Unknown" category assign karna better hai.

πŸ’¬ Crack the Interview:

Q1: mode() DataFrame kyun return karta hai Series ki jagah?
Ans: Kyunki data multi-modal ho sakta hai β€” ek se zyada values same highest frequency rakh sakti hain. DataFrame multiple modes ko rows mein represent karta hai. [0] se first mode milta hai jo usually sufficient hota hai.

Q2: Numerical columns mein mean, median aur mode β€” kab kaunsa use karein?
Ans: Normal distribution: mean. Skewed distribution: median. Categorical/discrete data: mode. Bimodal distribution mein mode sabse useful hai kyunki woh dono peaks identify karta hai.

Q3: Mode imputation mein bias introduce hone ka risk kya hai?
Ans: Haan, significant risk hai. Agar 30% data missing hai aur sabko mode se fill karein toh mode category artificially dominant ho jayegi. Group-wise mode imputation se yeh bias reduce hota hai lekin completely eliminate nahi hota. ML mein iterative imputation better hai.

4. std() β€” Standard Deviation (Data Spread Measure)

πŸ” Kya Hai: std() Standard Deviation calculate karta hai β€” yeh measure karta hai ki data values mean se kitni door spread hain. Low std = values tightly clustered hain mean ke around. High std = values widely scattered hain. Yeh variance ka square root hota hai.

🎯 Kyu Use Hota Hai: Sirf average jaanna kaafi nahi β€” do companies ki average salary same ho sakti hai lekin ek mein sab 60K kamayein aur doosri mein kuch 20K kuch 1 Lakh. std() yeh inequality/variability quantify karta hai. Risk assessment, quality control, aur consistency measurement mein critical hai.

πŸ’‘ Kab Use Hota Hai: Z-Score outlier detection (Formula: (value-mean)/std), financial risk assessment (portfolio volatility), manufacturing quality control (tolerance limits), aur feature scaling (StandardScaler divides by std).

πŸ’» Real-World Code Examples:

Example 1: Department-wise salary consistency compare karna β€” konsa department mein zyada pay inequality hai.

# Overall salary spread
print(f"Salary Std: β‚Ή{employees['Salary'].std():,.2f}")
# Department-wise consistency comparison
dept_stats = employees.groupby("Department")["Salary"].agg(["mean", "std"]).round(2)
dept_stats["CV%"] = ((dept_stats["std"] / dept_stats["mean"]) * 100).round(1)
print(dept_stats.sort_values("CV%", ascending=False))

Example 2: E-commerce delivery time consistency check β€” standard delivery window within 1 std.

delivery_mean = ecommerce["Delivery_Days"].mean()
delivery_std  = ecommerce["Delivery_Days"].std()
print(f"Avg Delivery: {delivery_mean:.1f} days")
print(f"Std Dev: {delivery_std:.1f} days")
print(f"68% orders delivered between {delivery_mean-delivery_std:.1f} - {delivery_mean+delivery_std:.1f} days")

πŸ“Š Expected Output:

# Salary Std: β‚Ή28,500.80
# Department Consistency:
# Department   mean       std        CV%
# Marketing    62400.00   25800.50   41.3%  # Most unequal!
# IT           82500.00   22100.00   26.8%
# Finance      75200.50   18500.25   24.6%
# HR           55800.75   12400.00   22.2%  # Most consistent

# Avg Delivery: 4.2 days
# Std Dev: 1.8 days
# 68% orders delivered between 2.4 - 6.0 days

βœ… Best Practices:

  • Pandas std() by default ddof=1 use karta hai (sample standard deviation). Population std ke liye std(ddof=0) pass karein. ML aur statistics mein sample std (ddof=1) standard hai.
  • Different units/scales ke columns compare karne ke liye Coefficient of Variation (CV = std/mean * 100) use karein β€” yeh unit-independent comparison enable karta hai.
  • Normal distribution mein: 68% data within Β±1 std, 95% within Β±2 std, 99.7% within Β±3 std (Empirical Rule).

πŸ’¬ Crack the Interview:

Q1: ddof=0 aur ddof=1 ka practical difference kya hai?
Ans: ddof=0 population std deta hai (N se divide). ddof=1 sample std deta hai (N-1 se divide) β€” Bessel's correction kehte hain isse jo sample se population estimate karne ka bias correct karta hai. Analytics mein almost hamesha ddof=1 (default) use hota hai.

Q2: Coefficient of Variation (CV) kyun important hai std se zyada?
Ans: Std absolute value hai β€” β‚Ή20,000 std salary mein zyada hai lekin property price mein negligible hai. CV relative measure hai (percentage) jo different scales ke data fairly compare karta hai. 30% CV hamesha high variability indicate karega regardless of unit.

Q3: StandardScaler mein std() ka kya role hai?
Ans: StandardScaler formula: (X - mean) / std. Yeh z-score transformation hai jo data ko mean=0 aur std=1 kar deta hai. Isse sab features same scale par aa jaate hain jo gradient-based ML algorithms (Linear Regression, SVM, Neural Networks) ke liye critical hai.

5. var() β€” Variance (Squared Spread Measure)

πŸ” Kya Hai: var() Variance calculate karta hai β€” yeh mean se har value ke deviation ka squared average hai. Mathematically: var = stdΒ². Variance data ki variability ka fundamental measure hai jo statistical tests, ANOVA, aur ML algorithms mein directly use hota hai.

🎯 Kyu Use Hota Hai: Variance statistical modeling ka backbone hai β€” ANOVA (group comparison) mein within-group vs between-group variance compare hoti hai. PCA mein maximum variance wale components select hote hain. Low variance features ML models mein useless hote hain aur remove kiye jaate hain.

πŸ’‘ Kab Use Hota Hai: ANOVA statistical tests, PCA (Principal Component Analysis), feature selection (VarianceThreshold), portfolio risk calculation (financial variance), aur statistical model assumptions verification mein.

πŸ’» Real-World Code Examples:

Example 1: Low variance features identify karke ML model se remove karna.

# Calculate variance of all numeric columns
numeric_cols = employees.select_dtypes(include=["number"]).columns
variances = employees[numeric_cols].var().sort_values()
print("Column Variances:")
print(variances.round(2))
# Identify near-zero variance columns (useless for ML)
low_var = variances[variances 0.01].index.tolist()
print(f"\nLow Variance Columns (Remove): {low_var}")

Example 2: Department-wise salary variance compare karke pay equity analysis.

dept_variance = employees.groupby("Department")["Salary"].var().sort_values(ascending=False)
print("Salary Variance by Department:")
print(dept_variance.round(2))
print(f"\nVerify: stdΒ² = var β†’ {employees['Salary'].std()**2:.2f} = {employees['Salary'].var():.2f}")

πŸ“Š Expected Output:

# Column Variances:
# Flag_Active      0.00    # ⚠️ Near-zero β€” remove!
# Rating           0.85
# Age            103.02
# Salary   812295640.64    # High variance β€” important feature

# Salary Variance by Department:
# Marketing    665640025.00   # Highest inequality
# IT           488410000.00
# HR           153760000.00   # Most equitable

# Verify: stdΒ² = var β†’ 812295640.64 = 812295640.64 βœ…

βœ… Best Practices:

  • var() ki value squared units mein hoti hai (salary variance "rupeesΒ²" mein). Interpretation ke liye std() (same units) zyada intuitive hai. Variance mathematical calculations mein use hota hai.
  • ML mein VarianceThreshold feature selector use karein: from sklearn.feature_selection import VarianceThreshold; selector = VarianceThreshold(threshold=0.01)
  • var() bhi ddof=1 default use karta hai Pandas mein (sample variance). NumPy mein ddof=0 default hai β€” cross-library calculations mein consistent rakhein.

πŸ’¬ Crack the Interview:

Q1: Variance aur Standard Deviation mein preference kab kisko dein?
Ans: Reporting/interpretation ke liye std() use karein (same units). Mathematical/statistical calculations (ANOVA, PCA) mein variance directly use hota hai kyunki additive property hai β€” two variables ki combined variability var se calculate hoti hai, std se nahi.

Q2: PCA mein variance ka kya role hai?
Ans: PCA new axes (principal components) dhundhta hai jahan data ka maximum variance capture ho. First PC sabse zyada variance explain karta hai. "95% variance explained" matlab naye components mein original data ki 95% information preserve hai.

Q3: Zero variance column kya indicate karta hai?
Ans: Zero variance matlab column mein sabhi values identical hain β€” koi variation nahi hai. Aise columns ML models ko koi useful information nahi dete aur remove karne chahiye. Example: ek column jismein sab rows mein "India" likha hai.

6. sum() / count() / min() / max() β€” Quick Aggregation Functions

πŸ” Kya Hai: sum() total calculate karta hai, count() non-null values count karta hai, min() smallest value, max() largest value return karta hai. Yeh chaar fundamental aggregation functions hain jo data summarization ki building blocks hain.

🎯 Kyu Use Hota Hai: Business mein sabse basic questions yahi hote hain β€” total revenue kitna hai (sum), kitne orders aaye (count), cheapest product kya hai (min), highest salary kya hai (max). Yeh functions standalone bhi use hote hain aur groupby() ke saath combined bhi.

πŸ’‘ Kab Use Hota Hai: Revenue totals, transaction counts, salary ranges, age boundaries, inventory levels, aur har woh scenario jahan quick numerical summary chahiye β€” individually ya groupby ke saath.

πŸ’» Real-World Code Examples:

Example 1: E-commerce business ka complete financial snapshot ek hi block mein.

print("πŸ“Š Business Dashboard")
print(f"Total Revenue:   β‚Ή{ecommerce['Revenue'].sum():,.2f}")
print(f"Total Orders:    {ecommerce['OrderID'].count():,}")
print(f"Cheapest Order:  β‚Ή{ecommerce['Revenue'].min():,.2f}")
print(f"Biggest Order:   β‚Ή{ecommerce['Revenue'].max():,.2f}")
print(f"Revenue Range:   β‚Ή{ecommerce['Revenue'].max() - ecommerce['Revenue'].min():,.2f}")

Example 2: Employee data mein age aur salary boundaries check karke impossible values detect karna.

# Data quality check using min/max
print(f"Age Range: {employees['Age'].min()} to {employees['Age'].max()}")
print(f"Salary Range: β‚Ή{employees['Salary'].min():,.0f} to β‚Ή{employees['Salary'].max():,.0f}")
# Flag impossible values
if employees["Age"].min() 0 or employees["Age"].max() > 120:
    print("⚠️ Invalid age values detected!")
# count() vs len() for missing value detection
print(f"\nTotal Rows: {len(employees)}")
print(f"Salary Non-Null: {employees['Salary'].count()}")
print(f"Missing Salaries: {len(employees) - employees['Salary'].count()}")

πŸ“Š Expected Output:

# πŸ“Š Business Dashboard
# Total Revenue:   β‚Ή3,67,51,250.00
# Total Orders:    15,000
# Cheapest Order:  β‚Ή150.00
# Biggest Order:   β‚Ή2,45,000.00
# Revenue Range:   β‚Ή2,44,850.00

# Age Range: 18 to 65       βœ… Valid!
# Salary Range: β‚Ή15,000 to β‚Ή8,50,000
# Total Rows: 10000
# Salary Non-Null: 9720
# Missing Salaries: 280

βœ… Best Practices:

  • count() sirf non-null values count karta hai β€” len(df) total rows deta hai including NaN. Difference = missing values count. Yeh quick missing value detection technique hai.
  • min()/max() string columns par bhi kaam karte hain β€” alphabetically first/last value return karte hain. Dates par earliest/latest date milti hai.
  • sum() boolean columns par lagane se True values count hota hai: df['Is_Active'].sum() = active records count.

πŸ’¬ Crack the Interview:

Q1: count() aur len() mein kya fundamental difference hai?
Ans: len(df) total rows deta hai NaN samajhit. count() sirf non-null values count karta hai. Agar 10000 rows mein 280 NaN hain toh len=10000 lekin count=9720. Isliye count() data quality metric hai.

Q2: sum() mein skipna=False dene se kya hoga agar NaN values hain?
Ans: Agar ek bhi NaN hai toh entire result NaN return hoga β€” kyunki NaN + any number = NaN. Default skipna=True hai jo NaN ignore karke baaki values sum karta hai. Production mein hamesha default use karein.

Q3: idxmin() aur idxmax() kya karte hain aur min()/max() se kaise different hain?
Ans: min()/max() actual minimum/maximum value return karte hain. idxmin()/idxmax() us value ka INDEX (row label) return karte hain. df.loc[df['Salary'].idxmax()] se highest salary waale employee ki puri row mil jaati hai.

7. groupby() β€” Category-Wise Data Summarization

πŸ” Kya Hai: groupby() Pandas ka sabse powerful function hai jo data ko categories mein split karke har group par separately aggregation apply karta hai. Yeh SQL ke GROUP BY ka equivalent hai. Split β†’ Apply β†’ Combine methodology follow karta hai.

🎯 Kyu Use Hota Hai: Real analysis hamesha category-wise hota hai β€” department-wise average salary, city-wise total revenue, month-wise order count, product-wise rating. Overall averages misleading hoti hain β€” groupby() granular insights deta hai jahan actual patterns chhupe hote hain.

πŸ’‘ Kab Use Hota Hai: Department-wise analysis, region-wise sales, time-based trends (monthly/quarterly), customer segment comparison, product category performance, aur basically jab bhi "X ke basis par Y kya hai" type question answer karna ho.

πŸ’» Real-World Code Examples:

Example 1: Department-wise complete salary analysis β€” count, mean, median, max sab ek saath.

# Simple groupby + single aggregation
dept_salary = employees.groupby("Department")["Salary"].mean().sort_values(ascending=False)
print(dept_salary)
# Multiple columns groupby
dept_city = employees.groupby(["Department", "City"])["Salary"].mean().round(2)
print(dept_city.head(10))

Example 2: E-commerce mein category-wise top selling products find karna.

# Category-wise sales performance
category_stats = ecommerce.groupby("Category").agg(
    Total_Revenue=("Revenue", "sum"),
    Total_Orders=("OrderID", "count"),
    Avg_Order_Value=("Revenue", "mean")
).sort_values("Total_Revenue", ascending=False).round(2)
print(category_stats)

πŸ“Š Expected Output:

# Department-wise Salary:
# IT           82500.00
# Finance      75200.50
# Marketing    62400.00
# HR           55800.75

# Category-wise Sales:
# Category      Total_Revenue  Total_Orders  Avg_Order_Value
# Electronics   1,25,00,000    3200          3906.25
# Fashion       89,50,000      4500          1988.89
# Home          52,00,000      2800          1857.14
# Books         18,50,000      4500          411.11

βœ… Best Practices:

  • groupby() ke baad .reset_index() lagayein agar result ko regular DataFrame jaisa use karna hai β€” bina reset ke multi-level index ban jaata hai.
  • Multiple aggregations ke liye named aggregation syntax use karein (Example 2 jaisa) β€” readable output milta hai meaningful column names ke saath.
  • groupby().transform() use karein jab group-level result ko original DataFrame size mein broadcast karna ho β€” jaise group mean se difference calculate karna.

πŸ’¬ Crack the Interview:

Q1: groupby() mein transform() aur agg() mein kya difference hai?
Ans: agg() grouped/reduced output deta hai (groups = rows). transform() original DataFrame ki size preserve karta hai β€” har row ko uske group ki aggregate value se replace karta hai. Group mean se deviation nikalna ho toh: df['col'] - df.groupby('group')['col'].transform('mean').

Q2: groupby() lazy evaluation kya hai?
Ans: df.groupby('col') akela call karne par koi computation nahi hoti β€” sirf GroupBy object banta hai. Actual computation tab hoti hai jab aggregation function (.mean(), .sum()) call ho. Isliye groupby() itself instant hai regardless of data size.

Q3: groupby() mein NaN key values ka kya hota hai?
Ans: Default mein NaN keys drop ho jaate hain groupby output se. Include karne ke liye: df.groupby('col', dropna=False) use karein. Yeh important hai jab NaN itself ek meaningful category ho (jaise "No Department Assigned").

8. agg() β€” Multiple Aggregations Ek Saath

πŸ” Kya Hai: agg() (aggregate) multiple aggregation functions ko ek single call mein apply karta hai β€” same column par different functions ya different columns par different functions. Yeh groupby() ke saath milke complete multi-metric summary tables banata hai.

🎯 Kyu Use Hota Hai: Real reporting mein ek metric kaafi nahi hota β€” department ki sirf average salary se kaam nahi chalega, count, min, max, median sab chahiye ek saath. agg() se ek call mein multiple perspectives mil jaate hain jo complete picture deta hai.

πŸ’‘ Kab Use Hota Hai: Dashboard data preparation, management reports, multi-KPI analysis tables, comprehensive group summaries, aur jab bhi ek hi data par multiple statistical measures simultaneously chahiye.

πŸ’» Real-World Code Examples:

Example 1: Department-wise comprehensive salary report β€” multiple metrics ek saath.

# Same column par multiple aggregations
salary_report = employees.groupby("Department")["Salary"].agg(
    ["count", "mean", "median", "std", "min", "max"]
).round(2)
salary_report["range"] = salary_report["max"] - salary_report["min"]
print(salary_report)

Example 2: Different columns par different aggregation functions apply karna (Named Aggregation).

# Named aggregation β€” different functions on different columns
dept_summary = employees.groupby("Department").agg(
    Employee_Count=("EmployeeID", "count"),
    Avg_Salary=("Salary", "mean"),
    Avg_Age=("Age", "mean"),
    Max_Salary=("Salary", "max"),
    Avg_Rating=("Rating", "mean")
).round(2).sort_values("Avg_Salary", ascending=False)
print(dept_summary)

πŸ“Š Expected Output:

# Salary Report:
# Department  count   mean      median    std       min     max      range
# IT          3200    82500.00  78000.00  22100.00  32000   250000   218000
# Finance     2000    75200.50  72000.00  18500.25  35000   185000   150000
# HR          2400    55800.75  54000.00  12400.00  28000   120000   92000

# Named Aggregation:
# Department  Employee_Count  Avg_Salary  Avg_Age  Max_Salary  Avg_Rating
# IT          3200            82500.00    32.50    250000      4.20
# Finance     2000            75200.50    38.40    185000      3.85
# HR          2400            55800.75    34.20    120000      3.65

βœ… Best Practices:

  • Named Aggregation syntax (Example 2) zyada readable hai β€” output columns ko directly meaningful names milte hain bina post-renaming ke.
  • Custom aggregation functions bhi pass kar sakte hain: .agg(range_func=('col', lambda x: x.max()-x.min()))
  • agg() ke baad .reset_index() lagayein agar flat DataFrame chahiye β€” dashboard tools aur visualization libraries flat structure prefer karte hain.

πŸ’¬ Crack the Interview:

Q1: agg() mein list format aur dictionary format mein kya difference hai?
Ans: List format ['mean','sum'] same column par multiple functions apply karta hai β€” MultiIndex columns bante hain. Named aggregation format New_Name=('col','func') different columns par different functions lagata hai β€” flat readable columns bante hain. Named aggregation recommended hai.

Q2: Custom lambda function agg() mein kaise pass karein?
Ans: .agg(IQR=('Salary', lambda x: x.quantile(0.75) - x.quantile(0.25))) β€” koi bhi custom calculation define kar sakte hain. Lekin lambda functions string functions se slower hote hain kyunki optimize nahi ho paate.

Q3: agg() aur apply() groupby ke saath kab kaunsa use karein?
Ans: agg() standard aggregation functions (sum, mean, count) ke liye β€” vectorized aur fast. apply() complex custom functions ke liye jo multiple columns access karte hain ya conditional logic rakhte hain. Performance: agg() >> apply(). Pehle agg() try karein, na chale toh apply().

9. pivot_table() β€” Excel-Style Cross-Tabulation Summary

πŸ” Kya Hai: pivot_table() Excel ki pivot table ka Pandas version hai. Yeh data ko rows aur columns ke intersection par summarize karta hai β€” jaise Department (rows) Γ— City (columns) ka average salary. Multidimensional analysis ka sabse powerful tool hai.

🎯 Kyu Use Hota Hai: groupby() linear summaries deta hai lekin cross-tabulation perspective nahi. pivot_table() se 2-dimensional matrix banti hai β€” rows mein ek category, columns mein doosri, aur cells mein aggregated values. Management dashboards aur business reports isi format mein hote hain.

πŸ’‘ Kab Use Hota Hai: Department Γ— City salary matrix, Month Γ— Category revenue matrix, Region Γ— Product sales matrix β€” jab bhi 2 categorical dimensions par ek numeric value summarize karni ho tab pivot_table() perfect choice hai.

πŸ’» Real-World Code Examples:

Example 1: Department Γ— City ki average salary ka cross-tabulation matrix banana.

salary_pivot = pd.pivot_table(
    employees,
    values="Salary",
    index="Department",
    columns="City",
    aggfunc="mean",
    margins=True,
    margins_name="Overall"
).round(0)
print(salary_pivot)

Example 2: E-commerce mein Month Γ— Category revenue heatmap data banana with multiple aggfuncs.

ecommerce["Month"] = pd.to_datetime(ecommerce["OrderDate"]).dt.month_name()
revenue_pivot = pd.pivot_table(
    ecommerce,
    values="Revenue",
    index="Month",
    columns="Category",
    aggfunc=["sum", "count"],
    fill_value=0
)
print(revenue_pivot)

πŸ“Š Expected Output:

# Salary Pivot (Department Γ— City):
# City        Mumbai   Delhi    Bangalore  Overall
# Department
# IT          85000    82000    92000      82500
# Finance     78000    72000    76000      75200
# HR          58000    52000    57000      55800
# Overall     72000    68000    74000      68450

# Revenue Pivot (Month Γ— Category):
#            sum                          count
# Category   Electronics  Fashion  Home   Electronics  Fashion  Home
# January    520000       320000   180000  250         420      180
# December   980000       750000   420000  350         680      310

βœ… Best Practices:

  • margins=True lagayein β€” row aur column totals/averages automatically add ho jaate hain jo grand total aur subtotals ka kaam karte hain.
  • fill_value=0 lagayein NaN cells ko 0 se replace karne ke liye β€” jab kisi combination ka data exist na kare (jaise January mein koi specific category sell nahi hui).
  • Multiple aggfuncs list mein pass kar sakte hain: aggfunc=['sum','mean','count'] β€” MultiIndex columns bante hain jo comprehensive view dete hain.

πŸ’¬ Crack the Interview:

Q1: pivot_table() aur groupby() mein kab kaunsa use karein?
Ans: Single dimension grouping ke liye groupby(). Two-dimensional cross-tabulation ke liye pivot_table(). pivot_table() internally groupby() hi use karta hai lekin result ko 2D matrix format mein reshape karta hai jo visualization aur reporting ke liye better hai.

Q2: pivot_table() aur pd.pivot() mein kya difference hai?
Ans: pd.pivot() simple reshaping karta hai bina aggregation ke β€” duplicate index-column combinations par error deta hai. pivot_table() aggregation function apply karta hai (default mean) aur duplicates handle karta hai. Real analysis mein hamesha pivot_table() use karein.

Q3: pd.crosstab() aur pivot_table() mein kya difference hai?
Ans: crosstab() frequency counts ke liye optimized hai (default aggfunc='count'). pivot_table() zyada flexible hai β€” koi bhi aggfunc use kar sakte hain. crosstab() primarily categorical Γ— categorical analysis ke liye hai, pivot_table() generic purpose ke liye.

10. cumsum() / cummax() / cummin() β€” Cumulative/Running Statistics

πŸ” Kya Hai: cumsum() running total calculate karta hai β€” har row par ab tak ka cumulative sum. cummax() ab tak ka maximum, cummin() ab tak ka minimum track karta hai. Yeh time-series aur sequential data analysis ke fundamental tools hain.

🎯 Kyu Use Hota Hai: Business mein running totals critical hain β€” YTD (Year-to-Date) revenue, MTD sales target progress, running maximum stock price (all-time high), cumulative customer acquisition count. Yeh functions time ke saath growth aur trends track karte hain.

πŸ’‘ Kab Use Hota Hai: YTD/MTD revenue tracking, cumulative customer growth charts, running maximum/minimum in stock prices, progressive target achievement tracking, aur Pareto analysis (80/20 rule) mein cumulative percentage calculate karne ke liye.

πŸ’» Real-World Code Examples:

Example 1: Monthly revenue data mein YTD (Year-to-Date) cumulative revenue aur target achievement track karna.

# Monthly revenue sorted by date
monthly = ecommerce.groupby(pd.to_datetime(ecommerce["OrderDate"]).dt.to_period("M"))["Revenue"].sum().reset_index()
monthly.columns = ["Month", "Revenue"]
monthly["YTD_Revenue"]  = monthly["Revenue"].cumsum()
monthly["Running_Max"]  = monthly["Revenue"].cummax()
monthly["Best_Month?"]  = monthly["Revenue"] == monthly["Running_Max"]
print(monthly)

Example 2: Pareto Analysis β€” top products finding using cumulative percentage.

# Product-wise revenue sorted descending
product_rev = ecommerce.groupby("Product")["Revenue"].sum().sort_values(ascending=False)
product_rev_df = product_rev.reset_index()
product_rev_df["Cumulative_Revenue"] = product_rev_df["Revenue"].cumsum()
product_rev_df["Cumulative_%"] = (product_rev_df["Cumulative_Revenue"] / product_rev_df["Revenue"].sum() * 100).round(1)
top_products = product_rev_df[product_rev_df["Cumulative_%"] 80]
print(f"Top {len(top_products)} products generate 80% revenue!")

πŸ“Š Expected Output:

# YTD Revenue Tracking:
# Month    Revenue   YTD_Revenue  Running_Max  Best_Month?
# Jan      450000    450000       450000       True
# Feb      380000    830000       450000       False
# Mar      520000    1350000      520000       True  # New record!
# ...
# Dec      890000    6750000      890000       True  # All-time best!

# Pareto Analysis:
# Top 15 products generate 80% revenue!  (Out of 200 total products)

βœ… Best Practices:

  • Cumulative functions apply karne se pehle data ko correctly sort karein (date-wise ya relevant order mein) β€” unsorted data par cumsum() meaningless results dega.
  • Group-wise cumulative sum ke liye: df.groupby('Category')['Revenue'].cumsum() β€” har group ka separate running total calculate hota hai.
  • cumprod() bhi available hai β€” compound growth rate calculate karne ke liye: (1 + df['Returns']).cumprod() investment growth track karta hai.

πŸ’¬ Crack the Interview:

Q1: cumsum() mein NaN values ka kya effect padta hai?
Ans: Default mein cumsum() NaN ko skip karta hai β€” NaN row par previous cumulative value retain hota hai. skipna=False dene par NaN ke baad sab values NaN ho jayengi kyunki NaN + anything = NaN.

Q2: Pareto Analysis (80/20 Rule) cumsum() se kaise implement karein?
Ans: Data ko descending order mein sort karein, cumsum() lagayein, total se divide karke percentage banayein, 80% mark tak filter karein. Yeh batata hai ki top kitne items/customers 80% revenue contribute karte hain β€” resource allocation decisions ke liye critical hai.

Q3: expanding() aur cumsum() mein kya difference hai?
Ans: cumsum() sirf sum ke liye hai. expanding() general purpose hai β€” expanding().mean() se running average, expanding().std() se running std, koi bhi function apply kar sakte hain. cumsum() expanding().sum() ka shortcut hai.

Conclusion: Statistics & Aggregation Quick Reference Matrix

Apne statistical requirement ke basis par sahi function chunye:

Task / Requirement Function Key Note
Average / Central Value mean() Outlier-sensitive β€” symmetric data ke liye
Robust Center (Skewed Data) median() Outlier-resistant β€” skewed data ke liye
Most Common Category mode() Categorical data β€” returns DataFrame
Data Spread Measurement std() / var() std for reporting, var for statistical tests
Quick Totals & Ranges sum/count/min/max Building blocks of all aggregation
Category-Wise Analysis groupby() Split-Apply-Combine pattern
Multiple Metrics Together agg() Named aggregation for clean output
2D Cross-Tab Summary pivot_table() margins=True for grand totals
Running Totals & Trends cumsum/cummax/cummin Sort data first β€” order matters!

Next Post Preview: Masterclass Part 10

Next masterclass mein hum cover karenge: Relationships & Correlation Functions β€” corr(), cov(), crosstab(), scatter analysis aur 9 powerful tools jo variables ke beech hidden relationships discover karte hain.

Happy Coding & Stay Analytically Pure! πŸš€

πŸ‘€
Jatin Kumar
Data Analyst & Educator

Python, SQL, Power BI aur Excel mein practical tutorials likhta hoon β€” taaki data analytics seekhna aasan ho. Portfolio: jatinanalytics.co.in

Portfolio LinkedIn GitHub Kaggle All Articles
Share:

πŸ’¬ Comments (0)

Spam/links allowed nahi hain β€” respectful comments welcome!

Loading comments...

Was this article helpful?