What Are All Python Pandas. Agg Functions?

What Are All Python Pandas. Agg Functions?

I am talking about Python Pandas .agg() function, this one:

meanData = all_data.groupby(['Id'])[features].agg('mean')

So, it can do things like:

  • Mean
  • Median
  • Sum
  • Max
  • Min
  • Std

What else can it do? I found nothing on the official documentation page:

5 Answers

You can find the full list in documentation under pandas.core.groupby.GroupBy.some-function-name in the left menu.

List currently includes many aggregation functions:

pipe, all, any, bfill, backfill, count, cumcount, cummax, cummin, cumprod, cumsum, ffill, first, head, last, max, mean, median, min, ngroup, nth, ohlc, pad, prod, rank, pct_change, size, sem, std, sum, var, tail

The aggregations that work with .agg() include:

Mean - df.agg('mean')

Median - df.agg('median')

Mode - df.agg('mode')

Sum - df.agg('sum')

Count - df.agg('count')

Max - df.agg('max')

Min - df.agg('min')

Standard Deviation - df.agg('std')

Variance - df.agg('var')

Skewness - df.agg('skew')

Kurtosis - df.agg('kurt')

The relevant parts of pandas/core/base.py (here, line 298):

def _try_aggregate_string_function(self, arg, *args, **kwargs):
    """
    if arg is a string, then try to operate on it:
    - try to find a function (or attribute) on ourselves
    - try to find a numpy function
    - raise

    """
    f = getattr(self, arg, None)
    if f is not None:
        if callable(f):
            return f(*args, **kwargs)
    f = getattr(np, arg, None)
    if f is not None:
        return f(self, *args, **kwargs)

    raise ValueError("{arg} is an unknown string function".format(arg=arg))

Essentially it tries to introspect using the string as a function, and then tries the same with numpy, in case it's a builtin. If not it returns a ValueError.

I'd be happy if someone who knows more than me could clarify more, but if not, hope this helps.

It can be just about any function that can be applied on a DataFrame object.

print(dir(DataFrame))

When func is a string type, the func name is looked up in the available attributes of the DataFrame object that the .agg method is invoked on.

While it gives similar result when doing a division of elements in a DataFrame to write,

df = DataFrame([1,2,3,4])    
df.agg('true_div', 0, 2)

In the real world, you find that the method doing the operation is invoked in a direct manner on the DataFrame

df = DataFrame([1,2,3,4])
df.true_div(2)

The answers here and here give what you are looking but I want to add one of agg usage from one of my experience. I created a manual class to call as a function and use it in agg to show flexibility of agg -->

The class:

class Quantile:
def __init__(self, the_quantile):
    self.the_quantile = the_quantile
    
def __call__(self, the_array):
    return np.quantile(the_array.dropna(), self.the_quantile)

The agg code block I used it in:

df.groupby('SomeGroups')['SomeValues'] \
  .agg(count = 'count',
       nunique = 'nunique',
       size = 'size',
       minimum = 'min',
       idx_min = 'idxmin',
       mean = 'mean',
       median = 'median',
       maximum = 'max',
       idx_max = 'idxmax',
       std = 'std',
       mad = 'mad',
       quantile_0 = Quantile(0),
       quantile_25 = Quantile(0.25),
       quantile_50 = Quantile(0.5),
       quantile_57 = Quantile(0.57), #This was to show I can call any quantile 
       quantile_90 = Quantile(0.9),
       quantile_95 = Quantile(0.95),
       quantile_98 = Quantile(0.98),
       quantile_100 = Quantile(1))

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Maya Lin-Takahashi
Author

Maya Lin-Takahashi

Maya is a hardware enthusiast who tests and reviews smart home devices, smartphones, wearables, and audio gear. She focuses on practical consumer value and build quality.