I am talking about Python Pandas .agg() function, this one:
meanData = all_data.groupby(['Id'])[features].agg('mean')
So, it can do things like:
- Mean
- Median
- Sum
- Max
- Min
- Std
What else can it do? I found nothing on the official documentation page:
5 Answers
You can find the full list in documentation under pandas.core.groupby.GroupBy.some-function-name in the left menu.
List currently includes many aggregation functions:
pipe, all, any, bfill, backfill, count, cumcount, cummax, cummin, cumprod, cumsum, ffill, first, head, last, max, mean, median, min, ngroup, nth, ohlc, pad, prod, rank, pct_change, size, sem, std, sum, var, tail
The aggregations that work with .agg() include:
Mean - df.agg('mean')
Median - df.agg('median')
Mode - df.agg('mode')
Sum - df.agg('sum')
Count - df.agg('count')
Max - df.agg('max')
Min - df.agg('min')
Standard Deviation - df.agg('std')
Variance - df.agg('var')
Skewness - df.agg('skew')
Kurtosis - df.agg('kurt')
The relevant parts of pandas/core/base.py (here, line 298):
def _try_aggregate_string_function(self, arg, *args, **kwargs):
"""
if arg is a string, then try to operate on it:
- try to find a function (or attribute) on ourselves
- try to find a numpy function
- raise
"""
f = getattr(self, arg, None)
if f is not None:
if callable(f):
return f(*args, **kwargs)
f = getattr(np, arg, None)
if f is not None:
return f(self, *args, **kwargs)
raise ValueError("{arg} is an unknown string function".format(arg=arg))
Essentially it tries to introspect using the string as a function, and then tries the same with numpy, in case it's a builtin. If not it returns a ValueError.
I'd be happy if someone who knows more than me could clarify more, but if not, hope this helps.
It can be just about any function that can be applied on a DataFrame object.
print(dir(DataFrame))
When func is a string type, the func name is looked up in the available attributes of the DataFrame object that the .agg method is invoked on.
While it gives similar result when doing a division of elements in a DataFrame to write,
df = DataFrame([1,2,3,4])
df.agg('true_div', 0, 2)
In the real world, you find that the method doing the operation is invoked in a direct manner on the DataFrame
df = DataFrame([1,2,3,4])
df.true_div(2)
The answers here and here give what you are looking but I want to add one of agg usage from one of my experience. I created a manual class to call as a function and use it in agg to show flexibility of agg -->
The class:
class Quantile:
def __init__(self, the_quantile):
self.the_quantile = the_quantile
def __call__(self, the_array):
return np.quantile(the_array.dropna(), self.the_quantile)
The agg code block I used it in:
df.groupby('SomeGroups')['SomeValues'] \
.agg(count = 'count',
nunique = 'nunique',
size = 'size',
minimum = 'min',
idx_min = 'idxmin',
mean = 'mean',
median = 'median',
maximum = 'max',
idx_max = 'idxmax',
std = 'std',
mad = 'mad',
quantile_0 = Quantile(0),
quantile_25 = Quantile(0.25),
quantile_50 = Quantile(0.5),
quantile_57 = Quantile(0.57), #This was to show I can call any quantile
quantile_90 = Quantile(0.9),
quantile_95 = Quantile(0.95),
quantile_98 = Quantile(0.98),
quantile_100 = Quantile(1))