x/exp/stats: new package with Mean, Median, more

[hemanth0525]

Description:

This proposal aims to enhance the Go standard library’s math ( math/stats.go )package by introducing several essential statistical functions. The proposed functions are:

• Mean: Calculates the average value of a data set.
• Median: Determines the middle value when the data set is sorted.
• Mode: Identifies the most frequently occurring value in a data set.
• Variance: Measures the spread of the data set from the mean.
• StdDev: Computes the standard deviation, providing a measure of data dispersion.
  and many more....

Motivation:

The inclusion of these statistical functions directly in the math package will offer Go developers robust tools for data analysis and statistical computation, enhancing the language's utility in scientific and financial applications. Currently, developers often rely on external libraries for these calculations, which adds dependencies and potential inconsistencies. Integrating these functions into the standard library will:

• Provide Comprehensive Statistical Analysis: These functions will facilitate fundamental statistical measures, aiding in more thorough data analysis and better understanding of data distributions.
• Ensure Reliable Behavior: Functions are designed to handle edge cases, such as empty slices, to maintain predictable and accurate results.
• Optimize Performance and Accuracy: Implemented with efficient algorithms to balance performance with calculation accuracy.
• Increase Utility: Reduces the need for third-party libraries, making statistical computation more accessible and consistent within the Go ecosystem.

Design:

The functions will be added to the existing math package, ensuring they are easy to use and integrate seamlessly with other mathematical operations. Detailed documentation and examples will be provided to illustrate their usage and edge case handling.

Examples:

• Mean:

  mean := math.Mean([]float64{1, 2, 3, 4, 5})

• Median:

  median := math.Median([]float64{1, 3, 3, 6, 7, 8, 9})

• Mode:

  mode := math.Mode([]float64{1, 2, 2, 3, 4})

• Variance:

  variance := math.Variance([]float64{1, 2, 3, 4, 5})

• StdDev:

  stddev := math.StdDev([]float64{1, 2, 3, 4, 5})

---

@gabyhelp's overview of this issue: #69264 (comment)

[gabyhelp]

Related Issues and Documentation

• math: implement statistical functions for mean, median, mode, variance, and stddev #69195

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[ianlancetaylor]

In general the math package aims to provide the functions that are in the C++ standard library <math>.

[hemanth0525]

Thanks for the feedback! I get that the math package is meant to mirror the functions in C++'s <cmath>, but I think adding some built-in stats functions could be a nice improvement. A lot of developers deal with stats regularly, so having these in the standard library could make things easier without stepping too far from the package’s core purpose. Happy to chat more about it if needed!

[earthboundkid]

Can you do some detective work to see how people are dealing with this in open source Go now? Is there some go-stats package that has a million stars on Github? Are there ten libraries that are each imported five hundred times? Seeing that something has big demand already is important for bringing something that could be in a third party library into the standard library. Otherwise this will just get closed with "write a third party library." Which has certainly happened to me more than once!

[hemanth0525]

I’ve done some digging into how statistical functions are currently being handled in the Go community. While libraries like Gonum and others provide statistical methods, there's no single source of truth or dominant package in this space, and many are designed for more complex or specialized tasks. However, the basic statistical functions we're proposing—like Mean, Median, Mode, Variance, and StdDev—are foundational for a wide range of applications, from simple data analysis to more advanced scientific and financial computations.

By integrating these into the standard library, we'd eliminate the need for external dependencies for basic tasks, which is in line with Go's philosophy of having a strong standard library for common use cases. While third-party packages are an option, including these functions in the math package would make Go more self-sufficient for everyday statistical needs, benefiting developers who want a simple, reliable way to compute these without resorting to third-party solutions.

[seankhliao]

for common use cases

this is the part where we need to see evidence. especially considering the existence of libraries like gonum, how often does the need arise for functions like those proposed where you wouldn't need the extra functionality that other libraries provide.

[jimmyfrasche]

For what it's worth, python has a statistics package in its standard library: https://docs.python.org/3/library/statistics.html

It would be nice to have a simple package everyone agrees on for common use cases, but that doesn't necessarily need to be in std.

[randall77]

These functions sound pretty simple, but I think there's actually a lot of subtlety here. For instance, what does Mean do for rounding? Do we need to use Kahan's algorithm? What if the sum at some point rounds up to +Inf?

[doggedOwl]

Can you do some detective work to see how people are dealing with this in open source Go now? Is there some go-stats package that has a million stars on Github? Are there ten libraries that are each imported five hundred times? Seeing that something has big demand already is important for bringing something that could be in a third party library into the standard library. Otherwise this will just get closed with "write a third party library." Which has certainly happened to me more than once

in my experience everytime some numeric problems comes up gonum lib is suggested. they have a stats package https://pkg.go.dev/gonum.org/v1/gonum@v0.15.1/stat