[rrd-users] Save and Calculate Percentil

[Peter]

Hi,
I use rrdtool to store measurement data in rrd files used by Cacti.
The measurement data have an interval of 50 ms.
I need Average, Max, 99% and 5% Percentil.
I used the lowest possible step size in the rrd files:1 second.
For Average & Max I created separate rrd files with RRAs:
1s: for a week , 1 min: for a month , 5 min: for a year
My program now calculate every second the max & avg values and updates the
rrd files.
This works fine, also in Cacti.

Is a solution possible for the percentiles?
The prefered solution would be: based on the 50 ms data.
But a 50 ms step size is not possible.

And percentiles of small intervals are not so interesting.
I could calculate the percentiles in my program for intervals
5/10/15/30/60 min, 3/6/12/24 hours

But how should I define the rrd file and update the rra for this intervals?
Now the RRA are calculated from rrdupdate by the given consolidation
function.
Or can I set each RRA manually?


And Cacti need also a consolition function to create a DEF for rrdgraph ,
i.e. for MAX
DEF:a="/Data/rrd/DataMax.rrd":"DATAMAX":MAX \
Which RRA use rrdgraph (called by Cacti)?
If rrdgraph use always the RRA with the smallest matching intervall, so I
can use LAST or MAX (for 99%), MIN (for 5% ).


Peter




















--
View this message in context: http://rrd-mailinglists.937164.n2.nabble.com/Save-and-Calculate-Percentil-tp7582357.html
Sent from the RRDtool Users Mailinglist mailing list archive at Nabble.com.

[Rafal Gwizdala]

There's no 'percentile' consolidation function and it can't be derived from
other statistical functions available in rrdtool, so i think it would be
better to calculate the percentiles before storing them in RRD.
This would of course require storing the data for extended periods of time,
and to calculate 24h percentile you'd have to store 24 h worth of samples,
which could be difficult.
Here's a link to some approximate method which doesn't require you to store
all samples because it's based on histograms. And histograms are an
'additive' measure, so you can add 24 hourly histograms together to
calculate a daily percentile.
http://techblog.molindo.at/2009/11/efficiently-tracking-response-time-percentiles.html

I'm also doing some percentile calculations for app performance data, but
I'm using a different approach: a fixed time window of 5 minutes and I
calculate multiple measures for that period: average, median, 95th, 98th
and 99th percentile. These measures are then writtend to a RRD file every
minute.

Post by Peter
Hi,
I use rrdtool to store measurement data in rrd files used by Cacti.
The measurement data have an interval of 50 ms.
I need Average, Max, 99% and 5% Percentil.
I used the lowest possible step size in the rrd files:1 second.
1s: for a week , 1 min: for a month , 5 min: for a year
My program now calculate every second the max & avg values and updates the
rrd files.
This works fine, also in Cacti.
Is a solution possible for the percentiles?
The prefered solution would be: based on the 50 ms data.
But a 50 ms step size is not possible.
And percentiles of small intervals are not so interesting.
I could calculate the percentiles in my program for intervals
5/10/15/30/60 min, 3/6/12/24 hours
But how should I define the rrd file and update the rra for this intervals?
Now the RRA are calculated from rrdupdate by the given consolidation
function.
Or can I set each RRA manually?
And Cacti need also a consolition function to create a DEF for rrdgraph ,
i.e. for MAX
DEF:a="/Data/rrd/DataMax.rrd":"DATAMAX":MAX \
Which RRA use rrdgraph (called by Cacti)?
If rrdgraph use always the RRA with the smallest matching intervall, so I
can use LAST or MAX (for 99%), MIN (for 5% ).
Peter
--
http://rrd-mailinglists.937164.n2.nabble.com/Save-and-Calculate-Percentil-tp7582357.html
Sent from the RRDtool Users Mailinglist mailing list archive at Nabble.com.
_______________________________________________
rrd-users mailing list
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

[Rafal Gwizdala]

Oops, I think I didn't really understand your question...
The answer should be: store your percentiles separately (in separate DS-es)
and choose the aggregation function that makes most sense in your case, for
example the average.

Post by Rafal Gwizdala
There's no 'percentile' consolidation function and it can't be derived
from other statistical functions available in rrdtool, so i think it would
be better to calculate the percentiles before storing them in RRD.
This would of course require storing the data for extended periods of
time, and to calculate 24h percentile you'd have to store 24 h worth of
samples, which could be difficult.
Here's a link to some approximate method which doesn't require you to
store all samples because it's based on histograms. And histograms are an
'additive' measure, so you can add 24 hourly histograms together to
calculate a daily percentile.
http://techblog.molindo.at/2009/11/efficiently-tracking-response-time-percentiles.html
I'm also doing some percentile calculations for app performance data, but
I'm using a different approach: a fixed time window of 5 minutes and I
calculate multiple measures for that period: average, median, 95th, 98th
and 99th percentile. These measures are then writtend to a RRD file every
minute.

Post by Peter
Hi,
I use rrdtool to store measurement data in rrd files used by Cacti.
The measurement data have an interval of 50 ms.
I need Average, Max, 99% and 5% Percentil.
I used the lowest possible step size in the rrd files:1 second.
1s: for a week , 1 min: for a month , 5 min: for a year
My program now calculate every second the max & avg values and updates the
rrd files.
This works fine, also in Cacti.
Is a solution possible for the percentiles?
The prefered solution would be: based on the 50 ms data.
But a 50 ms step size is not possible.
And percentiles of small intervals are not so interesting.
I could calculate the percentiles in my program for intervals
5/10/15/30/60 min, 3/6/12/24 hours
But how should I define the rrd file and update the rra for this
intervals?
Now the RRA are calculated from rrdupdate by the given consolidation
function.
Or can I set each RRA manually?
And Cacti need also a consolition function to create a DEF for rrdgraph ,
i.e. for MAX
DEF:a="/Data/rrd/DataMax.rrd":"DATAMAX":MAX \
Which RRA use rrdgraph (called by Cacti)?
If rrdgraph use always the RRA with the smallest matching intervall, so I
can use LAST or MAX (for 99%), MIN (for 5% ).
Peter
--
http://rrd-mailinglists.937164.n2.nabble.com/Save-and-Calculate-Percentil-tp7582357.html
Sent from the RRDtool Users Mailinglist mailing list archive at
Nabble.com.
_______________________________________________
rrd-users mailing list
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

[Peter]

Hi Rafal,
/"choose the aggregation function that makes most sense in your case, for
example the average."/
That's the problem: If I calculate and store 10-minute values, which
function is for an hour / day good?
For a small factor between 'store' interval and rrdgraph intervall MIN / MAX
can be approximated used.

So my idea is to calculate & store for 99% and 5% percentil the data in
RRA for some intervals i.e. 15/30/60 min, 3/6/12/24 hours.
And hope, rrdgraph choose the best fitting RRA.

But how can I bring the data in a rrd file? rrdupdate is not possible.
Possible every 15 minute: rrddump > add the required values in the xml file

rrdrestore.

But then the RRA are growing infinitely?

Or better: 2 x 7 RRD files with only one RRA. This can updated with CF LAST.
But then I have a problem with Cacti. Which File is the Datasource?
So dump all 7 RRD to xml, Merge it together to an format with all RRA, and
restore this RRD to show it in Cacti.

Peter












Peter




--
View this message in context: http://rrd-mailinglists.937164.n2.nabble.com/Save-and-Calculate-Percentil-tp7582357p7582360.html
Sent from the RRDtool Users Mailinglist mailing list archive at Nabble.com.

[Rafal Gwizdala]

Peter, I think there should be no problem with writing to 7 different RRD
files with different resolution. I don't know Cacti very well but I think
it's possible to create a graph with data from several RRD files, so this
shouldn't be a problem.
Or just one RRD file with 7 input variables and 7 RRAs corresponding to all
your sampling resolutions and CF LAST aggregation (rrdupdate it every 15
minutes, just repeat previous values for percentiles that change less often)
All in all, I think rrdupdate should do the job without having to
dump/restore anything (but I never did anything similar with rrd)
R

[Steve Shipway]

The problem with calculating Percentiles accurately is that you need all the
data samples to calculate it; you cannot progressively increment the
calculation in the way you can with something like MAX or AVG.

As a result, you end up with one of three situations --

1. You calculate the percentile on the fly based on the AVG data. This is
how Routers2(MRTG) does it; the downside is that, as the RRA consolidation
step size increases, so does the inaccuracy of the Percentile calculation as
you lose detail with each averaging step.

2. Keep the high-detail PDP, and calculate the percentile from them. This
is how Routers2 does it if there exists a high-granularity RRA (usually,
there isn't one), and how the RRDTool PERCENTILE function does it. The
downside with this is that it can potentially be very CPU intensive if there
are a very large number of PDP to analyse.

3. Keep the high-detail PDP in the RRA prep area and consolidate in the RRA.
This is not currently possible, as there is not a 'PERCENTILE' type RRA.
The likely reason for this is that the PDP prep area is simply not large
enough to hold the potentially huge number of individual PDP that would be
necessary; RRA consolidation is progressive (eg: newavg = (n*oldavg +
newpdp)/(n+1)) and so needs only a small prep area.

So, in summary, you cannot do this in the RRA. You can use the PERCENTILE
function to do it on the fly at graph time, but this is potentially CPU
intensive, and will be progressively more inaccurate unless you keep
high-resolution RRA data to cover the entire period of the graph.

Steve

Steve Shipway
***@auckland.ac.nz