[rrd-users] RRA of unaltered data

[Thomas Nilsson]

My shell script:
----
./rrdtool create test2.rrd \
--start now --step=5 \
DS:ok:GAUGE:10:0:100 \
RRA:LAST:0:1:10

for i in 2 20 4 40 6 60 8 80 10 100
do
sleep 5
./rrdtool update test2.rrd -t ok N:$i
echo "Updated DS \"ok\" with value $i"
done

./rrdtool dump test2.rrd
----

Wen I run my script it output this:

Updated DS "ok" with value 2
Updated DS "ok" with value 20
Updated DS "ok" with value 4
Updated DS "ok" with value 40
Updated DS "ok" with value 6
Updated DS "ok" with value 60
Updated DS "ok" with value 8
Updated DS "ok" with value 80
Updated DS "ok" with value 10
Updated DS "ok" with value 100
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE rrd SYSTEM "http://oss.oetiker.ch/rrdtool/rrdtool.dtd">
<!-- Round Robin Database Dump -->
<rrd>
<version>0003</version>
<step>5</step> <!-- Seconds -->
<lastupdate>1448379137</lastupdate> <!-- 2015-11-24 16:32:17 CET -->

<ds>
<name> ok </name>
<type> GAUGE </type>
<minimal_heartbeat>10</minimal_heartbeat>
<min>0.0000000000e+00</min>
<max>1.0000000000e+02</max>

<!-- PDP Status -->
<last_ds>100</last_ds>
<value>2.9769000000e+02</value>
<unknown_sec> 0 </unknown_sec>
</ds>

<!-- Round Robin Archives -->
<rra>
<cf>LAST</cf>
<pdp_per_row>1</pdp_per_row> <!-- 5 seconds -->

<params>
<xff>0.0000000000e+00</xff>
</params>
<cdp_prep>
<ds>
<primary_value>4.6702324000e+01</primary_value>
<secondary_value>0.0000000000e+00</secondary_value>
<value>NaN</value>
<unknown_datapoints>0</unknown_datapoints>
</ds>
</cdp_prep>
<database>
<!-- 2015-11-24 16:31:30 CET / 1448379090 --> <row><v>2.0000000000e+00</v></row>
<!-- 2015-11-24 16:31:35 CET / 1448379095 --> <row><v>9.6584168000e+00</v></row>
<!-- 2015-11-24 16:31:40 CET / 1448379100 --> <row><v>1.3225379200e+01</v></row>
<!-- 2015-11-24 16:31:45 CET / 1448379105 --> <row><v>1.9166821600e+01</v></row>
<!-- 2015-11-24 16:31:50 CET / 1448379110 --> <row><v>2.5747227200e+01</v></row>
<!-- 2015-11-24 16:31:55 CET / 1448379115 --> <row><v>2.8526251200e+01</v></row>
<!-- 2015-11-24 16:32:00 CET / 1448379120 --> <row><v>3.8413156800e+01</v></row>
<!-- 2015-11-24 16:32:05 CET / 1448379125 --> <row><v>3.7715062400e+01</v></row>
<!-- 2015-11-24 16:32:10 CET / 1448379130 --> <row><v>5.1284544000e+01</v></row>
<!-- 2015-11-24 16:32:15 CET / 1448379135 --> <row><v>4.6702324000e+01</v></row>
</database>
</rra>
</rrd>

RRDtool 1.5.5 Copyright by Tobias Oetiker <***@oetiker.ch>
Compiled Nov 24 2015 16:14:27

Questions
1. Is there a way to get the values I add to show up unaltered in the RRA?
Tried with LAST, AVERAGE, MIN and MAX in the RRA declaration.

2. How are the values in my RRA calculated?

[Simon Hobson]

Post by Thomas Nilsson
1. Is there a way to get the values I add to show up unaltered in the RRA?
Tried with LAST, AVERAGE, MIN and MAX in the RRA declaration.
2. How are the values in my RRA calculated?

These together must be one of the most frequently asked questions, to which there is a simple answer :
http://rrdtool.vandenbogaerdt.nl

In particular, "Rates, normalizing and consolidating" http://rrdtool.vandenbogaerdt.nl/process.php

The latter will answer question 2. Once you understand that, then you'll understand the answer to question 1 which is: ensure that the normalisation process is a "null operation". That means you must supply data on *exactly* step boundaries, for every step, only once per step, and missing no steps out.

So in your example, you must ensure that the timestamp of every update is on *exactly* a multiple of 5 seconds since unix epoch - not "now", not "a 5 second sleep plus the execution time of a few statements after the previous one", but on exactly a multiple of 5 seconds.
Thus (in bash) you can do :
s=5
t=date +%s
t=$( ( $t / $s ) * $s )
rrdtool update ... $t:$v

This will ensure that the values stored will be exactly what you put in - but only until there is some consolidation.

But in general, if you want to store exact values, and especially if they are produced at irregular times, then RRD is probably not the right tool.