Use the same unit for all data in /api/dashboard

Ideas & Suggestions
1

I am implementing Zabbix monitoring of my storage nodes and I can do it entirely in Zabbix with one issue: Zabbix can’t represent the number of bytes as floating point (it’s out of range). As a result, I have to represent this as an integer.

However, I can’t just use the raw numbers that come back for storage since they are in GB, so I need to multiply by 1024 * 1024 * 1024 to convert from GB to B (Zabbix does strange things when the unit is GB, displaying KGB instead of TB for example). Unfortunately, the integer truncation happens before the multiplication meaning I’ll only ever see whole-GB values and never fractional GBs in charts.

A simple solution to this would be to unify on bytes as the unit for the dashboard API. Currently, the units are mixed which already causes confusion:

  • Data using GB as the unit:
    • diskSpace
    • bandwidth.used
    • bandwidth.available
  • Data using B as the unit:
    • bandwidth.egress
    • bandwidth.ingress

This mixing of units is strange as it is. Calculations become a lot easier if everything is represented in bytes. There’s no precision loss as a result of using floating-point, and we don’t have to try to infer the units in use.

I may write a simple proxy to perform this conversion for me before Zabbix sees the data, but it would be great if this could be corrected so that a proxy isn’t required to pull consistent data.

1 Like
2

I’ve developed the following python-flask proxy that corrects this problem for me.

#!/usr/bin/env python3

from flask import Flask, request, jsonify
import requests

app = Flask('storj-dashboard')

def GBtoB(v):
    return int(v * 1024 * 1024 * 1024)

GBpaths = [
    'data.diskSpace.used',
    'data.diskSpace.available',
    'data.bandwidth.used',
    'data.bandwidth.available',
]

def mutatePath(obj, path, fn):
    p = path.split('.')
    last = p.pop()

    o = obj

    for i in p:
        o = o[i]

    o[last] = fn(o[last])

@app.route('/api/dashboard')
def dashboard():
    r = requests.get('http://localhost:14002/api/dashboard')
    d = r.json()

    for i in GBpaths:
        mutatePath(d, i, GBtoB)

    return jsonify(d)