Log-Exporter for Prometheus with Grafana Dashboard

kevink · February 19, 2021, 8:03pm

oops, that’s definitely a bug and not intentionally. thanks. I fixed it. All sources are now updated. Thanks for the report! I should check my dashboards more carefully next time

fmoledina · February 19, 2021, 8:17pm

All good, I just noticed it when I was pilfering the per node disk usage details from your dashboard to include in mine .

Krawi · July 19, 2021, 6:33pm

Hi all

I’m looking into the combined dashboard and have a question about the disk free numbers:

Node dashboard shows this:

Grafana dashboard shows higher value (Free+Trash ???):
Combined Dashboard

Which dashboard is wrong?

Skyblockpro1 · July 21, 2021, 11:32am

Oh I seem to see what’s happening I think The exporter see the trash as free space

Not 100% sure but when you add the values on the official dashboard it gives you the free space in exporter (approximately)

Skyblockpro1 · August 22, 2021, 8:31am

Hey guys I’m currently stuck.
I’m trying to create a graph that will display the delta of profits over 24H i seem not to be able to do it
any help will be much apprecheated
Thanks

kevink · August 22, 2021, 12:42pm

sum(delta(storj_payout_currentMonth{type=~"payout|held",nodename=~"$node.*"}[24h])) / 100

Skyblockpro1 · August 22, 2021, 3:34pm

Thanks it seems to work

Krawi · January 10, 2022, 7:50am

Hello, i have some strange behavior for a panel:

As you can see this specific panel shows still the old names of the satellites, other panels are showing the new names, but not this panel.

Krawi · May 4, 2022, 7:07am

Because of changes in the logs since V1.53.1 the log-exporter fails to provide parts of its data.

Here are the two files i modified to make it working again and create a new docker image.

Create a new Image

git clone https://github.com/kevinkk525/storj-log-exporter
cd storj-log-exporter
<copying new files>
sudo docker build -t storj-log-exporter-new .
(Could take some time, be patient)

Now there should be a new (local, not from Docker Hub) container image in your list → execute
docker images
for the list.

docker-compose.yml

  storj-log-exporter-new:
    image: storj-log-exporter-new:latest
    container_name: storj-log-exporter-new
    restart: unless-stopped
    user: "1000:1000"
    networks:
      - default
#    ports: 
#      - 9144:9144
    volumes:
      - type: bind
        source: <path_to_your_logfiles>
        target: /app/logs
    command: -config /app/config.yml

config.yml

global:
  config_version: 3
input:
  type: file
  path: /app/logs/node.log
  readall: false # Read from the beginning of the file? False means we start at the end of the file and read only new lines.
imports:
- type: grok_patterns
  dir: /app/logstash-patterns-core/patterns
grok_patterns:
metrics:
- type: counter
  name: storj_pieces_delete_successful
  help: Total number of deleted pieces
  match: '%{TIMESTAMP_ISO8601:UNWANTED}\s*%{LOGLEVEL}\s*piecedeleter\s*delete piece sent to trash\s*\{\"Process\"\: \"storagenode\"\, \"Satellite ID\"\: \"%{DATA:SatelliteID}\"\, \"Piece ID\"\: \"%{DATA:PieceID}\"\}'
  labels:
    satellite: '{{.SatelliteID}}'
    url: '{{if eq .SatelliteID "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}}ap1.storj.io:7777{{else if eq .SatelliteID "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}}us1.storj.io:7777{{else if eq .SatelliteID "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}}eu1.storj.io:7777{{else if eq .SatelliteID "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}}europe-north-1.tardigrade.io:7777{{else if eq .SatelliteID "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}}saltlake.tardigrade.io:7777{{else if eq .SatelliteID "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}}us2.storj.io:7777{{end}}'  
- type: counter
  name: storj_pieces_delete_failed
  help: Total number of failed deletes
  match: '%{TIMESTAMP_ISO8601:UNWANTED}\s*%{LOGLEVEL}\s*piecedeleter\s*could not send delete piece to trash\s*\{\"Process\"\: \"storagenode\"\, \"Satellite ID\"\: \"%{DATA:SatelliteID}\"\, \"Piece ID\"\: \"%{DATA:PieceID}\"\}'
  match: '%{TIMESTAMP_ISO8601:UNWANTED}\s*%{LOGLEVEL}\s*piecedeleter\s*delete failed\s*\{\"Process\"\: \"storagenode\"\, \"Satellite ID\"\: \"%{DATA:SatelliteID}\"\, \"Piece ID\"\: \"%{DATA:PieceID}\"\}'
  labels:
    satellite: '{{.SatelliteID}}'
    url: '{{if eq .SatelliteID "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}}ap1.storj.io:7777{{else if eq .SatelliteID "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}}us1.storj.io:7777{{else if eq .SatelliteID "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}}eu1.storj.io:7777{{else if eq .SatelliteID "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}}europe-north-1.tardigrade.io:7777{{else if eq .SatelliteID "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}}saltlake.tardigrade.io:7777{{else if eq .SatelliteID "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}}us2.storj.io:7777{{end}}'  

- type: counter
  name: storj_pieces_upload_started
  help: Total number of uploads started
  match: '%{TIMESTAMP_ISO8601:UNWANTED}\s*%{LOGLEVEL}\s*piecestore\s*upload started\s*\{\"Process\"\: \"storagenode\"\, \"Piece ID\"\: \"%{DATA:PieceID}\"\, \"Satellite ID\"\: \"%{DATA:SatelliteID}\"\, \"Action\"\: \"%{DATA:Action}\"\, \"Available Space\"\: %{INT:AvailableSpace}\}'
  labels:
    satellite: '{{.SatelliteID}}' 
    url: '{{if eq .SatelliteID "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}}ap1.storj.io:7777{{else if eq .SatelliteID "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}}us1.storj.io:7777{{else if eq .SatelliteID "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}}eu1.storj.io:7777{{else if eq .SatelliteID "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}}europe-north-1.tardigrade.io:7777{{else if eq .SatelliteID "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}}saltlake.tardigrade.io:7777{{else if eq .SatelliteID "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}}us2.storj.io:7777{{end}}'  
    type: '{{if eq .Action "PUT"}}usage{{else if eq .Action "PUT_REPAIR"}}repair{{else}}{{.Action}}{{end}}'
- type: counter
  name: storj_pieces_upload_successful
  help: Total number of successful uploads
  match: '%{TIMESTAMP_ISO8601:UNWANTED}\s*%{LOGLEVEL}\s*piecestore\s*uploaded\s*\{\"Process\"\: \"storagenode\"\, \"Piece ID\"\: \"%{DATA:PieceID}\"\, \"Satellite ID\"\: \"%{DATA:SatelliteID}\"\, \"Action\"\: \"%{DATA:Action}\"\, \"Size\"\: %{INT:Size}\}'
  labels:
    satellite: '{{.SatelliteID}}'   
    url: '{{if eq .SatelliteID "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}}ap1.storj.io:7777{{else if eq .SatelliteID "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}}us1.storj.io:7777{{else if eq .SatelliteID "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}}eu1.storj.io:7777{{else if eq .SatelliteID "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}}europe-north-1.tardigrade.io:7777{{else if eq .SatelliteID "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}}saltlake.tardigrade.io:7777{{else if eq .SatelliteID "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}}us2.storj.io:7777{{end}}'  
    type: '{{if eq .Action "PUT"}}usage{{else if eq .Action "PUT_REPAIR"}}repair{{else}}{{.Action}}{{end}}'
- type: counter
  name: storj_pieces_upload_canceled
  help: Total number of canceled uploads
  match: '%{TIMESTAMP_ISO8601:UNWANTED}\s*%{LOGLEVEL}\s*piecestore\s*upload canceled\s*\{\"Process\"\: \"storagenode\"\, \"Piece ID\"\: \"%{DATA:PieceID}\"\, \"Satellite ID\"\: \"%{DATA:SatelliteID}\"\, \"Action\"\: \"%{DATA:Action}\"\, \"Size\"\: %{INT:Size}\}'
  labels:
    satellite: '{{.SatelliteID}}'  
    url: '{{if eq .SatelliteID "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}}ap1.storj.io:7777{{else if eq .SatelliteID "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}}us1.storj.io:7777{{else if eq .SatelliteID "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}}eu1.storj.io:7777{{else if eq .SatelliteID "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}}europe-north-1.tardigrade.io:7777{{else if eq .SatelliteID "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}}saltlake.tardigrade.io:7777{{else if eq .SatelliteID "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}}us2.storj.io:7777{{end}}'  
    type: '{{if eq .Action "PUT"}}usage{{else if eq .Action "PUT_REPAIR"}}repair{{else}}{{.Action}}{{end}}'
- type: counter
  name: storj_pieces_upload_rejected
  help: Total number of rejected uploads
  match: '%{TIMESTAMP_ISO8601:UNWANTED}\s*%{LOGLEVEL}\s*piecestore\s*upload rejected\s*\{\"Process\"\: \"storagenode\"\, \"Piece ID\"\: \"%{DATA:PieceID}\"\, \"Satellite ID\"\: \"%{DATA:SatelliteID}\"\, \"Action\"\: \"%{DATA:Action}\"(\, \"Size\"\: %{INT:Size})?\}'
  labels:
    satellite: '{{.SatelliteID}}'  
    url: '{{if eq .SatelliteID "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}}ap1.storj.io:7777{{else if eq .SatelliteID "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}}us1.storj.io:7777{{else if eq .SatelliteID "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}}eu1.storj.io:7777{{else if eq .SatelliteID "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}}europe-north-1.tardigrade.io:7777{{else if eq .SatelliteID "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}}saltlake.tardigrade.io:7777{{else if eq .SatelliteID "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}}us2.storj.io:7777{{end}}'  
    type: '{{if eq .Action "PUT"}}usage{{else if eq .Action "PUT_REPAIR"}}repair{{else}}{{.Action}}{{end}}'
- type: counter
  name: storj_pieces_upload_failed
  help: Total number of failed  uploads
  match: '%{TIMESTAMP_ISO8601:UNWANTED}\s*%{LOGLEVEL}\s*piecestore\s*upload failed\s*\{\"Process\"\: \"storagenode\"\, \"Piece ID\"\: \"%{DATA:PieceID}\"\, \"Satellite ID\"\: \"%{DATA:SatelliteID}\"\, \"Action\"\: \"%{DATA:Action}\"(\,\s*"error": \"%{DATA:Error}\",\s*"errorVerbose": \"%{DATA:ErrorVerbose}\")?(\, \"Size\"\: %{INT:Size})?\}'
  labels:
    satellite: '{{.SatelliteID}}'   
    url: '{{if eq .SatelliteID "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}}ap1.storj.io:7777{{else if eq .SatelliteID "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}}us1.storj.io:7777{{else if eq .SatelliteID "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}}eu1.storj.io:7777{{else if eq .SatelliteID "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}}europe-north-1.tardigrade.io:7777{{else if eq .SatelliteID "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}}saltlake.tardigrade.io:7777{{else if eq .SatelliteID "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}}us2.storj.io:7777{{end}}'  
    type: '{{if eq .Action "PUT"}}usage{{else if eq .Action "PUT_REPAIR"}}repair{{else}}{{.Action}}{{end}}'
    error: '{{.Error}}'

- type: counter
  name: storj_pieces_download_started
  help: Total number of downloads started
  match: '%{TIMESTAMP_ISO8601:UNWANTED}\s*%{LOGLEVEL}\s*piecestore\s*download started\s*\{\"Process\"\: \"storagenode\"\, \"Piece ID\"\: \"%{DATA:PieceID}\"\, \"Satellite ID\"\: \"%{DATA:SatelliteID}\"\, \"Action\"\: \"%{DATA:Action}\"\}'
  labels:
    satellite: '{{.SatelliteID}}'    
    url: '{{if eq .SatelliteID "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}}ap1.storj.io:7777{{else if eq .SatelliteID "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}}us1.storj.io:7777{{else if eq .SatelliteID "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}}eu1.storj.io:7777{{else if eq .SatelliteID "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}}europe-north-1.tardigrade.io:7777{{else if eq .SatelliteID "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}}saltlake.tardigrade.io:7777{{else if eq .SatelliteID "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}}us2.storj.io:7777{{end}}'  
    type: '{{if eq .Action "GET"}}usage{{else if eq .Action "GET_REPAIR"}}repair{{else if eq .Action "GET_AUDIT"}}audit{{else}}{{.Action}}{{end}}'
- type: counter
  name: storj_pieces_download_successful
  help: Total number of successful downloads
  match: '%{TIMESTAMP_ISO8601:UNWANTED}\s*%{LOGLEVEL}\s*piecestore\s*downloaded\s*\{\"Process\"\: \"storagenode\"\, \"Piece ID\"\: \"%{DATA:PieceID}\"\, \"Satellite ID\"\: \"%{DATA:SatelliteID}\"\, \"Action\"\: \"%{DATA:Action}\"\}'
  labels:
    satellite: '{{.SatelliteID}}'    
    url: '{{if eq .SatelliteID "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}}ap1.storj.io:7777{{else if eq .SatelliteID "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}}us1.storj.io:7777{{else if eq .SatelliteID "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}}eu1.storj.io:7777{{else if eq .SatelliteID "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}}europe-north-1.tardigrade.io:7777{{else if eq .SatelliteID "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}}saltlake.tardigrade.io:7777{{else if eq .SatelliteID "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}}us2.storj.io:7777{{end}}'  
    type: '{{if eq .Action "GET"}}usage{{else if eq .Action "GET_REPAIR"}}repair{{else if eq .Action "GET_AUDIT"}}audit{{else}}{{.Action}}{{end}}'
- type: counter
  name: storj_pieces_download_canceled
  help: Total number of canceled downloads
  match: '%{TIMESTAMP_ISO8601:UNWANTED}\s*%{LOGLEVEL}\s*piecestore\s*download canceled\s*\{\"Process\"\: \"storagenode\"\, \"Piece ID\"\: \"%{DATA:PieceID}\"\, \"Satellite ID\"\: \"%{DATA:SatelliteID}\"\, \"Action\"\: \"%{DATA:Action}\"\}'
  labels:
    satellite: '{{.SatelliteID}}'  
    url: '{{if eq .SatelliteID "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}}ap1.storj.io:7777{{else if eq .SatelliteID "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}}us1.storj.io:7777{{else if eq .SatelliteID "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}}eu1.storj.io:7777{{else if eq .SatelliteID "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}}europe-north-1.tardigrade.io:7777{{else if eq .SatelliteID "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}}saltlake.tardigrade.io:7777{{else if eq .SatelliteID "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}}us2.storj.io:7777{{end}}'  
    type: '{{if eq .Action "GET"}}usage{{else if eq .Action "GET_REPAIR"}}repair{{else if eq .Action "GET_AUDIT"}}audit{{else}}{{.Action}}{{end}}'
- type: counter
  name: storj_pieces_download_failed
  help: Total number of failed downloads
  match: '%{TIMESTAMP_ISO8601:UNWANTED}\s*%{LOGLEVEL:LOGLEVEL}\s*piecestore\s*download failed\s*\{\"Process\"\: \"storagenode\"\, \"Piece ID\"\: \"%{DATA:PieceID}\"\, \"Satellite ID\"\: \"%{DATA:SatelliteID}\"\, \"Action\"\: \"%{DATA:Action}\"(\,\s*"error": \"%{DATA:Error}\",\s*"errorVerbose": \"%{DATA:ErrorVerbose}\")?\}'
  labels:
    satellite: '{{.SatelliteID}}'    
    url: '{{if eq .SatelliteID "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}}ap1.storj.io:7777{{else if eq .SatelliteID "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}}us1.storj.io:7777{{else if eq .SatelliteID "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}}eu1.storj.io:7777{{else if eq .SatelliteID "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}}europe-north-1.tardigrade.io:7777{{else if eq .SatelliteID "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}}saltlake.tardigrade.io:7777{{else if eq .SatelliteID "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}}us2.storj.io:7777{{end}}'  
    type: '{{if eq .Action "GET"}}usage{{else if eq .Action "GET_REPAIR"}}repair{{else if eq .Action "GET_AUDIT"}}audit{{else}}{{.Action}}{{end}}'
    error: '{{if eq .Action "GET_AUDIT"}}.Error{{else}}use of closed network connection{{end}}' # workaround for all unique write tcp error messages..

- type: counter
  name: storj_log_levels
  help: Total number of loglevel messages in log
  match: '%{TIMESTAMP_ISO8601:UNWANTED}\s*%{LOGLEVEL:LOGLEVEL}\s*%{DATA}\}'
  labels:
    loglevel: '{{.LOGLEVEL}}'

- type: counter
  name: storj_log_errors
  help: Total number of error messages in log and their type
  match: '%{TIMESTAMP_ISO8601:UNWANTED}\s*ERROR\s*%{DATA:TYPE}\s*\{%{DATA}\}'
  labels:
    loglevel: 'ERROR'
    type: '{{.TYPE}}'

server:
  protocol: http
  port: 9144
  path: /

fredondo · August 22, 2022, 4:15pm

Hi,

I try to run dashboard , everything looks fine but no susses for logs. All grok_exporter are ignored.

HELP grok_exporter_lines_total Total number of log lines processed by grok_exporter.

TYPE grok_exporter_lines_total counter

grok_exporter_lines_total{status=“ignored”} 11487
grok_exporter_lines_total{status=“matched”} 0

Regards,
Fernando

lyoth · June 18, 2023, 10:06pm

do you have an updated config.yml for this? I’m trying to implement this on my system.

Krawi · June 19, 2023, 6:40am

Replace every occurance of

\"Process\"

with

\"process\"

in the config.yml, it’s just a small change in the logs output to lowercase.

A good tool for the next change in the logs: https://grokdebugger.com/

asturking · November 1, 2023, 11:39pm

Hello, I want to set up this exporter to have the maximum information possible, but I’m not sure what the default path for the node.log file should be.
Or if you need to redirect the Docker log output to a file or what needs to be done.

remo · November 2, 2023, 4:49am

You should follow theese steps to redirect your logs to the location you want.
Usually logs redirected to same location where storagenode data folder is.
On linux it looks like this in config.yaml :

configures log encoding. can either be ‘console’, ‘json’, ‘pretty’, or ‘gcloudlogging’.

log.encoding: “console”

the minimum log level to log

log.level: info

can be stdout, stderr, or a filename

log.output: /app/config/node.log

asturking · November 2, 2023, 11:49am

Thanks!
I thought it could be executed from another host like the exporter that connects to the web interface.

Now I have it up and running, but it seems that it doesn’t collect all the data correctly. I see there’s a pull request on GitHub to address this change in the latest versions.

It would be great if @kevink could launch the combined dashboard.

asturking · November 2, 2023, 12:06pm

My bad, the dashboard actualy exist

hackschnitzel09 · December 22, 2023, 5:51pm

Hi, I have issues with the logs i get only the download fail cancel sucess srartet and the upload failed. But all the rest is empty. I changed "Process" to "process" with find and replace. Did anything change?

Walter1 · February 10, 2024, 12:33pm

Hello together,

thanks alot for the Log-Exporter. I’ve been using this now almost for a year and couldn’t operate properly without it.

Do know how to make the Grafana-Dashboard accessible through the internet? This would be great, right now I can reach it only by LAN with computer:3000 for example.

Thanks and kind regards,

Krawi · February 10, 2024, 1:54pm

Hello Walter

First option is portforwarding (like a node), but this gives anyone access to your Dashboard, so not really secure.
Second option is using tunneling through SSH.
Third option would be a VPN at home, perhaps your router can handle that itself.