Monitoring dCache with ELK

From Public PIC Wiki
Revision as of 09:01, 7 August 2015 by Mcaubet (talk | contribs) (Created page with "= PIC & ELK = * PIC has an ELK (ElasticSearch + LogStash + Kibana) service in order to monitor the different ''batch'' & ''storage'' services. * A Kibana4 Dashboard can be acc...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

PIC & ELK

  • PIC has an ELK (ElasticSearch + LogStash + Kibana) service in order to monitor the different batch & storage services.
  • A Kibana4 Dashboard can be accessed in the following link:
https://kibana4.pic.es:8443/#/dashboard?_g=%28%29

Monitoring Dashboards

Monitoring the number of transfers, stores, restores, removes & hits in time per VO

  • Link: dCache Statistics in Time per VO
  • In this Dashboard we can find how many stores and restores have run in the last X hours (default value is 1 week), as well as the amount of transfers and hits. Number of removed files is also shown.

Monitoring transfers

  • A plot showing information about moved data in the last X hours for the top 20 dCache pools. Here also is specified how many data has been moved by protocol and which protocols have been used per pool.
  • A plot showing information about moved data in the last X hours for all dCache pools. Here also is specified how many data has been moved by protocol and which protocols have been used per pool.
  • A stack plot showing the amount of data moved by DN.
  • A window with percentil statistics.

Transferred data by protocol

  • Link: dCache Transferred Bytes (Read & Writes) Dashboard
  • In this Dashboard we have several plots, basically showing the amount of data moved by each protocol.
  • A big plot is showing the amount of data moved by all protocols
  • We have also 2 plots showing the amount of data moved due to stores and restores.