Difference between revisions of "Monitoring dCache with ELK"
Jump to navigation
Jump to search
| (2 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| − | = PIC & ELK = | + | = PIC & ELK '''(obsolete)''' = |
* PIC has an ELK (ElasticSearch + LogStash + Kibana) service in order to monitor the different ''batch'' & ''storage'' services. | * PIC has an ELK (ElasticSearch + LogStash + Kibana) service in order to monitor the different ''batch'' & ''storage'' services. | ||
* A Kibana4 Dashboard can be accessed in the following link: '''[https://kibana4.pic.es:8443/#/dashboard?_g=%28%29 Kibana4]''' | * A Kibana4 Dashboard can be accessed in the following link: '''[https://kibana4.pic.es:8443/#/dashboard?_g=%28%29 Kibana4]''' | ||
| Line 6: | Line 6: | ||
== Monitoring the number of transfers, stores, restores, removes & hits in time per VO == | == Monitoring the number of transfers, stores, restores, removes & hits in time per VO == | ||
* Link: '''[https://kibana4.pic.es:8443/#/dashboard/dCache-Statistics-in-Time-per-VO?_g=%28%29&_a=%28filters:!%28%29,panels:!%28%28col:1,id:Count-%23hits-in-time-per-VO,row:6,size_x:6,size_y:5,type:visualization%29,%28col:7,id:Count-%23removes-in-time-per-VO,row:6,size_x:6,size_y:5,type:visualization%29,%28col:1,id:Count-%23restores-in-time-per-VO,row:11,size_x:6,size_y:5,type:visualization%29,%28col:1,id:Count-%23transfers-in-time-per-VO,row:1,size_x:12,size_y:5,type:visualization%29,%28col:7,id:Count-%23stores-in-time-per-VO,row:11,size_x:6,size_y:5,type:visualization%29%29,query:%28query_string:%28analyze_wildcard:!t,query:%27*%27%29%29,title:%27dCache%20Statistics%20in%20Time%20per%20VO%27%29 dCache Statistics in Time per VO]''' | * Link: '''[https://kibana4.pic.es:8443/#/dashboard/dCache-Statistics-in-Time-per-VO?_g=%28%29&_a=%28filters:!%28%29,panels:!%28%28col:1,id:Count-%23hits-in-time-per-VO,row:6,size_x:6,size_y:5,type:visualization%29,%28col:7,id:Count-%23removes-in-time-per-VO,row:6,size_x:6,size_y:5,type:visualization%29,%28col:1,id:Count-%23restores-in-time-per-VO,row:11,size_x:6,size_y:5,type:visualization%29,%28col:1,id:Count-%23transfers-in-time-per-VO,row:1,size_x:12,size_y:5,type:visualization%29,%28col:7,id:Count-%23stores-in-time-per-VO,row:11,size_x:6,size_y:5,type:visualization%29%29,query:%28query_string:%28analyze_wildcard:!t,query:%27*%27%29%29,title:%27dCache%20Statistics%20in%20Time%20per%20VO%27%29 dCache Statistics in Time per VO]''' | ||
| − | * In this Dashboard we can find how many stores and restores have run in the last X hours (default value is | + | * In this Dashboard we can find how many stores and restores have run in the last X hours (default value is 15 minutes), as well as the amount of transfers and hits. Number of removed files is also shown. |
== Monitoring transfers == | == Monitoring transfers == | ||
Latest revision as of 15:16, 13 June 2022
PIC & ELK (obsolete)
- PIC has an ELK (ElasticSearch + LogStash + Kibana) service in order to monitor the different batch & storage services.
- A Kibana4 Dashboard can be accessed in the following link: Kibana4
Monitoring Dashboards
Monitoring the number of transfers, stores, restores, removes & hits in time per VO
- Link: dCache Statistics in Time per VO
- In this Dashboard we can find how many stores and restores have run in the last X hours (default value is 15 minutes), as well as the amount of transfers and hits. Number of removed files is also shown.
Monitoring transfers
- Link: dCache Transfer Dashboard
- Here we have different plots:
- A plot showing information about moved data in the last X hours for the top 20 dCache pools. Here also is specified how many data has been moved by protocol and which protocols have been used per pool.
- A plot showing information about moved data in the last X hours for all dCache pools. Here also is specified how many data has been moved by protocol and which protocols have been used per pool.
- A stack plot showing the amount of data moved by DN.
- A window with percentil statistics.
Transferred data by protocol
- Link: dCache Transferred Bytes (Read & Writes) Dashboard
- In this Dashboard we have several plots, basically showing the amount of data moved by each protocol.
- A big plot is showing the amount of data moved by all protocols
- We have also 2 plots showing the amount of data moved due to stores and restores.