Peerplays_Blockchain/peerplays_migrated

Fork 0

[Enhancement]: Push the blockchain logs to elasticsearch based on the blockchain cluster name #28

New issue

Closed

opened 2020-09-14 18:28:14 +00:00 by prandnum · 10 comments

prandnum commented

2020-09-14 18:28:14 +00:00

(Migrated from gitlab.com)

Logs are pushed to "peerplays-YYYY-MM" index.
This needs to be enhanced to "peerplays-{BLOCKCHAIN-CLUSTERNAME}-YYYY-MM" so that the indices would be named

"peerplays-elizabeth-2020-09"
"peerplays-gladiator-2020-09"
"peerplays-beatrice-2020-09"

This will help us in having a internal centralized logging server for all the blockchain clusters and help in segregating and querying the logs easier.

https://peerplays.atlassian.net/wiki/spaces/PIX/pages/364609555/ElasticSearch+Plugin+for+Peerplays

Logs are pushed to "peerplays-YYYY-MM" index. This needs to be enhanced to "peerplays-{BLOCKCHAIN-CLUSTERNAME}-YYYY-MM" so that the indices would be named - "peerplays-elizabeth-2020-09" - "peerplays-gladiator-2020-09" - "peerplays-beatrice-2020-09" This will help us in having a internal centralized logging server for all the blockchain clusters and help in segregating and querying the logs easier. https://peerplays.atlassian.net/wiki/spaces/PIX/pages/364609555/ElasticSearch+Plugin+for+Peerplays

prandnum commented

2020-09-14 18:32:06 +00:00

(Migrated from gitlab.com)

changed the description

prandnum commented

2020-09-14 18:34:38 +00:00

(Migrated from gitlab.com)

changed the description

gizord commented

2020-09-21 09:07:18 +00:00

(Migrated from gitlab.com)

I recommend you use logstash instead of adding this straight into your backend.
I believe that the logging facilities of graphene/peerplays allow for dumping logs into a file. Then parse those with logstash and push it to ES from there. Don't forget to shrink/remove the logs, e.g. with logrotate.

I recommend you use logstash instead of adding this straight into your backend. I believe that the logging facilities of graphene/peerplays allow for dumping logs into a file. Then parse those with logstash and push it to ES from there. Don't forget to shrink/remove the logs, e.g. with logrotate.

bobinson commented

2020-09-21 10:54:40 +00:00

(Migrated from gitlab.com)

@gizord - yea, this is a ELK deployment that we use for logs. This is more of an internal task and we also planning to make one of the nodes public for anyone to quickly access logs.

@gizord - yea, this is a ELK deployment that we use for logs. This is more of an internal task and we also planning to make one of the nodes public for anyone to quickly access logs. ![image](/uploads/ab713c8d7b799f630466153505f80372/image.png)

serkixenos commented

2021-02-17 12:44:50 +00:00

(Migrated from gitlab.com)

This can be achieved by setting elasticsearch-index-prefix property in config file.

Default value is

peerplays-

And resulting index name is peerplays-YYYY-MM

If set to

peerplays-gladiator-

Resulting index name would be

peerplays-gladiator-YYYY-MM-

This can be achieved by setting elasticsearch-index-prefix property in config file. Default value is peerplays- And resulting index name is peerplays-YYYY-MM If set to peerplays-gladiator- Resulting index name would be peerplays-gladiator-YYYY-MM-

serkixenos (Migrated from gitlab.com) closed this issue

2021-02-17 12:44:51 +00:00

bobinson (Migrated from gitlab.com) reopened this issue

2021-02-17 13:05:38 +00:00

bobinson commented

2021-02-17 13:06:06 +00:00

(Migrated from gitlab.com)

@prandnum - can you prepare a doc for this ?

bobinson commented

2021-02-17 13:06:07 +00:00

(Migrated from gitlab.com)

assigned to @prandnum

prandnum commented

2021-02-17 14:10:09 +00:00

(Migrated from gitlab.com)

Sure will work on it and publish it in gitbook

prandnum commented

2021-02-22 16:48:26 +00:00

(Migrated from gitlab.com)

Now we can configure peerpays blockchain operations index and peerplays blockchain objects using elasticsearch-index-prefix and es-objects-index-prefix parameters respectively.

prandnum (Migrated from gitlab.com) closed this issue

2021-02-22 16:48:47 +00:00

prandnum commented

2021-02-25 18:16:39 +00:00

(Migrated from gitlab.com)

Documentation will be tracked using https://gitlab.com/PBSA/peerplays/-/issues/46