Hi, since ES doesn't support the query history feature, we have two options
1- We can develop a snippet code with any language we choose to log the query into a database or file or whatever we want.
2- You are like me into “touching” infrastructure and servers, you can follow these easy steps with me.
What we will be using/need?
1- ES service
2- GCP project
3- SlowLog
4- kubectl
Port forwarding you ES service
Before this, we need to be connected to the cluster so we can list our service and forward it
$ kubectl get po$ kubectl port-forward elasticsearch-0 9200:9200
Activate SlowLog
SlowLog is actually a service that gives us an idea about queries performance and here we will be just doing some tiny hack because if we need to get all the query history and not only the slow ones we need to put the threshold at 0
$ curl -XPUT -d '{"index.search.slowlog.threshold.query.warn" : "0ms","index.search.slowlog.threshold.fetch.warn": "0ms"}' http://127.0.0.1:9200/your_index/_settingsCreate a sink in GCP
Create a sink in the logging section/log router
PS: Create a big query service or bucket we will need it after
1- Sink Name: We need to specify the sink name
2- Sink Service: We can choose sink service, it means where we want to save our logs and we can choose between big-query, Bucket … for me, it is easiest to choose a Bucket
3- Sink Destination: select for example the bucket that we created before we go into this
after we did this we can edit the search query of the logging service, for me, I need only the log that slow log show it not all the ES logs so we can write
resource.type=”k8s_container”
resource.labels.cluster_name=”your_cluster_name”
resource.labels.namespace_name=”default”
resource.labels.container_name=”elasticsearch”
textPayload:”i.s.s.query"and that is all folks ….
Peace from Tunisia
