elasticsearch - Remove fields by their name pattern -


we using logstash elasticsearch log of out application events. of event holds fields dynamically named.
want apply filter removed or merged them before entering elasticsearch.
example :

{     "root": {       "eventtype": "info",       "timestamp": 20150713153757.758     },     "event": {       "message": "itemsviews created in 1 msec",       "cache_11542": true,       "cache_10242": false,       "cache_55240": 124     }   }  

in case remove fields starting "cache_" under object event.

so output elasticsearch

{     "root": {       "eventtype": "info",       "timestamp": 20150713153757.758     },     "event": {       "message": "itemsviews created in 1 msec"   }   } 

is there way define filler in logstash configuration file achieve ?

many in advance.

looks ruby filter solution @magnus-bäck points out might solution. had suggested the mutate filter using "remove_field" array in conjunction gsub filter. gsub regex match cache* fields can renamed variable use in mutate. however, since have n-number of cache fields, ruby script better. :)


Comments