Append field to json with logstash The 2019 Stack Overflow Developer Survey Results Are In Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern) 2019 Community Moderator Election Resultshow to make logstash read kannel logLogstash: Handshake failed i/o timeoutlogstash regex match in if conditiongeoip logstash filter not working in Debianlogstash - take 2 - filter to send messages from IntelMQ/python/redis to ELKRsyslog doesn't send logs to logstash server on portlogstash: Trying to extract substrings from pathLogStash Json ParserErrorLogstash Unrecognized service Amazon LinuxLogstash output to nginx
Drawing vertical/oblique lines in Metrical tree (tikz-qtree, tipa)
ELI5: Why do they say that Israel would have been the fourth country to land a spacecraft on the Moon and why do they call it low cost?
Do working physicists consider Newtonian mechanics to be "falsified"?
Does Parliament hold absolute power in the UK?
Why can't devices on different VLANs, but on the same subnet, communicate?
My body leaves; my core can stay
Is this wall load bearing? Blueprints and photos attached
Could an empire control the whole planet with today's comunication methods?
Is it ethical to upload a automatically generated paper to a non peer-reviewed site as part of a larger research?
What do I do when my TA workload is more than expected?
What was the last x86 CPU that did not have the x87 floating-point unit built in?
Intergalactic human space ship encounters another ship, character gets shunted off beyond known universe, reality starts collapsing
What can I do if neighbor is blocking my solar panels intentionally?
Why are PDP-7-style microprogrammed instructions out of vogue?
Windows 10: How to Lock (not sleep) laptop on lid close?
Why not take a picture of a closer black hole?
Keeping a retro style to sci-fi spaceships?
Is an up-to-date browser secure on an out-of-date OS?
Can I visit the Trinity College (Cambridge) library and see some of their rare books
How to determine omitted units in a publication
Word for: a synonym with a positive connotation?
Can the DM override racial traits?
Word to describe a time interval
Simulating Exploding Dice
Append field to json with logstash
The 2019 Stack Overflow Developer Survey Results Are In
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)
2019 Community Moderator Election Resultshow to make logstash read kannel logLogstash: Handshake failed i/o timeoutlogstash regex match in if conditiongeoip logstash filter not working in Debianlogstash - take 2 - filter to send messages from IntelMQ/python/redis to ELKRsyslog doesn't send logs to logstash server on portlogstash: Trying to extract substrings from pathLogStash Json ParserErrorLogstash Unrecognized service Amazon LinuxLogstash output to nginx
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
my logstash config is:
input
kafka
bootstrap_servers => "172.16.70.4:9092"
group_id => "all_topic"
topics => ["topic1", "topic2"]
auto_offset_reset => "earliest"
decorate_events => true
output
if ([@metadata][kafka][topic] == "topic1")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic1"
codec => line format => "%message"
if ([@metadata][kafka][topic] == "topic2")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic2"
codec => line format => "%message"
and message is json format from server 172.16.70.4:id:123456,time:20190225,color:black,result:1
I want after send to server 172.16.70.44 add field tag to message, this like:id:123456,time:20190225,color:black,result:1,tag:server1
Update
input
kafka
bootstrap_servers => "172.16.70.4:9092"
group_id => "all_topic"
topics => ["topic1", "topic2"]
auto_offset_reset => "earliest"
decorate_events => true
filter
json
source => "message"
add_field => "tag" => "hello"
add_tag => [ "mytag" ]
id => "my_id"
remove_field => [ "color" ]
mutate add_field => "tag" => "%[message][my_tag]"
mutate add_field => "tag" => "my_tag"
mutate add_field => "[@metadata][tag]" => "my_tag"
mutate add_field => "[docs][test]" => "my_tag"
output
if ([@metadata][kafka][topic] == "topic1")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic1"
codec => line format => "%message"
if ([@metadata][kafka][topic] == "topic2")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic2"
codec => line format => "%message"
I checked all method and filter but nothing happens. Field or tag not added or field not remove and output is the original json message.
logs logstash
asked Apr 9 at 6:11
pyramid13pyramid13
120616
add a comment |
my logstash config is:
input
kafka
bootstrap_servers => "172.16.70.4:9092"
group_id => "all_topic"
topics => ["topic1", "topic2"]
auto_offset_reset => "earliest"
decorate_events => true
output
if ([@metadata][kafka][topic] == "topic1")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic1"
codec => line format => "%message"
if ([@metadata][kafka][topic] == "topic2")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic2"
codec => line format => "%message"
and message is json format from server 172.16.70.4:id:123456,time:20190225,color:black,result:1
I want after send to server 172.16.70.44 add field tag to message, this like:id:123456,time:20190225,color:black,result:1,tag:server1
Update
input
kafka
bootstrap_servers => "172.16.70.4:9092"
group_id => "all_topic"
topics => ["topic1", "topic2"]
auto_offset_reset => "earliest"
decorate_events => true
filter
json
source => "message"
add_field => "tag" => "hello"
add_tag => [ "mytag" ]
id => "my_id"
remove_field => [ "color" ]
mutate add_field => "tag" => "%[message][my_tag]"
mutate add_field => "tag" => "my_tag"
mutate add_field => "[@metadata][tag]" => "my_tag"
mutate add_field => "[docs][test]" => "my_tag"
output
if ([@metadata][kafka][topic] == "topic1")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic1"
codec => line format => "%message"
if ([@metadata][kafka][topic] == "topic2")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic2"
codec => line format => "%message"
I checked all method and filter but nothing happens. Field or tag not added or field not remove and output is the original json message.
logs logstash
asked Apr 9 at 6:11
pyramid13pyramid13
120616
The JSON filter plugin should be able to do this.
– Haxiel
Apr 9 at 10:22
3
I'm voting to close this question as off-topic because it is primarily about configuring a pipeline on Logstash, which is a cross-platform tool.
– Haxiel
Apr 9 at 10:24
add a comment |
my logstash config is:
input
kafka
bootstrap_servers => "172.16.70.4:9092"
group_id => "all_topic"
topics => ["topic1", "topic2"]
auto_offset_reset => "earliest"
decorate_events => true
output
if ([@metadata][kafka][topic] == "topic1")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic1"
codec => line format => "%message"
if ([@metadata][kafka][topic] == "topic2")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic2"
codec => line format => "%message"
and message is json format from server 172.16.70.4:id:123456,time:20190225,color:black,result:1
I want after send to server 172.16.70.44 add field tag to message, this like:id:123456,time:20190225,color:black,result:1,tag:server1
Update
input
kafka
bootstrap_servers => "172.16.70.4:9092"
group_id => "all_topic"
topics => ["topic1", "topic2"]
auto_offset_reset => "earliest"
decorate_events => true
filter
json
source => "message"
add_field => "tag" => "hello"
add_tag => [ "mytag" ]
id => "my_id"
remove_field => [ "color" ]
mutate add_field => "tag" => "%[message][my_tag]"
mutate add_field => "tag" => "my_tag"
mutate add_field => "[@metadata][tag]" => "my_tag"
mutate add_field => "[docs][test]" => "my_tag"
output
if ([@metadata][kafka][topic] == "topic1")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic1"
codec => line format => "%message"
if ([@metadata][kafka][topic] == "topic2")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic2"
codec => line format => "%message"
I checked all method and filter but nothing happens. Field or tag not added or field not remove and output is the original json message.
logs logstash
asked Apr 9 at 6:11
pyramid13pyramid13
120616
my logstash config is:
input
kafka
bootstrap_servers => "172.16.70.4:9092"
group_id => "all_topic"
topics => ["topic1", "topic2"]
auto_offset_reset => "earliest"
decorate_events => true
output
if ([@metadata][kafka][topic] == "topic1")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic1"
codec => line format => "%message"
if ([@metadata][kafka][topic] == "topic2")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic2"
codec => line format => "%message"
and message is json format from server 172.16.70.4:id:123456,time:20190225,color:black,result:1
I want after send to server 172.16.70.44 add field tag to message, this like:id:123456,time:20190225,color:black,result:1,tag:server1
Update
input
kafka
bootstrap_servers => "172.16.70.4:9092"
group_id => "all_topic"
topics => ["topic1", "topic2"]
auto_offset_reset => "earliest"
decorate_events => true
filter
json
source => "message"
add_field => "tag" => "hello"
add_tag => [ "mytag" ]
id => "my_id"
remove_field => [ "color" ]
mutate add_field => "tag" => "%[message][my_tag]"
mutate add_field => "tag" => "my_tag"
mutate add_field => "[@metadata][tag]" => "my_tag"
mutate add_field => "[docs][test]" => "my_tag"
output
if ([@metadata][kafka][topic] == "topic1")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic1"
codec => line format => "%message"
if ([@metadata][kafka][topic] == "topic2")
kafka
bootstrap_servers => "172.16.70.44:9092"
topic_id => "new_topic2"
codec => line format => "%message"
I checked all method and filter but nothing happens. Field or tag not added or field not remove and output is the original json message.
logs logstash
logs logstash
asked Apr 9 at 6:11
pyramid13pyramid13
120616
asked Apr 9 at 6:11
pyramid13pyramid13
120616
edited Apr 9 at 10:44
pyramid13
asked Apr 9 at 6:11
pyramid13pyramid13
120616
asked Apr 9 at 6:11
pyramid13pyramid13
120616
asked Apr 9 at 6:11
pyramid13pyramid13
120616
120616
The JSON filter plugin should be able to do this.
– Haxiel
Apr 9 at 10:22
3
I'm voting to close this question as off-topic because it is primarily about configuring a pipeline on Logstash, which is a cross-platform tool.
– Haxiel
Apr 9 at 10:24
add a comment |
The JSON filter plugin should be able to do this.
– Haxiel
Apr 9 at 10:22
3
I'm voting to close this question as off-topic because it is primarily about configuring a pipeline on Logstash, which is a cross-platform tool.
– Haxiel
Apr 9 at 10:24
The JSON filter plugin should be able to do this.
– Haxiel
Apr 9 at 10:22
The JSON filter plugin should be able to do this.
– Haxiel
Apr 9 at 10:22
3
3
I'm voting to close this question as off-topic because it is primarily about configuring a pipeline on Logstash, which is a cross-platform tool.
– Haxiel
Apr 9 at 10:24
I'm voting to close this question as off-topic because it is primarily about configuring a pipeline on Logstash, which is a cross-platform tool.
– Haxiel
Apr 9 at 10:24
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f511377%2fappend-field-to-json-with-logstash%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f511377%2fappend-field-to-json-with-logstash%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
The JSON filter plugin should be able to do this.
– Haxiel
Apr 9 at 10:22
3
I'm voting to close this question as off-topic because it is primarily about configuring a pipeline on Logstash, which is a cross-platform tool.
– Haxiel
Apr 9 at 10:24