Zabbix Performance tunning (with Proxies)
We setup a distributed monitoring cluster with one Zabbix (3.4.7) Server and 8 Proxies:
Zabbix:
OS: Debian Stretch
CPU: 16*2.27GHz
RAM: 48GB
Disk: Raid1 10K (Non-SSD)
LogFile=/var/log/zabbix/zabbix_server.log
PidFile=/var/run/zabbix/zabbix_server.pid
DBName=zabbix
DBUser=zabbix
DBHost=127.0.0.1
DBPort=3307
LogFileSize=0
DBPassword=****
Timeout=4
AlertScriptsPath=/etc/zabbix/alert.d/
FpingLocation=/usr/bin/fping
LogSlowQueries=3000
Include=/etc/zabbix/zabbix_server.conf.d/*.conf
StartAlerters=10
StartPollers=80
StartPollersUnreachable=80
StartTrappers=20
StartPingers=30
StartEscalators=5
CacheSize=8G
StartDBSyncers=16
HistoryCacheSize=2048M
TrendCacheSize=256M
ValueCacheSize=10G
HistoryIndexCacheSize=2G
ExternalScripts=/etc/zabbix/alert.d/
SSHKeyLocation=/nonexistent/.ssh
Proxy:
OS: Debian Stretch
CPU: 15*2.5GHz
RAM: 6GB
Disk: Raid1 10K (Non-SSD)
Server=XXXX
Hostname=zbx-lte
LogFile=/var/log/zabbix/zabbix_proxy.log
LogFileSize=0
PidFile=/var/run/zabbix/zabbix_proxy.pid
SocketDir=/var/run/zabbix
DBName=zabbix
DBUser=zabbix
DBPassword=159753
ConfigFrequency=600
DataSenderFrequency=1
StartPollers=240
StartPollersUnreachable=80
StartTrappers=20
StartPingers=80
SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
CacheSize=1G
StartDBSyncers=16
HistoryCacheSize=2048M
HistoryIndexCacheSize=2G
Timeout=6
ExternalScripts=/usr/lib/zabbix/externalscripts
FpingLocation=/usr/bin/fping
LogSlowQueries=3000
We're monitoring near 1650 nodes (snmp, icmp, agent, ssh ,external scripts and external applications) by Zabbix
Since (about ) 2 month ago , we saw so many lags in non-icmp graphs (Pic: 5.png) on specific zabbix proxy (zbx-lte in pics)
this graph is related to a device (with snmp v2). (there are many other graph like this on this proxy)
I capture network traffic of relevant zabbix proxy , For one of items , zabbix proxy sends only 16 queries instead of 60 queries (with interval 1m for 1 hour) .
If I change this device to monitor by Zabbix Server or any other proxies everything work properly
It sounds there is a problem on this proxy
Please help me to find the root cause.
performance proxy zabbix
add a comment |
We setup a distributed monitoring cluster with one Zabbix (3.4.7) Server and 8 Proxies:
Zabbix:
OS: Debian Stretch
CPU: 16*2.27GHz
RAM: 48GB
Disk: Raid1 10K (Non-SSD)
LogFile=/var/log/zabbix/zabbix_server.log
PidFile=/var/run/zabbix/zabbix_server.pid
DBName=zabbix
DBUser=zabbix
DBHost=127.0.0.1
DBPort=3307
LogFileSize=0
DBPassword=****
Timeout=4
AlertScriptsPath=/etc/zabbix/alert.d/
FpingLocation=/usr/bin/fping
LogSlowQueries=3000
Include=/etc/zabbix/zabbix_server.conf.d/*.conf
StartAlerters=10
StartPollers=80
StartPollersUnreachable=80
StartTrappers=20
StartPingers=30
StartEscalators=5
CacheSize=8G
StartDBSyncers=16
HistoryCacheSize=2048M
TrendCacheSize=256M
ValueCacheSize=10G
HistoryIndexCacheSize=2G
ExternalScripts=/etc/zabbix/alert.d/
SSHKeyLocation=/nonexistent/.ssh
Proxy:
OS: Debian Stretch
CPU: 15*2.5GHz
RAM: 6GB
Disk: Raid1 10K (Non-SSD)
Server=XXXX
Hostname=zbx-lte
LogFile=/var/log/zabbix/zabbix_proxy.log
LogFileSize=0
PidFile=/var/run/zabbix/zabbix_proxy.pid
SocketDir=/var/run/zabbix
DBName=zabbix
DBUser=zabbix
DBPassword=159753
ConfigFrequency=600
DataSenderFrequency=1
StartPollers=240
StartPollersUnreachable=80
StartTrappers=20
StartPingers=80
SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
CacheSize=1G
StartDBSyncers=16
HistoryCacheSize=2048M
HistoryIndexCacheSize=2G
Timeout=6
ExternalScripts=/usr/lib/zabbix/externalscripts
FpingLocation=/usr/bin/fping
LogSlowQueries=3000
We're monitoring near 1650 nodes (snmp, icmp, agent, ssh ,external scripts and external applications) by Zabbix
Since (about ) 2 month ago , we saw so many lags in non-icmp graphs (Pic: 5.png) on specific zabbix proxy (zbx-lte in pics)
this graph is related to a device (with snmp v2). (there are many other graph like this on this proxy)
I capture network traffic of relevant zabbix proxy , For one of items , zabbix proxy sends only 16 queries instead of 60 queries (with interval 1m for 1 hour) .
If I change this device to monitor by Zabbix Server or any other proxies everything work properly
It sounds there is a problem on this proxy
Please help me to find the root cause.
performance proxy zabbix
Load average is quite high, considering the 16 CPUs. Could you check the iowait of the system and share a screenshot ?
– Simone Zabberoni
yesterday
@SimoneZabberoni top screenshot added
– Sassan torabkheslat
21 hours ago
add a comment |
We setup a distributed monitoring cluster with one Zabbix (3.4.7) Server and 8 Proxies:
Zabbix:
OS: Debian Stretch
CPU: 16*2.27GHz
RAM: 48GB
Disk: Raid1 10K (Non-SSD)
LogFile=/var/log/zabbix/zabbix_server.log
PidFile=/var/run/zabbix/zabbix_server.pid
DBName=zabbix
DBUser=zabbix
DBHost=127.0.0.1
DBPort=3307
LogFileSize=0
DBPassword=****
Timeout=4
AlertScriptsPath=/etc/zabbix/alert.d/
FpingLocation=/usr/bin/fping
LogSlowQueries=3000
Include=/etc/zabbix/zabbix_server.conf.d/*.conf
StartAlerters=10
StartPollers=80
StartPollersUnreachable=80
StartTrappers=20
StartPingers=30
StartEscalators=5
CacheSize=8G
StartDBSyncers=16
HistoryCacheSize=2048M
TrendCacheSize=256M
ValueCacheSize=10G
HistoryIndexCacheSize=2G
ExternalScripts=/etc/zabbix/alert.d/
SSHKeyLocation=/nonexistent/.ssh
Proxy:
OS: Debian Stretch
CPU: 15*2.5GHz
RAM: 6GB
Disk: Raid1 10K (Non-SSD)
Server=XXXX
Hostname=zbx-lte
LogFile=/var/log/zabbix/zabbix_proxy.log
LogFileSize=0
PidFile=/var/run/zabbix/zabbix_proxy.pid
SocketDir=/var/run/zabbix
DBName=zabbix
DBUser=zabbix
DBPassword=159753
ConfigFrequency=600
DataSenderFrequency=1
StartPollers=240
StartPollersUnreachable=80
StartTrappers=20
StartPingers=80
SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
CacheSize=1G
StartDBSyncers=16
HistoryCacheSize=2048M
HistoryIndexCacheSize=2G
Timeout=6
ExternalScripts=/usr/lib/zabbix/externalscripts
FpingLocation=/usr/bin/fping
LogSlowQueries=3000
We're monitoring near 1650 nodes (snmp, icmp, agent, ssh ,external scripts and external applications) by Zabbix
Since (about ) 2 month ago , we saw so many lags in non-icmp graphs (Pic: 5.png) on specific zabbix proxy (zbx-lte in pics)
this graph is related to a device (with snmp v2). (there are many other graph like this on this proxy)
I capture network traffic of relevant zabbix proxy , For one of items , zabbix proxy sends only 16 queries instead of 60 queries (with interval 1m for 1 hour) .
If I change this device to monitor by Zabbix Server or any other proxies everything work properly
It sounds there is a problem on this proxy
Please help me to find the root cause.
performance proxy zabbix
We setup a distributed monitoring cluster with one Zabbix (3.4.7) Server and 8 Proxies:
Zabbix:
OS: Debian Stretch
CPU: 16*2.27GHz
RAM: 48GB
Disk: Raid1 10K (Non-SSD)
LogFile=/var/log/zabbix/zabbix_server.log
PidFile=/var/run/zabbix/zabbix_server.pid
DBName=zabbix
DBUser=zabbix
DBHost=127.0.0.1
DBPort=3307
LogFileSize=0
DBPassword=****
Timeout=4
AlertScriptsPath=/etc/zabbix/alert.d/
FpingLocation=/usr/bin/fping
LogSlowQueries=3000
Include=/etc/zabbix/zabbix_server.conf.d/*.conf
StartAlerters=10
StartPollers=80
StartPollersUnreachable=80
StartTrappers=20
StartPingers=30
StartEscalators=5
CacheSize=8G
StartDBSyncers=16
HistoryCacheSize=2048M
TrendCacheSize=256M
ValueCacheSize=10G
HistoryIndexCacheSize=2G
ExternalScripts=/etc/zabbix/alert.d/
SSHKeyLocation=/nonexistent/.ssh
Proxy:
OS: Debian Stretch
CPU: 15*2.5GHz
RAM: 6GB
Disk: Raid1 10K (Non-SSD)
Server=XXXX
Hostname=zbx-lte
LogFile=/var/log/zabbix/zabbix_proxy.log
LogFileSize=0
PidFile=/var/run/zabbix/zabbix_proxy.pid
SocketDir=/var/run/zabbix
DBName=zabbix
DBUser=zabbix
DBPassword=159753
ConfigFrequency=600
DataSenderFrequency=1
StartPollers=240
StartPollersUnreachable=80
StartTrappers=20
StartPingers=80
SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
CacheSize=1G
StartDBSyncers=16
HistoryCacheSize=2048M
HistoryIndexCacheSize=2G
Timeout=6
ExternalScripts=/usr/lib/zabbix/externalscripts
FpingLocation=/usr/bin/fping
LogSlowQueries=3000
We're monitoring near 1650 nodes (snmp, icmp, agent, ssh ,external scripts and external applications) by Zabbix
Since (about ) 2 month ago , we saw so many lags in non-icmp graphs (Pic: 5.png) on specific zabbix proxy (zbx-lte in pics)
this graph is related to a device (with snmp v2). (there are many other graph like this on this proxy)
I capture network traffic of relevant zabbix proxy , For one of items , zabbix proxy sends only 16 queries instead of 60 queries (with interval 1m for 1 hour) .
If I change this device to monitor by Zabbix Server or any other proxies everything work properly
It sounds there is a problem on this proxy
Please help me to find the root cause.
performance proxy zabbix
performance proxy zabbix
edited 21 hours ago
asked yesterday
Sassan torabkheslat
3619
3619
Load average is quite high, considering the 16 CPUs. Could you check the iowait of the system and share a screenshot ?
– Simone Zabberoni
yesterday
@SimoneZabberoni top screenshot added
– Sassan torabkheslat
21 hours ago
add a comment |
Load average is quite high, considering the 16 CPUs. Could you check the iowait of the system and share a screenshot ?
– Simone Zabberoni
yesterday
@SimoneZabberoni top screenshot added
– Sassan torabkheslat
21 hours ago
Load average is quite high, considering the 16 CPUs. Could you check the iowait of the system and share a screenshot ?
– Simone Zabberoni
yesterday
Load average is quite high, considering the 16 CPUs. Could you check the iowait of the system and share a screenshot ?
– Simone Zabberoni
yesterday
@SimoneZabberoni top screenshot added
– Sassan torabkheslat
21 hours ago
@SimoneZabberoni top screenshot added
– Sassan torabkheslat
21 hours ago
add a comment |
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53944786%2fzabbix-performance-tunning-with-proxies%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53944786%2fzabbix-performance-tunning-with-proxies%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Load average is quite high, considering the 16 CPUs. Could you check the iowait of the system and share a screenshot ?
– Simone Zabberoni
yesterday
@SimoneZabberoni top screenshot added
– Sassan torabkheslat
21 hours ago