In this series of articles, we will create central log management using ELK (Elaticsearch, Logstash, Kibana). In this article of the series, I will only talk about the steps we will follow and the environments. At the end of the article I will share the whole topology.
We will perform ELK and beat installations hands-on. We will talk about how to make a basic central log management structure according to the scenario below and we will create this scenario step by step. In the end, we will show you how to do mail notification when something we didn’t want happened.
Servers we will collect logs:
- 3 Microsoft Server 2019 Web Servers
- 1 Linux Ubuntu Server 2019.10 Server
Logs to be Collected in Central Log Management Infrastructure:
Logs to be Collected From Windows Servers:
- We will collect Event Logs with winlogbeat.
- We will collect CPU, RAM and Disk usage logs with metricbeat.
- We will collect ICMP and Http logs at the Network Level with packetbet.
- We will copy IIS logs from Web Servers with filebeat.
On the Linux server, we will collect syslog information with filebeat.
Finally, I will tell you how to create an alert when a situation we want to follow occurs.
Environments:
1)Elasticsearch:
IP Address:10.250.2.221
OS:Ubuntu 19.10
2)Logstash:
IP Address:10.250.2.222
OS:Ubuntu 19.10
3)Kibana:
IP Address:10.250.2.223
OS:Ubuntu 19.10
4)Web Server 1:
IP Address:10.250.2.224
OS: Windows Server 2019
5)Web Server 2:
IP Address:10.250.2.225
OS: Windows Server 2019
6)Web Server 3:
IP Address:10.250.2.226
OS:Windows Server 2019
7)Linux Server:
IP Address:10.250.2.227
OS:Linux Server 2019.10
We will follow the steps below in our article series:
- We will collect Windows event logs with winlogbeat.
- We will collect the CPU, RAM and Disk Logs of Windows servers with Metricbeat.
- We will collect the network package logs with packetbeat (for this we will need to install winpcap on the windows server first).
- We will collect IIS logs with filebeat.
- We will transfer Linux server syslogs to Logstash with file beat.
- We will filter through logstash and edit the data formats and export them to elasticsearch.
- Using the indices created on Elasticsearch, we will present a web interface to the end user by visualizing with Kibana.
In the next article, we will start installing our structure step by step by installing elasticsearch on Ubuntu Server 2019 with IP address 10.250.2.221.
You can read our other articles in this serie from the links below.
Install Elasticsearch on ubuntu server 19.10
Install Logstash on Ubuntu Server 19.10
Install Kibana on Ubuntu Server 19.10
Install Winlogbeat on Windows Server 2019
Configure Logstash to Read log files Windows
Create Kibana Dashboards For Windows Event Logs
