M.HAJJI
Comprehensive Installation Guide: Docker, Splunk, Elastic Stack & Cribl
Step 1: Install Docker
Docker enables containerization, allowing you to run complex applications like Splunk and Elastic Stack in isolated, reproducible environments. This simplifies deployment, scaling, and maintenance.
For production environments, ensure Docker is installed following best security practices such as limiting root access and enabling TLS for Docker API.
# On Ubuntu/Debian sudo apt update sudo apt install -y docker.io docker-compose sudo systemctl start docker sudo systemctl enable docker # Verify Docker installation docker --version docker-compose --version # On Windows or macOS # Download Docker Desktop from https://www.docker.com/get-started
Step 2: Install Splunk Enterprise with Docker Compose
Splunk Enterprise is a powerful platform for searching, monitoring, and analyzing machine-generated data via a web-style interface. Running Splunk inside Docker containers provides flexibility and isolation, making deployments cleaner and easier to manage.
Security tip: Always set a strong, complex password for the admin account, and consider integrating Splunk with LDAP or SSO for enterprise authentication.
version: '2.4'
volumes:
opt-splunk-etc: {}
opt-splunk-var: {}
services:
splunkenterprise:
container_name: splunk
hostname: splunkenterprise
image: splunk/splunk:latest
mem_limit: 2G
environment:
- SPLUNK_START_ARGS=--accept-license
- SPLUNK_ENABLE_LISTEN=9997
- SPLUNK_ADD=tcp 1514
- SPLUNK_PASSWORD=your_password
volumes:
- opt-splunk-etc:/opt/splunk/etc
- opt-splunk-var:/opt/splunk/var
ports:
- "8000:8000"
- "9997:9997"
- "8088:8088"
- "1514:1514"
- "8089:8089"
Step 3: Explanation for Splunk Docker Compose Setup
- Persistent Volumes: Volumes
opt-splunk-etcandopt-splunk-varpersist Splunk configurations and indexed data outside the container lifecycle. - Memory Limit: Restricting memory to 2GB ensures stable container operation without overwhelming host resources.
- Port Exposure: Common ports exposed allow web UI access, receiving data from forwarders, HTTP event collection, and syslog ingestion.
- Environment Variables: Accepts license and configures admin password to allow automated startup without manual intervention.
This reflects a professional approach to containerized enterprise software deployment, essential for infrastructure and security roles.
Step 4: Install Elastic Stack Using the Elastic Container Repo
Elastic Stack (Elasticsearch, Kibana, Beats, Logstash) is an open-source suite for centralized logging, analytics, and visualization. This containerized setup includes security best practices like TLS encryption, password management, and health checks.
To simplify installation, clone the GitHub repository below which contains the Docker Compose files, certificate generation logic, and setup scripts:
git clone https://github.com/mohamdhajji/elastic-secure-docker.git
cd elastic-secure-docker
cp .env.example .env # then set your passwords and stack version
./install.sh
The setup service handles automated certificate creation and secure password setup for the elastic and kibana_system users before starting the main services.
services:
setup:
image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
container_name: ecp-elasticsearch-security-setup
volumes:
- certs:/usr/share/elasticsearch/config/certs:z
user: "0"
command: >
bash -c '
if [ x${ELASTIC_PASSWORD} == x ]; then
echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
exit 1;
elif [ x${KIBANA_PASSWORD} == x ]; then
echo "Set the KIBANA_PASSWORD environment variable in the .env file";
exit 1;
fi;
if [ ! -f certs/ca.zip ]; then
echo "Creating CA";
bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
unzip config/certs/ca.zip -d config/certs;
fi;
if [ ! -f certs/certs.zip ]; then
echo "Creating certs";
echo -ne "instances:\n" " - name: elasticsearch\n" " dns:\n" " - ecp-elasticsearch\n" " - localhost\n" " ip:\n" " - 127.0.0.1\n" " - name: kibana\n" " dns:\n" " - ecp-kibana\n" " - localhost\n" " ip:\n" " - 127.0.0.1\n" " - name: fleet-server\n" " dns:\n" " - ecp-fleet-server\n" " - localhost\n" " ip:\n" " - 127.0.0.1\n" > config/certs/instances.yml;
bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
unzip config/certs/certs.zip -d config/certs;
cat config/certs/elasticsearch/elasticsearch.crt config/certs/ca/ca.crt > config/certs/elasticsearch/elasticsearch.chain.pem
fi;
echo "Setting file permissions"
chown -R root:root config/certs;
find . -type d -exec chmod 750 {} \;;
find . -type f -exec chmod 640 {} \;;
echo "Waiting for Elasticsearch availability";
until curl -s --cacert config/certs/ca/ca.crt https://ecp-elasticsearch:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
echo "Setting kibana_system password";
until curl -s -X POST --cacert config/certs/ca/ca.crt -u elastic:${ELASTIC_PASSWORD} -H "Content-Type: application/json" https://ecp-elasticsearch:9200/_security/user/kibana_system/_password -d "{\"password\":\"\KIBANA_PASSWORD\"}" | grep -q "^{}"; do sleep 10; done;
echo "All done!";
'
healthcheck:
test: ["CMD-SHELL", "[ -f config/certs/elasticsearch/elasticsearch.crt ]"]
interval: 1s
timeout: 5s
retries: 120
elasticsearch:
depends_on:
setup:
condition: service_healthy
image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
container_name: ecp-elasticsearch
volumes:
- certs:/usr/share/elasticsearch/config/certs
- esdata01:/usr/share/elasticsearch/data
ports:
- ${ES_PORT}:9200
restart: always
environment:
- node.name=ecp-elasticsearch
- cluster.name=${CLUSTER_NAME}
- ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
- bootstrap.memory_lock=true
- discovery.type=single-node
- xpack.security.enabled=true
- xpack.security.http.ssl.enabled=true
- xpack.security.http.ssl.key=certs/elasticsearch/elasticsearch.key
- xpack.security.http.ssl.certificate=certs/elasticsearch/elasticsearch.chain.pem
- xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.http.ssl.verification_mode=certificate
- xpack.security.http.ssl.client_authentication=optional
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.key=certs/elasticsearch/elasticsearch.key
- xpack.security.transport.ssl.certificate=certs/elasticsearch/elasticsearch.crt
- xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.security.transport.ssl.client_authentication=required
volumes:
certs:
esdata01:
Step 5: Install Cribl Stream with Docker Compose
Cribl Stream allows for routing, shaping, and reducing log data before forwarding it to systems like Splunk or Elastic. This helps optimize data ingestion and reduces costs by filtering unnecessary data early.
Using Docker Compose for Cribl provides ease of management and ensures a consistent environment across deployments.
Proficiency in log pipeline optimization and data reduction techniques is a strong asset for security monitoring and big data roles.
version: '3'
services:
cribl:
container_name: cribl
image: cribl/cribl:latest
ports:
- "9000:9000"
volumes:
- cribl-data:/opt/cribl/var
- ./cribl.conf:/opt/cribl/conf/cribl.conf
environment:
- CRIBL_LICENSE=your_license_key
volumes:
cribl-data:
