Observability Project: Difference between revisions
Jump to navigation
Jump to search
m
→Technologies
m (→Technologies) |
|||
| (5 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
==Latest Progress== | ==Latest Progress== | ||
* 30-Apr-2024: Configuring cloudlet to use hidden password, start automatically from service daemon, automatically refresh my [https://github.com/VincentDirks/SplunkDockerCompose.git SplunkDockerCompose] repo, and keep service file during redeploys. | |||
* 29-Apr-2024: Setting up Splunk to run in cloudlets. Tried running Splunk image directly, but found that a docker-compose.yml file was a bit easier. At this stage I am able to run the Splunk image. I found that the cloudlets were not executing the docker compose up command, and I created a systemd daemon service for docker compose. | |||
* 24-Apr-2024: Learn about and setting up DNS etc. to link my new domain dirks.nz | * 24-Apr-2024: Learn about and setting up DNS etc. to link my new domain dirks.nz | ||
* 23-Apr-2024: Setup Node.js container in Cloudlets to fetch and run head from main branch of repo below. If it's running you can access it [https://vid-nodejs-in-the-cloud.mel.cloudlets.com.au/ here] | * 23-Apr-2024: Setup Node.js container in Cloudlets to fetch and run head from main branch of repo below. If it's running you can access it [https://vid-nodejs-in-the-cloud.mel.cloudlets.com.au/ here] | ||
| Line 22: | Line 24: | ||
* Reverse proxy in front of my current CV Wiki and to capture the traffic. | * Reverse proxy in front of my current CV Wiki and to capture the traffic. | ||
* Log aggregator & event analyser to hold the data and start analysing it | * Log aggregator & event analyser to hold the data and start analysing it | ||
==Plan/Backlog== | |||
# <s>Run a basic Express.js demo in a cloudlet node with a recent Node.js image</s> [DONE] | |||
# <s>Implement access to it from my new `dirks.nz` domain, using encryption (https)</s> [DONE] | |||
# Run Splunk in cloudlets '''[In progress]''' | |||
# Implement a reverse proxy using library from npm (for traffic to/from above Express.js demo) | |||
# Create Splunk HEC (Http Event Collector) (check if config persists through redeploys) | |||
# If needed, investigate data persistence through use of volumes from Docker <=> Cloudlet <=> Backup/Archive | |||
# Send traffic events to Splunk HEC from Express demo (create indexes, set data retention policies, check data persistence through redeploys) | |||
# Investigate if reverse proxy can be used for accessing Splunk instance UI, and consider recording splunk UI access logs | |||
# Try implement auth in reverse proxy to require login before being able to access some URL paths (eg. Splunk UI) | |||
# Implement MediaWiki in a cloudlet, and copy content from my existing cvwiki | |||
# Route traffic to/from new cvwiki via reverse proxy, & log the traffic | |||
# Create an UI Automation suite to test cvwiki using Playwright | |||
# Run the above suite in cloudlet and record results to splunk | |||
# Schedule the above cloudlet to run daily | |||
=Current Implementation= | |||
=Technologies= | =Technologies= | ||
| Line 46: | Line 66: | ||
The free license removes the auth parts and gives everyone admin access, I will investigate restricting access to it later. | The free license removes the auth parts and gives everyone admin access, I will investigate restricting access to it later. | ||
==Language (javascript) | ==Language (javascript), Runtime (Node.js), Framework (Express)== | ||
For now, I have better knowledge, skills, experience using these than others. I recognise that there may well be better solutions out there, but for now I will favour these because it feels more comfortable. | |||
For | |||
That said though, and given that I'll be using containerized solutions, it should be easy to try other options later. | |||
===Code Quality & Inspection Tools=== | ===Code Quality & Inspection Tools=== | ||