Observability Project: Difference between revisions
Jump to navigation
Jump to search
m
→Current Implementation
m (→Technologies) |
|||
| (11 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
==Latest Progress== | ==Latest Progress== | ||
* 30-Apr-2024: Configuring cloudlet to use hidden password, start automatically from service daemon, automatically refresh my [https://github.com/VincentDirks/SplunkDockerCompose.git SplunkDockerCompose] repo, and keep service file during redeploys. | |||
* 29-Apr-2024: Setting up Splunk to run in cloudlets. Tried running Splunk image directly, but found that a docker-compose.yml file was a bit easier. At this stage I am able to run the Splunk image. I found that the cloudlets were not executing the docker compose up command, and I created a systemd daemon service for docker compose. | |||
* 24-Apr-2024: Learn about and setting up DNS etc. to link my new domain dirks.nz | * 24-Apr-2024: Learn about and setting up DNS etc. to link my new domain dirks.nz | ||
* 23-Apr-2024: Setup Node.js container in Cloudlets to fetch and run head from main branch of repo below. If it's running you can access it [https://vid-nodejs-in-the-cloud.mel.cloudlets.com.au/ here] | * 23-Apr-2024: Setup Node.js container in Cloudlets to fetch and run head from main branch of repo below. If it's running you can access it [https://vid-nodejs-in-the-cloud.mel.cloudlets.com.au/ here] | ||
| Line 22: | Line 24: | ||
* Reverse proxy in front of my current CV Wiki and to capture the traffic. | * Reverse proxy in front of my current CV Wiki and to capture the traffic. | ||
* Log aggregator & event analyser to hold the data and start analysing it | * Log aggregator & event analyser to hold the data and start analysing it | ||
==Plan/Backlog== | |||
# <s>Run a basic Express.js demo in a cloudlet node with a recent Node.js image</s> [DONE] | |||
# <s>Implement access to it from my new `dirks.nz` domain, using encryption (https)</s> [DONE] | |||
# Run Splunk in cloudlets '''[In progress]''' | |||
# Implement a reverse proxy using library from npm (for traffic to/from above Express.js demo) | |||
# Create Splunk HEC (Http Event Collector) (check if config persists through redeploys) | |||
# If needed, investigate data persistence through use of volumes from Docker <=> Cloudlet <=> Backup/Archive | |||
# Send traffic events to Splunk HEC from Express demo (create indexes, set data retention policies, check data persistence through redeploys) | |||
# Investigate if reverse proxy can be used for accessing Splunk instance UI, and consider recording splunk UI access logs | |||
# Try implement auth in reverse proxy to require login before being able to access some URL paths (eg. Splunk UI) | |||
# Implement MediaWiki in a cloudlet, and copy content from my existing cvwiki | |||
# Route traffic to/from new cvwiki via reverse proxy, & log the traffic | |||
# Create an UI Automation suite to test cvwiki using Playwright | |||
# Run the above suite in cloudlet and record results to splunk | |||
# Schedule the above cloudlet to run daily | |||
=Current Implementation= | |||
==Express Demo Running in Cloudlets== | |||
This step's objectives were to | |||
# Standup a cloudlet node using an inbuilt Node.JS image | |||
# Link it to my code repo [https://github.com/VincentDirks/node.js-in-the-cloud.git node.js-in-the-cloud] | |||
# Access it from a subdomain of dirks.nz | |||
# Implement encryption (https) | |||
# Redirect calls to http to https, as well as any calls direct to the environment | |||
The solution is accessible at https://nodejs-in-the-cloud.dirks.nz/ | |||
My code repo is [https://github.com/VincentDirks/node.js-in-the-cloud.git node.js-in-the-cloud] | |||
A Git repo is added to the cloudlets Deployment Manager, deployed it to the Node.js container, with settings to monitor the repo for changes, so that the updates are fetched and container restarted automatically. | |||
Created an nginx node to configure encryption and redirects | |||
Using the cloudlets Let's Encrypt Free SSL addon to generate certs | |||
fetch the changed nginx conf files and add to repo and update readme there to reflect | |||
TBD | |||
=Technologies= | =Technologies= | ||
| Line 46: | Line 87: | ||
The free license removes the auth parts and gives everyone admin access, I will investigate restricting access to it later. | The free license removes the auth parts and gives everyone admin access, I will investigate restricting access to it later. | ||
== | ==Language (javascript), Runtime (Node.js), Framework (Express)== | ||
For now, I have better knowledge, skills, experience using these than others. I recognise that there may well be better solutions out there, but for now I will favour these because it feels more comfortable. | |||
That said though, and given that I'll be using containerized solutions, it should be easy to try other options later. | |||
I have better knowledge, skills, experience using these than | |||
===Code Quality & Inspection Tools=== | ===Code Quality & Inspection Tools=== | ||
I would like to set these up, but haven't as yet. | I would like to set these up, but haven't as yet. | ||
==Development Environment== | ==Development Environment (VS Code)== | ||
* VS Code | * VS Code | ||
* VS Code Extensions: TBD | * VS Code Extensions: TBD | ||