Observability Project
Contents
Latest Progress
- 24-Apr-2024: Learn about and setting up DNS etc. to link my new domain dirks.nz
- 23-Apr-2024: Setup Node.js container in Cloudlets to fetch and run head from main branch of repo below. If it's running you can access it here
- 22-Apr-2024: Setup basic Express.js project in new github repo
Old items are moved to Progress
Introduction
This is a personal development project to observe this web site (https://dirksonline.net/cvwiki). I will be striving to monitor reliability, functionality, and importantly who and how it's being used.
Background
Testing strives to find things that might surprise someone who matters. Hence, we need to know the people that matter, and what matters to them. Identifying the individuals (& types of people) that matter to the project and observe what they do for the project. To do our jobs well it is imperative we create good mental models of the people, and it is essential to continue to observe, and update, our models because they form the foundations from which we derive our testing and quality assessments.
Users are almost always at the pinnacle of people that matter, having good models of the types of users, and what they choose to do, is critical to the success of most projects and products.
I want to learn about the people who use my CVWiki web site, so I'm going to develop some observability solutions, and in the process deepen my knowledge and experience in the technologies used in observing web sites.
I am a keen agilist, and will aim to maximize the work not done. Implementing minimalist solutions, that require the least amount of work from me, for my current context. I will favour resources I already have at hand, and the technologies I already know about. But that said, there are some large gaps I need to cover and there will be plenty new to discover and learn, I expect that some decisions may need review down the track as I discover better technologies for my solutions.
Architecture
- Modular microservices style architecture, with the parts of the system running in containers
- Reverse proxy in front of my current CV Wiki and to capture the traffic.
- Log aggregator & event analyser to hold the data and start analysing it
Technologies
Hosting Platform
and the associated PaaS
A while back a friend of mine said he was using these folks in Australia and was very happy with them, hence I moved my cpanel based stuff to them quite a while ago, and I've been happy with the service myself too.
For this project I am learning about the cloudlets containerisation service. It would seem that I can create nodes using
- a Cloudlets certified Node.js container
- a Docker Image
- Kubernetes
Data Collection & Analysis Platform
I have used this platform for a number of my past employers and projects and really like it. I recognise it isn't a free, nor an open source solution such as Grafana, Prometheus, ELK, Graylog etc. but it's what I know currently and will favour it until I have time and need for a better fitting solution.
I will be using the docker-splunk single containerized instance of Splunk Enterprise, with the free license.
The free license removes the auth parts and gives everyone admin access, I will investigate restricting access to it later.
Programming Language
- Javascript
- Node.js
- npm
- Express.js
I have better knowledge, skills, experience using these than other languages and libraries. I recognise that there may be better solutions but I will favour these for now because it feels more comfortable. Given that I'll be using containerized solutions, it should be easy to try other languages later.
Code Quality & Inspection Tools
I would like to set these up, but haven't as yet.
Development Environment
- VS Code
- VS Code Extensions: TBD
Progress
Old items from Latest Progress will be moved to here
