Infrastructure review report

What I reviewed

• The cloud account: servers, database, file storage, firewalls and bills — with access to view the configuration, not the data.
• The GitHub organisation: repositories, members and how code reaches production.
• Server configuration, and where logs are kept and for how long.
• A conversation with the team lead and the developer who deploys.

Summary for management

The three biggest risks

1. The database backup is kept on the same server as the database itself. If that server fails, both are gone.
2. Two former contractors still have access to the cloud and the code — they could delete the servers, even though they no longer work with you.
3. One person deploys to production from their own laptop, with a key that opens every server.

Has restoring from backups ever been tested?

No evidence found: there is no written procedure, no trace of test restores, and the team does not remember ever trying one.

Where are you overpaying for cloud?

About £70 a month: the staging server runs around the clock, two volumes are not attached to any server, and old snapshots are never deleted.

Risks by severity

For each one: what I found, what could happen, and which of the six jobs fixes it.

Critical

data or money can be lost through a single failure, or by someone who already has access.

High

harm is possible if something else also happens: the site goes down, someone leaves, a password leaks.

Medium

no direct harm, but recovering from a failure would take longer, and the gaps pile up.

Low

money is being overspent, no risk.

1. Critical

   Database backup on the same server

   What I found

   PostgreSQL runs on its own server, and the nightly backup (pg_dump) is written to that same server’s disk. There is no copy anywhere else.

   What could happen

   If the disk fails or the server is deleted by mistake, both the database and the backup are lost. Even when the backup survives, up to a day of orders is gone.

   Which job

   Infrastructure as code

2. Critical

   Former contractors still have access

   What I found

   Two former contractors are still in the cloud team and the GitHub organisation. Two-factor authentication on GitHub is not enforced.

   What could happen

   They, or anyone who steals their password, could delete the servers or copy the code — and nobody would notice.

   Which job

   No separate job needed: your administrator can remove them and make two-factor authentication mandatory.

3. High

   Deploys run from one laptop

   What I found

   The deploy script and the SSH key exist only on one developer’s laptop. The same key logs in to every server as root.

   What could happen

   While that person is on holiday or after they leave, nobody ships fixes. If their laptop goes missing, the key opens every server.

   Which job

   CI/CD pipeline

4. High

   Passwords in the repository

   What I found

   A .env file with the database password and the payment provider’s API key is in the git history. All 8 developers can see the repository.

   What could happen

   If any developer’s account or laptop is compromised, the database and payments are exposed. Deleting the file does not help — the history keeps the keys, so they have to be changed.

   Which job

   Secrets management

5. High

   Database reachable from the internet

   What I found

   The firewall allows connections to the PostgreSQL port from any IP address. The only protection is a password — the same one that is in the repository.

   What could happen

   Anyone with the password can connect to the database from anywhere, not just from your servers.

   Which job

   Infrastructure as code

6. High

   Outages are reported by customers

   What I found

   Nobody checks the site from the outside, and there are no alerts. Logs sit on each server’s disk and are deleted after 7 days.

   What could happen

   The shop can be down for hours before anyone notices. After a week it is no longer possible to find out what happened.

   Which job

   Logging and observability

7. Medium

   Servers configured by hand

   What I found

   Nginx and system settings were changed directly on the servers and are not written down anywhere. Staging runs a different PostgreSQL version from production.

   What could happen

   A failed server cannot be rebuilt quickly — nobody knows what was changed on it. Some bugs only show up in production.

   Which job

   Infrastructure as code

8. Medium

   Unsupported operating system

   What I found

   Every server runs Ubuntu 20.04 — its standard support has already ended.

   What could happen

   Security patches no longer arrive, unless extended support is switched on.

   Which job

   Infrastructure as code

9. Low

   Overpaying for cloud

   What I found

   The staging server runs around the clock, two volumes are not attached to any server, and 14 old snapshots are never deleted.

   What could happen

   About £70 a month goes on things nobody uses.

   Which job

   No separate job needed — it is sorted out along with the first job.

What I do not recommend

You do not need Kubernetes right now. For three servers and eight developers, the current Docker Compose setup is enough — deployed through CI/CD rather than from a laptop. It is worth coming back to once starting and updating services becomes manual work again.

What I did not check

• The payment and email-sending accounts — I had no access to them.
• The DNS provider account — a former contractor controls it, and access needs to be taken back.
• The security of the application code itself — this review does not assess it.

Where to start

1. Straight away, without me: your administrator removes the former contractors, makes two-factor authentication mandatory and takes back the DNS account.
2. First job — database backups in a second location and a tested restore (“Infrastructure as code”).
3. Next — passwords out of the repository (“Secrets management”) and deploys through CI/CD rather than from one laptop.