AI application monitoring.

First monitored application

The first target is an industrial AI chat application. Monitoring verifies that the application opens, the conversation works and answers remain grounded in the agreed material. A simple server availability check is not enough: the service can be online while the chat itself is broken.

Solution

I developed automated testing that uses the application from the user’s perspective. It can run daily or weekly and checks that the application responds as expected. If there is an interruption or an answer fails the agreed criteria, the system sends an email alert. This enables a rapid response before the customer has to report the problem.

Proof of value

The first scheduled run in August 2026 failed for a real reason: an API key had been removed while another project was being cleaned up, and customers received an authentication error. Monitoring found the fault before a customer complaint. It was located and fixed the same day, then verified both in a browser and in the next automated run.

My role

I built the system from design to deployment: architecture, browser-based application mapping, automated test scripts, configuration for multiple monitored applications, scheduling and alerts. The work is a practical example of using AI agents to build production-grade monitoring, not merely demos.

Benefits and limits

Automated monitoring helps ensure that published AI applications remain reliable through use and updates. Regular testing reduces outages and incorrect answers while enabling maintenance to react quickly. The method can be introduced gradually and adapted to each application.

Tools

• Playwright
• Node.js
• GitHub Actions