↓Skip to main content

Devops

From Monoliths to AI Proxies: Real-World Strategy for Testing and Evolving LLM Integrations

Ai Software Architecture Application Development Devops Software Architecture LLM Proxy Pattern AI Architecture Best-Practices Large Language Models LLM Integration Model Context Protocol (MCP) Testing Testing AI Systems
Integrating Large Language Models (LLMs) into production systems presents unique architectural, testing, and operational challenges. This article shares practical insights and solutions from real-world experience integrating LLMs into a customer interaction platform. It covers the evolution from a monolithic to a more modular AI Proxy architecture pattern, strategies for testing, deploying and monitoring LLMs, and the emerging Model Context Protocol (MCP) standard. Application developers and software architects will learn proven practices to build robust, reliable and responsible LLM-powered systems.

Monitoring your application locally with NewReclic

Programming Devops Java Performance Newrelic

The New Relic Digital Intelligence Platform provides actionable insights to drive digital business results. You can monitor your application and infrastructure performance so you can quickly resolve issues and improve digital customer experiences.

Following instruction should help you to connect your application to NewRelic platform and customize application events sent to the platform.

External Site Monitoring

Devops Web

How did you know that your site is down? From your customers? Then it’s time to start using some external monitoring tool.

Logging Policy

Programming Devops Logging

There are different points of view on how logging levels should be used in code. I will share mine.

My assumption is: “There should be no errors in logs when everything is fine.”

Developing in "Dirty Trunk"

Devops
The “dirty trunk” branching strategy involves committing directly to the main branch, with CI builds triggered on each commit. While simple, it requires discipline and quick fixes to maintain build stability. The article discusses pros, cons, and best practices for this approach.

Jolokia: HTTP/JSON bridge for JMX

Devops Jmx Tomcat Java

Very often there is a need to monitor the Java application server. For example, external monitoring tool, like Nagious/Zenoss/Zabbix needs to get some metrics, like heap memory usage or thread count.

Usual way to get that metrics is to setup access to application server via JMX.

But, sometimes, it is not possible to leave some other port opened for JMX and the only port available is HTTP(80 or 8080) or HTTPS(443 or 8443).

Here the Jolokia comes to rescue!

Links: Continuous Deployment, Versioning and Git

Devops Git Maven Continuous Deployment
The post shares resources on using Git for versioning and branching, including an article on versioning Maven projects with Git, an overview of the popular GitFlow branching model, and a free Git client called SourceTree that supports GitFlow. Additional links cover Git workflows and the GitFlow project on GitHub.

Establishing Customizable Tomcat Configuration

Devops Tomcat Spring

Deploying to Apache Tomcat often requires making changes to default configuration. These changes are often environment specific. Also, when upgrading a Tomcat to new version you need to be sure that all your custom changes have not been lost and were applied to new configuration. To deal with all that stuff Tomcat via separation of the configuration. This post contains step-by-step instruction will help you to establish custom tomcat configuration.