Stalled to UAT-ready in eight weeks.

The challenge

The client had built an internal AI assistant to help sales teams find information faster.

Instead of switching between Salesforce, brochures, transaction data, internal SOPs, project documents and other systems, sales users could ask one assistant questions such as:

• Which units are available within a given budget?
• What is the floor plan for a specific property?
• How does a project compare with nearby developments?
• What recent transactions have taken place in the area?
• Who should I contact internally for a particular customer issue?

The business case was straightforward: reduce the time sales teams spent searching for information and help them respond to customers faster.

The product was already built and had begun reaching users.

The problem was that it could not yet be relied on.

Users would sometimes get errors. Some searches returned incorrect or incomplete results. Certain conversations would stop working altogether. When something failed, the engineering team often had little visibility into why.

The system needed to move from “works in a demo” to “can be trusted during day-to-day sales conversations.”

What we found

The immediate symptoms looked unrelated: errors, failed lookups, broken brochure retrieval, project-name mismatches and incomplete Salesforce filters.

Underneath them were a few broader problems.

Failures were difficult to see

Some components treated technical failures as if no data had been found.

For example, if a search service became unavailable, the assistant could respond with “no data found” instead of reporting that the service had failed.

This made genuine outages indistinguishable from valid empty results and significantly slowed diagnosis.

A failed response could damage the next conversation

Certain interrupted tool calls were being saved into conversation history in an invalid state.

The immediate request might fail, but more importantly, the saved conversation could then fail again on every subsequent message.

Users effectively ended up with permanently broken threads.

The system had accumulated too much complexity

The assistant had been designed around an earlier generation of AI models that required significant engineering support.

Over time, additional parsing rules, conversation-management logic, retrieval workarounds and specialised components had been layered on top.

Those decisions had originally been reasonable.

But model capabilities had improved while the surrounding architecture had remained largely unchanged.

The result was a system carrying substantial complexity that was no longer helping it.

In several places, the workarounds had become a source of errors themselves.

Our approach

Rather than rewriting the product, we focused on stabilising the existing system and simplifying it where possible.

1. Make the system observable

Our first major change was to introduce structured tracing across the assistant.

We added visibility into:

• user requests
• AI responses
• data lookups
• external system calls
• database queries
• errors and retries

This immediately changed how problems were diagnosed.

Issues that had remained unresolved for weeks could now be traced to their actual source instead of being reproduced and guessed at.

Observability was later integrated into the client’s existing Azure monitoring environment, avoiding the need for an additional operational platform.

2. Make failures explicit

We removed patterns that silently converted technical failures into empty results.

When an external service is unavailable, the assistant now knows that a service failed and can communicate that appropriately instead of inventing an answer or claiming that no information exists.

We also fixed the conversation-state issue that could permanently break a thread.

Importantly, existing damaged conversations could repair themselves when users returned to them. Users did not need to start over or have data manually corrected.

3. Simplify the agent architecture

Once the system was observable and stable enough to measure, we began removing complexity.

The model deployment was upgraded and older workarounds were removed gradually, one change at a time.

This included eliminating custom parsing logic and replacing a large bespoke conversation workflow with smaller, purpose-built components responsible for areas such as:

• model fallback
• project context
• user permissions
• error handling
• long-conversation management
• recovery from interrupted requests

The objective was not simply cleaner code.

A simpler system meant fewer hidden interactions, fewer failure modes, lower operating overhead and easier future development.

4. Make structured-data queries safer and more reliable

Several important questions required the assistant to query structured business data.

We rebuilt this flow so that generated queries are validated before execution and can automatically correct themselves when the database returns an error.

Critical rules are enforced by the application rather than left to the AI model.

For example, access restrictions determining which records a user can see are validated programmatically.

This means security does not depend on the model remembering an instruction correctly.

The same query engine was then reused across multiple business data domains, making additional integrations easier to support.

5. Improve project and document retrieval

Real sales conversations rarely use perfect catalogue names.

Users type abbreviations, partial names and spelling mistakes.

Instead of allowing approximate matching to happen invisibly deep inside the retrieval system, we introduced a dedicated project lookup capability.

The assistant can now identify likely matches and, when necessary, ask the user to choose between ambiguous projects.

Document retrieval was also consolidated and search indexes were cleaned to remove duplicate and low-quality content.

This improved both maintainability and the quality of information presented to the assistant.

6. Reduce unnecessary tools and dependencies

The system had accumulated overlapping tools and retrieval components.

We removed unused capabilities, combined duplicate tools and eliminated redundant infrastructure.

The assistant’s toolset became smaller and more clearly defined.

This matters because an AI agent must decide which tool to use for every request. A smaller, better-defined set of tools reduces ambiguity and improves reliability.

Working on a live system

This was a rescue engagement, not a greenfield build.

The application had users, a fixed UAT timeline and existing infrastructure that needed to remain operational.

We therefore avoided a large rewrite.

Significant changes were broken into small, independently reversible releases. Each change documented:

• what was changing
• why it was changing
• which parts of the system were affected
• how success would be evaluated

This allowed the product to continue moving toward UAT while the underlying architecture was improved.

Results

Over eight weeks, the system moved from an unstable implementation toward a substantially more reliable UAT-ready product.

Key outcomes included:

• structured observability across the AI workflow
• resolution of the failure that caused conversations to become permanently unusable
• clearer handling of external-service outages
• safer and more reliable structured-data querying
• deterministic enforcement of user access rules
• improved handling of incomplete and misspelled project names
• integration of additional transaction-data sources
• improved Salesforce filtering
• simplified document retrieval
• fewer overlapping tools and infrastructure dependencies
• deployment to UAT and continued fixes based on live user feedback

The engineering footprint also became smaller.

Across the engagement, more code was removed than added.

87 commits were delivered, with a net reduction of approximately 4,100 lines of code, while additional capabilities were introduced.

That was an important outcome in itself.

The product became more capable by becoming less complicated.

What remained

Not every item was completed during the engagement.

Two areas remained for future work:

• broader improvements to web-search routing and comparison queries
• export of large structured query results into tabular files

Automated evaluation was also not fully implemented during the engagement. Testing and verification relied on traces and manual evaluation against representative business queries.

For that reason, we do not claim numerical accuracy, latency or reliability targets that were not independently measured.

The broader lesson

This engagement reinforced a pattern we increasingly see in enterprise AI systems.

The first challenge is getting an AI assistant to work.

The harder challenge is making it predictable enough for people to rely on.

That requires more than choosing a better model. It requires visibility into failures, clear boundaries around what the model can do, reliable integrations with enterprise systems, and regular removal of workarounds that are no longer necessary.

In this case, some of the biggest improvements came not from adding more AI, but from simplifying the engineering around it.

How Nond helped

Nond helps companies move AI systems from promising prototypes to dependable production applications.