Bolting an LLM Onto a Legacy .NET App Without Breaking Production

The brief was familiar. "Add AI" to an ASP.NET Web Forms invoicing app that has been quietly running for half a dozen enterprise customers since 2013. The app cannot go down. The codebase has no tests. The director wants a "draft my email" feature on the customer detail page. You are alone on this. Good luck.

I've now done this dance enough times to have a pattern that doesn't end in tears.

The pattern: sidecar, not surgery

Don't refactor the monolith. Build a tiny .NET 8 minimal API service that talks to your LLM provider, deploy it next to the existing app, and call it from the legacy code over HTTP. The legacy app stays a 2013 codebase. The sidecar is modern, isolated, deployable on its own, and rollback-able with a feature flag.

The mental model is the strangler fig. You're not replacing the old tree, you're growing a new one beside it. The first branch happens to talk to OpenAI.

The sidecar in 25 lines

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddHttpClient("llm", c =>
{
    c.BaseAddress = new Uri(builder.Configuration["LLM:Endpoint"]!);
    c.DefaultRequestHeaders.Add("api-key", builder.Configuration["LLM:Key"]);
    c.Timeout = TimeSpan.FromSeconds(20);
});

var app = builder.Build();

app.MapPost("/draft-email", async (DraftRequest req, IHttpClientFactory f) =>
{
    var http = f.CreateClient("llm");
    var payload = new
    {
        model = "gpt-4o-mini",
        messages = new[]
        {
            new { role = "system", content = "Write a polite customer email." },
            new { role = "user",   content = req.Context }
        },
        max_tokens = 400
    };
    var res = await http.PostAsJsonAsync("/chat/completions", payload);
    return Results.Ok(await res.Content.ReadFromJsonAsync<LlmResponse>());
});

app.Run();

record DraftRequest(string Context);
record LlmResponse(string Text);

Why it works

The legacy app changes by exactly two lines. An HTTP call from the code-behind. No NuGet upgrades, no Framework-to-Core migration, no risk to the invoicing path that pays for your team's coffee. The sidecar owns the messy parts: retries, timeouts, prompt versions, switching providers when one of them has a bad afternoon. None of that ever touches the monolith.

When to use it

Anytime "legacy app + AI feature" is the brief. Especially when the legacy app has no test coverage, the team is small, and the AI bit is genuinely additive (not replacing core logic). The sidecar keeps your blast radius the size of one feature. That is what lets you sleep.

When not to

If the AI feature needs deep access to legacy state, long EF transactions, in-memory caches, weird DbContext lifetimes, the HTTP boundary will fight you. Inline it instead, but only if you've got at least some test coverage. And don't use this pattern on a greenfield .NET app. Just build it integrated from day one. Sidecars are a coping mechanism, not a goal.

Deployment topology

flowchart LR
    User[User browser] --> LB[Load balancer]
    LB --> Legacy["Legacy ASP.NET Web Forms<br/>(unchanged)"]
    Legacy -- HTTP --> Sidecar[".NET 8 LLM Sidecar<br/>(new, isolated)"]
    Sidecar --> OpenAI[("OpenAI / Azure OpenAI")]
    Sidecar -. on failure .- FeatureFlag{Feature flag<br/>off?}
    FeatureFlag -- yes --> Legacy

Conclusion

Put the sidecar call behind a feature flag from day one. Even if you only ever flip it once. Future you, debugging at 3am while the LLM provider has its weekly incident, will thank present you for the ten lines that let ops turn the AI feature off without a deploy.