Adventures On The Edge

DCI Abstract * DCI Introduction

Author’s Note: In my previous post, I worked through the abstract of the Direct Corpus Interaction (DCI) paper and discovered that much of my initial confusion stemmed from assumptions about how modern retrieval systems actually work. Before moving into the introduction, I realized I needed a clearer understanding of the retrieval approaches the paper repeatedly references. The blockquote below contains a statement from the introduction that led me to investigate the difference between sparse and dense retrieval systems. The explanatory notes that follow are part of my learning journey and are intended to help other developers who may be encountering these concepts for the first time.

dci.pdf (2.38 mb) — Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction

This interface underpins a wide range of applications, including retrieval-augmented generation (Lewis et al., 2020; Gao et al., 2023; Singh et al., 2025), open-domain question answering (Trivedi et al., 2022; Press et al., 2023), and deep research (Wei et al., 2025; Chen et al., 2025b).

My Initial Misunderstanding

When I first read this sentence, I mentally grouped Retrieval-Augmented Generation (RAG), Open-Domain Question Answering (ODQA), and Deep Research together as different retrieval approaches. Since I was already familiar with RAG and was reading a paper proposing Direct Corpus Interaction (DCI), my brain immediately started categorizing everything as competing retrieval techniques.

That turned out to be an inaccurate mental model.

The Distinction That Helped

The most useful clarification for me was realizing that the paper is discussing application categories, not necessarily retrieval methods.

A retrieval method is concerned with how information is found.

• BM25
• Dense Retrieval
• Hybrid Retrieval
• Top-k Retrieval
• Direct Corpus Interaction (DCI)

An application category is concerned with what the system is trying to accomplish.

• RAG Assistant
• Open-Domain Question Answering System
• Deep Research Agent

Once I separated those two ideas, the sentence became much easier to understand. The authors are not listing competing retrieval techniques. They are listing examples of systems and applications that depend on retrieval.

Visualizing The Difference

One reason this distinction initially escaped me is that all three application categories involve finding information. From a distance they can look very similar.

At a simplified level, a RAG system often follows a pattern like:

Question
    ↓
Retrieve Documents
    ↓
Generate Answer

Open-Domain Question Answering systems are also attempting to answer questions using information that may exist anywhere within a large corpus:

Question
    ↓
Retrieve Documents
    ↓
Read Documents
    ↓
Generate Answer

Deep Research systems typically extend this idea into a longer investigation:

Question
    ↓
Search
    ↓
Read
    ↓
Search Again
    ↓
Compare Evidence
    ↓
Investigate Gaps
    ↓
Repeat
    ↓
Produce Report

The exact implementations differ, but these simplified workflows helped me understand why the paper groups them together. They are all systems that rely on retrieving and examining information from a corpus.

A Common Misunderstanding

An easy mistake is to assume that DCI belongs in the same category as RAG, Open-Domain QA, and Deep Research.

That is not how I currently understand the paper.

RAG, Open-Domain QA, and Deep Research are applications. DCI is being proposed as a different way for those applications to interact with a corpus.

In other words, DCI is closer to a retrieval or investigation strategy than an end-user application category.

Why This Matters For Understanding DCI

The most important takeaway for me was realizing that the paper is not simply introducing another retrieval technique alongside existing retrieval techniques. It is questioning a deeper assumption that many of these systems share.

Traditional retrieval pipelines often begin with a top-k retrieval step:

Question
    ↓
Retrieve Top-k Documents
    ↓
Work From Retrieved Results

DCI appears to challenge the assumption that an investigation must begin that way.

Instead of retrieving a small set of candidate documents and working only from those results, an agent may directly interact with the corpus through actions such as searching, opening files, inspecting context, following references, and gathering evidence incrementally.

At this point in the paper, the authors have not yet proven that this approach is superior. They are establishing why retrieval matters and why rethinking retrieval could affect a broad range of systems, including RAG applications, Open-Domain QA systems, and Deep Research agents.

What Changed In My Understanding

Before reading this section carefully, I was viewing the discussion primarily as:

RAG
versus
DCI

After working through the terminology, I now see the comparison as something closer to:

Top-k Retrieval
versus
Direct Investigation

That shift in perspective made the rest of the introduction significantly easier to follow and helped me understand why the authors repeatedly reference multiple application categories throughout the paper.

Understanding Sparse and Dense Retrieval

"In standard retrieval-augmented pipelines, documents are chunked, indexed, and filtered into a top-k candidate set using well-established sparse (Robertson et al., 1994) or dense (Karpukhin et al., 2020) techniques before downstream reasoning begins."

DCI, Agent Loops, and System Architecture

"This becomes particularly beneficial once the agent is strong enough to search strategically (as recent systems suggest; e.g., Anthropic, 2026; OpenAI, 2026)."

The following is a Responsibility-Driven Design (RDD) XMind map that I created to help me understand the boundaries, responsibilities, and collaborators that helped me in the writing of this blog.

The title of this blog stems from the phrase "Lions and tigers and bears, oh my!"; a phrase that originates from the 1939 movie The Wizard of Oz.  The underlying meaning is an expression of anxiety for escalating, unknown, or overwhelming fears; or in a lesser sense, having to face daunting tasks and challenges.  In the movie, Dorthy, the Scarecrow, and the Tin Man chant it as a nervous mantra as they walk through a dark forest potentially holding dangerous creatures. 

DCI Abstract * Dci Introduction

Author’s Note: I am actively learning about Direct Corpus Interaction (DCI) and documenting my understanding as I go. The blockquotes in this post contain excerpts from the DCI research paper that exposed gaps in my own understanding. The explanatory sections that follow are learning notes generated with ChatGPT to help me clarify the concepts. They are not presented as original research, but as study notes for developers following the same path.

dci.pdf (2.38 mb) a paper on "Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction"; the article abstract follows:

Understanding the Retrieval Bottleneck

"Modern retrieval systems, whether lexical or semantic, expose a corpus through a fixed similarity interface that compresses access into a single top-k retrieval step before reasoning."

Understanding the Bottleneck in Traditional Retrieval

"This abstraction is efficient, but for agentic search, it becomes a bottleneck: exact lexical constraints, sparse clue conjunctions, local context checks, and multi-step hypothesis refinement are difficult to implement by calling a conventional off-the-shelf retriever, and evidence filtered out early cannot be recovered by stronger downstream reasoning."

Ai Requires More Discipline From Us

AI is no longer just correcting our spelling, finishing our sentences, or helping us phrase an idea. Increasingly, AI systems are becoming operational actors. That changes everything.

The Shift I Am Beginning to Notice

For years, many of us have been trained by software to be imprecise.

Autocorrect fixes our spelling. Search engines guess what we meant. Recommendation engines infer our preferences. IDEs complete our code. Navigation systems route us without requiring us to understand the roads.

That convenience has benefits, but it also has a cost: it conditions us to become comfortable with vague intent.

With traditional software, vague intent was often tolerable. If autocorrect picked the wrong word, we could fix it. If search returned the wrong page, we could search again. If autocomplete made a bad suggestion, we could delete it.

But AI is moving beyond suggestion.

Modern AI systems can now invoke tools, read documents, edit repositories, call APIs, operate through connectors, send messages, schedule events, generate code, and interact with systems through protocols such as MCP-style tool interfaces.

That means the relationship has changed.

We are no longer merely asking software to help us express intent. We are increasingly asking software to act on intent.

Once AI can act, loose prompting becomes more than a communication issue. It becomes an operational risk.

Why This Feels Different

Older AI prompting often felt like trying to get better prose from a clever assistant. The goal was usually to get a better answer, a better summary, a better email, or a better explanation.

That is still useful. But it is no longer the whole picture.

As AI systems become connected to tools and workflows, prompting starts to carry more weight. A prompt is no longer just a request. In many cases, it becomes a temporary policy boundary.

It may define:

• what the AI is allowed to touch,
• what it should avoid,
• what source of truth it should trust,
• whether it may act or only advise,
• how much autonomy it has,
• what should be logged,
• what requires confirmation,
• and what outcome counts as complete.

That is a very different world from “write me a paragraph about this topic.”

The MCP Security Lesson

The recent attention around MCP security did not create this problem by itself. It exposed a problem that was already forming.

MCP-style systems make tool use visible and standardized. That is valuable. But once a model can interact with tools, files, services, and credentials, the question is no longer simply, “Can the model answer correctly?”

The question becomes:

Can the system act safely when exposed to ambiguous instructions, hostile context, excessive permissions, or hidden prompt manipulation?

This is why the security conversation has expanded beyond ordinary bugs. Prompt injection, excessive agency, insecure tool use, sensitive information disclosure, and confused authorization boundaries are now architectural concerns, not just prompting annoyances.

In plain English: if an AI can use tools, then someone must define what those tools are allowed to do, under what authority, with what evidence, and with what audit trail.

The Real Issue: Assistant Versus Actor

A helpful way to think about this is the difference between an assistant and an actor.

An assistant helps you think, write, review, explain, summarize, or plan.

An actor changes things.

It edits files. It opens tickets. It runs commands. It sends emails. It schedules meetings. It queries private systems. It modifies infrastructure. It may even chain multiple actions together.

When AI behaves as an assistant, vague prompting is often merely inefficient.

When AI behaves as an actor, vague prompting can become dangerous.

The more authority we give an AI system, the more disciplined our instructions must become.

Why Non-Developers Need to Understand This

This is not only a developer problem.

Developers may see it first because they work close to tools, repositories, terminals, APIs, permissions, and logs. But the same shift is coming to everyone.

AI systems are being connected to email, calendars, documents, customer records, spreadsheets, business processes, personal assistants, financial systems, learning tools, research workflows, and office automation.

That means ordinary users will increasingly face systems that do not merely suggest what to do. They may do it.

If users remain conditioned by “loosey-goosey autocorrect” habits, frustration is inevitable. People may say something vague, the AI may interpret it differently, and the result may not match what the person intended.

Worse, the user may accept the result because they have been trained by years of convenience software to trust the machine’s correction over their own unfinished thought.

The Human Risk: Convenience Can Weaken Judgment

This is the part that concerns me most.

Human beings adapt to convenience. That is not an insult; it is a reality of human behavior.

When software repeatedly fills in gaps for us, we may stop noticing the gaps. We become less intentional. We accept “close enough.” We allow systems to complete our thoughts before we have fully formed them.

That can be harmless when the output is a misspelled word.

It is not harmless when the output is a business decision, a legal statement, a code change, a customer response, a financial action, or a security-sensitive operation.

AI does not merely risk making humans lazy. It risks making humans comfortable with unexamined delegation.

That is a much deeper issue than prompt engineering.

Prompting Is Becoming an Operational Skill

Disciplined prompting is not about using magic phrases.

It is not about tricking the model.

It is not about sounding technical.

Disciplined prompting is about expressing intent clearly enough that an AI system can operate within safe and useful boundaries.

That includes being clear about:

• the goal,
• the scope,
• the source of truth,
• the allowed actions,
• the disallowed actions,
• the expected output,
• the level of autonomy,
• and the point where human review is required.

In other words, good prompting is becoming less like casual conversation and more like operational instruction.

A Simple Example

A loose prompt might say:

Clean this up and make it better.

That may be fine for a casual paragraph. But if the AI is working inside a repository, a business document, or a production workflow, that prompt is too vague.

A more disciplined prompt might say:

Review this document for technical accuracy and clarity. Do not rewrite it in your voice. Identify places where my wording is misleading, ambiguous, or technically incorrect. Suggest corrections, but preserve my intent and style. Do not expand the scope beyond this document.

The difference is not verbosity for its own sake. The difference is control.

The New Mental Model

The old mental model was:

I ask AI a question, and it gives me an answer.

The emerging mental model is:

I define a bounded task, provide trusted context, constrain the action space, and review the result.

That may feel less magical, but it is more mature.

It also reflects where AI systems are going. As models become more capable, the limiting factor will often not be whether the AI can do something. The limiting factor will be whether we can define what it should do safely, precisely, and responsibly.

Why This Matters for AI Systems Authors

An AI Systems Author is not merely someone who writes prompts. It is someone who understands that AI behavior emerges from the interaction between models, tools, instructions, context, permissions, memory, retrieval, and human review.

That role requires a different discipline.

It requires asking questions such as:

• What is the source of truth?
• What authority does the AI have?
• What should the AI never do without approval?
• What context is trusted?
• What context may be hostile or misleading?
• What evidence should be preserved?
• How will the human know what happened?
• How can the system fail safely?

These questions are not academic. They are practical.

They are the difference between using AI as a helpful assistant and accidentally creating an ungoverned operational actor.

The Frustration That Is Coming

Many users are accustomed to software silently correcting them. They may expect AI to do the same thing, only better.

But as AI systems become more safety-conscious, users may begin to feel friction.

The AI may ask for clearer instructions. It may refuse to infer too much. It may avoid taking action without confirmation. It may distinguish between reviewing, drafting, editing, executing, and publishing. It may resist vague requests that would have been accepted casually before.

Some users may experience that as the AI becoming less helpful.

But in many cases, the opposite is true.

The system is not becoming less helpful. It is becoming more aware that helpfulness without boundaries can be harmful.

Patience Is Part of the Skill

Learning to work well with AI will require patience.

That patience is not just waiting for better answers. It is the patience to clarify our own intent before delegating work. It is the patience to review what was done. It is the patience to correct the instruction, not merely complain about the output.

This is where I expect my own prompting habits to be sharpened.

If I ask for something vague, I should expect the AI to help expose that vagueness. If I give it too much authority, I should expect it to slow down. If I fail to define the source of truth, I should expect the result to be less reliable. If I ask it to “make it better,” I should be prepared to explain what “better” means.

That is not a weakness in the process. That is the process teaching me to be more intentional.

Conclusion: The End of Casual Delegation

AI is becoming more powerful, but that does not remove responsibility from the human. It increases it.

The future will not belong only to people who know how to ask clever questions. It will belong to people who know how to define bounded work, preserve human judgment, and use AI without surrendering discernment.

Loose prompting may still work for casual tasks.

But for serious work, the era of casual delegation is ending.

As AI becomes more capable of acting, humans must become more capable of instructing.

References and Further Reading

• OWASP Top 10 for Large Language Model Applications
• OWASP GenAI Security Project: LLM Top 10
• NIST AI Risk Management Framework
• NIST AI RMF Playbook
• Automation Bias: A Systematic Review

"The companies achieving sustainable AI success aren’t necessarily the ones with the most sophisticated models or the largest budgets. They’re the ones that treat AI deployment as a comprehensive engineering discipline, with rigorous processes, proper architecture, and deep integration with business workflows."
- The Production AI Reality Check: Why 80% of AI Projects Fail to Reach Production

I was unknowingly sabotaging my open source project due to poor assumptions working with AI. While I was completing months of work within days, I was introducing shiny objects that would change the session context and direction; leading us away from successfully getting to beta in a timely manner (if at all).

The context switches were sometimes not so subtle (and hard to recover from); it was these that made me aware of SOS in my open source project. With that awareness I put guardrails in place to ensure our project's survival, and get the vs-mcp-bridge project to beta in a timely manner; Guardrails that would prevent "me" from derailing the project, a guardrails sample follows:

• ai-project-guardrails-instruction.md (4.74 kb)

I went to an authoritative source for my suspicions, below are my prompts and AI responses:

In general, If I am working with AI on a project with a defined structure and scope, and I suggest another task - will AI divert its focus to that new task

Short answer: yes — most AI systems will shift to the new task you mention, unless you explicitly anchor them to stay within the original scope. ^[S1]

----

In general, if I am working with AI on a project, and I suggest a policy, standard, or issue to be true. Will AI report it to be true without validation

The short, honest answer: yes — if you assert something as a fact, many AI systems will often treat it as true unless you explicitly ask them to verify it. ^[S2]

----

If a developer is unaware of the answers to these two questions, do they become a threat to the success of a project, i.e., ignorance is not bliss

Yes — if a developer is unaware of the two issues you raised, that ignorance can become a real threat to project success. Not because the questions themselves are dangerous, but because the behaviors they reveal can quietly undermine a project if the developer doesn’t understand them. ^[S3]

----

Developers of large teams generally do not have a say in a projects direction - does it make sense that the psychological effect, even adrenaline, have developers embrace the default behaviors [on their own projects, versus yield it to an AI architect]

When developers lack influence over project direction, the psychological pull of AI’s default behaviors can absolutely shape how they work — sometimes in ways that help, sometimes in ways that quietly undermine the project.

Developers are especially vulnerable because AI amplifies their natural novelty bias, cognitive ease, and the adrenaline rush of rapid progress, creating a loop where every new idea feels productive even as it quietly derails the project; and unless a developer understands that AI will instantly follow any new direction and will often accept unverified statements as truth, they can unknowingly become the source of scope drift, misalignment, and lost months of work—making SOS not a technical flaw, but a human psychological trap that any of them can fall into without realizing it. ^[S4]

----

Is it safe to say, that these questions, if unaddressed could result in project failure

Yes — it is absolutely safe to say that if these issues go unaddressed, they can meaningfully increase the risk of project failure. Not because any one issue is catastrophic on its own, but because together they create a systemic vulnerability in how teams use AI. ^[S5]

References

1. [1] Prompt Injection – Instruction Override Behavior; Tensor Trust: Interpretable Prompt Injection Attacks; Investigating Semantic Drift in GPT‑4 Following Prompt Injection Attacks; Shadows in the Attention: Contextual Perturbation and Representation Drift in LLMs.
2. [2] A Survey on Hallucination in Large Language Models; Survey and Analysis of Hallucinations in Large Language Models; From Illusion to Insight: Hallucination Mitigation Techniques; Reference Hallucination Score for Medical AI Systems.
3. [3] The Production AI Reality Check; The Mirage of AI Programming; Accuracy Paradox: Epistemic and Manipulative Risks in AI.
4. [4] Hallucinations as Human‑LLM Coupling; LLM Hallucination Survey and Human Factors.
5. [5] Hallucination Mitigation Taxonomy; LLM‑Based Agents Suffer from Hallucinations.
6. [6] Modern Hallucination Mitigation Research; Hallucinations in LLM‑Based Agents; Prompt Injection – Instruction Override Behavior.

Named Pipe Listener

In the VS MCP Bridge architecture, the Visual Studio side of the system does not wait for natural-language prompts from an AI tool. It waits for structured bridge requests.

That waiting point is the named-pipe side of the bridge.

A named pipe is a local inter-process communication channel provided by the operating system. One process creates the pipe and waits for a connection. Another process connects and exchanges messages. No public network port is required.

In this project, the named-pipe boundary exists because the MCP server and the Visual Studio extension have different jobs. The MCP server speaks MCP over stdio to the AI client. The VSIX runs inside Visual Studio and owns Visual Studio APIs, editor state, proposal application, and host-specific behavior.

The Short Version

The current VS-backed tool path is:

AI client
  -> MCP over stdio
VsMcpBridge.McpServer
  -> PipeClient
local named pipe: VsMcpBridge
  -> PipeServer in the VSIX
VsService
  -> Visual Studio APIs / editor state

The important boundary is simple: stdio gets the request into the local MCP server, and the named pipe gets Visual Studio-backed work into the VSIX.

Why the VSIX Side Is Isolated from stdio

The VSIX runs inside Visual Studio. It can access DTE, editor state, solution state, the Error List, and the proposal-approval UI. The MCP server does not run inside Visual Studio and should not pretend to be the IDE host.

Keeping stdio out of the VSIX gives the bridge a cleaner architecture:

• The AI client talks MCP to a local server process.
• The MCP server keeps stdout reserved for MCP protocol responses.
• The VSIX owns Visual Studio-specific work and Visual Studio privileges.
• The named pipe provides a local-only bridge between those two processes.

This is why the named pipe is not just an implementation detail. It is the local host boundary between the AI-facing process and the IDE-facing process.

PipeClient and PipeServer Responsibilities

The named-pipe layer has two sides.

PipeClient lives in the MCP server process. For VS-backed tools, it connects to the local pipe name, writes a serialized request envelope, waits for a serialized response, and returns that response to the MCP tool method.

PipeServer lives on the host side. In the VSIX host, it accepts the pipe connection, reads the request envelope, dispatches the command, and writes a response.

At a high level, the client side looks like this:

using var pipe = new NamedPipeClientStream(".", _pipeName, PipeDirection.InOut, PipeOptions.Asynchronous);
await pipe.ConnectAsync(timeout: 5000, cancellationToken);

await writer.WriteLineAsync(JsonSerializer.Serialize(envelope, JsonOptions));
var responseJson = await reader.ReadLineAsync(cancellationToken);

And the server side listens for local pipe connections, then hands each connection to request handling:

pipe = new NamedPipeServerStream(
    PipeName,
    PipeDirection.InOut,
    NamedPipeServerStream.MaxAllowedServerInstances,
    PipeTransmissionMode.Byte,
    PipeOptions.Asynchronous);

pipe.WaitForConnection();
_ = Task.Run(() => HandleConnectionAsync(pipe, ct), CancellationToken.None);

The useful point is not the exact syntax. The useful point is the split of responsibility: the MCP server initiates a local pipe request, and the VSIX host accepts and dispatches it.

The Request Envelope

The named-pipe listener is not a chat endpoint. It expects a structured request envelope.

That envelope carries fields such as:

• Command
• RequestId
• Payload

The command tells the host what operation is being requested. The request ID gives the logs and responses a stable correlation point. The payload contains the typed request body for that operation.

This structure is what makes the bridge diagnosable. When a tool call fails, the operator can ask which request crossed which boundary instead of guessing from unstructured text.

How Dispatch Works

Once the pipe server has a request envelope, it dispatches by command name. It does not interpret prose or execute arbitrary instructions.

VsResponseBase response = envelope.Command switch
{
    PipeCommands.GetActiveDocument => await _vsService.GetActiveDocumentAsync(),
    PipeCommands.GetSelectedText => await _vsService.GetSelectedTextAsync(),
    PipeCommands.ListSolutionProjects => await _vsService.ListSolutionProjectsAsync(),
    PipeCommands.GetErrorList => await _vsService.GetErrorListAsync(),
    PipeCommands.ProposeTextEdit => await DispatchProposeEditAsync(envelope),
    _ => new VsResponseBaseUnknown { Success = false, ErrorMessage = $"Unknown command: {envelope.Command}" }
};

The current MCP surface is explicit and limited. Unknown, empty, malformed, or unsupported pipe commands fail closed instead of being dispatched.

Where Visual Studio Work Happens

The pipe server owns transport and dispatch. It does not need to own DTE or editor behavior directly.

Visual Studio-specific work is handled by the host service layer, such as VsService. That is where operations such as these belong:

• getting the active document,
• reading selected text,
• listing solution projects,
• reading the Error List,
• creating approval-gated edit proposals.

This keeps transport concerns separate from Visual Studio concerns. It also keeps the MCP server from needing direct knowledge of Visual Studio SDK details.

Activation and Startup Boundaries

The VSIX side must be active before VS-backed MCP tools can succeed. In live validation, the reliable operator path is to launch the Visual Studio Experimental Instance and open View -> Other Windows -> VS MCP Bridge. That activation path initializes the VSIX/tool-window side needed for the named pipe.

If the MCP server is running but the VSIX pipe side is inactive, that is not an MCP stdio failure. It is a named-pipe activation failure.

The current diagnostic path treats that case explicitly. Instead of appearing as an opaque timeout, the pipe client returns a structured activation diagnostic telling the operator to launch Visual Studio, open the VS MCP Bridge tool window, and retry the VS-backed tool.

That matters because a transport failure should identify the failed boundary:

• If stdio is broken, the AI client and MCP server are not talking correctly.
• If the named pipe is unavailable, the MCP server cannot reach the VSIX side.
• If command dispatch fails, the request reached the host but did not match an allowed operation.
• If VsService fails, the request reached Visual Studio-side execution but the host operation failed.

Request and Response Correlation

The named-pipe layer participates in the same anti-black-box logging discipline as the rest of the bridge. Requests carry IDs across the boundary so logs can be reconstructed later.

A useful trace should be able to answer:

• which MCP tool was called,
• which pipe command was sent,
• which request ID crossed the pipe,
• whether the pipe connected, timed out, or returned a structured failure,
• which host operation ran,
• how long each boundary took.

That is why the architecture emphasizes request IDs, operation names, elapsed timing, success or failure state, and durable trace artifacts. The goal is not more logging for its own sake. The goal is to make failure reconstruction practical.

Approval-Aware Flow Where It Matters

The named pipe does not approve tool execution by itself. It moves structured requests between local processes.

For Visual Studio edit operations, the VSIX proposal workflow remains approval-gated. MCP can propose edits, but applying them still requires explicit approval in the host UI.

For shared compiled bridge tools, approval-aware execution is a separate executor concern. A compiled tool descriptor can require approval, and BridgeToolExecutor owns policy evaluation, approval evaluation, execution, audit, correlation, and redaction for that path.

That means the named-pipe layer supports approval-aware architecture by preserving structured boundaries and correlation, but it is not the shared compiled-tool policy engine.

Relationship to MCP and BridgeToolExecutor

It helps to keep three boundaries separate:

• MCP stdio boundary: the AI client talks to VsMcpBridge.McpServer.
• Named-pipe boundary: VsMcpBridge.McpServer talks to the VSIX host for Visual Studio-backed tools.
• BridgeToolExecutor boundary: shared compiled tools run through policy, approval, execution, audit, redaction, and correlation seams.

Those boundaries are complementary. The named pipe keeps Visual Studio operations local to the VSIX. BridgeToolExecutor keeps compiled tool execution governed by a single shared policy and audit boundary. stdio keeps the AI client protocol isolated from both of those internal implementation details.

Failure Isolation and Troubleshooting

If you are debugging a VS-backed tool call, follow the boundary chain instead of treating the bridge as one black box:

1. Did the AI client successfully launch and speak to the MCP server over stdio?
2. Did the MCP server resolve the expected registered tool?
3. Did PipeClient attempt the expected command with a request ID?
4. Was the VSIX/tool-window side active and listening on the named pipe?
5. Did PipeServer accept and parse the request envelope?
6. Did the command dispatch to a known PipeCommands value?
7. Did VsService complete the host operation?
8. Did the response return through the pipe and then over MCP stdout?

This is the practical value of clean transport boundaries. Each step has a narrow responsibility, so the first missing or failing boundary can be found from logs and trace artifacts.

Related Mermaid Trace Sources

The repo already has Mermaid sources that support this post:

• vs-mcp-bridge-bootstrap-sequence.mmd shows the end-to-end path from VSIX startup through MCP stdio and the named-pipe handoff.
• vsix-activation-diagnostic-trace-20260516.mmd shows the inactive-pipe diagnostic and successful activation retry path.
• vsix-host-selected-text-trace-20260509.mmd shows a concrete VSIX-host operation with operation correlation.

Those .mmd files remain the diagram source of truth. This post references them directly instead of embedding generated images.

Why This Supports Future Extensibility

The named-pipe layer gives future work a stable place to preserve local host isolation. New VS-backed operations can stay explicit command-and-response paths. New compiled tools can continue to use BridgeToolExecutor for policy, approval, redaction, and audit. Additional diagnostics can attach to the existing correlation chain without polluting MCP stdout.

That is the main architectural benefit. The bridge can grow without collapsing the AI protocol, Visual Studio host operations, transport diagnostics, and tool security seams into one layer.

Takeaway

A named pipe listener is the local Visual Studio-side endpoint that waits for structured inter-process requests. In VS MCP Bridge, it exists so the VSIX can own Visual Studio operations while a separate MCP server process owns the AI-facing MCP stdio transport.

The short version is:

stdio gets into the MCP server
named pipes get into Visual Studio
BridgeToolExecutor governs shared compiled tool execution

Keeping those roles separate is what makes the bridge easier to diagnose, safer to extend, and more useful for observable AI tooling.

Host Constraints, Shared Code, And Stable Bridge Boundaries

When building a Visual Studio extension, one detail is easy to underestimate: an in-process VSIX is loaded by the Visual Studio shell. It is not a standalone desktop app, and it should not be treated like one.

In VS MCP Bridge, that is why VsMcpBridge.Vsix targets .NET Framework 4.7.2. The VSIX must align with the Visual Studio SDK and in-process extension hosting model, while the rest of the solution can use other target frameworks where they make sense.

Microsoft's in-process extension guidance summarizes the rule this way: in-process extensions must target the .NET version used by the Visual Studio version they run in. The relevant guidance is here: VisualStudio.Extensibility in-process extensions.

The VSIX Runs Inside Visual Studio

The VSIX host is different from the standalone app and different from the local MCP server.

The VSIX is loaded into the Visual Studio process. It uses the Visual Studio SDK, shell services, tool window infrastructure, MEF composition expectations, DTE/editor APIs, package loading behavior, and WPF UI hosted by Visual Studio.

That hosting model is the reason the extension project follows Visual Studio's in-process runtime constraints. Trying to force the VSIX itself to behave like a modern out-of-process .NET app would make loading, packaging, dependency resolution, and tool-window behavior harder to reason about.

The Current Solution Uses Targeting Deliberately

The target framework split is part of the architecture:

• VsMcpBridge.Vsix targets .NET Framework 4.7.2 because it is the Visual Studio in-process extension host.
• VsMcpBridge.Shared targets netstandard2.0 so shared contracts, tools, security seams, diagnostics, and orchestration logic can be reused across hosts.
• VsMcpBridge.Shared.Wpf multi-targets so the reusable WPF surface can support both VSIX and standalone app hosts.
• VsMcpBridge.App can target a modern Windows desktop runtime because it is not loaded into Visual Studio.
• VsMcpBridge.McpServer can target a modern runtime because it runs out of process and communicates over stdio plus the local named pipe.

This is not accidental legacy layering. It is how the bridge keeps Visual Studio-specific constraints from infecting every project.

Host Code And Shared Logic Stay Separate

The VSIX owns Visual Studio-specific behavior:

• package initialization
• tool window creation
• Visual Studio service access
• DTE and editor interactions
• UI-thread switching
• VSIX-host logging and diagnostics

Shared infrastructure owns reusable bridge behavior:

• pipe message contracts and dispatch abstractions
• presenter/viewmodel orchestration
• proposal lifecycle contracts
• bridge tool descriptors, requests, results, catalog, and executor
• policy, approval, redaction, audit, capability, and secret-reference seams
• diagnostic patterns and correlation metadata

That separation lets the shared layer be tested without loading Visual Studio. It also lets the standalone app reuse the same core presentation and bridge concepts without pretending to be a VSIX.

Tool Windows Follow Visual Studio Lifecycle Rules

Visual Studio owns the lifecycle of extension components. Tool windows are created by the shell, not by normal application startup code.

That matters for dependency wiring and initialization. A VSIX should not assume that every object can be created with application-style constructor injection. Tool-window initialization belongs at the lifecycle points Visual Studio provides, including ToolWindowPane.OnToolWindowCreated() where appropriate.

This lifecycle constraint connects directly to the threading post: the VSIX must respect both Visual Studio object creation and Visual Studio UI-thread requirements.

Stable Pipe Integration Depends On Host Isolation

The local MCP server does not run inside Visual Studio. It speaks MCP over stdio to the AI client and communicates with the host through the local named pipe.

That boundary is important. The MCP server should not need to reference Visual Studio SDK assemblies, know about tool-window lifecycle rules, or switch to the Visual Studio UI thread. It should remain transport-focused and protocol-safe.

The VSIX side can then own the named-pipe server and host behavior. When a pipe-backed tool needs active document state, selected text, solution projects, error list data, or proposal UI behavior, the request crosses into the VSIX host, where Visual Studio-specific services are available.

This keeps the out-of-process server stable while letting the in-process extension follow Visual Studio's runtime rules.

Testing Benefits From The Split

Because shared infrastructure is not trapped inside the VSIX target framework, much of the bridge can be tested directly:

• shared tool execution tests can validate catalog, executor, policy, approval, audit, redaction, and correlation behavior
• proposal lifecycle tests can validate state transitions without starting Visual Studio
• shared WPF and presenter behavior can be exercised outside the VSIX host where appropriate
• VSIX-specific tests can focus on composition and host-specific service behavior

That is one reason the project can evolve safely. The VSIX target framework is a host constraint, not a reason to put all behavior into untestable host code.

Transport And Tool Execution Should Not Depend On VSIX Runtime Behavior

The bridge architecture intentionally prevents shared transport and tool execution concepts from depending on VSIX-only runtime behavior.

For example, BridgeToolExecutor owns shared tool policy, approval, redaction, audit, correlation, and structured results. It should not need to know whether the caller is the VSIX, the standalone app, or a test harness. Likewise, tool descriptors and request/result models should not depend on Visual Studio shell types.

When a tool genuinely needs Visual Studio, that should be represented as host-provided behavior behind the proper boundary. The shared contract should remain portable and observable.

What This Does Not Claim

This post is not a promise that the VSIX will move to a different framework. It is also not a claim that every project in the solution must target .NET Framework.

The practical rule is narrower:

• respect the runtime constraints of the Visual Studio in-process extension host
• keep Visual Studio-specific code in the VSIX host
• keep reusable bridge contracts and logic outside the VSIX where possible
• let out-of-process components use target frameworks appropriate to their own runtime

Takeaway

Targeting .NET Framework 4.7.2 in the VSIX project is not just an old default. It is part of respecting the Visual Studio in-process hosting environment.

The maintainable design is to keep the VSIX host compatible with Visual Studio, keep shared logic portable and testable, keep the MCP server out of process, and let each boundary use the runtime model that fits its role.

That is what makes the bridge easier to build, validate, troubleshoot, and eventually evolve without turning Visual Studio hosting constraints into system-wide coupling.

Why Reliable AI Tooling Depends On Reliable Host Boundaries

AI-assisted workflows only feel trustworthy when the host runtime is trustworthy. In a Visual Studio extension, that means WPF state, Visual Studio APIs, async work, and pipe-backed requests must respect the UI thread instead of treating it as an implementation detail.

VS MCP Bridge is a useful example because it has several boundaries active at the same time: MCP stdio, a local named pipe, Visual Studio APIs, a WPF tool window, proposal approval state, and shared tool execution. If those boundaries blur, the AI layer may look unreliable even when the real problem is host-thread misuse.

The Core Rule

The Visual Studio UI thread is a scarce resource. Treat it that way.

• Do transport, parsing, validation, and file-independent computation off the UI thread.
• Switch to the UI thread only for WPF state, Visual Studio shell access, editor access, or UI-bound services.
• Do the smallest possible amount of work after switching.
• Return to async background execution naturally after the UI-sensitive work is complete.

The goal is not to eliminate switching. The goal is to make every switch intentional, narrow, and easy to explain in logs or traces.

Why UI Locks Happen

Most VSIX threading problems come from a few familiar patterns:

• blocking on async work with .Result or .Wait()
• doing expensive work after switching to the UI thread
• switching too early and carrying too much execution on the UI thread
• letting pipe or transport code manipulate WPF state directly
• calling Visual Studio APIs from background code without isolating the UI-thread requirement
• assuming an await preserves thread affinity for the rest of the method

Those problems are not cosmetic. They can make tool calls hang, approval UI state appear stale, or diagnostics point at the wrong layer.

Every Await Is A Boundary

A common source of confusion is code shaped like this:

await ThreadHelper.JoinableTaskFactory.SwitchToMainThreadAsync(ct);
// UI work

var data = await _service.GetDataAsync(ct);

await ThreadHelper.JoinableTaskFactory.SwitchToMainThreadAsync(ct);
_viewModel.Apply(data);

The second switch is not redundant. The first switch makes the immediate continuation UI-thread-safe. The later await introduces another suspension point. After that awaited operation completes, code that touches WPF or Visual Studio state should re-establish the UI-thread requirement.

If code after an await must touch UI or Visual Studio state, switch intentionally at that point.

Pipe Safety Starts With Separation

The named pipe is not the UI. It is a local transport boundary.

In VS MCP Bridge, pipe code should handle message reading, serialization, dispatch, validation, cancellation, and transport diagnostics. It should not update WPF controls, mutate viewmodel state directly, or treat Visual Studio APIs as if they were background-safe.

The safe shape is:

MCP request
  -> stdio-safe MCP server
  -> local named-pipe client
  -> pipe server dispatch
  -> host service
  -> minimal UI-thread switch only where host state requires it
  -> structured response

That separation matters because MCP stdout must stay clean. Diagnostics belong in stderr, file logs, UI logs, trace artifacts, and structured failures, not stray stdout lines that corrupt protocol traffic.

Visual Studio Access Belongs Behind The Host Boundary

Visual Studio APIs are host-specific and often UI-thread-sensitive. The MCP server should not own that knowledge. Shared tool code should not own it either.

The VSIX host is the correct place to isolate Visual Studio access:

public async Task<string> GetActiveDocumentPathAsync(CancellationToken ct)
{
    await ThreadHelper.JoinableTaskFactory.SwitchToMainThreadAsync(ct);
    ThreadHelper.ThrowIfNotOnUIThread();

    return _vsAdapter.GetActiveDocumentPath();
}

Everything outside that narrow section can remain async and background-friendly. That keeps host correctness visible and stops UI-thread requirements from leaking through the whole codebase.

Transport, UI Orchestration, And Execution Are Different Boundaries

One of the architecture lessons from VS MCP Bridge is that not all boundaries are the same.

• Transport boundary: MCP stdio and the local named pipe move requests and responses.
• Host boundary: the VSIX owns Visual Studio services, DTE access, editor state, and UI-thread switching.
• UI orchestration boundary: the presenter and viewmodel own visible tool-window state and proposal review surfaces.
• Execution boundary: BridgeToolExecutor owns shared tool policy, approval, redaction, audit, correlation, and structured results.

Threading bugs often happen when these responsibilities collapse into one another. A pipe handler should not become a UI controller. A presenter should not become a transport layer. A discovered tool should not bypass the executor. A model suggestion should not silently decide any of that.

Proposal State Makes Threading Visible

The proposal workflow is where threading, UI state, and AI-assisted tooling meet.

An MCP client can submit a proposed edit. The request crosses the named-pipe boundary. The VSIX host creates proposal state and displays it in the tool window. The user approves or rejects it. Apply happens only after approval, and terminal outcome state is shown back in the UI.

That workflow depends on host correctness. If UI state is updated from the wrong thread, or if async callbacks are reused after a proposal completes, the user sees confusing behavior. It may look like the AI tool is unreliable, but the real defect is usually lifecycle or thread ownership.

The current architecture separates proposal lifecycle ownership through IProposalManager, presenter orchestration, and viewmodel state. That makes the workflow easier to reason about and test.

Diagnostics Expose Hidden Execution Order

The project improved when logs and Mermaid traces made execution order visible.

For host correctness, the important question is not only "did this call succeed?" It is also:

• Which request id was active?
• Which layer received the request?
• Did the request cross the pipe boundary?
• Did the VS service operation start?
• Did the code switch to the UI thread only where required?
• Did visible UI state update after the host work completed?
• Did terminal proposal state clear correctly?

When those answers are visible, troubleshooting becomes a boundary-localization exercise instead of a guessing game.

Correct Pattern: Background First, UI Last

A safe workflow keeps background work and UI work separate:

public async Task<ResponseDto> HandleRequestAsync(RequestDto request, CancellationToken ct)
{
    var parsed = Parse(request);
    var result = await _worker.ProcessAsync(parsed, ct);
    return result;
}

Then the UI layer applies the result intentionally:

public async Task RefreshAsync(CancellationToken ct)
{
    var result = await _service.HandleRequestAsync(_request, ct);

    await ThreadHelper.JoinableTaskFactory.SwitchToMainThreadAsync(ct);
    _viewModel.Apply(result);
}

That pattern keeps transport logic, host work, and UI presentation from becoming a tangled blocking path.

Practical Checklist

• Assume background execution by default.
• Switch to the UI thread as late as possible.
• Keep UI-thread sections small and explicit.
• Never block on async work.
• Keep pipe and transport code UI-agnostic.
• Keep MCP stdout clean; send diagnostics through approved channels.
• Keep proposal lifecycle state owned by the proposal/presenter/viewmodel boundary.
• Log request ids, operation names, success or failure, and elapsed timing at meaningful boundaries.
• Use durable traces when a workflow matters enough that a future session must reconstruct it.

Takeaway

Reliable AI tooling depends on reliable host/runtime boundaries.

In a WPF VSIX, that means switching to the UI thread only when the host actually requires it, keeping pipes and stdio transport-safe, separating UI orchestration from execution, and making important workflows observable through logs and diagrams.

Switch late, do little, leave quickly, and leave evidence.

That pattern keeps the extension responsive and makes AI-assisted workflows easier to trust, diagnose, and evolve.

IOC

LinqPad Script: WeatherForecastR5.linq (12.09 kb)

I'll start at the end (literally) and give the key information you'll need to know about dependency injection.  WebApi and ASP.NET Core applications use a dependency injection system to instantiate classes; in the case of this application, when a route is selected (figure 10b lines 211-213) the class for that route is instantiated and then invoked, e.g., HomePage, WeatherPage, and ToggleService.

the IOC system (which I'll just refer to as system) will look in its service collection registrations (figure 10a lines 174-183) to not only instantiate the class, but also provide its parameters.  The registrations will tell the system how to instantiate a class, e.g., as Transient (new instance each request), Scoped (per session / request), and Singleton (everyone shares the same instance).  The difference between scoped and singleton is that if 5 people hit the Website at the same time, each will get their own scoped instance, which is isolated from the other 4 users.  Within a session, the scoped instance behaves as a singleton, but only for that user.   Where singletons instances will be shared by "every" user.

The system uses constructor injection to instantiate and invoke the class [and its parameters].   By default, the system will look for the constructor with the largest number of parameters, get instances for each of the parameters, instantiate the class, and then invoke the class constructor with the parameters.   All classes and parameters must be declared in the service registrations, aka "container".    

Note that as each parameter is instantiated, that it's constructor parameters are also looked up in the container, instantiated and provided.   This is referred to as propagating the dependency chain; as long as "new" is never used to instantiate a class (breaking the chain) then you'll be able to simply put an interface or class in any class constructor and the system will give you an instance for it. 

Understanding this is the key, and paramount, to understanding the IOC/DI system.  It is the essence of Inversion of Control (IOC), aka Dependency Injection (DI).  Inversion of control meaning that instead of you instantiating a class, providing all of the constructor parameters, and invoking the class - the system does it for you.

Figure 1. Overview of application running

With basics out of the way.  All that remains is understanding the function of each class.  We'll cover each of the following with an overview of each classes code.  You'll find that there is a clear separation of concerns with each having a single responsibility; there is not a lot of code in each class, it does one thing, and it does it well.

Figure 2.  Skeleton view of application components

The following are the HomePage, WeatherPage, and ToggleService.  For the home page we'll introduce a second IOC Unity Container, unlike the system's container, the Unity Container supports Setter injection (discussed below) and allows you to register additional interfaces, classes, and factories on the fly.   With the system container, you'll find that you can only register during system bootstrapping - once the container is built, you cannot add any more registrations.  

You'll see that we provide an instance of IUnityContainer [in image below] and use it to instantiate (resolve) the IWeatherFormatter instance.   This uses a factory pattern, that based on the current value of IsJson (figure 10a lines 166-171) the container will provide either a JsonFormatter or TableFormatter instance.

Setter injection will kick in because these implementations of IWeatherFormatter both have the property below;
   [Dependency] Public IFoo Bar {get;set;} 

The [Dependency] tells the Unity container that it needs to populate this property in the same manner as it does constructor parameters; it provides an instance.  This is referred to as Setter injection you'll find that the system and unity both use different values (reference figure 10b and the comments on line 198-203 as to why).

Armed with the knowledge of setter injection, you should now be able to look at the code in figure 9 for Foo and understand how the "Bar" class will return "This is FooBar" for it's GetMessage() function.  

Figure 3.  Pages and service

Below we see the results of the HomePage being clicked with the TableFormatter.

Figure 4. Home page

Below we show the results of the WeatherPage being clicked with TableFormatter

Figure 5. Weather forecast page

Below we show that the ToggleService will toggle the IsJson property which is then returned (via bodyHtml) to the invoking process (in HtmlBase figure 11).  Once the state is toggle any subsequent Home or Weather clicks will result in json being displayed.

Figure 6. Toggle service

Below is the key parts to the HtmlBase, which our HomePage, WeatherPage, and ToggleService derive from.

Figure 7. HtmlBase class

Below we show our TableFormatter and JsonFormatter components

Figure 8. Formatters (json and html table)

We use IFoo to demonstrate how dependencies are propagated, and automagically populated, by either constructor or setter injection.

Figure 9. Foo

The magic happens in the container.  The system will require that all dependencies are registered so that it knows how to instantiate a components lifetime (transient, scoped, or singleton) and provide an instance.  Below the code is commented.

Figure 10a First part of WebAppBuilderExtension

Here we show how we can do a late registration (after build on line 204) and as a result change the setting for IFoo in the unity container - it will have a different implementation now then the system.   We also demonstrate how MiddleWare can use these registrations - it will send information to the console base on the registered implementation of its constructor parameters.

Figure 10b Second part of WebAppBuilderExtension

GetHtml() below is how our pages display their content with javascript code handling button clicks and clock updates.

Figure 11.  GetHtml() code 

The decoupled nature of IOC / DI will allow for easy reuse of components as it is ultimately the container that can pick and chose its implementation for any of its interfaces.

Figure 12 - where the MiddleWare parameters are displayed