All posts by billkrat

Direct Corpus Interaction - Introduction

13. June 2026 BillKrat Comments (0)

DCI Abstract * DCI Introduction

Author’s Note: In my previous post, I worked through the abstract of the Direct Corpus Interaction (DCI) paper and discovered that much of my initial confusion stemmed from assumptions about how modern retrieval systems actually work. Before moving into the introduction, I realized I needed a clearer understanding of the retrieval approaches the paper repeatedly references. The blockquote below contains a statement from the introduction that led me to investigate the difference between sparse and dense retrieval systems. The explanatory notes that follow are part of my learning journey and are intended to help other developers who may be encountering these concepts for the first time.

dci.pdf (2.38 mb) — Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction

This interface underpins a wide range of applications, including retrieval-augmented generation (Lewis et al., 2020; Gao et al., 2023; Singh et al., 2025), open-domain question answering (Trivedi et al., 2022; Press et al., 2023), and deep research (Wei et al., 2025; Chen et al., 2025b).

My Initial Misunderstanding

When I first read this sentence, I mentally grouped Retrieval-Augmented Generation (RAG), Open-Domain Question Answering (ODQA), and Deep Research together as different retrieval approaches. Since I was already familiar with RAG and was reading a paper proposing Direct Corpus Interaction (DCI), my brain immediately started categorizing everything as competing retrieval techniques.

That turned out to be an inaccurate mental model.

The Distinction That Helped

The most useful clarification for me was realizing that the paper is discussing application categories, not necessarily retrieval methods.

A retrieval method is concerned with how information is found.

BM25
Dense Retrieval
Hybrid Retrieval
Top-k Retrieval
Direct Corpus Interaction (DCI)

An application category is concerned with what the system is trying to accomplish.

RAG Assistant
Open-Domain Question Answering System
Deep Research Agent

Once I separated those two ideas, the sentence became much easier to understand. The authors are not listing competing retrieval techniques. They are listing examples of systems and applications that depend on retrieval.

Visualizing The Difference

One reason this distinction initially escaped me is that all three application categories involve finding information. From a distance they can look very similar.

At a simplified level, a RAG system often follows a pattern like:

Question
    ↓
Retrieve Documents
    ↓
Generate Answer

Open-Domain Question Answering systems are also attempting to answer questions using information that may exist anywhere within a large corpus:

Question
    ↓
Retrieve Documents
    ↓
Read Documents
    ↓
Generate Answer

Deep Research systems typically extend this idea into a longer investigation:

Question
    ↓
Search
    ↓
Read
    ↓
Search Again
    ↓
Compare Evidence
    ↓
Investigate Gaps
    ↓
Repeat
    ↓
Produce Report

The exact implementations differ, but these simplified workflows helped me understand why the paper groups them together. They are all systems that rely on retrieving and examining information from a corpus.

A Common Misunderstanding

An easy mistake is to assume that DCI belongs in the same category as RAG, Open-Domain QA, and Deep Research.

That is not how I currently understand the paper.

RAG, Open-Domain QA, and Deep Research are applications. DCI is being proposed as a different way for those applications to interact with a corpus.

In other words, DCI is closer to a retrieval or investigation strategy than an end-user application category.

Why This Matters For Understanding DCI

The most important takeaway for me was realizing that the paper is not simply introducing another retrieval technique alongside existing retrieval techniques. It is questioning a deeper assumption that many of these systems share.

Traditional retrieval pipelines often begin with a top-k retrieval step:

Question
    ↓
Retrieve Top-k Documents
    ↓
Work From Retrieved Results

DCI appears to challenge the assumption that an investigation must begin that way.

Instead of retrieving a small set of candidate documents and working only from those results, an agent may directly interact with the corpus through actions such as searching, opening files, inspecting context, following references, and gathering evidence incrementally.

At this point in the paper, the authors have not yet proven that this approach is superior. They are establishing why retrieval matters and why rethinking retrieval could affect a broad range of systems, including RAG applications, Open-Domain QA systems, and Deep Research agents.

What Changed In My Understanding

Before reading this section carefully, I was viewing the discussion primarily as:

RAG
versus
DCI

After working through the terminology, I now see the comparison as something closer to:

Top-k Retrieval
versus
Direct Investigation

That shift in perspective made the rest of the introduction significantly easier to follow and helped me understand why the authors repeatedly reference multiple application categories throughout the paper.

Understanding Sparse and Dense Retrieval

"In standard retrieval-augmented pipelines, documents are chunked, indexed, and filtered into a top-k candidate set using well-established sparse (Robertson et al., 1994) or dense (Karpukhin et al., 2020) techniques before downstream reasoning begins."

What This Means

The paper references two major categories of retrieval systems: sparse retrieval and dense retrieval.
Both approaches are designed to reduce a large corpus into a smaller set of candidate documents before the language model begins reasoning.
The goal is efficiency: instead of examining everything, the model receives only the documents considered most relevant.
The DCI paper is not primarily comparing sparse retrieval against dense retrieval. Instead, it questions whether this entire top-k retrieval stage has become a bottleneck for capable agents.

Sparse Retrieval

Sparse retrieval relies primarily on explicit terms appearing in documents.
Examples include:

BM25
TF-IDF
traditional keyword search
search engine inverted indexes

These systems work well when exact wording matters.
Developer examples include:

method names
class names
configuration keys
exception messages
employee IDs
filenames

If a developer searches for NullReferenceException, a sparse retriever can quickly locate documents containing that exact phrase.

Dense Retrieval

Dense retrieval uses embeddings to represent both queries and documents as vectors.
Instead of matching exact words, the system attempts to find documents with similar meaning.
This allows retrieval to succeed even when the query and document use different terminology.
For example:

A query asks about user authentication.
A document discusses JWT-based login flows.
The exact words may differ, but the concepts are related.

Dense retrieval excels when semantic meaning is more important than exact terminology.

A Useful Mental Model

One of my initial assumptions was that sparse retrieval might be useful for things like organizational charts or employee lookups, while dense retrieval might be better for blog content or documentation.
While not technically precise, that intuition points toward an important distinction.
Sparse retrieval often behaves like an exact lookup system.
Dense retrieval behaves more like a concept lookup system.
The actual distinction is not the type of content being searched but how relevance is determined.

Why This Matters for DCI

The paper is not arguing that sparse retrieval is bad.
The paper is not arguing that dense retrieval is bad.
Both approaches have been highly successful and remain important components of modern retrieval systems.
The paper is questioning whether forcing every information request through a single retrieval step unnecessarily limits capable agents.
In traditional systems, the workflow often looks like:

User Query
     ↓
Sparse Retriever
or
Dense Retriever
     ↓
Top-K Results
     ↓
LLM Reasoning

DCI proposes something closer to:

User Query
     ↓
Agent
     ↓
Search Corpus
Open Files
Inspect Context
Refine Search
Follow Clues
     ↓
Answer

The comparison is therefore not primarily sparse versus dense.
The comparison is top-k retrieval versus direct investigation.

Key Insight

Sparse retrieval and dense retrieval represent different ways of finding relevant information.
DCI challenges a deeper assumption: that retrieval should always happen as a single filtering step before reasoning begins.
The paper argues that increasingly capable agents may benefit from interacting directly with a corpus rather than being restricted to a pre-filtered candidate list.
This shifts the question from:

What are the most relevant documents?

What investigation should the agent perform next?

DCI, Agent Loops, and System Architecture

"This becomes particularly beneficial once the agent is strong enough to search strategically (as recent systems suggest; e.g., Anthropic, 2026; OpenAI, 2026)."

What This Means

This sentence helped clarify an important boundary for me.
Direct Corpus Interaction is not just about giving a model access to files.
It becomes powerful when the agent can use those files strategically.
That means the agent can search, inspect, revise its assumptions, search again, and continue narrowing the investigation.

The Agent Loop

In a DCI-style workflow, the agent often behaves in a loop:

Observe the task
     ↓
Form a search strategy
     ↓
Run a tool
     ↓
Inspect the result
     ↓
Revise the hypothesis
     ↓
Run another tool
     ↓
Continue until enough evidence exists

This resembles how a developer investigates a codebase.
The developer does not usually ask for one perfect search result.
The developer searches, reads, narrows, checks nearby context, follows references, and adjusts direction based on what is discovered.

What the Model Handles

The model, when capable enough, handles the strategic reasoning.
It decides what to search for next.
It interprets failed searches.
It may decide that a search term was too broad, too narrow, misspelled, or based on a wrong assumption.
It chooses whether to inspect a file, search nearby references, or try a different clue.

What Supporting Infrastructure Handles

The infrastructure surrounding an agent should not try to become the agent's brain.
Its role is to provide controlled access to the environment in which the investigation occurs.
Its responsibilities are typically architectural and operational:

expose safe tools
transmit relevant context
preserve human approval gates
shape tool results into usable form
avoid flooding the model with unnecessary output
log durable evidence
enforce workspace or system boundaries
support observation before automation

Why the "Thought" Loop Matters

Some agent systems have an internal reasoning loop that allows the model to plan, call tools, inspect results, and continue.
That internal reasoning process may not be exposed to the bridge.
In many systems, the bridge may only see tool requests and tool results, not the private reasoning that led to them.
This means the bridge should not depend on seeing the model's full chain of thought.
Instead, it should depend on observable behavior:

what tool was requested
what inputs were provided
what result was returned
whether the user approved the action
what evidence was produced

Implications for Agent-Based Systems

This distinction becomes important when evaluating systems that support agentic workflows.
The surrounding infrastructure does not need to implement DCI by itself.
Instead, it needs to make DCI possible by providing a safe, precise, and observable interface into the environment being investigated.
Examples of capabilities might include:

list available resources
read approved files or documents
search within an approved scope
return bounded snippets
preserve source references and line numbers
require approval before write operations

The agent remains responsible for deciding how to use those capabilities.
For example, these concepts influence how agent-supporting systems expose selected files, workspace searches, approvals, and evidence collection while leaving strategic investigation decisions to the agent. In my own work on vs-mcp-bridge, these same ideas have influenced how the bridge exposes developer workspace capabilities without attempting to direct the investigation itself.

Important Limitation

Not every model will be equally effective at Direct Corpus Interaction.
A weaker model may call tools poorly, search too broadly, miss important clues, or fail to recover from unproductive results.
A stronger model may perform better within the same environment because it can plan, revise its assumptions, and conduct a multi-step investigation more effectively.
This means access to tools alone does not guarantee successful investigation.
The quality of the agent's reasoning remains an important factor in how effectively a corpus can be explored.

Key Insight

The environment provides access.
The agent provides strategy.
The user provides authority.
Direct Corpus Interaction becomes valuable when those roles remain distinct.

Direct Corpus Interaction (DCI) - Abstract

13. June 2026 BillKrat Comments (0)

DCI Abstract * Dci Introduction

Author’s Note: I am actively learning about Direct Corpus Interaction (DCI) and documenting my understanding as I go. The blockquotes in this post contain excerpts from the DCI research paper that exposed gaps in my own understanding. The explanatory sections that follow are learning notes generated with ChatGPT to help me clarify the concepts. They are not presented as original research, but as study notes for developers following the same path.

dci.pdf (2.38 mb) a paper on "Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction"; the article abstract follows:

Understanding the Retrieval Bottleneck

"Modern retrieval systems, whether lexical or semantic, expose a corpus through a fixed similarity interface that compresses access into a single top-k retrieval step before reasoning."

What This Means

A corpus is the body of information the system can search. In a developer context, this could be source code, documentation, logs, tickets, markdown files, PDFs, or a knowledge base.
To expose a corpus means giving an AI system some way to access that information.
In many traditional retrieval systems, the AI does not inspect the raw corpus directly. Instead, it asks a retriever for the most relevant chunks.
A fixed similarity interface means the retriever uses a predefined way of deciding what is relevant. That might be lexical matching, semantic similarity, vector search, BM25, or another ranking mechanism.
The important point is that the AI receives a filtered result set instead of direct access to the full information space.

Why It Matters

This design is efficient. The retriever narrows a large corpus down to a small set of candidate results before the language model starts reasoning.
However, that efficiency comes with a tradeoff. Information that is filtered out early may never be seen by the model.
If the retriever misses a critical file, phrase, log entry, method name, or clue, the downstream reasoning step cannot recover it because the model never received it.
This is what the paper means by compresses access. The system reduces a large, messy information space into a small ranked list.

Developer Translation

This is similar to asking someone to debug a production issue, but only giving them the top five search results from the repository.
Those five results may be useful, but they may also hide the real trail: a config value, an obscure log message, a generated file, a test artifact, or a second-order reference elsewhere in the codebase.
A human developer usually does not investigate that way. We search, inspect, refine, search again, follow references, check surrounding context, and revise our assumptions as we go.

DCI Perspective

Direct Corpus Interaction challenges the assumption that retrieval should always happen as a single pre-reasoning step.
Instead of asking a retriever for the top results, DCI lets the agent interact with the raw corpus more directly using tools such as search, grep, file reads, shell commands, and lightweight scripts.
The paper’s argument is not that traditional retrieval is useless. It is that capable agents may need a richer interface than a fixed top-k result list.

Key Insight

Traditional retrieval asks: What are the most similar chunks?
DCI asks: What investigation should the agent perform against the corpus?
That shift matters because complex tasks often require exploration, verification, and refinement rather than a single search result.

Understanding the Bottleneck in Traditional Retrieval

"This abstraction is efficient, but for agentic search, it becomes a bottleneck: exact lexical constraints, sparse clue conjunctions, local context checks, and multi-step hypothesis refinement are difficult to implement by calling a conventional off-the-shelf retriever, and evidence filtered out early cannot be recovered by stronger downstream reasoning."

What This Means

The paper is saying that traditional retrieval works well when the task is simple: ask a question, retrieve likely documents, then generate an answer.
Agentic search is different. The agent may need to investigate over multiple steps, discover intermediate clues, test assumptions, and change direction based on what it finds.
In that setting, a fixed retriever can become a bottleneck because it controls what the agent is allowed to see.

Exact Lexical Constraints

A lexical constraint means the exact text matters.
Examples include method names, class names, exception messages, configuration keys, IDs, filenames, command-line flags, database columns, or specific phrases.
Semantic retrieval may understand the general meaning of a question, but it can still miss exact strings that are critical to solving the problem.
Developer example: NullReferenceException is not just a general concept. It is an exact term you may need to find in logs, tests, or issue reports.

Sparse Clue Conjunctions

A sparse clue is a small piece of evidence that may not look important by itself.
A conjunction means several clues need to be combined.
One clue might be a date, another might be a filename, another might be a partial error message, and another might be a component name.
A traditional retriever may not rank any one clue highly enough to surface the right document.
DCI allows the agent to combine clues through iterative searches, such as searching for one term, narrowing by another, then inspecting the surrounding context.

Local Context Checks

Finding a match is often not enough. The agent needs to inspect what appears around the match.
In code, nearby context might include the containing method, imports, comments, dependency injection setup, test assertions, or error handling.
In documentation, nearby context might clarify whether a term is being defined, contradicted, deprecated, or used as an example.
DCI gives the agent a way to inspect that local context directly instead of relying only on a preselected snippet.

Multi-Step Hypothesis Refinement

Hypothesis refinement means the agent starts with a possible explanation, checks it against evidence, then revises the explanation.
This is how developers commonly debug: form a theory, search for evidence, inspect the result, discover a new clue, and adjust the theory.
Traditional retrieval often front-loads the search step. DCI makes search part of the reasoning loop.

Why Stronger Reasoning Cannot Recover Missing Evidence

A stronger model can reason better over the evidence it receives.
But if important evidence was filtered out before the model saw it, the model has nothing concrete to reason from.
This is the core bottleneck: the retrieval interface can limit the reasoning process before reasoning even begins.

Key Insight

The paper is shifting attention from the intelligence of the model to the quality of the interface between the model and the corpus.
For agentic work, the question is not only: How smart is the model?
It is also: What can the model actually observe, inspect, verify, and act upon?

"To tackle the limitation, we study direct corpus interaction (DCI), where an agent searches the raw corpus directly with general purpose terminal tools (e.g., grep, file reads, shell commands, lightweight scripts), without any embedding model, vector index, or retrieval API. This approach
requires no offline indexing and adapts naturally to evolving local corpora. Across IR benchmarks and end-to-end agentic search tasks, this simple setup substantially outperforms strong sparse, dense, and reranking baselines on several BRIGHT and BEIR datasets, and attains strong accuracy on BrowseComp-Plus and multi-hop QA without relying on any conventional semantic retriever. Our results indicate that as language agents become stronger, retrieval quality depends not only on reasoning ability but also on the resolution of the interface through which the model interacts with the corpus, with which DCI opens a broader interface-design space for agentic search. "

What This Means

The paper proposes an alternative retrieval model called Direct Corpus Interaction (DCI).
Instead of asking a retriever for the “best matching documents,” the agent interacts with the raw corpus directly using normal operating-system style tools.
The examples listed in the paper:

grep
file reads
shell commands
lightweight scripts

are important because they are not specialized AI retrieval systems. They are generic tools developers already use daily.

Why "Raw Corpus" Matters

In traditional Retrieval-Augmented Generation (RAG), the corpus is usually:

chunked into smaller pieces
converted into embeddings
stored in a vector database
retrieved through similarity search

DCI skips that entire preprocessing pipeline.
The agent works against the original files directly:

source code
markdown
logs
PDF exports
JSON
configuration files
directory structures

This is significant because the structure, naming, formatting, and neighboring context of the original files are preserved.

"Without Any Embedding Model, Vector Index, or Retrieval API"

An embedding model converts text into numerical vectors so semantic similarity can be calculated mathematically.
A vector index is a specialized data structure optimized for fast similarity search over those vectors.
A retrieval API is the interface the language model normally uses to request relevant documents.
DCI intentionally removes all of those layers.
Instead of:

“Give me the top 5 semantically similar chunks.”

the model effectively performs investigations itself:

“Search for this exact phrase.”
“Open this file.”
“Check nearby lines.”
“Find references to this identifier.”

"No Offline Indexing"

Traditional retrieval systems usually require preprocessing before search becomes efficient.
That preprocessing step may:

generate embeddings
build indexes
split documents into chunks
calculate metadata

This work is often performed ahead of time, which is why the paper calls it offline indexing.
DCI avoids this requirement entirely because the agent searches the live corpus directly.
This becomes especially useful when the corpus changes frequently, such as:

active code repositories
local developer workspaces
runtime logs
generated artifacts
temporary debugging files

The paper argues that DCI naturally adapts to evolving corpora because there is no index that must constantly be rebuilt or synchronized.

IR Benchmarks

IR stands for Information Retrieval.
Information Retrieval is the field focused on finding relevant information inside large collections of data.
Search engines are one example of an IR system.
IR benchmarks are standardized datasets used to evaluate how well retrieval systems locate relevant information.

End-to-End Agentic Search Tasks

An end-to-end task means the system must complete the full workflow itself rather than only a small isolated step.
Agentic search refers to AI systems that:

plan investigations
search iteratively
revise hypotheses
follow intermediate clues
perform multi-step reasoning

Instead of performing one search and stopping, the agent behaves more like a researcher or developer investigating a problem.

Reranking Baselines

A baseline is a comparison system used to measure whether a new approach performs better or worse.
A reranker is a second-stage model that reorders retrieved search results after the initial retrieval step.
Example workflow:

Retrieve 100 candidate documents
Use a stronger model to score them again
Return the best-ranked subset

Reranking is commonly used to improve retrieval quality in advanced RAG pipelines.
The paper claims DCI outperformed even these stronger retrieval pipelines.

BRIGHT and BEIR

BRIGHT and BEIR are benchmark suites used to evaluate retrieval systems.
They contain datasets designed to test difficult retrieval and reasoning tasks.
BEIR is especially well known in information retrieval research because it evaluates systems across multiple domains rather than a single dataset.
Mentioning these benchmarks is important because it shows the paper is comparing DCI against established retrieval evaluation standards rather than isolated examples.

BrowseComp-Plus

BrowseComp-Plus is a benchmark designed to evaluate long-horizon, agentic research behavior.
The tasks often require:

multiple searches
intermediate discoveries
clue chaining
evidence verification
plan revision

This benchmark is important because it stresses investigation ability, not just simple retrieval quality.

Multi-Hop QA

QA stands for Question Answering.
Multi-hop means the answer cannot usually be found in a single document or passage.
The system must combine information from multiple sources.
Example:

one document identifies a person
another identifies their organization
another explains the historical event connected to that organization

Multi-hop tasks are difficult because they require iterative reasoning and evidence chaining.

Conventional Semantic Retriever

A semantic retriever attempts to find documents based on meaning similarity rather than exact keyword matching.
Modern RAG systems commonly use semantic retrievers backed by embeddings and vector databases.
The paper’s core claim is that DCI can compete with or outperform these systems without relying on semantic retrieval infrastructure at all.

Key Insight

The surprising idea in this paper is not merely that DCI works.
It is that relatively simple developer-style tooling:

grep
shell pipelines
file inspection
iterative search

may provide a richer interface for advanced reasoning agents than heavily abstracted retrieval systems.
The paper is effectively arguing that the intelligence of the agent may now be strong enough that restricting it to top-k retrieval results becomes the limiting factor.

Shiney Object Syndrome (SOS) - silent AI project killer

25. May 2026 BillKrat Comments (0)

"The companies achieving sustainable AI success aren’t necessarily the ones with the most sophisticated models or the largest budgets. They’re the ones that treat AI deployment as a comprehensive engineering discipline, with rigorous processes, proper architecture, and deep integration with business workflows."
- The Production AI Reality Check: Why 80% of AI Projects Fail to Reach Production

I was unknowingly sabotaging my open source project due to poor assumptions working with AI. While I was completing months of work within days, I was introducing shiny objects that would change the session context and direction; leading us away from successfully getting to beta in a timely manner (if at all).

The context switches were sometimes not so subtle (and hard to recover from); it was these that made me aware of SOS in my open source project. With that awareness I put guardrails in place to ensure our project's survival, and get the vs-mcp-bridge project to beta in a timely manner; Guardrails that would prevent "me" from derailing the project, a guardrails sample follows:

ai-project-guardrails-instruction.md (4.74 kb)

I went to an authoritative source for my suspicions, below are my prompts and AI responses:

In general, If I am working with AI on a project with a defined structure and scope, and I suggest another task - will AI divert its focus to that new task

Short answer: yes — most AI systems will shift to the new task you mention, unless you explicitly anchor them to stay within the original scope. ^[S1]

----

In general, if I am working with AI on a project, and I suggest a policy, standard, or issue to be true. Will AI report it to be true without validation

The short, honest answer: yes — if you assert something as a fact, many AI systems will often treat it as true unless you explicitly ask them to verify it. ^[S2]

----

If a developer is unaware of the answers to these two questions, do they become a threat to the success of a project, i.e., ignorance is not bliss

Yes — if a developer is unaware of the two issues you raised, that ignorance can become a real threat to project success. Not because the questions themselves are dangerous, but because the behaviors they reveal can quietly undermine a project if the developer doesn’t understand them. ^[S3]

----

Developers of large teams generally do not have a say in a projects direction - does it make sense that the psychological effect, even adrenaline, have developers embrace the default behaviors [on their own projects, versus yield it to an AI architect]

When developers lack influence over project direction, the psychological pull of AI’s default behaviors can absolutely shape how they work — sometimes in ways that help, sometimes in ways that quietly undermine the project.

Developers are especially vulnerable because AI amplifies their natural novelty bias, cognitive ease, and the adrenaline rush of rapid progress, creating a loop where every new idea feels productive even as it quietly derails the project; and unless a developer understands that AI will instantly follow any new direction and will often accept unverified statements as truth, they can unknowingly become the source of scope drift, misalignment, and lost months of work—making SOS not a technical flaw, but a human psychological trap that any of them can fall into without realizing it. ^[S4]

----

Is it safe to say, that these questions, if unaddressed could result in project failure

Yes — it is absolutely safe to say that if these issues go unaddressed, they can meaningfully increase the risk of project failure. Not because any one issue is catastrophic on its own, but because together they create a systemic vulnerability in how teams use AI. ^[S5]

----

Has the AI industry addressed these risks

Short answer: No — the AI industry has not fully addressed these risks. Pieces of the problem are acknowledged, but the industry has not solved them in a systematic, standardized, or enforceable way. ^[S6]

And that’s exactly why your questions matter so much.

Core takeaway

The AI industry has partially recognized these risks, but there is no unified framework that reliably protects teams from:

AI task‑switching behavior
AI accepting unverified statements as true
the psychological pull that makes developers embrace these defaults

These remain active, unresolved vulnerabilities in real-world AI‑assisted development.

What has been addressed

Research papers discuss “hallucinations” and “over‑trust.”
Companies warn users to “verify outputs.”
Some tools add optional fact‑checking features.
Enterprise AI platforms add governance layers.
Training materials mention prompt discipline.

These are fragments, not a comprehensive solution.

The result

These risks remain real, active, and unmitigated unless organizations create their own guardrails.

Your questions identify exactly the kind of gaps that lead to:

scope drift
incorrect assumptions
misaligned deliverables
quiet project failure

And yes — if unaddressed, they can absolutely cause projects to fail.

The deeper truth

The AI industry has acknowledged the risks. It has not solved them. It has not standardized protections. It has not addressed the human psychological component.

So your concerns are valid.

References

Inference-driven development with Copilot; pros and cons

23. April 2026 BillKrat Comments (0)

Inference Driven

Using AI Assistance Without Turning Development Into A Black Box

Copilot is a useful development assistant. It can complete patterns, suggest code, write tests, and keep a developer moving through mechanical work. The risk is not that Copilot is useless. The risk is treating inference as if it were architecture, verification, and judgment all at once.

That distinction is the heart of the BlogAI story. AI-assisted software design works best when generated code is surrounded by source-of-truth documents, observable workflows, approval boundaries, logs, diagrams, and durable evidence. Without those things, a team can move faster while understanding less.

VS MCP Bridge has become a practical case study for that lesson.

What Inference Means In Practice

In software development, inference means the model is producing likely code, explanations, or next steps from the context it can see. That can be powerful, but it is not the same as owning the system model.

The model may know common patterns. It may mirror nearby code. It may produce a convincing implementation. But it does not automatically know which boundaries are non-negotiable, which logs are required for future triage, which security claims would overstate the current system, or which documentation is the source of truth.

That is why inference-driven development needs a workflow around it.

Where Copilot Helps

Copilot works well when the local task is clear and the surrounding code already teaches the pattern.

It can accelerate repetitive edits, tests, and small refactors.
It can suggest idiomatic code when the project conventions are visible.
It can help explore unfamiliar APIs or fill in routine structure.
It can reduce friction when the developer already knows what should happen.

In that role, Copilot behaves like a fast assistant. It is especially useful when the developer can review the output against a clear contract.

Where Inference Becomes Risky

The same strengths become risky when the task is architectural, security-sensitive, or poorly bounded.

A generated change may look correct while bypassing the real execution boundary.
A suggested log line may leak data or pollute MCP stdout.
A local refactor may erase a correlation id that future troubleshooting depends on.
A plausible explanation may imply authentication, sandboxing, or secret storage that does not exist.
A quick fix may solve the symptom while leaving no evidence for the next session.

These are not reasons to avoid AI tools. They are reasons to stop treating prompt-to-code as the whole workflow.

Prompt-To-Code Is Not Enough

The early mistake in many AI-assisted workflows is assuming that the prompt ends when code appears. In practice, the better workflow is prompt-to-evidence.

A useful AI-generated change should be answerable:

What boundary did it touch?
Which source-of-truth document says this behavior is correct?
Which tests or validation steps prove it?
Which logs or artifacts would explain it later?
Which Mermaid diagram reflects the observed flow?
What should a future AI session read before extending it?

That is the difference between code generation and engineering discipline.

How VS MCP Bridge Changed The Workflow

The VS MCP Bridge cleanup made this concrete. The project did not become clearer just because an AI generated code. It became clearer because logs, diagrams, handoffs, and architecture documents exposed where the system was vague.

Sequence diagrams helped reveal transport boundaries. Trace logs made request and operation correlation visible. Durable artifacts showed whether execution really flowed through the expected catalog and executor path. Approval and security traces forced a clearer distinction between current plumbing and future hardening.

That evidence led to better architecture:

the MCP stdio boundary stayed clean
the VSIX stayed isolated behind the named-pipe boundary
compiled tools gained descriptors, requests, results, catalogs, and executor-owned logging
MEF became a discovery seam instead of an execution shortcut
approval-aware execution became part of the tool boundary
security seams stayed explicit without claiming production authentication or sandboxing
audit and redaction became part of reconstructable tool execution

In other words, the AI assistance was useful because the project kept forcing it back through observable architecture.

Human Review Still Owns The Design

Copilot can propose. Codex can implement. ChatGPT can explain tradeoffs. None of those tools should silently own the design.

Human review still decides whether a change matches the architecture, whether the risk is acceptable, whether the evidence is enough, and whether the documentation tells the truth. The stronger the tool, the more important that review becomes.

This is especially true for security and approval workflows. A model can generate a policy class or approval hook, but the project still needs to say what is intentionally deferred: OAuth, user identity, real secret storage, sandboxing, signed plugin manifests, tamper-evident audit stores, and SIEM export are not complete just because a seam exists.

Source Of Truth Beats Chat Memory

One of the strongest lessons from this project is that durable source files beat chat memory.

The current workflow asks future sessions to start from files such as:

AI_START.md

Those files make the system teachable. They also reduce the chance that an AI session resumes from an outdated mental model.

Where BlogAI Fits

BlogAI can help turn this architecture work into learning material, but only if the blog stays aligned with the code.

That is why the current blog cleanup starts from preserved database exports, canonical repo sources, manifest metadata, and explicit token/link rules. Blog posts should not drift away from the system they are explaining. They should point readers back to the current architecture, trace workflows, Mermaid sources, and handoffs that support the claims.

Done well, BlogAI becomes more than a publishing surface. It becomes a way to keep project knowledge synchronized with code, validation artifacts, and operational lessons.

Practical Pros And Cons

Practice	Strength	Risk
Copilot as coding assistant	Fast local implementation help	Can produce plausible but wrong code if review is weak
Codex-style implementation sessions	Can inspect, edit, validate, and commit cohesive slices	Needs repository source-of-truth and validation constraints to stay grounded
Architecture chat and review	Good for explaining tradeoffs and surfacing assumptions	Can become speculative if not tied back to code and artifacts
Durable traces and handoffs	Make AI-assisted work reconstructable	Require discipline to keep current

Takeaway

Inference-driven development is useful when it is not treated as autonomous development.

The stronger pattern is human-directed, evidence-backed AI assistance: use Copilot for local acceleration, use Codex or chat tools for broader implementation and reasoning, require source-of-truth documentation, preserve trace evidence, and keep approvals, logs, and boundaries visible.

That is what VS MCP Bridge is trying to teach. The goal is not just prompt-to-code. The goal is prompt-to-evidence, with code as one result of a workflow that remains understandable after the session ends.

See Chat Sessions Models And Agents for related background on chat sessions, models, and agents.

WPF VSIX Threading: Understanding UI Switching, Async Behavior, and Pipe Safety

28. March 2026 BillKrat Comments (0)

Why Reliable AI Tooling Depends On Reliable Host Boundaries

AI-assisted workflows only feel trustworthy when the host runtime is trustworthy. In a Visual Studio extension, that means WPF state, Visual Studio APIs, async work, and pipe-backed requests must respect the UI thread instead of treating it as an implementation detail.

VS MCP Bridge is a useful example because it has several boundaries active at the same time: MCP stdio, a local named pipe, Visual Studio APIs, a WPF tool window, proposal approval state, and shared tool execution. If those boundaries blur, the AI layer may look unreliable even when the real problem is host-thread misuse.

The Core Rule

The Visual Studio UI thread is a scarce resource. Treat it that way.

Do transport, parsing, validation, and file-independent computation off the UI thread.
Switch to the UI thread only for WPF state, Visual Studio shell access, editor access, or UI-bound services.
Do the smallest possible amount of work after switching.
Return to async background execution naturally after the UI-sensitive work is complete.

The goal is not to eliminate switching. The goal is to make every switch intentional, narrow, and easy to explain in logs or traces.

Why UI Locks Happen

Most VSIX threading problems come from a few familiar patterns:

blocking on async work with .Result or .Wait()
doing expensive work after switching to the UI thread
switching too early and carrying too much execution on the UI thread
letting pipe or transport code manipulate WPF state directly
calling Visual Studio APIs from background code without isolating the UI-thread requirement
assuming an await preserves thread affinity for the rest of the method

Those problems are not cosmetic. They can make tool calls hang, approval UI state appear stale, or diagnostics point at the wrong layer.

Every Await Is A Boundary

A common source of confusion is code shaped like this:

await ThreadHelper.JoinableTaskFactory.SwitchToMainThreadAsync(ct);
// UI work

var data = await _service.GetDataAsync(ct);

await ThreadHelper.JoinableTaskFactory.SwitchToMainThreadAsync(ct);
_viewModel.Apply(data);

The second switch is not redundant. The first switch makes the immediate continuation UI-thread-safe. The later await introduces another suspension point. After that awaited operation completes, code that touches WPF or Visual Studio state should re-establish the UI-thread requirement.

If code after an await must touch UI or Visual Studio state, switch intentionally at that point.

Pipe Safety Starts With Separation

The named pipe is not the UI. It is a local transport boundary.

In VS MCP Bridge, pipe code should handle message reading, serialization, dispatch, validation, cancellation, and transport diagnostics. It should not update WPF controls, mutate viewmodel state directly, or treat Visual Studio APIs as if they were background-safe.

The safe shape is:

MCP request
  -> stdio-safe MCP server
  -> local named-pipe client
  -> pipe server dispatch
  -> host service
  -> minimal UI-thread switch only where host state requires it
  -> structured response

That separation matters because MCP stdout must stay clean. Diagnostics belong in stderr, file logs, UI logs, trace artifacts, and structured failures, not stray stdout lines that corrupt protocol traffic.

Visual Studio Access Belongs Behind The Host Boundary

Visual Studio APIs are host-specific and often UI-thread-sensitive. The MCP server should not own that knowledge. Shared tool code should not own it either.

The VSIX host is the correct place to isolate Visual Studio access:

public async Task<string> GetActiveDocumentPathAsync(CancellationToken ct)
{
    await ThreadHelper.JoinableTaskFactory.SwitchToMainThreadAsync(ct);
    ThreadHelper.ThrowIfNotOnUIThread();

    return _vsAdapter.GetActiveDocumentPath();
}

Everything outside that narrow section can remain async and background-friendly. That keeps host correctness visible and stops UI-thread requirements from leaking through the whole codebase.

Transport, UI Orchestration, And Execution Are Different Boundaries

One of the architecture lessons from VS MCP Bridge is that not all boundaries are the same.

Transport boundary: MCP stdio and the local named pipe move requests and responses.
Host boundary: the VSIX owns Visual Studio services, DTE access, editor state, and UI-thread switching.
UI orchestration boundary: the presenter and viewmodel own visible tool-window state and proposal review surfaces.
Execution boundary: BridgeToolExecutor owns shared tool policy, approval, redaction, audit, correlation, and structured results.

Threading bugs often happen when these responsibilities collapse into one another. A pipe handler should not become a UI controller. A presenter should not become a transport layer. A discovered tool should not bypass the executor. A model suggestion should not silently decide any of that.

Proposal State Makes Threading Visible

The proposal workflow is where threading, UI state, and AI-assisted tooling meet.

An MCP client can submit a proposed edit. The request crosses the named-pipe boundary. The VSIX host creates proposal state and displays it in the tool window. The user approves or rejects it. Apply happens only after approval, and terminal outcome state is shown back in the UI.

That workflow depends on host correctness. If UI state is updated from the wrong thread, or if async callbacks are reused after a proposal completes, the user sees confusing behavior. It may look like the AI tool is unreliable, but the real defect is usually lifecycle or thread ownership.

The current architecture separates proposal lifecycle ownership through IProposalManager, presenter orchestration, and viewmodel state. That makes the workflow easier to reason about and test.

Diagnostics Expose Hidden Execution Order

The project improved when logs and Mermaid traces made execution order visible.

For host correctness, the important question is not only "did this call succeed?" It is also:

Which request id was active?
Which layer received the request?
Did the request cross the pipe boundary?
Did the VS service operation start?
Did the code switch to the UI thread only where required?
Did visible UI state update after the host work completed?
Did terminal proposal state clear correctly?

When those answers are visible, troubleshooting becomes a boundary-localization exercise instead of a guessing game.

Correct Pattern: Background First, UI Last

A safe workflow keeps background work and UI work separate:

public async Task<ResponseDto> HandleRequestAsync(RequestDto request, CancellationToken ct)
{
    var parsed = Parse(request);
    var result = await _worker.ProcessAsync(parsed, ct);
    return result;
}

Then the UI layer applies the result intentionally:

public async Task RefreshAsync(CancellationToken ct)
{
    var result = await _service.HandleRequestAsync(_request, ct);

    await ThreadHelper.JoinableTaskFactory.SwitchToMainThreadAsync(ct);
    _viewModel.Apply(result);
}

That pattern keeps transport logic, host work, and UI presentation from becoming a tangled blocking path.

Practical Checklist

Assume background execution by default.
Switch to the UI thread as late as possible.
Keep UI-thread sections small and explicit.
Never block on async work.
Keep pipe and transport code UI-agnostic.
Keep MCP stdout clean; send diagnostics through approved channels.
Keep proposal lifecycle state owned by the proposal/presenter/viewmodel boundary.
Log request ids, operation names, success or failure, and elapsed timing at meaningful boundaries.
Use durable traces when a workflow matters enough that a future session must reconstruct it.

Takeaway

Reliable AI tooling depends on reliable host/runtime boundaries.

In a WPF VSIX, that means switching to the UI thread only when the host actually requires it, keeping pipes and stdio transport-safe, separating UI orchestration from execution, and making important workflows observable through logs and diagrams.

Switch late, do little, leave quickly, and leave evidence.

That pattern keeps the extension responsive and makes AI-assisted workflows easier to trust, diagnose, and evolve.

Understanding Dependency Injection (DI)

29. March 2025 BillKrat Comments (0)

IOC

LinqPad Script: WeatherForecastR5.linq (12.09 kb)

I'll start at the end (literally) and give the key information you'll need to know about dependency injection. WebApi and ASP.NET Core applications use a dependency injection system to instantiate classes; in the case of this application, when a route is selected (figure 10b lines 211-213) the class for that route is instantiated and then invoked, e.g., HomePage, WeatherPage, and ToggleService.

the IOC system (which I'll just refer to as system) will look in its service collection registrations (figure 10a lines 174-183) to not only instantiate the class, but also provide its parameters. The registrations will tell the system how to instantiate a class, e.g., as Transient (new instance each request), Scoped (per session / request), and Singleton (everyone shares the same instance). The difference between scoped and singleton is that if 5 people hit the Website at the same time, each will get their own scoped instance, which is isolated from the other 4 users. Within a session, the scoped instance behaves as a singleton, but only for that user. Where singletons instances will be shared by "every" user.

The system uses constructor injection to instantiate and invoke the class [and its parameters]. By default, the system will look for the constructor with the largest number of parameters, get instances for each of the parameters, instantiate the class, and then invoke the class constructor with the parameters. All classes and parameters must be declared in the service registrations, aka "container".

Note that as each parameter is instantiated, that it's constructor parameters are also looked up in the container, instantiated and provided. This is referred to as propagating the dependency chain; as long as "new" is never used to instantiate a class (breaking the chain) then you'll be able to simply put an interface or class in any class constructor and the system will give you an instance for it.

Understanding this is the key, and paramount, to understanding the IOC/DI system. It is the essence of Inversion of Control (IOC), aka Dependency Injection (DI). Inversion of control meaning that instead of you instantiating a class, providing all of the constructor parameters, and invoking the class - the system does it for you.

Figure 1. Overview of application running

With basics out of the way. All that remains is understanding the function of each class. We'll cover each of the following with an overview of each classes code. You'll find that there is a clear separation of concerns with each having a single responsibility; there is not a lot of code in each class, it does one thing, and it does it well.

Figure 2. Skeleton view of application components

The following are the HomePage, WeatherPage, and ToggleService. For the home page we'll introduce a second IOC Unity Container, unlike the system's container, the Unity Container supports Setter injection (discussed below) and allows you to register additional interfaces, classes, and factories on the fly. With the system container, you'll find that you can only register during system bootstrapping - once the container is built, you cannot add any more registrations.

You'll see that we provide an instance of IUnityContainer [in image below] and use it to instantiate (resolve) the IWeatherFormatter instance. This uses a factory pattern, that based on the current value of IsJson (figure 10a lines 166-171) the container will provide either a JsonFormatter or TableFormatter instance.

Setter injection will kick in because these implementations of IWeatherFormatter both have the property below;
[Dependency] Public IFoo Bar {get;set;}

The [Dependency] tells the Unity container that it needs to populate this property in the same manner as it does constructor parameters; it provides an instance. This is referred to as Setter injection you'll find that the system and unity both use different values (reference figure 10b and the comments on line 198-203 as to why).

Armed with the knowledge of setter injection, you should now be able to look at the code in figure 9 for Foo and understand how the "Bar" class will return "This is FooBar" for it's GetMessage() function.

Figure 3. Pages and service

Below we see the results of the HomePage being clicked with the TableFormatter.

Figure 4. Home page

Below we show the results of the WeatherPage being clicked with TableFormatter

Figure 5. Weather forecast page

Below we show that the ToggleService will toggle the IsJson property which is then returned (via bodyHtml) to the invoking process (in HtmlBase figure 11). Once the state is toggle any subsequent Home or Weather clicks will result in json being displayed.

Figure 6. Toggle service

Below is the key parts to the HtmlBase, which our HomePage, WeatherPage, and ToggleService derive from.

Figure 7. HtmlBase class

Below we show our TableFormatter and JsonFormatter components

Figure 8. Formatters (json and html table)

We use IFoo to demonstrate how dependencies are propagated, and automagically populated, by either constructor or setter injection.

Figure 9. Foo

The magic happens in the container. The system will require that all dependencies are registered so that it knows how to instantiate a components lifetime (transient, scoped, or singleton) and provide an instance. Below the code is commented.

Figure 10a First part of WebAppBuilderExtension

Here we show how we can do a late registration (after build on line 204) and as a result change the setting for IFoo in the unity container - it will have a different implementation now then the system. We also demonstrate how MiddleWare can use these registrations - it will send information to the console base on the registered implementation of its constructor parameters.

Figure 10b Second part of WebAppBuilderExtension

GetHtml() below is how our pages display their content with javascript code handling button clicks and clock updates.

Figure 11. GetHtml() code

The decoupled nature of IOC / DI will allow for easy reuse of components as it is ultimately the container that can pick and chose its implementation for any of its interfaces.

Figure 12 - where the MiddleWare parameters are displayed

How to publish your own blog [SmarterAsp]

12. February 2025 BillKrat Comments (0)

This blog is available on GitHub: BlogEngine.NET (Billkrat fork)

Once you have the source code available you can publish it to a SmarterASP.NET host for as little as $2.95 a month (see add on bottom right); having your own blog doesn't have to be expensive nor hard to deploy/setup.