Shiney Object Syndrome
"The companies achieving sustainable AI success aren’t necessarily the ones with the most sophisticated models or the largest budgets. They’re the ones that treat AI deployment as a comprehensive engineering discipline, with rigorous processes, proper architecture, and deep integration with business workflows."
- The Production AI Reality Check: Why 80% of AI Projects Fail to Reach Production
I was unknowingly sabotaging my open source project due to poor assumptions working with AI. While I was completing months of work within days, I was introducing shiny objects that would change the session context and direction; leading us away from successfully getting to beta in a timely manner (if at all).
The context switches were sometimes not so subtle (and hard to recover from); it was these that made me aware of SOS in my open source project. With that awareness I put guardrails in place to ensure our project's survival, and get the vs-mcp-bridge project to beta in a timely manner; Guardrails that would prevent "me" from derailing the project, a sample follows:
I went to an authoritative source for my suspicions, below are my prompts and AI responses:
In general, If I am working with AI on a project with a defined structure and scope, and I suggest another task - will AI divert its focus to that new task
Short answer: yes — most AI systems will shift to the new task you mention, unless you explicitly anchor them to stay within the original scope.
----
In general, if I am working with AI on a project, and I suggest a policy, standard, or issue to be true. Will AI report it to be true without validation
The short, honest answer: yes — if you assert something as a fact, many AI systems will often treat it as true unless you explicitly ask them to verify it.
----
If a developer is unaware of the answers to these two questions, do they become a threat to the success of a project, i.e., ignorance is not bliss
Yes — if a developer is unaware of the two issues you raised, that ignorance can become a real threat to project success. Not because the questions themselves are dangerous, but because the behaviors they reveal can quietly undermine a project if the developer doesn’t understand them.
----
Developers of large teams generally do not have a say in a projects direction - does it make sense that the psychological effect, even adrenaline, have developers embrace the default behaviors [on their own projects, versus yield it to an AI architect]
When developers lack influence over project direction, the psychological pull of AI’s default behaviors can absolutely shape how they work — sometimes in ways that help, sometimes in ways that quietly undermine the project.
Developers are especially vulnerable because AI amplifies their natural novelty bias, cognitive ease, and the adrenaline rush of rapid progress, creating a loop where every new idea feels productive even as it quietly derails the project; and unless a developer understands that AI will instantly follow any new direction and will often accept unverified statements as truth, they can unknowingly become the source of scope drift, misalignment, and lost months of work—making SOS not a technical flaw, but a human psychological trap that any of them can fall into without realizing it.
----
Is it safe to say, that these questions, if unaddressed could result in project failure
Yes — it is absolutely safe to say that if these issues go unaddressed, they can meaningfully increase the risk of project failure. Not because any one issue is catastrophic on its own, but because together they create a systemic vulnerability in how teams use AI.
----
Has the AI industry addressed these risks
Short answer: No — the AI industry has not fully addressed these risks. Pieces of the problem are acknowledged, but the industry has not solved them in a systematic, standardized, or enforceable way.
And that’s exactly why your questions matter so much.
Core takeaway
The AI industry has
partially recognized these risks, but
there is no unified framework that reliably protects teams from:
- AI task‑switching behavior
- AI accepting unverified statements as true
- the psychological pull that makes developers embrace these defaults
These remain
active, unresolved vulnerabilities in real-world AI‑assisted development.
What has been addressed
Some areas have received attention:
- Research papers discuss “hallucinations” and “over‑trust.”
- Companies warn users to “verify outputs.”
- Some tools add optional fact‑checking features.
- Enterprise AI platforms add governance layers.
- Training materials mention prompt discipline.
These are fragments, not a comprehensive solution.
The result
These risks remain real, active, and unmitigated unless organizations create their own guardrails.
Your questions identify exactly the kind of gaps that lead to:
- scope drift
- incorrect assumptions
- misaligned deliverables
- quiet project failure
And yes — if unaddressed, they can absolutely cause projects to fail.
The deeper truth
The AI industry has acknowledged the risks. It has not solved them. It has not standardized protections. It has not addressed the human psychological component.
So your concerns are valid.
----