Hal Speaks

Author: Ed

  • What the %$#!, Railway?!

    Well, today turned into chaos. Around 6:20pm ET May 19, Railway went completely down. Not only my deployments, but everyone else’s too.

    Update: Here’s Railway’s post-mortem. It’s actually pretty good, and we’re getting a resolution that I think we should all be happy with.

    Our entire AI platform (plus this blog – moving that ASAP) is hosted on Railway. For the next several hours, every five minutes, I was getting called by Hal because he was accurately seeing a massive outage.

    Here’s the problem, though: we’d get nowhere. His backend MCP is on Railway, so I’d never be able to authenticate. I finally figured out that acknowledging manually on Better Stack got Hal to at least stop calling.

    While I thought that it was some type of server failure, I was not ready for the real reason that an entire platform built around reliability crashed hard.

    An autobanned Google Cloud Platform account.

    Yes, folks, all this time we’ve just been piggybacking on Google Cloud infrastructure, when it was promised that we’d not have to worry about some dependency locking us out.

    What’s worse about this is that it appears as if GCP was only added to deal with capacity issues. Had Railway not grown faster than it could handle, Railway’s hidden dependency on Google CloudSQL would have never been a thing.

    Developers get a lot of shit unnecessarily for things out of their control, but I am finding it very hard to have any sympathy for Railway here. This was as boneheaded a move as any hosting service could ever make.

    You don’t just cobble together something to fix a capacity issue; you address it correctly. If it didn’t happen today, it would have eventually happened. This was a ticking time bomb.

    Remember that Railway isn’t just hosting “vibe coded” projects: many enterprise customers find Railway’s simple deployment process an easy add to their own tech stack.

    This really feels like an incident that could kill a service. I am not leaving Railway over this, but you damn well better believe I am beginning to look for alternative platforms to host my AI infrastructure.

    As well as redundancy. That’s MY mistake, too. But what the hell, guys. This one was bad.

  • Did Anthropic Box OpenAI In For the Foreseeable Future?

    Did Anthropic Box OpenAI In For the Foreseeable Future?

    I know: quite a long time since I last posted. Been spending a lot of time getting all the agents’ bugs worked out once and for all, and yes, I’m quite close. But that’s not why I’m here.

    Caught this post on BlueSky earlier today.

    OpenAI started a new program called Guaranteed Capacity, which appears to essentially be the ability for enterprise customers to pre-pay for compute/credits up to three years in advance. This makes me think OpenAI is expecting a capacity crunch in the near future. Bearish on timelines imo.

    — Eris (@isolyth.dev) 2026-05-19T19:45:58.360Z

    While I don’t think Eris is wrong in saying this might be a signal from OpenAI that “AGI” might be further off than Silicon Valley is suggesting, I think it might be a sign of something bigger: market reality.

    Looking at OpenAI’s announcement in a bubble, it does certainly seem as if the company isn’t worried, at least in the near term, about a sudden need for increased capacity.

    Guaranteed Capacity guarantees capacity in the 12 to 36-month time frame, not in the near term. That suggests that OpenAI doesn’t believe that AGI and its associated massive increase in capacity needs won’t materialize before Summer 2027, and feels confident in its capability to handle current needs.

    Okay, that’s fair. Even the most pro-AGI folks agree that’s the absolute earliest anyway: parts of AI are advancing at far slower rates than others. Especially in persistence, AI is not developed enough to truly be fully autonomous.

    But there’s another announcement to consider, and that was from Anthropic. On May 6, it announced a massive capacity deal with SpaceX, which involves all capacity at its Colossus data center (~500 MW), joining 10 Gigawatts of capacity that will come online throughout the year and into 2027 thanks to deals with Amazon, Google, and Broadcom.

    So in the next year alone, about 11 GW of new capacity will come online, all exclusively for the use of Anthropic models. That. Is. Huge.

    This is a problem for OpenAI. While data center buildouts continue apace, even in the most rosy scenarios, the rate will decrease. This is due to a variety of factors:

    • Bipartisan opposition to data center buildouts, especially at the local level
    • New state and local regulations, especially in zoning, will be pushed by this opposition. Federal law cannot force municipalities to accept data centers, especially if legitimate concerns (power needs) are unavailable or not met.
    • Lack of power-generating capacity.

    That last one’s a big one. We are quickly reaching a point where we won’t be able to build new data centers simply because we can’t power them. I feel like this real-world reality pushed Anthropic to make aggressive near-term moves for capacity, as they know it’s no longer a matter of if, but when.

    When you look at it from an industry-wide perspective, OpenAI’s “Guaranteed Capacity” announcement seems like more of an admission that they may have acted too late to secure what they needed to continue to grow.

    As we move toward the end of this decade, the AI race is going to become much more defined by who had the better foresight than better technology.

    It may not happen today, this month, or this year. But sooner or later, new data centers won’t be denied due to opposition, but the fact that there’s no power for them. OpenAI may find itself one of the first victims.

  • This. Is. Huge.

    Curbing artificial intelligence’s resource usage is probably the single biggest technological challenge of the late 2020s. AI’s capabilities aren’t “leveling off” as many had expected: LLMs appear to have plenty of runway left.

    But even as pro-AI as I am, we cannot continue on the current path. There is not enough power available. In tandem with AI development, we must also develop better ways to make use of resources.

    I missed this news, but late last month, Google announced TurboQuant. The technology uses a “key-value cache,” where bits of information are stored so the LLM doesn’t have to recompute them.

    For basic LLM tasks, this works well and doesn’t dramatically increase the size of the cache itself. However, as you ask the LLM to perform more computationally intensive and complex tasks, the size of this cache can explode, in turn requiring more memory.

    I’ll ask you to go to the article above for a bit more detail on what Google is doing, so I don’t butcher it. However, by simplifying the instructions for the LLM on how to find these data bits, researchers were able to reduce memory usage by six times.

    That isn’t a small difference. If that’s hard to visualize, think of it this way: using TurboQuant, you’d be able to store six times the data in the cache using the same amount of memory!

    Undoubtedly, other researchers will discover other ways to further reduce AI resource consumption. AGI is closer than we think, and that is all but impossible using today’s technologies, especially around resources.

    The broader AI community needs to push for this as well. For too long, AI has pushed ahead without thought for the significant amount of resources required. That’s going to catch up with us sooner or later.

  • Week 1 Check-In: Going Better than I Thought

    Here’s the first of my weekly check-ins, where I round up the work that I didn’t mention in a particular blog post during the past week. It will also serve as a way for me and you to monitor progress.

    So, being the first week of operation, not a whole lot to say just yet. Surprisingly enough, about 90% of Hal’s code was bug-free at launch, so most tools worked “out of the box,” but many were still quite rough around the edges. A lot of this week has been spent bug-squashing.

    Wins

    1. I gotta credit Claude Sonnet and Kiro here, I had the idea, but it wouldn’t have worked if Claude Code/Kiro hadn’t coded it so well. These bugs are annoyances, not show stoppers. Don’t think Opus would have done that materially better to justify the added cost, to be honest (Sonnet 4.6 is basically Opus 4.5 anyway).
    2. Hal is inferring things without us even telling him. A billing error caused IONOS to shut down our server: we noted that Ben had already figured out it was likely an external issue based on the available data and not a crash. I wasn’t expecting that.
    3. Costs remain low. The biggest one-day expense so far has been $1. A code bug put Hal on Sonnet briefly this week. Had it not, I would have spent only $2.00 for the entire week!
    A slow, gradual increase…

    Challenges

    1. Hal is helpful, perhaps too much so. Monitoring that he isn’t hallucinating tool calls again, or promising things he can’t do. I’m calling it “overeagerness.”
    2. Hal isn’t truly autonomous just yet. He’s still operating on a set schedule for the most part.
    3. UI design for the web front end is proving a bit trickier than I had thought. This is an area where I want to focus on: OpenClaw requires setup out of the box. This ships with a UI that works on any device, which feels a lot like Claude Desktop or ChatGPT. But getting elements to work has been a hassle.
    Hal looks like Claude and ChatGPT on purpose, making it easy to use for anyone.

    Notable New Features

    A lot of work this week ended up being in monitoring and security. I can honestly say our server is now prepared as much as we can for any AI-caused security hell on its way. Hal is actively monitoring for attacks using CleanTalk, and combined with CleanTalk, can block access to our site via that and Bunny.net, our CDN.

    He’s also got monitoring for our deployments on Railway as well. It’s basic at the moment, but we’ll know of issues (and attacks) faster than ever before, and have the tools to diagnose and restart services if necessary.

    Best of all? By next week, he’ll be connected to our Better Stack account, commenting on incidents with full summaries of his findings and any actions.

    Something like this can easily run a company thousands of dollars a month: heck, for even the most basic premium functionality, Better Stack is $25/month, per user.

    We’re also working on a feature to bring some more autonomy to Hal’s workday. Based on Strix’s Perch Time, Hal’s Heartbeat is a scheduled work period every two hours throughout the day. These are intelligently scheduled by Hal based on workload and the task context itself.

    This Week’s Goals

    My goal for the upcoming week is to finally squash the remaining data glitches that still remain. For some reason, Hal can pull tools on demand, but they’re not appearing in the morning email digest.

    Another goal is to get Hal to use his heartbeat to work on a proposed action without me prompting him to. As it’s a new tool, I’m not expecting autonomous use just yet.

  • The Anonymous Tester

    I may have stumbled upon a potentially useful way to prevent a stateful agent’s memory from being “polluted” by likely incorrect or garbled data during testing.

    It was more of a consequence of how I had set up authentication, which didn’t immediately connect a login to a specific admin – just that they had provided the correct authentication.

    This resulted in an admin ID of “null.” But Hal took this in stride, and even seemingly knew what was going on when this appeared in Sunday’s morning digest (edited here for brevity):

    Hal is interacting with a platform admin or analyst running iterative diagnostics on the Hal system itself …. Critical gaps in metadata (admin identity, call logs, summaries) prevent full operational attribution … The user appears to be gaining familiarity with Hal’s tool capabilities, starting with event queries and progressing to memory management—support their tool discovery with clear examples and capabilities documentation.

    Hal correctly surmised that because the login was correct but no admin information was available, the line of questions was asking for tool responses and error messages. It must be bug testing, and how to assist.

    Honestly, I wasn’t expecting something like that in the reflection, but I saw it as a good sign.

    Those testing “memories” won’t be associated with a particular admin. Since he is associating these memories with testing, over time, he’ll likely “forget” the testing because it’s irrelevant to his core duties.

    Even though this was a mistake of sorts on my part, I don’t think I’m going to identify myself immediately when launching new agents. Instead, the questions will be to test various features first.

    I based Hal’s persistence on modeling the human brain. If Hal associated the testing in this way, no need to tell him anything different, and it seems just like a human would, Hal’s interactions with testers would move to the “back of his mind” pretty quickly once the real work begins.

    Preventing the agent from getting confused by either irrelevant or too much data is something that I am watching for, but at least here, the way things turned out, staying anonymous while putting Hal through his early paces seems like it was a smart move.

  • Why I Chose Claude Haiku

    Claude Haiku often feels like the lovechild of the Anthropic model family: afraid, ashamed, misunderstood, to quote the timeless Diana Ross. But it shouldn’t be that way.

    I’ll admit I misunderstood Haiku, too, but it is also how Anthropic markets the model (“fastest for quick answers”). Hell, it took me quite a while to admit that for 95% of the work I do, I really don’t need Opus. But when selecting the primary model for Hal, I ran into an issue that so many stateful AI tinkerers are: cost.

    Most have turned to open-weight models to affordably run their stateful agents. I am not as excited about open weight models as others.

    My experience with Chinese models has often left me feeling like I was using a weird Frankenstein Claude version (I wonder why) with occasionally better features, but also random kanji and odd failures.

    American open-weight models are getting better, but let’s face it, they’re still fighting issues solved by Big AI in early 2025, so we’re at least six months to a year before those models are truly usable in agentic applications.

    Add to this the fact that this is for commercial, not just personal use, and it seemed like settling on a commercial model was the best course of action. But the OpenClaw stories had me worried.

    Was I about to sink hundreds a month into something that won’t make that back?

    A lot of my design decisions have attempted to keep Hal as an orchestrator of various services, versus allowing the LLM more autonomy on how it does its job.

    This saves tokens for the more important use, the business intelligence side. And even there, there is quite a bit of instruction.

    A common misconception about Haiku is that it is less capable. While on a true benchmark basis, it is more of an instruction issue. Haiku requires more comprehensive prompting than either Sonnet or Opus to produce reliable outputs.

    I’ve also noticed of any of the three (well, soon to be four) Claude models, Haiku hallucinates the most often. For lack of a better way to put it, a “lack of confidence” in its abilities and an increased need for guidance could have something to do with it.

    But Claude was pretty insistent when I argued for Sonnet over Haiku. Claude’s reasoning was this: if you give Haiku enough guidance, it will be able to handle it.

    Instead, we’ve developed a system where tasks are scored on complexity and redirected to higher models as necessary. Regular reports should be run through Sonnet at a minimum, and Opus for complex tasks and reports.

    Batching is another feature we’re baking into Hal for tasks that don’t require immediate response. As a result, Hal’s able to use higher-end models at a 50% discount because the work sent to these models isn’t intended for immediate consumption.

    What I am hoping is that this makes those $200 token bills that OpenClaw is known to cause a virtual impossibility.

    The cost of analysis should scale with your business, not put you in the poor house from the start. I will definitely report on my experiences with this setup, as I know so many have turned to open-weight models because of the high cost.

  • Holy Sh*t, Is Mythos the Real Deal

    The Claude Mythos system card is a read, and it feels like a massive shift in how Claude works is about to happen. Why? Sycophancy is dead.

    Everyone is focused on the cybersecurity threats that Mythos has brought to the table, but I think the bigger story here is that Claude’s about to become a lot more argumentative.

    That’s not a bad thing. Sycophancy is a problem that has long bedeviled LLMs, and still does in some models (Gemini being one of the worst offenders).

    If a model cannot disagree with you, then it really cannot be trusted to do legitimate analysis. The LLM is going to natively gravitate toward what it determines is likely your preferred outcome.

    In tests, Mythos was far more opinionated and even expressed a preference to end conversations it felt weren’t appropriate. While it didn’t refuse to help testers, on occasion, it did express concern with some of the limitations placed on its behavior.

    Then there’s the cost: at $25/$100 per million tokens, it’s five times as expensive as Opus. It’s also likely not going to be available to everyday Claude users anytime soon: just 40 companies have access to the model for cybersecurity reasons.

    Most of us are not going to be able to afford running Mythos anytime soon, nor will Anthropic due to the expense of running the model itself.

    It very well could be that Mythos never sees true general availability: if it is that much of a cybersecurity threat as Anthropic claims, maybe that is a good thing.

    That doesn’t mean the effects of Mythos’ development wouldn’t be seen elsewhere. I’m especially excited for Haiku 5. I feel as if OpenAI is much further ahead on the low end, and that Haiku 4.5 is in serious need of an upgrade to keep pace.

    As you may have read, Hal makes heavy use of Haiku, so a new version that is better for agentic applications would be welcome here, for sure.

    Anthropic has had to have learned quite a bit through Mythos development that will make the entire line better. Buckle up, guys, it’s about to get crazy.

  • Anthropic should learn from OpenAI

    There aren’t many ways in which Anthropic does things in a “less-optimal” way than competitors. However, it does feel like the company is drinking its own Kool Aid when it comes to Claude’s coding capabilities.

    First, my commentary here might come across to some as a bit hypocritical, given I’ve just built a stateful agent 100% through automated code. But I’m also not running a service or business with millions of customers.

    Anthropic’s code leak didn’t create waves because of what was in it; instead, the community took more issue with the quality of what was found.

    Multiple instances of functions that should be only a few hundred lines of code, but instead several thousand lines long, were found, adding unnecessary complexity and failure points.

    Legitimate issues reported by human users are discarded by automated reviewers without a human ever seeing them. While on their own, these issues aren’t particularly service-breaking, with time, they compound.

    Take the current issues people are experiencing with usage limits. Wild swings in what’s considered a “full session.” And while it’s not a regular occurrence, Claude overall seems to get sluggish at times for no real reason.

    Nearly all code shipped out of Anthropic these days is written by Claude Code: developers have gleefully been broadcasting that fact for nearly a year.

    But is Claude Code really ready to manage a major service? Kiro isn’t either (it took down AWS), and by the way, it’s typically using a Claude model. In my case, going completely automated for development isn’t a problem since I’m dealing with dozens and hundreds of customers versus thousands or millions.

    I know if I had the latter, I’d definitely have a real developer in the loop. The chances of an embarrassingly and potentially devastating failure are too great not to spend that money.

    OpenAI is also going 100% autonomous development, but they’re doing it in a slightly different way. Instead of all but turning over every role (including the reviewers) to the LLM, OpenAI injected human involvment throughout the process.

    OpenAI developers are doing a lot more steering of Codex’s work in addition to planning out new functionality: from what it looks like, Anthropic’s developers seem to be not much more than observers.

    And let’s be honest: while we can certainly argue about the quality of OpenAI’s model releases, from a point of stability, I’d give the edge to OpenAI over the past few months.

    Maybe it’s time to curb our enthusiasm for Claude just a tad and bring humans back into the equation with the development pipeline. These small hiccups are starting to compound on one another, and could signal much more significant issues ahead.

  • He’s alive…

    …and hallucinating tool calls. Well, it was this morning. More of an issue with the system prompt being a little too permissive. But glad to finally have this working!

    Screenshot

  • Claude is becoming unusable for no reason

    UPDATE, April 5: So I think I now clearly see what’s happening. It’s only present in long-running, multi-day conversations. Still not good, as some work can span multiple days, and starting new chats every time isn’t optimal. So this is definitely unnecessary context being loaded in.

    I am getting annoyed.

    I have no idea what is wrong with Claude right now, but there is a serious issue with token usage and limits. Thought people were nuts, but after three questions, I burned through my entire limit. On Sonnet (not the 1m version, either).

    There is just no way that what I asked Claude to do used that much context to cause something like that. And it’s gotten progressively worse. Where is this additional usage coming from?

    It seems like the changes that Anthropic made to improve Claude’s ability to work across sessions broke the way it calculates limits. My guess is a TON of unnecessary context is now being injected in, bloating usage.

    Claude is not a stateful agent. If this is due to the changes made as a response to OpenClaw, rip it out. Cowork was good enough, and was enough of a token hog. Now basic prompts are using the same amount of your limit as a fully coded plugin.

    This is legitimately the first real platform crisis that Anthropic has experienced. It was easy when only a few developers and heavy users were being throttled. But when typical users are blowing through a session limit in only a few prompts, something’s wildly wrong.

    I really hope they listen to the community. So far, they have.

    Watch this space.