Technical Decisions
32 recent decisions
Stay current on NVIDIA drivers - build the automation, and do not ship RLC-AI again without the latest drivers
Sep 7, 2026 · technical · medium85% confidence
Greg flagged in #distinguished-leaders that the NVIDIA contract obliges CIQ to distribute the latest drivers within 30 days of release, and that CIQ has drifted. Peter took it to Nathan Blackham and Justin Haynes in their group DM the next morning, told both that he had not had the 30-day obligation in his head either, and asked them to put some sort of process in place to stay current. Nathan opened ENGD-839 for the automation; Peter told him to slot it right around where the next RLC-AI drop lands. Separately he told Product the gate plainly: it makes no sense to put out RLC-AI again without updating to the latest drivers at the very least.
People: Nathan Blackham, Justin Haynes, Greg Kurtzer, Brady Dibble
Hold the write-static-code guardrail at the 98 percent it actually buys - concede the escape case publicly and still refuse to mandate replacing third-party MCPs
Sep 2, 2026 · technical · high94% confidence
At the 9/2 engineering all-hands Peter clarified the missive he had sent two weeks earlier about writing static code rather than pointing Claude at an MCP server. He gave the concrete failure it came from: someone stood up a Claude skill pointed straight at the HubSpot MCP server, and it deleted an org worth of data and 14 customers. The rule he wants: have the AI write Python or JavaScript, scope the token to exactly what that script needs, and let the skill know only that the script exists - not that the MCP server does. Jamin Collins pushed back hard and correctly - an over-helpful agent will eventually route around any tool that is not an actual guard, and once it learns it can, it will stop using the guard entirely. Peter conceded it in front of the whole org twice - he is right, he is completely right - and then held the line anyway: I do not think it is worth throwing out a pattern that works 98 percent of the time because of the 2 percent. If there is a better pattern you want to pitch, I am all for it. He named the tradeoff explicitly: the full fix is replacing each third-party MCP, which means an admin approval process per service, and I do not want to constrain our usage and our experimentation right now. Once we find that a certain set of tooling is really valuable, then it is worth investing in doing exactly that. He framed it as accepting a little bit of risk so that we can experiment as a team, and set the target as getting rid of the 100 percent failure case, not solving every edge case.
People: Jamin Collins, Andrew Jorgensen, Stephen Moody, Nathan Blackham
Judge the Atlas code on its engineering merits alone - keep it, port it off the laptop, and stop building skills with direct data access
Sep 2, 2026 · technical · high94% confidence
After a deep dive through the revops-motherbrain repo and a walkthrough with Karl Lowenbjer, Tabatha Wilmot and Chris Baek, Peter delivered a written verdict on 8/31. He opened by refusing the question he was not asked: I am making ZERO statements about whether what this code was written to do is something we need at CIQ - that is a discussion for others. On the code itself: keep the bulk of it. API, frontend, warehouse and agents are independently separable with no cross dependencies. Data access is SQL direct rather than through hallucination-prone skill paths, read-only is enforced at the right layers, the MCP server sits above those layers, test-to-code ratio is roughly 2:3 and the documentation has not drifted. Fixes named: pull the business logic out of the code into a human-readable rule set (he called it the most valuable part of the project), delete the dead Trust Anchor, remove a write token that should be read-only, de-duplicate the mql LLM screen defined in two codebases. The real problem is what runs on the contractor laptop - Claude skills against the claude.ai MCP connector with direct HubSpot write scope, running without asking permission, bypassing the pipeline-approval and closed-edit rules HubSpot enforces on humans. Verdict on the open PRs: approve the move to a consolidated server, do NOT proceed with the interim laptop-resident lead router, fix it properly instead. And a standing instruction: the process of building out skills with direct access to data should stop immediately. In the 8/31 sync with Bjorn and Chris Baek he also confirmed he could pick up and make good use of the code if the contractor who wrote it did not stay - separating the keep-the-code question from the keep-the-person question.
People: Karl Lowenbjer, Chris Baek, Bjorn Hovland, Tabatha Wilmot
Scope the Atlas review to how it works, not whether it is worth it - and restate the rule that Claude must call a script, never an MCP directly
Aug 28, 2026 · technical · medium90% confidence
In the Motherbrain / Atlas sync with Tabatha Wilmot, Karl Lowenbjer and Chris Baek, Peter set the terms of his own code review. He told them outright that he would not wade into whether Atlas is valuable - that is a Tabatha-and-sales question - and that what he wants to answer is how it works, how it is architected, how it stores and persists data, how it does security protections, what should run on a laptop versus centralized, and what the scheduler should be. He asked for the scripts and skills that still live only on Tabatha laptop to be pushed into the repo first so he can read both ends of the system, asked for half-sentence direct answers to Marlon open questions rather than more narrative, and committed to a four-to-five-hour deep dive by Saturday evening with a follow-up the middle of next week. When Tabatha described reps closing deals wrongly through the HubSpot MCP, Peter named the cause - Claude writing code at runtime against a system of record - and restated the standing rule that Claude should never call an MCP directly, it should call a script that structurally cannot do that.
People: Tabatha Wilmot, Karl Lowenbjer, Chris Baek
Fuzzball Substrate can be an option for the Kubernetes engine, never the only runtime - ship on existing rails or nobody buys the train
Aug 24, 2026 · technical · medium73% confidence
In the Aug 24 Kubernetes Weekly Sync, Jonathon Anderson raised whether the new CIQ Kubernetes engine should use Fuzzball Substrate as its default or its only container runtime, framing it as the question that would drive whether Substrate gets open-sourced - a Substrate-by-default engine would put Substrate in far more places and let CIQ target the same worker nodes with both Fuzzball Orchestrate and the Kubernetes engine. Bjorn Hovland split it immediately: default yes, only no, on the grounds that forcing one runtime reduces the usefulness of Kubernetes and CIQ products should be independently viable with value-add tie-ins. Peter argued against both halves from the buyer side, taking the position of a hyperscaler already running Kubernetes and evaluating a switch to CIQ: either CIQ sits on the same Kubernetes they already sit on, which is an easy sale to reason about, or CIQ hands them a pile of validation hoops to clear before a contract can be signed. He compressed it to a rule - it has to include existing rail widths, otherwise nobody is going to buy our trains - and then kept the door open rather than closing it, noting defaults can change over time as Fuzzball starts getting traction. Jonathon withdrew the framing as overstated. Chris Wolford then supplied the decisive fact, that Substrate cannot be the Kubernetes default because the engine would fail Kubernetes conformance testing and lose certification, and proposed it as a user-selectable option in the Pro tier instead. Jonathon concluded the open-source question is therefore deferred, since it only mattered if Substrate were the shipping default.
People: Jonathon Anderson, Bjorn Hovland, Chris Wolford, Brian Dawson
Extend the existing Claude PII policy unchanged to the new AI systems - and settle that HubSpot data is not restricted customer data
Aug 18, 2026 · technical · medium83% confidence
Brady Dibble raised a data-access question about additional AI systems on 2026-08-15, treating the data he had lost access to as uniformly restricted. Peter refused to answer it per-system. He ruled that (1) the things you lost are not all the same - some are restricted and some are not, (2) Hubspot data is not customer data, and a list of customers is just fine while PII is not, (3) as far as I am concerned the same rules can apply to the two systems you just listed, and (4) there should be no barrier to moving approved data to them, but they should not open the door to more data than we currently allow to Claude. He pointed Brady at the existing policy, noting Michelle has it.
People: Brady Dibble, Michelle Novicio
Institute the inspectable-function rule for AI data access as company-wide hygiene - and refuse to make it either a shared library or retroactive
Aug 11, 2026 · technical · high97% confidence
At the Friday AI Committee, Peter re-delivered the rule he had set once before and watched not land. He used Mini-Me as the worked example: AI helped create the capability, but AI is not part of the runtime function of accessing the repositories. Claude cannot get to Slack except through that function. It does not have the keys. He asked for exactly one thing - if you are pulling data in or out of a CIQ system, do it through a function that you write and could inspect - and explicitly declined two obvious extensions. He would not make it a shared implementation, because there is no value in building a library of these things and because forcing an MCP server would block people from experimenting with technology being stood up today. He would not apply it retroactively, because a year from now we are going to have 100 times as many tools running around the company, so he is way more worried about the giant tsunami of stuff that is coming. He named three reasons rather than one: inspectability, 100 percent repeatability, and teaching the company the right pattern, since Claude is going to make everybody at the company a developer and that is incredibly terrifying if we do not teach people to be better developers. He extended the bar to local models when asked, and licensed Michelle to publish the message with his name attached.
People: Michelle Novicio, Ryan Smith, Brian Dawson, Stephen Moody, Steve Wallace, Brady Dibble
Endura scope for CIQs own pipelines - include Fuzzball, Warewulf, Ascender and Depot; exclude Apptainer
Jul 28, 2026 · technical · medium79% confidence
Eric Sheridan of Infrared Security updated Peter on the Endura proposal for securing CIQ build pipelines, noting Nathan and Justin had reviewed the Koji integration strategy positively and that Bjorn was excited about using Endura to protect Rocky core packages. Eric asked whether scope should extend beyond Rocky to Fuzzball, Warewulf, Apptainer and Ascender. Peter replied to include Fuzzball, Warewulf and Ascender, said Apptainer should not be necessary, and added a product Eric had not asked about - Depot - on the grounds that projects which serve CIQ products to customers should be included as well. Pricing follows this scope, and a call with Greg and Bjorn was set for Wednesday.
People: Eric Sheridan, Nathan Blackham, Justin Haynes, Bjorn Hovland, Greg Kurtzer
Require all agent/LLM access to systems of record to go through inspectable static code, never direct LLM access
Jul 24, 2026 · technical · high80% confidence
Peter opened the 7/24 Jira Automation call with Karl Lowenbjer by setting a condition before taking Karls agenda. The rule: no LLM gets direct access to a system of record. In every instance the LLM writes code that accesses the system, and that code is inspectable by a human or by another LLM. He stated he wants this true for everything Karls team builds, and that as long as it is true he is comfortable. He explicitly accepted that the filters themselves may be wrong - we might make a mistake in terms of what we pull in and out and that is fine, I do not mind mistakes - but the mechanism is non-negotiable. Karl confirmed Atlas already works this way. Peter then closed the topic without further discussion.
People: Karl Lowenbjer, Tabatha Wilmot, Michelle Novicio, Chris Baek
Icicle wind-down — stop tracking; preserve patent + Fuzzball-side optionality
May 29, 2026 · technical · medium65% confidence
In the 5/28 Nathan 1:1, Peter formally took Icicle off his active tracking list. Nathan concluded that Icicle is not going to make money for CIQ; Peter agreed and said I am checking it off my list. I am done watching it. The wind-down is not a kill — patent prosecution continues (CIQ filed a day before Dell/Google filed similar patents), and Icicle survives as a possible Fuzzball-side opportunistic checkbox feature for hobbyist or small-deployment workloads. What was cut is Peter attention as a recurring tracked item.
People: Nathan Blackham
Icicle viability gate: AI inference benchmark on H100 decides go/no-go
May 7, 2026 · technical · high88% confidence
Set a clear decision gate for the Icicle project: viability is determined by performance on a real-world AI inference workload, not synthetic benchmarks. Omer to run the RLC Pro AI benchmark on an H100 GPU. 2-3x synthetic CPU/memory degradation is acceptable IF power savings are significant for AI inference; otherwise project gets punted.
People: Nathan Blackham, Ryan Smith, Omer, Ahmer Mumtaz, Jeff Uphoff, Damen Knight, Ani Fox Bochenkov, Bjorn Hovland
Fuzzball SaaS Prepaid Token Billing Model with Unified Portal Backend
Mar 21, 2026 · technical · high93% confidence
Drove adoption of prepaid token ('CIQ Tokens') billing model for Fuzzball SaaS MVP, with unified Portal backend as single source of truth for all user accounts and token balances. Free tier deferred to fast-follow. MVP requires account + credit card to run workflows.
People: Greg Kurtzer, Bjorn Hovland, Jonathon Anderson, Chris Wolford
Gate Fuzzball SaaS GTM on Working Prototype — Assign Michelle
Mar 21, 2026 · technical · medium92% confidence
Agreed to move forward on Fuzzball SaaS GTM but gated it on having a working prototype first. Suggested Michelle as the resource to stand up a prototype on top of existing Fuzzball, accepting she's not the ideal person but can make progress. Rejected doing it 'out of band' — keep in normal product flow.
People: Greg Kurtzer, Michelle, Jonathon Anderson
AI Governance Single-Track Pivot for ISO 42001
Mar 13, 2026 · technical · high91% confidence
Pivoted AI governance from dual-track (internal vs products) to single rigorous model because CIQ products (RLCAI, Fuzzball, Werewolf) now directly integrate AI, changing the liability profile.
People: Steve Wallace, Stephen Moody, Brady Dibble, Chris Baek, Ryan Smith, Tabatha Wilmot, Brian Dawson, Michelle Novicio
Asset Management — Backend-First, Bridge to Views
Mar 11, 2026 · technical · low74% confidence
Advocated for deciding where the asset management backend should live first, then building bridges to expose it wherever needed (Jira, Rippling). Explicitly disagreed with Greg's framing that building bridges adds unnecessary complexity.
People: Greg Kurtzer, Christina, Mariah Rippee
Toyota POC — No Hotfix, Demo MPI and PBS Separately
Mar 5, 2026 · technical · high82% confidence
Decided NOT to rush a hotfix for Toyota's urgent out-of-scope MPI-via-PBS request before their Thursday director meeting. Team will demo MPI and PBS as separate working components, explain the integration bug is known, and commit to fix in ~1 week by the March 17 Reno meeting.
People: Chris Wolford, Jonathan Anderson, Bjorn Hovland, Art Tyde, Yoshi
Demanded measurable success criteria for LinuxLM project
Mar 3, 2026 · technical · medium69% confidence
Greg proposed training/fine-tuning a Linux-expert foundation LLM (LinuxLM). Peter pushed back by demanding explicit success criteria — deployment plan, evaluation methodology, and clear value proposition — before endorsing the project.
People: Greg Kurtzer, Bjorn Hovland
Assign Max as RLC-AI benchmarking plan owner with RHEL comparisons
Feb 20, 2026 · technical · medium65% confidence
Assigned Max Spevack as owner of the RLC-AI benchmarking plan in response to Greg's question about ownership. Directed that RHEL comparisons be added to exit criteria. Bjorn owns product definition (what to benchmark), Max owns technical execution (how to benchmark accurately). Max will create a one-page methodology document for the Humane pitch.
People: Max Spevack, Greg Kurtzer, Bjorn Hovland, Nathan Blackham, Damon
Directed Fuzzball team to improve logging and error observability
Feb 20, 2026 · technical · medium81% confidence
After AMD MI300 troubleshooting meeting, directed Fuzzball team (Jonathon Anderson, David Horn) that the product needs better logging and error visibility. Customers should be able to self-diagnose issues via log files instead of requiring live troubleshooting meetings with CIQ engineers.
People: Jonathon Anderson, David Horn, Chris Wolford
AMD RLC Plus Strategy: Speed-to-Market with Minimal Scope
Feb 4, 2026 · technical · high90% confidence
Decided to prioritize speed-to-market for the RLC Plus AMD co-marketing launch. The initial build will use upstream AMD packages (pre-built ROCm), the kernel driver (not upstream DKMS), and enable EPEL. Deferring the more robust in-house rebuild until market traction is proven. Justin Haynes to draft proposal and decision matrix.
People: Justin Haynes, Nathan Blackham, Brady Dibble, Brian Dawson, Peter Nelson
AI-First PR Process - Tests Over Human Review
Feb 3, 2026 · technical · medium85% confidence
Decided that PR process should evolve to prioritize automated test suites over human code review for AI-generated code. If a comprehensive test suite passes, the code is considered production-ready regardless of its origin.
People: Greg Kurtzer, Cedric
Project Shackleton - RESF contingency infrastructure
Jan 30, 2026 · technical · medium95% confidence
Build a parallel mirror of all RESF infrastructure in AWS (Git repos, Koji, vault/pub, Mattermost history) with goal of restoring Rocky Linux builds within two weeks if Lewis triggers his kill switch. Everything built with CDK and Ansible for repeatable deployment.
People: Nathan Blackham, Dieter, Peter Nelson, Max Spevack
PR standards in AI era - own the test suite, not the code
Jan 30, 2026 · technical · medium88% confidence
In an AI-enabled world, engineers should own the test suite and exit criteria, not necessarily every line of code. Quality comes from tests passing, not from reading every line. Engineer accountability shifts from I wrote this code to I own that this code passes these tests.
People: Max Spevack, Nathan Blackham, Justin, Peter Nelson
CVE automation architecture - simple state machine, 1 CVE per commit
Jan 30, 2026 · technical · high92% confidence
CVE automation should be built as a simple state machine with clear exit criteria at each step. Each commit addresses exactly one CVE. The orchestrator should be stupid-simple - just moving between states. Steps: Research -> Rebase -> Build -> Test -> MR -> Final Build -> Integration Test -> Promote to Beta -> Integration Test -> Production.
People: Max Spevack, Nathan Blackham, Justin Haynes, David Gomez
Committed to ensuring Greg technical direction reaches Justin
Jan 26, 2026 · technical · medium82% confidence
Committed to redirecting Justin to follow Greg architectural guidance on object storage/depot, and to be the conduit ensuring Greg technical direction reaches engineering clearly. Greg flagged that depot work was not in line with past directives.
People: Greg Kurtzer, Justin Haynes, Nathan Blackham
Approved proceeding with x86 RESF shadow infrastructure first
Jan 22, 2026 · technical · medium80% confidence
Approved Jonathan Dieter proceeding with the RESF shadow infrastructure build using x86 architecture first, deferring non-x86 builders (S390X, RISC-V, PPC64LE) to address later. The 4-week MVP timeline is feasible for x86.
People: Jonathan Dieter
Koji Build System Priority - RESF Risk Mitigation
Jan 13, 2026 · technical · high75% confidence
Directed Nathan to prioritize Koji cluster standup with Dieter. Goal is to have independent build infrastructure in place so CIQ is not dependent on RESF. Asked for timeline if this became top priority, and indicated Dieter should reprioritize accordingly.
People: Nathan Blackham, Dieter
Partner/User Management Tech Debt - Accept for Speed
Jan 12, 2026 · technical · medium95% confidence
Explicitly acknowledged and accepted that Partner Portal, Fuzzball SaaS, and Portal Depot will have separate user/account systems rather than integrating them. Flagged the future cleanup cost but chose speed over architectural purity.
People: Lindsay Aamodt, Tabatha, Dave Dickerson, Ramesh
CVE Strategy - Eventually Consistent Model
Dec 29, 2025 · technical · high78% confidence
Aligned with Max on new approach to CVE patching: adopt an eventually consistent model that prioritizes rapid patching over perfect upfront testing. Accept a small error rate (e.g., 5%) as a necessary trade-off for speed, with fixes handled by COE.
People: Max Spevack, Nathan Blackham, Jeff Uphoff, David Gomez
AI Bot Architecture Decision
Dec 28, 2025 · technical · medium73% confidence
Decided to build a web app to front the AI bot, allowing curated outputs to be shared with Sarah and others without granting direct data access to underlying Slack/email/Jira data.
People: Stephen Moody, Sarah Almaraz
CVE Remediation - Direct Intervention Required
Dec 24, 2025 · technical · high79% confidence
Identified unacceptable lack of urgency from Nathan team on NARF-created CVEs. Will take direct action to address performance issues next week.
People: Nathan Blackham, Max Spevack, Brady Dibble
RLC 9.7 Launch Path Decision
Dec 19, 2025 · technical · high78% confidence
Participated in RLC 9.7 Launch planning meeting to decide path forward on release and rework priorities.
People: Nathan Blackham, Justin Haynes, Max Spevack, Brady Dibble, Sarah Almaraz, Brian Dawson