# Jameson Stone > Public writing, paintings, projects, and studio directory by Jameson Stone. Canonical site: https://jamesonstone.io/ Curated index: https://jamesonstone.io/llms.txt ## Home [Jameson Stone](https://jamesonstone.io/) Just put the AGI in the bag, bro. ## Blog [Writing archive](https://jamesonstone.io/blog/) ## About Source: https://jamesonstone.io/about/ # About I am Jameson Stone. This site collects short notes, older writing, paintings, and project records. ## Profiles - [GitHub](https://github.com/jamesonstone) - [LinkedIn](https://www.linkedin.com/in/jameson-stone-66b521196) ## Contact Email: jameson AT stone DOT tc ## Reasoning Machines Source: https://jamesonstone.io/reasoning-machines/ # Reasoning Machines Connected work across software, systems, and applied AI. - [Reasoning Machines](https://reasoningmachines.co/) My independent studio for software, systems, and ideas. - [Applied Symbolics](https://github.com/appliedsymbolics) Software for operational work. Explore [Ambient OS](https://ambientos.dev/) and [OpenCarrier](https://opencarrier.dev/) . - [OpenVital](https://openvital.co/) AI applied to healthcare. The healthcare arm of Reasoning Machines. - [Limina Dev](https://limina-dev.github.io/) Warehouse and logistics software, including mapr and sortr. - [Spectral7](https://github.com/spectral7-ltd) Applied AI and operational software spanning logistics, communications, and knowledge systems. Research tools: [Scout](https://scout.jamesonstone.io/) · [Narrowband](https://narrowband.jamesonstone.io/) [All projects & source code](https://jamesonstone.io/projects/) ## Projects Source: https://jamesonstone.io/projects/ # Projects Public projects are things you can open and use. Code projects are source repositories, CLIs, and workflow tools. I keep them separate here so each list stays easy to scan. ## Public Projects - ### [Hyperlite](https://github.com/jamesonstone/hyperlite) Attention for active Git work A native macOS window and CLI for tracking open pull requests, review feedback, merge conflicts, and GitHub Actions across your projects. [Read the guide](https://github.com/jamesonstone/hyperlite/blob/main/docs/USER_GUIDE.md) [Source](https://github.com/jamesonstone/hyperlite) - ### [Kit](https://github.com/jamesonstone/kit) Repository-local contracts for coding agents A provider-neutral CLI that installs agent contracts and contextual rules, maintains project memory, and validates repository guidance. [Get started](https://github.com/jamesonstone/kit#how-kit-works) [Source](https://github.com/jamesonstone/kit) - ### [Mint](https://jamesonstone.io/mint/) Semantic versioning and release automation A CLI and GitHub Action for computing versions, writing changelogs, and managing releases across production, non-production, and sandbox environments. [Read the docs](https://jamesonstone.io/mint/) [Source](https://github.com/jamesonstone/mint) ## Code Projects - [kit](https://github.com/jamesonstone/kit) : A portable, document-first workflow harness for specs, planning, implementation, review, and handoff. - [yp](https://github.com/jamesonstone/yp) : A tiny path-copying CLI for sending files, directories, or picker selections to the clipboard. - [kp](https://github.com/jamesonstone/kp) : A local prompt utility for keeping reusable coding-agent prompts close at hand in the shell. - [event-collector](https://github.com/jamesonstone/event-collector) : An HTTP service and CLI for collecting generic event envelopes with hash validation and idempotency. - [sso-testkit](https://github.com/jamesonstone/sso-testkit) : A Go CLI for checking SSO readiness with scenario files, token-exchange probes, and redacted reports. - [radar](https://github.com/jamesonstone/radar) : A Lightweight Process Observer for Agents. - [mint](https://github.com/jamesonstone/mint) : Compute the Next Version, Write the Changelog. Mint the Release. - [ding](https://github.com/jamesonstone/ding) : Get Invoices Paid. - [beacon](https://github.com/jamesonstone/beacon) : Signal Layer for Coding Agents. - [flx](https://github.com/jamesonstone/flx) : Source Control Rebuilt for the Agentic Era, with Git-Native Interoperability. - [hyperlite](https://github.com/jamesonstone/hyperlite) : NextGen Signal Layer for Coding Agents. - [rungrid](https://github.com/jamesonstone/rungrid) : Unified Development Workspace Observability. - [ghostgc](https://github.com/jamesonstone/ghostgc) : Garbage Collection for Abandoned AI Coding Runtimes. - [loopc](https://github.com/jamesonstone/loopc) : PID-driven Process Control. - [kura](https://github.com/jamesonstone/kura) : Storehouse of CLI Commands. - [narrowband](https://github.com/jamesonstone/narrowband) : Discovering Meaningful problems worth solving. - [scout](https://github.com/jamesonstone/scout) : Finding Innovative AI Signals via Research Distillation. - [outpost](https://github.com/jamesonstone/outpost) : Durable Alerts From Committed Events. - [touchstone](https://github.com/jamesonstone/touchstone) : Personal Benchmark of LLM Performance and Efficiency. - [alfred-mac-emoji-picker](https://github.com/jamesonstone/alfred-mac-emoji-picker) : Type e or emoji in Alfred to open the built-in macOS emoji picker (Ctrl+Cmd+Space). ## Archive - project-aurora: Handling everything between development and operations with Kubeless. - polymer: An event-to-Kafka translation layer written in Kotlin. - opportunity-rover / curiosity-rover: End-to-end Debezium database migration experiments. - orpheus: Portable storage with Git. - ack: A chatbot for development and operations workflows. - jib: Programmatic Kubernetes manifest creation with Kustomize. - bulkhead: API gateway experiments with backoff behavior. - tf-R: Declarative reconciliation enforced by a control loop. ## Paintings Source: https://jamesonstone.io/paintings/ # Paintings A small visual archive. - ![A Mote of Dust Suspended in a Sunbeam](https://jamesonstone.io/assets/images/paintings/amoteofdust.jpg) ## A Mote of Dust Suspended in a Sunbeam The vastness of space against a tiny blue dot, drawn from Carl Sagan's pale blue dot reflection. - ![Continuity of Complexity](https://jamesonstone.io/assets/images/paintings/continuity.jpg) ## Continuity of Complexity A painting about how every subject grows more complex the closer you look at it. - ![The Light That Beckons](https://jamesonstone.io/assets/images/paintings/thelightthatbeckons.jpg) ## The Light That Beckons A first painting about decisions, divergence, and finding humor in the course of life. ## Published posts ## The Internet was Always Just a Transport Layer Source: https://jamesonstone.io/blog/2026/11/02/the-internet-was-always-just-a-transport-layer.html Date: 2026-11-02T15:10:00+00:00 Nobody wants to learn your navigation. They want the information, the product, or the result. We built an economy around making people visit websites, then confused the visit with the value. Agents expose that confusion. Compare the prices. Check the inventory. Book the appointment. The user gets what they came for without touring five interfaces and dismissing five cookie banners. Your database still matters. Your funnel has a problem. Dead internet theory worries about a web without people. But people do not need to be present for work to happen on their behalf. Fewer human visits can mean less wasted human time. Calling that death mistakes attendance for usefulness. The internet connects systems. We put storefronts on top and started treating the storefront as the infrastructure. Agents have no reason to preserve that arrangement. Beauty survives this. A mural does not need a conversion rate. A room can be worth entering because somebody made it extraordinary. A website can earn attention the same way. The experience has value when someone actually wants the experience. But making someone navigate a website to retrieve a fact is not an experience worth defending. It is work we have learned to tolerate. Build something people want to experience, or provide something useful enough that their agents come looking for it. Expecting attention simply because you put an interface in the way is a business model with an expiration date. ## Complexity Generators Source: https://jamesonstone.io/blog/2026/10/28/complexity-generators.html Date: 2026-10-28T15:40:00+00:00 A Complexity Generator repeatedly creates work for other people while reporting their own work as complete. The output may be a document, a process, a specification or a working prototype. It looks like progress because something now exists. Its producer has a visible deliverable. Everyone downstream inherits the task of deciding what it means, where it fits and who will keep it working. AI amplifies this trait. As [implementation becomes commoditized](https://jamesonstone.io/blog/2026/10/06/implementation-is-commoditized.html) , producing artifacts takes less effort. Their interpretation, integration and maintenance still consume organizational capacity. A person can now generate obligations much faster than an organization can absorb them. The [shallow expert](https://jamesonstone.io/blog/2026/10/12/the-shallow-expert.html) now has a tool that can generate documentation, code and process almost without limit. Each addition arrives with a plausible explanation, reinforcing the impression that its creator understands the system. The ability to generate a process becomes confused with the judgment required to improve one. More structure accumulates without making the work simpler. Twelve hours with a coding agent can produce an impressive solution to a problem nobody has. Getting it to run feels like validation. The receiving team must then spend time evaluating it, establishing why it has no practical use and explaining that back to its creator. The organization pays for the output even when it discards it. “Have your agent read it” changes how the material is consumed. Someone still has to resolve contradictions, reconcile it with the existing system and own the consequences. A summary cannot accept responsibility. The cost can compound. Each addition must coexist with earlier decisions, interfaces and commitments. Another prototype can create another integration project. Another specification can create another meeting. Output accelerates while the organization accumulates unfinished obligations. The Complexity Generator introduces complexity and leaves simplification to somebody else. Throwing work over the wall is how they declare completion: the unresolved decisions, integration and consequences become the receiving team’s problem. Their productivity depends on counting the artifact while excluding the work it creates. Complexity Generators can work extraordinarily hard and produce negative value. The organizational failure is mistaking their output for progress. ## Team Joiners Source: https://jamesonstone.io/blog/2026/10/28/team-joiners.html Date: 2026-10-28T12:14:11+00:00 The useful divide in applied AI is between the mentally flexible and the team joiners. One camp treats tools, strategies and beliefs as revisable. The other picks a side and organizes its judgment around belonging. Team joining offers a shortcut. It narrows what deserves attention and supplies ready-made explanations for why the chosen approach remains superior. Contradictory evidence becomes easier to dismiss when a community has already written the rebuttal. Judgment becomes advocacy. The “Claude-pilled” version illustrates the problem. Someone uses Claude extensively, learns its strengths, adapts their prompts and becomes effective with it. That fluency can then be mistaken for proof of universal superiority. Other models receive less practice and less patience. Familiarity gets credited entirely to capability. A useful test of mental flexibility is whether someone can recognize concrete value in a competing model. Can they explain where it wins? Can they name evidence that would change their preference? If every comparison must end with the same winner, the conclusion came before the evaluation. In product development and research, that attachment creates an opening. Team joiners protect their priors. The mentally flexible are ready to abandon all priors when a better outcome demands it. They welcome the discovery that their understanding is obsolete: it means something previously out of reach may now be possible. Team joiners have a position to defend. The mentally flexible have a result to improve. That difference is a source of alpha. ## Applied Philosophy Source: https://jamesonstone.io/blog/2026/10/25/applied-philosophy.html Date: 2026-10-25T15:56:00+00:00 Engineering turns scientific theory into things that exist. Applied philosophy would do the same with philosophical ideas: design them, implement them, and confront what happens outside the argument. It’s like Schrödinger, with real cats. ## Superposition IV: Cheap Execution, Expensive Frames Source: https://jamesonstone.io/blog/2026/10/25/superposition-cheap-execution.html Date: 2026-10-25T15:24:00+00:00 Always-on agents change the opportunity cost of an unexecuted idea. A possibility that once competed for an engineer’s working hours can now become work in progress overnight. More ambitions become feasible. Unused execution capacity can consequently begin to feel like a failure of imagination: if the machine can keep working, why has nothing worth doing been assigned to it? That pressure embeds an objective before it announces itself as a productivity problem. Keeping capacity occupied becomes evidence of seriousness, and a growing workload becomes evidence of ambition. Yet neither establishes that the resulting work deserves to exist. Accepting limits is a substantive challenge here: the demand for perpetual execution may be another inherited premise, not an unavoidable consequence of technical capability. Cheap execution can also create expensive framing. The first description generates a plan, artifacts, and dependencies. Redirection now requires deciding which of them survive. Their production cost may have been trivial, but the attention required to evaluate and abandon them is not. The interpretation acquires momentum through the obligations it creates. A partition can be resized, an alert automated, a log populated. Each local success confirms the usefulness of delegation while leaving the decomposition intact. A more encompassing abstraction might ask what operational understanding is actually required: what has changed, what matters, and when intervention is justified. That could dissolve several tasks rather than accelerate their execution. The scarce capacity is then the ability to change levels of explanation. A list of solvable problems can conceal a single misidentified need. Assigning every item to an agent preserves that mistake at greater throughput. An impressive volume of completed work can coexist with a failure to understand why the work was necessary. This is where ambition requires a harder test than output. Can the objective itself be revised after execution becomes available? Can a question survive long enough to expose the assumptions that made the task list seem inevitable? A capable agent makes implementation easier to justify. The more consequential achievement may be an abstraction that makes implementation unnecessary. Part 4 of 4. [Previous: Superposition III: Before the Prompt](https://jamesonstone.io/blog/2026/10/25/superposition-before-the-prompt.html) ## Superposition III: Before the Prompt Source: https://jamesonstone.io/blog/2026/10/25/superposition-before-the-prompt.html Date: 2026-10-25T15:23:00+00:00 The influence of AI may begin before the conversation does. Repeated interaction teaches which formulations elicit useful responses. Context, explicit objectives, and decomposable tasks become familiar ways to make a request intelligible. The further possibility is that these conventions begin selecting which thoughts are permitted to become requests at all. This is a hypothesis about anticipatory accommodation, not an established consequence of Kahneman’s framework. An intended listener can enter the composition of a thought. A difficult intuition is translated into the kind of question expected to receive a productive answer; eventually the translation may precede awareness of the intuition. The model need not choose the objective if its anticipated response helps determine what qualifies as an objective. Successful prompting can also teach ritual. A formulation works, its structure is repeated, and contingent details acquire the status of necessities. Intermittent success complicates the diagnosis: did the instruction improve the result, or did a variable system happen to produce a better answer? Without subtraction or comparison, proficiency can become a repertoire of superstitions. The slot machine teaches a technique for pulling the handle, then the technique becomes an account of how the machine works. Physical disconnection is insufficient if that account remains the medium of thought. A walk can remain an exercise in composing prompts. Conversely, using AI need not surrender conceptual independence if its formulations remain available for rejection. The distinction concerns whether an abstraction can survive long enough to be evaluated without first demonstrating compatibility with the tool. Cycles of intensive research and unstructured reflection could make that distinction easier to maintain. Research supplies material; an interval without immediate elaboration leaves room for relationships the initial question did not anticipate. This is a proposed discipline to test, not a guaranteed recipe for originality. Its criterion is whether a different question becomes possible, including one that makes the original request unnecessary. Better prompting alone cannot establish that the right problem was selected. Part 3 of 4. [Previous: Superposition II: Narrative](https://jamesonstone.io/blog/2026/10/25/superposition-narrative.html) · [Next: Superposition IV: Cheap Execution, Expensive Frames](https://jamesonstone.io/blog/2026/10/25/superposition-cheap-execution.html) ## Superposition II: Narrative Source: https://jamesonstone.io/blog/2026/10/25/superposition-narrative.html Date: 2026-10-25T15:22:00+00:00 An idea does not arrive as an isolated proposition. It emerges amid unfinished work, accumulated disappointments, ambitions, and a story about what a life is becoming. That context helps determine why a particular connection becomes interesting. Extracting the sentence can preserve its apparent content while severing the relationship that made it worth thinking about. Kahneman’s [System 1/System 2 account](https://kahneman.scholar.princeton.edu/document/8) distinguishes automatic, associative impressions from effortful monitoring and reasoning. Context and expectations influence what comes readily to mind; deliberation can inspect an impression or accept it. This does not map neatly onto superposition and articulation. Intuition can suppress ambiguity, while deliberate thought can reopen incompatible interpretations. Slowness is no guarantee of correction. The proposed extension concerns the narrative surrounding an intuition. An idea about eliminating unnecessary work might express an insight into an organization. It might also protect an identity built around being indispensable. The same proposition can serve both functions. Establishing its validity would not establish that pursuing it serves the life in which it arose. Three questions therefore need separate answers: why did the idea occur, what warrants believing it, and why does it matter? Personal significance neither validates nor refutes a claim. But removing that significance does not produce a complete account of what deserves attention. Kahneman and Riis separately distinguish [lived experience from its remembered evaluation](https://kahneman.scholar.princeton.edu/document/7) ; that distinction concerns another dimension of judgment, not another name for System 1 and System 2. An agent can investigate a proposition while missing the ambition it expresses. It can also elaborate that ambition so convincingly that its original purpose escapes examination. The philosophical task is selective decoupling: assess the claim without using biography as evidence, then restore enough context to ask whether the claim addresses the need that produced it. Abstraction should permit movement between those levels. An idea stripped of its context can be logically defensible and existentially beside the point. Part 2 of 4. [Previous: Superposition I: Articulation](https://jamesonstone.io/blog/2026/10/25/superposition.html) · [Next: Superposition III: Before the Prompt](https://jamesonstone.io/blog/2026/10/25/superposition-before-the-prompt.html) ## Superposition I: Articulation Source: https://jamesonstone.io/blog/2026/10/25/superposition.html Date: 2026-10-25T15:21:00+00:00 An idea exists in superposition until articulation assigns it state. Several possible abstractions coexist before language distinguishes their boundaries and relationships. Explaining the idea selects one configuration. The resulting precision can conceal that selection: a description appears to reveal what the idea was, although it has helped determine what the idea becomes. Choosing a noun establishes a category. Choosing a verb assigns agency or causation. An analogy imports relationships that may never have belonged to the intuition. These commitments make an idea examinable, but they also determine which objections become intelligible. A formulation can exclude an alternative before either has been considered explicitly. AI rapidly elaborates the provisional interpretation. An incidental noun becomes an organizing principle; an analogy acquires the authority of a mechanism. The argument can become increasingly coherent while losing fidelity to the possibility that prompted it. Internal consistency establishes relationships within the explanation. It cannot establish that this was the explanation worth developing. The danger is difficult to detect because coherence rewards continued refinement. A contradiction looks like something to repair inside the framework. Yet it might be evidence that the framework’s boundaries were mistaken. Improving the explanation and recovering the original question can demand opposite actions: one adds distinctions that protect the argument; the other discards the distinction around which it was built. Preserving superposition means treating articulation as an experiment. A second formulation should be allowed to contradict the first, rather than becoming an amendment to it. The useful comparison concerns what each formulation makes visible, what it renders irrelevant, and which assumptions both quietly preserve. An intuition deserves scrutiny, not immunity; unresolvedness alone is no evidence of depth. But neither should an articulate explanation acquire ownership of the thought merely because it arrived first. Part 1 of 4. [Next: Superposition II: Narrative](https://jamesonstone.io/blog/2026/10/25/superposition-narrative.html) ## Introducing Mint Source: https://jamesonstone.io/blog/2026/10/18/introducing-mint.html Date: 2026-10-18T11:40:00+00:00 [Mint v1.0.0](https://github.com/jamesonstone/mint/releases/tag/v1.0.0) is out. It turns Git history into versioned releases: resolve the next semantic version from conventional commits, generate release notes, create an annotated tag, and publish a GitHub Release. Those steps belong together. The version describes a set of changes. The notes explain them. The tag identifies the source. Keeping them in one release workflow makes it easier to trace a published version back to the code that produced it. GitHub Actions is the normal way to use Mint. The action builds Mint on the runner, so contributors don’t need to install a local binary. A minimal, manually triggered release workflow looks like this: ``` name: Release on: workflow_dispatch jobs: release: runs-on: ubuntu-latest permissions: contents: write steps: - uses: actions/checkout@v4 with: fetch-depth: 0 - uses: jamesonstone/mint@v1.0.0 with: command: release-publish owner: ${{ github.repository_owner }} repo: ${{ github.event.repository.name }} github-token: ${{ github.token }} ``` Run this from the trusted release branch. The full checkout supplies the history and tags Mint needs to resolve the version. The job needs write access to publish its tag and GitHub Release. Mint refuses to move a conflicting tag. Monorepos can scope versioning to component paths and use prefixes such as backend/v1.2.3. Application workflows still own their builds, container publishing, and deployments; Mint handles release state. The optional CLI exposes the same tooling for manual work. Start with [Mint Release Tooling on GitHub Marketplace](https://github.com/marketplace/actions/mint-release-tooling) , or read the [integration guide](https://jamesonstone.io/mint/) . Mint is open source under Apache-2.0. ## No Sacred Cows in AI Source: https://jamesonstone.io/blog/2026/10/17/no-sacred-cows-in-ai.html Date: 2026-10-17T13:29:38+00:00 AI discoveries outpace productization. New capabilities reach users before they’re integrated into reliable products, leaving users to build the workflows around them. The danger is canonizing whatever happens to work. Useful intuition becomes prescriptive doctrine, and accumulated experience becomes an impediment to adaptation. Musashi wrote, [“You should not have a favourite weapon.”](https://www.yama-dojo.ca/resources/Miyamoto%20Musashi%20-%20Book%20of%20Five%20Rings.pdf#page=11) Skills illustrate the problem. Their present utility establishes neither permanent superiority nor immunity from obsolescence. “Fat skills” don’t escape that scrutiny. Novel packaging doesn’t establish conceptual novelty, and procedural complexity requires evidence of corresponding benefit. I predict the eventual obsolescence of all current scaffolding that compensates for missing AI capabilities. AI develops as trained ecosystems: models, tools, memory, and feedback. Useful behavior can be absorbed into the system, leaving external recipes redundant or actively restrictive. Prescribing yesterday’s workaround can prevent tomorrow’s system from exploiting its capabilities. Economic and reputational attachment compounds the problem. When skills become someone’s business or identity, the incentives favor elaborating the method over acknowledging its obsolescence. The only antidote to stagnation is flexible thinking: revisable priors, continually tested against changing capabilities. Intuition should accelerate experimentation without dictating its conclusions. Each model release warrants reassessing the assumptions embedded in our workflows. Without that discipline, experience ossifies into doctrine. We become proficient at preserving techniques whose original justification has disappeared. Test subtraction as rigorously as addition. Every technique must keep earning its place through demonstrated utility. The model eats everything. ## The Shallow Expert Source: https://jamesonstone.io/blog/2026/10/12/the-shallow-expert.html Date: 2026-10-12T12:00:00+00:00 I’ve been the shallow expert at times. It’s easy to mistake a convincing explanation for understanding, especially when it agrees with what you already think. Coding agents make that mistake easier. They can research a subject, compare approaches, and help build a solution. The help is real. But the finished work can give us confidence we haven’t earned. The cost of producing authoritative-sounding analysis has fallen faster than the cost of evaluating it. That gap is where shallow expertise takes hold. We know enough to repeat the conclusion, but not enough to explain the assumptions behind it. We may not even recognize what’s missing. The report reads well. Our confidence grows. Then the conditions change. A dependency behaves differently. A small change has a large effect elsewhere. Or a rare surprise breaks the model entirely. That risk never goes away. Understanding cannot predict every surprise, but it can recognize where the model ends and ask what happens beyond it. This takes non-linear thinking. It means following dependencies and feedback loops, then seeing when the original problem needs to be reframed. Finding that question is the higher-order skill. The question always matters more than the answer. Forty-two tells us nothing until we know what was asked. A team needs that reasoning to make a decision. Handing it a 150-page document does not establish that we’ve done the work. If we cannot explain the system ourselves, we’re asking everyone else to shoulder the burden of understanding it. The person questioning an assumption is doing necessary work. Defending the document against them only leaves the gap intact. “Brevity is the soul of wit.” Bring the team a clear account of what matters: the assumptions, the tradeoffs, the consequences, and what would change the recommendation. Leave room for questions. Anyone asking a team to act on an answer should be able to reason beyond it. ## Implementation is Commoditized Source: https://jamesonstone.io/blog/2026/10/06/implementation-is-commoditized.html Date: 2026-10-06T12:00:00+00:00 Implementation is commoditized. Coding agents are now good enough to take a well-described problem and turn it into working software on their own. Not always elegantly. Not always efficiently. But reliably enough that writing code is no longer the scarce part of building software. That changes more than it seems. The layer between For most of the history of software, code sat between the product manager and the engineer as a layer of abstraction. The PM described what the business needed. The engineer translated it into code. Because that translation was hard, slow, and expensive, it became a moat. But code was always a subset of something larger: the domain. Every line encoded a decision about how some messy, real-world system actually behaves. The billing edge case. The regulatory carve-out. The instrument that fails in a way nobody documented. The code was just the most legible artifact of that understanding. Because the translation was so expensive, it was easy to mistake the layer for the value. Engineers became “the people who write code,” and the domain complexity they absorbed along the way was treated as a side effect. The layer is gone Now a PM can describe a feature to an agent and get working code back. The translation layer has collapsed, and with it the moat built on the scarcity of people who could turn intent into software. What remains is everything the code was abstracting. An agent will implement what you ask for. It is much worse at knowing what you should have asked for. It doesn’t know that the “simple” status field is load-bearing for three downstream teams, or that the obvious refactor breaks a contract nobody wrote down. Those aren’t implementation problems. They’re domain problems, and they don’t disappear because the typing got cheaper. What engineers are paid for Engineers are, as they have always been, paid to own domain complexity. Not typing. Not syntax. The judgment about what the system should do, where it will break, and which tradeoffs are worth making. Subtlety, nuance, and non-linear thinking still live outside the explicit goals of how language models are trained and reinforced. In most cases they’re actively discouraged by the training set, which rewards the confident, the common, and the directly on-task answer. The model converges on the median. The domain lives in the exceptions. The code was never the moat. The understanding was. It still is. ## Jameson's Law of Information Traversal Source: https://jamesonstone.io/blog/2026/10/05/jamesons-law-of-information-traversal.html Date: 2026-10-05T13:00:00+00:00 Almost everything you see about AI, whether in the news, on social media, or in whatever else makes its way to you, is wrong. But it isn’t just wrong. It’s a whole spectrum of wrong. Most of it is wrong because the subject is complicated, and the person summarizing it for you doesn’t entirely understand it themselves. A large share is wrong on purpose, built to persuade you toward a conclusion they already own. The rest is wrong because wrong is entertaining (flat earth, birds aren’t real, etc). The small slice that’s right is right narrowly. It has to be. Any topic of sufficient complexity can’t be broadly right. By definition, in fact. Each of these has the same cause: distance. Every retelling moves the idea one step further from the people who understand it, and every step trades a little truth for a little reach. What arrives at you is whatever traveled best, not whatever was most accurate. So the key to living online is to pay attention to everything, ignore 90% of it, and constantly assess the final 10% for intent. in summary: Jameson’s Law of Information Traversal Expanded Formula: As information about a complex subject moves farther from its source, it tends to become easier to communicate and less accurate. What survives is increasingly selected for simplicity, persuasion, or entertainment rather than fidelity. Jameson’s Law of Information Traversal The farther a complex idea travels from its source, the more it optimizes for transmission over truth. Jameson’s Law of Information Traversal Corollary: The more consequential a claim is to you, the closer to the source you should go. ## Intelligence as a System of Beliefs Source: https://jamesonstone.io/blog/2026/07/01/intelligence-as-a-system-of-beliefs.html Date: 2026-07-01T13:00:00+00:00 I’ve been converging on a different definition of intelligence. Intelligence is the capacity to discover, refine, and maintain a system of beliefs whose fitness generalizes across domains. This is intentionally different from defining intelligence as knowledge, memory, or raw computational ability. Knowledge is local. Intelligence transfers. Think about the people who seem capable of solving almost any problem. They can debug software, repair a car, untangle an organizational problem, reason about finance, or quickly become effective in an entirely new discipline. These domains have almost nothing in common on the surface, yet the same people consistently perform well across all of them. Why? Because they aren’t relying on domain-specific skills. They’re applying the same underlying beliefs. Beliefs like: - Every effect has a cause. - Symptoms are rarely the root cause. - Good experiments reduce uncertainty. - Systems can be understood through their state and interactions. - Constraints explain behavior. - Simplicity should be exhausted before complexity. - Every complex problem is simply a compilation of many smaller, less complex problems. These aren’t facts. They’re beliefs about how the world behaves. More importantly, they’re beliefs that continue to prove useful regardless of the domain. The word belief is deliberate. A belief doesn’t have to be objectively true. It only has to be useful. Science itself operates this way. Every scientific theory is, in some sense, a belief about reality. Some theories eventually prove incomplete or even incorrect, yet remain extraordinarily useful. Newtonian mechanics is “wrong” under relativity, but it’s still fit enough to design bridges, predict trajectories, and send spacecraft to the Moon. Its value comes from its predictive power, not its claim to ultimate truth. That’s why I prefer the word fitness over truth. Truth may be unknowable. Fitness is observable. A belief is fit if it consistently produces accurate predictions and successful decisions across many different environments. Viewed this way, intelligence is not a collection of isolated skills. It’s an evolving worldview. A network of beliefs that has survived repeated contact with reality because it continues to make sense of new situations. The people we call “intelligent” aren’t necessarily the ones who know the most. They’re the ones whose beliefs keep working. ## Getting Started Avro Schema and Debezium Source: https://jamesonstone.io/blog/2020/03/04/avro-schema.html Date: 2020-03-04T17:00:00+00:00 ![schematics](https://jamesonstone.io/assets/images/blog/avro-schema.png) [Apache Avro](https://avro.apache.org/) is a data serialization system. More verbosely, Avro is a row-oriented remote procedure call and data serialization framework developed within Apache’s Hadoop project. It uses JSON for defining data types and protocols, and serializes data in a compact binary format Avro provides: - Rich data structures. - A compact, fast, binary data format. - A container file, to store persistent data. - Remote procedure call (RPC). - Simple integration with dynamic languages. Code generation is not required to read or write data files nor to use or implement RPC protocols. Code generation as an optional optimization, only worth implementing for statically typed languages. Source: [quickstart](https://github.com/phunt/avro-rpc-quickstart#whats-apache-avro) [Debezium](https://debezium.io/) is a change data capture implementation that: Debezium is an open source distributed platform for change data capture. Start it up, point it at your databases, and your apps can start responding to all of the inserts, updates, and deletes that other apps commit to your databases. Debezium is durable and fast, so your apps can respond quickly and never miss an event, even when things go wrong. Building an Avro Schema Example Let’s start by building an avro schema by hand, and then generating the resulting classes for use. For this setup, we’ll use tutorials point [site](https://www.tutorialspoint.com/avro/avro_schemas.htm) ``` { "type" : "record", "namespace" : "com.jstone28.winch", "name" : "Employee", "fields" : [ { "name" : "message" , "type" : "string" }, ] } ``` type - primitive or derived type; “record” indicates namespace - the namespace where the object resides name - combined with the namespace it defines the identifier for object; when in a field, it services as the name of the nested object fields - holds the JSON data for the object The [specification](https://avro.apache.org/docs/current/spec.html) provides an excellent resource. Automatically Generating Objects As someone who does not come from the JVM world, the plugin system and automagical process that author these generated objects are still a mystery to me. I will, one day, dig into the source of the gradle plugin system and the respective avro plugins to track down how this all gets put together but I’ll save that for another blog. For now, we can add the following to the build.gradle.kts and settings.gradle.kts to have access to the generated object at the classpath represented by the namespace and name of record. For example: build.gradle.kts ``` plugins { id("com.commercehub.gradle.plugin.avro") version "0.16.0" } repositories { mavenCentral() maven(url = "https://packages.confluent.io/maven/") } dependencies { implementation("io.confluent:kafka-avro-serializer:5.2.1") implementation("io.confluent:kafka-schema-registry-client:5.3.0") implementation("org.apache.avro:avro:1.8.2") } ``` settings.gradle.kts ``` pluginManagement { repositories { jcenter() gradlePluginPortal() maven(url = "https://dl.bintray.com/gradle/gradle-plugins") } } ``` Streaming Database Changes with Debezium In the past, we’ve setup Debezium to stream data changes in the [MySQL](https://jamesonstone.io/blog/2020/03/01/kotlinconsumer.html) and [PostgreSQL](https://jamesonstone.io/blog/2020/01/25/debezium.html) database. Today, we’ll use it to inspect the message being produced by the Debezium connector, and pushed into Kafka for consumption. ![topic view](https://jamesonstone.io/assets/images/blog/topic-ui-avro.png) By default, Debezium uses Avro binary data format and schema. Using a project like [fast-data-dev](https://github.com/lensesio/fast-data-dev) you can quickly visualize these items. ![schema UI](https://jamesonstone.io/assets/images/blog/schema-ui.png) ## Creating a Kafka Consumer with Kafka, Kotlin, PostgreSQL, and Debezium Source: https://jamesonstone.io/blog/2020/03/01/kotlinconsumer.html Date: 2020-03-01T17:00:00+00:00 Continuing with tools of the trade, we’re going to explore writing a Consumer for a Kafka topic in Kotlin. We’ll generate these messages with debezium via an http endpoint so that the process will be end to end. Lucky for us, we can build on some of what we’ve already been working on to implement this solution. We’ll do our work in the [winch project](https://gitlab.com/jstone28/winch) What’s Required - [Polymer](https://gitlab.com/jstone28/polymer) - [Instrumentation](https://gitlab.com/jstone28/instrumentation) - [Fast Data Dev](https://hub.docker.com/r/landoop/fast-data-dev) - [Avro Schema](https://avro.apache.org/docs/current/spec.html) - [Winch](https://gitlab.com/jstone28/winch) Our kotlin test application Overview ![winch](https://jamesonstone.io/assets/images/blog/winch-overview.png) We’ll setup instrumentation and polymer to send a continuous flow of messages into fast-data-dev (kafka) where we can consume them by winch and print “received!” when we get a new message Getting Started Let’s make sure we can get messages into fast-data-dev in an automated fashion first. ``` ./scripts/startup ``` This should start up all required containers and register the debezium connector with kafka-fast-data-dev. Now e can navigate to localhost:3030 and check the connector-ui to make sure we’ve registered correctly. ![topic-ui](https://jamesonstone.io/assets/images/blog/topic-ui.png) Now that we’ve verified the connector’s attached correctly. Let’s move on to kicking off instrumentation with polymer. ``` ./gradlew runenv ``` To get the most control out of our test environment, we’ll run instrumentation locally. To do this, clone instrumentation and run the following command from the root directory: ``` make run ``` ![instrumentation](https://jamesonstone.io/assets/images/blog/instrumentation.png) Ok now we can see message generated by instrumentation via polymer, next we can move to actually writing the consumer. We’re going to start with the most simple and straight forward example from around the internet. Using [this](https://www.confluent.io/blog/apache-kafka-spring-boot-application/) blog post from the guys over at [confluent](https://www.confluent.io/) to get started: We create a service to consume the simple-message-topic message ``` package com.jstone28.winch.services import org.springframework.kafka.annotation.KafkaListener; import org.springframework.stereotype.Service; @Service class SimpleMessageTopicConsumer { @KafkaListener(topics = ["simple-message-topic"], groupId = "group_id") fun processMessage(message: String) { println("got message:" + message) } } ``` - @Service - “an operation offered as an interface that stands alone in the model, with no encapsulated state” (Evans, 2003); also a specialization of @Component allowing for implementation classes to be autodetected through classpath scanning - SimpleMessageTopicConsumer - name of class - @KafkaListener - “Annotation that marks a method to be the target of a Kafka message listener on the specified topics.” [Springboot docs](https://docs.spring.io/spring-kafka/api/org/springframework/kafka/annotation/KafkaListener.html#groupId--) - topic - the topic for which the KafkaListener should listen for. - groupId - override default group.id property for the consumer factory of this value for this listener only - processMessage name of our function With this consumer, we can take advantage of the built-in springboot functionality to process message found ![kafka consumer](https://jamesonstone.io/assets/images/blog/instrumentation.png) ## Getting Started with Kubeless Source: https://jamesonstone.io/blog/2020/02/26/kubeless.html Date: 2020-02-26T17:00:00+00:00 [Kubeless](https://github.com/kubeless/kubeless) is a Kubernetes native serverless framework. It works by creating a “function” Custom Resource Definition (CRD). These functions are serverless functions that execute when called. Installing Install the kubeless cli for function deployment ``` brew install kubeless ``` Create the namespace and function CRD from kubeless ``` kubectl create ns kubeless kubectl create -f https://github.com/kubeless/kubeless/releases/download/v1.0.6/kubeless-v1.0.6.yaml ``` Quick Start Deploy the hello function into the cluster ``` kubeless function deploy hello --runtime python2.7 \ --from-file src/hello.py \ --handler test.hello ``` Open a proxy to localhost:8080 ``` kubectl proxy -p 8080 & ``` Create a test call ``` curl -L --data '{"Another": "Echo"}' \ --header "Content-Type:application/json" \ localhost:8080/api/v1/namespaces/default/services/hello:http-function-port/proxy/ ``` View the function pod logs ``` 10.244.0.1 - - [23/Feb/2020:22:34:07 +0000] "GET /healthz HTTP/1.1" 200 2 "" "kube-probe/1.15" 0/117 {'event-time': None, 'extensions': {'request': }, 'event-type': None, 'event-namespace': None, 'data': {u'Another': u'Echo'}, 'event-id': None} 10.244.0.1 - - [23/Feb/2020:22:34:37 +0000] "POST / HTTP/1.1" 200 19 "" "curl/7.64.1" 0/9402 10.244.0.1 - - [23/Feb/2020:22:34:37 +0000] "GET /healthz HTTP/1.1" 200 2 "" "kube-probe/1.15" 0/143 ``` Sources [Kubeless Quickstart](https://kubeless.io/docs/quick-start/) ## [Pt 2] Installing and Configuring the Kubernetes Dashboard with Kind Source: https://jamesonstone.io/blog/2020/02/22/pt2-k8s-dashboard.html Date: 2020-02-22T17:00:00+00:00 ![Photo by Steve Harvey on Unsplash](https://jamesonstone.io/assets/images/blog/part2-k8s-dashboard.png) In [Part 1](https://jamesonstone.io/blog/2020/02/22/pt1-k8s-dashboard.html) we got a Kubernetes Dashboard up and running along side a small application called [kuard](https://github.com/kubernetes-up-and-running/kuard) to visualize what was happening in the cluster. In this post we’ll explore the best practices when working with the Kubernetes Dashboard. Before getting on, I would be remiss not to mention a blog post by Joe Beda on the topic of the [Kubernetes Dashboard](https://blog.heptio.com/on-securing-the-kubernetes-dashboard-16b09b1b7aca) . He goes in depth on the different aspects of configuring the dashboard and the “gotchas” associated with lenient security practices. There was also an excellent [TGIK](https://www.youtube.com/watch?v=od8TnIvuADg) also done by Joe Beda covering this topic. Authentication and the Kubernetes Dashboard Before we discuss any sort of authentication pattern, we should first and foremost declare that the Kubernetes Dashboard should never be exposed outside the cluster. Only those with cluster access, should be able to reach the dashboard. ![dashboard auth](https://jamesonstone.io/assets/images/blog/dashboard1.png) Authentication with the Kubernetes Dashboard can be handled in many ways. The two that were given at log in is via kubeconfig, and the second by token. The kubeconfig is available (by default) in the home directory usually under the ~/.kube/config path. The token can be acquired by searching the secret associated with the service account used to access the cluster. The kubernetes dashboard does integration with OIDC which means github, google, facebook, and other auth providers can be configured to accept or deny user access. How to handle authentication is up to your organization, and since the early days of the dashboard, there have been many improvements to its security. Just remember the golden rule: never, ever open access to the dashboard outside the cluster. General Usage I won’t go to deeply here since how you use the dashboard is entirely up to you, however, I will point out a feature that I’m sure you’re interested in. That is, in pod exec. Before we start, I should mention a trend I’ve seen amongst new Kubernetes users; that trend is the wanting to treat Kubernetes as a set of servers rather than a general pool of resources. Because of this perspective, new Kubernetes users tend to want to log into every pod and execute actions there. This is not an advisable technique because pods are ephemeral. You can expect them to be destroyed and recreated at a feverish pace. Changes must be made at the infrastructure (infrastructure as code) level so that they continue as pods and nodes are created and destroyed. ![exec](https://jamesonstone.io/assets/images/blog/exec.png) This feature is protected by RBAC service accounts so be sure that you have the correct access before attempting to run commands in the pod. Conclusion The Kubernetes Dashboard is an interesting tool that is easy enough to deploy and grok. But, like most things in the Kubernetes world, if you’re looking for something specific, you should probably find and deploy something specific. All that having been said, if you’re only looking for general information about your cluster, look no further than the Kubernetes Dashboard. ## [Pt 1] Installing and Configuring the Kubernetes Dashboard with Kind Source: https://jamesonstone.io/blog/2020/02/22/pt1-k8s-dashboard.html Date: 2020-02-22T17:00:00+00:00 ![Photo by Marek Szturc on Unsplash](https://jamesonstone.io/assets/images/blog/dashboard.png) This [process](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/) has been well documented by the various Kubernetes teams. My overview is basically replicant with details that I deem valuable. Under no circumstances is this a definitive guide [Kind](https://github.com/kubernetes-sigs/kind) is Kubernetes in Docker and it’s a great alternative to using a cloud provider for a quick and dirty kubernetes instance. There are, of course, some shortcomings with using kind but they’re not going to impede us in our quest to install and configure the kubernetes dahsboard. Kubernetes [Dashboard](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/#deploying-containerized-applications) is a Web User Interface for a Kubernetes cluster. It has long been controversial because of its proneness to vulnerabilities. In this blog, we’re going to install, lock down, and configure the Kubernetes. Getting Started Let’s start by creating a kind cluster. If you don’t have kind installed, you can install it with brew: ``` brew install kind ``` Next let’s start up a new cluster ``` kind create cluster ``` And then we’ll point our kubeconfig at the newly created cluster ``` export KUBECONFIG="$(kind get kubeconfig-path --name="kind")" ``` Great, now we have a kind cluster and we’re ready to get installing Installation To deploy the Kubernetes dashboard (not enabled by default), we’ll apply yaml from the Kubernetes project itself ``` kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta8/aio/deploy/recommended.yaml ``` This yaml can be found at the [URL](https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta8/aio/deploy/recommended.yaml) listed above. Let’s quickly breakdown what’s being installed: Create a new [Namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) called kubernetes-dashboard ``` apiVersion: v1 kind: Namespace metadata: name: kubernetes-dashboard ``` Create a [Service Account](https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/) in the newly create kubernetes-dashboard namespace called kubernetes-dashboard ``` apiVersion: v1 kind: ServiceAccount metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard ``` Create a new [Service](https://kubernetes.io/docs/concepts/services-networking/service/) for the dashboard using a [clusterIP](https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types) type (IP only available within the cluster) ``` kind: Service apiVersion: v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard spec: ports: - port: 443 targetPort: 8443 selector: k8s-app: kubernetes-dashboard ``` Next we create three [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) and a [ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/) . Our secretes are listed as opaque meaning they can be arbitrary sets of key value pairs. The secrets define kubernetes-dashboard-certs, kubernetes-dashboard-csrf, and kubernetes-dashboard-key-holder respectively. The ConfigMap simply defines an empty map of kubernetes-dashboard-settings. We’ll explore what these are later in the configuration section. ``` --- apiVersion: v1 kind: Secret metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-certs namespace: kubernetes-dashboard type: Opaque --- apiVersion: v1 kind: Secret metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-csrf namespace: kubernetes-dashboard type: Opaque data: csrf: "" --- apiVersion: v1 kind: Secret metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-key-holder namespace: kubernetes-dashboard type: Opaque --- kind: ConfigMap apiVersion: v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-settings namespace: kubernetes-dashboard ``` Create a [Role](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#default-roles-and-role-bindings) for in cluster RBAC best practices ``` kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard rules: # Allow Dashboard to get, update and delete Dashboard exclusive secrets. - apiGroups: [""] resources: ["secrets"] resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"] verbs: ["get", "update", "delete"] # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map. - apiGroups: [""] resources: ["configmaps"] resourceNames: ["kubernetes-dashboard-settings"] verbs: ["get", "update"] # Allow Dashboard to get metrics. - apiGroups: [""] resources: ["services"] resourceNames: ["heapster", "dashboard-metrics-scraper"] verbs: ["proxy"] - apiGroups: [""] resources: ["services/proxy"] resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"] verbs: ["get"] ``` Create a [ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) which gives the dashboard access to all namespaces ``` kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard rules: # Allow Metrics Scraper to get metrics from the Metrics server - apiGroups: ["metrics.k8s.io"] resources: ["pods", "nodes"] verbs: ["get", "list", "watch"] ``` Next, create the [RoleBinding](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding) for our [Service Account](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) and [ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) ``` --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: kubernetes-dashboard subjects: - kind: ServiceAccount name: kubernetes-dashboard namespace: kubernetes-dashboard --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: kubernetes-dashboard roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: kubernetes-dashboard subjects: - kind: ServiceAccount name: kubernetes-dashboard namespace: kubernetes-dashboard ``` Finally, we arrive at the dashabord [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) . This is the piece that actually deploys our dashboard. ``` kind: Deployment apiVersion: apps/v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard spec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: k8s-app: kubernetes-dashboard template: metadata: labels: k8s-app: kubernetes-dashboard spec: containers: - name: kubernetes-dashboard image: kubernetesui/dashboard:v2.0.0-beta8 imagePullPolicy: Always ports: - containerPort: 8443 protocol: TCP args: - --auto-generate-certificates - --namespace=kubernetes-dashboard # Uncomment the following line to manually specify Kubernetes API server Host # If not specified, Dashboard will attempt to auto discover the API server and connect # to it. Uncomment only if the default does not work. # - --apiserver-host=http://my-address:port volumeMounts: - name: kubernetes-dashboard-certs mountPath: /certs # Create on-disk volume to store exec logs - mountPath: /tmp name: tmp-volume livenessProbe: httpGet: scheme: HTTPS path: / port: 8443 initialDelaySeconds: 30 timeoutSeconds: 30 securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true runAsUser: 1001 runAsGroup: 2001 volumes: - name: kubernetes-dashboard-certs secret: secretName: kubernetes-dashboard-certs - name: tmp-volume emptyDir: {} serviceAccountName: kubernetes-dashboard nodeSelector: "beta.kubernetes.io/os": linux # Comment the following tolerations if Dashboard must not be deployed on master tolerations: - key: node-role.kubernetes.io/master effect: NoSchedule ``` The Installation doesn’t leave us shorthanded. As its last two items, it provides a metrics scrapper and metrics dashboard for monitoring the cluster and dashboard. ``` kind: Service apiVersion: v1 metadata: labels: k8s-app: dashboard-metrics-scraper name: dashboard-metrics-scraper namespace: kubernetes-dashboard spec: ports: - port: 8000 targetPort: 8000 selector: k8s-app: dashboard-metrics-scraper --- kind: Deployment apiVersion: apps/v1 metadata: labels: k8s-app: dashboard-metrics-scraper name: dashboard-metrics-scraper namespace: kubernetes-dashboard spec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: k8s-app: dashboard-metrics-scraper template: metadata: labels: k8s-app: dashboard-metrics-scraper annotations: seccomp.security.alpha.kubernetes.io/pod: 'runtime/default' spec: containers: - name: dashboard-metrics-scraper image: kubernetesui/metrics-scraper:v1.0.1 ports: - containerPort: 8000 protocol: TCP livenessProbe: httpGet: scheme: HTTP path: / port: 8000 initialDelaySeconds: 30 timeoutSeconds: 30 volumeMounts: - mountPath: /tmp name: tmp-volume securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true runAsUser: 1001 runAsGroup: 2001 serviceAccountName: kubernetes-dashboard nodeSelector: "beta.kubernetes.io/os": linux # Comment the following tolerations if Dashboard must not be deployed on master tolerations: - key: node-role.kubernetes.io/master effect: NoSchedule volumes: - name: tmp-volume emptyDir: {} ``` After applying these manifests, you should see the following outputted in the CLI ``` $→ kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta8/aio/deploy/recommended.yaml namespace/kubernetes-dashboard created serviceaccount/kubernetes-dashboard created service/kubernetes-dashboard created secret/kubernetes-dashboard-certs created secret/kubernetes-dashboard-csrf created secret/kubernetes-dashboard-key-holder created configmap/kubernetes-dashboard-settings created role.rbac.authorization.k8s.io/kubernetes-dashboard created clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created deployment.apps/kubernetes-dashboard created service/dashboard-metrics-scraper created deployment.apps/dashboard-metrics-scraper created ``` We can attach to the dashboard by creating a proxy into the cluster. Kubectl handles all the plumbing, and gives us a nice window on our own localhost into the cluster. ``` kubectl proxy ``` The dashboard will be available after the proxy is created at: http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/ ![auth method](https://jamesonstone.io/assets/images/blog/dashboard1.png) Let’s get started with the Service Accounts Bearer Token ``` kubectl -n kube-system describe secret $( kubectl -n kube-system get secret | \ awk '/^deployment-controller-token-/{print $1}' ) | \ awk '$1=="token:"{print $2}' ``` Installing an App Let’s get started by deploying an application on to our cluster so we can see things move around on the dashboard. A good place to start is with [kuard](https://github.com/kubernetes-up-and-running/kuard) . ``` kubectl run --restart=Never --image=gcr.io/kuar-demo/kuard-amd64:blue kuard kubectl port-forward kuard 8080:8080 ``` Now let’s switch back to the dashboard ``` http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/ ``` We can see the pod that’s been spun up for kaurd ![dashboard2](https://jamesonstone.io/assets/images/blog/dashboard2.png) From here we can mess with Kuard’s features to simulate different use cases within the cluster ![kuard](https://jamesonstone.io/assets/images/blog/kuard.png) Conclusion We’ll follow up in [Part 2](https://jamesonstone.io/blog/2020/02/22/pt2-k8s-dashboard.html) configuring the dashboard. ## Docker Compose, Kotlin, and Kafka: Why Can't I Connect Source: https://jamesonstone.io/blog/2020/02/15/compose-kafka.html Date: 2020-02-15T17:00:00+00:00 ![Photo by Clint Adair on Unsplash](https://jamesonstone.io/assets/images/blog/compose-kafka-connect.png) Building a simple docker-compose environment to test docker environments is the bread and butter of application development. It’s usually the first step in unifying development amongst a team of engineers. This week, I had to do just this very thing for a PoC I was developing. [Polymer](https://gitlab.com/jstone28/polymer) was the outcome of the effort. The general idea behind the PoC is creating a “translation layer” between an application firing events ([instrumentation](https://gitlab.com/jstone28/instrumentation) ) and an instance of kafka running a consumer watching a single topic. A visual representation of this architecture looks like this: ![polymer overview](https://jamesonstone.io/assets/images/blog/polymer-overview.png) The Problem ``` Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient) ``` There seems to be an inability to access Kafka from any other application on the docker generated network. Here’s our starting docker-compose.yml ``` version: "3.1" services: zookeeper: container_name: zookeeper image: confluent/zookeeper ports: - 2181:2181 - 2888:2888 - 3888:3888 kafka: image: debezium/kafka:1.0 container_name: kafka hostname: kafka ports: - 9092:9092 environment: KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://0.0.0.0:9092 KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092 ZOOKEEPER_CONNECT: zookeeper:2181 volumes: - /var/run/docker.sock:/var/run/docker.sock depends_on: - zookeeper polymer: build: context: . dockerfile: ./Dockerfile container_name: polymer ports: - 8080:8080 depends_on: - kafka instrumentation: image: registry.gitlab.com/jstone28/instrumentation container_name: instrumentation depends_on: - polymer ``` Our end to end flow is to have instrumentation call into polymer which will then publish the event to a Kafka topic (simple-message-topic). This mimics a board scope of use cases, but it’s really meant to exercise polymer, to ensure a translation layer can be built and maintained. To get things rolling, Let’s start up Kafka and Zookeeper (kafka is dependent on zookeeper so calling up on kafka will result in both coming up) ``` docker-compose up kafka ``` Next, let’s open a new shell and open a consumer via the kafka shell scripts watching the topic: simple-message-topic ``` docker-compose exec kafka /kafka/bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic simple-message-topic --from-beginning ``` Once this is open, we should see the consumer shell spit out: ``` [2020-02-15 21:15:09,477] WARN [Consumer clientId=consumer-console-consumer-49289-1, groupId=console-consumer-49289] Error while fetching metadata with correlation id 2 : {simple-message-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) ``` Now that we have a consumer setup, let’s open up another shell and start firing events. Start Up Polymer ``` docker-compose up polymer ``` We can have instrumentation fire events to polymer or we can kick them off manually by making a POST request through [POSTMAN](https://docs.api.getpostman.com/?version=latest) with a JSON body. In our case we’re using: ``` { "message": "simple-message-message" } ``` If you attempt that POST request now, you’ll see and endless stream of the following logs in the Polymer output: ``` polymer| 2020-02-15 21:22:23.673 INFO 90 --- [main] c.jstone28.polymer.PolymerApplicationKt: Started PolymerApplicationKt in 3.416 seconds (JVM running for 4.06) ``` So what’s going on here? Let’s pull an image from Robin Moffatt’s [blog](https://rmoff.net/2018/08/02/kafka-listeners-explained/) ![kafka](https://jamesonstone.io/assets/images/blog/kafka.png) This helps pull together some ideas. There’s another line in the blog that points out something I’ve been overlooking: LISTENERS are what interfaces Kafka binds to. ADVERTISED_LISTENERS are how clients can connect. That statement, more or less, points out what I’ve been overlooking. I don’t understand Kafka well enough. I am arbitrarily setting environment variables and hoping that it works. Time to do some reading. Let’s go at this from the other side, we know that the Springboot module instantiating kafka is trying to do so at localhost. Instead, let’s update the configuration to point to kafka:9092. Before now, I didn’t know about resource/application.yml or resource/application.properties but this solved the issue. ``` spring: kafka: producer: bootstrap-servers: kafka:9092 key-serializer: org.apache.kafka.common.serialization.StringSerializer value-serializer: org.apache.kafka.common.serialization.StringSerializer ``` And because we solved it from the Springboot module’s perspective, we can revert the original docker-compose.yml ``` version: "3.1" services: zookeeper: container_name: zookeeper image: confluent/zookeeper ports: - 2181:2181 - 2888:2888 - 3888:3888 kafka: image: debezium/kafka:1.0 container_name: kafka hostname: kafka ports: - 9092:9092 environment: ZOOKEEPER_CONNECT: zookeeper:2181 volumes: - /var/run/docker.sock:/var/run/docker.sock depends_on: - zookeeper polymer: build: context: . dockerfile: ./Dockerfile container_name: polymer ports: - 8080:8080 depends_on: - kafka instrumentation: image: registry.gitlab.com/jstone28/instrumentation container_name: instrumentation depends_on: - polymer ``` Note that we reduced the number of kafka environment variable Now, when we send a message with a POST, we can see the consumer echo: ``` { "message": "simple-message-message" } ``` ## On Team Chat Channels in Slack Source: https://jamesonstone.io/blog/2020/02/01/team-slack.html Date: 2020-02-01T17:00:00+00:00 ![chat](https://jamesonstone.io/assets/images/blog/slack.png) Photo by Pankaj Patel on Unsplash I had a conversation in a live meeting that triggered a Proustian moment of a team long left in the past. The conversation was specifically in regards to the purpose of Team Channels in Slack. And, the Proustian moment was one of forgotten fortune… Early in my career I joined a start up, and boy was I lucky to join this start up in particular, because this team was young, motivated, and talented. I credit the experience for advancing my career a decade in three years. The team itself was so gung-ho because, well, we didn’t know any better. We were all at the stage in life where work was the most important thing. Many of us just having finished college (or, in my case, still wrapping it up). The entire team was co-located so we did a large portion of our speaking in the trenches together, however, off hours, as few and far between as they were, brought more communication in the form of IRC and, eventually, Hipchat. There were many different chat communication styles at that organization ranging from me (at the time) saying very very little in fear of making a mistake (which I made plenty of in due course) to one team member who would, almost literally, stream his consciousness into chat throughout the day. My communication style was bore mostly out of ignorance. Ignorance for good software development and not wanting everyone to know. And, likewise, ignorance of the importance of communication. This team member, who constantly streamed his consciousness into the chat, became an exemplar. At the time, I sort of took for granted that there would be a stream of new messages in the team channel almost as if someone were having a jovial conversation of maximal importance which included questions, opinions, jokes, stories, and rapport with the occasional response to a comment. If we fast forward today, I now realize that this teammate’s modus operandi was a strong indicator of a good, healthy team culture. One that’s very rarely found nowadays. I miss it very much. There was an entirely different level of connectedness between our teammates because of this one teammate. He brought together our disparate conversations into a cohesive fabric of self conversation for which we all could enjoy and interact with. In thinking about it further, it feels as though there have been many advances in chat communication with the likes of [Slack](https://slack.com/) , [Mattermost](https://mattermost.com/) , and [Discord](https://discordapp.com/) but with these advances, came a sense of disconnectedness. Whether it comes from a place of ignorance of the importance of communication or, as was previously the case of myself, feeling like an imposter, the result is the same: near silent channels with everyone trying to guess what everyone else is going to say. ∞∞∞ I guess I’m writing this as a confirmation; that the course to chart forward is one of radical openness – clear albeit sometimes murky – conversation that exposes my internal dialog and thinking. Something everyone can look upon as a collective weaving of their thoughts, comments, opinions, questions, jokes. I don’t know where this teammate is nowadays but, in some sense, it doesn’t matter. His examples live on through me. ## Change Data Capture with Debezium, Kafka, Postgres, and Golang Source: https://jamesonstone.io/blog/2020/01/25/debezium.html Date: 2020-01-25T17:00:00+00:00 ![debezium](https://jamesonstone.io/assets/images/blog/debezium.png) [Debezium](https://debezium.io/documentation/faq/) Is a tool used to handle change data capture (CDC). You can think of CDC as an event system for your database. Debezium works by hooking into Postgres’ Write Ahead Logs (WAL) to capture events. It ties itself together with a encode/unencode library to publish json messages to a Kafka topic. Last week I created a PoC to test how consuming debezium messages from a kafka might translate into and end to end application. I’m sad to report that I wasn’t able to entirely complete it (and likely won’t) because it became obvious 5/6ths the way through that the remaining logic was going to make this PoC intractable against its contenders. Overview ![overview](https://jamesonstone.io/assets/images/blog/debezium-overview.png) The purpose of the PoC was to determine if debezium was an appropriate solution for handling interim migration from one database to another database instance by following the events occurring in one database and populating the other with those same events. The project breaks down into two separate repos. One for a producer, [opportunity-rover](https://gitlab.com/jstone28/opportunity-rover) ; and one for the consumer of those messages, [curiosity-rover](https://gitlab.com/jstone28/curiosity-rover) . Opportunity house all of the plumbing in terms of docker-compose environments and general setup. The [Opportunity README](https://gitlab.com/jstone28/opportunity-rover/blob/master/README.md) defines how to get started, but spoiler: you clone and run ./startup The process that gets kicked off does the following: - Starts up Postgres, Kafka, Zookeeper, Debezium (connect), Opportunity, and finally Curiosity. - It then makes a curl call with the debezium information into Kafka’s [Connector](https://docs.confluent.io/current/connect/managing/index.html) feature - A terminal is opened within kafka to watch messages coming through on the topic specified by the connector - Curiosity has already begun watching on that same topic mimicking a consumer ready to push the messages into its own database - A User can make a call to localhost:3030/insert which will prompt a username record insert Outcome The result of the PoC was abandonment. The reason for the abandonment came when we started looking at how the messages were displayed on the topic. For example: ``` Message: {"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"string","optional":true,"field":"username"}],"optional":true,"name":"postgres.public.opportunity.Value","field":"before"},{"type":"struct","fields":[{"type":"string","optional":true,"field":"username"}],"optional":true,"name":"postgres.public.opportunity.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":false,"field":"schema"},{"type":"string","optional":false,"field":"table"},{"type":"int64","optional":true,"field":"txId"},{"type":"int64","optional":true,"field":"lsn"},{"type":"int64","optional":true,"field":"xmin"}],"optional":false,"name":"io.debezium.connector.postgresql.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"}],"optional":false,"name":"postgres.public.opportunity.Envelope"},"payload":{"before":null,"after":{"username":"user-0.4809437189067354"},"source":{"version":"1.0.0.Final","connector":"postgresql","name":"postgres","ts_ms":1579630649193,"snapshot":"false","db":"opportunity","schema":"public","table":"opportunity","txId":494,"lsn":23433960,"xmin":null},"op":"c","ts_ms":1579630649195}} ``` This is the message outputted by debezium and left for consumption by Curiosity. We quickly realized that the logic required to translate this message into a sql statement and (possibly) subsequent API calls was going to be an insurmountable amount against just doing a traditional double-write-with-snapshot effort. Overall it was fairly straight forward save for using kafka with docker containers. Docker for mac does some spooky stuff with localhost, which I could never get really figured out. I broke down and ran everything within the docker-compose network, and there everything was accessible. ## A Post Commit Git Hook for Semantic Version Updates, Docker Build, Tag, and Push Source: https://jamesonstone.io/blog/2020/01/18/post-commit-semver.html Date: 2020-01-18T17:00:00+00:00 ![post](https://jamesonstone.io/assets/images/blog/POSTCOMMIT.png) In our next installment, we’re going to pick up a semantic version, docker build, tag, push off the cutting room floor. This small function runs as a post-commit, that is the function that runs after the commit has been accepted. In the life cycle of git hooks, it runs after the pre-commit hook but before the pre-push. ``` #!/bin/bash # This hook handles our deployment model. It allows us to make a commit # that will build, tag, deploy, and push a new version of PDS to the # container registry. # This hook is required because Jenkins doesn't have enough # permissions to execute these actions this. # VARIABLES: # SEMVER_TYPE - the type of sem ver extracted from the commit # VERSION - the version (after being bumped) to tag and push with # DEPLOYMENT_COMMIT - boolean check of deployment commit # DEPLOY_STRING - "deploy-" DEPLOY_STRING="deploy-" REGISTRY="registry.github.com/jstone28/jstone28.github.io" # check last commit message for [deploy-dev], [deploy-stage], [deploy-prod] DEPLOYMENT_COMMIT=$(git log -1 --pretty=oneline --abbrev-commit) ## update VERSION file update_version() { # evaluate the current version at runtime CURRENT_MAJOR_VERSION=$(awk -F'.' '{print FILENAME, $1}' < VERSION | awk '{$1=$1};1') CURRENT_MINOR_VERSION=$(awk -F'.' '{print FILENAME, $2}' < VERSION | awk '{$1=$1};1') CURRENT_PATCH_VERSION=$(awk -F'.' '{print FILENAME, $3}' < VERSION | awk '{$1=$1};1') case $SEMVER_TYPE in major) echo $((CURRENT_MAJOR_VERSION+1))."$CURRENT_MINOR_VERSION"."$CURRENT_PATCH_VERSION" > VERSION ;; minor) echo "$CURRENT_MAJOR_VERSION".$((CURRENT_MINOR_VERSION+1))."$CURRENT_PATCH_VERSION" > VERSION ;; patch) echo "$CURRENT_MAJOR_VERSION"."$CURRENT_MINOR_VERSION".$((CURRENT_PATCH_VERSION+1)) > VERSION ;; esac } ## Git tag with VERSION representation git_tag() { git tag v"$VERSION" git push origin --tags # must be after the deploy+1 commit } ## commit version updates git_commit() { git add . # we add anything here so that we can include CHANGELOG git commit -m ":rocket: version updates for deployment v${VERSION}" --no-verify } ## docker build with version tag and latest docker_build() { docker build -t "$REGISTRY":"$VERSION" -t "$REGISTRY":latest . } ## docker push docker_push() { docker push "$REGISTRY":"$VERSION" docker push "$REGISTRY":latest } ##################### # pre-push function # ##################### if [[ "$DEPLOYMENT_COMMIT" == *"$DEPLOY_STRING"* ]]; then echo "" echo "Deployment commit detected. Performing tag, build, and push..." echo "" SEMVER_TYPE=$(echo "$DEPLOYMENT_COMMIT" | sed -e 's/.*\[\(.*\)\].*/\1/' | cut -f2- -d-) update_version VERSION=$(cat VERSION) git_commit git_tag docker_build docker_push else echo "" echo "Deployment commit not detect." echo "" fi ``` The purpose behind this git hook is as the comment describes. Jenkins, at least with the version of credentials I’ve been delegated, doesn’t have enough runtime permissions to execute these actions. This git hook is clearly meant as pipeline logic but because of those permission, they’ve become repo logic. Let’s walk through what’s going on: First and fore most we setup some variables and grab the commit message to store in a variable. Next we declare a few functions to use in our main. The comments of these respective commands define their actions. Next we declare our main function. Our first check is that the commit message defines the deploy string. That deploy string is the one we defined at the top of the file deploy- in practice, however, the conditional here is checking for anything as a delineator as well (we used [deploy-patch/minor/major] for example [deploy-patch]). If a the deploy string isn’t detectd, the whole post-commit is skipped and the message Deployment commit not detected. is printed after the commit. The deploy string is unique enough that the chances of it being used in normal course are fairly low; and it discrete enough that it doesn’t drastically change the commit message. An example deployment commit message might look like: ``` git commit -m ":sparkles: added new create/ endpoint to application [deploy-minor]" ``` Once the deploy string is detected, a deluge of commands take place: First and foremost, we extract the Semantic Version type. This will allow up to update the VERSION file appropriately, with the correct Semantic Version update, based on the change made. Next we set the new version as the VERSION variable and commit the updated Semantic Version bump: git commit -m ":rocket: version updates for deployment v${VERSION}" --no-verify We run this command as --no-verify to avoid any redundant checking based on any other git hooks that may be active (note: because this commit doesn’t contain the deploy string, it won’t trigger this specific git hook again). After we’ve committed the VERSION file bump, we git tag that version as well. After we’ve git tagged it, we push the tags to the remote. In our final steps of this post-commit hook we build and tag with the VERSION file’s contents, and finally we push to the container registry. To Sum up We’ve done the following: - Declared, in a commit message, that we’re ready to deploy this version - Bumped the Semantic Version according to that same commit message - Updated the VERSION file to house the new Semantic Version - Committed the new Semantic Version to the remote - git tagged the latest version with the correspondingly updated VERSION file contents - docker built and tagged the image with the new version - docker push the newly built and tagged version to the remote registry That was a lot but I hope it was helpful. I think I have one more iteration of this change from a different approach. I may add that one next. /J ## Why Management Structures Dictate Software Architecture Source: https://jamesonstone.io/blog/2020/01/18/manage-struct.html Date: 2020-01-18T17:00:00+00:00 ![structure](https://jamesonstone.io/assets/images/blog/MANAGE_STRUCT.png) [Conway’s Law](https://en.wikipedia.org/wiki/Conway%27s_law) a perennial favorite for being quoted around the office defines how software architecture tends towards the communication (and by extension, the management structure) of the organization […] organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations. — M. Conway Why is it that this law from the 1960 still holds true today? I think I’ve gotten some where in my thinking about why that is. Unfortunately, it’s not all that innovative or surprising but using [Occam’s Razor](https://en.wikipedia.org/wiki/Occam%27s_razor) seems to be the least amount of amount of over head to setup. Put simply, the reason why Conway’s Law holds true comes down to consolidation of power by individuals within a hierarchy. Let’s back up a second and add some context. Generally speaking, there are two career tracks for the software engineer: individual contributor and engineering manager. The first is what it says on the tin but the second is a little more nuanced. Technically speaking, an engineering manager should be parallel in the hierarchy from the individual contributor, however, often this dynamic is changed or exploited. Either by an over-eager engineering manager or individual contributors looking to abdicate responsibility. Either way, a small number of individuals end up making the decision for the organization. By virtue of that fact, software architectures grow in the directions they’re allotted. Let’s take an example from my past experience: I worked at a large organization many years ago that was tasked with creating a content management system. The organization of the two teams was two heads of the project (one individual contributor, one engineering manager), spread out across two floors 2nd and 24th, and one siloed to the API while the other was to create consumers of the API to do various tasks. That second team (the team I was on) was broken up into many sub-teams building various features. Now, based on the pretense, how do you think the project laid itself out? If you guessed that the API ended up doing things nobody really needed, and had features nobody really used whilst, on the other side of the proverbial fence, the API consumers ended up building applications that used the API in nonsensical ways, and often had to rig a bunch of scaffolding and additional small extraneous applications to get their desired feature set. But how did we get there? One step at a time, really. At the top of our management hierarchy, we had two leaders who weren’t cooperating. They also weren’t not cooperating but it was enough to have a trickle down effect to us boots on the ground. We worked independent of our cohorts on the API team and ended up with a set of applications that weren’t really suited for the API; likewise, we ended up with an API that served something other than what we were doing. When I stood back, a year after the project had been in maintenance, I could have easily drawn you a map of the command structure just based on how the application worked. I was almost depressed when acknowledging that fact lo those many years ago. And the fact that I still see teams making this same mistake today. It’s like the realization you have when you realize “Peopleware” was written in 1987 but everything it outlines still exists today, some 33 years later. /J ## A Semantic Versioning Function for the Makefile Source: https://jamesonstone.io/blog/2020/01/17/makefile-semver.html Date: 2020-01-17T17:00:00+00:00 ![make](https://jamesonstone.io/assets/images/blog/MAKE_SEMVER.png) I’ve been spending a lot of time in Jenkins lately; most certainly against my will. Jenkins offers all of the downfalls of a decentralized open source project: - Disjointed road map - Feature incomplete - Difficult to decipher UI - Multiple implementations of the same feature - Redundant methods within a single feature - Much much more! It’s because of this that I’ve had to move a lot of the logic I would normally keep in the pipeline, into the project itself. Such as Semantic Versioning Function: ``` .PHONY: increment-version increment-version: @echo -n "Add CHANGELOG entry? Y to exit for entry creation [Y/n] " && read ans && [ $${ans:-N} = n ] @# evaluate the current version at runtime $(eval CURRENT_MAJOR_VERSION := $(shell awk -F'.' '{print FILENAME, $$1}' < VERSION)) $(eval CURRENT_MINOR_VERSION := $(shell awk -F'.' '{print FILENAME, $$2}' < VERSION)) $(eval CURRENT_PATCH_VERSION := $(shell awk -F'.' '{print FILENAME, $$3}' < VERSION)) # major ifeq ($(SEMVER),$(filter $(SEMVER), MAJOR major)) @echo $$[$(CURRENT_MAJOR_VERSION)+1].$(ZERO).$(ZERO) > VERSION endif # minor ifeq ($(SEMVER),$(filter $(SEMVER), MINOR minor)) @echo $(CURRENT_MAJOR_VERSION).$$[$(CURRENT_MINOR_VERSION)+1].$(ZERO) > VERSION endif # patch ifeq ($(SEMVER),$(filter $(SEMVER), PATCH patch)) @echo $(CURRENT_MAJOR_VERSION).$(CURRENT_MINOR_VERSION).$$[$(CURRENT_PATCH_VERSION)+1] > VERSION endif @echo "version updated." ``` The first prompt asks the user if they want to enter a CHANGELOG entry. If they user answers yes, the make command is exited with an error. This allows the user to make the change to the CHANGELOG before the versioning takes place. Alternatively, if the user answers that they do not need to enter a CHANGELOG entry, the rest of the command will execute. This function works in conjunction with a VERSION file which contains a plain Semantic Version such as 0.0.3 ``` make increment-version SEMVER=patch/minor/major ``` The type of change is set to the SEMVER variable and that prompts the update mechanism to make the corresponding change. For example: If the current version is 1.0.3 and we run the following command: ``` make increment-version SEMVER=patch ``` The resulting version in the VERSION file will be 1.0.4. If we again run the increment-version command: ``` make increment-version SEMVER=minor ``` The resulting version then will be 1.1.0. If we run the increment-version command a final time: ``` make increment-version SEMVER=major ``` The VERSION file will read 2.0.0 I hope this snippet is useful to someone. I didn’t spend too much time putting it together but it’s, at least, the 10th time I’ve written something like this. /J ## What Weather Forecasts Tell Us About Progress and Technology Source: https://jamesonstone.io/blog/2020/01/12/weather-pre.html Date: 2020-01-12T17:00:00+00:00 ![weather](https://jamesonstone.io/assets/images/blog/WEATHER.png) I was sitting next to the fire listening to the rain when it struck me: weather forecasts seem to be getting less and less accurate. In fact, even the most popular of applications is occasionally 10+ degrees wrong. It runs contradictory to what seems intuitive; there are clearly more satellites, more organizations, more people looking at the skies than ever before. How could that contribute to a less accurate report? The answer lies in a couple of factors: - Too much data (Signal v. Noise) - The need to grab the headline - The fear of being wrong (or unprepared) Too Much Data (Signal v. Noise) The end of the 2010’s saw the rise of data; data in every form, from every avenue, about everything. As a result, organizations all over the world took a hard look their monitization strategy and realized they were leaving money on the table. Every interaction with a client now held multiple realized potentials: one for the product, one for the customer, and one for any data inadvertently given away in the process. All this data, regardless of relative importance is labeled as “high priority” and then handed over to Data Engineering. Data Engineering is an interesting field because of its fatal mix of statics, machine learning, and probability, without any practical expertise in psychology, product development, or human-computer interaction. What we will some day come to realize is that quality of data matters. And that data interpretation is just as important as the data itself. Data Engineering shouldn’t be entirely responsible for attempting to rationalize data nor should they be given that responsibility. Rather, a team of individuals with radically different backgrounds should come to own the process of interpreting data. This is the only sustainable solution in the fight to separate signal from noise. The Need to Grab the Headline When Michael Jackson passed away, LA news was abuzz with activities but no organization (NBC, ABC, CBS, local) was willing to report news they couldn’t confirm. You could say this is a feat of journalistic integrity or due diligence, or even simple CYA protocols; whatever it was, it left the door open. TMZ, never being the one to let integrity get in the way of reporting, reported on the death of Michael Jackson almost immediately. It wasn’t until several hours later did the other organizations follow suite. The outcome of that whole debacle? TMZ became a nationally recognized source of news. The traditional news outlets must have taken a long hard look at themselves and realized that, to remain relevant among today’s unprecedented media interconnectedness they needed to report the biggest headlines they could for every story they reported on. Evidence of this is the level of insanity that weather terms have reached. A few examples I’ve heard recently: Gustnado, Firenado, Sea Smoke, Snowmageddon, Winter Storm - X, Superstorm, and on and on. These terms grab the attention of the passerby; often, they give substance to an otherwise benign event. In the past 10 years, I’ve been a part of almost 10 different weather events but only 1 ever really amounted to anything for my region (Sandy). To bring this all home, the effects of attempting to grab the headline as quickly as possible means that the news is occasionally wrong at a higher percentage than it has been historically. Fear of Being Wrong (or unprepared) Weather presents two vitally important attributes that drive news outlets to this problem (the fear of being wrong): outcomes and deadlines. Those two items are the fear and loathing of every news writer and reporter around the world. Why? Because it means it’s the civic duty of news outlets to ensure you’re informed prior to a big weather event. The issue today is that we stopped paying attention to when the news outlets were wrong. Instead, we only focus on when they are right. That is to say, when the news tells you there’s going to be 5 feet of snow and sub-zero temperatures, and you walk outside and it’s sunny and 65, nothing happens to that new outlet. In the opposite situation, the news outlet is put on trial, perhaps metaphorically although occasionally in reality. The effect of being too worried about an outcome means that the organization will always err on the side of caution, and therefore almost always wrong. Conclusion The effects of what we do today aren’t known. Some may find solace in that, others: fear. Regardless of your reaction, you can arm yourself for the future by simply paying attention: - Does what your working require quality or speed? If only one, which one? - Are you taking calculated risk? Are you pushing what you know to the extent that you can learn more? - Can you separate the Signal from the Noise? Can you see what’s relevant, but more importantly, let go the data that is not? Reality is a complex place: weather, software, news very little of it makes sense in the moment. But, like most things, looking back, it’s the only thing that could have possibly been. /J ## What an IHOP Can Teach About Team Culture Source: https://jamesonstone.io/blog/2019/12/28/i-hop.html Date: 2019-12-28T17:00:00+00:00 ![ihop](https://jamesonstone.io/assets/images/blog/IHOP.png) What can an IHOP teach us about team dynamics and team culture? A lot, actually. This entry isn’t so much a technical view of the Dine Brand Global group, but rather an observation about watching other teams, outside software development react to the same or similar problems. We spend a great deal of time, as engineers, trying to justify why our team problems are more complicated, more difficult than other industries but the fact remains: if you remove software from the equation, you can easily boil down all problems to the same set that every team deals with. I’m seated in the same seat as last time though that makes sense because last time I was here it was roughly the same time. This IHOP, near my home, is always buzzing. It’s an odd blend of a parking lot that’s too small, a restaurant built in the 70s that’s always nearly full, and a busy road to tie it all together. Every time I’m in this IHOP, I feel like I’m a part of a well rehearsed play. The kitchen is at the center of this IHOP – a nervous system of sorts. If you look closely, the cooks actually set the pace for the restaurant. Walk into a poorly run franchise and you’ll notice the cadence of the kitchen isn’t quite right. It’s like a band trying to keep pace with a drummer who’s not keeping tempo. This is fundamentally important to the IHOP team. Namely, that both ends of the process, input and output, function as efficiently as possible. Food cannot sit idle so, if the kitchen is producing, the waitstaff has to be moving quickly. Perhaps it’s breakfast foods that lend itself well to this paradigm or the simple nature of the foods on the IHOP menus, however, because the pace is also set quite high, there’s little time for the waitstaff to do anything else besides be 100% engaged; idle hands, as they say. If you look at this from both perspectives: the kitchen is the heartbeat. The waitstaff is the blood coursing through the body. The waitstaff collects the orders, and hands them to the cooks, the cooks in turn (and moving as efficiently as possible), return the completed orders. The waiters, conscious of idle finished food, rush to return the now completed orders to their respective customers whilst also inputting new orders. The pace is self feeding, it’s self referential. The cooks move fast because the waitstaff is moving fast and the waitstaff moves quickly because of the cooks. Pace is so very important to team culture and team dynamics but it’s often overlooked. As a leader, if someone is telling you to slow down, your response shouldn’t be to slow down. It should be to find people who can keep up with your pace. And, if your mind isn’t flooded with ideas of potentiality, in all likelihood you’re working too slow; however, this is of less focus because the market will inform you of your failings just the same. The short 5 minute wait before my food arrives is punctuated by our polite and skillful waitress keeping a vigilant eye over mine and all the tables in her section. This particular waitress is well into her 50s but I’ve also had the waiter one section over and the one working the back section too. It’s odd to see such uniformity in action without direct communication: you get the same service whether it’s here or there, this waiter or that waitress. For the moment, I’m captivated by our waitress, finding herself a free moment, recleaning and readjusting chairs and tables in the next section over. The idea that sits just below the surface at every IHOP is the idea of decision autonomy. Every person is given complete control over their domain. They act and react to events as they see fit; and because this same decision autonomy exists at every level, it overlays itself creating a military-like precision of leadership. This is a fundamental ingredient to every successful team because it doesn’t just persist in the here and now but extends over time; meaning all new members to the team are instantly indoctrinated, and thus carry on the culture as it’s imparted on them. That’s enough for now. I’m going to finish my food and get out of here. I’ve got pace to keep. /J