Republican leaders at the state and national levels are propelling Turning Point USA, the conservative advocacy group co-founded by Charlie Kirk, further into K-12 schools following the activist’s assassination earlier this month.

The organization, which is known for its presence on college campuses, says it already has more than 1,000 chapters in high schools across the country and 48 representatives on staff meant to help students organize.

Since the 31-year-old’s death, Kirk’s allies have pledged to expand the group’s reach and impact, and the organization has reported a surge in new interest. It received 54,000 inquiries about starting new chapters within days of Kirk’s death, a Turning Point USA spokesperson told CNN last week.

Turning Point USA is also getting an assist from Republican leaders. The U.S. Department of Education last week announced it was partnering with the organization, along with dozens of other conservative groups, to launch a coalition to produce educational programming for schools and universities in advance of America’s 250th birthday next year.

And prior to announcing his resignation Wednesday, Oklahoma’s state education chief vowed this week to put a Turning Point USA chapter into every high school in his state as part of a partnership with the organization.

State Superintendent Ryan Walters—who has pursued a number of conservative causes in public schools, instructing schools to teach the Bible and ordering schools to show a video of him praying for Trump, for example—said it was an effort to “fight back against the liberal propaganda, pushed by the radical left, and the teachers’ unions.”

A news release from the state education department laid out instructions for students on how to start a local chapter.

“These are the highest levels of state and federal government dictating what students will receive in their grades nine through 12 years, and that runs counter to what we’ve been doing in this country,” said Amy Binder, a professor of sociology at Johns Hopkins University who has studied and written extensively about how the left and right engage with students in campus politics.

“I suspect that the superintendent who says that he’s going to mandate that is not mandating Young Democratic Socialists of America chapters in the high schools as well,” she added.

Republican state leaders have also vowed to revoke the licenses of teachers over controversial posts about Kirk.

Turning Point USA did not respond to a request for an interview, though the chief education officer for Turning Point Education, Hutz Hertzberg, said in the U.S. Department of Education announcement of the new civics coalition that the group “is more resolved than ever to advance God-centered, virtuous education for students flourishing across our nation.”

How Turning Point USA appeals to young voters

For decades, politically conservative groups have developed an infrastructure to organize college students at off-campus locations, tell them they’re being indoctrinated on campus, coach them in activism, and help them network with others, Binder said.

By contrast, she said, politically liberal groups have engaged students directly on campus without the same focus on developing expansive, off-campus networks and attending conferences where students’ expenses are covered.

Turning Point USA meaningfully parlayed decades’ worth of Republican strategy into attracting younger people into the conservative fold.

Kirk used social media, podcasts, and YouTube in an attempt to meet young people where they are. In the process, he became a key ally of President Donald Trump.

Kirk also had an extensive history of making comments that critics have deemed racist, sexist, homophobic, and Islamophobic.

Conservative youth activism is more likely to be sponsored by outside organizations, said Hava Gordon, a sociology professor at the University of Denver who has studied youth activism. Turning Point USA provides resources to help students secure teacher sponsors and craft campaigns—which is not typical for youth activism, particularly in high schools, she said.

Making it easier to establish chapters runs in direct contrast to what other student interest groups—such as gay-straight alliances, climate action groups, and student unions—have encountered, Gordon said.

It can be a battle for students to get school support, she said, especially amid Republican efforts that, for years, have sought to restrict educators’ discussions about sensitive political topics through “divisive concepts” laws that are in effect in at least 20 states.

“All of the sudden, we have the right arguing that youth activism is an urgent issue that has to happen when, in fact, most youth activists, I think, have been running into obstacles to organize on their their high school campuses,” she said.

Other organizations elsewhere on the political spectrum provide resources and programming for students and might have high school chapters. For instance, the ACLU this summer staged advocacy institutes that drew hundreds of high school students to Washington for discussion on presidential power, immigration, racial justice, and transgender rights, among other topics. The organization says it has seen a surge of interest during the Trump administration.

But those types of organizations do not use the same tactics as Turning Point USA, which include “teaching students how to be provocative on campus, how to have eyebrow-raised confrontational events on campus, how to invite provocative speakers on campus, how to get under the skin of liberals on campus,” Binder said.

Turning Point USA’s website shows its footprint across the United States, with activism hubs in each region. It describes itself as “promoting freedom-loving, American values. Students champion these initiatives by organizing into student-led chapters and activism hubs.”

The organization supplies an activism kit for local chapters, and provides online resources that include presentations, games, recommended publications, and activity sheets on topics like “Socialism Sus,” “Big gov scares” and “Taxes are shady.”

It shares videos on “hot topics” beyond those themes on “dealing with pushback,” “becoming a leader,” “Should children be able to transition?” and “Feminism, Therapy, and Gold Diggers: What’s Happening to American Women?”

Those resources are not meant to get into the policy weeds. They’re more to “catch people by the throat and get them on your side,” Binder said.

“They’re meant to capture imagination. They’re meant to be provocative. They’re meant to be kind of snarky and easily understood,” she said. “Building a cadre of future leaders who are policy wonks—that’s not what TPUSA is doing. That exists elsewhere in the right ecosystem, but that is definitely not what TPUSA is doing.”

Kirk’s efforts also focused in part on rooting out what he and allies considered left-wing bias in schools and colleges through creating a “professor watch list,” a school board watch list, and launching a private school network in response to “woke” curriculum in public schools. (Researchers have found there’s no rampant left-wing bias among public school teachers “indoctrinating” students into hating the country.)

Kirk’s efforts align with broader conservative pushes

Kirk and his allies realized their organizing and influence activities weren’t just about getting young people out to vote every few years. They’re about helping them construct their identities, Binder said.

“So that’s why they want to be not only on college campuses, but in high schools as well. And it’s effective,” Binder said. “I think the outpouring that we’re seeing now in the wake of Charlie Kirk’s death is a real indicator that the ubiquity and the tone that they’ve used in their videos and their other appearances have been very effective in emotionally just getting the attention of young people.”

And there’s some historical precedent, Gordon said. Youth organizing around Republican Barry Goldwater’s 1964 presidential campaign involved advocating for conservative values more broadly, and many of the activists involved went on to work in politics.

“The Trump administration is looking at Turning Point students as a future inroads into the conservative movement, into the Republican Party, and into Trumpism more specifically,” Gordon said.

Police lured the man to a meeting and arrested him after accessing a private WhatsApp group with colleagues

Want LBC stories before everyone else? Set us as your Preferred Source on Google

By Issy Clarke

An airline employee was arrested by Dubai police after he shared images with colleagues in a private WhatsApp group of bomb damage caused by the Middle East conflict.

Loading audio...

Police accessed the closed WhatsApp group chat, saved the evidence and told the man to come to a meeting before arresting him.

The offending image showed smoke rising above a building after the March 2026 strikes and had only been shared in the private group chat.

He remains in detention on charges including publishing information deemed harmful to state interests, the maximum sentence of which is two years.

Read more: Dubai 'arrests survivors of Iranian drone strike after they sent images of explosion aftermath to loved ones'

Read more: British holidaymaker, 60, arrested in Dubai for 'filming missiles'

Radha Stirling, chief executive of London-based advocacy group Detained in Dubai, said Dubai police had "explicitly confirmed they are conducting electronic surveillance operations capable of detecting private WhatsApp messages."

She said people were being tracked, identified, and arrested not for public statements, but for private exchanges between colleagues."

'Companies like WhatsApp must answer urgent questions about user privacy." she added.

Ms Stirling continued: "If private communications can be detected and used as the basis for arrest by overreaching or hypersensitive states, users worldwide need clarity on how their data is being accessed."

The police report said authorities learned of the material's existence "'through electronic monitoring operations".

A special team from the Electronic and Cybercrime Department was told to find the account holder who shared the video.

The airline worker was tracked down, lured to a meeting and arrested by police.

The case was then escalated to State Security Prosecution. He remains in detention.

The UAE government owns majority holdings in telecom companies Etisalat and Du. This gives security services the power to observe all communications on their networks.

The Arab state has also used the Israeli-developed software Pegasus which allows agents to listen into private calls and read messages, even if they are shared on encrypted apps like WhatsApp,.

The spyware can infect a device even without the user activating a link - such as via a WhatsApp call, even if it isn't answered.

Once inside, it can access all WhatsApp messages, logos and contacts.

Ms Stirling said other tourists, airline crew and residents have reported being detained for sending, receiving or keeping content even when they did not share it.

-- KV: --

Posted 18th April 2026 at 11:59 pm · Follow me on Mastodon, Bluesky, Twitter or subscribe to my newsletter

I’ve spent roughly the last decade and some change as a software engineer, and recently decided to start a solo consultancy.

I’m focused on helping SMEs sort out the messy back-office parts of the business: spreadsheet glue, brittle internal workflows, poor reporting, awkward integrations, backend/platform problems, and AI workflows that need to do real work rather than just look good in a demo.

I’m not really interested in becoming a generic agency. I’d rather work with businesses that already feel operational pain and need someone technical to help untangle it properly.

For those of you who’ve made this jump:

* how did you get your first real project? * what kind of outreach actually worked? * did your first few clients come from network, content, cold outreach, partnerships, subcontracting, or somewhere else?

Also, if anyone knows SMEs or operators dealing with this sort of mess, I’d be glad to chat.

As a gesture of goodwill, I’m offering the first 5 clients 10 hours free to help get an initial project moving.

You can find me over at https://crescita.cc

In this section, we will derive a variant of gcd that is ~2x faster than the one in the C++ standard library.

#Euclid’s Algorithm

Euclid’s algorithm solves the problem of finding the greatest common divisor (GCD) of two integer numbers $a$ and $b$, which is defined as the largest such number $g$ that divides both $a$ and $b$:

This is true, because if $g = \gcd(a, b)$ divides both $a$ and $b$, it should also divide $(a \bmod b = a - k \cdot b)$, but any larger divisor $d$ of $b$ will not: $d > g$ implies that $d$ couldn’t divide $a$ and thus won’t divide $(a - k \cdot b)$.

The formula above is essentially the algorithm itself: you can simply apply it recursively, and since each time one of the arguments strictly decreases, it will eventually converge to the $b = 0$ case.

The textbook also probably mentioned that the worst possible input to Euclid’s algorithm — the one that maximizes the total number of steps — are consecutive Fibonacci numbers, and since they grow exponentially, the running time of the algorithm is logarithmic in the worst case. This is also true for its average running time if we define it as the expected number of steps for pairs of uniformly distributed integers. The Wikipedia article also has a cryptic derivation of a more precise $0.84 \cdot \ln n$ asymptotic estimate.

There are many ways you can implement Euclid’s algorithm. The simplest would be just to convert the definition into code:

int gcd(int a, int b) {
    if (b == 0)
        return a;
    else
        return gcd(b, a % b);
}

You can rewrite it more compactly like this:

int gcd(int a, int b) {
    return (b ? gcd(b, a % b) : a);
}

You can rewrite it as a loop, which will be closer to how it is actually executed by the hardware. It won’t be faster though, because compilers can easily optimize tail recursion.

int gcd(int a, int b) {
    while (b > 0) {
        a %= b;
        std::swap(a, b);
    }
    return a;
}

You can even write the body of the loop as this confusing one-liner — and it will even compile without causing undefined behavior warnings since C++17:

int gcd(int a, int b) {
    while (b) b ^= a ^= b ^= a %= b;
    return a;
}

All of these, as well as std::gcd which was introduced in C++17, are almost equivalent and get compiled into functionally the following assembly loop:

; a = eax, b = edx
loop:
    ; modulo in assembly:
    mov  r8d, edx
    cdq
    idiv r8d
    mov  eax, r8d
    ; (a and b are already swapped now)
    ; continue until b is zero:
    test edx, edx
    jne  loop

If you run perf on it, you will see that it spends ~90% of the time on the idiv line. This isn’t surprising: general integer division works notoriously slow on all computers, including x86.

But there is one kind of division that works well in hardware: division by a power of 2.

#Binary GCD

The binary GCD algorithm was discovered around the same time as Euclid’s, but on the other end of the civilized world, in ancient China. In 1967, it was rediscovered by Josef Stein for use in computers that either don’t have division instruction or have a very slow one — it wasn’t uncommon for CPUs of that era to use hundreds or thousands of cycles for rare or complex operations.

Analogous to the Euclidean algorithm, it is based on a few similar observations:

1. $\gcd(0, b) = b$ and symmetrically $\gcd(a, 0) = a$;
2. $\gcd(2a, 2b) = 2 \cdot \gcd(a, b)$;
3. $\gcd(2a, b) = \gcd(a, b)$ if $b$ is odd and symmetrically $\gcd(a, b) = \gcd(a, 2b)$ if $a$ is odd;
4. $\gcd(a, b) = \gcd(|a − b|, \min(a, b))$, if $a$ and $b$ are both odd.

Likewise, the algorithm itself is just a repeated application of these identities.

Its running time is still logarithmic, which is even easier to show because in each of these identities one of the arguments is divided by 2 — except for the last case, in which the new first argument, the absolute difference of two odd numbers, is guaranteed to be even and thus will be divided by 2 on the next iteration.

What makes this algorithm especially interesting to us is that the only arithmetic operations it uses are binary shifts, comparisons, and subtractions, all of which typically take just one cycle.

#Implementation

The reason this algorithm is not in the textbooks is because it can’t be implemented as a simple one-liner anymore:

int gcd(int a, int b) {
    // base cases (1)
    if (a == 0) return b;
    if (b == 0) return a;
    if (a == b) return a;
    if (a % 2 == 0) {
        if (b % 2 == 0) // a is even, b is even (2)
            return 2 * gcd(a / 2, b / 2);
        else            // a is even, b is odd (3)
            return gcd(a / 2, b);
    } else {
        if (b % 2 == 0) // a is odd, b is even (3)
            return gcd(a, b / 2);
        else            // a is odd, b is odd (4)
            return gcd(std::abs(a - b), std::min(a, b));
    }
}

Let’s run it, and… it sucks. The difference in speed compared to std::gcd is indeed 2x, but on the other side of the equation. This is mainly because of all the branching needed to differentiate between the cases. Let’s start optimizing.

First, let’s replace all divisions by 2 with divisions by whichever highest power of 2 we can. We can do it efficiently with __builtin_ctz, the “count trailing zeros” instruction available on modern CPUs. Whenever we are supposed to divide by 2 in the original algorithm, we will call this function instead, which will give us the exact number of bits to right-shift the number by. Assuming that the we are dealing with large random numbers, this is expected to decrease the number of iterations by almost a factor 2, because $1 + \frac{1}{2} + \frac{1}{4} + \frac{1}{8} + \ldots \to 2$.

Second, we can notice that condition 2 can now only be true once — in the very beginning — because every other identity leaves at least one of the numbers odd. Therefore we can handle this case just once in the beginning and not consider it in the main loop.

Third, we can notice that after we’ve entered condition 4 and applied its identity, $a$ will always be even and $b$ will always be odd, so we already know that on the next iteration we are going to be in condition 3. This means that we can actually “de-evenize” $a$ right away, and if we do so we will again hit condition 4 on the next iteration. This means that we can only ever be either in condition 4 or terminating by condition 1, which removes the need to branch.

Combining these ideas, we get the following implementation:

int gcd(int a, int b) {
    if (a == 0) return b;
    if (b == 0) return a;
    int az = __builtin_ctz(a);
    int bz = __builtin_ctz(b);
    int shift = std::min(az, bz);
    a >>= az, b >>= bz;
    while (a != 0) {
        int diff = a - b;
        b = std::min(a, b);
        a = std::abs(diff);
        a >>= __builtin_ctz(a);
    }
    return b << shift;
}

It runs in 116ns, while std::gcd takes 198ns. Almost twice as fast — maybe we can even optimize it below 100ns?

For that we need to stare at its assembly again, in particular at this block:

; a = edx, b = eax
loop:
    mov   ecx, edx
    sub   ecx, eax       ; diff = a - b
    cmp   eax, edx
    cmovg eax, edx       ; b = min(a, b)
    mov   edx, ecx
    neg   edx
    cmovs edx, ecx       ; a = max(diff, -diff) = abs(diff)
    tzcnt ecx, edx       ; az = __builtin_ctz(a)
    sarx  edx, edx, ecx  ; a >>= az
    test  edx, edx       ; a != 0?
    jne   loop

Let’s draw the dependency graph of this loop:

Modern processors can execute many instructions in parallel, essentially meaning that the true “cost” of this computation is roughly the sum of latencies on its critical path. In this case, it is the total latency of diff, abs, ctz, and shift.

We can decrease this latency using the fact that we can actually calculate ctz using just diff = a - b, because a negative number divisible by $2^k$ still has $k$ zeros at the end of its binary representation. This lets us not wait for max(diff, -diff) to be computed first, resulting in a shorter graph like this:

Hopefully you will be less confused when you think about how the final code will be executed:

int gcd(int a, int b) {
    if (a == 0) return b;
    if (b == 0) return a;
    int az = __builtin_ctz(a);
    int bz = __builtin_ctz(b);
    int shift = std::min(az, bz);
    b >>= bz;
    while (a != 0) {
        a >>= az;
        int diff = b - a;
        az = __builtin_ctz(diff);
        b = std::min(a, b);
        a = std::abs(diff);
    }
    return b << shift;
}

It runs in 91ns, which is good enough to leave it there.

If somebody wants to try to shave off a few more nanoseconds by rewriting the assembly by hand or trying a lookup table to save a few last iterations, please let me know.

#Acknowledgements

The main optimization ideas belong to Daniel Lemire and Ralph Corderoy, who had nothing better to do on the Christmas holidays of 2013.

I regularly hear people asking which programming language to learn, and then reeling off a list of very similar languages (“Should I learn Java, C#, C++, Python, or Ruby?”). In response I usually tell them that it doesn’t really matter, as long as they get started. There are fundamentals behind them.

What do I mean when I say fundamentals? If you have an array or list of items and you’re going to loop over it, that is the same in any imperative language. There is straightforward iteration

int[10] arr;
for (int i = 0; i < 10; i++) {
    // do something with arr[i]
}

and there is iterating over all unordered combinations

int[10] arr;
for (int i = 0; i < 10; i++) {
    for (int j = i+1; j < 10; j++) {
        // do something with arr[i] and arr[j]
    }
}

and a few other patterns, but those patterns are basically the same in C, Java, Python, or Fortran. Having neural pathways that fluently express intention in these patterns, the same way you express thoughts in sentence structures in English, are fundamentals.

But not all languages have the same set of patterns. The patterns for looping in C or Python are very different from the patterns of recursion in Standard ML or Prolog. The way you organize a program in Lisp, where you name new language constructs, is very different from how you organize it in APL, where fragments of symbol sequences are both the definitions of behavior and become the label for that behavior in your mind.

These distinct collections of fundamentals form various ur-languages. Learning a new language that traces to the same ur-language is an easy shift. Learning one that traces to an unfamiliar ur-language requires significant time and effort and new neural pathways.

I am aware of seven ur-languages in software today. I’ll name them for a type specimen, the way a species in paleontology is named for a particular fossil that defines it and then other fossils are compared to the type specimen to determine their identity. The ur-languages are:

• ALGOL
• Lisp
• ML
• Self
• Forth
• APL
• Prolog

ALGOL

Characteristics. Programs consist of sequences of assignments, conditionals, and loops, organized into functions. Many languages add module systems, ways of defining new data types, polymorphism, or alternate control flow constructs like exceptions or coroutines.

Examples. Most common programming languages trace to this ur-language. ALGOL itself included ALGOL 58, ALGOL 60, ALGOL W, and ALGOL 68. Assembly languages for mainstream processors, Fortran, C, C++, Python, Java, C#, Ruby, Pascal, JavaScript and Ada all trace to this ur-language.

History. This is the oldest ur-language, going back to Ada Lovelace formulating programs for Babbage’s analytical engine. The machine and assembly languages for all Eckert-Mauchly architecture computers, going back to EDVAC and the first Univacs were of this form, as were all early attempts at higher level languages, starting with Grace Hopper’s A-0 and going through Fortran and COBOL. In the 1960’s the academic computer science community developed structured programming to make programming in these languages more manageable, which led to ALGOL 60, which basically all members of the class derive from.

Over time, members of this family accrete features taken from other ur-languages. In the 1980’s, notions from the Self ur-language were grafted onto many of these language in the form of classes as a way to define data types and do polymorphism. Since 2010, ideas from the ML ur-language have been appearing.

Lisp

Characteristics. Lisp consists of prefix expressions enclosed in parentheses, for example

(+ 2 3)
(defun square (x)
  (* x x))
(* (square 3) 3)

This syntax seems bizarre, but the language also has a built-in representation of lists as a data structure as parentheses around the space separated items (e.g., (1 2 3 4)). Thus the code is in the form of a list, and Lisp systems let you define macros that take a list, modify it, and pass that modified code to the compiler.

Lisps tend to behave like some other ur-language when writing most code (usually ALGOL or ML), but are distinguished by the macro system that lets the programmer redefine the semantics of the language. Common Lisp, for example, has a loop syntax, but it is defined as a macro, not built into the language.

Examples. There were many early Lisps, but the community achieved a consensus in Common Lisp. Meanwhile, Sussman and Steele explored how much could be done with functions and produced Scheme. Several other special purpose Lisps such as Lush (for numerical computing), AutoLISP (the scripting language for AutoCAD), and Emacs Lisp (the language used to implement editing behavior in the Emacs editor) have been used. In recent years Clojure has emerged as a third major branch of the Lisp family.

History. Lisp is about a year younger than Fortran, which makes it the second oldest language still in use today. Its origins were in a purely mathematical question: how do you write down a mathematical structure you can define that can evaluate its own expressions? John McCarthy provided an answer in 1958, which then got implemented on a computer. That mathematical background made early Lisps awkward fits for the machines they were on. Questions about memory and CPU cycles were irrelevant to the mathematics, and things like garbage collection had to be invented to make it work.

There was a period in the late 1970’s and early 1980’s when machines were specially built to run Lisp from the ground up. Much of today’s integrated development environments was invented on those machines. Lisp itself was the vehicle of choice for most artificial intelligence research in that period, and when artifical intelligence’s hype in the 1980’s failed to deliver, the field, and Lisp with it, crashed into what is called the “AI Winter.” Lisps remain stubbornly alive to this day, especially as computers gained power and other languages adopted many of the features that originally made them awkward to implement.

Characteristics. ML languages are defined by functions being first class values and a type system in the Hindley-Milner family that is adequate to represent different kinds of functions and tagged unions. All iteration is done by recursion, as in

sum : list of int -> int
sum [] = 0
sum (x:xs) = x + sum xs

or by defining functions that encapsulate the iteration pattern and take another function to implement the behavior.

map : ('a -> 'a) -> list of 'a -> list of 'a
map _ [] = []
map f (x:xs) = (f x) : (map f xs)

Some languages in the family (Miranda and Haskell) are lazy by default, that is, they do not evaluate anything until it is actually needed. Others explore extensions of the type system in various directions. OCaml attempts to merge notions from the Self ur-language. Agda and Idris mix values and types (what is called dependent type systems) and 1ML mixes modules and types.

Examples. ML spawned CaML (Cambridge ML), Standard ML, OCaml, and a whole related family such as Miranda, Haskell, and today the dependently typed languages like Agda and Idris.

History. ML was the metalanguage (thus the name) for a theorem proving program developed in Cambridge, England. The language escaped from that context and was popular in Europe, particularly in England and France.

Self (object oriented languages)

Characteristics. A program consists of a set of objects that can receive and send messages to each other. All behavior is implemented in this way. You create a new object by sending a message to an existing object. You do conditionals by having a variable which refers to either the true object or the false object. Both take a message with two parameters, a function to run on true, and a function to run on false. The true object runs the first function. The false object runs the second. The calling code does not know which it is sending to, only that it is sending a message. Loops are the same. Indeed, by creating and inserting appropriate objects into the right places you can entirely redefine the semantics of the language.

These languages usually have their source stored in a live environment rather than text files. The programmer modifies the live system and saves its new state rather than compile files to produce a system.

Examples. The two important examples are Smalltalk and Self. A whole range of languages implement message passing to objects in some subset of the language. This kind of partial import is usually referred to as “object oriented programming.” Most of these are modeled on Smalltalk. JavaScript is the exception, and derives from Self’s classless object system.

The ideas were taken in two other important directions.

First, Common Lisp’s object system generalized the idea of choosing what code is run based on what object receives a message. It disconnects behavior from the objects and instead the runtime chooses which behavior to run based on all the parameters involved, not just one.

Second, Erlang switched the notion of a thread of execution jumping from object to object to run various code and instead had parallel threads of execution that explicitly listen for and send messages.

History. Smalltalk was the original language, developed at Xerox Parc in the late 1970’s and 1980’s. There were a variety of commercial Smalltalk systems in the 1980’s, and IBM used Smalltalk to develop its programming tools for other languages (the collection of tools known as VisualAge). Today Smalltalk largely survives as the open source Pharo Smalltalk.

A lot of work was done on how to make Smalltalk run fast and efficiently, culminating in the Strongtalk project. Strongtalk is historically important because its discoveries became the basis of the HotSpot just-in-time compiler for Java.

Smalltalk inherited the notion of a value and its type from earlier languages, and implemented the idea of a class. All objects had a class that gave their type, and the class was used to construct objects of that type. Self disposed of the notion of class and worked solely with objects. As this is a purer form, I have chosen Self as the type specimen for this ur-language.

Forth (stack languages)

Characteristics. Stack languages are an inverse of Lisp, and share the grammar of Hewlett Packard reverse Polish notation calculators. They have a data stack. When you write a literal like the number 42, it is pushed to the stack. When you write the name of a function, it takes no explicit parameters. Instead it operates on the stack. Simple arithmetic looks quite backwards

2 3 + 5 *

and function definitions are equally terse. In most Forth variants, : defines a new word, in this case square. When square is called it is the same as calling dup, which duplicates the top element of the stack, followed by *, which multiplies the top two elements.

: square dup * ;
3 square

Forth allows programmers to intercept the parser and replace it with their own code, so the grammar is entirely replaceable. It is common to see Forth programs that define small languages, such as a Fortran subset or a way to directly ASCII parse diagrams giving packet layouts or the transitions of state machines.

Examples. Forth in all its many variants, PostScript, Factor, Joy (a purely functional language that uses a mathematical formulation of composition in place of the stack).

History. Forth was originally written in 1970 to control radio telescopes, but then spread broadly in embedded systems. It is sufficiently easy to bootstrap a Forth system that there are dozens of variations created by different programmers for thir own purposes.

PostScript emerged in the 1980’s as a flexible means to describe documents to printers. It is much more limited in many ways than Forth, but defines primitives related to graphical layouts in the language.

APL (array languages)

Characteristics. Everything in the language is an (n dimensional) array. Operators are one or two symbols long, and implement high level operations over these arrays. The result is so terse that the sequences of symbols become the label for an operation rather than giving it another name. For example, to calculate the average of an array in variable x, you would write

(+⌿÷≢) x

Examples. APL, J, K. The higher order operations over arrays have been partially exported into many environments, such as MATLAB, NumPy, and R.

History. APL began as a mathematical notation created by Kenneth Iverson in the 1960’s. He then implemented it on a computer. It has enjoyed a niche following ever since among people doing heavy calculations. Its descendant, K, was very popular in financial settings.

Prolog (logic languages)

Characteristics. Programs consist of facts, either “ground” facts such as Bob is Ed and Jane’s father,

father(bob, ed).
father(bob, jane).

or non-ground facts which define how to derive a fact from other facts by putting in variables (which are capitalized)

grandfather(X, Y) :- father(X, Z), father(Z, Y).

Prolog runtimes take these facts and a query on them and searches for a result for the query. And it turns out that if you choose the right structure for defining facts, this is Turing complete.

The terms that form facts in Prolog are a native data type in their own right that can be created and then fed to the runtime, the same way that Lisp’s macros or Forth’s parser replacement work.

Because Prolog programs are basically searches, they are tuned rather the way database queries are, adjusting the order in which things are searched and cutting off paths that will not yield anything as early as possible.

Examples. Prolog, Mercury, Kanren. The vast majority of programming around this ur-language takes place in Prolog itself — the community is impressively unified.

History. In the 1970’s, logicians in France realized that they could express programs in terms of first order logic, and began trying to implement this. In the 1980’s the Japanese fifth generation computer project bet heavily on Prolog, and when that project failed, Prolog went down in reputation with it.

Meanwhile, decades of research continued into how to make Prolog runtimes smart enough to be efficient in most cases and how to add new capabilities, such as numerical constraints (yielding constraint logic programming).

Prolog tends to show up in niches. Type checking for Java was for many years implemented in Prolog, as was Facebook’s original source code search tool.

What to do with this

For most programmers some or all of these will seem very exotic. It is worth spending some time with each of them for the neural pathways they will make you grow and the possibilities they introduce. Very often two things that seems completely different when viewed through an ALGOL lens turn into a trivial comparison seen through a different lens.

First, every programmer needs to know a language in the ALGOL family well.

Second, learn a language in the Prolog family: SQL. This is after the ALGOL family, SQL will give you the most mileage in your career. I’ve collected the most common stumbling blocks people hit when learning SQL into a free course.

Then, once you’ve got these two, it’s worth branching out. Learning a new language that traces to an unfamiliar ur-language each year will pay dividends. The languages I would suggest today in each of these families, and maybe in this order, are:

• Lisp: PLT Racket
• ML: Haskell
• Self: Self
• Prolog: Prolog¹
• Forth: gForth²
• APL: K (via ok)

If you do a lot of numerical work, learn K earlier. If you do lots of embedded programming, learn gForth earlier. But the order is not important, nor is the exact language. You could learn Standard ML or OCaml instead of Haskell, Common Lisp instead of PLT Racket, and Factor instead of gForth with absolute impunity. These are not-wrong suggestions as opposed to perfect answers.

Mentioned in Writing programs

In a dramatic middle-of-the-night stand off, a bipartisan set of lawmakers pushing for true reform and privacy protections for Americans bought us some more time to fight! They are holding out for, at a minimum, the requirement of an actual probable cause warrant for FBI access to information collected under the mass spying program known as 702.

A reauthorization with virtually no changes was defeated because a core group of lawmakers held strong; they know that people are hungry for real reform that protects the privacy of our communications. We now have a 10-day extension to continue to push Congress to pass a real reform bill. 

The Lawmakers rallied late Thursday night to reject a proposed amendment that made gestures at privacy protections, but it would not have improved on the status quo and would have reauthorized Section 702 for five more years to boot. 

Take action

TELL congress: 702 Needs Reform

Section 702 is rife with problems, loopholes, and compliance issues that need fixing. The National Security Agency collects full conversations being conducted by and with targets overseas – including by and with Americans in the U.S. –  and stores them in massive databases. The NSA then allows other agencies, including the Federal Bureau of Investigation, to access untold amounts of that information. In turn, the FBI takes a “finders keepers” approach to this data: they reason that since it's already collected under one law, it’s OK for them to see it. 

Under current practice, the FBI can query and even read the U.S. side of that communication without a warrant. What’s more, victims of this surveillance  won’t even know and have very few ways of finding out that their communications have been surveilled. EFF and other civil liberties advocates have been trying for years to know when data collected through Section 702 is used as evidence against them.  

Reforming Section 702 is even more urgent because of revelations hinted at by Senator Ron Wyden’s public statements concerning a “secret interpretation” of the law that enables surveillance of Americans, and a public  “Dear Colleague” letter he sent to fellow Senators about FBI abuse of Section 702. 

That’s right—the way the government conducts mass surveillance is so secret and unaccountable even the way they interpret the law is classified. 

 “In many cases these will be law-abiding Americans having perfectly legitimate, often sensitive, conversations,” Wyden wrote. “These Americans could include journalists, foreign aid workers, people with family members overseas - even women trying to get abortion medication from an overseas provider. Congress has an obligation to protect our country from foreign threats and protect the rights of these and other Americans.” 

We have 10 days to make it clear to Congress: 702 needs real reforms. Not a blanket  reauthorization. Not lip service to change. Real reform.

Take action

TELL congress: 702 Needs Reform

WASHINGTON – Congressman Michael Baumgartner (WA‑05) introduced the Multilateral Alignment of Technology Controls on Hardware (MATCH) Act, a bipartisan bill designed to strengthen U.S. national security by closing critical gaps in export controls on semiconductor manufacturing equipment (SME). Original cosponsors include Chairman of the Select Committee on China John Moolenaar (MI‑02) and Reps. Rich McCormick (GA‑07), Bill Huizenga (MI‑04), Jefferson Shreve (IN‑06), Mike Lawler (NY‑17), John Mannion (NY‑22), Jared Golden (ME‑02), Josh Riley (NY‑19), Maggie Goodlander (NH‑02) and Suhas Subramanyam (VA-10). Senators Pete Ricketts (R‑NE) and Andy Kim (D‑NJ) are introducing companion legislation in the Senate.

China continues to aggressively subsidize its semiconductor industry, replicating the state‑driven strategies that enabled its dominance in solar panels, EV batteries, and other advanced manufacturing sectors. As a result, Chinese‑made legacy chips are now widely embedded in U.S. weapons systems, intelligence platforms, and critical infrastructure. Even in leading‑edge AI chip production, Chinese firms such as Huawei are rapidly advancing.

While the United States has imposed extensive export controls to slow China’s semiconductor indigenization, U.S. allies have not fully matched these measures. This misalignment has left critical gaps that China continues to exploit.

“China has made it abundantly clear that it intends to dominate the technologies that underpin both our economy and our national defense. The United States cannot afford to leave open back doors that allow the Chinese Communist Party to acquire the tools it needs to leap ahead in semiconductor manufacturing. I introduced the MATCH Act to ensure that America and our allies move in lockstep to close these gaps, defend our technological edge, and safeguard the supply chains that power everything from our weapons systems to our critical infrastructure. This is about protecting American workers, American innovation, and American security for the long haul.” – Congressman Michael Baumgartner

“The bipartisan MATCH Act will close loopholes, create a level playing field for U.S. and allied toolmakers, and ensure the next decade of growth in chip manufacturing – and the jobs that come with it – happens in the United States and allied countries, not China. Semiconductor manufacturing equipment is a crucial advantage we have against China’s military and technological ambitions, and as the Select Committee recently documented in a bipartisan investigation, China exploited loopholes in current export controls to buy chip-making equipment tools as part of its strategy to dominate in chips. There is an urgent need to pass this bipartisan legislation and protect our advantage in chip making.”  –Select Committee on China Chairman John Moolenaar

“The ability to design and produce semiconductors lies at the heart of the technology competition with Communist China,” said Senator Ricketts.  “SME is an important dual-use technology.  For too long, our export controls have been a patchwork of entity-based restrictions that Beijing easily bypasses using front companies.  At the same time, U.S. restrictions on SME are stronger than those of our close allies and partners.  This status quo puts American companies last.  The MATCH Act strengthens our controls and creates a level playing field for U.S. companies.”

“To remain the global leader in the AI race, the United States cannot allow critical technologies to slip through the cracks. The MATCH Act is about closing those gaps and making sure our allies stand shoulder-to-shoulder with us in protecting the tools that power our economy and national defense,” said Congressman Rich McCormick. “By aligning export controls, we strengthen our industrial base in tandem with American innovation to ensure that the future of semiconductor manufacturing, and the jobs it creates, remains firmly in the United States and among our trusted partners.”

“As the Chairman of the House Subcommittee overseeing export controls, I’ve seen how China is exploiting access to American and allies chipmaking equipment to modernize its military and pursue global AI dominance. The bipartisan MATCH Act advances the security, economic prosperity, and technological leadership of the United States by protecting critical chipmaking equipment from the Chinese Communist Party.” – Congressman Bill Huizenga

“America leads the world in semiconductor technology. That lead was earned strategically and is never guaranteed. When we are out of sync with our allies on vital issues like these, it creates openings that China will be quick to exploit,” said Congressman Jefferson Shreve. “The bipartisan MATCH Act brings the United States and our partners into alignment so we can protect the tools and technologies that matter most against our foreign adversaries. This bill ensures fairness for American companies in this field and strength for our national security. If we want to keep the jobs, innovation, and supply chains of the future out of the hands of China, we need to act with clarity and urgency.”

“China is moving aggressively to dominate the semiconductor supply chain, and we must work alongside our allies to coordinate our export controls to stop the CCP’s efforts. This legislation helps protect American innovation, secure our supply chains, and ensures a level playing field with our allies to counter China’s chip production. Most of all, it’s a matter of national security” said Congressman Mike Lawler. 

Congressman John W. Mannion said, “American workers, innovation, and ingenuity built the world’s most advanced semiconductor industry. With the United States ready to lead the next generation of global memory chip manufacturing and research, I’m going to protect that future and make sure we stay ahead. The MATCH Act safeguards the tools, technology, jobs, and know-how that will keep the United States ahead of China and at the leading edge of the 21st century technologies the world depends on.” 

“The United States should have policies in place to ensure a level playing field for American manufacturing, including export controls to push back against China’s efforts to dominate a semiconductor market that is critical to our economy and our national security. I’m proud to be a part of a bipartisan coalition supporting the MATCH Act, which takes a balanced approach to protecting and supporting American innovation and jobs.” – Congressman Jared Golden 

“America, not China, should lead the world in AI to ensure that AI works for American workers, American innovation, and America’s national security,” said Congresswoman Maggie Goodlander. “Our bipartisan MATCH Act is about closing dangerous loopholes that China is exploiting, standing shoulder to shoulder with our allies on a level playing field, and winning the AI race against China.” 

Key provisions of the MATCH Act include:

• Country-Wide Prohibition on “Chokepoint” SME: Prohibits the sale of the most essential SME to any destination inside a country of concern.  This includes, at a minimum, Deep Ultraviolet (DUV) immersion lithography and cryogenic etch tools for advanced and legacy chips.
• Tighter Restrictions on China’s National Champions: Designates as covered facilities all fabs run by ChangXin Memory Technologies (CXMT), Hua Hong, Huawei, Semiconductor Manufacturing International Corp (SMIC), and Yangtze Memory Technologies Corp (YMTC), including all subsidiaries and affiliates.  Applies Entity-List-like restrictions on exports, servicing, and technical support to these facilities of all items subject to the Export Administration Regulations.
• Leverage for Diplomatic Negotiations: Supports diplomatic negotiations with deadlines for aligning controls.  Includes a National Security Waiver if additional time is required. 
• Creates a Level Playing Field:  SME is a critical dual-use component that supports China’s military modernization.  The MATCH Act ensures that controls will apply uniformly to U.S. and allied countries, in the interests of our collective national security. If allies cannot demonstrate progress within the 150-day deadline, the Act directs the Department of Commerce to implement controls unilaterally.  It expands U.S. jurisdiction over foreign-produced items that use U.S. software, technology, or components (applying the “Foreign Direct Product Rule”).

MATCH Act Endorsements: 

“A strategy that aligns the U.S. and its allies in stopping the flow of advanced semiconductor manufacturing equipment to our adversaries is key to protecting U.S. technological advantages. When foreign suppliers of advanced chipmaking tools are not subject to the same rules as U.S. firms, it undermines the objectives of U.S. export controls. Closing gaps in the export control regime by partnering with our allies first and imposing extraterritorial controls second is a prudent strategy that advances U.S. national security.” – Jacob Feldgoise, Senior Data Research Analyst, Center for Security and Emerging Technology (CSET).

“Preserving and enhancing export controls on semiconductor manufacturing are one of the most important things that America can do to win the AI race against China. The MATCH Act is an important tool to help ensure American dominance in this area and to hold our allies’ feet to the fire in holding the line on export controls on the most advanced technology humanity has produced.” – Dmitri Alperovitch, Chairman, Silverado Policy Accelerator

“For too long, the United States has borne the economic and political costs of semiconductor export controls while leaving room for foreign suppliers to free ride, backfill, or slow-roll their alignment. The MATCH Act creates a simple rule: either our allies match our controls on the most important tools and components for advanced semiconductors, or the United States closes the loopholes itself. That is how you turn a theoretical chokepoint into a real one, securing our AI supply-chain and technology leadership for years to come.” – Samuel Hammond, Chief Economist, Foundation for American Innovation 

“For years, gaps in semiconductor equipment controls have undermined America’s most effective technology denial strategy. Although the United States has acted to restrict chokepoint tools, our allies don’t always follow suit — and China continues to exploit the difference.” – Ryan Fedasiuk, Fellow, China and Technology, American Enterprise Institute

“The MATCH Act is critical to protecting U.S. and allied dominance in advanced chipmaking, which is the foundation of our leadership over China in AI. Advanced chips are the lifeblood of AI and are one of the few things that China struggles to manufacture, as it cannot make these chips without U.S. and allied equipment. However, China is exploiting loopholes and asymmetries in U.S. and allied export controls on chipmaking equipment and components to purchase hundreds of billions of dollars’ worth of advanced tools; chipmaking equipment is the largest export from the Netherlands to China, the second largest from Japan, and the third largest from the United States. The MATCH Act would close many of these loopholes and turn off the tap, which is among the most important steps we can take to preserve long-term U.S. allied leadership in semiconductor production, toolmaking, and AI.” – Chris McGuire, Senior Fellow for China and Emerging Technologies, Council on Foreign Relations

“Export controls on chipmaking tools are the foundation of America’s technology competition strategy with China. They are directly responsible for U.S. leadership in emerging technologies such as AI and quantum computing and defense technologies of the future. The MATCH Act would decisively close gaps in the controls that risk undermining their effectiveness in the long term, thus giving America an enduring lead in the technologies that will reshape the security landscape.” – Saif M. Khan, Former Director for Technology and National Security, National Security Council

“America and its allies have a massive lead over China in advanced AI chip production, but keeping that edge requires that we and our friends be on the same page. Allied export controls on semiconductor manufacturing equipment suffer from meaningful gaps that China continues to exploit. We must ensure our allies align their export controls with American standards — through diplomacy, if possible, but unilaterally if necessary. The stakes are too high to wait.” – Chris Griswold, Policy Director, American Compass

“FDD Action supports the MATCH Act, which focuses on strengthening and aligning export controls on semiconductor manufacturing equipment. These tools are essential to protecting U.S. technological leadership and national security. By closing gaps and reinforcing coordination with allies, this bill helps prevent critical technologies from being diverted in ways that could undermine U.S. national security.” – Alexandria Paolozzi Moore, FDD Action Senior Director of Government Affairs

You can read the bill text here. 

The world in which IPv6 was a good design

Last November I went to an IETF meeting for the first time. The IETF is an interesting place; it seems to be about 1/3 maintenance grunt work, 1/3 extending existing stuff, and 1/3 blue sky insanity. I attended mostly because I wanted to see how people would react to TCP BBR, which was being presented there for the first time. (Answer: mostly positively, but with suspicion. It kinda seemed too good to be true.)

Anyway, the IETF meetings contain lots and lots of presentations about IPv6, the thing that was supposed to replace IPv4, which is what the Internet runs on. (Some would say IPv4 is already being replaced; some would say it has already happened.) Along with those presentations about IPv6, there were lots of people who think it's great, the greatest thing ever, and they're pretty sure it will finally catch on Any Day Now, and IPv4 is just a giant pile of hacks that really needs to die so that the Internet can be elegant again.

I thought this would be a great chance to really try to figure out what was going on. Why is IPv6 such a complicated mess compared to IPv4? Wouldn't it be better if it had just been IPv4 with more address bits? But it's not, oh goodness, is it ever not. So I started asking around. Here's what I found.

Buses ruined everything

Once upon a time, there was the telephone network, which used physical circuit switching. Essentially, that meant moving connectors around so that your phone connection was literally just a very long wire ("OSI layer 1"). A "leased line" was a very long wire that you leased from the phone company. You would put bits in one end of the wire, and they'd come out the other end, a fixed amount of time later. You didn't need addresses because there was exactly one machine at each end.

Eventually the phone company optimized that a bit. Time-division multiplexing (TDM) and "virtual circuit switching" was born. The phone company could transparently take the bits at a slower bit rate from multiple lines, group them together with multiplexers and demultiplexers, and let them pass through the middle of the phone system using fewer wires than before. Making that work was a little complicated, but as far as we modem users were concerned, you still put bits in one end and they came out the other end. No addresses needed.

The Internet (not called the Internet at the time) was built on top of these circuits. You had a bunch of wires that you could put bits into and have them come out the other side. If one computer had two or three interfaces, then it could, if given the right instructions, forward bits from one line to another, and you could do something a lot more efficient than a separate line between each pair of computers. And so IP addresses ("layer 3"), subnets, and routing were born. Even then, with these point-to-point links, you didn't need MAC addresses, because once a packet went into the wire, there was only one place it could come out. You used IP addresses to decide where it should go after that.

Meanwhile, LANs got invented as an alternative. If you wanted to connect computers (or terminals and a mainframe) together at your local site, it was pretty inconvenient to need multiple interfaces, one for each wire to each satellite computer, arranged in a star configuration. To save on electronics, people wanted to have a "bus" network (also known as a "broadcast domain," a name that will be important later) where multiple stations could just be plugged into a single wire, and talk to any other station plugged into the same wire. These were not the same people as the ones building the Internet, so they didn't use IP addresses for this. They all invented their own scheme ("layer 2").

One of the early local bus networks was arcnet, which is dear to my heart (I wrote the first Linux arcnet driver and arcnet poetry way back in the 1990s, long after arcnet was obsolete). Arcnet layer 2 addresses were very simplistic: just 8 bits, set by jumpers or DIP switches on the back of the network card. As the network owner, it was your job to configure the addresses and make sure you didn't have any duplicates, or all heck would ensue. This was kind of a pain, but arcnet networks were usually pretty small, so it was only kind of a pain.

A few years later, ethernet came along and solved that problem once and for all, by using many more bits (48, in fact) in the layer 2 address. That's enough bits that you can assign a different (sharded-sequential) address to every device that has ever been manufactured, and not have any overlaps. And that's exactly what they did! Thus the ethernet MAC address was born.

Various LAN technologies came and went, including one of my favourites, IPX (Internetwork Packet Exchange, though it had nothing to do with the "real" Internet) and Netware, which worked great as long as all the clients and servers were on a single bus network. You never had to configure any addresses, ever. It was beautiful, and reliable, and worked. The golden age of networking, basically.

Of course, someone had to ruin it: big company/university networks. They wanted to have so many computers that sharing 10 Mbps of a single bus network between them all became a huge bottleneck, so they needed a way to have multiple buses, and then interconnect - "internetwork," if you will - those buses together. You're probably thinking, of course! Use the Internet Protocol for that, right? Ha ha, no. The Internet protocol, still not called that, wasn't mature or popular back then, and nobody took it seriously. Netware-over-IPX (and the many other LAN protocols at the time) were serious business, so as serious businesses do, they invented their own thing(s) to extend the already-popular thing, ethernet. Devices on ethernet already had addresses, MAC addresses, which were about the only thing the various LAN protocol people could agree on, so they decided to use ethernet addresses as the keys for their routing mechanisms. (Actually they called it bridging and switching instead of routing.)

The problem with ethernet addresses is they're assigned sequentially at the factory, so they can't be hierarchical. That means the "bridging table" is not as nice as a modern IP routing table, which can talk about the route for a whole subnet at a time. In order to do efficient bridging, you had to remember which network bus each MAC address could be found on. And humans didn't want to configure each of those by hand, so it needed to figure itself out automatically. If you had a complex internetwork of bridges, this could get a little complicated. As I understand it, that's what led to the spanning tree poem, and I think I'll just leave it at that. Poetry is very important in networking.

Anyway, it mostly worked, but it was a bit of a mess, and you got broadcast floods every now and then, and the routes weren't always optimal, and it was pretty much impossible to debug. (You definitely couldn't write something like traceroute for bridging, because none of the tools you need to make it work - such as the ability for an intermediate bridge to even have an address - exist in plain ethernet.)

On the other hand, all these bridges were hardware-optimized. The whole system was invented by hardware people, basically, as a way of fooling the software, which had no idea about multiple buses and bridging between them, into working better on large networks. Hardware bridging means the bridging could go really really fast - as fast as the ethernet could go. Nowadays that doesn't sound very special, but at the time, it was a big deal. Ethernet was 10 Mbps, because you could maybe saturate it by putting a bunch of computers on the network all at once, not because any one computer could saturate 10 Mbps. That was crazy talk.

Anyway, the point is, bridging was a mess, and impossible to debug, but it was fast.

Internet over buses

While all that was happening, those Internet people were getting busy, and were of course not blind to the invention of cool cheap LAN technologies. I think it might have been around this time that the ARPANET got actually renamed to the Internet, but I'm not sure. Let's say it was, because the story is better if I sound confident.

At some point, things progressed from connecting individual Internet computers over point-to-point long distance links, to the desire to connect whole LANs together, over point-to-point links. Basically, you wanted a long-distance bridge.

You might be thinking, hey, no big deal, why not just build a long distance bridge and be done with it? Sounds good, doesn't work. I won't go into the details right now, but basically the problem is congestion control. The deep dark secret of ethernet bridging is that it assumes all your links are about the same speed, and/or completely uncongested, because they have no way to slow down. You just blast data as fast as you can, and expect it to arrive. But when your ethernet is 10 Mbps and your point-to-point link is 0.128 Mbps, that's completely hopeless. Separately, the idea of figuring out your routes by flooding all the links to see which one is right - this is the actual way bridging typically works - is hugely wasteful for slow links. And sub-optimal routing, an annoyance on local networks with low latency and high throughput, is nasty on slow, expensive long-distance links. It just doesn't scale.

Luckily, those Internet people (if it was called the Internet yet) had been working on that exact set of problems. If we could just use Internet stuff to connect ethernet buses together, we'd be in great shape.

And so they designed a "frame format" for Internet packets over ethernet (and arcnet, for that matter, and every other kind of LAN).

And that's when everything started to go wrong.

The first problem that needed solving was that now, when you put an Internet packet onto a wire, it was no longer clear which machine was supposed to "hear" it and maybe forward it along. If multiple Internet routers were on the same ethernet segment, you couldn't have them all picking it up and trying to forward it; that way lies packet storms and routing loops. No, you had to choose which router on the ethernet bus is supposed to pick it up. We can't just use the IP destination field for that, because we're already using that for the final destination, not the router destination. Instead, we identify the desired router using its MAC address in the ethernet frame.

So basically, to set up your local IP routing table, you want to be able to say something like, "send packets to IP address 10.1.1.1 via the router at MAC address 11:22:33:44:55:66." That's the actual thing you want to express. This is important! Your destination is an IP address, but your router is a MAC address. But if you've ever configured a routing table, you might have noticed that nobody writes it like that. Instead, because the writers of your operating system's TCP/IP stack are stubborn, you write something like "send packets to IP address 10.1.1.1 via the router at IP address 192.168.1.1."

In truth, that really is just complicating things. Now your operating system has to first look up the ethernet address of 192.168.1.1, find out it's 11:22:33:44:55:66, and finally generate a packet with destination ethernet address 11:22:33:44:55:66 and destination IP address 10.1.1.1. 192.168.1.1 shows up nowhere in the packet; it's just an abstraction at the human level.

To do that pointless intermediate step, you need to add ARP (address resolution protocol), a simple non-IP protocol whose job it is to convert IP addresses to ethernet addresses. It does this by broadcasting to everyone on the local ethernet bus, asking them all to answer if they own that particular IP address. If you have bridges, they all have to forward all the ARP packets to all their interfaces, because they're ethernet broadcast packets, and that's what broadcasting means. On a big, busy ethernet with lots of interconnected LANs, excessive broadcasts start becoming one of your biggest nightmares. It's especially bad on wifi. As time went on, people started making bridges/switches with special hacks to avoid forwarding ARP as far as it's technically supposed to go, to try to cut down on this problem. Some devices (especially wifi access points) just make fake ARP answers to try to help. But doing any of that is a hack, albeit sometimes a necessary hack.

Death by legacy

Time passed. Eventually (and this actually took quite a while), people pretty much stopped using non-IP protocols on ethernet at all. So basically all networks became a physical wire (layer 1), with multiple stations on a bus (layer 2), with multiple buses connected over bridges (gotcha! still layer 2!), and those inter-buses connected over IP routers (layer 3).

After a while, people got tired of manually configuring IP addresses, arcnet style, and wanted them to auto-configure, ethernet style, except it was too late to literally do it ethernet style, because a) the devices had already been manufactured with ethernet addresses, not IP addresses, and b) IP addresses were only 32 bits, which is not enough to just manufacture them forever with no overlaps, and c) just assigning IP addresses sequentially instead of using subnets would bring us back to square one: it would just be ethernet over again, and we already have ethernet.

So that's where bootp and DHCP came from. Those protocols, by the way, are special kinda like ARP is special (except they pretend not to be special, by technically being IP packets). They have to be special, because an IP node has to be able to transmit them before it has an IP address, which is of course impossible, so it just fills the IP headers with essentially nonsense (albeit nonsense specified by an RFC), so the headers might as well have been left out. (You know these "IP" headers are nonsense because the DHCP server has to open a raw socket and fill them in by hand; the kernel IP layer can't do it.) But nobody would feel nice if they were inventing a whole new protocol that wasn't IP, so they pretended it was IP, and then they felt nice. Well, as nice as one can feel when one is inventing DHCP.

Anyway, I digress. The salient detail here is that unlike real IP services, bootp and DHCP need to know about ethernet addresses, because after all, it's their job to hear your ethernet address and assign you an IP address to go with it. They're basically the reverse of ARP, except we can't say that, because there's a protocol called RARP that is literally the reverse of ARP. Actually, RARP worked quite fine and did the same thing as bootp and DHCP while being much simpler, but we don't talk about that.

The point of all this is that ethernet and IP were getting further and further intertwined. They're nowadays almost inseparable. It's hard to imagine a network interface (except ppp0) without a 48-bit MAC address, and it's hard to imagine that network interface working without an IP address. You write your IP routing table using IP addresses, but of course you know you're lying when you name the router by IP address; you're just indirectly saying that you want to route via a MAC address. And you have ARP, which gets bridged but not really, and DHCP, which is an IP packet but is really an ethernet protocol, and so on.

Moreover, we still have both bridging and routing, and they both get more and more complicated as the LANs and the Internet get more and more complicated, respectively. Bridging is still, mostly, hardware based and defined by IEEE, the people who control the ethernet standards. Routing is still, mostly, software based and defined by the IETF, the people who control the Internet standards. Both groups still try to pretend the other group doesn't exist. Network operators basically choose bridging vs routing based on how fast they want it to go and how much they hate configuring DHCP servers, which they really hate very much, which means they use bridging as much as possible and routing when they have to.

In fact, bridging has gotten so completely out of control that people decided to extract the layer 2 bridging decisions out completely to a higher level (with configuration exchanged between bridges using a protocol layered over IP, of course!) so it can be centrally managed. That's called software-defined networking (SDN). It helps a lot, compared to letting your switches and bridges just do whatever they want, but it's also fundamentally silly, because you know what's software defined networking? IP. It is literally and has always been the software-defined network you use for interconnecting networks that have gotten too big. But the problem is, IPv4 was initially too hard to hardware accelerate, and anyway, it didn't get hardware accelerated, and configuring DHCP really is a huge pain, so network operators just learned how to bridge bigger and bigger things. And nowadays big data centers are basically just SDNed, and you might as well not be using IP in the data center at all, because nobody's routing the packets. It's all just one big virtual bus network.

It is, in short, a mess.

Now forget I said all that...

Great story, right? Right. Now pretend none of that happened, and we're back in the early 1990s, when most of that had in fact already happened, but people at the IETF were anyway pretending that it hadn't happened and that the "upcoming" disaster could all be avoided. This is the good part!

There's one thing I forgot to mention in that big long story above: somewhere in that whole chain of events, we completely stopped using bus networks. Ethernet is not actually a bus anymore. It just pretends to be a bus. Basically, we couldn't get ethernet's famous CSMA/CD to keep working as speeds increased, so we went back to the good old star topology. We run bundles of cables from the switch, so that we can run one cable from each station all the way back to the center point. Walls and ceilings and floors are filled with big, thick, expensive bundles of ethernet, because we couldn't figure out how to make buses work well... at layer 1. It's kinda funny actually when you think about it. If you find sad things funny.

In fact, in a bonus fit of insanity, even wifi - the ultimate bus network, right, where literally everybody is sharing the same open-air "bus" - we almost universally use wifi in a mode, called "infrastructure mode," which simulates a giant star topology. If you have two wifi stations connected to the same access point, they don't talk to each other directly, even when they can hear each other just fine. They send a packet to the access point, but addressed to the MAC address of the other node. The access point then bounces it back out to the destination node.

HOLD THE HORSES LET ME JUST REVIEW THAT FOR YOU. There's a little catch there. When node X wants to send to Internet node Z, via IP router Y, via wifi access point A, what does the packet look like? Just to draw a picture, here's what we want to happen:

X -> [wifi] -> A -> [wifi] -> Y -> [internet] -> Z

Z is the IP destination, so obviously the IP destination field has to be Z. Y is the router, which we learned above that we specify by using its ethernet MAC address in the ethernet destination field. But in wifi, X can't just send out a packet to Y, for various reasons (including that they don't know each other's WPA2 encryption keys). We have to send to A. Where do we put A's address, you might ask?

No problem! 802.11 has a thing called 3-address mode. They add a third ethernet MAC address to every frame, so they can talk about the real ethernet destination, and the intermediate ethernet destination. On top of that, there are bit fields called "to-AP" and "from-AP," which tell you if the packet is going from a station to an AP, or from an AP to a station, respectively. But actually they can both be true at the same time, because that's how you make wifi repeaters (APs send packets to APs).

Speaking of wifi repeaters! If A is a repeater, it has to send back to the base station, B, along the way, which looks like this:

X -> [wifi] -> A -> [wifi-repeater] -> B -> [wifi] -> Y -> [internet] -> Z

X->A uses three-address mode, but A->B has a problem: the ethernet source address is X, and the ethernet destination address is Y, but the packet on the air is actually being sent from A to B; X and Y aren't involved at all. Suffice it to say that there's a thing called 4-address mode, and it works pretty much like you think.

(In 802.11s mesh networks, there's a 6-address mode, and that's about where I gave up trying to understand.)

Avery, I was promised IPv6, and you haven't even mentioned IPv6

Oh, oops. This post went a bit off the rails, didn't it?

Here's the point of the whole thing. The IETF people, when they were thinking about IPv6, saw this mess getting made - and maybe predicted some of the additional mess that would happen, though I doubt they could have predicted SDN and wifi repeater modes - and they said, hey wait a minute, stop right there. We don't need any of this crap! What if instead the world worked like this?

• No more physical bus networks (already done!)
• No more layer 2 internetworks (that's what layer 3 is for)
• No more broadcasts (layer 2 is always point-to-point, so where would you send the broadcast to? replace it with multicast instead)
• No more MAC addresses (on a point-to-point network, it's obvious who the sender and receiver are, and you can do multicast using IP addresses)
• No more ARP and DHCP (no MAC addresses, no so mapping IP addresses to MAC addresses)
• No more complexity in IP headers (so you can hardware accelerate IP routing)
• No more IP address shortages (so we can go back to routing big subnets again)
• No more manual IP address configuration except at the core (and there are so many IP addresses that we can recursively hand out subnets down the tree from there)

Imagine that we lived in such a world: wifi repeaters would just be IPv6 routers. So would wifi access points. So would ethernet switches. So would SDN. ARP storms would be gone. "IGMP snooping bridges" would be gone. Bridging loops would be gone. Every routing problem would be traceroute-able. And best of all, we could drop 12 bytes (source/dest ethernet addresses) from every ethernet packet, and 18 bytes (source/dest/AP addresses) from every wifi packet. Sure, IPv6 adds an extra 24 bytes of address (vs IPv4), but you're dropping 12 bytes of ethernet, so the added overhead is only 12 bytes - pretty comparable to using two 64-bit IP addresses but having to keep the ethernet header. The idea that we could someday drop ethernet addresses helped to justify the oversized IPv6 addresses.

It would have been beautiful. Except for one problem: it never happened.

Requiem for a dream

One person at work put it best: "layers are only ever added, never removed."

All this wonderfulness depended on the ability to start over and throw away the legacy cruft we had built up. And that is, unfortunately, pretty much impossible. Even if IPv6 hits 99% penetration, that doesn't mean we'll be rid of IPv4. And if we're not rid of IPv4, we won't be rid of ethernet addresses, or wifi addresses. And if we have to keep the IEEE 802.3 and 802.11 framing standards, we're never going to save those bytes. So we will always need the "IPv6 neighbour discovery" protocol, which is just a more complicated ARP. Even though we no longer have bus networks, we'll always need some kind of simulator for broadcasts, because that's how ARP works. We'll need to keep running a local DHCP server at home so that our obsolete IPv4 light bulbs keep working. We'll keep needing NAT so that our obsolete IPv4 light bulbs can keep reaching the Internet.

And that's not the worst of it. The worst of it is we still need the infinite abomination that is layer 2 bridging, because of one more mistake the IPv6 team forgot to fix. Unfortunately, while they were blue-skying IPv6 back in the 1990s, they neglected to solve the "mobile IP" problem. As I understand it, the idea was to get IPv6 deployed first - it should only take a few years - and then work on it after IPv4 and MAC addresses had been eliminated, at which time it should be much easier to solve, and meanwhile, nobody really has a "mobile IP" device yet anyway. I mean, what would that even mean, like carrying your laptop around and plugging into a series of one ethernet port after another while you ftp a file? Sounds dumb.

The killer app: mobile IP

Of course, with a couple more decades of history behind us, now we know a few use cases for carrying around a computer - your phone - and letting it plug into one ethernet port wireless access point after another. We do it all the time. And with LTE, it even mostly works! With wifi, it works sometimes. Good, right?

Not really, because of the Internet's secret shame: all that stuff only works because of layer 2 bridging. Internet routing can't handle mobility - at all. If you move around on an IP network, your IP address changes, and that breaks any connections you have open.

Corporate wifi networks fake it for you, bridging their whole LAN together at layer 2, so that the giant central DHCP server always hands you the same IP address no matter which corporate wifi access point you join, and then gets your packets to you, with at most a few seconds of confusion while the bridge reconfigures. Those newfangled home wifi systems with multiple extenders/repeaters do the same trick. But if you switch from one wifi network to another as you walk down the street - like if there's a "Public Wifi" service in a series of stores - well, too bad. Each of those gives you a new IP address, and each time your IP address changes, you kill all your connections.

LTE tries even harder. You keep your IP address (usually an IPv6 address in the case of mobile networks), even if you travel miles and miles and hop between numerous cell towers. How? Well... they typically just tunnel all your traffic back to a central location, where it all gets bridged together (albeit with lots of firewalling) into one super-gigantic virtual layer 2 LAN. And your connections keep going. At the expense of a ton of complexity, and a truly embarrassing amount of extra latency, which they would really like to fix, but it's almost impossible.

Making mobile IP actually work¹

So okay, this has been a long story, but I managed to extract it from those IETF people eventually. When we got to this point - the problem of mobile IP - I couldn't help but ask. What went wrong? Why can't we make it work?

The answer, it turns out, is surprisingly simple. The great design flaw was in how the famous "4-tuple" (source ip, source port, destination ip, destination port) was defined. We use the 4-tuple to identify a given TCP or UDP session; if a packet has those four fields the same, then it belongs to a given session, and we can deliver it to whatever socket is handling that session. But the 4-tuple crosses two layers: internetwork (layer 3) and transport (layer 4). If, instead, we had identified sessions using only layer 4 data, then mobile IP would have worked perfectly.

Let's do a quick example. X port 1111 is talking to Y port 80, so it sends a packet with 4-tuple (X,1111,Y,80). The response comes back with (Y,80,X,1111), and the kernel delivers it to the socket that generated the original packet. When X sends more packets tagged (X,1111,Y,80), then Y delivers them all to the same server socket, and so on.

Then, if X hops IP addresses, it gets a new name, say Q. Now it'll start sending packets with (Q,1111,Y,80). Y has no idea what that means, and throws it away. Meanwhile, if Y sends packets tagged (Y,80,X,1111), they get lost, because there is no longer an X to receive them.

Imagine now that we tagged sockets without reference to their IP address. For that to work, we'd need much bigger port numbers (which are currently 16 bits). Let's make them, say, 128 or 256 bits, some kind of unique hash.

Now X sends out packets to Y with tag (uuid,80). Note, the packets themselves still contain the (X,Y) addressing information, down at layer 3 - that's how they get routed to the right machine in the first place. But the kernel doesn't use the layer 3 information to decide which socket to deliver to; it just uses the uuid. The destination port (80 in this case) is only needed to initiate a new session, to identify what service you want to connect to, and can be ignored or left out after that.

For the return direction, Y's kernel caches the fact that packets for (uuid) go to IP address X, which is the address it most recently received (uuid) packets from.

Now imagine that X changes addresses to Q. It still sends out packets tagged with (uuid,80), to IP address Y, but now those packets come from address Q. On machine Y, it receives the packet and matches it to the socket associated with (uuid), notes that the packets for that socket are now coming from address Q, and updates its cache. Its return packets can now be sent, tagged as (uuid), back to Q instead of X. Everything works! (Modulo some care to prevent connection hijacking by impostors.²)

There's only one catch: that's not how UDP and TCP work, and it's too late to update them. Updating UDP and TCP would be like updating IPv4 to IPv6; a project that sounded simple, back in the 1990s, but decades later, is less than half accomplished (and the first half was the easy part; the long tail is much harder).

The positive news is we may be able to hack around it with yet another layering violation. If we throw away TCP - it's getting rather old anyway - and instead use QUIC over UDP, then we can just stop using the UDP 4-tuple as a connection identifier at all. Instead, if the UDP port number is the "special mobility layer" port, we unwrap the content, which can be another packet with a proper uuid tag, match it to the right session, and deliver those packets to the right socket.

There's even more good news: the experimental QUIC protocol already, at least in theory, has the right packet structure to work like this. It turns out you need unique session identifiers (keys) anyhow if you want to use stateless packet encryption and authentication, which QUIC does. So, perhaps with not much work, QUIC could support transparent roaming. What a world that would be!

At that point, all we'd have to do is eliminate all remaining UDP and TCP from the Internet, and then we would definitely not need layer 2 bridging anymore, for real this time, and then we could get rid of broadcasts and MAC addresses and SDN and DHCP and all that stuff.

And then the Internet would be elegant again.

¹ Edit 2017-08-16: It turns out that nothing in this section requires IPv6. It would work fine with IPv4 and NAT, even roaming across multiple NATs.

² Edit 2017-08-15: Some people asked what "some care to prevent connection hijacking" might look like. There are various ways to do it, but the simplest would be to do something like the SYN-ACK-SYNACK exchange TCP does at connection startup. If Y just trusts the first packet from the new host Q, then it's too easy for any attacker to take over the X->Y connection by simply sending a packet to Y from anywhere on the Internet. (Although it's a bit hard to guess which 256-bit uuid to fill in.) But if Y sends back a cookie that Q must receive and process and send back to Y, that ensures that Q is at least a man-in-the-middle and not just an outside attacker (which is all TCP would guarantee anyway). If you're using an encrypted protocol (like QUIC³), the handshake can also be protected by your session key.

³ Edit 2017-10-24: Besides QUIC, there are several other candidates for such a protocol, including MinimaLT. I didn't mention MinimaLT originally because it wasn't part of my original conversation with the IETF people, but I don't mean to imply that QUIC is the only possible option as a roaming-capable TCP replacement. In fact, MinimaLT is the first protocol I heard of that elegantly solved the roaming problem. Future solutions that might get adopted, including by QUIC, will likely be modeled after MinimaLT's solution.

Update 2020-07-09: I've posted more thoughts on IPv4/IPv6 migration and interoperability on the Tailscale blog.

The only time they see it is when it impedes their access to a site.

Yet still the operator emblazons this brand on the interstitial. Why?

Does he think he's Tony Stark in Sokovia, or what?

On April 17, engineers at NASA’s Jet Propulsion Laboratory (JPL) in Southern California sent commands to shut down an instrument aboard Voyager 1 called the Low-energy Charged Particles experiment, or LECP. The nuclear-powered spacecraft is running low on power, and turning off the LECP is considered the best way to keep humanity’s first interstellar explorer going.

NASA/JPL-Caltech

The LECP has been operating almost without interruption since Voyager 1 launched in 1977 — almost 49 years. It measures low-energy charged particles, including ions, electrons, and cosmic rays originating from our solar system and galaxy. The instrument has provided critical data about the structure of the interstellar medium, detecting pressure fronts and regions of varying particle density in the space beyond our heliosphere. The twin Voyagers are the only spacecraft that are far enough from Earth to provide this information.

Like Voyager 2, Voyager 1 relies on a radioisotope thermoelectric generator, a device that converts heat from decaying plutonium into electricity. Both probes lose about 4 watts of power each year. After almost a half-century in space, power margins have grown razor thin, requiring the team to conserve energy by shutting off heaters and instruments while making sure the spacecraft don’t get so cold that their fuel lines freeze.

During a routine, planned roll maneuver on Feb. 27, Voyager 1’s power levels fell unexpectedly. Mission engineers knew any additional drop in power could trigger the spacecraft’s undervoltage fault protection system, which would shut down components on its own to safeguard the probe, requiring recovery by the flight team — a lengthy process that carries its own risks.

The Voyager team needed to act first.

“While shutting down a science instrument is not anybody’s preference, it is the best option available,” said Kareem Badaruddin, Voyager mission manager at JPL. “Voyager 1 still has two remaining operating science instruments — one that listens to plasma waves and one that measures magnetic fields. They are still working great, sending back data from a region of space no other human-made craft has ever explored. The team remains focused on keeping both Voyagers going for as long as possible.”

Far-out plan

The choice of which instrument to turn off next wasn’t made in the heat of the moment. Years ago, the Voyager science and engineering teams sat down together and agreed on the order in which they would shut off parts of the spacecraft while ensuring the mission can continue to conduct its unique science. Of the 10 identical sets of instruments that each spacecraft carries, seven have been shut off so far. For Voyager 1, the LECP was next on that list. The team shut off the LECP on Voyager 2 in March 2025.

Because Voyager 1 is more than 15 billion miles (25 billion kilometers) from Earth, the sequence of commands to shut down the instrument will take 23 or so hours to reach the spacecraft, and the shutdown process itself will take about three hours and 15 minutes to complete. One part of the LECP — a small motor that spins the sensor in a circle to scan in all directions — will remain on. It uses little power (0.5 watts), and keeping it running gives the team the best chance of being able to turn the instrument back on someday if they find extra power.

What comes next

Engineers are confident that shutting down the LECP will give Voyager 1 about a year of breathing room. They are using the time to finalize a more ambitious energy-saving fix for both Voyagers they call “the Big Bang,” which is designed to further extend Voyager operations. The idea is to swap out a group of powered devices all at once — hence the nickname — turning some things off and replacing them with lower-power alternatives to keep the spacecraft warm enough to continue gathering science data.

The team will implement the Big Bang on Voyager 2 first, which has a little more power to spare and is closer to Earth, making it the safer test subject. Tests are planned for May and June 2026. If they go well, the team will attempt the same fix on Voyager 1 no sooner than July. If it works, there is even a chance that Voyager 1’s LECP could be switched back on.

DC Agle / Calla Cofield
Jet Propulsion Laboratory, Pasadena, Calif.
818-354-5011 / 626-808-2469
agle@jpl.nasa.gov / calla.e.cofield@jpl.nasa.gov

NASA Science Editorial Team

April 17, 2026 4:46PM

Categories

tl;dr: on Apple Silicon, a WebAssembly module's linear memory can be shared directly with the GPU: no copies, no serialization, no intermediate buffers. The CPU and GPU read and write the same physical bytes. End-to-end, it works: a Wasm guest fills a matrix in its linear memory, the GPU reads it, computes, writes back, and the guest sees the result through the same pointer, same memory, zero copies.

Normally Wasm and GPUs are separated by an expensive serialization boundary: on most hardware, getting data from a VM sandbox to an accelerator means copying across a bus. Apple Silicon's Unified Memory Architecture erases that boundary (no bus, same physical memory), and what falls out is a runtime where Wasm is the control plane and the GPU is the compute plane, with near-zero overhead between them.

I'm building something called Driftwood that exploits this for stateful AI inference ... and this post is about the foundation (how the zero-copy chain works, what I measured, what it opens up). Still early, still poking at it.

Why this is normally hard

Quick background, for anyone who doesn't live in this stack: WebAssembly gives you a sandbox. Your module gets a flat byte array (linear memory) and that's the universe ... everything outside is mediated by "host" function calls. The whole point is isolation, portability, determinism.

GPUs also want a flat byte array, but a specific kind: page-aligned, pinned, accessible to the DMA engine. On a discrete GPU (think NVIDIA, or AMD), that memory sits across a PCIe bus from the CPU, so getting data from a Wasm module's linear memory to the GPU means: copy out of the sandbox into host memory, then copy across the bus into GPU memory. Two copies, two latency hits, and an awkward impedance mismatch between "isolated VM" and "hardware accelerator."

Apple Silicon changes the physics. The CPU and GPU share the same physical memory (Apple's Unified Memory Architecture) ... no bus! A pointer the CPU can read, the GPU can also read, from the same DRAM. The real question: can you thread that pointer through the layers of abstraction (the Wasm runtime, the GPU API) without anyone making a defensive copy along the way?

Turns out ... you can!

Three links. I validated each one on its own before trying to compose them: it's the kind of thing where if you skip the isolation step and the whole pipeline breaks, you have no idea "which joint is leaking".

Link 1: mmap gives you page-aligned memory. On ARM64 macOS, mmap with MAP_ANON | MAP_PRIVATE returns 16 KB-aligned addresses. This isn't a lucky accident, it happens to be the ARM64 page size, and mmap aligns by contract. The alignment matters because Metal requires it.

Link 2: Metal accepts that pointer without copying. MTLDevice.makeBuffer(bytesNoCopy:length:) wraps an existing pointer as a Metal buffer. On Apple Silicon, this is the zero-copy path, i.e. the GPU accesses the same physical memory the CPU does. I verified pointer identity: the MTLBuffer.contents() pointer equals the original mmap pointer. I verified no hidden copies: RSS delta was 0.03 MB (measurement noise), compared to 16.78 MB for the explicit-copy path. And, same compute latency either way.

Link 3: Wasmtime lets you bring your own allocator. Wasmtime's MemoryCreator trait lets you control how linear memory is allocated. Instead of letting Wasmtime call mmap internally, you provide the backing memory yourself. I implement MemoryCreator to return our own mmap region, and Wasmtime's memory.data_ptr() returns exactly the pointer I handed it. The Wasm module reads and writes through Wasmtime's memory API; the GPU reads and writes through the Metal buffer; both are operating on the same bytes.

The composition: allocate an mmap region, hand it to both Wasmtime (as the actor's linear memory) and Metal (as a GPU buffer). The Wasm module writes data at known offsets, the GPU computes on it in place, and the results appear in the module's linear memory with no copies and no explicit data transfer.

I tested the full chain with a 128×128 matrix multiply: the Wasm module fills matrices A and B, the GPU runs a GEMM shader, the module reads result C back. Zero errors across 16,384 elements. Small test, but it's the kind of thing where either it all lines up or you get garbage, so zero errors is the signal I wanted.

What I measured

Three things I cared about: pointer identity (is it actually zero-copy?), memory overhead (any hidden copies sneaking in?), and correctness (does the GPU see what Wasm wrote?).

  Measurement                     Zero-copy path     Copy path
  ─────────────────────────────────────────────────────────────
  Pointer identity                mmap == MTLBuffer   different addrs
  RSS delta (16 MB region)        0.03 MB             16.78 MB
  GEMM latency (128×128)          ~6.75 ms            ~6.75 ms
  Correctness (16K elements)      0 errors            0 errors

The latency equivalence makes sense: on UMA, the compute itself is identical either way. The memory picture is where it shows up: the zero-copy path has essentially no overhead for making data GPU-accessible, and the copy path doubles your memory footprint.

At small tensor sizes, nobody cares. At the scale of KV caches in transformer inference (hundreds of megabytes per conversation) it's the difference between fitting four actors in memory or two. That's the regime I actually want to operate in, so the memory part matters.

From zero-copy to inference

So now I've got a primitive: Wasm and the GPU share memory with no overhead. What do you do with it?

I plugged the chain into Apple's MLX framework and ran Llama 3.2 1B Instruct from a Wasm actor: a full transformer decoder written in Rust, compiled to a native host runtime, driving inference on the Apple Silicon GPU through host function calls. (I was too lazy to wire up a custom kernel path from scratch, and ... MLX was there)

Measured latencies, running Llama 3.2 1B (4-bit quantized, 695 MB) on a 2021 M1 Macbook Pro (old personal laptop, I'll re-evaluate on a proper Mac Studio someday when I can get my hands on it ):

  Operation                    Latency
  ──────────────────────────────────────
  Model load (safetensors)     229 ms      (one-time)
  Prefill (5 tokens)           106 ms
  Per-token generation          ~9 ms
  Host function boundary        negligible

The host function boundary (the Wasm-to-GPU dispatch) isn't measurable against the inference cost. Anyone who's worked with sandboxed runtimes has probably winced at the thought of crossing that boundary per dispatch. On this hardware, it's not a thing.

KV cache portability

Transformers maintain a key-value cache that accumulates context across conversation turns, which is normally ephemeral (kill the process, lose the cache, start over). If you've tried running local inference, you know the feeling.

Because the cache lives in GPU-accessible memory that I control, I can serialize it. So, I dump the KV cache to safetensors format (standard ML tensor serialization, nothing exotic) and restore it later, on the same machine, or a different machine, or potentially against a different model on a different machine! That last one I haven't tested across meaningfully different architectures yet ... we'll see.

  Operation                    Latency      Size
  ───────────────────────────────────────────────────
  Serialize (24 tokens)        1.1 ms       1.58 MB (~66 KB/token)
  Restore from disk            1.4 ms
  Re-prefill from scratch      67.7 ms      (the alternative)
  ───────────────────────────────────────────────────
  Speedup from restore:        5.45×
  Round-trip fidelity:         bit-identical (10/10 tokens match)

5.45× at 24 tokens, and the ratio improves with context length: restore time is nearly constant, re-prefill scales linearly. At 4,096 tokens, restore would be around 100× faster than recomputation (I haven't actually pushed it to 4,096 yet; that's napkin math extrapolating from the constant-vs-linear shape).

This is the basis for stateful actor mobility: freezing a conversation mid-exchange, moving it somewhere else, thawing it with full context intact. The Wasm module's linear memory captures the actor's logical state; the KV cache captures the inference engine's accumulated context. Together: a portable snapshot of a running AI conversation (or, at least, that's the plan ).

What's being built

Driftwood is a runtime for stateful Wasm actors with GPU inference. The zero-copy chain is the foundation: on top of it I'm going to add on actor snapshots (freeze and resume any conversation), checkpoint portability (move inference state across machines), and multi-model support (the snapshot format is model-agnostic, so in theory the actor's identity survives model swaps ... which might work, will revisit once I test it).

This is all early, still stitching things together. But the "physics" works: Wasm and the GPU can share memory on Apple Silicon with zero overhead, the KV cache is portable, and a full transformer runs from a sandboxed actor at native speed. The next things I want to poke at: whether the snapshot really survives a model swap, whether the chain holds up on larger models, and whether I'm missing some obvious reason this will fall over at scale. Slow and steady ...

More on the actor model and snapshot architecture in a future post, once I've actually shipped something past the "physics works" stage.

Here I'll share my first impressions with ROCm and Strix Halo and how I've set up everything.

128GB efficiently shared between the CPU and GPU.

I'm used to working with Ubuntu, so I stuck with it in the supported 24.04 LTS version, and just followed the official installation instructions.

It seems that things wouldn't work without a BIOS update: PyTorch was unable to find the GPU. This was easily done on the BIOS settings: it was able to connect to my Wifi network and download it automatically.

Also on the BIOS settings, you might need to make sure you set the reserved video memory to a low value and let the memory be shared between the CPU and GPU using the GTT. The reserved memory can be as low as 512MB.

Implications:

• The CPU is not able to use the GPU reserved memory.
• The GPU can use the total of Reserved + GTT, but utilizing both simultaneously can be less efficient than a single large GTT pool due to fragmentation and addressing overhead.
• Some legacy games or software sadly might see the GPU memory as 512 MB and refuse to work, this has not happened to me so far though.

Then on /etc/default/grub, I've made this change:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash ttm.pages_limit=32768000 amdgpu.gttsize=114688"

and then ran sudo update-grub.

Note that amdgpu.gttsize shouldn't include the whole system memory, you should leave some memory (I read from 4GB to 12GB) reserved to the CPU (Total memory minus reserved GPU minus GTT) for the sake of the stability of the Linux kernel.

This was somewhat tricky because of the weird dependency graph of PyTorch, but eventually I've got it working with:

[project]
name = "myproject"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.13"
dependencies = [
    "torch==2.11.0+rocm7.2",
    "triton-rocm",
]
[tool.uv]
environments = ["sys_platform == 'linux'"]
[[tool.uv.index]]
name = "pytorch-rocm"
url = "https://download.pytorch.org/whl/rocm7.2"
explicit = true
[tool.uv.sources]
torch = { index = "pytorch-rocm" }
torchvision = { index = "pytorch-rocm" }
triton-rocm = { index = "pytorch-rocm" }

and you can even add it this your .bashrc:

alias pytorch='''uvx --extra-index-url https://download.pytorch.org/whl/rocm7.2 \
    --index-strategy unsafe-best-match \
    --with torch==2.11.0+rocm7.2,triton-rocm \
    ipython -c "import torch; print(f\"ROCM: {torch.version.hip}\"); \
    print(f\"GPU available: {torch.cuda.is_available()}\"); import torch.nn as nn" -i
'''

podman run --rm -it --name qwen-coder --device /dev/kfd --device /dev/dri \
--security-opt label=disable --group-add keep-groups -e HSA_OVERRIDE_GFX_VERSION=11.5.0 \
-p 8080:8080 -v /some_path/models:/models:z  ghcr.io/ggml-org/llama.cpp:server-rocm \
-m /models/qwen3.6/model.gguf -ngl 99 -c 327680 --host 0.0.0.0 --port 8080 \
--flash-attn on --no-mmap

Note that you can easily download the model with:

uvx hf download Qwen/Qwen3.6-35B-A3B --local-dir /some_path/models/qwen3.6

And convert to gguf with the convert_hf_to_gguf.py script from the llama.cpp repo:

git clone https://github.com/ggerganov/llama.cpp.git /some_path/llama.cpp

cd /some_path/models/qwen3.6 &&
uvx --extra-index-url https://download.pytorch.org/whl/rocm7.2 \
    --index-strategy unsafe-best-match \
    --with torch==2.11.0+rocm7.2,triton-rocm,transformers \
    ipython /some_path/llama.cpp/convert_hf_to_gguf.py \
    -- . --outfile model.gguf

I'm using a Podman to run Opencode, see my repo on how set it up.

And this is my config to have it work with Llama.cpp:

{
    "$schema": "https://opencode.ai/config.json",
    "provider": {
        "local": {
            "options": {
                "baseURL": "http://localhost:8080/v1",
                "apiKey": "any-string",
                "reasoningEffort": "auto",
                "textVerbosity": "high",
                "supportsToolCalls": true
            },
            "models": {
                "qwen-coder-local": {}
            }
        }
    },
    "model": "local/qwen-coder-local",
    "permission": {
        "*": "ask",
        "read": {
            "*": "allow",
            "*.env": "deny",
            "**/secrets/**": "deny"
        },
        "bash": "allow",
        "edit": "allow",
        "glob": "allow",
        "grep": "allow",
        "websearch": "allow",
        "codesearch": "allow",
        "webfetch": "allow"
    },
    "disabled_providers": [
        "opencode"
    ]
}

So as I promised, my first impressions are: so far, so good, I was able to play with PyTorch and run Qwen3.6 on llama.cpp with a large context window. There were some rough edges, but I think it was quite worth it.

Computer chips that cram billions of electronic devices into a few square inches have powered the digital economy and transformed the world. Scientists may be on the cusp of launching a similar technological revolution — this time using light.

In a significant advance toward that goal, National Institute of Standards and Technology (NIST) scientists and collaborators have pioneered a way to make integrated circuits for light by depositing complex patterns of specialized materials onto silicon wafers. These so-called photonics chips use optical devices such as lasers, waveguides, filters and switches to shuttle light around and process information. The new advance could provide a big boost for emerging technologies such as artificial intelligence, quantum computers and optical atomic clocks.

Making circuitry for light as powerful and ubiquitous as circuitry for electrons is one of today’s technological frontiers, says Scott Papp, a NIST physicist whose group led the research, published this week in Nature. “We’re learning to make complex circuits with many functions, cutting across many application areas.”

Light Speed

When it comes to information transfer and processing, light can do things that electricity can’t. Photons — particles of light — are far zippier than electrons at working their way through circuits.

Laser light is also essential for controlling powerful, emerging quantum technologies such as optical atomic clocks and quantum computers.

But several hurdles remain before integrated photonics can truly hit its stride. One involves lasers. High-quality, compact and efficient lasers exist in only a few wavelengths, or colors, of light. For example, semiconductor lasers are very good at generating infrared light with a wavelength of 980 nanometers, or billionths of a meter — a color just outside the range of human vision. 

Emerging technologies such as optical atomic clocks and quantum computers need laser light in many other colors as well. The lasers that produce those colors are big, costly and power-hungry, effectively confining these quantum technologies to a handful of special-purpose labs. 

By integrating lasers into circuits on chips, scientists hope to help quantum technologies become cheaper and more portable, so they can start to fulfill their vast promise.

A Multilayered Approach

The new NIST photonics chip is a bit like a layer cake. NIST physicists Papp and Grant Brodnik, along with colleagues, started with a standard wafer of silicon coated with silicon dioxide (glass) and lithium niobate, a so-called nonlinear material that can change the color of light coming into it. 

The researchers then added pieces of metal to electrically control how the circuits convert one color of light to others. The scientists also created other metal-lithium niobate interfaces that allowed them to rapidly turn light on and off within the circuits — a crucial ability for data processing and high-speed routing.

The icing on the cake, so to speak, was a second nonlinear material called tantalum pentoxide, or tantala. Tantala can transform light in ways that feel like magic, taking in a single laser color and putting out the full rainbow of visible light colors plus a wide range of infrared wavelengths. Papp and colleagues have spent years developing techniques to fabricate circuits out of tantala without heating it up, allowing the material to be deposited onto other materials without damaging them. 

By patterning the different materials on top of each other in a three-dimensional stack, the researchers produced a single chip that efficiently routes light between layers. That allowed them to merge the light-manipulating wizardry of tantala with the controllability of lithium niobate. The new technique “allows seamless integration,” says Brodnik. “The real power is that tantala can be added to existing circuitry.”

Ultimately, the researchers were able to fit roughly 50 fingernail-sized chips containing 10,000 photonic circuits, each outputting a unique color, onto a wafer roughly the size of a beer coaster. “We can create all these different colors, just by designing circuits,” says Papp.

One Chip, Many Potential Uses

Quantum technologies such as clocks and computers could be among the biggest beneficiaries of integrated photonics. These devices often use arrays of atoms to store and process information. For each type of atom, physicists need lasers tailored to the atom’s internal quantum energy levels. For example, rubidium atoms, commonly used in quantum computers and clocks, respond to red light with a wavelength of 780 nanometers. Strontium atoms, another popular choice, “see” blue light at 461 nanometers. Shine other colors on the atoms and nothing happens.

The bulky, costly and complicated lasers needed to produce these bespoke colors have been a major hindrance to getting quantum computers and optical clocks out of the lab and into the field, where they could have big impacts. Cheap, low-power, portable optical clocks, for example, could help predict volcanic eruptions and earthquakes, offer an alternative to GPS for positioning and navigation, and help scientists investigate scientific mysteries such as the nature of dark matter. Quantum computers could offer new ways to study the physics and chemistry of drugs and materials.

Integrated photonic circuits aren’t just for quantum. Papp believes NIST’s photonics chips could help efficiently shuttle signals between the specialized chips used by tech firms, potentially making AI-based tools more powerful and efficient. Tech companies are also interested in using photonics to improve virtual reality displays.

While NIST’s chips aren’t yet ready for mass production, the technique used to create them provides a path forward, Papp and Brodnik say. The NIST scientists collaborated with experts at Octave Photonics, a Louisville, Colorado-based startup company founded by former NIST researchers that’s now working to scale up the technology.

“When you see the chip glowing in the lab, taking in invisible light and making all this visible light in one integrated chip — it’s obvious how many potential applications there could be,” says Papp.

Paper: Grant M. Brodnik, Grisha Spektor, Lindell M. Williams, Jizhao Zang, Alexa R. Carollo, Atasi Dan, Jennifer A. Black, David R. Carlson and Scott B. Papp. Monolithic 3D integration of tantalum pentoxide nonlinear photonics. Nature. Published online April 15, 2026. DOI: 10.1038/s41586-026-10379-w

Back in November last year, I started a new job at Intercom, and one of the first projects I got to work on was improving the Intercom monolith CI with some of my new colleagues.

Interestingly, I never got around to talking about CI on this blog, even though I consider it to be one of my main areas of expertise. That topic is way beyond the subject I’d like to talk about here, but just to give a bit of context, a key driver in CI performance and user experience is how fast you can get a Ruby process ready to run tests.

When working with very large test suites, it becomes essential to run tests in parallel. If you have a test suite that runs in say, 1 hour, on paper, you can run it in 15 minutes on 4 workers, or in 6 minutes on 10 workers, and 1 minute on 60 workers.

But that’s a bit too simplistic, in practice, a CI test runner has two phases.

First, a setup phase that all runners have to go through, which includes fetching the source code, getting backing services like the database ready, and booting the application. Once the setup phase is done, the workers can start doing the actually useful work of running tests.

So using the same 1-hour test suite, but now with a 1-minute setup phase, will now take 16 minutes if you are using 4 workers but 2 minutes if you are using 60 parallel workers. That’s a much worse user experience, but also means half of your compute isn’t spent doing the actual work, likely increasing your costs.

All this to say that parallelizing test suites has diminishing returns that are entirely tied to how costly setting up a worker is. The worker setup time is like a fixed cost toll, hence reducing it both improves user experience and reduces cost.

Given that the Intercom monolith CI runs with 1350 parallel workers by default, one second is optimized out of the setup time has 1350 times more impact than a second optimized out of a particular test, and saves over 20 minutes of compute per build.

Hence, while the team also worked on speeding up various slow tests and factories, I personally was very focused on reducing the setup time, shaving every second or even split seconds I could find.

As part of this effort, I looked into speeding up the application boot time, and if you’re a Rubyist, you probably know about Bootsnap.

What Does Bootsnap Even Do?

While Bootsnap has been in the default Rails gemfile for almost a decade now, and is very popular even in non-Rails codebases, based on chats I had with people online or at conferences, I suspect many people don’t quite understand exactly what it is doing. So let me explain just one of the optimizations it performs.

When you require a file (what is internally referred to as a “feature”), Ruby has to perform a very expensive linear search in its load path, something that looks a bit like this:

def search_load_path(feature)
  if path.end_with?(".rb", ".so")
    $LOAD_PATH.each do |load_path|
      absolute_path = File.join(load_path, feature)
      return absolute_path if File.exist?(absolute_path)
    end
    return nil
  else
    search_load_path("#{path}.rb", "#{path}.so")
  end
end
def require_internal(feature)
  return false if $LOADED_FEATURES.include?(feature)
  absolute_path = if File.absolute_path?(feature)
    feature
  else
    search_load_path(feature)
  end
  unless absolute_path
    raise LoadError, "cannot load such file: #{feature}"
  end
  load(absolute_path)
end

The problem with this code loading mechanism is that, while simple, it scales very badly.

A clean Ruby process starts with approximately 8 paths in its load path, so require is relatively cheap, worst case scenario, Ruby will query the file system 16 times.

But then, every single gem you add to your gemfile adds one extra entry in $LOAD_PATH, and will in turn likely call require even more times. Meaning that starting a Ruby program isn’t linear, but much worse. The cost is roughly O(N*M) with N being $LOAD_PATH.size and M being $LOADED_FEATURES.size.

In other words, an application with 400 gems is likely way more than twice as slow to boot compared to an application with 200 gems.

That’s a problem Aaron Patterson explained in his GORUCO 2015 talk, and that talk in turn inspired me to write bootscale, which we used with success in the Shopify monolith for a while, but while very effective, it was quite brittle, so it remained mostly confidential.

Later on, my former colleague Burke Libbey, reimplemented the same idea, but in a much more robust and cleaner way, giving birth to bootsnap, and facilitating its adoption across the community.

But enough history, let’s dive into what it does.

Load Path Caching.

While this is not the only thing Bootsnap does, its main feature is load path caching.

The idea is simple, instead of repeatedly testing the existence of files over and over, Bootsnap eagerly scan all directories in $LOAD_PATH to build a large map of all potentially requirable files, as to provide a way to look them up with just a O(1) hash lookup.

It’s a bit over-simplified, but in essence, Bootsnap’s cache is just a big hash:

@cache = {
  "active_support/core_ext.rb" => "/gems/activesupport-8.1.2/lib/active_support/core_ext.rb",
  "active_support/json.rb" => "/gems/activesupport-8.1.2/lib/active_support/json.rb",
  ...
}

Which then allows to decorate Kernel.require, as to cheaply translate relative paths into absolute ones, hence entirely sidestepping Ruby’s slow search mechanism:

def require(feature)
  unless File.absolute_path?(feature)
    if feature.end_with(".rb", ".so")
      feature = Bootsnap::LoadPathCache.lookup(feature)
    else
      feature = Bootsnap::LoadPathCache.lookup("#{feature}.rb") || Bootsnap::LoadPathCache.lookup("#{feature}")
    end
  end
  require_without_bootsnap(feature)
end

There are, of course, many subtle corner cases Bootsnap has to deal with to reproduce Ruby’s behavior as accurately as possible, but conceptually, Bootsnap is quite simple and reliable.

Thanks to this cache, instead of scanning and checking the existence of up to 2*N files in every require call, we only need to pay a one-time cost, which is quickly amortized.

Cache Invalidation

Now, the problem with adding a cache is that you need to know when it’s no longer valid, and as the famous maxim says, cache invalidation is one of the hardest problems in programming.

Hence, Bootsnap can’t just persist its cache across CI builds, as any added or removed file in any of the load paths MUST invalidate the cache.

The way Bootsnap does it is that in the cache, it records the mtime of the scanned directories. Whenever you add or remove a file in a directory, the directory’s mtime is updated. However, its parent directory mtime is left unchanged:

require "fileutils"
FileUtils.rm_rf("/tmp/test/")
FileUtils.mkdir_p("/tmp/test/dir/subdir")
p File.mtime("/tmp/test/dir").to_f        # 1776351416.5027087
p File.mtime("/tmp/test/dir/subdir").to_f # 1776351416.5027075
File.write("/tmp/test/dir/subdir/file.txt", "1")
p File.mtime("/tmp/test/dir").to_f        # 1776351416.5027087
p File.mtime("/tmp/test/dir/subdir").to_f # 1776351416.502805

If file and directory mtime were updated recursively, that would be extremely powerful for many tools like Bootsnap, however I suspect it wasn’t done this way out of performance concerns.

As such, Bootsnap has to recursively check the mtime of all directories in all load paths whenever it needs to revalidate the cache. That’s cheaper than rebuilding the full cache, but still potentially thousands of stat(2) syscalls, so quite costly.

More importantly, on CI systems it’s relatively common to check out code using git, and git doesn’t care about mtime¹. Which means in many cases, this cache won’t be re-usable across builds or machines, hence it will need to be rebuilt every time, making the scanning performance important.

N+1 Syscalls

On the Intercom monorepo, scanning all load paths was just shy of a second², and as I said previously, even a split second was important to me, given it was part of the “setup time”. So I was keen on finding ways to improve it.

To understand the issue, let’s look at a very simplified implementation of Bootsnap’s load path scanner:

def scan(dir_path, requirables = [], directories = [])
  Dir.foreach(dir_path) do |name|
    path = "#{absolute_dir_path}/#{name}"
    if File.directory?(path)
      directory << path
      scan(path, requirables, directories)
    elsif name.end_with?(".rb", ".so")
      requirables << name
    end
  end
end

Simply put, for each entry of a directory, if it’s also a directory, we record it in the list and recurse, otherwise, if it has an extension we care about (.rb, .so, .bundle, etc) we add it to the list of requirable files.

At that stage, you might wonder why all this code isn’t just Dir["**/*.{rb,so}"], but Bootsnap needs to exactly match Ruby’s behavior otherwise it could change the behavior of programs. So while it is far fetched to imagine a project with directories named something.rb or somethingelse.so, assuming such directories don’t exist would be a correctness bug.

There’s also some other subtleties, like needing to keep a list of all directories, even the ones not yet containing requirable files, so as to be able to revalidate the cache.

As mentioned, the real version is noticeably more complex, but from a performance standpoint, they’re equivalent.

Anyways, the problem with that path scanner, is that it’s essentially the system programming equivalent to what N+1 queries are to web programming, except with system calls.

While they’re much faster than a database query, syscalls are still something you wish to avoid or minimize when working at this level of abstraction, as they involve a context switch into the kernel.

The actual cost of a syscall depends on which system the program work on. For instance, Linux syscalls are generally much faster than macOS ones, and some calls don’t even need a context switch. On the other side, some macOS syscalls like open(2) have a massive overhead because of some security features.

In this specific case, for each entry in the directory, we’ll call File.directory?, which results in a stat(2) syscall. But this N+1 syscall was a long-known issue even in early UNIX programs written in C, that’s why at least on Linux and BSD, readdir(3), which is the API to read the content of a directory, exposes a d_type member, allowing us to know whether that directory entry is a directory, a regular file, or something else without needing to issue a stat(2) call.

Unfortunately, while Ruby does use d_type internally to speed up methods like Dir[], it doesn’t expose it to the Dir.foreach block.

This wasn’t a new issue for me, I knew about that problem back in 2020, as back then, I was already looking at speeding up Bootsnap and Zeitwerk (which have the same issue). That’s why back then, I opened a feature request for a Dir.scan method, unfortunately that ticket never got any traction.

So I thought it was time to try again.

Implementing Dir.scan

Instead of reviving the old ticket, I decided to try again from scratch, and this time to include a prototype implementation. Instead of adding a new method, I decided extend the existing Dir methods like foreach, so that they’d yield a second parameter to represent the file type as a symbol.

This initial prototype sped up recursively walking directories by about 2x. Soon after, Nobuyoshi Nakada, AKA nobu noticed my pull request and implemented an alternative version that yielded File::Stat objects instead of symbols, which I thought was much more elegant, so I opened a new feature request proposing his API.

But from experience, I knew that even in the best-case scenario, I’d need to wait for the next developer meeting before I’d get an OK from Matz, which means it would only make it into Ruby 4.1.

Waiting a full year to improve Bootsnap wasn’t very satisfactory, but since Bootsnap already ships with a C extension, I thought I could just implement that API in Bootsnap itself, to get the performance improvement immediately.

Benchmarked on Intercom’s monolith (only the repo, not the dependencies) showed the same 2x improvement:

ruby 3.4.4 (2025-05-14 revision a38531fd3f) +PRISM [arm64-darwin25]
Warming up --------------------------------------
                orig     1.000 i/100ms
                 opt     1.000 i/100ms
Calculating -------------------------------------
                orig      1.988 (± 0.0%) i/s  (502.94 ms/i) -     10.000 in   5.031382s
                 opt      4.297 (± 0.0%) i/s  (232.70 ms/i) -     22.000 in   5.120236s
Comparison:
                orig:        2.0 i/s
                 opt:        4.3 i/s - 2.16x  faster

Bootsnap was now able to scan ~32k files in ~10k repositories in 230ms, while the previous implementation needed 500ms.

Later on, my Ruby feature request was discussed at the developer meeting, and a few concerns were raised, notably it was considered that changing the signature of existing methods could cause backward compatibility issues.

Instead, after a few rounds of discussion, we settled on a new method: Dir.scan:

Dir.scan(path) do |name, type|
  case type
  when :directory
    # ...
  when :link
    # ...
  when :file
    # ...
  end
end

This new feature will be available in Ruby 4.1.0.

Other Path Methods

While this N+1 issue was definitely the main hotspot, and this 2x win felt good, I tend to treat performance gains like mushroom hunting. When you find a mushroom, it usually means that it’s an area where they grow well, and that no other mushroom hunter has passed by recently.

Well, in my experience, unoptimized code is the same. If you find a piece of code that is much slower than it could be, it suggests nobody ever needed it to be faster, hence it’s likely the same for other pieces of code in the same area.

In this case, another Ruby method Bootsnap calls a lot was File.join, and while it wasn’t a major hotspot, it still was visible on boot profile, so I figured it was worth looking into.

But how are you supposed to tell if some code is slower than it should be?

What commonly slows down a given method is its handling of corner cases, so a good comparison point is a naive implementation that only considers the happy path. In our case, the most common usage of File.join by far is basically just a concatenation:

def file_join(parent, child)
  "#{parent}/#{child}"
end

So if we benchmark this simplistic implementation against the real File.join, we should have a vague idea of how much performance is left on the table:

# frozen_string_literal: true
require "benchmark/ips"
dir = "/Users/byroot/src/github.com/byroot/ruby/build"
entry = "path/to/file.txt"
Benchmark.ips do |x|
  x.report("File.join") { File.join(dir, entry) }
  x.report("interpolation") { "#{dir}/#{entry}" }
  x.compare!(order: :baseline)
end

ruby 4.0.2 (2026-03-17 revision d3da9fec82) +YJIT +PRISM [arm64-darwin25]
Warming up --------------------------------------
           File.join   429.375k i/100ms
       interpolation     1.560M i/100ms
Calculating -------------------------------------
           File.join      4.336M (± 0.2%) i/s  (230.65 ns/i) -     21.898M in   5.050870s
       interpolation     17.501M (± 0.5%) i/s   (57.14 ns/i) -     88.905M in   5.079969s
Comparison:
           File.join:  4335527.0 i/s
       interpolation: 17501462.6 i/s - 4.04x  faster

Bingo! A 4x difference really didn’t pass the smell test, so I immediately profiled File.join called 10 million times in a loop:

Full profile

What immediately surprised me was that File.join was spending over half its time in encoding-related functions (rb_enc_*), most notably 33% in rb_enc_mbclen:

/**
 * Queries the number of bytes of the character at the passed pointer.
 *
 * @param[in]  p    Pointer to a character's first byte.
 * @param[in]  e    End of the string that has `p`.
 * @param[in]  enc  Encoding of the string.
 * @return     If the character at `p` does  not end until `e`, number of bytes
 *             between `p`  and `e`.   Otherwise the number  of bytes  that the
 *             character at `p` is encoded.
 *
 * @internal
 *
 * Strictly speaking there  are chances when `p`  points to a middle  byte of a
 * wide character.   This function  returns "the  number of  bytes from  `p` to
 * nearest of either `e` or the next character boundary", if you go strict.
 */
int rb_enc_mbclen(const char *p, const char *e, rb_encoding *enc);

Without even looking at the code, this told me there was a large potential for an easy optimization, because File.join, like all other Ruby methods handling paths, rejects paths encoded with non-ASCII compatible encodings:

>> File.join("a".encode(Encoding::UTF_16LE), "b".encode(Encoding::UTF_16LE))
# => 'File.join': path name must be ASCII-compatible (UTF-16LE): "a" (Encoding::CompatibilityError)

So I thought this could be some leftover from a long time ago that could be pruned, hence I started digging into the git history, and found that this multi-byte encoding support was added by nobu in January 2012 (commit ed469831), whereas the code that rejects non-ASCII compatible encoding was only added in October 2012 (commit ad54de2a), again by nobu.

Unfortunately, neither commit message was really explicit in its intent, nor linked to a bug ticket or anything like that. Still, it did look like back in 2012, nobu tried to solve some issues with multi-byte paths, but it was ultimately decided to only accept ASCII-compatible encodings and reject the others.

But a few years of working on Ruby taught me never to assume nobu made a mistake, so before jumping to that conclusion, I figured I’d ask him, just in case:

And after a few hours, he answered me:

Indeed, it was no mistake, but some sort of corner case I didn’t know about involving the Japanese Shift JIS encoding. This wasn’t the first time it happened to me, and probably won’t be the last.

Anyways, in such cases, there is a Wikipedia page that helped me multiple times: Japanese language and computers³. But let me explain the problem here.

ASCII Compatibility

Ruby supports over a hundred string encodings, and some of them are defined as “ASCII-compatible”, which isn’t a very well-defined concept. According to Ruby, both UTF-8 and Shift JIS are ASCII-compatible:

>> Encoding::UTF_8.ascii_compatible?
=> true
>> Encoding::Shift_JIS.ascii_compatible?
=> true

Which in a way isn’t wrong, because both are ASCII superset, meaning valid ASCII is both valid UTF-8 and valid Shift JIS.

However, UTF-8’s killer feature is that it’s way more ASCII compatible than previous multi-byte encodings. All UTF-8 multibyte characters only use codes outside the ASCII range (so higher than 127). Thanks to this, simple ASCII operations like searching for a specific character in the ASCII range can remain simple fixed-length operations:

def backslash?(string)
  string.each_byte do |byte|
    return true if byte == 0x5c # `\` is 0x5c in ASCII 
  end
  false
end

In other words, with UTF-8, if you see a 0x5c byte, you know for sure it’s a backslash character, whereas with Shift-JIS, it may be a backslash character, or it may be a continuation byte of a multi-byte character. For example, 構 is encoded as 0x8d 0x5c. Hence, you can’t efficiently treat Shift-JIS as ASCII, you must use a lookup table to check the width of every character, which is what rb_enc_mbclen does (mbclen -> multi-byte character length), and it’s a very costly operation compared to just iterating over a stream of bytes.

But ultimately, it’s fair to assume the overwhelming majority of paths passed to File.join are encoded in UTF-8 or even pure-ASCII, as such, I could implement a fast path for these encodings and keep the more complex algorithm for the others.

That’s something I already did a few years prior for various string methods, such that Ruby already had a helper to check for such encodings:

static inline bool
rb_str_encindex_fastpath(int encindex)
{
    // The overwhelming majority of strings are in one of these 3 encodings,
    // which are all either ASCII or perfect ASCII supersets.
    // Hence you can use fast, single byte algorithms on them, such as `memchr` etc,
    // without all the overhead of fetching the rb_encoding and using functions such as
    // rb_enc_mbminlen etc.
    // Many other encodings could qualify, but they are expected to be rare occurrences,
    // so it's better to keep that list small.
    switch (encindex) {
      case ENCINDEX_ASCII_8BIT:
      case ENCINDEX_UTF_8:
      case ENCINDEX_US_ASCII:
        return true;
      default:
        return false;
    }
}

Using this helper, I implemented a fastpath for File.join, using single byte comparisons, and that’s when I realized the multi-byte checks weren’t the only thing slowing down File.join and several other path handling methods.

Reverse Search

After every path segment it concatenates, File.join would call chompdirsep to find whether the segment had a trailing path separator. That’s necessary because File.join avoids duplicate separators:

>> File.join("foo/", "/bar")
=> "foo/bar"

But there was something very wrong with its implementation:

static char *
chompdirsep(const char *path, const char *end, rb_encoding *enc)
{
    while (path < end) {
        if (isdirsep(*path)) {
            const char *last = path++;
            while (path < end && isdirsep(*path)) path++;
            if (path >= end) return (char *)last;
        }
        else {
            Inc(path, end, enc);
        }
    }
    return (char *)path;
}

As you can see, the function receives the start and end pointers of the string, and is supposed to return the position of the last meaningful separator, so that extra trailing separators are eliminated by File.join:

>> File.join("foo///", "/bar")
=> "foo/bar"

The logical way to implement such a function would be to start looking from the back of the string, but here it was scanning the entire string, meaning longer paths were disproportionately slower to join than shorter paths.

I’m not one hundred percent sure why it was implemented that way, probably because the multi-byte aware Inc macro was readily available, and implementing a Dec macro would have been a bit trickier, but technically it should have been doable.

In my case, I only cared about optimizing the fast path, so I inlined a single-byte version of it, which searches for duplicate separators from the end of the string:

long trailing_seps = 0;
while (isdirsep(name[len - trailing_seps - 1])) {
    trailing_seps++;
}
rb_str_set_len(result, len - trailing_seps);

And while I was in there, I kept looking for other opportunities.

C Strings

The profile was showing 6.7% of time spent in rb_string_value_cstr, which, after fixing the multi-byte encoding, was now a much bigger deal.

What that function does is that it ensures that a given Ruby string is also a valid “C string”, which implies two things:

• The string is NULL terminated.
• The string does not contain any NULL bytes.

Most Ruby methods dealing with path, do reject strings containing NULL bytes because that’s not valid for a file or directory name:

>> File.join("foo\0bar", "baz")
(irb):1:in 'File.join': string contains null byte (ArgumentError)

However, we actually don’t really care here if the string is NULL-terminated or not, as all we’re doing is concatenating it, we’re not passing it to any C-level API that expects a NULL-terminated string. So rb_string_value_cstr wasn’t really the right function to call, hence I could replace it with rb_str_null_check, which only checks the content of the string.

Variadic Arguments

Another hotpot from the profile was the 10% spent in rb_ary_new_from_values, which, as its name indicates, creates a new array.

The reason is that File.join has some pretty flexible arguments:

>> File.join("a", "b", "c") == File.join("a", ["b", ["c"]])
=> true

So to simplify the implementation, File.join was defined to receive all its arguments in an args Array:

static VALUE
rb_file_s_join(VALUE klass, VALUE args)
{
    return rb_file_join(args);
}

To avoid that extra allocation and copying, I changed it to not create the Array object, and instead receive a pointer into the stack and the number of arguments:

static VALUE
rb_file_s_join(int argc, VALUE *argv, VALUE klass)
{
    return rb_file_join(argc, argv);
}

Allowing for not allocating that extra array in the simpler cases.

Result

All this combined made the common usages of File.join over 7 times faster:

compare-ruby: ruby 4.1.0dev (2026-01-17T14:40:03Z master 00a3b71eaf) +PRISM [arm64-darwin25]
built-ruby: ruby 4.1.0dev (2026-01-18T12:55:15Z spedup-file-join 5948e92e03) +PRISM [arm64-darwin25]
warming up....
|              |compare-ruby|built-ruby|
|:-------------|-----------:|---------:|
|two_strings   |      2.477M|   19.317M|
|              |           -|     7.80x|
|many_strings  |    547.577k|   10.298M|
|              |           -|    18.81x|
|array         |    515.280k|  523.291k|
|              |           -|     1.02x|
|mixed         |    621.840k|  635.422k|
|              |           -|     1.02x|

And now, on Ruby 4.1.0dev, using File.join for two simple paths is faster than using string interpolation:

ruby 4.1.0dev (2026-04-11T18:26:22Z compact-ar-table 06507da144) +YJIT +PRISM [arm64-darwin25]
Warming up --------------------------------------
           File.join     1.944M i/100ms
       interpolation     1.716M i/100ms
Calculating -------------------------------------
           File.join     21.750M (± 0.4%) i/s   (45.98 ns/i) -    108.860M in   5.005112s
       interpolation     19.012M (± 0.6%) i/s   (52.60 ns/i) -     96.111M in   5.055419s
Comparison:
           File.join: 21750287.3 i/s
       interpolation: 19012105.0 i/s - 1.14x  slower

If you are curious, you can read the full pull request.

Other Methods

After finding such low-hanging fruits in File.join, I figured other path handling methods likely had similar issues, and I applied similar optimizations to:

• File.basename
• File.dirname
• File.extname
• File.expand_path

Not that any of these were massive hotspots to my knowledge, but I saw no reason not to optimize them too.

Introduction

This article describes a real PostgreSQL production incident caused by transaction ID wraparound, a failure mode that is both silent and severe. The incident ultimately resulted in a complete write outage. The failure did not occur immediately after a configuration change, nor was it triggered by high load, traffic growth, or infrastructure problems.

In PostgreSQL, every write transaction is assigned a transaction ID (XID). These transaction IDs are drawn from a finite, global counter that advances continuously as transactions are executed. To safely reuse transaction IDs, PostgreSQL requires that old row versions be periodically frozen. Freezing marks data as permanently visible, preventing transaction IDs from aging indefinitely. If freezing does not occur in time, transaction IDs continue to advance toward a hard safety limit.

What makes transaction ID wraparound particularly dangerous is not merely its existence, but the way it manifests. A database can operate normally for months or even years while silently approaching the limit. CPU usage appears healthy. I/O patterns remain stable. Query performance shows no gradual degradation. There are no warning signs that resemble traditional capacity or performance issues.

When the safety limit is finally reached, PostgreSQL has no safe way to continue accepting write transactions without risking data corruption. At that point, PostgreSQL intentionally blocks all write activity. The database effectively becomes read-only, and recovery shifts from routine tuning to an urgent operational incident. The incident described in this article occurred in an environment with a stable and modest workload. There was no traffic spike, no abnormal query behavior, and no operational change immediately preceding the outage. The failure occurred simply because sufficient time had passed without transaction ID freezing being completed.

It is also important to note that this class of failure cannot be meaningfully reproduced through short-lived simulations or conventional performance testing. Transaction ID wraparound is governed by cumulative transaction counts and long-term aging of data. In most environments, it emerges only after months or years of normal operation, making it easy to overlook during testing, staging validation, or initial production rollout.

The purpose of this article is to explain what transaction ID wraparound is, why it is particularly dangerous in production environments, how configuration decisions made long ago can silently lead to it, and why understanding the underlying transaction ID math is essential for running PostgreSQL safely in production.

How Transaction IDs and the Wraparound Problem Work in PostgreSQL

PostgreSQL uses multi-version concurrency control (MVCC) to manage concurrent access to data. Instead of updating rows in place, each change creates a new version of the row while older versions remain until they are no longer needed. To determine which row versions are visible, PostgreSQL assigns a transaction ID (XID) to every write transaction. That XID is stored with the row version and compared against transaction snapshots during reads. Transaction IDs are global across the entire PostgreSQL cluster and are allocated sequentially from a 32-bit counter, making the total XID space finite.

Although the counter can technically represent about four billion values, PostgreSQL enforces a much earlier safety boundary. When the age of the oldest unfrozen transaction ID approaches approximately 2 billion, PostgreSQL considers the system unsafe.

To prevent data corruption caused by transaction ID reuse, PostgreSQL relies on freezing. During vacuum operations, sufficiently old row versions have their transaction IDs replaced with a special frozen marker, making them permanently visible and safe. If freezing does not happen in time and the 2-billion threshold is reached, PostgreSQL deliberately blocks all write operations. INSERT, UPDATE, DELETE, and many DDL commands begin to fail. The database effectively becomes read-only until the issue is resolved.

This behavior is intentional and protective. The system may appear healthy for months or years, but once the limit is crossed, the failure is sudden and absolute.

A Little of Backstory

The system belonged to a mid-sized B2B SaaS organization. PostgreSQL was the primary OLTP database and had been running in production for several years. Database ownership was largely developer-driven. The development team handled schema changes, performance tuning, and most configuration decisions. This worked reasonably well because the workload was stable and modest. The application workload characteristics were unremarkable:

• Approximately 10 write transactions per second
• Short-lived transactions with autocommit enabled
• No batch processing
• No reporting or analytics on the primary

From monitoring dashboards, the system appeared healthy. CPU, memory, and I/O usage were consistently within normal ranges.

A Configuration Decision Made Earlier

Several months before I joined the organization, the team experienced a performance issue related to disk I/O. Autovacuum activity was visible during the incident and was assumed to be contributing to the problem. As a temporary mitigation, autovacuum was disabled on several tables, and in some cases at broader scope. The immediate performance issue was resolved. The system stabilized. Development moved forward.

Autovacuum, however, was never re-enabled. Over time, this configuration became accepted as normal. The databases continued to function, and no immediate problems surfaced. The long-term implications of disabling autovacuum were not actively considered.

When I joined the organization, there was no active database incident. Around that time, I took on core database responsibilities. As expected, my initial focus was on foundational operational work. I validated backups, tested restore procedures, reviewed disaster recovery readiness, and checked replication and failover behavior. These were necessary and correct priorities. The databases had been running for years without visible issues, and there was no immediate signal that something fundamental was wrong.

Transaction ID wraparound health was not reviewed during that initial phase.

The Incident: One Month Later

Approximately one month after I joined and took core responsibility, the incident occurred. The application team reported that write operations were failing. INSERT statements failed. UPDATE statements failed. DDL statements failed.

PostgreSQL had entered transaction ID wraparound protection mode. The database was effectively read-only. At this point, the system was already past the warning phase. PostgreSQL had made a deliberate decision to protect data correctness.

Recovery Under Pressure

The immediate goal was to restore write capability. This was not a tuning exercise; it was a recovery operation. The work involved identifying and terminating long-running or idle transactions that were holding old snapshots, followed by aggressive manual VACUUM FREEZE operations on the most affected tables.

The process was stressful and messy. Freezing had to be pushed hard enough to advance transaction IDs beyond the danger zone while keeping the system stable. Eventually, sufficient freezing was completed, write operations resumed, and the system stabilized.

The Realization: This Was Not an Isolated Case

Once the immediate incident was resolved, it became clear that this could not be treated as a one-off problem.The root cause was not a recent change. It was a configuration decision made years earlier. I started checking other PostgreSQL production systems owned by the same teams. The first query I ran was simple:

SELECT relname, age(relfrozenxid)
FROM pg_class
WHERE relkind = 'r'
ORDER BY age(relfrozenxid) DESC;

The results were concerning. Multiple production databases had tables with very high transaction ID age. In many cases, autovacuum was explicitly disabled on those tables. This was the moment when the seriousness of the situation became clear. The initial incident was not bad luck. It was an early warning.

How Autovacuum Prevents Transaction ID Wraparound

In normal operation, PostgreSQL relies on autovacuum to prevent transaction ID wraparound. Autovacuum periodically scans tables to remove dead row versions and freeze old rows, ensuring transaction IDs do not age indefinitely.

Each table in PostgreSQL tracks a value, called relfrozenxid, which represents the oldest transaction ID in that table that has not yet been frozen. As transactions continue across the cluster, the age of relfrozenxid increases unless vacuuming successfully freezes older row versions. The current state of transaction ID aging can be observed using the following query:

SELECT relname, age(relfrozenxid)
FROM pg_class
WHERE relkind = 'r'
ORDER BY age(relfrozenxid) DESC;
PostgreSQL enforces a hard safety threshold controlled by:

To protect against transaction ID reuse, PostgreSQL enforces a hard safety threshold controlled by the parameter:

SHOW autovacuum_freeze_max_age;

By default, this value is 200,000,000. When the age of a table’s relfrozenxid approaches this limit and freezing cannot progress, PostgreSQL deliberately blocks write operations to prevent data corruption.

If autovacuum is disabled or prevented from running effectively, freezing does not occur automatically. Transaction IDs continue to age silently until the safety threshold is reached, at which point the issue surfaces abruptly as a production outage.

The Math: Why the Failure Was not instant, but Inevitable

Transaction ID consumption depends only on write rate and time. In this system:

• 10 transactions / second
• Transactions per minute: 10 × 60 = 600
• Transactions per hour: 600 × 60 = 36,000
• Transactions per day: 36,000 × 24 = 864,000

So, Transaction IDs consumed per day: 864,000. The PostgreSQL wraparound safety threshold is 200,000,000. The time to reach wraparound risk:  200,000,000 ÷ 864,000 ˜ 231 days.  That is approximately 7.5 months.

No spike was required. No growth was required. Time alone was sufficient.

Unused Tables also cause the wraparound problem

One important detail that made this incident worse was the presence of tables that were no longer in active use. In PostgreSQL, transaction IDs increase globally for the entire database, but freezing is handled at the table level. This difference is easy to miss.

In this environment, several tables had been created earlier for testing or for features that were later abandoned. Because those tables were considered “not really in use,” autovacuum had been disabled on them. Once autovacuum was disabled, those tables stopped being frozen. Their transaction ID state remained fixed at the point of their last insert or vacuum.

Meanwhile, the rest of the application continued to run normally. Write transactions kept happening on other tables, and transaction IDs continued to advance every day. After enough time passed, one of these unused tables ended up having the oldest unfrozen transaction ID in the entire database. At that point, PostgreSQL treated it as a wraparound risk for the whole system.

The key point is simple: a table does not need to be actively used to be dangerous. A single forgotten test table with autovacuum disabled is enough to trigger transaction ID wraparound.

Why SQL Server Does Not Face This Problem

SQL Server does not experience this specific failure mode because it does not rely on a finite, globally aging transaction ID in the way PostgreSQL does.

In PostgreSQL, row versions store transaction IDs that come from a finite global counter. Those IDs must eventually be reused, which is why PostgreSQL requires freezing. If freezing does not occur in time, PostgreSQL must stop write operations to avoid data corruption.

SQL Server uses a different internal approach. Transaction ordering and version visibility are based on Log Sequence Numbers (LSNs) rather than reusable transaction IDs. LSNs are monotonically increasing values generated by the transaction log and are not reused in a way that creates wraparound ambiguity.

When SQL Server uses row versioning (such as snapshot isolation or read committed snapshot), older row versions are tracked using LSNs and stored in tempdb. Cleanup depends on active transactions and log truncation, not on reaching a global identifier reuse limit. If SQL Server cannot clean up old versions, the impact is operational rather than absolute. tempdb may grow, performance may degrade, or blocking may increase. However, SQL Server does not need to halt all write operations to preserve correctness.

In short, PostgreSQL must enforce a hard stop when transaction ID reuse becomes unsafe, while SQL Server avoids this specific problem by using an LSN-based design instead of finite, reusable transaction IDs.

Conclusion

This production incident was not caused by load, traffic growth, or inefficient queries. It was caused by transaction ID freezing not occurring over an extended period of time. The databases appeared healthy for years. The workload was stable. Monitoring showed nothing unusual. The risk accumulated quietly as a function of time.

The incident surfaced only after I had taken core responsibility, and the recovery required aggressive and careful intervention. Subsequent checks showed that the same risk existed across other systems as well. PostgreSQL behaved exactly as designed. It protected data integrity by stopping writes before corruption could occur.

Transaction ID wraparound is not an edge case. It is a predictable outcome when freezing is ignored or misunderstood. Understanding the math behind transaction ID consumption and treating autovacuum as a safety mechanism rather than a tuning option is essential for running PostgreSQL in production.

or: How I Learned to Stop Worrying and Admire Electronics

I have been feeling hopeless ever since Claude started doing most of my work. I wanted to put the fun back in life by doing something - something which i used to feel back in college, back when i was doing dumb shit - building silly things in python or whatnot.

I thought how crazy it will be to build a robot? after all, Claude is so smart - why can’t i just prompt the shit out of it and build my own silly robot?

So i started browsing internet, checking videos on youtube on how to build a robot. Turns out, it does need a lot of initial investment - atleast ₹20,000 (~$200). i thought i will end up wasting my money by buying all these components and then not able to stitch it together.

While researching, i find out - i don’t know anything about anything.

i don’t know how electricity works.

what’s the deal with so many electronics component?

why is every one of these thing look like a silicon chip?

what’s a breadboard?

and why is everyone saying don’t forget to check voltage with multimeter?

Watching bunch of youtubers making DIY robot didn’t help either.

So i thought what if i can make “Hello, World!” equivalent in robotics?
I looked around and finally aligned myself to make a teeny-tiny robot car - and use my phone as remote control.

To make robot car or what most tutorial videos call it on youtube, “Arduino 4WD Robot Car”, you need to buy bunch of small things and stitch it together.

To keep things simple - let’s just assume there is nothing robotic about it. just imagine the toy car you had in your childhood - there was a battery and a motor and maybe few wheels.

when you put one wire to the +ive terminal and another wire to -ive terminal - it makes motor move. if you change the terminals - the motor moves in opposite direction.

everything we are going to do will be based on this principle - Voltage difference moves the motor

What makes motors move is actually the Voltage difference b/w its terminals.

Voltage, Current and Resistance - the holy trinity - it always confused me back in school days.

After watching bunch of videos, talking to LLMs - i kinda picture these things as following:

Voltage is just difference b/w potential energy between two points

Think of water flowing down from mountain top.

• At the top of mountain - when water flows down - it will move fast toward the ground/sea level. why? because there is height difference and gravity.

• Now if the mountain has rough edges, bunch of rocks - the water will move slower.

• Height difference b/w mountain top and ground -> Potential Energy -> Voltage

• The water flow rate -> how many electrons moved per second -> Current

• The rocks, rough edges -> Resistance

Same principle applies to batteries. When battery dies - the potential difference b/w its terminals goes to zero. the resistance is fixed. the number of electrons remains the same in wire before and after. but the current goes to zero because there is no force to move the current.

In a way, you can say that Voltage is like a gravity for electrons - the difference make the electrons move

A small note on convention: the glowing beads above show the direction of current (+ → motor → −), which is what every schematic, datasheet, and wiring diagram uses. Electrons are negatively charged, so they physically drift in the opposite direction (− → motor → +). Same phenomenon, two different arrows — we’ll stick with current throughout this post.

Alright! let’s go back to assembling the car

1. 4WD chassis kit^[1] - 4 wheels, 4 motors, a chassis and wires.

Note: it won’t come in assembled form. you need to assemble it yourself, do soldering to stick the wires on the motor terminals etc.

2. 18650 Li-ion battery^[2] - 2 Batteries specifically 18650 Li-ion battery + Battery Holder

These are most common rechargeable batteries - used in bunch of electronic items.

The amazing thing about 18650 Li-ion battery is that initial Tesla cars used to have this exact same battery - hundreds of 18650 cells enclosed in a box.

3. TP4056^[3] Charger for battery - you will soon need it The charger looks weird. but it cost only ₹14(~15 Cents). you need to use Type C to charge it. One battery at a time only, and it does take time.

4. Multimeter^[4] - The most important purchase you are ever going to make here is multimeter.

5. ESP32^[5] - the brain of your car

6. TB6612FNG^[6] - this is a MOSFET based motor module - it’s the most important component here. this thing makes voltage dance.

7. M-to-M jumper wires^[7] and M-to-F jumper wires^[8]

8. Soldering iron kit^[9]

9. Breadboard^[10] - Buy the bigger one so that you can do bunch of experiments at home with capacitors, resistance etc

10. Buck Converter^[11] - To protect ESP32 from battery’s high voltage

First assemble the wheels, motors and chassis to create our base toy car. Make sure to test the wheel by connecting directly to battery and write down the direction it moves.

i wrote down explicitly and marked on the chassis

a. On Top left - when Red -> +ive and Black -> -ive terminal on battery. the wheel moves forward

b. On Rear left - When Red -> +ive and Black -> -ive terminal on battery. the wheel moves backward.

This is critical step. Remember the wheel direction and connectivity with battery terminals.

This is the brain of your car. you will write logic(code) to trigger on,off on the pins. these pins are called GPIO (General Purpose Input Output) pins. a pin can be either in high(3.3V) or low(0V) state. the ESP32 is not going to directly power the motor - it is only going to give instructions.

Now to control the speed of motor/wheel - you need to control the Voltage or Current. If you pass the higher Current - the wheel will move faster. if you pass lower Current or Voltage - the motor will move slower.

From V = I x R - Since input V is supplied by those 2x 18650 Li-ion Batteries.
You can control the input Current by tweaking the Resistance. If you put a higher resistance before the motor - you can effectively reduce the Current it can get. but higher Resistance means high temperature - not the best way to handle effective Voltage or Current

Smart people have figured out long ago that to control the Voltage - you don’t need to add Resistance and waste energy in form of heat. You need to modulate the electric signals.

The term duty cycle describes the proportion of ‘on’ time to the regular interval or ‘period’ of time; a low duty cycle corresponds to low power, because the power is off for most of the time. ^[12]

This sits between ESP32 and Motors. It listens to the ESP32 signals and uses it to control the Voltage with Pulse Width Modulation (PWM).
This H-Bridge is MOSFET based and used to control the speed and direction of the wheel. you can read more about TBFNG internals^[13] here

You need to flash/burn the code in ESP32 so that it can listen to your inputs and control the TB6612FNG. The code part is pretty easy - you can take the references from web or use Claude to build those things. the most important thing i learned is - it’s all MMIO (Memory Mapped IO^[14]).

You are writing program in C by interacting with predefined memory addresses on those pins in ESP32 which eventually controlling the hardware. This brilliant video by Artful Bytes^[15] explained this mind blowing concept - i am still not sure i understand it but it’s mindblowing nonetheless

My initial goal was to just move one side of wheels - with fixed set of instructions. Forward - Stop - Backward.

You need to follow the instructions to wire the component - follow the pins - pins will follow the current. you will write programs to trigger 0/1 on the pins.

You can follow the detailed wiring instructions^[17] here

It took a lot of time to make this work because the H-Bridge i bought initially was not working properly.
I got tired of checking voltage b/w pins and arguing with Claude but somehow i persisted because i just wanted to make it work.

Finally after everything - when the wheels started moving - that was my Eureka!! moment.

Most important thing i learned while doing all of this is

Just like everything is file in linux. i think everything is voltage in electronics

Now i am glad Claude and other LLM exist in this world.

As Scott Galloway^[18] say Life is so rich

I tried Claude Design yesterday and I have a theory for how this whole thing shakes out.

As product teams scaled and design needed to justify itself inside engineering orgs, it was pushed toward systemization — and Figma invented its own primitives to make that work: components, styles, variables, props, and so on. Some concepts are borrowed from programming, some aren’t, and the whole thing doesn’t neatly map onto anything. Guidance evolves, migrations pile up, and if you want to automate any of it you’re stuck with a handful of shoddy plugins. The beast is hairy enough that entire design roles now specialize in wrangling the system itself.

There’s always been a tense push-pull between Figma and code over what the source of truth should be. Figma won over Sketch partially by staking its claim there — their tooling would be canonical.

That victory had a hidden cost. By nature of having a locked-down, largely-undocumented format that’s painful to work with programmatically, Figma accidentally excluded themselves from the training data that would have made them relevant in the agentic era. LLMs were trained on code, not Figma primitives, so models never learned them. As code becomes easier for designers to write and agents keep improving, the source of truth will naturally migrate back to code. And all the baroque infrastructure Figma had to introduce over the past decade will look nuts by comparison. Why fuss around in a lossy approximation of the thing when you can work directly in the medium where it will actually live? If we want to make pottery, why are we painting watercolors of the pot instead of just throwing the clay?

At work, we’ve spent quite a bit of time back-porting design changes made directly in code back to Figma and it is not fun. I can’t share that file, but for a fair comparison, this is Figma’s own design system file for their product. I have to assume it was built by the most competent design system team you can find. And yet…

These are Figma’s own files. Built by their own team. This is the gold standard.

Imagine debugging a color that looks wrong. You check the component. The component uses a variable. The variable is aliased to another variable. That variable references a mode. The mode is overridden at the instance level. The instance lives inside a nested component with a library swap applied. At this point, you’re either considering picking up code or moving to the countryside and becoming a sheep farmer because one more minute of this will make you lose your goddamn mind.

So as the source of truth shifts back to code, Figma is left in an odd spot: holding a largely manual, pre-agentic system that nobody in their right mind would design from scratch today.

I think design tooling forks into two distinct shapes from here — and there’s almost a clock resetting between Figma and every other tool competing to answer the same question they answered in 2016: who can help me, a designer, get my ideas out fastest?

Spoiler: it’s not Figma Make. Figma Make feels like it primarily benefits people who have already drunk the Kool-Aid — it reads from Figma styles, component libraries, and proprietary props (or, as I like to call them, Prop Props), and it’s the only tool in this new landscape still pretending the design file is canonical. It’s the tool for people who want/have to stay inside the system.

Claude Design is the first of those two tools, which takes the opposite bet. There’s an Arts and Crafts principle called “truth to materials” — the idea that a thing should be honest about what it is and how it’s made, rather than masquerading as something else. Figma ended up being the opposite of this: a set of extremely rigid schemas with a free-form “just vibes, man” costume over the top. Like a Type-A personality physically incapable of relaxing, forced to perform chill while internally screaming that your frames aren’t nested and your tokens are detached and nothing is on the grid. Claude Design, for all its roughness, is at least honest about what it is: HTML and JS all the way down.

And it has a massive structural advantage: its sibling is Claude Code. Eventually, I can see Claude Design just dumping things directly into Claude Code and vice versa. Claude Design’s onboarding already lets you import your repos. The feedback loop between design and implementation — which has been a source of friction since the beginning of time — becomes a single conversation.

The other tool that emerges from this moment will have no expectation of code at all. It’ll be a pure exploration environment — somewhere to drop rectangles, stack layer styles, fuss with blend modes and gradients, and go completely nuts, unconstrained by systems or prompting conventions. Maybe it’s an iPad app with Pencil support where you just quickly sketch a bunch of rectangles. 37signals could do something really funny right now. Or maybe it goes in the opposite direction — something more like Photoshop that goes all-in on high-fidelity compositing and lets our imaginations run wild, now that we’re no longer beholden to the ceiling of what you can do with CSS effects. Doesn’t it seem kinda weird how for 90% of its life, Figma’s only layer effect was a drop shadow or a blur?

Figma’s Sketch moment is rapidly approaching. And if you said that sentence to a Victorian child, they would probably have a stroke.

Post Script

The following are messages meant only for the teams behind Sketch and Figma. If neither apply to you, you can skedaddle.

To Figma: I can see a world where this post does numbers in the Figma internal Slack. If that’s the case and you’re reading this from Figma: this wouldn’t have happened if you hired me last year when I was interviewing. Your loss, big dawg.

To Sketch: GET YOUR HEADS OUTTA YOUR ASSES AND GIVE EM HELL. ADD PARTICLE EFFECTS. ADD DEBOSSING EFFECTS. MESH TRANSFORMS. FUCK IT, ADD METAL SHADERS. GO NUTS. STOP COASTING OFF OF BEING MAC NATIVE. QUIT DRINKING COCOA AND GET THIRSTY FOR BLOOD.

To mom: Sorry for cursing.

The scene is right out of the 1950s with students pecking away at manual typewriters, the machines dinging at the end of each line.

Once each semester, Grit Matthias Phelps, a German language instructor at Cornell University, introduces her students to the raw feeling of typing without online assistance. No screens, online dictionaries, spellcheckers or delete keys.

The exercise started in spring 2023 as Phelps grew frustrated with the reality that students were using generative AI and online translation platforms to churn out grammatically perfect assignments.

“What’s the point of me reading it if it’s already correct anyway, and you didn’t write it yourself? Could you produce it without your computer?” said Phelps.

She wanted students to understand what writing, thinking and classrooms were like before everything turned digital. So, she found a few dozen old manual typewriters in thrift shops and online marketplaces, and created what her syllabus calls an “analog” assignment.

It might be premature to say that typewriters are making a comeback beyond Cornell’s campus. But the revival is part of a national trend toward old-school testing methods like in-class pen-and-paper exams and oral tests to prevent AI use for assignments on laptops.

Typewriters bring ‘old days’ taste of doing one thing at a time

Students arrived for class on a recent analog day to find typewriters at the desks, some with German and some with QWERTY keyboards.

“I was so confused. I had no idea what was happening. I’d seen typewriters in movies, but they don’t tell you how a typewriter works,” said Catherine Mong, 19, a freshman in Phelps’ Intro to German class. “I didn’t know there was a whole science to using a typewriter.”

Like a rotary phone, the manual typewriter appears simple but is not intuitive to the smartphone generation. Phelps demonstrated how to feed the paper manually, striking the keys with force but not so hard the letters would smudge. She explained that the dinging bell signifies the end of a line and the need to manually return the carriage to start the next line. (“Oh,” said one student, “that’s why it’s called ‘return.'”)

“Everything slows down. It’s like back in the old days when you really did one thing at a time. And there was joy in doing it,” said Phelps, who brings in her two children, aged 7 and 9, to serve as “tech support” and ensure no one has their phones out.

Students welcomed having fewer distractions

The assignment carries lessons beyond simply how to use a typewriter, which is the whole point.

“It dawned on me that the difference with typing on a typewriter is not just how you interact with the typewriter, but how you interact with the world around you,” said computer science major Ratchaphon Lertdamrongwong, a sophomore, whose class had to write a critique of a German movie they’d watched.

In the absence of screens, there are no notifications to distract you as you write. Without every answer readily available at his fingertips, he asked his classmates for help, which Phelps heartily encouraged.

“While writing the essay, I had to talk a lot more, socialize a lot more, which I guess was normal back then,” Lertdamrongwong said, referring to the typewriter era. “But it’s drastically different from how we interact within the classroom in modern times. People are always on a laptop, always on the phone.”

Without a delete key and the ability to correct every mistake, he paused to think more intentionally about his writing.

“This might sound bad, but I was forced to actually think about the problem on my own instead of delegating to AI or Google search,” he said.

Manual machines were a workout for pinky fingers

Most students found their pinkies weren’t strong enough to touch-type, so they typed more slowly, pecking at the keyboard with their index fingers.

Mong, the freshman, faced the added challenge of a recently broken wrist, requiring her to use just one hand. The self-described perfectionist was initially frustrated with how messy her page looked with odd spacing between certain letters and misspellings. (Phelps told students to backspace and type ‘X’s over errors.)

“This thing I handed in had pencil marks all over it and definitely did not look clean or finished. But it’s part of the process of learning that you’re going to make mistakes,” said Mong, who found the assignment of typing a poem “fun and challenging.”

She embraced the odd spacing and played with the visual boundaries of the page to indent and fragment lines in the style of poet E.E. Cummings. It took several sheets of paper and many mistakes, all of which Mong saved.

“I’m probably going to hang them on my wall,” Mong said. “I’m kind of fascinated by typewriters. I told all my friends, I did a German test on a typewriter!”

The Associated Press’ education coverage receives financial support from multiple private foundations. AP is solely responsible for all content. Find AP’s standards for working with philanthropies, a list of supporters and funded coverage areas at AP.org.

Sixteen bets made $100,000 accurately predicting the timing of the US airstrikes against Iran on 27 February. Later, a single user would make over $550,000 after betting that Ayatollah Ali Khamenei would topple, just moments before his assassination by Israeli forces. On 7 April, right before Donald Trump announced a temporary ceasefire with Iran, traders bet $950m that oil prices would come down. They did.

These bets and other well-timed wagers accurately predicted the precise timing of major developments in the US-Israel war with Iran, creating huge windfalls and raising concerns among lawmakers and experts over potential insider trading.

Betting – once largely siloed to sporting events – has now spread to include contracts on news events where insider information could give some traders an advantage.

The proliferation of online betting markets like Polymarket and Kalshi has allowed bets on virtually any news event. It's also easier than ever to buy commodity derivatives like oil futures, where traders gamble on what the price of oil will be in the future.

Leaders of some US federal agencies and some members of Congress said they want to crack down on suspicious trading taking place across different marketplaces, but it's unclear how much headway regulators will make.

"Is the problem that we don't have legislation or that we don't have enforcement capabilities?" said Joshua Mitts, a law professor at Columbia University. "To have a law that can't really be enforced effectively given the technological limitations, it's sort of putting the cart before the horse."

Perfect timing

On the night of 27 February, the day before the US and Israel would carry out strikes on Iran, an unusual influx of about 150 accounts on Polymarket placed bets that the US would strike Iran the next day. A New York Times analysis found the bets totaled $855,000, with 16 accounts pocketing more than $100,000 each.

Soon after, a single anonymous Polymarket user, under an account named "Magamyman", made over $553,000 after betting that Khamenei would be "removed" from power just moments before he was killed by an Israeli airstrike, according to a complaint filed to the Commodity Futures Trading Commission (CFTC), the federal agency that regulates futures markets, by Public Citizen, a consumer advocacy group. The complaint also cites a crypto-analytics firm that identified six "suspected insiders" who made a total of $1.2m on Polymarket after Khamenei was killed.

The well-timed surge of wagers were seen again on 7 April, when at least 50 Polymarket accounts placed bets that the US and Iran would reach a ceasefire hours before Trump would announce it on a Truth Social post. Earlier, the president had said "a whole civilization will die tonight" if Iran did not open the strait of Hormuz.

But traders weren't just active on Polymarket: there were similar surges of oil futures trading activity just hours before Trump announced updates to the conflict that would lower oil prices.

On 23 March, traders placed $580m in bets on the oil futures market just 15 minutes before Trump said on social media that the US was having "productive" talks with Iran, according to the Financial Times. The traders made a windfall after Trump's comments triggered a sell-off in the oil markets that made oil prices plummet.

The same thing happened again on 7 April, this time when traders spent $950m on oil futures, betting that the price of oil would fall just hours before the ceasefire with Iran was announced.

"We can't say from the outset whether any of these trades were illegal. Any one of them could be lucky, and any one of them could be based on lawful information," said Andrew Verstein, a law professor at the University of California at Los Angeles. "But many of them bear the hallmarks of suspicious trades that would naturally warrant investigation."

'A wild west'

For those who closely follow trading patterns, the rush of activity that happened before these events seem too big to simply be bets hedging on luck.

"Not only the timing, but the amount of these bets makes it look very likely that someone had insider knowledge … and placed very, very substantial bets on it," said Craig Holman, a government affairs lobbyist for Public Citizen who filed the group's complaint to the CFTC.

Holman said he is skeptical about how bold the CFTC will be in its investigations given its current structure under the Trump administration. The commission typically has five bipartisan members that are appointed by the president. Now the CFTC has one sole commissioner – Michael Selig, who Trump appointed at the end of 2025 and has positioned himself as friendly toward prediction markets.

Over the last few months, the CFTC has been roiled in fights with state legislatures who argue that regulation of these online betting marketplaces belong to the states.

Kalshi, Polymarket's competitor, was temporarily banned in Nevada after the state sued the company for offering contacts in the state without a gambling license. Arizona meanwhile filed criminal charges against the company for allowing people to place bets on elections. In both cases, Kalshi denied any wrongdoing and has argued that the CFTC has exclusive jurisdiction over online prediction markets.

"It's a wild west phase, when we're talking about the prediction market industry, and now it's spilled over into the stock market as well."

Anonymous sources told Reuters and Bloomberg that the CFTC launched an investigation into the oil futures trades that were placed on 27 March and 7 April, though the agency has not publicly announced it is conducting an investigation.

Speaking to Congress this week, Selig said that the agency is prepared to go after those who are suspected of insider trading, warning "we will find you and you will face the full force of the law", but said that the commission would not issue any new regulations until it has five seated commissioners.

Polymarket did not respond to request for comment. In a statement, White House spokesperson Davis Ingle said "federal employees are subject to government ethics guidelines that prohibit the use of nonpublic information for financial benefit".

"Any implication that administration officials are engaged in such activity without evidence is baseless and irresponsible reporting," Ingle said. "The CFTC will always uphold its duty to monitor fraud, manipulation and illicit activity daily."

Risky bets

Federal law prohibits government employees, including those working for Congress or the White House, from using non-public information for personal profit.

In late March, a bipartisan group of representatives introduced a bill that would ban members of Congress and senior staff within the federal government from participating in prediction market contracts related to political events or policy decisions.

But experts warn that insider trading law is complicated, and the new technology that makes it easier to place bets online leaves a complicated paper trail that can be hard to follow.

Historically, insider trading takes place when a person uses exclusive information about a company to buy or sell stocks right before information becomes public. These types of illegal trades are regulated by the Securities and Exchange Commission (SEC), which regulates the stock exchanges.

Insider futures trading could be seen as a subset of this typical insider trading, but the territory is new.

"The trick is that there are essentially no clean cases of people getting in trouble for commodity futures insider trading," Verstein said. "The law there is just not well developed."

In a paper published last month, Mitts, the Columbia law professor, and other researchers screened more than 200,000 "suspicious wallet-market pairs" between February 2024 and February 2026 and found that traders in this group achieved a nearly 70% win rate, making $143m in well-timed bets tied to everything from the capture of former Venezuelan leader Nicolás Maduro to Taylor Swift's engagement to Travis Kelce. The paper notes that informed traders face fewer legal constraints by trading on platforms like Polymarket or Kalshi because these markets still operate in a legal gray area.

"The challenge here is that this trading is occurring through the blockchain or other anonymized means, so it is going to be quite difficult for a regulator enforcement authority or prosecutor to determine the identity of the trader," Mitts said. "They would also have to prove the trader traded on the basis of information that had been wrongly misappropriated."

But the stakes are high. Insider trading involving classified military information can lead to distrust of both markets and governments.

"Unlike corporate insider trading, there's a lot of ways for the government to make itself be correct. You can just make the war that would occur, and that's concerning because then the real economy is being distorted," Verstein said. "Real decisions, including perhaps financial decisions, are being distorted by financial bets."

Initial Load

Cumulative

current slide ID:

Layers Loaded

By Daniel Wood, Connie Hanzhang Jin, Brent Jones and JeffBrady

---

Recently, the USDA updated its plant hardiness map for the first time in 11years.

If you’re a gardener — and everybody can be a gardener, even on a balcony or a stoop — this is a bigdeal!

The updated map opens up new possibilities for home gardeners, but there are limits. Let’s explore how the map has changed and what this means for yourgarden.

scroll

What does your hardiness zone tellyou?

Some people might think their hardiness zone tells them which plants they can grow. In reality, it’s a little morecomplicated.

Your zone measurement is an average of the coldest yearly temperature in your area over the past 30years.

This measurement, which predicts an area’s coldest temperatures, is only useful for plants that have to survive thewinter.

They’re called perennials: You plant them once and they come back after each winter if they’re given the right environment to survive. Think things like trees, shrubs and woodyplants.

Windmill palm

zones 7-11

Hydrangea

zones 3-9

Azalea

zones 6-9

Lavender

zones 5-9

Windmill palm

zones 7-11

Hydrangea

zones 3-9

Azalea

zones 6-9

Lavender

zones 5-9

Windmill palm

zones 7-11

Hydrangea

zones 3-9

Azalea

zones 6-9

Lavender

zones 5-9

And for predicting winter plant survival, knowing an area’s hardiness zone is a big help to gardeners, says Todd Rounsaville, a horticulturist with the USDA who was involved with creating the new map. He explains that the hardiness zone “is really one of the best predictors of winter survival and plant survival in general in thelandscape.”

He advises gardeners to use the map as one very important tool of many in their risk assessmenttoolbox.

“Because the USDA map has really become the industry standard for rating things, it’s pretty rare that you will not see a zone rating on a plant, either on the tag or on a website,” hesays.

Knowing what your average coldest temperature is helps rightsize your expectations about what might grow in your area. Live in Chicago’s Zone 6a? You can be assured that no citrus plants will survive your winter. Instead, try an apple tree. The apple tree is that kid you grew up with who wore shorts all winter. It needs the cold temperatures to setfruit.

Live in Miami’s Zone 11a? No apples for you. Instead, grow dragonfruit!

What does your hardiness zone not tell you?

On its own, your hardiness zone can’t tell you exactly what to grow in yourarea.

For example, parts of these three areas — Juneau, Alaska; Boston, Mass.; and Santa Fe, N.M. — are all in USDA’s Zone7a.

Juneau, Alaska

Boston, Mass.

Santa Fe, N.M.

Juneau, Alaska

Boston, Mass.

Santa Fe, N.M.

Juneau,

Alaska

Boston,

Mass.

Santa Fe,

N.M.

“We know intuitively that the same plants can’t grow in these places,” Rounsavillesays.

While Juneau may have relatively temperate winters, it also is extremely wet, averaging over 80 inches of snow a year. Santa Fe, on the other hand, is extremely dry, with much hotter summer temperatures than Juneau. Boston has both temperate winters and summers. It gets a good amount of rain but not nearly enough to sustain Juneau’s rainforest plants. It gets plenty of heat but is colder and wetter in the winter, making it inhospitable for desert dwellers, like cactuses and othersucculents.

But all three cities rarely get below zero degrees each winter, so they are classified as the samezone.

So when you hear that your zone has changed, here are some things to keep inmind:

1 The hardiness map says nothing about your extreme lowest temperature

Just because your average lowest winter temperature has changed, doesn’t mean the temperature will never dip below your hardinesszone.

scroll

2 The hardiness map says nothing about the frequency of extreme coldweather

Your poor plants have to stay outside all winter, so the duration and frequency of cold weather matters for plantsurvival.

“If you’re naked and you run through a freezer, it’s not going to kill you,” says Andrew Bunting, vice president of horticulture at the Pennsylvania Horticultural Society. “If you run into the freezer and have to stay there for an extended period of time, it’s probably going to killyou.”

If extreme, out-of-zone weather occurs during a quick cold snap, steps can be taken to protect your plants with temporary blankets or other shelters. Pots can be broughtinside.

But if the extreme lows persist, tender plants will struggle to survive. Your hardiness zone does not take any of this intoaccount.

3 The hardiness map can’t tell you if your plants will survive thesummer

Summer temperature extremes matter a great deal but are not reflected in the USDA hardinessmap.

Let’s look again at Juneau and Santa Fe, much of which are in Zone 7a. Juneau’s all-time high temperature was 90º F in 1975. Summer days in Santa Fe routinely reach the 90s. Some shade- and cool-weather-loving plants like ferns and hostas will thrive in Juneau but struggle mightily in a place like Santa Fe. Likewise, a cactus accustomed to high temperatures would struggle to thrive in the cooler summer temperatures of Juneau, to say nothing of the overwhelmingrainfall.

Because of this tricky problem, there have been attempts to create a corresponding map that helps gardeners know which plants might survive summer in theirarea.

In 1997, the American Horticultural Society released a heat zone map that measured the average number of times per year that the temperature of an area exceeds 86ºF.

American Horticultural Society

But this map didn’t become well known among gardeners. On a recent visit to a plant nursery outside Washington, D.C., nearly every plant tag had a USDA hardiness zone, but only one, out of the several dozen checked, had the AHS heat zonelisted.

Above 86º F, plants from cooler climates rapidly becomestressed.

Because of these complexities, more plant survival factors should be included in the 2023 map, says Tony Avent, who runs Juniper Level Botanic Garden and Plant Delights Nursery in Raleigh,N.C.

“If [these metrics] had been factored in, that would have given you a much more applicable map,” says Avent, who was a member of the committee that put together the 2012 version of themap.

“And that’s the part that’s a littledisappointing.”

But including more plant survival factors in the USDA hardiness map runs the risk of creating an overly complicated map and muddying its intended use, Rounsavillesays.

“In a perfect world, we could infinitely break down where plants will grow well, but that’s very hard to do and produce a map that is, you know, coherent but at a local resolution,” Rounsavillesays.

Since the USDA plant hardiness zone can’t tell you everything about how a plant will fare in your garden, it’s a good idea to turn to local plant experts for guidance. Local nurseries and botanical gardens can be great resources for in-depth knowledge of the area and recent warming or coolingtrends.

New plant varieties are constantly being bred with improvements such as increased hardiness, bloom count, bloom length or colorcombinations.

Some nursery owners like Avent enjoy experimenting with these plants. He and his team grow many varieties of plants — both typical and unconventional — to figure out which plants they can bring to market inRaleigh.

“We live to kill plants,” Avent says. He estimates that they’ve killed over 50,000 plant varieties in his career. Every one they kill, they record in adatabase.

If my zone changed, can I plant new thingsnow?

Maybe, and maybe you already did! It’s possible you or your neighbors may have already noticed some of these climatic changes and have been experimenting with plant varieties that were once unusual for yourarea.

Keep in mind that the new USDA map is backward looking; it represents changes that have already taken place over the past 30years.

In the 7a-7b Philadelphia suburbs, Bunting notes two perennials that he has noticed surviving Philadelphia winters in recentyears.

“It used to be [that] if you had a camellia, it was in a little courtyard with lots of protection, maybe even wrapped [in protective cloth] for the winter.” But now, “It’s perfectly hardy. Same with figs. People used to wrap figs. You don’t have to do thatanymore.”

Of course, your mileage may vary. As Bunting notes, where you plant a perennial in your yard — whether sheltered or in the open — matters. Some areas get southern exposure and lots of sun, others are behind a house, or under a tree. Every yard has many distinct microclimates, and learning how to harness these subtle differences in your yard can help you plant more ambitious varieties with moreconfidence.

“Gardeners know that if they’re near paved surfaces or brick and mortar structures, that there’s a lot of radiant heat that those absorb during the day,” Rounsaville says. “And they can really push hardiness zones through the winter to help with plantsurvival.”

With that, you have what you need to start a garden. Big orsmall.

Happy planting!

Explore the map

Start over with a new location

Before GPS, how did aircraft navigate? One important technique was celestial navigation: navigating from the positions of the stars, planets, or the sun. While celestial navigation is accurate, cannot be jammed, and doesn't require any broadcast infrastructure, it is a difficult and time-consuming process to perform manually. In the early 1960s, an automated system was developed for the B-52 bomber to automatically track stars and compute navigation information. Digital computers weren't suitable at the time, so the star tracking system performed trigonometric calculations with an electromechanical analog computer called the Angle Computer.1

The photo above shows the mechanism inside the Angle Computer.2 Although it may look like a gyroscope or IMU (Inertial Measurement Unit), it is completely different and nothing is spinning. The Angle Computer physically models the "celestial sphere", with a complicated mechanism inside that moves a pointer that represents the position of a star. The corresponding angles (the azimuth and altitude) are read out electrically through devices called synchros, providing information to the navigation system through bundles of wires. In this article, I'll give an overview of how celestial navigation works and explain how the Angle Computer performs its calculations.

The Astro Compass system

The Angle Computer is one piece of the Astro Compass, a system that locked onto a star and produced a highly accurate heading (i.e., compass direction), accurate to a tenth of a degree. While the heading is the main output from the Astro Compass, the navigator can also use it to determine position, using the "lines of position" technique described later.

The Astro Compass navigation system was built around the "Astro Tracker" (above), the optical system that tracks a star. The Astro Tracker was mounted on the aircraft with the 4-inch glass dome protruding from the top of the fuselage. This unit contains a tracking telescope, which used a photomultiplier tube to detect the light from a star. A gyroscope and a complicated system of motors provided a "stable platform", keeping the telescope precisely vertical even as the aircraft tilted and moved. A prism rotated and tilted to aim the telescope at a particular star.3

The Astro Compass system is bewilderingly complicated, consisting of 19 components (above) to support the Astro Tracker.4 On the right are the ten amplifier and computer components that controlled the system; the Angle Computer is in the lower right. On the left are the nine control and indicator panels that were used by the B-52's navigator. The photo below shows four of these panels in use in a B-52 in 1972.

Controlling the Astro Compass

The Astro Compass has an interesting user interface, letting you input one value at a time by rotating a knob. First, you use the Master Control Panel to select a data value such as the clock time, SHA (Sidereal Hour Angle) for star #1, or Declination for star #3. Then you turn the "Set Control" knob clockwise or counterclockwise to scroll through the data values until the proper value is reached. Each knob on the Master Control Panel has a different geometrical shape, allowing the user to distinguish the knobs by feel. The Master Control Panel is visible in the lower left corner of the photo above, within easy reach of the navigator.

Each data value has a separate electromechanical display. The photo below shows a Star Data display, indicating the sidereal hour angle and the declination for a star. I removed the cover so you can see how the digital display actually consists of analog dials rotated by motors under synchro control. The system has three Star Data displays, so it can hold the positions of three stars at a time. Getting fixes from three different stars is useful when computing lines of position. The system uses one star at a time, but you can quickly change stars by flipping the Star switch on the Master Control Panel.

But how did the navigator obtain the information to put into the Astro Compass, since the sun, moon, stars, and planets are in constant motion?5 The necessary celestial information is published in a book called the Air Almanac. The US Government started publishing the Air Almanac in 1941, issuing a new volume every four months. The Almanac had a sheet for each day, providing celestial data on 10-minute intervals. The first column has the time (GMT, Greenwich Mean Time)6 while the other columns give the position of the sun, an important value called the First Point of Aries (symbol ♈︎), the positions of the visible planets, and the position of the moon. A separate table and chart provided the locations of stars; the stars don't have daily updates since they are almost stationary.7 (The Air Almanac is now online; you can download the 2026 Air Almanac here.)

The navigational triangle: Computing a star's position

The Air Almanac provides star coordinates in a global coordinate system, but the Astro Compass needed to know star coordinates in the aircraft's local coordinate system. Determining the star's position requires changing the coordinate system by using spherical trigonometry and something called the navigational triangle. There's a fair bit of terminology involved, which I'll explain in this section.

The Astro Tracker, like many telescopes, is aimed by using azimuth and altitude. Suppose you go into your yard, point at the horizon, and turn 360° in a circle; the direction you're pointing is called the azimuth. The point directly overhead is called the zenith. Now swing your arm upwards 90° from the horizon to the zenith. That angle is called the altitude. (Confusingly, the term "altitude" is used both for the angle of a star and the height of an aircraft.) Thus, if you point at a particular star, you can describe its position with two angles: your horizontal rotation from north gives the azimuth, and the angle up from the horizon gives the altitude.8 This system is called the horizontal coordinate system, as it is based on the horizon. (The word "horizontal" comes from "horizon", by the way.) This is a local coordinate system since other locations will have a different azimuth and altitude for the star. The azimuth and altitude constantly vary with time because the Earth's rotation makes the star appear to move.

The equations for the altitude and azimuth are complicated, with sines, cosines, arcsine, and arctangent. To see why the equations are complicated, consider a time-exposure photo of star trails. As the Earth rotates, each star forms a circle around Polaris, the North Star. To trace out this circular path, the altitude and azimuth vary in a trigonometric way. This computation is performed electromechanically by the Angle Computer, as will be explained later.

Now let's switch to how the position of a star is defined in the Air Almanac (for example), independently of your local position. Pretend that the stars are on the surface of a large sphere that surrounds the Earth, called the celestial sphere. The stars are stationary on the surface of the celestial sphere, while the Earth rotates once a (sidereal)9 day in the middle. Thus, as you look up at the celestial sphere, you see the stars moving. You can extend the Earth's equator out to the celestial sphere, defining the celestial equator. Likewise, the celestial sphere has celestial poles, matching the Earth's poles. On the Earth, you specify a location (such as the airplane's location) with latitude and longitude (red). Latitude is measured from the equator, and longitude is measured from a fixed meridian (orange). The 0° meridian is arbitrarily defined to pass through Greenwich (England, not Connecticut). Similarly, the position of a star is specified by the angle from the celestial equator (called declination instead of latitude) and the angle from the meridian (called the sidereal hour angle or SHA instead of longitude).10

But what meridian is the starting point—0°—when measuring a star's Sidereal Hour Angle? The celestial equator matches the Earth's equator, but this won't work for the Greenwich meridian because it is constantly in motion. Instead, the 0° celestial meridian is arbitrarily defined as the position where the sun crosses the equator at the vernal equinox (the start of spring). If you consider the position of the sun on the celestial sphere, the sun will travel around the sphere once a year. Because the Earth's axis is tilted, the sun will be above the equator half the year and below the equator half the year, crossing the equator at the vernal equinox (March) and the autumnal equinox (September).

This reference point on the celestial sphere is called the First Point of Aries, represented by the symbol ♈︎ (horns of a ram); you might remember this symbol from the Air Almanac. At this point, the sun is in the constellation Pisces. So why is this point called the First Point of Aries and not Pisces? Back in 130 BCE, the ancient Greek astronomer Hipparchus defined the First Point of Aries as the starting point for the sun's motion. In that distant era, the sun was in the constellation Aries at the equinox, not in Pisces as it is today. It turns out that the direction of the Earth's axis isn't fixed, but moves in a 26,000-year cycle called the precession of the equinoxes.11 A 26,000-year cycle may seem irrelevant, but it's fast enough that the sun has moved from Aries to Pisces since Hipparchus's time. (And the equinox has moved 1° more since the B-52 was first produced!)

(All this talk of Aries and Pisces may sound like astrology, and, yes, there is a direct connection. Aries is the first zodiac sign, starting at the vernal equinox, typically March 21. The equinox's precession is "backwards", so the equinox has moved to Pisces, the last zodiac sign. Astronomically, the equinox will move into the constellation Aquarius around 2600 CE, but astrologers disagree on whether the Age of Aquarius has started; perhaps the 1960s was the dawning of the Age of Aquarius.)

How do you convert the star's fixed coordinate to the Earth's rotating coordinate? First, you look up the angle between the Greenwich meridian and the celestial meridian of Aries at a particular time. This angle (purple) is called the Greenwich Hour Angle of Aries (GHA ♈︎). Next, you look up the star's Sidereal Hour Angle (SHA). Adding them gives you the star's Greenwich Hour Angle (red), the angle between the Greenwich meridian and the star. Subtracting the aircraft's longitude gives you the Local Hour Angle (LHA, not shown), the angle between the aircraft's meridian and the star. (Note that these steps are simply addition and subtraction, so a mechanical system can easily do them with differential gears.)

The final step, obtaining the azimuth and altitude, requires tricky spherical trigonometry. The yellow triangle is the navigational triangle, a spherical triangle on the surface of the celestial sphere. The upper vertex is the North Pole, the red vertex is the airplane's zenith (i.e., directly above the airplane), and the final vertex is the star. Two sides of the triangle and an angle (purple) are known, so the remaining angles and sides can be solved with spherical trigonometry. Specifically, the first side (purple) is 90°-declination, the second side is 90°-latitude,12 and the angle between is the LHA (Local Hour Angle). Solving for the angle at the zenith gives the azimuth (blue), while solving for the third side gives 90°-altitude (green, the angle down from the zenith to the star).

Thus, the key problem is solving the navigational triangle. Navigators could solve the navigational triangle by looking up angles in a thick book of "sight reduction" tables and performing some math. But how could the process be automated? That was the purpose of the Angle Computer.

The Angle Computer

The job of the Angle Computer was to solve the navigational triangle mechanically. Its inputs were the star's declination, altitude, and local hour angle. From these, it computed the star's altitude and azimuth at the aircraft's current position.13

The concept behind the Angle Computer is that it physically modeled the celestial sphere with a half-sphere, 2 5/8" in radius. A star pointer was mechanically positioned on the surface of this sphere, using the star's declination and local hour angle, adjusted by the latitude of the viewer. The star pointer moved a readout mechanism that translated the star's position into the azimuth and altitude at the specified location. Thus, the Angle Computer mechanically converted between the coordinate systems by using a physical representation, solving the navigational triangle.

The diagram below shows how the star pointer is positioned on the two-dimensional surface of the sphere, using a complicated mechanism inside the sphere. The U-shaped declination arm swings up and down, corresponding to the star's declination (angle above the celestial equator). Meanwhile, the declination arm constantly rotates around the polar axis, as specified by the LHA (Local Hour Angle). In one (sidereal) day, the mechanism will make a full cycle, corresponding to the Earth's spin. Finally, the latitude arm moves the mechanism up or down, corresponding to the viewer's latitude. On the right, three gears provide the inputs for latitude, LHA, and declination.

A separate mechanism provides the altitude and azimuth outputs, driven by the star pointer. The key is the semicircular azimuth arc, which represents the arc from the viewer's horizon to the zenith, oriented to a particular azimuth. The star pointer is attached to the azimuth arc through a slider, so as the star pointer moves, it moves the slider along the azimuth arc and also rotates the azimuth arc. Specifically, the azimuth arc represents the line from the horizon to the zenith at a particular azimuth. The position of the slider on the azimuth arc corresponds to the altitude, from 0° at the horizon to 90° at the zenith.14. The azimuth arc rotates around the zenith point, which is at the back of the azimuth arc; this rotation indicates the azimuth value. As the azimuth arc rotates, it turns a gear at the zenith, providing the azimuth output. The slider arc has teeth on it; as the slider moves, these teeth rotate a second gear, providing the altitude output.

From the back, the numerous synchro transmitters, synchro control transformers, and motors are visible. Even though the computation itself is mechanical, the Angle Computer has numerous electrical components. In the top half, the synchro transmitters provide electrical outputs of the azimuth and altitude. (A synchro transmitter uses fixed and moving coils to convert a shaft rotation angle into a three-wire electrical signal.) The large gear provides the altitude output. In the lower half, the longer cylinders are motors that move the Angle Computer's mechanisms. The motors are directed to rotate to a particular position through a feedback loop: synchro control transformers provide feedback to the external servo amplifiers that power the motors.

Partially disassembling the Angle Computer shows the complex gear trains inside, linking the synchros, motors, and the physical mechanism. The squat brass-colored units in the lower center are differential assemblies to add or subtract signals.15 One of the drive motors, a long cylinder, is visible in the lower right.

The Line of Position

Although the heading was the primary output from the Astro Compass, the Astro Compass could also help determine the location of the aircraft, using a technique called the celestial line of position. This technique was discovered in 1837 and became heavily used for navigating ships with a sextant. It could also be used onboard an aircraft.

To understand the line of position, suppose you go outside and find a star directly overhead. If you measure the altitude—the angle from the horizon to the star—with a sextant, the angle will be 90°, since it is overhead. Now, suppose you teleport 60 nautical miles away in any direction. The sextant will now show an altitude of 89° to the star, since a nautical mile is conveniently defined to match one minute of angle (one-sixtieth of a degree). Alternatively, if you measure an altitude of 89° to the star, you know you are 60 miles away from the original point under the star (called the sub-stellar point). Likewise, if you measure 88° to the star, you're on a circle with radius of 120 nautical miles around the sub-stellar point. If you measure, say, an altitude of 40°, you know you're on a very large circle with radius of 3000 miles around the sub-stellar point. So how does this help with navigation?

Suppose you're on a boat in the middle of the Pacific and you have a rough idea of where you are, say within 100 miles, but you want to find your exact position. Put a dot on the map where you think you are. Next, pick a star and work out what the angle to the star should be from your position. Measure the altitude with your sextant. Suppose you expected 50° but measured 51°. You now know that you're somewhere on a circle with radius of 2940 miles around the distant sub-stellar point. This doesn't seem very useful. However, since the angle was 1° more than expected, you know that the circle is 60 miles closer to that distant point than your estimated position. Moreover, since you have some idea of where you are, you know that you're on the part of this circle near your estimated location. And since you're looking at a small part of a big circle, you can approximate it by a line. So you can go back to your map, move 60 miles closer to the star from your estimated point, and draw a perpendicular line. This is your line of position, and you know that you're on this line (more or less).

Knowing that you're on a line isn't too useful, but you can repeat the process with a star in a different part of the sky. Maybe this time the angle is 2° smaller than expected, so you can draw a line of position 120 miles further away from your estimated position, in a different direction. The two lines cross, indicating a position where you (probably) are.16 Normally, you repeat the process with a third star, giving you three lines of position, providing a position and an idea of its accuracy.

The Astro Compass used the display above to show the star's azimuth and the distance in miles from the assumed location to the line of position, called the Altitude Intercept. With this information, the navigator could draw a line of position on the map. The navigator repeated the process with two more stars to get a location fix.17

Conclusion

The Angle Computer is a relic from a time when a mechanical analog computer was the best way to solve a problem, but the computer was also electrical. Although a mechanical apparatus solved the navigational triangle, it was moved into position by motors, and the output was transmitted electrically through wires. Moreover, the Angle Computer was driven by electronic amplifiers and feedback circuits that used both vacuum tubes and transistors.

The designers of the Astro Compass considered multiple approaches to computing the navigational triangle (details). The first was to use small electromechanical devices called resolvers that convert a physical rotation into sine and cosine values. By combining six resolvers with amplifiers, the altitude and azimuth could be obtained. The resolver solution was rejected as being too large and requiring a precision power supply. The second approach was to use a digital computer to determine the solution. This solution was rejected because in 1963, a digital computer was expensive, slow, and less reliable. The final approach, which was adopted, was to build a mechanical, physical model of the celestial sphere. Thus, the Angle Computer resided at the uneasy intersection of physical mechanisms, electrical circuits, vacuum tubes, and solid-state electronics, soon to be obsoleted by digital computers.

I plan to write more about the Astro Compass system. For updates, follow me on Bluesky (@righto.com), Mastodon (@[email protected]), or RSS. Thanks to Richard for supplying the Astro Compass hardware.

AI statement: I didn't use AI to write this article (details).

Notes and references

Fuzix OS

For source code and licenses please visit the Github repository. The matching tag is '0.4'.

Install Images

Install images for 0.4 targets. See the git source README files in Kernel/platform-* for install information.

Fuzix 0.4 Release Notes

Overview Of Changes

The core of the Fuzix kernel remains much the same for this release. A number of bugs have been fixed and some interfaces improved. The networking layer has been completely reworked to be more modular so that in future it can run in a different address space to the kenel on 8bit machines.

Executable formats have changed. The 8080, 8085 and Z80 binary formats are now properly unified so that 8085 and Z80 can run 8080 binaries directly. The 68HC11 and 6803 formats are somewhat different but the syscall ABI is arranged so that the 68HC11 can run 6803 binaries.

The 32bit binaries that were using a bodged Linux binflt format are now using a.out with some small extensions to handle the relocation maps. This should hopefully now become a stable executable format for the future.

Building has, where possible, been made easier. The tool chains remain a bit of pain because of the fact many are somewhat obscure, and those that are not tend to get broken on a regular basis forcing specific releases to be used. The actual system build however now has a "make diskimage" target that puts together all the pieces for a bootable system in one go rather than requiring the builder understands the finer details of the system in question and how to merge all the pieces together.

The make environment is much better than it was. It is still terrible however so a "make clean" is needed when switching processors and a "make kclean" is strongly recommended when tweaking any kernel config options or switching target. A lot of the make rules have been merged which should make the problem of sorting out all the rules and dependencies more tractable for 0.5.

Naming Changes

The N8VEM project rebranded as 'Retrobrew' at the request of its founder who had ceased to be so involved. The 0.3 release fixed most N8VEM naming, 0.4 completes this.

There is now a distinction between RC2014 (the product line) and RCbus (the bus standard). In particular the bus has been extended beyond the original concept and now has its own standard document and keepers. Fuzix should now only use the 'RC2014' nomenclature for official RC2014 products but there may be a few that have been missed. Systems that were previously rc2014-xyz are now rcbus-xyz.

Dropped Systems For Now

• Pentagon
• Pentagon 1024
• Scorpion

None of the testers for this work are currently available.

Not Tested (even on emulator)

• P112 (no working emulator, no access to machine)
• SocZ80 (no emulator, need to fix system)

Supported Processors

6303 / 6803

Hitachi 6303 and Motorola 6803 processors. These are supported by the CC68 compiler chain and tools derived from cc65 and specifically designed for Fuzxi. Currently the only target board is the rcbus 6803/6303 processor card. Floating point is not supported, but adding it would only require someone writes the basic underlying soft fp routines (add, subtract etc).

Other targets should not be hard to add.

6502 / 65C02 / 65C816

These are supported by cc65 (v2.18 or later). Due to compiler limits floating point is not supported. 65C816 is treated as a 65C02 with extras because of the lack of an open 65C816 compiler.

Currently this port targets the RCbus 65C02/65C816 cards and the PZ1. Most of the "classic" 6502 systems have neither the memory or the I/O, and in most cases even when they are upgraded with some of the late era processor upgrades still lack decent I/O.

6809

This port uses gcc 6809 and lwtools. It supports a range of classic and modern systems including Dragon, Tandy COCO, Thomson and RCBus machines. You may need to use an old lwtools (eg 4.13) if using a modern lwtoools and it reports a segmentation fault or similar from lwasm.

68HC11

The final generation of the 6800 processor line this port uses gcc and has a very different ABI to the 6800/6803. It can run 6800 and 6803 binaries however, but not 6303. At the moment this port targets the Mini11 SBC and the RCbus 68HC11 card.

68000

Motorola 68000 series processors with flat memory space. The port is now a lot more stable and has a sensible binary format. Processors up to 68EC020 are catered for. An additional memory model has been added for systems with low memory and it is now just about possible to run Fuzix on a 128K system without relying on fast disks. GCC and binutils need to be built in a particular way for pre 68020 processors otherwise they will insert unsupported instructions. See the README for 68000.

8080

The 8080 is supported using the Fuziz C compiler. This is a new compiler so there may be a few bugs left to shake out.

8085

This port now uses the full 8085 instruction set (including the stuff Intel decided to not to document). It uses a new compiler built specifically for Fuzix. This port is thus currently a little bit flaky and there are tool and system bugs to nail down. Performance is several times faster than the pure 8080 build as the extra instructions make a huge difference when executing high level languages. Floating point is not yet supported as it needs the base low level FP routines for 8085 writing

The supported target is the rcbus 8085 card with onboard bank MMU.

ARM

ARM M0 is supported using gcc and targetting the Raspberry Pi Pico. ARM M4 targets for the DK-TM4C129X and EK-TM4C129X.

ESP8266

This target is specific to the ESP8266 variant of the Tensilica L106.

NS32K

A complete port for MMUless NS32K processors. This port is new as of 0.4. It targets the in-progress RCbus NS32FX16 processor card design.

Z80 / Z180 / 64180 / Z84C1X

These processors are all supported by the main tree using a fork of SDCC 3.8. The newer SDCC changes a lot of calling conventions so no move to it has been made at this point. That may change in the future.

Supporting code libraries make most flat Z180 systems trivial and provide all the mapping and peripheral support.

Other Processors

These are processors which are in the tree but not yet fully functional. In some cases this is merely used to help catch portability problems in the libraries.

6800

Work in progress to extend CC68 and Fuzix to the original 6800/6808 processor line. Not yet usable

8086/8088/80C188/80C186

This is currently just the base sketches for a future PC and Rcbus-80C188 port.

ESP32

An initial experimental port only

EZ80

Initial support for booting on an ez80 based platform. In this case the Jee Retro development platform. This should form a good basis for enabling any other ez80 platform, but none have been enabled yet.

MSP430X

Retired

PDP11

Tracking the state of the GCC PDP11 compiler and binutils. Not yet at a point the toolchain works well enough.

Rabbit 2000/3000

At this point used as a build check on the user space. Hopefully some Rabbit board enabling will happen during the 0.5 work.

RiscV 32

Tool chain testing and portability work. At the moment this is another compiler chain that we break. In theory Fuzix should be able to run on some of the upcoming and recent RiscV micro-controllers with 128K or so of SRAM.

TMS9995

This target is being used to slowly debug the C compiler. Not a useable port as the compiler is still a fair way from working correctly.

WRX6

Warrex CPU6. Being used for compiler debug but also on hold until the documentation of the system is in a better state to make progress.

Z280

Work in progress. The Z280 is very Z80 like but the different privilege structure and interrupt behaviour meand that for the Fuzix at least this will need its own variant of the low level core code.

Supported Systems

For more details on each system consult the relevant README.md in that Kernel/platform-xxxx directory. There are othee platforms but if not listed here they are likely works in progress or special cases.

2063

John Winan's 2063-Retro system as featured in his "John's Basement" youtube series. A fairly classic Z80 Retro system with somewhat slow bitbang SD card interface.

68K-nano

The 68K nano design from Matt Sarnoff. A minimalist 68000 system with 16bit IDE interface. This makes a very nice and easy to build little Fuzix box.

Ampro Littleboard

The Ampro Littleboard was a classic Z80 CP/M board designed to be the same size as a floppy disk. This port requires the Plus version of the board with SCSI controller.

Amstrad NC100

An pre-laptop portable word processing machine with PCMCIA slot.

Amstrad NC200

The sequel to the NC100 with a floppy drive and much nicer display.

CPM22

A port that uses a customisation block plus the CP/M 2.2 BIOS. Intended to make it practical to port Fuzix to systems like S.100 machines where the BIOS is basically the architecture and each machine tends to be a bit unique. If you need to port Fuzix to a platform and do not want to deal with the Fuzix and C side of things this may be a good basis. At the moment it is very basic, but easily extensible to cover other gaps in the CP/M BIOS.

COCO2

Support for the Tandy COCO2 with 64K RAM and CF or SD card adapter. The kernel is partly flashed onto a cartridge bank to make it all fit nicely. Should also work with a Dragon 64.

COCO3

The Tandy COCO3

Cromemco

Z80 based system with 8" floppy disk interface. The hard disk is not currently supported due to lack of documentation/example code.

DK/EK-TM4C129X

An ARM development board.

Dragon 32

There are two ports specifically aimed at the Dragon machines. Each supports one of Tormod Volden's SD and memory extensions - the NX and the later more featured MOOH. This port should, with the right build options, also work on COCO machines with the same interface.

Dyno

A fairly generic flat memory Z180 retro system.

Easy-Z80

Sergey Kiselev's EasyZ80. Very similar to the RCbus and RC2014 systems with 512/512K RAM but integrated onto a single board with battery back up and also IM2 interrupt mode support.

ESP8266

Fuzix for the ExpressIf microcontroller. The onboard wireless is not supported because a) it's not documented and b) we stole all the memory it uses and repurposed it. It does however support using a WizNet 5500 for internet connectivity.

Genie EG64/3

Support for the Video Genie (aka Dick Smith System 80, PMC-80 etc) with the Genie EG64/3 CP/M adapter or the equivalent TRS80 Lubomir soft banker.

KC87

Robotron KC87, KC85/1 and Z9001. Not the KC85/4. East German systems produced by Robotron-Meßelektronik.

MB020

A simple flat memory 68020 board by Bill Shen

Micro80

A retro Z8C415 based machine using the single chip integrated Z80 and I/O.

Microbee

The classic Australian system yet almost unknown elsewhere. There are a long series of these machines with growing power. At least 128K and some kind of decent disk interface is required. Colour is recommended as the Fuzix kernel does not try and deal with video update flicker on the older video.

Mini11

Low chip count 68HC11 SBC with 512K of RAM driven directly off a 68HC11, and the upper address pins driven by the 68HC11 GPIO lines.

MSX

The are two MSX ports. The MSX1 port is a cartridge based port that can run on any 64K mmeory machine with Sunrise style IDE. There is no support for things like the MegaRAM at this point.

The MSX2 port requires V9938 or higher video, MSX2 style memory and at this point a MegaFlashROM with SD card. There is no support for MSX1 systems with the orignal video and MSX2 memory banking although that would be good to add.

MTX

Memotech MTX512 with banked memory and suitable disk adapter. An obscure but rather beautiful British machine.

Multicomp09

FPGA based retro system.

N8

The N8 is a fusion of Z180 retrocomputer and sort of not quite MSXish things.

Nascom

Nascom II or III with CP/M PAL, multiple memory banks and CF adapter on the Z80 PIO plus an RTC.

P112

DX Designs P112. An early 'retro' system using a Zilog ESCC.

PCW8256/512/9256/9512/10

The classic Amstrad wordprocessor / CP/M machine. Still somewhat of a work in progress. Note that the PCW16 is a completely different architecture and not supported.

Pentagon

Easten block ZX Spectrum clone. This port is targetted at systems with 256/512K of RAM. For the 128K machines just use the 128K Spectrum targets

Pentagon 1024

The 1MB Pentagon with additional mapping capabilities

Pico68K

A breadboard 68000 system using an ACIA and VIA plus bitbang SD card.

RBC Mark 4

The Retrobrew (formerly N8VEM) Mark 4. A Z180 based design for the ECB bus. Note that 0.4 now requires RomWBW firmware.

RBC Mini M68K

John Coffman's mini 68K system.

RC2014

There are two Fuzix ports aimed at "official" RC2014 targets.

The RC2014 port requires a 512K RAM/ROM card and supports the bigger RC2014 configurations along with a lot of other compatible RCBus hardware.

The RC2014-tiny port is a "because we can" port that puts the core of the Fuzix kernel in the pageable ROM and pages it in and out in order to run Fuzix on the 64K + pageable ROM setup. This works but isn't recommended. Note that you can (and should!) jumper the ROM card for a 28C256 not a 27C256 part if you are doing this, that way you can quickly erase it and update it.

RCBus (Z80 and compatible)

The following RCbus compatible systems have their own ports referenced elsewhere in this document.

• 2063 (with adapter)
• Easy Z80
• RIZ180
• SC108
• SC111
• SC720
• Simple80
• T68KRC
• ZRC

The SBC64/MBC64/ZRCC have their own rcbus-sbc64 target. In general the other Z80 boards are compatible and operate with the RC2014 tree. This knows how to handle Z180 processor cards with the banked memory card, and various variant systems and I/O options.

The rcbus-z180 target handles the many near identical flat Z180 designs that use the RCBus with a 512/512K flat 20bit memory space as well as the modification sometimes used to allow for 1MB RAM.

RCBus (Other)

There are specific ports for RCbus systems with the following processors: 6303 (and 6803), 6502, 6809, 8085, 68008, NS32K. The card requirements vary by port. Please consult the port documentation for detail.

Rhyophyre

A Z180 SBC with uPD7220 GDC video. The video is not supported in Fuzix at this time.

RIZ180

A minimal system Z180 design by Bill Shen

Raspberry Pi Pico

Support for the Pi Pico embedded ARM board

Sam Coupe

A "super ZX Spectrum" machine that proved to be too little, too late to survive the shift away from 8bit home computers.

SBC 2G

A banked memory design that draws heavily on Grant Searle's machine.

SBC v2

Retrobrew SBC v2. Z80 base ECB bus card with simple banked memory. This target is documented heavily and is designed to be a reference for anyone trying to understand how to port Fuzix.

SC108

Small Computer Central design with CPU, RAM and OM on one card. Also supports the SC114.

SC111

Small Computer Central Z180 design. This tree has been kept apart from the unification of various related Z180 RCbus systems because it also supports the ability to boot from SCM firmware. With RomWBW firmware the SC111 can run either.

SC720

Small Computer Central Z80 SBC. Has a different more limited memory mapping model to the standard rcbus systems.

Scorpion

Another Eastern block ZX Spectrum clone that differs somewhat from the Pentagon designs.

Searle

Fuzix for a slightly modified version of the Grant Searle classic design. The modifications consist of a single wire and resistor mod to use the full 128K of RAM and a tweak to allow an external timer to provide a timer tick.

Sinclair ZX Spectrum

There are multiple ports for the variants of this system. For the clones and not quite compatible systems please see

• Pentagon
• Pentagon 1024
• Scorpion
• TC2068

The following configurations have ports

128K or later with DivIDE/DivMMC

This port uses the DivIDE/DivMMC memory in low space in order to run Fuzix on the Spectrum systems that lack the ability to map RAM low directly. This port also knows about the ZX-Uno extensions and will use them.

128K or later with SpectraNet and a disk controller

Similar to the DivIDE/DivMMC this port uses the SpectraNet memory in the same way, and needs a disk controller such as a ZX-MMC to go with it.

+2A or +3 with ZX-MMC or similar disk controller

Using the additional memory features on the +2A and +3 machines in order to do rather better memory banking.

Simple80

Bill Shen's glueless Z80 minimal design. Requires either a rev1 board or a small board mod to correct the memory banking error in the original.

SmallZ80

TG Consulting Z80 system

SocZ80

High speed FPGA based Z80 (T80) platform. Runs at 128MHz and supports 8MB of RAM.

TC2068

Timex reworking of the Sinclair ZX Spectrum. Incompatible, buggy and a bit of a flop. It does however have support for RAM/ROM cartridges and has a nice 512 pixel wide video mode.

Tiny68K / T68KRC

A 68000 based system with 16MB of RAM and a CF interface. The T68KRC has less RAM but an RCbus connector. Both are supported.

TO8

Thomson TO8. An initial port to the TO8 platform.

Tom's SBC

Z80 retro computer design with banked EPROM and 64K (or 128K with mod) RAM. Fuzix can run from banked EPROM for a ROM based system or from RAM with the 128K modifier.

TRS80

Tandy model 1 and model 3 systems (and clones) with an Alpha Supermem or Selector are supported along with the Model 4. The 128K model 4 is supported without a memory extender although the Dave Huffman style mod is supported. The XLR8R is not directly supported at this time.

VZ200

VTech Laser 200 with the SDDrive adapter.

YAZ180

Yet another Z180 system.

Z1013

An Easten block design and probably candidate for 'worst home computer keyboard ever shipped'. Fuzix supports a suitable configuration which in practice probably means the modern re-creation.

Z50Bus

For all practical purposes the Z50Bus systems except the Linc80 are software equivalent to the RCbus ones and use the same kernel. The Linc80 has its own port but that requires building a DIY memory expansion.

Z80 MBC-2

A small Z80 based system with an Arduino I/O subsystem.

Z80 Membership Card

Sunrise EV stackable retrocomputer. Fuzix requires the CPU/RAM/SD card combination.

Z80Pack

Z80 emulated CP/M platform. Useful for debugging and development purposes.

Z80Retro

Peter Wilson's Z80 Retro design

Zeta V2

Sergey's floppy disks sized Z80 system with banked memory, PPIDE and floppy controller.

ZRC

Bill Shen's ZRC. A minimal Z80 system using a CPLD. The machine has 2MB of DRAM. Fuzix has no idea at this point how to use it all because it's not clear what you need 2MB for on a small Z80 system with no graphics.

Mastodon

Bedraggled from a walk in the rain, Sherry Turkle shows up begging for a latte. She's left her wallet in her hotel room. She's exhausted, she says, and could do with a coffee. "You can see it's not my most perky morning. But I'm really thrilled to be meeting with you."

These aren't just pleasantries – Turkle has a serious point to make. As professor of the social studies of science and technology at MIT and the founder and current director of the MIT Initiative on Technology and Self, she has spent over three decades studying the way people interact with machines, and is growing increasingly worried about the amount of human interaction people are happy to delegate to robots or carry out over phones and computers. As a human, within seconds of meeting her in person, I can interpret the complexities of her mood – the tired part, and the happy to be here part. "This is a complex dance that we know how to do to each other," she says. A dance she fears is being forgotten.

Turkle wasn't always this interested in technology. Born in Brooklyn in 1948, she studied in Paris before returning to do her PhD in sociology and psychology at Harvard. By 1978 she had just written her first book, on French psychoanalysis, when MIT hired her to study the sociology of sciences of the mind. "I began to hear students talking about their minds as machines, based on the early personal computers they had." They'd use phrases like "debugging" or "don't talk to me until I clear my buffer". "I'd never heard any of this stuff before."

So Turkle began to study the way that artificial intelligence was taking hold in everyday life, at a time when these interactions with machines were pretty raw. She "literally was at the right place at the right time."

The place being MIT, home to some of the pioneers of artificial intelligence and social robotics, and the birthplace of perhaps the most sophisticated, and endearing, social robots. Turkle tested these anthropomorphic robots on children, "computer virgins". In one study she observed how children would bond with the robots, which were programmed to respond with human-like emotions, in a way they wouldn't with other toys. "This becomes a tremendously significant relationship for the child," she says, "and then it will get broken or disappoint, and the child will go ballistic. My research group went berserk at how much damage we felt we'd done."

Turkle was "smitten with the subject and stayed with it for 30 years". In the early days she was labelled as a "cyber diva". "People thought I was very pro-computer. I was on the cover of Wired magazine." Then things began to change. In the early 80s,"we met this technology and became smitten like young lovers," she says, but today our attachment is unhealthy. In her latest book, Alone Together: Why We Expect More from Technology and Less from Each Other, Turkle says we have reached a point she calls the "robotic moment" – where we delegate important human relationships, in particular interactions at "the most vulnerable moments in life" – childhood and old age – to robots. "We are so worried about Asperger's, so worried about the way we communicate with faces. To me, as somebody who likes technology, this is just playing with fire."

Turkle frequently takes calls from journalists seeking comments on the latest story about robots in nursing homes, teacherbot programmes or nannybots to look after children. She sees married couples who prefer to have their fights online. "My studies of funerals are hilarious," she says. "Everybody's texting. When I ask them about it, they say, 'Yeah, I do it during the boring bits.' So that's the question: what's does it mean as a society that we are there for the boring bits?"

She is particularly concerned about the effect on children. "I am a single mum. I raised my daughter, and she was very listened to." Today our phones are always on, and always on us. Parents are too busy texting to watch their kids, she cautions. There's been a spike in playground accidents. "These kids are extremely lonely. We are giving everybody the impression that we aren't really there for them. It's toxic." This is what she means by "alone together" – that our ability to be in the world is compromised by "all that other stuff" we want to do with technology.

For many these are inconvenient truths, and lately Turkle has come to be seen as a naysayer, even a technophobe. She is no longer the cover girl for Wired. "This time they didn't even review my book." In fact, the initial reviews of Alone Together, Turkle says, can be summarised as "everybody likes Facebook, can't she just get with the programme?" This, she adds, is unfair to the 15 years of research behind it. "I mean, give me the credit. I didn't do a think piece. I was reporting. People tell me they wish [iPhone companion] Siri were their best friend. I was stunned. You can't make this stuff up."

Turkle is optimistic that people will begin to want to reclaim their privacy, to turn back to their relationships with real people. Yet she concedes that the lure of technology is such that it's a tough challenge. "Online you become the self you want to be." But the downside? We lose the "raw, human part" of being with each other. She points to our early morning meeting, for example. She's tired, and we could have done the interview over Skype. "Online I am perfect," she says. "But what's the worst that can happen here? You write a story that says, 'Bedraggled from her walk in the rain, she shows up begging for a latte? So what? You pretty much see me as I am. And I'm willing to say that's a good thing."

A technical postmortem on how one person with zero coding experience built a self-evolving AI swarm across 11 platforms using 900 accounts, 56 GitHub Actions workflows, and $0 in compute costs.

A real-world production migration from DigitalOcean to Hetzner dedicated, handling 248 GB of MySQL data across 30 databases, 34 Nginx sites, GitLab EE, Neo4j, and live mobile app traffic — with zero downtime.

Why We Migrated⌗

Running a software company in Turkey has become increasingly expensive over the last few years. Skyrocketing inflation and a dramatically weakening Turkish Lira against the US dollar have turned dollar-denominated infrastructure costs into a serious burden. A bill that felt manageable two years ago now hits very differently when the exchange rate has multiplied several times over.

Every month, we were paying $1,432 to DigitalOcean for a droplet with 192GB RAM, 32 vCPUs, 600GB SSD, two block volumes (1TB each), and backups enabled. The server was fine — but the price-to-performance ratio had stopped making sense.

Then we discovered the Hetzner AX162-R.

That’s $14,388 saved per year — for a server that’s objectively more powerful in every dimension. The decision was easy.

I’ve been a DigitalOcean customer for nearly 8 years. They have a great product and I have no complaints about reliability or developer experience. But looking at those numbers now, I cannot help feeling a bit sad about all the extra money I left on the table over the years. If you are running steady-state workloads and not actively using DO’s ecosystem features, do yourself a favor and check dedicated server pricing before your next renewal.

What We Were Running⌗

This wasn’t a toy project. The stack included:

• 30 MySQL databases (248 GB of data)
• 34 Nginx virtual hosts across multiple domains
• GitLab EE (42 GB backup)
• Neo4J Graph DB (30 GB graph database)
• Supervisor managing dozens of background workers
• Gearman job queue
• Several live mobile apps serving hundreds of thousands of users

Old server: CentOS 7 — long past its end-of-life, but still running in production. New server: AlmaLinux 9.7 — a RHEL 9 compatible distribution and the natural successor to CentOS. This migration was also an opportunity to finally escape an OS that hadn’t received security updates in years.

The naive approach — change DNS, restart everything, hope for the best — wasn’t acceptable. Instead, we designed a proper migration path with six phases:

Phase 1 — Full stack installation on the new server Nginx (compiled from source with identical flags), PHP (via Remi repo, with the same .ini config files from the old server), MySQL 8.0, Neo4J Graph DB, GitLab EE, Node.js, Supervisor, and Gearman. Every service had to be configured to match the old server’s behavior before we touched a single DNS record.

SSL certificates were handled by rsyncing the entire /etc/letsencrypt/ directory from the old server to the new one. After the migration was complete and all traffic was flowing through the new server, we force-renewed all certificates in one shot:

certbot renew --force-renewal

Phase 2 — Web files cloned with rsync The entire /var/www/html directory (~65 GB, 1.5 million files) was cloned to the new server using rsync over SSH with the --checksum flag for integrity verification. We ran a final incremental sync right before cutover to catch any files changed after the initial clone.

Phase 3 — MySQL master to slave replication Rather than taking the database offline for a dump-and-restore, we set up live replication. The old server became master, the new server a read-only slave. We used mydumper for the initial bulk load, then started replication from the exact binlog position recorded in the dump metadata. This kept both databases in real-time sync until the moment of cutover.

Phase 4 — DNS TTL reduction We scripted the DigitalOcean DNS API to lower all A and AAAA record TTLs from 3600 to 300 seconds — without touching MX or TXT records (changing mail record TTLs can cause deliverability issues). After waiting one hour for old TTLs to expire globally, we were ready to cut over in under 5 minutes.

Phase 5 — Old server nginx converted to reverse proxy We wrote a Python script that parsed every server {} block across all 34 Nginx site configs, backed up the originals, and replaced them with proxy configurations pointing to the new server. This meant that during DNS propagation, any request still hitting the old IP was silently forwarded. No user would see a disruption.

Phase 6 — DNS cutover and decommission A single Python script hit the DigitalOcean API and flipped all A records to the new server IP in seconds. The old server remained as a cold standby for one week, then was shut down.

The key insight: at no point did we have a window where the service was unavailable. Traffic was always being served — either directly or through the proxy.

The MySQL Migration⌗

This was the most complex part of the entire operation.

Dumping the Data⌗

We used mydumper instead of the standard mysqldump — and it made an enormous difference. By leveraging the new server’s 48 CPU cores for parallel export and import, what would have taken days with a traditional single-threaded mysqldump was completed in hours. If you’re migrating a large MySQL database and you’re not using mydumper/myloader, you’re doing it the hard way.

mydumper \
  --threads 32 \
  --compress \
  --trx-consistency-only \
  --skip-definer \
  --chunk-filesize 256 \
  -v 3 \
  --outputdir /root/mydumper_backup/

The main dump’s metadata file recorded the binlog position at the time of the snapshot:

File: mysql-bin.000004
Position: 21834307

This would be our replication starting point.

Transferring the Dump to the New Server⌗

Once the dump was complete, we transferred it to the new server using rsync over SSH. With 248 GB of compressed chunks, this was significantly faster than any other transfer method:

rsync -avz --progress /root/mydumper_backup/ root@NEW_SERVER:/root/mydumper_backup/

The --compress flag in mydumper paid off here — compressed chunks transferred much faster over the wire.

Loading the Data⌗

myloader \
  --threads 32 \
  --overwrite-tables \
  --ignore-errors 1062 \
  --skip-definer \
  -v 3 \
  --directory /root/mydumper_backup/

The MySQL 5.7 to 8.0 Problem⌗

Being stuck on CentOS 7 meant we were also stuck on MySQL 5.7 — an outdated version that had been running in production for years. Before the migration, we ran mysqlcheck --check-upgrade to verify that our data was compatible with MySQL 8.0. It came back clean, so we installed the latest MySQL 8.0 Community on the new server. The performance improvement across all our projects was immediately noticeable — query execution times dropped significantly thanks to MySQL 8.0’s improved optimizer and InnoDB enhancements.

That said, the version jump did introduce one tricky problem.

After import, the mysql.user table had the wrong column structure — 45 columns instead of the expected 51. This caused mysql.infoschema to be missing, breaking user authentication.

Fix:

systemctl stop mysqld
mysqld --upgrade=FORCE --user=mysql &

But this failed the first time with:

ERROR: 'sys.innodb_buffer_stats_by_schema' is not VIEW

The sys schema had been imported as regular tables instead of views. Solution:

DROPDATABASEsys;

Then rerun the upgrade. Success.

Setting Up MySQL Replication⌗

With both dumps imported, we configured the new server as a replica of the old one:

CHANGEMASTERTOMASTER_HOST='OLD_SERVER_IP',MASTER_USER='replicator',MASTER_PASSWORD='...',MASTER_PORT=3306,MASTER_LOG_FILE='mysql-bin.000004',MASTER_LOG_POS=21834307;STARTSLAVE;

Almost immediately, replication stopped with error 1062 (Duplicate Key). This happened because our dump was taken in two passes — during the gap between them, rows were written to certain tables, and now both the imported dump and the binlog replay were trying to insert the same rows.

The fix: