# when I ran ESM TypeScript code the wrong way:
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module

# when I imported an ESModule from a CommonJS .js file:
Error [ERR_REQUIRE_ESM]: require() of ES Module .../lodash.js from .../index.cjs not supported

# when I imported an ESModule from a .ts file:
error TS1479: The current file is a CommonJS module whose imports will produce 'require' calls

# when I used ES6 import syntax in a .js file:
SyntaxError: Cannot use import statement outside a module

These errors are all related to importing, typechecking, and loading modules. The JavaScript ecosystem moves fast, and things changing over the last few years was not a surprise. However, it was surprising to see so many errors related to such a core piece of the language!

How we got here

Modules in JavaScript and TypeScript have changed significantly over time:

• Years ago, there was no module system for JavaScript and TypeScript. A number of solutions sprang up around ways to declare and load modules: IIFEs, LabJS, AMD, require.js, TypeScript namespaces, and more. Tooling support and interop were hit or miss.
• CommonJS emerged as a standard-by-convention for modules, across browser, server, JavaScript, and TypeScript.
• When ES6 came out, folks started switching over to import and export syntax (from CommonJS’s require and module.exports), using tools like Babel and TypeScript to compile code down to CommonJS.
• CommonJS can be challenging to statically analyze, and uses an inefficient, synchronous module loading algorithm at runtime. ES Modules were introduced as the way to use import and export, while at the same time improving code load times at runtime.
• ES Modules introduced significant complexity for NodeJS in particular: instead of reusing the .js and .ts file extensions, ES Modules in NodeJS require either using .mjs, or setting type=module in your package.json. Interoperating these modules with an ecosystem-ful of CommonJS remains painful.

Current state, by the numbers

I was curious – since ES Modules (import/export) were introduced in 2015, and NodeJS has supported type=module/commonjs, .mjs, and .cjs, with the goal of replacing .js, since 2019, to what degree have these new conventions been adopted?

I answered this with data, using two approaches:

1. Looking at the most starred JavaScript and TypeScript repos on Github
2. Looking at the most downloaded packages on NPM

The results are not rosy. After 5+ years, adoption of ES Modules remains weak:

1. Between 9-27% of JavaScript/TypeScript projects declare themselves to be ES Modules via the type (and lesser-used exports) fields in their package.jsons.
2. Less than 6% of JavaScript/TypeScript files declare that they are ES Modules via the .mjs, .cjs, .mts, etc. file extensions.

Note that these ranges come from the two approaches I used to estimate the numbers. Head here for more detailed data and code.

How do we fix it?

This helps explain why it’s so painful to interoperate ES Modules and CommonJS across both NodeJS and TypeScript: enough libraries use ES Modules that for many projects you need to either use ES Modules, or figure out how to interoperate ES Modules with your CommonJS code. At the same time, enough code still uses CommonJS that you often need to figure out how to include that legacy code in your otherwise-ES Module project.

The benefits of ES Modules are significant. Rolling everything back to CommonJS is not the way forward. Is there more we can do to simplify the ecosystem, and push harder on adoption? Some ideas:

1. We should kill .mjs, .cjs, .mts, etc. The vast majority of projects use type=module in their package.json, rather than file extensions. It would simplify things considerably if we drop support for these new file extensions and stick to .js, .jsx, .ts, and .tsx.
2. We should make type=module the default for new package.json files for the npm init, yarn init, and pnpm init commands. Package managers’ publish commands should warn when type is not set to module.
3. We should upgrade the most common libraries used by the community to ES Modules, either manually or through automated pull requests (this feels like something that can be semi-automated).
4. The NPM registry can require an explicit module field on new packages, making it clear when a package intentionally uses CommonJS (eg. because it targets legacy NodeJS versions).
5. NodeJS can officially drop support for require and module.exports in a future version, creating a bit more pressure to migrate.

I’d love to hear others’ thoughts. Have you also felt the pain of interoperating ES Modules and CommonJS?

Discuss this post on HackerNews or on Threads.

I am fortunate to have the opportunity to have a setup like this. How I ended up here is a long story, which I will save for another post. For this post, I’d like to talk about what I have learned about effectively contributing to an engineering organization when you are isolated from your coworkers physically, temporally, and organizationally.

Learning to work again

Before moving to Japan, I served as the tech lead for Facebook Groups. That meant organizing meetings, building slide decks and spreadsheets, writing scoping documents, responding quickly to chat messages and emails to keep teams unblocked… You know what I wasn’t doing a lot of? Coding.

After moving to Japan, with so little timezone overlap (9am my time is 4pm in San Francisco, 7pm in New York) I suddenly found myself with nearly the whole day free. My coworkers went offline a couple hours after my day started, until London woke up for the last hour of my working day. That meant nearly zero meetings and few synchronous chats.

Due to the incompatible timezones, I started attending group meetings less and less. At the same time, my coworkers naturally took me out of the critical paths of important projects. If anything was time-sensitive, I was no longer a good person to take care of it. I wouldn’t be included in chats and email threads anymore. My directors wouldn’t ask me to put together quick planning docs after exec reviews, nor would I be included in those reviews in the first place. This stung, at first – I was used to being included in meetings, and learning about new information before others did. These are the things that made me feel important! I didn’t want to give them up. But I had no choice.

At first I struggled to figure out how to spend my new-found free time, and how to feel good about my contributions to Instagram. I experimented with a few ideas: hacking on new product ideas with the other engineers in Japan, mentoring engineers in our Singapore office (larger than the one in Tokyo), fixing tech debt that others left behind.

It turned out that most of this work wasn’t easy to do. It was hard to effectively work on product while disconnected from the bulk of our US-based product organization; and it was hard to be a good mentor while having little context on what was going on in the Singapore office.

However, I did find that writing code again was rewarding. In my first two weeks in Japan, I landed more code than I had in the previous year in Menlo Park. I’m an engineer, and this made me happy – I missed coding! While hacking on a few product ideas (Instagram Maps and QR Codes) and fixing tech debt around our Python codebase, I kept a running list of opportunities I found along the way: abstractions that were repeatedly slowing down engineers and causing reliability issues, small improvements we could make to the type safety of our libraries, new infrastructure that we could build and ways we could make existing infrastructure better.

Leaning in

I started to shift the kinds of things I worked on, to focus more on simplifying our codebase, improving our infrastructure, and making our engineers more productive. I had done infra and DevX work in the past (some of it open source), but it had always been limited to 20% time projects. It was never my main gig. I considered myself a product engineer, but found myself sliding deeper into the world of infrastructure and DevX.

This shift was not something I did intentionally. Because my day no longer filled up with meetings and obligations that other people put on my plate, coding and fixing the issues I found while coding gradually took up more and more of my time, until six months into being in Japan I realized that it had become my main work. I felt happy about the work I was doing, and satisfied to write code again. I started to gain a reputation as an engineer that knows the codebase and has good technical opinions, despite having been at Instagram a relatively short time, due to how much code I was churning out and the pain points I was fixing for other engineers. (By this point, I was in the top 1% of engineers at Instagram by code output.)

Reflecting back on this a year later, I realize I was doing the work that other engineers should have been doing, but often couldn’t due to the realities of being in an organization that demands more of your time and attention as you grow more senior. Famously, at Meta even senior engineers are expected to code (we don’t have many architect types), but their level of code output is often not the same as more junior engineers. I had essentially turned myself into an intern, coding 80% of the day. This was a powerful change, which let me identify and execute on opportunities that others simply couldn’t.

Shifting how I operate

As I did this, I had to get comfortable leading from behind. I spun out a number of projects, recruiting engineers in the US and EMEA to lead and take over as soon as an effort developed sigificant communciation or coordination components. This meant delegating, and letting these emerging leaders be the public faces of projects that I might have previously spent longer incubating myself. I felt anxious about getting credit for this work, but quickly found that my worries were misplaced, and that my reputation continued to grow, and engineers and managers around the organization were not only aware of my work, but were appreciative of the opportunities I created for others and the impact this was having on Instagram. By becoming a more effective delegator, I was able to grow more engineers more quickly than I had in the past. It was gratifying, and it let me deliver results without having to participate in most meetings and synchronous communications myself.

Around this time, I shifted my communication style from trying to make my previous approach work in a new environment, to using my odd timezone to my advantage:

1. Instead of rapid back-and-forth over chat, I caught up on accumulated messages in the mornings and spent the first hour of every day responding in long form. I had to anticipate what the person at the other end of the chat was really asking, how they would respond, and what else they would need to know. By taking time to word a better response, I avoided time-consuming back-and-forth for both of us.
2. Instead of meetings, I used Workplace posts (Meta’s internal version of Facebook Group posts) to share thoughts, start discussions, and make announcements. This had the benefit of serving as a system of record and a form of documentation, while at the same time being more inclusive of people in different timezones and those who would not have been invited to or would not have felt comfortable speaking up at a meeting.
3. I focused my 1-2 hours of available meeting time each day on the conversations that absolutely had to be synchronous. This meant spacing weekly 1-1s out to biweekly or monthly, and turning down meetings more often.
4. I began flying to the US every quarter or so for in-person time. No matter how good I felt about my newfound ability to work asynchronously, there is simply no alternative to seeing your coworkers in person, catching up about their families and lives without the ticking clock of a virtual meeting, and grabbing beer after-hours to kvetch and ideate and get to know each other as friends.

I also had to adjust my schedule to better manage my own health. When I worked in California, I would drive from San Francisco to get to Menlo Park around 10am, then head home around 6 or 7pm. Now, since my time overlap with the US was in the morning, I had to shift my meeting times earlier, often to 8 or 9am. At the same time, it is important for my mental health to have a morning routine: I like to run, make coffee, and study in the morning; to avoid compromising on this, I shifted my schedule up, waking up at 6:30am to have time for these must-haves.

Closing

Overall, the move turned out well for me: I have learned to work effectively from a very different timezone, and am happy with the results. I don’t think this setup would work well for everyone, and I feel lucky that I was able to iterate to a working style that is good for me. I am grateful that my organization supports and appreciates the work I do, and is accommodating of the way I do it. I hope my experience resonates with others, and am eager to learn about others’ experiences in similar setups.

Discuss this post on Threads.

For some things, time is more important than timing:

• Investing (long term). Investing for the long term is all about holding a diverse set of assets for as long as possible. It doesn’t matter if you invest at discounted price, because (a) most of your gains come from the long-term effects of compounding, not from the price you bought at, and (b) you’ll probably time it wrong anyway.
• Learning. It’s never too late to learn a new skill or expertise. But the earlier you start, the more you will be rewarded with better opportunities, improved decision making, and higher job satisfaction. Like an investment, your knowledge will compound to improve all the choices you make and things you do throughout life.
• Relationships. Building your social network will give you access to opportunities and friends/partners you wouldn’t have had otherwise. The longer you do this, the more you’ll benefit yourself and others.

For other things, timing is more important than time:

• Investing (short term). I once sold my three best Pokemon cards on eBay for $0.99 plus shipping; today, those cards would have fetched at least 100x that. Short-term investing is about timing the market to sell at a peak, buy at a trough, or exploit an inefficiency before everyone else catches on.
• Starting a business. Many good business ideas failed because they were tried at the wrong time – either the market wasn’t ready for them, or the tech wasn’t ready. Successful businesses don’t just have a great idea and execute it well, but they need to do it at just the right time.
• Genes. For a mutated gene to take off in a population, it has to give its carriers a reproductive advantage in the environment at the time. A drought resistance gene isn’t useful in a year with heavy rain.

What’s the difference between these two classes of phenomena? Things that compound are all about time; the longer you do them, the faster the benefits build on each other. Things that are about exploiting a new opportunity are about timing; if you don’t do them at the right time, you won’t be rewarded. Try not to confuse the two.

Of course, this is the same tree that grows ぎんなん (Ginkgo nuts), an East Asian delicacy found in many dishes, 茶碗蒸し (Chawanmushi) for example.

Ginkgo has been around so long, it predates the dinosaurs! And we still eat it! How cool is that. This got me thinking – what are the oldest foods we consume today?

Criteria:

1. Must be edible by humans
2. Must be morphologically unchanged since its fossil age

Photo	Kingdom	Species	Common name	Age (years)
	Animalia	Tachypleus tridentatus	Horseshoe crab	480M
	Plantae	Ginkgo biloba	Maidenhair nuts	290M
	Plantae	Bryoria fremontii	Wila	250M?
	Plantae	Cladonia rangiferina	Reindeer lichen	250M?
	Plantae	Cycas revoluta	Sago palm	200M
	Plantae	Araucaria araucana	Monkey puzzle tree nuts	160M
	Plantae	Equisetum arvense	Horsetail	140M
	Plantae	Welwitschia	-	112M
	Plantae	Osmundastrum cinnamomeum	Cinnamon fern	70M
	Plantae	Trapa natans	Water caltrop nuts	66M
	Plantae	Nelumbo lutea, Nelumbo nucifera	Lotus	65M+

Note: I’m a hobbyist, and not a paleobotanist. Additions and edits are welcome, if I misclassified or missed anything.

If you’re building a web app at scale you have plenty of options for data fetching, transport, caching, and management:

• fetch, XMLHTTPRequest, etc. (REST)
• Swagger (REST)
• Falcor (REST)
• Apollo (GraphQL)
• Relay (GraphQL)
• Protobuf
• gRPC
• …

Each of these does a few things well, leaving the rest to you. What do you need to do well to be a good data fetching library?

Fetching Data

• Consistency when fetching data. When your app fetches new data, the UI should automatically update everywhere that data is used.
• Request batching. When multiple requests are made at the same time, they should be combined into one request, when reasonable, to avoid hitting browser-imposed concurrency limits.
• Avoiding network waterfalls. When fetching data for a component, you should make one request that includes data for all its descendents too, to avoid lots of network round trips. GraphQL solves this with fragments, but it’s still painful in modern clients (eg. Relay does this by requiring you manually compose fragments up the React tree so they’re fetched together, then pass fragment pointers down the tree to tell Relay where to look for fetched data; it also requires a compilation step).
• UX when fetching data. It’s a big burden for engineers to have to manually add loading spinners and error states, and engineers often forget.
• Colocation. You should declare data dependencies close to where they’re used, ideally in the same file. Same principle as colocating CSS with your components.
• File size. You shouldn’t have to ship kilobytes of metadata describing your data schema to clients in order to fetch data.
• Type safety. Your code should know what type of data it got from the server.
• Versioning. Updating your data model should be backwards-compatible; clients should be forwards-compatible.
• Realtime updates. Sometimes, you want data to be pushed, not pulled (eg. for things like notifications). But the APIs for pushing and pulling data are often different, both on the client and on the server.

Updating Data

• Consistency when updating data. When you update a piece of data, your data fetching library should automatically refetch dependent data that will change after the request completes (eg. if you update Person.FirstName, Person.FullName should automatically be refetched).
• Optimistic updates. The UI should optimistically render the success state before a server response comes back, if there’s a high chance the request will succeed. This makes your UI feel snappy and fun to use. Today’s data fetching libraries make this a lot of manual work.
• Request queueing. Requests that depend on one another need to be queued; independent requests can be sent in parallel. Otherwise, network weather and server queuing can cause your client to end up in a state that’s inconsistent with the server.
• UX when updating data. Product engineers shouldn’t need to manually render loading spinners and success and error notifications.
• Durability when updating data. Exponential backoff for retrying idempotent requests and idempotence tokens for network failures; eager persistance to disk for client/server failures.
• Type safety. You shouldn’t be able to send invalid input to the server, for some definition of invalid.
• Versioning. Updates to your mutation APIs should be backwards-compatible; clients should be forwards-compatible.

Why?

I think the reason this hasn’t been solved is it’s a hard problem that’s at the intersection of a bunch of fast-moving technologies:

1. Programming languages (to infer what data depends on what)
2. Databases (client-side stores are small databases)
3. Server APIs and protocols (REST, GraphQL, RPC, etc.)
4. UI frameworks (at this point, React has won)
5. User experience and perceived performance

So far, every attempt at solving data fetching has either tightly coupled all of these layers, leading to trouble gaining mass adoption (think Meteor); other solutions have tackled just one layer, ignoring the rest. This smells to me like the abstractions & APIs for each layer are wrong: you shouldn’t need tight coupling to solve data fetching. Instead, you need better abstractions.

Is this a problem that can be solved?

Thousands of people gathered outside the Capitol building in Washington DC today, some of them going into the building, some with guns. A woman got shot. All of this in response to Donald Trump’s remark:

We’re going to try and give our Republicans the kind of pride and boldness that they need to take back our country.

This is the latest in an escalating, years-long trend of different people seeing completely different realities in the US. What is to some a long-running trend of declining violence feels to others like a sharp increase; what’s to some a trend of increasing wealth and prosperity, is to others shrinking opportunity; what is to some a fair election, is to others an unfair steal.

It’s a good thing that different people have different political philosophies and values – competition of ideas is good, and leads to better ideas in the long run. We need conservatives and liberals and libertarians and socialists to ask questions and argue for ideas that others don’t.

There have always been radicals, good and bad, but the public largely agreed on most things most of the time, when it comes to the facts. Why has this changed? Or, maybe it’s just taking longer for people to converge on a common set of facts on a given issue?

⁂⁂⁂

I just finished Walter Lippmann’s 1922 classic Public Opinion. In it he argues that different people often draw completely different conclusions from the same facts due to three things:

1. Sampling. There’s a big universe of facts out there, and each media outlet reports on a tiny piece of this universe.

2. Stereotypes. When you read a news story, you unconsciously pattern match and associate it to related examples you have in your mind (poor people are lazy, young mothers are desparate, immigrants are criminals, etc.).

3. Context. When you read a news story, you subconsciously have some fascet of your own identity in mind (as a Republican, as a pro-choice person, etc.).

All three are in effect when stories are reported on and consumed. It’s a series of lenses that samples, then distorts, the truth in a way that given the same real-world event, different people may come to completely different conclusions about what happened.

For example, one person might read a CNN article about how voter turnout has been great this election, picture in their mind a stereotype of a concerned citizen going out to due their civic duty and vote for the first time, and feel proud about how this will help their party, the Democrats, win.

Concurrently, another person might listen to their trusted Breitbart podcast about several documented cases of voter fraud – which invokes in their mind a stereotype of a crazed liberal trying to push their unpopular policy by whatever means necessary – and feel their duty as a patriot to do something about it.

Both stories may have grains of truth within them, but by the time they are reported on and consumed by people, these grains are far gone and the two peoples’ resulting perceptions couldn’t be more different.

⁂⁂⁂

I’d add a #1b to Lippmann’s series of lenses: Fake News.

Fake News amplifies #1 (sampling) significantly. It was less of a problem in Lippmann’s time since the News world was much smaller, and only highly edited and fact-checked publications – with their reputations at stake – had enough circulation to influence public opinion nationally.

Lippmann might have called Fake News “rumors”, not “news”, since they definitely existed in and before his time (think medieval stories about Jews putting Christian babies’ blood in Matzos). What’s changed now is rumors have national circulation.

⁂⁂⁂

So, where do we go from here?

I think we should use Lippmann’s framework for public opinion as a roadmap for how to unify public opinion a bit more:

• Sampling:
  1. Regulate news stations, and force them to give air time to the other side too (the FCC’s Fairness Doctrine Act did this in the US till 1987 – last year there was a bill to bring it back, H.R. 4401).
  2. 1000x government grants for public media, to make it possible for stations like NPR to compete with Fox, CNN, NYTimes, etc. in terms of national distribution, and to spur more competition in public media.
  3. Encourage social media companies to downrank content from people whose past content has been disputed.
• Stereotypes:
  1. Improve public education by paying teachers more, breaking up unions, increasing competition in public schools (with vouchers, etc.), and funding early childhood and after-school programs.
  2. Increase arts grants to spur empathy and imagination.
  3. Invest in ways to get rural folks to move to cities for a few years, and vice-versa. People need to see the other side as concrete people, not abstract evils.
• Context:
  1. Invest in a national counter-propaganda (in the Bernays sense) that promotes shared values (democracy, freedom, decency) over the tribal propagandas of today.
  2. Discourage tribalism and build a stonger a national identity with all of the above.

type Props = {
  href?: string
  onClick?: () => void
  text: string
}

function Button(props: Props) {
  if ('href' in props) {
    return (
      <a className="Button" href={props.href}>
        {props.text}
      </a>
    )
  }
  if ('onClick' in props) {
    return (
      <button className="Button" onClick={props.onClick}>
        {props.text}
      </button>
    )
  }
  throw ReferenceError('You must pass either href or onClick to <Button />')
}

let a = <Button text="Click me" href="https://github.com" />
let b = <Button text="Click me" onClick={() => {}} />

// OK. Should be an error
let c = <Button text="Click me" href="https://github.com" onClick={() => {}} />

// OK. Throws an error at runtime
let d = <Button text="Click me" />

In this example, Button is a React component that takes a prop text, and either an href or onClick prop controlling what happens when you press the button. Either href or onClick might be defined, and we throw a runtime exception if neither of them are defined.

This code has a few problems:

1. The error case (when neither href nor onClick are passed in) is caught at runtime. Can we catch it at compile time instead?
2. As a person using <Button />, I don’t know if I should pass in href, onClick, or both.
3. We only handled 2 cases in our component’s implementation, but there are actually 4 cases to handle!

When you have an object type with nullable, optional, or union-typed fields, the number of states the object can be in quickly explodes. This example has two optional fields, leading to 2²=4 states. If it had 3 optional fields that would be 2³=8 states, 4 would be 2⁴=16 states, and so on. The number of states can grow fast, slowing down typechecking, introducing subtle bugs in your code, and making your APIs a little bit harder to reason about.

When possible, use unions of objects instead of objects with nullable fields.

Since TypeScript unions are not disjoint by default, you’ll want to also add a special “tag” field on each object in the union: a field that’s typed as a literal (string, number, boolean, etc.) that hints to TypeScript that the union is disjoint – that is, that props must be the first object in the union, or the second, but not an object that has fields from both. (Concretely, if you don’t add the field, TypeScript won’t complain about c below):

type Props =
  | {text: string; type: 'href'; href: string}
  | {text: string; type: 'onClick'; onClick(): void}

function Button(props: Props) {
  if (props.type === 'href') {
    return (
      <a className="Button" href={props.href}>
        {props.text}
      </a>
    )
  }
  return (
    <button className="Button" onClick={props.onClick}>
      {props.text}
    </button>
  )
}

let a = <Button text="Click me" type="href" href="https://github.com" />
let b = <Button text="Click me" type="onClick" onClick={() => {}} />

// Compile-time error
let c = (
  <Button
    text="Click me"
    type="href"
    href="https://github.com"
    onClick={() => {}}
  />
)

// Compile-time error
let d = <Button text="Click me" />

This approach enforces at compile time that either href or onClick, and not both, are passed as props to <Button />. That means we don’t need to throw runtime exceptions anymore, and we can trust TypeScript to verify that we’ve covered our cases.

Note that you can’t always use this approach. There are times when it’s impossible or needlessly wordy to split out your object of unions into a union of objects. Keep this technique in mind for the times that it does make your code more terse, safe, and easy to use.

When can that happen?

• Exceptions bubble up through the call tree
• State has to be lifted up to the root of a call tree
• If a function is async, its ancestors must be too
• Centralized services also centralize any decentralized services that call them
• In a physical system, if you know the position and momentum of an object A and it collides with another object B that you don’t know one of those quantities for, then you no longer know them about A. (caveat: I’m no physicist)

Concretely:

async function a() {
  // a has to be async to await b()
  await b()
}

async function b() {
  // b has to be async to await c()
  await c()
}

As a programmer, that seems bad. Bubbling up breaks encapsulation and makes things harder to compose. If I have a nice application and need to mark a function deep in the app async, why do I need to update all of its callers to be async too? Why should a parent know about its grandchild?

For good reason. At runtime, some contagious things are special:

• Exceptions bubble up through the call stack
• await pauses execution

We want to model these runtime behaviors in our language:

• Asynchronous code needs to be awaited before continuing. As a consumer of an asynchronous function, you want to know that it’s asynchronous so that you can treat it as special.
• State is a function parameter, because the function’s computation depends on it. As a consumer, you need to pass the state to it.
• Functions that throw are modeled with a throws clause, Either, Err, or nothing, depending on the language. As a consumer, you might want to know that something throws so you can handle it.

In some languages, all of these are captured in a function’s return type. The idea is that these aren’t implementation details – consumers really should know about them. These languages model runtime behavior using types.

• Async: Future/Promise
• State: The State monad, or passing a callback down the call stack
• Exception: Either, Err, throws

But, you sometimes want a trap door:

• Async: .then, React suspense boundaries
• State: Mutable state, React local state
• Exceptions: try/catch, React error boundaries

These are ways to avoid contagion.

What unites all of those? They’re effects, that different languages model differently. Some languages keep them implicit (like throw in languages that don’t have throws clauses), some make them explicit (the State monad).

There’s nothing inherently contagious about these things, as evidenced by the languages that support trap doors for them. It’s up to language designers to say “this thing should be contagious” or “this thing shouldn’t”.

What should be contagious, but isn’t? What is contagious, but shouldn’t be?

Why’d I write the book? I love TypeScript, and I didn’t see any great resource out there that explained not just the language and build system features, but went deeper to explain why things were designed the way they were, how the different parts of the language fit together, how that compares to other languages out there, and how it all looks in practice. Those are the sorts of things you can only learn through lots of research and practice, and that was sorely missing from the handful of relatively brief and practical TypeScript guides out there today.

The rest of this post is a brief laundry list of tips for writing your own technical book.

Getting a book deal

Make ‘em fight for it. One day, I got a cold email from Packt Publishing asking me to write a book on TypeScript. I’d actually been thinking about the idea for a few months, having sent a series of unreturned cold emails to an O’Reilly acquisitions editor pitching a handful of topics to write about (one that’s close to my heart, which I hope to revisit someday, is a book that teaches you functional programming in JavaScript, starting from basic middle school algebra). I really wanted to write a book for O’Reilly since they published a bunch of the technical books that I’d learned from over the years (Zakas’ excellent High Performance JavaScript and Grigorik’s High Performance Browser Networking are a couple of my favorites). So I forwarded Packt’s email to O’Reilly, and got a response the next day that O’Reilly was interested, and that I should send over an outline and a timeline. We had a signed contract within the week – I’d get 10% of print sales, plus 25% of digital sales.

Pad the timeline. Like any other engineering project, the time you think it will take to write a book is going to end up being a fraction of the time it actually takes. Why? Even if you’re a subject matter expert with an outline in hand, there are a lot of unknowns: the time it takes to do research; the time it takes to come up with good examples; the time it takes to derive reference implementations; the number of times you delete and re-write sections to get them right; the overhead of coordinating contributors, and so on. Take your estimate, and pad it like you would an engineering estimate. For example, I estimated 8 months, while it actually ended up taking 15.

Writing

Start with an outline. Without a clear idea of what you’ll cover, it’s easy to get off-topic. It’s also easy to get burnt out from the sense of having a lot of content left to cover with no clear sense of progress or of what’s left. With an outline, it’s easy to know how far you’ve come, and how long you have left to go.

Schedule regular time to write: every morning, every weekend, etc. With a full-time job, it can be hard to find time to write. And when you’re not writing, you might feel guilty that you’re not writing, which can lead to stress. That’s why blocking off regular time to write is key. My schedule was most Saturdays and Sundays, from 9am to 6pm. I also noticed that mornings were by far the most productive for me: while in the morning I could often get into a state of flow, this was much harder to do at night. Rinse and repeat: doing this every weekend, for months at a time, I steadily chipped away at my outline.

Get feedback early, get feedback often. Like starting a business, you’ll often have intuition about what problem needs to be solved by your book. And like a business, you’re going to get a lot of the details wrong at first. To make sure you’re on the right track, get as much feedback as you can from potential readers: show your outline to colleagues before you write, run your drafts by friends and readers recommended by your editor (I asked O’Reilly for this on a couple of occasions); get as much feedback as you can, as early as you can. In Andy Grove’s words, “a common rule we should always try to heed is to detect and fix any problem at the lowest-value stage possible”.

Keep a running list of stylistic conventions. You’re going to have to form a point of view on a huge number of stylistic conventions as you write. For me, that included things like whether to use tabs or spaces, semicolons or no semicolons, how to consistently represent inferred types in code blocks (I used comments), how to consistently represent evaluated results in code blocks (I used comments prefixed with “evaluates to”), how to represent function and method names when talking about a function (fn, fn(), .fn, or .fn()?), and so on. There’s a lot of decisions to make, and you won’t be able to make all of them upfront. Keep a running list, and when you make up your mind about one, go back and update your manuscript to conform to that convention. The earlier you can make up your mind, the easier it will be to update everything.

Use lots of TODOs to keep track of what’s left. Anytime you want to revisit something – a missing code sample, a section that reads weird and should be re-worded, a section that you don’t feel like writing at this very moment and want to circle back on later – leave a TODO. It’s easy to grep for later on, and it’ll help make sure you don’t miss anything.

Editing

When editing, print out your manuscript and edit it with a red pen. Yes, I mean with a real pen, on real paper. There’s a lot of issues you’ll catch this way that you won’t notice on screen: from grammatical errors and formatting, to discontinuities, to explanations that could use more work. Also, it feels fancy to sit at a cafe and edit a 300-page stack of papers in pen (unless you’re a teacher, in which case you’re thinking “that’s cute”).

Read your manuscript all the way through. It might sound surprising, but after months spent writing it’s not easy to pause and take the time to read throught everything you just wrote. When you read all the way through your manuscript, you’ll notice issues that you wouldn’t have noticed when reading through a section or a chapter at a time. Certain kinds of problems you’ll only be able to find this way: things like stylistic inconsistencies (eg. you used capitalized comments in one place, and lowercased in another place), discontinuities (eg. you covered a topic twice, or forgot to give context for something), and so on. This sort of editing can be exhausting, but is critical to a high-quality result. Think of it as dogfooding.

Expert reviewers won’t give you as much as you want. While reviewers suggested by O’Reilly were super helpful as a sanity check, they won’t give you the deep domain-specific insights and pushback that you want. They’ll review for high level structure and comprehension, but won’t go into details like “is this the right way to derive a Promise type?” or “is this statement about module loading technically correct?”. More generally–

Getting good technical editors is hard. While O’Reilly provided good technical editors, I had to seek out subject matter experts myself and convince them to look at my manuscript. To do that, I asked a few friends, and emailed a bunch of people that I’ve met via Github and on Meetup to ask if they’d like to take a pass through my manuscript. Of course, that’s a giant ask that comes with a significant time commitment and little pay-off (O’Reilly can front technical editors a few hundred bucks as a thank you, which can feel inadequate for the 10+ hours an editor might spend editing your manuscript). Despite that, Daniel Rosenwasser from the TypeScript team offered to step in and edit. His contributions were the most valuable edits I recieved by an order of magnitude, and I’m immensely grateful for his taking the time to give feedback on my manuscript.

Misc.

While getting good technical editors is hard, getting good contributors is harder. Initially, I was hoping to get a range of contributions by subject matter experts: someone to talk about module loading, someone to talk about reactive programming, people to talk about React, Vue, and Angular. Turns out, a lot of subject matter experts don’t have time to do that. Of the dozen contributors that I planned to recruit, I ended up with two, one of which O’Reilly suggested. For these two contributors, it took a lot of work to get their contributions to a place where I could integrate them with my manuscript: back-and-forth on what content they should include, updating formatting, and integrating their sections with the rest of the manuscript. Eventually, I ended up re-working their sections significantly. As an author you have a lot of opinions about how to best present content, which can be hard to reconcile with other contributors – unlike collaborating on code, where you can enforce conventions with frameworks and linters, it’s a lot harder to be on the same page when it comes to writing style.

Writing is lonely (Ben Horowitz said it first). No one knows better than you whether to write A or B, what’s the right way to explain something, or whether or not your solution is actually correct. No one can tell you whether one way to explain something is better than another way. You’re going to spend an inordinate amount of time by yourself, at a cafe or at home, going back and forth on different ways to do something, wincingly playing out in your mind the reactions from the community (“This is the worst idea ever!”; Or, “this is ground-breaking stuff!”). Explore different ways to do a thing and run it by as many people as you can; then, pick an option – no one else can do that for you.

If you’re just starting to write your technical book, or have been thinking about taking the leap, I hope this list helps you. Am I missing anything? Shoot me an email with edits, additions, or insinuations.

• Do you treat derived data and non-derived data the same?
• Should users interact with the two the same way?
• Do you re-compute dependent data in a lazy or eager way?
• How much control do you need over re-computing derived data?

A popular solution is selector libraries like Reselect. The way they work is:

1. A changes
2. Define a function (“selector”) S that takes State A and returns State B
3. When you want to read B, you call S with A

To make this pattern work efficiently, S of course has to be memoized, and State A immutable for that memoization to work. This approach makes the following tradeoffs:

• You treat derived and non-derived data fundamentally differently
• Users interact with the two differently
• You re-compute derived data in a lazy way
• You don’t have control over how derived data is re-computed

Undux makes a different set of choices when it comes to derived data.

With Undux, you treat derived data the same as non-derived data, both from the store’s point of view and your user’s point of view. That’s because from your Component’s point of view, the two are the same. In fact, if your component (or its container) knows what’s derived and what isn’t, that might be a sign of poor encapsulation.

To derive data in Undux, you just compute it with an effect. Optionally, prefix its key with something:

store
  .on('networkData')
  .pipe(map(postProcessData))
  .subscribe(store.set('derived/processedNetworkData'))

This approach makes different tradeoffs than selector libraries do:

• You treat derived and non-derived data the same
• Users interact with the two the same (less a namespace like derived/, if you want)
• You recompute data in an eager way by default, though you can make it lazy if you want
• You have fine control over how to re-compute derived data:

store
  .on('networkData')
  .pipe(
    debounceTime(200),
    filter((_) => _ !== null),
    map(postProcessData)
  )
  .subscribe(store.set('derived/processedNetworkData'))

Why does Undux do it this way? Isn’t there something holy about a clean source of truth?

My view is there isn’t. Fundamentally, all data is derived. Even the list of users you got back from a GraphQL endpoint – that’s a function of two things:

users = F(initialState, API request)

Or, a count of how many times you pressed a button. That’s derived from:

count = F(initialCount, Stream(clicks))

That is, users are derived from an initial state and a network request. And a click count is derived from an initial count and a stream of clicks.

Selector libraries draw a line between primary state – stuff like initial state, state from the network, state from user intreactions – and n-ary state that’s computed from that state. But isn’t all state (except intial state) computed when it comes down to it? Because JavaScript doesn’t have first-class monads, you don’t usually think of it like that. But you should.

And when you do, you realize it makes life easier to forget about drawing lines about where state comes from, and letting components interact with all state the same way. It’s a simpler mental model, it’s easier to reason about, it’s better enapsulation, and with Undux, is more powerful too.

What do you think?