About Projects Blog Tags Email GitHub Twitter Press Feed

20 Jan 2016
The best books I read in 2015

Fiction I loved reading in 2015 that related to gender, race, and class

  • This Burns My Heart by Samuel Park - Heartbreaking, amazing, gorgeous. A woman in South Korea who marries wrong and what that means for her freedom and her life.
  • The Crimson Petal and the White by Michael Faber - A prostitute who tries to get out by becoming the live-in mistress and nanny for a rich guy. Class issues, where book smarts are not enough.
  • The Beacon at Alexandria by Gillian Bradshaw - Roman girl wants to be a doctor, so she disguises herself as a boy and apprentices herself to a Jew. Such Alanna wow &c!
  • Sisters of the Revolution: A Feminist Speculative Fiction Anthology edited by Ann and Jeff Vandermeer - Packed full of spectacular stories!
  • A Door Into Ocean by Joan Slonczewski - A planet of women near a co-ed planet. Politics, gender, nifty scifi.
  • Carnival by Elizabeth Bear - A matriarchal world where “stud” males (as opposed to gay “gentle” males) have very few rights, a gay male couple as ambassadors from a world more like ours but with many issues of its own, and of course alien life forms. I swear this is not porn. Honest to gosh. The thing I really loved about this one was that there were complicated ethical issues with all represented worlds.

Non-fiction I loved reading in 2015 that related to gender, race, and class

  • The New Jim Crow: Mass Incarceration in the Age of Colorblindness by Michelle Alexander - You basically have to read this to understand our criminal justice system. It’s fucked up and horrible and real. A perfect companion piece to Torture & Democracy, which you should also read.
  • Picking Cotton: Our Memoir of Injustice and Redemption by Jennifer Thompson-Cannino and Ronald Cotton with (ghostwriter) Erin Torneo - A white woman and the black man she (unintentionally) falsely accused of rape, their separate stories, and the way they met and became friends when they finally did the fucking DNA testing and got him out of prison many years later.
  • Wifework: What Marriage Really Means for Women by Susan Maushart - Really fascinating and disturbing. Some of it feels like obvious bullshit. Some of it cuts to the quick.

Other fiction I loved reading in 2015

  • The Expanse Book 2: Caliban’s War by James S.A. Corey - Fantastic! So much better than the first one and all subsequent ones! All about the risks and dangers of communicating and acting with imperfect knowledge.
  • The Fate of Mice by Susan Palwick - Short stories, pretty great.
  • All She Was Worth by Miyuki Miyabe, translated by Alfred Birnbaum - Detective story, in Japan, largely discussing consumer debt.
  • The Siren by Tiffany Reisz - “Erotic romance”. No, but this was great! It really spoke to the tension between who you love and what you need, when they don’t coincide.
  • Aunty Lee’s Delights: A Singaporean Mystery and Aunty Lee’s Deadly Specials: A Singaporean Mystery by Ovidia Yu - I had to read these on my way to Singapore, of course. Fun fluff!
  • The Three-Body Problem by Cixin Liu - I mostly enjoyed this one because I kept thinking it was about different concepts during various phases along the way reading through it. (Like, for a while there I was convinced it was about the color grue.)

Books I loved reading in 2015 that related to art

  • The Natural Way to Draw by Nicolaides - Particularly great for the stage I was at earlier in the year, I think. Lots of good exercises, and more philosophical. For instance, a big takeaway for me is that when doing gesture drawings you want to feel tension in your body echoing the pose.
  • Watercolor for the Fun of it: getting started by John Lovett - Really helpful exercises and tips, especially re playing with texture.
  • Tate Watercolour Manual: Lessons from the Great Masters by Tony Smibert - Big lessons: different kinds of brushstrokes, and I really should play with tonal value more.
  • The Urban Sketcher: Techniques for Seeing and Drawing on Location by Marc Taro Holmes - I love this author and I love his art and I kinda want to be him when I grow up and this was the best!
  • Local Color: Seeing Place Through Watercolor by Mimi Robinson - Advice on mixing color in various ways, and now I totally want to build my own color palettes for various seasons and walks &c!
  • Urban Watercolor Sketching: A Guide to Drawing, Painting, and Storytelling in Color by Felix Scheinberger - Some good stuff about luminosity, being sparing with color, drawing attention to the important bits, and storytelling in here. Also I love the sketches throughout the book.
  • Experimental Drawing by Robert Kaupelis - I got lots of great ideas for drawing exercises from this.
  • Creative Sketching Workshop: Inspiration, Tips and Exercises for Sketching on the Move edited by Pete Scully
  • Urban Sketching: the complete guide to techniques by Thomas Thorspecken

Books I loved reading in 2015 that related to programming and tech

  • Storm Applied: Strategies for real-time event processing by Sean T. Allen, Matthew Jankowski and Peter Pathirana - So useful! (Storm is a tool I work with and reading this made my job easier, so that was nice.)
  • Seven Concurrency Models in Seven Weeks: When Threads Unravel by Paul Butcher - I was pleasantly surprised by how many of these I was already familiar with, but overall it was a good overview with suggestions for interesting follow-up reading at the end of each section.
  • Functional Programming in Scala by Paul Chiusano and Runar Bjarnason - Lots of stuff I already knew, but pretty clear explanations of some FP concepts, and I like finding good articulations of familiar concepts to help with sharing them.
  • Being Geek: The Software Developer’s Career Handbook by Michael Lopp - My big takeaway here was learning about trickle lists, which turn out to be a pretty useful tool.
  • Beautiful Code: Leading Programmers Explain How They Think edited by Andy Oram & Greg Wilson

Other non-fiction I loved reading in 2015

  • House Rules by Rachel Sontag - A memoir about a fucked up family. Very hard and compelling to read. I felt nauseous through most of it, and yet keep recommending it to people. There’s something particularly intense about reading about families which are obviously horrible but don’t involve any physical abuse.
  • It’s complicated: the social lives of networked teens by Danah Boyd - Actually really interesting discussions of privacy as a social norm rather than technological mandate.
  • The Lone Samurai: The Life of Miyamoto Musashi by William Scott Wilson - Christ, what an asshole. And yet in some ways I want to be him when I grow up.
  • Social Engineering: The Art of Human Hacking by Christopher Hadnagy - Hilariously reminiscent of the time a hypnotist watched me try to bully my boyfriend into reminding me to create a tea blend I wanted to try a week later and then told me that I’m very good at neuro-linguistic programming.

Books I loved re-reading in 2015

  • The Last Samurai by Helen DeWitt - This remains one of my favorite books of all time.
  • Unwind by Neal Shusterman - A world in which abortion is illegal but parents can legally have their children unwound between the ages of 13 and 18.
  • Deathless by Catherynna M. Valente - Still gives me a lot of very complicated feels.
  • Narbonic: The Perfect Collection Books One and Two by Shaenon K. Garrity - Look, it’s about a cute evil mad scientist and her adorable henchman named Dave. Of course I love it! This is basically the story of my marriage!

Total number of censored favorites not appearing in this post: 10
Total number of books read in 2015: 160

02 Jan 2015
The best books I read in 2014

This was a rough year for me and books - I had a lot of family deaths and a very short commute and started an awesome new job, all of which got in the way, as you might imagine. I only ended up reading 79 books total, which a barely over half my usual count for a year :shamefaced:. Luckily, I came across a few gems to share with you!

Fiction I loved reading in 2014

  • Spirits Abroad by Zen Cho - Absolutely spectacular Malaysian fantasy short stories. This is the big must read of the year, people. Get to it.
  • Ancillary Justice by Ann Leckie - Written from the POV of a starship’s AI in a human body from a culture with no concept of gender. Really fantastic and original scifi. Read while drinking amazing tea. (Ancillary Sword, its sequel, was fun but not really as great as its predecessor.)
  • A Guide for the Perplexed by Dara Horn - Memory and family and complicated stories and choosing stories and survival and acceptance and hope and redemption and pain passed down through generations. My entire family needs to read this one.
  • Fledgling by Octavia Butler - Vampires done with issues of race and gender in society.
  • The Pattern Scars by Caitlin Sweet - Tremendously twisted fantasy novel - seers, magic, mind control, trigger warning for sexual abuse, lots of horrors and betrayals. No heroes.
  • The Witch in the Almond Tree by C.S.E. Cooney - Gotta admit, she writes great sexy witch stories. Though this short story of hers was somehow even hotter (deeply NSFW, though).
  • The Steel Remains by Richard K. Morgan - Fantasy novel, queer characters, discrimination, brutality.
  • Mission Child by Maureen F. McHugh - Cultural and gender identity, another book with themes of getting lost and isolated from one’s culture and becoming someone else, or maybe not.
  • The Lies of Locke Lamora, Red Seas Under Red Skies, and The Republic of Thieves by Scott Lynch - Really fun fantasy novel trilogy!
  • Graceling by Kristin Cashore - A light fantasy novel, where some people have graces (superpowers, basically). Our protagonist is a teenage girl with a superpower that makes her great at killing, which of course means she unwillingly IS the brute squad for her uncle, the king. But then the story really gets started.
  • The Ghost Bride by Yangsze Choo - Great little novel about marriage prospects and Malay Chinese afterlife beliefs.
  • Last Days by Joyce Carol Oates - Great short stories. “If I had it in me to love anyone I would love you… but you know the Holy Ghost saw fit to depart from me leaving just this husk and whited sepulchre, and I can’t lift a finger in rebellion. ‘The Spirit bloweth where it will.’ Do you know what that means? That means everything.
  • The Birthday of the World and other stories by Ursula K. Le Guin - Lots and lots on gender and interesting weird relationship structures.

Non-fiction I loved reading in 2014

  • How Doctors Think by Jerome Groopman, M.D. - Mentions a doctor who kept a log of all mistakes he made. Ever since reading this, I kinda want to keep a journal of all bugs I fix/encounter. My main take-aways: Always generate a short list of alternatives, even when you think you know the answer. When talking to doctors: Ask what body parts are near where you’re having your symptoms; what body part might be causing the symptom? What else could it be? What’s the worst thing it could be? Is there anything that doesn’t fit? Is it possible I have more than one problem? (list on p263)
  • Key Concepts in Life and Death: Inside Moves and Under the Stones Techniques by Richard Hunter - Really great Go book!
  • The Unix Programming Environment by Brian W. Kernighan and Rob Pike - I went through this one slowly, with lots of interruptions, over the course of about a year - I actually counted it for last year’s best-of list because I started it in 2013, but I’m counting it again because I finished it in 2014 and it’s still spectacular. I can absolutely see it as a book I can go back and reread and expect to get more out of each time.
  • The Writing Life by Annie Dillard - I read this book because these quotes from it spoke deeply to me, and it lived up to them completely.
  • A Field Guide to Getting Lost by Rebecca Solnit - Beautiful, resonant, a perfect read while traveling through Mexico with my friend Bonnie this past spring.
  • And the Band Played On: Politics, People and the AIDS Epidemic by Randy Shilts - I kept having to close this book to remember how to breathe and stop crying. Intense and amazing and heartbreaking and probably not what I should’ve been reading in this year full of deaths, but so worth it.
  • Something Like an Autobiography by Akira Kurosawa - Yes! He has an exuberant, fascinating view on life.
  • The Fire Next Time by James Baldwin - Race in America. Painful, important.
  • Nothing to Envy: Ordinary Lives in North Korea by Barbara Demick - I had absolutely no sense of what life is like there before reading a few glimpses this year, and this was the most riveting of them all.
  • The Good Women of China by Xinran - Xinran is basically the Studs Terkel of China.
  • Factory Girls by Leslie T. Chang - Migrant young women from rural China working in factories. Particularly striking to read from my current context of the tech industry here.

Books I loved re-reading in 2014

  • for colored girls who have considered suicide / when the rainbow is enuf by Ntozake Shange - One of my favorite books of poetry of all time. Still makes me cry.
  • Earth by David Brin - I still love this novel as much as I did when I first read it probably 15ish years ago - if anything, it’s only become more strikingly relevant.
  • Alien Sex edited by Ellen Datlow - Short stories about the obvious. Not erotica, honest - it’s just a bunch of really good, weird scifi.
  • Vox by Nicholson Baker - This one really is hot, though. It’s sort of a novel about a single phone sex conversation, except it’s also thoughtful and neurotic and nerdy and brilliant and bizarre.
  • Under the Skin by Michel Faber - Had to reread because apparently there’s now a movie? This book has itched at the back of my head for years and years. If you aren’t familiar with the plot, I strongly suggest reading it without reading any reviews of spoilers first. It’s extremely disturbing and told perfectly.
  • Blindsight by Peter Watts - Way, way better than its recently released sequel. Vampires without being boring. Cognitive science and how our brains work. Breathtakingly good, really.

08 May 2014
git SHAmend! a quick way to amend changes into an older commit

When I work on projects with slow test suites, my workflow often ends up looking sort of like this: I make some changes on a branch, run the tests that seem likely to be relevant locally, and then push the branch off to tddium (or whatever) to see if any of the other tests fail unexpectedly.

I like each of my commits to be clean and green in isolation, to make digging through project history easier in the future. To that end, when I make a bunch of commits before running all my tests, I often end up fixing bugs and using interactive rebase to merge the fixes into earlier commits before merging my branch in. (So long as I’m the only person working on that branch, at least!)

It’s a reasonable enough process - stage my changes for the fix, commit them as a WIP, then open up interactive rebase to amend them as a fixup to the right commit from earlier. That’s so many steps, though! And I’m so lazy! I really just want to be able to run something like git shamend SHA_FOR_EARLIER_COMMIT instead. (Or better yet, git-smend or git-sm for short!)

So, I wrote git-shamend to solve this problem for me. The full script is available here - just copy it to /usr/local/bin (or wherever you prefer to keep such things) and you’ll be able to use it with git shamend SHA_TO_AMEND.

I initially planned on writing SHAmend! using git’s low-level (“plumbing”) commands, but all my twitter buddies told me I shouldn’t feel guilty about building on top of porcelain (git’s high-level commands) instead if I wanted to.

The meat of how it works is like this:

First, we get the SHA for the reference you pass in. This avoids problems that could come up if you pass in something like HEAD^, whose meaning changes whenever you add a new commit (as we do later on).

  SHA_TO_AMEND=$(git rev-parse "$@")

If the reference you pass in is a commit that’s in your current branch…

  if git merge-base --is-ancestor $SHA_TO_AMEND HEAD

…then git-shamend commits your staged changes, marked as a fixup (which is an amendment that retains the original commit message) to that earlier commit…

  git commit --fixup $SHA_TO_AMEND

…and if you have any remaining unstaged changes…

  git diff-index --quiet HEAD

…stashes them so they don’t interfere with the upcoming rebase…

  git stash

…and runs an interactive rebase automatically to get that fixup amended properly to the earlier commit you specified.

  GIT_EDITOR=true git rebase -i --autosquash "$SHA_TO_AMEND^"

Wait, it runs an interactive rebase automatically? That sounds kinda weird. It’s interactive, but it’s not!

Git uses the environment variable $GIT_EDITOR to figure out which editor open up to allow you to move commits around when running an interactive rebase. Setting that to “true” here causes git to use true as your editor for this command, where true is just a tiny unix program that does nothing except exit with a successful exit code.

So from git’s perspective, it runs an interactive rebase and opens up an editor for you to move around commits, which ‘you’ close successfully. Great, that’s all git-rebase needs from, er, ‘you’ - assuming there are no conflicts, it can handle the rest on its own!

This works because the --autosquash flag tells git-rebase to put fixup commits in the right place for you before opening up the editor, so there’s really nothing you need to do to get things sorted out right.

From your perspective, git just kinda does its thing without bothering you. Gotta love that.

But what if something does go wrong? If the rebase exits unsuccessfully (that is, with a non-zero exit code)…

  if [ $? -ne 0 ]

…then git-shamend aborts the rebase and resets that fixup commit it created earlier, to clean up after itself. (And echos a warning, natch. All that stuff is in the actual git-shamend script.)

  git rebase --abort
  git reset --soft HEAD^

And at the very end, if you did have any unstaged changes that were stashed earlier…

  if [ $NOTHING_TO_STASH -ne 0 ]

…they’re popped from the stash, to make return your working directory to its pre-SHAmend!ing state.

  git stash pop

Now I can smend smend smend as much as I want, way more efficiently!

02 May 2014
Finding words that sound alike but are spelled wildly differently

I’ve been working on search stuff lately, and we needed some wordlists to help test search results that match only because they sound similar to the query, and not because they’re spelled similarly.

Turns out we couldn’t find a pre-existing wordlist of homophones (words that sound the same but are spelled differently) that are dramatically different in spelling. And our QA team especially wanted some examples of people’s names that meet those criteria.

So, sure, I figured that’d be fun and quick to throw together for them!

It’s a lot like finding anagrams - the basic structure was a dict (a hash map, for the non-Python folks reading this) keyed by the phonetic encoding of each word. Each key pointed to a nested dict, which included an array of words which phonetically matched the key and a bool indicating whether it fit my criteria or not. In the end, all matching words were spit into stdout as a list of comma-separated homophones.

I determined whether words were spelled differently enough by checking whether a small enough percentage of their trigrams were the same. (I also had a minumum length set, so I’d be sure to have enough trigrams per word to be worth checking for match percentage.

(It was kinda neat to find something that felt more like an interview puzzle than anything else, but was actually useful for my day job. Oh hey, look, those skills are occasionally actually useful! Now you don’t have to feel weird about all the time you spent learning how to solve these sorts of puzzles!)

Sweet and simple and fun! Here’s my script and a few of the wordlists I created with it, since I figure other people may also find this sort of thing useful when testing search implementations. (FYI, if you’re using something other than a metaphone/doublemetaphone soundalike algorithm and trigrams for misspellings, you may want to make some adjustments.)

24 Apr 2014
My new favorite vim/tmux bug

This week, I’m grateful that my coworkers know to come grab me if something seriously weird is going on, because it fills me with so much glee! I mean, WHAT.

minimal repro:
On Suffolk (one of our machines), open tmux, open vim, open new terminal tab.
Vim gets “lililililililill” inserted in current file, and beeps a lot
If the file already has content, it prepends i and appends ll to ~10 lines, and sometimes capitalizes something


I’m going to skim over some of the details so that this remains a blog post and not an endless excited ramble, but! This is approximately how figuring this nonsense out went!

Initial poking around

When does the problem happen? When you open a new bash tab or window, or enter any command in any bash session.

“lililililililill” looks very suspicious. Is that a macro or something hiding in one of the vim registers? Use :reg to check the contents of the vim registers - nope, nothing fishy in there!

Is there anything funky in our tmux config? ~/.tmux.conf doesn’t exist, and a quick googling around didn’t turn up anything on any other standard sorts of tmux config files. Fair enough, put that aside for the moment.

Poked around a bit more to define the edges of the problem:

  • Happens both in iterm and terminal
  • Happens after restarting vim, restarting tmux, reinstalling tmux, reimaging Suffolk
  • Does not happen in vim/macvim outside of tmux.
  • Does not happen in any other of the handful of machines that were checked.
  • Does not happen in vim in tmux when ssh’d into another machine.

(One exception to that last one - a coworker said he was able to replicate it when ssh’d into a remote coworker’s machine. But when we tried to replicate that, it didn’t happen. An isolated datum, potentially relevant, but highly suspect. To this day, I’m pretty convinced that folks got mixed up and it never really happened in the first place - happens to the best of us, and it doesn’t fit with any of the other evidence.)

Cool, we got the lay of the land. So! What changed recently?

Ah, this machine was newly reimaged. Maybe we have new broken or incompatible versions of some things?

We were told that the tmux version should be frozen as part of our install script. Do you believe everything you’re told? No? Good! You guessed it, we had totally different versions of vim, tmux, bash, and OS X on this machine than on other machines which do not exhibit the same problem.

Around this time I started up a Google doc to keep track of everything we were trying, because once things start to look complicated I know I won’t be able to remember everything I’ve tried. Especially when multiple people are involved! And it’s a horrible waste of time to repeat experiments out of forgetfulness, or even worse, lose potentially relevant data. I won’t bore you with a full list of versions and reinstallation steps, but boy do I have all the details in my notes.

Point being, we downgraded bash, tmux, and vim to match the versions working on other machines, but the problem remained.

At this point, I was sadly told that the machine was just going to get a bunch of stuff reinstalled and I shouldn’t spend any more time poking at it. Sadness! But okay, fair enough, it was getting in people’s way and the show must go on.

But wait! Things don’t magically solve themselves after all!

Imagine my delight when I came in the next morning and heard that the reinstalling stuff hadn’t fixed the problem! I’d been super bummed the day before to have my mystery stolen away from me, so this was very exciting! I hung out with a coworker for a bit to give him pointers on how to look into Elasticsearch bugs, then ran off to the biggest mystery of the week.

Aha! We noticed that we have a tmux-related vim plugin in our vim config - tmux-config. Bonus points for anyone who feels like stopping here to look at that and guess how this story ends. ^___^

I didn’t have much time to play with it in the moment, but the very best thing happened - we were able to replicate it on any machine recently reimaged with our new workstation setup script! This meant I was able to get the bug onto my laptop! AW YEAH.

Wheeeeee I got to take my bug home with me to play with!

I sat down to take a closer look at that tmux-config plugin.

~/.vim/bundle/tmux-config/tmux-autowrite/autowrite-vim.sh creates a preexec function that’s called whenever you start up a new bash session, or enter any command in bash.

Line 31 reads:

  tmux send-keys -t $pane ^\\\ ^n F19 WriteAll

Commenting out that line causes the problem to go away. Running tmux send-keys -t %0 ^\\ ^n F19 WriteAll manually in another bash window causes the bug to manifest regardless. Perfect! What is this thing trying to do, and what is it actually doing?

Ah! In that same plugin, ~/.vim/bundle/tmux-config/plugin/autowrite.vim:45 defines this relevant mapping:

  map <silent> <F19>WriteAll :silent! wall<CR>

My notes from the moment my jaw dropped after one look at that:

I’m not sure what ^\\ and ^n are supposed to do - they don’t seem to be doing anything.

The rest is a mapping set in autowrite.vim:45 to save vim buffers when you do other stuff in the terminal, basically trying to mimic the way we have macvim set up to save on blur.

I’m not sure why yet, but F19 is what toggles capitalization and makes the damn beep. It only does the capitalization in vim inside tmux, not in vim outside tmux.

And then ‘WriteAll’ is interpreted as a normal vim command -

  • ri replace the character under the cursor with an i
  • e takes you to the end of a word
  • A takes you to the end of the line and puts you into insert mode
  • then ll is inserted at the end of the line

Not sure why the preexec function gets run multiple times with each bash session/command, but that’s what must be happening!


This is around the time I settled in to perform a series of experiments.

What happens if I send F19 alone with tmux send-keys?


  tmux send-keys -t %0 F19


BEEP BEEP BEEP &c, and if the file open in vim has any contents, the next <= 3 characters get their case toggled.

Huh, case gets toggled. Not just capitalized. Toggled. That’s interesting.

What happens if I hit F19 in vim outside of tmux?

Aw, hell, my MacBook doesn’t even have an F19 key! Yeargh. Fine, whatever, I went and installed KeyRemap4MacBook so I could remap fn-fn to F19 to test stuff with.

Result: No beeping or case toggling.

Why does F19 cause beeping/case toggling in vim inside tmux but not in vim outside tmux?

Am I super confident that my mapping worked properly? I mean, I tested it with EventViewer, but how realistic is that? Does tmux send-keys somehow send something different than what my mapping thinks I’m sending now?

How else can I test that F19 is what it claims to be?

I did some googling around, and learned that you can actually check how keystrokes are encoded in bash by opening up your terminal, hitting control-v, then hitting a key.

Whoa, neat, that seems useful! I checked encodings to see if I could find a difference, and oho, that jumped out at me!

  • Inside tmux, F19 is encoded as ^[[33~
  • (in our bash outside tmux, it’s ^[[18;2~ instead, dunno why)

HOLD ON. Look at that more closely: inside tmux, F19’s encoding ends in ‘3~’, which is exactly the command in vim that you’d expect to toggle case for 3 characters - COINCIDENCE? I THINK NOT.

Wait a second. 3~ looks super familiar for another reason! Oh, right, I’d noticed earlier that ~/.vim/bundle/tmux-config/plugin/autowrite.vim:35 set up some function keys like so:

  if &term == "screen-256color"
    set t_F3=^[[25~
    set t_F4=^[[26~
    set t_F5=^[[28~
    set t_F6=^[[29~
    set t_F7=^[[31~
    set t_F8=^[[32~
    set t_F9=^[[33~

My eyes had skimmed over that bit earlier, because it looked like it only went up to F9. I didn’t bother to verify that assumption, just moved right past it. DAMNIT. Time to search the vim docs!

OH HEY t_F9 refers to F19 in vim ARGH ARGH ARGH HOW DID I MISS THAT.

So, that’s inside a conditional. It doesn’t always happen. What’s &term? Well, it’s whatever $TERM is in bash. Okay, let’s verify that!

Inside tmux, $TERM was set to ‘screen’. Outside tmux, $TERM was set to ‘xterm-256color’.

256color… oh hell.

Some googling around turned up this useful answer on setting up tmux to handle xterm-style function key inputs. Setting that option did in fact make the bug go away! But that option wasn’t set on any of our other computers, and we have all these other things hinting at a different solution.

Time to search that tmux-config plugin for screen-256color to see where it comes up OH GODDAMNIT ~/.vim/bundle/tmux-config/plugin/tmux.conf:9 sets:

  set-option -g default-terminal "screen-256color"

With that option set, the conditional in autowrite.vim is satisfied, and (when vim is restarted after that option being set and all the vim plugins are sourced) t_F9 (which is secretly F19) is mapped to [33~.


To sum up

(1) tmux wasn’t set to handle xterm-style function key inputs, because our tmux.conf wasn’t actually being copied

  • from: ~/.vim/bundle/tmux-config/tmux-autowrite/tmux.conf
  • to: ~/.tmux.conf

(2) THEREFORE, tmux hadn’t received this config from our tmux.conf:

    set-option -g default-terminal "screen-256color"

(3) SO, $TERM inside tmux was “screen” and outside tmux was “xterm-256color”

(4) This means tmux wasn’t set to handle xterm-style function keys (such as F19). This isn’t super-clear, to be fair. The clear way to set tmux to receive xterm function keys properly would be with “setw -g xterm-keys on”

(5) Vim checks $TERM to see if function keys are available. See the tmux FAQ. If they’re not, the character codes sent by the function keys are interpreted literally.

(6) We actually have vim set to interpret the higher function keys explicitly in autowrite.vim:35 - if $TERM is “screen-256color” (which happens explicitly in that tmux.conf we weren’t using) then t_F9 (which is F19) is set to ^[[33~

(7) Why? Because (as I verified with control-v) inside tmux, F19 is encoded as ^[[33~

(8) Since we never explicitly set it otherwise, $TERM inside tmux was set to “screen” - which means that the condition in our autowrite.vim:35 was never met, and thus t_F9 was never set to ^[[33~ in vim.

(9) Because t_F9 was never mapped properly in our vim config, when that preexec function ran and bash sent “^\\ ^n F19 WriteAll” to tmux via tmux send-keys, vim escaped into normal mode because of ^\\ ^n and then interpreted the rest literally as ^[[33~WriteAll.

(10) And because the literal string ^[[33~WriteAll wasn’t mapped in vim (only <F19>WriteAll was!), each character was interpreted as a separate vim command, not part of a single mapping as intended.

^[[33~WriteAll as interpreted as a series of vim commands
  • ^[ is escape
  • [3 doesn’t do anything (as far as I can tell)
  • 3~ toggles case for the next three characters
  • W takes you to the start of the next WORD
  • ri replace the character under the cursor with an i
  • te takes you to just before the next e
  • A takes you to the end of the line and puts you into insert mode, and then
  • ll is inserted at the end of the line

Long story short, the fix was:

  ln -s ~/.vim/bundle/tmux-config/tmux.conf ~/.tmux.conf

Process-related takeaways

Absence of evidence IS evidence of absence - we noticed pretty early on that there was no ~/.tmux.conf, then moved on, figuring that okay, guess there isn’t anything weird in the config. Next time, if something is missing that seems like a likely place to look, I want to think of looking at whether analogous config files exist on working machines to compare sooner.

Verify ALL assumptions sooner (or at least the easy-to-check ones) - I noticed that t_F9 thing way earlier and skimmed past it, assuming that surely t_F9 referred to F9. That’s an assumption that would’ve been super quick to verify! Gotta verify assumptions as they’re made, especially ones that are quick and easy to check out.

Edit: And via the great discussion of this post on Hacker News: “Every bug in existence is a story of different software components doing exactly what they were told to.” (Unless you count cosmic ray bugs, natch.)