Bare Fardel

Now That The Twitter API Will Be Extortion-Level Expensive...

01 March 2023

Last summer, I got fed up with the atrocious way in which Twitter displays conversations (To wit: It Doesn't), and spent a few weeks going to a coffee shop before work to write my own vastly superior Twitter app. A few weeks ago, Twitter announced the free tier API will be discontinued, which my app depends on. I apparently will have to pay $100/month for API access. This an insane price. I am not paying it, so my app will soon cease to function. Here is what the app was and how it worked, so that you too can know the joy of what Twitter could be, if you spent 40 hours of time in a coffee shop to write your own Twitter app (which some might (understandably) view as a form of insanity).

Here's The App

What Are You (The Reader) Looking At Here, Exactly?
You are looking a list of every conversation from all of the people I have followed. This means every tweet they made, and every reply, traced back to the "original tweet" to give the full conversation context.

Why Did I (The Writer) Build This, Exactly?

There are a few people on Twitter who say interesting things, or reply to other interesting things, and give me new things to think about that I wouldn't have otherwise maybe ever thought about. I like to see these things.

I also of course follow a few accounts whose tweets are socially corrosive, ill-advised, and generally rude, because it's fun (in a soul-corroding sort of way) to watch one of Your People say something destructively clever about Those Other Idiots. But let's forget about that, because it's embarrassing to admit I enjoy it.

I encounter a problem here. I want to see all of the conversations from everyone I follow. The default Twitter timeline does not show you these conversations. The default timeline is an algorithmic timeline which contains a mania-inducing array of ads, tweets I didn't ask to see from "influencers" I've never heard of, and maybe a few tweets that got "good engagement" from that the people I DID ask to follow (but none of their replies!). If I go into my settings and switch my timeline to the "Chronological timeline", I get a slightly better experience: I at least now now see the Tweets that my Followees posted themselves, in the order in which they posted them, but I still won't see their replies to conversations. This is not satisfactory, because they may have replied to a really interesting conversation full of thoughts that I otherwise wouldn't have ever thought about.

An example:

Fantastic question & reply, and despite the fact that I follow Andy, none of these tweets will show up on the Twitter home page, no matter if your timeline is algorithmic or chronological.

This is an example with just 3 tweets, but I follow quite a few programmers, some of whom have decided with all the powers of their radiant intellects to use Twitter, the most limited and nuance-free discussion platform ever created, to engage in complicated technical discussions, and so it is extremely common to have 2+ cranky and pedantic bastards talking past each other for 20 or 30 tweet-replies. And by default you can't see these replies. Fucking preposterous! I'm here to see the pedants at play in the fields of the Lord (Elon), so why aren't they shown to me?

Eventually, this state of affairs became too much for me, and I decided to build the Twitter UI that I want to use. The UI that by all rights, should exist. A UI that lets me quickly and easily browse all of the conversations between everyone I follow: the deranged & schizophrenic shitposters, the ludicrously pedantic technophiles, and of course, the Interesting People.

For The Curious, A Couple of Drawings

The "holy moly maybe something better is possible" moment came when I realized that you can think of every conversation on Twitter as a tree.

Here are some diagrams of Twitter conversations display as trees. Imagine all of the circles in the diagram below are tweets. Each grey box is a collection of all the conversations stemming from some initial tweet ( The topmost circle in each section). The ones below are replies, each replying to the one that its arrow points to. Purple circles are tweets from someone that you follow. All other colors, you don't follow. Since I am interested in the conversations exclusively from the people I follow, what my app does is display every unique path in these conversation-trees that ends in a purple circle. A green line is drawn around each of these conversations. When I click Next/Previous in my app, I am cycling between each of these green-circled paths.

Fetching this data from Twitter & assembling these unique conversation trees was a fun puzzle. It was made easier by the fact that trees are a form of a Graph, which is a data structure used to think about many mathematical problems, and so pretty much any question you might want to ask about a graph (How do I get every unique path that ends in a purple circle?), some mathematician figured out how to answer 50+ years ago. Though I doubt they had my particular use-case in mind while figuring out their theorems.

There are many limitations in the app: I never bothered to get retweets or quote-retweets working, only tweets & replies. Images show up, but videos have to be viewed on the Twitter website. There is also the potential to recursively fetch tweets until the heat-death of the universe, if someone popular tweeted something like "Here's the list of trending things that I pretend to do for my morning routine, post yours!" (These threads are the 2020s equivalent of the cursed email chains of the early 2000s: "Forward this to everyone you know or the corporealized ghost of Mother Mary will disembowel all of your aunts at midnight")

There are also things missing that that I view as enhancements: No like/comment/retweet/view counters, no wasted space for the reply box (I don't ever post/reply), no sponsored content, no trending content. Nothing superfluous to the core conversation-reading experience.

So, Where's The Code?

My app violates every single term of the Twitter API Display Policy. Thus, I've not made the code public. This software is only for me. Or will be, until it ceases to live.

From Mad Max (2015)