QCon London: Meta software engineer Zahan Malkani, speaking at QCon London, said that Threads was shipped in just five months in order to benefit from Twitter disillusionment. The plan succeeded but the team now faces heavy technical debt as a result, and is gradually removing the Instagram code.
QCon London is a vendor-neutral developer conference with approaching 1300 attendees from across Europe.
“It was January last year,” said Malkani. “As many people abandoned Twitter, they were looking for somewhere else to hang out … we had a fleeting window of opportunity that could disappear as quickly as it appeared.”
That was a software development challenge, and according to Malkani, “every shortcut was on the table.” The team set out four milestones, but time to market was so critical that each milestone was designed to be “a possible end state itself, where we could ship if needed,” said Malkani. The first milestone was simply “being able to log in to make a text post that gets associated with your account.” The actual product shipped was something like milestone 3.5, he said. “We never got anywhere near M4.”
The team promised the leadership that something would be ready by the summer, the trick being that “we had no intention of building this from scratch.” The reason was that Instagram, the social image sharing network, already supported most of the key features. Therefore, “we reused Instagram wholesale for the back end. It literally just is the Instagram back end with some custom functionality for Threads,” said Malkani. In the same way, the client apps for each platform were stripped-down versions of the Instagram apps. “We just reordered the layout. We put the caption on top and the media on the bottom,” he said.
Although it achieved the fast shipping goal, the approach “has major downsides,” Malkani told QCon. “You’re accumulating massive debt using a code base for something new that it wasn’t designed to serve.”
Malkani shared some of the technical details of the Instagram stack. Most of the business logic lives in a Python binary called Distillery, he said, which is “probably the largest Django deployment in the world.” This in turn talks to other Meta services to get data and links.
The demand for Threads when it launched exceeded Meta’s expectations. Just hours before it went live, Malkani recalls having to restart a database cache service in order to scale it up to handle a hundred times the capacity it was provisioned for. “A million people downloaded our app and onboarded to try it out in the first hour,” he said. This rose to 100 million in the first five days, though demand then tailed off.
What though of the technical debt? Reusing Instagram code meant “reusing it with all its edge case handling, and all its warts, right? … we have a big project underway to disentangle that,” said Malkani.
Following the launch the team had to pivot quickly to add important features, and that this required gradually detaching Threads from its base as an Instagram fork. An example is fediverse interop based on the ActivityPub protocol, which could not be done with the Instagram broadcast sharing stack because of the way Instagram integrates with the rest of Meta. “We essentially need to rewrite our business logic and back-end stack, which is ongoing,” Malkani told attendees.
A big issue is data models, Malkani said, where Threads needs different ones than Instagram; and building on a sub-optimal data model is costly to unpick. Asked whether he would have done anything differently, he said that while he did not regret reusing the Instagram code, he said that “I would take a few architectural decisions differently” and spend more time on the data model. “Maybe it would push us back a month or something, but it would have been worth it.”