Gianfranco's Best of August 2024 Reading List

Welcome to the August 2024 edition of my monthly reading list.  

This month, I've curated my favorite essays offering insights into the latest developments in AI, societal impacts of technology, strategic business thinking, financial trends, and personal growth.

If you only have a few minutes, these three posts were my favorite, and are included in the list below:

The AI Supply Chain Tug of War (David Cahn/Sequoia Capital)

• Cahn exposes the game of "hot potato" being played with risk in the AI supply chain. While everyone wants a piece of the AI pie, no one wants to be left holding the bag if/when demand evaporates. This has led to Big Tech becoming the reluctant guarantor of the entire system, absorbing massive risk through aggressive capital expenditure and propping up demand. The question is, how long can this unsustainable game continue before someone gets burned?

Building A Generative AI Platform (Chip Huyen)

• Chip argues that effective AI application development necessitates augmenting models with external context, implementing robust guardrails, and strategically employing routers, gateways, and caching mechanisms. This deep dive exposes the intricate workings of AI applications, emphasizing the critical need for developers to move beyond basic model deployment and embrace a holistic engineering approach for reliable and scalable AI solutions.

The big stack game of LLM Poker (Sarah Tavel)

• Tavel identifies the game of chicken being played by tech giants. Fueled by the potential of increasingly sophisticated AI and the existential threat of falling behind, these companies are in a race to the bottom. Colossal spending on rapidly outdated models creates a temporary, but fertile period for AI development. The question is, how long until this unsustainable burn rate leads to a shakeout, and who will be left standing?

Artificial Intelligence

• The AI Supply Chain Tug of War - Sequoia Capital
  • Excerpt: "They absorb risk from their downstream partners Nvidia and TSMC through large orders that generate huge short-term profits for these companies. They also absorb risk from upstream partners: The cloud companies are the largest source of funding for frontier model companies and they subsidize end customers in the form of low API prices and bundled credits."
  • Read More
• A Deep Dive on AI Inference Startups - Eastwind Substack
  • Excerpt: "How much revenue is really up for grabs? If we look at the PaaS market as an example, the entire category likely generates under $2B in revenue (Heroku was in the ~$500M ARR range as of 2022, and current PaaS darling Vercel is at the $100M ARR range). It stands to reason that the aggregate current-day revenues for generative AI abstraction are well under $1B (given PaaS encapsulates so many different services)."
  • Read More
• Building A Generative AI Platform - Huyen Chip
  • Excerpt: "A model gateway is an intermediate layer that allows your organization to interface with different models in a unified and secure manner. The most basic functionality of a model gateway is to enable developers to access different models – be it self-hosted models or models behind commercial APIs such as OpenAI or Google – the same way. A model gateway makes it easier to maintain your code. If a model API changes, you only need to update the model gateway instead of having to update all applications that use this model API."
  • Read More
• Adversarial Attacks on LLMs - Lilian Weng
  • Excerpt: "The use of large language models in the real world has strongly accelerated by the launch of ChatGPT. We (including my team at OpenAI, shoutout to them) have invested a lot of effort to build default safe behavior into the model during the alignment process (e.g. via RLHF). However, adversarial attacks or jailbreak prompts could potentially trigger the model to output something undesired."
  • Read More
• Evolution of Databases in the World of AI Apps - Chips Ahoy Capital
  • Excerpt: "The key here is that investors seem to have forgotten that MDB didn’t win because they had the best document database, but because of the developer ecosystem, capabilities built around it, ease of use, and vendor integrations. The same should apply to Gen AI native apps, vendors aren’t going to win just because they have the best vector search capability."
  • Read More
• Open-endedness is all we’ll need [Agentic AI] - Air Street Press
  • Excerpt: "Integrating new general purpose technologies takes time. There was a 40 year gap between the creation of the first central power stations in the 1880s and the productivity impact of electrification beginning to appear in statistics. This was because it took time to replace or retrofit existing infrastructure, develop new practices and processes, and manage the overlap between old and new systems."
  • Read More
• The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery - Sakana AI
  • Excerpt: "Tokyo-based frontier lab Sakana AI, recently published a paper on AI Scientist, an end-to-end framework designed to automate the generation of research ideas, implementation, and the production of research papers. This is a creatively and highly compute-efficient way of potentially creating new research (for as little as $15 a paper)."
  • Read More
• The big stack game of LLM poker - Sarah Tavel
  • Excerpt: "Not surprisingly, when GPT3.5 launched in November 2022 it was head and shoulders ahead of any competitive model and cost $0.0200 for 1000 tokens. It's $0.0005 now – 2.5% of its original pricing in just 1.5 years. I can’t remember another technology that has commoditized as quickly as LLMs. It’s a dynamic that makes it almost impossible to rationalize any ROI at this stage in the game because any investment in a LLM is almost instantly depreciated by the next version."
  • Read More
• New LLM Pre-training and Post-training Paradigms - Sebastian Raschka
  • Excerpt: "The blog post explores new strategies for training and refining large language models to improve efficiency and performance, particularly in practical business applications."
  • Read More
• The ROI of AI (It's a Dollar Auction) - Fabricated Knowledge
  • Excerpt: "[Mark Zuckerberg] “I think bubbles are interesting because a lot of the bubbles ended up being things that were very valuable over time and it's just more of a question of timing, like you're asking, right? Even the dot com bubble, you know, it's like there's all this fiber laid and it ended up being super valuable, but it just wasn't as valuable as quickly as people thought.”"
  • Read More

Technology and Society

• Why the CrowdStrike bug hit banks hard - Bits About Money
  • Excerpt: "CrowdStrike maintains that they do not understand it to be the case that a bad actor intentionally tried to bring down global financial infrastructure and airlines by using them as a weapon. No, CrowdStrike did that themselves, on accident, of their own volition. But this demonstrates the problem pretty clearly: if a junior employee tripping over a power cord at your company brings down computers worldwide, the bad guys have a variety of options for achieving directionally similar aims by attacking directionally similar power cords."
  • Read More
• Fast - Patrick Collison
  • Excerpt: "Apollo 8. On August 9 1968, NASA decided that Apollo 8 should go to the moon. It launched on December 21 1968, 134 days later."
  • Read More
• Herd Mentality – Why are there so many AI accelerators? - Digits to Dollars
  • Excerpt: "It is noticeable that many of the very large AI accelerator fund raises that have taken place this year have been led by non-traditional venture investors. The best known firms all got their fingers burned by the first TPU-era wave, and have been mostly absent from the latest deals. Probably the best example of this is Sequoia which has published two pieces cautioning against the AI hype. Those notes are in part meant for Sequoia’s own LPs, explaining why the firm is not investing here."
  • Read More
• The Game Theory of AI CapEx - Sequoia Capital
  • Excerpt: "For example, once everyone has 100k clusters, big tech companies will need to figure out what to do with their 50k and 25k clusters. We’ve heard a few industry experts make comments along the lines of: No one will ever train a frontier model on the same data center twice—by the time the model has been trained, the GPUs will have become outdated, and frontier cluster sizes will have grown."
  • Read More

Business Thinking

• Data is Just an Added Sense - Commoncog
  • Excerpt: "You don’t decide to go on a hike blindfolded if you are naturally sighted; you are doing yourself a disservice if you do so. With humans, the impairment of a sense has a specific connotation: it leads us to say that a person is ‘disabled’. In business, however, the loss of a sense is sometimes used as a badge of honour, such as when LinkedIn influencers crow that being ‘data-driven’ is bad, and that they’re better off being ‘data-informed’ instead."
  • Read More
• Things I learned 40 years ago about strategy, layoffs, capital intensity - Tech Can Be Better
  • Excerpt: "He liked to say “Your strategy is what you ship”, putting the emphasis on what you actually can get done repeatedly. You can have all kinds of grand dreams and stratagems, but your customer sees what you actually ship, and they make their purchase and usage decisions based on that. It is easy to fool yourself about your strategy; it is easy to lose sight of what your customer is actually experiencing."
  • Read More

Finance and Economics

• Mr. Market Miscalculates - Oaktree Capital
  • Excerpt: "But instead, we see that an optimistic market is capable of ignoring individual pieces of bad news until a critical mass of bad news builds up, at which time a tipping point is reached, the optimists surrender, and a rout begins. Rudiger Dornbush’s great quote about economics is highly applicable here: “. . . things take longer to happen than you think they will, and then they happen faster than you thought they could.” Or as my partner Sheldon Stone says, “The air goes out of the balloon much faster than it went in.”"
  • Read More
• When To Stop Burning Cash - Capital Gains (The Diff)
  • Excerpt: "It's long been the case that companies consume more capital than they produce early on. If that weren't true, you would expect the financial system in general to be a lot smaller, since everyone would fund growth from operations. That's a world where fewer people spend all day, or all their life, looking at spreadsheets. But it's also a world where economic activity is more weighted towards mature companies, instead of emerging startups."
  • Read More
• Understanding Elasticity - Capital Gains (The Diff)
  • Excerpt: "If oil illustrates one extreme, with highly inelastic supply and demand in the short run, digital goods are on the other side: the supply elasticity is almost perfect, and it barely makes sense to talk about the supply curve for something like a tweet or a Spotify play—the marginal cost is close to zero, so it almost always makes sense to deliver an incremental unit of the product. At least, that’s true as long as the cost of serving a page means looking up and displaying static content, rather than running inference."
  • Read More
• CardsFTW #117: Interest - CardsFTW
  • Excerpt: "But what if you do not pay in full, even if you pay on time? Interest will start accruing on your purchases. Not only will interest accrue on your purchases that you didn't pay in full, but it will also accrue on new purchases with no grace period. This means that as soon as you make the purchase, interest will start accruing."
  • Read More
• Is Land-Use Regulation Holding Back Construction Productivity? - Construction Physics
  • Excerpt: "Stricter land-use regulations force builders to spread their efforts over a large number of relatively small projects, limiting the number of homes they’re able to build. This, in turn, limits their ability to invest in better homebuilding technology or otherwise take advantage of economies of scale."
  • Read More
• A Model Proposal for Fintech Account Insurance - CardsFTW
  • Excerpt: "FDIC insurance only applies to the underlying bank failing, not to the fintech program’s failure. For all practical purposes, there is no insurance for a fintech program. If a major fintech debit card program went bankrupt tomorrow and we lost access to the ledger, those users might also be out of their funds."
  • Read More

Life

• Learning to Give a Damn - Altos Venture Capital Blog
  • Excerpt: "If a soldier does not care to clean up the truck when he reports back in, then he probably did not bother refueling. Furthermore, there is a high probability that he is not taking care of the truck every day.” At that moment, he grew very serious, and said, “You have to care. You have to make sure your soldiers care. Otherwise, you and I will be explaining to the soldier’s loved ones why his truck ran out of fuel, did not reach the destination, and got killed by enemy fire."
  • Read More
• 0152 – Letter To A Young Songwriter - Visakan Veerasamy
  • Excerpt: "You might think that you ought to try to write the best song that you possibly can, but you’d be wrong. That’s actually a trap that you should try your hardest to avoid, especially when you’re just starting out. Why? Because your concept of ‘best’ is a work-in-progress and a moving target. Why spend a year working on writing the best song you can, when you could instead spend a year improving your idea of what makes a song great?"
  • Read More

Podcast

• Invest Like the Best - Gavin Baker - AI, Semiconductors, and The Robotic Frontier - Patrick O'Shaughnessy
  • Listen here

Interesting Charts: