- Gianfranco's Newsletter
- Posts
- Gianfranco's Best of May 2024 Reading List
Gianfranco's Best of May 2024 Reading List
The top essays on AI, finance, hyperscalers, semiconductors, and more in this curated May 2024.
Gianfranco Filice
June 06, 2024
Welcome to the May 2024 edition of my monthly reading list.
This month, I've curated my favorite essays across various categories: Artificial Intelligence, Finance and Economics, Hyperscalers, Life and Meaning, Semiconductors, Life and Technology, and Crypto/Web3.
Artificial Intelligence
The Economics of Generative AI - Apoorv Agrawal
Excerpt: “Bottom line: semis layer has captured ~88% of all gross profits in the Gen AI ecosystem (vs 5% for semis in the cloud stack).”
The Evolving Landscape of LLM Evaluation - Sebastian Ruder
Excerpt: “Most popular benchmarks are either directly available on the web or may have been uploaded in different forms on GitHub or other platforms. Current models are trained on much of the Internet, with newer models trained on more recent snapshots of CommonCrawl. Unless filtering measures are taken to specifically remove benchmark data from pre-training, models are invariably exposed to test data through their pre-training. While they may not memorize every example, training on the data makes it more likely for the model to produce the correct prediction.”
How Good Are the Latest Open LLMs? And Is DPO Better Than PPO? - Sebastian Raschka, PhD
Excerpt: “This is a very interesting finding because, as the Llama 3 blog post notes, according to the Chinchilla scaling laws, the optimal amount of training data for an 8 billion parameter model is much smaller, approximately 200 billion tokens. Moreover, the authors of Llama 3 observed that both the 8 billion and 70 billion parameter models demonstrated log-linear improvements even at the 15 trillion scale. This suggests that we (that is, researchers in general) could further enhance the model with more training data beyond 15 trillion tokens.”
The future of foundation models is closed-source - John Luttig
Excerpt: “Frontier models were trained on the corpus of the internet, but that data source is a commodity – model differentiation over the next decade will come from proprietary data, both via model usage and private data sources. Open-source models have no feedback loop between production usage and model training, so they foot the bill for all incremental training data, whereas closed-source models drive compounding value with data from incremental usage. If Meta differentiates their model based on their social graph or user feedback, they’ll want to capture that value via their closed products, and not share it with the world.”
Finance and Economics
Should Schools Teach Financial Literacy? - Capital Gains
Excerpt: “When I was in middle school, I wondered whether I'd ever care about the definition of "mitochondria," but it actually turned out to be useful information: now I can more easily understand jokes about how school teaches us lots of things that won't have direct practical applications to our lives, and doesn't tell us much about how to do taxes, whether or not to consider an adjustable-rate mortgage, and the advantages and disadvantages of a Roth IRA.”
America’s fiscal outlook is disastrous, but forgotten - The Economist
Excerpt: “The IMF reckons that loose fiscal policy has become a bigger contributor to its stubbornly high inflation than other factors affecting demand and supply. And inflation is, of course, the reason why the Fed has maintained elevated interest rates, thereby raising the government’s financing costs. Back in January markets expected about 1.5 percentage points in rate cuts this year. If those were to materialise, the Treasury would end up paying about $1.2trn in interest during 2024, according to analysts at the Bank of America. If, however, the Fed holds rates at their current level, as many investors now anticipate, the Treasury’s interest payments will instead reach $1.6trn. The difference—$400bn—adds more than a percentage point to the federal deficit.”
Is Japan having a currency crisis? - Noahpinion
Excerpt: “What really matters is the amount of debt relative to Japan’s power to tax its economy. When interest rates go up, the government has to pay higher interest costs. When your debt is over 130% of GDP, that is a big budgetary strain. If Japan can’t raise taxes to pay those costs, it will have to borrow yet more just to keep paying interest — and if it does that, things can get very bad very quickly.”
Moonshots or Buybacks - Capital Gains
Excerpt: “But there are other reasons to invest in pure R&D without direct commercial applications. The skills involved in the pure research side often have some useful spillovers to the more commercial development that turns an idea into something with a price tag. So investing in R&D is a way to buy a sort of talent buffer: when business is good, researchers can focus on things that won't be practical for a decade, but when the business is threatened, there's a talent bench that can be redirected towards the projects that matter more in the next quarter or two.”
The business of wallets - Bits about Money
Excerpt: “There are classically two ways to make money in financial product innovation: you charge the customer for stocks, an ongoing and often percentage-based fee to custody their assets for an arbitrarily long time, or you charge them for flows, a per-instance and sometimes scaling-with-size fee on transactions specifically. Very frequently, from the perspective of a single user, stocks are priced and flows are free, or vice versa. In the basic bank account, stocks are priced (via the interest rate spread on deposits) but transactions are free or close to it. In credit card processing for businesses, stocks are free-ish but flows (incoming transactions from customers) are priced.”
The Impact of Debt - Oaktree Capital
Excerpt: “Not just market volatility, but . . . world and life volatility: recessions, wars, divorces, illness, moves, floods, changes of heart, etc.” With no debt, he postulates, we’re likely to survive all but the most infrequent, most volatile events. But in a succession of illustrations, Housel shows that as the level of one’s indebtedness increases, the range of volatility one can withstand narrows, until at a very high level of debt, only the tamest of environments are survivable. As Housel puts it, “ as debt increases, you narrow the range of outcomes you can endure in life.”
Hyperscalers
CEO Andy Jassy’s 2023 Letter to Shareholders - Amazon
Excerpt: [On Amazon’s Approach to Primitives] “As AWS unveiled these building blocks over time (we now have over 240 at builders’ disposal—meaningfully more than any other provider), whole companies sprang up quickly on top of AWS (e.g. Airbnb, Dropbox, Instagram, Pinterest, Stripe, etc.), industries reinvented themselves on AWS (e.g. streaming with Netflix, Disney+, Hulu, Max, Fox, Paramount), and even critical government agencies switched to AWS (e.g. CIA, along with several other U.S. Intelligence agencies). But, one of the lesser-recognized beneficiaries was Amazon’s own consumer businesses, which innovated at dramatic speed across retail, advertising, devices (e.g. Alexa and Fire TV), Prime Video and Music, Amazon Go, Drones, and many other endeavors by leveraging the speed with which AWS let them build. Primitives, done well, rapidly accelerate builders’ ability to innovate.”
Big Tech Capex and Earnings Quality - Base Hit Investing
Excerpt: “Think about how much profit needs to be generated annually to earn acceptable returns on this capex: a 10% return would require $15 billion of additional after tax profits in year 1. As Buffett points out, if you require a 10% return on a $150 billion investment but get nothing in year 1, then you’d need $32 billion in year 2, and just one more year of deferred returns would require a massive $50 billion profit in year 3.”
Life and Meaning
How to Find a Partner like Charlie Munger - Alchemy
Excerpt: “Say yes to the dinner. Buffett and Munger met when the latter was in Omaha for his father’s funeral. Imagine if Buffett had declined to go out because he’d rather … read another annual report, watch TV, or simply because he didn’t feel like being social that night. You have to make space in your life for meeting new people.”
Semiconductors
America's semiconductor policy is missing a key piece - Noahpinion
Excerpt: “Smaller node sizes correspond to higher performance. The most modern GPUs and CPUs, which often feature in news stories, require quite small node sizes. That being said, the majority of the chips which allow for modern technologies are higher node sizes and less computationally intensive.”
How to Build a $20 Billion Semiconductor Fab - Construction Physics
Excerpt: “A modern microchip has features on the order of 50 nanometers in width, or around 1/2000th the width of a human hair.1 Materials are placed in layers a few atoms thin. Creating objects this small requires ultra-precise manufacturing equipment, and a production environment that can screen out as many sources of interference as possible; every rogue speck of dust or tiny fluctuation in electrical voltage. And these conditions must be maintained not in the rarefied conditions of an experimental lab, but in a mass production facility that is producing hundreds of millions of microchips every year. The combination of ultra-precision and high-volume production results in some of the most complex, expensive factories in the world.”
Science and Technology
The death (again) of the internet as we know it - Noahpinion
Excerpt: “The internet as we know it has already died once. In the 2010s, the rise of smartphones and mass social media (Twitter/Facebook/Instagram) caused what internet veterans refer to as an Eternal September event, for the entire internet. “Eternal September” is an old slang term for when a bunch of normal folks flood into a previously cozy, boutique online space. When the average person got a high-speed pocket computer that linked them 24/7 to the world of social media, the internet ceased to be the domain of weirdos and hobbyists, and became the town square for our entire society.”
2023 Annual Letter - Chamath Palihapitiya
Excerpt: “While I believe it is premature to declare the winners, as of today, I see two clear areas of value creation over the long term. The first is proprietary data. While we always talked about data as being the “new oil” over the past decade, we are finally starting to see how it can be monetized to help enhance the capabilities of AI models. This trend is exemplified by the recent partnership between Reddit and Google, where access to Reddit’s vast, diverse dataset will be leveraged to train Google’s Gemini model to be more sophisticated and context-aware.”
Crypto / Web3
New Launches - Part 1 - Cobie
Excerpt: “This “phantom market” that happens pre-token is a delusion. It is not discovering the natural price based on the dynamics of supply and demand. It is simply finding the absolute highest price that a VC investor is willing to pay. This dynamic pushes valuations to prices that the market is unable to bear, evidenced by the graveyard of tokens from 2020-2022 that are trading significantly below their private market valuations.”
GG - Cryptotainment Era - Wrongalot
Excerpt: “When you don’t know what to build - you build more tech. If you don’t know what to do with money - you do finance. If you are bored - you browse memes on the internet. Crypto does it all in one escapist movement.”
One Interesting Chart:
Please reply to this email if you found this insightful or want to see short summaries for each of these articles, papers, and links in the future.
Reply