특정회사 제품을 언급하고 싶지는 않은데요,
많은 커뮤니티에서 잘못된 오해를 많이 낳고 있는 부분들이 있어서 바로 잡아야 할 필요성이 크다보니 공유를 해봅니다.
바로 Groq 에서 최근 라마2 70B를 굉장히 세게 홍보를 한 부분인데요, 솔직히 말씀드려보자면, 이렇게 이해가 안되는 홍보를 하는 AI 반도체 회사들이 미국 뿐 아니라 한국에도 많습니다.. 안타까운 마음이 너무나 큽니다.. 물론 그렇다고 문제점을 공개적으로 지적하기 극히 어려워서 대부분은 그냥 내부 비밀로 평가내용을 공개하지 않는 경우가 대부분입니다 (그런데 개인/기관 투자자분들은...) 그럼에도 이번 경우는 좀 너무했고.. 왜 너무했다고 생각하는지 다른 분들이 공개를 이미 하고있기에 저도 살짝..
라마2 모델을 구동하는데, DRAM은 없고 SRAM만 있다보니 자그마치 500개가 넘는 칩을 사용해야 하는데요, (게다가 그렇게 칩을 많이 쓰고서도 batch size는...) 문제점을 열거하면 10페이지도 채울 수 있지만.. 아래에 어떤 분이 간단히 분석한 내용 공유드립니다 (100% 동의하는건 아니지만 대강 제가 왜 이 칩의 데모에 심각한 문제가 있다고 보는지 이해하실 수 있으리라 봅니다)
특정 칩의 데모나 평가를 해야할 경우, 아래 내용과 같은 분석이 (사실 이것말고도 평가해야할 내용들은 많지만) 최소한 제시가 되어야, 칩을 제대로 평가하고 싶은 생각이 듭니다.
그리고 여담이지만... 어떤 칩이 만약.. LLM에서 엔비디아를 넘어섰다..? (--> 전 그런 예는 인텔 하바나 제품정도를 제외하면 본적이 없습니다. 넘어섰다고 한다면 TCO 관점에서 모델 사이즈, batch size 등의 중요한 셋팅가정을 함께 안내하고 throughput과 latency 둘 다 보여주셔야 합니다) nvidia는 범용 GPU라서 LLM에 특화된 칩이 나와야한다? (--> nvidia만큼 LLM에 특화된 AI 반도체가 있나요? LLM에서 최강자라서 지금 nvidia 제품들이 잘 팔리고 있다고 봐야겠죠. LLM이 전부는 아니지만.. A100, H100같은 칩들은 아예 tensor core와 HBM 성능에 몰빵했습니다. 심지어 transformer 가속 기능도 있습니다. 이런 칩이 LLM에 특화된 칩입니다)
Yangqing Jia
@jiayq
Probably the first operation cost analysis of owning
@GroqInc hardware to run Llama2-70b.
First of all, let me say I am a big fan of Groq. Great performance, great potential. The below is just a showcase how challenging things might be when rivaling the industry lead, but given time I look forward to it.
1. Each Groq card has a memory of 230MB. For the LLaMA 70b model, assuming int8 quantization and completely disregarding the memory consumption for inference, the minimum number of cards needed is 305. In reality, more are needed, with reports indicating 572 cards, so we will calculate based on 572 cards.
2. The price of each Groq card is $20,000, therefore, the cost for purchasing 572 cards is $11.44 million. Of course, due to sales strategies and benefit of scale, the price per card might be much lower, but let's calculate using the list price for now.
3. For 572 cards, and the average power consumption per card of 185W, the total power consumption is 105.8kW excluding peripherals. (Note, actual consumption will be higher)
4. Currently, the average price per kW per month in data centers is about $200, meaning the annual electricity cost is 105.8 * 200 * 12 = $254,000.
5. Basically, using 4 H100 cards can achieve half the performance of Groq, meaning an 8-card H100 box is roughly equivalent in capability to the above. The nominal maximum power of an 8-card H100 is 10kW (actually about 8-9 kW), so the annual electricity cost is $24,000 or slightly lower.
6. Today, the price for an 8-card H100 box is about $300,000.
7. Therefore, if operated for three years, Groq's hardware purchase cost is $11.44 million, and operational cost is $762,000. For an 8-card H100 box, the hardware purchase cost is $300,000, and operational cost is $72,000 or slightly lower.
These are all rough numbers. Please kindly point out any outrageous errors I made above.
많은 커뮤니티에서 잘못된 오해를 많이 낳고 있는 부분들이 있어서 바로 잡아야 할 필요성이 크다보니 공유를 해봅니다.
바로 Groq 에서 최근 라마2 70B를 굉장히 세게 홍보를 한 부분인데요, 솔직히 말씀드려보자면, 이렇게 이해가 안되는 홍보를 하는 AI 반도체 회사들이 미국 뿐 아니라 한국에도 많습니다.. 안타까운 마음이 너무나 큽니다.. 물론 그렇다고 문제점을 공개적으로 지적하기 극히 어려워서 대부분은 그냥 내부 비밀로 평가내용을 공개하지 않는 경우가 대부분입니다 (그런데 개인/기관 투자자분들은...) 그럼에도 이번 경우는 좀 너무했고.. 왜 너무했다고 생각하는지 다른 분들이 공개를 이미 하고있기에 저도 살짝..
라마2 모델을 구동하는데, DRAM은 없고 SRAM만 있다보니 자그마치 500개가 넘는 칩을 사용해야 하는데요, (게다가 그렇게 칩을 많이 쓰고서도 batch size는...) 문제점을 열거하면 10페이지도 채울 수 있지만.. 아래에 어떤 분이 간단히 분석한 내용 공유드립니다 (100% 동의하는건 아니지만 대강 제가 왜 이 칩의 데모에 심각한 문제가 있다고 보는지 이해하실 수 있으리라 봅니다)
특정 칩의 데모나 평가를 해야할 경우, 아래 내용과 같은 분석이 (사실 이것말고도 평가해야할 내용들은 많지만) 최소한 제시가 되어야, 칩을 제대로 평가하고 싶은 생각이 듭니다.
그리고 여담이지만... 어떤 칩이 만약.. LLM에서 엔비디아를 넘어섰다..? (--> 전 그런 예는 인텔 하바나 제품정도를 제외하면 본적이 없습니다. 넘어섰다고 한다면 TCO 관점에서 모델 사이즈, batch size 등의 중요한 셋팅가정을 함께 안내하고 throughput과 latency 둘 다 보여주셔야 합니다) nvidia는 범용 GPU라서 LLM에 특화된 칩이 나와야한다? (--> nvidia만큼 LLM에 특화된 AI 반도체가 있나요? LLM에서 최강자라서 지금 nvidia 제품들이 잘 팔리고 있다고 봐야겠죠. LLM이 전부는 아니지만.. A100, H100같은 칩들은 아예 tensor core와 HBM 성능에 몰빵했습니다. 심지어 transformer 가속 기능도 있습니다. 이런 칩이 LLM에 특화된 칩입니다)
Yangqing Jia
@jiayq
Probably the first operation cost analysis of owning
@GroqInc hardware to run Llama2-70b.
First of all, let me say I am a big fan of Groq. Great performance, great potential. The below is just a showcase how challenging things might be when rivaling the industry lead, but given time I look forward to it.
1. Each Groq card has a memory of 230MB. For the LLaMA 70b model, assuming int8 quantization and completely disregarding the memory consumption for inference, the minimum number of cards needed is 305. In reality, more are needed, with reports indicating 572 cards, so we will calculate based on 572 cards.
2. The price of each Groq card is $20,000, therefore, the cost for purchasing 572 cards is $11.44 million. Of course, due to sales strategies and benefit of scale, the price per card might be much lower, but let's calculate using the list price for now.
3. For 572 cards, and the average power consumption per card of 185W, the total power consumption is 105.8kW excluding peripherals. (Note, actual consumption will be higher)
4. Currently, the average price per kW per month in data centers is about $200, meaning the annual electricity cost is 105.8 * 200 * 12 = $254,000.
5. Basically, using 4 H100 cards can achieve half the performance of Groq, meaning an 8-card H100 box is roughly equivalent in capability to the above. The nominal maximum power of an 8-card H100 is 10kW (actually about 8-9 kW), so the annual electricity cost is $24,000 or slightly lower.
6. Today, the price for an 8-card H100 box is about $300,000.
7. Therefore, if operated for three years, Groq's hardware purchase cost is $11.44 million, and operational cost is $762,000. For an 8-card H100 box, the hardware purchase cost is $300,000, and operational cost is $72,000 or slightly lower.
These are all rough numbers. Please kindly point out any outrageous errors I made above.
Bloomberg는 뉴스로 잘 알려져있지만 금융 데이터 및 거래 소프트웨어 Bloomberg Terminal로 큰 돈 벌고 있음. ESPN은 스포츠팬들을 위한 미디어 회사.
FreightWave는 Bloomberg와 Espn을 섞어둔 짬뽕.
전 세계 화물과 공급망 관련 최신 데이터를 제공하면서 고객들이 가격이나 수요 등을 분석, 모니터, 예측할 수 있게 도와주는 소프트웨어를 팔고 있으며 동시에 업계 관계자와 팬들을 위한 인사이트 있는 미디어 콘텐츠들을 제공중.
미디어 회사로 퍼널 최상단을 비용 $0으로 공략하고 광고비와 소프트웨어 두 경로로 매출을 만들어내니 무적의 회사가 아닌가 싶어 뉴스레터 독자들에게만 슬쩍 알려주고 싶었음.
무제한 테스트 해볼 수 있는 런치패드를 만들면? 초기 GTM 비용을 제로에 가깝게 만들수 있다면?
FreightWave는 소프트웨어 Sonar를 만들고 바로 자사 미디어 사이트에 배너걸고 모객부터 했음. 실패했다? 반응이 없다? 다른 소프트웨어를 만들고 다시 또 덤벼들수 있는 비용과 시간이 존재함. 이게 일반적인 소프트웨어 회사와 차별되는 지점 첫번째. 실패 비용과 시간 낭비가 매우 적음. 시장의 시선부터 잡으니 어떤 제품/서비스를 내놓아도 바로바로 판단을 할 수 있다.
FreightWave는 화물, 공급망 관련 미디어 콘텐츠 마켓쉐어 65%, 전 채널 합쳐서 일 방문자 300만명 달성했음. What the Truck! 이라는 유튜브 콘텐츠도 만든다.
그리고 최근 창업자 Craig은 비행기 매거진 회사를 매입했다.
지난 수십년을 가만히 지켜보면 베어마켓이든 불마켓이든 매거진은 수십년을 살아남았고 그만큼 독자의 관심도와 깊이 또한 남다르다. Craig은 너무나 많은 미디어 회사들이 제대로 수익화를 하고 있지 않다고 보고있음. 구독료는 시작에 불과함. 소프트웨어가 아니여도 실물상품이나 서비스를 팔 수 있고 실제로 그의 비행기 관련 회사는 짧은 기간에 매우 빠르게 크고 있다고 함.
FreightWave는 Bloomberg와 Espn을 섞어둔 짬뽕.
전 세계 화물과 공급망 관련 최신 데이터를 제공하면서 고객들이 가격이나 수요 등을 분석, 모니터, 예측할 수 있게 도와주는 소프트웨어를 팔고 있으며 동시에 업계 관계자와 팬들을 위한 인사이트 있는 미디어 콘텐츠들을 제공중.
미디어 회사로 퍼널 최상단을 비용 $0으로 공략하고 광고비와 소프트웨어 두 경로로 매출을 만들어내니 무적의 회사가 아닌가 싶어 뉴스레터 독자들에게만 슬쩍 알려주고 싶었음.
무제한 테스트 해볼 수 있는 런치패드를 만들면? 초기 GTM 비용을 제로에 가깝게 만들수 있다면?
FreightWave는 소프트웨어 Sonar를 만들고 바로 자사 미디어 사이트에 배너걸고 모객부터 했음. 실패했다? 반응이 없다? 다른 소프트웨어를 만들고 다시 또 덤벼들수 있는 비용과 시간이 존재함. 이게 일반적인 소프트웨어 회사와 차별되는 지점 첫번째. 실패 비용과 시간 낭비가 매우 적음. 시장의 시선부터 잡으니 어떤 제품/서비스를 내놓아도 바로바로 판단을 할 수 있다.
FreightWave는 화물, 공급망 관련 미디어 콘텐츠 마켓쉐어 65%, 전 채널 합쳐서 일 방문자 300만명 달성했음. What the Truck! 이라는 유튜브 콘텐츠도 만든다.
그리고 최근 창업자 Craig은 비행기 매거진 회사를 매입했다.
지난 수십년을 가만히 지켜보면 베어마켓이든 불마켓이든 매거진은 수십년을 살아남았고 그만큼 독자의 관심도와 깊이 또한 남다르다. Craig은 너무나 많은 미디어 회사들이 제대로 수익화를 하고 있지 않다고 보고있음. 구독료는 시작에 불과함. 소프트웨어가 아니여도 실물상품이나 서비스를 팔 수 있고 실제로 그의 비행기 관련 회사는 짧은 기간에 매우 빠르게 크고 있다고 함.
❤1
Large language models (LLMs) have demonstrated excellent zero-shot generalization to new language tasks. However, effective utilization of LLMs for zero-shot visual question-answering (VQA) remains challenging, primarily due to the modality disconnection and task disconnection between LLM and VQA task. End-to-end training on vision and language data may bridge the disconnections, but is inflexible and computationally expensive. To address this issue, we propose \emph{Img2Prompt}, a plug-and-play module that provides the prompts that can bridge the aforementioned modality and task disconnections, so that LLMs can perform zero-shot VQA tasks without end-to-end training. In order to provide such prompts, we further employ LLM-agnostic models to provide prompts that can describe image content and self-constructed question-answer pairs, which can effectively guide LLM to perform zero-shot VQA tasks. Img2Prompt offers the following benefits: 1) It can flexibly work with various LLMs to perform VQA. 2)~Without the needing of end-to-end training, it significantly reduces the cost of deploying LLM for zero-shot VQA tasks. 3) It achieves comparable or better performance than methods relying on end-to-end training. For example, we outperform Flamingo \cite{Deepmind:Flamingo2022} by 5.6\% on VQAv2. On the challenging A-OKVQA dataset, our method even outperforms few-shot methods by as much as 20\%.
https://arxiv.org/abs/2212.10846
https://arxiv.org/abs/2212.10846
arXiv.org
From Images to Textual Prompts: Zero-shot VQA with Frozen Large...
Large language models (LLMs) have demonstrated excellent zero-shot generalization to new language tasks. However, effective utilization of LLMs for zero-shot visual question-answering (VQA)...
https://news.hada.io/topic?id=13377&utm_source=slack&utm_medium=bot&utm_campaign=T05AXQMJY68
Y Combinator(YC)는 스타트업에게 더 많은 사람들이 참여하기를 바라는 아이디어와 분야에 대해 자주 논의함
이러한 아이디어들을 모아 '스타트업 요청(Request for Startups, RFS)'이라는 전통을 이어가며 공유함
아래에 나열된 RFS는 최신 버전이며, YC에 지원하기 위해 반드시 이 아이디어들 중 하나를 선택할 필요는 없음
USING MACHINE LEARNING TO SIMULATE THE PHYSICAL WORLD
- Diana Hu and Jared Friedman
Many essential software tools work by simulating the world using known principles of physics and chemistry. Weather prediction, computational fluid dynamics for designing rockets and airplanes, and tools for drug discovery that predict the interactions of molecules — today many of these are based on running a full physics simulation of the world. These are very computationally heavy since they are solving complex multivariate mathematical equations.
It turns out that AI models are general functional approximators that can also solve and predict problems like these without needing to explicitly know about physics. This results in predictions that are much less computationally expensive and can be completed in minutes or seconds on much smaller computers rather than taking days/weeks and super computers.
We’re interested in companies replacing existing simulations with ML-based ones, along with companies using ML-based simulations to open new markets currently unaddressable.
APPLYING MACHINE LEARNING TO ROBOTICS
- Diana Hu and Jared Friedman
Robotics hasn’t yet had its GPT moment, but we think it’s close.
YC has followed robotics closely for two decades. In fact, one of YC’s founders, Trevor Blackwell, is a pioneering roboticist who built the first dynamically balancing bipedal robot.
For decades, everyone has known that robots are the future, as any science fiction novel will show. But that future proved elusive because previous generations of robots were expensive and brittle, requiring controlled conditions. With the rapid improvements in foundation models, it’s finally possible to make robots that have human-level perception and judgment. That’s been the missing piece.
While consumer use-cases feature heavily in science fiction, some of the overlooked and most immediately addressable applications for robots are B2B. Specifically, we think promising areas are industrial use-cases like Gecko Robotics (W16), which builds inspection robots, and farm use-cases like Bear Flag Robotics (W18), which builds autonomous tractors and was acquired by John Deere.
We’re interested in funding people building software tools to help other people to make robots, along with people building the robots themselves.
Y Combinator(YC)는 스타트업에게 더 많은 사람들이 참여하기를 바라는 아이디어와 분야에 대해 자주 논의함
이러한 아이디어들을 모아 '스타트업 요청(Request for Startups, RFS)'이라는 전통을 이어가며 공유함
아래에 나열된 RFS는 최신 버전이며, YC에 지원하기 위해 반드시 이 아이디어들 중 하나를 선택할 필요는 없음
USING MACHINE LEARNING TO SIMULATE THE PHYSICAL WORLD
- Diana Hu and Jared Friedman
Many essential software tools work by simulating the world using known principles of physics and chemistry. Weather prediction, computational fluid dynamics for designing rockets and airplanes, and tools for drug discovery that predict the interactions of molecules — today many of these are based on running a full physics simulation of the world. These are very computationally heavy since they are solving complex multivariate mathematical equations.
It turns out that AI models are general functional approximators that can also solve and predict problems like these without needing to explicitly know about physics. This results in predictions that are much less computationally expensive and can be completed in minutes or seconds on much smaller computers rather than taking days/weeks and super computers.
We’re interested in companies replacing existing simulations with ML-based ones, along with companies using ML-based simulations to open new markets currently unaddressable.
APPLYING MACHINE LEARNING TO ROBOTICS
- Diana Hu and Jared Friedman
Robotics hasn’t yet had its GPT moment, but we think it’s close.
YC has followed robotics closely for two decades. In fact, one of YC’s founders, Trevor Blackwell, is a pioneering roboticist who built the first dynamically balancing bipedal robot.
For decades, everyone has known that robots are the future, as any science fiction novel will show. But that future proved elusive because previous generations of robots were expensive and brittle, requiring controlled conditions. With the rapid improvements in foundation models, it’s finally possible to make robots that have human-level perception and judgment. That’s been the missing piece.
While consumer use-cases feature heavily in science fiction, some of the overlooked and most immediately addressable applications for robots are B2B. Specifically, we think promising areas are industrial use-cases like Gecko Robotics (W16), which builds inspection robots, and farm use-cases like Bear Flag Robotics (W18), which builds autonomous tractors and was acquired by John Deere.
We’re interested in funding people building software tools to help other people to make robots, along with people building the robots themselves.
GeekNews
YC의 Requests for Startups - Spring 2024 | GeekNews
Y Combinator(YC)는 스타트업에게 더 많은 사람들이 참여하기를 바라는 아이디어와 분야에 대해 자주 논의함이러한 아이디어들을 모아 '스타트업 요청(Request for Startups, RFS)'이라는 전통을 이어가며 공유함아래에 나열된 RFS는 최신 버전이며, YC에 지원하기 위해 반드시 이 아이디어들 중 하나를 선택할 필요는 없음로봇공학에 머신
NEW DEFENSE TECHNOLOGY
- Jared Friedman and Gustaf Alströmer
The US is now engaged in large-scale conflicts in several regions that threaten to change our world. While the US has historically led the world in defense technology, the defense contractors it depends on have grown slow and inefficient, bloated by decades of cost-plus contracts.
SpaceX showed the world that a private space company could be vastly more effective than the publicly-funded United Launch Alliance. New companies that sell to the DoD like Palantir and Anduril are showing that the same thing is true for defense tech.
Silicon Valley was born in the early 20th century as an R&D area for the US military. Early Silicon Valley companies were largely funded by the DoD and played a key role in WWII by building military radar, code-breaking equipment, and components for the atomic bomb.
This decade is the time to return Silicon Valley to these roots.
BRING MANUFACTURING BACK TO AMERICA
- Jared Friedman
The UK became the world’s richest country in the 19th century by being the workshop of the world. The US did the same in the 20th century. But in recent decades, we’ve given up this role. The hollowing out of US manufacturing has led to social and political division and left us in a precarious place geopolitically.
Bringing manufacturing back to America is one of the biggest areas of bipartisan agreement, and the CHIPS act which was passed in 2022 proves that the US government will put serious money behind this objective.
Other changes in the world have set the stage for a resurgence of US manufacturing. New ML-based robotics systems will make it possible to automate far more, which will reduce the cost-of-labor arbitrage that pushed manufacturing to other countries in the first place. Companies like SpaceX and Tesla have trained an entire generation of engineers in how to build an American company that makes physical products but operates like a startup.
We know this works because we’ve had experience working with some of the leading companies in this space. Astranis (W16) is building telecommunications satellites in the heart of San Francisco, in a building that used to build warships for the US Navy during WWII. Gecko Robotics (W16), based in America’s old industrial heartland of Pittsburgh, builds robots that do industrial inspections. Solugen (W17) makes industrial chemicals from a large-scale plant in Houston.
NEW SPACE COMPANIES
- Jared Friedman and Dalton Caldwell
The cost to reach orbit is falling fast, having fallen over 10x since SpaceX’s first launch in 2006. A startup can now build and launch a satellite on just a seed round.
If you think about how many kilograms of payload get launched into space today, imagine how many will be sent up in one year, in five years, in ten years, and so on.
If we are entering a future with access to space being as routine and inexpensive as commercial air travel, shipping or trucking… what new businesses does that unlock?
Building a space company might scare founders by seeming too ambitious, but surprisingly, it is not necessarily harder than building a software company. YC has funded many space companies — Astranis, Relativity Space, Stoke, and many others — and their success rate has been no lower, and maybe higher, than our other companies.
CLIMATE TECH
- Gustaf Alströmer
We have a fair chance of avoiding catastrophic climate change if startups offer commercial solutions to decarbonize society or remove carbon from the atmosphere.
We’re interested in funding people building in these five top-level buckets: Energy Related, Science Required, Climate Adaptation, Green Fintech, and Carbon Accounting & Offsets.
- Jared Friedman and Gustaf Alströmer
The US is now engaged in large-scale conflicts in several regions that threaten to change our world. While the US has historically led the world in defense technology, the defense contractors it depends on have grown slow and inefficient, bloated by decades of cost-plus contracts.
SpaceX showed the world that a private space company could be vastly more effective than the publicly-funded United Launch Alliance. New companies that sell to the DoD like Palantir and Anduril are showing that the same thing is true for defense tech.
Silicon Valley was born in the early 20th century as an R&D area for the US military. Early Silicon Valley companies were largely funded by the DoD and played a key role in WWII by building military radar, code-breaking equipment, and components for the atomic bomb.
This decade is the time to return Silicon Valley to these roots.
BRING MANUFACTURING BACK TO AMERICA
- Jared Friedman
The UK became the world’s richest country in the 19th century by being the workshop of the world. The US did the same in the 20th century. But in recent decades, we’ve given up this role. The hollowing out of US manufacturing has led to social and political division and left us in a precarious place geopolitically.
Bringing manufacturing back to America is one of the biggest areas of bipartisan agreement, and the CHIPS act which was passed in 2022 proves that the US government will put serious money behind this objective.
Other changes in the world have set the stage for a resurgence of US manufacturing. New ML-based robotics systems will make it possible to automate far more, which will reduce the cost-of-labor arbitrage that pushed manufacturing to other countries in the first place. Companies like SpaceX and Tesla have trained an entire generation of engineers in how to build an American company that makes physical products but operates like a startup.
We know this works because we’ve had experience working with some of the leading companies in this space. Astranis (W16) is building telecommunications satellites in the heart of San Francisco, in a building that used to build warships for the US Navy during WWII. Gecko Robotics (W16), based in America’s old industrial heartland of Pittsburgh, builds robots that do industrial inspections. Solugen (W17) makes industrial chemicals from a large-scale plant in Houston.
NEW SPACE COMPANIES
- Jared Friedman and Dalton Caldwell
The cost to reach orbit is falling fast, having fallen over 10x since SpaceX’s first launch in 2006. A startup can now build and launch a satellite on just a seed round.
If you think about how many kilograms of payload get launched into space today, imagine how many will be sent up in one year, in five years, in ten years, and so on.
If we are entering a future with access to space being as routine and inexpensive as commercial air travel, shipping or trucking… what new businesses does that unlock?
Building a space company might scare founders by seeming too ambitious, but surprisingly, it is not necessarily harder than building a software company. YC has funded many space companies — Astranis, Relativity Space, Stoke, and many others — and their success rate has been no lower, and maybe higher, than our other companies.
CLIMATE TECH
- Gustaf Alströmer
We have a fair chance of avoiding catastrophic climate change if startups offer commercial solutions to decarbonize society or remove carbon from the atmosphere.
We’re interested in funding people building in these five top-level buckets: Energy Related, Science Required, Climate Adaptation, Green Fintech, and Carbon Accounting & Offsets.
The financial opportunity of building in this space is massive: an estimated $3-10 trillion in EBITDA will be up for grabs. Recent legislation will also significantly accelerate the existing market trends. The Inflation Reduction Act will spend an estimated $800B in the US alone over 10 years. To put that into perspective, it is almost 10x the $90B 2008 bill that catalyzed the US solar, battery, and EV industries into existence.
Y Combinator has funded well over 100 climate tech startups, and together they are worth over $10B. Building in climate tech is a once-in-a-generation opportunity.
COMMERCIAL OPEN SOURCE COMPANIES
- Nicolas Dessaigne and Diana Hu
Open source companies move more quickly than closed source companies. For developer tools, being open source is a powerful way to gain developer adoption. But it’s also a great way for startups to become mature and sell to enterprises a lot sooner. Ultimately, open source companies succeed when they become the standard choice for software engineers.
Very technical founders are at a strong advantage here, as the sales motion relies more on the technical merits of the project rather than strong sales tactics. It’s more natural for technical founders to talk to users who are engineers just like them, and they can iterate faster since they’ll get feedback from the open source community.
YC has funded over 150 open source companies including Gitlab (W15), Docker (S10), Apollo (S11), Supabase (S20) to name a few, and we want to fund more.
SPATIAL COMPUTING
- Diana Hu
AR/VR as the new personal computing platform has been in the works for over a decade. But it’s only recently, with the launches of the Apple Vision Pro and the Meta Quest 3, that we are getting close. The user experience is getting better, rendering power is increasing, and hand/eye tracking has improved dramatically — but there’s still work to be done.
We would like to see a new set of startups building software on these devices, solving practical use cases that go beyond gaming. There are so many challenges still to solve with discovering best use cases, best UX/UI practices, and more — we are excited to work with founders that are at the frontier of this tech.
NEW ENTERPRISE RESOURCE PLANNING SOFTWARE
- Dalton Caldwell
As companies get larger they end up adopting some software suite to help run their business. This piece of software is widely known as an “ERP”, or Enterprise Resource Planning software. You can think of this software as the operating system that a business runs on.
ERPs are usually known to be expensive, painful to implement, and disliked by users, yet are absolutely necessary and the very definition of business critical to its customers.
We would like to see new startups that build software that helps businesses run. Ideally that software would be loved by its customers for its flexibility and ease of use. This type of software is so valuable and important that we can imagine that there is the opportunity for dozens of new massively successful vendors.
DEVELOPER TOOLS INSPIRED BY EXISTING INTERNAL TOOLS
- Dalton Caldwell
If a developer has worked at a company with some amount of success, they have likely encountered tools or frameworks that were built by programmers at the company to help solve their own particularly painful or repetitive problems. These tools tend to have funny internal nicknames and for the most part never see the light of day.
When aspiring founders try to come up with new startup ideas they often don’t realize that the internal tools they had at prior jobs are a great place to get inspiration from.
We would like to see more startups created that are inspired by these types of homegrown tools, because it’s likely that if it’s very useful at one company, it’s very useful at others. The lineage of all software tools can often be traced back to something a programmer built to get their job done, and there is no reason to doubt this won’t continue to be true.
EXPLAINABLE A.I.
- Diana Hu and Nicolas Dessaigne
Y Combinator has funded well over 100 climate tech startups, and together they are worth over $10B. Building in climate tech is a once-in-a-generation opportunity.
COMMERCIAL OPEN SOURCE COMPANIES
- Nicolas Dessaigne and Diana Hu
Open source companies move more quickly than closed source companies. For developer tools, being open source is a powerful way to gain developer adoption. But it’s also a great way for startups to become mature and sell to enterprises a lot sooner. Ultimately, open source companies succeed when they become the standard choice for software engineers.
Very technical founders are at a strong advantage here, as the sales motion relies more on the technical merits of the project rather than strong sales tactics. It’s more natural for technical founders to talk to users who are engineers just like them, and they can iterate faster since they’ll get feedback from the open source community.
YC has funded over 150 open source companies including Gitlab (W15), Docker (S10), Apollo (S11), Supabase (S20) to name a few, and we want to fund more.
SPATIAL COMPUTING
- Diana Hu
AR/VR as the new personal computing platform has been in the works for over a decade. But it’s only recently, with the launches of the Apple Vision Pro and the Meta Quest 3, that we are getting close. The user experience is getting better, rendering power is increasing, and hand/eye tracking has improved dramatically — but there’s still work to be done.
We would like to see a new set of startups building software on these devices, solving practical use cases that go beyond gaming. There are so many challenges still to solve with discovering best use cases, best UX/UI practices, and more — we are excited to work with founders that are at the frontier of this tech.
NEW ENTERPRISE RESOURCE PLANNING SOFTWARE
- Dalton Caldwell
As companies get larger they end up adopting some software suite to help run their business. This piece of software is widely known as an “ERP”, or Enterprise Resource Planning software. You can think of this software as the operating system that a business runs on.
ERPs are usually known to be expensive, painful to implement, and disliked by users, yet are absolutely necessary and the very definition of business critical to its customers.
We would like to see new startups that build software that helps businesses run. Ideally that software would be loved by its customers for its flexibility and ease of use. This type of software is so valuable and important that we can imagine that there is the opportunity for dozens of new massively successful vendors.
DEVELOPER TOOLS INSPIRED BY EXISTING INTERNAL TOOLS
- Dalton Caldwell
If a developer has worked at a company with some amount of success, they have likely encountered tools or frameworks that were built by programmers at the company to help solve their own particularly painful or repetitive problems. These tools tend to have funny internal nicknames and for the most part never see the light of day.
When aspiring founders try to come up with new startup ideas they often don’t realize that the internal tools they had at prior jobs are a great place to get inspiration from.
We would like to see more startups created that are inspired by these types of homegrown tools, because it’s likely that if it’s very useful at one company, it’s very useful at others. The lineage of all software tools can often be traced back to something a programmer built to get their job done, and there is no reason to doubt this won’t continue to be true.
EXPLAINABLE A.I.
- Diana Hu and Nicolas Dessaigne
Would you trust an AI to diagnose you? Would you swear that a model is unbiased? Or more simply, how can we be sure that a model doesn’t hallucinate an answer?
Understanding model behavior is very challenging, but we believe that in contexts where trust is paramount it is essential for an AI model to be interpretable. Its responses need to be explainable.
For society to reap the full benefits of AI, more work needs to be done on explainable AI. We are interested in funding people building new interpretable models or tools to explain the output of existing models.
L.L.MS FOR MANUAL BACK OFFICE PROCESSES IN LEGACY ENTERPRISES
- Tom Blomfield
In pretty much every old, large company, there are huge teams of people running manual processes. They’re hidden away from the end customer (hence “back office” rather than “front office”), so we don’t tend to encounter them very often in our day-to-day lives.
Often there was just enough ambiguity in these tasks that they were very difficult to automate before the existence of LLMs. In other cases, software engineers had simply never even come into contact with these processes, so automation had never seriously been considered. People continue to do this repetitive work in the same way they have for decades.
LLMs allow whole categories of manual processes to be automated in ways that weren’t possible until recently. Where there’s linguistic ambiguity or some amount of subjective evaluation needed, LLMs come into their own.
Examples might be:
QA and compliance reviews of customers service chats
Figuring out medical billing codes and insurance reimbursement at a hospital
Assessing applications for a mortgage or a business loan
Transaction monitoring, sanctions screening and anti money-laundering investigations
Filing paperwork with the state authorities after a dealership sells a car
The problem for most software engineers is that they’ve never encountered these kinds of back office processes before. The biggest hurdle is often uncovering one of these processes to tackle.
A.I. TO BUILD ENTERPRISE SOFTWARE
- Harj Taggar
Enterprise software has a reputation among smart programmers as being boring to work on. You have to do sales and because each potential customer wants something slightly different, you end up writing bloated software to try and please them all.
But what if AI could change how enterprise software gets built and sold? The core of what every customer wants is the same — they just want it customized around the edges.
AI is good at writing code — especially when you give it an existing codebase to learn from. So what if instead of long enterprise sales cycles you just give customers a simple starter product and have them tell your AI how they want it customized? In the future, every enterprise could have their own custom ERP, CRM or HRIS that is continually updating itself as the company itself is changing.
A product based on this premise would be highly disruptive to large incumbents, because now they can’t win by just copying you and adding another feature to their bloated software. Now they would have to completely change their whole conceptual approach to building software.
Maybe the AI will get so good at this that it can think up new types of enterprise software that don’t even exist yet. Building this AI would be an interesting technical challenge and if you’re excited about building AI that can code, enterprise software is the most profitable software to build.
STABLECOIN FINANCE
- Brad Flora
Stablecoins are digital currencies that peg their value to some external reference. This is typically the U.S. dollar, but it can be other fiat currencies, assets, or even other digital currencies. Their transactions are recorded on a digital ledger, usually a blockchain. This means they can be traded at any time of day between any two wallets on the same network, transactions settle in seconds, and fees are a fraction of what you see in traditional finance.
Understanding model behavior is very challenging, but we believe that in contexts where trust is paramount it is essential for an AI model to be interpretable. Its responses need to be explainable.
For society to reap the full benefits of AI, more work needs to be done on explainable AI. We are interested in funding people building new interpretable models or tools to explain the output of existing models.
L.L.MS FOR MANUAL BACK OFFICE PROCESSES IN LEGACY ENTERPRISES
- Tom Blomfield
In pretty much every old, large company, there are huge teams of people running manual processes. They’re hidden away from the end customer (hence “back office” rather than “front office”), so we don’t tend to encounter them very often in our day-to-day lives.
Often there was just enough ambiguity in these tasks that they were very difficult to automate before the existence of LLMs. In other cases, software engineers had simply never even come into contact with these processes, so automation had never seriously been considered. People continue to do this repetitive work in the same way they have for decades.
LLMs allow whole categories of manual processes to be automated in ways that weren’t possible until recently. Where there’s linguistic ambiguity or some amount of subjective evaluation needed, LLMs come into their own.
Examples might be:
QA and compliance reviews of customers service chats
Figuring out medical billing codes and insurance reimbursement at a hospital
Assessing applications for a mortgage or a business loan
Transaction monitoring, sanctions screening and anti money-laundering investigations
Filing paperwork with the state authorities after a dealership sells a car
The problem for most software engineers is that they’ve never encountered these kinds of back office processes before. The biggest hurdle is often uncovering one of these processes to tackle.
A.I. TO BUILD ENTERPRISE SOFTWARE
- Harj Taggar
Enterprise software has a reputation among smart programmers as being boring to work on. You have to do sales and because each potential customer wants something slightly different, you end up writing bloated software to try and please them all.
But what if AI could change how enterprise software gets built and sold? The core of what every customer wants is the same — they just want it customized around the edges.
AI is good at writing code — especially when you give it an existing codebase to learn from. So what if instead of long enterprise sales cycles you just give customers a simple starter product and have them tell your AI how they want it customized? In the future, every enterprise could have their own custom ERP, CRM or HRIS that is continually updating itself as the company itself is changing.
A product based on this premise would be highly disruptive to large incumbents, because now they can’t win by just copying you and adding another feature to their bloated software. Now they would have to completely change their whole conceptual approach to building software.
Maybe the AI will get so good at this that it can think up new types of enterprise software that don’t even exist yet. Building this AI would be an interesting technical challenge and if you’re excited about building AI that can code, enterprise software is the most profitable software to build.
STABLECOIN FINANCE
- Brad Flora
Stablecoins are digital currencies that peg their value to some external reference. This is typically the U.S. dollar, but it can be other fiat currencies, assets, or even other digital currencies. Their transactions are recorded on a digital ledger, usually a blockchain. This means they can be traded at any time of day between any two wallets on the same network, transactions settle in seconds, and fees are a fraction of what you see in traditional finance.
There’s been much debate about the utility of blockchain technology, but it seems clear that stablecoins will be a big part of the future of money. We know this because YC companies have been effectively incorporating stablecoins into their operations for years now – for cross-border payments, to reduce transaction fees and fraud, to help users protect savings from hyperinflation. This utility is so straightforward it seems inevitable traditional finance will follow suit.
In fact we’re seeing signs of this. PayPal recently issued its own stablecoin. Major banks have started offering custody services and making noise about issuing their own.
It all looks a bit like digital music’s transition from the realm of outlaw file sharing in the early 2000s to becoming the norm as players like Apple entered the market. Importantly, those major players were all outmatched in the end by Spotify, a startup founded during that same transition moment.
$136b worth of stablecoins have been issued to date but the opportunity seems much more immense still. Only about seven million people have transacted with stablecoins to date, while more than half a billion live in countries with 30%+ inflation. U.S. banks hold $17T in customer deposits which are all up for grabs as well. And yet the major stablecoin issuers can be counted on one hand and the major liquidity providers with just a few fingers.
We would like to fund great teams building B2B and consumer products on top of stablecoins, tools and platforms that enable stablecoin finance and more stablecoin protocols themselves.
A WAY TO END CANCER
- Surbhi Sarna
The technology to diagnose cancer at an early stage already exists. Since most cancers are now treatable if caught early enough, this technology would dramatically reduce cancer deaths if rolled out widely and affordably.
The technology we’re talking about is an MRI. Modern MRIs are sensitive enough to detect cancer masses as small as a millimeter.
Some companies are already having success on a small scale offering MRIs to patients for a high cash price. However, there is backlash from the medical community as MRIs also create incidental findings (or false positives), that cost our healthcare system valuable time and money to investigate.
For this to work, the world would need to scale up the number of MRI scans it does by at least 100x. Doing that will require innovations in the MRI hardware, the AI algorithms to interpret scans and reduce false positives, and the business models and consumer marketing to make it a viable business. We’re interested in funding companies looking to tackle this multifaceted problem.
While much exciting progress is being made on cancer therapeutics, finding cancer early enough for our existing therapeutics to be curative might be the opportunity with the greatest potential impact.
FOUNDATION MODELS FOR BIOLOGICAL SYSTEMS
- Surbhi Sarna
The vast majority of scientific innovation fails – either on the bench during early experimentation or while in clinical trials.
Foundation models built around the vast amount of data we now have will not only enable scientists to know what path to pursue much quicker than before, but have the potential to unlock new scientific approaches to disease. Foundation models built around text and images are enabling the next-generation of consumer products; we believe foundation models built around biological systems will do the same for healthcare.
We are interested in funding highly technical founders building foundational models from scratch in any part of biology or medicine.
THE MANAGED SERVICE ORGANIZATION MODEL FOR HEALTHCARE
- Surbhi Sarna
Private equity is consuming small and large private clinics all over the country. By the time more junior healthcare workers are paid, they only make a fraction of what they are billing. This causes them to be overworked but underpaid, as much of the revenue goes to overhead and the private equity owner of the clinic.
A new startup model has emerged as an alternative to PE ownership: the MSO (Managed Service Organizations) model.
In fact we’re seeing signs of this. PayPal recently issued its own stablecoin. Major banks have started offering custody services and making noise about issuing their own.
It all looks a bit like digital music’s transition from the realm of outlaw file sharing in the early 2000s to becoming the norm as players like Apple entered the market. Importantly, those major players were all outmatched in the end by Spotify, a startup founded during that same transition moment.
$136b worth of stablecoins have been issued to date but the opportunity seems much more immense still. Only about seven million people have transacted with stablecoins to date, while more than half a billion live in countries with 30%+ inflation. U.S. banks hold $17T in customer deposits which are all up for grabs as well. And yet the major stablecoin issuers can be counted on one hand and the major liquidity providers with just a few fingers.
We would like to fund great teams building B2B and consumer products on top of stablecoins, tools and platforms that enable stablecoin finance and more stablecoin protocols themselves.
A WAY TO END CANCER
- Surbhi Sarna
The technology to diagnose cancer at an early stage already exists. Since most cancers are now treatable if caught early enough, this technology would dramatically reduce cancer deaths if rolled out widely and affordably.
The technology we’re talking about is an MRI. Modern MRIs are sensitive enough to detect cancer masses as small as a millimeter.
Some companies are already having success on a small scale offering MRIs to patients for a high cash price. However, there is backlash from the medical community as MRIs also create incidental findings (or false positives), that cost our healthcare system valuable time and money to investigate.
For this to work, the world would need to scale up the number of MRI scans it does by at least 100x. Doing that will require innovations in the MRI hardware, the AI algorithms to interpret scans and reduce false positives, and the business models and consumer marketing to make it a viable business. We’re interested in funding companies looking to tackle this multifaceted problem.
While much exciting progress is being made on cancer therapeutics, finding cancer early enough for our existing therapeutics to be curative might be the opportunity with the greatest potential impact.
FOUNDATION MODELS FOR BIOLOGICAL SYSTEMS
- Surbhi Sarna
The vast majority of scientific innovation fails – either on the bench during early experimentation or while in clinical trials.
Foundation models built around the vast amount of data we now have will not only enable scientists to know what path to pursue much quicker than before, but have the potential to unlock new scientific approaches to disease. Foundation models built around text and images are enabling the next-generation of consumer products; we believe foundation models built around biological systems will do the same for healthcare.
We are interested in funding highly technical founders building foundational models from scratch in any part of biology or medicine.
THE MANAGED SERVICE ORGANIZATION MODEL FOR HEALTHCARE
- Surbhi Sarna
Private equity is consuming small and large private clinics all over the country. By the time more junior healthcare workers are paid, they only make a fraction of what they are billing. This causes them to be overworked but underpaid, as much of the revenue goes to overhead and the private equity owner of the clinic.
A new startup model has emerged as an alternative to PE ownership: the MSO (Managed Service Organizations) model.
The MSO model enables doctors to run their own clinics by (1) providing them software that can handle back office tasks such as billing and scheduling and (2) channeling patients to them.
These functions are largely what PE ownership provides. Doctors who are part of an MSO model can continue to run small, physician-owned practices while competing successfully with large, PE-owned conglomerates.
YC has funded several companies doing this in different verticals: Nourish (nutritionists), LunaJoy (mental health for women), Finni Health (autism care for children), and others.
We are interested in investing in this MSO (Managed Service Organizations) model across every vertical in healthcare.
ELIMINATING MIDDLEMEN IN HEALTHCARE
- Surbhi Sarna
The US spends more money per person on healthcare than any other developed nation, yet our patient outcomes are no better. Much of our spend goes to paying middlemen — which in our view includes everyone not directly providing care to patients.
A recent report on medicare spending on drugs found that 70% of spend went to middlemen (primarily PBMs, wholesalers, and pharmacies) and only 30% to the pharmaceutical companies who make the drugs. Similar dynamics exist in every other vertical — hospital care, medical equipment, insurance, etc.
There are many ways startups could attack these inefficiencies, from using AI to automate repetitive human jobs to exploring new and better business models for providing care. In the spirit of Jeff Bezos’ “your margin is my opportunity”, we believe it’s possible to build a highly profitable business and make the system more efficient at the same time.
BETTER ENTERPRISE GLUE
- Pete Koomen
Most enterprise software requires customers to write a lot of custom code. Large vendors like Oracle, Salesforce, and Netsuite each support multibillion dollar ecosystems of consultants and independent software vendors (“ISVs”) who help customize these products and connect them to other software on behalf of their clients.
This “glue code” — ETL pipelines, integrations, and custom workflows — is the dark matter of the enterprise software universe.
YC has funded successful companies in this space, including Zapier (S12), Fivetran (W13) and Airbyte (W20). These products help companies build glue code for common use cases.
By generating custom code for uncommon, company-specific use cases, large language models have the potential to eliminate the need for glue code altogether. We would like to see more startups working on solving this problem.
SMALL FINE-TUNED MODELS AS AN ALTERNATIVE TO GIANT GENERIC ONES
- Nicolas Dessaigne
Giant generic models with a lot of parameters are very impressive. But they are also very costly and often come with latency and privacy challenges. Fortunately, smaller open-source models like Llama2 and Mistral have already demonstrated that, when finely tuned with appropriate data, they can yield comparable results at a fraction of the cost.
Moreover, as new hardware continues to be integrated into our phones and laptops, the prospect of running these models at the edge becomes increasingly feasible, unlocking a multitude of new use cases.
We are eager to support companies engaged in developing or fine-tuning such specialized models or creating tools to facilitate their construction.
These functions are largely what PE ownership provides. Doctors who are part of an MSO model can continue to run small, physician-owned practices while competing successfully with large, PE-owned conglomerates.
YC has funded several companies doing this in different verticals: Nourish (nutritionists), LunaJoy (mental health for women), Finni Health (autism care for children), and others.
We are interested in investing in this MSO (Managed Service Organizations) model across every vertical in healthcare.
ELIMINATING MIDDLEMEN IN HEALTHCARE
- Surbhi Sarna
The US spends more money per person on healthcare than any other developed nation, yet our patient outcomes are no better. Much of our spend goes to paying middlemen — which in our view includes everyone not directly providing care to patients.
A recent report on medicare spending on drugs found that 70% of spend went to middlemen (primarily PBMs, wholesalers, and pharmacies) and only 30% to the pharmaceutical companies who make the drugs. Similar dynamics exist in every other vertical — hospital care, medical equipment, insurance, etc.
There are many ways startups could attack these inefficiencies, from using AI to automate repetitive human jobs to exploring new and better business models for providing care. In the spirit of Jeff Bezos’ “your margin is my opportunity”, we believe it’s possible to build a highly profitable business and make the system more efficient at the same time.
BETTER ENTERPRISE GLUE
- Pete Koomen
Most enterprise software requires customers to write a lot of custom code. Large vendors like Oracle, Salesforce, and Netsuite each support multibillion dollar ecosystems of consultants and independent software vendors (“ISVs”) who help customize these products and connect them to other software on behalf of their clients.
This “glue code” — ETL pipelines, integrations, and custom workflows — is the dark matter of the enterprise software universe.
YC has funded successful companies in this space, including Zapier (S12), Fivetran (W13) and Airbyte (W20). These products help companies build glue code for common use cases.
By generating custom code for uncommon, company-specific use cases, large language models have the potential to eliminate the need for glue code altogether. We would like to see more startups working on solving this problem.
SMALL FINE-TUNED MODELS AS AN ALTERNATIVE TO GIANT GENERIC ONES
- Nicolas Dessaigne
Giant generic models with a lot of parameters are very impressive. But they are also very costly and often come with latency and privacy challenges. Fortunately, smaller open-source models like Llama2 and Mistral have already demonstrated that, when finely tuned with appropriate data, they can yield comparable results at a fraction of the cost.
Moreover, as new hardware continues to be integrated into our phones and laptops, the prospect of running these models at the edge becomes increasingly feasible, unlocking a multitude of new use cases.
We are eager to support companies engaged in developing or fine-tuning such specialized models or creating tools to facilitate their construction.
Continuous Learning_Startup & Investment
https://twitter.com/eladgil/status/1760314361544163824
MSFT seems to have increased Azure revenue by $5B or so a year via AI.
This makes its $10B investment in OpenAI seem small.
Most of the funding of LLMs by $ actually comes from big tech.
How long does this continue?
This makes its $10B investment in OpenAI seem small.
Most of the funding of LLMs by $ actually comes from big tech.
How long does this continue?
Continuous Learning_Startup & Investment
https://twitter.com/eladgil/status/1760314361544163824
What are the new foundation model architectures? How do we think about this in an agentic world? Maybe there is more to learn from AlphaGo than chat?