Spent two hours with Marc Andreessen, who gave me a masterclass on how to think, learn, read, research, and write.
Here's what I learned:
1. Read, read, read... then read some more.
2. Many of your best ideas will emerge in fits of rage or frustration. Channel the fury. Smash the keyboard. Lean into the passion. Torch the page with your energy.
3. Marc doesn't have much of a formal writing process. He thinks and thinks, and when epiphany strikes, he hammers out an outline as fast as possible to get his ideas on paper. Then, he turns it into a full article.
4. Marc's motto for writing and thinking: "Strong views, weakly held." Put yourself out there, but stay on the hunt for dissenting opinions from smart and respectful people.
5. Online writing tolerates and even encourages stylistic idiosyncrasies that traditional publishing would not accommodate. Lean into them.
6. The world is awash in bad content. You need to punch through. Snappy one-liners and genuine conviction are two ways to do that.
7. Marc's been reading online for as long as anybody on the planet, and the biggest thing that's surprised him is how political the Internet's become. Something changed between ~2013-2015. The Internet was once an escape from political debates. Now it's a hotbed of them.
8. Writing software is halfway between writing a novel and building a bridge.
9. Play around with communication tools. Push the limits. Doesn't matter what the rules are. When Marc felt constrained by Twitter's 140-character limit, he started replying to his own tweets and invented the Twitter thread.
10. On the quest for good ideas, surround yourself with "lateral thinkers" who can't help but come up with variant perspectives on everything they see. They won't always be right, but they're always challenge your thinking.
11. Media formats are cyclical. Nietzsche wrote in aphorisms and Twitter is aphorisms-as-a-service. Hip-hop brought back poetry. Montaigne pioneered the essay format and blogs brought them back into vogue.
12. People should write more manifestos.
13. Marc's nomination for the best living American novelist: James Ellroy.
14. GPT has revealed how much writing is pure pablum. Bland, lifeless, uninsightful, unoffensive, and not worth the price of the ink it was printed with.
15. "With GPT, every writer now has a writing partner who can do an infinite amount of grunt work without complaining."
16. "ChatGPT plagiarism is a complete non-issue. If you can't out-write a machine, what are you doing writing?"
17. Marc writes from the heart. He doesn't do much editing and likes to provide reading recommendations instead of directly citing his sources.
18. The person who writes down the plan in an organization has tremendous power. If you want to find the up-and-comers at a tech company, look into who's writing the plan. Though they may not be coming up with all the ideas, you'll know they have the energy, motivation, and skills to organize and communicate ideas in a written form.
19. Marc uses a barbell approach to consume information. He focuses on what's happening right now while also reading a lot of things that were written 10+ years ago. The content is either timely or timeless, with almost nothing in between.
Here's what I learned:
1. Read, read, read... then read some more.
2. Many of your best ideas will emerge in fits of rage or frustration. Channel the fury. Smash the keyboard. Lean into the passion. Torch the page with your energy.
3. Marc doesn't have much of a formal writing process. He thinks and thinks, and when epiphany strikes, he hammers out an outline as fast as possible to get his ideas on paper. Then, he turns it into a full article.
4. Marc's motto for writing and thinking: "Strong views, weakly held." Put yourself out there, but stay on the hunt for dissenting opinions from smart and respectful people.
5. Online writing tolerates and even encourages stylistic idiosyncrasies that traditional publishing would not accommodate. Lean into them.
6. The world is awash in bad content. You need to punch through. Snappy one-liners and genuine conviction are two ways to do that.
7. Marc's been reading online for as long as anybody on the planet, and the biggest thing that's surprised him is how political the Internet's become. Something changed between ~2013-2015. The Internet was once an escape from political debates. Now it's a hotbed of them.
8. Writing software is halfway between writing a novel and building a bridge.
9. Play around with communication tools. Push the limits. Doesn't matter what the rules are. When Marc felt constrained by Twitter's 140-character limit, he started replying to his own tweets and invented the Twitter thread.
10. On the quest for good ideas, surround yourself with "lateral thinkers" who can't help but come up with variant perspectives on everything they see. They won't always be right, but they're always challenge your thinking.
11. Media formats are cyclical. Nietzsche wrote in aphorisms and Twitter is aphorisms-as-a-service. Hip-hop brought back poetry. Montaigne pioneered the essay format and blogs brought them back into vogue.
12. People should write more manifestos.
13. Marc's nomination for the best living American novelist: James Ellroy.
14. GPT has revealed how much writing is pure pablum. Bland, lifeless, uninsightful, unoffensive, and not worth the price of the ink it was printed with.
15. "With GPT, every writer now has a writing partner who can do an infinite amount of grunt work without complaining."
16. "ChatGPT plagiarism is a complete non-issue. If you can't out-write a machine, what are you doing writing?"
17. Marc writes from the heart. He doesn't do much editing and likes to provide reading recommendations instead of directly citing his sources.
18. The person who writes down the plan in an organization has tremendous power. If you want to find the up-and-comers at a tech company, look into who's writing the plan. Though they may not be coming up with all the ideas, you'll know they have the energy, motivation, and skills to organize and communicate ideas in a written form.
19. Marc uses a barbell approach to consume information. He focuses on what's happening right now while also reading a lot of things that were written 10+ years ago. The content is either timely or timeless, with almost nothing in between.
Q: How can a startup build a great recruiting pipeline?
In the clip below, Naval Ravikant explains:
"You have to build a great pipeline. Your best pipeline is going to come from your personal contacts. You literally have to sit everyone at your company down and tell them to name the 10 best people they've ever worked with or gone to school with. I don't care what they're doing now. I don't care if they're getting their PhD. I don't care if they're starting their own company. I don't care if they just joined Google a month ago."
He continues:
"You get those 10 best people and then you just pound the pavement. You have lunch, coffee, breakfast, dinner, whatever with each of them. Even if they're not available for recruiting, they're going to give you a few people. Demand three people from them. And keep in touch because when they become loose, you want them thinking of you."
He also mentions that other companies in your space could also be a great source of talent, in addition to founders of companies that have tried to build something similar to you but failed.
In the clip below, Naval Ravikant explains:
"You have to build a great pipeline. Your best pipeline is going to come from your personal contacts. You literally have to sit everyone at your company down and tell them to name the 10 best people they've ever worked with or gone to school with. I don't care what they're doing now. I don't care if they're getting their PhD. I don't care if they're starting their own company. I don't care if they just joined Google a month ago."
He continues:
"You get those 10 best people and then you just pound the pavement. You have lunch, coffee, breakfast, dinner, whatever with each of them. Even if they're not available for recruiting, they're going to give you a few people. Demand three people from them. And keep in touch because when they become loose, you want them thinking of you."
He also mentions that other companies in your space could also be a great source of talent, in addition to founders of companies that have tried to build something similar to you but failed.
초기 투자 집중하는 펀드에서 받은 내용중 일부...
Software multiples have come down a bit in the past quarter to a median of 5.4x forward revenue, with 8x for the top quartile.
(ImageSource: BVP Cloud Index as of 10/3/23) 즉...소프트웨어 회사들 기업가치는 미래 12개월 매출의 5.4배... 아주 톱 회사들은 8배 정도로 내려왔다. 얼마전 100X 이상까지 올라가던 시절은 잊는게 좋다.
2021년 Q4부터 투자는 계속 줄고 있다. 지난분기 (Q3, 2024) 에는 $73B 이 투자되었는데 그것은 Q4 2019 이후 제일 낮은 금액이였고 또 10,095 회사가 투자 받았는데 Q3 2020 년 이후 제일 낮았다.
시드에서 Series A 받는게 무지 어려워졌다. 이젠 A 라운드 받으려면 약 월매출 5천만원 이상 넘으면서 이익을 내기 시작하던지... 가까워야된다.
---
이게 지금 미국 초기 기업들 사정이다. 좀 더 큰 기업들은 여기서 선을 이어서 이해하면 얼만큼 어려운지 이해 하기 쉽다.
그래서... 왠만한 회사들은 계속 추가 펀딩이 되겠지 생각하고 사업을 하면 안된다. 무조건 가지고 있는 자금으로 이익을 내면서 살아남는데 추가 자금이 꼭 필요 없어야만... 펀딩이 되는 시절이 당분간 계속 될거다.
Software multiples have come down a bit in the past quarter to a median of 5.4x forward revenue, with 8x for the top quartile.
(ImageSource: BVP Cloud Index as of 10/3/23) 즉...소프트웨어 회사들 기업가치는 미래 12개월 매출의 5.4배... 아주 톱 회사들은 8배 정도로 내려왔다. 얼마전 100X 이상까지 올라가던 시절은 잊는게 좋다.
2021년 Q4부터 투자는 계속 줄고 있다. 지난분기 (Q3, 2024) 에는 $73B 이 투자되었는데 그것은 Q4 2019 이후 제일 낮은 금액이였고 또 10,095 회사가 투자 받았는데 Q3 2020 년 이후 제일 낮았다.
시드에서 Series A 받는게 무지 어려워졌다. 이젠 A 라운드 받으려면 약 월매출 5천만원 이상 넘으면서 이익을 내기 시작하던지... 가까워야된다.
---
이게 지금 미국 초기 기업들 사정이다. 좀 더 큰 기업들은 여기서 선을 이어서 이해하면 얼만큼 어려운지 이해 하기 쉽다.
그래서... 왠만한 회사들은 계속 추가 펀딩이 되겠지 생각하고 사업을 하면 안된다. 무조건 가지고 있는 자금으로 이익을 내면서 살아남는데 추가 자금이 꼭 필요 없어야만... 펀딩이 되는 시절이 당분간 계속 될거다.
👍3
<본격 AI 하드웨어 시대 개막: 구글 하드웨어 이벤트 현장 후기!!> 🤖
1. 구글(Google)이 픽셀8, 픽셀8 프로, 픽셀 워치2, 픽셀 버즈 프로 등 일련의 하드웨어 제품을 공개.
2. 모든 하드웨어의 가장 중요한 기능은 AI, 머신러닝으로 귀결.
3. 픽셀8 프로는 구글의 생성형 AI(generative AI) 모델이 탑재된(on-device) 첫 번째 하드웨어.
4. 여러 명의 피사체가 등장하는 사진을 찍을 때 각 인물의 얼굴을 개별로 수정, 모두가 잘 나온 최종본 하나를 만들 수 있는 식.
5. 영상을 촬영했는데, 등장인물의 대화에 비해 주변 소음이 너무 크다면 소음만 줄이거나 키울 수 있음.
6. 지하철 내부 등 시끄러운 환경에서 통화할 때 소음을 없애 목소리만 또렷이 들을 수 있음. (with 픽셀 버즈 프로)
7. 회의 때 전화가 걸려 오면 직접 받지 않고 AI가 받게 해 나 대신 상대방과 간단한 통화를 하게 만드는 것도 가능(Call Screening). 받는 사람이 AI라고 인식하지 못할 정도의 자연스러운 목소리가 전달됨.
8. 예컨대 택배 기사님한테 전화가 걸려오면 “문 앞에 두세요”를 문자가 아닌 음성(대화 방식)으로 AI가 전할 수 있음.
9. 구글 어시스턴트는 AI 챗봇 ’바드(Bard)’로 업그레이드. 보다 복잡한 물음을 던지고 답을 얻을 수 있음.
10. 소프트웨어(AI)가 하드웨어를 정의하는 시대, AI 기술의 폭발적 발전으로 양질전환, 퀀텀 점프, 패러다임 시프트가 시작된 시기라는 걸 절감.
더밀크 The Miilk @followers
1. 구글(Google)이 픽셀8, 픽셀8 프로, 픽셀 워치2, 픽셀 버즈 프로 등 일련의 하드웨어 제품을 공개.
2. 모든 하드웨어의 가장 중요한 기능은 AI, 머신러닝으로 귀결.
3. 픽셀8 프로는 구글의 생성형 AI(generative AI) 모델이 탑재된(on-device) 첫 번째 하드웨어.
4. 여러 명의 피사체가 등장하는 사진을 찍을 때 각 인물의 얼굴을 개별로 수정, 모두가 잘 나온 최종본 하나를 만들 수 있는 식.
5. 영상을 촬영했는데, 등장인물의 대화에 비해 주변 소음이 너무 크다면 소음만 줄이거나 키울 수 있음.
6. 지하철 내부 등 시끄러운 환경에서 통화할 때 소음을 없애 목소리만 또렷이 들을 수 있음. (with 픽셀 버즈 프로)
7. 회의 때 전화가 걸려 오면 직접 받지 않고 AI가 받게 해 나 대신 상대방과 간단한 통화를 하게 만드는 것도 가능(Call Screening). 받는 사람이 AI라고 인식하지 못할 정도의 자연스러운 목소리가 전달됨.
8. 예컨대 택배 기사님한테 전화가 걸려오면 “문 앞에 두세요”를 문자가 아닌 음성(대화 방식)으로 AI가 전할 수 있음.
9. 구글 어시스턴트는 AI 챗봇 ’바드(Bard)’로 업그레이드. 보다 복잡한 물음을 던지고 답을 얻을 수 있음.
10. 소프트웨어(AI)가 하드웨어를 정의하는 시대, AI 기술의 폭발적 발전으로 양질전환, 퀀텀 점프, 패러다임 시프트가 시작된 시기라는 걸 절감.
더밀크 The Miilk @followers
👍2
AI startup funding was CRAZY today 💵
- Cortica raises a $40M Series D
- rabbit raises a $20M Series A
- Vayu Robotics raises a $12.7M Series A
- Move AI raises a $10M Seed
- Induced AI raises a $2.3M Seed
Want to know what they do? 🧵
Amount: $40M 🎉
Round: Series D
Investor: CVS Health Ventures, LRVHealth
Quick Intro 👉 Cortica’s mission is to design and deliver life-changing care - one child, one family, one community at a time.
🚨 rabbit
Amount: $20M 🎉
Round: Series A
Investor: Khosla Ventures
Quick Intro 👉 Rabbit is building a custom, AI-powered UI layer designed to sit between a user and any operating system.
@VayuRobotics
Amount: $12.7M 🎉
Round: Seed
Investor: Khosla Ventures
Quick Intro 👉 Building the foundation model for robotics – the next gen of AI to power perception and motion. They envision intelligent systems will advance safe and sustainable human productivity.
@MoveAI_
Amount: $10M 🎉
Round: Seed
Investor: Warner Music Group
Quick Intro 👉 Move AI’s mission is to empower millions of creators by harnessing the potential of generative AI to digitize movement and democratize animation at scale.
@InducedAI
Amount: $2.3M 🎉
Round: Seed
Investor: Sam Altman, Peak XV Partners
Quick Intro 👉 Induced AI offers an AI-based native browser robotic process automation (RPA) platform.
https://x.com/chiefaioffice/status/1709675847769096598?s=46&t=h5Byg6Wosg8MJb4pbPSDow
- Cortica raises a $40M Series D
- rabbit raises a $20M Series A
- Vayu Robotics raises a $12.7M Series A
- Move AI raises a $10M Seed
- Induced AI raises a $2.3M Seed
Want to know what they do? 🧵
Amount: $40M 🎉
Round: Series D
Investor: CVS Health Ventures, LRVHealth
Quick Intro 👉 Cortica’s mission is to design and deliver life-changing care - one child, one family, one community at a time.
🚨 rabbit
Amount: $20M 🎉
Round: Series A
Investor: Khosla Ventures
Quick Intro 👉 Rabbit is building a custom, AI-powered UI layer designed to sit between a user and any operating system.
@VayuRobotics
Amount: $12.7M 🎉
Round: Seed
Investor: Khosla Ventures
Quick Intro 👉 Building the foundation model for robotics – the next gen of AI to power perception and motion. They envision intelligent systems will advance safe and sustainable human productivity.
@MoveAI_
Amount: $10M 🎉
Round: Seed
Investor: Warner Music Group
Quick Intro 👉 Move AI’s mission is to empower millions of creators by harnessing the potential of generative AI to digitize movement and democratize animation at scale.
@InducedAI
Amount: $2.3M 🎉
Round: Seed
Investor: Sam Altman, Peak XV Partners
Quick Intro 👉 Induced AI offers an AI-based native browser robotic process automation (RPA) platform.
https://x.com/chiefaioffice/status/1709675847769096598?s=46&t=h5Byg6Wosg8MJb4pbPSDow
X (formerly Twitter)
Chief AI Officer on X
AI startup funding was CRAZY today 💵
- Cortica raises a $40M Series D
- rabbit raises a $20M Series A
- Vayu Robotics raises a $12.7M Series A
- Move AI raises a $10M Seed
- Induced AI raises a $2.3M Seed
Want to know what they do? 🧵
- Cortica raises a $40M Series D
- rabbit raises a $20M Series A
- Vayu Robotics raises a $12.7M Series A
- Move AI raises a $10M Seed
- Induced AI raises a $2.3M Seed
Want to know what they do? 🧵
장안의 화제 논문 “GPT-4V(ision)을 디벼보자 - The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)”
GPT-4V의 이미지 이해 능력이 어디까지 가능한지를 탐구한 논문인데요.
ChatGPT가 처음 나왔을 때 정도의 충격입니다. 이미지 판별, 디텍팅, OCR은 물론이고 X-Ray 분석과 밈의 이해와 설명까지합니다.
핵심은 기존의 모든 이미지와 관련된 AI 모델의 능력을 GPT-4V 하나가 전부 발휘하고 있다는 것인데요. GPT-3가 기존의 모든 자연어와 관련된 AI 모델의 능력을 전부 하나의 모델로 가능하게 된 상황과 같습니다.
100가지의 능력을 하나의 모델로 가능하게 되었을 때 단순히 100배의 능력이 발휘되는 것이 아니라, 능력이 기하급수적으로 점프하여 10,000배 이상의 능력을 발휘 할 수 있게 되었다는 것이 핵심이라고 봅니다.
즉, GPT-3로 인해 AI 기술과 업계가 완전히 바뀐 것과 같은 상황이 다시 온 것이라고 봐도 무방할 것 같습니다. (아직은 개별 비전 태스크의 성능의 수준면에서 보면 GPT-3.5 수준 정도로 생각됩니다만, Vision이 GPT-4 수준으로 올라오는 것은 시간문제겠죠.)
안보신 분들은 꼭 한 번 보시기 바랍니다. 이미지만 봐도 어떤 일들이 가능한지와 앞으로 발전하게 될 모습을 충분히 알 수 있습니다.
https://arxiv.org/abs/2309.17421
GPT-4V의 이미지 이해 능력이 어디까지 가능한지를 탐구한 논문인데요.
ChatGPT가 처음 나왔을 때 정도의 충격입니다. 이미지 판별, 디텍팅, OCR은 물론이고 X-Ray 분석과 밈의 이해와 설명까지합니다.
핵심은 기존의 모든 이미지와 관련된 AI 모델의 능력을 GPT-4V 하나가 전부 발휘하고 있다는 것인데요. GPT-3가 기존의 모든 자연어와 관련된 AI 모델의 능력을 전부 하나의 모델로 가능하게 된 상황과 같습니다.
100가지의 능력을 하나의 모델로 가능하게 되었을 때 단순히 100배의 능력이 발휘되는 것이 아니라, 능력이 기하급수적으로 점프하여 10,000배 이상의 능력을 발휘 할 수 있게 되었다는 것이 핵심이라고 봅니다.
즉, GPT-3로 인해 AI 기술과 업계가 완전히 바뀐 것과 같은 상황이 다시 온 것이라고 봐도 무방할 것 같습니다. (아직은 개별 비전 태스크의 성능의 수준면에서 보면 GPT-3.5 수준 정도로 생각됩니다만, Vision이 GPT-4 수준으로 올라오는 것은 시간문제겠죠.)
안보신 분들은 꼭 한 번 보시기 바랍니다. 이미지만 봐도 어떤 일들이 가능한지와 앞으로 발전하게 될 모습을 충분히 알 수 있습니다.
https://arxiv.org/abs/2309.17421
❤3
Brian Balfour is the founder and CEO of Reforge, former VP of Growth at HubSpot, and co-founder of three other startups.
I've been looking forward to having Brian on the pod for quite a while, and so instead of talking through the typical growth loops and product frameworks, we made this a very special and unique episode. Brian dug through a Notion doc he keeps of lessons he's learned from his career and life (which includes over 100!), and chose ten of the most important and meaningful to share.
Here's a peek:
▫️ Lesson 1: Inspect the work, not the person.
▫️ Lesson 2: Tell me what it takes to win; then tell me the cost.
▫️ Lesson 3: Problems never end (and that’s okay).
▫️ Lesson 4: The year is made in the first six months.
▫️ Lesson 5: Growth is a system between acquisition, retention, and monetization. Change one and you affect them all.
• Lesson 6: Do the opposite.
• Lesson 7: Use cases, not personas.
• Lesson 8: Solving for everyone is solving for no one.
• Lesson 9: Find sparring partners, not mentors or coaches.
• Lesson 10: 2x+ the activation energy for things that need to change
I found this conversation incredibly informative and inspiring, and I know you will too.
Listen now 👇
YouTube: https://lnkd.in/gX29QbFA
I've been looking forward to having Brian on the pod for quite a while, and so instead of talking through the typical growth loops and product frameworks, we made this a very special and unique episode. Brian dug through a Notion doc he keeps of lessons he's learned from his career and life (which includes over 100!), and chose ten of the most important and meaningful to share.
Here's a peek:
▫️ Lesson 1: Inspect the work, not the person.
▫️ Lesson 2: Tell me what it takes to win; then tell me the cost.
▫️ Lesson 3: Problems never end (and that’s okay).
▫️ Lesson 4: The year is made in the first six months.
▫️ Lesson 5: Growth is a system between acquisition, retention, and monetization. Change one and you affect them all.
• Lesson 6: Do the opposite.
• Lesson 7: Use cases, not personas.
• Lesson 8: Solving for everyone is solving for no one.
• Lesson 9: Find sparring partners, not mentors or coaches.
• Lesson 10: 2x+ the activation energy for things that need to change
I found this conversation incredibly informative and inspiring, and I know you will too.
Listen now 👇
YouTube: https://lnkd.in/gX29QbFA
lnkd.in
LinkedIn
This link will take you to a page that’s not on LinkedIn
I gave a talk at Seoul National University.
I noscriptd the talk “Large Language Models (in 2023)”. This was an ambitious attempt to summarize our exploding field.
Video: https://youtu.be/dbo3kNKPaUA
Slides: https://docs.google.com/presentation/d/1636wKStYdT_yRPbJNrf8MLKpQghuWGDmyHinHhAKeXY/edit?usp=sharing
Trying to summarize the field forced me to think about what really matters in the field. While scaling undeniably stands out, its far-reaching implications are more nuanced. I share my thoughts on scaling from three angles:
1) Change in perspective is necessary because some abilities only emerge at a certain scale. Even if some abilities don’t work with the current generation LLMs, we should not claim that it doesn’t work. Rather, we should think it doesn’t work yet. Once larger models are available many conclusions change.
This also means that some conclusions from the past are invalidated and we need to constantly unlearn intuitions built on top of such ideas.
2) From first-principles, scaling up the Transformer amounts to efficiently doing matrix multiplications with many, many machines. I see many researchers in the field of LLM who are not familiar with how scaling is actually done. This section is targeted for technical audiences who want to understand what it means to train large models.
3) I talk about what we should think about for further scaling (think 10000x GPT-4 scale). To me scaling isn’t just doing the same thing with more machines. It entails finding the inductive bias that is the bottleneck in further scaling.
I believe that the maximum likelihood objective function is the bottleneck in achieving the scale of 10000x GPT-4 level. Learning the objective function with an expressive neural net is the next paradigm that is a lot more scalable. With the compute cost going down exponentially, scalable methods eventually win. Don’t compete with that.
In all of these sections, I strive to describe everything from first-principles. In an extremely fast moving field like LLM, no one can keep up. I believe that understanding the core ideas by deriving from first-principles is the only scalable approach.
I noscriptd the talk “Large Language Models (in 2023)”. This was an ambitious attempt to summarize our exploding field.
Video: https://youtu.be/dbo3kNKPaUA
Slides: https://docs.google.com/presentation/d/1636wKStYdT_yRPbJNrf8MLKpQghuWGDmyHinHhAKeXY/edit?usp=sharing
Trying to summarize the field forced me to think about what really matters in the field. While scaling undeniably stands out, its far-reaching implications are more nuanced. I share my thoughts on scaling from three angles:
1) Change in perspective is necessary because some abilities only emerge at a certain scale. Even if some abilities don’t work with the current generation LLMs, we should not claim that it doesn’t work. Rather, we should think it doesn’t work yet. Once larger models are available many conclusions change.
This also means that some conclusions from the past are invalidated and we need to constantly unlearn intuitions built on top of such ideas.
2) From first-principles, scaling up the Transformer amounts to efficiently doing matrix multiplications with many, many machines. I see many researchers in the field of LLM who are not familiar with how scaling is actually done. This section is targeted for technical audiences who want to understand what it means to train large models.
3) I talk about what we should think about for further scaling (think 10000x GPT-4 scale). To me scaling isn’t just doing the same thing with more machines. It entails finding the inductive bias that is the bottleneck in further scaling.
I believe that the maximum likelihood objective function is the bottleneck in achieving the scale of 10000x GPT-4 level. Learning the objective function with an expressive neural net is the next paradigm that is a lot more scalable. With the compute cost going down exponentially, scalable methods eventually win. Don’t compete with that.
In all of these sections, I strive to describe everything from first-principles. In an extremely fast moving field like LLM, no one can keep up. I believe that understanding the core ideas by deriving from first-principles is the only scalable approach.
YouTube
Large Language Models (in 2023)
I gave a talk at Seoul National University.
I noscriptd the talk “Large Language Models (in 2023)”. This was an ambitious attempt to summarize our exploding field.
Trying to summarize the field forced me to think about what really matters in the field. While…
I noscriptd the talk “Large Language Models (in 2023)”. This was an ambitious attempt to summarize our exploding field.
Trying to summarize the field forced me to think about what really matters in the field. While…
London-based autone (YC S22) has raised $4.5M (~€4.27M) in seed funding to help business maximize their growth through data-driven inventory optimization.
Today, retailers have to make 1000s of complex operational decisions every day, all impacting their bottom line. This problem is being currently tackled with Excel or legacy systems, both of which no longer fit for purpose.
Founded by Adil Bouhdadi and Harry Glucksmann Cheslaw, two business scientists with over a decade of experience building successful data-driven supply chains at places like Kering and LVMH, autone is a platform that lets retailers make optimal decisions, easily and quickly. It ingests a retailer's data, generates recommendations, and then allows users to approve a given action.
Autone covers topics including product pricing, inventory replenishment, and re-ordering with the goal of covering all operational processes.
Congrats to the team on the seed!
Today, retailers have to make 1000s of complex operational decisions every day, all impacting their bottom line. This problem is being currently tackled with Excel or legacy systems, both of which no longer fit for purpose.
Founded by Adil Bouhdadi and Harry Glucksmann Cheslaw, two business scientists with over a decade of experience building successful data-driven supply chains at places like Kering and LVMH, autone is a platform that lets retailers make optimal decisions, easily and quickly. It ingests a retailer's data, generates recommendations, and then allows users to approve a given action.
Autone covers topics including product pricing, inventory replenishment, and re-ordering with the goal of covering all operational processes.
Congrats to the team on the seed!
https://twitter.com/AlexReibman/status/1710160221719654421
For one day only, Lightspeed Venture Partners invited San Francisco’s top AI entrepreneurs to showcase what’s possible with AI
Live product and raw code only. No bullshit.
Here are the top demos from the
@cerebral_valley
x
@lightspeedvp
AI coworking day (🧵)
For one day only, Lightspeed Venture Partners invited San Francisco’s top AI entrepreneurs to showcase what’s possible with AI
Live product and raw code only. No bullshit.
Here are the top demos from the
@cerebral_valley
x
@lightspeedvp
AI coworking day (🧵)
X (formerly Twitter)
Alex Reibman on X
For one day only, Lightspeed Venture Partners invited San Francisco’s top AI entrepreneurs to showcase what’s possible with AI
Live product and raw code only. No bullshit.
Here are the top demos from the @cerebral_valley x @lightspeedvp AI coworking day…
Live product and raw code only. No bullshit.
Here are the top demos from the @cerebral_valley x @lightspeedvp AI coworking day…
👍2
I bought a couple of Chinese microphones, I wear them and turn them on all day recording everything I speak, at the end of the day the files are processed with OpenAi’s Whisper and transformed into text files from which the information is extracted.
Since the early days of the iPhone and Android, the smartphone has reigned supreme. Now, entrepreneurs and tech giants are racing to deliver AI in new devices and gadgets to challenge the dominant device.
Companies race to make AI you can wear
https://www.axios.com/2023/10/04/ai-wearables-meta-humane-tab-rewind?utm_campaign=editorial&utm_source=twitter&utm_medium=social
Companies race to make AI you can wear
https://www.axios.com/2023/10/04/ai-wearables-meta-humane-tab-rewind?utm_campaign=editorial&utm_source=twitter&utm_medium=social
Axios
The race for AI you can wear
Meta and others are shipping glasses, pendants and pins to harness the power of generative AI.
https://flair.ai/
커머스에서 사용하는 디자인들을 쉽게 사용할 수 있게 도와주는 AI Tool인데 백만 유저가 넘었다고 하네요.
https://twitter.com/i/status/1708340444546109820
커머스에서 사용하는 디자인들을 쉽게 사용할 수 있게 도와주는 AI Tool인데 백만 유저가 넘었다고 하네요.
https://twitter.com/i/status/1708340444546109820
Flair.ai
AI Product Photo Generator & Editor | Create E-Commerce Images
Create studio-quality e-commerce photoshoots in seconds with our drag-and-drop AI editor. Try it free today.
Today we're officially opening applications for Llama Impact Grants.
Full details & application ➡️ https://bit.ly/45lqz7z
From now until November 15, organizations across the globe can submit proposals for how they'd like to utilize Llama 2 to address challenges across three different tracks: education, environment & open innovation. The goal of the program is to identify and support the most compelling applications of Llama 2 for societal benefit.
Following a two-phase proposal review process, three $500,000 grants will be awarded to winning teams to implement their solutions.
We can't wait to see what you'll build!
Full details & application ➡️ https://bit.ly/45lqz7z
From now until November 15, organizations across the globe can submit proposals for how they'd like to utilize Llama 2 to address challenges across three different tracks: education, environment & open innovation. The goal of the program is to identify and support the most compelling applications of Llama 2 for societal benefit.
Following a two-phase proposal review process, three $500,000 grants will be awarded to winning teams to implement their solutions.
We can't wait to see what you'll build!
AI at Meta
Llama Impact Grants - AI at Meta
We are launching a challenge to encourage a diverse set of public, non-profit, and for-profit entities to use Llama 2 to address environmental, education and other important challenges.