if you are intrigued about Q Learning but only know about PPO (policy gradients) and RLHF, this paper of John Schulman (OpenAI cofounder) is worth reading:
arxiv.org/abs/1704.06440. It shows that both are functionally equivalent in the entropy-regularized setting.

The AI industry is in a curious state right now.

Billions are being spent on capex, credits and tokens…yet few new incremental customer revenues are being generated -
at least that I see.

Two potential explanations:

1) AI is an efficiency play so companies will deploy agents and automation to reduce existing infrastructure/costs, keeping most end use cases the same. So AI generates OpEx savings “under the hood”.

2) VCs are feeding startups with billions that consume AI compute looking for new use cases. The AI compute complex booms and books real revenues but if the startups don’t find product-market fit soon then these revenues will shrivel because the startups won’t get more funding and will go bankrupt.

What are some startups making money using AI to enable a new, valuable product to end customers/users?

실적이 부진한 임원에게 제프가 한 이야기라는데 이 반대로만 하면...
1. 당신은 고객에게 충실하지 못했고..
2. 완전히 주인의식을 갖지 못했고..
3. 나 자신과 팀을 위해 더 높은 기준을 설정하지 못하고..
4. 충분히 크게 생각하지 못하고...
5. 신속하게 결정하고 행동하지 못하고...
6. 내 성과가 부족한 것이 확실해졌을 때 나 스스로 공개적이고 단호하게 비판하지 못했다..... 라고 실랄하게 질책을 받았다고 한다.

OpenAI 안에는 2020년에 생긴 Applied라는 팀이 ChatGPT를 담당한다고 함. PMF 찾는 초기 스타트업처럼 움직이는게 목표였다고. 그리고 리서치 팀하고 분리가 되어있지만 긴밀하게 협업하는 구조를 가지고 있음.

https://newsletter.pragmaticengineer.com/p/inside-openai-how-does-chatgpt-ship

"The more neatly you fit into society, the less free you actually are."

@naval