Pepemedia 🇺🇦🏳️🌈
Photo
New original meme for y'all, in response to:
(https://news.1rj.ru/str/atellschannel/4736)
(https://news.1rj.ru/str/atellschannel/4736)
Telegram
киберпсалмы & mathюки
"With GPT-3 costing around $4.6 million in compute, than would put a price of $8.6 billion for the compute to train "GPT-4".
If making bigger models was so easy with parameter partitioning from a memory point of view then this seems like the hardest challenge, but you do need to solve the memory issue to actually get it to load at all.
However, if you're lucky you can get 3-6x compute increase from Nvidia A100s over V100s, https://developer.nvidia.com/blog/nvidia-ampere-architecture-in-depth/
But even a 6x compute gain would still put the cost at $1.4 billion.
Nvidia only reported $1.15 billion in revenue from "Data Center" in 2020 Q1, so just to train "GPT-4" you would pretty much need the entire world's supply of graphic cards for 1 quarter (3 months), at least on that order of magnitude."
Looks like where will be no significantly larger model in GPT4 for us :C
source - https://www.reddit.com/r/MachineLearning/comments/i49jf8/d_biggest_roadblock_in_making_gpt4_a_20_trillion/
If making bigger models was so easy with parameter partitioning from a memory point of view then this seems like the hardest challenge, but you do need to solve the memory issue to actually get it to load at all.
However, if you're lucky you can get 3-6x compute increase from Nvidia A100s over V100s, https://developer.nvidia.com/blog/nvidia-ampere-architecture-in-depth/
But even a 6x compute gain would still put the cost at $1.4 billion.
Nvidia only reported $1.15 billion in revenue from "Data Center" in 2020 Q1, so just to train "GPT-4" you would pretty much need the entire world's supply of graphic cards for 1 quarter (3 months), at least on that order of magnitude."
Looks like where will be no significantly larger model in GPT4 for us :C
source - https://www.reddit.com/r/MachineLearning/comments/i49jf8/d_biggest_roadblock_in_making_gpt4_a_20_trillion/
just have found an amazing site - https://code.likeagirl.io/
really awesome concept that may inspire someone
really awesome concept that may inspire someone
Code Like A Girl
Welcome to Code Like A Girl, a space that celebrates redefining society's perceptions of women in technology. Share your story with us!
I have a good news and a bad news.
Good news: original mass effect trilogy will be continued.
Bad news: original mass effect trilogy will be continued.
https://youtu.be/Lg-Ctg6k_Ao
Good news: original mass effect trilogy will be continued.
Bad news: original mass effect trilogy will be continued.
https://youtu.be/Lg-Ctg6k_Ao
YouTube
The Next Mass Effect - Official Teaser Trailer
Get a sneak peek at the next chapter of the Mass Effect universe. A veteran team at BioWare is in the early stages of what is on the horizon for the Mass Effect franchise, and we are excited to show you where we’ll be heading next!