https://youtu.be/o0Ulj7fYLgw geth on the hype chariot mine own dudes to beginneth our sloweth and depress'd journey
YouTube
novemthree - In The Soil (official music video from the upcoming 2021 album)
We are glad to present a new music video "In The Soil" by mysterious psych-folk project novemthree. This song will be a part of the new full-lengh album after six years of silence to be released on COD label in COD noizes series. Filmed by AE Fox.
-
Мы рады…
-
Мы рады…
Pepemedia 🇺🇦🏳️🌈 via @nowplaybot
Matt Elliott – Hating the Player, Hating the Game
Idk who is this, but the vocal sounds gorgeous in a headphones
Forwarded from Попячечная 🌚
Из коментов вынесу:
This sparked an interesting memory for me. I was once working with a customer who was producing on-board software for a missile. In my analysis of the code, I pointed out that they had a number of problems with storage leaks. Imagine my surprise when the customers chief software engineer said "Of course it leaks". He went on to point out that they had calculated the amount of memory the application would leak in the total possible flight time for the missile and then doubled that number. They added this much additional memory to the hardware to "support" the leaks. Since the missile will explode when it hits its target or at the end of its flight, the ultimate in garbage collection is performed without programmer intervention.https://youtu.be/K3W61Ae0M9g this is so damn beautiful
YouTube
Dear Nia
This is not a good summation of the many-worlds interpretation of quantum mechanics. This is what happens when you stay up reading about it while imbibing moderate-to-inadvisable amounts of beer. (I’m sure Nia and Soren were fine in the end. (More stuff below.))…
Pepemedia 🇺🇦🏳️🌈
Photo
New original meme for y'all, in response to:
(https://news.1rj.ru/str/atellschannel/4736)
(https://news.1rj.ru/str/atellschannel/4736)
Telegram
киберпсалмы & mathюки
"With GPT-3 costing around $4.6 million in compute, than would put a price of $8.6 billion for the compute to train "GPT-4".
If making bigger models was so easy with parameter partitioning from a memory point of view then this seems like the hardest challenge, but you do need to solve the memory issue to actually get it to load at all.
However, if you're lucky you can get 3-6x compute increase from Nvidia A100s over V100s, https://developer.nvidia.com/blog/nvidia-ampere-architecture-in-depth/
But even a 6x compute gain would still put the cost at $1.4 billion.
Nvidia only reported $1.15 billion in revenue from "Data Center" in 2020 Q1, so just to train "GPT-4" you would pretty much need the entire world's supply of graphic cards for 1 quarter (3 months), at least on that order of magnitude."
Looks like where will be no significantly larger model in GPT4 for us :C
source - https://www.reddit.com/r/MachineLearning/comments/i49jf8/d_biggest_roadblock_in_making_gpt4_a_20_trillion/
If making bigger models was so easy with parameter partitioning from a memory point of view then this seems like the hardest challenge, but you do need to solve the memory issue to actually get it to load at all.
However, if you're lucky you can get 3-6x compute increase from Nvidia A100s over V100s, https://developer.nvidia.com/blog/nvidia-ampere-architecture-in-depth/
But even a 6x compute gain would still put the cost at $1.4 billion.
Nvidia only reported $1.15 billion in revenue from "Data Center" in 2020 Q1, so just to train "GPT-4" you would pretty much need the entire world's supply of graphic cards for 1 quarter (3 months), at least on that order of magnitude."
Looks like where will be no significantly larger model in GPT4 for us :C
source - https://www.reddit.com/r/MachineLearning/comments/i49jf8/d_biggest_roadblock_in_making_gpt4_a_20_trillion/