Forwarded from DataEng
Свет увидела новая распределенная SQL база данных на Rust (учебная): https://github.com/erikgrinaker/toydb
Автор также подробно описал её архитектуру: https://github.com/erikgrinaker/toydb/blob/master/docs/architecture.md
Автор также подробно описал её архитектуру: https://github.com/erikgrinaker/toydb/blob/master/docs/architecture.md
GitHub
GitHub - erikgrinaker/toydb: Distributed SQL database in Rust, written as an educational project
Distributed SQL database in Rust, written as an educational project - erikgrinaker/toydb
Forwarded from Инжиниринг Данных (Dmitry Anoshin)
Dremio будет проводить панель по озерам данных с топовыми спикерами онлайн https://www.dremio.com/subsurface/live?_ga=2.203573072.1949877702.1626636985-1007574036.1626636985
Dremio
Save the Date for Subsurface LIVE 2023!
Great website to explore and learn about Azure in an interactive way.
https://azurecharts.com/learning
https://azurecharts.com/learning
Azurecharts
Azure Learning Explorer
Discover published Azure Learning modules and learning paths for services of your interest
A comprehensive technology-agnostic course on how to architect distributed cloud applications.
YouTube
Architecting Distributed Cloud Applications
A comprehensive technology-agnostic course on how to architect distributed cloud applications. PLEASE SHARE MY CHANNEL WITH OTHERS
Ars Technica: Big tech companies are at war with employees over remote work.
https://arstechnica.com/gadgets/2021/08/vaccines-reopenings-and-worker-revolts-big-techs-contentious-return-to-the-office/
https://arstechnica.com/gadgets/2021/08/vaccines-reopenings-and-worker-revolts-big-techs-contentious-return-to-the-office/
Ars Technica
Big tech companies are at war with employees over remote work
CEOs want workers back at their desks. Employees and the virus have other plans.
I know that "remote vs office" topic is all over the place, but still, what do you think, does commute or other perks worth going to office, or do you miss your colleagues?
Anonymous Poll
34%
Remote
5%
Office
61%
Hybrid
Hudi, Iceberg and Delta Lake: Data Lake Table Formats Compared
https://lakefs.io/hudi-iceberg-and-delta-lake-data-lake-table-formats-compared/
https://lakefs.io/hudi-iceberg-and-delta-lake-data-lake-table-formats-compared/
Git for Data - lakeFS
Hudi vs Iceberg vs Delta Lake: Detailed Comparison
Discover how Hudi, Iceberg, and Delta Lake compare in data lake table formats, focusing on performance, scalability, updates, and platform compatibility.
Above and many other interesting news you can find from this post.
Telegram
Инжиниринг Данных
Привет! Вероятно тебе будет интересно запостить
https://blog.jetbrains.com/big-data-tools/2021/08/03/data-engineering-annotated-monthly-july-2021/
https://blog.jetbrains.com/big-data-tools/2021/08/03/data-engineering-annotated-monthly-july-2021/
This one worth highlighting too. What can be neater than architecture design as (Python) code 😎
Mingrammer
Examples · Diagrams
Here are some more examples.
dataeng2021.jpg
242.2 KB
Data engineering in 2021 by datastack.tv (reference)
If you work in one of these areas:
- Back-end development
- Data analytics
- Data science
- DevOps
and you are based in Armenia and want to become a data engineer, reach out to me (check channel denoscription).
- Back-end development
- Data analytics
- Data science
- DevOps
and you are based in Armenia and want to become a data engineer, reach out to me (check channel denoscription).
After sort and distribution keys, now also column compression encoding can be optimized automagically 😉
Amazon
Amazon Redshift announces Automatic Table Optimization
Came across this comparison while reading Firebolt white paper. Not sure why authors used a Redshift cluster which is much larger than 1TB dataset, maybe to add more computational power. Anyway, don't get scared by these clusters' prices.
Fivetran
Cloud Data Warehouse Benchmark | Blog | Fivetran
Our newest benchmark compares price, performance and differentiated features for Redshift, Snowflake, BigQuery, Databricks and Synapse.
It seems these days it's all about Ops, DevOps, MLOps, DevSecOps and now we have Ops for data, DataOps 😎. And yes, this term can be heard more recently, but not sure if this is something new. I guess data engineers were already covering these aspects.
https://www.linkedin.com/posts/firebolt_how-vimeo-keeps-data-intact-with-85-billion-activity-6833790832726810624-Sdjs
https://www.linkedin.com/posts/firebolt_how-vimeo-keeps-data-intact-with-85-billion-activity-6833790832726810624-Sdjs
Linkedin
Firebolt on LinkedIn: How Vimeo Keeps Data Intact with 85 Billion Events Per Month
WHEN IT'S CLEAR YOU NEED A DATA OPS TEAM
Lior Solomon, VP Data Engineering at Vimeo shares his own experience on The Data Engineering Show:
* What made him…
Lior Solomon, VP Data Engineering at Vimeo shares his own experience on The Data Engineering Show:
* What made him…