Great website to explore and learn about Azure in an interactive way.
https://azurecharts.com/learning
https://azurecharts.com/learning
Azurecharts
Azure Learning Explorer
Discover published Azure Learning modules and learning paths for services of your interest
A comprehensive technology-agnostic course on how to architect distributed cloud applications.
YouTube
Architecting Distributed Cloud Applications
A comprehensive technology-agnostic course on how to architect distributed cloud applications. PLEASE SHARE MY CHANNEL WITH OTHERS
Ars Technica: Big tech companies are at war with employees over remote work.
https://arstechnica.com/gadgets/2021/08/vaccines-reopenings-and-worker-revolts-big-techs-contentious-return-to-the-office/
https://arstechnica.com/gadgets/2021/08/vaccines-reopenings-and-worker-revolts-big-techs-contentious-return-to-the-office/
Ars Technica
Big tech companies are at war with employees over remote work
CEOs want workers back at their desks. Employees and the virus have other plans.
I know that "remote vs office" topic is all over the place, but still, what do you think, does commute or other perks worth going to office, or do you miss your colleagues?
Anonymous Poll
34%
Remote
5%
Office
61%
Hybrid
Hudi, Iceberg and Delta Lake: Data Lake Table Formats Compared
https://lakefs.io/hudi-iceberg-and-delta-lake-data-lake-table-formats-compared/
https://lakefs.io/hudi-iceberg-and-delta-lake-data-lake-table-formats-compared/
Git for Data - lakeFS
Hudi vs Iceberg vs Delta Lake: Detailed Comparison
Discover how Hudi, Iceberg, and Delta Lake compare in data lake table formats, focusing on performance, scalability, updates, and platform compatibility.
Above and many other interesting news you can find from this post.
Telegram
Инжиниринг Данных
Привет! Вероятно тебе будет интересно запостить
https://blog.jetbrains.com/big-data-tools/2021/08/03/data-engineering-annotated-monthly-july-2021/
https://blog.jetbrains.com/big-data-tools/2021/08/03/data-engineering-annotated-monthly-july-2021/
This one worth highlighting too. What can be neater than architecture design as (Python) code 😎
Mingrammer
Examples · Diagrams
Here are some more examples.
dataeng2021.jpg
242.2 KB
Data engineering in 2021 by datastack.tv (reference)
If you work in one of these areas:
- Back-end development
- Data analytics
- Data science
- DevOps
and you are based in Armenia and want to become a data engineer, reach out to me (check channel denoscription).
- Back-end development
- Data analytics
- Data science
- DevOps
and you are based in Armenia and want to become a data engineer, reach out to me (check channel denoscription).
After sort and distribution keys, now also column compression encoding can be optimized automagically 😉
Amazon
Amazon Redshift announces Automatic Table Optimization
Came across this comparison while reading Firebolt white paper. Not sure why authors used a Redshift cluster which is much larger than 1TB dataset, maybe to add more computational power. Anyway, don't get scared by these clusters' prices.
Fivetran
Cloud Data Warehouse Benchmark | Blog | Fivetran
Our newest benchmark compares price, performance and differentiated features for Redshift, Snowflake, BigQuery, Databricks and Synapse.
It seems these days it's all about Ops, DevOps, MLOps, DevSecOps and now we have Ops for data, DataOps 😎. And yes, this term can be heard more recently, but not sure if this is something new. I guess data engineers were already covering these aspects.
https://www.linkedin.com/posts/firebolt_how-vimeo-keeps-data-intact-with-85-billion-activity-6833790832726810624-Sdjs
https://www.linkedin.com/posts/firebolt_how-vimeo-keeps-data-intact-with-85-billion-activity-6833790832726810624-Sdjs
Linkedin
Firebolt on LinkedIn: How Vimeo Keeps Data Intact with 85 Billion Events Per Month
WHEN IT'S CLEAR YOU NEED A DATA OPS TEAM
Lior Solomon, VP Data Engineering at Vimeo shares his own experience on The Data Engineering Show:
* What made him…
Lior Solomon, VP Data Engineering at Vimeo shares his own experience on The Data Engineering Show:
* What made him…
In data engineering landscape there are always new interesting project. And sometimes only way you hear about them is by talking to other data professionals. So here are two cool projects I learned about from a friend:
1. Presidio: Data protection and anonymization library from Microsoft
2. Trino: a new query engine from creators of Presto
1. Presidio: Data protection and anonymization library from Microsoft
2. Trino: a new query engine from creators of Presto
microsoft.github.io
Home - Microsoft Presidio
PII anonymization for text, images, and structured data.
It seems one tool, dbt, is driving demand for new analytics engineer specialization. Spark is popular too,and often considered as main tool for data engineers, but it did not create a specialization.
Medium
Analytics Engineer: The Newest Data Career Role
Is it analytics? Is it engineering? What do these people do?
Data Engineering Annotated Monthly – August 2021 | The Big Data Tools Blog
https://blog.jetbrains.com/big-data-tools/2021/09/06/data-engineering-annotated-monthly-august-2021/
https://blog.jetbrains.com/big-data-tools/2021/09/06/data-engineering-annotated-monthly-august-2021/
The JetBrains Blog
Data Engineering Annotated Monthly – August 2021 | The Big Data Tools Blog
August is usually a quiet month, with vacations taking their toll. But data engineering never stops. I’m Pasha Finkelshteyn and I will be your guide through this month’s news, my impressions of the de