#AWS Dev Day about Databases and Analytics
#Kafka #EMR #Redshift
https://pages.awscloud.com/emea_dba_devday.html
#Kafka #EMR #Redshift
https://pages.awscloud.com/emea_dba_devday.html
Amazon
AWS Support and Customer Service Contact Info | Amazon Web Services
On this page, you’ll find info regarding the different ways to get in touch with AWS support, including Sales, Technical, Compliance, and Login support.
Interesting website for remote job opportunities.
Հետաքրքիր կայք հեռակա աշխատանքի համար։
#remotework
https://www.remotefit.io/
Հետաքրքիր կայք հեռակա աշխատանքի համար։
#remotework
https://www.remotefit.io/
www.remotefit.io
Remotefit: Find a remote job with a great culture fit
Find a remote job in engineering, product, design, management, sales, or customer service, and work from anywhere.
If you still do not know what а columnar data format is you should read this article.
#Parquet #Arrow
https://towardsdatascience.com/apache-arrow-read-dataframe-with-zero-memory-69634092b1a
#Parquet #Arrow
https://towardsdatascience.com/apache-arrow-read-dataframe-with-zero-memory-69634092b1a
Medium
Apache Arrow: Read DataFrame With Zero Memory
Theoretical & practical introduction to Arrow file format
A good summary of Kinesis Data Streams
Մեր հայրենակցի կողմից 😉
https://dev.solita.fi/2020/05/28/kinesis-streams-part-1.html
Մեր հայրենակցի կողմից 😉
https://dev.solita.fi/2020/05/28/kinesis-streams-part-1.html
/dev/solita
Mastering AWS Kinesis Data Streams, Part 1
I have been working with AWS Kinesis Data Streams for several years now, dealing with over 0.5TB of streaming data per day. Rather than telling you about all the reasons why you should use Kinesis Data Streams (plenty is written on that subject), I'll talk…
So #python runtime is overall fastest for #AWS #Lambda. Didn't see it coming.
https://medium.com/the-theam-journey/benchmarking-aws-lambda-runtimes-in-2019-part-i-b1ee459a293d
https://medium.com/the-theam-journey/benchmarking-aws-lambda-runtimes-in-2019-part-ii-50e796d3d11b
https://medium.com/the-theam-journey/benchmarking-aws-lambda-runtimes-in-2019-part-i-b1ee459a293d
https://medium.com/the-theam-journey/benchmarking-aws-lambda-runtimes-in-2019-part-ii-50e796d3d11b
Medium
Benchmarking AWS Lambda runtimes in 2019 (part I)
Have you ever wondered whether your AWS Lambda could be faster if you used a different runtime?
Here is a great resource for getting familiar with AWS Data Analytics solutions even if you are not planning to take exam. It is very interactive and visually pleasing course.
https://www.aws.training/Details/eLearning?id=46612
https://www.aws.training/Details/eLearning?id=46612
Forwarded from DataEng
CAP теорема для дата инженеров: https://www.analyticsvidhya.com/blog/2020/08/a-beginners-guide-to-cap-theorem-for-data-engineering/
Analytics Vidhya
A Beginner's Guide to CAP Theorem for Data Engineering
CAP theorem helps to handle your distributed database systems when a few database servers refuse to communicate with each other.
If you noticed, recently I was not actively posting in the channel because I was busy preparing for an exam. I want to share this guide if you are considering to pass it as well.
https://towardsdatascience.com/becoming-an-aws-certified-data-analytics-new-april-2020-4a3ef0d9f23a?gi=b2f48e1e3986
https://towardsdatascience.com/becoming-an-aws-certified-data-analytics-new-april-2020-4a3ef0d9f23a?gi=b2f48e1e3986
Medium
Becoming an AWS Certified Data Analytics — NEW April 2020
A guide to become the next AWS Certified Data Analytics expert.
Interesting new feature rich data warehouse with unusual pricing approach base on AWS. Especially I like dedup feature.
https://www.dataengineeringpodcast.com/firebolt-cloud-data-warehouse-episode-148/
https://www.firebolt.io/
https://www.dataengineeringpodcast.com/firebolt-cloud-data-warehouse-episode-148/
https://www.firebolt.io/
Data Engineering Podcast
Building A Better Data Warehouse For The Cloud At Firebolt - Episode 148
Data warehouse technology has been around for decades and has gone through several generational shifts in that time. The current trends in data warehousing are oriented around cloud native architectures that take advantage of dynamic scaling and the separation…
Forwarded from DataEng
Карта навыков современного дата инженера: https://github.com/datastacktv/data-engineer-roadmap
Неплохо дополняет мою статью: https://khashtamov.com/ru/data-engineer/
Неплохо дополняет мою статью: https://khashtamov.com/ru/data-engineer/
GitHub
GitHub - datastacktv/data-engineer-roadmap: Roadmap to becoming a data engineer in 2021
Roadmap to becoming a data engineer in 2021. Contribute to datastacktv/data-engineer-roadmap development by creating an account on GitHub.
Forwarded from DataEng
В Amazon Redshift стал доступен функционал работы с БД поверх HTTPS: https://aws.amazon.com/ru/about-aws/whats-new/2020/09/announcing-data-api-for-amazon-redshift/
Amazon
Announcing Data API for Amazon Redshift
AWS Glue now supports serverless streaming ETL
https://aws.amazon.com/about-aws/whats-new/2020/04/aws-glue-now-supports-serverless-streaming-etl/
https://aws.amazon.com/about-aws/whats-new/2020/04/aws-glue-now-supports-serverless-streaming-etl/
Amazon
AWS Glue now supports serverless streaming ETL
You may find above updates from AWS random and not important but there is a general tendency for me. Basically you can do one thing in many ways and there is better integration between different ETL types: streaming, batch and CDC. However it is hard sometimes to choose between services. #AWS
Data Quality Implementation in Data Warehouses | Toptal
https://www.toptal.com/database/data-warehouse-data-quality-process?utm_campaign=Toptal%20Engineering%20Blog&utm_medium=email&_hsmi=94506066&_hsenc=p2ANqtz-96vNKU-BhpAA5fMC-8gJ3pDBq23ob6VF1lvqOxgYVoATaLUYMQexXEDzJN9c-dFoeH5APIz07aa8hrA2PL9wlSXo1PA62qBFqdkzcTTTOV4AIoWbE&utm_content=94506066&utm_source=hs_email
https://www.toptal.com/database/data-warehouse-data-quality-process?utm_campaign=Toptal%20Engineering%20Blog&utm_medium=email&_hsmi=94506066&_hsenc=p2ANqtz-96vNKU-BhpAA5fMC-8gJ3pDBq23ob6VF1lvqOxgYVoATaLUYMQexXEDzJN9c-dFoeH5APIz07aa8hrA2PL9wlSXo1PA62qBFqdkzcTTTOV4AIoWbE&utm_content=94506066&utm_source=hs_email
Toptal Engineering Blog
Data Quality Implementation in Data Warehouses
Data quality is a crucial element of any successful data warehouse solution. As the complexity of data warehouses increases, so does the need for data quality processes.
Very useful and relevant blog post about data deletion in a data lake. Besides suggested solution I would like to mention also using Delta Lake as alternative. And finally, it would be great if the author has mentioned cost considerations .
https://aws.amazon.com/blogs/big-data/how-to-delete-user-data-in-an-aws-data-lake/
https://aws.amazon.com/blogs/big-data/how-to-delete-user-data-in-an-aws-data-lake/
Amazon
How to delete user data in an AWS data lake | Amazon Web Services
General Data Protection Regulation (GDPR) is an important aspect of today’s technology world, and processing data in compliance with GDPR is a necessity for those who implement solutions within the AWS public cloud. One article of GDPR is the “right to erasure”…
Amazing statistics about data.
https://www.datanami.com/2020/09/04/10-big-data-statistics-that-will-blow-your-mind/?utm_source=rss&utm_medium=rss&utm_campaign=10-big-data-statistics-that-will-blow-your-mind
https://www.datanami.com/2020/09/04/10-big-data-statistics-that-will-blow-your-mind/?utm_source=rss&utm_medium=rss&utm_campaign=10-big-data-statistics-that-will-blow-your-mind
Datanami
10 Big Data Statistics That Will Blow Your Mind
They call it “big data” for a reason--it's really, really big. But getting your head wrapped around the growth of information digitization is not easy.