The Data Lake paradigm is often considered the scalable successor of the more curated Data Warehouse approach when it comes to democratization of data. However, many who went out to build a centralized Data Lake came out with a data swamp of unclear responsibilities…

Take advantage of today's sky-high demand for data engi…

Which analytical platform to choose?

See why enterprises are embracing a cloud-everywhere strategy with HPE GreenLake. Join HPE President and CEO Antonio Neri for a special broadcast on April 4. Learn about the next set of cloud services launching on HPE GreenLake edge-to-cloud platform.

JUST ENOUGH AZURE DATA EXPLORER FOR CLOUD ARCHITECTS This article provides a high-level overview of all the key capabilities in Azure Data Explorer, positioning, and resources, and is intended to demystify Azure Data Explorer for cloud architects.  WHAT…

Your genetic code is probably already in a database, without you ever giving a sample or permission. This video is sponsored by Brilliant. The first 200 people to sign up via https://brilliant.org/veritasium get 20% off a yearly subnoscription.

▀▀▀▀▀▀▀▀▀…

Just what was it that took Facebook, Instagram & WhatsApp offline on 4th October 2021? - Dr Steve Bagley investigates!

previously noscriptd "Facebook's Day Off"

Facebook announcement on the outage: https://bit.ly/C_FB-Outage
The visualisation software Steve…

A curated list of emerging Technologies in Data Engineering.

For many modern data scientists, Python is the programming language that was used for their everyday work — as a consequence, the data analysis would be done using one of the most data packages…

Streamlit is an open-source Python framework for data scientists and AI/ML engineers to deliver interactive data apps – in only a few lines of code.

Digital transformation is a key aspect of the new programming model of intelligent devices across the cloud. One of the key types of data that's arisen from this new application paradigm is telemetry. Telemetry data is everywhere: IoT sensors, app logs, web…

At its recent Ignite conference, Microsoft announced the new Azure OpenAI Service in preview, allowing access to OpenAI’s API through the Azure platform. This new Azure Cognitive Service will give customers access to OpenAI’s powerful GPT-3 models, along…

Nov 2022: Newer version of the product is now available to be used for this post. AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, machine learning (ML), and application development.…

Amazon Redshift is a fully managed, petabyte-scale data warehouse service in the cloud. A star schema is a database organization structure optimized for use in a data warehouse. In a star schema, a dimension is a structure that categorizes the facts and measures…

Learn more about the latest release of Apache Spark™, version 3.2, including pandas API on Spark, Adaptive Query Execution, and ANSI mode and how you can begin using it through Databricks Runtime 10.0.

The Apache Spark team has integrated the Pandas API in the product's latest 3.2 release. With this change, dataframe processing can be scaled to multiple clusters or multiple processors in a single machine using the PySpark execution engine.

Browse functional programming jobs, salaries, blogs and learning resources! Scala jobs, Haskell jobs, Clojure jobs and more.

There are so many data roles out there. Data engineer, analytics engineer, analyst, BI Developer, ETL Developer, Data modeler and so many more.

But what is the difference.

What do all of these data roles do?

In this video we will focus on three roles and…