Top 5 websites where obtain DataSets for your Data Science projects
In the life cycle of any Data science project, it begins by understanding the requirements and objectives of the business problem, in this phase, it is essential to have knowledge about the problem to be solved, ask the right questions and define the...
Read: https://dofdata.hashnode.dev/top-5-websites-where-obtain-datasets-for-your-data-science-projects
In the life cycle of any Data science project, it begins by understanding the requirements and objectives of the business problem, in this phase, it is essential to have knowledge about the problem to be solved, ask the right questions and define the...
Read: https://dofdata.hashnode.dev/top-5-websites-where-obtain-datasets-for-your-data-science-projects
Migrate your Sybase ASE database to Amazon RDS for SQL Server in near-real time
Read: https://aws.amazon.com/blogs/database/migrate-your-sybase-ase-database-to-amazon-rds-for-sql-server-in-near-real-time/
Read: https://aws.amazon.com/blogs/database/migrate-your-sybase-ase-database-to-amazon-rds-for-sql-server-in-near-real-time/
ETL, ELT and Data Pipelines, Data Integration
ETL has historically been used for batch workloads on a large scale. They're being used for real-time streaming as well.
popular ETL tools: IBM Infosphere information server, AWS Glue, Improvado, Skyvia, Hevo, Informatica.
ETL ( Extract Load Transfo...
Read: https://pooj124.hashnode.dev/etl-elt-and-data-pipelines-data-integration
ETL has historically been used for batch workloads on a large scale. They're being used for real-time streaming as well.
popular ETL tools: IBM Infosphere information server, AWS Glue, Improvado, Skyvia, Hevo, Informatica.
ETL ( Extract Load Transfo...
Read: https://pooj124.hashnode.dev/etl-elt-and-data-pipelines-data-integration
Рецепт готовки Apache Kafka: как создавался Data Lake на 80 Тb
Kafka позволит вам грамотно организовать работу с большим массивом данных, но в процессе может подкинуть проблем. Иногда придется устроить танцы с бубном, чтобы программа продолжила работать, а не рухнула в момент запуска.
О не очень стандартном использовании Apache Kafka и лайфхаках по созданию Data Lake на его основе нам рассказал Михаил Кобик, директор департамента инфраструктурных решений в SMART Technologies SOFT. В 2017 году перед командой Михаила встала непростая задачка - создать хранилище данных на 80 Tb. В распоряжении был спек, примерные нагрузки и абсолютное непонимание, что с этим делать со стороны заказчика.
Читать: https://habr.com/ru/post/681070/
Kafka позволит вам грамотно организовать работу с большим массивом данных, но в процессе может подкинуть проблем. Иногда придется устроить танцы с бубном, чтобы программа продолжила работать, а не рухнула в момент запуска.
О не очень стандартном использовании Apache Kafka и лайфхаках по созданию Data Lake на его основе нам рассказал Михаил Кобик, директор департамента инфраструктурных решений в SMART Technologies SOFT. В 2017 году перед командой Михаила встала непростая задачка - создать хранилище данных на 80 Tb. В распоряжении был спек, примерные нагрузки и абсолютное непонимание, что с этим делать со стороны заказчика.
Читать: https://habr.com/ru/post/681070/
Легально ли брать контент из базы данных?
Далеко не все компании и пользователи задумываются о легальности использования чужой базы данных без согласия его владельца. А зря, это может привести к штрафам и судебным разбирательствам. Давайте попробуем разобраться, когда брать информацию из базы данных легально, а когда это является нарушением.
Читать: https://habr.com/ru/post/681090/
Далеко не все компании и пользователи задумываются о легальности использования чужой базы данных без согласия его владельца. А зря, это может привести к штрафам и судебным разбирательствам. Давайте попробуем разобраться, когда брать информацию из базы данных легально, а когда это является нарушением.
Читать: https://habr.com/ru/post/681090/
Creating Polymorphic Relationships using Node.js and Sequelize
In my last article, I explained what are polymorphic relationships. This blogs is a continuation of the same where I will demonstrate how to create a polymorphic relationship in node.js using the popular ORM sequelize.
Sequelize is one of the most po...
Read: https://blogs.aayushkurup.dev/creating-polymorphic-relationships
In my last article, I explained what are polymorphic relationships. This blogs is a continuation of the same where I will demonstrate how to create a polymorphic relationship in node.js using the popular ORM sequelize.
Sequelize is one of the most po...
Read: https://blogs.aayushkurup.dev/creating-polymorphic-relationships
Aayush Kurup's Blog
This blog talks about how create polymorphic relation in node.js.
This blog talks about how create polymorphic relation in node.js and sequelize. It is a very practical and hands on blog about polymporphic relationships.
Everything you can learn about Data Analytics
In the last few years, Data analysis has become one of the most demanded skills in the job market. Let's discuss the several applications of data analytics, and the rate of skill development one has to deal with to have a better understanding and car...
Read: https://saffronclouds.hashnode.dev/everything-you-can-learn-about-data-analytics
In the last few years, Data analysis has become one of the most demanded skills in the job market. Let's discuss the several applications of data analytics, and the rate of skill development one has to deal with to have a better understanding and car...
Read: https://saffronclouds.hashnode.dev/everything-you-can-learn-about-data-analytics
SQL Server & Powershell: быстро, просто, параллельно. И больше не нужно выбирать 2 из 3-х
В настоящей статье описывается процесс параллельного выполнения операций над объектами базы данных Microsoft SQL Server (таблицы, индексы и т.д.) с использованием инструментария Powershell. Наиболее часто используемый сценарий - обслуживание таблиц в базе, но возможно исполнение любых ваших запросов в параллельном режиме. Если для вас это интересно, то...
...добро пожаловать
Читать: https://habr.com/ru/post/681166/
В настоящей статье описывается процесс параллельного выполнения операций над объектами базы данных Microsoft SQL Server (таблицы, индексы и т.д.) с использованием инструментария Powershell. Наиболее часто используемый сценарий - обслуживание таблиц в базе, но возможно исполнение любых ваших запросов в параллельном режиме. Если для вас это интересно, то...
...добро пожаловать
Читать: https://habr.com/ru/post/681166/
👍1
Data Engineering Lifecycle
Architecting the Data Platform
Layers of a data platform architecture, A layer represents functional components that perform a specific set of tasks in the data
Data Ingestion or Data Collection layer
Data Storage and Integration Layer
Data Proces...
Read: https://pooj124.hashnode.dev/data-engineering-lifecycle
Architecting the Data Platform
Layers of a data platform architecture, A layer represents functional components that perform a specific set of tasks in the data
Data Ingestion or Data Collection layer
Data Storage and Integration Layer
Data Proces...
Read: https://pooj124.hashnode.dev/data-engineering-lifecycle
Case Study: Danny's Dinner
Introduction
Danny seriously loves Japanese food so, at the beginning of 2021, he decides to embark upon a risky venture and opens up a cute little restaurant that sells his 3 favourite foods: sushi, curry and ramen.
Danny’s Diner is in need of your ...
Read: https://itsadityagupta.hashnode.dev/case-study-dannys-dinner
Introduction
Danny seriously loves Japanese food so, at the beginning of 2021, he decides to embark upon a risky venture and opens up a cute little restaurant that sells his 3 favourite foods: sushi, curry and ramen.
Danny’s Diner is in need of your ...
Read: https://itsadityagupta.hashnode.dev/case-study-dannys-dinner
Use PostgreSQL the Right Way!
This post on Medium summarizes some situations and configurations I’ve had to adapt to work with the PostgreSQL database; let’s get to the tips.
To learn more:
https://jlgjosue.medium.com/use-postgresql-the-right-way-7d200aa7999f
Thanks for reading, ...
Read: https://josueluzardogebrim.hashnode.dev/use-postgresql-the-right-way
This post on Medium summarizes some situations and configurations I’ve had to adapt to work with the PostgreSQL database; let’s get to the tips.
To learn more:
https://jlgjosue.medium.com/use-postgresql-the-right-way-7d200aa7999f
Thanks for reading, ...
Read: https://josueluzardogebrim.hashnode.dev/use-postgresql-the-right-way
Creating database tables and adding data in Magento 2 Adobe Commerce
Create a db schema
Reference https://devdocs.magento.com/guides/v2.4/extension-dev-guide/declarative-schema/db-schema.html
Path will be a similar path to this
app\code\AzCodez\CustomerViewing\etc\db_schema.xml
Add this code to set up your table. Mo...
Read: https://blog.azcodez.com/creating-database-tables-and-adding-data-in-magento-2-adobe-commerce
Create a db schema
Reference https://devdocs.magento.com/guides/v2.4/extension-dev-guide/declarative-schema/db-schema.html
Path will be a similar path to this
app\code\AzCodez\CustomerViewing\etc\db_schema.xml
Add this code to set up your table. Mo...
Read: https://blog.azcodez.com/creating-database-tables-and-adding-data-in-magento-2-adobe-commerce
Introduction To Database Management Systems
If you want a simple and sweet introduction to databases and database management systems, then you are at the right place, because after reading this blog post you will be fully equipped with a basic understanding of databases and systems to manage t...
Read: https://yuvraj01.hashnode.dev/introduction-to-database-management-systems
If you want a simple and sweet introduction to databases and database management systems, then you are at the right place, because after reading this blog post you will be fully equipped with a basic understanding of databases and systems to manage t...
Read: https://yuvraj01.hashnode.dev/introduction-to-database-management-systems
Big Data Platforms
V's of Big Data :
Velocity: Velocity is the speed at which data accumulates. Data is being generated extremely fast, in a process that never stops. Near or real-time streaming, local, and cloud-based technologies can process information very quickl...
Read: https://pooj124.hashnode.dev/big-data-platforms
V's of Big Data :
Velocity: Velocity is the speed at which data accumulates. Data is being generated extremely fast, in a process that never stops. Near or real-time streaming, local, and cloud-based technologies can process information very quickl...
Read: https://pooj124.hashnode.dev/big-data-platforms
Database Operation from the command line
#Connecting to a Database
sql2csv: is tool in csvkit that allows you to pull data from the database and convert it to csv. It executes an SQL query on a large variety of SQL databases (e.g MS SQL, MySQL, Oracle, PostgreSQL, Sqlite).
Documentation
sql...
Read: https://rukayat-balogun.hashnode.dev/database-operation-from-the-command-line
#Connecting to a Database
sql2csv: is tool in csvkit that allows you to pull data from the database and convert it to csv. It executes an SQL query on a large variety of SQL databases (e.g MS SQL, MySQL, Oracle, PostgreSQL, Sqlite).
Documentation
sql...
Read: https://rukayat-balogun.hashnode.dev/database-operation-from-the-command-line
Duplicates in SQL
Duplicate records waste resources in DB and often introduce errors in computation. In this blog we will see how to check for duplicates and how to deal with them.
Identifying the duplicate entries
The first step should be to define which column or co...
Read: https://asrathore.hashnode.dev/duplicates-in-sql
Duplicate records waste resources in DB and often introduce errors in computation. In this blog we will see how to check for duplicates and how to deal with them.
Identifying the duplicate entries
The first step should be to define which column or co...
Read: https://asrathore.hashnode.dev/duplicates-in-sql
Amit Kumar's blog
Duplicates in SQL
All you need to know for handling duplicates in SQL.
👍2
Хранение и управление доступом на данные в хранилище S3
Привет! Сегодня хотим рассказать о, как использовать бакеты, хранить данные, настраивать политики и управлять доступом на данные при работе c объектным хранилищем S3.
Читать: https://habr.com/ru/post/681376/
Привет! Сегодня хотим рассказать о, как использовать бакеты, хранить данные, настраивать политики и управлять доступом на данные при работе c объектным хранилищем S3.
Читать: https://habr.com/ru/post/681376/
Database Design: From Novice to Professional [Ch.2: Guided Tour of the Development Process]
In the previous chapter we talked about how bad design can lead to problems in the future
in the two incoming chapters we will talk about the process you can follow that may help you in designing your database.
before we start we need to know that f...
Read: https://3omar.hashnode.dev/database-design-from-novice-to-professional-ch2-guided-tour-of-the-development-process
In the previous chapter we talked about how bad design can lead to problems in the future
in the two incoming chapters we will talk about the process you can follow that may help you in designing your database.
before we start we need to know that f...
Read: https://3omar.hashnode.dev/database-design-from-novice-to-professional-ch2-guided-tour-of-the-development-process
Database Design: From Novice to Professional [Ch.3: Initial Requirements and Use Cases]
In this chapter we will foucs on the Analysis part but in the abstract world focus on what the user can do in our system.
Remember the 4 process thinking, Now we consider part of the first step from real-world problem to eventual real-world solution ...
Read: https://3omar.hashnode.dev/database-design-from-novice-to-professional-ch3-initial-requirements-and-use-cases
In this chapter we will foucs on the Analysis part but in the abstract world focus on what the user can do in our system.
Remember the 4 process thinking, Now we consider part of the first step from real-world problem to eventual real-world solution ...
Read: https://3omar.hashnode.dev/database-design-from-novice-to-professional-ch3-initial-requirements-and-use-cases
Data Engineering - Performance Tuning and Troubleshooting
Data Pipelines - Performance Threats
Scalability in the face of increasing datasets and workloads
Application failures
Scheduled jobs not functioning accurately
Tool Incompatibilities
Data Pipelines - Performance Metrics
Latency: time taken for se...
Read: https://pooj124.hashnode.dev/data-engineering-performance-tuning-and-troubleshooting
Data Pipelines - Performance Threats
Scalability in the face of increasing datasets and workloads
Application failures
Scheduled jobs not functioning accurately
Tool Incompatibilities
Data Pipelines - Performance Metrics
Latency: time taken for se...
Read: https://pooj124.hashnode.dev/data-engineering-performance-tuning-and-troubleshooting
4 Critical Features for a Modern Payments System
Read: https://www.mongodb.com/blog/post/critical-features-modern-payments-system
Read: https://www.mongodb.com/blog/post/critical-features-modern-payments-system