Data Science Interview Preparation Book 👇👇
https://www.instagram.com/reel/C2fN6c7Nb_G/?igsh=ZzUzZ2lmOWhxY2c5
https://www.instagram.com/reel/C2fN6c7Nb_G/?igsh=ZzUzZ2lmOWhxY2c5
👍4
Comment "Excel" to get this excel step by step guide 👇
https://www.instagram.com/reel/C2h2GJDtU0q/?igsh=MThzenYyaGh1OHE2YQ==
https://www.instagram.com/reel/C2h2GJDtU0q/?igsh=MThzenYyaGh1OHE2YQ==
👍9
Prompt Engineering in itself does not warrant a separate job.
Most of the things you see online related to prompts (especially things said by people selling courses) is mostly just writing some crazy text to get ChatGPT to do some specific task. Most of these prompts are just been found by serendipity and are never used in any company. They may be fine for personal usage but no company is going to pay a person to try out prompts 😅. Also a lot of these prompts don't work for any other LLMs apart from ChatGPT.
You have mostly two types of jobs in this field nowadays, one is more focused on training, optimizing and deploying models. For this knowing the architecture of LLMs is critical and a strong background in PyTorch, Jax and HuggingFace is required. Other engineering skills like System Design and building APIs is also important for some jobs. This is the work you would find in companies like OpenAI, Anthropic, Cohere etc.
The other is jobs where you build applications using LLMs (this comprises of majority of the companies that do LLM related work nowadays, both product based and service based). Roles in these companies are called Applied NLP Engineer or ML Engineer, sometimes even Data Scientist roles. For this you mostly need to understand how LLMs can be used for different applications as well as know the necessary frameworks for building LLM applications (Langchain/LlamaIndex/Haystack). Apart from this, you need to know LLM specific techniques for applications like Vector Search, RAG, Structured Text Generation. This is also where some part of your role involves prompt engineering. Its not the most crucial bit, but it is important in some cases, especially when you are limited in the other techniques.
Most of the things you see online related to prompts (especially things said by people selling courses) is mostly just writing some crazy text to get ChatGPT to do some specific task. Most of these prompts are just been found by serendipity and are never used in any company. They may be fine for personal usage but no company is going to pay a person to try out prompts 😅. Also a lot of these prompts don't work for any other LLMs apart from ChatGPT.
You have mostly two types of jobs in this field nowadays, one is more focused on training, optimizing and deploying models. For this knowing the architecture of LLMs is critical and a strong background in PyTorch, Jax and HuggingFace is required. Other engineering skills like System Design and building APIs is also important for some jobs. This is the work you would find in companies like OpenAI, Anthropic, Cohere etc.
The other is jobs where you build applications using LLMs (this comprises of majority of the companies that do LLM related work nowadays, both product based and service based). Roles in these companies are called Applied NLP Engineer or ML Engineer, sometimes even Data Scientist roles. For this you mostly need to understand how LLMs can be used for different applications as well as know the necessary frameworks for building LLM applications (Langchain/LlamaIndex/Haystack). Apart from this, you need to know LLM specific techniques for applications like Vector Search, RAG, Structured Text Generation. This is also where some part of your role involves prompt engineering. Its not the most crucial bit, but it is important in some cases, especially when you are limited in the other techniques.
👍27❤1
Popular Python packages for data science:
1. NumPy: For numerical operations and working with arrays.
2. Pandas: For data manipulation and analysis, especially with data frames.
3. Matplotlib and Seaborn: For data visualization.
4. Scikit-learn: For machine learning algorithms and tools.
5. TensorFlow and PyTorch: Deep learning frameworks.
6. SciPy: For scientific and technical computing.
7. Statsmodels: For statistical modeling and hypothesis testing.
8. NLTK and SpaCy: Natural Language Processing libraries.
9. Jupyter Notebooks: Interactive computing and data visualization.
10. Bokeh and Plotly: Additional libraries for interactive visualizations.
1. NumPy: For numerical operations and working with arrays.
2. Pandas: For data manipulation and analysis, especially with data frames.
3. Matplotlib and Seaborn: For data visualization.
4. Scikit-learn: For machine learning algorithms and tools.
5. TensorFlow and PyTorch: Deep learning frameworks.
6. SciPy: For scientific and technical computing.
7. Statsmodels: For statistical modeling and hypothesis testing.
8. NLTK and SpaCy: Natural Language Processing libraries.
9. Jupyter Notebooks: Interactive computing and data visualization.
10. Bokeh and Plotly: Additional libraries for interactive visualizations.
👍39
Understanding Bias and Variance in Machine Learning
Bias refers to the error in the model when the model is not able to capture the pattern in the data and what results is an underfit model (High Bias).
Variance refers to the error in the model, when the model is too much tailored to the training data and fails to generalise for unseen data which refers to an overfit model (High Variance)
There should be a tradeoff between bias and variance. An optimal model should have Low Bias and Low Variance so as to avoid underfitting and overfitting.
Techniques like cross validation can be helpful in these cases.
➖➖➖➖➖➖➖➖➖➖➖➖➖➖
Bias refers to the error in the model when the model is not able to capture the pattern in the data and what results is an underfit model (High Bias).
Variance refers to the error in the model, when the model is too much tailored to the training data and fails to generalise for unseen data which refers to an overfit model (High Variance)
There should be a tradeoff between bias and variance. An optimal model should have Low Bias and Low Variance so as to avoid underfitting and overfitting.
Techniques like cross validation can be helpful in these cases.
➖➖➖➖➖➖➖➖➖➖➖➖➖➖
👍34❤1
✅ Best Telegram channels to get free coding & data science resources
https://news.1rj.ru/str/addlist/V3itvQONC4BlZTU5
✅ Free Courses with Certificate:
https://news.1rj.ru/str/free4unow_backup
https://news.1rj.ru/str/addlist/V3itvQONC4BlZTU5
✅ Free Courses with Certificate:
https://news.1rj.ru/str/free4unow_backup
👍5
Top 10 essential data science terminologies
1. Machine Learning: A subset of artificial intelligence that involves building algorithms that can learn from and make predictions or decisions based on data.
2. Big Data: Extremely large datasets that require specialized tools and techniques to analyze and extract insights from.
3. Data Mining: The process of discovering patterns, trends, and insights in large datasets using various methods such as machine learning and statistical analysis.
4. Predictive Analytics: The use of statistical algorithms and machine learning techniques to predict future outcomes based on historical data.
5. Natural Language Processing (NLP): The field of study that focuses on enabling computers to understand, interpret, and generate human language.
6. Neural Networks: A type of machine learning model inspired by the structure and function of the human brain, consisting of interconnected nodes that can learn from data.
7. Feature Engineering: The process of selecting, transforming, and creating new features from raw data to improve the performance of machine learning models.
8. Data Visualization: The graphical representation of data to help users understand and interpret complex datasets more easily.
9. Deep Learning: A subset of machine learning that uses neural networks with multiple layers to learn complex patterns in data.
10. Ensemble Learning: A technique that combines multiple machine learning models to improve predictive performance and reduce overfitting.
Credits: https://news.1rj.ru/str/datasciencefree
ENJOY LEARNING 👍👍
1. Machine Learning: A subset of artificial intelligence that involves building algorithms that can learn from and make predictions or decisions based on data.
2. Big Data: Extremely large datasets that require specialized tools and techniques to analyze and extract insights from.
3. Data Mining: The process of discovering patterns, trends, and insights in large datasets using various methods such as machine learning and statistical analysis.
4. Predictive Analytics: The use of statistical algorithms and machine learning techniques to predict future outcomes based on historical data.
5. Natural Language Processing (NLP): The field of study that focuses on enabling computers to understand, interpret, and generate human language.
6. Neural Networks: A type of machine learning model inspired by the structure and function of the human brain, consisting of interconnected nodes that can learn from data.
7. Feature Engineering: The process of selecting, transforming, and creating new features from raw data to improve the performance of machine learning models.
8. Data Visualization: The graphical representation of data to help users understand and interpret complex datasets more easily.
9. Deep Learning: A subset of machine learning that uses neural networks with multiple layers to learn complex patterns in data.
10. Ensemble Learning: A technique that combines multiple machine learning models to improve predictive performance and reduce overfitting.
Credits: https://news.1rj.ru/str/datasciencefree
ENJOY LEARNING 👍👍
👍21❤1
👍6
✅ Best Telegram channels to get free coding & data science resources
https://news.1rj.ru/str/addlist/ID95piZJZa0wYzk5
✅ Free Courses with Certificate:
https://news.1rj.ru/str/free4unow_backup
https://news.1rj.ru/str/addlist/ID95piZJZa0wYzk5
✅ Free Courses with Certificate:
https://news.1rj.ru/str/free4unow_backup
👍4❤1
Why is it require to split our data into three parts: train, validation, and test?
• The training set is used to fit the model, i.e. to train the model with the data.
• The validation set is then used to provide an unbiased evaluation of a model while fine-tuning hyperparameters. This improves the generalization of the model.
• Finally, a test data set which the model has never "seen" before should be used for the final evaluation of the model. This allows for an unbiased evaluation of the model. The evaluation should never be performed on the same data that is used for training. Otherwise the model performance would not be representative.
• The training set is used to fit the model, i.e. to train the model with the data.
• The validation set is then used to provide an unbiased evaluation of a model while fine-tuning hyperparameters. This improves the generalization of the model.
• Finally, a test data set which the model has never "seen" before should be used for the final evaluation of the model. This allows for an unbiased evaluation of the model. The evaluation should never be performed on the same data that is used for training. Otherwise the model performance would not be representative.
👍15👎1
What are the main assumptions of linear regression?
There are several assumptions of linear regression. If any of them is violated, model predictions and interpretation may be worthless or misleading.
1) Linear relationship between features and target variable.
2) Additivity means that the effect of changes in one of the features on the target variable does not depend on values of other features. For example, a model for predicting revenue of a company have of two features - the number of items a sold and the number of items b sold. When company sells more items a the revenue increases and this is independent of the number of items b sold. But, if customers who buy a stop buying b, the additivity assumption is violated.
3) Features are not correlated (no collinearity) since it can be difficult to separate out the individual effects of collinear features on the target variable.
4) Errors are independently and identically normally distributed (yi = B0 + B1*x1i + ... + errori):
i) No correlation between errors (consecutive errors in the case of time series data).
ii) Constant variance of errors - homoscedasticity. For example, in case of time series, seasonal patterns can increase errors in seasons with higher activity.
iii) Errors are normaly distributed, otherwise some features will have more influence on the target variable than to others. If the error distribution is significantly non-normal, confidence intervals may be too wide or too narrow.
There are several assumptions of linear regression. If any of them is violated, model predictions and interpretation may be worthless or misleading.
1) Linear relationship between features and target variable.
2) Additivity means that the effect of changes in one of the features on the target variable does not depend on values of other features. For example, a model for predicting revenue of a company have of two features - the number of items a sold and the number of items b sold. When company sells more items a the revenue increases and this is independent of the number of items b sold. But, if customers who buy a stop buying b, the additivity assumption is violated.
3) Features are not correlated (no collinearity) since it can be difficult to separate out the individual effects of collinear features on the target variable.
4) Errors are independently and identically normally distributed (yi = B0 + B1*x1i + ... + errori):
i) No correlation between errors (consecutive errors in the case of time series data).
ii) Constant variance of errors - homoscedasticity. For example, in case of time series, seasonal patterns can increase errors in seasons with higher activity.
iii) Errors are normaly distributed, otherwise some features will have more influence on the target variable than to others. If the error distribution is significantly non-normal, confidence intervals may be too wide or too narrow.
👍20
🔐"Key Python Libraries for Data Science:
Numpy: Core for numerical operations and array handling.
SciPy: Complements Numpy with scientific computing features like optimization.
Pandas: Crucial for data manipulation, offering powerful DataFrames.
Matplotlib: Versatile plotting library for creating various visualizations.
Keras: High-level neural networks API for quick deep learning prototyping.
TensorFlow: Popular open-source ML framework for building and training models.
Scikit-learn: Efficient tools for data mining and statistical modeling.
Seaborn: Enhances data visualization with appealing statistical graphics.
Statsmodels: Focuses on estimating and testing statistical models.
NLTK: Library for working with human language data.
These libraries empower data scientists across tasks, from preprocessing to advanced machine learning."
Numpy: Core for numerical operations and array handling.
SciPy: Complements Numpy with scientific computing features like optimization.
Pandas: Crucial for data manipulation, offering powerful DataFrames.
Matplotlib: Versatile plotting library for creating various visualizations.
Keras: High-level neural networks API for quick deep learning prototyping.
TensorFlow: Popular open-source ML framework for building and training models.
Scikit-learn: Efficient tools for data mining and statistical modeling.
Seaborn: Enhances data visualization with appealing statistical graphics.
Statsmodels: Focuses on estimating and testing statistical models.
NLTK: Library for working with human language data.
These libraries empower data scientists across tasks, from preprocessing to advanced machine learning."
👍24
Top 5 data science concepts 👇
1. Machine Learning: Machine learning is a subset of artificial intelligence that focuses on developing algorithms and models that can learn from and make predictions or decisions based on data. It involves techniques such as supervised learning, unsupervised learning, and reinforcement learning to analyze and interpret patterns in data.
2. Data Visualization: Data visualization is the graphical representation of data to help users understand complex datasets and identify trends, patterns, and insights. It involves creating visualizations such as charts, graphs, maps, and dashboards to communicate data effectively and facilitate data-driven decision-making.
3. Statistical Analysis: Statistical analysis is the process of collecting, exploring, analyzing, and interpreting data to uncover patterns, relationships, and trends. It involves using statistical methods such as hypothesis testing, regression analysis, and probability theory to draw meaningful conclusions from data and make informed decisions.
4. Data Preprocessing: Data preprocessing is the initial step in the data analysis process that involves cleaning, transforming, and preparing raw data for analysis. It includes tasks such as data cleaning, feature selection, normalization, and handling missing values to ensure the quality and reliability of the data before applying machine learning algorithms.
5. Big Data: Big data refers to large and complex datasets that exceed the processing capabilities of traditional data management tools. It involves storing, processing, and analyzing massive volumes of structured and unstructured data to extract valuable insights and drive informed decision-making. Techniques such as distributed computing, parallel processing, and cloud computing are used to handle big data efficiently.
Data Science Resources for Beginners
👇👇
https://drive.google.com/drive/folders/1uCShXgmol-fGMqeF2hf9xA5XPKVSxeTo
Share with credits: https://news.1rj.ru/str/datasciencefun
ENJOY LEARNING 👍👍
1. Machine Learning: Machine learning is a subset of artificial intelligence that focuses on developing algorithms and models that can learn from and make predictions or decisions based on data. It involves techniques such as supervised learning, unsupervised learning, and reinforcement learning to analyze and interpret patterns in data.
2. Data Visualization: Data visualization is the graphical representation of data to help users understand complex datasets and identify trends, patterns, and insights. It involves creating visualizations such as charts, graphs, maps, and dashboards to communicate data effectively and facilitate data-driven decision-making.
3. Statistical Analysis: Statistical analysis is the process of collecting, exploring, analyzing, and interpreting data to uncover patterns, relationships, and trends. It involves using statistical methods such as hypothesis testing, regression analysis, and probability theory to draw meaningful conclusions from data and make informed decisions.
4. Data Preprocessing: Data preprocessing is the initial step in the data analysis process that involves cleaning, transforming, and preparing raw data for analysis. It includes tasks such as data cleaning, feature selection, normalization, and handling missing values to ensure the quality and reliability of the data before applying machine learning algorithms.
5. Big Data: Big data refers to large and complex datasets that exceed the processing capabilities of traditional data management tools. It involves storing, processing, and analyzing massive volumes of structured and unstructured data to extract valuable insights and drive informed decision-making. Techniques such as distributed computing, parallel processing, and cloud computing are used to handle big data efficiently.
Data Science Resources for Beginners
👇👇
https://drive.google.com/drive/folders/1uCShXgmol-fGMqeF2hf9xA5XPKVSxeTo
Share with credits: https://news.1rj.ru/str/datasciencefun
ENJOY LEARNING 👍👍
👍25❤1
The first channel on Telegram that offers exciting questions, answers, and tests in data science, artificial intelligence, machine learning, and programming languages
👇👇
https://news.1rj.ru/str/DataScienceInterviews
👇👇
https://news.1rj.ru/str/DataScienceInterviews
👍9
New developers: whenever you work on something interesting, write it down in a document which you keep updating. This will be very helpful when you need to create a resume or have to talk about your achievements in an interview. (Or for college essays.)
I can guarantee you that if you don't do this, you will forget half the interesting things you've done; and for a majority of us, our brains are experts in convincing us that we haven't really done anything interesting.
I can guarantee you that if you don't do this, you will forget half the interesting things you've done; and for a majority of us, our brains are experts in convincing us that we haven't really done anything interesting.
👍16
🔥WEBSITES TO GET FREE DATA SCIENCE CERTIFICATIONS🔥
👌. Kaggle: http://kaggle.com
👌. freeCodeCamp: http://freecodecamp.org
👌. Cognitive Class: http://cognitiveclass.ai
👌. Microsoft Learn: http://learn.microsoft.com
👌. Google's Learning Platform: https://developers.google.com/learn
👌. Kaggle: http://kaggle.com
👌. freeCodeCamp: http://freecodecamp.org
👌. Cognitive Class: http://cognitiveclass.ai
👌. Microsoft Learn: http://learn.microsoft.com
👌. Google's Learning Platform: https://developers.google.com/learn
👍8
I often get asked- what's the BEST Certification for #datascience or #machinelearning?
👉My answer is: none
The reality is that certification don't matter for data science.
This is not commerce. we are not using the same techniques over and over again to solve well-defined problems.
The problems are challenging, the data is messy and numerous techniques are used.
So if you've wondering which certification you should get, Save yourself,some mental energy and stop thinking about it- they are not really matter.
👉 Instead, grab a dataset and start playing with it.
👉 Start applying what you know and trying to solve interesting problems, learn something new every day.
👉 Here are few places to grab datasets to get you started
Google: https://toolbox.google.com/datasetsearch
Kaggle: https://www.kaggle.com/datasets
US Government Dataset: www.data.gov
Quandl: https://www.quandl.com/
UCI
ML repo: http://mlr.cs.umass.edu/ml/datasets.html
World Bank🏦: https://data.worldbank.org/
👉My answer is: none
The reality is that certification don't matter for data science.
This is not commerce. we are not using the same techniques over and over again to solve well-defined problems.
The problems are challenging, the data is messy and numerous techniques are used.
So if you've wondering which certification you should get, Save yourself,some mental energy and stop thinking about it- they are not really matter.
👉 Instead, grab a dataset and start playing with it.
👉 Start applying what you know and trying to solve interesting problems, learn something new every day.
👉 Here are few places to grab datasets to get you started
Google: https://toolbox.google.com/datasetsearch
Kaggle: https://www.kaggle.com/datasets
US Government Dataset: www.data.gov
Quandl: https://www.quandl.com/
UCI
ML repo: http://mlr.cs.umass.edu/ml/datasets.html
World Bank🏦: https://data.worldbank.org/
👍15