🖥 How to Install Deep Seek Locally Using Ollama LLM on Ubuntu 24.04

A detailed tutorial from TecMint demonstrating how to install and run the DeepSeek model locally on Linux (Ubuntu 24.04) using Ollama.

The guide covers all installation steps: updating the system, installing Python and Git, configuring Ollama to control DeepSeek, and running the model via the command line or using a convenient Web UI.

▪️ The guide also includes instructions for automatically launching the Web UI at system startup via systemd, which makes working with the model more comfortable and accessible.

Suitable for those who want to explore the possibilities of working with large language models without being tied to cloud services, providing full control over the model and its settings.

▪️ Read

🔗 01. Small Language Models - An Emerging Technique in AI

Unlike large language models, which rely on vast amounts of data, small language models focus on high-quality, curated training datasets. This approach allows them to potentially outperform larger models in specific tasks, especially when specialized training is applied.

💡 Key Advantages of Small Language Models:

1. Compact Size: Small language models are significantly smaller in size compared to their larger counterparts. This compactness makes inference (the process of making predictions) much easier and more efficient, as they do not require large GPUs or extensive computational resources.

2. Efficient Training: Training small language models is more efficient because they do not need to process "essentially unlimited" data. This reduces the computational resources required for both training and inference.

3. Easier Deployment: One of the most promising aspects of small language models is their potential for deployment on edge devices. While this capability is still emerging, the instructor predicts that we will soon see small language models customized for specific hardware, such as drones, phones, or other devices. This would enable these models to perform specialized tasks directly on the device, without the need for cloud-based processing.

4. Specialization: Small language models can be tailored for specific tasks, potentially outperforming larger models in those areas. This makes them highly suitable for applications where task-specific performance is more critical than general-purpose capabilities.

💡 Future Prospects:

The video highlights that small language models are likely to play a significant role in the future of edge-based computing. As hardware capable of supporting machine learning models becomes more prevalent, small language models could be integrated into a wide range of devices, enabling real-time, on-device AI capabilities.

💡 Conclusion:

Small language models represent a promising area of research in AI, offering several advantages over large language models, including efficiency, ease of deployment, and the potential for task-specific optimization. As the technology evolves, we can expect to see these models increasingly used in edge devices, driving innovation in specialized AI applications. Understanding the benefits and potential of small language models is essential for anyone interested in the future of AI and machine learning.

🔗 02. Demo of Phi - A Small Language Model by Microsoft Research

Phi is a small language model developed by Microsoft Research, as one of the most notable examples of small language models created recently. Phi is designed with 2.7 billion parameters, making it significantly smaller than many large language models, yet it demonstrates impressive performance due to its focus on high-quality, textbook-level training data.

💡 Key Features of Phi:

1. High-Quality Data: Unlike large language models that rely on vast amounts of data, Phi is trained on curated, high-quality datasets. This focus on quality over quantity allows Phi to achieve superior performance in tasks such as reasoning, language understanding, and mathematical problem-solving.

2. Performance Benchmarks: Despite its smaller size, Phi outperforms larger models like Llama 2 in specific areas. For example, it achieves more than three times better performance in mathematical tasks and nearly double the coding performance compared to other small models. This demonstrates that small language models can compete with or even surpass larger models in specialized tasks.

3. Efficiency and Speed: One of the key advantages of Phi is its compact size (only 1.96 gigabytes), which makes it easy to run on standard hardware. The presenter demonstrates how Phi can be quickly executed using tools like cURL or Python, and it provides fast responses to queries, such as calculating the square root of 16 or generating equations for linear optimization.

4. Specialization: Phi's ability to excel in specific tasks, such as math and coding, highlights the potential of specialized small language models. The presenter suggests that this could be a future trend, where small models are tailored for particular applications, allowing them to run efficiently on smaller devices and in smaller form factors.

💡 Running Phi:

The video provides a practical demonstration of how to run Phi using the Mozilla Llama file. The process is straightforward, requiring only a simple command to execute the model. The presenter shows how Phi can quickly respond to prompts, showcasing its speed and accuracy in real-time.

💡Future Implications:

The presenter emphasizes that Phi represents a promising direction in AI development. By focusing on specialized, high-quality training data, small language models like Phi can achieve surprisingly good performance while being more efficient and easier to deploy. This could lead to a future where small language models are increasingly used in edge devices and other resource-constrained environments.

💡 Conclusion:

Phi is a canonical example of how small language models can leverage high-quality data and specialized training to outperform larger models in specific tasks. Its compact size, efficiency, and speed make it a powerful tool for applications requiring real-time, on-device AI capabilities. As the field evolves, we can expect to see more small language models like Phi being developed for specialized tasks, driving innovation in AI and machine learning.

🔗 03. Getting started with a llamafile

Llamafile is a project by Mozilla that simplifies the process of running large language models (LLMs) locally. Llamafile is designed to make it easy for users to deploy and run LLMs on their own machines, offering significant advantages in terms of privacy, cost, and performance.

💡 Key Features of Llamafile:

1. Single-File Distribution: Llamafile allows users to distribute and run LLMs using a single executable file. This is made possible by leveraging the Cosmopolitan Libc library, which compresses everything into one file, making the process straightforward and user-friendly.

2. Model Agnostic: Although the project is named "Llamafile," it is not tied to any specific large language model. Users can run various models, such as LLaVA (which can read images) or Mixtral, a high-performing open-source model. Mixtral, in particular, is highlighted for its Apache 2 license and strong performance, making it a popular choice for local deployment.

3. Ease of Use: Running a model with Llamafile is as simple as downloading the file and executing a command (e.g., ./run). The project also provides a Python API that mimics the OpenAI API, allowing users to transition from proprietary models to open-source ones seamlessly. Additionally, users can interact with the model using cURL commands, making it accessible for different use cases.

4. Privacy and Cost Benefits: One of the main advantages of running LLMs locally with Llamafile is privacy. Users do not need to send their data to external servers, ensuring that sensitive information remains secure. Additionally, running models locally is free, as users only need their own hardware, avoiding the costs associated with cloud-based APIs.

5. Performance: Local models run with Llamafile offer lower latency compared to external APIs, resulting in faster response times. The narrator demonstrates this by running a Python "Hello World" function and other tasks, showing how quickly the model can generate responses using local hardware (e.g., a Mac GPU).

💡 Practical Demonstration:

The video includes a live demonstration of how to set up and run a model using Llamafile. The narrator downloads the Mixtral model (a 30 GB file) and executes it locally. The model is then tested with various prompts, such as generating a Python function, showcasing its speed and accuracy. The narrator also explains how to reset the model to its default state and customize its behavior.

💡 Advantages of Using Llamafile:

- Privacy: Data remains on the user's machine, ensuring confidentiality.
- Cost-Effective: No need to pay for cloud-based services; users only need their own hardware.
- Performance: Local execution reduces latency, providing faster responses.
- Flexibility: Users can run different models and interact with them using Python or cURL.

💡 Conclusion:

Llamafile is a powerful and user-friendly tool for running large language models locally. Its single-file distribution, ease of use, and strong performance make it an attractive option for developers and researchers looking to leverage LLMs without relying on external services. The project also emphasizes the importance of privacy and cost savings, making it a compelling choice for those who want to explore local AI deployment. The narrator encourages viewers to try Llamafile and experience its benefits firsthand.

🔗 04. Demo of LLaVA - A Small Language Model for Vision and Language Tasks

LLaVA (Large Language and Vision Assistant) is a small language model specifically designed for multi-modal vision and language tasks. LLaVA is an example of how smaller, specialized models can outperform larger models in specific domains, particularly in computer vision and image understanding.

💡 Key Features of LLaVA:

1. Specialized for Vision and Language: LLaVA is optimized for multi-modal tasks, meaning it can process both text and images. This makes it particularly useful for applications that require understanding and describing visual content, such as image captioning or accessibility tools.

2. Small and Efficient: Although LLaVA is larger than some other small models, it is still significantly smaller than traditional large language models (e.g., 30 GB models). Its compact size allows it to run efficiently on local hardware, such as a Mac with an M Series chip, without requiring extensive computational resources.

3. High Performance: Despite its smaller size, LLaVA delivers fast and accurate results in vision-related tasks. The instructor demonstrates how LLaVA can quickly analyze and describe images, often faster than a human could interpret the same visual information.

💡 Running LLaVA with Llamafile:

The instructor uses Llamafile, a tool that packages large language models into a single binary file, to run LLaVA locally. Llamafile simplifies the process of deploying and running models like LLaVA, making it accessible for users who want to experiment with local AI models.

💡 Advantages of Small, Specialized Models:

1. Task-Specific Optimization: LLaVA is optimized for computer vision tasks, allowing it to perform these tasks more efficiently than general-purpose models. This specialization leads to faster performance and better accuracy in its domain.

2. Accessibility Applications: The instructor suggests that LLaVA could be particularly useful for accessibility applications, such as generating alt text for images in educational courses or other workflows. This makes it a valuable tool for developers and educators who need to create accessible content.

3. Local Execution: Running LLaVA locally with Llamafile ensures privacy and low latency, as the data does not need to be sent to external servers. This makes it ideal for applications where data security and real-time performance are important.

💡 Conclusion:

LLaVA is a powerful example of how small, specialized language models can excel in specific tasks, such as multi-modal vision and language understanding. Its ability to quickly and accurately describe images makes it a valuable tool for applications like accessibility, education, and content creation. By using tools like Llamafile, users can easily deploy and run LLaVA locally, benefiting from its efficiency, speed, and privacy. The instructor encourages viewers to explore LLaVA and consider its potential for specialized AI applications.