Friday Daily Thread: r/Python Meta and Free-Talk Fridays

# Weekly Thread: Meta Discussions and Free Talk Friday 🎙️

Welcome to Free Talk Friday on /r/Python! This is the place to discuss the r/Python community (meta discussions), Python news, projects, or anything else Python-related!

## How it Works:

1. Open Mic: Share your thoughts, questions, or anything you'd like related to Python or the community.
2. Community Pulse: Discuss what you feel is working well or what could be improved in the /r/python community.
3. News & Updates: Keep up-to-date with the latest in Python and share any news you find interesting.

## Guidelines:

All topics should be related to Python or the /r/python community.
Be respectful and follow Reddit's Code of Conduct.

## Example Topics:

1. New Python Release: What do you think about the new features in Python 3.11?
2. Community Events: Any Python meetups or webinars coming up?
3. Learning Resources: Found a great Python tutorial? Share it here!
4. Job Market: How has Python impacted your career?
5. Hot Takes: Got a controversial Python opinion? Let's hear it!
6. Community Ideas: Something you'd like to see us do? tell us.

Let's keep the conversation going. Happy discussing! 🌟

/r/Python
https://redd.it/1gwub4n

MetaDataScraper: A Python Package for scraping Facebook page data with ease!

Hey everyone! 👋

I’m excited to introduce MetaDataScraper, a Python package designed to automate the extraction of valuable data from Facebook pages. Whether you're tracking follower counts, post interactions, or multimedia content like videos, this tool makes scraping Facebook page data a breeze. No API keys or tedious manual effort required — just pure automation! 😎

Usage docs here at ReadTheDocs.

# Key Features:

Automated Extraction: Instantly fetch follower counts, post texts, likes, shares, and video links from public Facebook pages.
Comprehensive Data Retrieval: Get detailed insights from posts, including text content, interactions (likes, shares), and multimedia (videos, reels, etc.).
Loginless Scraping: With the LoginlessScraper class, no Facebook login is needed. Perfect for scraping public pages.
Logged-In Scraping: The LoggedInScraper class allows you to login to Facebook and bypass the limitations of loginless scraping. Access more content and private posts if needed.
Headless Operation: Scrapes data silently in the background (without opening a visible browser window) — perfect for automated tasks or server environments.
Flexible & Easy-to-Use: Simple setup, clear method calls, and works seamlessly with Selenium WebDriver.

# Example Usage:

1. Installation: Simply install via pip:

​

pip install MetaDataScraper

2) Loginless Scraping (no Facebook login required):

from MetaDataScraper import

/r/Python
https://redd.it/1gwn9yd

Finally launched my portfolio with Django

https://preview.redd.it/0wec41h4pa2e1.png?width=1904&format=png&auto=webp&s=0b06f10f26406892d7a7a85f02f0b3c334630595

After years of working with Django, I always postponed building my own personal site. Recently, I decided it was time, and that’s how eriktaveras.com came to life.

# What’s included?

Backend: Django to manage projects and a contact form with spam protection (rate limiting and content detection).
Frontend: Tailwind CSS for a clean design and Alpine.js for light interactivity.
Extras: Automatic Telegram notifications whenever someone submits the contact form.

I’m also working on adding a blog and still uploading more projects to the portfolio, so it’s very much a work in progress.

# What I learned

Using Tailwind CSS for quick, efficient design.
Combining Django with lightweight frontend tools like Alpine.js.
Building a secure contact form without relying on external services.

If you’re curious, feel free to check it out at www.eriktaveras.com. I’d love to hear your feedback or ideas for improvement!

Thanks for reading! 🚀

/r/django
https://redd.it/1gwlx9o

How do you handle speeding up frequent reads on aggregations without redundancy risks?

I've built an internal tool for Estimating, Inventory, Schedule & Dispatch, Job Costing & Reconciliation at a construction contractor business. We're using postgres. Now that much of the operational functionality is there with proper normalization, I'm building out dashboards that do a lot of aggregation on deeply nested fields.

So the (possibly misguided/skill issue?) goal is to persist some aggregated values to distant parent model objects. But the values can never be out of sync!

I've implemented the new GeneratedField with db_persist=True in a number of places, which just simplifies some things, but as I understand it I can't use a GeneratedField to sum a value on a child related model.

So there's a few options I'm aware of, and I'm curious what you use in production environments where data validity and integrity is vital (this affects what people are paid, records for taxes, etc).

Side effects in the child model's `save()` method override
1. Slow on save
2. Error prone, No guarantees on data integrity
3. Tons of clutter and poor maintainability in models . py
Django Signals to update affected parent fields
1. Slow on save
2. Does this roll back

/r/django
https://redd.it/1gwt719

Networking applications should not be opening sockets

From my first development project involving networking I was hooked. I also found some areas of networking software a bit unresolved. There was some strong modeling for people who make networking components but that seemed to peter out after the sockets library. Nobody seemed to have a good compelling way to bundle all that block I/O, byte framing, encoding/decoding, message dispatching etc into something that was reused from project to project.

I finally did something about this and have produced a software library. I also wrote a discussion paper that is the first link in the readme of the following github repo. The repo contains demonstration modules that are referred to in the other readme links.

Networking is not about sockets

Is there anyone else out there that has thought along similar lines? Has anyone seen something better?

/r/Python
https://redd.it/1gw3hwi

Use python to build mk Converter

🚀 Check out my URL/PDF/DOCX to Markdown Converter!

Hey fellow developers! 👋

I'm super excited to share a tool I've been working on that I think might make your life a bit easier. You know that annoying process of converting documents to Markdown? Well, I built something to handle that!

What does it do?
- Converts web pages to Markdown with just a URL
- Transforms PDF files to Markdown (using pdfplumber)
- Converts DOCX files to clean Markdown
- Lets you preview the rendered result right there
- Comes with copy/download buttons for quick access

I built it using FastAPI for the backend (it's crazy fast! ⚡) and kept the frontend super clean and simple. You literally just paste a URL or upload a file, hit convert, and boom! 💥 You've got your Markdown.

Why I made this:
I got tired of manually converting docs for my documentation work, and thought others might find this useful too. Plus, I wanted to learn more about FastAPI and document processing in Python.

Tech stack:
- FastAPI (because who doesn't love async Python? 🐍)
- pdfplumber for PDF parsing
- python-docx for Word docs
- marked.js for the preview
- Basic HTML/CSS/JS for the frontend

The code is open source, and I'd love to get your feedback or contributions!

/r/Python
https://redd.it/1gwfscj

Project Guide: AI-Powered Documentation Generator for Codebases

What My Project Does:
Project Guide is an AI-powered tool that analyzes codebases and automatically generates comprehensive documentation. It aims to simplify the process of understanding and navigating complex projects, especially those written by others.

Target Audience:
This tool is intended for developers, both professionals and hobbyists, who work with existing codebases or want to improve documentation for their own projects. It's suitable for production use but can also be valuable for learning and project management.

Comparison:
Unlike traditional documentation tools that require manual input, Project Guide uses AI to analyze code and generate insights automatically. It differs from static analysis tools by providing higher-level, context-aware documentation that explains project architecture and purpose.

Showcase:
Ever wished your project could explain itself? Now it can! 🪄 Project Guide uses AI to analyze your codebase and generate comprehensive documentation automagically.

Features:
🔍 Deep code analysis
📚 Generates detailed developer guides
🎯 Identifies project purpose and architecture
🗺️ Creates clear documentation structure
🤖 AI-powered insights
📝 Markdown-formatted output
🔄 Recursive directory analysis
🎨 Well-organized documentation

Check it out: https://github.com/sojohnnysaid/project-guide

Going through codebases that someone else wrote is hard, no matter how long you've been at this. This tool

/r/Python
https://redd.it/1gx2515

11 Python Boilerplate Code Snippets Every Developer Needs

Python's simplicity makes it a favorite among developers, especially in trending fields like AI, machine learning, and automation. But let's face it—repeating boilerplate code can be a drag. That’s where Python snippets come in!

From validating emails to shuffling lists, we’ve rounded up 11 essential Python boilerplate snippets to simplify your daily tasks and supercharge your workflow:

# 🔍 1. Validate Email Formats (Regex Simplified)

Use regular expressions to validate email strings efficiently:

pythonCopy codeimport re
def validateemail(email):
emailpattern = re.compile(r'^a-zA-Z0-9._%+-+@a-zA-Z0-9.-+\.a-zA-Z{2,}$')
return bool(emailpattern.match(email))


# ✂️ 2. Slice Strings & Lists Like a Pro

Access sub-elements directly without loops for cleaner code:

pythonCopy codemystring = "Hello, World!"
print(mystring[0:5]) # Output: Hello


# 🔄 3. Compare Words: Are They Anagrams?

Quickly check if two strings are anagrams with `collections.Counter`:

pythonCopy codefrom collections import Counter
def areanagrams(word1, word2):
return Counter(word1) == Counter(word2)


/r/Python
https://redd.it/1gx1nwb

Running concurrent tasks for streaming in a flask route

Hi guys I'm trying to figure out the best way to solve my issue, whether it be threads, or asyncio, or something other than flask.

Heres my route handler:

routehandler():
def streamresponse():
def process(connection):
dosomething()

processingthread = CancellableThreadWithDBConnection(target=process)
processingthread.start()

while not processingdone:
try:
yield json.dumps("")


/r/flask
https://redd.it/1gwq16h

Async or new thread?

Hi guys my flask route is streaming and “yield”s data every 1 second to check if the client connection has been closed. Meanwhile I want the actual route handler logic to run.

Right now I create a separate thread in the route handler to run the actual logic then just have a while loop with yield “” in the main thread.

But this just seems so hacky since I have to terminate the child thread from the main thread if the client closed the connection and yield “” threw a generator exit error.

I saw that flask has an event loop and just wanted to check with you all and see if anyone has had experience with it. Obviously it’s a much better solution if it works. Thanks!

/r/flask
https://redd.it/1gw6gxe

Fellow django developers let's connect! Let's learn and create something together!

I'm creating a discord channel where developers can just chat, mentor other people, and even create project together. We'd be happy if you join our community!

Discord link: https://discord.gg/SD5b4NP4Sq

/r/djangolearning
https://redd.it/1gx7yyi

Data model

Hi there, fellows,
I have the feeling i am wasting a lot of time reading the documentation of the flask-sqlalchemy flask-sqlalchemy....#define-models without doing real progress.

I seek here some advices to reach my goal faster: load a pandas dataframe into a nice class like ExcelData()
I can already load an excel and display it via route and template, but i now want to save it into a DB via a class. My skills seems to be bloked at this step.

Any hints? Link? Template, Tuto? Indian YouTuber?

/r/flask
https://redd.it/1gxagf9

🚀 Feature Friday: assertNumQueries!

Today's Feature Friday reaches back into Django's history for a small-but-powerful tool: assertNumQueries!

This method from TransactionTestCase helps you write tests that verify the number of queries made by a piece of code.

It is a great way to check DB performance and catch regressions or "n+1" issues (when you accidentally make a single DB query for every object in a loop instead of loading everything up front from the database).

You can pass a function to assertNumQueries, or use it as a context manager, as shown in the example below:

from django.test import TransactionTestCase
from .services import myfunctionthathitsthedb

class MyTest(TransactionTestCase):

def testdbperformance(self):
# called directly
self.assertNumQueries(7, myfunctionthathitsthedb)
# used as a context manager
with self.assertNumQueries(2):
Person.objects.create(name="Aaron")


/r/django
https://redd.it/1gx8b9a

Django mastery?

Hi I want to ask how I would master Django?

How to start making projects in Django

Please seniors give me advice

/r/djangolearning
https://redd.it/1gx7hy3

no error but still error

https://preview.redd.it/vozsjofxci2e1.png?width=1920&format=png&auto=webp&s=859fe72c3562d2b37573fca76afbb2ebcd897c05

after clicking approve, function should run which is

https://preview.redd.it/81yit1s5di2e1.png?width=1920&format=png&auto=webp&s=6b06cc038f86829f199c1f056d1148171bebfbf6

now this function confirm_order_route is not printing any statements and in this i have a called a subfunction confirm_order

https://preview.redd.it/tqml8qhpdi2e1.png?width=1920&format=png&auto=webp&s=251373932e24fa1ff6fbc86631c6b1adb26a855e

now this function is deleting booked_order ( record in table in models.py) but not adding data to table confirmed_order , idk why it is not working , not showing any error , it is deleting but not inserting

below i am sharing my tables (models.py)

confirmedOrder model

professional model and bookedorder model

customer model

please help!

/r/flask
https://redd.it/1gxgyma

what is the best approach to deploy django rest framework

what is the best approach to deploy django rest framework

/r/djangolearning
https://redd.it/1gxnrbf

Need advice on reducing latency and improving throughput in Django app

Hey r/django community! I'm struggling with performance issues in my Django application and could really use some expert advice.

**Current Setup:**

* Django 4.2
* PostgreSQL database
* Running on AWS EC2 t2.medium
* \~10k daily active users
* Serving mainly API endpoints and some template views
* Using Django REST Framework for API endpoints

**Issues I'm facing:**

1. Average response time has increased to 800ms (used to be around 200ms)
2. Database queries seem to be taking longer than expected
3. During peak hours, server CPU usage spikes to 90%+
4. Some endpoints timeout during high traffic

**What I've already tried:**

* Added database indexes on frequently queried fields
* Implemented Redis caching for frequently accessed data
* Used Django Debug Toolbar to identify slow queries
* Set up django-silk for profiling
* Added select\_related() and prefetch\_related() where possible

Despite these optimizations, I'm still not getting the performance I need. My main questions are:

1. What are some common bottlenecks in Django apps that I might be missing?
2. Are there specific Django settings I should tune for better performance?
3. Should I consider moving to a different database configuration (e.g., read replicas)?
4. What monitoring tools do you recommend for identifying performance bottlenecks?
5. Any recommendations for load testing tools to simulate high traffic scenarios?

Thanks in advance for any help! Let me

/r/django
https://redd.it/1gxotrj

D Accepted NeurIPS 2024 paper claimed to be solving a novel problem as first work, but ignores 5 prior works

At NeurIPS 2024 I found a paper that got accepted that positions its main contribution in the form of “Existing algorithms for X ignore Y. We adapt algorithm Z for X to account for Y”.

On OpenReview I see that the reviewers in particular praised the novelty of the work, and recognised Y as an important aspect that had been ignored in the field of X.

Now the interesting bit: co-authors and I published a paper in Springer’s Machine Learning journal in 2023 that also proposes an algorithm for X that account for Y. We were also not the first to study the problem setting of X with Y: our paper’s related work section discusses 4 papers that have all proposed algorithms for X that account for Y. One is even from NeurIPS (2017), and the oldest one dates back to 2012 (an AAAI paper).

The authors of this 2024 NeurIPS paper completely missed all this prior literature and believed they were the first, and so did all the reviewers.

This week I e-mailed the authors of this NeurIPS 2024 paper and they acknowledged that these works (mine + the 4 others) indeed were all working on the same problem setting, mentioned that they

/r/MachineLearning
https://redd.it/1gxooqv

Saturday Daily Thread: Resource Request and Sharing! Daily Thread

# Weekly Thread: Resource Request and Sharing 📚

Stumbled upon a useful Python resource? Or are you looking for a guide on a specific topic? Welcome to the Resource Request and Sharing thread!

## How it Works:

1. Request: Can't find a resource on a particular topic? Ask here!
2. Share: Found something useful? Share it with the community.
3. Review: Give or get opinions on Python resources you've used.

## Guidelines:

Please include the type of resource (e.g., book, video, article) and the topic.
Always be respectful when reviewing someone else's shared resource.

## Example Shares:

1. Book: "Fluent Python" \- Great for understanding Pythonic idioms.
2. Video: Python Data Structures \- Excellent overview of Python's built-in data structures.
3. Article: Understanding Python Decorators \- A deep dive into decorators.

## Example Requests:

1. Looking for: Video tutorials on web scraping with Python.
2. Need: Book recommendations for Python machine learning.

Share the knowledge, enrich the community. Happy learning! 🌟

/r/Python
https://redd.it/1gxm8u3

Python isn't just glue, it's an implicit JIT ecosystem

Writing more Rust recently led me to a revelation about Python. Rust was vital to my original task, but only a few simplifications away, the shorter Python version leapt to almost as fast. I'd stumbled from a cold path to a hot path...

This is my argument that Python, through a number of features both purposeful and accidental, ended up with an implicit JIT ecosystem, well-worn trails connecting optimized nodes, paved over time by countless developers.

I'm definitely curious to hear how this feels to others.
I've been doing Python half my life (almost two decades) and Rust seriously for the last few years.
I love both languages deeply but the pendulum has now swung back towards Python not as I won't use Rust but as I feel my eyes are now open as to how when and how I should use Rust.

Python isn't just glue, it's an implicit JIT ecosystem

/r/Python
https://redd.it/1gxlogn

Junior Dev Looking for Django Collab

Hello , I'm a junior dev wanted to get my hands dirty with some Django projects. If you're interested in teaming up or have a project in mind, drop me a message.

/r/django
https://redd.it/1gxtses