Hardest Problem in Computer Science: Centering Things
https://tonsky.me/blog/centering/
@DevMisc
#web #css #learn
https://tonsky.me/blog/centering/
@DevMisc
#web #css #learn
❤2
How to read C type declarations
http://www.unixwiz.net/techtips/reading-cdecl.html
@DevMisc
#c #learn #misc
http://www.unixwiz.net/techtips/reading-cdecl.html
@DevMisc
#c #learn #misc
🔥2😁1
Python's many command-line utilities
Every command-line tool included with Python. These can be run with python -m module_name.
https://www.pythonmorsels.com/cli-tools/
@DevMisc
#python #utils #cli
Every command-line tool included with Python. These can be run with python -m module_name.
https://www.pythonmorsels.com/cli-tools/
@DevMisc
#python #utils #cli
❤3👍1
Hacking Millions of Modems (and Investigating Who Hacked My Modem)
https://samcurry.net/hacking-millions-of-modems
@DevMisc
#security #writeup #misc
The author discovered that someone was intercepting and replaying his web traffic, which indicated his home network or modem had been compromised. After investigating, he found the IP address belonged to DigitalOcean and had previously been used for phishing and malware campaigns. The author was unable to keep the potentially compromised modem when getting a replacement, so he had to give it back to his ISP, Cox. Years later, the author's cybersecurity friends looked into the incident further and discovered the IP address was part of a domain generation algorithm used by malware. They also found that the Cox business portal had an exposed API that could be used to access and control customer modems without authorization. The author reported the vulnerabilities to Cox, who quickly fixed the issues.
https://samcurry.net/hacking-millions-of-modems
@DevMisc
#security #writeup #misc
🔥3
Sei pays out $2M bug bounty
https://usmannkhan.com/bug%20reports/2024/06/17/sei-bug-report.html
@DevMisc
#security #crypto #misc
In April 2024, the author discovered and reported two critical bugs to the Sei Network blockchain. The first bug would have caused the chain to halt if exploited, while the second bug would have allowed an attacker to freely transfer funds out of any account. The Sei Foundation awarded the author $75,000 and $2,000,000 respectively for these reports. The issues were caught after auditing but before being deployed to mainnet, preventing any funds from being put at risk. The author provided detailed technical explanations of how the bugs worked and how they were ultimately fixed by the Sei team.
https://usmannkhan.com/bug%20reports/2024/06/17/sei-bug-report.html
@DevMisc
#security #crypto #misc
🔥6
⚠️ RegreSSHion: RCE in OpenSSH's server, on glibc-based Linux systems
https://www.qualys.com/2024/07/01/cve-2024-6387/regresshion.txt
@DevMisc (🟠 comments)
#security #ssh #linux
- The vulnerability is a regression of a previous issue (CVE-2006-5051) that was introduced in OpenSSH 8.5p1 in October 2020.
- The vulnerability allows for remote code execution as root on glibc-based Linux systems due to the SIGALRM handler calling non-async-signal-safe functions like syslog().
- Older vulnerable OpenSSH versions like 3.4p1 and 4.2p1 can be exploited by interrupting free() calls and leveraging heap corruption techniques like unlink() and House of Mind.
- Newer vulnerable versions like 9.2p1 can be exploited by interrupting malloc() calls and corrupting FILE structures to gain arbitrary code execution.
- Precise timing and network delay mitigation techniques are critical to winning the signal handler race condition.
- The exploit requires carefully crafting the heap layout and leveraging leftover data from previous allocations.
- OpenBSD is not vulnerable because it uses a safer syslog_r() function in its SIGALRM handler.
- The vulnerability is present in the default configuration of OpenSSH and affects the privileged sshd process.
- Significant effort and multiple iterations were required to develop reliable exploits for the different OpenSSH versions.
- The research demonstrates the continued need for vigilance in secure software development, as even a well-designed system like OpenSSH can have subtle regressions that introduce critical vulnerabilities.
https://www.qualys.com/2024/07/01/cve-2024-6387/regresshion.txt
@DevMisc (
#security #ssh #linux
Please open Telegram to view this post
VIEW IN TELEGRAM
🔥2
Reverse Engineering Ticketmaster's Rotating Barcodes
https://conduition.io/coding/ticketmaster/
@DevMisc
#web #rev #misc
- TicketMaster has moved away from traditional printable PDF tickets in favor of a proprietary "SafeTix" system that uses rotating barcodes displayed on a mobile device.
- The rotating barcodes are meant to prevent ticket fraud, but the author argues they create significant usability issues, especially when cell service is poor at venues.
- TicketMaster markets SafeTix as a solution to ticket scalping and fraud, but the author believes the real motivations are to lock users into TicketMaster's ecosystem and make it harder to resell tickets outside their platform.
- The author was able to reverse engineer the SafeTix system and discovered it uses Time-based One-Time Passwords (TOTPs) along with a static bearer token to authenticate tickets.
- With the TOTP secrets and bearer token, the author could theoretically generate valid barcodes and bypass TicketMaster's security measures.
- TicketMaster makes it easy to extract the necessary token information by printing it to the browser console when the barcode is loaded.
- The author is uncertain about the lifetime of the TOTP tokens, but believes they may only be valid for up to 20 hours before the event based on TicketMaster's documentation.
- The author developed a tool called "TicketGimp" that can render valid SafeTix barcodes using the extracted token information.
- The author is highly critical of TicketMaster's practices, accusing them of using technology to exclude and disadvantage customers for their own financial gain.
- The author calls on TicketMaster's developers to have more integrity and use their technical skills responsibly, rather than enabling TicketMaster's "cruel business practices".
https://conduition.io/coding/ticketmaster/
@DevMisc
#web #rev #misc
❤2👍2
Python has too many package managers
https://dublog.net/blog/so-many-python-package-managers/
@DevMisc
#python #pip #extra
- Python has a fragmented package and environment management ecosystem, with over a dozen different tools, each with their own strengths and weaknesses.
- The lack of a standardized, efficient, and user-friendly package manager in Python has been an "inexcusable pain-in-the-ass" for many years.
- Other programming languages like Rust, C#, and JavaScript have developed more cohesive and widely-loved package management solutions like Cargo, NuGet, and npm.
- Python's legacy package manager pip has historically had poor dependency resolution, only recently adding backtracking capabilities. It also lacks environment management features.
- The proliferation of various configuration files like requirements.txt, setup.py, Pipfile, environment.yml etc. has led to a lot of redundancy and lack of standardization in Python package management.
- The introduction of PEP 621 in 2020 aimed to consolidate dependencies and configuration into a single pyproject.toml file, leading to the emergence of new tools like Poetry, PDM, Flit, and Hatch.
- Poetry is currently the closest Python tool to the Cargo experience, but it suffers from slow dependency resolution, especially for large projects.
- Conda is a popular choice for data scientists and experimentalists as it can manage non-Python dependencies, but it lacks some features like lock files and can be cumbersome for production use.
- The Rust community's influence is evident in promising new Python package management tools like uv, which aims to be a fast, Cargo-like drop-in replacement for pip.
- The Python community still lacks a cohesive, standardized, and widely-adopted package management solution, but tools like uv hold promise for the future.
https://dublog.net/blog/so-many-python-package-managers/
@DevMisc
#python #pip #extra
💯3👍1👏1
Why German Strings are Everywhere
https://cedardb.com/blog/german_strings/
@DevMisc
#cpp #data #misc
- Developed by Umbra (CedarDB's predecessor)
- Adopted by DuckDB, Apache Arrow, Polars, and Facebook Velox
German Strings are a custom string type highly optimized for data processing. They offer significant improvements over traditional C and C++ string implementations.
Key Features:
- 128-bit struct representation (vs. 192 bits in C++)
- Short string optimization for strings ≤12 characters
- Long string format with 4-char prefix for quick comparisons
- Immutable design for better performance and concurrency
- Storage classes: persistent, transient, temporary
Advantages:
- Space-efficient, fitting in two CPU registers
- Reduced allocations and data movement
- Easier parallelization due to immutability
- Flexible lifetime management with storage classes
- Optimized for common database operations (comparisons, sorting)
Trade-offs:
- Requires careful consideration of string usage and lifetime
- Updates are more expensive (but rare in database systems)
- Maximum string length limited to 4 GiB
https://cedardb.com/blog/german_strings/
@DevMisc
#cpp #data #misc
👍5
Counting Bytes Faster Than You'd Think Possible
https://blog.mattstuchlik.com/2024/07/21/fastest-memory-read.html
@DevMisc
#asm #cpp #optimization
- The author was able to significantly optimize a byte-counting program, achieving a ~550x speedup over a naive implementation.
- The key optimization was using an interleaved memory access pattern, reading from different 4KB pages in a round-robin fashion, instead of sequential access.
- This interleaved access pattern takes advantage of the "Streamer" hardware prefetcher in modern CPUs, which can maintain separate forward and backward access streams for each 4KB page.
- Interleaving 8 different 4KB pages was found to be the optimal approach, providing up to a 30% performance boost over sequential access.
- The author also unrolled the inner loop to process 2 cache lines (64 bytes) at a time, and added a prefetch instruction to fetch the next set of data.
- The final solution uses AVX2 SIMD instructions to perform the byte counting in a highly efficient manner.
- The author was able to achieve a ranking of #13 on the HighLoad leaderboard with this optimized solution.
- The interleaved memory access pattern seems to be an under-discussed optimization technique, with the author not recalling seeing it used in other code.
- The author encourages readers to share any other memory-based optimizations they are aware of, as the author is interested in learning about them.
- The document provides the full source code for the optimized byte-counting program, allowing readers to study and potentially apply the techniques in their own work.
https://blog.mattstuchlik.com/2024/07/21/fastest-memory-read.html
@DevMisc
#asm #cpp #optimization
❤1🤯1
Scaling One Million Checkboxes to 650M checks
https://eieio.games/essays/scaling-one-million-checkboxes/
@DevMisc (🟠 comments)
#learn #fun #scaling #misc
- The website "One Million Checkboxes" (OMCB) launched on June 26th, 2024 and unexpectedly went viral, attracting millions of users and checkbox checks within the first few days.
- The initial architecture used a single Flask server, nginx reverse proxy, and Redis for state management, but this was quickly overwhelmed by the surge in traffic.
- Key principles for scaling the site included bounding costs, embracing short-term solutions, using simple self-hosted tech, and keeping the experience global.
- Scaling efforts involved adding more Flask servers, implementing batching and connection pooling, and capping bandwidth usage with Linux tc to control costs.
- Bugs like allowing checkbox checks beyond the 1 million limit caused issues that required quick fixes, like truncating the bitset.
- Adding a Redis replica helped spread the load, though finding the private IP address was a challenge.
- Ensuring clients received consistent, up-to-date checkbox state required adding timestamps and logic to handle stale updates.
- Rewriting the backend in Go provided a significant performance boost, allowing the implementation of a "sunsetting" feature to freeze checked boxes over time.
- Using Redis and Lua noscripts made the sunsetting logic simple and race condition-free.
- The author learned valuable lessons about building for the unpredictable nature of the internet, validating their belief in demand for constrained anonymous interactions, and the benefits of launching quickly versus extensive planning.
https://eieio.games/essays/scaling-one-million-checkboxes/
@DevMisc (
#learn #fun #scaling #misc
Please open Telegram to view this post
VIEW IN TELEGRAM
🔥2
TOTP tokens on my wrist with the smartest dumb watch
https://blog.singleton.io/posts/2022-10-17-otp-on-wrist/
@DevMisc (🟠 comments)
#c #fun #misc
- The author has replaced the logic board of a classic Casio F-91W watch with a new ARM Cortex M0+ powered board from Sensor Watch.
- The new board allows the watch to be programmed and customized, including adding features like TOTP (time-based one-time password) support for two-factor authentication.
- The author was able to set up TOTP support for their Google and GitHub accounts, allowing them to access the OTP codes directly on their wrist.
- The author also created a new "ratemeter" watchface that can be used to measure rates, such as rowing strokes or cadence.
- The document provides detailed instructions on how to add TOTP secrets to the watchface code and how the ratemeter watchface was implemented.
- The Sensor Watch project provides a clean and easy to modify set of watchfaces and complications that can be customized.
- The F-91W watch case, combined with the new programmable board, offers a powerful and hackable platform with long battery life.
- The author highlights the availability of a WASM-based emulator that makes it easy to test and play with the custom builds.
- The document mentions other interesting watchfaces available in the Sensor Watch project, including a pulsometer and orrery.
- The author recommends getting a Sensor Watch from Oddly Specific Objects, though they have no affiliation with the company.
https://blog.singleton.io/posts/2022-10-17-otp-on-wrist/
@DevMisc (
#c #fun #misc
Please open Telegram to view this post
VIEW IN TELEGRAM
"We ran out of columns" - The best, worst codebase
https://jimmyhmiller.github.io/ugliest-beautiful-codebase
@DevMisc
#sql #fun #badcode
- The database was the central component of the system, with a table called "Merchants" that had over 500 columns due to running out of columns in the original "Merchants" table.
- The "SequenceKey" table was a single-column, single-row table used to generate IDs, demonstrating a creative but unconventional solution.
- The system had a manually maintained "calendar" table to track login access, which was a fragile and outdated approach.
- The employee data was reloaded from a CSV file every morning, with an email-based process to replicate the data to headquarters.
- There was a normalized copy of the database, but it required 7 joins to go from the "Merchants" table to a phone number, showing the complexity.
- The codebase was a mix of VB and C#, with a proliferation of JavaScript frameworks and custom modifications.
- The "shipping manager" application was built in a weekend by a single developer named Gilfoyle, who was known for not checking in his code.
- The author discovered a bug related to the shipping queue that was caused by a SOAP service client doing all the side effects instead of the service itself.
- The "Merchants Search" page was optimized by a senior developer named Justin, who was able to make significant improvements by decoupling the page into separate endpoints.
- The codebase, despite its flaws, allowed for a sense of freedom and creativity, with developers carving out their own "little worlds of sanity" within the larger monolithic application.
https://jimmyhmiller.github.io/ugliest-beautiful-codebase
@DevMisc
#sql #fun #badcode
❤2🔥1
How to Get or Create in PostgreSQL
https://hakibenita.com/postgresql-get-or-create
@DevMisc (🟠 comments)
#postgres #database #learn #misc
- Implementing "get or create" functionality correctly in PostgreSQL can be tricky, with potential issues around race conditions, concurrency, and bloat.
- A simple INSERT statement is not idempotent, as executing it with the same input twice will trigger a unique constraint violation error.
- To provide idempotency, the process needs to handle two situations: 1) if the tag already exists, return the existing tag, and 2) if the tag does not exist, create it and return the new tag.
- Using a unique constraint violation to handle "get or create" can lead to bloat, as new rows are first inserted and then marked as dead if a duplicate is found.
- Checking if a tag exists before inserting it (the "look before you leap" approach) can suffer from time-of-check to time-of-use issues when used concurrently.
- The "ask for forgiveness" approach using INSERT ON CONFLICT DO NOTHING is a better way to handle "get or create" without generating bloat.
- Concatenating results from the target table and a WITH clause using UNION ALL can handle visibility issues with data-modifying statements in the WITH clause.
- Even the "ask for forgiveness" approach can suffer from concurrency issues, as a race condition can still occur between checking if a tag exists and inserting it.
- Using INSERT ON CONFLICT with a DO NOTHING clause is the most robust solution, providing idempotency, concurrency safety, and preventing bloat.
- Starting in PostgreSQL 17, the MERGE statement with RETURNING can provide an alternative approach that doesn't require unique or exclusion constraints on the target table.
https://hakibenita.com/postgresql-get-or-create
@DevMisc (
#postgres #database #learn #misc
Please open Telegram to view this post
VIEW IN TELEGRAM
❤2
Handling Concurrency Without Locks
https://hakibenita.com/django-concurrency
@DevMisc
#sql #reliability #learn #misc
- Concurrency issues can be difficult to recognize and often get overlooked, leading to hard-to-debug bugs.
- It's tempting to dismiss concurrency issues due to perceived low likelihood, but they can still crop up unexpectedly under high load.
- Locking is a common approach to handling concurrency, but locks can be overused and lead to performance issues.
- The database is the lowest common denominator for coordinating locks across multiple processes and servers.
- The "ask for forgiveness" (EAFP) approach, where you try an operation and handle exceptions, is often more Pythonic than checking conditions in advance.
- In PostgreSQL, when an exception occurs within a transaction, it can block further commands until the transaction ends, requiring special handling.
- Using SELECT FOR UPDATE can lock rows to prevent race conditions, but this can also cause performance issues with high concurrency.
- Incrementing counters directly in the database, using an F expression, can avoid race conditions without the need for explicit locking.
- Combining database-level updates with RETURNING to immediately fetch the updated object can optimize the process further.
- The key is to keep concurrency issues in mind, avoid dismissing them due to perceived low likelihood, and use the most appropriate concurrency control mechanisms for the specific situation.
https://hakibenita.com/django-concurrency
@DevMisc
#sql #reliability #learn #misc
Can you convert a video to pure CSS?
https://dgerrells.com/blog/can-you-convert-a-video-to-pure-css
@DevMisc (🟠 comments)
#css #web #fun
- It is possible to convert a video into a pure CSS animation, where each pixel of the video is represented by an animated CSS element.
- This technique involves downscaling the video, extracting the pixel data, and generating a massive CSS animation with keyframes for each pixel.
- The process can be optimized by skipping keyframes where the pixel color hasn't changed, but this introduces some visual artifacts.
- An alternative approach uses CSS box-shadows to represent the video pixels, which is simpler and more performant, especially on Chrome browsers.
- Browser support and performance varies greatly, with Safari handling much larger CSS animations than Chrome before crashing.
- The final CSS-based video animation can be further converted into an animated GIF using a library like GIF.js.
- The motivation behind this project seems to be more about the technical challenge and novelty rather than practical application.
- The author emphasizes the importance of style over substance, suggesting this approach could be used for startup landing pages to send a message, even if it crashes most browsers.
- The author hints at the possibility of creating a new file format called ".vibcss" to standardize this CSS-based video approach.
- Overall, the document showcases an innovative, if impractical, technique to push the boundaries of what is possible with CSS animations.
https://dgerrells.com/blog/can-you-convert-a-video-to-pure-css
@DevMisc (
#css #web #fun
Please open Telegram to view this post
VIEW IN TELEGRAM
👍2