"We ran out of columns" - The best, worst codebase

- The database was the central component of the system, with a table called "Merchants" that had over 500 columns due to running out of columns in the original "Merchants" table.
- The "SequenceKey" table was a single-column, single-row table used to generate IDs, demonstrating a creative but unconventional solution.
- The system had a manually maintained "calendar" table to track login access, which was a fragile and outdated approach.
- The employee data was reloaded from a CSV file every morning, with an email-based process to replicate the data to headquarters.
- There was a normalized copy of the database, but it required 7 joins to go from the "Merchants" table to a phone number, showing the complexity.
- The codebase was a mix of VB and C#, with a proliferation of JavaScript frameworks and custom modifications.
- The "shipping manager" application was built in a weekend by a single developer named Gilfoyle, who was known for not checking in his code.
- The author discovered a bug related to the shipping queue that was caused by a SOAP service client doing all the side effects instead of the service itself.
- The "Merchants Search" page was optimized by a senior developer named Justin, who was able to make significant improvements by decoupling the page into separate endpoints.
- The codebase, despite its flaws, allowed for a sense of freedom and creativity, with developers carving out their own "little worlds of sanity" within the larger monolithic application.

https://jimmyhmiller.github.io/ugliest-beautiful-codebase

@DevMisc
#sql #fun #badcode

Handling Concurrency Without Locks

- Concurrency issues can be difficult to recognize and often get overlooked, leading to hard-to-debug bugs.
- It's tempting to dismiss concurrency issues due to perceived low likelihood, but they can still crop up unexpectedly under high load.
- Locking is a common approach to handling concurrency, but locks can be overused and lead to performance issues.
- The database is the lowest common denominator for coordinating locks across multiple processes and servers.
- The "ask for forgiveness" (EAFP) approach, where you try an operation and handle exceptions, is often more Pythonic than checking conditions in advance.
- In PostgreSQL, when an exception occurs within a transaction, it can block further commands until the transaction ends, requiring special handling.
- Using SELECT FOR UPDATE can lock rows to prevent race conditions, but this can also cause performance issues with high concurrency.
- Incrementing counters directly in the database, using an F expression, can avoid race conditions without the need for explicit locking.
- Combining database-level updates with RETURNING to immediately fetch the updated object can optimize the process further.
- The key is to keep concurrency issues in mind, avoid dismissing them due to perceived low likelihood, and use the most appropriate concurrency control mechanisms for the specific situation.

https://hakibenita.com/django-concurrency

@DevMisc
#sql #reliability #learn #misc

Floating points between zero and one

TL;DR: the number of float representations between 0.0 and 1.0 is the same as between 1.0 and +inf

https://chadnauseam.com/coding/random/floating-points-between-zero-and-one

@DevMisc
#lowlevel #fun #misc

Password protect a static HTML page, decrypted in-browser in JavaScript

Safely encrypt and password protect the content of your public static HTML file, to be decrypted in-browser without any back-end - to serve it over static hosting like Netlify, GitHub pages, etc. (see a live example)

https://github.com/robinmoisson/staticrypt

@DevMisc
#fun #web #cryptography #tools #extra

Programming Zero Knowledge Proofs: From Zero to Hero

- Zero Knowledge Proofs (ZKPs) allow one party (the prover) to prove to another party (the verifier) that they know some secret information without revealing that information.
- ZKPs have the key properties of privacy (proving something without revealing anything else) and succinctness (the proof stays roughly the same size regardless of the complexity of the computation).
- ZKPs can be programmed by writing special programs called circuits, which specify constraints that must be satisfied.
- The process of generating a ZKP involves a trusted setup to create a proving key and verification key, which can then be used to generate and verify proofs.
- Implementing basic ZKP programs involves defining constraints, compiling the circuit, performing a trusted setup, generating a witness, and generating/verifying proofs.
- Improving ZKP programs often involves adding more complex constraints, such as checking that inputs are not equal to 1.
- ZKPs can be used to implement cryptographic primitives like digital signatures, using hash functions and commitments instead of public-key cryptography.
- Implementing a digital signature scheme with ZKPs involves generating a private/public key pair (identity secret/identity commitment) and using them to sign and verify messages.
- ZKPs make it possible to implement complex cryptographic protocols, like group signatures, in a more straightforward way compared to traditional cryptographic techniques.
- Writing effective ZKP programs requires developing an intuition for how to express constraints in a way that satisfies the requirements of the underlying arithmetic circuits.

https://zkintro.com/articles/programming-zkps-from-zero-to-hero

@DevMisc
#cryptography #zkp #misc

@DevMisc
#meme #iot #extra

Securing a Linux Server

A guide to secure and harden a Linux server install.

https://kenhv.com/blog/securing-a-linux-server

@DevMisc
#security #devops #learn

How does cosine similarity work?

When working with LLM embeddings, it is often important to be able to compare them. Cosine similarity is the recommended way to do this.

https://tomhazledine.com/cosine-similarity/

@DevMisc
#embeddings #llm #learn

const fn: Pure Functions in Rust

I was taught formal methods at university but these ultra-safe development techniques are expensive, require using unusual external verification languages, and most damning for web and application developers, they slow down iteration.
After graduating and getting a webdev job, I despaired that the safety and guarantees of the formal systems that I had been introduced to weren't available to me as a web developer.
I was going to have to act if I wanted to live in a different world.

https://www.youtube.com/watch?v=voRBS0r4EyI

@DevMisc
#rust #fp #purity #learn

The sorry state of OpenSSL usability

OpenSSL is a widely used but poorly documented software, making it difficult for users to figure out how to use basic functionality like generating an RSA key. The documentation is scattered across multiple websites, often contradictory, and assumes a level of cryptographic knowledge that many users lack. Even simple tasks like determining the key format can be challenging due to the lack of clear guidance. The author highlights several usability issues with OpenSSL, such as the default use of a weak 512-bit RSA key, and the lack of warnings or guidance when using deprecated interfaces. The author argues that improving OpenSSL's usability through better documentation, user testing, and avoiding unnecessary forks could go a long way in making this critical piece of software more accessible to a wider audience.

https://jameshfisher.com/2017/12/02/the-sorry-state-of-openssl-usability/

@DevMisc
#cryptography #openssl #learn

Replacing FastAPI with Rust — A series

The author is starting a blog series exploring the possibility of replacing the FastAPI Python web framework with the Rust programming language for their use case. FastAPI is the author's go-to backend framework, but they are interested in moving beyond some of the limitations of Python. The key requirements the author is hoping to achieve with a Rust-based solution are high performance, type safety, and ease of deployment. The author is unsure where this exploration will lead, but is excited to document the process and see if they can create a compelling alternative to FastAPI. Readers are encouraged to follow along, provide feedback, and suggest ideas for the blog series.

1. Intro (defining requirements)
2. Research (rweb, rocket, paperclip)
3. Trying Actix (actix-web)
4. A Solution (rweb, dropshot, rocket with okapi)
5. Rocket 0.5 (rocket, async tests, memory management)
6. AWS Lambda (AWS CDK)

@DevMisc
#rust #fastapi #rocket #learn

Understanding Rust futures by going way too deep 🦀

The text is a deep dive into understanding Rust futures. It starts by setting up a basic Rust project and adding dependencies like tokio, color-eyre, and tracing. It then demonstrates how to make a simple HTTP request using the reqwest library. The author explains that futures in Rust are just state machines that don't do anything until they are polled, which is demonstrated through creating a "dumb future". The text then dives deeper into the internals of Rust futures, showing how to inspect the call stack when things go wrong, even using unsafe code to cause a segmentation fault. The overall message is that Rust futures are powerful but require understanding their underlying mechanics to use them effectively.

https://fasterthanli.me/articles/understanding-rust-futures-by-going-way-too-deep

@DevMisc
#rust #async #learn

So you want to deploy mTLS

mTLS is just like TLS, but with one extra consideration. In mTLS, both the client and server are authenticated, whereas in standard TLS only the server is authenticated. That’s where the m comes from: mutual.

https://otterize.com/blog/so-you-want-to-deploy-mtls

@DevMisc
#cryptography #security #tls #learn

Cursed Knowledge

Things we wish we didn't know.

https://immich.app/cursed-knowledge/

@DevMisc
#fun #misc #learn

Understanding and Preventing Race Conditions in Web Applications

- Race conditions can lead to data corruption, crashes, and security vulnerabilities if not properly addressed.
- Using a simple increment operation to update a shared counter is vulnerable to race conditions and can result in incorrect counts.
- Locking the database row using "SELECT FOR UPDATE" is not sufficient to prevent race conditions, as it does not guarantee atomicity.
- Transactions are essential to ensure atomicity and prevent race conditions, but simply using a transaction is not enough - the specific SQL query used must also be atomic.
- Updating the counter using an atomic SQL query (e.g. "UPDATE table SET views = views + 1 WHERE id = $1") is a more scalable and efficient solution compared to locking rows.
- When dealing with game economies and player balances, it's crucial to enforce constraints at the database level (e.g. CHECK constraints) to prevent negative balances.
- Transactions do not inherently prevent all race conditions - the specific implementation and isolation level used is important.
- Avoid using "LOCK TABLE" as it can lead to performance issues and scalability problems.
- Testing for race conditions by simulating concurrent requests is essential to identify and address such issues.
- There is no single silver bullet solution - each approach has trade-offs that must be carefully considered based on the specific requirements of the application.

https://gavide.hashnode.dev/prevent-race-conditions-in-your-api

@DevMisc [#Original ❤️]
#security #sql #backend #learn

Gaining access to anyones browser without them even visiting a website

- Arc Browser
- arc boosts can contain arbitrary javanoscript
- arc boosts are stored in firestore
- the arc browser gets which boosts to use via the creatorID field
- we can arbitrarily chage the creatorID field to any user id
- $2000 bounty
- possible RCE with xss on privileged pages (chrome://settings)
- arc sends every url you visit to firebase queries

https://kibty.town/blog/arc/

@DevMisc
#security #firebase #learn