• 0 Posts
  • 11 Comments
Joined 3 years ago
cake
Cake day: July 3rd, 2023

help-circle

  • Deadlocks aren’t generally too much of an issue with an MVCC DB like Postgres (though it certainly can happen if you use explicit locks and such), but yeah ultimately it’s all about the kind of workload we’re talking about. I honestly think of it much less like a database and more like a high-performance synchronization mechanism that can cross process boundaries. There’s many times where you want to coordinate the behavior of many independent processes (like horizontally-scaled services), such as rate-limiting your requests to an external service. Implementing a distributed rate-limiting algorithm with a RDBMS is a pretty poor choice, because the data used for any synchronization is ephemeral, unimportant, and short-lived (and this, ACID guarantees are pretty irrelevant), and you would incur a lot of contention overhead for shared resources. You might even get bottlenecks that prevent you from achieving the rate you’re trying to limit yourself to. Something like Valkey gives you simple, fast atomic writes with built-in TTL mechanisms, which is perfect for a rate-limiting system. There are many other examples like this where it makes a lot of sense.

    Or consider even something incredibly simple like, I dunno, a realtime user counter. Valkey has an atomic primitive operation for this built-in (INCR). With an RDBMS, this is not trivial as you scale up the number of clients, because either you have to deal with a ton of row-level contention (if the counter is stored in a single row that gets updated), or you end up hammering your disk with increasingly large aggregate operations (if there’s a row-per-user and the total is computed across them, somehow).

    As I said, it’s really just very different workloads where they are valuable.


  • I’m not sure what you mean by “standard DB queue” (perhaps you mean a message queue, I guess?), but yeah Redis (or really, Valkey, now, since Redis tried to change license on everyone) is a very important tool in the toolbox for system design. In-memory databases trade persistence guarantees for raw throughput, and are very well-suited for different workloads than a traditional database is. It’s great for caching, rate-limiting, realtime analytics, leaderboards… Basically anything with ephemeral data or data that can easily be reconstructed from some other source of truth. It’s an optimization tool that can significantly reduce the load on your primary application database for certain workloads (and achieve higher throughput).




  • expr@programming.devtomemes@lemmy.world•Bad news
    link
    fedilink
    arrow-up
    3
    ·
    29 days ago

    Our interview process has the candidate implement a couple (extremely simplistic) API routes, with the specific approach left up to the candidate. It’s more about having something they can talk through and evolve than it is about us looking for any specific approach.

    Basically it exists as a way for us to have a starting point for talking about scalability topics, asking the candidate how they would evolve their implementation as we scale up the system. Does a pretty good job of giving us an idea of how they think through system design problems.


  • I was not talking about your specific lawsuit, or TLS.

    Companies trust other companies all the time, and it’s foundational to most all SLA/SLOs. Any time I’ve voiced concerns around how AI companies are using the data we are giving them (like giving them access to our codebase), it’s brushed off as “we have an agreement with them”. It’s just a load of hogwash. They can and will abuse all data they have access to, just as they have done thus far.

    In this particular case, we are talking about a data center, and it is not at all reasonable to assume that the data that flows to said data center is in any way protected, especially one run by Musk.





  • “not suited for human life”? Really? 10°C is… incredibly mild. Most humans are comfortable at that temperature with just a jacket and maybe a hat.

    Cold weather is all about actually dressing appropriately. Every person I’ve ever heard bitch and moan about it are never dressed appropriately. With the right clothing, you can be comfortable at most any low temperature (obviously, extremely low temperatures require specialized clothing/equipment).

    The same cannot be said for hot weather, where extreme heat is both dangerous and unavoidable.