10 years ago, I'd said "at least third normal form"... but today: Whatever gets the job done. When the application is not really dependent on weird queries (e.g. just a blog), screw the normal forms and design your schema to use the least number of queries for a certain task. Nobody understands three lines of code of queries with left and right joins.
On the other hand, if your bookkeeping application uses a database try to keep things as tidy as possible.
I'd say this applies to virtually all best practices, patterns, architectures, etc. If you're doing something very simple, who cares about modularity or any kind of code hygiene? I don't. But what happens in reality? Simple and small systems or experiments grow, one little addition at a time, and we all know the mess that ensues.
So in my understanding, the question posed only applies to at least moderately complex systems, which is where engineering skills matter. And in that context, learning what distinguishes a good database design is obviously very valuable, not to say crucial.
> Nobody understands three lines of code of queries with left and right joins.
Not sure if you're being flippant, but a) this is not true, and more importantly b) why is it that we don't expect programmers to be at least as fluent in SQL as in other, less important, languages?
If you’re doing any moderately complex analysis of the data in your database, the ORM will quickly start falling down. Abstracting the query layer into the application codebase is nice, and mapping entities to objects is nice, but ORM is not a silver bullet. Learning what makes for a good schema vs a bad schema and how to avoid N+1 loading or query problems is important no matter what.
ORMs aren’t bad, but learn their escape hatches or else you’ll have a hard time doing more complicated things.
It also usually forces your design towards the entities themselves rather than the specific way they’re stored, which positions you better for switching to a completely different storage system in the future if, for instance, it’s becoming too slow or expensive to maintain everything in a traditional big name RDBMS.
> forces your design towards the entities themselves
I agree that it's very important to not let the physical schema leak into the rest of the system, and to have a strong conceptual model (aka entities and relations). This has been well understood for almost half a century: https://en.wikipedia.org/wiki/Three-schema_approach
But I don't think ORMs are in any special position to help with this. They typically introduce so much other confusion that they tend to divert attention from designing both a good physical schema and a good conceptual model, and maintain a sensible mapping between the two. This can be done with RDBMS views for example, with a fraction of the overhead of an ORM. Most ORM-based code bases I've seen leak tons of db-level details.
> switching to a completely different storage system in the future
Designing for this eventuality is not healthy IMO. If you get there, it will be a so-called "good problem to have" and you will have to deal with whatever unique challenges you face at that level. We might as well be writing code with the possibility of "switching to a completely different programming language in the future" in mind. Yes, clean, modular code will help, but beyond that, not committing to the capabilities of the tools you have chosen will harm you system.
I disagree with pretty much everything you said. :)
I think there are plenty of bad ORMs and there are plenty of ways to use the good ones in a bad way, but that doesn’t mean that they aren’t providing the value I mentioned. For instance Entity Framework Core with code-first migrations has you designing the data models themselves, then wiring up relationships and other metadata (indexes, keys, etc.) in the DB context itself - your actual entities are completely portable and have nothing to do with the db itself outside of being used by it.
And sure, needing to switch to another storage system may be a good problem to have... that doesn’t mean you should explicitly tie all of your code to one particular RDBMS. If a user is a user is a user, it shouldn’t matter to anything else in your codebase how or where it is stored, it should still be the same entity. Moving those users from your SQL Server to Mongo or to a third party like Auth0 or an Azure/AWS/etc. federated directory service doesn’t change the fact that every user has an ID, an email, a name, etc.
It's well-known to be a topic that splits opinion, so I'm not surprised we disagree :) To me, "designing the data model", "wiring up relationships", etc doesn't require an ORM. On the other hand, I do agree it's good to have some tooling around it and that's something many more bare-bones frameworks (ORM or not) are lacking.
I don't hear people talk about "coding for the web, but design so that you can easily switch to deploy as a Windows desktop app." Or "write it in Python, but in such a way that we can easily swap to OCaml." It seems to me databases are uniquely treated this way, as some kind of disposable, simple piece of side equipment. Again, modular code will always be easier to migrate, but I prefer to take full advantage of db capabilities, as it results in much less code and frees up time and mental space to focus on a good conceptual model and physical schema, among other things.
I've never used EF, so I might not see what you are seeing.
> It seems to me databases are uniquely treated this way, as some kind of disposable, simple piece of side equipment
This is exactly right - lots of people are still cargo-culting rules of thumb that no longer make any sense.
This was an artifact of the last generation's commercial DB market. Open source DBs weren't "there" yet; a combination of real limitations and risk-conservatism kept companies shoveling huge amounts of money at vendors for features and stability now provided by `apt-get install postgresql-server`.
If you just lit seven figures on fire for a database license, you're not hungry to do it again, so you wanted all your software to be compatible with whichever vendor you just locked yourself in to. And certain DB vendors are very well known for brass-knuckle negotiation; if you could credibly threaten to migrate to $competition instead of upgrading, it was one of the few actually useful negotiating levers available.
Today, open source DBs are better than the commercial ones in many situations, certainly not worse in general use, and the costs of running a bunch of different ones are far lower. Not to mention, the best way to win a software audit is to run zero instances of something.
Very useful historical perspective, thanks! Confirms what I had pieced together, that DBs used to be a big liability for organizations, with a special clan (DBAs) of people gatekeeping and introducing patterns that programmers found infuriating. Hence the hatred towards stored procedures, layered schemas, and databases in general. It's probably important to keep stressing, as you do, how different things are now. It's only been a fews years that Postgres has had row level security for example.
DBAs still have their place. In my shop, we have more DBAs than infrastructure people.
When you have a small team working on a given tool that only really needs to manage its own data, it really doesn't matter. But some point, you do need expert gatekeepers to tell engineers when they're Doing It Wrong when there are many heterogenous clients accessing large datastores for different purposes, complex audit requirements, etc.
Yes specialization is often useful. But the divide between developers and DBAs seems to have been similar to the dev/ops divide. Probably still is in many places. There is always a need for seniors or specialists to guide work, I'm not against that. But something like DevOps for RDBMS is needed. DevDat?
The number of teams who design a data model & ORM layer "just to make it easy to move later on": Lots
The number of teams who eventually move to a different data store? Almost zero.
Getting "locked in" to a database is a non-issue. In fact you should get locked into a database system, provided you picked a good one to start with. Most teams never even scratch the surface of what a powerful DB like Postgres can do for them and it breaks my heart every time.
No it doesn't. Surfing had it's boom and bust and now is a normal sport/hobby like skiing or everything else.
In the future one could also argue: tech industry contradicts basic capitalism: nowadays anybody has a smartphone, the phones cannot get any better, and everyone takes care of their smartphone because they don't want to pay alot money for a new one.
Yeah I'm decidedly confused about the author's conception of surfing. They seem to be under an impression that the sport/hobby has inherent philosophical values.
I think that's a common conception of many surfers. Surfing is somewhat entrenched in philosophy. Surfers have philosophical concepts of individualism, territory, ecology, morality, and existential concepts surrounding their place in nature and what is truly important in life to be happy.
Surfers can be considered to live their life like any spiritual individual. Their _god_ is the ocean, their pursuit is happiness in the moment recognizing risk and danger is a necessary component for a full life.
I wonder how that came about, and how a culture like that could have come to persist into the 2020s?
My initial thought is to the limited range to actually peruse the sport, leading to a closer knit culture. It's coastal dependent and coast culture always has a distinct identity in my experience, more so than Mountain culture in my experience.
It's more that the core that would consider themselves the core have some widely shared values. Lot's of "lifestyle"/"adventure" sports are similar. And these are the people that started the brands that ended up not living up to their values.
Surfing is inherently limited in its scope by the number of surfable beaches. There's not room for unbounded expansion of the industry, unless you count surf-inspired lifestyle apparel.
IMO, the key to understanding statistics is learning how to quantify uncertainty over datasets. The key to cheating is to either downplay, (or overplay), the interpretation of this uncertainty as it pertains to the conclusions you can draw.
I see a lot phrases like "studies show that" these days. Somehow in our science-based, sophisticated society everybody likes to throw studies at each other to prove their own narrative. But I don't understand them. I am not that statistical illiterate, I know the difference of mean, mode and median and stdev (and when to use what). When I dig deeper into one study I'll find hypothesis testing methods like p-values, r^2 and whatever ("our hypothesis was proven because p > 0.9"). But here my knowledge ends. If p>0.9 is that good? or did they just tune the data to get that high p-values? or is the whole method garbage and the study could not get replicated with the same p value?
And I want to know how to cheat with statistics, because since these studies are made by people, whom might get paid to prove a certain point (e.g. "My institute gets paid by Mars, hence I'll downplay the effects on health of sugar in daily nutrition and amplify the positives effects of <some chemical found in chocolate>"). or they just want to show significance for their research, because they worked the last 10 years on it and it's "their baby".
That is a nice take on digital minimalism, but in my opinion you could also just delete most of the attention-sink-apps and enable black-and-white-screen on the smart phone you're currently using, instead of buying yet another device.
Ah, that's a good reference for a more modern "control loop" idea. I agree that there are many similarities with that specification and I need to read in more depth. Thanks!
Honestly, there's limited advantage over vanilla HTML5 if HTML is actually what you were going to write. The website at moxie.rs is plain HTML/CSS/etc. The reason to reach for tools like this on the web is if you want interactive state, complex data transformations, etc.
I think it will be hard for moxie-dom to compete directly on ergonomics with purely web-focused tools (especially JS frameworks), because the underlying tools will always need to maintain some distance from platform semantics.
I've never understood all this excitement about REST. If you need a special service, just code a small shell program and register it at /etc/inetd.conf. if you need security, pipe your text file thru OpenSSL/pgp or use ssh instead.
>You need the right inventives to tackle these challenges. And frankly governments need to reduce the friction in terms of money allocation and regulations to get things done.
Churches did this in the past (with all the bad side effects). But I guess in more secular regions government needs jump in (with all the bad side effects).
What about the software side? Fairphone 3 supports Android 9, Fairphone 2 supports Android 7.
I see no use in a smartphone with replaceble hardware parts if I cannot install at least security patches for my phones OS on the long run.
Fairphone has had very solid software support. It is together with Nexus and Oneplus the hardware most enthusiasts choose, so there is a functioning community around it.
It is not only well supported in LineageOS, but also most alternative phone OSes such as UBports (Ubuntu Touch), Sailfish and postmarketOS have or have had people actively using the hardware.
It's quite a good track record considering the Fairphone 2 is already twice the age the minimum software update guarantee from Google for their own hardware.
Absolutely not. I don't download Windows updates from "enthusiasts" or "the community", and I don't consider community security patches to be "solid software support".
There's nothing wrong with LineageOS and I'm not criticizing their work. They do a great job keeping phones supported past the time when manufacturers have dropped support. But the Android community is far too willing to trust ROMs downloaded from forums that include who-knows-what and calling it "solid software support".
Solid software support is updates from the manufacturer, the company I originally trusted to provide updates when I bought the phone. If I have to rely on the work of people who once called themselves "Team Douche" [1] because the company who released the original software has stopped support, I'm not considering that "solid software support".
You don't download Windows updates from your laptop manufacturer either (unless you have a Surface.)
Comparing LineageOS with the custom ROMs on XDA really misrepresents their professionalism. LineageOS development does not take place on XDA. It takes place on their own infrastructure like any other high-quality open source operating system. Builds are made from public sources and signed by their build server (they definitely don't just include "who-knows-what.") They ship security patches faster than any manufacturer (except Google,) and I don't see any reason not to trust them as much as or more than your manufacturer. LOS-supported phones are well supported.
>You don't download Windows updates from your laptop manufacturer either
True, but that's a minor difference. The normal way to get Android OS updates is through your device manufacturer, like the normal way to get Windows updates is from Microsoft. The normal way to get Windows updates is not from a torrent found on some random forum, and that's not the normal way to get Android updates either.
>Comparing LineageOS with the custom ROMs on XDA really misrepresents their professionalism
I agree and that's why I said I'm not calling them out... but again they're not the normal way to get OS updates. It's out-of-band and while LineageOS may be professional, it's not professional to have to abandon the normal update mechanism just to continue to receive OS updates.
LineageOS is still community support, and community support is not "solid software support". It's community support.
What's normal shouldn't really matter. If anything, the "normal" way of receving Android updates from your manufacturer is kind of bad. I bet if laptops followed the same model, you'd see exactly the same things, like late security updates and devices losing support after two-three years. In general, I think I'd rather see my OS distributed by people who make software rather than people who make hardware.
> I agree and that's why I said I'm not calling them out
You kind of are. You've mentioned that you don't want to download OS updates from forums a number of times in response to people mentioning LineageOS, but official LOS ports are not distributed through forums (or torrents for that matter.) You also brought up an old, somewhat embarrassing name that their development team apparently used to call themselves. It seems a lot like you're trying to discredit them.
I'm not calling out Lineage as bad, I'm calling them out as unofficial community support, which I am saying is not "strong software support". Lineage is not distributed through forums but Lineage is a fork of CyanogenMod which got its start... being distributed through forums. From some random group of people who chose to use an embarrassing name. Lineage would not exist today if CyanogenMod hadn't been so popular. And the Android software update process was so broken that people had no choice but to download this software, distributed on a forum, from a group of people who willingly chose a wildly unprofessional name for themselves. Does that sound normal? Does that sound okay? Is any other OS distributed in this fashion?
I'm not trying to discredit Lineage. I'm trying to discredit the Android software update process. Lineage makes it better but it is not strong software support if you have to rely on the community to deliver out-of-band security updates.
Lineage is better than nothing, but only because the current Android update process is so broken. If Android was any other OS, getting updates from the hardware manufacturer or the community would both be laughed at. This process only seems normal on Android because the Android update process is so hopelessly broken because the original software vendor refuses to distribute their own updates.
But Debian images are from enthusiasts and the community. Also, your windows updates don't come from HP, and they often break things. Lineage isn't much different than most Linux distributions.
My guess is maybe you'd be happier with an iThing.
>Conflating LineageOS with forum sourced ROMs is FUD
Nonsense. CyanogenMod started off as a forum-sourced ROM on XDA just like anything else [1]. It may have grown beyond that, but only because people blindly trusted it and installed it on their phones 10 years ago, and XDA is still distributing hacked ROMs in the exact same fashion today.
I'd say a Fairphone is better supported than a Nexus (no experience with Oneplus). For example, the Nexus 5 is no longer supported, even by lineage (https://wiki.lineageos.org/devices/hammerhead/), although they do say:
> A build guide is available for developers that would like to make private builds, or even restart official support.
Edit: I should add that although I've already ordered my Fairphone 3 I've not had a chance to actually test it out yet. If anyone is interested I can post a 25 EUR discount referral link?
Does it receive real security updates (i.e. all the CVEs that are supposed to be fixed by that patch level are actually fixed), or "whatever we can reasonably fix" updates where e.g. the kernel is left on an outdated version without fixes because the fixes never got backported and upgrading the kernel is impossible due to limited support from e.g. the SoC vendor?
Is that really support throughout the stack? I understand they can provide userland updates, but kernel and driver updates? I’m not sure it’s possible for anyone apart from the chip manufacturer to do that, unless the drivers are open source.
That's like saying that you insist on using the Windows XP image that came with your laptop forever and will only update through the vendor rather than just installing the latest Ubuntu image.
We really need to stop giving a fuck about manufacturer images and demand that the industry ceases to lock bootloaders so we don't have to wait on them to update anything.
Unlocked bootloaders everywhere would be nice but there are already manufacturers that give you that across the range. The big issue is that none of the drivers are upstream so even with LineageOS you're stuck on and old and often unpatched kernel. When a new Android version comes around it's common for support to be dropped for devices. Right now there's no Ubuntu-like experience with Android. Get a Pixel or an Android One phone if you care at all about security and access to new Android versions.
No, it's like saying I want to get Windows Updates from Microsoft and not some random person on a forum. It would be great if Google published a full Android release that worked on all phones and had the manufacturer's best experience in mind, but they don't. So the next closest thing, the company I choose to put my trust in, is the maker of the phone. The company who originally installed the OS in the first place.
I want updates from a company who can be held accountable. Not some random ROM from XDA.
I bought one of the second batches of the first Fairphone - I love the idea and the phone was great for a proof of concept, but it was disappointing to find it was never going to be upgraded beyond Android 4.4.
It kind of broke the promise of repairable, sustainable hardware if the chipset provider obsoleted the device through lack of updates in less than a year of ownership.
While 4.4 wasn't brilliant, I could have continued on with it - but I was uncomfortable using such an increasingly insecure OS.
FP2 still gets Android security patches - the main problem by now is that the drivers for the hardware are not properly updated to Android 7 (due to the manufacturer not providing any updates)
The Fairphone 2 can run Android 9 via LineageOS. The official versions (the one with and the one without OpenGapps) still get security updates. Can you say the same about Nexus 5X with Android 8.1?
On the other hand, if your bookkeeping application uses a database try to keep things as tidy as possible.