If you're starting a new operating system project, you might consider using Zig[1] instead of C.
It does all the low level stuff that C does but it has some safety features as well as generics.
The standard library is usable in freestanding mode, even the data structures that allocate memory, because everything takes an allocator as a parameter.
When I started this, I considered writing it I rust. However, I really wanted this to be the tutorial I would have wanted to read several years ago when I first developed an interest in OS, and a "modern" system language like zig or rust would have hurt that goal.
What? No. Because way more people know C than rust. I wanted to be able to reach a wider audience. I know I would have looked elsewhere if I saw found this but I didn't know the language.
There is ATS for the task, with all the C features plus proofs, quantified constraints, dependent types, linear types, interface for pluggable SAT solvers and so much more.
I'm talking about legit languages people are proposing to replace C with that are actually any good (except C2, it was an afterthought and I regret it because it was out of place and unlike the other languages there purposeless). Ada and perhaps C++17 is the only language people are actually recommending to replace C.
Yeah, given the current market situation, you are right.
However, there are quite a few lessons on the manuals, papers and sample code from those languages, to learn about other approaches of doing systems programing.
Lessons that one can bring back to life.
For example, are you aware that ESPOL, the systems programming language for the Burroughs B5500 in 1961, about 10 years before C was created, already had the concept of UNSAFE code blocks?
Burroughs B5500 is still sold nowadays as Unisys ClearPath mainframe.
Or that IBM implemented their research OS and compiler toolchain for their very first RISC processor in PL/8, a safe systems programming language, using a compiler architecture similar to how LLVM looks like?
IBM mainframes still use PL/I variants, although new systems code is a mix of C and C++.
So plenty of ideas there to learn, even if no one is going to start writing new code in those languages.
> Ada and perhaps C++17 is the only language people are actually recommending to replace C.
I would still add Swift to that list, even if that turns out to be just like Objective-C, mainly specific to Apple platforms. Given Apple's roadmap for it.
Swift is more applications programming than OS or systems. It does have those capabilities, but no one is using it for that and it's not really the intent.
However, to the extent that swift (and obj-c!) are used for systems programming, those programs are useable outside of apple platforms.
Swift? I know Apple says they intend it to be usable for OS development, but I would want to see some real OS work (either from them or from others) before being convinced it is a real contender (or is there, and did I miss it?)
Officially launchd and the Dock are now written in Swift (disclosed at WWDC 2017), so even though they aren't drivers or something, they are already a bit lower level than just apps.
Swift 4.0 got all the Unsafe* APIs stabilized, maybe they would be using it a bit more.
In any case, Apple also suffers an Objective-C vs Swift, just like Microsoft used to have C# (Longhorn, Singularity, Midori) vs C++.
Which means, even if it possible to use a certain language from technology point of view, many political wars have to be won as well.
I don't know if this is my conservatism speaking or not, but one problem I see with this approach is that you wouldn't learn as much because more batteries are included. Having to implement more than just a kernel (like the standard library and userland) gives you more of a chance to learn intangibles like what is or isn't appropriate to include in an API or what programs will actually be useful given your overall design. Also, there could be the problem of not having as many resources available in the language you're writing your OS in. That's a huge deal for a less experienced programmer.
That all being said, Zig looks pretty okay. It would be a good candidate to write an OS if you had to because the standard library looks trivially portable so you could test the userland on a known working kernel before getting it working on your new kernel.
As an alternative to this you could use Nim[1] which compiles to C. If you, like myself, dislike C, Nim can be used as a "better C" (you can disable the Nim GC and forget about its standard library, then use Nim as a thin layer over C with modules, generics and other nice goodies)
The extra information is all linked to from the main parts. The reason it's also on the main page is so anyone looking for that info independently of this particular tutorial might find it more easily.
this is excellent, i've been thinking about this as well recently. my goal has been to create a small device/appliance using raspberry pi. i tried out lots of operating systems: raspbian, ubuntu-mate, ubuntu core, windows iot, fedora, etc. i needed a lightweight x windows environment, a decent package manager, and modern linux kernel, overall none of these really seemed to match my needs. basically raspbian was too heavyweight, it was a desktop environment with way too many applications like libre office, etc that i just didn't need. ubuntu core required the use of snaps which are based on docker, i'm hosting an electron app so that's just overkill, i'm thinking in terms of preserving memory, too since there's only 1g on the rpi 3 model b so everything has to be optimized. windows iot requires a windows license, a definite non-starter for me. i even looked at chromium and android, but those didn't seem to be a good match because i want to host a golang service and i don't have a touchscreen. i even looked at fuchsia, but that seems too experimental.
i even considered doing my own os, but ultimately my goal is to build an appliance so that would be a major distraction. dietpi seems good, i will probably pursue that, but thanks for writing the article.
my requirement is to start up a browser, preferably chrome, and open up a certain page, this page is essentially like a digital scoreboard that is shown on a tv. electron embeds chromium and can start up an application super quickly, of course it requires nodejs and a bunch of nodejs libraries, but seems around 79M for the binary, my electron app requires a few nodejs libraries, but this is all text so nominal.
i explored using some scripts to autostart firefox and open up a certain page, but the fact that i needed kiosk plugins from third-parties just ended that for me, i don't want the browser frame to show up. electron is a bit overkill, but thus far it gets exactly what i want done. that piece will be optimized once i find the time.
i suppose this is a one-off project for you or are you creating a device for distribution? if you are creating the latter, how did you clone this to other devices, i'm assuming dd?
I'll probably prep the "gold master" SD card and then leave it to our SD duplicator.
I'm aiming to distribute SD cards that make some of our Pi Zero kits - Radio, Mood Light, etc - near frictionless to get up and running.
I found the use of Qemu + multistrap in this tooling easy to get my head around and I additionally wanted a stateless RAM disk so customers couldn't fubar it.
I didn't have much luck with Buildroot.
I evaluated TinyCore/PiCors too and I love the concept, but it's just a little contrived to ship to beginners. The way it mounts whole packages of overlays/dependencies is clever though- I am doing something similar (inspired by Open Pandora's pnd format) to add a custom build of VLC to PiratePython for our radio kit. Package the whole app + dependencies into a container image, drop it onto the SD card, mount and simlink everything into place.
i also found that there are high level services that are needed to control the device, simple things like restart, changing ip's, upgrading, backing up, etc. i started to create a higher level library for these things using golang, basically a restful service, in this way it could be controlled via an application instead of having to use ssh, this is good for consumer like devices, but obviously opens up a can of security issues.
i will share the code (make it public via github) and write up my experiences developing for this device.
will give it a try, i think raspbian is sort of targeted at the education market for students that need a full blown desktop computer for various things. the fact that it's running debian as a base is great. how does this differ from dietpi?
DietPi is smaller, but I'm not sure exactly what it's missing that Raspbian Lite has. I've looked into it before, but I don't remember the answer. They're both Debian-based.
i noticed dietpi has a great list about the various hardware sbc's and their performance ratings for things like file system storage, etc. time to also check out the asus tinker board or odroid c2.
that's something i considered, but i also wanted to take the opportunity to learn more about creating a tight knit distribution. i could see a lot of appliances coming out of these rpi like devices, there definitely needs to be a tight distribution purpose built for people who want to build appliances similar to mine. i've also always had this task burning in my list of things to do to do an lfs type project.
my device connects to any hdmi enabled tv and provides a digital scoreboard for sports, basically i'm running a chromium browser (via electron) and autostarting that when the device starts. the device is then controlled by an ios app. i ended up using ubuntu-mate, but i don't think i even need gnome, maybe something along the lines of windowmaker, flux, or lightdm would do. i need to save microsd space so that i can have users upload media like images and video, and ubuntu mate really hogs up 4G and above which forces me to have to supply a larger microsd card which ups the total cost.
An in-between option might be to use https://github.com/drtyhlpr/rpi23-gen-image -- it generates a Debian distribution for you with (more or less) only the packages you specify.
The most recent image I built ended up being around 700MB, but I included some stuff you may not need, so you might be able to get that smaller.
i was once a fan of arch linux, loved having the control of compiling everything. how well is this supported for raspberry pi? it's always nice to know that vendors have tested the exact hardware that i'm using extensively, even if it's just another arm device that looks and smells like the others. also, what package manager is used by arch? it's been like 10 years since i last used it.
I can't confirm this, but it feels like Arch linux has surged in popularity over the last ~3-4 years. I ditched Debian for Arch in 2016 and haven't looked back.
If I remember correctly, TCL uses a very simple overlay system based on squashfs. Most (all?) of the tooling and package management stuff is also written in bash.
i found this, seems it's been supported for a long time. i'm not ecstatic about the wrappers in ruby/python for gpio, but then again i don't need it at this point in time.
I am a great fan of FreeBSD and I run it as my main OS on one of my two laptops that I use daily.
That being said, last time I tried to run FreeBSD on a Radpberry Pi was not a pleasant experience. That was several years ago though so things may have improved.
FreeBSD/arm64 now has official binary packages for 12-CURRENT. RPi3 runs in 64-bit mode with all 4 cores. For the most part, everything is good. But the vc4 KMS driver is not ported yet, so no GPU accelerated desktop for now.
Very nice! I've been considering attempting to port an OS that I've been writing to an ARM device like the Pi, but I've never really looked into it much. With that said, I'm surprised the booting portion is so easy, I was under the impression that for the Raspberry Pi (And ARM devices in general) it was more complex. I suppose though, this is started after the bootloader that's normally on the SD card correct? I would imagine that makes things a fair bit simpler if that's the case, since the bootloader would take care of all the secret sauce.
Yep the proprietary bootloader on the Raspberry Pi handles most initialisation leaving the SoC in a usable state. If you want to see how things like DRAM init work you can take a look at the open source reverse engineered firmware at https://github.com/christinaa/rpi-open-firmware . It isn't complete but it is just about enough to also boot the Linux kernel.
I do plan to implement paging. I already manage all of the physical pages, I just haven't enabled the mmu yet as I wanted to get more fun features in first
I just want to thank you for posting this over here, I've been looking for something like this. I need something where I can develop and understand code as I go, so not a completed system, but that also provides final code so I can have something that actually works. I tried following the OSDev pages to try an x86 kernel, but I gave up when I couldn't get interrupts to fire.
I think for the cost, a RasPi is always a nice board to tinker with. And having a bunch of MicroSD cards to experiments makes it easy to test multiple scenarios easily.
It does all the low level stuff that C does but it has some safety features as well as generics.
The standard library is usable in freestanding mode, even the data structures that allocate memory, because everything takes an allocator as a parameter.
Here are some examples:
* i386 OS: https://github.com/AndreaOrru/zen
* ARM (raspberry pi) OS: https://github.com/andrewrk/clashos
[1]: http://ziglang.org/