> Current Common Lisp implementations can usually support both image-oriented and source-oriented development. Image-oriented environments (for example, Squeak Smalltalk) have as their interchange format an image file or memory dump containing all the objects present in the system, which can be later restarted on the same or distinct hardware. By contrast, a source-oriented environment uses individual, human-readable files for recording information for reconstructing the project under development; these files are processed by the environment to convert their contents into material which can be executed.
Am I reading this right that people can (and do??) use images as a complete replacement for source code files?
All the magic of Smalltalk is in the development tools that work by means of introspection into the running image, writing source code in text files causes you to lose all that. Add to that the fact that Smalltalk when written as source files is quite verbose.
Smalltalk does have standard text source file format, but that format is best described as human-readable, not human-writable. The format is essentially a sequence of text blocks that represent operations done to the image in order to modify it to a particular state interspersed with "data" (mostly method source code, but the format can store arbitrary stuff as the data blocks).
One exception to this is GNU Smalltalk which is meant to be used with source files and to that end uses its own more sane source file syntax.
> Am I reading this right that people can (and do??) use images as a complete replacement for source code files?
Images are not replacements of source code files. Images are used in addition to source code files. Source code is checked in. Images are created and shipped. The image lets you debug things live if you've got to. You can introspect, live debug, live patch and do all the shenanigans. But if you're making fixes, you'd make the changes in source code, check it in, build a new image and ship that.
in smalltalk you make the changes in the image while it is running. the modern process is that you then export the changes into a version control system. originally you only had the image itself. apparently squeak has objects inside that go back to 1977:
https://lists.squeakfoundation.org/archives/list/squeak-dev@...
with originally i meant before the use of version control systems became common and expected. i don't know the actual history here, but i just found this thread that looks promising to contain some interesting details: https://news.ycombinator.com/item?id=15206339 (it is also discussing lisp which bring this subthread back in line with the original topic :-)
that's very interesting, thank you, i should have realized that even early on there had to be a way to share code between images. (and i don't know why i missed that comment before responding myself)
but, doesn't building a new system image involve taking an old/existing image, adding/merging all the changes, and then release new image and sources file from that?
in other words, the image is not recreated from scratch every time and it is more than just a cache.
what is described there is the process of source management in the absence of a proper revision control system. obviously when multiple people work on the same project, somewhere the changes need to be tracked and merged.
but that doesn't change the fact that the changes first happen in an image, and that you could save that image and write out a new sources file.
The image is not stand-alone: there should also be a sources file and a changes file (and of course a virtual machine).
"When you use a browser to access a method, the system has to retrieve the source code for that method. Initially all the source code is found in the file we refer to as the sources file. … As you are evaluating expressions or making changes to class descriptions, your actions are logged onto an external file that we refer to as the changes file. If you change a method, the new source code is stored on the changes file, not back into the sources file. Thus the sources file is treated as shared and immutable; a private changes file must exist for each user."
1984 "Smalltalk-80 The Interactive Programming Environment" page 458
~
The image is a cache. For a reproducible process, version and archive source-code.
1984 "Smalltalk-80 The Interactive Programming Environment" page 500
"At the outset of a project involving two or more programmers: Do assign a member of the team to be the version manager. … The responsibilities of the version manager consist of collecting and cataloging code files submitted by all members of the team, periodically building a new system image incorporating all submitted code files, and releasing the image for use by the team. The version manager stores the current release and all code files for that release in a central place, allowing team members read access, and disallowing write access for anyone except the version manager."
I've never heard of anybody doing it, but in theory it could work.
SBCL (and maybe others) use a "core image" to bootstrap at startup. It's not unheard of for people to build a custom core image with the packages they use a lot from the REPL. It's become less common as computers have gotten faster, and most people use systems like Quicklisp or Roswell to automatically get updates and load from source. Of course the SBCL core image is generated from the compiler source code when building it, and the dependencies are loaded and compiled from source initially, too, so there's still going to be source code files around.
You could, in theory, start with the compiled SBCL image, exclusively type code into the REPL, save the image and exit, and then restart with the new image and continue adding code via the REPL. I really doubt anybody uses that workflow exclusively, though. At the very least most people will eventually save the code they entered in the REPL into a source file once they've debugge it and got it working.
I imagine some systems may start out by tinkering with definitions in the REPL in the live system, and then as it grows, the best definition of the system is found in the current state of the REPL, rather than any more formal specification of the system – including by source code.
At some point maybe the system state will be captured into source code for longer term maintenance, but I can totally see the source code being secondary to the current state of the system during exploration.
After all, that's how I tend to treat SQL databases early on. The schema evolves in the live server, and only later do I dump it into a schema creation script and start using migrations to change it.
Besides all answers, this concept exists in modern IDEs like Eclipse, anything JetBrains, Netbeans, Visual Studio,...
Even though their appear to be file based, the plugins API makes use of a virtual filesystem that allows for managing the code as if it was the image based concepts from Smalltalk, Lisp and other systems like Cedar, Mesa, Oberon,....
Also something that many don't think about, databases with stored procedures.
I was using Apple Notes for some “math thinking” the other week. A killer feature for me would be an easy way to input various math Unicode characters (I was just copy and pasting them).
> The biggest downside of Racket is that you can't build up your environment incrementally the way you can with Common Lisp/Sly. When you change anything in your source you reload REPL state from scratch.
I don’t quite understand… I’m using Racket in emacs/SLIME and I can eval-last-sexp, regions, etc.
Ah, I'm using racketmode which doesn't support live state buildup (and the builtin GUI doesn't either). What exactly is your setup? SLIME only has a Common Lisp backend, it doesn't support Racket to my knowledge.
EDIT: ok with geiser and geiser-racket incremental state buildup works really well. I rescind my objection!
I think that should work in racket-mode as well. You can easily send individual sexps to the repl and add to the live state. However, one thing that CL does that Racket doesn't do (afaik) is when you change a data type (e.g. alter a struct), it automatically ensures live code uses the new types. In Racket by contrast I have to either carefully go through all affected forms and send them to the repl, or send the whole buffer to the repl. This does make the whole experience feel more static than in CL.
I learned recently that Racket is an accepted language on LeetCode, which solved the problem “when am I ever going to write lisp in real life…” for me. It’s provided a great excuse.
I have really been enjoying writing it! Paredit and SLIME are addictive.
I tried this, but found it annoying that it will add a slight delay. Totally makes sense if you've been running on caps lock -> escape for a long time. I've bound caps lock -> ctrl and left ctrl -> escape.
I’m in a similar camp to the OP. For me, my joy doesn’t come from building - it comes from understanding. Which incidentally has actually made SWE not a great career path for me because I get bored building features, but that’s another story…
For me, LLMs have been a tremendous boon for me in terms of learning.
Am I reading this right that people can (and do??) use images as a complete replacement for source code files?
reply