Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Building Forth (Book) #68

Open
Immortalin opened this issue Apr 11, 2018 · 40 comments
Open

Building Forth (Book) #68

Immortalin opened this issue Apr 11, 2018 · 40 comments

Comments

@Immortalin
Copy link

Immortalin commented Apr 11, 2018 •
edited

Hi All,

I had been planning to write a book on building Forth for quite a well but never quite got around to it. That was back in 2016 before I knew Forthhub existed. Since communities like Clojure have produced books like the famous O'Reilly Cookbook series and the Rust Programming Language Book, I am going to open up the repo so all can contribute. I will be writing a fair chunk of it, editing contributions and also finding a publisher for dead tree form.
Here's the link, please feel free to bookmark it and I will set up a ebook build system soon.

https://github.com/Immortalin/Building_Forth
@Immortalin
Copy link
Author

Immortalin commented Apr 11, 2018 •
edited

The main content will cover bootstrapping Forth from a scripting language (Python, Ruby or perhaps Go) and then finally to full Forth bootstrapping. Perhaps a implementation in C too. Along the way concepts such as indirect threading will be covered. War stories, famous usages of Forth and any other Forth related content are also welcomed. For code, I am particularly interested in easy, simple implementations of the does and create macros and also a full Forth x86 assembler. The simpler the better. Remember our audiences are not embedded programmers, this is a book for general programmers.

@Immortalin
Copy link
Author

Of particular interest is the relation of Forth to modern stacks, i.e. what role can Forth play in facilitating unikernels and Internet of Things

@Immortalin
Copy link
Author

For bare metal Forth OS, QEMU + this
can be used to skip the whole modern PC initialization mess.

@Immortalin
Copy link
Author

WebAssembly would also be another great target option

@alexshpilkin
Copy link
Member

There is also Pure64 to handle initialization, but the initialization is only tedious, given some architectural knowledge, whereas any more or less modern x86-64 assembly is pure madness. (I think I have two half-functional Forth assemblers for x86 where the instruction descriptions turned out to not be expressive enough in the middle of the table; 80186 is easy, though.)

Frankly, I don't think getting more targets up is that difficult, it's the first target that's the problem. For a book, you might want to choose something clean — ARMv4, maybe, or MSP430, or possibly even the JVM. I looked at WebAssembly, and it looks rather convoluted to me, but it might be the sorry state of the documentation.

@massung
Copy link
Member

massung commented Apr 11, 2018

Obviously anything someone would like to do to further Forth or get new people understanding it is great; I commend - and would endeavor to assist with - any project like this where I could.

That said, I personally feel that if there was Forth "book" that needed written, it would be to take Brad Rodriguez's very good, and yet very dated, Moving Forth web pages and update them both in terms of focus (the documents do bounce around a bit) and their targets; while the Z80 and 6502 are still quite popular for emulating and hobbyists, can you even get a Z8 or PDP-11 any more? Certainly teaching with code that targets C or JavaScript would help many newcomers.

His pages are very old, and I'm constantly in a minor state of panic thinking the domain could be lost at any time.

Of course, as I type this, all I can think to myself is, "if you want this, why not just do it instead of hoping someone else does?" ;-)

@scotws
Copy link

scotws commented Apr 11, 2018 •
edited

@massung You'll be happy to hear that the indefatigable Juergen Pintaske has reformatted Rodriguez' texts and published them in e-book format on Amazon (https://www.amazon.de/Moving-Forth-Internals-Processor-English-ebook/dp/B07BTV7QVR/) as part of his Forth Bookshelf.

@alex-shpilkin I agree with you that x86 assembler is, ah, daunting. One possible target, especially for a book such as this, would be the free RISC-V ISA (https://riscv.org/). It seems to be gaining real-world traction with Nvidia and Western Digital getting behind it for some of their projects, and there is now actual silicon out in the world that you can buy (https://www.sifive.com/products/hifive1/). This would allow you to create a bare-metal Forth with a more modern slant.

@scotws
Copy link

scotws commented Apr 11, 2018

Sorry, I just remembered that Mecrisp-Forth has already been (experimentally) ported to RISC-V assembler, see https://www.forth-ev.de/article.php/20180306213428204 for the announcement (in German) and https://sourceforge.net/projects/mecrisp/files/ for the actual files.

@pebhidecs
Copy link

pebhidecs commented Apr 11, 2018 via email

@massung
Copy link
Member

massung commented Apr 11, 2018

@scotws Wow, thanks for that link! I'm so happy that is the case. While Kindle is fine, if there was a physical copy I would have clicked "buy" already. ;-)

@Brad-R
Copy link

Brad-R commented Apr 11, 2018 via email

@AntonErtl
Copy link

AntonErtl commented Apr 11, 2018 via email

@monsonite
Copy link

monsonite commented Apr 11, 2018 via email

@AntonErtl
Copy link

AntonErtl commented Apr 11, 2018 via email

@pebhidecs
Copy link

pebhidecs commented Apr 11, 2018 via email

@Immortalin
Copy link
Author

@Brad-R would you be interested in donating content or helping out? Moving Forth would be a great guide and template for an updated book in the same vein

@yunfan
Copy link

yunfan commented Apr 12, 2018

x86 is boring, but still we all use it. i guess there's arm and mips could be considered too, also the riscv, since the author of qemu has wrote a online emulator for it, its easy to try for us

for arm, there's rpi and many other clones, for mips just check our wifi aps. and for the internet of things, please give esp8266 a chance, i knew someone has already done(punnyforth)

also i am very glad you mentioned unikernel. its a worth to try. finally there's someone focus on cloud instead of his outdated microcontroller

@quozl
Copy link

quozl commented Apr 12, 2018

give esp8266 a chance, i knew someone has already done(punnyforth)

CForth too.

@yunfan
Copy link

yunfan commented Apr 12, 2018

@quozl thanks for completed me. but i prefer asm based implementation, although c implmented one might be more portable(which made me strugle :[ )

@MitchBradley
Copy link

I am the author of an ASM-based Forth system (Forthmacs) that has been ported to, at least, 680x0, SPARC, MIPS, x86, PA-RISC, PowerPC, Alpha, SaxPY, ARM32, and ARM64. That system has served as the base of an application (Open Firmware) that shipped on several million computers from several different companies.
I'm also the author of CForth, a C-based Forth system that runs on pretty much anything.
As such, I know quite a bit about the tradeoffs between ASM and C based Forths.
Personally, I rarely use the ASM based system anymore. All the tools that I care about, with the sole exception of the interactive assembly language debugger, run the same on both, sharing most of the same source code. There was a time when an ASM Forth was quite useful to me - when my job involved debugging "first silicon" of new microprocessors. At that time, one often could not trust C compilers to generate correct code, and even when they did, it was helpful to dissect problems down to the assembly language level when you could not trust the silicon.
Those days are over. It has been nearly two decades since I have had my hands on first silicon of a new micro. I'd venture to guess that few people on this list have ever had that privilege/responsibility. Furthermore, C compilers are extremely good now, often generating better code than most humans can. At the beginning of my career, there was a time when I had found at least one code generation bug in every compiler I had ever used. Now, I can't remember the last time I found such a bug.
The turning point for me came when I needed to convert a hand-code ASM program for AVR into C. That program, which I had written, had a tightly coded inner loop that did bit timing for a pulse-position infrared modulator. It was becoming difficult to maintain because of an ever-increasing feature set. It needed to be rewritten in C for ease of maintenance, but I didn't have time to go through a full testing and qualification cycle for the new version. Realizing that the AVR C compiler generated good code, I was able to write the C code so that the compiler generated exactly the same - byte for byte - machine code as the old ASM version. So we were able to switch over to the new code base with no additional testing. From then on, new mods could be tested as time and schedule permitted.
With CForth, I have been able to target new instruction sets essentially instantly. That happened with at least ARM-THUMB (a different instruction encoding than ARM32), x86-64, xtensa (esp8266 and esp32). At this point I have just stopped caring about instruction sets. For a lot of my career they were extremely important; now they are uninteresting.

@quozl
Copy link

quozl commented Apr 12, 2018

I agree with @MitchBradley. Unless you have a good reason to code in assembly, it's not worth the bother or the slow down you get by avoiding compiler optimisers. If it is worth it, then at least only in Forth words that are critical to an application, and only if the speed improves. Hand assembly just doesn't always help.

@yunfan
Copy link

yunfan commented Apr 12, 2018

no i am not prefer asm for showoff, its just for me, using asm could understand much of the details.

also i'd saw some embed project's c code(maybe 8051), which's code organization is really ugly, i think that might be the source of bad impression of embeding c code

also i prefer tiny things, not only the executable , but also the tools, this is also the reason i like forth
dont worry, i dont hate cforth, its just if i need to gave my choice for only 1 forth, then i prefer asm implementation

@Immortalin
Copy link
Author

Immortalin commented Apr 12, 2018 •
edited

Using C in a step kind of defeats the point of Forth - a self contained system that is NOT yet another variation on the Bell Labs computing vision

@Immortalin
Copy link
Author

Immortalin commented Apr 12, 2018 •
edited

The only other system that has cultural similarity with Forth is Lisp and that's on another totally different abstraction level altogether. Beyond the islands of Forth and Lisp, everything else is just a sea of curly braces. I have nothing against C per se just that the whole point of Forth is that it brings something new to the table, a table that's currently only occupied by C/C++ and perhaps now Rust.

@Immortalin
Copy link
Author

Though we could always just bring in LLVM..

@MitchBradley
Copy link

C has been described as a "universal assembler". It works reasonably well for that purpose. You can use as few runtime library routines, or as many, as you wish. I have done pretty much every variant, as needs demanded, from bare-metal where my code was running starting at the reset vector, with zero dependence on any code that I didn't write, to fully-hosted systems where Forth sits on top of a huge stack of external OS code, to Forth inside the OS kernel acting as a resident debugger, to tethered systems where Forth runs on a host and takes control of an app on another processor via a tiny stub of a few hundred bytes of code.
Your idea of "the purpose of Forth" is obviously quite different from mine. For me, Forth is a way to do whatever I need to do at the moment. Sometimes it's bare metal, sometimes it's a small part of an enormous system.

@Brad-R
Copy link

Brad-R commented Apr 12, 2018 via email

@alexshpilkin
Copy link
Member

@Brad-R MSP430 is great, actually—even a full commented listing of the whole kernel (a la TeXbook appendix B) should be short enough, although that’s not probably not what you’re going for. It also has very real implementations and development tools, unlike the RISC-V.

One request: if the ebook prepared by Pintaske is going to be the definitive version, could it be released in a DRM-free way? I’m fine with paying for it, but don’t like restrictions that prevent proper archiving.

@Immortalin
Copy link
Author

@MitchBradley good point! What would be your recommendations for the book?

@Mecrisp
Copy link

Mecrisp commented Apr 13, 2018 via email

@MitchBradley
Copy link

I think there are enough Forth implementations already. I would like to see more activity around applications.

1 similar comment
@MitchBradley
Copy link

I think there are enough Forth implementations already. I would like to see more activity around applications.

@Brad-R
Copy link

Brad-R commented Apr 13, 2018 via email

@monsonite
Copy link

monsonite commented Apr 14, 2018 via email

@larsbrinkhoff
Copy link
Member

There's one more Forth option for the ESP8266:
https://github.com/niclash/forthright

@niclash
Copy link
Member

niclash commented Apr 16, 2018 via email

@cwpjr
Copy link
Member

cwpjr commented Apr 24, 2018 via email

@yunfan
Copy link

yunfan commented Apr 24, 2018

@cwpjr can i have a look of that pdf version?

@pebhidecs
Copy link

pebhidecs commented Apr 24, 2018 via email

@cwpjr
Copy link
Member

cwpjr commented May 13, 2018 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests