Idea: Tiny text editor

[jkotlinski]

v is bloated, code is over 4 kb. Another problem is that text buffer grows unbounded.

I propose to remove v from default included modules and replace it with t, tiny text editor.

The vision for t is that memory consumption will be no more 1 kb, including text buffer, and never exceed this fixed size.

[Whammo]

A sliding window open file text editor?

[burnsauce]

As long as you leave v on the disk, I'm happy :)

[jkotlinski]

@Whammo I haven't thought it through properly. But it would be nice to have the regular Forth "virtual memory" setup, where you have maybe three 1 kb-blocks mapped to RAM, that are swapped in/out to disk as needed.

One problem is that random access is not possible with regular PRG/SEQ files. But I think it should probably work fine to use REL files for Forth source code instead.

In short, I think I'd like a more block-like setup for source code, just to avoid the practical problems that come with having files of unlimited length. Of course, this is a very big and deep change so I'm not even sure if I'd ever get started with this one :-)

[burnsauce]

One problem is that random access is not possible with regular PRG/SEQ files. But I think it should probably work fine to use REL files for Forth source code instead.

Yes it is. See U1 aka BLOCK-READ. You need to navigate the blocks by yourself but you can read a PRG file a block at a time.

[polluks]

I prefer the REL approach. You don't have to manage the disk structure yourself, but let the DOS do it :-)

[jkotlinski]

That would be my hope, too, that REL files would require less and more simple code. But there could be drawbacks with that approach that I’m not aware of.

…

On Sat, 26 Dec 2020 at 02:13, Stefan ***@***.***> wrote: I prefer the REL approach. You don't have to manage the disk structure yourself, but let the DOS do it :-) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#311 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAY34O22AYQX5QL4R5Y7HVDSWU2DFANCNFSM4VBXO3UA> .

[Whammo]

Copying screens to memory seems to be the key to swift pagination, and reading the screen as a file seems to be the key to saving. Perhaps if the compiler were interfaced as a device it would be interesting?

[Whammo]

Evaluate works nicely though! :)

[Whammo]

RLE would also speed pagination and compiling.

[Whammo]

We're almost there, it's easy to read and write sectors.

[polluks2]

If you use REL the DOS seeks the record, but raw sectors require your own calculation.

[Whammo]

So a line-length record REL file created on the fly by reading the PRG, edited by inserting and deleting records then saved to PRG.

[Whammo]

Although, maximum record size is 254 bytes. Four of these make one screen with four bytes left over for each record.
These four bytes could be logical forward and backward links for out-of-sequence inserts to be justfied at save.
For a continuous scroll you would only need 1500 bytes in RAM

[Whammo]

Each sector has a track and sector pointer to the next sector in the file. It also points out when it is the last sector in the file.

[jkotlinski]

This feels like the most important remaining improvement.. Hope to find time and energy to start working on it this year.

[jkotlinski]

I have been thinking about how to do file access, especially SEQ versus REL files. The pros and cons are pretty deep.

The main benefit with plain SEQ files, without any extras, is that it is minimally complicated. Especially when transfering to/from PC.

I think a way forward might be to add some of the File word set, enough to allow random read/write/append access to SEQ files. Under the hood, it would read and write sectors directly.

This setup is not super efficient. Maybe the worst is when inserting or erasing space in the start of a file, then the old file contents would need to be completely re-written, to handle the move to a new position.

A way to mitigate this problem is to do screen-based editing. Inserting/erasing screens is not something that one would do all the time, so it is kind of OK if it is slow.

Navigating between screens might be slow, as disk i/o happens as a result of navigating. But that is probably livable and hard to avoid.

[jkotlinski]

Pygmy Forth has been mentioned as an elegant model for block-based file editing: https://github.com/utoh/pygmy-forth/blob/master/pygmy.txt

gforth also has block-based text editing: https://stackoverflow.com/questions/48837115/does-gnu-forth-have-an-editor

[ekipan]

I've mentioned before that durexForth is my first time touching the C64. I've never written any programs that touch the disk yet, but in the Forth spirit of keeping things simple how about saving each block in a separate file named, say, "b001" "b002" etc.? Probably a bit wasteful of disk space, but if I understand saveb and loadb correctly it seems like the simplest reasonable thing. I wonder if that's what you meant when you said

But I think it should probably work fine to use REL files for Forth source code instead.

[jkotlinski]

saving each block in a separate file named, say, "b001" "b002" etc.?

Technically, that would work just fine. And you are right, it would be the simplest reasonable thing. Maybe it is really the best idea. The files that comes with durexForth would stay as is, but the blocks created by the tiny text editor would be stored like "b001" "b002".

What I had in mind with REL files was more like how Gforth is described in the link two comments above. You create a file "mygame" and that file internally has 1024 byte big blocks.

[ekipan]

I'm working on this "simplest" thing right now, a block with a single buffer. I think it's debugged now? Scratching files in VICE is sometimes flaky for me when I'm using snapstates (maybe I'm doing it wrong).

Some edit history of this post:

• Forgot the scratch the file first, fixed the code.
• Several trivial code edits. Now moved to a gist.

[jkotlinski]

I have some idea to create Block wordset, that a future editor can be built on. I should just get working on it.

Some functionality for an editor could be:

F1=previous block
F3=next block
F5=save
F7=execute
F8=exit

Maybe that is all functionality needed.

[ekipan]

About that. I might have written one (z: a block editor) as an exercise to see if I could, over the last week.

It's not quite as small as you wanted. 5.3K of source compiles to 2058 bytes. But it's rather well-featured. Only thing I've left to add really is line join. I didn't really want to make a repo because then I'd have to take responsibility for it :P

[ekipan]

Though now I want to write one without any interactivity at all, just commands at the interpreter using the C64 screen editor. I'm sure it'll be a lot smaller. It was a fun exercise though.

[ekipan]

Well, I think I wrote it. Less than 400 bytes compiled 🎉.

require block
marker -- \ -tt--

create scr 1 ,
: edit dup scr ! block drop ;
: line 32 * scr @ block + ;
: wipe 0 line 1024 bl fill ;
: scrub 0 line dup 1024 + swap
  DO i c@ bl max 'Z' min i c! LOOP ;

\ [u-] ttype [-] llist aa bb cc dd
: 00. 0 <# # # #> type space ;
: tt  dup 00. ." rr " line 32 type cr ;
: ttt DO i tt LOOP ;
: aa  8 0 ttt ;
: bb  16 8 ttt ;
: cc  24 16 ttt ;
: dd  32 24 ttt ;
: ll  aa bb key drop cc dd ;

\ "[u-] wwhiteout rreplace
: in- source >in ! drop ;
: in/ source >in @ /string 32 min in- ;
: blf 32 bl fill ;
: ww  line blf in- ;
: rr  line dup blf in/ rot swap move ;

\ "[u-] iinsert xxdelete
: xx  >r r@ 1+ line r@ line
  992 r> 32 * - move 31 ww ;
: ii  >r r@ line r@ 1+ line
  992 r@ 32 * - move r> rr ;

Would have preferred bblank and ddelete, but aa bb cc dd are good listing names.

Either it needs a load word that patches 32-character lines into \ or you just use ( comments in your blocks 😛

vs buffer is a good source of text to play with. $a001 0 line 1024 move ss. aa bb cc dd ll type lines with rr already on them so you can screen edit and press enter as though you were using the basic editor. Or you can replace the rr with ww ii xx.

With parse in durexForth v5 I think you could rewrite in- in/:

: in/ $d parse ;
: in- in/ 2drop ;

[ekipan]

The interpreter's ok prompt does like to overwrite line numbers. I could at-xy after an rr but that's also slightly inconvenient to use, having to cursor back to where you were.

[jkotlinski]

I haven't dug into those editors of yours yet, but I really like the code!

[jkotlinski]

Fastloader is a real good point. Actually, that speaks a bit for the blocks-as-prg-files concept - since there are fastloader cartridges that speed up LOAD for free. I will try to do some measurements on this, right now I am mostly guessing.

[jkotlinski]

OK... when testing, it seems faster to load a 1024 byte big .prg file with LOAD, than to load 4 sectors with the U1 command. So yeah, then it seems to me, that there is not much benefit with the sector approach.

[jkotlinski]

Another Block word set... the next hurdle will be to do LOAD, which requires modifying the interpreter.

https://github.com/jkotlinski/durexforth/blob/blocks/forth/block.fs

[ekipan]

An alternative I was thinking of writing is instead of tracking lru, having a 1-to-1 mapping blk->buf, say 4 buffers and 3 and. So blocks 1, 5, 9, etc are stored in buffer 1, blocks 2, 6, 10 in buffer 2, etc. Adjacent blocks would stay in memory while paging in an editor.

[jkotlinski]

Hmmm... what? How can four blocks be in a single block buffer? I don't get it.

EDIT: OK, hmm... does it mean that if you select block 0, 1, 2, or 3, the buffers would always contain blocks 0, 1, 2 and 3?

[ekipan]

1 block would loadb "b01" into buffer 1. If it gets updated then 5 block would flush and then load into the same buffer. Blocks 1 2 3 4 5 6 7 8 are always assigned to buffers 1 2 3 0 1 2 3 0.

[jkotlinski]

OK, that is pretty clever, I will try to update the code :-)

[ekipan]

There are pros and cons of course. Probably a bit smaller/faster code, but more restrictive in use. Can't have blocks 1 and 5 in memory at the same time.

[ekipan]

I guess 99KiB to play around in is fine but I did like the extra freedom of a third digit. :P

[jkotlinski]

Umm... how many 1024 bytes files will actually fit on a 1541?

[ekipan]

Way less than 999, sure, so most of the number space would be empty, just available if the programmer wanted to put their programs there. It's not really important though, and it has some cost in the code.

When I looked it up I found capacity is 170KiB? And I have no idea the overhead of files (I presume a 1024 bytes file wastes a bunch of disk space).

[jkotlinski]

It seems like there is plenty of room even after writing 99 blocks, so I added the third digit.

[Whammo]

Large files could be loaded by changing the pointer to the first sector, and subbing a last sector mark where needed.

[ekipan]

Something I took note of that's relevant for anyone following the thread: the filename virtual mapping introduces a (small) problem.

[Whammo]

1.1.4 Some Pacts about a 1541 Diskette
Number of Tracks: 35
Sectors per Track: 17 - 21
Bytes per block: 256
Total number of blocks: 683
Number of free blocks 644
Entries in the directory: 144

[ekipan]

Should I maybe make a separate show & tell discussion thread to track my 2 editors instead of continuing to post here?

Edit: Made. Cross-ref.

[jkotlinski]

A note about progress so far. The "one-file-per-block" approach seemed to work, but felt inefficient on a real drive. The main problem is that the drive head needs to move between track 18 and the data tracks a lot.

[jkotlinski]

Another idea for block management is mentioned in the Commodore 1541 Users Guide.

Allocate disk blocks with the B-A (BLOCK-ALLOCATE) command, and keep track of their locations in a .seq file. In that way, random block-access can coexist perfectly well with regular files. This seems like a really nice solution to me.

[ekipan]

I've adapted your block.fs to my own needs and style which I understand is not really mergeable into dF, but two points of interest:

1. Should empty-buffers also perhaps erase the dirty flags? I'm not sure but I think I might be surprised if I emptied-buffers and then a later block load created a file b000. Maybe save-buf should check if the assigned bbi is not 0.
2. By using a fixed buffer for both the filename and the scratch command you can delete much of the string handling code.

Though it sounds like you want to explore other options besides the file-per-block concept. FYI if you are interested.

[Whammo]

I used to factor all my code in a similar manner. Then I saw all the work put into it's streamlining my submissions, and I asked myself, "Why should Johan have all the fun?"
😆

[jkotlinski]

A challenge: How to port this code to Forth.

[Whammo]

Direct access drive programming with durexForth #389

[Whammo]

10 open the command channel
20 open a buffer call it channel 5
30 write something say, "DATA" to the buffer
40 allocate variables, track 1 sector 1
50 send block-allocate to the command channel
I have to assume after the command string is semicolon separated binary data.
60 read error channel
70 - if already allocated try again
80 send block-write to command channel
90 ?

[jkotlinski]

That was not so difficult, to create a block-allocating word. The io module is really helpful.

\ block-allocate. returns -1 on success
: b-a ( drive track sector -- flag )
<# 0 #s bl hold 2drop
   0 #s bl hold 2drop
   0 #s bl hold
       'a' hold
       '-' hold
       'b' hold #>
$f $f open ioabort $f chkin ioabort
chrin begin chrin drop readst until
clrchn $f close '0' = ;

[Whammo]

I still have not mastered format

☹️

[ekipan]

1. <# sets a pointer after a fixed buffer,
2. hold moves the pointer back and stores in it,
3. #> 2drops, then gives you the pointer plus the length. That's the basics.

# does a 2divide by base, converts to digit and holds. #s does # in a loop. sign holds a '-' if given a negative number. I would tell you to dump to get an idea but dump itself prints numbers so that wouldn't work :P

[jkotlinski]

Not nearly ready for merge yet, but there is the start of another BLOCK system here: #554

It implements BLOCK, BUFFER, FLUSH, SAVE-BUFFERS, UPDATE, EMPTY-BUFFERS and LIST. LOAD is still to be done.

Before using those words, one needs to do a one-time setup thing: call 20 CREATE-BLOCKS to grab 20 Forth blocks on disk, and save their sector locations in a file named blocks. After that, there are 20 Forth blocks that are ready to use.

I hope this setup will work very well, but let me know if you have any concerns.

[ekipan]

Since block.fs and v.fs are incompatible, I wonder if it's worth inventing a word, say prohibit:

( wordlist.fs )
: prohibit ( "name" -- )
parse-name 2dup find-name if
rvs ." has " type abort then 2drop ;

That you could use like:

( v.fs )
prohibit ---block---
marker ---editor---
( ... )

( block.fs )
prohibit ---editor---
require io
marker ---block---
( ... )

[jkotlinski]

I think it is fine that they cannot be used simultaneously. It is a temporary problem, the long-term aim is to retire v.

It will be warned for in the documentation.