/
parse-word.html
258 lines (190 loc) · 8.38 KB
/
parse-word.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
<title>PARSE-WORD</title>
<h3><code>PARSE-WORD</code></h3>
[ <a href="rfds.html">RfDs/CfVs</a> | <a href="proposals.html">Other proposals</a> ]
<!--
<h4>Poll standings</h4>
See <a href="#voting">below</a> for voting instructions.
<h5>Systems</h5>
<pre>
[ ] conforms to ANS Forth.
[ ] already implements the proposal in full since release [ ]:
[ ] implements the proposal in full in a development version:
[ ] will implement the proposal in full in release [ ].
[ ] will implement the proposal in full in some future release.
There are no plans to implement the proposal in full in [ ].
[ ] will never implement the proposal in full:
</pre>
<h5>Programmers</h5>
<pre>
[ ] I have used (parts of) this proposal in my programs:
[ ] I would use (parts of) this proposal in my programs if the systems
I am interested in implemented it:
[ ] I would use (parts of) this proposal in my programs if this
proposal was in the Forth standard:
[ ] I would not use (parts of) this proposal in my programs.
</pre>
<h5>Informal results</h5>
-->
<h4>Status</h4>
Due to the <a href="#name-conflict">name conflict</a>, I have shelved
the RfD with this name. However, there has been enough support for
the new name PARSE-NAME for this functionality, so this RfD is
replaced by the <a href="parse-name.html">PARSE-NAME RfD</a>.
<h4>Change history</h4>
<dl>
2005-04-22 A section on white space was added. A name conflict was
discovered, a strawpoll for a new name was held, and this RfD was
replaced by the one for PARSE-NAME.
</dl>
<h4>Problem</h4>
How do we parse a word from the input stream?
<p>PARSE does not skip leading delimiters, and you cannot specify that
you want to parse for white space.
<p>WORD skips leading delimiters, but you cannot specify parsing for
white space, it creates a counted string (not the preferred
representation), the length of the string is therefore limited, it
requires a separate buffer (typically limitinmg the string size even
more, and the copying to that buffer consumes time); WORD also
requires passing a delimiter, although skipping leading delimiters
only makes sense for white-space delimiters. ANS Forth does not
specify the lifetime of the resulting string precisely.
<h4>Proposal</h4>
<pre>
PARSE-WORD ( "<whitespaces>name<whitespace>" -- c-addr u ) CORE-EXT
</pre>
Skip leading white space and parse name delimited by a white space
character.
<p>c-addr is the address within the input buffer and u is the length
of the selected string. If the parse area is empty or contains only
white space, the resulting string has length zero.
<h4>Typical Use</h4>
<pre>
PARSE-WORD some-name TYPE
</pre>
<h4>Remarks</h4>
<h5>Lifetime</h5>
The lifetime of the resulting string is specified implicitly through
"within the input buffer", as is done in PARSE; i.e., the string will
be usable until the next input buffer is read, for whatever reason
(REFILL, INCLUDED, etc.). Should the lifetime be made more explicit?
<h5>Existing practice</h5>
ANS Forth mentions a PARSE-WORD with essentially the same definition
in <a
href="http://www.complang.tuwien.ac.at/forth/dpans-html/dpansa6.htm#A.6.2.2008">A.6.2.2008</a>.
Open Firmware also defines PARSE-WORD with the same definition. The
only difference between these definitions and the current definition
is that the current definition makes it explicit what happens when
there is only white space in the input buffer.
<p>Several systems have implemented a PARSE-WORD compatible with this
specification, e.g., Gforth and Quartus.
<p><a name="name-conflict">A number of systems have been named that
define a PARSE-WORD incompatible with this specification</a> (e.g.,
they often pass a delimiter on the stack). The systems mentioned are
MPE's VFX Forth and all MPE v6+ embedded targets, MinForth, CHForth,
F83, Jforth, 4th. Of these systems VFX, MinForth and CHForth are ANS
Forth implementations, F83, 4th and JForth are not (although 4th
partially stays close to ANS Forth). Coos Haak (CHForth) <a
href="http://groups.google.at/groups?selm=4da9kpvgmu3s%24.gpk0xcgp5agg.dlg%4040tude.net&output=gplain">indicated</a>
that the next version of CHForth will have a PARSE-WORD compatible to
this specification.
<p>PARSE-WORD in Mops works like the one proposed here, but it refills
the input buffer if the parse area is empty or contains only white
space.
<h5>Names</h5>
Given the differences in behaviour of existing words with that name,
we need a different name for this word.
People have proposed the following names:
<pre>
NextWord (conflict: exists with different meaning in Win32Forth)
(existing practice: SP-Forth/4)
TOKEN
Supported by: Alex McDonald
Could live with it: Bernd Paysan
Strongly opposed: Michael L Gassanenko
NAME (existing practice: exists with this meaning in Gforth)
Supported by: Coos Haak
Supported by: Albert van der Horst
Could live with it: Bernd Paysan
(Michael Gassenenko points out that it does not fit with >NAME NAME>)
EXTRACTWORD
EXTRACT-WORD
Supported by: Ward McFarland
GET-WORD
GET-NAME
GET-TOKEN
PARSE-NAME
Supported by: Stephen Pelc
Second choice: Albert van der Horst
Could live with it: Ward McFarland
Could also support: Alex McDonald
Could live with it: Marcel Hendrix
Can live with it: David N. Williams
Supported by: Anton Ertl
Is happy with it: Mike Hore
PARSE-STREAM
</pre>
Some people support PARSE-WORD despite the conflict: Charles Melice,
Ward McFarland, Bernd Paysan.
<h5>What is white space?</h5>
I believe that the only white space allowed in ANS Forth programs
consists of BL (section 3.4.1.1, treatment of control characters is
implementation-defined), but this proposal was written such that it
would not need rewriting if the definition of white space was extended
in the future.
<p>One other probably widely used and accepted white space character
is TAB, but extending the definition of white space may be better left
to another RfD.
<p>Systems that support TAB: Gforth, VFX Forth and MPE v6+ embedded
systems.
<h5>Implementation and Tests</h5>
<ul>
<li><a href="reference-implementations/parse-word.fs">Reference implementation</a>
<li><a href="tests/parse-word.fs">Tests</a>
</ul>
<h4>Comments</h4>
Andrey Cherezov writes: "In the SP-Forth/4 (spf.sourceforge.net) there
is 'NextWord' word compatible with this specification."
<!--
<h4><a name="voting">Voting instructions</a></h4>
Fill out the appropriate ballot(s) below and mail it/them to me
<anton@mips.complang.tuwien.ac.at>. Your vote will be published
(including your name (without email address) and/or the name of your
system) here. You can vote (or change your vote) at any time by
mailing to me, and the results will be published here.
<p>Note that you can be both a system implementor and a programmer, so
you can submit both kinds of ballots.
<h4>Ballot for systems</h4>
If you maintain several systems, please mention the systems separately
in the ballot. Insert the system name or version between the
brackets. Multiple hits for the same system are possible (if they do
not conflict).
<pre>
[ ] conforms to ANS Forth.
[ ] already implements the proposal in full since release [ ].
[ ] implements the proposal in full in a development version.
[ ] will implement the proposal in full in release [ ].
[ ] will implement the proposal in full in some future release.
There are no plans to implement the proposal in full in [ ].
[ ] will never implement the proposal in full.
</pre>
If you want to provide information on partial implementation, please
do so informally, and I will aggregate this information in some way.
<h4>Ballot for programmers</h4>
Just mark the statements that are correct for you (e.g., by putting an
"x" between the brackets). If some statements are true for some of
your programs, but not others, please mark the statements for the
dominating class of programs you write.
<pre>
[ ] I have used (parts of) this proposal in my programs.
[ ] I would use (parts of) this proposal in my programs if the systems
I am interested in implemented it.
[ ] I would use (parts of) this proposal in my programs if this
proposal was in the Forth standard.
[ ] I would not use (parts of) this proposal in my programs.
</pre>
If you feel that there is closely related functionality missing from
the proposal (especially if you have used that in your programs), make
an informal comment, and I will collect these, too. Note that the
best time to voice such issues is the RfD stage.
-->
<hr><a href="http://www.complang.tuwien.ac.at/anton/">Anton Ertl</a>