The maximum unicode code point is `10FFFF` but in the escapes table it states code points can only go up to `10FFF`. I think this is an error, since most emojis are well above that number, and other languages allow escapes up to `10FFFF`
this does a few things with identifiers and strings:
- removes `"` from the set of identifier characters, to prevent ambiguity with string
- adds `bare-identifier` and `escaped-string` non-terminals (i think less mixing of terminal/non-terminal is easier to read?)
- moves string ahead of bare-identifier in identifier, for easier implementation with ordered-choice parsers
- moves raw-string into string, so you can have raw-string identifiers
Previously slashdash was allowed anywhere `ws` was, and could be
followed by any legal slashdashable token regardless of position. This
caused some ambiguous parsing and enabled constructions we really don't
want.
The new version simply allows a slashdash to preceed each commentable
token, thus restricting its use to locations where the token is
otherwise valid.
The Unicode standard listed VT in its table in §5.8 but that was just a
table of acronyms, and VT was only listed because Microsoft Word used it
as a line separator. VT is not part of its recommendation for characters
to treat as line terminators.