GNU bug report logs - #66988
30.0.50; treesit-forward-sexp not working properly in js-ts-mode and tsx-ts-mode

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 66988 in the body.
You can then email your comments to 66988 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Loïc Lemaître <loic.lemaitre <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 30.0.50; treesit-forward-sexp not working properly in js-ts-mode and
 tsx-ts-mode
Date: Tue, 7 Nov 2023 15:56:09 +0100

Hi Emacs team,

Here the steps to demonstrate the bug :

1. Compile Emacs from master branch with tree-sitter support
2. Install javascript and tsx languages
3. Run Emacs
4. Create a new buffer
5. Turn major mode to either js-ts-mode or tsx-ts-mode
6. Past the following content into the buffer:
(
  <div>
  </div>
);
7. Place point before opening parenthese
8. M-x forward-sexp (which will call treesit-forward-sexp)

=> New position is right after the semi-colon instead of being before 
the semi-colon.

Note that the bug disappear if the buffer content is changed for :
const component = (
  <div>
  </div>
);

But previous content, while not being very usefull, is valid JSX, as far 
as I know.
I use this syntax for unit test purpose, since it is very short.

Thanks !

Loïc Lemaître

In GNU Emacs 30.0.50 (build 5, x86_64-pc-linux-gnu, GTK+ Version
 3.24.33, cairo version 1.16.0) of 2023-11-05 built on
 loic-Latitude-E5470
Repository revision: f0c0ff6bf23ec667ff5487fd94b7f46803ea00ac
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12201001
System Description: Ubuntu 22.04.3 LTS

Configured using:
 'configure --with-native-compilation=aot --with-tree-sitter
 --with-json'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBSELINUX LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG
RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER X11 XDBE
XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
  value of $LANG: fr_FR.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: TypeScript[TSX]

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(pp shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils typescript-ts-mode js
c-ts-common treesit json map byte-opt imenu cc-mode cc-fonts cc-guess
cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs comp
cl-seq comp-cstr cl-extra help-mode warnings icons rx gv bytecomp
byte-compile time-date subr-x cl-loaddefs cl-lib rmc iso-transl tooltip
cconv eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel term/x-win x-win term/common-win x-dnd touch-screen
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq
simple cl-generic indonesian philippine cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button
loaddefs theme-loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget keymap hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo gtk x-toolkit xinput2 x multi-tty move-toolbar
make-network-process native-compile emacs)

Memory information:
((conses 16 125398 22424) (symbols 48 9683 4) (strings 32 28484 2423)
 (string-bytes 1 1002643) (vectors 16 20759)
 (vector-slots 8 406881 18596) (floats 8 38 27) (intervals 56 891 0)
 (buffers 992 15))

Message #8 received at 66988 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Loïc Lemaître <loic.lemaitre <at> gmail.com>,
 66988 <at> debbugs.gnu.org, Yuan Fu <casouri <at> gmail.com>,
 Theodor Thornhill <theo <at> thornhill.no>
Subject: Re: bug#66988: 30.0.50; treesit-forward-sexp not working properly in
 js-ts-mode and tsx-ts-mode
Date: Fri, 10 Nov 2023 03:41:00 +0200

Hi!

On 07/11/2023 16:56, Loïc Lemaître wrote:
> Hi Emacs team,
> 
> Here the steps to demonstrate the bug :
> 
> 1. Compile Emacs from master branch with tree-sitter support
> 2. Install javascript and tsx languages
> 3. Run Emacs
> 4. Create a new buffer
> 5. Turn major mode to either js-ts-mode or tsx-ts-mode
> 6. Past the following content into the buffer:
> (
>    <div>
>    </div>
> );
> 7. Place point before opening parenthese
> 8. M-x forward-sexp (which will call treesit-forward-sexp)
> 
> => New position is right after the semi-colon instead of being before 
> the semi-colon.
> 
> Note that the bug disappear if the buffer content is changed for :
> const component = (
>    <div>
>    </div>
> );
> 
> But previous content, while not being very usefull, is valid JSX, as far 
> as I know.
> I use this syntax for unit test purpose, since it is very short.

Thanks for the report.

The patch below should fix it.

Yuan, what do you think? A similar change (bos and eos anchors) might be 
useful for other things and other modes.

Alternatively, treesit-thing-settings could be interpreted to imply full 
matches, then the code using it should not only match against the 
regexps but also check that the entire string (type name) is matched.

Also Cc'ing Theodor.

diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
index 5a669fdbd42..d81fa9ed322 100644
--- a/lisp/progmodes/js.el
+++ b/lisp/progmodes/js.el
@@ -3843,6 +3843,7 @@ js--treesit-sexp-nodes
     "undefined"
     "arguments"
     "pair"
+    "parenthesized_expression"
     "jsx")
   "Nodes that designate sexps in JavaScript.
 See `treesit-thing-settings' for more information.")
@@ -3886,7 +3887,7 @@ js-ts-mode

     (setq-local treesit-thing-settings
                 `((javascript
-                   (sexp ,(regexp-opt js--treesit-sexp-nodes))
+                   (sexp ,(format "\\`%s\\'" (regexp-opt 
js--treesit-sexp-nodes)))
                    (sentence ,(regexp-opt js--treesit-sentence-nodes))
                    (text ,(regexp-opt '("comment"
                                         "template_string"))))))

Message #11 received at 66988 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Theodor Thornhill <theo <at> thornhill.no>, 66988 <at> debbugs.gnu.org,
 Loïc Lemaître <loic.lemaitre <at> gmail.com>,
 Yuan Fu <casouri <at> gmail.com>
Subject: Re: bug#66988: 30.0.50; treesit-forward-sexp not working properly
 in js-ts-mode and tsx-ts-mode
Date: Fri, 10 Nov 2023 09:37:38 +0200

> @@ -3843,6 +3843,7 @@ js--treesit-sexp-nodes
>      "undefined"
>      "arguments"
>      "pair"
> +    "parenthesized_expression"
>      "jsx")
>    "Nodes that designate sexps in JavaScript.
>  See `treesit-thing-settings' for more information.")

I had the same problem and to make this mode more usable had to use such patch:

diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
index 5a669fdbd42..4c07fbd94b7 100644
--- a/lisp/progmodes/js.el
+++ b/lisp/progmodes/js.el
@@ -3825,7 +3825,9 @@ js--treesit-sentence-nodes
 See `treesit-thing-settings' for more information.")
 
 (defvar js--treesit-sexp-nodes
-  '("expression"
+  '("expression" ;; SHOULD NOT MATCH "expression_statement", BUT SHOULD MATCH "parenthesized_expression"
+    "parenthesized_expression"
+    "formal_parameters"
     "pattern"
     "array"
     "function"
@@ -3843,7 +3845,13 @@ js--treesit-sexp-nodes
     "undefined"
     "arguments"
     "pair"
-    "jsx")
+    "jsx"
+    "statement_block"
+    "object"
+    "object_pattern"
+    "named_imports"
+    "class_body"
+    )
   "Nodes that designate sexps in JavaScript.
 See `treesit-thing-settings' for more information.")
 
PS:
Also tried to replace
  (setq-local treesit-sexp-type-regexp (regexp-opt js--treesit-sexp-nodes))
with
  (setq-local treesit-sexp-type-regexp (rx-to-string `(seq bol (or ,@js--treesit-sexp-nodes) eol)))

Message #14 received at 66988 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Theodor Thornhill <theo <at> thornhill.no>, 66988 <at> debbugs.gnu.org,
 Loïc Lemaître <loic.lemaitre <at> gmail.com>
Subject: Re: bug#66988: 30.0.50; treesit-forward-sexp not working properly in
 js-ts-mode and tsx-ts-mode
Date: Fri, 10 Nov 2023 18:41:20 -0800

> On Nov 9, 2023, at 5:41 PM, Dmitry Gutov <dmitry <at> gutov.dev> wrote:
> 
> Hi!
> 
> On 07/11/2023 16:56, Loïc Lemaître wrote:
>> Hi Emacs team,
>> Here the steps to demonstrate the bug :
>> 1. Compile Emacs from master branch with tree-sitter support
>> 2. Install javascript and tsx languages
>> 3. Run Emacs
>> 4. Create a new buffer
>> 5. Turn major mode to either js-ts-mode or tsx-ts-mode
>> 6. Past the following content into the buffer:
>> (
>>   <div>
>>   </div>
>> );
>> 7. Place point before opening parenthese
>> 8. M-x forward-sexp (which will call treesit-forward-sexp)
>> => New position is right after the semi-colon instead of being before the semi-colon.
>> Note that the bug disappear if the buffer content is changed for :
>> const component = (
>>   <div>
>>   </div>
>> );
>> But previous content, while not being very usefull, is valid JSX, as far as I know.
>> I use this syntax for unit test purpose, since it is very short.
> 
> Thanks for the report.
> 
> The patch below should fix it.
> 
> Yuan, what do you think? A similar change (bos and eos anchors) might be useful for other things and other modes.
> 
> Alternatively, treesit-thing-settings could be interpreted to imply full matches, then the code using it should not only match against the regexps but also check that the entire string (type name) is matched.

I regret not doing this by default for treesit-indent-rules and traverse functions. Now it’s hard to change without creating confusion and breaking backward compatibility. I wonder if there are good way to smoothly transition to match full names by default…

Treesit-thing-settings can be changed to match full names, but only if we can change treesit-indent-rules and friends too. Otherwise it would be too confusing.

Yuan

Message #17 received at 66988 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: dmitry <at> gutov.dev, 66988 <at> debbugs.gnu.org, theo <at> thornhill.no,
 loic.lemaitre <at> gmail.com
Subject: Re: bug#66988: 30.0.50;
 treesit-forward-sexp not working properly in js-ts-mode and
 tsx-ts-mode
Date: Sat, 11 Nov 2023 09:35:56 +0200

> Cc: Theodor Thornhill <theo <at> thornhill.no>, 66988 <at> debbugs.gnu.org,
>  Loïc Lemaître <loic.lemaitre <at> gmail.com>
> From: Yuan Fu <casouri <at> gmail.com>
> Date: Fri, 10 Nov 2023 18:41:20 -0800
> 
> > Alternatively, treesit-thing-settings could be interpreted to imply full matches, then the code using it should not only match against the regexps but also check that the entire string (type name) is matched.
> 
> I regret not doing this by default for treesit-indent-rules and traverse functions. Now it’s hard to change without creating confusion and breaking backward compatibility. I wonder if there are good way to smoothly transition to match full names by default…
> 
> Treesit-thing-settings can be changed to match full names, but only if we can change treesit-indent-rules and friends too. Otherwise it would be too confusing.

Is it feasible to have a variable that controls whether the full
matches are implied in these APIs?  Then we could start by making it
optional, and at some later time make it the default.

Message #20 received at 66988 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Yuan Fu <casouri <at> gmail.com>
Cc: Theodor Thornhill <theo <at> thornhill.no>, 66988 <at> debbugs.gnu.org,
 Loïc Lemaître <loic.lemaitre <at> gmail.com>
Subject: Re: bug#66988: 30.0.50; treesit-forward-sexp not working properly in
 js-ts-mode and tsx-ts-mode
Date: Sat, 11 Nov 2023 12:49:21 +0200

On 11/11/2023 04:41, Yuan Fu wrote:
> I regret not doing this by default for treesit-indent-rules and traverse functions. Now it’s hard to change without creating confusion and breaking backward compatibility. I wonder if there are good way to smoothly transition to match full names by default…
> 
> Treesit-thing-settings can be changed to match full names, but only if we can change treesit-indent-rules and friends too. Otherwise it would be too confusing.

It's my understanding that the current implementation, when it doesn't 
use a full match, is a potential bug in every single instance.

Perhaps you have an example of when partial match is intended and 
beneficial? If so, we can just go through all other regexps and wrap 
them in \` and \'. And should.

Message #23 received at 66988 <at> debbugs.gnu.org (full text, mbox):

From: Loïc Lemaître <loic.lemaitre <at> gmail.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>, 66988 <at> debbugs.gnu.org,
 Yuan Fu <casouri <at> gmail.com>, Theodor Thornhill <theo <at> thornhill.no>
Subject: Re: bug#66988: 30.0.50; treesit-forward-sexp not working properly in
 js-ts-mode and tsx-ts-mode
Date: Sat, 11 Nov 2023 16:43:06 +0100

[Message part 1 (text/plain, inline)]

Thanks for the patch ! It fixes the bug.
But unfortunatly, there is another similar bug in 
/treesit-forward-sexp/, that you can reproduce with that example:
({(<A></A>)});

/(treesit-forward-sexp)/ does not work as expected for both the opening 
parentheses and the brace.

I have checked that it is not a regression due to the patch. That said,  
the patch changes the results (that are not what we expect in any cases).

Loïc

Le 10/11/2023 à 02:41, Dmitry Gutov a écrit :
> Hi!
>
> On 07/11/2023 16:56, Loïc Lemaître wrote:
>> Hi Emacs team,
>>
>> Here the steps to demonstrate the bug :
>>
>> 1. Compile Emacs from master branch with tree-sitter support
>> 2. Install javascript and tsx languages
>> 3. Run Emacs
>> 4. Create a new buffer
>> 5. Turn major mode to either js-ts-mode or tsx-ts-mode
>> 6. Past the following content into the buffer:
>> (
>>    <div>
>>    </div>
>> );
>> 7. Place point before opening parenthese
>> 8. M-x forward-sexp (which will call treesit-forward-sexp)
>>
>> => New position is right after the semi-colon instead of being before 
>> the semi-colon.
>>
>> Note that the bug disappear if the buffer content is changed for :
>> const component = (
>>    <div>
>>    </div>
>> );
>>
>> But previous content, while not being very usefull, is valid JSX, as 
>> far as I know.
>> I use this syntax for unit test purpose, since it is very short.
>
> Thanks for the report.
>
> The patch below should fix it.
>
> Yuan, what do you think? A similar change (bos and eos anchors) might 
> be useful for other things and other modes.
>
> Alternatively, treesit-thing-settings could be interpreted to imply 
> full matches, then the code using it should not only match against the 
> regexps but also check that the entire string (type name) is matched.
>
> Also Cc'ing Theodor.
>
> diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
> index 5a669fdbd42..d81fa9ed322 100644
> --- a/lisp/progmodes/js.el
> +++ b/lisp/progmodes/js.el
> @@ -3843,6 +3843,7 @@ js--treesit-sexp-nodes
>      "undefined"
>      "arguments"
>      "pair"
> +    "parenthesized_expression"
>      "jsx")
>    "Nodes that designate sexps in JavaScript.
>  See `treesit-thing-settings' for more information.")
> @@ -3886,7 +3887,7 @@ js-ts-mode
>
>      (setq-local treesit-thing-settings
>                  `((javascript
> -                   (sexp ,(regexp-opt js--treesit-sexp-nodes))
> +                   (sexp ,(format "\\`%s\\'" (regexp-opt 
> js--treesit-sexp-nodes)))
>                     (sentence ,(regexp-opt js--treesit-sentence-nodes))
>                     (text ,(regexp-opt '("comment"
>                                          "template_string"))))))
>

[Message part 2 (text/html, inline)]

Message #26 received at 66988 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Loïc Lemaître <loic.lemaitre <at> gmail.com>,
 66988 <at> debbugs.gnu.org, Yuan Fu <casouri <at> gmail.com>,
 Theodor Thornhill <theo <at> thornhill.no>
Subject: Re: bug#66988: 30.0.50; treesit-forward-sexp not working properly in
 js-ts-mode and tsx-ts-mode
Date: Sun, 12 Nov 2023 01:40:22 +0200

On 11/11/2023 17:43, Loïc Lemaître wrote:
> Thanks for the patch ! It fixes the bug.
> But unfortunatly, there is another similar bug in 
> /treesit-forward-sexp/, that you can reproduce with that example:
> ({(<A></A>)});

The problem in this case is that the code doesn't parse (one of the 
nodes in the parse tree is ERROR). Removing either the curlies, or the 
outer parens pair makes the code valid and the behavior correspondingly 
better.

Although for treesit-forward-sexp to jump between curlies in

  {(<A></A>)};

we'll also need to add "statement_block" to js--treesit-sexp-nodes.

Message #29 received at 66988 <at> debbugs.gnu.org (full text, mbox):

From: Loïc Lemaître <loic.lemaitre <at> gmail.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>, 66988 <at> debbugs.gnu.org,
 Yuan Fu <casouri <at> gmail.com>, Theodor Thornhill <theo <at> thornhill.no>
Subject: Re: bug#66988: 30.0.50; treesit-forward-sexp not working properly in
 js-ts-mode and tsx-ts-mode
Date: Sun, 12 Nov 2023 13:10:14 +0100

Sorry for my example that is actually not valid JSX...
Yours demontrates better the remaining issue.

Thanks

Loïc

Le 12/11/2023 à 00:40, Dmitry Gutov a écrit :
> On 11/11/2023 17:43, Loïc Lemaître wrote:
>> Thanks for the patch ! It fixes the bug.
>> But unfortunatly, there is another similar bug in 
>> /treesit-forward-sexp/, that you can reproduce with that example:
>> ({(<A></A>)});
>
> The problem in this case is that the code doesn't parse (one of the 
> nodes in the parse tree is ERROR). Removing either the curlies, or the 
> outer parens pair makes the code valid and the behavior 
> correspondingly better.
>
> Although for treesit-forward-sexp to jump between curlies in
>
>   {(<A></A>)};
>
> we'll also need to add "statement_block" to js--treesit-sexp-nodes.

Message #32 received at 66988 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Theodor Thornhill <theo <at> thornhill.no>, 66988 <at> debbugs.gnu.org,
 Loïc Lemaître <loic.lemaitre <at> gmail.com>
Subject: Re: bug#66988: 30.0.50; treesit-forward-sexp not working properly in
 js-ts-mode and tsx-ts-mode
Date: Tue, 14 Nov 2023 22:32:24 -0800

> On Nov 11, 2023, at 2:49 AM, Dmitry Gutov <dmitry <at> gutov.dev> wrote:
> 
> On 11/11/2023 04:41, Yuan Fu wrote:
>> I regret not doing this by default for treesit-indent-rules and traverse functions. Now it’s hard to change without creating confusion and breaking backward compatibility. I wonder if there are good way to smoothly transition to match full names by default…
>> Treesit-thing-settings can be changed to match full names, but only if we can change treesit-indent-rules and friends too. Otherwise it would be too confusing.
> 
> It's my understanding that the current implementation, when it doesn't use a full match, is a potential bug in every single instance.

That’s a good point.

> Perhaps you have an example of when partial match is intended and beneficial? If so, we can just go through all other regexps and wrap them in \` and \'. And should.

I don’t, except for a very few cases where I saved typing a few characters.

You have a good point. I think most people instinctively write the full match in their code anyway, so changing to the full match should be fine. We can start from master and see if the world ends or not. 

Yuan

Message #35 received at 66988 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: dmitry <at> gutov.dev, 66988 <at> debbugs.gnu.org, theo <at> thornhill.no,
 loic.lemaitre <at> gmail.com
Subject: Re: bug#66988: 30.0.50; treesit-forward-sexp not working properly in
 js-ts-mode and tsx-ts-mode
Date: Wed, 15 Nov 2023 14:19:53 +0200

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Tue, 14 Nov 2023 22:28:34 -0800
> Cc: Dmitry Gutov <dmitry <at> gutov.dev>,
>  Theodor Thornhill <theo <at> thornhill.no>,
>  66988 <at> debbugs.gnu.org,
>  loic.lemaitre <at> gmail.com
> 
> > Is it feasible to have a variable that controls whether the full
> > matches are implied in these APIs?  Then we could start by making it
> > optional, and at some later time make it the default.
> 
> It’s feasible, but I don’t think it’s TRT. The major mode author should have the control over whether the full match is implied, not the user. It doesn’t make sense to toggle the variable either. If you change that variable, major mode code must also change to be correct.

So what do you suggest that we do about this issue?

Message #38 received at 66988 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Dmitry Gutov <dmitry <at> gutov.dev>, 66988 <at> debbugs.gnu.org,
 Theodor Thornhill <theo <at> thornhill.no>, loic.lemaitre <at> gmail.com
Subject: Re: bug#66988: 30.0.50; treesit-forward-sexp not working properly in
 js-ts-mode and tsx-ts-mode
Date: Tue, 14 Nov 2023 22:28:34 -0800

> On Nov 10, 2023, at 11:35 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> Cc: Theodor Thornhill <theo <at> thornhill.no>, 66988 <at> debbugs.gnu.org,
>> Loïc Lemaître <loic.lemaitre <at> gmail.com>
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Fri, 10 Nov 2023 18:41:20 -0800
>> 
>>> Alternatively, treesit-thing-settings could be interpreted to imply full matches, then the code using it should not only match against the regexps but also check that the entire string (type name) is matched.
>> 
>> I regret not doing this by default for treesit-indent-rules and traverse functions. Now it’s hard to change without creating confusion and breaking backward compatibility. I wonder if there are good way to smoothly transition to match full names by default…
>> 
>> Treesit-thing-settings can be changed to match full names, but only if we can change treesit-indent-rules and friends too. Otherwise it would be too confusing.
> 
> Is it feasible to have a variable that controls whether the full
> matches are implied in these APIs?  Then we could start by making it
> optional, and at some later time make it the default.

It’s feasible, but I don’t think it’s TRT. The major mode author should have the control over whether the full match is implied, not the user. It doesn’t make sense to toggle the variable either. If you change that variable, major mode code must also change to be correct.

Yuan

Message #41 received at 66988 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Dmitry Gutov <dmitry <at> gutov.dev>, 66988 <at> debbugs.gnu.org,
 Mattias Engdegård <mattias.engdegard <at> gmail.com>,
 Theodor Thornhill <theo <at> thornhill.no>,
 Loïc Lemaître <loic.lemaitre <at> gmail.com>
Subject: Re: bug#66988: 30.0.50; treesit-forward-sexp not working properly in
 js-ts-mode and tsx-ts-mode
Date: Sat, 18 Nov 2023 10:57:56 -0800

> On Nov 15, 2023, at 4:19 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Tue, 14 Nov 2023 22:28:34 -0800
>> Cc: Dmitry Gutov <dmitry <at> gutov.dev>,
>> Theodor Thornhill <theo <at> thornhill.no>,
>> 66988 <at> debbugs.gnu.org,
>> loic.lemaitre <at> gmail.com
>> 
>>> Is it feasible to have a variable that controls whether the full
>>> matches are implied in these APIs?  Then we could start by making it
>>> optional, and at some later time make it the default.
>> 
>> It’s feasible, but I don’t think it’s TRT. The major mode author should have the control over whether the full match is implied, not the user. It doesn’t make sense to toggle the variable either. If you change that variable, major mode code must also change to be correct.
> 
> So what do you suggest that we do about this issue?

We change every treesit function that takes a regexp for matching node type names to imply full match. I think most people wrote code as if these functions use full match, so the breakage should be small. And I can forecast this change in as many channels as I can.

Yuan

Message #44 received at 66988 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: dmitry <at> gutov.dev, 66988 <at> debbugs.gnu.org, mattias.engdegard <at> gmail.com,
 theo <at> thornhill.no, loic.lemaitre <at> gmail.com
Subject: Re: bug#66988: 30.0.50; treesit-forward-sexp not working properly in
 js-ts-mode and tsx-ts-mode
Date: Sun, 19 Nov 2023 07:48:02 +0200

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sat, 18 Nov 2023 10:57:56 -0800
> Cc: Dmitry Gutov <dmitry <at> gutov.dev>,
>  Theodor Thornhill <theo <at> thornhill.no>,
>  66988 <at> debbugs.gnu.org,
>  Loïc Lemaître <loic.lemaitre <at> gmail.com>,
>  Mattias Engdegård <mattias.engdegard <at> gmail.com>
> 
> 
> 
> > On Nov 15, 2023, at 4:19 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > 
> >> From: Yuan Fu <casouri <at> gmail.com>
> >> Date: Tue, 14 Nov 2023 22:28:34 -0800
> >> Cc: Dmitry Gutov <dmitry <at> gutov.dev>,
> >> Theodor Thornhill <theo <at> thornhill.no>,
> >> 66988 <at> debbugs.gnu.org,
> >> loic.lemaitre <at> gmail.com
> >> 
> >>> Is it feasible to have a variable that controls whether the full
> >>> matches are implied in these APIs?  Then we could start by making it
> >>> optional, and at some later time make it the default.
> >> 
> >> It’s feasible, but I don’t think it’s TRT. The major mode author should have the control over whether the full match is implied, not the user. It doesn’t make sense to toggle the variable either. If you change that variable, major mode code must also change to be correct.
> > 
> > So what do you suggest that we do about this issue?
> 
> We change every treesit function that takes a regexp for matching node type names to imply full match. I think most people wrote code as if these functions use full match, so the breakage should be small. And I can forecast this change in as many channels as I can.

Fine by me, so let's do it.

Message #47 received at 66988 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Dmitry Gutov <dmitry <at> gutov.dev>, 66988 <at> debbugs.gnu.org,
 Mattias Engdegård <mattias.engdegard <at> gmail.com>,
 Theodor Thornhill <theo <at> thornhill.no>,
 Loïc Lemaître <loic.lemaitre <at> gmail.com>
Subject: Re: bug#66988: 30.0.50; treesit-forward-sexp not working properly in
 js-ts-mode and tsx-ts-mode
Date: Sat, 18 Nov 2023 22:19:15 -0800

> On Nov 18, 2023, at 9:48 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Sat, 18 Nov 2023 10:57:56 -0800
>> Cc: Dmitry Gutov <dmitry <at> gutov.dev>,
>> Theodor Thornhill <theo <at> thornhill.no>,
>> 66988 <at> debbugs.gnu.org,
>> Loïc Lemaître <loic.lemaitre <at> gmail.com>,
>> Mattias Engdegård <mattias.engdegard <at> gmail.com>
>> 
>> 
>> 
>>> On Nov 15, 2023, at 4:19 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>> 
>>>> From: Yuan Fu <casouri <at> gmail.com>
>>>> Date: Tue, 14 Nov 2023 22:28:34 -0800
>>>> Cc: Dmitry Gutov <dmitry <at> gutov.dev>,
>>>> Theodor Thornhill <theo <at> thornhill.no>,
>>>> 66988 <at> debbugs.gnu.org,
>>>> loic.lemaitre <at> gmail.com
>>>> 
>>>>> Is it feasible to have a variable that controls whether the full
>>>>> matches are implied in these APIs?  Then we could start by making it
>>>>> optional, and at some later time make it the default.
>>>> 
>>>> It’s feasible, but I don’t think it’s TRT. The major mode author should have the control over whether the full match is implied, not the user. It doesn’t make sense to toggle the variable either. If you change that variable, major mode code must also change to be correct.
>>> 
>>> So what do you suggest that we do about this issue?
>> 
>> We change every treesit function that takes a regexp for matching node type names to imply full match. I think most people wrote code as if these functions use full match, so the breakage should be small. And I can forecast this change in as many channels as I can.
> 
> Fine by me, so let's do it.

Ok, I’ll start preparing the patch and news. Meanwhile, folks, let me know if anyone has objections.

Yuan

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.