GNU bug report logs - #67977
30.0.50; tree-sitter: Emacs crashes when accessing treesit-nodes in a narrowed buffer

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 67977 in the body.
You can then email your comments to 67977 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Denis Zubarev <dvzubarev <at> yandex.ru>
To: bug-gnu-emacs <at> gnu.org
Subject: 30.0.50; tree-sitter: Emacs crashes when accessing treesit-nodes in
 a narrowed buffer
Date: Sat, 23 Dec 2023 02:18:20 +0300

Steps to reproduce:
1. emacs -Q
2. M-x find-file /tmp/t.py
3. paste there
start=1
def _init(self, param1, param2, param3=False):
    self._param1 = param2
    self._param2 = param2
    self._param3 = param3
4. python-ts-mode
5. select two last lines and M-x narrow-to-region
6. answer all prompts
7. Put cursor on the last self
8. M-x eval-expression
(progn
(setq temp-node (treesit-node-at (point)))
(sit-for 2)
(garbage-collect)
(message "node %s" temp-node))
9. Emacs crashes or prints node that contains garbage
It should print "node #<treesit-node identifier in 112-116>"
It works ok in the widen buffer or if the whole function is narrowed.

Lisp Backtrace:
"message" (0xffffc0c0)
"progn" (0xffffc1e0)
"eval" (0xffffc338)
"eval-expression" (0xffffc440)
"funcall-interactively" (0xffffc438)
"command-execute" (0xffffc7b8)
"execute-extended-command" (0xffffc8f0)
"funcall-interactively" (0xffffc8e8)
"command-execute" (0xffffcc18)

bt:
#0  0x00007ffff338c4f4 in ts_language_symbol_count () at /usr/local/lib/libtree-sitter.so.0
#1  0x00007ffff338c81e in ts_language_symbol_name () at /usr/local/lib/libtree-sitter.so.0
#2  0x0000555555806920 in Ftreesit_node_type (node=node <at> entry=XIL(0x555556de81d5)) at treesit.c:1864
#3  0x00005555557a3b91 in print_vectorlike_unreadable (buf=0x7fffffff6590 "", escapeflag=false, printcharfun=XIL(0), obj=<optimized out>) at print.c:2049
#4  print_object (obj=<optimized out>, printcharfun=XIL(0), escapeflag=false) at print.c:2633
#5  0x00005555557a4742 in Fprin1_to_string (object=XIL(0x555556de81d5), noescape=XIL(0x30), overrides=overrides <at> entry=XIL(0)) at print.c:813
#6  0x000055555576f3e8 in styled_format (nargs=<optimized out>, args=<optimized out>, message=<optimized out>) at editfns.c:3662
#7  0x00005555557702e9 in Fformat_message (args=0x7fffffffc0c0, nargs=<optimized out>) at editfns.c:3415
#8  Fmessage (args=0x7fffffffc0c0, nargs=<optimized out>) at editfns.c:3212
#9  Fmessage (nargs=<optimized out>, args=0x7fffffffc0c0) at editfns.c:3181

print treesit_node from frame #2:

$1 = {
  context = {4294928528, 32767, 4294928440, 32767},
  id = 0x7fff20ff67e0,
  tree = 0x5555557a0efc <strout+572>
}




In GNU Emacs 30.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version
 3.24.33, cairo version 1.16.0) of 2023-12-23 built on NUC-here
Repository revision: 9c86dd52475e0ad65359bc964fbe0d62b9d3e464
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101004
System Description: Ubuntu 22.04.3 LTS

Configured using:
 'configure --with-modules --with-native-compilation=aot
 --with-imagemagick --with-json --with-tree-sitter --with-xft'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ
IMAGEMAGICK JPEG JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2
M17N_FLT MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP
SOUND THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER X11 XDBE XIM XINPUT2
XPM GTK3 ZLIB

Important settings:
  value of $LC_MONETARY: ru_RU.UTF-8
  value of $LC_NUMERIC: ru_RU.UTF-8
  value of $LC_TIME: ru_RU.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Debbugs

Minor modes in effect:
  winum-mode: t
  global-anzu-mode: t
  anzu-mode: t
  dap-tooltip-mode: t
  dap-ui-many-windows-mode: t
  dap-ui-mode: t
  treemacs-filewatch-mode: t
  treemacs-git-mode: t
  treemacs-fringe-indicator-mode: t
  dap-auto-configure-mode: t
  dap-mode: t
  save-place-mode: t
  global-so-long-mode: t
  global-git-commit-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  recentf-mode: t
  projectile-mode: t
  header-line-indent-mode: t
  which-key-mode: t
  savehist-mode: t
  better-jumper-mode: t
  better-jumper-local-mode: t
  global-company-mode: t
  company-mode: t
  vertico-mode: t
  marginalia-mode: t
  evil-goggles-mode: t
  evil-escape-mode: t
  evil-snipe-override-mode: t
  evil-snipe-mode: t
  pcre-mode: t
  server-mode: t
  gcmh-mode: t
  global-hl-line-mode: t
  hl-line-mode: t
  winner-mode: t
  smartparens-global-mode: t
  ws-butler-global-mode: t
  undo-fu-session-global-mode: t
  undo-fu-mode: t
  global-flycheck-mode: t
  persp-mode: t
  doom-modeline-mode: t
  solaire-global-mode: t
  solaire-mode: t
  hexl-follow-ascii: t
  global-evil-surround-mode: t
  evil-surround-mode: t
  dirvish-override-dired-mode: t
  direnv-mode: t
  +lsp-optimization-mode: t
  evil-mode: t
  evil-local-mode: t
  +popup-mode: t
  override-global-mode: t
  general-override-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  window-divider-mode: t
  minibuffer-regexp-mode: t
  buffer-read-only: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/home/denin/.emacs.d/.local/straight/build-30.0.50/ivy/elpa hides /home/denin/.emacs.d/.local/straight/build-30.0.50/lispy/elpa
/home/denin/.emacs.d/.local/straight/build-30.0.50/transient/transient hides /usr/local/share/emacs/30.0.50/lisp/transient
/home/denin/.emacs.d/.local/straight/build-30.0.50/bind-key/bind-key hides /usr/local/share/emacs/30.0.50/lisp/bind-key
/home/denin/.emacs.d/.local/straight/build-30.0.50/use-package/use-package-delight hides /usr/local/share/emacs/30.0.50/lisp/use-package/use-package-delight
/home/denin/.emacs.d/.local/straight/build-30.0.50/use-package/use-package-lint hides /usr/local/share/emacs/30.0.50/lisp/use-package/use-package-lint
/home/denin/.emacs.d/.local/straight/build-30.0.50/use-package/use-package-jump hides /usr/local/share/emacs/30.0.50/lisp/use-package/use-package-jump
/home/denin/.emacs.d/.local/straight/build-30.0.50/use-package/use-package-diminish hides /usr/local/share/emacs/30.0.50/lisp/use-package/use-package-diminish
/home/denin/.emacs.d/.local/straight/build-30.0.50/use-package/use-package-ensure hides /usr/local/share/emacs/30.0.50/lisp/use-package/use-package-ensure
/home/denin/.emacs.d/.local/straight/build-30.0.50/use-package/use-package hides /usr/local/share/emacs/30.0.50/lisp/use-package/use-package
/home/denin/.emacs.d/.local/straight/build-30.0.50/use-package/use-package-bind-key hides /usr/local/share/emacs/30.0.50/lisp/use-package/use-package-bind-key
/home/denin/.emacs.d/.local/straight/build-30.0.50/use-package/use-package-core hides /usr/local/share/emacs/30.0.50/lisp/use-package/use-package-core
/home/denin/.emacs.d/.local/straight/build-30.0.50/project/project hides /usr/local/share/emacs/30.0.50/lisp/progmodes/project
/home/denin/.emacs.d/.local/straight/build-30.0.50/gdb-mi/gdb-mi hides /usr/local/share/emacs/30.0.50/lisp/progmodes/gdb-mi
/home/denin/.emacs.d/.local/straight/build-30.0.50/xref/xref hides /usr/local/share/emacs/30.0.50/lisp/progmodes/xref
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-ref hides /usr/local/share/emacs/30.0.50/lisp/org/ob-ref
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ol-irc hides /usr/local/share/emacs/30.0.50/lisp/org/ol-irc
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-crypt hides /usr/local/share/emacs/30.0.50/lisp/org/org-crypt
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-attach hides /usr/local/share/emacs/30.0.50/lisp/org/org-attach
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-mouse hides /usr/local/share/emacs/30.0.50/lisp/org/org-mouse
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-latex hides /usr/local/share/emacs/30.0.50/lisp/org/ob-latex
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ol-eww hides /usr/local/share/emacs/30.0.50/lisp/org/ol-eww
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-ditaa hides /usr/local/share/emacs/30.0.50/lisp/org/ob-ditaa
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-groovy hides /usr/local/share/emacs/30.0.50/lisp/org/ob-groovy
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-compat hides /usr/local/share/emacs/30.0.50/lisp/org/org-compat
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ol-info hides /usr/local/share/emacs/30.0.50/lisp/org/ol-info
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-protocol hides /usr/local/share/emacs/30.0.50/lisp/org/org-protocol
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-footnote hides /usr/local/share/emacs/30.0.50/lisp/org/org-footnote
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-id hides /usr/local/share/emacs/30.0.50/lisp/org/org-id
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/oc-biblatex hides /usr/local/share/emacs/30.0.50/lisp/org/oc-biblatex
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-C hides /usr/local/share/emacs/30.0.50/lisp/org/ob-C
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-colview hides /usr/local/share/emacs/30.0.50/lisp/org/org-colview
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-calc hides /usr/local/share/emacs/30.0.50/lisp/org/ob-calc
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-octave hides /usr/local/share/emacs/30.0.50/lisp/org/ob-octave
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-java hides /usr/local/share/emacs/30.0.50/lisp/org/ob-java
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-js hides /usr/local/share/emacs/30.0.50/lisp/org/ob-js
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-R hides /usr/local/share/emacs/30.0.50/lisp/org/ob-R
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-ruby hides /usr/local/share/emacs/30.0.50/lisp/org/ob-ruby
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-css hides /usr/local/share/emacs/30.0.50/lisp/org/ob-css
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ox-odt hides /usr/local/share/emacs/30.0.50/lisp/org/ox-odt
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ol-man hides /usr/local/share/emacs/30.0.50/lisp/org/ol-man
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-perl hides /usr/local/share/emacs/30.0.50/lisp/org/ob-perl
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ox-org hides /usr/local/share/emacs/30.0.50/lisp/org/ox-org
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ox-html hides /usr/local/share/emacs/30.0.50/lisp/org/ox-html
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-gnuplot hides /usr/local/share/emacs/30.0.50/lisp/org/ob-gnuplot
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-entities hides /usr/local/share/emacs/30.0.50/lisp/org/org-entities
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ol-bibtex hides /usr/local/share/emacs/30.0.50/lisp/org/ol-bibtex
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-goto hides /usr/local/share/emacs/30.0.50/lisp/org/org-goto
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob hides /usr/local/share/emacs/30.0.50/lisp/org/ob
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-inlinetask hides /usr/local/share/emacs/30.0.50/lisp/org/org-inlinetask
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-comint hides /usr/local/share/emacs/30.0.50/lisp/org/ob-comint
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ol-gnus hides /usr/local/share/emacs/30.0.50/lisp/org/ol-gnus
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ox-latex hides /usr/local/share/emacs/30.0.50/lisp/org/ox-latex
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-version hides /usr/local/share/emacs/30.0.50/lisp/org/org-version
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-agenda hides /usr/local/share/emacs/30.0.50/lisp/org/org-agenda
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-lisp hides /usr/local/share/emacs/30.0.50/lisp/org/ob-lisp
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-matlab hides /usr/local/share/emacs/30.0.50/lisp/org/ob-matlab
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org hides /usr/local/share/emacs/30.0.50/lisp/org/org
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-maxima hides /usr/local/share/emacs/30.0.50/lisp/org/ob-maxima
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-lua hides /usr/local/share/emacs/30.0.50/lisp/org/ob-lua
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-refile hides /usr/local/share/emacs/30.0.50/lisp/org/org-refile
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-faces hides /usr/local/share/emacs/30.0.50/lisp/org/org-faces
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-python hides /usr/local/share/emacs/30.0.50/lisp/org/ob-python
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-loaddefs hides /usr/local/share/emacs/30.0.50/lisp/org/org-loaddefs
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-lint hides /usr/local/share/emacs/30.0.50/lisp/org/org-lint
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ol-docview hides /usr/local/share/emacs/30.0.50/lisp/org/ol-docview
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ox-md hides /usr/local/share/emacs/30.0.50/lisp/org/ox-md
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ox-man hides /usr/local/share/emacs/30.0.50/lisp/org/ox-man
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-indent hides /usr/local/share/emacs/30.0.50/lisp/org/org-indent
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-mobile hides /usr/local/share/emacs/30.0.50/lisp/org/org-mobile
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-scheme hides /usr/local/share/emacs/30.0.50/lisp/org/ob-scheme
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ox hides /usr/local/share/emacs/30.0.50/lisp/org/ox
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-tangle hides /usr/local/share/emacs/30.0.50/lisp/org/ob-tangle
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/oc-csl hides /usr/local/share/emacs/30.0.50/lisp/org/oc-csl
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-shell hides /usr/local/share/emacs/30.0.50/lisp/org/ob-shell
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ox-beamer hides /usr/local/share/emacs/30.0.50/lisp/org/ox-beamer
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/oc-natbib hides /usr/local/share/emacs/30.0.50/lisp/org/oc-natbib
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-habit hides /usr/local/share/emacs/30.0.50/lisp/org/org-habit
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-emacs-lisp hides /usr/local/share/emacs/30.0.50/lisp/org/ob-emacs-lisp
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-persist hides /usr/local/share/emacs/30.0.50/lisp/org/org-persist
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-core hides /usr/local/share/emacs/30.0.50/lisp/org/ob-core
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ox-icalendar hides /usr/local/share/emacs/30.0.50/lisp/org/ox-icalendar
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-haskell hides /usr/local/share/emacs/30.0.50/lisp/org/ob-haskell
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-attach-git hides /usr/local/share/emacs/30.0.50/lisp/org/org-attach-git
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/oc hides /usr/local/share/emacs/30.0.50/lisp/org/oc
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-sed hides /usr/local/share/emacs/30.0.50/lisp/org/ob-sed
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-pcomplete hides /usr/local/share/emacs/30.0.50/lisp/org/org-pcomplete
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ox-koma-letter hides /usr/local/share/emacs/30.0.50/lisp/org/ox-koma-letter
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-list hides /usr/local/share/emacs/30.0.50/lisp/org/org-list
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-ctags hides /usr/local/share/emacs/30.0.50/lisp/org/org-ctags
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-dot hides /usr/local/share/emacs/30.0.50/lisp/org/ob-dot
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ol-rmail hides /usr/local/share/emacs/30.0.50/lisp/org/ol-rmail
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-forth hides /usr/local/share/emacs/30.0.50/lisp/org/ob-forth
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-sqlite hides /usr/local/share/emacs/30.0.50/lisp/org/ob-sqlite
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-cycle hides /usr/local/share/emacs/30.0.50/lisp/org/org-cycle
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-num hides /usr/local/share/emacs/30.0.50/lisp/org/org-num
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-src hides /usr/local/share/emacs/30.0.50/lisp/org/org-src
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-sql hides /usr/local/share/emacs/30.0.50/lisp/org/ob-sql
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-plot hides /usr/local/share/emacs/30.0.50/lisp/org/org-plot
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-clock hides /usr/local/share/emacs/30.0.50/lisp/org/org-clock
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-element hides /usr/local/share/emacs/30.0.50/lisp/org/org-element
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ox-ascii hides /usr/local/share/emacs/30.0.50/lisp/org/ox-ascii
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ol-mhe hides /usr/local/share/emacs/30.0.50/lisp/org/ol-mhe
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-table hides /usr/local/share/emacs/30.0.50/lisp/org/ob-table
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-plantuml hides /usr/local/share/emacs/30.0.50/lisp/org/ob-plantuml
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ox-texinfo hides /usr/local/share/emacs/30.0.50/lisp/org/ox-texinfo
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-ocaml hides /usr/local/share/emacs/30.0.50/lisp/org/ob-ocaml
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-datetree hides /usr/local/share/emacs/30.0.50/lisp/org/org-datetree
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ol-doi hides /usr/local/share/emacs/30.0.50/lisp/org/ol-doi
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-macs hides /usr/local/share/emacs/30.0.50/lisp/org/org-macs
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-macro hides /usr/local/share/emacs/30.0.50/lisp/org/org-macro
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/oc-basic hides /usr/local/share/emacs/30.0.50/lisp/org/oc-basic
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-fold hides /usr/local/share/emacs/30.0.50/lisp/org/org-fold
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-exp hides /usr/local/share/emacs/30.0.50/lisp/org/ob-exp
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-capture hides /usr/local/share/emacs/30.0.50/lisp/org/org-capture
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-fortran hides /usr/local/share/emacs/30.0.50/lisp/org/ob-fortran
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-lob hides /usr/local/share/emacs/30.0.50/lisp/org/ob-lob
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-awk hides /usr/local/share/emacs/30.0.50/lisp/org/ob-awk
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ol hides /usr/local/share/emacs/30.0.50/lisp/org/ol
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ol-w3m hides /usr/local/share/emacs/30.0.50/lisp/org/ol-w3m
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-makefile hides /usr/local/share/emacs/30.0.50/lisp/org/ob-makefile
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-lilypond hides /usr/local/share/emacs/30.0.50/lisp/org/ob-lilypond
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-archive hides /usr/local/share/emacs/30.0.50/lisp/org/org-archive
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-processing hides /usr/local/share/emacs/30.0.50/lisp/org/ob-processing
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-eshell hides /usr/local/share/emacs/30.0.50/lisp/org/ob-eshell
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-julia hides /usr/local/share/emacs/30.0.50/lisp/org/ob-julia
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-sass hides /usr/local/share/emacs/30.0.50/lisp/org/ob-sass
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-screen hides /usr/local/share/emacs/30.0.50/lisp/org/ob-screen
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/oc-bibtex hides /usr/local/share/emacs/30.0.50/lisp/org/oc-bibtex
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-duration hides /usr/local/share/emacs/30.0.50/lisp/org/org-duration
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-org hides /usr/local/share/emacs/30.0.50/lisp/org/ob-org
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-tempo hides /usr/local/share/emacs/30.0.50/lisp/org/org-tempo
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ol-bbdb hides /usr/local/share/emacs/30.0.50/lisp/org/ol-bbdb
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-fold-core hides /usr/local/share/emacs/30.0.50/lisp/org/org-fold-core
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-clojure hides /usr/local/share/emacs/30.0.50/lisp/org/ob-clojure
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-timer hides /usr/local/share/emacs/30.0.50/lisp/org/org-timer
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-keys hides /usr/local/share/emacs/30.0.50/lisp/org/org-keys
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ob-eval hides /usr/local/share/emacs/30.0.50/lisp/org/ob-eval
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-table hides /usr/local/share/emacs/30.0.50/lisp/org/org-table
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ol-eshell hides /usr/local/share/emacs/30.0.50/lisp/org/ol-eshell
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/ox-publish hides /usr/local/share/emacs/30.0.50/lisp/org/ox-publish
/home/denin/.emacs.d/.local/straight/build-30.0.50/org/org-feed hides /usr/local/share/emacs/30.0.50/lisp/org/org-feed
/home/denin/.emacs.d/.local/straight/build-30.0.50/soap-client/soap-inspect hides /usr/local/share/emacs/30.0.50/lisp/net/soap-inspect
/home/denin/.emacs.d/.local/straight/build-30.0.50/soap-client/soap-client hides /usr/local/share/emacs/30.0.50/lisp/net/soap-client
/home/denin/.emacs.d/.local/straight/build-30.0.50/eldoc/eldoc hides /usr/local/share/emacs/30.0.50/lisp/emacs-lisp/eldoc
/home/denin/.emacs.d/.local/straight/build-30.0.50/map/map hides /usr/local/share/emacs/30.0.50/lisp/emacs-lisp/map

Features:
(shadow hide-mode-line go-translate gts-engine-youdao
gts-engine-stardict gts-engine-deepl gts-engine-google-rpc
gts-engine-google gts-engine-bing gts-implements gts-faces gts-core
evil-args emacsbug vertico-directory winum sort smiley gnus-cite textsec
uni-scripts idna-mapping ucs-normalize uni-confusable textsec-check
gnus-async gnus-bcklg qp gnus-ml nndoc gnus-agent gnus-srvr gnus-score
score-mode nnvirtual gnus-msg nntp gnus-cache gnus-dup gnus-art mm-uu
mml2015 mm-view mml-smime smime dig gnus-sum gnus-group gnus-undo
gnus-start gnus-dbus gnus-cloud nnimap nnmail mail-source utf7 nnoo
gnus-spec gnus-int gnus-range gnus-win evil-collection-gnus gnus
nnheader range mm-archive url-cache evil-collection-tabulated-list
debbugs-gnu debbugs-compat evil-collection-debbugs debbugs soap-client
rng-xsd rng-dt rng-util xsd-regexp mule-util vertico-quick dumb-jump
evil-anzu anzu git-gutter-fringe fringe-helper git-gutter vc
bug-reference macrostep-c cmacexp evil-collection-macrostep macrostep
rainbow-delimiters tramp-archive tramp-gvfs dbus org-capture org-agenda
the-org-mode-expansions evil-collection-org smartparens-org org-yt
org-element org-persist org-id org-refile org-element-ast avl-tree org
ob-emacs-lisp org-table org-loaddefs evil-collection-calendar cal-menu
calendar cal-loaddefs ob ob-tangle ol ob-ref ob-lob ob-table ob-exp
org-macro org-src evil-collection-sh-script sh-script smie executable
ob-comint org-pcomplete org-list org-footnote org-entities
evil-collection-indent lsp-diagnostics lsp-headerline lsp-icons
lsp-modeline lsp-ui lsp-ui-flycheck lsp-ui-doc
evil-collection-lsp-ui-imenu lsp-ui-imenu lsp-ui-peek lsp-ui-sideline
lsp-ui-util lsp-zig lsp-tilt lsp-steep lsp-svelte lsp-sqls
lsp-ruby-syntax-tree lsp-ruby-lsp lsp-yaml lsp-xml lsp-vimscript
lsp-vhdl lsp-volar lsp-vetur lsp-html lsp-verilog lsp-vala lsp-v
lsp-typeprof lsp-ttcn3 lsp-toml lsp-terraform lsp-tex lsp-sorbet
lsp-solargraph lsp-semgrep lsp-rust lsp-rubocop lsp-rf lsp-ruff-lsp
lsp-remark lsp-racket lsp-r lsp-purescript lsp-pylsp lsp-pyls lsp-pwsh
lsp-php lsp-pls lsp-perlnavigator lsp-perl lsp-openscad lsp-ocaml
lsp-magik lsp-nix lsp-nim lsp-nginx lsp-move lsp-mint lsp-marksman
lsp-markdown lsp-lua lsp-kotlin lsp-json lsp-javascript lsp-idris
lsp-haxe lsp-groovy lsp-hack lsp-graphql lsp-glsl lsp-gleam lsp-go
lsp-completion lsp-gdscript lsp-fsharp lsp-fortran lsp-eslint lsp-erlang
lsp-emmet lsp-elixir lsp-elm lsp-dockerfile lsp-dhall lsp-d lsp-css
lsp-csharp gnutls lsp-crystal lsp-credo lsp-cmake lsp-clojure
lsp-semantic-tokens lsp-clangd lsp-beancount lsp-bash lsp-astro lsp-awk
lsp-ansible lsp-angular lsp-ada lsp-actionscript evil-collection-vc-git
vc-git vc-dispatcher jka-compr auto-minor-mode evil-ts-obj
evil-ts-obj-yaml evil-ts-obj-nix evil-ts-obj-cpp evil-ts-obj-bash
evil-ts-obj-python evil-ts-obj-avy evil-ts-obj-core evil-ts-obj-util
evil-ts-obj-conf cus-start outli org-faces org-keys oc lsp-pyright
dap-gdb-lldb dap-utils dap-lldb ccls ccls-member-hierarchy
ccls-inheritance-hierarchy ccls-call-hierarchy ccls-tree ccls-code-lens
ccls-semantic-highlight ccls-common smartparens-c cc-mode-expansions
cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine
cc-vars cc-defs dap-mouse dap-ui lsp-treemacs lsp-treemacs-generic
lsp-treemacs-themes treemacs-treelib treemacs-persp treemacs-projectile
treemacs-evil treemacs-nerd-icons treemacs treemacs-header-line
treemacs-compatibility treemacs-mode treemacs-bookmarks treemacs-tags
treemacs-interface treemacs-persistence treemacs-filewatch-mode
treemacs-follow-mode treemacs-rendering treemacs-annotations
treemacs-async treemacs-workspaces treemacs-dom treemacs-visuals
treemacs-fringe-indicator treemacs-faces treemacs-icons treemacs-scope
treemacs-themes treemacs-core-utils pfuture treemacs-logging
treemacs-customization treemacs-macros gdb-mi gdb-module hydra bui
bui-list bui-info bui-entry bui-core bui-history bui-button bui-utils
lsp-lens dap-python dap-mode dap-tasks dap-launch lsp-docker yaml
posframe dap-overlays lsp-mode lsp-protocol evil-collection-xref xref
spinner network-stream smartparens-markdown
evil-collection-markdown-mode markdown-mode edit-indirect noutline
outline lv inline ewoc hl-todo smartparens-python
python-el-fgallina-expansions pyvenv evil-collection-python python
saveplace evil-collection-so-long so-long delsel
evil-collection-minibuffer vertico-repeat tramp-sh jupyter-tramp
tramp-cache time-stamp jupyter-server jupyter-server-kernel jupyter-repl
jupyter-widget-client simple-httpd jupyter-client jupyter-kernel
jupyter-kernelspec jupyter-env jupyter-monads thunk jupyter-messages
hmac-def jupyter-mime shr pixel-fill kinsoku url-file svg xml
jupyter-rest-api url-http url-auth url-gw nsm websocket bindat
jupyter-base eieio-base tramp trampver tramp-integration tramp-message
tramp-compat xdg parse-time iso8601 tramp-loaddefs magit-diff
smerge-mode diff magit-core magit-autorevert magit-margin
magit-transient magit-process magit-mode git-commit magit-git magit-base
evil-collection-magit-section magit-section cursor-sensor crm
evil-collection-log-edit log-edit pcvs-util add-log with-editor
doom-snippets doom-snippets-lib yasnippet evil-collection-elisp-mode
elisp-mode recentf tree-widget projectile project evil-collection-grep
grep ibuffer-vc ibuf-ext evil-collection-ibuffer ibuffer
ibuffer-loaddefs mail-extr flycheck-popup-tip evil-collection-popup
popup vi-tilde-fringe display-line-numbers jinx message sendmail
yank-media puny rfc822 mml mml-sec evil-collection-epa epa epg rfc6068
epg-config gnus-util mm-decode mm-bodies mm-encode mail-parse rfc2231
rfc2047 rfc2045 mm-util ietf-drums mail-prsvr mailabbrev mail-utils
gmm-utils mailheader time-date evil-collection-which-key which-key
savehist better-jumper company-capf company-fuzzy ht ffap company
evil-collection-vertico vertico orderless marginalia evil-goggles
evil-easymotion evil-escape evil-snipe pcre2el rxt re-builder quail
ace-window avy evil-nerd-commenter evil-nerd-commenter-operator
evil-nerd-commenter-sdk smartparens-html html-mode-expansions sgml-mode
facemenu dom consult-flycheck evil-collection-consult consult
evil-collection-bookmark bookmark server pulse color autorevert
filenotify gcmh hl-line winner smartparens-config smartparens-text
smartparens loadhist ws-butler undo-fu-session undo-fu flycheck-package
package-lint evil-collection-imenu imenu evil-collection-finder finder
finder-inf lisp-mnt evil-collection-package-menu doom-packages package
browse-url url-handlers evil-collection-flycheck flycheck persp-mode
doom-modeline doom-modeline-segments doom-modeline-env
doom-modeline-core shrink-path f f-shortdoc shortdoc nerd-icons
nerd-icons-faces nerd-icons-data nerd-icons-data-mdicon
nerd-icons-data-flicon nerd-icons-data-codicon nerd-icons-data-devicon
nerd-icons-data-sucicon nerd-icons-data-wicon nerd-icons-data-faicon
nerd-icons-data-powerline nerd-icons-data-octicon
nerd-icons-data-pomicon nerd-icons-data-ipsicon doom-themes-ext-org
solaire-mode face-remap spacemacs-light-theme spacemacs-theme
chatgpt-shell shell-maker evil-collection-view view shell goto-addr
highlight-numbers parent-mode ielm find-func evil-collection-eshell
em-prompt eshell esh-mode esh-var esh-cmd generator esh-ext esh-opt
esh-proc esh-io esh-arg pcomplete esh-module esh-groups esh-util files-x
nhexl-mode disp-table hexl ascii-table evil-embrace embrace
expand-region text-mode-expansions er-basic-expansions
expand-region-core expand-region-custom evil-surround let-alist ob-core
org-cycle org-fold org-fold-core org-compat ob-eval org-version org-macs
turbo-log s evil-textobj-tree-sitter
evil-textobj-tree-sitter-thing-at-point evil-textobj-tree-sitter-core
treesit-langs treesit-faces treesit-langs-build dtrt-indent
evil-collection-tar-mode tar-mode evil-collection-arc-mode arc-mode
archive-mode url url-proxy url-privacy url-expand url-methods
url-history url-cookie generate-lisp-file url-domsuf url-util url-parse
auth-source password-cache url-vars mailcap treesit delight tree-sitter
tree-sitter-load tree-sitter-cli tsc tsc-dyn tsc-dyn-get
evil-collection-compile compile text-property-search
evil-collection-comint comint ansi-osc ansi-color dired-aux
dired-rainbow dired-hacks-utils dirvish transient format-spec eieio
eieio-core compat evil-collection-dired dired dired-loaddefs
tsc-obsolete direnv json map evil-collection-diff-mode
evil-collection-custom cus-edit cus-load wid-edit evil-collection
annalist diff-mode dash use-package-delight ibuf-macs targets evil
evil-integration evil-maps evil-commands reveal evil-jumps
evil-command-window evil-types evil-search evil-macros evil-repeat
evil-states evil-core advice evil-common thingatpt rect evil-vars ring
derived edmacro kmacro byte-opt use-package-bind-key bind-key comp
comp-cstr warnings icons comp-run comp-common rx doom-editor
doom-projects doom-ui easy-mmode doom-keybinds pp cl-extra help-mode
use-package-core bytecomp byte-compile general tex-site doom-start
doom-modules cl-seq doom doom-lib cl-macs cl-loaddefs cl-lib pcase gv
harfbuzz jansson dynamic-modules subr-x rmc iso-transl tooltip cconv
eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type mwheel
term/x-win x-win term/common-win x-dnd touch-screen tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo gtk
x-toolkit xinput2 x multi-tty move-toolbar make-network-process
native-compile emacs)

Memory information:
((conses 16 1704985 365936) (symbols 48 99737 3) (strings 32 355138 61068)
 (string-bytes 1 10376606) (vectors 16 185907) (vector-slots 8 3346556 129426)
 (floats 8 4450 11060) (intervals 56 62765 2766) (buffers 992 35))

Message #8 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Denis Zubarev <dvzubarev <at> yandex.ru>, Yuan Fu <casouri <at> gmail.com>
Cc: 67977 <at> debbugs.gnu.org
Subject: Re: bug#67977: 30.0.50;
 tree-sitter: Emacs crashes when accessing treesit-nodes in a narrowed
 buffer
Date: Sat, 23 Dec 2023 09:26:28 +0200

> From: Denis Zubarev <dvzubarev <at> yandex.ru>
> Date: Sat, 23 Dec 2023 02:18:20 +0300
> 
> 1. emacs -Q
> 2. M-x find-file /tmp/t.py
> 3. paste there
> start=1
> def _init(self, param1, param2, param3=False):
>     self._param1 = param2
>     self._param2 = param2
>     self._param3 = param3
> 4. python-ts-mode
> 5. select two last lines and M-x narrow-to-region
> 6. answer all prompts
> 7. Put cursor on the last self
> 8. M-x eval-expression
> (progn
> (setq temp-node (treesit-node-at (point)))
> (sit-for 2)
> (garbage-collect)
> (message "node %s" temp-node))
> 9. Emacs crashes or prints node that contains garbage

Thanks.

Yuan, this also happens on the emacs-29 branch, so we should try
fixing this crash ASAP.

Message #11 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Denis Zubarev <dvzubarev <at> yandex.ru>, 67977 <at> debbugs.gnu.org
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Sat, 23 Dec 2023 00:08:51 -0800

> On Dec 22, 2023, at 11:26 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Denis Zubarev <dvzubarev <at> yandex.ru>
>> Date: Sat, 23 Dec 2023 02:18:20 +0300
>> 
>> 1. emacs -Q
>> 2. M-x find-file /tmp/t.py
>> 3. paste there
>> start=1
>> def _init(self, param1, param2, param3=False):
>>    self._param1 = param2
>>    self._param2 = param2
>>    self._param3 = param3
>> 4. python-ts-mode
>> 5. select two last lines and M-x narrow-to-region
>> 6. answer all prompts
>> 7. Put cursor on the last self
>> 8. M-x eval-expression
>> (progn
>> (setq temp-node (treesit-node-at (point)))
>> (sit-for 2)
>> (garbage-collect)
>> (message "node %s" temp-node))
>> 9. Emacs crashes or prints node that contains garbage
> 
> Thanks.
> 
> Yuan, this also happens on the emacs-29 branch, so we should try
> fixing this crash ASAP.

Yeah. The node wants to print it’s type name (with ts_node_type), which access it’s parse tree, but the tree is already freed, that means the node is outdated and shouldn’t try to print it’s type name, but should rather print “outdated”.

But simply narrowing the buffer shouldn’t reparse the buffer and cause the parse tree to be freed. Anyway, let me see what’s going on.

Yuan

Message #14 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Denis Zubarev <dvzubarev <at> yandex.ru>, 67977 <at> debbugs.gnu.org
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Sat, 23 Dec 2023 19:00:34 -0800

> On Dec 23, 2023, at 12:08 AM, Yuan Fu <casouri <at> gmail.com> wrote:
> 
> 
> 
>> On Dec 22, 2023, at 11:26 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> 
>>> From: Denis Zubarev <dvzubarev <at> yandex.ru>
>>> Date: Sat, 23 Dec 2023 02:18:20 +0300
>>> 
>>> 1. emacs -Q
>>> 2. M-x find-file /tmp/t.py
>>> 3. paste there
>>> start=1
>>> def _init(self, param1, param2, param3=False):
>>>   self._param1 = param2
>>>   self._param2 = param2
>>>   self._param3 = param3
>>> 4. python-ts-mode
>>> 5. select two last lines and M-x narrow-to-region
>>> 6. answer all prompts
>>> 7. Put cursor on the last self
>>> 8. M-x eval-expression
>>> (progn
>>> (setq temp-node (treesit-node-at (point)))
>>> (sit-for 2)
>>> (garbage-collect)
>>> (message "node %s" temp-node))
>>> 9. Emacs crashes or prints node that contains garbage
>> 
>> Thanks.
>> 
>> Yuan, this also happens on the emacs-29 branch, so we should try
>> fixing this crash ASAP.
> 
> Yeah. The node wants to print it’s type name (with ts_node_type), which access it’s parse tree, but the tree is already freed, that means the node is outdated and shouldn’t try to print it’s type name, but should rather print “outdated”.
> 
> But simply narrowing the buffer shouldn’t reparse the buffer and cause the parse tree to be freed. Anyway, let me see what’s going on.


I pushed a fix and now it shouldn’t crash anymore. However, I’m yet not sure why at some point the buffer was widened. Is there any way to track who called widen?

Yuan

Message #17 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: dvzubarev <at> yandex.ru, 67977 <at> debbugs.gnu.org
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Sun, 24 Dec 2023 09:11:57 +0200

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sat, 23 Dec 2023 19:00:34 -0800
> Cc: Denis Zubarev <dvzubarev <at> yandex.ru>,
>  67977 <at> debbugs.gnu.org
> 
> >> Yuan, this also happens on the emacs-29 branch, so we should try
> >> fixing this crash ASAP.
> > 
> > Yeah. The node wants to print it’s type name (with ts_node_type), which access it’s parse tree, but the tree is already freed, that means the node is outdated and shouldn’t try to print it’s type name, but should rather print “outdated”.
> > 
> > But simply narrowing the buffer shouldn’t reparse the buffer and cause the parse tree to be freed. Anyway, let me see what’s going on.
> 
> 
> I pushed a fix and now it shouldn’t crash anymore. However, I’m yet not sure why at some point the buffer was widened. Is there any way to track who called widen?

Run Emacs under GDB with a breakpoint at Fwiden, then look at the
backtrace.  The command "xbacktrace", defined on src/.gdbinit, will
show a Lisp backtrace as well.

But I already did the above, and the answer is the expected one: it's
JIT font-lock, which calls font-lock-fontify-region, which does:

    (save-restriction
      (unless font-lock-dont-widen (widen))

And if you leave blink-cursor-mode and global-eldoc-mode on (which is
the default), you have also another caller: jit-lock-context-fontify
(which is called from a timer).

Does this answer your question?

Btw, I hope that these calls to 'widen' don't require unnecessary
reparsing by tree-sitter, do they?

Message #20 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Denis Zubarev <dvzubarev <at> yandex.ru>, 67977 <at> debbugs.gnu.org
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Tue, 26 Dec 2023 20:15:24 -0800

> On Dec 23, 2023, at 11:11 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Sat, 23 Dec 2023 19:00:34 -0800
>> Cc: Denis Zubarev <dvzubarev <at> yandex.ru>,
>> 67977 <at> debbugs.gnu.org
>> 
>>>> Yuan, this also happens on the emacs-29 branch, so we should try
>>>> fixing this crash ASAP.
>>> 
>>> Yeah. The node wants to print it’s type name (with ts_node_type), which access it’s parse tree, but the tree is already freed, that means the node is outdated and shouldn’t try to print it’s type name, but should rather print “outdated”.
>>> 
>>> But simply narrowing the buffer shouldn’t reparse the buffer and cause the parse tree to be freed. Anyway, let me see what’s going on.
>> 
>> 
>> I pushed a fix and now it shouldn’t crash anymore. However, I’m yet not sure why at some point the buffer was widened. Is there any way to track who called widen?
> 
> Run Emacs under GDB with a breakpoint at Fwiden, then look at the
> backtrace.  The command "xbacktrace", defined on src/.gdbinit, will
> show a Lisp backtrace as well.
> 
> But I already did the above, and the answer is the expected one: it's
> JIT font-lock, which calls font-lock-fontify-region, which does:
> 
>    (save-restriction
>      (unless font-lock-dont-widen (widen))
> 
> And if you leave blink-cursor-mode and global-eldoc-mode on (which is
> the default), you have also another caller: jit-lock-context-fontify
> (which is called from a timer).
> 
> Does this answer your question?

Yes, they do. Many thanks!

> Btw, I hope that these calls to 'widen' don't require unnecessary
> reparsing by tree-sitter, do they?

Yes, but only because we called treesit-node-at while the buffer is narrowed, which triggers a reparse. Font-lock and jit-lock themselves always access the parser with widened buffer so they don’t trigger reparse on their own. 

So it seems working in a narrowed buffer would trigger a lot of back-and-fortch reparse. I wonder if it’s worth optimizing for (eg, use two parsers behind the scenes, one for widened buffer and one for narrowed buffer).

Yuan

Message #23 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: dvzubarev <at> yandex.ru, 67977 <at> debbugs.gnu.org
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Wed, 27 Dec 2023 14:57:09 +0200

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Tue, 26 Dec 2023 20:15:24 -0800
> Cc: Denis Zubarev <dvzubarev <at> yandex.ru>,
>  67977 <at> debbugs.gnu.org
> 
> > Btw, I hope that these calls to 'widen' don't require unnecessary
> > reparsing by tree-sitter, do they?
> 
> Yes, but only because we called treesit-node-at while the buffer is narrowed, which triggers a reparse. Font-lock and jit-lock themselves always access the parser with widened buffer so they don’t trigger reparse on their own. 
> 
> So it seems working in a narrowed buffer would trigger a lot of back-and-fortch reparse. I wonder if it’s worth optimizing for (eg, use two parsers behind the scenes, one for widened buffer and one for narrowed buffer).

Could font-lock-dont-widen help, perhaps?

Message #26 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Denis Zubarev <dvzubarev <at> yandex.ru>, 67977 <at> debbugs.gnu.org
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Thu, 28 Dec 2023 00:07:51 -0800

> On Dec 27, 2023, at 4:57 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Tue, 26 Dec 2023 20:15:24 -0800
>> Cc: Denis Zubarev <dvzubarev <at> yandex.ru>,
>> 67977 <at> debbugs.gnu.org
>> 
>>> Btw, I hope that these calls to 'widen' don't require unnecessary
>>> reparsing by tree-sitter, do they?
>> 
>> Yes, but only because we called treesit-node-at while the buffer is narrowed, which triggers a reparse. Font-lock and jit-lock themselves always access the parser with widened buffer so they don’t trigger reparse on their own. 
>> 
>> So it seems working in a narrowed buffer would trigger a lot of back-and-fortch reparse. I wonder if it’s worth optimizing for (eg, use two parsers behind the scenes, one for widened buffer and one for narrowed buffer).
> 
> Could font-lock-dont-widen help, perhaps?

Yes. If font-lock doesn’t widen, then there wouldn’t be back-and-forth reparses. 

Yuan

Message #29 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Yuan Fu <casouri <at> gmail.com>, Eli Zaretskii <eliz <at> gnu.org>
Cc: Denis Zubarev <dvzubarev <at> yandex.ru>, 67977 <at> debbugs.gnu.org
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Thu, 28 Dec 2023 13:44:43 +0200

On 28/12/2023 10:07, Yuan Fu wrote:
> 
>> On Dec 27, 2023, at 4:57 AM, Eli Zaretskii<eliz <at> gnu.org>  wrote:
>>
>>> From: Yuan Fu<casouri <at> gmail.com>
>>> Date: Tue, 26 Dec 2023 20:15:24 -0800
>>> Cc: Denis Zubarev<dvzubarev <at> yandex.ru>,
>>> 67977 <at> debbugs.gnu.org
>>>
>>>> Btw, I hope that these calls to 'widen' don't require unnecessary
>>>> reparsing by tree-sitter, do they?
>>> Yes, but only because we called treesit-node-at while the buffer is narrowed, which triggers a reparse. Font-lock and jit-lock themselves always access the parser with widened buffer so they don’t trigger reparse on their own.
>>>
>>> So it seems working in a narrowed buffer would trigger a lot of back-and-fortch reparse. I wonder if it’s worth optimizing for (eg, use two parsers behind the scenes, one for widened buffer and one for narrowed buffer).
>> Could font-lock-dont-widen help, perhaps?
> Yes. If font-lock doesn’t widen, then there wouldn’t be back-and-forth reparses.

But then treesit major modes will be affected by user narrowing (e.g. if 
the user narrowed to inside the string, the buffer won't be highlighted 
as a string).

Message #32 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: casouri <at> gmail.com, dvzubarev <at> yandex.ru, 67977 <at> debbugs.gnu.org
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Thu, 28 Dec 2023 15:53:34 +0200

> Date: Thu, 28 Dec 2023 13:44:43 +0200
> Cc: Denis Zubarev <dvzubarev <at> yandex.ru>, 67977 <at> debbugs.gnu.org
> From: Dmitry Gutov <dmitry <at> gutov.dev>
> 
> >> Could font-lock-dont-widen help, perhaps?
> > Yes. If font-lock doesn’t widen, then there wouldn’t be back-and-forth reparses.
> 
> But then treesit major modes will be affected by user narrowing (e.g. if 
> the user narrowed to inside the string, the buffer won't be highlighted 
> as a string).

Why is that a problem?  When the user narrows the buffer, the part
outside the narrowing doesn't exist as far as Emacs is concerned.

Message #35 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: casouri <at> gmail.com, dvzubarev <at> yandex.ru, 67977 <at> debbugs.gnu.org
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Thu, 28 Dec 2023 18:16:30 +0200

On 28/12/2023 15:53, Eli Zaretskii wrote:
>> Date: Thu, 28 Dec 2023 13:44:43 +0200
>> Cc: Denis Zubarev<dvzubarev <at> yandex.ru>,67977 <at> debbugs.gnu.org
>> From: Dmitry Gutov<dmitry <at> gutov.dev>
>>
>>>> Could font-lock-dont-widen help, perhaps?
>>> Yes. If font-lock doesn’t widen, then there wouldn’t be back-and-forth reparses.
>> But then treesit major modes will be affected by user narrowing (e.g. if
>> the user narrowed to inside the string, the buffer won't be highlighted
>> as a string).
> Why is that a problem?  When the user narrows the buffer, the part
> outside the narrowing doesn't exist as far as Emacs is concerned.

That's not how it works in most major modes, at least since the 
introduction of font-lock-dont-widen 20 years ago.

Like its docstring says, the exceptions were supposed to be weird modes 
like RMAIL and Info which use narrowing for their own purposes (that 
seems buggy in Info's case, when 'C-x n w' breaks the intended display 
right away). But even Info-mode doesn't actually change 
font-lock-dont-widen, actually, because the apparent behavior would be 
the same. But it could.

I don't have a personal stake in this (I never use narrowing 
interactively). But maybe you'll want to make a poll, to ask the users 
that do.

Message #38 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Eli Zaretskii <eliz <at> gnu.org>, dvzubarev <at> yandex.ru, 67977 <at> debbugs.gnu.org
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Thu, 28 Dec 2023 23:00:18 -0800

> On Dec 28, 2023, at 8:16 AM, Dmitry Gutov <dmitry <at> gutov.dev> wrote:
> 
> On 28/12/2023 15:53, Eli Zaretskii wrote:
>>> Date: Thu, 28 Dec 2023 13:44:43 +0200
>>> Cc: Denis Zubarev<dvzubarev <at> yandex.ru>,67977 <at> debbugs.gnu.org
>>> From: Dmitry Gutov<dmitry <at> gutov.dev>
>>> 
>>>>> Could font-lock-dont-widen help, perhaps?
>>>> Yes. If font-lock doesn’t widen, then there wouldn’t be back-and-forth reparses.
>>> But then treesit major modes will be affected by user narrowing (e.g. if
>>> the user narrowed to inside the string, the buffer won't be highlighted
>>> as a string).
>> Why is that a problem?  When the user narrows the buffer, the part
>> outside the narrowing doesn't exist as far as Emacs is concerned.
> 
> That's not how it works in most major modes, at least since the introduction of font-lock-dont-widen 20 years ago.
> 
> Like its docstring says, the exceptions were supposed to be weird modes like RMAIL and Info which use narrowing for their own purposes (that seems buggy in Info's case, when 'C-x n w' breaks the intended display right away). But even Info-mode doesn't actually change font-lock-dont-widen, actually, because the apparent behavior would be the same. But it could.
> 
> I don't have a personal stake in this (I never use narrowing interactively). But maybe you'll want to make a poll, to ask the users that do.

I guess that depends on how you view narrowing, I assume most of the time the user considers narrowing to be something that narrows the view to a region, rather than effectively removing the rest of the buffer. IIRC We’ve had discussions on adding “types” to narrows but to no conclusion.

Anyway, if the reparse caused by narrowing prove to be problematic, I can implement the optimization I mentioned earlier, where we use two parsers, one for when buffer is widened and one for when buffer is narrowed. This will be in C and is transparent to lisp.

Yuan

Message #41 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Yuan Fu <casouri <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, dvzubarev <at> yandex.ru, 67977 <at> debbugs.gnu.org
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Fri, 29 Dec 2023 14:48:32 +0200

On 29/12/2023 09:00, Yuan Fu wrote:
> 
> 
>> On Dec 28, 2023, at 8:16 AM, Dmitry Gutov <dmitry <at> gutov.dev> wrote:
>>
>> On 28/12/2023 15:53, Eli Zaretskii wrote:
>>>> Date: Thu, 28 Dec 2023 13:44:43 +0200
>>>> Cc: Denis Zubarev<dvzubarev <at> yandex.ru>,67977 <at> debbugs.gnu.org
>>>> From: Dmitry Gutov<dmitry <at> gutov.dev>
>>>>
>>>>>> Could font-lock-dont-widen help, perhaps?
>>>>> Yes. If font-lock doesn’t widen, then there wouldn’t be back-and-forth reparses.
>>>> But then treesit major modes will be affected by user narrowing (e.g. if
>>>> the user narrowed to inside the string, the buffer won't be highlighted
>>>> as a string).
>>> Why is that a problem?  When the user narrows the buffer, the part
>>> outside the narrowing doesn't exist as far as Emacs is concerned.
>>
>> That's not how it works in most major modes, at least since the introduction of font-lock-dont-widen 20 years ago.
>>
>> Like its docstring says, the exceptions were supposed to be weird modes like RMAIL and Info which use narrowing for their own purposes (that seems buggy in Info's case, when 'C-x n w' breaks the intended display right away). But even Info-mode doesn't actually change font-lock-dont-widen, actually, because the apparent behavior would be the same. But it could.
>>
>> I don't have a personal stake in this (I never use narrowing interactively). But maybe you'll want to make a poll, to ask the users that do.
> 
> I guess that depends on how you view narrowing, I assume most of the time the user considers narrowing to be something that narrows the view to a region, rather than effectively removing the rest of the buffer. IIRC We’ve had discussions on adding “types” to narrows but to no conclusion.

Indeed.

> Anyway, if the reparse caused by narrowing prove to be problematic, I can implement the optimization I mentioned earlier, where we use two parsers, one for when buffer is widened and one for when buffer is narrowed. This will be in C and is transparent to lisp.

This definitely can work, but perhaps we should examine concrete use 
cases first. It could be that the caller would want the full parse tree 
available anyway, when the view was narrowed interactively by the user.

Then the actual advice from us would be to (save-restriction (widen 
...)) anyway, and the second parse tree would stay mostly unused.

It's a good thing to have fixed the crash, though, so the developers can 
try it both ways and decide what's best for them.

Message #44 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Denis Zubarev <dvzubarev <at> yandex.ru>,
 67977 <at> debbugs.gnu.org
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Fri, 29 Dec 2023 20:35:45 -0800

> On Dec 29, 2023, at 4:48 AM, Dmitry Gutov <dmitry <at> gutov.dev> wrote:
> 
> On 29/12/2023 09:00, Yuan Fu wrote:
>>> On Dec 28, 2023, at 8:16 AM, Dmitry Gutov <dmitry <at> gutov.dev> wrote:
>>> 
>>> On 28/12/2023 15:53, Eli Zaretskii wrote:
>>>>> Date: Thu, 28 Dec 2023 13:44:43 +0200
>>>>> Cc: Denis Zubarev<dvzubarev <at> yandex.ru>,67977 <at> debbugs.gnu.org
>>>>> From: Dmitry Gutov<dmitry <at> gutov.dev>
>>>>> 
>>>>>>> Could font-lock-dont-widen help, perhaps?
>>>>>> Yes. If font-lock doesn’t widen, then there wouldn’t be back-and-forth reparses.
>>>>> But then treesit major modes will be affected by user narrowing (e.g. if
>>>>> the user narrowed to inside the string, the buffer won't be highlighted
>>>>> as a string).
>>>> Why is that a problem?  When the user narrows the buffer, the part
>>>> outside the narrowing doesn't exist as far as Emacs is concerned.
>>> 
>>> That's not how it works in most major modes, at least since the introduction of font-lock-dont-widen 20 years ago.
>>> 
>>> Like its docstring says, the exceptions were supposed to be weird modes like RMAIL and Info which use narrowing for their own purposes (that seems buggy in Info's case, when 'C-x n w' breaks the intended display right away). But even Info-mode doesn't actually change font-lock-dont-widen, actually, because the apparent behavior would be the same. But it could.
>>> 
>>> I don't have a personal stake in this (I never use narrowing interactively). But maybe you'll want to make a poll, to ask the users that do.
>> I guess that depends on how you view narrowing, I assume most of the time the user considers narrowing to be something that narrows the view to a region, rather than effectively removing the rest of the buffer. IIRC We’ve had discussions on adding “types” to narrows but to no conclusion.
> 
> Indeed.
> 
>> Anyway, if the reparse caused by narrowing prove to be problematic, I can implement the optimization I mentioned earlier, where we use two parsers, one for when buffer is widened and one for when buffer is narrowed. This will be in C and is transparent to lisp.
> 
> This definitely can work, but perhaps we should examine concrete use cases first. It could be that the caller would want the full parse tree available anyway, when the view was narrowed interactively by the user.
> 
> Then the actual advice from us would be to (save-restriction (widen ...)) anyway, and the second parse tree would stay mostly unused.

I’m sure both strict and non-strict narrowing has their use-cases. For movement functions, we probably don’t want to widen; for indentation, I think both could be useful depending on, again, specific use-cases.

I don’t think any of use are motivated enough :-) We can wait until someone with a specific problem/requirement comes up.

> 
> It's a good thing to have fixed the crash, though, so the developers can try it both ways and decide what's best for them.

Right, Lisp programmers can decide whether they want to widen. And if there are enough important use-cases for widening, we can starting thinking about optimizing switching back-and-forth between narrowed and widened buffer.

Yuan

Message #47 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Denis Zubarev <dvzubarev <at> yandex.ru>
To: Yuan Fu <casouri <at> gmail.com>,
	Eli Zaretskii <eliz <at> gnu.org>
Cc: "67977 <at> debbugs.gnu.org" <67977 <at> debbugs.gnu.org>
Subject: Re: bug#67977: 30.0.50;
 tree-sitter: Emacs crashes when accessing treesit-nodes in a narrowed
 buffer
Date: Sat, 30 Dec 2023 19:21:35 +0300

[Message part 1 (text/html, inline)]

Message #50 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Denis Zubarev <dvzubarev <at> yandex.ru>
Cc: Eli Zaretskii <eliz <at> gnu.org>,
 "67977 <at> debbugs.gnu.org" <67977 <at> debbugs.gnu.org>
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Sat, 30 Dec 2023 12:23:18 -0800

> On Dec 30, 2023, at 8:21 AM, Denis Zubarev <dvzubarev <at> yandex.ru> wrote:
> 
> > I pushed a fix and now it shouldn’t crash anymore. However, I’m yet not sure why at some point the buffer was widened. Is there any way to track who called widen?
>  Thank you, It doesn't crash anymore.
>  > So it seems working in a narrowed buffer would trigger a lot of back-and-fortch reparse. I wonder if it’s worth optimizing for (eg, use two parsers behind the scenes, one for widened buffer and one for narrowed buffer).
>  I have performed a quite naive benchmark and haven't seen any significant slow down when inserting text in a narrowed buffer.

Right, when you type, since the only thing that access the parser is font-lock, which always widens the buffer, there’s no unnecessary reparse. If you invoke some function that access the parser while the buffer is narrowed, that’ll trigger a reparse, and the next time font-lock runs, it’ll widen and make the parser reparse the full buffer again.

Yuan

Message #53 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Yuan Fu <casouri <at> gmail.com>, Denis Zubarev <dvzubarev <at> yandex.ru>
Cc: Eli Zaretskii <eliz <at> gnu.org>,
 "67977 <at> debbugs.gnu.org" <67977 <at> debbugs.gnu.org>
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Sun, 31 Dec 2023 02:08:38 +0200

On 30/12/2023 22:23, Yuan Fu wrote:
> 
>> On Dec 30, 2023, at 8:21 AM, Denis Zubarev<dvzubarev <at> yandex.ru>  wrote:
>>
>>> I pushed a fix and now it shouldn’t crash anymore. However, I’m yet not sure why at some point the buffer was widened. Is there any way to track who called widen?
>>   Thank you, It doesn't crash anymore.
>>   > So it seems working in a narrowed buffer would trigger a lot of back-and-fortch reparse. I wonder if it’s worth optimizing for (eg, use two parsers behind the scenes, one for widened buffer and one for narrowed buffer).
>>   I have performed a quite naive benchmark and haven't seen any significant slow down when inserting text in a narrowed buffer.
> Right, when you type, since the only thing that access the parser is font-lock, which always widens the buffer, there’s no unnecessary reparse. If you invoke some function that access the parser while the buffer is narrowed, that’ll trigger a reparse, and the next time font-lock runs, it’ll widen and make the parser reparse the full buffer again.

The difference might also only be noticeable with larger files: 
tree-sitter is pretty fast by itself, so an extra reparse might not make 
a difference unless it triggers a full re-fontification or application 
of text properties over a large span.

Message #56 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Denis Zubarev <dvzubarev <at> yandex.ru>
To: Yuan Fu <casouri <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>,
 "67977 <at> debbugs.gnu.org" <67977 <at> debbugs.gnu.org>
Subject: Re: bug#67977: 30.0.50;
 tree-sitter: Emacs crashes when accessing treesit-nodes in a narrowed
 buffer
Date: Sun, 31 Dec 2023 13:39:10 +0300

[Message part 1 (text/html, inline)]

Message #59 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Denis Zubarev <dvzubarev <at> yandex.ru>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: casouri <at> gmail.com, 67977 <at> debbugs.gnu.org
Subject: Re: bug#67977: 30.0.50;
 tree-sitter: Emacs crashes when accessing treesit-nodes in a narrowed
 buffer
Date: Sun, 31 Dec 2023 14:56:25 +0200

> From: Denis Zubarev <dvzubarev <at> yandex.ru>
> Cc: Eli Zaretskii <eliz <at> gnu.org>,
> 	"67977 <at> debbugs.gnu.org" <67977 <at> debbugs.gnu.org>
> Date: Sun, 31 Dec 2023 13:39:10 +0300
> 
> Yes, if accessing nodes, then there is a slowdown.
>  
> The result is what benchmark-run returns.
>  
>  # func  buff-size  widen                 narrowed                
> ------------------------------------------------------------------
>       1  671 B      (0.22944 2 0.022735)  (0.520645 9 0.1100442)  
>      11  7 KiB      (0.23176 3 0.038288)  (0.9340281 9 0.116293   
>      31  20 KiB     (0.27746 3 0.039054)  (1.7671007 10 0.134515)
>  
> Also there were errors when running benchmark in the narrowed buffer:
> Error muted by safe_call: (internal--syntax-propertize 1482) signaled (args-out-of-range 1 1872)
> Error muted by safe_call: (internal--syntax-propertize 282) signaled (args-out-of-range 1 672)
> 
>  
> Recipe:
> 1. emacs -Q
> 2. paste the code to the buffer
> 3. M-x python-ts-mode
> 4. put the cursor in the place of |
>  
> 5. M-x eval-expression
>    (require 'benchmark)
>    (benchmark-run 200 (progn (insert "v")  (insert "=") (insert "f") (insert "(") (insert ")") (treesit-node-at
> (point)) (insert "\n") (font-lock-ensure) ))
> 6. M-x undo
>  
> For benchmark in narrowed state select the body of a function, starting from a documentation string
> and M-x narrow-to-region.
>  
> To increase buffer size copy the whole function:
> M-x eval-expression
> (dotimes (_ 10) (yank))
> 
>  
> Code:
> class TempC:
>     def func_call():
>         """documentation
>         """
>         var1 = 3
>         var2 = func(1, temp={'1':1, '2':2},
>                     temp1=TempC([1], [3]), ab='83, 8',)
>         var3= r'\n str'
>         var4 = f'temp {TempC("1"+"3")}|{v + "mon" +v}and {func()}'
>  
>         |
>  
>         varb =832
>         if varb:
>             pass
>         elif not varb is not None and varb == 3:
>             varb = 38
>         else:
>             pass
>  
>         for i in range():
>             print(i)
>  
>         def nested():
>             class Nested(Tempc):
>                 def __init__(self):
>                     self._init = True
>             return Nested()
>         return nested

Adding Stefan to the discussion.

Message #62 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Denis Zubarev <dvzubarev <at> yandex.ru>, Yuan Fu <casouri <at> gmail.com>,
 Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: Eli Zaretskii <eliz <at> gnu.org>,
 "67977 <at> debbugs.gnu.org" <67977 <at> debbugs.gnu.org>
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Sun, 31 Dec 2023 15:40:28 +0200

On 31/12/2023 12:39, Denis Zubarev wrote:
> Also there were errors when running benchmark in the narrowed buffer:
> Error muted by safe_call: (internal--syntax-propertize 1482) signaled 
> (args-out-of-range 1 1872)

That does look like a bug.

The patch below seems to fix it:

diff --git a/lisp/treesit.el b/lisp/treesit.el
index 264b95dc3a3..46ebadcf057 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1150,7 +1150,7 @@ treesit--pre-syntax-ppss
     (if (and new-start (< new-start start))
         (progn
           (setq treesit--syntax-propertize-start nil)
-          (cons new-start end))
+          (cons (max new-start (point-min)) end))
       nil)))

 ;;; Indent


Or maybe syntax-propertize itself should have a protection against going 
outside of bounds:

diff --git a/lisp/emacs-lisp/syntax.el b/lisp/emacs-lisp/syntax.el
index e35992298a6..61a9e79b59c 100644
--- a/lisp/emacs-lisp/syntax.el
+++ b/lisp/emacs-lisp/syntax.el
@@ -431,7 +431,7 @@ syntax-propertize
                      (if (or (null new)
                              (and (>= (car new) start) (<= (cdr new) 
end)))
                          nil
-                       (setq start (car new))
+                       (setq start (max (car new) (point-min)))
                        (setq end (cdr new))
                        ;; If there's been a change, we should go 
through the
                        ;; list again since this new position may

Message #65 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Yuan Fu <casouri <at> gmail.com>, Eli Zaretskii <eliz <at> gnu.org>,
 Denis Zubarev <dvzubarev <at> yandex.ru>,
 "67977 <at> debbugs.gnu.org" <67977 <at> debbugs.gnu.org>
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Mon, 01 Jan 2024 23:46:52 -0500

> diff --git a/lisp/treesit.el b/lisp/treesit.el
> index 264b95dc3a3..46ebadcf057 100644
> --- a/lisp/treesit.el
> +++ b/lisp/treesit.el
> @@ -1150,7 +1150,7 @@ treesit--pre-syntax-ppss
>      (if (and new-start (< new-start start))
>          (progn
>            (setq treesit--syntax-propertize-start nil)
> -          (cons new-start end))
> +          (cons (max new-start (point-min)) end))
>        nil)))
>
>  ;;; Indent

Sounds fine to me.  Often the caller of `syntax-propertize-function`
should widen beforehand, but in cases like `mmm-mode` indeed that's not
always desired.

> Or maybe syntax-propertize itself should have a protection against going
> outside of bounds:
>
> diff --git a/lisp/emacs-lisp/syntax.el b/lisp/emacs-lisp/syntax.el
> index e35992298a6..61a9e79b59c 100644
> --- a/lisp/emacs-lisp/syntax.el
> +++ b/lisp/emacs-lisp/syntax.el
> @@ -431,7 +431,7 @@ syntax-propertize
>                       (if (or (null new)
>                               (and (>= (car new) start) (<= (cdr new) end)))
>                           nil
> -                       (setq start (car new))
> +                       (setq start (max (car new) (point-min)))
>                         (setq end (cdr new))
>                         ;; If there's been a change, we should go through
>                           the
>                         ;; list again since this new position may

I think it's preferable for the expand-region function to perform this
test.  We could `cl-assert` that it's within BOB...EOB to help catch
such bugs (and clarify who's in charge of avoiding the problem), but
maybe we can mention it in the docstring.  But I personally consider
that anything that sends buffer positions is in charge to make sure that
they're within bounds, unless specifically documented otherwise, so
documenting it seems redundant.


        Stefan

Message #68 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Yuan Fu <casouri <at> gmail.com>, Eli Zaretskii <eliz <at> gnu.org>,
 Denis Zubarev <dvzubarev <at> yandex.ru>,
 "67977 <at> debbugs.gnu.org" <67977 <at> debbugs.gnu.org>
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Tue, 2 Jan 2024 15:34:35 +0200

On 02/01/2024 06:46, Stefan Monnier wrote:
>> diff --git a/lisp/treesit.el b/lisp/treesit.el
>> index 264b95dc3a3..46ebadcf057 100644
>> --- a/lisp/treesit.el
>> +++ b/lisp/treesit.el
>> @@ -1150,7 +1150,7 @@ treesit--pre-syntax-ppss
>>       (if (and new-start (< new-start start))
>>           (progn
>>             (setq treesit--syntax-propertize-start nil)
>> -          (cons new-start end))
>> +          (cons (max new-start (point-min)) end))
>>         nil)))
>>
>>   ;;; Indent
> Sounds fine to me.  Often the caller of `syntax-propertize-function`
> should widen beforehand, but in cases like `mmm-mode` indeed that's not
> always desired.

Thank you, I've pushed that fix.

>> Or maybe syntax-propertize itself should have a protection against going
>> outside of bounds:
>>
>> diff --git a/lisp/emacs-lisp/syntax.el b/lisp/emacs-lisp/syntax.el
>> index e35992298a6..61a9e79b59c 100644
>> --- a/lisp/emacs-lisp/syntax.el
>> +++ b/lisp/emacs-lisp/syntax.el
>> @@ -431,7 +431,7 @@ syntax-propertize
>>                        (if (or (null new)
>>                                (and (>= (car new) start) (<= (cdr new) end)))
>>                            nil
>> -                       (setq start (car new))
>> +                       (setq start (max (car new) (point-min)))
>>                          (setq end (cdr new))
>>                          ;; If there's been a change, we should go through
>>                            the
>>                          ;; list again since this new position may
> I think it's preferable for the expand-region function to perform this
> test.  We could `cl-assert` that it's within BOB...EOB to help catch
> such bugs (and clarify who's in charge of avoiding the problem), but
> maybe we can mention it in the docstring.  But I personally consider
> that anything that sends buffer positions is in charge to make sure that
> they're within bounds, unless specifically documented otherwise, so
> documenting it seems redundant.

Fair enough.

And I'll leave the question of adding cl-assert (or not) to you or 
others. Though might be that the signaled args-out-of-range will be a 
reliable indicator of this situation anyway.

Message #71 received at 67977 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Stefan Monnier <monnier <at> iro.umontreal.ca>,
 Denis Zubarev <dvzubarev <at> yandex.ru>,
 "67977 <at> debbugs.gnu.org" <67977 <at> debbugs.gnu.org>
Subject: Re: bug#67977: 30.0.50; tree-sitter: Emacs crashes when accessing
 treesit-nodes in a narrowed buffer
Date: Tue, 2 Jan 2024 14:58:13 -0800

> Though might be that the signaled args-out-of-range will be a reliable indicator of this situation anyway.

Right. The signal is pretty clear on what’s wrong.

Yuan

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.