GNU bug report logs - #24892
{s,}brk removed from FreeBSD 11.x and later, arm64 architecture

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 24892 in the body.
You can then email your comments to 24892 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: ashish.is <at> lostca.se (Ashish SHUKLA)
To: bug-gnu-emacs <at> gnu.org
Subject: {s,}brk removed from FreeBSD 11.x and later, arm64 architecture
Date: Mon, 07 Nov 2016 11:31:17 +0530

[Message part 1 (text/plain, inline)]

Hi,

As mentioned in the subject, {,s}brk syscalls are removed from arm64
architecture on FreeBSD 11.x[1]. This causes Emacs to fail to compile on those
platforms. Following is an excerpt from the build process:

--8<---------------cut here---------------start------------->8---
cc -c  -Demacs  -I. -I. -I../lib -I../lib                    -MMD -MF deps/gmalloc.d -MP    -Wno-switch -Wno-tautological-constant-out-of-range-compare -Wno-pointer-sign -Wno-string-plus-int -Wno-unknown-attributes  -O2 -pipe  -fno-strict-aliasing  gmalloc.c
cc -c  -Demacs  -I. -I. -I../lib -I../lib                    -MMD -MF deps/doc.d -MP    -Wno-switch -Wno-tautological-constant-out-of-range-compare -Wno-pointer-sign -Wno-string-plus-int -Wno-unknown-attributes  -O2 -pipe  -fno-strict-aliasing  doc.c
cc -Demacs  -I. -I. -I../lib -I../lib                    -MMD -MF deps/.d -MP    -Wno-switch -Wno-tautological-constant-out-of-range-compare -Wno-pointer-sign -Wno-string-plus-int -Wno-unknown-attributes  -O2 -pipe  -fno-strict-aliasing  -Wl,-znocombreloc   \
  -o temacs  vm-limit.o dispnew.o frame.o scroll.o xdisp.o menu.o  window.o charset.o coding.o category.o ccl.o character.o chartab.o bidi.o cm.o term.o terminal.o xfaces.o    emacs.o keyboard.o macros.o keymap.o sysdep.o buffer.o filelock.o insdel.o marker.o minibuf.o fileio.o dired.o cmds.o casetab.o casefiddle.o indent.o search.o regex.o undo.o alloc.o data.o doc.o editfns.o callint.o eval.o floatfns.o fns.o font.o print.o lread.o dynlib.oemacs-module.o syntax.o unexelf.o bytecode.o process.o gnutls.o callproc.o region-cache.o sound.o atimer.o doprnt.o intervals.o textprop.o composite.o xml.o kqueue.o  profiler.o decompress.o          terminfo.o lastfile.o gmalloc.o     ../lib/libgnu.a                 -lrt  -lexecinfo       -lutil -lncurses         -lpthread  -lm -lz
alloc.o: In function `Fmemory_limit':
alloc.c:(.text+0x65e8): undefined reference to `sbrk'
unexelf.o: In function `unexec':
unexelf.c:(.text+0x200): undefined reference to `sbrk'
gmalloc.o: In function `__default_morecore':
gmalloc.c:(.text+0x8): undefined reference to `sbrk'
cc: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[2]: *** [Makefile:596: temacs] Error 1
gmake[2]: Leaving directory '/scratch/ashish/work/emacs-25.1/src'
gmake[1]: *** [Makefile:398: src] Error 2
gmake[1]: Leaving directory '/scratch/ashish/work/emacs-25.1'
--8<---------------cut here---------------end--------------->8---

If you need any other information to work on this, please let me know.

References:
[1]  https://reviews.freebsd.org/rS300303

Thanks in advance
-- 
Ashish SHUKLA

“A designer knows he has achieved perfection not when there is nothing
left to add, but when there is nothing left to take away.”
                                              (Antoine de Saint-Exupéry)

Sent from my Emacs

[signature.asc (application/pgp-signature, inline)]

Message #12 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: ashish.is <at> lostca.se (Ashish SHUKLA)
Cc: 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later,
 arm64 architecture
Date: Mon, 07 Nov 2016 17:19:38 +0200

> From: ashish.is <at> lostca.se (Ashish SHUKLA)
> Date: Mon, 07 Nov 2016 11:31:17 +0530
> 
> As mentioned in the subject, {,s}brk syscalls are removed from arm64
> architecture on FreeBSD 11.x[1].

Are they mad?

Thanks for reporting this.

A "fix" for memory-limit is simple, like NS already does (but we will
need a configure-time test for sbrk).  The issue with unexelf is more
serious.  I guess we will have to use mmap (or the result of the
existing call to it) instead?

Message #15 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Ashish SHUKLA <ashish.is <at> lostca.se>
Cc: 24892 <at> debbugs.gnu.org
Subject: {s,}brk removed from FreeBSD 11.x and later, arm64 architecture
Date: Tue, 8 Nov 2016 09:54:45 -0800

[Message part 1 (text/plain, inline)]

Thanks for reporting that. Although I don't have easy access to that 
platform, I simulated the problem on FreeBSD 11 x86-64 and came up with 
the attached patch, which I installed into the Emacs master branch. 
Please give it a try if you can, and let me know either way. I doubt 
whether this patch will apply to Emacs 25 and have not investigated the 
details there.

[0001-Port-to-FreeBSD-11-AMD.patch (application/x-patch, attachment)]

Message #20 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 24892 <at> debbugs.gnu.org, ashish.is <at> lostca.se
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later,
 arm64 architecture
Date: Tue, 08 Nov 2016 22:11:22 +0200

> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Tue, 8 Nov 2016 09:54:45 -0800
> Cc: 24892 <at> debbugs.gnu.org
> 
> Thanks for reporting that. Although I don't have easy access to that 
> platform, I simulated the problem on FreeBSD 11 x86-64 and came up with 
> the attached patch, which I installed into the Emacs master branch. 
> Please give it a try if you can, and let me know either way. I doubt 
> whether this patch will apply to Emacs 25 and have not investigated the 
> details there.

Thanks!

I think we must have at least a workaround for this in Emacs 25.
Can't we use a similar solution in emacs-25, i.e. systems that don't
have sbrk will use their system malloc?  (I understand that on master
we use system malloc for more platforms, but we don't have to do that
on emacs-25.)

Message #23 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Tue, 8 Nov 2016 13:41:40 -0800

[Message part 1 (text/plain, inline)]

Following up on this bug report, the memory-limit function should be 
marked obsolescent and eventually removed, as its value has not been 
useful (or, as far as I can tell, used) for many years. Until it's 
removed it can just return 0. Proposed patch attached.

[0001-Make-memory-limit-obsolete.patch (application/x-patch, attachment)]

Message #26 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Wed, 09 Nov 2016 05:34:25 +0200

> Cc: 24892 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Tue, 8 Nov 2016 13:41:40 -0800
> 
> Following up on this bug report, the memory-limit function should be 
> marked obsolescent and eventually removed, as its value has not been 
> useful (or, as far as I can tell, used) for many years. Until it's 
> removed it can just return 0. Proposed patch attached.

I agree that the current implementation is not useful on many
platforms, but would it make sense to provide a different
implementation, which will be more useful?  For example, something
based on getrusage or perhaps the info in /proc?

Thanks.

Message #29 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: ashish.is <at> lostca.se (Ashish SHUKLA)
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
Subject: Re: {s,}brk removed from FreeBSD 11.x and later, arm64 architecture
Date: Wed, 09 Nov 2016 09:59:12 +0530

[Message part 1 (text/plain, inline)]

On Tue, 8 Nov 2016 09:54:45 -0800, Paul Eggert <eggert <at> cs.ucla.edu> said:
| Thanks for reporting that. Although I don't have easy access to that
| platform, I simulated the problem on FreeBSD 11 x86-64 and came up
| with the attached patch, which I installed into the Emacs master
| branch. Please give it a try if you can, and let me know either way. I
| doubt whether this patch will apply to Emacs 25 and have not
| investigated the details there.

Tried your diff on Emacs 25.1, it segfaulted:

,----[ emacs-25.1
| Dumping under the name emacs
| 82523 pure bytes used
| mv -f emacs bootstrap-emacs
| gmake -C ../lisp compile-first EMACS="../src/bootstrap-emacs"
| gmake[3]: Entering directory '/scratch/ashish/work/emacs-25.1/lisp'
| gmake[3]: Nothing to be done for 'compile-first'.
| gmake[3]: Leaving directory '/scratch/ashish/work/emacs-25.1/lisp'
| gmake -C ../admin/charsets cp51932.el
| gmake[3]: Entering directory '/scratch/ashish/work/emacs-25.1/admin/charsets'
| gmake[3]: Nothing to be done for 'cp51932.el'.
| gmake[3]: Leaving directory '/scratch/ashish/work/emacs-25.1/admin/charsets'
| gmake -C ../admin/charsets eucjp-ms.el
| gmake[3]: Entering directory '/scratch/ashish/work/emacs-25.1/admin/charsets'
| gmake[3]: Nothing to be done for 'eucjp-ms.el'.
| gmake[3]: Leaving directory '/scratch/ashish/work/emacs-25.1/admin/charsets'
| gmake -C ../admin/unidata all EMACS="../../src/bootstrap-emacs"
| gmake[3]: Entering directory '/scratch/ashish/work/emacs-25.1/admin/unidata'
| for f in `sed -n 's/^;; FILE: //p' < ../../lisp/international/charprop.el`; do \
|   [ -f ../../lisp/international/$f ] || exec gmake PHONY_EXTRAS=../../lisp/international/charprop.el ../../lisp/international/charprop.el; \
| done
| gmake[3]: Leaving directory '/scratch/ashish/work/emacs-25.1/admin/unidata'
| /bin/mkdir -p ../etc
| rm -f ../etc/DOC
| ../lib-src/make-docfile -d . \
|   dosfns.o msdos.o xterm.o xfns.o xmenu.o xselect.o xrdb.o xsmfns.o fringe.o image.o fontset.o dbusbind.o cygw32.o nsterm.o nsfns.o nsmenu.o nsselect.o nsimage.o nsfont.o macfont.o w32.o w32console.o w32fns.o w32heap.o w32inevt.o w32notify.o w32menu.o w32proc.o w32reg.o w32select.o w32term.o w32xfns.o w16select.o widget.o xfont.o ftfont.o xftfont.o ftxfont.o gtkutil.o xsettings.o xgselect.o termcap.o dispnew.o frame.o scroll.o xdisp.o menu.o  window.o charset.o coding.o category.o ccl.o character.o chartab.o bidi.o cm.o term.o terminal.o xfaces.o    emacs.o keyboard.o macros.o keymap.o sysdep.o buffer.o filelock.o insdel.o marker.o minibuf.o fileio.o dired.o cmds.o casetab.o casefiddle.o indent.o search.o regex.o undo.o alloc.o data.o doc.o editfns.o callint.o eval.o floatfns.o fns.o font.o print.o lread.o dynlib.o emacs-module.o syntax.o unexelf.o bytecode.o process.o gnutls.o callproc.o region-cache.o sound.o atimer.o doprnt.o intervals.o textprop.o composite.o xml.o kqueue.o  profiler.o decompress.o          > ../etc/DOC
| ../lib-src/make-docfile -a ../etc/DOC -d ../lisp \
|   loaddefs.el loadup.el abbrev.elc bindings.elc buff-menu.elc button.elc case-table.elc composite.elc cus-face.elc cus-start.elc custom.elc disp-table.elc dnd.elc dos-fns.elc dos-vars.elc dos-w32.elc dynamic-setting.elc electric.elc emacs-lisp/backquote.elc emacs-lisp/byte-run.elc emacs-lisp/cl-generic.elc emacs-lisp/cl-preloaded.elc emacs-lisp/eldoc.elc emacs-lisp/float-sup.elc emacs-lisp/lisp-mode.elc emacs-lisp/lisp.elc emacs-lisp/macroexp.elc emacs-lisp/map-ynp.elc emacs-lisp/nadvice.elc emacs-lisp/regexp-opt.elc emacs-lisp/syntax.elc emacs-lisp/tabulated-list.elc emacs-lisp/timer.elc env.elc epa-hook.elc facemenu.elc faces.elc files.elc font-core.elc font-lock.elc format.elc frame.elc fringe.elc help.elc image.elc indent.elc international/characters.elc international/cp51932.elc international/eucjp-ms.elc international/fontset.elc international/mule-cmds.elc international/mule-conf.elc international/mule.elc international/ucs-normalize.elc isearch.elc jit-lock.elc jka-cmpr-hook.elc language/burmese.elc language/cham.elc language/chinese.elc language/cyrillic.elc language/czech.elc language/english.elc language/ethiopic.elc language/european.elc language/georgian.elc language/greek.elc language/hebrew.elc language/indian.elc language/japanese.elc language/khmer.elc language/korean.elc language/lao.elc language/misc-lang.elc language/romanian.elc language/sinhala.elclanguage/slovak.elc language/tai-viet.elc language/thai.elc language/tibetan.elc language/utf-8-lang.elc language/vietnamese.elc ls-lisp.elc menu-bar.elc minibuffer.elc mouse.elc mwheel.elc newcomment.elc progmodes/elisp-mode.elc progmodes/prog-mode.elc register.elc replace.elc rfn-eshadow.elc scroll-bar.elc select.elc simple.elc startup.elc subr.elc term/common-win.elc term/internal.elc term/ns-win.elc term/pc-win.elc term/tty-colors.elc term/w32-win.elc term/x-win.elc textmodes/fill.elc textmodes/page.elc textmodes/paragraphs.elc textmodes/text-mode.elc tool-bar.elc tooltip.elc uniquify.elc vc/ediff-hook.elc vc/vc-hooks.elc version.elc w32-fns.elc w32-vars.elc widget.elc window.elc x-dnd.elc
| gmake -C ../leim leim-list.el EMACS="../src/bootstrap-emacs"
| gmake[3]: Entering directory '/scratch/ashish/work/emacs-25.1/leim'
| rm -f ../lisp/leim/leim-list.el
| EMACSLOADPATH= '../src/bootstrap-emacs' -batch --no-site-file --no-site-lisp -l international/quail \
|   --eval "(update-leim-list-file (unmsys--file-name \"./../lisp/leim\"))"
| Fatal error 11: Segmentation faultgmake[3]: *** [Makefile:134: ../lisp/leim/leim-list.el] Segmentation fault (core dumped)
| gmake[3]: Leaving directory '/scratch/ashish/work/emacs-25.1/leim'
| gmake[2]: *** [Makefile:490: ../lisp/leim/leim-list.el] Error 2
| gmake[2]: Leaving directory '/scratch/ashish/work/emacs-25.1/src'
| gmake[1]: *** [Makefile:398: src] Error 2
| gmake[1]: Leaving directory '/scratch/ashish/work/emacs-25.1'
| *** Error code 1
| 
| % lldb src/bootstrap-emacs -c leim/bootstrap-emacs.core                                                                       
| 
| (lldb) target create "src/bootstrap-emacs" --core "leim/bootstrap-emacs.core"
| Core file '/scratch/ashish/work/emacs-25.1-old/leim/bootstrap-emacs.core' (aarch64) was loaded.
| (lldb) bt all
| bootstrap-emacs was compiled with optimization - stepping may behave oddly; variables may not be available.
| * thread #1: tid = 101155, 0x0000000000513450 bootstrap-emacs`oblookup [inlined] AREF + 4 at lisp.h:1526, name = 'bootstrap-emacs', stop reason = signal SIGSEGV
|   * frame #0: 0x0000000000513450 bootstrap-emacs`oblookup [inlined] AREF + 4 at lisp.h:1526 [opt]
|     frame #1: 0x000000000051344c bootstrap-emacs`oblookup(obarray=<unavailable>, ptr="", size=14, size_byte=14) + 120 at lread.c:3961 [opt]
|     frame #2: 0x000000000051330c bootstrap-emacs`intern_1(str="", len=14) + 96 at lread.c:3779 [opt]
|     frame #3: 0x00000000004bbdec bootstrap-emacs`Fdo_auto_save [inlined] intern(str=<unavailable>) + 208 at lisp.h:3836 [opt]
|     frame #4: 0x00000000004bbde8 bootstrap-emacs`Fdo_auto_save(no_message=<unavailable>, current_only=0) + 204 at fileio.c:5514 [opt]
|     frame #5: 0x0000000000487230 bootstrap-emacs`shut_down_emacs(sig=11, stuff=0) + 216 at emacs.c:2000 [opt]
|     frame #6: 0x0000000000487060 bootstrap-emacs`terminate_due_to_signal(sig=11, backtrace_limit=40) + 112 at emacs.c:365 [opt]
|     frame #7: 0x00000000004a11c8 bootstrap-emacs`deliver_fatal_thread_signal [inlined] handle_fatal_signal(sig=11) + 128 at sysdep.c:1601 [opt]
|     frame #8: 0x00000000004a11bc bootstrap-emacs`deliver_fatal_thread_signal [inlined] deliver_thread_signal + 100 at sysdep.c:1575 [opt]
|     frame #9: 0x00000000004a1158 bootstrap-emacs`deliver_fatal_thread_signal(sig=<unavailable>) + 16 at sysdep.c:1613 [opt]
|     frame #10: 0x00000000004a27e4 bootstrap-emacs`handle_sigsegv(sig=<unavailable>, siginfo=<unavailable>, arg=<unavailable>) + 152 at sysdep.c:1695 [opt]
|     frame #11: 0x000000004065abb0 libthr.so.3`handle_signal(actp=0x00000000008c9a60, sig=11, info=0x00000000008c9ad0, ucp=0x00000000008c9b20) + 256 at thr_sig.c:244 [opt]
|     frame #12: 0x000000004065a318 libthr.so.3`thr_sighandler(sig=11, info=0x00000000008c9ad0, _ucp=0x00000000008c9b20) + 248 at thr_sig.c:189 [opt]
|     frame #13: 0x0000fffffffff000
|     frame #14: 0x0000000000405560 bootstrap-emacs`__start + 360
|     frame #15: 0x0000000040583630 ld-elf.so.1 at rtld_start.S:41
`----

On Emacs master branch, where diff is already present, it segfaults:

,----[ emacs master branch
| Loading /scratch/ashish/work/emacs-master/lisp/mouse.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/select.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/emacs-lisp/timer.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/isearch.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/rfn-eshadow.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/menu-bar.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/emacs-lisp/lisp.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/textmodes/page.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/register.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/textmodes/paragraphs.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/progmodes/prog-mode.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/emacs-lisp/lisp-mode.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/progmodes/elisp-mode.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/textmodes/text-mode.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/textmodes/fill.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/newcomment.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/replace.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/emacs-lisp/tabulated-list.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/buff-menu.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/emacs-lisp/float-sup.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/vc/vc-hooks.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/vc/ediff-hook.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/uniquify.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/electric.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/emacs-lisp/eldoc.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/cus-start.el (source)...
| Loading /scratch/ashish/work/emacs-master/lisp/tooltip.el (source)...
| Finding pointers to doc strings...
| Finding pointers to doc strings...done
| Dumping under the name emacs
| 82347 pure bytes used
| mv -f emacs bootstrap-emacs
| gmake -C ../lisp compile-first EMACS="../src/bootstrap-emacs"
| gmake[3]: Entering directory '/scratch/ashish/work/emacs-master/lisp'
| EMACSLOADPATH= '../src/bootstrap-emacs' -batch --no-site-file --no-site-lisp --eval '(setq load-prefer-newer t)'  -f batch-byte-compile emacs-lisp/macroexp.el
| Fatal error 11: Segmentation faultgmake[3]: *** [Makefile:282: emacs-lisp/macroexp.elc] Segmentation fault (core dumped)
| gmake[3]: Leaving directory '/scratch/ashish/work/emacs-master/lisp'
| gmake[2]: *** [Makefile:757: bootstrap-emacs] Error 2
| gmake[2]: Leaving directory '/scratch/ashish/work/emacs-master/src'
| gmake[1]: *** [Makefile:398: src] Error 2
| gmake[1]: Leaving directory '/scratch/ashish/work/emacs-master'
| 
| % lldb ./src/bootstrap-emacs -c ./lisp/bootstrap-emacs.core
| (lldb) target create "./src/bootstrap-emacs" --core "./lisp/bootstrap-emacs.core"
| Core file '/scratch/ashish/work/emacs-master/./lisp/bootstrap-emacs.core' (aarch64) was loaded.
| (lldb) bt all
| bootstrap-emacs was compiled with optimization - stepping may behave oddly; variables may not be available.
| * thread #1: tid = 101215, 0x0000000000514ea0 bootstrap-emacs`oblookup [inlined] AREF + 4 at lisp.h:1529, name = 'bootstrap-emacs', stop reason = signal SIGSEGV
|   * frame #0: 0x0000000000514ea0 bootstrap-emacs`oblookup [inlined] AREF + 4 at lisp.h:1529 [opt]
|     frame #1: 0x0000000000514e9c bootstrap-emacs`oblookup(obarray=<unavailable>, ptr="", size=14, size_byte=14) + 120 at lread.c:4044 [opt]
|     frame #2: 0x0000000000514d5c bootstrap-emacs`intern_1(str="", len=14) + 96 at lread.c:3862 [opt]
|     frame #3: 0x00000000004bd77c bootstrap-emacs`Fdo_auto_save [inlined] intern(str=<unavailable>) + 208 at lisp.h:3837 [opt]
|     frame #4: 0x00000000004bd778 bootstrap-emacs`Fdo_auto_save(no_message=<unavailable>, current_only=0) + 204 at fileio.c:5538 [opt]
|     frame #5: 0x0000000000488ab4 bootstrap-emacs`shut_down_emacs(sig=11, stuff=0) + 216 at emacs.c:2034 [opt]
|     frame #6: 0x00000000004888e4 bootstrap-emacs`terminate_due_to_signal(sig=11, backtrace_limit=40) + 128 at emacs.c:375 [opt]
|     frame #7: 0x00000000004a334c bootstrap-emacs`deliver_fatal_thread_signal [inlined] handle_fatal_signal(sig=11) + 128 at sysdep.c:1680 [opt]
|     frame #8: 0x00000000004a3340 bootstrap-emacs`deliver_fatal_thread_signal [inlined] deliver_thread_signal + 100 at sysdep.c:1654 [opt]
|     frame #9: 0x00000000004a32dc bootstrap-emacs`deliver_fatal_thread_signal(sig=<unavailable>) + 16 at sysdep.c:1692 [opt]
|     frame #10: 0x00000000004a496c bootstrap-emacs`handle_sigsegv(sig=<unavailable>, siginfo=<unavailable>, arg=<unavailable>) + 168 at sysdep.c:1777 [opt]
|     frame #11: 0x000000004065cbb0 libthr.so.3`handle_signal(actp=0x00000000008f4710, sig=11, info=0x00000000008f4780, ucp=0x00000000008f47d0) + 256 at thr_sig.c:244 [opt]
|     frame #12: 0x000000004065c318 libthr.so.3`thr_sighandler(sig=11, info=0x00000000008f4780, _ucp=0x00000000008f47d0) + 248 at thr_sig.c:189 [opt]
|     frame #13: 0x0000fffffffff000
|     frame #14: 0x00000000004dfa0c bootstrap-emacs`init_alloc [inlined] make_float(float_value=0) + 64 at alloc.c:2649 [opt]
|     frame #15: 0x00000000004df9cc bootstrap-emacs`init_alloc at alloc.c:7304 [opt]
|     frame #16: 0x000000000048929c bootstrap-emacs`main(argc=0, argv=<unavailable>) + 1944 at emacs.c:1214 [opt]
|     frame #17: 0x00000000004054c0 bootstrap-emacs`__start + 360
|     frame #18: 0x0000000040585630 ld-elf.so.1 at rtld_start.S:41
`----

Please let me know if you need more information.

Thanks!
-- 
Ashish SHUKLA

“Living on Earth may be expensive, but it includes an annual free trip
around the Sun.”

Sent from my Emacs

[signature.asc (application/pgp-signature, inline)]

Message #32 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Ashish SHUKLA <ashish.is <at> lostca.se>
Cc: 24892 <at> debbugs.gnu.org
Subject: Re: {s,}brk removed from FreeBSD 11.x and later, arm64 architecture
Date: Wed, 9 Nov 2016 00:26:12 -0800

Ashish SHUKLA wrote:
> Please let me know if you need more information.

Does the latest Emacs master build and run on the last FreeBSD ARM version that 
had sbrk?

If so, what is the difference in value there between sbrk (0) and ((byte *) 
old_bss_addr + old_bss_size), in the region of unexelf.c that computes one or 
the other?

If not, what is the most-recent FreeBSD ARM version that runs Emacs master, and 
what changed after that?

Message #35 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Wed, 9 Nov 2016 00:27:04 -0800

Eli Zaretskii wrote:
> would it make sense to provide a different
> implementation, which will be more useful?

It might if someone were to write it. It would not be likely to have the same 
API, though.

Message #38 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later,
 arm64 architecture
Date: Wed, 09 Nov 2016 10:46:25 +0100

On Nov 09 2016, Paul Eggert <eggert <at> cs.ucla.edu> wrote:

> If so, what is the difference in value there between sbrk (0) and ((byte
> *) old_bss_addr + old_bss_size), in the region of unexelf.c that computes
> one or the other?

The former includes the heap.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Message #41 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: ashish.is <at> lostca.se (Ashish SHUKLA)
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
Subject: Re: {s,}brk removed from FreeBSD 11.x and later, arm64 architecture
Date: Wed, 09 Nov 2016 18:53:43 +0530

[Message part 1 (text/plain, inline)]

On Wed, 9 Nov 2016 00:26:12 -0800, Paul Eggert <eggert <at> cs.ucla.edu> said:
| Ashish SHUKLA wrote:
|| Please let me know if you need more information.

| Does the latest Emacs master build and run on the last FreeBSD ARM
| version that had sbrk?

I believe so, but I am not sure. I'm just an unprivileged user on this ARM64
host, who recently received access to this host.

HTH
-- 
Ashish SHUKLA

“Maxwell's equations have  had a greater impact on  human history than
any ten presidents.”
                                                          ("Carl Sagan")

Sent from my Emacs

[signature.asc (application/pgp-signature, inline)]

Message #44 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Wed, 09 Nov 2016 17:49:19 +0200

> Cc: 24892 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Wed, 9 Nov 2016 00:27:04 -0800
> 
> Eli Zaretskii wrote:
> > would it make sense to provide a different
> > implementation, which will be more useful?
> 
> It might if someone were to write it. It would not be likely to have the same 
> API, though.

I think we could lift the implementation from
system_process_attributes, we report there the process memory size.

Message #47 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andreas Schwab <schwab <at> suse.de>
Cc: eggert <at> cs.ucla.edu, 24892 <at> debbugs.gnu.org, ashish.is <at> lostca.se
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later,
 arm64 architecture
Date: Wed, 09 Nov 2016 17:50:22 +0200

> From: Andreas Schwab <schwab <at> suse.de>
> Date: Wed, 09 Nov 2016 10:46:25 +0100
> Cc: 24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
> 
> On Nov 09 2016, Paul Eggert <eggert <at> cs.ucla.edu> wrote:
> 
> > If so, what is the difference in value there between sbrk (0) and ((byte
> > *) old_bss_addr + old_bss_size), in the region of unexelf.c that computes
> > one or the other?
> 
> The former includes the heap.

Should we perhaps ask the FreeBSD libc maintainers to help us find the
solution?

Message #50 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Andreas Schwab <schwab <at> suse.de>
Cc: 24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Wed, 9 Nov 2016 09:52:06 -0800

On 11/09/2016 01:46 AM, Andreas Schwab wrote:
>> If so, what is the difference in value there between sbrk (0) and ((byte
>> >*) old_bss_addr + old_bss_size), in the region of unexelf.c that computes
>> >one or the other?
> The former includes the heap.

Yes, and I'm curious about what the difference actually is, so that we 
can figure out how to compute it ourselves now that sbrk has been 
withdrawn. On FreeBSD 11.0 x86-64 (which still has sbrk) the difference 
is 0 for Emacs master, but on AMD apparently the value is something 
other than 0.

Message #53 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Richard Stallman <rms <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: eliz <at> gnu.org, 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later,
 arm64 architecture
Date: Wed, 09 Nov 2016 19:22:45 -0500

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

The point of memory-limit was to enable features useful for users.
If memory-limit doesn't work any more, those features don't either.

What exactly were those features?  What problems did they help
users deal with?

Are they completely unnecessary now?  If not, can we make them
function again?  Or provide other features to deal with the same
problems?



-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Message #56 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: rms <at> gnu.org
Cc: eliz <at> gnu.org, 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Wed, 9 Nov 2016 17:40:17 -0800

On 11/09/2016 04:22 PM, Richard Stallman wrote:
> The point of memory-limit was to enable features useful for users.
> If memory-limit doesn't work any more, those features don't either.
>
> What exactly were those features?  What problems did they help
> users deal with?
I assume memory-limit was an attempt to let Lisp code diagnose when 
memory was getting low in Emacs, circa 1992. The memory-limit function 
was added in commit 20d2471455526acfd5fe96681ea31f0eac88fae4 by Jim 
Blandy on 1992-10-03. As far as I can see the function was never 
successful, in the sense that this part of the code was being developed 
at the time and that Emacs eventually used other methods (e.g., calls 
from C to sbrk (0)) to get the information that memory-limit was returning.
> Are they completely unnecessary now?  If not, can we make them
> function again?  Or provide other features to deal with the same
> problems?

This particular feature, whatever it was, seems to have become 
unnecessary in late 1992, soon after it was added.

Message #59 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Wed, 9 Nov 2016 17:47:19 -0800

[Message part 1 (text/plain, inline)]

On 11/09/2016 07:49 AM, Eli Zaretskii wrote:
> we could lift the implementation from
> system_process_attributes, we report there the process memory size.

That number is not that relevant to the intent of memory-limit, and on 
my platform (Fedora 24 x86-64) returning 0 is a better approximation. 
That being said, we can add some help along those lines, in the attached 
patch, slightly modified from the original to suggest (alist-get 'vsize 
(process-attributes (emacs-pid))) for users who prefer the virtual 
memory size.

[0001-Make-memory-limit-obsolete.patch (application/x-patch, attachment)]

Message #62 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later,
 arm64 architecture
Date: Thu, 10 Nov 2016 10:52:00 +0100

On Nov 09 2016, Paul Eggert <eggert <at> cs.ucla.edu> wrote:

> On 11/09/2016 01:46 AM, Andreas Schwab wrote:
>>> If so, what is the difference in value there between sbrk (0) and ((byte
>>> >*) old_bss_addr + old_bss_size), in the region of unexelf.c that computes
>>> >one or the other?
>> The former includes the heap.
>
> Yes, and I'm curious about what the difference actually is,

17851608 bytes.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Message #65 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Thu, 10 Nov 2016 18:13:24 +0200

> Cc: 24892 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Wed, 9 Nov 2016 17:47:19 -0800
> 
> On 11/09/2016 07:49 AM, Eli Zaretskii wrote:
> > we could lift the implementation from
> > system_process_attributes, we report there the process memory size.
> 
> That number is not that relevant to the intent of memory-limit, and on 
> my platform (Fedora 24 x86-64) returning 0 is a better approximation. 

That's strange: how can zero be a useful approximation of the memory
footprint of a running process?  What does memory-limit return on your
system?

> That being said, we can add some help along those lines, in the attached 
> patch, slightly modified from the original to suggest (alist-get 'vsize 
> (process-attributes (emacs-pid))) for users who prefer the virtual 
> memory size.

Thanks, I think we should have a function that does this in, say,
simple.el, under a name such as emacs-memory-size, and point to that
in the obsolescence note.

>  DEFUN ("memory-limit", Fmemory_limit, Smemory_limit, 0, 0, 0,
> -       doc: /* Return the address of the last byte Emacs has allocated, divided by 1024.
> -This may be helpful in debugging Emacs's memory usage.
> -We divide the value by 1024 to make sure it fits in a Lisp integer.  */)
> +       doc: /* Return zero.  */)
>    (void)
>  {
> -  Lisp_Object end;
> -
> -#if defined HAVE_NS || !HAVE_SBRK
> -  /* Avoid warning.  sbrk has no relation to memory allocated anyway.  */
> -  XSETINT (end, 0);
> -#else
> -  XSETINT (end, (intptr_t) (char *) sbrk (0) / 1024);
> -#endif
> -
> -  return end;
> +  return make_number (0);
>  }

That's too drastic, IMO.  We will eventually do that, in time, but
doing that in the same commit that makes the function obsolete is too
soon.

Message #68 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Andreas Schwab <schwab <at> suse.de>
Cc: 24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Thu, 10 Nov 2016 08:23:40 -0800

On 11/10/2016 01:52 AM, Andreas Schwab wrote:
>> I'm curious about what the difference actually is,
> 17851608 bytes.

Thanks, and now I'm curious about where that difference comes from. I 
assume that unexelf.c code like this:

  #ifdef HAVE_SBRK
    new_break = sbrk (0);
  #else
    new_break = (byte *) old_bss_addr + old_bss_size + 17851608;
  #endif

would not be a good idea, even if it happens to work on this particular 
ARM64 build, as the number 17851608 must be specific to the platform 
and/or the build. (The number is 0 on x86-64, for example.) But how can 
I compute the number?

Would it work if we copied the old sbrk.S from the ARM64 C library, 
assemble and linked that, and then just call sbrk? (I don't have easy 
access to this platform so I can't try tricks like this myself.)

Message #71 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Thu, 10 Nov 2016 08:59:11 -0800

[Message part 1 (text/plain, inline)]

On 11/10/2016 08:13 AM, Eli Zaretskii wrote:
> how can zero be a useful approximation of the memory footprint of a 
> running process?

It's not, but memory-limit is not about memory footprint.

> What does memory-limit return on your system?

47668, representing about 47 MiB. In contrast, (alist-get 'vsize 
(process-attributes (emacs-pid))) returns 807344, representing about 788 
MiB. So 0 is numerically closer to the "correct" answer.

> I think we should have a function that does this in, say, simple.el, 
> under a name such as emacs-memory-size, and point to that in the 
> obsolescence note.

Something like that could be done in a later patch. However, the notion 
"memory size" is vague and prone to misinterpretation, so any later 
patch should be done carefully. And I'm leery of defining a function 
that nobody is asking for - if nobody has cared for many years that 
memory-limit has been returning bogus values, then why do we need a 
function at all?

> That's too drastic, IMO. We will eventually do that, in time, but 
> doing that in the same commit that makes the function obsolete is too 
> soon. 

OK, less-drastic patch attached.

[0001-Make-memory-limit-obsolete.patch (application/x-patch, attachment)]

Message #74 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later,
 arm64 architecture
Date: Thu, 10 Nov 2016 18:00:41 +0100

On Nov 10 2016, Paul Eggert <eggert <at> cs.ucla.edu> wrote:

> On 11/10/2016 01:52 AM, Andreas Schwab wrote:
>>> I'm curious about what the difference actually is,
>> 17851608 bytes.
>
> Thanks, and now I'm curious about where that difference comes from. I
> assume that unexelf.c code like this:
>
>   #ifdef HAVE_SBRK
>     new_break = sbrk (0);
>   #else
>     new_break = (byte *) old_bss_addr + old_bss_size + 17851608;
>   #endif
>
> would not be a good idea, even if it happens to work on this particular
> ARM64 build, as the number 17851608 must be specific to the platform
> and/or the build. (The number is 0 on x86-64, for example.) But how can I
> compute the number?

If you don't dump the heap then you miss everything allocated through
lisp_malloc that isn't explicitly copied to pure space.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Message #77 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Andreas Schwab <schwab <at> suse.de>
Cc: 24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Thu, 10 Nov 2016 09:22:16 -0800

On 11/10/2016 09:00 AM, Andreas Schwab wrote:
> If you don't dump the heap then you miss everything allocated through
> lisp_malloc that isn't explicitly copied to pure space.

Sure, but why isn't this a problem in FreeBSD 11.0 x86-64? On that 
platform, sbrk (0) == (byte *) old_bss_addr + old_bss_size when unexec 
is called, so why doesn't unexec miss the same objects on that platform? 
What's different about the ARM64 platform?

Message #80 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Thu, 10 Nov 2016 19:24:31 +0200

> Cc: 24892 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Thu, 10 Nov 2016 08:59:11 -0800
> 
> On 11/10/2016 08:13 AM, Eli Zaretskii wrote:
> > how can zero be a useful approximation of the memory footprint of a 
> > running process?
> 
> It's not, but memory-limit is not about memory footprint.

It is, in a way, although it only reports part of that footprint.

> > What does memory-limit return on your system?
> 
> 47668, representing about 47 MiB. In contrast, (alist-get 'vsize 
> (process-attributes (emacs-pid))) returns 807344, representing about 788 
> MiB. So 0 is numerically closer to the "correct" answer.

"Numerically closer" doesn't mean "useful".  Depending on what the
user wants, either result could be useful, but zero isn't.

> > I think we should have a function that does this in, say, simple.el, 
> > under a name such as emacs-memory-size, and point to that in the 
> > obsolescence note.
> 
> Something like that could be done in a later patch. However, the notion 
> "memory size" is vague and prone to misinterpretation, so any later 
> patch should be done carefully. And I'm leery of defining a function 
> that nobody is asking for - if nobody has cared for many years that 
> memory-limit has been returning bogus values, then why do we need a 
> function at all?

It only returned bogus results on some platforms.

FWIW, I'm frequently interested in the Emacs memory footprint, for
various reasons, so having a function in Emacs would be handy.

> OK, less-drastic patch attached.

Thanks, LGTM for master.

Message #83 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later,
 arm64 architecture
Date: Thu, 10 Nov 2016 18:33:01 +0100

On Nov 10 2016, Paul Eggert <eggert <at> cs.ucla.edu> wrote:

> Sure, but why isn't this a problem in FreeBSD 11.0 x86-64?

Perhaps you were just lucky?

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Message #86 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Andreas Schwab <schwab <at> suse.de>
Cc: 24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Thu, 10 Nov 2016 18:48:50 -0800

On 11/10/2016 09:33 AM, Andreas Schwab wrote:
> Perhaps you were just lucky?

Perhaps. Is there some way to arrange for similar luck on ARM64? I 
assume it's something involving disabling ASLR? (I thought we were 
already doing that, but maybe not....)

Message #89 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: ashish.is <at> lostca.se
Cc: schwab <at> suse.de, Paul Eggert <eggert <at> cs.ucla.edu>, 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Thu, 17 Nov 2016 19:27:05 +0200

Ashish,

As you see, we here lack expertise in FreeBSD/arm internals that's
needed to resolve this issue.  We need help from the FreeBSD/arm
experts.  Could you please ask some FreeBSD developer to chime in and
suggest how to fix the FreeBSD/arm build due to elimination of sbrk?

Emacs 25.2 will begin its pretest shortly, and it would be a shame if
we'd need to declare that Emacs can no longer be built on FreeBSD/arm,
due to this development in FreeBSD libc.

TIA

Message #92 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: ashish.is <at> lostca.se (Ashish SHUKLA)
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: schwab <at> suse.de, Paul Eggert <eggert <at> cs.ucla.edu>, 24892 <at> debbugs.gnu.org,
 ashish.is <at> lostca.se
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later,
 arm64 architecture
Date: Fri, 18 Nov 2016 17:59:44 +0530

[Message part 1 (text/plain, inline)]

On Thu, 17 Nov 2016 19:27:05 +0200, Eli Zaretskii <eliz <at> gnu.org> said:
| Ashish,

| As you see, we here lack expertise in FreeBSD/arm internals that's
| needed to resolve this issue.  We need help from the FreeBSD/arm
| experts.  Could you please ask some FreeBSD developer to chime in and
| suggest how to fix the FreeBSD/arm build due to elimination of sbrk?

| Emacs 25.2 will begin its pretest shortly, and it would be a shame if
| we'd need to declare that Emacs can no longer be built on FreeBSD/arm,
| due to this development in FreeBSD libc.

Hi,

Thanks for your time, looking at the issue. One of the FreeBSD developer has
pointed me to this ongoing discussion[1] on freebsd-hackers@ list.

References:
[1]  https://lists.freebsd.org/pipermail/freebsd-hackers/2016-November/050144.html

HTH
-- 
Ashish SHUKLA

“The probability of  someone watching you is  directly proportional to
the stupidity of your action.”

Sent from my Emacs

[signature.asc (application/pgp-signature, inline)]

Message #95 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Ashish SHUKLA <ashish.is <at> lostca.se>
Cc: freebsd-hackers <at> freebsd.org, 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Fri, 18 Nov 2016 08:21:04 -0800

Ashish SHUKLA wrote:

> One of the FreeBSD developer has
> pointed me to this ongoing discussion[1] on freebsd-hackers@ list.
>
> References:
> [1]  https://lists.freebsd.org/pipermail/freebsd-hackers/2016-November/050144.html

OK, thanks, I'll CC: freebsd-hackers. FreeBSD hackers: this is following up on 
this serious Emacs bug report, which says that Emacs no longer works on FreeBSD 
11.x arm64:

https://bugs.gnu.org/24892

The referenced FreeBSD discussion appears to be under the misapprehension that 
sbrk is useful only to estimate how much memory is in use, and since this use 
has long been wrong sbrk should be unceremoniously removed. Unfortunately, this 
assumption is incorrect for Emacs, as (bleeding-edge) Emacs uses its own 
allocator during its build process, and reverts to malloc only during user 
operation. (The build-time Emacs makes a copy of itself in a new executable, and 
relies on sbrk during the build.) Obviously this is dicey and we are planning to 
change Emacs to make it more portable; however, this will require nontrivial 
reengineering on our part.

In the meantime we respectfully request that sbrk functionality be kept in 
FreeBSD 11.x arm64 for a while. It is fine to mark it as deprecated or obsolete, 
or even rename it, but please do not remove the functionality entirely.

Message #98 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Ed Maste <emaste <at> freebsd.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: "freebsd-hackers <at> freebsd.org" <freebsd-hackers <at> freebsd.org>,
 24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later,
 arm64 architecture
Date: Fri, 18 Nov 2016 15:20:26 -0500

On 18 November 2016 at 11:21, Paul Eggert <eggert <at> cs.ucla.edu> wrote:
>
> In the meantime we respectfully request that sbrk functionality be kept in
> FreeBSD 11.x arm64 for a while. It is fine to mark it as deprecated or
> obsolete, or even rename it, but please do not remove the functionality
> entirely.

Hi Paul,

I want to clarify one point: arm64 support was first available in a
release in FreeBSD 11.0, without sbrk, and sbrk never existed on the
stable/11 branch.

It's important for us that emacs works on FreeBSD (including
FreeBSD/arm64) and there are folks willing to help make that happen. I
had a quick look at emacs' source, and it seems there's an
implementation that allocates memory out of a large array in .bss used
for some platforms - could we make use of that here?

Message #101 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Brooks Davis <brooks <at> freebsd.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: freebsd-hackers <at> freebsd.org, 24892 <at> debbugs.gnu.org,
 Ashish SHUKLA <ashish.is <at> lostca.se>
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Fri, 18 Nov 2016 20:23:28 +0000

[Message part 1 (text/plain, inline)]

On Fri, Nov 18, 2016 at 08:21:04AM -0800, Paul Eggert wrote:
> Ashish SHUKLA wrote:
> 
> > One of the FreeBSD developer has
> > pointed me to this ongoing discussion[1] on freebsd-hackers@ list.
> >
> > References:
> > [1]  https://lists.freebsd.org/pipermail/freebsd-hackers/2016-November/050144.html
> 
> OK, thanks, I'll CC: freebsd-hackers. FreeBSD hackers: this is following up on 
> this serious Emacs bug report, which says that Emacs no longer works on FreeBSD 
> 11.x arm64:
> 
> https://bugs.gnu.org/24892
> 
> The referenced FreeBSD discussion appears to be under the misapprehension that 
> sbrk is useful only to estimate how much memory is in use, and since this use 
> has long been wrong sbrk should be unceremoniously removed. Unfortunately, this 
> assumption is incorrect for Emacs, as (bleeding-edge) Emacs uses its own 
> allocator during its build process, and reverts to malloc only during user 
> operation. (The build-time Emacs makes a copy of itself in a new executable, and 
> relies on sbrk during the build.) Obviously this is dicey and we are planning to 
> change Emacs to make it more portable; however, this will require nontrivial 
> reengineering on our part.
> 
> In the meantime we respectfully request that sbrk functionality be kept in 
> FreeBSD 11.x arm64 for a while. It is fine to mark it as deprecated or obsolete, 
> or even rename it, but please do not remove the functionality entirely.

Under FreeBSD's stability rules if we shipped ANY release for a platforms
(e.g. arm64, riscv) with a system call, we must support it for the life
of the architecture (approximately FOREVER).  As such, if any other
solution is viable, we'd prefer that.

What does emacs actually need from sbrk()?  Could it get by with
something with the same interface allocating from .bss or does it need
its allocations to be at the end of .bss?  I'm happy to put some time
into a userspace workaround if one is viable.

-- Brooks

[signature.asc (application/pgp-signature, inline)]

Message #104 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Ed Maste <emaste <at> freebsd.org>
Cc: freebsd-hackers <at> freebsd.org, Brooks Davis <brooks <at> freebsd.org>,
 24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later, arm64
 architecture
Date: Fri, 18 Nov 2016 14:22:07 -0800

Ed Maste wrote:
> arm64 support was first available in a release in FreeBSD 11.0, without sbrk, and sbrk never existed on the stable/11 branch.

Thanks, I didn't know that. So Emacs has never worked in this environment.

> it seems there's an implementation that allocates memory out of a large array in .bss used for some platforms - could we make use of that here?

Quite possibly. Unfortunately the code that uses that array (in gmalloc.c) also 
uses sbrk to move the break past the end of the large array. The large array is 
intended for use only during the build process; during ordinary execution, sbrk 
is used.

Perhaps Emacs could work around the problem by supplying a user-space sbrk 
emulator, which uses mmap to support the old-fashioned model where the heap is 
contiguous. Is that something that could be done easily? That is, is there some 
way to reserve the address space for a potentially-large Emacs heap right after 
.bss, such that library calls to malloc and mmap won't use this address space? 
That might do the trick.

Brooks Davis wrote:
> What does emacs actually need from sbrk()?  Could it get by with something with the same interface allocating from .bss or does it need its allocations to be at the end of .bss?

I think more the latter. The problem with a fixed-size .bss is that whatever 
size we pick during the build is likely to be wrong for users. Also, Emacs saves 
the entire build-time .bss into an executable, so an enormous .bss will be 
counterproductive.

Message #107 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: freebsd-hackers <at> freebsd.org, brooks <at> freebsd.org, emaste <at> freebsd.org,
 ashish.is <at> lostca.se, 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later,
 arm64 architecture
Date: Sat, 19 Nov 2016 08:57:44 +0200

> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Fri, 18 Nov 2016 14:22:07 -0800
> Cc: freebsd-hackers <at> freebsd.org, Brooks Davis <brooks <at> freebsd.org>,
> 	24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
> 
> > it seems there's an implementation that allocates memory out of a large array in .bss used for some platforms - could we make use of that here?
> 
> Quite possibly. Unfortunately the code that uses that array (in gmalloc.c) also 
> uses sbrk to move the break past the end of the large array. The large array is 
> intended for use only during the build process; during ordinary execution, sbrk 
> is used.
> 
> Perhaps Emacs could work around the problem by supplying a user-space sbrk 
> emulator, which uses mmap to support the old-fashioned model where the heap is 
> contiguous. Is that something that could be done easily? That is, is there some 
> way to reserve the address space for a potentially-large Emacs heap right after 
> .bss, such that library calls to malloc and mmap won't use this address space? 
> That might do the trick.
> 
> Brooks Davis wrote:
> > What does emacs actually need from sbrk()?  Could it get by with something with the same interface allocating from .bss or does it need its allocations to be at the end of .bss?
> 
> I think more the latter. The problem with a fixed-size .bss is that whatever 
> size we pick during the build is likely to be wrong for users. Also, Emacs saves 
> the entire build-time .bss into an executable, so an enormous .bss will be 
> counterproductive.

I agree with everything Paul wrote.

There's one other factor you should be aware of: Emacs 25.2 will most
probably start its pretest in a week or so.  If this problem can be
solved for FreeBSD on arm64 by providing a drop-in replacement for
sbrk, we could include that in 25.2, which is a strictly bug-fix
release that can only accept safe bug fixes.  If not, the solution
will have to wait for Emacs 26.1, currently being developed on the
master branch, which is unlikely to be released before mid-2017 or
even the end of 2017.  Not a catastrophe, given that Emacs never
worked on this host, but something to consider.

(It is, of course, possible to provide a simple and safe solution for
25.2 and a more elaborate one for 26.1.)

Thanks.

Message #114 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later,
 arm64 architecture
Date: Sat, 19 Nov 2016 09:20:50 +0200

unblock 21966 by 24892
block 24655 by 24892
thanks

> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Fri, 18 Nov 2016 14:22:07 -0800
> Cc: freebsd-hackers <at> freebsd.org, Brooks Davis <brooks <at> freebsd.org>,
> 	24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
> 
> Ed Maste wrote:
> > arm64 support was first available in a release in FreeBSD 11.0, without sbrk, and sbrk never existed on the stable/11 branch.
> 
> Thanks, I didn't know that. So Emacs has never worked in this environment.

Since Emacs never worked on FreeBSD/arm64, we don't need to block the
release of 25.2 on this issue (although it would be nice to have it
resolved by 25.2).

Message #125 received at 24892 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: eggert <at> cs.ucla.edu
Cc: 24892 <at> debbugs.gnu.org
Subject: Re: bug#24892: {s, }brk removed from FreeBSD 11.x and later,
 arm64 architecture
Date: Sat, 02 Sep 2017 16:37:31 +0300

unblock 24655 by 24892
thanks

> Date: Sat, 19 Nov 2016 09:20:50 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 24892 <at> debbugs.gnu.org
> 
> unblock 21966 by 24892
> block 24655 by 24892
> thanks
> 
> > From: Paul Eggert <eggert <at> cs.ucla.edu>
> > Date: Fri, 18 Nov 2016 14:22:07 -0800
> > Cc: freebsd-hackers <at> freebsd.org, Brooks Davis <brooks <at> freebsd.org>,
> > 	24892 <at> debbugs.gnu.org, Ashish SHUKLA <ashish.is <at> lostca.se>
> > 
> > Ed Maste wrote:
> > > arm64 support was first available in a release in FreeBSD 11.0, without sbrk, and sbrk never existed on the stable/11 branch.
> > 
> > Thanks, I didn't know that. So Emacs has never worked in this environment.
> 
> Since Emacs never worked on FreeBSD/arm64, we don't need to block the
> release of 25.2 on this issue (although it would be nice to have it
> resolved by 25.2).

And because we've heard nothing since Nov 2016, it looks like this
issue shouldn't block Emacs 26.1, either.

Thanks.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.