GNU bug report logs - #13342
Clang, the FFI, and 8-bit signed integers

Previous Next

Package: guile;

Reported by: Peter Teeson <pteeson <at> me.com>

Date: Wed, 2 Jan 2013 23:06:02 UTC

Severity: normal

Tags: notabug

Merged with 13386

Done: ludo <at> gnu.org (Ludovic Courtès)

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 13342 in the body.
You can then email your comments to 13342 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Wed, 02 Jan 2013 23:06:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Peter Teeson <pteeson <at> me.com>:
New bug report received and forwarded. Copy sent to bug-guile <at> gnu.org. (Wed, 02 Jan 2013 23:06:03 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Peter Teeson <pteeson <at> me.com>
To: bug-guile <at> gnu.org
Cc: Peter Teeson <pteeson <at> me.com>
Subject: Errors trying to build Guile 2.0.7
Date: Wed, 02 Jan 2013 15:39:10 -0500
[Message part 1 (text/plain, inline)]
My environment is Mac Pro 4,1 Intel, MacOS 10.7.5 Lion
All required Guile libs are newly installed in /usr/local/

I get the following 2 errors, (A & B below) trying to build Guile.

(A) I solved this one by installing GNU readline-6.2.
Prior to that it must have used Apple's deadline which didn't have the rl_get_keymap_name
function.

You might wish to make a test for that function in configure and maybe make GNU readline a required lib?

-------- Tail Log for Make -----
…...
make  all-am
  CC       readline.lo
readline.c: In function 'init_bouncing_parens':
readline.c:445: warning: implicit declaration of function 'rl_get_keymap_name'
readline.c:445: warning: passing argument 1 of 'strncmp' makes pointer from integer without a cast
  CCLD     libguilereadline-v-18.la
Undefined symbols for architecture x86_64:
  "_rl_get_keymap_name", referenced from:
      _scm_init_readline in readline.o
ld: symbol(s) not found for architecture x86_64
collect2: ld returned 1 exit status
make[3]: *** [libguilereadline-v-18.la] Error 1
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
Gandalf:guile-2.0.7 pteeson$

----------------- readline.c:445 --------
  if (strncmp (rl_get_keymap_name (rl_get_keymap ()), "vi", 2))
=========================================================


(B) I ran Make Check after Make as I always do and this error is reported
--------------- Make Check --------------------
…..
PASS: test-asmobs
bad return from expression `(f-sum -1 2000 -30000 40000000000)': expected 39999971999; got 39999972255
FAIL: test-ffi
PASS: test-list
….
PASS: test-pthread-create-secondary
==================================
1 of 29 tests failed
Please report to bug-guile <at> gnu.org
==================================
make[5]: *** [check-TESTS] Error 1
make[4]: *** [check-am] Error 2
make[3]: *** [check] Error 2
make[2]: *** [check-recursive] Error 1
make[1]: *** [check-recursive] Error 1
make: *** [check] Error 2



[Message part 2 (text/html, inline)]

Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Thu, 03 Jan 2013 15:23:02 GMT) Full text and rfc822 format available.

Message #8 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Peter Teeson <pteeson <at> me.com>
Cc: 13342 <at> debbugs.gnu.org
Subject: Re: bug#13342: Errors trying to build Guile 2.0.7
Date: Thu, 03 Jan 2013 16:22:24 +0100
Hi!

Peter Teeson <pteeson <at> me.com> skribis:

> I get the following 2 errors, (A & B below) trying to build Guile.
>
> (A) I solved this one by installing GNU readline-6.2.
> Prior to that it must have used Apple's deadline which didn't have the rl_get_keymap_name
> function.

Yes, that’s because you were using the Readline “compatibility” layer of
libeditline, which is shipped as part of MacOS.

> You might wish to make a test for that function in configure and maybe make GNU readline a required lib?

There are tests checking for GNU Readline.  The problem is that
libeditline attempts to replicate Readline’s API, but not quite, hence
the problem.  That’s why ‘README’ insists on using GNU Readline.

> bad return from expression `(f-sum -1 2000 -30000 40000000000)': expected 39999971999; got 39999972255
> FAIL: test-ffi

This is a known issue when building Guile with LLVM/Clang:

  http://bugs.gnu.org/10015
  http://bugs.gnu.org/10681

It would be great if you could investigate.

Thanks,
Ludo’.




Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Fri, 04 Jan 2013 02:22:02 GMT) Full text and rfc822 format available.

Message #11 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: Peter Teeson <pteeson <at> me.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 13342 <at> debbugs.gnu.org
Subject: Re: bug#13342: Errors trying to build Guile 2.0.7
Date: Thu, 03 Jan 2013 21:21:00 -0500
Hi Ludo':
Thanks for your reply and your explanation.

On 2013-01-03, at 10:22 AM, Ludovic Courtès wrote:
>> (A) I solved this one by installing GNU readline-6.2.
>> Prior to that it must have used Apple's deadline which didn't have the rl_get_keymap_name
>> function.
> 
> Yes, that’s because you were using the Readline “compatibility” layer of
> libeditline, which is shipped as part of MacOS.

>> You might wish to make a test for that function in configure and maybe make GNU readline a required lib?
> 
> There are tests checking for GNU Readline.  The problem is that
> libeditline attempts to replicate Readline’s API, but not quite, hence
> the problem.  That’s why ‘README’ insists on using GNU Readline.
Before I did/do anything I always read the README ;}
Somehow I missed the README insisting - I still don't see it there;}
There is a cryptic statement on page 1
"It will also use the libreadline library if it is available." 

and buried later on at line 413 (using BBEdit)
"guile-readline:
      The glue code for using GNU readline with Guile.  This
      will be build when configure can find a recent enough readline
      library on your system."

I agree there should be something mentioning the Mac situation and insisting on on GNU Readline. Where do you think it should be explicitly mentioned? In the Special Instructions For Some Systems section?

>> bad return from expression `(f-sum -1 2000 -30000 40000000000)': expected 39999971999; got 39999972255
>> FAIL: test-ffi
> 
> This is a known issue when building Guile with LLVM/Clang:
> 
> http://bugs.gnu.org/10015
> http://bugs.gnu.org/10681
> 
> It would be great if you could investigate.

I'd be glad to try and help but please give me pointers on how to approach this.
This will be a learning exercise for me as I am familiar with Xcode and it's debugger (GDB and LLDB) but have only used it developing Cocoa apps.

Care to suggest how I should approach things?

thanks and respect….

Peter






Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Fri, 04 Jan 2013 02:54:01 GMT) Full text and rfc822 format available.

Message #14 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: Peter Teeson <peter.teeson <at> me.com>
To: Peter Teeson <pteeson <at> me.com>
Cc: 13342 <at> debbugs.gnu.org
Subject: Re: bug#13342: Errors trying to build Guile 2.0.7
Date: Thu, 03 Jan 2013 21:18:47 -0500
Hi Ludo':
Thanks for your reply and your explanation.

On 2013-01-03, at 10:22 AM, Ludovic Courtès wrote:
>> (A) I solved this one by installing GNU readline-6.2.
>> Prior to that it must have used Apple's deadline which didn't have the rl_get_keymap_name
>> function.
> 
> Yes, that’s because you were using the Readline “compatibility” layer of
> libeditline, which is shipped as part of MacOS.

>> You might wish to make a test for that function in configure and maybe make GNU readline a required lib?
> 
> There are tests checking for GNU Readline.  The problem is that
> libeditline attempts to replicate Readline’s API, but not quite, hence
> the problem.  That’s why ‘README’ insists on using GNU Readline.
Before I did/do anything I always read the README ;}
Somehow I missed the README insisting - I still don't see it there;}
There is a cryptic statement on page 1
"It will also use the libreadline library if it is available." 

and buried later on at line 413 (using BBEdit)
"guile-readline:
       The glue code for using GNU readline with Guile.  This
       will be build when configure can find a recent enough readline
       library on your system."

I agree there should be something mentioning the Mac situation and insisting on on GNU Readline. Where do you think it should be explicitly mentioned? In the Special Instructions For Some Systems section?

>> bad return from expression `(f-sum -1 2000 -30000 40000000000)': expected 39999971999; got 39999972255
>> FAIL: test-ffi
> 
> This is a known issue when building Guile with LLVM/Clang:
> 
> http://bugs.gnu.org/10015
> http://bugs.gnu.org/10681
> 
> It would be great if you could investigate.

I'd be glad to try and help but please give me pointers on how to approach this.
This will be a learning exercise for me as I am familiar with Xcode and it's debugger (GDB and LLDB) but have only used it developing Cocoa apps.

Care to suggest how I should approach things?

thanks and respect….

Peter





Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Fri, 04 Jan 2013 17:24:01 GMT) Full text and rfc822 format available.

Message #17 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Peter Teeson <pteeson <at> me.com>
Cc: 13342 <at> debbugs.gnu.org
Subject: Re: bug#13342: Errors trying to build Guile 2.0.7
Date: Fri, 04 Jan 2013 18:23:34 +0100
Hi,

Peter Teeson <pteeson <at> me.com> skribis:

> On 2013-01-03, at 10:22 AM, Ludovic Courtès wrote:
>>> (A) I solved this one by installing GNU readline-6.2.
>>> Prior to that it must have used Apple's deadline which didn't have the rl_get_keymap_name
>>> function.
>> 
>> Yes, that’s because you were using the Readline “compatibility” layer of
>> libeditline, which is shipped as part of MacOS.
>
>>> You might wish to make a test for that function in configure and maybe make GNU readline a required lib?
>> 
>> There are tests checking for GNU Readline.  The problem is that
>> libeditline attempts to replicate Readline’s API, but not quite, hence
>> the problem.  That’s why ‘README’ insists on using GNU Readline.
> Before I did/do anything I always read the README ;}
> Somehow I missed the README insisting - I still don't see it there;}
> There is a cryptic statement on page 1
> "It will also use the libreadline library if it is available." 
>
> and buried later on at line 413 (using BBEdit)
> "guile-readline:
>       The glue code for using GNU readline with Guile.  This
>       will be build when configure can find a recent enough readline
>       library on your system."
>
> I agree there should be something mentioning the Mac situation and insisting on on GNU Readline. Where do you think it should be explicitly mentioned? In the Special Instructions For Some Systems section?

Well, you’re right that it doesn’t really insist.  ;-)  It does mention
GNU Readline though, which is not what was being used here.

I’m reluctant to adding anything special about libeditline, since
they’re really failing to be compatible with Readline here, and it’s
something to be fixed in libeditline.

>>> bad return from expression `(f-sum -1 2000 -30000 40000000000)': expected 39999971999; got 39999972255
>>> FAIL: test-ffi
>> 
>> This is a known issue when building Guile with LLVM/Clang:
>> 
>> http://bugs.gnu.org/10015
>> http://bugs.gnu.org/10681
>> 
>> It would be great if you could investigate.
>
> I'd be glad to try and help but please give me pointers on how to approach this.

I’d first add printfs in test_ffi_sum (in test-ffi-lib.c) to see if it
gets the right arguments.  Depending on the result, I’d try to call
test_ffi_sum from C with the same arguments to see if it’s an
integer-wrapping issue.

I’ll try a Clang build when time permits to see what happens.

Thanks,
Ludo’.




Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Tue, 08 Jan 2013 16:10:03 GMT) Full text and rfc822 format available.

Message #20 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Peter Teeson <peter.teeson <at> me.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: bug-guile <at> gnu.org
Subject: Re: [PARTIALLY SOLVED] bug#13342: Errors trying to build Guile 2.0.7
Date: Tue, 08 Jan 2013 11:08:43 -0500
[Message part 1 (text/plain, inline)]
Hi Ludo:
I think the reason that we get this error is that the "a" argument is declared to be only 8 bits wide
but is passed in as -1 which is correctly represented as 0xFFFFFFFF

However an 8-bit representation is 0xFF which is only 255.

Here is how I tracked it down:
(0) Took the printf's from my test program and placed them in 

scm_t_int64 test_ffi_sum (scm_t_int8 a, scm_t_int16 b,
                          scm_t_int32 c, scm_t_int64 d);
scm_t_int64 test_ffi_sum (scm_t_int8 a, scm_t_int16 b,
                          scm_t_int32 c, scm_t_int64 d)
{
    scm_t_int64 sum;
    printf("scm_t_int64 d %" "ll" "d" " %#llX \n", d,d);
    printf("scm_t_int32 c %" "d" " %#X \n", c,c);
    printf("scm_t_int16 b %" "hd" " %#X \n", b,b);
    printf("scm_t_int8 a %" "hh" "d" " %#X \n", a,a);
    sum = d + c + b + a;
    printf("scm_t_int64 sum %" "ll" "d" " %#llX \n", sum,sum);

    a = -1; //  NOTE:re-assign of a!!!!
    printf("scm_t_int16 a %" "hd" " %#X \n", a,a);

    return d + c + b + a;
}

(1) Did make and make check with the following result

scm_t_int64 d 40000000000 0X9502F9000 
scm_t_int32 c -30000 0XFFFF8AD0 
scm_t_int16 b 2000 0X7D0 
scm_t_int8 a -1 0XFF 
scm_t_int64 sum 39999972255 0X9502F239F 
scm_t_int16 a -1 0XFFFFFFFF 
PASS: test-ffi


(2) Observe that the test passed. 
So I think my original suspicion seems to have been confirmed.

I do not know where to go from here but await your further instructions.

respect

Peter
[Message part 2 (text/html, inline)]

Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Tue, 08 Jan 2013 16:32:01 GMT) Full text and rfc822 format available.

Message #23 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Peter Teeson <peter.teeson <at> me.com>
To: bug-guile <at> gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: bug#13342: Errors trying to build Guile 2.0.7 [PARTIALLY SOLVED]
Date: Tue, 08 Jan 2013 11:30:36 -0500
[Message part 1 (text/plain, inline)]
Hi Ludo:

I believe my conjecture that -1 was correct based on the following:

(0) I took the printf statements from my test program and placed them in 
test-ffi-sum:

scm_t_int64 test_ffi_sum (scm_t_int8 a, scm_t_int16 b,
                          scm_t_int32 c, scm_t_int64 d);
scm_t_int64 test_ffi_sum (scm_t_int8 a, scm_t_int16 b,
                          scm_t_int32 c, scm_t_int64 d)
{
    scm_t_int64 sum;
    printf("scm_t_int64 d %" "ll" "d" " %#llX \n", d,d);
    printf("scm_t_int32 c %" "d" " %#X \n", c,c);
    printf("scm_t_int16 b %" "hd" " %#X \n", b,b);
    printf("scm_t_int8 a %" "hh" "d" " %#X \n", a,a);

    sum = d + c + b + a;
    printf("scm_t_int64 sum %" "ll" "d" " %#llX \n", sum,sum);

    a = -1; // NOTE re-assinging of a!
    printf("scm_t_int16 a %" "hd" " %#X \n", a,a);

    return d + c + b + a;
}

(1) Ran make 

Making all in standalone
make  all-am
  CC       libtest_ffi_la-test-ffi-lib.lo
  CCLD     libtest-ffi.la

and then make check

scm_t_int64 d 40000000000 0X9502F9000 
scm_t_int32 c -30000 0XFFFF8AD0 
scm_t_int16 b 2000 0X7D0 
scm_t_int8 a -1 0XFF 
scm_t_int64 sum 39999972255 0X9502F239F 
scm_t_int16 a -1 0XFFFFFFFF 
PASS: test-ffi

(2) Since scm_t_int8 is only 8 bits wide it's maximum value is 0xFF <=> 255
Reassigning "a" to be -1 allowed it to pass the test.

(3) Therefore I conclude that this is why the test fails (correctly IMHO).
You can only get a litre of milk into a litre jar!

(4) Where do we go from here?

respect

Peter
[Message part 2 (text/html, inline)]

Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Sun, 27 Jan 2013 14:12:02 GMT) Full text and rfc822 format available.

Message #26 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Peter Teeson <peter.teeson <at> me.com>
Cc: 13342 <at> debbugs.gnu.org
Subject: Re: [PARTIALLY SOLVED] bug#13342: Errors trying to build Guile 2.0.7
Date: Sun, 27 Jan 2013 15:11:32 +0100
Hi Peter,

Sorry for the late reply, and thanks for investigating.

I reproduced the bug on GNU/Linux with Clang 3.1.

Peter Teeson <peter.teeson <at> me.com> skribis:

> I think the reason that we get this error is that the "a" argument is declared to be only 8 bits wide
> but is passed in as -1 which is correctly represented as 0xFFFFFFFF
>
> However an 8-bit representation is 0xFF which is only 255.

This is part of the issue.

Consider this example:

--8<---------------cut here---------------start------------->8---
#include <stdint.h>

int64_t
test_sum (int8_t a, int64_t b)
{
  return a + b;
}
--8<---------------cut here---------------end--------------->8---

When compiled with GCC 4.6, the assembly is:

--8<---------------cut here---------------start------------->8---
test_sum:
.LFB0:
	.cfi_startproc
	movsbq	%dil, %rdi
	leaq	(%rdi,%rsi), %rax
	ret
	.cfi_endproc
--8<---------------cut here---------------end--------------->8---

With Clang 3.1, it is:

--8<---------------cut here---------------start------------->8---
test_sum:                               # @test_sum
	.cfi_startproc
# BB#0:
	movslq	%edi, %rax
	addq	%rsi, %rax
	ret
--8<---------------cut here---------------end--------------->8---

The ‘movsbq’ emitted by GCC arranges to keep only the 8 LSBs.  Clang
does no such thing, thus keeping all the bits of the first operand in
the addition.

I looked at Section 3.2.3 (“Parameter Passing”) of the SysV ABI x86_64
PS but couldn’t find any evidence as to what the correct behavior is.

However, on the caller side, both compilers emit the same code.  This
program:

--8<---------------cut here---------------start------------->8---
#include <stdint.h>

extern int64_t test_sum (int8_t a, int64_t b);

int64_t
foo (void)
{
  return test_sum (-1, 123132);
}
--8<---------------cut here---------------end--------------->8---

leads to the following assembly with both compilers:

--8<---------------cut here---------------start------------->8---
foo:                                    # @foo
	.cfi_startproc
	movl	$-1, %edi
	movl	$123132, %esi           # imm = 0x1E0FC
	jmp	test_sum                # TAILCALL
--8<---------------cut here---------------end--------------->8---

(And as we’ve seen, libffi does the same.)

So that seems to indicate that the callee code generated by Clang is
incorrect as it fails to discard the 24 MSBs of its first argument.

Could you report it as a Clang/LLVM bug (keeping 13342 <at> debbugs.gnu.org
Cc’d)?

Thanks!

Ludo’.




Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Sun, 27 Jan 2013 21:36:02 GMT) Full text and rfc822 format available.

Message #29 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: Peter Teeson <peter.teeson <at> me.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 13342 <at> debbugs.gnu.org, guile-devel <at> gnu.org
Subject: Re: [PARTIALLY SOLVED] bug#13342: Errors trying to build Guile 2.0.7
Date: Sun, 27 Jan 2013 16:35:04 -0500
[Message part 1 (text/plain, inline)]
Hi Ludo:
MacOS 10.7.5 Intel Dual CPU quad cores 

Many thanks for your reply. A nice demo but, no disrespect, maybe not really the issue.
Looking at the generated code from each compiler was going to be my next step after
taking a look at the stack and regs to see what got passed/pushed/popped in the call to test_sum.
Which is why I posted about how can I debug/step through the interpreter. 
I still don't understand how to do that even though I read through the manual.

My experiment in building guile with both compilers indicate they both produce the same error.
bad return from expression `(f-sum -1 2000 -30000 40000000000)': expected 39999971999; got 39999972255
I am still of the opinion that it's an interpreter issue and not a compiler one. 

I ask for help in confirming my understanding of this scheme code from the file 
/test-suite/standalone/test-ffi
;;
;; Multiple int args of differing types
;;
(define f-sum
  (pointer->procedure int64 (dynamic-func "test_ffi_sum" lib)
                      (list int8 int16 int32 int64)))
(test (f-sum -1 2000 -30000 40000000000)  ;; This calls the c function f-sum
      (+ -1 2000 -30000 40000000000))	     ;;  This executes the scheme interpreter to do the sum

Is my understanding correct??????

If so please explain to me how the interpreter knows that the -1 in this expression is an int8?
My suspicion is that it thinks it's an int16 and that's why we see the difference!!

I don't know how to pass an int8 of -1 to the interpreter in REPL mode.
Otherwise I'd try that.

============= from Terminal REPL session =============
Gandalf:guile-2.0.7 pteeson$ guile
GNU Guile 2.0.7
<snipped comments>

scheme@(guile-user)> (+ -1 2000 -30000 40000000000)
$1 = 39999971999
scheme@(guile-user)> ,q

which matches the fail message value.
bad return from expression `(f-sum -1 2000 -30000 40000000000)': expected 39999971999; got 39999972255

================= from http://pubs.opengroup.org/onlinepubs/009695399/basedefs/stdint.h.html =========== 
Limits of exact-width integer types
Minimum values of exact-width signed integer types:
{INTN_MIN}
	Exactly -(2 N-1)
Maximum values of exact-width signed integer types:
{INTN_MAX}
	Exactly 2N-1 -1
==========================================


[Message part 2 (text/html, inline)]

Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Mon, 28 Jan 2013 01:48:01 GMT) Full text and rfc822 format available.

Message #32 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 13342 <at> debbugs.gnu.org, Peter Teeson <peter.teeson <at> me.com>
Subject: Re: bug#13342: [PARTIALLY SOLVED] bug#13342: Errors trying to build
	Guile 2.0.7
Date: Sun, 27 Jan 2013 20:46:22 -0500
Hi Ludovic,

Thanks for looking into this!  I think I understand the problem now.

ludo <at> gnu.org (Ludovic Courtès) writes:
> Consider this example:
>
> #include <stdint.h>
>
> int64_t
> test_sum (int8_t a, int64_t b)
> {
>   return a + b;
> }
>
> When compiled with GCC 4.6, the assembly is:
>
> test_sum:
> .LFB0:
> 	.cfi_startproc
> 	movsbq	%dil, %rdi
> 	leaq	(%rdi,%rsi), %rax
> 	ret
> 	.cfi_endproc
>
> With Clang 3.1, it is:
>
> test_sum:                               # @test_sum
> 	.cfi_startproc
> # BB#0:
> 	movslq	%edi, %rax
> 	addq	%rsi, %rax
> 	ret
>
> The ‘movsbq’ emitted by GCC arranges to keep only the 8 LSBs.  Clang
> does no such thing, thus keeping all the bits of the first operand in
> the addition.

This is the key revelation, although I've reached a different conclusion
about where the bug is.

> I looked at Section 3.2.3 (“Parameter Passing”) of the SysV ABI x86_64
> PS but couldn’t find any evidence as to what the correct behavior is.

I read the same section, and although it is not as clear as I'd prefer,
my interpretation is that the caller is responsible for sign-extending
signed chars to ints.  This is also consistent with something I vaguely
remember reading in K&R long ago, namely that 'char' and 'short'
arguments are coerced to 'int' before making a function call.

Clang strictly requires callers to sign-extend, whereas GCC is tolerant
of callers who fail to do so.  IMO, both behaviors are permitted by the
ABI.

The problem is that libffi does *not* sign-extend arguments passed in
registers when making calls, which is IMO a bug that has gone (mostly)
unnoticed because of the tolerance and ubiquity of GCC.

> However, on the caller side, both compilers emit the same code.  This
> program:
>
> #include <stdint.h>
>
> extern int64_t test_sum (int8_t a, int64_t b);
>
> int64_t
> foo (void)
> {
>   return test_sum (-1, 123132);
> }
>
> leads to the following assembly with both compilers:
>
> foo:                                    # @foo
> 	.cfi_startproc
> 	movl	$-1, %edi
> 	movl	$123132, %esi           # imm = 0x1E0FC
> 	jmp	test_sum                # TAILCALL
>
> (And as we’ve seen, libffi does the same.)

No, libffi does *not* do the same.  Take a look at the relevant code:

  https://github.com/atgreen/libffi/blob/master/src/x86/ffi64.c#L488

As you can see in lines 487 and 488, arguments passed in registers are
never sign-extended, but rather zero-extended.  The register values are
then copied whole in the darwin-specific assembly stub:

  https://github.com/atgreen/libffi/blob/master/src/x86/darwin64.S#L61

Interestingly, arguments passed on the stack *are* sign-extended:

  https://github.com/atgreen/libffi/blob/master/src/x86/darwin64.S#L120

* * * * *

In summary, I think this is a bug in libffi.

Note that it has already been reported that the libffi testsuite shows
many failures on OS X Lion, and the failures appear to be related to
this precise issue:

  http://sourceware.org/ml/libffi-discuss/2012/msg00162.html

The libffi maintainer wrote "I'm going to chalk this up to compiler
bugs", based on his observation that the tests worked properly when
compiled with -O0.  I think it's time to raise this issue again on the
libffi-discuss mailing list.

In any case, it's certainly not a bug in Guile.  The bug is either in
LLVM/Clang or libffi, depending on how one chooses to interpret the
x86-64 API.

    Regards,
      Mark




Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Mon, 28 Jan 2013 02:41:01 GMT) Full text and rfc822 format available.

Message #35 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: Peter Teeson <peter.teeson <at> me.com>
To: Mark H Weaver <mhw <at> netris.org>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 13342 <at> debbugs.gnu.org
Subject: Re: bug#13342: [PARTIALLY SOLVED] bug#13342: Errors trying to build
	Guile 2.0.7
Date: Sun, 27 Jan 2013 21:40:21 -0500
Hi Mark:
Thanks for your comments and I agree I was wrong.
Ludovic pointed out GCC dropping off the MSB 24-bits whereas Clang does not.
You pointed out the lack of sign extension in the libffi implementation.

I would think it would be easier to make the change to libffi 
rather than expect changes to be made to the compilers.
But that's just my opinion.

Anyway glad that you wizards have nailed it.

OK if I bow out now and let others fix it?

respect….

Peter



Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Mon, 28 Jan 2013 12:58:01 GMT) Full text and rfc822 format available.

Message #38 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Mark H Weaver <mhw <at> netris.org>
Cc: 13342 <at> debbugs.gnu.org, control <at> debbugs.gnu.org,
	Peter Teeson <peter.teeson <at> me.com>
Subject: Re: bug#13342: [PARTIALLY SOLVED] bug#13342: Errors trying to build
	Guile 2.0.7
Date: Mon, 28 Jan 2013 13:56:43 +0100
retitle 13342 Clang, the FFI, and 8-bit signed integers
merge 13342 13386
thanks

Hello!

Mark H Weaver <mhw <at> netris.org> skribis:

> ludo <at> gnu.org (Ludovic Courtès) writes:

[...]

>> I looked at Section 3.2.3 (“Parameter Passing”) of the SysV ABI x86_64
>> PS but couldn’t find any evidence as to what the correct behavior is.
>
> I read the same section, and although it is not as clear as I'd prefer,
> my interpretation is that the caller is responsible for sign-extending
> signed chars to ints.

What exactly in that section makes you think so?

> Clang strictly requires callers to sign-extend, whereas GCC is tolerant
> of callers who fail to do so.  IMO, both behaviors are permitted by the
> ABI.
>
> The problem is that libffi does *not* sign-extend arguments passed in
> registers when making calls, which is IMO a bug that has gone (mostly)
> unnoticed because of the tolerance and ubiquity of GCC.

Oh, I see.

>> However, on the caller side, both compilers emit the same code.  This
>> program:
>>
>> #include <stdint.h>
>>
>> extern int64_t test_sum (int8_t a, int64_t b);
>>
>> int64_t
>> foo (void)
>> {
>>   return test_sum (-1, 123132);
>> }
>>
>> leads to the following assembly with both compilers:
>>
>> foo:                                    # @foo
>> 	.cfi_startproc
>> 	movl	$-1, %edi
>> 	movl	$123132, %esi           # imm = 0x1E0FC
>> 	jmp	test_sum                # TAILCALL
>>
>> (And as we’ve seen, libffi does the same.)
>
> No, libffi does *not* do the same.  Take a look at the relevant code:
>
>   https://github.com/atgreen/libffi/blob/master/src/x86/ffi64.c#L488

Indeed!

[...]

> In summary, I think this is a bug in libffi.
>
> Note that it has already been reported that the libffi testsuite shows
> many failures on OS X Lion, and the failures appear to be related to
> this precise issue:

Note that it’s not just Darwin: I tested with Clang on GNU/Linux.

>   http://sourceware.org/ml/libffi-discuss/2012/msg00162.html
>
> The libffi maintainer wrote "I'm going to chalk this up to compiler
> bugs", based on his observation that the tests worked properly when
> compiled with -O0.  I think it's time to raise this issue again on the
> libffi-discuss mailing list.

Would you like to email them?

Anyway, thanks for the thorough investigation!

Ludo’.




Changed bug title to 'Clang, the FFI, and 8-bit signed integers' from 'Errors trying to build Guile 2.0.7' Request was from ludo <at> gnu.org (Ludovic Courtès) to control <at> debbugs.gnu.org. (Mon, 28 Jan 2013 12:58:02 GMT) Full text and rfc822 format available.

Merged 13342 13386. Request was from ludo <at> gnu.org (Ludovic Courtès) to control <at> debbugs.gnu.org. (Mon, 28 Jan 2013 12:58:02 GMT) Full text and rfc822 format available.

Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Mon, 28 Jan 2013 16:41:02 GMT) Full text and rfc822 format available.

Message #45 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: Peter Teeson <peter.teeson <at> me.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Mark H Weaver <mhw <at> netris.org>, 13342 <at> debbugs.gnu.org,
	control <at> debbugs.gnu.org
Subject: Re: bug#13342: [PARTIALLY SOLVED] bug#13342: Errors trying to build
	Guile 2.0.7
Date: Mon, 28 Jan 2013 11:39:46 -0500
Hi:
Do you also need to retitle and merge 10015 and 10681?

These were the similar bugs you mentioned when I first reported the issue.

Peter

On 2013-01-28, at 7:56 AM, Ludovic Courtès wrote:
> retitle 13342 Clang, the FFI, and 8-bit signed integers
> merge 13342 13386
> thanks





Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Mon, 28 Jan 2013 16:56:01 GMT) Full text and rfc822 format available.

Message #48 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Peter Teeson <peter.teeson <at> me.com>
Cc: Mark H Weaver <mhw <at> netris.org>, 13342 <at> debbugs.gnu.org,
	control <at> debbugs.gnu.org
Subject: Re: bug#13342: [PARTIALLY SOLVED] bug#13342: Errors trying to build
	Guile 2.0.7
Date: Mon, 28 Jan 2013 17:55:05 +0100
merge 13342 10015 10681
thanks

Peter Teeson <peter.teeson <at> me.com> skribis:

> Do you also need to retitle and merge 10015 and 10681?

Indeed, thanks!

Ludo’.




Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Wed, 30 Jan 2013 08:35:02 GMT) Full text and rfc822 format available.

Message #51 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: 13342 <at> debbugs.gnu.org
Subject: Re: bug#13342: Errors trying to build Guile 2.0.7
Date: Wed, 30 Jan 2013 03:34:02 -0500
FYI, I've posted about this to the libffi-discuss mailing list.

http://sourceware.org/ml/libffi-discuss/2013/msg00012.html

    Mark




Added tag(s) notabug. Request was from ludo <at> gnu.org (Ludovic Courtès) to control <at> debbugs.gnu.org. (Wed, 30 Jan 2013 21:38:01 GMT) Full text and rfc822 format available.

Reply sent to ludo <at> gnu.org (Ludovic Courtès):
You have taken responsibility. (Wed, 30 Jan 2013 21:38:02 GMT) Full text and rfc822 format available.

Notification sent to Peter Teeson <pteeson <at> me.com>:
bug acknowledged by developer. (Wed, 30 Jan 2013 21:38:03 GMT) Full text and rfc822 format available.

Message #58 received at 13342-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Mark H Weaver <mhw <at> netris.org>
Cc: 13342-done <at> debbugs.gnu.org, control <at> debbugs.gnu.org
Subject: Re: bug#13342: Errors trying to build Guile 2.0.7
Date: Wed, 30 Jan 2013 22:37:07 +0100
tags 13342 notabug
thanks

Hello!

Mark H Weaver <mhw <at> netris.org> skribis:

> FYI, I've posted about this to the libffi-discuss mailing list.
>
> http://sourceware.org/ml/libffi-discuss/2013/msg00012.html

Thanks!  So, according to
<http://gcc.gnu.org/ml/gcc/2013-01/msg00449.html>, this is an LLVM bug.

Peter: would you like to report it over at LLVM?

Anyway, closing this bug now.  Thanks!

Ludo’.




Reply sent to ludo <at> gnu.org (Ludovic Courtès):
You have taken responsibility. (Wed, 30 Jan 2013 21:38:03 GMT) Full text and rfc822 format available.

Notification sent to Peter Teeson <peter.teeson <at> me.com>:
bug acknowledged by developer. (Wed, 30 Jan 2013 21:38:04 GMT) Full text and rfc822 format available.

Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Wed, 30 Jan 2013 22:14:02 GMT) Full text and rfc822 format available.

Message #66 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: Peter Teeson <pteeson <at> me.com>
To: 13342 <at> debbugs.gnu.org, 13386 <at> debbugs.gnu.org,
	Mark H Weaver <mhw <at> netris.org>,
	Ludovic Courtès <ludo <at> gnu.org>
Subject: bug#13342: closed (Re: bug#13342: Errors trying to build Guile 2.0.7)
	,
	bug#13386: closed (Re: bug#13342: Errors trying to build Guile 2.0.7)
Date: Wed, 30 Jan 2013 17:13:05 -0500
Hi guys:
Thanks for closing off those bugs.
I have read all the emails you linked in your close emails and 
completely understand the conversation. 

Specially the comment by Michael Matz.
That clearly confirms that it is an LLVM compiler bug.

Good job both of you for digging down deeper and making things so clear.
At least now we know.

Would you please report this over at LLVM since you did the work to discover it.

Thanks again for your great patience and kindness in helping my fumbling efforts.

respect…..

Peter





Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Thu, 31 Jan 2013 20:22:02 GMT) Full text and rfc822 format available.

Message #69 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: Peter Teeson <pteeson <at> me.com>
Cc: 13342 <at> debbugs.gnu.org,
	Ludovic Courtès <ludo <at> gnu.org>,
	13386 <at> debbugs.gnu.org
Subject: Re: bug#13342: closed (Re: bug#13342: Errors trying to build Guile
	2.0.7) ,
	bug#13386: closed (Re: bug#13342: Errors trying to build Guile 2.0.7)
Date: Thu, 31 Jan 2013 15:20:39 -0500
Hi Peter,

Peter Teeson <pteeson <at> me.com> writes:
> Would you please report this over at LLVM since you did the work to
> discover it.

Okay, I'll take care of it.

I'm also working with the libffi folks to work around this problem in
the interim.  The next libffi release (due out fairly soon) will
interface properly with the existing LLVM-compiled code on your system.

Thanks again for helping us track this down.

   Regards,
     Mark




Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Fri, 01 Feb 2013 02:07:02 GMT) Full text and rfc822 format available.

Message #72 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: Peter Teeson <pteeson <at> me.com>
To: Mark H Weaver <mhw <at> netris.org>
Cc: 13342 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>,
	13386 <at> debbugs.gnu.org
Subject: Re: bug#13342: closed (Re: bug#13342: Errors trying to build Guile
	2.0.7) ,
	bug#13386: closed (Re: bug#13342: Errors trying to build Guile 2.0.7)
Date: Thu, 31 Jan 2013 21:05:14 -0500
Hi Mark:
I read the thread on libffi-discuss.
When you think things are ready to test I'll be glad to do so on my Mac setup.

At present my boot system is MacOS 10.7.5 Lion.
But I can boot into previous versions (Leopard 10.5, Snow Leopard 10.6)

Just so you know….

Peter
On 2013-01-31, at 3:20 PM, Mark H Weaver wrote:
> I'm also working with the libffi folks to work around this problem in
> the interim.  The next libffi release (due out fairly soon) will
> interface properly with the existing LLVM-compiled code on your system.





Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Sat, 02 Feb 2013 13:13:01 GMT) Full text and rfc822 format available.

Message #75 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: Peter Teeson <pteeson <at> me.com>
Cc: 13342 <at> debbugs.gnu.org,
	Ludovic Courtès <ludo <at> gnu.org>,
	13386 <at> debbugs.gnu.org
Subject: Re: bug#13342: closed (Re: bug#13342: Errors trying to build Guile
	2.0.7) ,
	bug#13386: closed (Re: bug#13342: Errors trying to build Guile 2.0.7)
Date: Sat, 02 Feb 2013 08:11:26 -0500
[Message part 1 (text/plain, inline)]
Hi Peter,

Peter Teeson <pteeson <at> me.com> wrote:
> When you think things are ready to test I'll be glad to do so on my
> Mac setup.

I believe that this libffi patch will fix the problem.  Would you like
to test it?  It would also be helpful if you could run "make check" on
both libffi and Guile and let us know of any failures.  There's no need
to test on older versions of the OS.

    Thanks again!
        Mark


[SIGN_EXTEND_SINT_ARGS.patch (text/x-diff, inline)]
diff --git a/src/x86/ffi64.c b/src/x86/ffi64.c
index b8a823d..2014af2 100644
--- a/src/x86/ffi64.c
+++ b/src/x86/ffi64.c
@@ -484,8 +484,25 @@ ffi_call (ffi_cif *cif, void (*fn)(void), void *rvalue, void **avalue)
 		{
 		case X86_64_INTEGER_CLASS:
 		case X86_64_INTEGERSI_CLASS:
-		  reg_args->gpr[gprcount] = 0;
-		  memcpy (&reg_args->gpr[gprcount], a, size < 8 ? size : 8);
+		  /* Sign-extend integer arguments passed in general
+		     purpose registers, to cope with the fact that
+		     LLVM incorrectly assumes that this will be done
+		     (the x86-64 PS ABI does not specify this). */
+		  switch (arg_types[i]->type)
+		    {
+		    case FFI_TYPE_SINT8:
+		      *(SINT64 *)&reg_args->gpr[gprcount] = (SINT64) *((SINT8 *) a);
+		      break;
+		    case FFI_TYPE_SINT16:
+		      *(SINT64 *)&reg_args->gpr[gprcount] = (SINT64) *((SINT16 *) a);
+		      break;
+		    case FFI_TYPE_SINT32:
+		      *(SINT64 *)&reg_args->gpr[gprcount] = (SINT64) *((SINT32 *) a);
+		      break;
+		    default:
+		      reg_args->gpr[gprcount] = 0;
+		      memcpy (&reg_args->gpr[gprcount], a, size < 8 ? size : 8);
+		    }
 		  gprcount++;
 		  break;
 		case X86_64_SSE_CLASS:

Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Sun, 03 Feb 2013 00:07:02 GMT) Full text and rfc822 format available.

Message #78 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: Peter Teeson <pteeson <at> me.com>
To: Mark H Weaver <mhw <at> netris.org>
Cc: 13342 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>,
	13386 <at> debbugs.gnu.org
Subject: Re: bug#13342: closed (Re: bug#13342: Errors trying to build Guile
	2.0.7) ,
	bug#13386: closed (Re: bug#13342: Errors trying to build Guile 2.0.7)
Date: Sat, 02 Feb 2013 19:05:17 -0500
[Message part 1 (text/plain, inline)]
Hi Mark:
The short answer is:
make  check-am
…...
make  check-TESTS
PASS: test-system-cmds
…...
PASS: test-asmobs
PASS: test-ffi
PASS: test-list
……
===================
All 29 tests passed
===================


In more detail the following steps:
Unzipped libffi-3.0.11.tar.gz
Downloaded libffi-3.0.11-includedir-1.patch
Applied that patch
Had to manually apply your diff file patch

(I'll explain in more detail if you wish)

Built and installed libffi in the usual manner.

Then did the same process for guile-2.0.7.tar.gz
did ./configure && make && make check
got the above results.

Here's the summary
Totals for this test run:
passes:                 35268
failures:               0
unexpected passes:      0
expected failures:      6
unresolved test cases:  34
untested test cases:    1
unsupported test cases: 9
errors:                 0

PASS: check-guile


Anything else I can do to help?

Peter




[Message part 2 (text/html, inline)]

Information forwarded to bug-guile <at> gnu.org:
bug#13342; Package guile. (Sun, 03 Feb 2013 03:24:02 GMT) Full text and rfc822 format available.

Message #81 received at 13342 <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: Peter Teeson <pteeson <at> me.com>
Cc: 13342 <at> debbugs.gnu.org,
	Ludovic Courtès <ludo <at> gnu.org>,
	13386 <at> debbugs.gnu.org
Subject: Re: bug#13342: closed (Re: bug#13342: Errors trying to build Guile
	2.0.7) ,
	bug#13386: closed (Re: bug#13342: Errors trying to build Guile 2.0.7)
Date: Sat, 02 Feb 2013 22:22:16 -0500
Peter Teeson <pteeson <at> me.com> writes:
> …...
> PASS: test-asmobs
> PASS: test-ffi
> PASS: test-list
> ……
> ===================
> All 29 tests passed
> ===================

Excellent!  We can now submit the libffi patch upstream.

    Thanks,
      Mark




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Sun, 03 Mar 2013 12:24:03 GMT) Full text and rfc822 format available.

This bug report was last modified 11 years and 27 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.