GNU bug report logs - #66265
Make padding optional with base64

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 66265 in the body.
You can then email your comments to 66265 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Paul Millar <paul.millar <at> desy.de>
To: bug-coreutils <at> gnu.org
Subject: Make padding optional with base64
Date: Fri, 29 Sep 2023 11:46:18 +0200

Hi,

RFC 4648 says[1]:
>   In some circumstances, the use of padding ("=") in base-encoded data
>   is not required or used.

Currently, the 'base64' application always includes the padding when 
encoding, and prints an warning/error message (on stderr) if padding is 
omitted when decoding.  Decoding is nonetheless successful (the correct 
data is emitted on stdout) if the base64-encoded data omits the padding.

I think the base64 application should be updated to support 
base64-encoded data without padding.

My suggestion would be to add an option to base64 to control whether 
padding is added when encoding.  For decoding, it might make sense to 
add an option to control whether padding is expected.

(although, other approaches might be possible)

Cheers,
Paul.

[1] https://datatracker.ietf.org/doc/html/rfc4648#section-3.2

Message #8 received at 66265 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Millar <paul.millar <at> desy.de>, 66265 <at> debbugs.gnu.org
Subject: Re: bug#66265: Make padding optional with base64
Date: Fri, 29 Sep 2023 15:11:55 +0100

On 29/09/2023 10:46, Paul Millar wrote:
> Hi,
> 
> RFC 4648 says[1]:
>   >   In some circumstances, the use of padding ("=") in base-encoded data
>   >   is not required or used.
> 
> Currently, the 'base64' application always includes the padding when
> encoding, and prints an warning/error message (on stderr) if padding is
> omitted when decoding.  Decoding is nonetheless successful (the correct
> data is emitted on stdout) if the base64-encoded data omits the padding.
> 
> I think the base64 application should be updated to support
> base64-encoded data without padding.
> 
> My suggestion would be to add an option to base64 to control whether
> padding is added when encoding.  For decoding, it might make sense to
> add an option to control whether padding is expected.
> 
> (although, other approaches might be possible)
> 
> Cheers,
> Paul.
> 
> [1] https://datatracker.ietf.org/doc/html/rfc4648#section-3.2

I agree with this actually.

The main advantage of padding as I see it is when concatenating encoded data.
I.e. that's the only use case where there could be ambiguity in the received
encoded data, as otherwise one can auto assume appropriate padding based on
the input length. I can't see `base64` or `basenc --base64` being used in a
problematic streaming context like that. If the utils themselves read from
a stream in a _single invocation_ they'll deal with partial blocks appropriately.
Looking at it another way, I don't see how base64 data over _separate invocations_
of these utils could be handled now, as they just give errors in this case anyway.
Now there would be a slight loss in error detection as truncated data would
not be noticed, however that is the case for a third of truncated sizes already.

So on input I'm inclined to auto pad.

On output one can trivially remove padding with `tr -d =`,
so I'm inclined to leave that as is.

I.e. we shouldn't need a new option for any of this.

cheers,
Pádraig

Message #11 received at 66265 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Millar <paul.millar <at> desy.de>, 66265 <at> debbugs.gnu.org
Subject: Re: bug#66265: Make padding optional with base64
Date: Fri, 6 Oct 2023 16:36:23 +0100

[Message part 1 (text/plain, inline)]

On 29/09/2023 15:11, Pádraig Brady wrote:
> On 29/09/2023 10:46, Paul Millar wrote:
>> Hi,
>>
>> RFC 4648 says[1]:
>>    >   In some circumstances, the use of padding ("=") in base-encoded data
>>    >   is not required or used.
>>
>> Currently, the 'base64' application always includes the padding when
>> encoding, and prints an warning/error message (on stderr) if padding is
>> omitted when decoding.  Decoding is nonetheless successful (the correct
>> data is emitted on stdout) if the base64-encoded data omits the padding.
>>
>> I think the base64 application should be updated to support
>> base64-encoded data without padding.
>>
>> My suggestion would be to add an option to base64 to control whether
>> padding is added when encoding.  For decoding, it might make sense to
>> add an option to control whether padding is expected.
>>
>> (although, other approaches might be possible)
>>
>> Cheers,
>> Paul.
>>
>> [1] https://datatracker.ietf.org/doc/html/rfc4648#section-3.2
> 
> I agree with this actually.
> 
> The main advantage of padding as I see it is when concatenating encoded data.
> I.e. that's the only use case where there could be ambiguity in the received
> encoded data, as otherwise one can auto assume appropriate padding based on
> the input length. I can't see `base64` or `basenc --base64` being used in a
> problematic streaming context like that. If the utils themselves read from
> a stream in a _single invocation_ they'll deal with partial blocks appropriately.
> Looking at it another way, I don't see how base64 data over _separate invocations_
> of these utils could be handled now, as they just give errors in this case anyway.
> Now there would be a slight loss in error detection as truncated data would
> not be noticed, however that is the case for a third of truncated sizes already.
> 
> So on input I'm inclined to auto pad.
> 
> On output one can trivially remove padding with `tr -d =`,
> so I'm inclined to leave that as is.
> 
> I.e. we shouldn't need a new option for any of this.

Implementation attached

cheers,
Pádraig

[basenc-autopad.patch (text/x-patch, attachment)]

Message #16 received at 66265-done <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Millar <paul.millar <at> desy.de>, 66265-done <at> debbugs.gnu.org
Subject: Re: bug#66265: Make padding optional with base64
Date: Sat, 7 Oct 2023 15:00:31 +0100

Pushed.

Marking this as done.

cheers,
Pádraig

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.