Skip to content

date: report invalid UTF-8 lines in -f file/stdin input - #14280

Merged
cakebaker merged 1 commit into
uutils:mainfrom
alexchen-sys:fix-12920-date-invalid-utf8-rc
Sep 2, 2026
Merged

cakebaker merged 1 commit into
uutils:mainfrom
alexchen-sys:fix-12920-date-invalid-utf8-rc

Conversation

@alexchen-sys

@alexchen-sys alexchen-sys commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Problem

date -f reads lines via BufRead::lines().map_while(Result::ok). On a line that is not valid UTF-8, lines() yields an InvalidData error and map_while silently ends the iterator: the offending line and everything after it vanish with no diagnostic, and date exits 0 — even with --debug, which stays silent because it never sees the line.

GNU date (current git master) instead reports the line (non-printable bytes octal-escaped), keeps processing the rest of the input and exits 1:

$ printf 'Hello\xffx\n' > bad.txt && date -f bad.txt
date: invalid date 'Hello\377x'

Fix

Read the input with split(b'\n'), stripping a trailing \r, and decode each chunk as UTF-8. A chunk that fails to decode becomes an invalid-date error via the existing escape_invalid_bytes helper (already used for the same diagnostic for the -d argument), producing the GNU-style invalid date 'Hello\377x' message; iteration continues with the remaining lines and the final exit status is 1, matching GNU.

Tests

Three new tests in tests/by-util/test_date.rs:

  • a file with a single invalid-UTF-8 line fails with code 1 and the GNU-shaped message on stderr
  • a mixed file still prints the valid dates around the bad line while failing with code 1
  • the same input through stdin (-f -) behaves identically

Verified against a date binary built from GNU git master (aea70b2): same diagnostic (byte-for-byte in the C locale — GNU renders typographic quotes under UTF-8 locales), same exit code, remaining lines still processed.

Fixes #12920

Lines read via BufReader::lines() + map_while(Result::ok) made any
non-UTF-8 line look like end of input: 'date -f file' (and 'date -f -')
silently exited 0 without printing anything. GNU date prints
"date: invalid date '<octal-escaped line>'" for each such line,
keeps processing the remaining lines, and exits with code 1.

Switch to BufRead::split(b'\n') so raw bytes survive, keep the '\r'
stripping of lines(), and route from_utf8 failures through the existing
invalid-date error arm using escape_invalid_bytes() for the
GNU-style octal escaping.
@github-actions

Copy link
Copy Markdown

Binary size comparison:

Individual binary size comparison VS main (threshold: >=5% AND >=4 KB).

Total size of compared binaries: 152.18 MB (+1004 KB, +0.65%)

Significant per-binary changes:
  comm     1.12 MB ->    2.33 MB  (+1.21 MB, +107.29%)

@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)

@cakebaker
cakebaker merged commit b7519db into uutils:main Sep 2, 2026
99 of 102 checks passed
@cakebaker

Copy link
Copy Markdown
Contributor

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(date): a file that contains \xFF, will lead to no error even with --debug and exits with status code 0 whiches success

2 participants