View Issue Details

IDProjectCategoryView StatusLast Update
0000261fileGeneralpublic2021-04-27 19:39
Reporterbitstreamout Assigned Tochristos  
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionfixed 
Platformx86_64OSopenSUSEOS VersionTumbleweed
Product Version5.40 
Fixed in Version5.41 
Summary0000261: New version breaks subversion tests
DescriptionCurrently subversion build breaks at several points ... many of the breaks are caused by behaviour change of file in detecting ASCII text without newlines.
Steps To ReproduceWith version 4.30

echo xx | file -
/dev/stdin: ASCII text, with no line terminators

with version 5.40

echo -n xx | file -
/dev/stdin: data
TagsNo tags attached.

Activities

bitstreamout

2021-04-21 05:58

reporter   ~0003593

Just to correct typo ... the 'With version 4.30' should be 'With version 5.39'

```
/suse/werner> file --version
file-5.39
magic file from /etc/magic:/usr/share/misc/magic
/suse/werner> echo -n xx | file -
/dev/stdin: ASCII text, with no line terminators
/suse/werner>
```

bitstreamout

2021-04-21 08:38

reporter   ~0003594

Could it be that the condition

if (u < 3)

within the LOOKS_ macro should be

if (u < 2)

at least for ASCII and latin1

bitstreamout

2021-04-22 12:56

reporter   ~0003595

Duplicate of https://bugs.astron.com/view.php?id=256

bitstreamout

2021-04-22 15:25

reporter   ~0003596

Hmmm ... still problems with files without last newline

```
abuild@noether:~/rpmbuild/BUILD/subversion-1.14.1> wc -c /dev/shm/svn-test-work/working_copies/merge_tests-2/A/B/F/foo
3 /dev/shm/svn-test-work/working_copies/merge_tests-2/A/B/F/foo
abuild@noether:~/rpmbuild/BUILD/subversion-1.14.1> file /dev/shm/svn-test-work/working_copies/merge_tests-2/A/B/F/foo
/dev/shm/svn-test-work/working_copies/merge_tests-2/A/B/F/foo: data
abuild@noether:~/rpmbuild/BUILD/subversion-1.14.1> cat /dev/shm/svn-test-work/working_copies/merge_tests-2/A/B/F/foo && echo
foo
```

bitstreamout

2021-04-22 15:27

reporter   ~0003597

Test with file 5.39
```
file /abuild/oscbuild/openSUSE_Tumbleweed/dev/shm/svn-test-work/working_copies/merge_tests-2/A/B/F/foo
/abuild/oscbuild/openSUSE_Tumbleweed/dev/shm/svn-test-work/working_copies/merge_tests-2/A/B/F/foo: ASCII text, with no line terminators
```

bitstreamout

2021-04-22 15:47

reporter   ~0003598

The fails.log of subversion with file 5.40
fails.log (18,188 bytes)   
[[[
W: Unexpected output
W: EXPECTED STDOUT (match_all=True):
W: | Index: foo
W: | ===================================================================
W: | --- foo	(nonexistent)
W: | +++ foo	(working copy)
W: | @@ -0,0 +1,2 @@
W: | +xxx
W: | +yyy
W: ACTUAL STDOUT:
W: | Index: foo
W: | ===================================================================
W: | Cannot display: file marked as a binary type.
W: | svn:mime-type = application/octet-stream
W: | 
W: | Property changes on: foo
W: | ___________________________________________________________________
W: | Added: svn:mime-type
W: | ## -0,0 +1 ##
W: | +application/octet-stream
W: | \ No newline at end of property
W: DIFF STDOUT (match_all=True):
W: | --- EXPECTED STDOUT (match_all=True)
W: | +++ ACTUAL STDOUT
W: | @@ -1,7 +1,11 @@
W: |  Index: foo
W: |  ===================================================================
W: | ---- foo	(nonexistent)
W: | -+++ foo	(working copy)
W: | -@@ -0,0 +1,2 @@
W: | -+xxx
W: | -+yyy
W: | +Cannot display: file marked as a binary type.
W: | +svn:mime-type = application/octet-stream
W: | +
W: | +Property changes on: foo
W: | +___________________________________________________________________
W: | +Added: svn:mime-type
W: | +## -0,0 +1 ##
W: | ++application/octet-stream
W: | +\ No newline at end of property
W: CWD: /dev/shm/svn-test-work/working_copies/diff_tests-27
W: EXCEPTION: SVNLineUnequal
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/main.py", line 1927, in run
    rc = self.pred.run(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/testcase.py", line 178, in run
    result = self.func(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/diff_tests.py", line 2077, in diff_schedule_delete
    svntest.actions.run_and_verify_svn(expected_output_r1_base, [],
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/actions.py", line 339, in run_and_verify_svn
    return run_and_verify_svn2(expected_stdout, expected_stderr,
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/actions.py", line 379, in run_and_verify_svn2
    verify.verify_outputs("Unexpected output", out, err,
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/verify.py", line 531, in verify_outputs
    compare_and_display_lines(message, label, expected, actual, raisable)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/verify.py", line 504, in compare_and_display_lines
    raise raisable
svntest.main.SVNLineUnequal
FAIL:  diff_tests.py 27: scheduled deleted
]]]

[[[
=============== DIFFS NOT EQUAL ===================
Expected
('Index: A2/D/G/rho\n', '===================================================================\n', '--- A2/D/G/rho\t(nonexistent)\n', '+++ A2/D/G/rho\t(revision 8)\n', '@@ -0,0 +1 @@\n', '+88\n', '\\ No newline at end of file\n')
Actual:
Index: A2/D/G/rho

===================================================================

Cannot display: file marked as a binary type.

svn:mime-type = application/octet-stream



Property changes on: A2/D/G/rho

___________________________________________________________________

Added: svn:mime-type

## -0,0 +1 ##

+application/octet-stream

\ No newline at end of property

W: CWD: /home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline
W: EXCEPTION: Failure: Diffs not equal
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/main.py", line 1927, in run
    rc = self.pred.run(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/testcase.py", line 178, in run
    result = self.func(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/log_tests.py", line 2182, in log_diff
    compare_diff_output(r8diff, log_chain[2]['diff_lines'])
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/log_tests.py", line 722, in compare_diff_output
    raise svntest.Failure("Diffs not equal")
svntest.Failure: Diffs not equal
FAIL:  log_tests.py 36: 'svn log --diff'
]]]

[[[
W: =============================================================
Expected 'foo' and actual 'foo' in output tree are different!
=============================================================
EXPECTED NODE TO BE:
=============================================================
 * Node name:   foo
    Path:       svn-test-work/working_copies/merge_tests-2/A/B/F/foo
    Contents:   None
    Properties: {}
    Attributes: {'verb': 'Adding'}
    Children:  None (node is probably a file)
=============================================================
ACTUAL NODE FOUND:
=============================================================
 * Node name:   foo
    Path:       svn-test-work/working_copies/merge_tests-2/A/B/F/foo
    Contents:   None
    Properties: {}
    Attributes: {'verb': 'Adding  (bin)'}
    Children:  None (node is probably a file)

W: Unequal at node foo
W: Unequal at node F
W: Unequal at node B
W: Unequal at node A
W: Unequal at node merge_tests-2
W: Unequal at node working_copies
W: Unequal at node svn-test-work
W: Output of commit is unexpected
W: EXPECTED OUTPUT TREE:
W: ROOT
  +-- svn-test-work
        +-- working_copies
              +-- merge_tests-2
                    +-- A
                          +-- B
                                +-- F
                                      |-- Q
                                      |     |-- bar
                                      |     +-- bar2
                                      |-- Q2
                                      |-- foo
                                      +-- foo2

W: ACTUAL OUTPUT TREE:
W: ROOT
  +-- svn-test-work
        +-- working_copies
              +-- merge_tests-2
                    +-- A
                          +-- B
                                +-- F
                                      |-- Q
                                      |     |-- bar
                                      |     +-- bar2
                                      |-- Q2
                                      |-- foo
                                      +-- foo2

W: ACTUAL OUTPUT TREE:
svntest.wc.State(wc_dir, {
  'A/B/F/Q'           : Item(verb='Adding'),
  'A/B/F/Q/bar'       : Item(verb='Adding'),
  'A/B/F/Q/bar2'      : Item(verb='Adding'),
  'A/B/F/Q2'          : Item(verb='Adding'),
  'A/B/F/foo'         : Item(verb='Adding  (bin)'),
  'A/B/F/foo2'        : Item(verb='Adding'),
})
W: CWD: /home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline
W: EXCEPTION: SVNTreeUnequal
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/main.py", line 1927, in run
    rc = self.pred.run(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/testcase.py", line 258, in run
    return self._delegate.run(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/testcase.py", line 178, in run
    result = self.func(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/merge_tests.py", line 403, in add_with_history
    svntest.actions.run_and_verify_commit(wc_dir,
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/actions.py", line 1565, in run_and_verify_commit
    tree.compare_trees("output", actual, output_tree)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/tree.py", line 683, in compare_trees
    compare_trees(label, a_child, b_child,
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/tree.py", line 683, in compare_trees
    compare_trees(label, a_child, b_child,
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/tree.py", line 683, in compare_trees
    compare_trees(label, a_child, b_child,
  [Previous line repeated 4 more times]
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/tree.py", line 661, in compare_trees
    raise SVNTreeUnequal
svntest.tree.SVNTreeUnequal
FAIL:  merge_tests.py 2: merge and add new files/dirs with history
]]]

[[[
W: =============================================================
Expected 'foo' and actual 'foo' in output tree are different!
=============================================================
EXPECTED NODE TO BE:
=============================================================
 * Node name:   foo
    Path:       svn-test-work/working_copies/merge_tests-15/A/B/F/foo
    Contents:   None
    Properties: {}
    Attributes: {'verb': 'Adding'}
    Children:  None (node is probably a file)
=============================================================
ACTUAL NODE FOUND:
=============================================================
 * Node name:   foo
    Path:       svn-test-work/working_copies/merge_tests-15/A/B/F/foo
    Contents:   None
    Properties: {}
    Attributes: {'verb': 'Adding  (bin)'}
    Children:  None (node is probably a file)

W: Unequal at node foo
W: Unequal at node F
W: Unequal at node B
W: Unequal at node A
W: Unequal at node merge_tests-15
W: Unequal at node working_copies
W: Unequal at node svn-test-work
W: Output of commit is unexpected
W: EXPECTED OUTPUT TREE:
W: ROOT
  +-- svn-test-work
        +-- working_copies
              +-- merge_tests-15
                    +-- A
                          +-- B
                                +-- F
                                      |-- Q
                                      |     +-- bar
                                      +-- foo

W: ACTUAL OUTPUT TREE:
W: ROOT
  +-- svn-test-work
        +-- working_copies
              +-- merge_tests-15
                    +-- A
                          +-- B
                                +-- F
                                      |-- Q
                                      |     +-- bar
                                      +-- foo

W: ACTUAL OUTPUT TREE:
svntest.wc.State(wc_dir, {
  'A/B/F/Q'           : Item(verb='Adding'),
  'A/B/F/Q/bar'       : Item(verb='Adding'),
  'A/B/F/foo'         : Item(verb='Adding  (bin)'),
})
W: CWD: /home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline
W: EXCEPTION: SVNTreeUnequal
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/main.py", line 1927, in run
    rc = self.pred.run(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/testcase.py", line 258, in run
    return self._delegate.run(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/testcase.py", line 178, in run
    result = self.func(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/merge_tests.py", line 1342, in merge_skips_obstructions
    svntest.actions.run_and_verify_commit(wc_dir,
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/actions.py", line 1565, in run_and_verify_commit
    tree.compare_trees("output", actual, output_tree)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/tree.py", line 683, in compare_trees
    compare_trees(label, a_child, b_child,
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/tree.py", line 683, in compare_trees
    compare_trees(label, a_child, b_child,
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/tree.py", line 683, in compare_trees
    compare_trees(label, a_child, b_child,
  [Previous line repeated 4 more times]
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/tree.py", line 661, in compare_trees
    raise SVNTreeUnequal
svntest.tree.SVNTreeUnequal
FAIL:  merge_tests.py 15: merge should skip over unversioned obstructions
]]]

[[[
W: =============================================================
Expected 'foo' and actual 'foo' in output tree are different!
=============================================================
EXPECTED NODE TO BE:
=============================================================
 * Node name:   foo
    Path:       svn-test-work/working_copies/merge_tests-16/A/B/F/foo
    Contents:   None
    Properties: {}
    Attributes: {'verb': 'Adding'}
    Children:  None (node is probably a file)
=============================================================
ACTUAL NODE FOUND:
=============================================================
 * Node name:   foo
    Path:       svn-test-work/working_copies/merge_tests-16/A/B/F/foo
    Contents:   None
    Properties: {}
    Attributes: {'verb': 'Adding  (bin)'}
    Children:  None (node is probably a file)

W: Unequal at node foo
W: Unequal at node F
W: Unequal at node B
W: Unequal at node A
W: Unequal at node merge_tests-16
W: Unequal at node working_copies
W: Unequal at node svn-test-work
W: Output of commit is unexpected
W: EXPECTED OUTPUT TREE:
W: ROOT
  +-- svn-test-work
        +-- working_copies
              +-- merge_tests-16
                    +-- A
                          +-- B
                                +-- F
                                      |-- Q
                                      +-- foo

W: ACTUAL OUTPUT TREE:
W: ROOT
  +-- svn-test-work
        +-- working_copies
              +-- merge_tests-16
                    +-- A
                          +-- B
                                +-- F
                                      |-- Q
                                      +-- foo

W: ACTUAL OUTPUT TREE:
svntest.wc.State(wc_dir, {
  'A/B/F/Q'           : Item(verb='Adding'),
  'A/B/F/foo'         : Item(verb='Adding  (bin)'),
})
W: CWD: /home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline
W: EXCEPTION: SVNTreeUnequal
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/main.py", line 1927, in run
    rc = self.pred.run(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/testcase.py", line 258, in run
    return self._delegate.run(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/testcase.py", line 178, in run
    result = self.func(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/merge_tests.py", line 1634, in merge_into_missing
    svntest.actions.run_and_verify_commit(wc_dir,
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/actions.py", line 1565, in run_and_verify_commit
    tree.compare_trees("output", actual, output_tree)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/tree.py", line 683, in compare_trees
    compare_trees(label, a_child, b_child,
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/tree.py", line 683, in compare_trees
    compare_trees(label, a_child, b_child,
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/tree.py", line 683, in compare_trees
    compare_trees(label, a_child, b_child,
  [Previous line repeated 4 more times]
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/tree.py", line 661, in compare_trees
    raise SVNTreeUnequal
svntest.tree.SVNTreeUnequal
FAIL:  merge_tests.py 16: merge into missing must not break working copy
]]]

[[[
W: Unexpected output
W: EXPECTED STDOUT (unordered):
W: | Properties on 'G':
W: |   svn:mergeinfo
W: | Properties on 'G/M':
W: |   key
W: | Properties on 'G/file':
W: |   key
W: | Properties on 'G/M/N':
W: |   key
W: | Properties on 'G/M/file':
W: |   key
W: ACTUAL STDOUT:
W: | Properties on 'G':
W: |   svn:mergeinfo
W: | Properties on 'G/M':
W: |   key
W: | Properties on 'G/M/N':
W: |   key
W: | Properties on 'G/M/file':
W: |   key
W: |   svn:mime-type
W: | Properties on 'G/file':
W: |   key
W: |   svn:mime-type
W: DIFF STDOUT (unordered):
W: | --- EXPECTED STDOUT (unordered)
W: | +++ ACTUAL STDOUT
W: | @@ -3,6 +3,8 @@
W: |    key
W: |    key
W: |    svn:mergeinfo
W: | +  svn:mime-type
W: | +  svn:mime-type
W: |  Properties on 'G':
W: |  Properties on 'G/M':
W: |  Properties on 'G/M/N':
W: CWD: /dev/shm/svn-test-work/working_copies/merge_tests-134
W: EXCEPTION: SVNLineUnequal
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/main.py", line 1927, in run
    rc = self.pred.run(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/testcase.py", line 258, in run
    return self._delegate.run(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/testcase.py", line 178, in run
    result = self.func(sandbox)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/merge_tests.py", line 17418, in merge_properties_on_adds
    svntest.actions.run_and_verify_svn(expected_output, [],
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/actions.py", line 339, in run_and_verify_svn
    return run_and_verify_svn2(expected_stdout, expected_stderr,
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/actions.py", line 379, in run_and_verify_svn2
    verify.verify_outputs("Unexpected output", out, err,
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/verify.py", line 531, in verify_outputs
    compare_and_display_lines(message, label, expected, actual, raisable)
  File "/home/abuild/rpmbuild/BUILD/subversion-1.14.1/subversion/tests/cmdline/svntest/verify.py", line 504, in compare_and_display_lines
    raise raisable
svntest.main.SVNLineUnequal
FAIL:  merge_tests.py 134: merged directory properties are added
]]]

fails.log (18,188 bytes)   

bitstreamout

2021-04-23 06:58

reporter   ~0003599

Something goes wrong even with those commits for bug PR/256
```
abuild@noether:~/rpmbuild/BUILD/file-5.40> echo -e "fo" | $PWD/src/.libs/file -m $PWD/magic/magic -
/dev/stdin: ASCII text
abuild@noether:~/rpmbuild/BUILD/file-5.40> echo -e "xx" | $PWD/src/.libs/file -m $PWD/magic/magic -
/dev/stdin: data
abuild@noether:~/rpmbuild/BUILD/file-5.40> echo -e "hi" | $PWD/src/.libs/file -m $PWD/magic/magic -
/dev/stdin: ASCII text
abuild@noether:~/rpmbuild/BUILD/file-5.40> echo -en "hi" | $PWD/src/.libs/file -m $PWD/magic/magic -
/dev/stdin: ASCII text, with no line terminators
abuild@noether:~/rpmbuild/BUILD/file-5.40> echo -en "foo" | $PWD/src/.libs/file -m $PWD/magic/magic -
/dev/stdin: data
abuild@noether:~/rpmbuild/BUILD/file-5.40> echo -e "foo" | $PWD/src/.libs/file -m $PWD/magic/magic -
/dev/stdin: ASCII text
abuild@noether:~/rpmbuild/BUILD/file-5.40> echo -e "xxx" | $PWD/src/.libs/file -m $PWD/magic/magic -
/dev/stdin: data
```

bitstreamout

2021-04-23 07:36

reporter   ~0003600

I suggest the attached patch to count every ASCII character even if it appears several times
file-5.50-ascii.patch (699 bytes)   
From a806b7c99870f76c5fcf3d34f9d91f37685e1a1c Mon Sep 17 00:00:00 2001
From: Werner Fink <werner@suse.de>
Date: Fri, 23 Apr 2021 09:32:09 +0200
Subject: [PATCH] Count every ASCII character

Signed-off-by: Werner Fink <werner@suse.de>
---
 src/encoding.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git src/encoding.c src/encoding.c
index 31d4d125..686be210 100644
--- a/src/encoding.c
+++ b/src/encoding.c
@@ -282,8 +282,7 @@ looks_ ## NAME(const unsigned char *buf, size_t nbytes, file_unichar_t *ubuf, \
 	} \
 	u = 0; \
 	for (i = 0; i < __arraycount(dist); i++) { \
-		if (dist[i]) \
-			u++; \
+		u += dist[i]; \
 	} \
 	if (u < MIN(nbytes, 3)) \
 		return 0; \
-- 
2.28.0

file-5.50-ascii.patch (699 bytes)   

bitstreamout

2021-04-23 11:07

reporter   ~0003601

I see that commit 3096f87f823e1e936139e48d6a3bae9a95557861 had introduced the `if (dist[i]) u++` which misdetect smaller ASCII files with and without newlines

christos

2021-04-27 19:39

manager   ~0003603

The whole character count/distribution approach leads to more confusion as it tries to solve some corner cases. It is not worth using heuristics to resolve the corner cases. I've reverted the fix to PR/180 and that should bring back the original behavior.

Issue History

Date Modified Username Field Change
2021-04-20 11:24 bitstreamout New Issue
2021-04-21 05:58 bitstreamout Note Added: 0003593
2021-04-21 08:38 bitstreamout Note Added: 0003594
2021-04-22 12:56 bitstreamout Note Added: 0003595
2021-04-22 15:25 bitstreamout Note Added: 0003596
2021-04-22 15:27 bitstreamout Note Added: 0003597
2021-04-22 15:47 bitstreamout File Added: fails.log
2021-04-22 15:47 bitstreamout Note Added: 0003598
2021-04-23 06:58 bitstreamout Note Added: 0003599
2021-04-23 07:36 bitstreamout File Added: file-5.50-ascii.patch
2021-04-23 07:36 bitstreamout Note Added: 0003600
2021-04-23 11:07 bitstreamout Note Added: 0003601
2021-04-27 19:37 christos Assigned To => christos
2021-04-27 19:37 christos Status new => assigned
2021-04-27 19:39 christos Status assigned => resolved
2021-04-27 19:39 christos Resolution open => fixed
2021-04-27 19:39 christos Fixed in Version => 5.41
2021-04-27 19:39 christos Note Added: 0003603