View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
90 [file] General feature N/A 2019-07-04 19:12 2019-09-12 16:02
Reporter: Pysis Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: assigned Product Version: 5.37  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Adds Nintendo Console Wii U/Switch file formats
Description: Adds header detailed output support for these formats:
 - Yaz0
 - BYAML
 - BFRES
 - BLWP
 - STAT
 - BARS
 - BFSAR
 - SARC

Lightly Documents these formats "clearinghouse" style:
 - BTSND
 - RPL/RPX
 - BFMA
 - (Non-archive) ARC

Tested with local sample files and Bats-core tests:
 - tests/data/A-1.00.autoplacement_forbid.agstats
 - tests/data/MapTex_A-0_0.bfres
 - tests/data/yaz0.sarc
 - tests/data/AkkareLabBgm.bars
 - tests/data/_DistanceView_Dynamic.byml
 - tests/data/DummySound.bfsar
 - tests/data/yaz0/_DistanceView_TeraTree.sblwp
 - tests/data/yaz0/AncientArrow.sbitemico
 - tests/data/yaz0/MapOpen_00.sbmapopen
 - tests/data/yaz0/MapTex_A-0_0.sbmaptex
 - tests/data/yaz0/5000000000.grass.extm.sstera
 - tests/data/yaz0/_DistanceView_Dynamic.smubin
 - tests/data/yaz0/Demo109_1.sbreviewtex
 - tests/data/yaz0/A-1.00.sstats
 - tests/data/yaz0/Rollpict001.sbstftex
 - tests/data/A-1.00_Clustering.blwp

 - (tests/_setup.sh)
 - tests/stat.bats
 - tests/bfres.bats
 - tests/yaz0.bats
 - tests/sarc.bats
 - tests/byml.bats
 - tests/bfsar.bats
 - tests/blwp.bats
 - tests/bars.bats

~ fileutil -v
file-5.37
magic file from /usr/local/Cellar/libmagic/5.37/share/misc/magic
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: file.tests.2019-07-04.19-19.gz (2,485 bytes) 2019-07-04 20:20
https://bugs.astron.com/file_download.php?file_id=60&type=bug
file.2019-07-04.19-11.gz (3,550 bytes) 2019-07-04 20:20
https://bugs.astron.com/file_download.php?file_id=59&type=bug
file.2019-07-07.13-32.gz (27,358 bytes) 2019-07-07 17:32
https://bugs.astron.com/file_download.php?file_id=62&type=bug
file.2019-07-07.10-42.gz (2,777 bytes) 2019-07-07 17:32
https://bugs.astron.com/file_download.php?file_id=61&type=bug
file.2019-22-08.22-33.gz (1,748 bytes) 2019-08-23 02:35
https://bugs.astron.com/file_download.php?file_id=74&type=bug
Notes
(0003258)
Pysis   
2019-07-07 17:32   
From the file headers:
grep -oPiz '\n\n#-+\n([^\n]+?)\n#[^\n]+?\n' \
      console \
      | grep -oPiz '\n\n#-+\n([^\n]+?)\n' \
      | grep -oPiz '\w.+?\n' \
      | sed 's|\x0||g' \
      > reference/file_current_console_magics.txt

Only from the section headers:
grep -oPiz '\n\n#-+\n([^\n]+?)\n#\n' \
        console \
        | grep -oPiz '\w.+?\n' \
        | sed 's|\x0||g' \
        > reference/file_current_console_magic_sections.txt
(0003262)
christos   
2019-07-21 09:35   
Some of these are too weak (too few bytes of magic and the magic is text) so they would be produce spurious matches to text files. We should add these commented out. These days the minimum magic is around 32 bits. Can you restructure (where possible) the magic to look like:


0x0 string SARC
>0x6 beshort 0xFEFF SARC archive file, BOM Big Endian,
>> use nintendo_sark
>0x6 beshort 0xFFFE SARC archive file, BOM Little Endian,
>>0 use nintendo_sark

0 name nintendo_sark
!:ext sarc
>0x4 beshort x Header length %xh,
>0x8 belong x File size %d bytes,
>0x0C belong x Data offset %xh,
>0x10 beshort x Version Number %d

This way you need SARK and a valid BOM for a match. What do you think?
(0003286)
Pysis   
2019-08-23 02:18   
Sorry for the delay.

I saw that note, but I believe those 4 ASCII letters do technically suffice. I see what you mean as an alternative, and it seems ok, but the way I have it just seems cleaner and done.
(0003304)
christos   
2019-09-12 16:02   
The advantage of my approach is that it makes the effective magic 8 bytes... I.e. it requires SARC\xfe\xff or SARC\xff\xfe before a match instead of just SARK.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
105 [file] General major always 2019-09-11 06:12 2019-09-12 15:38
Reporter: lighe Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: feedback Product Version: 5.37  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Docx files exported by MacOS's Pages not recognized
Description: 1) Create a document in Pages
2) File > Export to > Word > Docx
3) file -i file.docx reports application/octet-stream; charset=binary instead of application/vnd.openxmlformats-officedocument.wordprocessingml.document; charset=binary

Seems to be with any content.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: Incorrect.docx (383,370 bytes) 2019-09-11 06:12
https://bugs.astron.com/file_download.php?file_id=78&type=bug
Notes
(0003289)
christos   
2019-09-11 14:48   
[10:46am] 191>./file macosx.docx
macosx.docx: Microsoft Word 2007+
[10:46am] 192>./file -i macosx.docx
macosx.docx: application/vnd.openxmlformats-officedocument.wordprocessingml.document; charset=binary
[10:46am] 193>./file -v
file-5.37
magic file from /usr/local/share/misc/magic

What do you get?
(0003290)
christos   
2019-09-11 14:50   
ah, there are some changes on HEAD which might have fixed the issue.
(0003299)
lighe   
2019-09-12 13:25   
I am getting application/octet-stream; charset=binary
(0003300)
lighe   
2019-09-12 13:39   
Sorry.
file -v reportd file-5.30. So it looks like I confused myself thinking I'm on the latest version!
(0003301)
christos   
2019-09-12 15:38   
please try with the latest version from github.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
95 [tcsh] General minor always 2019-08-01 14:59 2019-09-12 01:54
Reporter: mnowak Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 6.21.00  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: 'nice' test fails on OpenIndiana
Description: 68. commands.at:893: testing nice ...
/export/home/newman/ws/oi-userland/components/shell/tcsh/tcsh-6.21.00/commands.at:896: tcsh -f -c 'nice set var=1; echo $?var'
--- /dev/null 2019-08-01 16:49:34.000000000 +0000
+++ /export/home/newman/ws/oi-userland/components/shell/tcsh/build/amd64/testsuite.dir/at-groups/68/stderr 2019-08-01 16:49:36.642047795 +0000
@@ -0,0 +1 @@
+setpriority: Permission denied.
68. commands.at:893: 68. nice (commands.at:893): FAILED (commands.at:896)
Tags:
Steps To Reproduce: Run tests as user like this:

/usr/bin/env PATH=/usr/gnu/bin:/usr/bin:/usr/sbin:/usr/perl5/bin /usr/gnu/bin/make check

Here GNU tools take precedence to illumos ones otherwise even more tests fail.
Additional Information:
Attached Files: testsuite.log (91,561 bytes) 2019-08-01 14:59
https://bugs.astron.com/file_download.php?file_id=68&type=bug
Notes
(0003278)
christos   
2019-08-02 10:47   
I can't reproduce this, but I fixed the tests so that the ones that failed because /usr/bin/false returns 255 instead of 0 are fixed.
(0003279)
mnowak   
2019-08-02 16:23   
Right, now I see it pass too... Thanks.
(0003298)
christos   
2019-09-12 01:54   
Submitter can't reproduce


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
100 [file] General minor always 2019-08-16 14:50 2019-09-11 22:55
Reporter: SIGSTACKFAULT Platform: x86_64 (vbox)  
Assigned To: christos OS: Ubuntu  
Priority: low OS Version: 19.04  
Status: resolved Product Version: 5.35  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: HEAD  
    Target Version:  
Summary: device tree configuration (.dts) files listed as "C source"
Description: Device tree configuration files, `.dts` are listed as "C source" files despite being, well, not C source code.
Tags: magic
Steps To Reproduce: For example, I'm going to use .dts files from the zephyr RTOS.

    git clone https://github.com/zephyrproject-rtos/zephyr --depth 10
    file zephyr/boards/arm/nrf52840_mdk/nrf52840_mdk.dts

files in `zephyr/boards/*/*/*.dts` all do this. Some are listed as just "ASCII text". I'm not sure if that says anything about your C magic. One is marked as UTF-8 unicode but I think that's zephyr's fault.

`--extension` doesn't help either.
Additional Information:
Attached Files: reproduce.sh (98 bytes) 2019-08-16 14:50
https://bugs.astron.com/file_download.php?file_id=73&type=bug
Notes
(0003285)
SIGSTACKFAULT   
2019-08-16 15:04   
The magic string you're looking for is "/dts-v1/", but it's not necessarily at the start; there may be comments before it.
(0003297)
christos   
2019-09-11 22:55   
Added magic.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
104 [file] General minor have not tried 2019-09-10 21:04 2019-09-11 17:07
Reporter: Ilrandar Platform:  
Assigned To: christos OS: ArchLinux  
Priority: normal OS Version:  
Status: assigned Product Version: 5.37  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: pdf file incorrectly reported as `data`
Description: Some pdf files downloaded from the internet are incorrectly reported as `data` by file. Their associated mime-type is `application/octet-stream` and not `application/pdf`. I join such a pdf to this report.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: certificat_scolarité_l2_eco.pdf (1,184,843 bytes) 2019-09-10 21:04
https://bugs.astron.com/file_download.php?file_id=77&type=bug
Notes
(0003288)
christos   
2019-09-11 14:42   
These are the first few lines of the file:

HTTP/1.1 200 OK
Date: Tue, 10 Sep 2019 08:38:20 GMT
Server: Apache/2.4.38 (Debian)
Content-Disposition: attachment; filename="21808995-2019-certificat-scolarite.pdf"
Cache-Control: no-cache, private
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 58
Content-Length: 1184531
Content-Type: application/pdf

Here's where the pdf file starts:

%PDF-1.3

The tool you used to download it or the original file has junk in front. Of course some browsers ignore the junk and process it as a pdf file (because users want things to just work), but this is just crappy behavior. Most application will not open it properly, and it is also a security issue since you can masquerade files this way. It is also fragile. How many lines does it try to parse? 10? 1K of data? Who knows. Depends on the implementation. Of course file can also be modified to mimick this behavior at the loss of efficiency and encouraging people to produce junk...
(0003295)
Ilrandar   
2019-09-11 17:07   
Oh, I didn’t know I could open pdf files with a text editor.
I don’t think you should ignore junk in front of file. I just needed some way to get this file (and a few other) to be recognized as pdf files, but if I can just open them and get rid of the leading incorrect lines, I will just do it.
Thank you for your answer.
As far as I’m concerned, you can consider this issue closed.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
98 [file] General minor always 2019-08-11 08:47 2019-09-11 16:59
Reporter: johannes Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: feedback Product Version: 5.37  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: LLVM Assembly Language files (*.ll) are reported as C source
Description: Files containing the human-readable assembly language of the LLVM intermediate representation are mistakenly reported to be C source.
Perhaps we can add some simple patterns to distuingish them. The mime type could be "text/x-llvm" or "text/x-llvm-ir".
LLVM Language: https://llvm.org/docs/LangRef.html
Tags:
Steps To Reproduce: $ cat > main.ll << EOF
define i8 @main() {
    ret i8 123
}
EOF

$ file main.ll
main.ll: C source, ASCII text
Additional Information: Here are some magic patterns I've been using which seem to work

cat >> magic/Magdir/llvm <<EOF
0 search/8192 define LLVM Assembly Language text
!:mime text/x-llvm
>0 regex \^define

0 search/8192 declare LLVM Assembly Language text
!:mime text/x-llvm
>0 regex \^declare
EOF
Attached Files:
Notes
(0003294)
christos   
2019-09-11 16:59   
We need something stronger than that, otherwise many text files will match.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
102 [file] General major always 2019-08-30 02:59 2019-09-11 15:46
Reporter: Sergei Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.37  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: HEAD  
    Target Version:  
Summary: Incorrect fix in "PR/518: Fall back to use setlocale() for the OS's that don't support"
Description: Fix implemented by
https://github.com/file/file/commit/b74175e236c34beeb930c95bb25300c92a205cd3 is incorrect:

"setlocale" doesn't return previous locale like "uselocale" does. "setlocale" returns (man 3 setlocale):

> RETURN VALUE
> A successful call to setlocale() returns an opaque string that corresponds to the locale set. This string may be allocated in static storage. The
> string returned is such that a subsequent call with that string and its associated category will restore that part of the process's locale. The
> return value is NULL if the request cannot be honored.

So that "rx->old_lc_ctype = setlocale(LC_CTYPE, "C");" and the likes is utterly wrong. I suggest using the original method from the commit "c397fb230f70cfa1dc8f3d387f4df8ebec6c1a63" to save locale. Patch attached.

With best regards,
Sergei.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: 0001-Fix-locale-saving-when-using-setlocale.patch (2,179 bytes) 2019-08-30 02:59
https://bugs.astron.com/file_download.php?file_id=76&type=bug
Notes
(0003292)
christos   
2019-09-11 15:46   
patch applied, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
103 [file] General minor always 2019-09-01 04:41 2019-09-11 15:20
Reporter: stokito Platform:  
Assigned To: christos OS:  
Priority: high OS Version:  
Status: resolved Product Version: 5.37  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: HEAD  
    Target Version:  
Summary: ZStandard incorrect MIME type application/x-zstd
Description: In the commit "PR/573: Nick Terrell: Add zstd support" https://github.com/file/file/commit/5b8826616cb699b69f7a1cc7b5c0cbac4b5bed9b was added a support for
for ZStandard compressed files with "application/x-zstd" MIME type.
But latter the official MIME type was assigned without the "x-" i.e. "application/zstd":
https://www.iana.org/assignments/media-types/application/zstd
https://tools.ietf.org/search/rfc8478
https://github.com/facebook/zstd/issues/767
Tags: magic
Steps To Reproduce: $ touch test.txt
$ zstd test.txt
test.txt :1300.00% ( 0 => 13 bytes, test.txt.zst)
$ file -i test.txt.zst
test.txt.zst: application/x-zstd; charset=binary
Additional Information: All GUI archievers (Ark, File Roller, Engrampa) internally using the correct "application/zstd" MIME type
Attached Files:
Notes
(0003287)
stokito   
2019-09-04 19:46   
BTW, only 0xFD2FB528 was standardized so we can remove all previous magic signatures.
0xFD2FB522 Zstandard compressed data (v0.2)
0xFD2FB523 Zstandard compressed data (v0.3)
0xFD2FB524 Zstandard compressed data (v0.4)
0xFD2FB525 Zstandard compressed data (v0.5)
0xFD2FB526 Zstandard compressed data (v0.6)
0xFD2FB527 Zstandard compressed data (v0.7)
(0003291)
christos   
2019-09-11 15:20   
Fixed, thanks


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
99 [tcsh] General minor always 2019-08-13 14:44 2019-08-13 14:44
Reporter: xdelaruelle Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: new Product Version: 6.21.00  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: use of :q in back-tick context leads to erroneous extra history entries
Description: I am developing a tool called Modules (http://modules.sourceforge.net/) which enables to dynamically update user's shell environment. In short, this tool provides a shell alias called `module` which evaluates in the current shell the output of a script producing shell code.

The `module` command is currently defined this way:

  $ alias module 'eval "`/usr/share/Modules/libexec/modulecmd.tcl tcsh \!*:q`"'

Which leads to erroneous entries in the ~/.history file:

  #+1565705427
  module load foo bar
  #+1565705427
  load foo bar#+1565705430
  history

All arguments passed from the alias to the modulecmd.tcl script are added as a second history line. These erroneous entries seem to come from the use of the quote modifier `:q` within a back-tick context ``.

Some explanation on the use of the quote modifier :q in this alias: as the modulecmd.tcl script outputs shell codes, it contains sometime special characters like curly braces, so the result of the script execution is enclosed in double quotes to pass it to eval. In this situation to correctly obtain quoted arguments, the quote modifier is used.

The issue has been reproduced on tcsh versions 6.18, 6.19, 6.20 and 6.21.
Tags:
Steps To Reproduce: To reproduce the issue on a small example, here is a script that produces some shell code (to define an alias or set an environment variable):

$ cat ./dispatch
#!/bin/csh
if ( $#argv != 1 ) then
  echo echo should get exactly 1 arg
  exit 1
endif

switch ( $argv[1] )
case myname:
  echo alias myname getent\\ passwd\\ \\\$USER\ \\\|\\ awk\\ -F:\\ \\\'\\\{print\ \\\$5\\\}\\\';
  breaksw;
case "":
  echo setenv EMPTY 1;
  breaksw;
endsw

$ echo $tcsh
6.21.00

Here we define the shell alias that calls the script and evaluates the shell code this script outputs:

$ alias tuneenv 'eval "`./dispatch \!*:q`"'

Then we use the alias:

$ tuneenv ""
$ echo $EMPTY
1
$ tuneenv myname
$ myname
Xavier
$ exit

Looking at history file, erroneous entries can be seen:

$ tail ~/.history
#+1565702553
tuneenv ""
#+1565702553
""#+1565702559
echo $EMPTY
#+1565702565
tuneenv myname
#+1565702565
myname#+1565702567
myname
Additional Information: Some additional tests to demonstrate the need to enclose script result in double quotes to pass it to eval:

$ alias tuneenv 'eval `./dispatch \!*`'
$ tuneenv myname
Missing '}'.

So without the double quotes, the shell alias myname which contains curly braces cannot be set

Then if we enclose script result in double quotes, !* should get the quote modifier applied to correctly transmit quoted arguments:

$ alias tuneenv 'eval "`./dispatch \!*`"'
$ tuneenv myname
$ myname
Xavier
$ tuneenv ""
should get exactly 1 arg

Without :q, the "" argument is not transmitted to the dispatch script.
Attached Files: dispatch (279 bytes) 2019-08-13 14:44
https://bugs.astron.com/file_download.php?file_id=72&type=bug
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
97 [file] General minor have not tried 2019-08-06 14:41 2019-08-10 22:33
Reporter: jidanni Platform: Debian  
Assigned To: christos OS: Linux  
Priority: normal OS Version:  
Status: assigned Product Version: 5.37  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: File misses the alpha channel sometimes
Description: File misses the alpha channel in the second file!

$ set slope.png zaokeng_slope.png
$ identify -format '%[channels]\n' $@
srgba
srgba
$ file $@
slope.png: PNG image data, 640 x 220, 8-bit/color RGBA, non-interlaced
zaokeng_slope.png: PNG image data, 700 x 400, 8-bit/color RGB, non-interlaced
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: slope.png (17,410 bytes) 2019-08-06 14:41
https://bugs.astron.com/file_download.php?file_id=71&type=bug
zaokeng_slope.png (16,283 bytes) 2019-08-06 14:41
https://bugs.astron.com/file_download.php?file_id=70&type=bug
Notes
(0003283)
christos   
2019-08-10 13:27   
(Last edited: 2019-08-10 13:28)
Well, "identify" is right and wrong. The second file is not rgba, it is rgb with extra transparency information in a "tRNS" chunk. Effectively this means that both files contain alpha info, but they are encoded differently. file(1) only looks at the IHDR section and does not do further processing to see if there are any other chunks. While we can make file(1) smarter about png files by parsing more of the png-file and all the additional chunks, and print additional information about it, it hardly seems worth-while. For reference: https://en.wikipedia.org/wiki/Portable_Network_Graphics

(0003284)
jidanni   
2019-08-10 22:33   
(I was trying to scan for all files that would trigger https://bugs.chromium.org/p/chromium/issues/detail?id=991197 .)
(OK, as you understand the issue perhaps you should file an Imagemagick bug.)


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
96 [file] General feature always 2019-08-02 20:34 2019-08-05 13:10
Reporter: koala_man Platform:  
Assigned To: christos OS:  
Priority: low OS Version:  
Status: resolved Product Version: 5.37  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: HEAD  
    Target Version:  
Summary: Add support for Hermes compiled JavaScript bytecode
Description: Hi! I am one of the developers of Hermes, a new JavaScript engine for mobile that pre-compiles JavaScript into bytecode. For your consideration, here is magic that can recognize our bytecode files (64bit magic + 32bit integer version, little-endian):

    0 lequad 0x1F1903C103BC1FC6 Hermes compiled JavaScript bytecode,
    >8 lelong x version %d

I think there is a case for upstreaming this because: 1. the use of a 64bit magic number minimizes the risk of false positives, 2. it's convenient for mobile JS to be able to determine a file's bytecode version to check compatibility, and 3. there are already hundreds of millions of mobile devices carrying files in this format.

Please let me know what you think.
Tags: magic
Steps To Reproduce: I have attached `hello.hbc`, a simple `print("Hello World");` compiled into Hermes bytecode using `hermes -O -emit-binary -out hello.hbc hello.js`

Without the suggested magic, file's guess is `data`. With the magic, it says:

    hello.hbc: Hermes compiled JavaScript bytecode, version 60
Additional Information: Hermes is developed by Facebook and was open sourced in July 2019: https://hermesengine.dev/

The bytecode header and magic numbers can be found in: https://github.com/facebook/hermes/blob/a09052e901f4b8dc925e6d4bd0cc47171b90db5e/include/hermes/BCGen/HBC/BytecodeFileFormat.h#L24
Attached Files: hello.hbc (284 bytes) 2019-08-02 20:34
https://bugs.astron.com/file_download.php?file_id=69&type=bug
Notes
(0003282)
christos   
2019-08-05 13:10   
Fixed, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
80 [tcsh] General major always 2019-05-13 18:15 2019-08-02 17:22
Reporter: mnowak Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 6.21.00  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: gethost: ".../tcsh-6.21.00/host.defs", 36: Discarded
Description: Build of tcsh 6.21.00 on OpenIndiana 2019.04 (an illumos distribution) fails with:

./gethost /export/home/newman/ws/oi-userland/components/shell/tcsh/tcsh-6.21.00/host.defs >> tc.defs.c.tmp
gethost: "/export/home/newman/ws/oi-userland/components/shell/tcsh/tcsh-6.21.00/host.defs", 36: Discarded
gethost: "/export/home/newman/ws/oi-userland/components/shell/tcsh/tcsh-6.21.00/host.defs", 47: Discarded
...
gethost: "/export/home/newman/ws/oi-userland/components/shell/tcsh/tcsh-6.21.00/host.defs", 349: Discarded
gethost: "/export/home/newman/ws/oi-userland/components/shell/tcsh/tcsh-6.21.00/host.defs", 350: Discarded
gethost: Too many errors
make[1]: *** [Makefile:455: tc.defs.c] Error 1

Full build output is in the attached file.

tcsh-6.20.00 builds fine.

Let me know, should you need more information.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: tcsh.txt (22,330 bytes) 2019-05-13 18:15
https://bugs.astron.com/file_download.php?file_id=54&type=bug
Notes
(0003253)
mnowak   
2019-06-15 14:09   
Reported also at https://github.com/tcsh-org/tcsh/issues/14.
(0003271)
christos   
2019-08-01 14:03   
I built it with both cc and gcc and no issues:

christos@openindiana:~/src/tcsh$ cc -V
cc: Sun C 5.10 SunOS_i386 2009/06/03
usage: cc [ options] files. Use 'cc -flags' for details
christos@openindiana:~/src/tcsh$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/gcc/8/lib/gcc/i386-pc-solaris2.11/8.3.0/lto-wrapper
Target: i386-pc-solaris2.11
Configured with: /jenkins/jobs/oi-userland/workspace/components/developer/gcc-8/gcc-8.3.0/configure CC=/usr/gcc/6/bin/gcc CXX=/usr/gcc/6/bin/g++ F77=/usr/gcc/6/bin/gfortran FC=/usr/gcc/6/bin/gfortran CFLAGS= CXXFLAGS= FFLAGS=' -m32 -O3 ' FCFLAGS='-m32 -O3 ' LDFLAGS=-m32 PKG_CONFIG_PATH=/usr/lib/pkgconfig --prefix=/usr/gcc/8 --mandir=/usr/gcc/8/share/man --bindir=/usr/gcc/8/bin --libdir=/usr/gcc/8/lib --sbindir=/usr/gcc/8/sbin --sbindir=/usr/gcc/8/bin --libdir=/usr/gcc/8/lib --libexecdir=/usr/gcc/8/lib --host i386-pc-solaris2.11 --build i386-pc-solaris2.11 --target i386-pc-solaris2.11 --with-pkgversion='OpenIndiana 8.3.0-OI-0' --with-bugurl=https://bugs.openindiana.org --enable-plugins --enable-objc-gc --enable-initfini-array --enable-languages=c,c++,fortran,lto,objc --without-gnu-ld --with-ld=/usr/bin/ld --with-build-time-tools=/usr/gnu/i386-pc-solaris2.11/bin --disable-libitm enable_frame_pointer=yes --with-gnu-as --with-as=/usr/bin/gas 'BOOT_CFLAGS=-g -O2' LDFLAGS=-R/usr/gcc/8/lib
Thread model: posix
gcc version 8.3.0 (OpenIndiana 8.3.0-OI-0)
christos@openindiana:~/src/tcsh$
(0003272)
christos   
2019-08-01 14:07   
Cannot reproduce.
(0003274)
mnowak   
2019-08-01 14:47   
Right, thanks for looking into this. GCC 8.3.0 works for me as well. Is GCC 8 now required? By default we use GCC 6.5.0. GCC 7.4.0 fails for me the same way GCC 6 does.
(0003275)
christos   
2019-08-01 15:36   
No, I've built it before with gcc-5, gcc-6, gcc-7... I think something might be wrong with the opensolaris gcc packages of gcc-6/7...
(0003277)
christos   
2019-08-01 16:06   
I will resolve it since it seems like a compiler/environment-specific bug. Don't you agree?
(0003280)
mnowak   
2019-08-02 17:10   
It's actually the -O3 optimization we use by default, with -O2 we are good.

I think you can close this. Thanks!
(0003281)
christos   
2019-08-02 17:22   
gcc optimizer bug.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
88 [tcsh] General minor always 2019-06-27 01:14 2019-08-01 16:05
Reporter: sharifib Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 6.21.00  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: HEAD  
    Target Version:  
Summary: :q does not preserve empty strings
Description: The :q modifier appears to skip empty strings. This makes it very difficult to pass verbatim arguments to another program.
Tags:
Steps To Reproduce: Minimal example:
> echo $tcsh
6.21.00
> set arr= ( '1st arg' '' '3rd arg' )
> echo ${#arr}
3
> tcsh -c 'echo ${#argv}' $arr:q
2
> tcsh -c 'echo ${#argv}' '1st arg' '' '3rd arg'
3

Expected result:
3
3
3
Additional Information: This is not new behavior, but complicates writing scripts that want to maintain and pass a variable number of arguments to other programs, some of which could possibly be empty. For example, something like the env command which may consume some initial arguments but does an exec with the remaining verbatim arguments. In tcsh, attempting to do the same with exec $argv[2-]:q apparently doesn't preserve any of the empty arguments.
Attached Files:
Notes
(0003256)
sharifib   
2019-06-27 01:18   
Another example:
> set arr= ( '1st arg' '' '3rd arg' )
> set arr2= ( $arr:q )
> echo ${#arr}
3
> echo ${#arr2}
2

Expected result:
3
3
(0003276)
christos   
2019-08-01 16:05   
Fixed, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
94 [tcsh] General crash always 2019-07-26 21:38 2019-08-01 14:18
Reporter: swagstafff Platform: Mac  
Assigned To: christos OS: macOS  
Priority: normal OS Version: 10.14  
Status: resolved Product Version: 6.21.00  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: HEAD  
    Target Version:  
Summary: Apple patches not applied
Description: Apple has patched its Mac OS X/macOS tcsh installations [1] since c. 2007 [2]. These patches have never propagated upstream and lead to bugs when not applied on a Mac. Bugs include incorrect HOSTTYPE, OSTYPE, MACHTYPE; and crash with segmentation fault.

[1] https://opensource.apple.com/source/tcsh/
[2] https://opensource.apple.com/source/tcsh/tcsh-60/patches/

Tags:
Steps To Reproduce: host.defs.patch
Unpatched:
> echo $HOSTTYPE $OSTYPE $MACHTYPE
unknown bsd44 unknown

After patch (and expected):
> echo $HOSTTYPE $OSTYPE $MACHTYPE
intel-mac darwin x86_64

Before config_f.h.patch:
> nonexistent > /dev/null
nonexistent: Command not found.
Segmentation fault
Additional Information: The uploaded files incorporate Apple's latest patches ("tcsh-67") [3], with some adaptation for application to tcsh 6.21.00 and corrections for correct hosttype on Intel Macs. The patches have been tested on macOS 10.12.6 and 10.14.5.

[3] https://opensource.apple.com/source/tcsh/tcsh-67/patches/
Attached Files: tc.sig.h.patch (456 bytes) 2019-07-26 21:38
https://bugs.astron.com/file_download.php?file_id=67&type=bug
host.defs.patch (678 bytes) 2019-07-26 21:38
https://bugs.astron.com/file_download.php?file_id=66&type=bug
config_f.h.patch (279 bytes) 2019-07-26 21:38
https://bugs.astron.com/file_download.php?file_id=65&type=bug
Notes
(0003273)
christos   
2019-08-01 14:18   
Applied (except the KAI one, which is not appropriate).


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
58 [tcsh] General minor N/A 2019-01-30 01:12 2019-08-01 07:29
Reporter: PesoMemo Platform: PC  
Assigned To: christos OS: Windows  
Priority: normal OS Version: 10  
Status: feedback Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Unable to find pre-compiled tcsh64.exe or tcsh86.exe for tcsh version 6.20.00
Description: I've searched for hours to no avail - does anyone know where I can D/L a Windows 10 binary x64/x86 of tcsh 6.20.00 ?
If anyone has a copy can they post it to a URL/share site where I can D/L it?

TIA, Bill

PS: You can email me at PesoMemo <atsign> gmail.com
Tags: tcsh64.exe tcsh86.exe tcsh 6.20.00 windows
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0003270)
christos   
2019-08-01 07:29   
Sorry, I don't have a setup to build. Amol, used to provide the binaries. Please ask in the mailing lists.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
81 [tcsh] General major always 2019-05-17 13:58 2019-07-29 13:05
Reporter: oldpink Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 6.21.00  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: HEAD  
    Target Version:  
Summary: Filename expansion inside square brackets is apparently broken
Description: I have been running TCSH version 6.21.00 on my Slackware system since Patrick Volkderding made the new build of TCSH part of Slackware-current on May 10.
However, I ran into a really nasty issue with this new version of TCSH that breaks filename pattern matching within square brackets, specifically, I tried upgrading (via upgradepkg) several of my Slackware packages selectively by excluding some of the packages that I had downloaded so that I could install the excluded packages as additions instead of upgrading them.
I excluded the kernel packages, using the following command: upgradepkg [b-c,l-z]*.t?z
However, much to my dismay, it went ahead and upgraded the kernel-* packages anyway, deleting my three already installed kernel packages that I wanted to keep in place as secondary bootup and backups.
Sure enough, when I type "ls [b-c,l-z]*.t?z, it includes the kernel-* files in the list of files, too.
Obviously, this is not the way it should work, so I downloaded version 6.21.00 from source, then compiled it using the Slackware build script (called a SlackBuild) to see if reverting the previous kernel would correct the error, and Shazam! yes it works as it should, properly excluding the kernel-* files in my filespec.
I'm certain that this is a bug, and a very bad one at that, so I'll be using the previous version of TCSH for now, but I wanted to let you know about my problem with the current version so that maybe something could be done before someone unintentionally wipes out an entire file system or something else because of this bug.
Tags:
Steps To Reproduce: Here is the list of files I'm trying to exclude, with no other files in the directory where they are:
kernel-huge-4.19.44-x86_64-1.txz
kernel-source-4.19.44-noarch-1.txz
kernel-modules-4.19.44-x86_64-1.txz

Using Bash, I get the proper response with this command: ls [a-d,l-z]*
ls: cannot access '[a-d,l-z]*': No such file or directory

Using version 6.20.00 of TCSH: ls [a-d,l-z]*
/bin/ls: No match.

Using the buggy version 6.21.00 version of TCSH: ls [a-d,l-z]*
kernel-huge-4.19.44-x86_64-1.txz kernel-source-4.19.44-noarch-1.txz
kernel-modules-4.19.44-x86_64-1.txz

Clearly something is not right.
Additional Information:
Attached Files:
Notes
(0003246)
oldpink   
2019-05-17 14:04   
I made a typo, and I can't find a way to edit my original bug report, so I'm writing my correction down here in the note box to explain that I downloaded the source and built version 6.20.00, NOT 6.21.00 to see if the bug went away by using the previous TCSH.
Sorry about my error.
(0003269)
christos   
2019-07-29 13:05   
Fixed, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
93 [file] General minor always 2019-07-17 17:52 2019-07-23 08:51
Reporter: iaeiaeiaeiae@byom.de Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: assigned Product Version: 5.37  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: file incorrectly recognizes -static-pie binaries
Description: file recognizes binaries build with gcc using -static-pie as dynamically linked binaries.
Tags:
Steps To Reproduce: 1. Create a simple hello word C program
2. Compile it using gcc -fPIE -static-pie -o test test.c
3. Run file on the resulting binary

Output:

test: ELF 64-bit LSB pie executable, x86-64, version 1 (GNU/Linux), dynamically linked, BuildID[sha1]=4594afb38f3f60aaad10d08ab519fcaeae55ee70, for GNU/Linux 3.2.0, not stripped

It should actually be reported as statically linked, check with ldd:

$ ldd test
    statically linked

Additional Information:
Attached Files: testbin (806,560 bytes) 2019-07-22 18:29
https://bugs.astron.com/file_download.php?file_id=64&type=bug
Notes
(0003260)
christos   
2019-07-21 09:05   
I can't reproduce this

[5:04am] 10020>cc -static -fPIE -pie hello.c -o hello
[5:04am] 10021>./file -m ../magic/magic.mgc ./hello
./hello: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.32, BuildID[sha1]=f864bee36ab81bd40145389b987e8fe6b6d92696, not stripped
[5:04am] 10022>./file --version
file-5.37
magic file from /usr/local/share/misc/magic
(0003266)
iaeiaeiaeiae@byom.de   
2019-07-22 18:29   
"-static -pie" and "-static-pie" are not the same thing. You need a relatively recent gcc for the latter https://gcc.gnu.org/viewcvs/gcc?view=revision&revision=252034

I used gcc (GCC) 9.1.0.
(0003267)
christos   
2019-07-22 20:43   
I don't have an environment like that available. Can you just attach a binary?
(0003268)
iaeiaeiaeiae@byom.de   
2019-07-23 08:51   
The previous comment contained a binary.

https://bugs.astron.com/file_download.php?file_id=64&type=bug


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
68 [file] General feature have not tried 2019-02-24 16:23 2019-07-21 18:31
Reporter: valoq Platform:  
Assigned To: OS:  
Priority: normal OS Version:  
Status: new Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Native decompression formats
Description: Note: This is a copy of the original issue (https://bugs.astron.com/view.php?id=3) without the spam.


Currently file uses external applications to decompress certain file formats before analysing them.
This prevents effective sandboxing via seccomp.

This bug is to collect information and keep track of the progress of implementing all compression formats using libraries

Currently we have the folloging native decompression functions:
uncompressgzipped
uncompresszlib


The following external decompression tools are currently used and need to be implemented using their respective libraries:

gzip
uncompress
bzip2
lzip
xz
lrzip
lz4
zstd


Additional information:

most of the necessary source code can be found here:
https://nxr.netbsd.org/search?q=&project=src&defs=&refs=&path=usr.bin%2Fgzip&hist=
(thanks christos)
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0003223)
valoq   
2019-02-24 16:27   
@christos:
Do you want all compression to be handled within compress.c (using only this one file) or can we use additional files for each compression algorithm?

Also, it seems I cannot close my own issues, so if possible please close the old one: https://bugs.astron.com/view.php?id=3
(0003224)
christos   
2019-02-24 18:09   
I think each compression scheme can come in its own file and this way we can probably use an adapter so we don't have to modify all sources.
(0003226)
valoq   
2019-02-24 22:18   
After taking a look at the bsd implementations, I think we can at least use the xz (unxz.c) decompression and the bzip (unbzip2.c) decompression code.
                                                                                                                                                                                                                
One issue remaining is that they use file descriptors for in and output and not strings.
I am not quite sure how we should adapt the code to be used by file.
Also I still don't completely understand some of the decompression handling of file.
Since I am not an expert on C I am rather reluctant to mess with stuff like that, especially since it involves a widely used program like file.

Nonetheless I am looking to push this to get a working implementation of seccomp into file (Debian buster just disabled seccomp as well :( )
(0003228)
valoq   
2019-03-03 14:56   
@christos:
Can you please provide the adapter to allow for easier adoption of third party compression code. That would be a great help.
(0003265)
christos   
2019-07-21 18:31   
bzip, lzma, xz added by Christoph Biedl


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
92 [file] General minor always 2019-07-17 14:46 2019-07-21 09:59
Reporter: ulm Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.37  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: HEAD  
    Target Version:  
Summary: file-5.37 falsely identifies text file (PSF license) as audio/x-psf
Description: $ file -i PSF-2.2
PSF-2.2: audio/x-psf; charset=us-ascii
$ head -n1 PSF-2.2
PSF LICENSE AGREEMENT FOR PYTHON 2.2

file-5.36 had correctly identified the same file as "text/plain; charset=us-ascii".

Full file is here: https://gitweb.gentoo.org/repo/gentoo.git/plain/licenses/PSF-2.2
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0003264)
christos   
2019-07-21 09:59   
Fixed, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
91 [file] General minor always 2019-07-09 02:20 2019-07-21 09:40
Reporter: avellable Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.33  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: HEAD  
    Target Version:  
Summary: Ruby files are detected properly when require contains '.'
Description: Magic test fails to detect ruby file if there's a '.' in "require" statement. That is a valid statement.
Tags:
Steps To Reproduce: Sample File

1. Create a file application.rb
# application.rb
require './something.rb'

2. Run file on it
$ file application.rb
application.rb: ASCII text

Expected Result:
application.rb: Ruby script text, ASCII text

Actual Result:
application.rb: ASCII text

Additional Information: Solution:

The regex needs to include periods('.') too,

See attached fixed file for reference. When used,

$ file -m new_ruby application.rb
application.rb: Ruby script text, ASCII text

$ diff new_ruby /usr/share/file/magic/ruby
25c25
< 0 regex \^[[:space:]]*require[[:space:]]'[A-Za-z_/\.]+'
---
> 0 regex \^[[:space:]]*require[[:space:]]'[A-Za-z_/]+'
48c48
< 0 regex \^[[:space:]]*require[[:space:]]'[A-Za-z_/\.]+' Ruby script text
---
> 0 regex \^[[:space:]]*require[[:space:]]'[A-Za-z_/]+' Ruby script text
Attached Files: new_ruby (1,812 bytes) 2019-07-09 02:20
https://bugs.astron.com/file_download.php?file_id=63&type=bug
Notes
(0003259)
avellable   
2019-07-09 02:21   
I would be more than happy to create a patch. Is there a document about how to contribute?
(0003263)
christos   
2019-07-21 09:40   
Fixed, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
89 [file] General minor have not tried 2019-06-28 15:58 2019-07-21 09:22
Reporter: vitalyisaev2 Platform: amd64  
Assigned To: christos OS: Linux  
Priority: normal OS Version: Fedora 29  
Status: feedback Product Version: 5.37  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: MIME Types for PKI files
Description: I believe that libmagic should support MIME type detection for PKI files, like ".key", ".crt" and so on:

application/pkcs8 .p8 .key
application/pkcs10 .p10 .csr
application/pkix-cert .cer
application/pkix-crl .crl
application/pkcs7-mime .p7c
application/x-x509-ca-cert .crt .der
application/x-x509-user-cert .crt
application/x-pkcs7-crl .crl
application/x-pem-file .pem
application/x-pkcs12 .p12 .pfx
application/x-pkcs7-certificates .p7b .spc
application/x-pkcs7-certreqresp .p7r

Please see full list here: https://pki-tutorial.readthedocs.io/en/latest/mime.html
Tags:
Steps To Reproduce: Currently some of certificates are determined as "plain/text":

➜ file --mime-type ca.crt
ca.crt: text/plain
➜ file ca.crt
ca.crt: PEM certificate

Additional Information:
Attached Files: ca.crt (1,302 bytes) 2019-06-28 15:58
https://bugs.astron.com/file_download.php?file_id=58&type=bug
Notes
(0003261)
christos   
2019-07-21 09:22   
Can you try writing a patch for this? I am not sure if we can cover most of the cases because we don't know enough about what the files contain (and file does not pay attention to extensions).


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
82 [file] General feature N/A 2019-05-25 23:03 2019-07-04 18:54
Reporter: GerbilSoft Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.37  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: HEAD  
    Target Version:  
Summary: Xbox, Xbox 360, VGM, and SNDH improvements
Description: This is a small patch set with the following improvements:

Audio:
- VGM: Added YMZ284 and YMZ294 variants for AY-3-8910.
- SNDH: Added a check for ICE-compressed files.

Xbox:
- Added MIME types and file extensions for XBE and XEX files.
- XEX: Display the 32-bit media ID.
- XEX: Support XEX1 format. (early devkits)
- Added Xbox 360 packages. (from Xbox Live and/or system updates)
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: file.2019-05-25.19-00.tar.gz (3,282 bytes) 2019-05-25 23:03
https://bugs.astron.com/file_download.php?file_id=55&type=bug
Notes
(0003248)
christos   
2019-05-27 01:35   
Added, thanks.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
84 [file] General minor have not tried 2019-06-03 11:35 2019-06-28 15:48
Reporter: vitalyisaev2 Platform: amd64  
Assigned To: christos OS: Linux  
Priority: normal OS Version: Fedora 29  
Status: assigned Product Version: 5.36  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Wrong mime-type for audio wma file
Description: Hello, could you please check this audio file. It's defined as `video/x-ms-asf`, though it contains only music.

$ file Amy\&Amethyst.wma --mime-type
Amy&Amethyst.wma: video/x-ms-asf
$ file --version
file-5.36
magic file from /etc/magic:/usr/share/misc/magic
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: Amy&Amethyst.wma (140,547 bytes) 2019-06-03 11:35
https://bugs.astron.com/file_download.php?file_id=57&type=bug
Notes
(0003249)
vitalyisaev2   
2019-06-03 12:17   
Expected: audio/x-ms-wma
(0003252)
christos   
2019-06-08 22:41   
ASF is a container format that can contain various media files such as WMA (https://en.wikipedia.org/wiki/Advanced_Systems_Format)
file(1) does not know how to look inside (yet)
(0003257)
vitalyisaev2   
2019-06-28 15:48   
If I find some who is aware of video streaming / codec, I will try to ask, how they detect what's inside ASF.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
86 [file] General minor always 2019-06-09 13:00 2019-06-16 00:16
Reporter: Mababa Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: feedback Product Version: 5.36  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: wrong type reported for text files beginning with the characters "BIK"
Description: Any text file where the first three characters are "BIK" are reported as "Bink Video". The more text that follows these initial characters, the more bogus information "file" seems to report.

$ echo BIK hello | file -
/dev/stdin: Bink Video rev.h, 0 audio track
$ echo BIKING to work | file -
/dev/stdin: Bink Video rev.t, 1870078063 frames, 0 audio track
$ echo BIKes have more fun when they sleep | file -
/dev/stdin: Bink Video rev.e, 1852139639x1701344288, 1830839670 frames at rate 175138149/1819484281, 0 audio track
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0003255)
christos   
2019-06-16 00:16   
I made it a bit stronger.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
87 [file] General feature N/A 2019-06-12 17:56 2019-06-16 00:08
Reporter: dbrezack Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: feedback Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Support for Flattened Device Tree files (fdt)
Description: I think it would be helpful if the file command supported Flattened Device Tree files (fdt).

The file magic is "d0 0d fe ed"

These resources may be helpful
https://wiki.freebsd.org/FlattenedDeviceTree
https://github.com/superna9999/pyfdt

Thanks
Data Brezack
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0003254)
christos   
2019-06-16 00:08   
You mean dtb and it already does:

sun50i-h5-nanopi-neo-plus2.dtb: Device Tree Blob version 17, size=25978, boot CPU=0, string block size=1870, DT structure block size=23028


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
83 [file] General minor always 2019-05-30 20:35 2019-06-08 22:23
Reporter: j2j Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: feedback Product Version: 5.37  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Too many false hits "(Lepton " by Magdir/measure
Description: Hello,

when i run file command version 5.37 and some earlier versions on many
files with -k option i often get also misidentification messages
starting with "(Lepton 3.x)" or "(Lepton 2.x)". See appended output
fileLepton.txt.

When looking inside sources i see that such messages are triggered by
magic lines inside Magdir/measure. These magic lines should identify
DIY-Thermocam raw data files.

Apparently such file have no real magic patterns. So test for values in
some range is done by lines like
     9608 ubyte <19
     >9 ubyte <2

But these test lines are too generic or not precise. These tests are also
matched by ISO 9660 CD-ROM images and many other file types.
So more test lines are needed.

Unfortunately no contributor name or web site is mentioned. So i have no
information about that file format to fix weak magic lines.
Tags: magic
Steps To Reproduce:
Additional Information:
Attached Files: fileLepton.txt (34,203 bytes) 2019-05-30 20:35
https://bugs.astron.com/file_download.php?file_id=56&type=bug
Notes
(0003251)
christos   
2019-06-08 22:23   
Do you have a file I can check against? It does try 5 different byte values in the checker...


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
85 [file] General minor have not tried 2019-06-05 11:09 2019-06-08 22:18
Reporter: vitalyisaev2 Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.36  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: HEAD  
    Target Version:  
Summary: Support CSV detection
Description: It would be nice if we could distinguish csv files from other ASCII text files. It may be tricky, but perhaps we can rely on the number of commas and other widely used delimiters. If the amount of delimiters is same for the first N lines, it's like to be a csv file.

This tread may be also useful: https://www.unix.com/shell-programming-and-scripting/25477-how-validate-csv-file.html

Kind regards,
Vitaly
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0003250)
christos   
2019-06-08 22:18   
Added...


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
79 [file] General feature N/A 2019-05-10 03:52 2019-05-27 01:28
Reporter: cisba Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: HEAD  
    Target Version:  
Summary: [PATCH] add OpenTimestamps related magic entries
Description: https://opentimestamps.org/
https://en.wikipedia.org/wiki/OpenTimestamps
Tags: magic
Steps To Reproduce: 1. create a file .ots using the web interface at https://opentimestamps.org/
2. download the attacched file "opentimestamps"
3. execute "file -m opentimestamps <your-ots-file-name>.ots"
Additional Information:
Attached Files: opentimestamps (647 bytes) 2019-05-10 03:52
https://bugs.astron.com/file_download.php?file_id=53&type=bug
Notes
(0003247)
christos   
2019-05-27 01:28   
Added, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
74 [tcsh] General major always 2019-03-25 09:51 2019-05-08 18:06
Reporter: bitstreamout Platform:  
Assigned To: christos OS: Linux  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 6.21.00  
    Target Version:  
Summary: Bug in tcsh postcmd alias handling
Description: The postcmd alias destroys boolean handling withing loops
Tags:
Steps To Reproduce: simple script to test this on tcsh 6.18.0 as well as on 6.20.0

 # !/bin/tcsh
 alias postcmd false

 set counter=5

 while ($counter > 0)
   @ counter--
   echo $counter
   sleep 1
 end
 exit 0

... this should break for counter getting 0 but it does not
Additional Information:
Attached Files: debug (1,613 bytes) 2019-04-05 09:16
https://bugs.astron.com/file_download.php?file_id=47&type=bug
tcsh-postcmd.patch (3,043 bytes) 2019-04-08 11:38
https://bugs.astron.com/file_download.php?file_id=48&type=bug
tcsh-6.20.00-postcmd.patch (1,421 bytes) 2019-04-09 07:25
https://bugs.astron.com/file_download.php?file_id=49&type=bug
Notes
(0003232)
bitstreamout   
2019-04-05 09:16   
Hmmm .. .after some debugging it looks like the function dowhile() of sh.func.c is executed repeated if no postcmd alias is set whereas if one is set the dowhile() is reached only once and does never finish
(0003233)
bitstreamout   
2019-04-05 12:56   
The bug is also in tcsh-6.16.00
(0003238)
bitstreamout   
2019-04-08 11:38   
Looks like aliasrun() does destroy the current parse tree .... at least this patch make the test case work
(0003239)
bitstreamout   
2019-04-09 07:25   
This one avoids breaking the testsuit
(0003245)
christos   
2019-05-08 18:06   
Thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
78 [file] General minor have not tried 2019-05-07 11:32 2019-05-08 18:02
Reporter: jpcima Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.37  
    Target Version:  
Summary: [PATCH] add BambooTracker instrument, and version number detection
Description: Patch: https://github.com/file/file/compare/master...jpcima:bamboo-tracker.diff

Adds support for BTI files (instruments).
Extracts the version number from uint32 BCD field.
-> specification at https://github.com/rerrahkr/BambooTracker/tree/master/specs
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: Echo Piano Bell.bti (106 bytes) 2019-05-07 11:32
https://bugs.astron.com/file_download.php?file_id=52&type=bug
Notes
(0003244)
christos   
2019-05-08 18:02   
thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
77 [file] General minor always 2019-04-29 13:35 2019-05-06 21:24
Reporter: yarikoptic Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: feedback Product Version: 5.35  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: -k, --keep-going does not list other hits - the other one might come out empty
Description: Detected while trying to workaround recently added detection of .json as application/json and not as text/plain. I thought that I would obtain both new application/json and text/plain if I use --keep-going . But the 2nd entry comes out to be blank.

As you can see below it is not happening for some other files, and is not unique to json
Tags:
Steps To Reproduce: /tmp > file --version
file-5.35
magic file from /etc/magic:/usr/share/misc/magic

/tmp > echo '{"a": 1}' >| file.json

/tmp > file --mime-type file.json
file.json: application/json

/tmp > file --mime-type -k file.json
file.json: application/json\012- \012-

/tmp > file --mime-type -k /usr/bin/* | head
/usr/bin/[: application/x-pie-executable\012- application/octet-stream
/usr/bin/0alias: application/x-pie-executable\012- application/octet-stream
/usr/bin/0desktop: application/x-pie-executable\012- application/octet-stream
/usr/bin/0install: application/x-pie-executable\012- application/octet-stream
/usr/bin/0launch: application/x-pie-executable\012- application/octet-stream
/usr/bin/0store: application/x-pie-executable\012- application/octet-stream
/usr/bin/0store-secure-add: application/x-pie-executable\012- application/octet-stream
/usr/bin/2to3: text/x-python\012-
/usr/bin/2to3-2.7: text/x-python\012-
/usr/bin/2to3-3.4: text/x-python\012-


/tmp > file --mime-type -k /usr/bin/* > /tmp/usr-bin-mime-types
/tmp > grep '\012- .*\012' /tmp/usr-bin-mime-types | head
/usr/bin/ack: text/x-perl\012- text/x-c\012-
/usr/bin/aclocal-1.16: text/x-perl\012- \012-
/usr/bin/afmtodit: text/x-perl\012- \012-
/usr/bin/apt-file: text/x-perl\012- \012-
/usr/bin/apt-rdepends: text/x-perl\012- \012-
/usr/bin/apxs: text/x-perl\012- \012-
/usr/bin/aspell-import: text/x-perl\012- \012-
/usr/bin/autoheader: text/x-perl\012- \012-
/usr/bin/autom4te: text/x-perl\012- \012-
/usr/bin/automake-1.16: text/x-perl\012- \012-
Additional Information:
Attached Files:
Notes
(0003243)
christos   
2019-05-06 21:24   
Fixed on HEAD.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
73 [file] General minor always 2019-03-20 09:21 2019-04-09 18:34
Reporter: enkeli Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.36  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.37  
    Target Version:  
Summary: Reports Zip archive data for data with only one PK central dir signature
Description: File reports for the attached file "Zip archive data".

$ file data-not-zip
data-not-zip: Zip archive data

The only ZIP-like thing it contains is 0x06054b50 and it does not seem sufficient to me to mark it as ZIP.

File hexdump:
00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................

file v. 5.30 reports "data" which seems more precise to me
00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000040: 0000 0000 0000 0000 0050 4b05 0600 0000 .........PK.....
00000050: 0000 0000 0000 0000 0000 0000 0000 000a ................
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: data-not-zip (408 bytes) 2019-03-20 09:21
https://bugs.astron.com/file_download.php?file_id=46&type=bug
data-not-zip-2 (102 bytes) 2019-04-09 09:32
https://bugs.astron.com/file_download.php?file_id=51&type=bug
Notes
(0003235)
christos   
2019-04-07 18:23   
Can't reproduce it:

[2:20pm] 2551>xxd -r ../data-not-zip >! ../data
[2:20pm] 2552>src/file -m magic/magic.mgc data
data: data
[2:20pm] 2553>hexdump -C data
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000040 00 00 00 00 00 00 00 00 00 50 4b 05 06 00 00 00 |.........PK.....|
00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0a |................|
00000060
[2:20pm] 2554>src/file -v
file-5.36
magic file from /usr/local/share/misc/magic
(0003240)
enkeli   
2019-04-09 09:32   
Sorry, I've probably uploaded wrong file.

This one is definitely reported as ZIP (close this issue if I'm wrong). I'd expect "data" or something other than ZIP.

$ sha256sum data-not-zip
a95a3e433275508dfbcfbea7ba83ff6b69d582be25dd0f3a87e2e9739b735d34 data-not-zip
$ file --version
file-5.36
magic file from /usr/share/file/misc/magic
$ file data-not-zip
data-not-zip: Zip archive data
(0003242)
christos   
2019-04-09 18:34   
Fixed, thanks


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
76 [file] General minor always 2019-04-09 09:18 2019-04-09 18:29
Reporter: mohd-akram Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.36  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.37  
    Target Version:  
Summary: [PATCH] Fix Python 3.7 byte-compiled detection
Description: Should be checking 0x420d0d0a not 0x3e0d0d0a. See https://github.com/python/cpython/blob/3.7/Lib/importlib/_bootstrap_external.py#L259.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: patch-python37.diff (466 bytes) 2019-04-09 09:18
https://bugs.astron.com/file_download.php?file_id=50&type=bug
Notes
(0003241)
christos   
2019-04-09 18:29   
Fixed, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
71 [file] General major always 2019-03-12 13:10 2019-04-07 18:28
Reporter: edurdo100 Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: feedback Product Version: 5.33  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Problem with encoding of file larger than 262145 characterers
Description: Whe run command file with the next options:

file -bi $fichero
In case than file larger than 262145 encoding always appear us-ascii, when the file has the encoding iso-8859-1
Tags:
Steps To Reproduce: Create a file encoding like iso-8859-1 and more than 262145 characterers
Run command:
file -bi $fichero
Additional Information:
Attached Files:
Notes
(0003237)
christos   
2019-04-07 18:28   
Can you attach an example? Does the behavior change when you use -P bytes to a large number?


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
72 [file] General minor always 2019-03-20 01:54 2019-04-07 18:26
Reporter: tduffy Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.36  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.37  
    Target Version:  
Summary: magic/Magdir/apple reports as data (instead of ASCII)
Description: $ file magic/Magdir/apple
magic/Magdir/apple: data

Looks like there are some bogus bytes on line 91:

#:apple pdosp^Zøÿ
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0003236)
christos   
2019-04-07 18:26   
Fixed, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
75 [file] General minor always 2019-03-31 19:16 2019-04-07 18:05
Reporter: wylda Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.36  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.37  
    Target Version:  
Summary: file-5.36: configure --enable-zlib actually disables Zlib
Description: If "--enable-zlib" is explicitly given for configure, then "ZLIBSUPPORT" won't be set to 1, even if in my case I have:
enable_zlib=yes
ac_cv_header_zlib_h=yes
ac_cv_lib_z_gzopen=yes
I'm not a programmer, but i guess it is because of "elif" instead of "if"???

---snip---
if test "$enable_zlib" = "yes"; then
  if test "$ac_cv_header_zlib_h$ac_cv_lib_z_gzopen" != "yesyes"; then
    as_fn_error $? "zlib support requested but not found" "$LINENO" 5
  fi
elif test "$ac_cv_header_zlib_h$ac_cv_lib_z_gzopen" = "yesyes"; then
  $as_echo "#define ZLIBSUPPORT 1" >>confdefs.h
fi
---snip---
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0003234)
christos   
2019-04-07 18:05   
Fixed, thanks.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
70 [file] General feature always 2019-03-05 17:33 2019-03-07 17:22
Reporter: v3l0c1r4pt0r Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.37  
    Target Version:  
Summary: [PATCH] Add AIX backup and package format detection
Description: I have noticed that AIX default package format is not identified by file ("data" returned). I am enclosing the patch.

I've managed to find example package with first signature variant (could be downloaded from IBM after signing up). Unfortunately I didn't find any example for the second form (confirmed only with Wikipedia and original AIX /etc/magic).
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: 0001-Add-AIX-package-and-backup-file-magic.patch (766 bytes) 2019-03-05 17:33
https://bugs.astron.com/file_download.php?file_id=45&type=bug
Notes
(0003229)
christos   
2019-03-07 13:22   
both magic entries have the same description, is that correct?
(0003230)
v3l0c1r4pt0r   
2019-03-07 14:41   
Yes, that's correct. I haven't seen any explanation of what is the difference.
(0003231)
christos   
2019-03-07 17:22   
Committed, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
69 [file] General major always 2019-02-27 16:04 2019-03-02 01:08
Reporter: vinc17 Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.35  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.37  
    Target Version:  
Summary: mistakes Xorg.0.log file as JSON data
Description: file mistakes Xorg.0.log file as JSON data (which breaks viewing with "less" + LESSOPEN).
Tags:
Steps To Reproduce: Example on the first 5 lines:

[105215.998]
X.Org X Server 1.20.3
X Protocol Version 11, Revision 0
[105215.998] Build Operating System: Linux 4.9.0-8-amd64 x86_64 Debian
[105215.998] Current Operating System: Linux cventin 4.19.0-3-amd64 0000001 SMP Debian 4.19.20-1 (2019-02-11) x86_64

$ file Xorg.0.log
Xorg.0.log: JSON data
Additional Information: Also reported on the Debian BTS: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=922874
Attached Files:
Notes
(0003227)
christos   
2019-03-02 01:08   
Changed to be stricter.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
67 [file] General minor have not tried 2019-02-22 14:15 2019-02-23 03:10
Reporter: valoq Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Fix bugtracker links on website
Description: The official file website https://www.darwinsys.com/file/ have several broken links that point to gw.com which is not available.

The bugtracker link points to http://bugs.gw.com/ which is also dead

Please fix the linksto point to this bugtracker instead.
This bugtracker is pretty hard to find otherwise.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0003222)
christos   
2019-02-23 03:10   
Ian Darwin fixed it.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
66 [file] General feature N/A 2019-02-20 23:47 2019-02-23 01:17
Reporter: GerbilSoft Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.37  
    Target Version:  
Summary: Another batch of patches for various game console formats
Description: This patchset adds the following formats:
- Scaleform video (basic identification only)
- PowerVR 3.0 textures
- Portable Sound Format (PSF)
- SAP (Atari 8-bit audio)
- Nintendo audio formats: BRSTM, BCSTM, BFSTM, BCWAV
- Xbox XPR0 textures
- Xbox 360 executables (XEX)

Other changes:
- ADX: Added a MIME type; prevent conflicts with Targa images; fix AHX and add AHX (Dreamcast).
- GameCube, Wii: Added unofficial MIME types.
- Xbox executables (XBE): Show the game title and title ID.
- Khronos KTX: Correctly escape the '^' for khronos-ktx-endian-header.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: file.2019-02-20.18-40.tar.gz (9,531 bytes) 2019-02-20 23:47
https://bugs.astron.com/file_download.php?file_id=44&type=bug
Notes
(0003221)
christos   
2019-02-23 01:17   
Committed, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
61 [file] General minor always 2019-02-15 04:34 2019-02-19 20:35
Reporter: tmc Platform:  
Assigned To: christos OS: GNU/Linux  
Priority: normal OS Version:  
Status: feedback Product Version: 5.35  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Line endings misdetected for UTF16 files
Description: file misidentifies the line endings of little-endian UTF16 files (running on x86_64), while big-endian seems to work fine.
Also it doesn't seem to even try to to identify the line ending of an UTF32 file.

Also, I notice that file can't recognise UTF16/32 files without a BOM, which is a pity.
Tags:
Steps To Reproduce: file misdetects utf16le with CRLF line endings sometimes as CR line endings:

> printf "\xff\xfe\r\0\n\0" |file -
/dev/stdin: Little-endian UTF-16 Unicode text, with CR line terminators

...and sometimes as mixed CR, CRLF line endings:

> printf "\xff\xfe\r\0\n\0\r\0\n\0" |file -
/dev/stdin: Little-endian UTF-16 Unicode text, with CRLF, CR line terminators

Maybe it's skipping over the final \n? This example reinforces that idea:

> printf "\xff\xfe\n\0\r\0" | file -
/dev/stdin: Little-endian UTF-16 Unicode text

utf16be doesn't seem to show this problem:

> printf "\xfe\xff\0\r\0\n" | file -
/dev/stdin: Big-endian UTF-16 Unicode text, with CRLF line terminators

UTF32 line endings not identified:

> printf "\xff\xfe\0\0\r\0\0\0B\0\0\0" | file -
/dev/stdin: Unicode text, UTF-32, little-endian

Additional Information:
Attached Files:
Notes
(0003210)
christos   
2019-02-18 17:04   
Yes, the utf-16 detection had length issues, which have been fixed on HEAD. Now the utf-32 detection is not built-in and it is done in regular magic, this is why it does not get the CR/LF info right. I guess we should move the utf-32 detection to be built-in for consistency.
Here's the latest output:

[11:59am] 2601>cat uni
#!/bin/sh
doit() {
        printf "$1" | ./file -m ../magic//magic.mgc -
}

doit "\xff\xfe\r\0\n\0"
doit "\xff\xfe\r\0\n\0\r\0\n\0"
doit "\xff\xfe\n\0\r\0"
doit "\xfe\xff\0\r\0\n"
doit "\xff\xfe\0\0\r\0\0\0B\0\0\0"
[11:59am] 2602>./uni
/dev/stdin: Little-endian UTF-16 Unicode text, with CRLF line terminators
/dev/stdin: Little-endian UTF-16 Unicode text, with CRLF line terminators
/dev/stdin: Little-endian UTF-16 Unicode text, with CR, LF line terminators
/dev/stdin: Big-endian UTF-16 Unicode text, with CRLF line terminators
/dev/stdin: Unicode text, UTF-32, little-endian
(0003220)
christos   
2019-02-19 20:35   
Should be fixed now:
#!/bin/sh
doit() {
        printf "$1" | ./file -m /dev/null -
}

doit "\xff\xfe\r\0\n\0"
doit "\xff\xfe\r\0\n\0\r\0\n\0"
doit "\xff\xfe\n\0\r\0"
doit "\xfe\xff\0\r\0\n"
doit "\xff\xfe\0\0\r\0\0\0B\0\0\0"
doit "\xff\xfe\0\0\r\0\0\0\n\0\0\0"
doit "\0\0\xfe\xff\0\0\0\r\0\0\0\n"


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
64 [file] General major always 2019-02-18 08:50 2019-02-19 13:21
Reporter: spinpx Platform: x86_64  
Assigned To: christos OS: Debian  
Priority: urgent OS Version: 10  
Status: resolved Product Version: 5.35  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: ASAN: memcpy-param-overlap
Description: 描述 We build file with `--disable-libseccomp` by clang 4.0.0 and ASAN.
We ran the program with the input we provide without any other arguments.

The bugs exists in file 5.35.

ASAN report:
==1129930==ERROR: AddressSanitizer: memcpy-param-overlap: memory ranges [0x7ffcc4a0f360,0x7ffcc4a10861) and [0x7ffcc4a104f8, 0x7ffcc4a119f9) overlap
    #0 0x4add33 in __asan_memcpy /scratch/llvm/clang-4/xenial/final/llvm.src/projects/compiler-rt/lib/asan/asan_interceptors.cc:453:3
    0000001 0x54f86c in do_core_note /mnt/raid/user/chenpeng/FuzzingBench/file/file/src/readelf.c:755:4
    0000002 0x54d323 in donote /mnt/raid/user/chenpeng/FuzzingBench/file/file/src/readelf.c:1194:7
    0000003 0x54792a in dophn_core /mnt/raid/user/chenpeng/FuzzingBench/file/file/src/readelf.c:398:13
    0000004 0x5451b4 in file_tryelf /mnt/raid/user/chenpeng/FuzzingBench/file/file/src/elfclass.h:43:7
    0000005 0x51f29b in file_buffer /mnt/raid/user/chenpeng/FuzzingBench/file/file/src/funcs.c:305:8
    0000006 0x4f5b5d in file_or_fd /mnt/raid/user/chenpeng/FuzzingBench/file/file/src/magic.c:508:6
    0000007 0x4f5cd6 in magic_file /mnt/raid/user/chenpeng/FuzzingBench/file/file/src/magic.c:397:9
    0000008 0x4f3fd5 in process /mnt/raid/user/chenpeng/FuzzingBench/file/file/src/file.c:546:9
    #9 0x4f1c4b in main /mnt/raid/user/chenpeng/FuzzingBench/file/file/src/file.c:416:9
    0000010 0x7fbb0e64b09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)
    0000011 0x41d689 in _start (/mnt/raid/user/chenpeng/FuzzingBench/build/asan/install/bin/file+0x41d689)

Address 0x7ffcc4a0f360 is located in stack of thread T0 at offset 608 in frame
    #0 0x54f33f in do_core_note /mnt/raid/user/chenpeng/FuzzingBench/file/file/src/readelf.c:710

  This frame has 2 object(s):
    [32, 544) 'sbuf'
    [608, 768) 'pi' <== Memory access at offset 608 partially overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
Address 0x7ffcc4a104f8 is located in stack of thread T0 at offset 1592 in frame
    #0 0x546f7f in dophn_core /mnt/raid/user/chenpeng/FuzzingBench/file/file/src/readelf.c:346

  This frame has 3 object(s):
    [32, 64) 'ph32'
    [96, 152) 'ph64'
    [192, 8384) 'nbuf' <== Memory access at offset 1592 is inside this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: memcpy-param-overlap /scratch/llvm/clang-4/xenial/final/llvm.src/projects/compiler-rt/lib/asan/asan_interceptors.cc:453:3 in __asan_memcpy
Tags:
Steps To Reproduce: run:
# file sbo3
Additional Information:
Attached Files: sbo3 (9,351 bytes) 2019-02-18 08:50
https://bugs.astron.com/file_download.php?file_id=42&type=bug
Notes
(0003213)
christos   
2019-02-18 18:00   
I think this is the same as PR/63
(0003217)
spinpx   
2019-02-19 08:12   
CVE-2019-8906
(0003219)
christos   
2019-02-19 13:20   
The comment in the CVE is not correct though, it is not memcpy() that causes the overflow; it is the file_printable() that does not work with a non-NUL-terminated string.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
65 [file] General major always 2019-02-18 08:53 2019-02-19 13:18
Reporter: spinpx Platform: x86_64  
Assigned To: christos OS: Debian  
Priority: urgent OS Version: 10  
Status: resolved Product Version: 5.35  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: Possible stack corruption
Description: We build file with `--disable-libseccomp` by clang 4.0.0.
We ran the program with the input we provide without any other arguments.

The bugs exists in file 5.35.

Stack:
Program received signal SIGSEGV, Segmentation fault.
"#0 0x000000000043e271 in do_core_note (ms=<optimized out>, nbuf=<optimized out>, type=<optimized out>, swap=<optimized out>, namesz=<optimized out>, descsz=<optimized out>, noff=<optimized out>, doff=<optimized out>, flags=<optimized out>, size=<optimized out>, clazz=<optimized out>) at /mnt/raid/user/chenpeng/FuzzingBench/file/file/src/readelf.c:770",
0000001 donote (ms=<optimized out>, vbuf=<optimized out>, offset=3452, size=<optimized out>, clazz=<optimized out>, swap=<optimized out>, align=<optimized out>, flags=<optimized out>, notecount=<optimized out>, fd=<optimized out>, ph_off=<optimized out>, ph_num=<optimized out>, fsize=<optimized out>) at /mnt/raid/user/chenpeng/FuzzingBench/file/file/src/readelf.c:1194
0000002 0x0000000400001000 in ?? ()
0000003 0x080490a4000000a4 in ?? ()
0000004 0x00bbeaebd9000000 in ?? ()
0000005 0x00200001b8000000 in ?? ()
0000006 0x0d1263aa87c56e01 in ?? ()
0000007 0x2e8ca4eef0bf9884 in ?? ()
0000008 0xf8be17bd0299a906 in ?? ()
#9 0x4fcacce3342026ed in ?? ()
0000010 0x0000000100000012 in ?? ()
0000011 0x4274654e00000001 in ?? ()
0000012 0xcc45524f432d4453 in ?? ()
0000013 0x00008990ba88a2ca in ?? ()
0000014 0x80e700000004b800 in ?? ()
0000015 0x00000000002058eb in ?? ()
0000016 0x0000000000000000 in ?? ()

We run exploitable:
Description: Possible stack corruption
Short description: PossibleStackCorruption (7/22)
Hash: 457b692b06cd893f7646681b5874c47e.57ada0a7199ae15ca6cc5f54cdbeaa99
Exploitability Classification: EXPLOITABLE
Explanation: GDB generated an error while unwinding the stack and/or the stack contained return addresses that were not mapped in the inferior's process address space and/or the stack pointer is pointing to a location outside the default stack region. These conditions likely indicate stack corruption, which is generally considered exploitable.
Other tags: AccessViolation (21/22)
Tags:
Steps To Reproduce: run:
# file stack_corruption1
Additional Information:
Attached Files: stack_corruption1 (10,498 bytes) 2019-02-18 08:53
https://bugs.astron.com/file_download.php?file_id=43&type=bug
Notes
(0003214)
christos   
2019-02-18 18:07   
I think this is the same as the other note ones. Can you please try to see if the fix for PR/63 fixed it?
(0003218)
spinpx   
2019-02-19 08:16   
CVE-2019-8907
I verified it with newest git version. It fixed now!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
63 [file] General major always 2019-02-18 08:46 2019-02-19 13:17
Reporter: spinpx Platform: x86_64  
Assigned To: christos OS: Debian  
Priority: urgent OS Version: 10  
Status: resolved Product Version: 5.35  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: Stack buffer overflow 2
Description: We build file with `--disable-libseccomp` by clang 4.0.0 and ASAN.
We ran the program with the input we provide without any other arguments.

The bugs exists in file 5.35 and the newest git version commit 5b9408cbbd401c13873bf944d3085785547e9915 .

==1104585==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffd001ebb00 at pc 0x00000052240b bp 0x7ffd001eb6c0 sp 0x7ffd001eb6b8
READ of size 1 at 0x7ffd001ebb00 thread T0
    #0 0x52240a in file_printable /home/chenpeng/data/FuzzingBench/file/file-git/src/funcs.c:631:57
    0000001 0x550158 in do_core_note /home/chenpeng/data/FuzzingBench/file/file-git/src/readelf.c:762:8
    0000002 0x54db93 in donote /home/chenpeng/data/FuzzingBench/file/file-git/src/readelf.c:1197:7
    0000003 0x549826 in dophn_exec /home/chenpeng/data/FuzzingBench/file/file-git/src/readelf.c:1689:14
    0000004 0x545e2d in file_tryelf /home/chenpeng/data/FuzzingBench/file/file-git/src/elfclass.h:58:7
    0000005 0x51f29b in file_buffer /home/chenpeng/data/FuzzingBench/file/file-git/src/funcs.c:305:8
    0000006 0x4f5b5d in file_or_fd /home/chenpeng/data/FuzzingBench/file/file-git/src/magic.c:508:6
    0000007 0x4f5cd6 in magic_file /home/chenpeng/data/FuzzingBench/file/file-git/src/magic.c:397:9
    0000008 0x4f3fd5 in process /home/chenpeng/data/FuzzingBench/file/file-git/src/file.c:546:9
    #9 0x4f1c4b in main /home/chenpeng/data/FuzzingBench/file/file-git/src/file.c:416:9
    0000010 0x7fe9e3d9f09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)
    0000011 0x41d689 in _start (/mnt/raid/user/chenpeng/FuzzingBench/build/asan/install/bin/file+0x41d689)

Address 0x7ffd001ebb00 is located in stack of thread T0 at offset 768 in frame
    #0 0x54fbaf in do_core_note /home/chenpeng/data/FuzzingBench/file/file-git/src/readelf.c:713

  This frame has 2 object(s):
    [32, 544) 'sbuf'
    [608, 768) 'pi' <== Memory access at offset 768 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow /home/chenpeng/data/FuzzingBench/file/file-git/src/funcs.c:631:57 in file_printable
Shadow bytes around the buggy address:
  0x100020035710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100020035720: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100020035730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100020035740: 00 00 00 00 f2 f2 f2 f2 f2 f2 f2 f2 00 00 00 00
  0x100020035750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x100020035760:[f3]f3 f3 f3 f3 f3 f3 f3 00 00 00 00 00 00 00 00
  0x100020035770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100020035780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100020035790: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000200357a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000200357b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable: 00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone: fa
  Freed heap region: fd
  Stack left redzone: f1
  Stack mid redzone: f2
  Stack right redzone: f3
  Stack after return: f5
  Stack use after scope: f8
  Global redzone: f9
  Global init order: f6
  Poisoned by user: f7
  Container overflow: fc
  Array cookie: ac
  Intra object redzone: bb
  ASan internal: fe
  Left alloca redzone: ca
  Right alloca redzone: cb
==1104585==ABORTING
Tags:
Steps To Reproduce: run:
# file sbo2
Additional Information:
Attached Files: sbo2 (1,192 bytes) 2019-02-18 08:46
https://bugs.astron.com/file_download.php?file_id=41&type=bug
Notes
(0003212)
christos   
2019-02-18 17:47   
Thanks, should be fixed with:

/p/file/cvsroot/file/src/file.h,v <-- file.h
new revision: 1.202; previous revision: 1.201
/p/file/cvsroot/file/src/funcs.c,v <-- funcs.c
new revision: 1.101; previous revision: 1.100
/p/file/cvsroot/file/src/readelf.c,v <-- readelf.c
new revision: 1.161; previous revision: 1.160
/p/file/cvsroot/file/src/softmagic.c,v <-- softmagic.c
new revision: 1.277; previous revision: 1.276
(0003216)
spinpx   
2019-02-19 08:12   
CVE-2019-8905


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
62 [file] General major always 2019-02-18 08:42 2019-02-19 13:16
Reporter: spinpx Platform: Intel  
Assigned To: christos OS: Debian  
Priority: urgent OS Version: 10  
Status: resolved Product Version: 5.35  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: Stack buffer overflow
Description: We build file with `--disable-libseccomp` by clang 4.0.0 and ASAN.
We ran the program with the input we provide without any other arguments.

The bugs exists in file 5.35 and the newest git version commit 5b9408cbbd401c13873bf944d3085785547e9915 .

ASAN report:
==990598==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffea9a966a0 at pc 0x000000441461 bp 0x7ffea9a931d0 sp 0x7ffea9a92940
READ of size 8167 at 0x7ffea9a966a0 thread T0
    #0 0x441460 in printf_common(void*, char const*, __va_list_tag*) /scratch/llvm/clang-4/xenial/final/llvm.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors_format.inc:544:9
    0000001 0x442140 in vasprintf /scratch/llvm/clang-4/xenial/final/llvm.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1412:1
    0000002 0x51d538 in file_vprintf /home/chenpeng/data/FuzzingBench/file/file-git/src/funcs.c:62:8
    0000003 0x51da90 in file_printf /home/chenpeng/data/FuzzingBench/file/file-git/src/funcs.c:88:7
    0000004 0x54f602 in do_bid_note /home/chenpeng/data/FuzzingBench/file/file-git/src/readelf.c:569:7
    0000005 0x54d6f9 in donote /home/chenpeng/data/FuzzingBench/file/file-git/src/readelf.c:1185:7
    0000006 0x54814a in dophn_core /home/chenpeng/data/FuzzingBench/file/file-git/src/readelf.c:401:13
    0000007 0x5459d4 in file_tryelf /home/chenpeng/data/FuzzingBench/file/file-git/src/elfclass.h:43:7
    0000008 0x51f29b in file_buffer /home/chenpeng/data/FuzzingBench/file/file-git/src/funcs.c:305:8
    #9 0x4f5b5d in file_or_fd /home/chenpeng/data/FuzzingBench/file/file-git/src/magic.c:508:6
    0000010 0x4f5cd6 in magic_file /home/chenpeng/data/FuzzingBench/file/file-git/src/magic.c:397:9
    0000011 0x4f3fd5 in process /home/chenpeng/data/FuzzingBench/file/file-git/src/file.c:546:9
    0000012 0x4f1c4b in main /home/chenpeng/data/FuzzingBench/file/file-git/src/file.c:416:9
    0000013 0x7fc89d20c09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)
    0000014 0x41d689 in _start (/mnt/raid/user/chenpeng/FuzzingBench/build/asan/install/bin/file+0x41d689)

Address 0x7ffea9a966a0 is located in stack of thread T0 at offset 8384 in frame
    #0 0x54779f in dophn_core /home/chenpeng/data/FuzzingBench/file/file-git/src/readelf.c:349

  This frame has 3 object(s):
    [32, 64) 'ph32'
    [96, 152) 'ph64'
    [192, 8384) 'nbuf' <== Memory access at offset 8384 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow /scratch/llvm/clang-4/xenial/final/llvm.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors_format.inc:544:9 in printf_common(void*, char const*, __va_li
st_tag*)
Shadow bytes around the buggy address:
  0x10005534ac80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10005534ac90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10005534aca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10005534acb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10005534acc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x10005534acd0: 00 00 00 00[f3]f3 f3 f3 f3 f3 f3 f3 f3 f3 f3 f3
  0x10005534ace0: f3 f3 f3 f3 f3 f3 f3 f3 f3 f3 f3 f3 f3 f3 f3 f3
  0x10005534acf0: f3 f3 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00
  0x10005534ad00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10005534ad10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10005534ad20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable: 00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone: fa
  Freed heap region: fd
  Stack left redzone: f1
  Stack mid redzone: f2
  Stack right redzone: f3
  Stack after return: f5
  Stack use after scope: f8
  Global redzone: f9
  Global init order: f6
  Poisoned by user: f7
  Container overflow: fc
  Array cookie: ac
  Intra object redzone: bb
  ASan internal: fe
  Left alloca redzone: ca
  Right alloca redzone: cb
==990598==ABORTING

Tags:
Steps To Reproduce: run:
# file sbo1
Additional Information:
Attached Files: sbo1 (8,714 bytes) 2019-02-18 08:42
https://bugs.astron.com/file_download.php?file_id=40&type=bug
Notes
(0003211)
christos   
2019-02-18 17:32   
Should be fixed in
/p/file/cvsroot/file/src/readelf.c,v <-- readelf.c
new revision: 1.160; previous revision: 1.159

Thanks!
(0003215)
spinpx   
2019-02-19 08:11   
CVE-2019-8904


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
59 [file] General minor always 2019-01-30 11:43 2019-02-18 16:53
Reporter: magicus Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: feedback Product Version: 5.35  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Magic is missing for Java jmod and module image files
Description: There is no magic formula for Java jmod and image files (as produced by jlink); these are artefacts generated by Java since JDK 9.
Tags:
Steps To Reproduce: Actual results:

$ file jdk/jmods/java.base.jmod
jdk/jmods/java.base.jmod: Zip archive data

$ file jdk/lib/modules
jdk/lib/modules: data

Expected results:

$ file jdk/jmods/java.base.jmod
jdk/jmods/java.base.jmod: Java jmod module, version 1.0

$ file jdk/lib/modules
jdk/lib/modules: Java module image (little endian), version 1.0
Additional Information: Provided patch has been tested on all three types (jmod, BE image and LE image), and a sanity check that it did not break zip detection.
Attached Files: add-java-jmod-and-image.patch (1,190 bytes) 2019-01-30 11:43
https://bugs.astron.com/file_download.php?file_id=38&type=bug
Notes
(0003209)
christos   
2019-02-18 16:53   
Thanks! JM magic is too short. I only added it for v1.0 so that it does not generate false positives.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
60 [file] General minor have not tried 2019-02-07 06:20 2019-02-18 16:45
Reporter: silvioprog Platform: Linux  
Assigned To: christos OS: Ubuntu  
Priority: normal OS Version: 18.04  
Status: feedback Product Version: 5.35  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Build fails on mingw-w64 (even installing third party required packages)
Description: Hello,

I've tried to compile the "libmagic" (static only) for Win32 via cross-compile using the MinGW-w64 available on Ubuntu 18.04, but the build fails at configure time, even installing the library "libgnurx" from a unofficial PPA.
Tags: magic
Steps To Reproduce: Below the first steps I've tried:

$ wget -c ftp://ftp.astron.com/pub/file/file-5.35.tar.gz
$ tar -zxvf file-5.35.tar.gz
$ cd file-5.35/
$ ./configure --host=i686-w64-mingw32 --enable-static=yes --enable-shared=no
[snip]
...
checking for ctime_r... no
checking for asctime_r... no
checking for localtime_r... no
checking for gmtime_r... no
checking for pread... no
checking for strcasestr... no
checking for fmtcheck... no
checking for dprintf... no
checking for gzopen in -lz... no
checking for seccomp_init in -lseccomp... no
checking for regexec in -lgnurx... no
configure: error: libgnurx is required to build file(1) with MinGW
Additional Information: I've tried to solve the error "libgnurx is required to build file(1) with MinGW" installing the "gnurx" from a third party PPA (since it is not an official distribution, I really would like to not use it), but, just for testing, let's go to install it anyway:

sudo add-apt-repository ppa:tobydox/mingw-w64
sudo apt update
sudo apt install libgnurx-mingw-w64


Now the "configure" step get finished, but when I try to build the library:

make
[snip]
...
  CC       file.o
  CC       seccomp.o
  CCLD     file.exe
make[3]: Leaving directory '~/file-5.35/src'
make[2]: Leaving directory '~/file-5.35/src'
Making all in magic
make[2]: Entering directory '~/file-5.35/magic'
/bin/bash: line 3: file.exe: command not found
Cannot use the installed version of file () to
cross-compile file 5.35
Please install file 5.35 locally first
Makefile:808: recipe for target 'magic.mgc' failed
make[2]: *** [magic.mgc] Error 1
make[2]: Leaving directory '~/file-5.35/magic'
Makefile:399: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '~/file-5.35'
Makefile:331: recipe for target 'all' failed
make: *** [all] Error 2
Attached Files: config.log.tar.gz (15,579 bytes) 2019-02-07 06:23
https://bugs.astron.com/file_download.php?file_id=39&type=bug
Notes
(0003207)
silvioprog   
2019-02-07 06:23   
Full build log in attachment.
(0003208)
christos   
2019-02-18 16:44   
Looks like the file program "file.exe" built correctly and now it is trying to compile the magic file but it cannot execute it. It then tries to execute the installed copy and complains (correctly) because that will not work. Perhaps edit the Makefile and make the command point directly to the locally built file.exe and try again?


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
57 [file] General minor always 2018-12-27 10:11 2019-01-18 15:39
Reporter: alfredwu Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.35  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: docx files determined as zip file type.
Description: /usr/local/file-5.35/bin/file 62b587d18bae01c362c227bf45d7a08d.docx
62b587d18bae01c362c227bf45d7a08d.docx: Zip archive data, at least v1.0 to extract
Tags:
Steps To Reproduce: /usr/local/file-5.35/bin/file 62b587d18bae01c362c227bf45d7a08d.docx
62b587d18bae01c362c227bf45d7a08d.docx: Zip archive data, at least v1.0 to extract
Additional Information:
Attached Files: 62b587d18bae01c362c227bf45d7a08d.docx (18,871 bytes) 2018-12-27 10:11
https://bugs.astron.com/file_download.php?file_id=37&type=bug
Notes
(0003206)
christos   
2019-01-18 15:39   
Fixed, thanks!
/p/file/cvsroot/file/magic/Magdir/msooxml,v <-- msooxml
new revision: 1.11; previous revision: 1.10


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
55 [file] General minor have not tried 2018-12-17 11:37 2019-01-18 15:26
Reporter: pombredanne Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: feedback Product Version: 5.35  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Text file reported as data and application/octet-stream instead of text
Description: This was originally reported on https://web.archive.org/web/20150815064234/http://bugs.gw.com:80/my_view_page.php as #473 but was lost.

The attached file is reported as data and application/octet-stream instead of text



 
Tags:
Steps To Reproduce: run file and file -i on this file
Additional Information:
Attached Files: wildtest.txt (4,233 bytes) 2018-12-17 11:37
https://bugs.astron.com/file_download.php?file_id=36&type=bug
Notes
(0003205)
christos   
2019-01-18 15:26   
There are two non-ascii characters in the file:
$ diff text55.src.orig text55.src | cat -v
98,99c98,99
< 0000001 1 M-^E [^[:alnum:][:alpha:][:blank:][:cntrl:][:digit:][:graph:][:lower:][:print:][:punct:][:space:][:upper:][:xdigit:]]
< 1 1 ^? [^[:alnum:][:alpha:][:blank:][:digit:][:graph:][:lower:][:print:][:punct:][:space:][:upper:][:xdigit:]]
---
> 0000001 1 [^[:alnum:][:alpha:][:blank:][:cntrl:][:digit:][:graph:][:lower:][:print:][:punct:][:space:][:upper:][:xdigit:]]
> 1 1 [^[:alnum:][:alpha:][:blank:][:digit:][:graph:][:lower:][:print:][:punct:][:space:][:upper:][:xdigit:]]


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
56 [file] General minor have not tried 2018-12-17 11:39 2019-01-08 18:24
Reporter: pombredanne Platform:  
Assigned To: administrator OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Disallow anonymous user to post on tickets
Description: This is allowing for a lot of comment spam otherwise. See https://bugs.astron.com/view.php?id=3
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0003204)
administrator   
2019-01-08 18:24   
changed to Viewer.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
37 [tcsh] General crash always 2018-09-05 21:02 2018-12-16 18:46
Reporter: anonymous Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 6.21.00  
    Target Version:  
Summary: Pasting a unicode long-hyphen at the prompt crashes TCSH 6.19.01 + with LC_ALL=C
Description: If I do not set $LC_ALL, I can paste a long-hyphen character '–' into the prompt and it displays properly. If I instead set:

setenv LC_ALL C

Then try to paste the code, I get a segfault at ed.inputl.c GetNextCommand() at:
cmd = CurrentKeyMap[*ch];

I tracked the issue down to PR/437 commit 4b12ecbf108. Reverting this patch allows the paste to go through (it now displays escape characters instead of the hyphen, but doesn't crash).

Building on linux x86-64

tcsh --version: (x86_64-unknown-linux) options wide,nls,dl,al,kan,sm,rh,color,filec
Tags:
Steps To Reproduce: ./configure
make
./tcsh -f
setenv LC_ALL C
<paste –>
Additional Information:
Attached Files:
Notes
(0000135)
christos   
2018-12-16 18:46   
Fixed, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
50 [file] General minor always 2018-10-24 14:33 2018-12-16 03:40
Reporter: red Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.35  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: xlsx file is determined as zip
Description: Hello! xlsx file detected as application/zip.
Tags: zip
Steps To Reproduce: file --mime-type ./airports2.xlsx
./airports2.xlsx: application/zip
Additional Information:
Attached Files: airports2.xlsx (6,452 bytes) 2018-10-24 14:33
https://bugs.astron.com/file_download.php?file_id=33&type=bug
Notes
(0000100)
christos   
2018-11-05 18:39   
This has been fixed on HEAD.
(0000111)
christos   
2018-12-10 21:15   
Feedback timeout.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
29 [file] General major always 2018-08-15 19:18 2018-12-10 21:16
Reporter: Dave_Anderson Platform: Mac  
Assigned To: christos OS: MacOS  
Priority: high OS Version: 10.13.6  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: file command does not default to "don't follow symlinks" as claimed in --help
Description: Normally when I point the file command to a symlink I expect file output to indicate that I did so. Instead file tells me what it knows about the file that the the symlink is pointing to. Note that if I provide -h the behavior is as expected. Unfortunately the default acts as though I passed -L:

DAs-MacBook-Pro:~ DA$ file --version
file-5.31
DAs-MacBook-Pro:~ DA$ file --help
Usage: file [OPTION...] [FILE...]
Determine type of FILEs.

      --help display this help and exit
.
.
.
  -L, --dereference follow symlinks
  -h, --no-dereference don't follow symlinks (default)
 .
 .
 .
 DAs-MacBook-Pro:~ DA$ file `which python3`
/usr/local/bin/python3: Mach-O 64-bit executable x86_64
DAs-MacBook-Pro:~ DA$ file -L `which python3`
/usr/local/bin/python3: Mach-O 64-bit executable x86_64
DAs-MacBook-Pro:~ DA$ file -h `which python3`
/usr/local/bin/python3: symbolic link to ../Cellar/python/3.7.0/bin/python3
DAs-MacBook-Pro:~ DA$ which python3
/usr/local/bin/python3
DAs-MacBook-Pro:~ DA$ ls -l /usr/local/bin/python3
lrwxr-xr-x 1 DA admin 34 Aug 15 11:38 /usr/local/bin/python3 -> ../Cellar/python/3.7.0/bin/python3
DAs-MacBook-Pro:~ DA$ file /usr/local/bin/python3
/usr/local/bin/python3: Mach-O 64-bit executable x86_64
DAs-MacBook-Pro:~ DA$
Tags:
Steps To Reproduce: See above. Simply run the following:

file <full pathname of symlink>

or

file <relative pathname of symlink>
Additional Information:
Attached Files:
Notes
(0000059)
Dave_Anderson   
2018-08-15 19:20   
Unsure of this project's pri/sev definitions; please correct as needed.
(0000067)
christos   
2018-08-20 16:07   
Unfortunately I fix vendor-provided versions...

MacOS/X:
[11:58am] 3248>file -v
file-5.31
magic file from /usr/share/file/magic
[11:58am] 3249>ln -s /dev/null fol
[11:58am] 3250>file fol
fol: character special (3/2)
[11:58am] 3251>uname -a
Darwin vpn1-1.astron.com 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64
[11:58am] 3252>

NetBSD
[11:59am] 222>file -v
file-5.30
magic file from /usr/share/misc/magic
[11:59am] 223>ln -s /dev/null fol
[12:01pm] 224>file fol
fol: symbolic link to /dev/null
[12:02pm] 236>./file -v
file-5.34
magic file from /usr/local/share/misc/magic
[12:02pm] 237>./file ~/fol
/u/christos/fol: symbolic link to /dev/null
[12:02pm] 225>uname -a
NetBSD rebar.astron.com 7.99.59 NetBSD 7.99.59 (GENERIC) amd64


(0000068)
Dave_Anderson   
2018-08-20 18:26   
Not sure what that means.

What does vendor-provided mean? if file was provided by Apple(a vendor?) why would it be unfortunate that you will fix it?
(0000069)
christos   
2018-08-23 09:45   
The version I supply works as advertised. The version Apple supplies works differently as we've found out. I don't know why Apple changed it. I don't control what Apple distributes (the note about should read I *cannot* fix vendor supplied versions).
(0000070)
Dave_Anderson   
2018-08-23 18:25   
Thanks. Workaround implemented as follows:

DAs-MacBook-Pro:~ DA$ cat ~/.bash_profile
...
alias file='file -h'
...

-L can override as intended:

DAs-MacBook-Pro:~ DA$ file `which python3`
/usr/local/bin/python3: symbolic link to ../Cellar/python/3.7.0/bin/python3
DAs-MacBook-Pro:~ DA$ file -L `which python3`
/usr/local/bin/python3: Mach-O 64-bit executable x86_64
(0000112)
christos   
2018-12-10 21:16   
Requester has work-around.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
24 [file] General minor always 2018-08-02 11:17 2018-12-10 21:15
Reporter: anonymous Platform: Linux  
Assigned To: christos OS: Debian  
Priority: normal OS Version: 4.9.82  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: Invalid ZIP archive version
Description: file reports for a ZIP file (SHA-256 hash: d6125587ffe1ca30833eb3c2b2187b4c8c22584a0899e699fde94f44b3c978c8)

$ file d6125587ffe1ca30833eb3c2b2187b4c8c22584a0899e699fde94f44b3c978c8
d6125587ffe1ca30833eb3c2b2187b4c8c22584a0899e699fde94f44b3c978c8: Zip archive data, at least v?[0] to extract

file version 5.33 has same output, v 5.32 reported: "Zip archive data"
Tags: zip
Steps To Reproduce:
Additional Information:
Attached Files: a (153 bytes) 2018-08-03 11:42
https://bugs.astron.com/file_download.php?file_id=13&type=bug
Notes
(0000038)
christos   
2018-08-03 08:55   
Can you provide a sample file with the problem?
(0000041)
anonymous   
2018-08-03 11:42   
This is not the file that I've mentioned above, but output is same. (note that extract-version is set to 0)
(0000045)
christos   
2018-08-11 11:11   
Fixed on HEAD
(0000099)
anonymous   
2018-10-23 12:29   
I've checked the new version 5.35 and the output for this sample is still "Zip archive data, at least v?[0] to extract". Maybe I got it wrong, so I have more specific question: is "v?[0]" desired output?
Now, when I think of it version, version 0 is invalid value, so this is how You want to say "version is unknown, hence `v?`, because version in the file header is 0x00. hence `[0]`.
(0000110)
christos   
2018-12-10 21:15   
Yes, I think that's ok (to print v?[%d], version for the unhandled versions.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
35 [file] General major always 2018-08-29 13:35 2018-12-10 21:14
Reporter: anonymous Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: Video files detected as image/x-tga
Description: Some MPEG-1/2 Video (mpgv) files are detected as image/x-tga instead of video/mpeg.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: arthurDVDNTSCHiQ.vob (1,429,504 bytes) 2018-08-29 13:35
https://bugs.astron.com/file_download.php?file_id=20&type=bug
Notes
(0000083)
christos   
2018-10-01 19:18   
Can't reproduce with HEAD version.
(0000109)
christos   
2018-12-10 21:14   
Feedback timeout.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
28 [file] General minor always 2018-08-13 15:52 2018-12-10 21:12
Reporter: anonymous Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: file reports a dynamically linked library as ELF 32-bit LSB pie executable instead of ELF 32-bit LSB shared object
Description: file on the submitted file libtest_lib.so.0.8 created by cmake build on GNU/Hurd reports:

5.33-3 and 5.34-{1,2}:
ELF 32-bit LSB pie executable, Intel 80386, version 1 (SYSV),
dynamically linked,
BuildID[sha1]=f7eb0213a2606a5e3b1bc46d369d2fc47db1d8c5, with
debug_info, not stripped

5.29-3:
ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV),
dynamically linked,
BuildID[sha1]=f7eb0213a2606a5e3b1bc46d369d2fc47db1d8c5, not stripped

Due to that one test fails.
Tags:
Steps To Reproduce: Run the cmake tests. Debian version 3.12.1-1
Additional Information:
Attached Files: libtest_lib.so.0.8 (5,100 bytes) 2018-08-13 15:52
https://bugs.astron.com/file_download.php?file_id=19&type=bug
Notes
(0000052)
christos   
2018-08-14 11:28   
Please download and run and unmodified version of file to reproduce. I am not sure where the 5.34.x comes from...

./file -m ../magic/magic.mgc libtest_lib.so.0.8
libtest_lib.so.0.8: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, BuildID[sha1]=f7eb0213a2606a5e3b1bc46d369d2fc47db1d8c5, with debug_info, not stripped
(0000053)
anonymous   
2018-08-14 14:43   
file 5.34-2 is from Debian (as all versions mentioned here: https://tracker.debian.org/pkg/file. Upstream releases downloaded from: ftp://ftp.astron.com/pub/file/
There are a number of patches applied though. Dunno yet if they change the output.
(0000054)
anonymous   
2018-08-14 14:56   
After downloading file-5.34.tar.gz and building:
LD_LIBRARY_PATH=./src/.libs/libmagic.so.1 src/.libs/file ../libtest_lib.so.0.8
../libtest_lib.so.0.8: ELF 32-bit LSB pie executable, Intel 80386, version 1 (SYSV), dynamically linked, BuildID[sha1]=dfa3ab0e1cc5aeb84dd5f655ef53977fb87b0aa1, with debug_info, not stripped
Same result with:
LD_LIBRARY_PATH=./src/.libs/libmagic.so.1 src/.libs/file -m ./magic/magic.mgc ../libtest_lib.so.0.8
(0000055)
christos   
2018-08-14 15:12   
that is really curious now:
[11:11am] 125>./file -m ../magic/magic.mgc libtest_lib.so.0.8
libtest_lib.so.0.8: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, BuildID[sha1]=f7eb0213a2606a5e3b1bc46d369d2fc47db1d8c5, with debug_info, not stripped
[11:11am] 126>uname -a
Linux mb1 4.9.0-6-amd64 0000001 SMP Debian 4.9.88-1+deb9u1 (2018-05-07) x86_64 GNU/Linux
[11:11am] 127>cksum libtest_lib.so.0.8
1407326587 5100 libtest_lib.so.0.8
(0000056)
anonymous   
2018-08-14 17:09   
previous results were reported form a GNU/Hurd box:
uname -a
GNU hurd-sid 0.9 GNU-Mach 1.8+git20180728-486-dbg/Hurd-0.9 i686-AT386 GNU

GNU/Linux-amd64:
uname -a
Linux z97-4790k 4.9.0-2-amd64 0000001 SMP Debian 4.9.10-1 (2017-02-17) x86_64 GNU/Linux
file --version
file-5.34
magic file from /etc/magic:/usr/share/misc/magic
file ./libtest_lib.so.0.8
./libtest_lib.so.0.8: ELF 32-bit LSB pie executable, Intel 80386, version 1 (SYSV), dynamically linked, BuildID[sha1]=f7eb0213a2606a5e3b1bc46d369d2fc47db1d8c5, with debug_info, not stripped
wget .../ file-5.34.tar.gz
uncompresss; configure;make
 LD_LIBRARY_PATH=./src/.libs/libmagic.so.1 src/.libs/file ../libtest_lib.so.0.8
../libtest_lib.so.0.8: ELF 32-bit LSB pie executable, Intel 80386, version 1 (SYSV), dynamically linked, BuildID[sha1]=f7eb0213a2606a5e3b1bc46d369d2fc47db1d8c5, with debug_info, not stripped
Same result with:
LD_LIBRARY_PATH=./src/.libs/libmagic.so.1 src/.libs/file -m ./magic/magic.mgc ../libtest_lib.so.0.8
(0000057)
anonymous   
2018-08-14 17:14   
cksum ../libtest_lib.so.0.8
1407326587 5100 ../libtest_lib.so.0.8
(0000058)
anonymous   
2018-08-15 14:19   
Hi again. It seems like the difference is that the file generated by cmake for tests is executable on GNU/Hurd but not on GNU/Linux:
ls -l ../libtest_lib.so.0.8
-rwxr-xr-x 1 srs srs 7340 Aug 10 13:57 ../libtest_lib.so.0.8
file ../libtest_lib.so.0.8
../libtest_lib.so.0.8: ELF 32-bit LSB pie executable, Intel 80386, version 1 (SYSV), dynamically linked, BuildID[sha1]=dfa3ab0e1cc5aeb84dd5f655ef53977fb87b0aa1, with debug_info, not stripped

chmod -x ../libtest_lib.so.0.8
file ../libtest_lib.so.0.8
../libtest_lib.so.0.8: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, BuildID[sha1]=dfa3ab0e1cc5aeb84dd5f655ef53977fb87b0aa1, with debug_info, not stripped

from the rule: magic/Magdir/elf
16 leshort 3 ${x?pie executable:shared object},

The file I uploaded was probably stripped of the +x flag.
(0000060)
christos   
2018-08-16 06:11   
Yes, well while I can remove the code that checks for the shared object being executable, the question is why does hurd installed libraries with the execute bit set :-)
(0000061)
anonymous   
2018-08-16 08:01   
Well, Hurd does not normally install shared libraries with the executable flag on. Only a few ones are executable. I don't know why they are really.
However, the library libtest_lib.so.0.8 was created by test code in cmake (Debian version 3.12.1-1)
(0000062)
christos   
2018-08-16 11:14   
Isn't it better to file a bug with Hurd and ask them why some of the libraries are installed executable and others are not, plus fix the CMakefile not to add the execute bit?

For example running:
$ /lib/x86_64-linux-gnu/libc-2.24.so
GNU C Library (Debian GLIBC 2.24-11+deb9u3) stable release version 2.24, by Roland McGrath et al.
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 6.3.0 20170516.
Available extensions:
        crypt add-on version 2.1 by Michael Glad and others
        GNU Libidn by Simon Josefsson
        Native POSIX Threads Library by Ulrich Drepper et al
        BIND-8.2.3-T5B
libc ABIs: UNIQUE IFUNC
For bug reporting instructions, please see:
<http://www.debian.org/Bugs/>.

Is that a PIE binary or a shared object?
(0000064)
anonymous   
2018-08-16 18:27   
Well, some Linux libraries also have the executable flag set:
ls -l /lib/x86_64-linux-gnu/libc-2.27.so
-rwxr-xr-x 1 root root 1800248 Mar 3 11:47 /lib/x86_64-linux-gnu/libc-2.27.so
/lib/x86_64-linux-gnu/libc-2.27.so GNU C Library (Debian GLIBC 2.27-1) stable release version 2.27.
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 7.3.0.
libc ABIs: UNIQUE IFUNC
For bug reporting instructions, please see:
<http://www.debian.org/Bugs/>.
(0000065)
christos   
2018-08-18 14:33   
Yes, the real question is should it? What's the reason or benefit? Should the installation be sloppy about it?
(0000108)
christos   
2018-12-10 21:12   
This is not relevant anymore; the execute bit is not used anymore.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
26 [file] General minor always 2018-08-06 16:32 2018-12-10 21:12
Reporter: anonymous Platform: GNU/Linux  
Assigned To: christos OS: Kubuntu  
Priority: normal OS Version: 17.10  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: Regex match does not capture all characters to end of line
Description: I've got an Ansible Vault data file that I'm making magic for; the second and subsequent lines of the example file should be irrelevant (for magic purposes), but I added placeholder lines (to match my actual file) for grins.

On the fourth line of my magic config file (the one that starts with ">>&1 regex/1l") my understanding is that "regex/1l" means that I'm specifying a regular expression, and that regex will be searched for in the block starting at the character following the previous match and ending at the end of the line. (It's this last bit, "...and ending at the end of the line", that I suspect is where my understanding is incomplete.)

HOWEVER, for some reason the last character of the line is getting chopped off; as you can see in the reproduction, file is only printing "AES25" in the output, rather than "AES256". This agrees with my testing, where if I try to match the regex "AES256" it fails, but matching "AES25" works.
Tags:
Steps To Reproduce: %) file --version
file-5.32
magic file from /home/alumin/.magic:/etc/magic:/usr/share/misc/magic

%) cat file-testfile
$ANSIBLE_VAULT;1.1;AES256
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000

%) cat dot-magic-notworking
## Ansible Vault files
0 string $ANSIBLE_VAULT Ansible Vault data
>&1 regex/1l ([0-9]+(\.[0-9]+)+) \b, version %s
>>&1 regex/1l .* \b, using %s encryption
!:mime application/ansible-vault
!:strength +60

%) cat dot-magic-working ## The only difference is changing "1l" to "2l" on line 4
## Ansible Vault files
0 string $ANSIBLE_VAULT Ansible Vault data
>&1 regex/1l ([0-9]+(\.[0-9]+)+) \b, version %s
>>&1 regex/2l .* \b, using %s encryption
!:mime application/ansible-vault
!:strength +60

%) file --magic-file dot-magic-notworking testfile
testfile: Ansible Vault data, version 1.1, using AES25 encryption

%) file --magic-file dot-magic-working testfile
testfile: Ansible Vault data, version 1.1, using AES256 encryption
Additional Information: This testing is against version 5.32; apologies for not using the latest version, but it's what I have on my system and I'm kind of expecting this to be a problem with my magic file rather than a bug anyway, so I thought I might skip getting the latest version built and running unless it's necessary.

I've attached the three files in question; there's nothing special about them as far as I know, but I suppose you can't be too careful. :)
Attached Files: file-testfile (1,845 bytes) 2018-08-06 16:32
https://bugs.astron.com/file_download.php?file_id=17&type=bug
dot-magic-working (234 bytes) 2018-08-06 16:32
https://bugs.astron.com/file_download.php?file_id=16&type=bug
dot-magic-notworking (234 bytes) 2018-08-06 16:32
https://bugs.astron.com/file_download.php?file_id=15&type=bug
Notes
(0000048)
christos   
2018-08-11 12:13   
Fixed on HEAD.
(0000051)
anonymous   
2018-08-13 21:45   
Cool, I assume you mean https://github.com/file/file/commit/b186b4a4ad8b4e6927bb7fd578db572d44bd0712 ? Thanks for looking into it!
(0000107)
christos   
2018-12-10 21:12   
Verified fixed.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
54 [file] General minor always 2018-11-25 11:46 2018-12-10 21:09
Reporter: anonymous Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.33  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: Berkeley DB file detection error
Description: While trying file on a Berkeley DB file (detected as 'Berkeley DB (Btree, version 9, native byte-order)' with file 5.32), I get this result with file 5.33: ', created: Thu Jan 1 00:38:24 1970'. This issue seems to be solved in file-5.34, but my distro uses 5.33.
Tags:
Steps To Reproduce: run `file` on the attached file.
Additional Information: I found this issue while compiling diffoscope with python-file instead of python-magic. One of the tests (where I found the attached file) fails because the file is not recognised correctly. For now, I've simply disabled the test to be able to build that tool, but it's not ideal.

GuixSD (my distro) cannot update file to a newer version before at least a few months because it requires rebuilding the whole set of packages. However, we can apply patches to the 5.33 version to fix this issue. If there is any patch you can point us to, we can still apply it without having to rebuild everything. Thank you!
Attached Files: test1.db (32,768 bytes) 2018-11-25 11:46
https://bugs.astron.com/file_download.php?file_id=35&type=bug
Notes
(0000102)
anonymous   
2018-12-10 00:26   
https://cbdoilamericano.com/# where to buy cbd cream for pain
https://cbdoilamericano.com/#
https://cbdoilamericano.com/#
(0000106)
christos   
2018-12-10 21:09   
You can always compile a newer version of file; even if I fix 5.33, it is uncertain if your upstream will pick up the changes. Thanks for the report anyway.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
53 [file] General minor sometimes 2018-11-24 07:39 2018-12-10 21:07
Reporter: anonymous Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.35  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: Output is not always consistent with cso files
Description: When using the compression tool maxcso to compress PS2 iso files to cso files file(1) does not always recognize them as "Compressed ISO CD image" files and will sometimes recognize them as "data" instead. The files work fine in the PS2 emulator PCSX2 and when decompressed with maxcso they will match the original md5sum of the source iso.

$ file Atelier\ Iris\ *.cso
Atelier Iris - Eternal Mana [NTSC-U].cso: Compressed ISO CD image
Atelier Iris 2: Azoth of Destiny [NTSC-U].cso: data
Atelier Iris 3: Grand Phantasm [NTSC-U].cso: Compressed ISO CD image

$ file Atelier\ Iris\ *.iso
Atelier Iris - Eternal Mana [NTSC-U].iso: UDF filesystem data (version 1.5) ''
Atelier Iris 2: Azoth of Destiny [NTSC-U].iso: UDF filesystem data (version 1.5) ''
Atelier Iris 3: Grand Phantasm [NTSC-U].iso: UDF filesystem data (version 1.5) 'ATELIER_IRIS_GF'

I asked the maxcso developer about this issue his response can be seen in this github issue, in short the assumptions file(1) is making about cso files is not always correct.

https://github.com/unknownbrackets/maxcso/issues/26

Unfortunately due to legal reasons I can not provide example files.
Tags:
Steps To Reproduce: This can be reproduced by compressing various PS2 iso files with maxcso.

https://github.com/unknownbrackets/maxcso/
Additional Information:
Attached Files:
Notes
(0000105)
christos   
2018-12-10 21:07   
Added the popular 0x4000 blocksize.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
52 [file] General minor have not tried 2018-11-15 15:30 2018-12-10 21:02
Reporter: vitalyisaev2 Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.35  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: mp3 file is determined as "audio/mpegapplication/x-tar"
Description: Hi, please consider this file:

➜ attachments git:(UCS-5425) file -v
file-5.35
magic file from /etc/magic:/usr/share/misc/magic
➜ attachments git:(UCS-5425) file --mime-type audio.mp3
audio.mp3: audio/mpegapplication/x-tar

I think that the expected result is "audio/mpeg".
Tags:
Steps To Reproduce: Sources: ftp://ftp.astron.com/pub/file/file-5.35.tar.gz
Compiler: gcc 8.2
OS: Fedora 28
Additional Information: Also I would like to notice that at the place were I work now we use libmagic in many services, and we're interested in high quality of mime-type detection. Our QA engineers maintain set of files of various formats. I think that this set can be used as a foundation for modern libmagic's CI pipeline, which hopefully will prevent from such kind of regression bugs. If you're interested, please let me know.

Kind regards,
Vitaly
Attached Files: audio.mp3 (10,493 bytes) 2018-11-15 15:31
https://bugs.astron.com/file_download.php?file_id=34&type=bug
Notes
(0000104)
christos   
2018-12-10 21:02   
Can't reproduce with HEAD.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
51 [file] General minor always 2018-11-12 00:41 2018-12-10 20:59
Reporter: anonymous Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.35  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: Update MIME types for fonts
Description: file returns the deprecated application/vnd.ms-opentype and application/font-sfnt as MIME types for otf and ttf fonts, respectively. They should really be font/otf and font/ttf as seen in this table: https://www.iana.org/assignments/media-types/media-types.xhtml#table-font
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0000103)
christos   
2018-12-10 20:59   
Fixed, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
49 [file] General minor always 2018-10-18 14:33 2018-10-19 01:04
Reporter: giosh94mhz Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.36  
    Target Version:  
Summary: Normal TSV identified as Algol 68
Description: Rules for Algol68 are too broad, and gives some false results, especially with Tab separated text file.

Since Algol is pretty rare and TSV very common, this can be a boring issue. For now, I've attached a patch to use exact regex match and limited to only 1024 bytes.

A proper/better solution is to do multiple checks for a rare file type like this, but I'm not an Algol developer so I'm a bit clueless here. I think we should use a multiple match, which ensure that 2-3 algol instruction are in place (e.g PROC && MODE && REF, instead of PROC || MODE || REF).

Tags:
Steps To Reproduce: See sample.xls (actually TSV) attached, and a first non optimal solution in file-algol.patch
Additional Information:
Attached Files: sample.xls (62 bytes) 2018-10-18 14:33
https://bugs.astron.com/file_download.php?file_id=32&type=bug
file-algol.patch (1,069 bytes) 2018-10-18 14:33
https://bugs.astron.com/file_download.php?file_id=31&type=bug
Notes
(0000098)
christos   
2018-10-19 01:04   
applied, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
47 [file] General trivial always 2018-10-09 17:50 2018-10-09 21:44
Reporter: roramirez Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: [PATCH] Add mime and extension for AMR
Description: Add ext and mime for Adaptive Multi-Rate Codec
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: 855500e8ca155116bf7fdbe97c73d682b973b87c.patch (765 bytes) 2018-10-09 17:50
https://bugs.astron.com/file_download.php?file_id=29&type=bug
Notes
(0000095)
christos   
2018-10-09 21:44   
committed, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
23 [file] General minor have not tried 2018-07-30 15:38 2018-10-08 18:26
Reporter: vitalyisaev2 Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: JSON is determined as "text/plain"
Description: Hello!
I think that JSON files should have "application/json" content-type according to RFC4627 (http://www.ietf.org/rfc/rfc4627.txt)

Kind regards,
Vitaly

Tags:
Steps To Reproduce: [root@9826ba54a40a /]# cat /tmp/ololo.json
{
    "abc": "edf",
    "json": "crab",
    "ololo": [
        1,
        2,
        3
    ],
    "subcrab": {
        "name": "crab",
        "surname": "subcrab"
    }
}

[root@9826ba54a40a /]# file --version
file-5.34
magic file from /etc/magic:/usr/share/misc/magic
[root@9826ba54a40a /]# file --mime-type /tmp/ololo.json
/tmp/ololo.json: text/plain
Additional Information:
Attached Files: ololo.json (144 bytes) 2018-07-30 15:38
https://bugs.astron.com/file_download.php?file_id=12&type=bug
Notes
(0000040)
christos   
2018-08-03 09:15   
Yes, but that would require something in file to be able to parse json (so that it can determine if it is json)... What about a json file that has a syntax error? Should it be recognized as json?
(0000042)
anonymous   
2018-08-08 06:55   
Hi! I suppose that we should recognize it as json in all cases when it's possible. Even it has a syntax error. Of course this behaviour depends on json parser.
(0000043)
vitalyisaev2   
2018-08-08 07:43   
Perhaps libmagic could use some lightweight library that will just validate JSON (without building whole deserialized object in memory)...

I'm not sure if we are able to distinguish JSON with syntax error and arbitrary text file. From my point of view, falling back to "text/plain" with broken JSON would be a better option.
(0000044)
christos   
2018-08-11 11:06   
Fixed on HEAD.
(0000050)
vitalyisaev2   
2018-08-13 10:51   
Thank you, going to check it in a couple of days
(0000092)
vitalyisaev2   
2018-10-08 14:24   
Sorry for the delay with response, I've checked it, works good. I think we can close this issue.
(0000094)
christos   
2018-10-08 18:26   
Submitter verified it is fixed.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
46 [file] General minor have not tried 2018-10-08 14:12 2018-10-08 18:26
Reporter: vitalyisaev2 Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: docx file is determined as zip
Description: Hello, please consider this docx file which is determined as zip. It's valid and can be opened by LIbreOffice, but file (compiled from repository's HEAD) does not agree that it's docx file.

Thank you.
Tags:
Steps To Reproduce: file --mime-type /tmp/office.docx
/tmp/office.docx: application/zip
Additional Information:
Attached Files: office.docx (34,220 bytes) 2018-10-08 14:12
https://bugs.astron.com/file_download.php?file_id=28&type=bug
Notes
(0000093)
christos   
2018-10-08 18:26   
Made the zip search deeper; thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
32 [file] General minor have not tried 2018-08-24 23:09 2018-10-02 01:11
Reporter: Chai T. Rex Platform: x86-64, Kaby Lake  
Assigned To: christos OS: Ubuntu  
Priority: normal OS Version: 16.04.5  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: file declares that Bash process substitution uses a broken symlink
Description: Apologies if this has already been fixed.

In Ubuntu 16.04.5, with file-5.25, Bash process substitution fails, claiming that there's a broken symbolic link to a pipe. The same sort of symbolic link works just fine with other programs, so there's something wrong with file's detection of broken symlinks.
Tags:
Steps To Reproduce: 1. Note that Bash process substitution works in general, so the symlink that's involved must be properly readable:

    $ /bin/cat <( echo '#!/bin/sh' )
    #!/bin/sh

2. Note that file gives an error:

    $ /usr/bin/file <( echo '#!/bin/sh' )
    /dev/fd/63: broken symbolic link to pipe:[10713867]
Additional Information: The version of the Ubuntu APT package called file is 1:5.25-2ubuntu1.1.

    $ /usr/bin/file --version
    file-5.25
    magic file from /etc/magic:/usr/share/misc/magic
Attached Files:
Notes
(0000085)
christos   
2018-10-01 19:24   
$ ls -l <(echo "foo bar")
lr-x------ 1 christos users 64 Oct 1 15:22 /dev/fd/63 -> pipe:[3511315244]

So what happens if you readlink(2) /dev/fd/63 and then try to open the result instead of opening it directly? What's supposed to happen?
(0000087)
christos   
2018-10-01 20:00   
(Last edited: 2018-10-01 20:01)
This is only broken on Linux; other systems with a proper procfs/devfs implementation work just fine :-)

(0000091)
christos   
2018-10-02 01:11   
Should be fixed now.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
45 [file] General minor always 2018-09-17 07:54 2018-10-01 23:34
Reporter: sezero Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: unreadable output for macho fat files
Description: $ file --version
file-5.34
magic file from /home/sezero/opt/file-5.34/share/misc/magic
$ file wildmidi
wildmidi: Mach-O universal binary with 3 architectures: [x86_64:Mach-O 64-bit x86_64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL>] [ppc:Mach-O ppc executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL>]

This is a fat file with x86_64, i386 and ppc architectures. While
trying to output flags (useless IMHO, but it's just me), file loses
one of the arches (i386) and the output is pretty much unreadable.

As far as I can tell, this behavior began with file-5.27 (file-5.26
doesn't compile for me.) With file-5.25, the output was readable,
although it did lose the last arch, i.e. ppc:
wildmidi: Mach-O universal binary with 3 architectures: [x86_64: Mach-O 64-bit x86_64 executable] [i386: Mach-O i386 executable] []
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: wildmidi (73,772 bytes) 2018-10-01 19:22
https://bugs.astron.com/file_download.php?file_id=27&type=bug
Notes
(0000078)
christos   
2018-10-01 19:02   
Do you have a binary that reproduces this?
I get:
./file -m ../magic/magic.mgc /bin/sync
/bin/sync: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit x86_64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE>] [i386:Mach-O i386 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE|NO_HEAP_EXECUTION>]
(0000079)
christos   
2018-10-01 19:03   
waiting for example.
(0000084)
sezero   
2018-10-01 19:22   
The tarball here has the file I referred to in the original report:
https://github.com/Mindwerks/wildmidi/releases/download/wildmidi-0.3.14/wildmidi-0.3.14-macosx.tar.gz

Also attaching one of the files from that tarball here.
(0000088)
christos   
2018-10-01 20:43   
Hmm, something is fishy with that file: Apple's file binary finds 3 architectures, but only in the body, not in the headers:
$ file wildmidi
wildmidi: Mach-O universal binary with 3 architectures: [x86_64:Mach-O 64-bit executable x86_64] [ppc:Mach-O executable ppc]
wildmidi (for architecture x86_64): Mach-O 64-bit executable x86_64
wildmidi (for architecture i386): Mach-O executable i386
wildmidi (for architecture ppc): Mach-O executable ppc

Will look into it some more.
(0000089)
sezero   
2018-10-01 21:11   
lipo (from cctools-port) three arches in header:

$ /opt/cctools-port/bin/i686-apple-darwin9-lipo -detailed_info wildmidi
Fat header in: wildmidi
fat_magic 0xcafebabe
nfat_arch 3
architecture x86_64
    cputype CPU_TYPE_X86_64
    cpusubtype CPU_SUBTYPE_X86_64_ALL
    offset 4096
    size 19832
    align 2^12 (4096)
architecture i386
    cputype CPU_TYPE_I386
    cpusubtype CPU_SUBTYPE_I386_ALL
    offset 24576
    size 23976
    align 2^12 (4096)
architecture ppc
    cputype CPU_TYPE_POWERPC
    cpusubtype CPU_SUBTYPE_POWERPC_ALL
    offset 49152
    size 24620
    align 2^12 (4096)
(0000090)
christos   
2018-10-01 23:34   
Fixed to print the missing arch, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
31 [file] General text always 2018-08-23 17:05 2018-10-01 20:00
Reporter: anonymous Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: pstring documentation error
Description: The documentation states that:
                          B A byte length (default).
                          H A 4 byte big endian length.
                          h A 2 byte big endian length.
                          L A 4 byte little endian length.
                          l A 2 byte little endian length.
Looking at the source that is not correct. It should have been:
                          B A byte length (default).
                          H A 2 byte big endian length.
                          h A 2 byte little endian length.
                          L A 4 byte big endian length.
                          l A 4 byte little endian length.
Tags:
Steps To Reproduce:
Additional Information: Look in the source-code after CHAR_PSTRING_2_BE CHAR_PSTRING_2_LE CHAR_PSTRING_4_BE CHAR_PSTRING_4_LE
Attached Files:
Notes
(0000086)
christos   
2018-10-01 20:00   
thanks


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
38 [file] General trivial always 2018-09-10 11:56 2018-10-01 19:14
Reporter: bbc Platform:  
Assigned To: christos OS:  
Priority: low OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: Typo in text for Microsoft Roslyn
Description: Attached is a patch. I didn't know where to send it so here it is.
Tags:
Steps To Reproduce: Run: file foo.pdb
Additional Information:
Attached Files: fix_typo_roslyn.patch (604 bytes) 2018-09-10 11:56
https://bugs.astron.com/file_download.php?file_id=22&type=bug
Notes
(0000082)
christos   
2018-10-01 19:14   
thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
36 [file] General minor have not tried 2018-09-05 19:13 2018-10-01 19:12
Reporter: cbiedl Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: Remaining references to gw.com
Description: Seems there are some references left to the defunct gw.com domain.

Please fix `magic/Header` and `src/file.c` while `magic/Magdir/wsdl` probably cannot be cured.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0000081)
christos   
2018-10-01 19:12   
fixed, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
40 [file] General minor always 2018-09-15 15:46 2018-10-01 19:11
Reporter: anonymous Platform: x86_64 GNU/Linux  
Assigned To: christos OS: Arch Linux  
Priority: normal OS Version: 4.16.13-1  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: Wrong mime type of Canon CR2 raw image data
Description: The mime type of Canon CR2 raw image data is recognized as image/tiff.
In version 5.33 and older versions the mime type of Canon CR2 raw image data was determined correctly as image/x-canon-cr2.
Tags:
Steps To Reproduce: file --mime-type -b image.cr2
Additional Information: Example files can be downloaded from the pixls.us website.

E.g.:

http://raw.pixls.us/data/Canon/EOS%207D/RAW_CANON_EOS_7D-raw.CR2
Attached Files:
Notes
(0000080)
christos   
2018-10-01 19:11   
bump strength to beat tiff.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
39 [file] General trivial always 2018-09-13 16:08 2018-10-01 18:58
Reporter: jidanni Platform: Debian  
Assigned To: christos OS:  
Priority: low OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: Document jpeg "frames 3"
Description: Document on the man page if "frames 3" is talking about some kind of layer.
https://photo.stackexchange.com/questions/9606/do-jpgs-have-layers
Tags:
Steps To Reproduce: $ file ...jpg
... 1836x3264, frames 3
Additional Information: $ identify -verbose ...jpg
doesn't mention any frames.
Attached Files:
Notes
(0000077)
christos   
2018-10-01 18:58   
changed "frames" to "components"


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
34 [file] General feature always 2018-08-29 11:35 2018-10-01 18:53
Reporter: anonymous Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.33  
Product Build: Resolution: no change required  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: allow piping
Description: so i was wondering what kind of files live in my /usr/lib directory, so i naively tried:

find /usr/lib -type f | file

but only got the `file` usage info printed out.
so i assume we cannot pipe filepaths to `file`?

could we have this awesome feature?
Tags:
Steps To Reproduce: find /usr/lib -type f | file
Additional Information:
Attached Files:
Notes
(0000076)
christos   
2018-10-01 18:53   
find . -type f | file -f -


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
33 [file] General minor have not tried 2018-08-24 23:32 2018-10-01 18:51
Reporter: Chai T. Rex Platform: x86-64, Kaby Lake  
Assigned To: christos OS: Ubuntu  
Priority: normal OS Version: 16.04.5  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: file --help bug reporting location is outdated
Description: According to the definition of `help()` in the GitHub repo file/file's `file.c` source code (with relevant lines highlighted at https://github.com/file/file/blob/master/src/file.c#L643-L661), the location to report bugs given by `file --help` is:

    Report bugs to http://bugs.gw.com/

That domain is dead and should be updated to the current man page's URL (with relevant lines highlighted at https://github.com/file/file/blob/master/doc/file.man#L631-L638):

    Please report bugs and send patches to the bug tracker at
    .Pa http://bugs.astron.com/
    or the mailing list at
    .Aq file@astron.com
    (visit
    .Pa http://mailman.astron.com/mailman/listinfo/file
    first to subscribe).
Tags:
Steps To Reproduce: 1. View the source code.
2. Notice that `file --help` will produce the wrong bug-reporting URL.
Additional Information: It may also be good to change the URLs in both the `man` page and `file --help` output to HTTPS instead of HTTP as a simple way to avoid a few security troubles.
Attached Files:
Notes
(0000075)
christos   
2018-10-01 18:51   
thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
43 [file] General minor have not tried 2018-09-16 07:29 2018-10-01 18:49
Reporter: petk Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: Obsolete HAVE_STDDEF_H symbol
Description: The <stddef.h> header file is part of the standard C89 headers [1]
and on current systems can be included unconditionally.

Since file requires at least C89 or greater, the HAVE_STDDEF_H symbol
defined by Autoconf in configure.ac [2] can be ommitted and simplifed.

Refs:
[1] https://port70.net/~nsz/c/c89/c89-draft.html#4.1.2
[2] https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/autoconf/headers.m4

Thank you.
Tags:
Steps To Reproduce:
Additional Information: Pull request has been opened also at https://github.com/file/file/pull/41 for better overview.
Attached Files: 41.patch (2,120 bytes) 2018-09-16 07:29
https://bugs.astron.com/file_download.php?file_id=25&type=bug
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
41 [file] General minor always 2018-09-16 07:00 2018-10-01 18:49
Reporter: petk Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: Obsolete HAVE_LIMITS_H symbol
Description: The <limits.h> header file is part of the standard C89 headers [1] and
on current systems there is no need to manually check if header is
present anymore.

Since the code requires at least C89 or greater, the HAVE_LIMITS_H
symbol defined by Autoconf in configure.ac [2] can be removed and
simplifed.

Refs:
[1] https://port70.net/~nsz/c/c89/c89-draft.html#4.1.2
[2] https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/autoconf/headers.m4
Tags:
Steps To Reproduce:
Additional Information: Pull request for additional info and overview of this is at https://github.com/file/file/pull/39
Attached Files: 39.patch (3,246 bytes) 2018-09-16 07:00
https://bugs.astron.com/file_download.php?file_id=23&type=bug
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
44 [file] General minor have not tried 2018-09-17 00:26 2018-10-01 18:48
Reporter: anonymous Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: Obsolete HAVE_SIGNAL_H symbol
Description: The <signal.h> header file is part of the standard C89 headers [1]
and on current systems can be included unconditionally.

Since file requires at least C89 or greater, the HAVE_SIGNAL_H symbol
defined by Autoconf in configure.ac [2] can be ommitted and simplifed.

Refs:
[1] https://port70.net/~nsz/c/c89/c89-draft.html#4.1.2
[2] https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/autoconf/headers.m4

Thank you.
Tags:
Steps To Reproduce:
Additional Information: Pull request has been also opened on GitHub for better overview of the patch needed. https://github.com/file/file/pull/42
Attached Files: 42.patch (2,813 bytes) 2018-09-17 00:26
https://bugs.astron.com/file_download.php?file_id=26&type=bug
There are no notes attached to this issue.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
42 [file] General minor always 2018-09-16 07:06 2018-10-01 18:48
Reporter: petk Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: Obsolete HAVE_LOCALE_H symbol
Description: The <locale.h> header file is part of the standard C89 headers [1]
and on current systems can be included unconditionally.

Since file requires at least C89 or greater, the HAVE_LOCALE_H symbol
defined by Autoconf in configure.ac [2] can be ommitted and simplifed.

Refs:
[1] https://port70.net/~nsz/c/c89/c89-draft.html#4.1.2
[2] https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/autoconf/headers.m4

Thank you.
Tags:
Steps To Reproduce:
Additional Information: Pull request for additional info is at https://github.com/file/file/pull/40
Attached Files: 40.patch (1,645 bytes) 2018-09-16 07:06
https://bugs.astron.com/file_download.php?file_id=24&type=bug
Notes
(0000073)
christos   
2018-09-17 00:37   
Have you tried building with visual studio (is it c89?)?
(0000074)
petk   
2018-09-17 01:32   
Hello, I haven't build this with Visual Studio but I can check it out via some virtual machine how this works... Visual Studio 2015 and 2017 should both support C89 standard headers, yes. For example:
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/setlocale-wsetlocale?view=vs-2015


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
30 [file] General minor always 2018-08-16 18:47 2018-08-20 15:52
Reporter: cbiedl Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: Different behaviour for setparam with value zero
Description: Hi there,

the file program ignores any parameter set using --parameter if the value is zero (file.c:430). At the same time, an according magic_setparam call does set the value.

It might be questionable whether the value zero should be supported - in my opinion it should, even if just for tests and weird experiments. However, I'd appreciate if the file program and libmagic would show the same behaviour. The current situation created some confusion when trying to test effects of --parameter before using the function in an application.

Regards,
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0000066)
christos   
2018-08-20 15:52   
fixed


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
25 [file] General minor have not tried 2018-08-06 06:19 2018-08-11 12:18
Reporter: cbiedl Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: Magic strength computation underflow
Description: This doesn't look good:

    $ file --list | grep FORTRAN
    Strength = 18446744073709551613@7: FORTRAN program text [text/x-fortran]

The problem is in `apprentice_magic_strength`: The `val` variable is size_t but as result of substraction the computation might underflow.

The obvious approach was to guard `val` before such an operation but it turns out some rule start with `!:strength - 1` and the like, so that change would cause other and surprising damage.

Suggestion: Make `val` a signed long which should be big enough to avoid overflow. If the resulting value is still negative, set it to one. Also move that operation a block down to a place where it seems more appropriate.

This yields (as the only change)

    $ file --list | grep FORTRAN
    Strength = 2@7: FORTRAN program text [text/x-fortran]

which looks much better.

As a side effect, the strength of

    @42: Perl5 module source text

went down by one, so adjust this to maintain the current status which is the result of delicate tuning IIRC.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: strength-underflow.patch (1,546 bytes) 2018-08-06 06:19
https://bugs.astron.com/file_download.php?file_id=14&type=bug
Notes
(0000049)
christos   
2018-08-11 12:18   
Applied, but with long -> ssize_t


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
27 [file] General minor have not tried 2018-08-06 19:37 2018-08-11 11:32
Reporter: cbiedl Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: Fix HPROF detection
Description: Seems it wasn't my brightest day when I submitted "Java HPROF dumps" a few years ago. So: "short" should have been "byte", this breaks detection on big endian. And the creation stamp should be printed only if the previous line matched. Patch below.

Besides, the URL is 404.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: hprof.patch (657 bytes) 2018-08-06 19:37
https://bugs.astron.com/file_download.php?file_id=18&type=bug
Notes
(0000047)
christos   
2018-08-11 11:32   
Applied, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
15 [file] General feature N/A 2018-07-24 15:19 2018-08-11 11:27
Reporter: eschwartz Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: confirmed Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Try to acquire the "magic" name on the Python Package Index
Description: Per https://www.python.org/dev/peps/pep-0541/ an abandoned project can be deleted by the PyPI maintainers to clear the way for reusing the name.

The current "magic" package on https://pypi.org/project/magic/ is unmaintained since initial upload in 2003, it cannot even be installed as there is no code uploaded to PyPI, the "Project links: Download" points at a website that no longer exists (according to web.archive.org it disappeared sometime between 20121203 and 20130103), and I cannot find the original uploader on the internet more recently than 2003.

I suspect it would not be difficult to convince the PyPI maintainers of the validity of a claim by the official project. :)
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0000016)
christos   
2018-07-25 06:55   
I'll send mail to jp-py@jsnp.net who owns the project first.
(0000017)
christos   
2018-07-25 06:59   
I've sent mail, waiting for a response.
(0000021)
christos   
2018-08-01 09:02   
Sent mail to infrastructure-stuff@python.org to ask what to do next.
(0000046)
christos   
2018-08-11 11:27   
No answer there, opened: https://github.com/pypa/pypi-legacy/issues/802


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
22 [file] General minor have not tried 2018-07-26 20:57 2018-08-06 21:22
Reporter: cbiedl Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: Possible typo in magic.man: warning: macro `It' not defined
Description: Hello,

commit cc32246d2aa7cc6ecc2071d1d6ea3c6a7015f2f2
Author: Christos Zoulas <christos@zoulas.com>
Date: Fri Jun 22 20:39:49 2018 +0000

    Add quad indirect offsets

added ".It S2" to the first line of doc/magic.man, which triggers a warning by man about an undefined macro. Care to check?
Tags: compression
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0000031)
christos   
2018-08-01 10:31   
fixed, thanks


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
12 [file] General feature have not tried 2018-07-22 10:03 2018-08-02 06:33
Reporter: cbiedl Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: Please support EDID (Extended Display Identification Data)
Description: See https://bugs.debian.org/896932
File format description is at https://en.wikipedia.org/wiki/Extended_Display_Identification_Data

Tentative magic:

    0 string \x00\xFF\xFF\xFF\xFF\xFF\xFF\x00
    >19 byte x
    >>18 byte x EDID data, version %u.
    >>19 byte x \b%u

Also, I'd like to add

    >>17 ubyte+1990 <255 \b, manufactured %u

but there's an overrun so a value 22 is shown as 220, not 2012
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0000037)
christos   
2018-08-02 06:33   
Applied, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
10 [file] General minor always 2018-07-13 22:07 2018-08-02 06:26
Reporter: anonymous Platform: Debian  
Assigned To: christos OS: Linux  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: Please tech file to recognize the Sketchup 3D model format
Description: This is forwarded from <URL: https://bugs.debian.org/903693 >.

The Sketchup 3D model format is used by the Sketchup modelling tool
available from <URL: https://www.sketchup.com/ >.

This is how it is handled at the moment:

  % file 35vertical.skp
  35vertical.skp: Zip archive data, o
  % file --mime 35vertical.skp
  35vertical.skp: application/octet-stream; charset=binary
  %

According to <URL: https://www.filesuffix.com/en/extension/skp >, its
MIME type should be application/vnd.sketchup.skp. According to
<URL:https://www.iana.org/assignments/media-types/application/vnd.sketchup.skp>,
this format is not registered with IANA.

An example file can be downloaded from
<URL: https://www.thingiverse.com/thing:898198 >.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0000011)
cbiedl   
2018-07-22 07:56   
As apparently all sketchup files have an identical header (some framing and a ucs-2 encoded text), the magic is fairly simple:

0 string \xff\xfe\xff\x0e\x53\x00\x6b\x00\x65\x00\x74\x00\x63\x00\x68\x00\x55\x00\x70\x00\x20\x00\x4d\x00\x6f\x00\x64\x00\x65\x00\x6c\x00 SketchUp Model
!:mime application/vnd.sketchup.skp
!:ext skp

The Debian package will ship this from the next version on.

    Christoph

PS: Any chance we'll ever have the mailing list again?
(0000036)
christos   
2018-08-02 06:26   
Added magic, and the mailing list is in mailman.astron.com


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
14 [file] General minor have not tried 2018-07-22 12:50 2018-08-01 13:44
Reporter: cbiedl Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: feedback Product Version:  
Product Build: Resolution: open  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Issues with flif magic
Description: Debian bug: https://bugs.debian.org/864023

The current state of flif (Free Lossless Image Format) detection is fairly broken. Things don't getter by looking at the spec¹ as apparently implementation is somewhat different.

As far as I can tell from the flif sources and images I've created using the flif tool:

The dimension (width, height) information is at offset 6 (spec says: 7). They are certainly not short but "varint", a dynamic format that can hold arbitrary values without wasting space (more or less the way, a length information is stored in ASN.1).

And a minor issue: The trailing comma as in "8-bit/color," leads to doubled comma in the output.

No idea how to proceed from here. Maybe ask the original submitter for review, assuming they have a bigger collection of flif files?

Cheers,
    Christoph

¹ http://flif.info/spec.html
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0000035)
christos   
2018-08-01 13:44   
Hmm, I wrote code to parse varint, but it seems that all the image examples I have work fine with the non-varint magic...


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
16 [file] General minor have not tried 2018-07-24 20:05 2018-08-01 12:05
Reporter: cbiedl Platform: Linux  
Assigned To: christos OS: Debian  
Priority: normal OS Version: all  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: magic for AutoCAD Drawing Exchange Format
Description: Debian bug https://bugs.debian.org/702744

The original proposal introduced a lot of regressions; I reworked it
and found no mis-detection even after checking my huge collection.

Still this could take some improvement, especially the regular
expression should better be something less expensive.

So for the record, the file format is something like:

* Format is text, line-based, DOS line endings
* The first four lines are

    0
    SECTION
    2
    HEADER

  where at least the numbers might be indented with whitespace.
* Further below there might be a keyword like "AC1006", denoting the
  actual version.

Also, the \000 in the first pattern is just a dirty hack to avoid the
string "ASCII text, with CRLF line terminators" is appended to the
output. Which, although technically correct, adds more confusion than
help, YMMV.



# AutoCAD Drawing Exchange Format
0 regex \^[\ \t]*0\r?\000$
>1 regex \^[\ \t]*SECTION\r?$
>>2 regex \^[\ \t]*2\r?$
>>>3 regex \^[\ \t]*HEADER\r?$ AutoCAD Drawing Exchange Format
!:mime application/x-dxf
!:ext dxf

>>>>&1 search/8192 AC1006 \b, R10
>>>>&1 search/8192 AC1009 \b, R11/R12
>>>>&1 search/8192 AC1012 \b, R13
>>>>&1 search/8192 AC1014 \b, R14
>>>>&1 search/8192 AC1015 \b, version 2000
>>>>&1 search/8192 AC1018 \b, version 2004
>>>>&1 search/8192 AC1021 \b, version 2007
>>>>&1 search/8192 AC1024 \b, version 2010
Tags:
Steps To Reproduce:
Additional Information:
System Description Maintainer for the file package in Debian.
Attached Files:
Notes
(0000034)
christos   
2018-08-01 12:05   
Applied, thanks


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
19 [file] General minor have not tried 2018-07-26 04:56 2018-08-01 10:34
Reporter: cbiedl Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: Recognize ia64 and amd64 COFF files
Description: Debian bug: https://bugs.debian.org/697846

Please consider the attached path to identify some 64bit architecture COFF files.

Regards,

    Christoph
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: upstream.coff-amd64.patch (1,022 bytes) 2018-07-26 04:56
https://bugs.astron.com/file_download.php?file_id=10&type=bug
Notes
(0000033)
christos   
2018-08-01 10:34   
Fixed, thanks


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
21 [file] General minor have not tried 2018-07-26 20:45 2018-08-01 10:33
Reporter: cbiedl Platform: Linux  
Assigned To: christos OS: Debian  
Priority: normal OS Version: all  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 5.35  
    Target Version:  
Summary: [PATCH] Add png extensions
Description: Ubuntu bug https://bugs.launchpad.net/ubuntu/+source/file/+bug/1726096

Add extensions for png
Tags:
Steps To Reproduce:
Additional Information:
System Description Maintainer for the file package in Debian.
Attached Files: ubuntu-1726096.patch (434 bytes) 2018-07-26 20:45
https://bugs.astron.com/file_download.php?file_id=11&type=bug
Notes
(0000032)
christos   
2018-08-01 10:33   
Fixed, thanks!


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
4 [file] General minor N/A 2018-06-17 19:55 2018-08-01 10:22
Reporter: valoq Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Several issues reported by coverity
Description: The static code analysis tool coverity found several issues in file

https://scan.coverity.com/projects/linuxsandboxingproject-file

Since the tool does not provide a mean to extract the result in a readable form, the details can only be accessed after login

I have attached one example issue below
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: oobaccess (1,277 bytes) 2018-06-17 19:55
https://bugs.astron.com/file_download.php?file_id=3&type=bug
Notes
(0000006)
christos   
2018-06-23 16:15   
I don't see the problem with the attached code; I have asked the owners to give me access to see the rest of the coverity issues.
(0000030)
christos   
2018-08-01 10:22   
All coverity issues have been addressed; the one mentioned here is a false-positive.


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
6 [file] General crash always 2018-06-22 14:55 2018-08-01 09:05
Reporter: tobias Platform: i686  
Assigned To: christos OS: Linux  
Priority: normal OS Version: 4.17.2  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: out of boundary read in DER parser
Description: It is possible to trigger an out of boundary read in DER parser if a custom magic file is used.

Parsing the length of a tag allows UINT32_MAX which will overflow the check if enough memory is available.

It is therefore needed to check for an UINT32_MAX overflow before checking the available amount of data.
Tags:
Steps To Reproduce: $ mkdir ~/magic
$ cp der-magic ~/magic
$ file -m ~/magic poc.der
Segmentation fault (core dumped)
$ _
Additional Information:
Attached Files: der-magic (30 bytes) 2018-06-22 14:55
https://bugs.astron.com/file_download.php?file_id=7&type=bug
poc.der (28 bytes) 2018-06-22 14:55
https://bugs.astron.com/file_download.php?file_id=6&type=bug
file-5.33-der.patch (326 bytes) 2018-06-22 14:55
https://bugs.astron.com/file_download.php?file_id=5&type=bug
Notes
(0000003)
christos   
2018-06-23 15:15   
Patch applied thanks!
(0000029)
christos   
2018-08-01 09:05   
feedback timeout


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
5 [file] General minor always 2018-06-22 14:50 2018-08-01 09:05
Reporter: tobias Platform: i686  
Assigned To: christos OS: Linux  
Priority: normal OS Version: 4.17.2  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: seccomp is lacking 32 bit support
Description: The allowed system calls are not enough for 32 bit Linux systems with large file support.

I have extended the commands to pass installation (make install fails during magic creation) and at least basic usage.
Tags:
Steps To Reproduce: Run "./configure && make && make install" on a 32 bit Linux system which uses large file support.
Additional Information:
Attached Files: file-5.33-seccomp.patch (735 bytes) 2018-06-22 14:50
https://bugs.astron.com/file_download.php?file_id=4&type=bug
Notes
(0000004)
christos   
2018-06-23 16:09   
Patch applied, thanks!
(0000028)
christos   
2018-08-01 09:05   
feedback timeout


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
13 [file] General minor have not tried 2018-07-22 10:51 2018-08-01 09:04
Reporter: cbiedl Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: ELF: Please handle files without program headers gracefully
Description: Debian bug: https://bugs.debian.org/882310

libmagic reports files without a program header as "corrupted program header size". As mentioned in that report, there used to be a fix for this in Redhat¹ some whopping 15 years ago, but it apparently never was upstreamed. I took the liberty to forward-port it to 5.33

¹ https://bugzilla.redhat.com/attachment.cgi?id=95833&action=diff&context=patch&collapsed=&headers=1&format=raw
Tags:
Steps To Reproduce: See the Debian bug
Additional Information:
Attached Files: 0001-ELF-Handle-files-without-program-headers-gracefully.patch (1,565 bytes) 2018-07-22 10:51
https://bugs.astron.com/file_download.php?file_id=9&type=bug
Notes
(0000013)
christos   
2018-07-25 06:12   
Applied, thanks!
(0000027)
christos   
2018-08-01 09:04   
feedback timeout


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
2 [file] General minor always 2018-06-12 09:33 2018-08-01 09:04
Reporter: Hugal31 Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Too much false positive for DIY-Thermocam format
Description: I think the DIY-Thermocam format in the "measure" magic file is unwell defined and causes too many false positives.
For example :

# V1 or Lepton 2.x
9608 byte <19
>9600 use diy-thermocam-checker
>>9600 default x (Lepton 2.x),
>>>9600 use diy-thermocam-parser

A file with the byte 3840 set to 0xFF, for example, will trigger the first line, because it will be resolved as signed "-1", and "-1 < 19" is true.
I believe this behaviour is incorrect and can be solved by using "ubyte" and "ulefloat".
The same applies to the other rules, where I think changing the signed condition to an unsigned would greatly reduce the number of false positives.


I don't know the DIY-Thermocam format, and it was added by someone named "Harald Geyer". If someone know the file format or know how to contact him, please help to fix this magic file.
Tags:
Steps To Reproduce: Use underflow to trigger some rules. See file attached for example.
Additional Information:
Attached Files: measure-fake.bin (9,621 bytes) 2018-06-12 09:33
https://bugs.astron.com/file_download.php?file_id=2&type=bug
Notes
(0000005)
christos   
2018-06-23 16:13   
Fixed as suggested
(0000026)
christos   
2018-08-01 09:04   
feedback timeout


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
1 [file] General feature N/A 2018-06-09 17:14 2018-08-01 09:03
Reporter: GerbilSoft Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: New additions and improvements: ADX, DDS, KTX, Wii/GCN
Description: Here are a bunch of patches I've been queueing up for a while, but haven't been able to submit due to the previous bug tracker being down. There are 20 patches (corresponding to local commits).

Overall list of changes:
* Support for Sega CRI ADX audio files.
* GBS files have fixed 32-char text fields, so a NULL terminator might not be present.
* Added unofficial MIME types from XDG.
* Some Sega Wii games have "GCIX" magic for GVR instead of "GBIX".
* Detect Mozilla Mork databases.
* Detect new texture formats: Khronos KTX, Valve VTF, Valve VTF3 (PS3), ASTC
* ELF: Detect Wii U binaries (Cafe OS)
* GCN/Wii: Detect unencrypted images from RVT-H Readers and headered images from the official SDKs.
* DirectDraw Surface: Moved from msdos to images; added more detail; reuse the rules for Sega PVR (Xbox) files.
Tags:
Steps To Reproduce:
Additional Information:
Attached Files: file.2018-04-21.15-55.tar.gz (11,614 bytes) 2018-06-09 17:14
https://bugs.astron.com/file_download.php?file_id=1&type=bug
Notes
(0000007)
christos   
2018-06-23 16:43   
patches applied! many thanks!
(0000025)
christos   
2018-08-01 09:03   
feedback timeout


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
7 [file] General minor always 2018-06-22 21:54 2018-08-01 09:03
Reporter: bodo Platform:  
Assigned To: christos OS:  
Priority: low OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: python function magic.detect_from_filename does not handle non-existent file
Description: The python api provides the function magic.detect_from_filename. When the file given as argument does not exist, it throws the exception
ValueError: not enough values to unpack (expected 2, got 1)
which does not correctly represent the failure condition.

detect_from_filename calls _create_filemagic with the return value of mime_magic.file.
If the file does not exist, mime_magic.file returns "cannot open `foobar' (No such file or directory)".
_create_filemagic then goes on to split that value into mime_type and mime_encoding.
But as the error message does not contain a semicolon, this leads to the above exception.
Tags:
Steps To Reproduce: Do not have the file 'foobar' in the current directory and execute the following in the python intepreter:

>>> import magic
>>> magic.detect_from_filename('foobar')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.5/site-packages/magic.py", line 264, in detect_from_filename
    none_magic.file(filename))
  File "/usr/lib64/python3.5/site-packages/magic.py", line 251, in _create_filemagic
    mime_type, mime_encoding = mime_detected.split('; ')
ValueError: not enough values to unpack (expected 2, got 1)
Additional Information: versions:
- sys-apps/file-5.32 on gentoo linux with USE="+python"
- python-3.5.5
Attached Files:
Notes
(0000002)
christos   
2018-06-23 14:58   
Committed a change to raise the actual error as a ValueError
(0000008)
bodo   
2018-06-24 14:11   
Works fine in python2 now. Python3 gives:

TabError: inconsistent use of tabs and spaces in indentation

Please replace your tabs with 8 spaces.
(0000009)
christos   
2018-06-24 15:51   
Fixed, thanks!
(0000012)
christos   
2018-07-25 06:08   
Should be fixed in HEAD:

$ python
Python 2.7.13 (default, Feb 13 2017, 09:50:26)
[GCC 5.4.0] on netbsd7
Type "help", "copyright", "credits" or "license" for more information.
>>> import magic
>>> magic.detect_from_filename('foobar')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "magic.py", line 267, in detect_from_filename
    none_magic.file(filename))
  File "magic.py", line 254, in _create_filemagic
    raise ValueError(mime_detected)
ValueError: cannot open `foobar' (No such file or directory)
>>>
(0000024)
christos   
2018-08-01 09:03   
feedback timeout


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
11 [file] General minor always 2018-07-22 08:07 2018-08-01 09:03
Reporter: cbiedl Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Differing results from magic_file() and magic_buffer()
Description: The recent changes in file added inspection on the stat() system call result. This is of course not possible when magic_buffer() was call which does not know about the underlying file, if any.

As one of several results, the test suite of Adam Hupp's python-magic breaks as it silently compares the result of both functions and fails on any difference: https://github.com/ahupp/python-magic/issues/173

Can you please add a warning to the documentation of magic_buffer() this might have a different result/less detailed result? One example is some ELF files where the executable bit is taken into consideration.

And about that one, the gzip magic checks for the file size only AFAICT. Given the buffer size provided by any magic_buffer() caller, it should be possible to create a synthetic stat record containing at least that information.

Cheers,
    Christoph
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0000014)
christos   
2018-07-25 06:24   
Thanks. The ELF change that depended on stat(2) has been undone, but the magic_buffer and magic_file can still return different results, so I've documented it.
(0000023)
christos   
2018-08-01 09:03   
feedback timeout


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
17 [file] General minor have not tried 2018-07-24 22:45 2018-08-01 09:02
Reporter: cbiedl Platform: Linux  
Assigned To: christos OS: Debian  
Priority: normal OS Version: all  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: NULL check for ms in magic_setparam, magic_getparam?
Description: Hello,

all the public functions in magic.c that take a "struct magic_set" as parameter check at first whether it's NULL. However, magic_setparam and magic_getparam do not. By intention?

Next step was to ask whether the "val" void pointer shouldn't be checked before dereferencing it. And all the other pointers that are provided in various functions.

    Christoph
Tags:
Steps To Reproduce:
Additional Information:
System Description Maintainer for the file package in Debian.
Attached Files:
Notes
(0000015)
christos   
2018-07-25 06:27   
Thanks, for now I've added the ms == NULL tests for consistency.
(0000022)
christos   
2018-08-01 09:02   
feedback timeout


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
18 [file] General minor always 2018-07-25 03:23 2018-08-01 08:51
Reporter: anonymous Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: --mime-encoding reports a mangled string for .doc files
Description: Background in https://stackoverflow.com/a/45177322/113632


Calling --mime on a .doc file prints:

$ file --mime /tmp/example.doc
/tmp/example.doc: application/msword; charset=binary

Which leads me to believe --mime-encoding should print "binary", but instead it prints "application/mswordbinary":

$ file --mime-encoding /tmp/example.doc
/tmp/example.doc: application/mswordbinary

Which appears to be the concatenation of the type and the encoding. Experimenting with several other file types all print *just* their encoding (e.g. binary, us-ascii, etc.).
Tags:
Steps To Reproduce: 1. Get a .doc file
2. Run `file --mime-encoding YOUR_FILE.doc`
Additional Information:
Attached Files:
Notes
(0000018)
christos   
2018-07-25 07:16   
Fixed on HEAD, thanks!
(0000020)
christos   
2018-08-01 08:51   
feedback timeout


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
20 [file] General minor always 2018-07-26 13:04 2018-08-01 08:51
Reporter: anonymous Platform: x86_64  
Assigned To: christos OS: GNU/Linux  
Priority: normal OS Version: CentOS 7  
Status: resolved Product Version: 5.34  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: --mime-encoding --uncompress reports a mangled string for .tar.gz files
Description: Hello!
Have a problem similar to https://bugs.astron.com/view.php?id=18, but for .tar.gz files instead of .doc
Option --mime-encoding --uncompress (-z) for my .tar.gz file produces "application/x-tarbinary"

$ file --mime-encoding --uncompress example.tar.gz
example.tar.gz: application/x-tarbinary

$ file --mime --uncompress example.tar.gz
example.tar.gz: application/x-tar; charset=binary compressed-encoding=application/x-gzip; charset=binary

Suppose that instead of "application/x-tarbinary" program should just return "binary"
Tried with latest release 5.34 (compile from source) and with old pre-installed version 5.11
Tags:
Steps To Reproduce: 1. Create example.tar.gz archive with two non-empty text files in it;
2. Run `file --mime-encoding --uncompress example.tar.gz`
Additional Information:
Attached Files:
Notes
(0000019)
christos   
2018-08-01 08:51   
fixed, thanks


View Issue Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
8 [file] General minor N/A 2018-07-08 08:56 2018-07-10 03:16
Reporter: valoq Platform:  
Assigned To: christos OS:  
Priority: normal OS Version:  
Status: resolved Product Version:  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version:  
    Target Version:  
Summary: Make Bugtracker publicly accessible
Description: The bugtracker is currently only accessible after registration and login.
This makes contributing and researching bugs/errors harder.

Please consider to make the bugtracker publicly accessible without requiring a login
Tags:
Steps To Reproduce:
Additional Information:
Attached Files:
Notes
(0000010)
christos   
2018-07-10 03:16   
Added anonymous login.