View Issue Details

IDProjectCategoryView StatusLast Update
0000451fileGeneralpublic2023-05-21 17:19
Reportermhx Assigned Tochristos  
PrioritynormalSeverityfeatureReproducibilityN/A
Status resolvedResolutionfixed 
Fixed in Version5.45 
Summary0000451: Improved magic file for DwarFS file system images
DescriptionAuthor of DwarFS here, following up on issue 449 (which was resolved while I was writing an update).

I've been working on a magic file independently of @srjs when the issue was raised on the github issue tracker. I'm posting my magic file as well, as it uses stricter checks that prevent accidental matches, supports DwarFS images with headers, and outputs the block compression algorithm used.

One thing that I don't know how to solve is that files with headers (which are typically shell scripts) will be identified either as shell scripts or as DwarFS images, depending on the order in which the magic file rules are read. I don't know what approach the file utility typically takes in case of such "hybrid" files.
TagsNo tags attached.

Activities

mhx

2023-05-21 16:06

reporter  

dwarfs.magic (1,659 bytes)   
# dwarfs: file(1) magic for DwarFS File System Image files
# URL: https://github.com/mhx/dwarfs for details about DwarFS
# From: Marcus Holland-Moritz <github@mhxnet.de>

#### DwarFS Version Macro
0			name		dwarfsversion
>&0			byte		x		\b, version %d
>&1			byte		x		\b.%d

#### DwarFS Compression Macro
0			name		dwarfscompression
>&0			leshort		=0		\b, uncompressed
>&0			leshort		=1		\b, LZMA compression
>&0			leshort		=2		\b, ZSTD compression
>&0			leshort		=3		\b, LZ4 compression
>&0			leshort		=4		\b, LZ4HC compression
>&0			leshort		=5		\b, BROTLI compression

#### DwarFS files without header
## We first check against a DWARFS magic at the start of the file, then
## validate by checking the block count / section type to be all zeros
## for the first block. Finally, we check that the *next* block also
## has the correct DWARFS magic.
0			string		DWARFS
>&0x2A			string/b	\0\0\0\0\0\0
>>&(&0x02.q+0x0A)	string		DWARFS		DwarFS File System Image
>>>&0			use		dwarfsversion
>>&0			use		dwarfscompression

#### DwarFS files with header
## We search for a DWARFS magic in the first 64k of the file (images with
## headers longer than 64k won't be recognized), then  validate by checking
## the block count / section type to be all zeros for the first block.
## Finally, we check that the *next* block also has the correct DWARFS magic.
## If we find a DWARFS magic that doesn't pass validation, we continue with
## an indirect match recursively.
1			search/65536/b	DWARFS
>&0x2A			string/b	\0\0\0\0\0\0
>>&(&0x02.q+0x0A)	string		DWARFS		DwarFS File System Image (with header)
>>>&0			use		dwarfsversion
>>&0			use		dwarfscompression
>&-1			indirect	x
dwarfs.magic (1,659 bytes)   
magictest.dwarfs (839 bytes)
magictest-header.dwarfs (951 bytes)   
#!/bin/bash
# DWARFS
find test/ src/ include/ -type f -name '*.[ch]*' | xargs -d $'\n' clang-format -i
# DWARFS
DWARFS�2�i��i���$�/���d��F?�̜1�����uS}�l(�/� pb�`�p��hmۗ��1l�/����T��P?	�uӆG��껞�N��h��k�����w2Zlz���V�ǦP��X�@d��>.���(DWARFS�ULׂ���2��=4���$u�H���(68��)�b(�/�`��
fS<2`�M:CA������0�&�0�]�UE��ƈ{{#����R���:z�F�o",2-�)zQ����=u���
����w]��T���s�F�*]"	J��b�l�$CR�#՞ �0I�� ��/]��fae4�K1��/�W��=
�an<�`�T��Q~�O7U�ڝ3NU:�Q�^~�K+�YJM��d�n����fØ��–%u���V�������?� ����:��U8P�!�va�ײP%H�1	YJ�z�
fe��򉼔ߠ+-�"91��=��S��
V��Bs���zJe�RK�a75ی��wu-���,��y������1�DWARFS%ƭ����xit��1�+�b�Et���M��ZD1�@��z�Y�U+�o�5� �
/��)tp	����A�libdwarfs v0.7.0-RC4-7-g032c69c-dirtyformat.sh�DWARFS1�P8�!�
�H����@�Ⓤ.��$�{�1\�ؿY��	 �N�	
magictest-header.dwarfs (951 bytes)   

christos

2023-05-21 17:19

manager   ~0003937

Added, thanks!

Issue History

Date Modified Username Field Change
2023-05-21 16:06 mhx New Issue
2023-05-21 16:06 mhx File Added: dwarfs.magic
2023-05-21 16:06 mhx File Added: magictest.dwarfs
2023-05-21 16:06 mhx File Added: magictest-header.dwarfs
2023-05-21 17:19 christos Assigned To => christos
2023-05-21 17:19 christos Status new => assigned
2023-05-21 17:19 christos Status assigned => resolved
2023-05-21 17:19 christos Resolution open => fixed
2023-05-21 17:19 christos Fixed in Version => 5.45
2023-05-21 17:19 christos Note Added: 0003937