View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0000378 | file | General | public | 2022-08-30 10:35 | 2022-09-03 16:25 |
Reporter | bandieramonte | Assigned To | christos | ||
Priority | high | Severity | major | Reproducibility | always |
Status | closed | Resolution | fixed | ||
OS | Win 11 64 Bit | OS Version | 10.0 Build 22000 | ||
Summary | 0000378: errorcheck_negative_one throwing error on magic load | ||||
Description | Hi, I hope this is the right place to report this issue, otherwise, kindly point me to the right place. I'm trying to use libmagic with Python 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022, 23:13:41) [MSC v.1929 64 bit (AMD64)]. It is failing to load and I have tried everything I could find in Google to solve this. The trace of the error is shown in the "Additional Information" box. By using 'pip install', I have installed: python-magic-bin 0.4.14 python-magic 0.4.27 I have also the following DLLs within PATH reach (in the System32 folder): libgnurx-0.dll magic.mgc magic1.dll regex2.dll zlib1.dll magic.py The aforementioned magic files are of the 64 bit version, since I'm using a 64 bit version of Python. These files have been attached herein in case it helps. I tried to reinstall python-magic and python-magic-bin, up to no avail. The funny thing is that this was working before, and somehow it started to show this error. I can't recall any change I did in the system that affected this. Any help would be appreciated, since I'm pretty stuck here. Thank you. | ||||
Additional Information | Trace of error: ERROR saleor.product.product_images Thumbnail fetch failed [PID:12096:Thread-8 (process_request_thread)] Traceback (most recent call last): File "C:\Users\gpb\saleorTest\saleor\product\product_images.py", line 101, in get_thumbnail thumbnail = getattr(image_file, method)[used_size] File "c:\Users\gpb\saleorTest\saleor-venv\lib\site-packages\versatileimagefield\datastructures\sizedimage.py", line 141, in __getitem__ self.create_resized_image( File "c:\Users\gpb\saleorTest\saleor-venv\lib\site-packages\versatileimagefield\datastructures\sizedimage.py", line 196, in create_resized_image image, file_ext, image_format, mime_type = self.retrieve_image( File "c:\Users\gpb\saleorTest\saleor-venv\lib\site-packages\versatileimagefield\datastructures\base.py", line 162, in retrieve_image image_format, mime_type = get_image_metadata_from_file(image) File "c:\Users\gpb\saleorTest\saleor-venv\lib\site-packages\versatileimagefield\utils.py", line 152, in get_image_metadata_from_file mime_type = magic.from_buffer(file_like.read(1024), mime=True) File "c:\Users\gpb\saleorTest\saleor-venv\lib\site-packages\magic\__init__.py", line 191, in from_buffer m = _get_magic_type(mime) File "c:\Users\gpb\saleorTest\saleor-venv\lib\site-packages\magic\__init__.py", line 165, in _get_magic_type i = _instances[mime] = Magic(mime=mime) File "c:\Users\gpb\saleorTest\saleor-venv\lib\site-packages\magic\__init__.py", line 73, in __init__ magic_load(self.cookie, magic_file) File "c:\Users\gpb\saleorTest\saleor-venv\lib\site-packages\magic\__init__.py", line 332, in magic_load return _magic_load(cookie, coerce_filename(filename)) File "c:\Users\gpb\saleorTest\saleor-venv\lib\site-packages\magic\__init__.py", line 225, in errorcheck_negative_one raise MagicException(err) magic.MagicException: None | ||||
Tags | No tags attached. | ||||
|
magic.py (9,840 bytes)
""" magic is a wrapper around the libmagic file identification library. See README for more information. Usage: >>> import magic >>> magic.from_file("testdata/test.pdf") 'PDF document, version 1.2' >>> magic.from_file("testdata/test.pdf", mime=True) 'application/pdf' >>> magic.from_buffer(open("testdata/test.pdf").read(1024)) 'PDF document, version 1.2' >>> """ import sys import glob import os.path import ctypes import ctypes.util import threading from ctypes import c_char_p, c_int, c_size_t, c_void_p default_magic_file=None class MagicException(Exception): def __init__(self, message): super(MagicException, self).__init__(message) self.message = message class Magic: """ Magic is a wrapper around the libmagic C library. """ def __init__(self, mime=False, magic_file=None, mime_encoding=False, keep_going=False, uncompress=False): """ Create a new libmagic wrapper. mime - if True, mimetypes are returned instead of textual descriptions mime_encoding - if True, codec is returned magic_file - use a mime database other than the system default keep_going - don't stop at the first match, keep going uncompress - Try to look inside compressed files. """ self.flags = MAGIC_NONE if mime: self.flags |= MAGIC_MIME if mime_encoding: self.flags |= MAGIC_MIME_ENCODING if keep_going: self.flags |= MAGIC_CONTINUE if uncompress: self.flags |= MAGIC_COMPRESS if magic_file is None: magic_file = default_magic_file self.cookie = magic_open(self.flags) self.lock = threading.Lock() magic_load(self.cookie, magic_file) def from_buffer(self, buf): """ Identify the contents of `buf` """ with self.lock: try: return maybe_decode(magic_buffer(self.cookie, buf)) except MagicException as e: return self._handle509Bug(e) def from_file(self, filename): # raise FileNotFoundException or IOError if the file does not exist with open(filename): pass with self.lock: try: return maybe_decode(magic_file(self.cookie, filename)) except MagicException as e: return self._handle509Bug(e) def _handle509Bug(self, e): # libmagic 5.09 has a bug where it might fail to identify the # mimetype of a file and returns null from magic_file (and # likely _buffer), but also does not return an error message. if e.message is None and (self.flags & MAGIC_MIME): return "application/octet-stream" else: raise e def __del__(self): # no _thread_check here because there can be no other # references to this object at this point. # during shutdown magic_close may have been cleared already so # make sure it exists before using it. # the self.cookie check should be unnecessary and was an # incorrect fix for a threading problem, however I'm leaving # it in because it's harmless and I'm slightly afraid to # remove it. if self.cookie and magic_close: magic_close(self.cookie) self.cookie = None _instances = {} def _get_magic_type(mime): i = _instances.get(mime) if i is None: i = _instances[mime] = Magic(mime=mime) return i def from_file(filename, mime=False): """" Accepts a filename and returns the detected filetype. Return value is the mimetype if mime=True, otherwise a human readable name. >>> magic.from_file("testdata/test.pdf", mime=True) 'application/pdf' """ m = _get_magic_type(mime) return m.from_file(filename) def from_buffer(buffer, mime=False): """ Accepts a binary string and returns the detected filetype. Return value is the mimetype if mime=True, otherwise a human readable name. >>> magic.from_buffer(open("testdata/test.pdf").read(1024)) 'PDF document, version 1.2' """ m = _get_magic_type(mime) return m.from_buffer(buffer) libmagic = None # Let's try to find magic or magic1 dll = ctypes.util.find_library('magic') or ctypes.util.find_library('magic1') or ctypes.util.find_library('cygmagic-1') bin_dist_path = os.path.join(os.path.dirname(__file__), 'libmagic') if os.path.isdir(bin_dist_path): if sys.platform == 'darwin': dll = os.path.abspath(os.path.join(bin_dist_path, 'libmagic.dylib')) elif sys.platform == 'win32': dll = os.path.abspath(os.path.join(bin_dist_path, 'libmagic.dll')) default_magic_file = os.path.join(bin_dist_path, 'magic.mgc') # This is necessary because find_library returns None if it doesn't find the library if dll: libmagic = ctypes.CDLL(dll) if not libmagic or not libmagic._name: windows_dlls = ['magic1.dll','cygmagic-1.dll'] platform_to_lib = {'darwin': ['/opt/local/lib/libmagic.dylib', '/usr/local/lib/libmagic.dylib'] + # Assumes there will only be one version installed glob.glob('/usr/local/Cellar/libmagic/*/lib/libmagic.dylib'), 'win32': windows_dlls, 'cygwin': windows_dlls, 'linux': ['libmagic.so.1'], # fallback for some Linuxes (e.g. Alpine) where library search does not work } platform = 'linux' if sys.platform.startswith('linux') else sys.platform for dll in platform_to_lib.get(platform, []): try: libmagic = ctypes.CDLL(dll) break except OSError: pass if not libmagic or not libmagic._name: # It is better to raise an ImportError since we are importing magic module raise ImportError('failed to find libmagic. Check your installation') magic_t = ctypes.c_void_p def errorcheck_null(result, func, args): if result is None: err = magic_error(args[0]) raise MagicException(err) else: return result def errorcheck_negative_one(result, func, args): if result is -1: err = magic_error(args[0]) raise MagicException(err) else: return result # return str on python3. Don't want to unconditionally # decode because that results in unicode on python2 def maybe_decode(s): if str == bytes: return s else: return s.decode('utf-8') def coerce_filename(filename): if filename is None: return None # ctypes will implicitly convert unicode strings to bytes with # .encode('ascii'). If you use the filesystem encoding # then you'll get inconsistent behavior (crashes) depending on the user's # LANG environment variable is_unicode = (sys.version_info[0] <= 2 and isinstance(filename, unicode)) or \ (sys.version_info[0] >= 3 and isinstance(filename, str)) if is_unicode: return filename.encode('utf-8') else: return filename magic_open = libmagic.magic_open magic_open.restype = magic_t magic_open.argtypes = [c_int] magic_close = libmagic.magic_close magic_close.restype = None magic_close.argtypes = [magic_t] magic_error = libmagic.magic_error magic_error.restype = c_char_p magic_error.argtypes = [magic_t] magic_errno = libmagic.magic_errno magic_errno.restype = c_int magic_errno.argtypes = [magic_t] _magic_file = libmagic.magic_file _magic_file.restype = c_char_p _magic_file.argtypes = [magic_t, c_char_p] _magic_file.errcheck = errorcheck_null def magic_file(cookie, filename): return _magic_file(cookie, coerce_filename(filename)) _magic_buffer = libmagic.magic_buffer _magic_buffer.restype = c_char_p _magic_buffer.argtypes = [magic_t, c_void_p, c_size_t] _magic_buffer.errcheck = errorcheck_null def magic_buffer(cookie, buf): return _magic_buffer(cookie, buf, len(buf)) _magic_load = libmagic.magic_load _magic_load.restype = c_int _magic_load.argtypes = [magic_t, c_char_p] _magic_load.errcheck = errorcheck_negative_one def magic_load(cookie, filename): return _magic_load(cookie, coerce_filename(filename)) magic_setflags = libmagic.magic_setflags magic_setflags.restype = c_int magic_setflags.argtypes = [magic_t, c_int] magic_check = libmagic.magic_check magic_check.restype = c_int magic_check.argtypes = [magic_t, c_char_p] magic_compile = libmagic.magic_compile magic_compile.restype = c_int magic_compile.argtypes = [magic_t, c_char_p] MAGIC_NONE = 0x000000 # No flags MAGIC_DEBUG = 0x000001 # Turn on debugging MAGIC_SYMLINK = 0x000002 # Follow symlinks MAGIC_COMPRESS = 0x000004 # Check inside compressed files MAGIC_DEVICES = 0x000008 # Look at the contents of devices MAGIC_MIME = 0x000010 # Return a mime string MAGIC_MIME_ENCODING = 0x000400 # Return the MIME encoding MAGIC_CONTINUE = 0x000020 # Return all matches MAGIC_CHECK = 0x000040 # Print warnings to stderr MAGIC_PRESERVE_ATIME = 0x000080 # Restore access time on exit MAGIC_RAW = 0x000100 # Don't translate unprintable chars MAGIC_ERROR = 0x000200 # Handle ENOENT etc as real errors MAGIC_NO_CHECK_COMPRESS = 0x001000 # Don't check for compressed files MAGIC_NO_CHECK_TAR = 0x002000 # Don't check for tar files MAGIC_NO_CHECK_SOFT = 0x004000 # Don't check magic entries MAGIC_NO_CHECK_APPTYPE = 0x008000 # Don't check application type MAGIC_NO_CHECK_ELF = 0x010000 # Don't check for elf details MAGIC_NO_CHECK_ASCII = 0x020000 # Don't check for ascii files MAGIC_NO_CHECK_TROFF = 0x040000 # Don't check ascii/troff MAGIC_NO_CHECK_FORTRAN = 0x080000 # Don't check ascii/fortran MAGIC_NO_CHECK_TOKENS = 0x100000 # Don't check ascii/tokens |
|
I'm happy to report that upon downgrading python-magic to version 0.4.26, down from version 0.4.27, this issue was solved for me. On the one hand, there is a solution to this. On the other hand, this seems to reveal an issue with version 0.4.27. Maybe this information will be helpful. |
|
the package name we distribute is file_magic. |
|
We don't maintain this package; we maintain file-magic only. |
|
I see, I guess we can close this issue then. I will try to find the right place to report this. Thank you! |
|
We don't maintain that python package |
Date Modified | Username | Field | Change |
---|---|---|---|
2022-08-30 10:35 | bandieramonte | New Issue | |
2022-08-30 10:35 | bandieramonte | File Added: magic1.dll | |
2022-08-30 10:35 | bandieramonte | File Added: regex2.dll | |
2022-08-30 10:35 | bandieramonte | File Added: zlib1.dll | |
2022-08-30 10:35 | bandieramonte | File Added: libgnurx-0.dll | |
2022-08-30 10:35 | bandieramonte | File Added: magic.py | |
2022-08-31 17:09 | bandieramonte | Note Added: 0003809 | |
2022-09-01 10:00 | christos | Note Added: 0003810 | |
2022-09-01 12:27 | christos | Assigned To | => christos |
2022-09-01 12:27 | christos | Status | new => assigned |
2022-09-01 12:28 | christos | Status | assigned => feedback |
2022-09-01 12:28 | christos | Note Added: 0003812 | |
2022-09-01 19:24 | bandieramonte | Note Added: 0003814 | |
2022-09-01 19:24 | bandieramonte | Status | feedback => assigned |
2022-09-03 16:25 | christos | Status | assigned => closed |
2022-09-03 16:25 | christos | Resolution | open => fixed |
2022-09-03 16:25 | christos | Note Added: 0003815 |