View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000285 | file | General | public | 2021-08-26 09:25 | 2021-09-09 17:49 |
| Reporter | Benjamin | Assigned To | christos | ||
| Priority | normal | Severity | block | Reproducibility | always |
| Status | resolved | Resolution | fixed | ||
| Platform | Linux | OS | Ubuntu | OS Version | Ubuntu 18.04.5 |
| Fixed in Version | 5.41 | ||||
| Summary | 0000285: Python biding of libmagic crash when trying to detect encoding in multithread | ||||
| Description | Hello, When i use the following code in a multithreaded environment (some django workers using Huey or Dramatiq) magic.detect_from_content( bytes_file.read(MAX_BYTES) ).encoding I got some: File "/home/benjamin/.cache/pypoetry/virtualenvs/qvNem8AN-py3.9/lib/python3.9/site-packages/magic.py", line 284, in detect_from_content return _create_filemagic(mime_magic.buffer(byte_content), File "/home/benjamin/.cache/pypoetry/virtualenvs/qvNem8AN-py3.9/lib/python3.9/site-packages/magic.py", line 251, in _create_filemagic mime_type, mime_encoding = mime_detected.split('; ') ValueError: too many values to unpack (expected 2) With the reproductible code given in this ticket errors can be differents ("munmap_chunk(): invalid pointer", "free(): invalid size", "double free or corruption (fasttop)") It works correctly on a single thread. | ||||
| Steps To Reproduce | import threading import magic MAX_BYTES = 4096 def thread_function(): with open("/tmp/foo.csv", "rb") as bytes_file: print(magic.detect_from_content( bytes_file.read(MAX_BYTES) ).encoding) if __name__ == '__main__': threads = list() for index in range(3): thread = threading.Thread(target=thread_function) threads.append(thread) thread.start() for index, thread in enumerate(threads): thread.join() | ||||
| Additional Information | My ubuntu libmagic1 version is: 1:5.32-2ubuntu0.4 My file-magic python packaque is 0.4.0 | ||||
| Tags | magic | ||||
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2021-08-26 09:25 | Benjamin | New Issue | |
| 2021-08-26 09:25 | Benjamin | Tag Attached: magic | |
| 2021-09-09 17:49 | christos | Assigned To | => christos |
| 2021-09-09 17:49 | christos | Status | new => assigned |
| 2021-09-09 17:49 | christos | Status | assigned => resolved |
| 2021-09-09 17:49 | christos | Resolution | open => fixed |
| 2021-09-09 17:49 | christos | Fixed in Version | => 5.41 |
| 2021-09-09 17:49 | christos | Note Added: 0003641 |
