The seek implementations for files return ARCHIVE_FATAL if seek fails to
adjust the position. ARCHIVE_FAILED is used in case of pipes etc.
Also, if a file seek fails, the position is not adjusted.
Adjust the memory implementations to do the same.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Actually follow the bidder comment and return amount of bits. Has the
nice advantage of removing the int/ssize_t handling, even though it was
safe anyway.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If open functions fail to store client data, free internally allocated
memory before returning an error code.
Writer open functions must check magic themselves.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The newc format stores only 32-bit inode values. Assign unique
synthetic values from each full device and inode identity. This avoids
collisions after truncation while preserving hardlink groups.
Fixes#3314.
Exercise large inode values that collide after 32-bit truncation,
hardlink identity, and distinct device IDs. Update bsdcpio expectations
so successful synthesis produces no truncation warnings.
PPMd pulls input through its byte callback, so the regular input buffer does not track bytes consumed after crossing a read-block boundary. Advance the packed stream using the callback count and reject reads beyond the remaining packed data.
Resolves#3337
There are other such generators in this code, but this one was the
subject of a recent bug. It was unclear how it worked, since it broke
writing the header, continuation and final entries into three different
places.
I feel this version is more intuitive.
If upstream filter provides huge amount of bytes, e.g. if archive is
fully stored in memory, bidding accesses too many bytes. This could in
turn overflow int return value, possibly leading to signed integer
overflows.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If registering a format parser bidder fails or triggers a warning,
forward the information instead of silently turning it into ARCHIVE_OK.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Several archive writers duplicate identical hdrcharset option handling.
Share the validation and conversion logic across the affected cpio, tar,
and zip writers.
This reduces duplication without changing behavior.
When creating an archive from an @archive argument, configure the
input reader rather than passing the output writer to
archive_read_add_passphrase(). Use the reader for error reporting as
well.
Exercise --passphrase while copying entries from an encrypted
@archive into a new archive, then extract the result and verify the
copied file contents.
If a signature is too large, the bid value could overflow INT_MAX. With
its current implementation, it could even lead to a signed integer
overflow, which is undefined behavior.
Disallow huge signatures, which most likely won't impact anyone.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
append_archive_filename() returns after reporting an input archive open
failure so archive creation can continue. Release the input archive
reader before returning to avoid accumulating one reader for every
failed @archive argument.
The name was only needed to find the appropriate bidder for a filter.
Since this has been resolved by accessing the last bidder, since in these
situations the bidder is added first, then the filter, the name is not
needed anymore.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Appending a filter implies appending a bidder. Always use the last added
bidder, since it was just added. This avoids desync situations where
previously added bidders could erroneously match.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Use difftime() instead of changing time_t values by the half-year
range. This avoids signed overflow near the time_t limits without
changing normal output.
Use difftime() instead of subtracting time_t values when choosing the
date format. This avoids signed overflow for far past or future
timestamps without changing normal output.
test_read_format_zip_ppmd8_aes256_streaming fails with 6 assertion
failures on systems without a crypto library (openssl/nettle/mbedtls)
because it tries to decrypt a pre-built encrypted zipx file without
checking crypto availability first.
Add the same guard used by the other encrypted zip tests in this file.
Signed-off-by: Jaipaul Cheernam <jaipaul.cheernam@est.tech>
* In _checksum_init() and _checksum_final(), fail if the requested digest
is not supported (e.g. archive uses SHA256 but libarchive was compiled
without SHA256 support)
* In checksum_init(), fail with a meaningful error message if either
_checksum_init() call fails.
* In checksum_final(), simplify the logic and improve the error message.
* Make the various digest tests conditional on support for the digest
being tested.
Fixes: 6f10adcd59 ("xar: Add support for SHA256 and SHA512")
Extend the xar reader and writer to understand the SHA256 (header
cksum_alg 3, 32 bytes, style "sha256") and SHA512 (cksum_alg 4, 64
bytes, style "sha512") checksum algorithms in addition to the existing
SHA1 and MD5. These are the algorithm codes defined by the canonical
xar format, so archives written with them interoperate with Apple's xar,
pkgutil, and PackageKit.
Both the TOC ("toc-checksum") and per-file ("checksum") algorithms are
supported for writing. New algorithm handling is guarded by
ARCHIVE_HAS_SHA256 / ARCHIVE_HAS_SHA512 so builds without those digests
are unaffected. MAX_SUM_SIZE grows from 20 to 64 to hold a SHA512
digest.
3.8.8 ("make ARCHIVE_FATAL sticky in data-reading entry points",
commit e1f890dc) made archive_seek_data() set
a->archive.state = ARCHIVE_STATE_FATAL whenever the call returned
ARCHIVE_FATAL. That is correct for genuine I/O / parse failures
returned by a format's seek_data() implementation, but it also
poisoned the archive in two cases where seeking is simply
unsupported for the current format:
1. The seek_data == NULL branch in archive_seek_data() itself
(no format_seek_data_block registered: ustar/tar, the
streaming ZIP reader, ...).
2. rar5_seek_data() in archive_read_support_format_rar5.c, which
unconditionally returns ARCHIVE_FATAL because RAR5 is a
streaming unpacker. This was latent before 3.8.8 because
the read core did not make FATAL sticky; the stickiness
change exposed it.
Both are capability gaps, not stream corruption. ARCHIVE_FATAL
must always be sticky (it means the archive is irrecoverably
damaged), so both sites now return ARCHIVE_FAILED with an error
string instead. ARCHIVE_FAILED is recoverable: capability probes
such as
if (archive_seek_data(a, 0, SEEK_CUR) >= 0) ...
learn that seeking is unavailable while the archive remains
usable for subsequent reads. The sticky-on-genuine-FATAL part of
the original 3.8.8 change is preserved for actual ARCHIVE_FATAL
returns from a format's seek_data() implementation.
The seek_data == NULL branch also reports the condition better: a
format that never registers a seek hook is not illegal use of the
library, so the error is now ARCHIVE_ERRNO_MISC / "Cannot seek
data with this format" instead of ARCHIVE_ERRNO_PROGRAMMER /
"Internal error: No format_seek_data_block function registered".
The rar5 change also matches how the RAR4 reader already reports
the same situation (archive_read_support_format_rar.c:1339
returns ARCHIVE_FAILED for compressed RAR files).
This restores 3.8.7 behaviour for clients that probe seekability
before reading. VLC's skins2 stream extractor
(modules/stream_extractor/archive.c, archive_seek_subentry())
probes with archive_seek_data(a, 0, SEEK_CUR); on a non-seekable
format the poisoned state killed all later archive_read_data()
calls, breaking ZIP-packed .vlt skins on distributions shipping
libarchive 3.8.8.
Tests:
- test_archive_seek_data_unsupported: covers the seek_data==NULL
branch using a ustar archive.
- test_read_format_rar5_seek_data_unsupported: covers the
rar5_seek_data() branch using an existing RAR5 fixture, and
verifies the entry content with verify_data() after the probe.
Both fail on master and pass with this change.
Store the program filter description in the bidder state instead of the
active filter state.
The active state is freed by archive_read_close(), while the bidder state
remains valid until archive_read_free(). This keeps archive_filter_name()
from returning a dangling pointer.
Check reserved fields by comparing adjacent bytes after confirming the
first byte is zero. This preserves the memcmp fast path and removes
2 KiB from each ISO9660 reader state.
A varint field can actually have up to 10 bytes to resemble whole 64 bit
of a value. Use checked arithmetic to detect integer overflows.
Also, treat a varint which never has a byte without continuation bit as
faulty.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If there is no good reason to deviate from the rule, always use "f" for
filter just as we use "a" for archive.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
- If filter is meant, use "f" instead of "self" or "filter".
- If bidder is meant, use "b" instead of "self".
Matches writer filter style.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Use "struct filter *filter" pattern, with filter being the name of the
filter. This matches the format parser pattern.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
- Declare access variable at the beginning
- Remove unneeded cast (see warc parser, which never did it)
- Remove inline cast function
- Avoid unneeded brackets to unify style
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Even though int64_t would be the correct data type because no more data
than INT64_MAX can be written, use the easier solution for format
writers for now.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Files which cannot fit into a stream should never return success, which
can happen due to truncation on 32 bit systems.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The isdigit function is locale-dependent, which means that characters
outside of 0-9 range can return true here, e.g. "power of 2" for a few
locales on Windows.
Test for ASCII character range from '0' to '9' instead.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Use memcpy() for path components whose lengths are already known and
write the terminating null byte explicitly.
Cache the inserted component length instead of calculating it multiple
times while constructing the ustar pathname.
Replace the single-use basename helper with strrchr() and preserve the
existing rejection of pathnames ending in a slash.
Cache the member filename length to avoid repeated strlen() calls while
building and writing the archive header.
Continue searching after each replaced backslash instead of restarting
from the beginning of the filename.
This avoids repeatedly scanning the already converted prefix.
Check the allocation used to initialize cur_dirstr before accessing the
buffer.
Release the partially initialized strings, root entry, file registry,
and hardlink registry before returning ARCHIVE_FATAL.
Copy the old GNU tar header before reading sparse extension blocks.
Those reads may replace or free the callback buffer backing the original
header. Use the copy when parsing the common header fields.
Calculate the pathname length directly after validating the path.
The helper only wrapped strlen() and adjusted the length for directory
entries. Removing it keeps the calculation beside the validation and
reuses the existing file type value.
Add a test archive that reuses one read buffer for every 512-byte block.
Reading the sparse extension overwrites the main GNU header. Check that
mode, uid, gid, and mtime still come from the original header.
Format integer values directly at each call site and remove the custom
integer formatter.
This handles the full signed integer range and preserves unsigned byte
counters.
Clear format state and callbacks after running the previous format
destructor so a failed replacement setup cannot leave stale pointers.
Also unregister previous GNU tar state and require ARCHIVE_STATE_NEW
before switching to GNU tar.
If allocation of filter fails, gracefully handle it instead of
eventually triggering a NULL pointer dereference.
Allocate the filter in last step, because it's rather difficult to
cleanly revert in case of error.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Test that unregistering 7-Zip and XAR clears format state and is safe
to repeat.
Also test switching from 7-Zip to GNU tar and selecting GNU tar again.
Create free_data function for later reusing.
Also make sure that f->data is always set to NULL.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Writing an iso archive with a series of long names in it would eventually
cause an out-of-bounds write off the end of the rockridge extra data.
This happened when we allocated a net-new block for a file with a name
longer than would fit in two records spanning multiple blocks. We would:
- Fill up the extra_space in the last block
- Allocate a new block
- Write 250 + 5 (header) bytes of the name into it
- Try to copy 250 _more_ bytes out of the name, even if we did not have
space left.
AppleDouble detection only needs to inspect the final path component for
the "._" prefix.
Check the multibyte pathname first on all platforms and only fall back to
WCS when the multibyte pathname is unavailable. This avoids unnecessary
wide-character pathname conversion for ordinary tar entries when mac-ext
handling is enabled.
Entries without an available pathname now return no match instead of
ARCHIVE_FAILED from the boolean check.
Use a more specific error when resource header generation fails.
The old message said `cannot archive file`, but this path fails when
the generated WARC resource header exceeds the internal limit.
Check generated WARC header creation and writes.
Fail when the generated warcinfo header cannot be built. Treat a
resource header build failure as fatal because output may already exist.
Propagate write errors instead of marking the record as written.
Add memory-writer tests for generated WARC header write failures.
Cover both the warcinfo header and a resource header with warcinfo
disabled. These tests show that header write failures are hidden.
Add a per-format "align" option to the pax writer that pads each
regular file's pax extended header (creating one if necessary) so the
file data that follows begins on a multiple of the requested number of
bytes within the uncompressed archive stream. The value must be a
power of two and a multiple of the 512-byte tar block.
The alignment is applied to the stream feeding into any compression
filter, so the decompressed archive is aligned regardless of whether
the archive is compressed. This lets tools share file contents out of
an (uncompressed copy of an) archive using reflinks / copy_file_range(2)
instead of doing a full data copy on extraction.
The padding is carried in an ignorable "LIBARCHIVE.pad" pax record, so
existing readers extract the archive unchanged. Files shorter than the
alignment are left unpadded since they cannot share a whole block.
Add a test covering uncompressed and gzip-compressed archives and
document the option.
Signed-off-by: Daan De Meyer <daan@amutable.com>
A symlink entry named TRAILER!!! with a body large enough to grow the
read filter's copy buffer used to leave the cached name pointer stale
before the end-of-archive comparison. Read a static newc archive in
small blocks and assert the trailer is still detected (ARCHIVE_EOF).
The fix landed in 23b54da; this only adds coverage.
The uu filter can keep a partial encoded line in its internal input
buffer, then reach upstream EOF with no new bytes available. In that
case, d is NULL and avail_in is zero, but the buffered-input path still
passes d to memcpy() before processing the saved bytes.
Skip the append copy when avail_in is zero so the saved input is handled
without passing a null source pointer to memcpy().
Reject resource records when less data was written than Content-Length.
Do not append the WARC end marker after a short entry. Return fatal
because the record body has already been started.
This avoids writing a record whose length does not match its body.
Add a writer test for a resource entry shorter than Content-Length.
The test closes after a short write and checks for a fatal error. It
documents that the end marker must not hide a short body.
Rewrite WARC reader comments to use clearer wording and match the
style used by other libarchive format readers.
Keep the change comment-only while preserving existing behavior.
Cache the CPIO trailer-name check before consuming the pathname
buffer.
The reader kept a pointer returned by __archive_read_ahead() and used
it after __archive_read_consume(), and after possible symlink body
processing. Read-ahead pointers are views into libarchive input buffers
and should not be relied on after the associated bytes are consumed.
Store the TRAILER!!! decision while the pathname buffer is still valid,
then use the saved boolean after the remaining header handling.
Parse Content-Length only up to the located end-of-line instead of
passing a bounded header field to strtol(). Use int64_t for the parsed
entry size, ASCII digit checks, and checked arithmetic while accumulating
the value.
When a sparse map entry began before the current data block
(expected_offset < offset), verify_sparse_file() formed the intermediate
pointer "buff + (expected_offset - offset)" before adding the entry
size. That intermediate points before buff, which is undefined behavior;
with file4's large hole it trapped the test under UBSan
(-fsanitize=pointer-overflow). Fold the size into the displacement so the
whole byte offset is computed as one integer and added to buff in a
single, in-bounds step.
The standard has no fixed 10 bit limit. Instead, a pre-tree could
contain up to 16 bit codes, although this is very unlikely.
Grow the table if needed.
While at it, clarify now that max_bits are actually lookup_bits.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If a pre-tree is larger than 10 bit, libarchive fails to read the data.
According to spec, up to 16 bit are valid.
Example file provided by Duncan Horn. Thanks a lot!
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
- Huffman operations are up to 16 bit, thus uint16_t
- LZX itself can handle up to 32 bit, thus uint32_t
- Memory sizes are size_t
Otherwise:
- Use uint8_t to show where bit lengths are stored
- Use int to clarify that a status is stored
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The check against table size is redundant since Kraft's inequality
already verified that this operation cannot exceed table size anymore.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Use 0 as marker for maximum symbol count to be able to eventually switch
to another data type.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
This step is not actually needed, but will prevent static analysis
warnings that values might exist which have never been initialized.
"Invalid" values only exist outside of the addressable area, which can
happen if maximum code bit length is smaller than table bit length.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
It's actually the amount of symbols/codes in Huffman tree. Clarify it by
using the field name symbol_count instead.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Some lzx functions reverse the generic logic of 0 being success and
anything else being an error. It's debatable from a boolean point of
view but use the regular ARCHIVE_OK in case of success, which is 0.
Unifies with other lzx functions and general idea of libarchive
functions.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
We really have to check if Kraft's inequality is 1 (in our case it's
0x10000) or 0 for an empty tree. Anything else is an error.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
As long as the given index is in range of the table, the previous
creation of the lookup table takes care that all elements are valid.
This is a left over of actual tree removal done in commit
c253f0aae9.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Most if not all compilers today understand inline, but it's also just a
hint. Static functions will be inlined if it's worth it from compiler
perspective and inline functions will be still functions, too.
Remove it and let compilers decide. Increases portability for very old
compilers which cannot handle it.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
remaining was set to length but never decremented, so the loop only stopped at a NUL and read past fields passed without a trailing NUL (a read_ahead window sized by mbsnbytes). Decrement per byte and check the count before dereferencing *itp.
Drives the best-effort converter (charset iconv cannot open) through
archive_strncpy_l with a length-delimited buffer: once with trailing
non-NUL bytes after the declared length and once with an exact-length
heap allocation with no trailing NUL. Fails under ASan before the fix.
iso9660_options() checks the second byte of the iso-level option value
before checking that the first byte contains a valid ISO level.
For an empty option value such as iso9660:iso-level=,
archive_write_set_options() passes a value pointer to the terminating NUL
byte of the duplicated options string. Reading value[1] then reads one byte
past that heap allocation.
Check value[0] before reading value[1]. This keeps the same accepted values
while rejecting the empty value without reading past the option buffer.
The GNU sparse name builder trims trailing slash and /. components before
building the synthetic ustar entry name. If the sparse pathname is fully
pruned, the effective source length becomes zero and build_ustar_entry_name()
reads one byte before the pathname buffer.
Handle the all-pruned pathname case before calling the ustar name builder.
The synthetic name then preserves the root-directory case.
Read a Solaris pax entry whose SUN.holesdata value ends right after its
final decimal offset (no trailing space), which is the shape of every
valid map. The 1536-byte file has data at [0,512) and [1024,1536) with a
hole between them, so archive_entry_sparse must report exactly those two
data blocks. This pins the parser's loop-termination behavior for the
common case where the run consumes all value_length bytes.
Make sure that multiplications do not overflow (or if they overflow,
handle them properly).
This also means that proper data type size_t instead of int is used.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The -s replacement scanner consumes a backslash and then reads the
escaped byte. If the replacement ends in one literal backslash, the loop
can advance past the terminating NUL and read one byte past the
allocated replacement buffer.
Stop scanning when a backslash is followed by the terminating NUL. The
final literal append then preserves the trailing backslash.
The seekable ZIP reader checks whether __MACOSX/ entries are resource
fork entries by testing the basename prefix.
The existing guard used r - name, which is the basename offset from the
start of the filename, not the basename length. For a short basename such
as "__MACOSX/.", this could read one byte past the filename.
Check the remaining basename length before reading the resource fork prefix.
When build_ustar_entry_name() trims trailing '/' characters and '/.'
path elements from a pathname made entirely of separators, filename_end
can be reduced all the way back to src.
Handle this root-like pathname case before attempting to locate the
filename component, and emit a root-like ustar name instead.
Add a regression test that writes and reads back a long all-slash pax
pathname.
Signed-off-by: R4mbb <yong010301@gmail.com>
Consume pending data before EOF and skip handling.
Then skip only the unread body bytes plus the WARC record separator.
This keeps the reader at the correct input position after an entry has
been partially or fully read.
read_children() checked the directory record length byte before ensuring
that the record pointer was still inside the current logical block.
If directory records exactly filled the final block, the next loop
condition could read one byte past the image buffer.
Check the remaining block length before reading the record length byte.
tar_atol_base_n() read the next byte before checking whether any bytes
remained. If a numeric field ended at the caller-provided boundary, this
caused a 1-byte OOB read while preparing the next digit range check.
Read each digit only while bytes remain.
header_pax_extension() passes PAX attribute values without the trailing
newline to pax_attribute(). The SUN.holesdata parser read one byte past
the supplied value when the last numeric field ended at the value
boundary.
Handle length == 0 before checking *e.
read_body_to_string() already consumes the extension body according to
the header size. Use the resulting string length instead of decoding the
size field again in header_Solaris_ACL().
Add a test that extracts data from the first WARC resource record, then
moves to the second header.
This checks the bug where already-read entry data was not handled correctly
before skip handling.
If strnlen is not available, custom code is used. Make sure that the
custom code keeps premise of strnlen API for maximum compatibility.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Explain the algorithm and data structure involved in Huffman tree
handling as array. Hopefully simplifies upcoming code change
verifications.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
While investigating improvements to Zip's header reading,
I found an overlooked case in the read core: When a format
returned FAILED for a header read (indicating that the entry
could not be read but that the archive as a whole is still valid),
the core silently converted that to FATAL, ending the archive
at that point.
The change to the read core adds a new internal state "DATA_RECOVERY"
indicating that we're post-header (like the existing DATA state) but
that client requests to actually obtain data are not valid.
While investigating improvements to Zip's header reading,
I found an overlooked case in the read core: When a format
returned FAILED for a header read (indicating that the entry
could not be read but that the archive as a whole is still valid),
the core silently converted that to FATAL, ending the archive
at that point.
Two formats already returned FAILED for certain header issues:
Zip and LHA both indicated unreadable symlink entries in this way,
which should allow the rest of the archive to continue being read.
New tests here verify that we can now continue reading past
such entries, and that clients who ignore the FAILED result
get the same FATAL handling as other API misuse.
The WARC writer capped each data write against the full declared entry
size, but never reduced that value after accepting data. As a result,
repeated write calls could emit more bytes than the WARC Content-Length.
Track the remaining byte count for the current regular-file entry and
decrement it after each successful write. Further writes are then
clamped to the remaining length and return zero once the entry is full.
archive_acl_from_text_nl() parses ACL text from a pointer and a length
and does not require a NUL terminator; the tar reader passes a
__archive_read_ahead() window of exactly value_length bytes for
SCHILY.acl.access/.default/.ace. When an entry is empty (a
whitespace-only value, or trailing whitespace after a valid entry)
next_field() consumes the rest of the buffer as whitespace and leaves
field[0].start one past the end, which the '#'-comment probe then
dereferences. Capture the buffer end and skip the entry when its first
field starts there. The wide archive_acl_from_text_w is NUL terminated
and unaffected.
Drives archive_acl_from_text_nl() with exact-length, non-NUL-terminated
buffers whose first field is only whitespace, covering the empty-entry
and trailing-whitespace cases. Allow the test to include
archive_acl_private.h via __LIBARCHIVE_TEST.
Read the fixed metadata prefix of the PKWARE Unix extra field 0x000d.
The fixed portion contains atime, mtime, uid, and gid values. Require
the full 12-byte prefix before reading it, and leave the variable-length
data field unsupported for now.
Add a test for the PKWARE Unix extra field 0x000d.
The fixture stores a regular file with the 12-byte fixed metadata
prefix in both the local header and central directory. The test checks
that the ZIP reader exposes mtime, atime, uid, and gid metadata for
seekable and streaming reads.
WARC records require a known Content-Length. Reject regular file
entries without a valid size instead of treating them as zero.
Avoid casting archive_entry_size() through size_t before storing it in
the uint64_t WARC Content-Length field, since that can truncate large
sizes on 32-bit platforms.
If enough bytes have to be skipped, a signed integer overflow could
occur (most realistically on 32 bit systems). Use an unsigned type,
which could still overflow but has only negative impact on diagnostic
warning message.
Follow up of 02fa05675f but this time for
function find_newc_header.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
archive_write_pax_header() set hardlink entries to size zero for the
restricted pax format, then immediately set all hardlink entries to size
zero again.
Remove the narrower check because it is fully subsumed by the following
hardlink check.
No functional change intended.
header_common() parsed the tar mode field once when setting the file
type and again when setting permissions. Cache the parsed value and reuse
it for both setters.
archive_entry_set_filetype() and archive_entry_set_perm() already mask
the relevant bits, so passing the full header mode preserves behavior.
The permission-set guard is unchanged.
No functional change intended.
This is the fix for....
Update the common archive_read_data_into_fd() to truncate
output to the actual entry size from the most recent
archive_read_header result.
This addresses a hole in handling of certain subtly-malformed archives
that can have more data than declared in the header. Before this,
bsdtar explicitly truncates the entry data to the actual declared size
when extracting to a file, but did not do so when extracting to
stdout.
This is the test for....
Update the common archive_read_data_into_fd() to truncate
output to the actual entry size from the most recent
archive_read_header result.
This addresses a hole in handling of certain subtly-malformed archives
that can have more data than declared in the header. Before this,
bsdtar explicitly truncates the entry data to the actual declared size
when extracting to a file, but did not do so when extracting to
stdout.
Reverse an old experiment: When the 32-bit size field is provided, it
now must match the actual size exactly. We no longer permit it to
only match the least-significant 32 bits of the actual size.
--begin-history--
When developing the earliest versions of libarchive's Zip support, I
experimented with the idea of supporting greater-than-4GiB entries in
32-bit Zip archives. This proved to be technically feasible as long
as the _compressed_ data was small enough to keep the resulting
archive under 4GiB. But this also required us to interpret
the 32-bit uncompressed size field as representing only the
least-significant 32 bits of the size.
--end-history--
Test: If the uncompressed size stored in the archive differs
from the actual uncompressed size by a multiple of 2^32,
we should report some kind of error.
This currently goes undiagnosed because of an old attempt
to support larger-than-4GiB entries in plain 32-bit Zip
archives without necessarily requiring Zip64 extensions.
While this is technically feasible, it requires treating
the 32-bit size field ambiguously, which can lead to size
confusion from readers.
ArchiveProperties entries have a property ID, a size, and payload.
The reader parsed the ID and size, but did not consume the payload.
Payload bytes could then be read as the next property ID, which
misaligned the rest of header parsing.
Skip the payload in chunks so parsing stays aligned with 7-Zip/7zz.
If the raw parser ever treats a basically endless stream which
eventually overflows INT64_MAX, fail instead of overflowing
offset + size calculation, leading to subsequent issues in callers.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If an entry declares an uncompressed size, we should not return any
data beyond that point. This doesn't impact bsdtar and friends since
they independently truncate to the declared entry size, but other
clients may not be so careful.
Apparently discovered (but not reported) by @bikini
Reported by @dkgkdfg65
Tests for...
If an entry declares an uncompressed size, we should not return any
data beyond that point. This doesn't impact bsdtar and friends since
they independently truncate to the declared entry size, but other
clients may not be so careful.
Apparently discovered (but not reported) by @bikini
Reported by @dkgkdfg65
Add a 7z test archive with a non-empty ArchiveProperties payload.
The archive has one empty file and an ArchiveProperties entry with a
1-byte payload. It is accepted by 7-Zip/7zz.
This covers parsing when ArchiveProperties has payload bytes.
Avoid adding the encryption overhead directly to entry sizes when deciding
whether Zip64 is needed or when updating the stored compressed size.
For the Zip64 decision, compare against ZIP_4GB_MAX - additional_size. For
stored encrypted entries, use archive_ckd_add_i64() so the size update and
overflow check happen together.
Covers basic E8 translations, yet it lacks the handling of negative
values because this would require large archives.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The cab specification states that E8 translation shall be performed only
within the first 1 GB range of an uncompressed file, i.e. for the first
0x8000 CFDATA entries in a folder.
Take this limit properly into account.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Avoid allocating and filling metadata bitmaps when the 7z header says
that every file has the corresponding value.
Sparse metadata still uses the existing bitmap path.
The Windows disk reader supports wildcards by converting a path like
"foo/*" into a single tree rooted at "foo/". This does not play nicely
with symlink mode `H`, which treats the first item in each tree as a
dereferenced link (L) and all subsequent items as physical links (P). On
Windows, we take the first _match_ in L mode and all subsequent
descendants and matches in P mode.
This creates a behavioral difference between `bsdtar -c -H *` and
`bsdtar -c -H one two three`.
This commit changes how we handle H->L/P transitions when doing wildcard
matches. The transition from H to P is now done when we descend into our
first directory, and we switch back to H mode when we ascend out of
it[^1].
Closes#3222
[^1]: This does not matter in practice because tree is breadth-first
rather than depth-first, but I've done it in the interest of
completeness and not leaving ourselves caltrops to step on later.
The memory allocation in archive_read_set_callback_data2, which is
called by archive_read_set_callback_data, could fail.
Check result and properly handle failure.
Resolves#3210.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Flatten nested allAreDefined checks for attribute and timestamp bools.
Also initialize empty-stream entries as directories by default before
applying the optional EmptyFile bitmap.
No archive behavior change is intended.
Both supported BCJ2 layout branches store the chosen unpack size in
remaining.
main_stream_bytes_remaining is set from remaining immediately after
the branch, so the earlier assignments are redundant.
On LoongArch64 with 16KB pages, Linux tmpfs often uses Transparent
Huge Pages (THP) sized at 32MiB. When a test writes data followed
by a 32MiB hole, the kernel may allocate a single huge page folio
to back the entire range in RAM. Because the range is physically
backed by memory, SEEK_HOLE correctly (from the kernel's perspective)
identifies the range as DATA, failing the test's assertion. Increasing
MIN_HOLE to 64MiB ensures the hole exceeds the huge page size on
loongarch64, preserving the sparse nature of the file for the test.
Signed-off-by: Pluto Yang <yangyj.ee@gmail.com>
Use correct data types to reduce amount of casts. This should further
improve readability.
While at it, also improve white space usage.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
It's quite rare that "p" is not the iteration pointer but the base
pointer, while "b" is not the base pointer but the iteration pointer.
Use more descriptive variable names to assist auditors to figure out how
E8 call translation reversal works. It's already hard by itself.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
It's a file size and used for signed integer comparison. Since it's
literally a file size, its maximum value should be properly checked as
well.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Use more macro definitions instead of magic numbers to improve code
readability. Hopefully this explains a bit more where these values come
from.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Move the macro definitions up to the field where they are used, which is
done for other such fields as well.
This in turn allows to avoid the magic value 0 in initial assignment.
Use ST_RD_TRANSLATION instead for better documentation where the finite
state machine starts (and that this state cannot be reached again).
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The SFX scanner missed a valid 7-Zip header when exactly 32 bytes
remained in the lookahead buffer.
The ELF SFX path also missed ".data" when it was at the last valid
position in the section string table. It also allowed e_shstrndx to be
equal to e_shnum, even though that is one past the section table.
Tighten these checks so valid edge cases are handled correctly.
Add PE and ELF SFX fixtures for the SFX scanner boundary cases.
The PE fixture places an empty 7z archive at the final exact
32-byte scan window. The ELF fixture places ".data\0" at the final
valid section string-table position.
Register the fixtures for autotools distcheck.
Several ISO tests used a bz2 suffix even though they read .Z files
and verify ARCHIVE_FILTER_COMPRESS. Rename them to use a compress
suffix so the test names match the filter.
The write option setter checks used several tiny source files with the
same basic shape.
Keep the same DEFINE_TEST cases. Put the write-side option setter
checks in test_archive_write_set_options.c. Remove the extra source
files from the build lists.
The read option setter checks used several tiny source files with the
same basic shape.
Keep the same DEFINE_TEST cases. Put the read-side option setter
checks in test_archive_read_set_options.c. Remove the extra source
files from the build lists.
The close-twice read checks used three tiny source files.
Keep the same DEFINE_TEST cases. Put the fresh-object, fd-backed,
and filename-backed checks in one source file. Remove the two extra
files from the build lists.
This rather required porting them to assertIs{Reg,Dir,Symlink} and away
from lstat.
This is possible now that 9046e753ca fixed
a number of other symlink issues on Windows.
Commit e1f890dc62 fixed the underlying
root cause of these OOB checks: If a header is invalid and ARCHIVE_FATAL
is returned, do not allow data reads anymore.
The checks while reading CFDATA already looked rather off, since the
check exists in header parser and the value is never updated, but
prevent worst case scenario.
Keep the tests as regression tests if archive_read.c is modified again.
This makes the code easier to read and audit.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The comment states that a warning should be returned, but
97f964e3e0 explicitly changed that to
fatal. Adjust the comment and remove unneeded function call.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The function ar_parse_common_header can be called at exactly one
position in _ar_read_header, since it parses common information.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The __archive_read_consume function is binary: Either all bytes are
consumed or a failure occurred. No partial consumption can occur.
Simplify its calls to clarify this.
While at it, adjust style of callers as well.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Several filename encoding tests were copied from KOI8-R or EUC-JP
cases, but now use CP1251 or CP932/SJIS instead.
Update skip messages, conversion messages, and comments to match the
locales and byte sequences used by the tests.
Also fix a typo in an EUC-JP ZIP filename skip message.
archive_read_format_7zip_read_header() already returns before this
point if zip_entry is NULL. The folder pointer is also assigned from an
in-bounds folder array entry, so the extra NULL checks are redundant.
The later DONT_KNOW reset is redundant as well, since the same state is
handled earlier in the function. Remove the dead checks without changing
the encryption detection logic.
If a file close to SSIZE_MAX in size is mapped into memory and checked
for mtree, a signed integer overflow could occur.
While this this generally harmless and the scenario very unlikely, fix
the issue with a proper cast.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
`archive_read_format_7zip_read_header()` initialized `entry_crc32` with
`crc32(0, NULL, 0)`, which is equivalent to zero for this accumulator.
The same function also formatted the constant string `"7-Zip"` into an
internal buffer for every entry, then assigned it as the archive format name.
The format name can be set directly to the constant string, so the extra
buffer and per-entry `snprintf()` are unnecessary.
Remove the unused buffer and redundant per-entry work.
assertEqualIntA() uses the archive pointer to print archive error details
when the assertion fails. Passing `a` while the asserted expression is
`archive_free(a)` can make the failure path inspect the archive after it has
already been freed.
We used to cache the entire target path, which presented a problem when
we used that target path to write a symlink. Future archive entries
could write "through" that symlink because we cached it as safe.
If values are only parsed and verified but not used later on, skip field
assignments. This reduces amount of fields to audit.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Make sure that all bytes are actually skipped. Normally, the result can
be ignored because the bytes were read previously. Here, more bytes than
previously read will be skipped.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If more than UINT_MAX byte are available, e.g. if a very large archive
is mapped into memory, the BZ2/DEFLATE avail_in/avail_out values are
truncated.
Set to maximum allowed value in such cases.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
With correct data types in place, simplify the overflow check by using
checked arithmetic and simple comparison.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Follow idea of internal libarchive APIs:
- File offsets and sizes are int64_t
- Memory offsets and sizes are size_t
Also use correct data types for iterator variables.
This refactoring also makes parse_7zip_uint64 obsolete by turning it
into parse_7zip_int64.
Resolves GHSA-m4vx-w6j3-jp82.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Use checked arithmetic and unify unpack_streams check to cover all
cases. This should be a purely defensive measure.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Introduce parse_7zip_size, which performs the UMAX_ENTRY check.
Generally, use this function whenever memory allocation sizes or indices
into memory allocations are used, i.e. when size_t is meant.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Use checked arithmetic to avoid overflows. Use it also where checks are
already in place for better readability.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Since we have checked integer arithmetic now, use it.
Clarify that INT64_MAX is the maximum according to standard.
With that, switch to int64_t to avoid (unsigned long)-1 casts.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Verify that sum of numUnpackStreams will never overflow numDigests. For
this, also set numDigests to size_t because that's how the variable is
used later on.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If the minimum amount of bytes could overflow size_t after alignment,
allocate exact minimum amount instead.
This avoids integer overflow and will most likely lead to regular OOM
error handling instead.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The skip_stream function is supposed to skip a specified amount of bytes
from input stream. Since streams could be larger than 4 GB even on 32
bit systems, do not cast to size_t. Instead, read the data in small
chunks until all bytes are skipped.
This also helps to avoid large allocations on 64 bit systems.
Also verify that skipped_bytes tracking cannot overflow.
While at it, change return value to int because it's only needed for
status information, i.e. if operation was successful or not. This avoids
another cast issue with uint64_t values larger than INT64_MAX.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Exercise atou64() through a TOC numeric element (<inode>) read via the
expat backend, which hands the parser its raw, non-NUL-terminated
character-data buffer. Mirrors test_read_format_xar_base64_oob; build
with ASan for the out-of-bounds coverage.
The final seek check never advanced actual_offset for the trailing
hole created by the EOF pad_to(), so extracting any sparse file whose
last region is a hole failed with ARCHIVE_FATAL "Seek error".
This affects the systemd CI which does such an operation, and now
fails since PR https://github.com/libarchive/libarchive/pull/3128
was merged and appeared in Archlinux:
[ 269.636946] TEST-13-NSPAWN.sh[5439]: + run0 --pipe -u testuser importctl -m --user import-tar - inodetest2
[ 270.132545] TEST-13-NSPAWN.sh[5440]: Enqueued transfer job 7. Press C-c to continue download in background.
[ 270.137660] TEST-13-NSPAWN.sh[5440]: Exporting '/home/testuser/.local/state/machines/inodetest', saving to 'pipe:[79515]' with compression 'uncompressed'.
[ 270.148434] TEST-13-NSPAWN.sh[5441]: Enqueued transfer job 8. Press C-c to continue download in background.
[ 270.173410] TEST-13-NSPAWN.sh[5441]: Importing 'pipe:[79515]', saving as 'inodetest2'.
[ 270.178790] TEST-13-NSPAWN.sh[5441]: Operating on image directory '/home/testuser/.local/state/machines'.
[ 270.386155] TEST-13-NSPAWN.sh[5440]: Operation completed successfully.
[ 270.389171] TEST-13-NSPAWN.sh[5440]: Exiting.
[ 270.689994] TEST-13-NSPAWN.sh[5441]: Failed to unpack regular file 'testfile': Seek error
https://github.com/systemd/systemd/actions/runs/28159904022/job/83436896561?pr=42736
A local and small regression test is added that replicates that test's
behaviour.
Miscellaneous fixes for 3.8.8
* configure.ac: Remove unused checks
* libarchive: Fix `TIME_MAX` and `TIME_MIN`
* test: Fix off-by-one in bounds check
Fixes: 0e4fa9ff0d ("test: promote the win32-repairing systemf wrapper to all systemf calls")
* bsdunzip_test: Set umask before asserting permissions
Fixes: 34b3da0013 ("unzip: Add symlink tests")
* archive_write_disk_posix: Fix type issues
Fix type issues in the HFS+ (de)compression code, primarily to avoid
arithmetic overflow when calculating the block count during compression.
* archive_write_disk_posix: Miscellaneous cleanup
Use `S_IS*()` macros consistently and clean up some `#ifdef`s.
* archive_write_disk_posix: Symlinks can have AppleDouble data
* libarchive: Fix format string issues
(applies to 32-bit builds of libarchive)
Files within one ZF_BLOCK_SIZE of 4 GiB (which is, incidentally, the max
zisofs-compressible file size) would trigger an overflow in the
calculation of the number of block _pointers_ they would require. We
could have rejected those files, but the truth is that they just require
exactly 131073 block pointers (131072 for the file itself and one for
the trailer.)
Writing a file near 4 GiB causes an overflow when we calculate the
number of block pointers we need to store, which leads to us allocating
the wrong amount of memory to store them.
This is visible with ASan:
Exercising: libarchive 3.9.0dev zlib/1.3.1 liblzma/5.4.4 bz2lib/1.0.8 libzstd/1.5.6 cng/2.0 libb2/bundled
706: test_write_format_iso9660_zisofs_overflow
=================================================================
==40444==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x08c004b4 at pc 0x006acb3b bp 0x002edb44 sp 0x002edb38
WRITE of size 1 at 0x08c004b4 thread T0
#0 0x006acb3a in archive_le32enc libarchive\archive_endian.h:219
#1 0x006c81d0 in zisofs_write_to_temp libarchive\archive_write_set_format_iso9660.c:7760
#2 0x006ca1bd in write_iso9660_data libarchive\archive_write_set_format_iso9660.c:1785
- Use contrib/untar's system_mkdir for extended portability
- Compile warning function only if it's actually used
- Use %H:%M instead of %R in strftime for extended portability
- Introduce system_unlink to add special directory symlink handling
Copy missing functionality from libarchive and cpio.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Crafted TOC with a base64-encoded <name> of odd length, which used to
make strappend_base64() read past the character-data buffer. The trailing
incomplete base64 character is dropped; the rest still decodes.
tar: mark bsdtar as long-path aware on Windows
tar -C/--directory option calls SetCurrentDirectoryW(), which is limited to MAX_PATH (260) characters unless the process opts in to longpath support via an application manifest and enables a registry setting or sets an option in developer settings.
Verify that symlink creations and overwriting works, i.e. that dangling
symlinks are properly overwritten.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Verify that -u does not override files which have a newer timestamp.
This also verifies that creation of timestamps properly works.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Verify that -f does not create new files and also does not override
files which have a newer timestamp.
This also verifies that creation of timestamps properly works.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
bsdtar's -C/--directory option calls SetCurrentDirectoryW(), which is
limited to MAX_PATH (260) characters unless the process opts in to long
path support via an application manifest declaring longPathAware.
Extracting into (or changing into) a directory whose path exceeds 260
characters fails with "could not chdir".
Embed a longPathAware application manifest into bsdtar.exe so these
operations work for long destination paths. Both Windows build systems
(CMake and the autotools/Makefile.am build) embed it, using different
mechanisms per toolchain:
- MinGW/Clang (GNU ld) do not generate an application manifest, so it
is compiled from a resource script (bsdtar.rc) via windres. This
manifest replaces the toolchain default, so it also declares the
asInvoker execution level explicitly.
- MSVC's linker already embeds a manifest (resource id 1); a duplicate
RT_MANIFEST resource fails to link (CVT1100). Instead the manifest is
merged into the linker's default manifest (asInvoker) by adding it to
the target sources.
The local functions be32dec and le32dec lead to conflicts on NetBSD. Use
a proper prefix as suggested by Dustin L. Howett.
Resolves#3170.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The input file in test_P_encryption uses deflate, which is only
available if zlib is available. Skip the test if built without zlib.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Allocate enough memory with initial malloc for possible additions, which
removes any kind of error handling required in idr_extend_identifier.
The additional memory allocated was already large enough. Clarify
exactly where the required amount comes from by using named constants.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Do not set 0 as error code, since this will be printed as success. Use
proper error codes in various places for better error messages.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Add archive_ckd_add_size and archive_ckd_mul_size. Use them for checked
arithmetic throughout the source code where explicit SIZE_MAX checks
were performed.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
entry_bytes_remaining goes negative for a streaming entry after the AES
header is subtracted; zip_read_data_zipx_xz passed it to zip_read_decrypt
as a size_t, causing an out-of-bounds read.
Co-Authored-By: Marco <marco@mpreimann.de>
Set the third argument to NULL if the amount of actually read bytes is
not needed. This simplifies code audits.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If a filename is larger than INT_MAX, eventual size_t/int/size_t
conversions lead to huge size_t values and subsequent OOB reads.
Use correct values. Before this would lead to unnecessary writes, the
filename length field check already aborts the operation.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If the filename length cannot be represented as octal in its field,
return an error instead of silently truncating the value.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Archives are not necessarily 256 bytes long, e.g. the last part of a
multi volume archive. Only try to read up to 256 bytes when looking for
an archive or disk name, not necessarily exactly 256 bytes. This matches
the behavior of regular entry name lookups.
Also, disk names can be empty. Handle this as well.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The cab reader does not support multi volume archives, but it should be
able to read the files to print information about content.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Make sure that the content of the link can fit into a size_t. This
should be always true, but be cautious with 32 bit systems and very
weird filesystems (possibly through fuse).
I took SSIZE_MAX as upper limit due to signedness and eventual readlink
calls which would fail with larger values anyway.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Use size_t for avail_in, avail_out and stream_in for ppmd streams.
The fields avail_in and avail_out values are set in function decompress
based on size_t variables (t_avail_in/t_avail_out) and eventually
written back. The stream_in field is only incremented.
The actual use case happens within ppmd_read to support situations in
which not enough bytes are available. In such cases, more bytes are read
on demand but not written into next_in.
In such cases, avail_in can turn negative and next_in can point outside
of its allocated memory area.
Since stream_in is always incremented by one, it won't overflow on real
hardware, given that size_t would address the whole available heap
space.
Make sure that avail_in never turns negative (which allows the size_t
usage) and also guarantee that t_avail_in will never wrap around,
leading to a huge "used" value.
As a bonus, __archive_read_ahead can be reliably called with a NULL
argument now, since no more casting occurs for second argument, which
was missing in the test.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The cab_read_ahead_remaining function might return more bytes through
avail than initially asked for. The given limit is 255+1, i.e. the
maximum file name length.
While it's not a big deal for libarchive to handle file names longer
than that, the CAB format does not allow longer names.
Limit the amount of available bytes to the given argument for proper
checking.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If not enough bytes are available, __archive_read_ahead will return the
amount of bytes still available, which can be larger than 0. Only in
error cases, a negative value is returned.
Check the return pointer instead. It simplifies the error handling,
allows a NULL argument, and covers more truncation issues.
Right now, a NULL pointer with a non-zero size could be further
processed, which just asks for more technical or logical issues to
arise.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Avoid requesting the wide-character pathname when the tar reader only
needs to check whether a regular entry name ends in '/'.
archive_entry_pathname_w() can lazily convert the pathname to WCS. In the
common tar read path, the multibyte pathname is already available, so
checking it first avoids unnecessary per-entry conversion. The WCS fallback
is kept for cases where the multibyte pathname is unavailable.
The archive_integer.h header offers various integer operations to
libarchive, utilizing system functionality where possible.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
For empty-prefix ustar entries, copy the fixed-width name field
directly into the archive entry instead of first building a temporary
archive_string.
This avoids a temporary buffer allocation and intermediate copy in the
common case. It also fixes a small fatal-error leak by freeing the
temporary prefix/name buffer before returning on pathname conversion
failure.
lz4: Fix double-free on reallocation failure
Alternative version of https://github.com/libarchive/libarchive/pull/2945 which removes the test (which requires a modified malloc to actually fail the 4 MB allocation).
lz4_allocate_out_block() frees state->out_block without NULLing
the pointer. If the subsequent malloc fails, the function returns
ARCHIVE_FATAL with a dangling pointer. lz4_filter_close() later
calls free(state->out_block) again, triggering a double-free.
Also, state->out_block_size was updated before checking if malloc
succeeded, leaving inconsistent metadata on failure.
Fix both lz4_allocate_out_block() and lz4_allocate_out_block_for_legacy():
- NULL the pointer immediately after free
- Move size update to after malloc succeeds
- Reset size to 0 on allocation failure
Validate the accumulated unpack stream count before allocating
per-substream metadata.
Reject folders that declare multiple substreams without kSize metadata,
since the parser needs those sizes to describe the stream layout.
Drop the explicit digest zeroing loop. The arrays are allocated with
calloc(), so the entries are already initialized.
Add a 7z regression test for malformed SubStreamsInfo metadata that
declares more than one unpack stream without the kSize data needed to
describe those streams.
Store the archive as a .7z.uu file, matching the existing malformed
7z tests.
This is overly broad, and will reject some well-formed archives which
contain symlinks to trees which exist in the archive; however, this is
the best we can do without some rudimentary path parsing.
Fixes GHSA-q4gj-vh8c-67p8
This PR does not fix any reachable issue, but fixes the code in question nonetheless to prevent regressions in the future:
- Do not call `archive_copy_error` after `archive_read_free` to prevent a user after free bug
- Reset `vtable` to `NULL` to prevent `close` from being called after filter initialization error, since `data` is already freed and set to `NULL`, preventing a `NULL` pointer dereference
If a system with sizeof(wchar_t)=2 (e.g. Cygwin) tries to convert a wide
character string into a multi byte string representation, it
precalculates the required length with sizeof(wchar_t) instead of
MB_LEN_MAX. This can lead to short memory allocation for filenames which
have a shorter representation in wchar_t than in UTF-8.
Resolves GHSA-crm5-q56g-xw29.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
A system with sizeof(wchar_t)=2 (Cygwin on Windows) can trigger an out
of boundary write in archive_read_open_filenames_w when converting the
wide character string into a multi byte string.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The current finite state machine carefully handles short reads, i.e. the
loop can enter as often as needed until enough bytes arrive for the
current state to perform its actions.
This can be simplified by relying on __archive_filter_read_ahead to
return the amount of bytes actually needed. I assume that this did not
happen in the original code due to its age (2009) and evolution of
libarchive's internals over time.
Also, headers are only skipped at the beginning. As soon as the reader
starts returning data (ST_ARCHIVE reached), the filter pretty much
becomes a pass-through filter.
Split the initial lead and header skipping into its own function and
only keep track if the initial skipping was performed or not. This
greatly simplifies the reader function.
Also, it avoids book keeping of internal states and "total_in" tracking,
which I don't have to properly audit for edge cases anymore.
Last but not least, this refactoring properly reports truncated streams
now.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Allocate nulls only when needed, i.e. when sparse files are encountered.
This removes another malloc call from regular use cases.
While at it, call archive_set_error in case of allocation failure for
better error messages.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If lseek fails to seek to correct position, e.g. because the descriptor
is in append mode, do not use errno because it's not set.
In such a case, fall back to ARCHIVE_ERRNO_MISC.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The POSIX and Windows code did not check the result of
archive_entry_clone. Handle this error condition properly.
While at it, unify POSIX and Windows code a bit.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
LLP64 could theoretically overflow the 32 bit counter. It still needs a
lot memory to achieve this, but let's better be safe than sorry.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Check if allocation succeded. If not, properly handle the error instead
of eventually performing a NULL pointer dereference.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
We do not have to guess the amount of remaining bytes, since
archive_read_ahead returns the information.
Simplify the function.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
A malformed RAR5 archive with data_size=1 forces bytes_remaining
(ssize_t) to wrap to -2 when a compressed block header consumes
to_skip=3 bytes (CWE-191). That negative value is then implicitly
cast to size_t ~0 inside malloc(), requesting a ~16-exabyte
allocation — confirmed heap buffer overflow via ASAN/UBSan on a
48-byte crafted archive requiring no authentication.
Three guards added to archive_read_support_format_rar5.c:
1. Reject data_size > SSIZE_MAX before assigning to bytes_remaining
(CWE-195, unsafe unsigned-to-signed conversion)
2. Reject to_skip > bytes_remaining in process_block() before the
subtraction — this is the primary fix for the underflow (CWE-191)
3. Change cur_block_size == 0 to cur_block_size <= 0 in merge_block()
as defense-in-depth so that any negative bytes_remaining reaching
read_ahead() is caught before it becomes a malloc size (CWE-122)
A malformed RAR5 archive with data_size=1 forces bytes_remaining
(ssize_t) to wrap to -2 when a compressed block header consumes
to_skip=3 bytes (CWE-191). That negative value is then implicitly
cast to size_t ~0 inside malloc(), requesting a ~16-exabyte
allocation — confirmed heap buffer overflow via ASAN/UBSan on a
48-byte crafted archive requiring no authentication.
Reproducer: 48-byte crafted RAR5 archive; ASAN confirms
"allocation-size-too-big 0xfffffffffffffffe".
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Sometimes, 24 bit operations are needed. Enough users exist that moving
this into archive_endian.h is worth it.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Fix an unsigned integer overflow in get_pe_sfx_offset to properly
address the correct section.
Reported by shxz9u.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If vtable is not set to NULL, close function would be called during
shutdown. Since data is already freed and set to NULL, this would lead
to a NULL pointer dereference later on.
The called library functions should never fail though, so this is a
purely defensive measure against future lzma changes.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If archive_read_next_header in add_pattern_from_file would ever return
anything but ARCHIVE_OK or ARCHIVE_EOF, a use after free would occur
when copying error information.
Since this is impossible with current setup (format raw without any
further filter, thus only open_filename code), this change is a purely
defensive measure against future changes.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If a filename is larger than INT_MAX, an out of boundary write could
occur on 64 bit systems. Fix data types and ignore such huge file names.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If the original name cannot be duplicated, return ARCHIVE_FAILED instead
of ARCHIVE_WARN. The latter implies that the option is unknown, which is
not the case.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
All arithmetical operations are unsigned, and it makes sense to keep it
unsigned: The total_in value is written at the end of the stream and if
the value overflows, it's pretty much expected to be % UINT32_MAX.
Very unlikely that int64_t will ever overflow, but the fix is cheap.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The CreateSymbolicLinkW function is available since 0x0600 and is also
part of the Nano Server APIs. On earlier systems, don't even try.
Otherwise use it directly to simplify code.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
- Use a stack array for 22 bytes
- Entering the if-branch already implies that we will add data
- Use snprintf instead of strcpy
Even though snprintf is slower than strcpy, it's easier to verify and
since nobody complained so far about the malloc overhead, this should be
okay (for now).
As a bonus, this code cannot fail anymore, which previously meant that
file attributes were silently ignored.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The oldest version we support is 0x0502, while _WIN32_WINNT_XP is
0x0500. Drop support code for even older versions.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Define the macro WINAPI_FAMILY_PARTITION if it's missing, which can
happen on old Windows versions like Windows XP. Also add other missing
definitions to support compilation.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Use proper memory barrier while checking for availability of function
WaitForInputIdle.
Since Vista, InitOnceExecuteOnce is a very simple idiom to handle such a
singleton setup. For earlier setups, accept a possible data race and
recover gracefully if two concurrent threads performed the initial
setup.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Move a static variable into struct archive_write_disk, which is a safe
location from a thread perspective.
Only create and use it if it's really needed, i.e. for systems which
lack support for extended attributes.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
This avoids all possible concurrency issues.
As a bonus, turn the whole array const to move it into read-only
section.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
This avoids all possible concurrency issues.
As a bonus, turn the whole array const to move it into read-only
section.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
This avoids all possible concurrency issues.
As a bonus, turn the whole array const to move it into read-only
section.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
It is possible to trigger an out of boundary write on 32 bit systems
with around 1 GB of data (with a line consuming most of that data) when
opened with archive_read_open_memory.
Cap the amount of data read at once at 2 * UUENCODE_BID_MAX_READ to
allow range checks to take place before a possible SSIZE_MAX overflow
can occur through avail_in. Also, discard any line longer than
UUENCODE_BID_MAX_READ since this should definitely be more than
enough, especially since in_cnt check already takes care of that.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The check for end of file (`avail == 0`) happens after the check for a `NULL` return value of `__archive_read_filter_ahead`.
Remove it, because at this stage, it's definitely truncated, not just end of file.
7zip: fix a number of issues in zstd detection
- -Wunused-function when ZSTD_compressStream is unavailable
- Incorrect automatic selection of 7Z_ZSTD when ZSTD_compressStream is unavailable
- Other instances of HAVE_ZSTD_H not matching HAVE_LIBZSTD
Co-authored-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Without this fix, the 7zip writer will fall back to zstd (when it is the
last available option) even if it could not be linked, then fail at
runtime with an unexpected error message.
The XXH32_init function allocates memory. Check if the allocation was
successful and return ARCHIVE_FATAL on error.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Some filter comments were not updated when the execution of external
filter programs was introduced.
Sync them with reality, including the actually performed commands.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The 7-ZIP archives written by libarchive can be streamed. Support such
archives by using a fallback if seek is not supported: As long as the
target position is somewhere ahead in stream, consume bytes until the
position is reached.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The third argument (available bytes) is not always evaluated. It is
perfectly fine to pass a NULL argument if it's not needed.
Also, some checks of the third argument can be removed, because in case
of an error (< 0) or in case of end of stream (0) the return value is
NULL anyway. If a NULL check is already in place, just use it.
Simplifies code audits because it's obvious that the available bytes are
not needed within the function if they are never stored.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Do not assume that enough bytes will be provided by filter if not
explicitly requested. The requested size is 76, but some checks expect
116 bytes. If these were not supplied by filter, th afio header
detection erroneously skips headers which otherwise could be found.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If enough bytes have to be skipped, a signed integer overflow could
occur (most realistically on 32 bit systems). Use an unsigned type,
which could still overflow but has only negative impact on diagnostic
warning message.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Verify that afio headers are detected even on unfortunate filter read
splits, i.e. when magic is found but rest of header has yet to be
retrieved.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
process_base_block() returned ARCHIVE_RETRY for HEAD_MAIN (and
HFL_SKIP_IF_UNKNOWN) blocks without consuming the body bytes the
sub-parser did not read. rar5_read_header() then re-parsed the same
region, turning an O(1) skip into O(N) and letting a crafted RAR5 file
stall the reader (GHSA-9h2c-464f-j3hj).
Record the block body start and skip any unconsumed bytes through a
small helper rar5_skip_remaining_block() before returning ARCHIVE_RETRY.
Add two regression tests derived from test_read_format_rar5_stored, each
with extra unread bytes appended to a no-data block's body (HEAD_MAIN and
an unknown HFL_SKIP_IF_UNKNOWN block); both fail on master and pass with
the fix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
If __archive_read_filter_consume fails, report information about
detected truncation.
In many cases, the return value can be ignored since
__archive_read_ahead already performed the check. In this case, we never
read the data before, so add the check here.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Integer promotion stops at unsigned int, thus addition of 4 to a
uint32_t can lead to an integer overflow.
Use int64_t to support all values.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Switch the release-time config.guess/config.sub fetches from http:// to
https:// and add -fsSL so a failed fetch errors out instead of writing an
error page into the helper script (CWE-494).
Apply same logic of gzip_filter_read from commit
256c71ad4e
to consumer_header, which contains the identical issue.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The variable is never read and it only allows a very unlikely integer
overflow if more than INT64_MAX bytes are read.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The pointer arithmetic could overflow with a large min value. Prevent
this by using subtraction instead of addition.
Resolves#3025.
Co-authored-by: ypp <alen0421@gmail.com>
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The check for end of file (avail == 0) happens after the check for a
NULL return value of __archive_read_filter_ahead.
Remove it, because at this stage, it's definitely truncated, not just
end of file.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If not enough memory is available, skip tests. This can happen on 32 bit
systems with ASAN enabled. While these tests run perfectly fine if run
directly with libarchive_test, the test harness fails with eventual NULL
pointer dereferences, since these assert-checks do not stop processing.
Happens with lzma and xz and compression level 9.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Solaris iconv may perform an implementation-defined conversion for
non-identical characters instead of failing. In the tested conversions it
substitutes question marks and reports success, so these negative tests do
not match Solaris behavior.
On systems with a 32 bit time_t, archive_parse_date covers less values
than with a 64 bit time_t.
Adjust the expected return value for an overly large value.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The codec field is of type uint64_t, which is an unsigned long on many
64 bit platforms (LP64), but 32 bit platforms and LLP64 platforms cannot
use the l modifier for correct output.
Cast to uintmax_t for portable j support.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The Rock Ridge rr_move functionality for paths with a depth larger than 8
has a few bugs in it. For one, it might loop over the same entries over
and over. And second, rr_move/ directory entries are not verified to be
unique. Both issues can lead to NULL pointer dereferences.
Add a test case which highlights the NULL pointer dereference.
Based on issue #2936.
cpio: Fix UAF in error path
Add entry only after its full initialization into list. Otherwise the error handling of a failing strdup would have to unlink the entry again.
Fixes: 16ad931073 ("cpio reader: Validate pathname in record_hardlink")
Resolves#3053.
In case of errors, assertEqualIntA offers more information by accessing
archive's errno and error message.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The CMake build script was modifying global CMake variables (CMAKE_REQUIRED_LIBRARIES
and CMAKE_REQUIRED_INCLUDES) during crypto library checks (OpenSSL, MbedTLS, Nettle)
and Haiku libbsd checks without saving/restoring them using
CMAKE_PUSH_CHECK_STATE() and CMAKE_POP_CHECK_STATE().
This caused side effects where subsequent system-level checks (like
CHECK_TYPE_SIZE or CHECK_FUNCTION_EXISTS) inherited these library dependencies,
leading to incorrect feature detection in cross-compilation environments
(e.g., reporting that basic types like 'pid_t' are missing).
This patch ensures all such checks are properly scoped, improving build
robustness across different architectures (x86_64, AArch64, MIPS64, LoongArch64, etc.)
and cross-toolchain environments.
The slurp_central_directory function has been changed to always use
seek, which is not needed and also breaks streamable 7-Zip archives.
Resolves an issue observed in #3068.
Fixes: 71ef1c3852 ("7zip: Merge skip_sfx into get_data_offset")
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The C standard dictates that integer promotion increases data types up
to int or the largest argument in arithmetic.
This becomes problematic in `offset_in_buffer += 4 + frame_data_size`
which effectively is `size_t += int + uint32_t`.
If frame_data_size is UINT32_MAX, adding 4 overflows before the
overflown value is added to size_t.
On 32 bit systems, this can easily lead to an endless loop or out of
boundary access.
Fix this by properly checking if integer overflows would occur.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
It is possible to trigger an endless loop on 32 bit systems due to
a triggerable integer overflow.
Add a regression test for this.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Use size_t where size_t arguments are expected (and cast avail to size_t
because we know at this point that it cannot be negative).
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Use C comments, define variables at start of block, keep 80 chars per
line. While at it, turn a few variables const to clarify that they do
not change.
No functional change.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
In case of errors, assertEqualIntA accesses the given archive and prints
diagnostic information about errno and error string.
Since archive_read_free and archive_write_free free the memory of the
archive, this would just lead to memory issues in case of errors.
Use assertEqualInt instead.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
_warc_header() initialises a local archive_string hdr, fills it via
_popul_ehdr(), and frees it on the success path at the bottom. The
intermediate error return on r < 0 (the populated header exceeds
MAX_HDR_SIZE=512, reachable with a long enough pathname carried by
WARC-Target-URI) returns ARCHIVE_WARN without freeing hdr.
By the time _popul_ehdr returns -1 hdr already holds the WARC version
line, WARC-Type, WARC-Target-URI (long path), WARC-Date, Last-Modified,
WARC-Record-ID, and Content-Length headers --- ~1 KB+ per entry in
practice.
Reproduces with the bundled bsdtar against a regular file with a
~400+ char pathname:
ASAN_OPTIONS=detect_leaks=1 bsdtar --format=warc -cf out.warc <long_path>
=> LeakSanitizer: ~1 KB direct leak; stack pierces _popul_ehdr ->
_warc_header (archive_write_set_format_warc.c:245) ->
_archive_write_header -> ... -> main.
Code unchanged since the file's introduction in 2014; no existing test
exercises the long-pathname path. The warcinfo branch above (line ~217)
handles this correctly already (frees hdr unconditionally outside the
"if (r >= 0)" block); only the WT_RSRC branch was missed.
Identified by Neurolog, a code-analysis tool the reporter is developing
that combines Souffle Datalog with LLM-assisted fact extraction. The
reproducer was hand-validated under LeakSanitizer against current master.
If init_unpack fails to allocate memory for given window size, a
dangling pointer can lead to an eventual double free.
Resolves#3069.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
file_new() at archive_write_set_format_7zip.c:1688 calls free(file)
on the symlink-UTF8-failure branch, leaving file->utf16name
(allocated at line 1666) leaked. The two earlier free(file) calls
in this function (lines 1656, 1668) are correct because they happen
BEFORE utf16name is allocated, but the third one happens after.
The rest of the function uses file_free() on every other
post-utf16name error path; file_free() does free(file->utf16name)
followed by free(file), which is the cleanup convention. Replacing
free(file) with file_free(file) makes the symlink-error branch
consistent with everything else.
Reproduces with bsdtar in a non-UTF-8 locale with a non-UTF-8
symlink target:
ln -s "$(printf 'broken_\\xff\\xfe_link')" sym
LC_ALL=C ASAN_OPTIONS=detect_leaks=1 \
bsdtar --format=7zip -cf out.7z sym
=> LeakSanitizer: 30-48 byte direct leak; allocation site is
file_new (archive_write_set_format_7zip.c:1666).
Equivalent trigger: any libarchive caller that sets AE_IFLNK
filetype on an entry without ever calling
archive_entry_set_symlink() (then archive_entry_symlink_utf8()
returns NULL and the error branch fires).
Code unchanged since the 7zip writer's introduction; no existing
test exercises this error path.
Identified by Neurolog, a code-analysis tool the reporter is
developing that combines Souffle Datalog with LLM-assisted fact
extraction. The reproducer was separately validated under LeakSanitizer
against current master.
file_free() releases pathname, symlink, uname, gname, and hardlink but
omits fflags_text. When a XAR archive describes a file with <flags> or
<ext2> children (e.g. <SystemNoUnlink/>, <Compress/>), xml_parse_file_flags
/ xml_parse_file_ext2 populate xar_file->fflags_text via archive_strcat,
which heap-allocates. The buffer leaks on every file_free().
Reproduces with ASan+LSan via the bundled bsdtar:
bsdtar -tvf <xar-with-flags>
=> Direct leak of N bytes ... archive_strcat ... xml_parse_file_flags
Same shape as commit 6767cbe3 ("Free XAR xattr fstype metadata"), which
fixed the analogous miss in xattr_free().
Existing release of fflags_text in archive_string_free is a no-op when
the field was never populated (.s == NULL, free(NULL) is safe), so the
patch is harmless on the non-flags path.
Resolves#3058.
Avoid passing negative suffix offsets to the duplicate identifier
resolver. Clamp them where ISO9660 and Joliet identifiers decide
where the duplicate suffix should be written.
When libarchive is compiled with FreeBSD's native iconv instead of
libiconv, as happens with libarchive in the base system, we need to
configure iconv(3) to handle invalid sequences by returning -1, as
iconv_strncat_in_locale() assumes GNU iconv semantics.
Add entry only after its full initialization into list. Otherwise the
error handling of a failing strdup would have to unlink the entry again.
Fixes: 16ad931073 ("cpio reader: Validate pathname in record_hardlink")
Resolves#3053.
The integer-code dispatch table in archive_write_add_filter() maps
ARCHIVE_FILTER_LZOP to the function pointer archive_write_add_filter_lzip
(the lzip filter setter) instead of archive_write_add_filter_lzop.
Add a test for ISO9660 entries whose Joliet identifiers collide
after default identifier truncation. The writer should resolve
the duplicate identifiers successfully when the archive is closed.
Apply the same fix as done in commit
c246ec5d05 for non-wide character
strings.
Without this, empty strings lead to out of boundary read accesses.
Resolves#3046.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Make sure that empty wide character names do not lead to out of boundary
accesses.
Reported by shura-turtle.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The debug_buff array is populated but never read. If multiple threads
process compress streams, they access the same index variable, which
could lead to out of boundary writes.
Delete the debug code to fix this issue.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If an archive has more than UINT_MAX (4 GB) bytes available, the bzip2
filter will enter an endless loop.
Apply same logic of gzip filter from commit
256c71ad4e.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Low-distance repeat state is scoped to the current RAR3 LZ table. Reusing it after a new table starts can make the next match use the wrong offset, producing incorrect output and a file CRC error.
Reset lastlowoffset and numlowoffsetrepeats when parsing a new LZ table. Add a small handcrafted RAR3 fixture that leaves a low-distance repeat pending, switches tables, and verifies the decoded bytes after the boundary.
RAR3 recovery records can be stored as NEWSUB RR blocks with payloads larger than the old 1 MiB extended-header limit. These records are metadata-only for this reader, so after validating the header and guarding against header-size plus payload-size overflow, consume the remaining header bytes and payload instead of rejecting the archive.
Add a reference archive with a NEWSUB RR payload just over 1 MiB, and keep the malformed huge-size case covering the overflow/truncation path.
Currently, the code calls GetTempPathW to figure out required size for a
buffer larger enough to contain the temporary directory path, allocates
the memory, and then calls GetTempPathW again to populate the memory.
Since libarchive is designed with multi-threading in mind, the worst
situation would be that another thread modifies the environment variable
between these two calls.
Use a buffer of MAX_PATH + 1 (261) to basically cover all regular
situations. If long paths are enabled, reallocate until enough bytes
were available (32 kb is maximum) without another thread intefering.
Realistically, this will happen only once.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Reject malformed CPIO entries whose pathname field exceeds 1 MiB before asking the read-ahead layer to satisfy the padded pathname length.
This prevents newc archives with attacker-controlled c_namesize values from forcing large metadata read-ahead and pathname allocation during archive listing. Add a regression test that fails on the unpatched reader and passes once the cap is enforced.
Signed-off-by: datauwu <datauwu@users.noreply.github.com>
If multiple fflags are added to archive, a stack out of boundary is
triggered due to insufficient array allocation.
Also, if flags with long names are supplied, an out of boundary read can
occur.
Resolves GHSA-wfvr-54j8-47r9.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
It is possible to trigger an out of boundary stack write and out of
boundary reads with fflags.
Based on proof of concept reported by Cris Jacob Maamor.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
It is only relevant to know if a separator has to be inserted. This can
be simplified by only checking if string already contains characters.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Since libarchive already contains infrastructure to safely work with
strings, use it instead of manually constructing strings.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
window_buf and filtered_buf were allocated via calloc without checking
for NULL. Change init_unpack to return int and propagate ARCHIVE_FATAL
on allocation failure to the caller.
Add documentation for
archive_free
archive_position_compressed
archive_position_uncompressed
archive_read_open_filenames
archive_write_add_filter_by_name
archive_write_set_format_filter_by_ext_def
Note that archive_write_add_filter.* can return ARCHIVE_WARN, and
archive_read_support_filter.* can return ARCHIVE_FATAL
Chase rename of archive_position to archive_filter_bytes
The strcpy and sprintf functions are generally hard to reason about.
While they are safe in this context, I think, it's easy to refactor the
code to avoid them completely.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The code can be simplified to avoid strcpy usage. While not exactly a
much safer approach by manually adjusting characters in a string, this
attempt reduces size of libarchive and avoids unneeded copy operations.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The cmdline_set_path function contains logic to reallocate memory of
command path and copy a new value into it.
Inline a simpler free+strdup alternative into __archive_cmdline_parse.
Since the function is always called with empty data, free could be
removed as well. I've kept it that way just to make sure that it's 100 %
compatible with previous code.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Switching a multi-volume archive file with another active filter, e.g.
decompression, can lead to memory corruption due to modifying the wrong
private data (self->data).
Use highest upstream filter to replace the correct private data.
Resolves GHSA-qf8j-cq3h-8m2m.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Check that no memory corruption occurs when switching files which are
decompressed through filters.
Test case supplied by Stuart Wallace.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Use a dedicated variable for archive instead of referencing
self->archive again and again. Also, inline client_open_proxy.
While at it, fix space intendation.
No functional change.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Parsing a string into an integer also means that boundary checks have to
be performed. Check within atou64 as well as outside by verifying that
subsequent casts won't truncate numbers.
Use error handling available by callers.
Reported by Muhammed Hussein.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The result of atol8 is cast to mode_t, which is also unsigned.
This brings atol8 closer to atol10.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The xml_data function is void, but some functions called in there might
fail. Prepare an error propagation for upcoming changes.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The variable is size_t and argument expects a size_t. Do not cast to
int, which could only lead to trouble (in theory).
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Add a test which contains a mode which does not fit into mode_t and
check if such an entry is properly rejected.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
- Release memory in case of allocation failure.
- Treat failure to add an entry to rr_moved as fatal error
Fixes ASAN test run with test_write_format_iso9660_null_deref
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The XAR xattr cleanup helper released the xattr name string but not the parallel fstype archive_string. Release fstype from the same xattr_free() ownership boundary.
Add a focused XAR sample with xattr fstype metadata and exercise it through the public read/free path. This gives leak-checking builds coverage for the xattr cleanup ownership boundary.
The len variable is used later on. Introduce a new one (debug length) to
limit the amount of characters written to stderr.
Could be further improved with %.*s but let's keep it as simple as
possible, since DEBUG is not default and has to be manually set in
line 89.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
A malformed Rock Ridge CE entry can point back into a continuation
area that has already been processed for the same file. This causes
the ISO9660 reader to parse the same SUSP records repeatedly.
Track the end offset of the CE data already processed for each
file_info and reject later CE entries whose start offset is before
that point. This keeps forward nested continuations valid, including
same-block continuations, while rejecting backward or overlapping CE
chains.
This check is done in read_CE(), where continuation data is actually
processed.
Check that archive_entry_sparse_reset resets all internal variables.
Test case created and issue reported by Linke Li.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Avoid requesting NEWSUB extended data through read-ahead while parsing the header. The full NEWSUB block size is still validated and consumed, but the extended data is not required to be present in memory during header parsing.
Add a test for a malformed NEWSUB header with a large packed size.
Allocate enough memory for possible addition of 3 characters within the
range of 0-Z. Since UTF-16 is in use, allocate 6 bytes + 2 bytes for the
terminating NUL character.
Also keep in mind that "l" is already size in bytes, which means that a
multiplication of 2 is not needed (and prevented overflow issues with
longer filenames).
Resolves#2935.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
It is possible to trigger an out of boundary write with short filenames
which contain illegal ISO9660 characters. For these files, Joliet IDs
are generated. If multiple files lead to the same ID (which can happen
because illegal characters are replaced with an underscore), 3
characters/digits in the range of 0-Z are added.
The addition could overflow the memory.
Based on PoC in #2935.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
3 characters/digits base 36 means that 46656 combinations are possible.
If a directory with even more conflicting identifiers is encountered, the
code would trigger an endless loop.
Fail with an error in such a case instead.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Strings pointed to by these variables are actually modified. They point
to modifiable data areas (own stack arrays or argv arguments), so the
code does not erroneously modify them. Instead, clarify that they are
modifiable by removing the qualifier.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Since glibc-2.43:
For ISO C23, the functions bsearch, memchr, strchr, strpbrk, strrchr,
strstr, wcschr, wcspbrk, wcsrchr, wcsstr and wmemchr that return pointers
into their input arrays now have definitions as macros that return a
pointer to a const-qualified type when the input argument is a pointer
to a const-qualified type.
Signed-off-by: Rudi Heitbaum <rudi@heitbaum.com>
The supplied nanoseconds of time keyword could be truncated due to
casting from int64_t to long (relevant for Windows and x86), resulting
in an incorrect value.
Since the implementation already caps the value at specific limits for
bug compatibility, just use the correct data type for parsing to not
make things worse.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
According to 7zip standard, up to 15 bytes can be used for the Codec ID,
but not more than 63 bit shall be used (by today):
- Switch to uint64_t to avoid 32 bit unsigned long on Windows
- Verify that Codec ID parser does not overflow the integer
If the Codec ID does not fit, treat it as a file format error.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If no flags are supplied, anchor flags are supposed to be not special.
This means that ^ at the beginning of a pattern should be treated as a
regular character.
This breaks current behavior, but complies with comments in code, i.e.
archive_pathmatch.h line 41/42.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The errno value is not correct here. If the error string of a failed
match is printed, use also its error code.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Patterns with a lot of asterisks may overflow the call stack, crashing
the application. Check the recursion depth. If it is too deep, fail
with an error.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
These functions can return negative values, in which case operation
itself failed. While internal libarchive libraries handle these cases,
the tools don't. Check for negative values in them as well.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
(UBSan occasionally finds something interesting and
often reports whacky non-bugs like this one. "Fixing"
it will make the real UB bugs easier to identify, so...)
According to C's integer promotion rules, `unsigned short` gets
promoted to _signed_ `int`, and shifting into the sign bit of an `int`
is technically UB. Explicit cast to `unsigned` quiets UBSan.
1. The XAR writer's path normalization code uses strcpy() to move
parts of a path string within the same buffer. The source and
destination ranges overlap, which is undefined behavior for strcpy().
2. Failure to check string length before accessing the last character
of a path component. For empty components (e.g., //), the length is 0,
and length-1 underflows to SIZE_MAX.
The bracket expression matching [ in the pathmatch engine fails to
handle malformed patterns, specifically when a closing ] is missing or
when high-byte characters are used. The scanner advances the pattern
pointer beyond the allocated buffer.
The ACL parser fails to validate buffer length when processing PAX
attributes (SCHILY.acl.access/default). The next_field() function
attempts to read a separator character from a pointer even when the
remaining length is zero.
Reported-by: Kamil Frankowicz
If the basename is '.', it means it is the root directory ('/'). Do not
append '/' to '.', as this will produce a path '/.', resulting in an
invalid mtree entry.
For example, using base.txz from FreeBSD:
tar -c -f - --format=mtree @base.txz | mtree -C
Only the Unix epoch format `@<timestamp>` can have a number with
more than 4 digits. So let's break out the numeric parsing into
a standalone uint64 parser and use it separately to parse epoch
timestamps (which are only limited by the range of time_t) and
other date components.
It also avoids a time-consuming leap-year correction for
nonsensically large year values.
Check for truncations before casting uint64_t to size_t for 32 bit
system safety.
Reported by Asaf Meizner.
Resolves additional notes in GHSA-rf5v-vf7c-6wvg.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
`list.h` contains a list of all the tests and is generated
by grepping the test source files for `DEFINE_TEST`.
Previously, it was generated at configure time.
This meant that if you added a new test to an existing
source file, you had to manually reconfigure.
This adds the necessary dependencies so that `list.h`
is regenerated whenever any C test source changes.
This ensures that new tests are always discovered automatically.
Note: If someone wants to update the autoconf-based
build system to do this, please send a PR.
As noted by @dunhor, the logic here was a little unclear.
Reorganize to do all the checks, then set up the decode
parameters only if the checks pass.
This has no functional change, it's just a legibility improvement.
The change to return FAILED for entry-specific issues uncovered
flaws in RAR5 handling of filter types:
* Supported filter types are verified in `parse_filter` and
were being also checked in `run_filter` -- the duplication
confused the error handling here.
* `do_uncompress_file` was only checking for `FATAL` from
the upstream filter logic, so failed to properly pass
`FAILED` errors through
The change to return `FAILED` instead of `FATAL` for issues that
impact a single entry (but don't necessarily terminate the entire archive)
created a bug in header parsing since `FAILED` wasn't handled in a
header-check loop.
Thank goodness for fuzzing CI!
archive_read_format_7zip_read_data_skip used to coerce any negative
skip_stream() return into ARCHIVE_FATAL. That is wrong in principle:
ARCHIVE_FAILED can legitimately propagate up from setup_decode_folder()
through read_stream() and skip_stream(), and the wrapper should not
upgrade it.
In the current encryption-partially test case this is empirically a
no-op because skip_stream() still returns ARCHIVE_FATAL via a second,
deeper code path through extract_pack_stream(). An inline TODO comment
flags that asymmetry for a follow-up audit.
Follow-up to 9fa772ab. An audit of the rar5 reader found many more
ARCHIVE_FATAL returns in data-decode paths that should be ARCHIVE_FAILED
so the caller can move on to the next entry after a corrupt one:
apply_filters Unsupported filter type
parse_tables Truncated huffman tables, decode failures (8 sites)
parse_block_header Unsupported header size, block checksum error
do_uncompress_block Decode code length / distance slot failures
Distance pointer overflow
decode_number / decode_code_length failures
merge_block Block size == 0 during merge
do_uncompress_file Invalid window size declaration
verify_checksums CRC32 / BLAKE2 mismatch
Programmer assertions, ENOMEM, true I/O errors, and propagation of
copy_string's window-buf-NULL FATAL return are intentionally kept as
ARCHIVE_FATAL because they are not recoverable per-entry conditions.
Follow-up to 4f148608. A code review found additional ARCHIVE_FATAL
returns in RAR4 data-decode paths that should be ARCHIVE_FAILED so the
caller can move on to the next entry:
read_data_block Truncated RAR file data
read_data_compressed PPMd "Invalid symbol" (3 sites)
parse_codes Zero window size is invalid
add_value Prefix found (second site)
make_table_recurse Huffman tree was not created
make_table_recurse Invalid location to Huffman tree specified
These are all per-entry parse/decode failures. As with the earlier
batch, the rar4 input position is tracked by rar_br_fillup so
read_data_skip will correctly advance past the damaged entry, and
RAR4 solid mode is not supported, so subsequent entries are not at
risk from a half-consumed shared decoder state.
Per-entry data errors (encryption, invalid filters, bad bitstream) now
return ARCHIVE_FAILED instead of ARCHIVE_FATAL. Update tests that were
asserting the old incorrect ARCHIVE_FATAL return codes.
The one exception is test_read_format_7zip_encryption_partially line 71,
which asserts ARCHIVE_FATAL after archive_read_next_header on the entry
following an encrypted entry: 7zip cannot skip an encrypted entry (the
decode-folder setup fails), so the skip legitimately returns ARCHIVE_FATAL
and the archive is done.
setup_decode_folder() returned ARCHIVE_FATAL for both header-level and
data-level encryption/filter errors. Header encryption is a true
archive-fatal condition; data encryption is per-entry.
Distinguish the two by returning ARCHIVE_FATAL when decoding archive
headers (header==1) and ARCHIVE_FAILED when decoding file content
(header==0). Fix the call site in read_stream() to propagate the
actual return value rather than mapping all errors to ARCHIVE_FATAL.
Two per-entry errors in the RAR5 data-reading path were returning
ARCHIVE_FATAL, preventing iteration over subsequent entries:
- "Reading encrypted data is not currently supported": the encryption
is per-entry; unencrypted entries in the same archive remain readable.
- "Invalid filter encountered": a malformed filter spec in one entry
does not make the rest of the archive unreadable.
Change both to ARCHIVE_FAILED.
ARCHIVE_FATAL means the entire archive is unreadable and no further
operations are valid. ARCHIVE_FAILED means the current entry cannot
be processed but iteration over subsequent entries may still succeed.
The RAR4 decompressor was returning ARCHIVE_FATAL from a large number
of data-parsing failures (invalid Huffman prefix, invalid PPMd sequence,
bad CRC, invalid symbol, etc.) that are per-entry errors. Because each
entry's compressed data region can be skipped using the packed_size
recorded in its file header, a decompressor error does not prevent
reading the next entry's header.
Change all such per-entry errors in the data-reading path
(read_data_stored, read_data_compressed, parse_codes, create_code,
add_value, make_table_recurse, expand, copy_from_lzss_window,
copy_from_lzss_window_to_unp) to return ARCHIVE_FAILED. OOM errors
and true I/O failures (rar_br_preparation truncated-data) remain
ARCHIVE_FATAL.
Three entry points in archive_read.c could return ARCHIVE_FATAL from
the format layer without setting a->archive.state = ARCHIVE_STATE_FATAL,
so subsequent API calls would not see the archive as fatally broken:
- archive_read_data_skip() unconditionally reset state to HEADER even
when the format's skip returned ARCHIVE_FATAL.
- archive_seek_data() and _archive_read_data_block() forwarded FATAL
from the format layer without recording it in the archive state.
Fix all three so that ARCHIVE_FATAL causes the state to become
ARCHIVE_STATE_FATAL, consistent with the existing behavior of
archive_read_next_header().
Clear invalid data when a header is invalid.
Check for data being larger than the allocated buffer.
Thanks to @ReverseWarrior for pointing out this issue,
and providing the fix.
An empty pathname caused a one-byte OOB read before the heap buffer in
write_path() due to two compounding bugs:
(1) misuse of bitwise & instead of logical &&, and
(2) missing gaurd for an empty pathname
isint() and isint_w() previously clamped values >= INT_MAX to INT_MAX
and returned success, allowing malformed ACL text to silently set IDs
to an arbitrary sentinel value. Change them to return -1 (a new
"overflow" indication) instead, and update all callers in both the
NFS4 and POSIX parsers (narrow and wide) to treat overflow as
ARCHIVE_WARN and skip the offending entry.
Add test_acl_nfs4_text.c with four test functions covering NFS4 ACL
text round-trips, audit/alarm entry types, numeric-ID handling
including the overflow boundary (INT_MAX - 1 accepted, INT_MAX
rejected), and malformed-entry error paths.
Improves the parsing of timestamps in a couple of ways:
* Saturate when timestamps exceed the range of time_t.
In particular, this provides more rational behavior on
systems with 32-bit time_t.
* Validate the format of overlong pax timestamps.
We previously failed to check that high-resolution
timestamps had only digits in the fractional part.
We now notice and ignore those with a warning.
archive_acl_text_len() counted the trailing ":id" digits only when
ARCHIVE_ENTRY_ACL_STYLE_EXTRA_ID was set, but archive_acl_to_text_l()
always writes them for USER/GROUP entries whose name is NULL. With a
7-digit or larger id the allocated buffer was too short, causing
append_id() to write past its end.
Fix the estimator to also count the extra colon and digits when the
name is NULL, matching the serializer's logic.
The wide serializer (archive_acl_to_text_w) had the opposite problem:
it passed id=-1 to append_entry_w() for NULL-name entries regardless
of the id value, causing a garbage character to be written in the name
field and the numeric id to be omitted entirely. Fix it to mirror the
narrow serializer by setting id = ap->id when wname is NULL.
Add tests for both the narrow and wide paths.
Darwin covers a wide range of platforms with similar but not identical
sets of libraries. MD5, SHA1 and SHA2 are available from libsystem on
all of these, but macOS also has them in libc and libmd. Restricting
our search to only libsystem means we can run configure on macOS and get
a config.h that also works for other Darwin platforms.
Also clarify that all systems will handle these SFX files, no matter on
which system libarchive is used.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If 7zip header cannot be found within roughly first 8 MB of data, skip
detection and assume that file is not a valid 7zip archive.
This avoids various 32 bit truncation issues as well as out of memory
conditions.
Reported by various individuals:
- Amemoyoi
- b4sh5i
- Jie Zhu
- Tikket
- wooseokdotkim
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Use desired data types: Offsets in files are (u)int64_t, sizes in memory
are (s)size_t. Use signedness when needed or specified by APIs.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Return an int for error information and supply offset through a given
argument. This fits other function declarations and makes it much easier
to differentiate between status and "return value".
While at it, merge fallback mechanism of both functions.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
This removes the function skip_sfx which makes slurp_central_directory
and archive_read_format_7zip_bid use the same code now.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Its functionality is split off from archive_read_format_7zip_bid
and returns offset to actual data, i.e. it handles self extracting
(SFX) files if offset 0 is not already a 7zip magic.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Prepare content of archive_read_format_7zip_bid to be split into its own
function which eventually can be reused by slurp_central_directory.
No real functional change (split if-blocks).
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Directly request an unsigned char pointer. This removes casts within the
function check_7zip_header_in_sfx and its surrounding caller code.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
We allocate space early on to support the advertised number of
files. A malicious archive can set a nonsensical value here to exhaust
memory. This adds a check comparing the number of files to the number
of streams and the size of the total header.
Note that the just-added test does not actually fail without this.
The existing code recovers if the allocation fails, which it typically
will. The new check tightens the limit so that we reject nonsensical
file counts and avoid problems from large memory allocations.
Extend test coverage to ensure all supported filters can be appended
to an archive reader before it is opened, matching the behavior
required to fix#2514.
Added tests for: BZIP2, LRZIP, LZ4, LZIP, LZMA, ZSTD, RPM, UU, and NONE.
Calling archive_read_append_filter(a, ARCHIVE_FILTER_COMPRESS) would
previously trigger a crash because compress_bidder_init() attempted to
read header bits from the upstream filter immediately. If the archive
was not yet opened (common when setting up filters), the upstream filter
state was not ready for reading.
This commit defers the header reading and decompressor initialization
until the first read operation (lazy initialization), consistent with
other filter implementations in libarchive.
Added test_read_append_compress_filter.
Bug: https://github.com/libarchive/libarchive/issues/2514
Test: ./libarchive_test test_read_append_compress_filter
Initialize the Huffman table to invalid values, which doesn't otherwise
affect the computation but avoids use of uninitialized values upon
extraction of some archives (as reported by `valgrind`).
Currently, libarchive_{random,util}.c use a couple bcrypt functions
regardless of whether HAVE_BCRYPT_H is defined as there are no other
implementations for Windows, but the actual <bcrypt.h> header is
included only under this macro.
To be able to build libarchive with ENABLE_CNG=OFF (for example, to
prefer a different crypto/digest engine) on Windows, don't guard
the include in these two files. In that case, bcrypt will still be
used, but only as an RNG.
This won't break anything because, as mentioned above, bcrypt is
used unconditionally here and if it's not present in the system,
the library won't build either way, with or without the change.
At least until we implement an RNG for Windows based on something
else.
Signed-off-by: Alexander Lobakin <alobakin@mailbox.org>
If a checksum has been provided through archive_entry and no data is
written, then digest internal memory is not released.
Fix this by always calling final functions, even if result is not
needed.
Using cmake within VSCode leads to creation of files which are not covered
by .gitignore so far. Covers Linux and Windows builds.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The function isofile_gen_utility_names could resolve .. directory
entries in a way that dirname will start with "../". If this happens,
the while-loop is unable to detect this because it forwards until the
cursor detects a slash again.
Fix this by also taking "../" at the beginning into account. Such an
entry can happen if "../../" points before the top directory.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If dot dot directory entries point before top directory, it can happen
that "../" will stay at the start of the path.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The isofile_gen_utility_names function normalizes directories, including
dot dot directory entries. If such an entry has multiple slahes and leads
to the top directory, then the new path erroneously becomes absolute.
Skip multiple slashes.
If rp is not NULL, then it points to a slash already. Takes this into
account to unify the rp and dirname cases a bit more.
Resolves#2937.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Normalizing dot dot directory entries with multiple slashes leads to
memory issues further down the call stack.
Reported in #2937.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Resolving paths like "dir/../filename" to "filename" can lead
to a strcpy call with overlapping memory. Use memmove instead,
which already happens at times in isofile_gen_utility_names.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Ubuntu patches zlib to use DFLTCC hardware DEFLATE acceleration on s390x
on compression levels 1-6, causing levels 1 and 6 to produce output with
the same lengths. Avoid the resulting test failure by comparing the
output lengths of levels 1 and 9 instead.
Tested on Ubuntu 24.04 on s390x hardware.
Fixes: https://github.com/libarchive/libarchive/issues/1515
There have been multiple instances of test cases being added to the CMake
build but not the Autotools one, thus omitting them from the released dist
tarball. Prevent this by testing the CMake build during `make distcheck`.
Remove an #include controlled by a preprocessor symbol that nothing
defines. I'm not sure if this has ever been needed, or what for, but
it serves no purpose today.
The link resolver is a helper utility that tracks linked
entries so they can be correctly restored. Clients add link information
to the link resolver and incrementally query it to correctly
link entries as they are restored to disk. The link resolver
incrementally releases entries as they are consumed in order
to minimize memory usage.
The `archive_entry_linkresolver_free()` method cleans up
by repeatedly querying the cache and freeing each entry.
But this conflicted with the incremental clean up,
leading to double-frees of leftover items.
The easy fix here is to have `archive_entry_linkresolver_free()`
just repeatedly query the list without trying to free, relying
on the incremental clean up mechanism.
Credit: tianshuo han reported the issue and suggested the fix.
A crafted ISO with pz_log2_bs=2 and pz_uncompressed_size=0xFFFFFFF9
causes an integer overflow in the block pointer allocation in
zisofs_read_data(). On 32-bit, (ceil+1)*4 wraps size_t to 0, malloc(0)
returns a tiny buffer, and the code writes ~4GB past it.
The pz_log2_bs validation fix prevents this. Add a regression test with
a crafted 48KB ISO that triggers the overflow on unfixed 32-bit builds.
Those 9 are not used anywhere outside the file (the actual
functionality is exported as a callback structure).
Make them static for a bit better compiler optimization
opportunities and, more important, to avoid symbol conflict
when static linking libarchive and any library which uses
the original Ppmd*.c from the LZMA SDK (like minizip-ng).
Also remove a couple declarations and macros not used
anywhere at all while we're here.
Signed-off-by: Alexander Lobakin <alobakin@mailbox.org>
These two filters failed to correctly set a name when
being registered, which prevented them from working
correctly with archive_read_append_filter()
Thanks to @fdegros for the test case demonstrating the lrzip failure
If the -R command line argument is supplied with user/group names
multiple times, memory leaks occur.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The anchor characters ^ and $ have only special meanings if they are
located at the beginning (^) or at the end ($) of the pattern. And even
then they are supposed to be only special if flags are set.
If they are located within the pattern itself, they are regular
characters regardless of flags.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
- Keep a whitespace between text and brackets
- No whitespace between text and colon
- No newline at end of error message
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
By only removing periods from error messages in Windows specific code,
but not adjusting its POSIX counterpart, the test fails on Windows but
not on POSIX systems.
Fix this by removing the period in test and in POSIX error messages.
Fixes: 3e0819b59e ("libarchive: Remove period from error messages")
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Calling archive_set_error with a message and errno already indicates
that a failure occurred. Only a minority of functions did that: Unify
with the rest.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The error messages are mostly written without a period. This makes
sense, because they can be accompanied with a strerror(errno) call,
giving more information: most likely a colon is appended.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The buff variable is only used in entry_to_archive. Moving it into the
specific code block where it is actually used reduces its visibility and
thus makes it easier to read the code:
Since Windows indeed uses unsigned for read, this makes it much easier
to verify that buff never grows and cannot be too large.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The API allows to set int64_t uid/gid values. When writing pax archives,
such large values are properly set in the USTAR header of actual data in
base256, i.e. everything works.
The pax header entries might be missing though because the check
truncates these values to unsigned int. Larger values could be truncated
in a way that they seem smaller than (1 << 18).
The check in line 1427 which sets the uid/gid values into the PAX header
block is correct, truncating the actual value to max. octal
representation.
This is a purely defensive change to support parsers which actually
allow such large uid/gid values but do not understand base256 encoding.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Make sure that supplied values cannot overflow int64_t, which would be
an undefined behavior. Also do not accept empty strings as numbers.
Last but not least, check values before casting them to target data
types to avoid truncations.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Make sure that supplied option values fit into data types. If not,
return an error instead of silently truncating values.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Otherwise an unknown error is printed, which does not help the user
(especially since a custom message is printed in this case already).
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Make sure that supplied arguments actually fit into target data types.
Print an error if numbers are too large instead of silently truncating
them.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Only a minority of calls to lafe_errc state explicitly that an error
occurred. Remove these few cases for a more unified argument handling.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Do not print that at least one mode has to be supplied. Exactly one mode
has to be supplied instead.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Complain only if contradicting mode options are supplied. Passing the
same argument multiple times should not lead to errors.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If a filter option is recognized but its value is invalid, return
ARCHIVE_FAILED instead of ARCHIVE_WARN. The latter is used for unknown
options, e.g. at the end of the option setter functions.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The buffer has a fixed 1024 limit, which can be smaller than the maximum
allowed path length. Increasing the limit to 4096 would partially help,
but since leading spaces are stripped, input lines could be valid and
longer.
Use getline instead.
Let cpio_rename work with the entry directly. In this initial step, it
moves buff into the stack.
The comment suggests that this was done in the past already.
### Summary
This PR fixes a NULL pointer dereference in `archive_acl_from_text_nl()` (located in `archive_acl.c`) that occurs when parsing malformed PAX tar archives containing a short "default" ACL prefix.
### Technical Details
This is a variant of the bug previously fixed in the wide-character version `archive_acl_from_text_w()` via commit 7a6549a6 (Issue #2744).
The vulnerability exists because `st = field[n].start + 1` is calculated before verifying if the field length is zero. In cases of malformed entries, `field[n].start` can be NULL, leading to Undefined Behavior/SIGILL. This patch applies the same logic from the wide-char fix: moving the pointer increment after the length guard.
### Validation
- **Environment:** Ubuntu 24.04, compiled with AddressSanitizer (ASan) and UndefinedBehaviorSanitizer (UBSan).
- **Reproduction:** Verified that the minimized PAX tar archive provided in issue #2904 no longer triggers the crash.
- **Regression:** All existing library tests pass.
Closes#2904
On LLP64 systems like Windows 11, overly large uid/gid values would be
truncated.
Same is true on Linux 32 bit.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
LLP64 systems like Windows 11 have a 32 bit long. Cast int64_t to
long long for better output.
Same is true for 32 bit systems.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
When parsing a malformed CAB file, the skip routine (cab_checksum_finish) blindly calculated the checksum on an uninitialized cfdata->memimage. This patch adds a NULL check before the checksum calculation and includes a standalone test case with a minimized malformed payload to prevent regressions.
When a PPMd-compressed block updates dictionary_size, the LZSS window
from a prior block is not reallocated. The allocation guard only checks
if dictionary_size is zero or the window pointer is NULL, not whether
the existing window is large enough. This allows copy_from_lzss_window()
to read past the allocated buffer.
Fix the guard to also check whether the current window is undersized.
Add bounds checks in copy_from_lzss_window() and parse_filter() as
defense in depth.
The zisofs block size exponent (pz_log2_bs) read from the Rock Ridge ZF
extension entry is used directly in shift expressions without validation.
The zisofs specification only permits values 15, 16, or 17 (corresponding
to 32K, 64K, and 128K block sizes).
When pz_log2_bs >= 64 on 64-bit systems (or >= 32 on 32-bit), the
expression (size_t)1UL << pz_log2_bs is undefined behavior per C11
6.5.7. On 32-bit systems, a large exponent also causes the block pointer
allocation size computation (ceil + 1) * 4 to overflow to zero, leading
to a heap buffer overflow write after malloc(0).
Fix: reject any pz_log2_bs outside the range [15, 17] by disabling
zisofs for the entry (file->pz = 0), which prevents the zisofs
decompression path from executing.
Found by fuzzing with ASAN/UBSAN.
`archive_read_support_format_cab` allocates a fresh context structure on
each call before registering the CAB format with libarchive. On the
second call, however, the registration step reports the format is
already registered, so the function frees the newly allocated context
structure — it is not needed, since the one from the first call is
already in use.
The context structure contains a `ws` field of type `archive_wstring`.
During initialization, `archive_wstring_ensure` is called on that field,
which performs its own heap allocation.
The cleanup path described above frees the context structure without
also releasing the memory owned by the `ws` field, causing a leak.
Fixed by calling `archive_wstring_free` on the `ws` field before freeing
the context structure.
Fixes#1980.
When the same archive_read object registers rar5 format more than once,
__archive_read_register_format returns ARCHIVE_WARN (duplicate bid).
The error path then called rar5_cleanup(ar), which dereferences
a->format to get the rar5 context — but a->format is NULL until
archive_read_open() is called, causing a SIGSEGV. The newly allocated
rar struct was also leaked.
Fix by introducing rar5_deinit() (the inverse of rar5_init) and using
it in both rar5_cleanup() and the registration error path. On failure,
the locally allocated rar is freed directly without going through
a->format. The function now always returns ARCHIVE_OK, consistent with
all other format handlers.
Fixes#1963.
If a file declares more than 8192 filters at once, without consuming
them, then the filter allocation function leaks memory.
The backing array used for storing filter pointers can hold up to 8192
pointers. After that it won't accept any new entries. The problem was
that the caller code didn't check if the backing array has accepted the
pointer; it silently assumed the pointer was registered, disposing the
only variable that was holding the pointer to allocated memory.
The fix is to fail the creation of a new filter structure if the backing
array is full. This will result in failure to unpack, but having more
than 8192 filters in one file at the same time seems unlikely.
Fixes issue #2891
The fuzzer fails to update entries[i] after archive_entry_linkify()
modifies the pointer (caching or swapping entries). This causes the
cleanup loop to free entries that were already freed via spare or
the drain loop, resulting in heap-use-after-free.
Fix: add entries[i] = entry after the linkify call to track ownership.
Calculate the filename length only once during read_file_data
initialization and replace strcpy with memcpy.
As a side effect it also helps silencing valgrind warnings due to strcpy
word-based optimization
https://github.com/libarchive/libarchive/issues/2887
If run as root (as is the case in CI), switch to an unprivileged user
(default: nobody) before running each test, and switch back after.
This makes it possible to write tests that rely on file permissions.
Note that tests that use the UID or GID must now check the EUID / EGID
instead as they will be different while the test is running. The only
way to avoid that is to run each test case in a child process, which
would hugely increase the complexity of test_run().
When create_dir() fails to create a directory, it immediately checks to
see if the directory already exists, which can happen if it's been given
something like foo/../foo. Unfortunately, this clobbers errno, which
means that create_dir() always reports ENOENT, regardless of the actual
error. Fix this by only performing this extra check if errno is EEXIST,
then reset errno to either EEXIST or ENOTDIR depending on the outcome.
The header is a series of blocks, most of which contain
values of just a few bytes (sizes, times, etc). The only
exceptions are the filename and directory name attributes,
which will be limited by the MSDOS/Windows maximum file length
limit. So it seems unlikely that this will ever exceed 64k.
(If we find counter-examples, we can easily extend this limit.)
Previously, we tested many combinations of FreeBSD version, build system, and filesystem.
In practice, that's redundant, and we've started seeing these excessive requests get throttled by Cirrus.
Cutting back to just 3 combinations should suffice and reduce the risk of throttling.
If an incorrect option value has been passed to a filter, it is possible
that library operations continue without even printing a warning.
This can happen because the special value "ARCHIVE_WARN - 1" is only
checked for filter issues, not format issues. Since this special value
is larger than ARCHIVE_FAILED, such failures are silently discarded.
Fix this by checking for this magic value for formats as well.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The crash reported in this issue is caused by a segmentation fault
that happens when a pointer `p` passed to `check_7zip_header_in_sfx`
is dereferenced. That argument `p` is defined as `p = buff + offset` in
`archive_read_format_7zip_bid` with `offset` determined by a call
to `find_elf_data_sec`.
Within `find_elf_data_sec`, a 64-bit ELF section offset is cast
to `ssize_t`. For a sufficiently large offset, such as an unsigned
64-bit value exceeding `SSIZE_MAX`, the `ssize_t` will be negative,
which leads to `p` to point before the start of the buffer.
This patch eliminates the problem by:
1. Reading the ELF section offset into a `uint64_t`.
2. Validating that the offset does not exceed `SSIZE_MAX`.
3. If the offset is invalid, breaking the loop to retain the safe
default `SFX_MIN_ADDR`.
These checks ensure that `min_addr` is always within the bounds of the buffer.
This PR includes a regression test `test_malformed3` in
`test_read_format_7zip_malformed.c` using a crafted reproducer file.
CodeMender verified with AddressSanitizer that the crash is resolved and the
test suite passes.
Co-authored-by: CodeMender <codemender-patching@google.com>
Fixes: https://issues.oss-fuzz.com/issues/470963352
Verify that archive_entry_acl_from_text() and
archive_entry_acl_from_text_w() return ARCHIVE_WARN instead of
crashing when given a bare "d" or "default" string with no
subsequent tag field.
Without the accompanying fix in archive_acl.c this test triggers
a NULL-pointer dereference (SEGV) in archive_acl_from_text_w().
When parsing a short "default" ACL prefix (e.g. L"d") with no
subsequent tag field, field[n] is left as {NULL, NULL} and the
code dereferences it unconditionally in the switch statement,
causing a SEGV.
Add a zero-length check after computing the field length so that
malformed entries are skipped with ARCHIVE_WARN, matching the
documented contract. Also move the st pointer computation after
the guard to avoid dereferencing a NULL start pointer.
Fixeslibarchive/libarchive#2744
If an entry pathname is set only by WCS or UTF-8, it may not have any
MBS representation in the archive's hdrcharset. Do not crash or create
an archive with an empty pathname. Furthermore, the entry pathname may
not have any MBS representation in the current locale. Do not report a
`(null)` pathname in the error message.
On Windows, since commit ae54394104 (Implement a string conversion
interface..., 2011-05-09, v3.0.0a~398), `archive_string_append_from_wcs`
no longer returns an error on use of replacement characters in non-C
locales. Restore the error to avoid silent use of replacement
characters, and for consistency across platforms.
Prior to commit c30f279475 (Complete support for UTF8 encoding
conversion, 2020-05-31, v3.5.0~32^2), `archive_mstring_get_mbs_l` returned
an error on failure to convert WCS to MBS. Restore the error so callers
can distinguish conversion failure from having no string in any form.
On Windows, `archive_mstring_get_mbs_l` from WCS with `sconv != NULL`
internally calls `archive_string_append_from_wcs_in_codepage` more than
once, with both `sconv != NULL` and `sconv == NULL`. Due to another bug
in the `sconv == NULL` case, we cannot enable all test combinations yet.
This change adds support for the following test scenarios:
- 15.0-RELEASE - ZFS
- 15.0-STABLE - UFS
- 15.0-STABLE - ZFS
This additional testing aims to catch issues with 15.x, as well as
ensure libarchive use doesn't regress when run on ZFS-based hosts.
Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
Building on illumos currently fails with:
```
ld: fatal: unrecognized option '--gc-sections'
```
This happens because `--gc-sections` isn't supported on illumos `ld`.
This patch updates CMakeLists.txt to skip unsupported linker options on
illumos. The flags used on other operating systems are optimizations
that don't affect correctness, so this change is safe.
Complete the refactoring of all 25 fuzzers:
- Remove duplicate Buffer struct definitions from 15 format fuzzers
- Remove duplicate DataConsumer class from 7 API fuzzers
- Update consume_bytes() calls to match new signature
- All fuzzers now use shared helpers from fuzz_helpers.h
This eliminates ~1000 lines of duplicated code.
The test runner already lists available tests if it fails to parse the
command line, but add a -l option to explicitly do this without also
printing an error message and a summary of options.
Adds a 'bsdtar-recovery' Android build target for use in Android recoveries
as a static binary, and fixes some build failures on the get-go.
Tested on halium-7.1, halium-9.0 & halium-13.0.
Change-Id: I9b656e7016d4bf21517e2edb18f2a7733edc6982
This reverts commit d8aaf88c9f, reversing
changes made to ee49ac8106.
tree_reopen() and tree_dup() return NULL only of they
are unable to allocate memory. Otherwise libarchive enters
ARCHIVE_FATAL if trying to walk an enterable but unreadable
directory.
__archive_ensure_cloexec_flag() operates only on fd >= 0
so there is no need to skip it
I have reimplemented the check around fdopendir()
Reported by: Christian Weisgerber from OpenBSD
Uses the sed-like way (and Java-like, and .Net-like, and Javascript-like…) to fix this issue of advancing the string to be processed by one if the match is zero-length.
Fixeslibarchive/libarchive#2725 and solves libarchive/libarchive#2438.
Since LZ4 and zstd share the same format for skippable frames, we need
to skip over these frames when trying to detect the format of compressed
data. Let's read up to something like 64kb of data when performing this
scanning.
Note that the LZ4 specification advises against starting with a skippable
frame, but doesn't forbid it:
> For the purpose of facilitating identification, it is discouraged to
> start a flow of concatenated frames with a skippable frame. If there
> is a need to start such a flow with some user data encapsulated into
> a skippable frame, it's recommended to start with a zero-byte LZ4
> frame followed by a skippable frame. This will make it easier for
> file type identifiers.
Resolves#2692.
This is expected to fail until a followup commit, because lz4 and zstd
skippable frames are the same format and we don't skip over those when
performing format detection (yet).
Relates to #2692.
Whenever we need to create a temporary file while writing to disk on a
POSIX system, try to create it in the same directory as the final file
instead of the current working directory. The target directory can
reasonably be expected to be writable (and if it isn't, creating the
file will fail anyway), but the current working directory may not be.
While here, consistently use __archive_mkstemp(), and increase the
template from six to eight random characters.
Fixes: 2e73ea3a7d ("Fix max path-length metadata writing (#2243)")
Fixes: e12c955dca ("Unify temporary directory handling")
Raise the maximum size of Mac metadata from 4 MiB to 10 MiB, as that is
the value used by Apple themselves in the version of libarchive included
in Darwin.
Provide preprocessor macros for two recurring magic numbers in the zip
support code: the length of the local file header (30 bytes) and the
maximum allowable size for Mac metadata (4 MiB).
In archive_util.c, we have a private function named get_tempdir() which
is used by __archive_mktemp() to get the temporary directory if the
caller did not pass one.
In archive_read_disk_entry_from_file.c, we use the same logic with a
slight twist (don't trust the environment if setugid) to create a
temporary file for metadata.
Merge the two by renaming get_tempdir() to __archive_get_tempdir() and
unstaticizing it (with a prototype in archive_private.h).
When compiling libarchive using clang in module mode a special
module.modulemap file describes the structure of the header files
so that they can be imported modularly. Having this file makes
it easier for modular uses of the library out of the box so that
clients don't need to write their own, potentially making errors
in doing so.
Add a module.modulemap in the public header file location so that
clang and related tools can find it easily.
Our tar header parsing tracks a count of bytes that need to be
consumed from the input. After each header, we skip this many bytes,
discard them, and reset the count to zero. The `V` header parsing
added the size of the `V` entry body to this count, but failed to
check whether that size was negative. A negative size (from
overflowing the 64-bit signed number parsing) would decrement this
count, potentially leading us to consume zero bytes and leading to an
infinite loop parsing the same header over and over.
There are two fixes here:
* Check for a negative size for the `V` body
* Check for errors when skipping the bytes that
need to be consumed
Thanks to Zhang Tianyi from Wuhan University for finding
and reporting this issue.
Depending on header search path ordering, we can easily
confuse libarchive_fe/err.h with the system header.
Rename ours to lafe_err.h to avoid the confusion.
Rename libarchive_fe/err.c to match.
We reuse the compression buffer to format the gzip header,
but didn't check for an overlong gzip original_filename.
This adds that check. If the original_filename is
over 32k (or bigger than the buffer in case someone shrinks
the buffer someday), we WARN and ignore the filename.
In archive_write_header(), if the format method or a filter flush method
fails, we set the archive state to fatal, but we did not do this in
archive_write_data() or archive_write_finish_entry(). There is no good
reason for this discrepancy. Not setting the archive state to fatal
means a subsequent archive_write_free() will invoke archive_write_close()
which may retry the operation and cause archive_write_free() to return
an unexpected ARCHIVE_FATAL.
If a fatal error occurs, the closer will not be called, so neither will
BZ2_bzCompressEnd(), and we will leak memory. Fix this by calling it a
second time from the freer. This is harmless in the non-error case as
it will see that the compression state has already been cleared and
immediately return BZ_PARAM_ERROR, which we simply ignore.
The closer will not be called if a fatal error occurs, so the current
arrangement results in a memory leak. The downside is that the freer
may be called even if we were not fully constructed, so it needs to
perform additional checks. On the other hand, knowing that the freer
always gets called and will free the client state simplifies error
handling in the opener.
Close all the file descriptors in the range [3 ..
sysconf(_SC_OPEN_MAX)-1] before executing a filter program to avoid
leaking file descriptors into subprocesses.
Bug: https://github.com/libarchive/libarchive/issues/2520
This fixes an unhelpful "Couldn't visit directory: Unknown error: -1" message.
Fixes: 3311bb52cb ("Bring the code supporting directory traversals from bsdtar/tree.[ch] into archive_read_disk.c and modify it. Introduce new APIs archive_read_disk_open and archive_read_disk_descend.")
RAR5 reader had inconsistent sanity checks for directory entries that
declare data. On one hand such declaration was accepted during the
header parsing process, but at the same time this was disallowed during
the data reading process. Disallow logic was returning the
ARCHIVE_FAILED error code that allowed the client to retry, while in
reality, the error was non-retryable.
This commit adds another sanity check during the header parsing logic
that disallows the directory entries to declare any data. This will make
clients fail early when such entry is detected.
Also, the commit changes the ARCHIVE_FAILED error code to ARCHIVE_FATAL
when trying to read data for the directory entry that declares data.
This makes sure that tools like bsdtar won't attempt to retry unpacking
such data.
Fixes issue #2714.
This commit fixes multiple issues found in the function that parses
extra fields found in the "file"/"service" blocks.
1. In case the file declares just one extra field, which is an
unsupported field, the function returns ARCHIVE_FATAL.
The commit fixes this so this case is allowed, and the unsupported
extra field is skipped. The commit also introduces a test for this
case.
2. Current parsing method of extra fields can report parsing errors in
case the file is malformed. The problem is that next iteration of
parsing, which is meant to process the next extra field (if any),
overwrites the result of the previous iteration, even if previous
iteration has reported parsing error. A successful parse can be
returned in this case, leading to undefined behavior.
This commit changes the behavior to fail the parsing function early.
Also a test file is introduced for this case.
3. In case the file declares only the EX_CRYPT extra field, current
function simply returns ARCHIVE_FATAL, preventing the caller from
setting the proper error string. This results in libarchive returning
an ARCHIVE_FATAL without any error messages set. The PR #2096 (commit
adee36b00) was specifically created to provide error strings in case
EX_CRYPT attribute was encountered, but current behavior contradicts
this case.
The commit changes the behavior so that ARCHIVE_OK is returned by the
extra field parsing function in only EX_CRYPT is encountered, so that
the caller header reading function can properly return ARCHIVE_FATAL
to the caller, at the same time setting a proper error string. A test
file is also provided for this case.
This PR should fix issue #2711.
Use posix_spawn() with POSIX_SPAWN_CLOEXEC_DEFAULT on systems that
define this constant, in order to avoid leaking file descriptors into
subprocesses.
Bug: https://github.com/libarchive/libarchive/issues/2520
to avoid use of undefined content of buf, in case when custom
locale makes the result string longer than buf length.
Signed-off-by: Marcin Mikula <marcin@helix.pl>
Some experiments showed strange things happen if you
provide an invalid type value when appending a new ACL entry.
Guard against that, and while we're here be a little more
paranoid elsewhere against bad types in case there is another
way to get them in.
When testing the feature with `bsdtar -acf test.zip --options
zip:compression=zstd …` on a tree of ~100MB, the execution would
appear to "hang" while writing a multi-gigabytes ZIP file.
setup_mac_metadata currently concates the template after TMPDIR without
adding a path separator, which causes mkstemp to create a temporary file
next to TMPDIR instead of in TMPDIR. Add a path separator to the
template to ensure that the temporary file is created under TMPDIR.
I hit this while rebuilding libarchive in nixpkgs. Lix recently started
using a dedicated build directory (under /nix/var/nix/builds) instead of
using a directory under /tmp [1]. nixpkgs & Lix support (optional)
on macOS sandboxing. The default sandbox profile allows builds to access
paths under the build directory and any path under /tmp. Because the
build directory is no longer under /tmp, some of libarchive's tests
started to fail as they accessed paths next to (but not under) the build
directory:
cpio/test/test_basic.c:65: Contents don't match
Description: Expected: 2 blocks
, options=
file="pack.err"
0000_62_73_64_63_70_69_6f_3a_20_43_6f_75_6c_64_20_6e_bsdcpio: Could n
0010_6f_74_20_6f_70_65_6e_20_65_78_74_65_6e_64_65_64_ot open extended
0020_20_61_74_74_72_69_62_75_74_65_20_66_69_6c_65_0a_ attribute file.
Sandbox: bsdcpio(11215) deny(1) file-write-create /nix/var/nix/builds/nix-build-libarchive-3.8.0.drv-7tar.md.5EUrQu
[1]: https://lix.systems/blog/2025-06-24-lix-cves/
If it's already known that we use variables for calculations with
size_t, use size_t for them directly instead of int, even if values
fit.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The off_t datatype in Windows is 32 bit, which leads to issues when
handling files larger than 2 GB.
Add a wrapper around fstat/stat calls to return a struct which has a
properly sized st_size variable. On systems with an off_t representing
the actual system limits, use the native system calls.
This also fixes mtree's checkfs option with large files on Windows.
Fixes https://github.com/libarchive/libarchive/issues/2685
Fixes 89b8c35ff4
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
It is possible to handle entries and files with sizes which do not fit
into off_t of the current system (Windows always has 32 bit off_t and
32 bit systems without large file support also have 32 bit off_t).
Set sizes to 0 in such cases. The fstat system call would return -1 and
set errno to EOVERFLOW, but that's not how archive_entry_set_size acts.
It would simply ignore negative values and set the size to 0.
Actual callers of archive_entry_stat from foreign projects seem to not
even check for NULL return values, so let's try to handle such cases as
nice as possible.
Affects mtree's checkfs option as well (Windows only, 32 bit systems
would simply fail in fstat/stat).
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The new `__la_wopen` wrapper is a copy of `__la_open` that
expects--rather than converts--a wcs parameter.
The `sopen` variants are offered as "more secure" variants of `open` and
`wopen`; I cannot vouch for their security, but some build systems are
strict about the use of "banned insecure APIs".
I've confirmed that `_wsopen_s` and `_open_s` are present in the Windows
Vista SDK.
I did not confirm that they are available in the Windows XP Platform
SDK, in part because in e61afbd463 (2016!) Tim says:
> I'd like to completely remove support for WinXP and earlier.
For write, 0 may not mean an error at all. We need to instead check for the length not being the same.
With fwrite, because 0 could mean an error, but not always. We must check that we wrote the entire file!
Note that unlike write, fwrite's description according to POSIX does not mention returning a negative type at all. Nor does it say you can retry unlike write.
Finally, with write, we need to check less than 0, not 0, as 0 is a valid return and does not mean an error.
Make sure that the string table size is not smaller than 6 (and also
not larger than SIZE_MAX for better 32 bit support).
Such small values would lead to a large loop limit which either leads to
a crash or wrong detection of a ".data" string in possibly uninitialized
memory.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The 64 bit format requires at least 63 bytes, so increase this limit.
Such small binaries most likely don't contain 7zip data anyway.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If a sparse hole is located at the end of an entry, then the tar
parser returns ARCHIVE_EOF while updating the offset where 0 bytes of
data will follow.
If archive_read_data encounters such an ARCHIVE_EOF return value, it
has to recheck if the offsets (data offset and output offset) still
match. If they do not match, it has to keep filling 0 bytes.
This changes assumes that it's okay to call archive_read_data_block
again after an EOF. As far as I understood the parsers so far, this
should be okay, since it's always ARCHIVE_EOF afterwards.
Fixes https://github.com/libarchive/libarchive/issues/1194
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If an entry reaches its end of file, the offset is not necessarily
the same as unp_size. This is especially true for links which have
a "0 size body" even though the unpacked size is not 0.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
When _warc_read encounters end of entry, it adds 4 bytes to the last
offset for \r\n\r\n separator, which is never written. Ignore these
bytes since they are not part of the returned entry.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If xar_read_data has no further data, set offset to end of entry,
not to total size of parsed archive so far.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The string constants can be used directly for comparison, which makes
this code robust against future changes which could lead to names being
longer than str could hold on stack.
Also removes around 100 bytes from compiled library (with gcc 15).
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If zlib is not supported, do not run tests to avoid false positives.
Also adjust tests to support latest gzip versions (1.10+) which store
less information for improved reproducibility. The gzip binary is
used as a fallback if zlib is not available.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If a unix system has no iconv support, the best effort function will
be unable to convert KOI8 to UTF-8. Skip the test if such support is
missing.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If no encryption support exists, the -P option will always fail.
"Skip" the test by making sure that there really is no encryption
support according to libarchive functions.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Some functions might return -1 in case of library error. Use an
own return value if a stub function was used for better error
messages.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
After importing the latest libarchive into FreeBSD, Shawn Webb @
HardenedBSD noted that the test build is broken when FORTIFY_SOURCE=2
while building the base system. Braced initializer lists are a special
case that need some extra fun parentheses when we're dealing with the
preprocessor.
While it's not a particularly common setup, the extra parentheses don't
really hurt readability all that much so it's worth fixing for wider
compatibility.
Fixes: libarchive/libarchive#2657
Ignoring SIGCHLD gets passed to child processes. Doing that has
influence on waitpid, namely that zombie processes won't be
created. This means that a status can never be read.
We can't enforce this in library, but libarchive's tools can be
protected against this by enforcing default handling.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Use pid_t since waitpid returns a pid_t. Also check for a negative
return value in writer as well to avoid reading the possibly
unitialized status value.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Calling CloseHandle multiple times for the same handle can lead to
exceptions while debugging according to documentation.
Mimic the waitpid handling for success cases to behave more like the
Unix version which would "reap the zombie".
Doing this for an unsuccessful call is off, but the loop is never
entered again, so I guess it's okay and worth it to reduce the amount
of Windows specific definitions in source files.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If the waitpid version for Windows fails, preserve the error code and
avoid overwriting it with a possible CloseHandle error.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The archive_utility_string_sort function won't be part of the 4.0.0 API
anymore. No users were found and such a task should be done outside of
the library.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The utility function "archive_utility_string_sort" is a custom qsort
implementation. Since qsort is specified in C11 and POSIX.1-2008
which libarchive is based on, use system's qsort directly.
The function is not used directly in libarchive, so this is a good
way to save around 500 bytes in resulting library without breaking
compatibility for any user of this function (none found).
Also allows more than UINT_MAX entries which previously were limited
by data type and (way earlier) due to recursion.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Test cases already get a C locale, which is sufficient for this test.
IF LC_TIME was not previously set, the used en_US.UTF-8 would stay
as an environment variable, possibly affecting other test cases.
Since en_US.UTF-8 is not guaranteed to be available, C is a better
choice.
Fixes https://github.com/libarchive/libarchive/issues/2560
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Reset current locale settings through setlocale and also all
environment variables which might affect test cases which
spawn children through systemf which in turn would call setlocale
on their own, e.g. bsdtar.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Explicitly use goto to turn a recursive call into an iterative one.
Most compilers do this on their own with default settings, but MSVC
with default settings would create a binary which actually performs
recursive calls.
Fixes call stack overflow in binaries compiled with low optimization.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The sparse 1.0 parser skips lines with comments. The amount of skipped
bytes is stored in a ssize_t variable, although common 32 bit systems
allow files larger than 4 GB.
Gracefully handle files with more than 2 GB bytes full of comments to
prevent integer truncations.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If a pax global header specifies a negative size, it is possible to
reduce variable `unconsumed` by 512 bytes, leading to a re-reading
of the pax global header. Fortunately the loop verifies that only one
global header per entry is allowed, leading to a later ARCHIVE_FATAL.
Avoid any form of negative size handling and fail early.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Skip all entry bytes after sparse entries were encountered. This matches
GNU tar behavior.
I have adjusted (and fixed) the existing test case for this. The test
case test_read_format_gtar_sparse_skip_entry did not work with GNU tar.
In #2558 it was explained that the pax size always overrides the header
size (correct). Since the pax size in the test case was way larger than
the actual entry bytes in archive, GNU tar choke on the test file.
The libarchive parser did not skip any bytes not already read due to
references by sparse entries, so the huge pax size was not detected.
By adjusting the test case to have a leftover byte (only 3 bytes are
referenced through sparse entry now, leaving one extra byte) with a
correct pax size and an invalid header size (after all it is overridden
by pax size), GNU tar works and libarchive gets off its 512 byte
alignment, not being able to read the next entry.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Consecutive to 16fd043f51
IID_ISequentialStream is required by the code.
This GUID is defined in uuid.lib or libuuid.a in mingw-w64. It is required
to link with that library to get the definition of the GUID. Some toolchains
add it by default but not all.
Some error messages already use the ll length modifier, which results
in raw formatter output, i.e. "%lld" instead of a number.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If a pax attribute has a 0 length value and no newline, the tar reader
gets out of sync with block alignment.
This happens because the pax parser assumes that variable value_length
(which includes the terminating newline) is at least 1. To get the
real value length, 1 is subtracted. This result is subtracted from
extsize, which in this case would lead to `extsize -= -1`, i.e.
the remaining byte count is increased.
Such an unexpected calculation leads to an off-by-one when skipping
to the next block. In supplied test case, bsdtar complains that the
checksum of the next block is wrong. Since the tar parser was not
properly 512 bytes aligned, this is no surprise.
Gracefully handle such a case like GNU tar does and warn the user that
an invalid attribute has been encountered.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
AppleDouble extension entries are present as separate files immediately
preceding the corresponding real files. In libarchive, we process the
entire metadata file (headers + data) as if it were a header in the real
file. However, the code forgets to reset the accumulated header state
before parsing the real file's headers. In one code path, this causes
the metadata file's name to be used as the real file's name.
Specifically, this can be triggered with a tar containing two files:
1. A file named `._badname` with pax header containing the `path` attribute
2. A file named `goodname` _with_ a pax header but _without_ the `path` attribute
libarchive will list one file, `._badname` containing the data of `goodname`.
This code is pretty brittle and we really should let the client deal with
it :(
Fixes#2510.
Signed-off-by: Zhaofeng Li <hello@zhaofeng.li>
Pax extended headers may specify negative time values for files older
than the epoch.
Adjust the code to clear values to 0.0 more often and set ps to
INT64_MIN to have a proper error specifier, because the parser does
not allow anything below -INT64_MAX.
Fixes https://github.com/libarchive/libarchive/issues/2562
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Iterating over a size_t with unsigned could lead to an endless loop
while adding uid/gid to a list which already counts 4 billion
entries.
I doubt that this can ever happen, given that the routines become
very slow with insertions, but better be safe than sorry.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Turn unmatched_count into a size_t to support as many entries as
possible on the machine.
If more than INT_MAX entries are not matched, truncate the result
of archive_match_path_unmatched_inclusions for external callers.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The count fields are merely used to check if a list is empty or not.
A check for first being not NULL is sufficient and is already in
place while iterating over the linked elements (count is not used).
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The operations for key and node comparison depend on the platform
libarchive is compiled for. Since these values do not change
during runtime, set them only once during initialisation.
Further simplify the code by declaring only one "rb_ops" with
required functions based on platform.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The cygwin FAQ states that __CYGWIN__ is defined when building for a
Cygwin environment. Only a few test files check (inconsistently) for
CYGWIN, so adjust them to the recommended __CYGWIN__ definition.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Cast address of "version" to BYTE pointer for CryptGetProvParam.
Fix "major" variable assignment for picky compilers like MSVC.
The "length" variable is an in/out variable. It must be set to the size
of available memory within "version". Right now it is undefined behavior
and 0 would crash during runtime.
Fixes https://github.com/libarchive/libarchive/issues/2628
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Bumps the all-actions group with 4 updates:
`actions/checkout` from 4.2.1 to 4.2.2
`actions/upload-artifact` from 4.4.3 to 4.6.2
`github/codeql-action` from 3.26.12 to 3.28.18
`ossf/scorecard-action` from 2.4.0 to 2.4.1
We should not get here, but given that the check exists, we should not let it happen if this is NULL because otherwise we just dereference it later on.
When looping over program header entries (e_shnum)
we need to increment sec_tbl_offset by e_shentsize
and not by fixed values.
Fixes OSS-Fuzz issue 418349489
Copy ae digests to mtree_entry. This simplifies porting non-archive
formats to archive formats while preserving supported message
digests specifically in cases where recomputing digests is not
viable.
Signed-off-by: Nicholas Vinson <nvinson234@gmail.com>
The size_t to int conversion is especially required on Windows systems
to support their int-based functions. These variables should be properly
checked before casts. This avoids integer truncations with large
strings.
I prefer size_t over int for sizes and adjusted variables to size_t
where possible to avoid casts.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If vsnprintf fails with errno EOVERFLOW, the results are very platform
dependent but never useful. The implementation in glibc fills bytes with
blanks, FreeBSD fills them with zeros, OpenBSD and Windows set first
byte to '\0'.
Just stop processing and don't print anything, which makes it follow
the OpenBSD and Windows approach.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The vsnprintf calls might return INT_MAX with very long strings.
Prevent a signed integer overflow when taking an additional nul
byte into account.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
If the format buffer shall not be further increased in size, the
length value mistakenly takes the terminating nul byte into account.
This is in contrast to a successful vsnprintf call.
Also use the correct string length if fallback to stack buffer is
required.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The stack buffer is never cleared, which can become an issue depending
on vsnprintf implementation's behavior if -1 is returned. The code
would eventually fall back to stack buffer which might be not
nul terminated.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Hi,
This PR adds support for setting a forced mtime on all written files
(`--mtime` and `--clamp-mtime`) in bsdtar.
The end goal will be to support all functionalities in
<https://reproducible-builds.org/docs/archives/#full-example>, namely
`--sort` and disabling other attributes (atime, ctime, etc.).
Fixes#971.
## History
- [v1](https://github.com/zhaofengli/libarchive/tree/forced-mtime-v1):
Added `archive_read_disk_set_forced_mtime` in libarchive. As a result,
it was only applied when reading from the filesystem and not from other
archives.
- [v2](https://github.com/zhaofengli/libarchive/tree/forced-mtime-v2):
Refactored to apply the forced mtime in `archive_write`.
- v3 (current): Reduced libarchive change to exposing
`archive_parse_date`, moved clamping logic into bsdtar.
---------
Signed-off-by: Zhaofeng Li <hello@zhaofeng.li>
Co-authored-by: Dustin L. Howett <dustin@howett.net>
A filter block size must not be larger than the lzss window, which is
defined
by dictionary size, which in turn can be derived from unpacked file
size.
While at it, improve error messages and fix lzss window wrap around
logic.
Fixes https://github.com/libarchive/libarchive/issues/2565
---------
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Co-authored-by: Tim Kientzle <kientzle@acm.org>
If a system is capable of handling 4 billion nodes in memory, a double
free could occur because of an unsigned integer overflow leading to a
realloc call with size argument of 0. Eventually, the client will
release that memory again, triggering a double free.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
To detect 7z SFX files, libarchive currently searches for the 7z header
in a hard-coded addr range of the PE/ELF file
(specified via macros SFX_MIN_ADDR and SFX_MAX_ADDR). This causes it to
miss SFX files that may stray outside these values (libarchive fails to
extract 7z SFX ELF files created by recent versions of 7z tool because
of this issue). This patch fixes the issue by finding a more robust
starting point for the 7z header search: overlay in PE or the .data
section in ELF. This patch also adds 3 new test cases for 7z SFX to
libarchive.
Fixes https://github.com/libarchive/libarchive/issues/2075
---------
Co-authored-by: Masoud Mehrabi Koushki <masoud.mehrabi.koushki1@huawei.com>
Co-authored-by: Martin Matuška <martin@matuska.de>
This new test archive contains a C hello world executable built like so
on a ubuntu 24.04 machine:
```
int main(int argc, char *argv[]) {
printf("hello, world\n");
return 0;
}
```
`powerpc-linux-gnu-gcc hw.c -o hw-powerpc -Wall`
The test archive that contains this executable was created like so,
using 7-Zip 24.08: `7zz a -t7z -m0=lzma2 -mf=ppc
libarchive/test/test_read_format_7zip_lzma2_powerpc.7z hw-powerpc`
The new test archive is required because the powerpc filter for lzma is
implemented in liblzma rather than in libarchive.
This commit adds support for reading and writing XAR archives on Windows
using the built-in xmllite library. xmllite is present in all versions
of Windows starting with Windows XP.
With this change, no external XML library (libxml2, expat) is required
to read or produce XAR archives on Windows.
xmllite is a little bit annoying in that it's entirely a COM API--the
likes of which are annoying to use from C.
Signed-off-by: Dustin L. Howett <dustin@howett.net>
Depends on e619342dfaCloses#1811
A few small tweaks to improve reading/writing of the legacy GNU tar
format.
* Be more tolerant of redundant 'K' and 'L' headers
* Fill in missing error messages for redundant headers
* New test for reading archive with redundant 'L' headers
* Earlier identification of GNU tar format in some cases
These changes were inspired by Issue #2434. Although that was determined
to not technically be a bug in libarchive, it's relatively easy for
libarchive to tolerate duplicate 'K' and 'L' headers and we should be
issuing appropriate error messages in any case.
The refactoring of https://github.com/libarchive/libarchive/pull/2553
introduced three issues:
1. Introduction of a modifiable global static variable
This violates the goal of having no global variables as stated in [the
README.md](https://github.com/libarchive/libarchive/blob/b6f6557abb8235f604eced6facb42da8c7ab2a41/README.md?plain=1#L195)
which in turn leads to concurrency issues. Without any form of mutex
protection, multiple threads are not guaranteed to see the correct
min/max values. Since these are not needed in regular use cases but only
in edge cases, handle them in functions with local variables only.
Also the global variables are locale-dependent which can change during
runtime. In that case, future calls leads to issues.
2. Broken 32 bit support
The writers for zip and others affected by the previously mentioned PR
and test-suite on Debian 12 i686 are broken, because the calculation of
maximum MS-DOS time is not possible with a 32 bit time_t. Treat these
cases properly.
3. Edge case protection
Huge or tiny int64_t values can easily lead to unsigned integer
overflows. While these do not affect stability of libarchive, the
results are still wrong, i.e. are not capped at min/max as expected.
In total, the functions are much closer to their original versions again
(+ more range checks).
---------
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Make sure that size_t casts do not truncate the value of packed_size on
32 bit systems since it's 64 bit. Extensions to RAR format allow 64 bit
values to be specified in archives.
Also verify that 64 bit signed arithmetics do not overflow.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
It is possible to trigger a call stack overflow by repeatedly entering
the rar_read_ahead function. In normal circumstances, this recursion is
optimized away by common compilers, but default settings with MSVC keep
the recursion in place. Explicitly turn the recursion into a goto-loop
to avoid the overflow even with no compiler optimizations.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Reset avail_in and next_in if the next entry of a split archive is
parsed to always update its internal structure to access next bytes when
cache runs empty.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Support header sizes larger than 32 bit even on 32 bit systems, since
these normally have large file support. Otherwise an unsigned integer
overflow could occur, leading to erroneous parsing on these systems.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Most source files use tabs instead of spaces, but
archive_read_support_format_rar.c uses spaces most of the time. A few
lines contain a mixture of tabs and spaces, which leads to poorly
formatted output with many default settings.
Unify the style. No functional change and preparation for upcoming
changes to get rid of white space diffs.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
PR #2258 hardened AFIO parsing by converting all inode values >= 2^63 to
zero values. This turns out to be problematic for filesystems that use
very large inode values; it results in all such files being viewed as
hardlinks to each other.
PR #2587 partially addressed this by instead considering inode values >=
2^63 as invalid and just ignoring them. This prevented the accidental
hardlinking, but at the cost of losing all hardlinks that involved large
inode values.
This PR further improves things by stripping the high order bit from
64-bit inode values in the AFIO parser. This allows them to be mostly
preserved and should allow hardlinks to get properly processed in the
vast majority of cases. The only false hardlinks would be when there are
inode values that differ exactly in the high order bit, which should be
very rare.
A full solution will require expanding inode handling to use unsigned
64-bit values; we can't do that without a major version bump, but this
PR also sets the stage for migrating inode support in a future
libarchive 4.0.
The calculations for the suffix and prefix can increment the endpoint
for a trailing slash. Hence the limits used should be one lower than the
maximum number of bytes.
Without this patch, when this happens for both the prefix and the
suffix, we end up with 156 + 100 bytes, and the write of the null at the
end will overflow the 256 byte buffer. This can be reproduced by running
```
mkdir -p foo/bar
bsdtar cvf test.tar foo////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////bar
```
when bsdtar is compiled with Address Sanitiser, although I originally
noticed this by accident with a genuine filename on a CHERI capability
system, which faults immediately on the buffer overflow.
Make sure to not skip past end of file for better error messages. One
such example is now visible with rar testsuite. You can see the
difference already by an actually not useless use of cat:
```
$ cat .../test_read_format_rar_ppmd_use_after_free.rar | bsdtar -t
bsdtar: Archive entry has empty or unreadable filename ... skipping.
bsdtar: Archive entry has empty or unreadable filename ... skipping.
bsdtar: Truncated input file (needed 119 bytes, only 0 available)
bsdtar: Error exit delayed from previous errors.
```
compared to
```
$ bsdtar -tf .../test_read_format_rar_ppmd_use_after_free.rar
bsdtar: Archive entry has empty or unreadable filename ... skipping.
bsdtar: Archive entry has empty or unreadable filename ... skipping.
bsdtar: Error exit delayed from previous errors.
```
Since the former cannot lseek, the error is a different one
(ARCHIVE_FATAL vs ARCHIVE_EOF). The piped version states explicitly that
truncation occurred, while the latter states EOF because the skip past
the end of file was successful.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The size_t data type is only 32 bit on 32 bit sytems while off_t is
generally 64 bit to support files larger than 2 GB.
If an entry is declared to be larger than 4 GB and the entry shall be
skipped, then 32 bit systems truncate the requested amount of bytes.
This leads to different interpretation of data in tar files compared to
64 bit systems.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Solves a Windows compile issue when OpenSSH/mbedTLS/Nettle is activated
and on the build system's paths by making the Windows API backend higher
priority on Windows (meaning that only RIPEMD160 will use
OpenSSH/mbedTLS/Nettle anymore).
Fixes#2536 and starts on #2320.
If a warc archive claims to have more than INT64_MAX - 4 content bytes,
the inevitable failure to skip all these bytes could lead to parsing
data which should be ignored instead.
The test case contains a conversation entry with that many bytes and if
the entry is not properly skipped, the warc implementation would read
the conversation data as a new file entry.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
The skip functions are limited to 1 GB for cases in which libarchive
runs on a system with an off_t or long with 32 bits. This has negative
impact on 64 bit systems.
Instead, make sure that _all_ subsequent functions truncate properly.
Some of them already did and some had regressions for over 10 years.
Tests pass on Debian 12 i686 configured with --disable-largefile, i.e.
running with an off_t with 32 bits.
Casts added where needed to still pass MSVC builds.
---------
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
My attempt to fix#2404 just made the confusion between the size of the
extracted file and the size of the contents in the tar archive worse
than it was before.
@ferivoz in #2557 showed that the confusion stemmed from a point where
we were setting the size in the entry (which is by definition the size
of the file on disk) when we read the `GNU.sparse.size` and
`GNU.sparse.realsize` attributes (which might represent the size on disk
or in the archive) and then using that to determine whether to read the
value in ustar header (which represents the size of the data in the
archive).
The confusion stems from three issues:
* The GNU.sparse.* fields mean different things depending on the version
of GNU tar used.
* The regular Pax `size` field overrides the value in the ustar header,
but the GNU sparse size fields don't always do so.
* The previous libarchive code tried to reconcile different size
information as we went along, which is problematic because the order in
which this information appears can vary.
This PR makes one big structural change: We now have separate storage
for every different size field we might encounter. We now just store
these values and record which one we saw. Then at the end, when we have
all the information available at once, we can use this data to determine
the size on disk and the size in the archive.
A few key facts about GNU sparse formats:
* GNU legacy sparse format: Stored all the relevant info in an extension
of the ustar header.
* GNU pax 0.0 format: Used `GNU.sparse.size` to store the size on disk
* GNU pax 0.1 format: Used `GNU.sparse.size` to store the size on disk
* GNU pax 1.0 format: Used `GNU.sparse.realsize` to store the size on
disk; repurposed `GNU.sparse.size` to store the size in the archive, but
omitted this in favor of the ustar size field when that could be used.
And of course, some key precedence information:
* Pax `size` field always overrides the ustar header size field.
* GNU sparse size fields override it ONLY when they represent the size
of the data in the archive.
Resolves#2548
Hello,
- The `CMAKE_COMPILER_IS_*` variables are deprecated and
`CMAKE_C_COMPILER_ID` can be used in this case instead.
- The legacy `endif()` command argument also simplified to avoid
repeating the condition.
Moving archive_entry_set_digest() to the public API simplifies porting
non-archive formats to archive formats while preserving supported
message digests specifically in cases where recomputing digests is not
viable.
The lafe_errc function adds a newline by itself already, so do not
insert one into the message.
You can reproduce with the following commands:
```
touch archive.tar
bsdtar -xf archive.tar -C /non-existing
```
```
bsdtar --exclude ""
```
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Hi,
please find my approach to fix the CVE-2025-1632 and CVE-2025-25724
vulnerabilities in this pr.
As both error cases did trigger a NULL pointer deref (and triggered
hopefully everywhere a coredump), we can safely replace the actual
information by a predefined invalid string without breaking any
functionality.
---------
Signed-off-by: Peter Kaestle <peter@piie.net>
Adding missing librairies to `archive_version_details()`'s output. I put
"system" if the library doesn't give a way to query its version and
"bundled" if there's a choice between the system copy of a library and a
bundled one and we took the bundled copy (Only one library in that case,
libb2. Maybe also xxhash in the future?).
I would have a question for the Windows specialists though: is there a
way to query the interface version of a CNG cryptographic provider?
Because I know of a way for Crypto API providers but I haven't found any
for CNG ones, despite `<bcrypt.h>` having an interface version
structure.
Fixes#2300.
As remarked in #2521, this test has unreachable code on Windows, which
triggers a build failure in development due to warnings-as-errors.
(Release versions should not have warnings-as-errors.)
The outer if checks !S_ISDIR(a->st.st_mode), so we know that the file
being overwritten is not a directory, and thus we can rename(2) over it
if we want to, but whether we can use a temporary regular file is a
property of the file being extracted. Otherwise, when replacing a
regular file with a directory, we end up in this case and create a
temporary regular file for the new directory, but with the permissions
of the directory (which likely includes x), and rename it over the top
at the end. Depending on where the archive_entry came from, it may have
a non-zero size that also isn't ovewritten with 0 (e.g. if it came from
stat(2)) and so the API user may then try to copy data (thus failing if
read(2) of directories isn't permitted, or writing the raw directory
contents if it is), but if the size is zero as is the case for this tar
test then it will end up not writing any data and "successfully"
overwrite the file with an empty file, not a directory.
`Int32x32To64` macro internally truncates the arguments to int32, while
`time_t` is 64-bit on most/all modern platforms. Therefore, usage of
this macro creates a Year 2038 bug.
I detailed this issue a while ago in a writeup, and spotted the same
issue in this repository when updating the list of affected
repositories:
<https://cookieplmonster.github.io/2022/02/17/year-2038-problem/>
A few more notes:
1. I changed all uses of `Int32x32To64` en masse, even though at least
one of them was technically OK and used with int32 parameters only. IMO
better safe than sorry.
2. This is untested, but it's a small enough change that I hope the CI
success is a good enough indicator.
Endianness is easy to determine at runtime, but detecting this a single
time and then reusing the cached result might require API changes.
However we can use compile-time detection for some known compiler macros
without API changes fairly easily. Let's start by enabling this for
Clang and GCC.
This new test archive contains a C hello world executable built like so
on a ubuntu 24.04 machine:
```
#include <stdio.h>
int main(int argc, char *argv[]) {
printf("hello, world\n");
return 0;
}
```
`powerpc-linux-gnu-gcc hw.c -o hw-powerpc -Wall`
The test archive that contains this executable was created like so,
using 7-Zip 24.08:
`7zz a -t7z -m0=deflate -mf=ppc
libarchive/test/test_read_format_7zip_deflate_powerpc.7z hw-powerpc`
This test fails in the first commit in this PR, and passes in the second
commit.
When the -s/regexp/replacement/ option was used with the b flag more
than once, the result of the previous substitution was appended to the
previous subject instead of replacing it. Fixed it by making sure the
subject is made the empty string before the call to realloc_strcat().
That in effect makes it more like a realloc_strcpy(), but creating a new
realloc_strcpy() function for that one usage doesn't feel worth it.
Resolves Issue libarchive/libarchive#2414
Co-authored-by: Stephane Chazelas <stephane@chazelas.org>
We previously told make to run as many threads as it likes on these CI
jobs, but that might sometimes hit resource limits like RAM or the
allowed number of open files.
These numbers were found experimentally by using `sysctl -n hw.ncpu` on
mac and `nproc` on linux.
`i4le()` returns an unsigned int, so `'%d'` is incorrect.
Reported by `clang -Wformat`. (Many more such fixes to come, but this is
the simplest set of them.)
7-Zip 24.05 and liblzma 5.5.1alpha added a RISC-V BCJ filter. Let's
enable this combination if we can.
Note that this does not allow the use of the RISC-V filter with other
compressors.
Previously skipped tests were reported like this when running the *_test
binaries:
```
4: test_acl_platform_nfs4 ok (S)
```
Let's make this more obvious:
```
4: test_acl_platform_nfs4 skipped
```
This plumbing is required for cmake/ctest to recognise and report
skipped tests.
Now skipped tests in cmake ci jobs are reported like so:
```
Start 7: libarchive_test_acl_platform_posix1e_read
7/785 Test #7: libarchive_test_acl_platform_posix1e_read ................................***Skipped 0.02 sec
```
And there is a list of skipped tests shown at the end of the test run.
Prior to this change, the ci autoconf jobs weren't looking for homebrew
headers or libraries unless pkg-config was used, so for example the
"MacOS (autotools)" ci job wasn't testing lz4 or zstd code.
Relates to #2426.
A few of libarchive's CI jobs don't find all the local support libraries
that they could be using. This change makes it easier to see which of
them are used.
We currently use XZ Utils 5.6.3 on windows CI jobs, but the Windows
(msvc)
job which uses cmake seems to only be looking for the old library name,
liblzma.lib:
```
-- Looking for lzma_auto_decoder in C:/Program Files (x86)/xz/lib/liblzma.lib
-- Looking for lzma_auto_decoder in C:/Program Files (x86)/xz/lib/liblzma.lib - not found
-- Looking for lzma_easy_encoder in C:/Program Files (x86)/xz/lib/liblzma.lib
-- Looking for lzma_easy_encoder in C:/Program Files (x86)/xz/lib/liblzma.lib - not found
-- Looking for lzma_lzma_preset in C:/Program Files (x86)/xz/lib/liblzma.lib
-- Looking for lzma_lzma_preset in C:/Program Files (x86)/xz/lib/liblzma.lib - not found
-- Could NOT find LibLZMA (missing: LIBLZMA_HAS_AUTO_DECODER LIBLZMA_HAS_EASY_ENCODER LIBLZMA_HAS_LZMA_PRESET) (found version "5.6.3")
```
We need to update build/ci/github_actions/ci.cmd to look for lzma.lib
instead.
The fallback for when `getline` is not implemented in libc was not
compiling due to the fact that the definition for it was missing, so add
the definition.
I have been using this for years without realizing it decompresses rar.
+ add rar to supported decompression formats
+ use section references to link sections (this makes them clickable in
GUIs)
+ add paragraph breaks for consistent spacing
+ pdtar is not this program, so use Sy per mdoc style guide
+ do almost the same in reverse for bsdtar
+ remove parenthetical around a complete sentance
Thank you so much, this is wonderful software.
This change fixes the autotools build to work with xz-utils 5.6.3, which
changed library names on windows, and fixes a couple of tests that I
noticed had dependencies on liblzma.
Moving the tests' integer reading functions to test_utils so that they
all use the same as well as moving the few using the archive_endian
functions over to the test_utils helper.
Follow-up from libarchive#2390.
When the pax `size` field is present, we should ignore the size value in
the ustar header. In particular, this fixes reading pax archives created
by GNU tar with entries larger than 8GB.
Note: This doesn't impact reading pax archives created by libarchive
because libarchive uses tar extensions to store an accurate large size
field in the ustar header. GNU tar instead strictly follows ustar in
this case, which prevents it from storing accurate sizes in the ustar
header.
Resolves#2404
These two new test archives contain a C hello world executable built
like so on a ubuntu 24.04 machine:
```
#include <stdio.h>
int main(int argc, char *argv[]) {
printf("hello, world\n");
return 0;
}
```
`sparc64-linux-gnu-gcc hw.c -o hw-sparc64 -Wall`
The two test archives that contain this executable were created like so,
using the https://github.com/tehmul/p7zip-zstd fork of 7-Zip:
`7z a -t7z -m0=zstd -mf=SPARC
libarchive/test/test_read_format_7zip_zstd_sparc.7z hw-sparc64`
`7z a -t7z -m0=lzma2 -mf=SPARC
libarchive/test/test_read_format_7zip_lzma2_sparc.7z hw-sparc64`
Two test files are required, because the 7zip reader code has two
different paths, one for lzma and one for all other compressors.
The test_read_format_7zip_lzma2_sparc test is expected to pass, because
LZMA BCJ filters are implemented in liblzma.
The test_read_format_7zip_zstd_sparc test is expected to fail in the
first commit, because libarchive does not currently implement the SPARC
BCJ filter. The second commit will make test_read_format_7zip_zstd_sparc
pass.
Two problems are prompting this revert:
* In order to change the Create OS value to "Windows", we would need to
record other data (such as `external_attributes`) in Windows format as
well.
* Changing the Create OS value doesn't actually fix the
filename-encoding issue that originally motivated this PR
This reverts commit 755af84301.
The `-P` flag is uppercase, so the test file should be named
test_option_P_upper.c for consistency with the other test files in this
directory.
Sorry about the noise.
Submitting this test at the request of @kientzle on issue #2041.
Note that failure is currently expected, as the feature it tests is not
yet implemented!
This commit prepares the XAR writer for another XML writing backend.
Almost everything in this changeset leaves the code identical to how
it started, except for a new layer of indirection between the xar writer
and the XML writer.
The things that are not one-to-one renames include:
- The removal of `UTF8Toisolat1` for the purposes of validating UTF-8
- The writer code made a copy of every filename for the purposes of
checking whether it was Latin-1 stored as UTF-8. In xar, Non-Latin-1
gets stored Base64-encoded.
- I've replaced this use because (1) it was inefficient and (2)
`UTF8Toisolat1` is a `libxml2` export.
- The new function has slightly different results than the one it is
replacing for invalid UTF-8. Namely, it treats illegal UTF-8 "overlong"
encodings of Latin-1 codepoints as _invalid_. It operates on the principle
that we can determine whether something is Latin-1 based entirely on how
long the sequence is expected to be.
- The move of `SetIndent` to before `StartDocument`, which the
abstraction layer immediately undoes. This is to accommodate XML writers
that require indent to be set _before_ the document starts.
PPMD may come later but I'd rather first iron out style issues with the
ones needing only to wire up libraries already-used in Libarchive before
going at the ones possibly requiring implementing algorithms as well.
Closes#1046 and resolves#1179.
Cygwin 3.5.4 (same applies for 3.5.3),
I get a compile error as shown below after a simple ./configure and
make. Adding <windef.h> solves the problem.
Co-authored-by: vco <god@universe.sys>
Bumps the all-actions group with 4 updates:
actions/checkout from 4.1.6 to 4.2.1
actions/upload-artifact from 4.3.3 to 4.4.3
github/codeql-action from 3.25.6 to 3.26.12
ossf/scorecard-action from 2.3.3 to 2.4.0
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This is the first part of converting the project to use SPDX license
identifiers instead using the verbose license text.
The patches are semi-automated and I've went through manually to ensure
no license changes were made. That said, I would welcome another pair of
eyes, since I am only human.
See https://github.com/libarchive/libarchive/issues/2298
---------
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
People really should never, ever, ever use libarchive internal headers. And they definitely should not expect libarchive internal headers to work in a C++ compiler. (C++ and C are really just not that compatible.)
However, people do a lot of things they shouldn't: Avoid the reserved C++ keyword `template`
Followup from #2346
Add libbsd to make/cmake configuration for linking readpassphrase on
Haiku.
Maybe there is a better way to do this for cmake, I'm not that familiar
with it.
Co-authored-by: vco <god@universe.sys>
I previously tried to find documentation on how symlinks are expected to
be stored in 7zip files, however the best reference I could find was
[here](https://py7zr.readthedocs.io/en/latest/archive_format.html). That
site suggests that symlink paths are stored as UTF-8 encoded strings:
Currently, the RAR5 code always reports
`ARCHIVE_READ_FORMAT_ENCRYPTION_UNSUPPORTED` for
`archive_read_has_encrypted_entries`, nor does it set any of the
entry-specific properties, even though it has enough information to
properly report this information. Accurate reporting of encryption is
super useful for applications because reporting an error message such as
"the archive is encrypted, but we don't currently support encryption" is
a lot better than a not generally useful `errno` value and a
non-localizable error string with a confusing and unpredictable error
message.
Fixes#1661
Change to read absolute symlinks as verbatim paths instead of NT paths:
as far as I can see, libarchive can deal with verbatim paths while it
can't with NT ones.
Fixes#2274.
Remove the incorrect 4th argument from `AC_CHECK_FUNCS` calls. The macro
uses only three arguments, so it was ignored anyway. Furthermore, in at
least once instance it was wrong -- due to a typo in `attr/xatr.h`
header name.
The tar header parsing overhaul in #2127 introduced a systematic
mishandling of truncated files: the code incorrectly checks for whether
a given read operation failed, and ends up dereferencing a NULL pointer
in this case. I've gone back and double-checked how
`__archive_read_ahead` actually works (it returns NULL precisely when it
was unable to satisfy the read request) and reworked the error handling
for each call to this function in archive_read_support_format_tar.c
Resolves#2353
Resolves https://issues.oss-fuzz.com/issues/42537231
OSS-Fuzz managed to construct a small gzip input that decompresses into
another gzip input with an extremely large filename field. This causes
libarchive to hang processing the inner gzip.
Address this by rejecting any gzip input where the filename or comment
fields exceed 1MiB.
Credit: OSS-Fuzz
No functional change, just a tiny style improvement.
Use `crc32_computed` to refer to the crc32 that the reader has computed
and `crc32_read` to refer to the value that we read from the archive.
That hopefully makes this code a tiny bit easier to follow. (It confused
me recently when I was double-checking something in this area, so I
thought an improvement here might help others.)
We always print the error message with or without -v, but for some
reason, we were omitting the path being processed. Simplify so that we
always print the full error including context.
This fixes various code quality issues I encountered while chasing a
memory leak reported by test automation. I failed to reproduce the
memory leak, but I hope you find this useful nonetheless.
PR #2127 failed to clean up the linkpath storage between entries. As a
result, after the first hard/symlink entry in a pax format archive, all
subsequent entries would get the same link information.
I'm really unsure how this bug failed to trip CI. I'll do some digging
in the test suite before I merge this.
Resolves#2331 , #2337
P.S. Thanks to Brad King for noting that the linkpath wasn't being
managed correctly, which was a big hint for me.
Synchronize the last use of `attr/xattr.h` to support using
`sys/xattr.h` instead. The former header is deprecated on GNU/Linux, and
this replacement makes it possible to build libarchive without the
`attr` package.
Fix memory leaks introduced by #2127:
* `struct tar` member `entry_linkpath` was moved at the same time as
other members were removed, but its cleanup was accidentally removed
with the others.
* `header_pax_extension` local variable `attr_name` was not cleaned up.
Resolves#2336
Some ISO images don't have valid timestamps for the root directory
entry. Parsing such timestamps can generate nonsensical results, which
in one case showed up as an unexpected overflow on a 32-bit system.
Add some validation logic that can check whether a 7-byte or 17-byte
timestamp is reasonable-looking, and use this to ignore invalid
timestamps in various locations. This also requires us to be a little
more careful about tracking which timestamps are actually known.
Resolves issue #2329
Followup to #2318 which accidentally made zlib required.
Tested locally by increasing the version in CMakeLists.txt to 1.4.1
(which does not exist yet), and confirming that the build reports that a
suitable version of zlib was not found, while the build continued..
Before writing a zip entry, its' pathname might be modified for two
reasons:
1. Path using Windows path separators will be converted to POSIX style.
2. Path using local encoding will be transcoded if a target charset is
set.
Must make sure these two mechanisms can coexist without overwriting each
other.
zlib 1.2.0 added this improvement for inflate:
"Raw inflate no longer needs an extra dummy byte at end"
libarchive does not feed zlib extra data beyond end of stream, so it
does not work with zlib < 1.2.0.
With 26 and 27, the sub-test is pushing 2G and 4G memory respectively.
There is no particular reason why we need to push for higher limits
here, so let's pick 23 with weights around 0.25G. The test suite overall
is in the 0.25 - 0.5G range and this fits perfectly.
Closes: https://github.com/libarchive/libarchive/issues/2080
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Fix `test_write_format_zip_stream` failure when `HAVE_ZLIB_H` is not
defined.
If `libz` is present, `zip` archives would be compressed by default,
which requires `zip_version=20`. Otherwise, the archive is not
compressed and only requires `zip_version=10`. I'm building libarchive
on a machine not intended for developing, so basicly there's no optional
dependencies like `libz` available, guess that's why nobody else has
reported this issue.
Pax introduced new headers that appear _before_ the legacy
headers. So pax archives require earlier properties to
override later ones.
Originally, libarchive handled this by storing the early
headers in memory so that it could do the actual parsing
from back to front. With this scheme, properties from
early headers were parsed last and simply overwrote
properties from later headers.
PR #2127 reduced memory usage by parsing headers in the
order they appear in the file, which requires later headers
to avoid overwriting already-set properties. Apparently,
when I made this change, I did not fully consider how charset
translations get handled on Windows, so failed to consistently
recognize when the path or linkname properties were in fact
actually set. As a result, the legacy path/link values (which have
no charset information) overwrote the pax path/link values (which
are known to be UTF-8), leading to the behavior observed in
#2248. This PR corrects this bug by adding additional
tests to see if the wide character path or linkname properties
are set.
Related: This bug was exposed by a new test added in #2228
which does a write/read validation to ensure round-trip filename
handling. This was modified in #2248 to avoid tickling the bug above.
I've reverted the change from #2248 since it's no longer necessary.
I have also added some additional validation to this test to
help ensure that the intermediate archive actually is a pax
format that includes the expected path and linkname properties
in the expected places.
It would seem as though #2127 conflicted with my change #2228.
I previously thought that the writer was putting info into the archive
that strings were encoded in UTF-8, but I'm not so sure of that
anymore... In any case, explicitly setting `hdrcharset` on the reader as
well is a reasonable alternative and something we do already.
The RAR5 reader is using a small stack of cached pointers to submit the
rendered data to the caller. In malformed files it's possible for this
pointer cache to be desynchronized with the memory buffer those pointers
are pointing to, making libarchive crash on invalid memory access.
OSS-Fuzz Issue: 70024
In particular, this ensures that we cannot overflow rounding-up
calculations. Recent tar changes put in a lot of sanity limits on the
sizes of particular kinds of data, but the usual behavior in most cases
was to skip over-large values. The skipping behavior required
rounding-up and accumulating values that could potentially overflow
64-bit integers. This adds some coarser checks that fail more directly
when an entry claims to be more than 1 exbibyte (2^60 bytes), avoiding
any possibility of numeric overflow along these paths.
OSS-Fuzz Issue: 70062
This is somewhat academic, since we don't actually expose any of the
ISO9660 header information that is stored in 17-byte date format, but
inspection revealed an off-by-one error in the parsing here.
This also proved a nice motivation to fill in some verification in our
most basic ISO9660 test case.
Currently updating archivemount which does
```c
pwd = getpwuid(st.st_uid);
if(pwd)
archive_entry_set_uname(node->entry, strdup(pwd->pw_name));
grp = getgrgid(st.st_gid);
if(grp)
archive_entry_set_gname(node->entry, strdup(grp->gr_name));
```
and I'm assuming the strdups are actually leaks? The manual is silent on
this.
Previous code added `.XXXXXX` to the end of the filename to write the
mac metadata. This is a problem if the filename is at or near the
filesystem max path length. This reuses the same code used by
create_tempdatafork to ensure that the filename is not too long.
The fuzzer constructed an AFIO (CPIO variant) archive that had a
rediculously large ino value, which caused an overflow of a signed
64-bit intermediate.
There are really three issues here:
* The CPIO parser was using a signed int64 as an intermediate type for
parsing numbers in all cases. I've addressed the overflow here by using
a uint64_t in the parser core, but left the resulting values as int64_t.
* The AFIO header parsing had no guards against rediculously large
values; it now rejects an archive when the ino or size fields (which are
allowed to be up to 16 hex digits long) overflow int64_t to produce a
negative value.
* The archive_entry would accept negative values for gid/uid/size/ino.
I've altered those so that these fields treat any negative value as zero
for these fields.
There was one test that actually verified that we could read a field
with size = -1. I've updated that to verify that the resulting size is
zero instead.
OSS-Fuzz Issue: 70019
The Rar5 reader would read the name size, then read the name, then check
whether the name size was beyond the maximum size allowed. This can
result in a very large memory allocation to read a name. Instead, check
the name size before trying to read the name in order to avoid excessive
allocation.
OSS-Fuzz Issue: 70017
I went through ~50 findings of SAST reports and identified a few of them
as true positives. I might still have missed some intended uses or some
magic in the code so please provide feedback if you think some of these
shouldn't be applied and why.
I explained the changes in the separate comments.
Microsoft's static analysis tool found some vulnerabilities from
unguarded null references that I changed in
[microsoft/cmake](https://github.com/microsoft/cmake). Pushing these
changes upstream so they can be added to
[kitware/cmake](https://github.com/Kitware/CMake).
The code currently uses `archive_entry_hardlink` to determine if an
entry is a hardlink, however on Windows, this call will fail if the path
cannot be represented in the current locale. This instead checks to see
if any entry in the `archive_mstring` is set.
All three parts of this change effectively stem from the same
assumption: most of the code in `archive_string.c` assumes that MBS <->
UTF-8 string conversion can be done directly and efficiently. This is
not quite true on Windows, where conversion looks more like MBS <-> WCS
<-> UTF-8. This results in a few inefficiencies currently present in the
code.
First, if the caller is asking for either the MBS or UTF-8 string, but
it's not currently set on the `archive_mstring`, then on Windows, it's
more efficient to first check if the WCS is set and do the conversion
with that. Otherwise, we'll end up doing a wasteful intermediate step of
converting either the MBS or UTF-8 string to WCS, which we already have.
Second, in the `archive_mstring_update_utf8` function, it's more
efficient on Windows to first convert to WCS and use that result to
convert to MBS, as opposed to the fallback I introduced in a previous
change, which converts UTF-8 to MBS first and disposes of the
intermediate WCS, only to re-calculate it.
We noticed an issue where we had an archive that, if you skipped the
first entry and tried to extract the second, you'd get a failure saying
`Truncated 7-Zip file body`. Turns out that this is because the first
file in the archive is a multiple of 65,536 bytes (the size of the
uncompressed buffer) and therefore after `read_stream` skipped all of
the first file, `uncompressed_buffer_bytes_remaining` was set to zero
(because all data was consumed) and then it calls
`get_uncompressed_data` with `minimum` set to zero. This then saw that
`minimum > zip->uncompressed_buffer_bytes_remaining` evaluated to false,
causing us to read zero bytes, which got interpreted as a truncated
archive.
The fix here is simple: we now always call `extract_pack_stream` when
`uncompressed_buffer_bytes_remaining` is zero before exiting the
skipping loop.
In order to match cpio output, format the reference date with _at least_
12 bytes instead of _exactly_ 12 bytes. This should fix a gratuitous
test failure on certain systems that default to multi-byte locales.
The CSRG ISOs have a non-standard PVD layout with a 68-byte root
directory record (rather than the 34-byte record required by
ECMA119/ISO9660). I built a test image with this change and modified the
ISO9660 reader to accept it.
While I was working on the bid logic to recognize PVDs, I added a number
of additional correctness checks that should make our bidding a bit more
accurate. In particular, this should more than compensate for the
weakened check of the root directory record size.
Resolves#2232
This rebuilds the tar reader to parse all header data incrementally as
it appears in the stream.
This definitively fixes a longstanding issue with unsupported pax
attributes. Libarchive must limit the amount of data that it reads into
memory, and this has caused problems with large unknown attributes. By
scanning iteratively, we can instead identify an attribute by name and
then decide whether to read it into memory or whether to skip it without
reading.
This design also allows us to vary our sanity limits for different pax
attributes (e.g., an attribute that is a single number can be limited to
a few dozen bytes while an attribute holding an ACL is allowed to be a
few hundred kilobytes). This allows us to be a little more resistant to
malicious archives that might try to force allocation of very large
amounts of memory, though there is still work to be done here.
This includes a number of changes to archive_entry processing to allow
us to consistently keep the _first_ appearance of any given value
instead of the original architecture that recursively cached data in
memory in order to effectively process all the data from back-to-front.
Resolves#1855Resolves#1939
Note: this is a partial cherry-pick from
https://github.com/libarchive/libarchive/pull/2095, which I'm going to
go through and break into smaller pieces in hopes of getting some things
in while discussion of other things can continue.
There's basically two fixes here:
The first is to check for the presence of the WCS pathname on Windows
before failing since the conversion from WCS -> MBS might fail. Later
execution already handles such paths correctly.
The second is to set the converted link name on the target entry where
relevant. Note that there has been prior discussion on this here:
https://github.com/libarchive/libarchive/pull/2095/files#r1531599325
certain rar files seem to have the lowest possible address here, so flip
the argument order to correctly evaluate this instead of invoking UB
(caught via sanitize=undefined)
---
the backtrace looks something like:
```
* frame #0: 0x00007a1e3898727b libarchive.so.13`execute_filter [inlined] execute_filter_e8(filter=<unavailable>, vm=<unavailable>, pos=<unavailable>, e9also=<unavailable>) at archive_read_support_format_rar.c:3640:47
frame #1: 0x00007a1e3898727b libarchive.so.13`execute_filter(a=<unavailable>, filter=0x00007a1e39e2f090, vm=0x00007a1e31b1efd0, pos=<unavailable>) at archive_read_support_format_rar.c:0
frame #2: 0x00007a1e38983ac3 libarchive.so.13`read_data_compressed [inlined] run_filters(a=0x00007a1e34209700) at archive_read_support_format_rar.c:3395:8
frame #3: 0x00007a1e38983a9e libarchive.so.13`read_data_compressed(a=0x00007a1e34209700, buff=0x00007a1e31a01fd8, size=0x00007a1e31a01fd0, offset=0x00007a1e31a01fc0, looper=1) at archive_read_support_format_rar.c:2083:12
frame #4: 0x00007a1e38981b10 libarchive.so.13`archive_read_format_rar_read_data(a=0x00007a1e34209700, buff=0x00007a1e31a01fd8, size=0x00007a1e31a01fd0, offset=0x00007a1e31a01fc0) at archive_read_support_format_rar.c:1130:11
frame #5: 0x00006158bc5d30d3 file-roller`extract_archive_thread(result=0x00007a1e3711e2b0, object=<unavailable>, cancellable=0x00007a1e3870bf20) at fr-archive-libarchive.c:999:17
frame #6: 0x00007a1e39928d6d libgio-2.0.so.0`run_in_thread(job=<unavailable>, c=<unavailable>, _data=0x00007a1e326e9740) at gsimpleasyncresult.c:899:5
frame #7: 0x00007a1e3990614e libgio-2.0.so.0`io_job_thread(task=<unavailable>, source_object=<unavailable>, task_data=0x00007a1e2307fc20, cancellable=<unavailable>) at gioscheduler.c:75:16
frame #8: 0x00007a1e399433bf libgio-2.0.so.0`g_task_thread_pool_thread(thread_data=0x00007a1e35c18ab0, pool_data=<unavailable>) at gtask.c:1583:3
frame #9: 0x00007a1e39db77e8 libglib-2.0.so.0`g_thread_pool_thread_proxy(data=<unavailable>) at gthreadpool.c:336:15
frame #10: 0x00007a1e39db5bfb libglib-2.0.so.0`g_thread_proxy(data=0x00007a1e378147d0) at gthread.c:835:20
frame #11: 0x00007a1e3a0b5c7b ld-musl-x86_64.so.1`start(p=0x00007a1e31a02170) at pthread_create.c:208:17
frame #12: 0x00007a1e3a0b8a8b ld-musl-x86_64.so.1`__clone + 47
```
note the 0xd which is 14 which is NegateOverflow in ubsan:
```
(lldb) x/1i $pc
-> 0x7a1e3898727b: 67 0f b9 40 0d other ud1l 0xd(%eax), %eax
```
for reference, the totally legal rar file is
https://img.ayaya.dev/05WYGFOcRPN9 , and this seems to only crash when
extracted via file-roller (or inside nautilus)
It appears that there are xar archives (in the form of Apple .pkg files)
that contain TOCs with duplicated name elements:
```xml
<file id="25">
<data> ... </data>
<type>file</type>
<name>PackageInfo</name>
<name>PackageInfo</name>
<name>PackageInfo</name>
</file>
```
When libarchive encounters one such file, it will produce an
archive_entry named PackageInfoPackageInfoPackageInfo.
To produce a test archive, the XAR writer was modified to emit two name
elements.
On Windows, the MBS pathname might be null if the string was set with a
WCS that can't be represented by the current locale. This is handled
properly by the rest of the code, but there's a sanity check that does
not make the proper distinction.
Note: this is a partial cherry-pick from
https://github.com/libarchive/libarchive/pull/2095, which I'm going to
go through and break into smaller pieces in hopes of getting some things
in while discussion of other things can continue.
There's no bug fix here - this just adds a test to verify that zip
creation when using the _w functions works as expected on Windows.
Note: this is a partial cherry-pick from
https://github.com/libarchive/libarchive/pull/2095, which I'm going to
go through and break into smaller pieces in hopes of getting some things
in while discussion of other things can continue.
This ensures that the buffer is properly initialized and does not
contain any leftover data from previous operations. It is used later in
the `archive_entry_copy_hardlink_l` function call and could be
uninitialized.
On Windows, if you are using `archive_entry_link_resolver` and give it
an entry that links to past entry whose pathname was set using a "wide"
string that cannot be represented by the current locale (i.e. WCS -> MBS
conversion fails), this code will crash due to a null pointer read. This
updates to use the `_w` function instead on Windows.
Note: this is a partial cherry-pick from
https://github.com/libarchive/libarchive/pull/2095, which I'm going to
go through and break into smaller pieces in hopes of getting some things
in while discussion of other things can continue.
On legacy systems the OS supplied `sys/queue.h` may lack the required
macros, so to avoid having to verify if the version of queue.h is of
use, opt to always to `la_queue.h` which will match expectations.
Allows libarchive to build on legacy Darwin where `STAILQ_FOREACH` would
be missing from `sys/queue.h`.
Resolves#2220
When using Clang in "MSVC mode" (i.e. clang-cl), command line arguments
are interpreted as MSVC would interpret them, at least when there are
conflicts. This means that `-Wall` - potentially among other switches -
is interpreted _dramatically_ differently by clang-cl compared to
"normal" Clang.
In CMake, this can be detected by testing for `if (MSVC)` in addition to
compiler id test, which is what I do here.
Note: this is a partial cherry-pick from #2095, which I'm going to go
through and break into smaller pieces in hopes of getting some things in
while discussion of other things can continue.
The tar utility reads from stderr to receive user input even when stdin
is a pipe. That is unfortunately unsupported on Windows.
The nearest equivalent is to reopen and read from the console input
handle.
Closes#2215
The parsers for 7zip and xar contain memory leaks. Also the testsuite
7zip leaks memory. Plug them.
Noticed while running test suite with address sanitizer on Linux.
Turning window_mask into ssize_t and adjusting the signature of
circular_memcpy reduces the amount of required casts. Since window_mask
depends on value window_size, which is ssize_t, this change is safe.
Also turned extra_data_size into int64_t, which is a no-op on 64 bit
systems but prevents 32 bit systems from truncating these huge values in
archives. Since these systems most likely have large file support, this
could have meant truncation -- in theory.
Reaching EOF for first time sets the correct amount of available bytes,
but each subsequent call returns 0.
Do not forget that the copy buffer can already contain data and return
the amount of bytes left in there.
See added test case.
This actually disabled CRC checks on regular builds and enabled it if
CRC checks were supposed to be disabled.
All other occurrences properly use ifndef instead of ifdef.
Bumps the all-actions group with 2 updates:
`actions/checkout` from 4.1.5 to 4.1.6
`github/codeql-action` from 3.25.5 to 3.25.6
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
`blocklength` should be bigger than `3` (channel count)
`byteoffset` should not be bigger than `2` (does not make sense as per the
last loop)
`src` should not overlap with `dst`.
There is no allocation in this function so it should be safe to return
early.
Security: GHSA-9qqv-q4qw-mf8m
If libarchive is built without zlib support, it is possible to trigger a
NULL pointer dereference with specially crafted 7zip files.
It takes multiple conditions to be able to reach the issue in crc32 and
all 7zip-specific ones are fixed with this PR.
Most of the functions already use size_t for sizes. Do the same for
xml_data. The libraries expat and libxml2 do not support strings larger
than 2 GB (limit is less than this) but it fixes one last Visual Studio
compiler warning which I missed previously.
Use casts where appropriate (values cannot exceed data type limits).
Fixes following warnings:
warning C4244: '=': conversion from 'int64_t' to 'long', possible loss
of data
warning C4244: '=': conversion from 'uint64_t' to 'size_t', possible
loss of data
Updates `actions/checkout` from 4.1.4 to 4.1.5
Updates `github/codeql-action` from 3.25.3 to 3.25.5
Updates `ossf/scorecard-action` from 2.3.1 to 2.3.3
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Adjust type of variables for their specific use case
Add a new inline function to unify casting (and clarify code blocks)
Use definition to explain magic number (and reduce casts)
Use `hpos` instead of magic number to highlight that additional bytes
will be parsed
Switch from int64_t to ssize_t for l and body, which is a no-op on 64
bit systems. On 32 bit systems, this change is okay because these
variables interact with other ssize_t variables in this scope.
Fixes compiler warning regarding line 584 in which ssize_t and int64_t
are mixed in calculations:
warning C4244: '=': conversion from 'int64_t' to 'long'
The cast in ppmd_read function is safe even on 32 bit systems. It is
called byte for byte which makes it impossible to actually reach
SIZE_MAX on any real world hardware.
Fixes Visual Studio warnings:
warning C4244: '=': conversion from 'uint64_t' to 'size_t', possible
loss of data
warning C4244: 'function': conversion from 'int64_t' to 'size_t',
possible loss of data
This setenv() call may clobber the memory pointed to by lang.
It is also insufficient, since you don't run in a clean environment, so
LANG may be overridden by an inherited LC_ALL or LC_CTYPE, or by the
user's .profile (remember that system() does not execute the command
directly, but passes it to a shell).
Reported-By: dag-erling (quoting)
copy_from_lzss_window_to_unp unnecessarily took an `int` parameter where
both of its callers were holding a `size_t`.
A lzss opcode chain could be constructed that resulted in a negative
copy length, which when passed into memcpy would result in a very, very
large positive number.
Switching copy_from_lzss_window_to_unp to take a `size_t` allows it to
properly bounds-check length.
In addition, this patch also ensures that `length` is not itself larger
than the destination buffer.
Security: CVE-2024-20696
If EOF is encountered while reading the new filename after choosing 'r',
avoid out of boundary access and usage of undefined memory content by
treating it the same way as if the question itself was not answered.
Processing excessively long lines could lead to out of boundary writes
or denial of service due to O(n^2) runtime complexity.
The OOB is properly fixed with first commit. The second commit stops
processing of lines which are longer than uu allows due to its
specification.
The comp size could be around INT_MAX on huge archives, which would lead
to eventual integer truncation to size_t in archives with version 1
headers when fixed value 2 is added to comp_size on 32 bit systems.
This fix is a no-op on 64 bit systems because size_t and uint64_t are of
same size there.
Cast any of cd_offset or cd_size to int64_t to avoid truncation of
result because both variables are of type uint32_t.
The calculation happens before comparison with current_offset, so it is
not automatically expanded to int64_t during calculation.
If an RPM file contains a huge header which is larger than 4 GB then
libarchive starts parsing the RPM header as actual archive instead of
skipping it.
Switched to uint64_t from size_t for proper 32 bit support as well.
When working with matches, the code does not care about the actual
amount of times when it matched, but just if it matched so far at least
once or never.
Turning the counter into a boolean flag has the advantage that even
insanely huge archives will never lead to integer overflow here.
If call of archive_read_support_format_raw fails, do not override the
error return value with the return value of
archive_read_support_format_empty(). Instead, return error code as expected.
Ensure that `src` won't move ahead of `dst`, so `src` will not OOB.
Since `dst` won't move in this function, and we are only increasing `src`
position, this check should be enough. It should be safe to early return
because this function does not allocate resources.
The code here had a couple of bad code patterns that seem to have been
copied throughout:
* Checking errno after strtol() -- Standard C doesn't seem to actually
require this, so we shouldn't rely on it
* Casting the result of strtol() directly to `int`. This loses
information prematurely.
Instead, I've added `l` as a temporary of type `long`, use that to hold
the result of `strtol()` until it can be checked. I've also removed the
`errno` tests in favor of checking the end pointer value.
The limit for --strip-components has been raised to 100 000.
Updates actions/checkout from 4.1.2 to 4.1.3
Updates actions/upload-artifact from 4.3.1 to 4.3.3
Updates github/codeql-action from 3.24.8 to 3.25.2
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
The bsdtar manual page claims that setting zstd:threads to 0 tells zstd
to use as many threads as there are cores in the system, but it actually
disables multi-threading. Replace 0 with the number of configured
processors.
While here, add a previously missing overflow check.
Co-authored-by: Martin Matuska <martin@matuska.de>
A file may have only one link target at a time. Otherwise the internal
link structure could loop. Besides, a hard link realistically can only
link to one file, not multiple ones.
Consider such an archive invalid.
Co-authored-by: Martin Matuska <martin@matuska.de>
This is a clone of the reverted pull request #2100. Due to the latest
circumstances we should require a more deep review of each new PR.
---------
Co-authored-by: Justin Vreeland <j_vreeland@apple.com>
The uint64_t variable propertiesSize is eventually casted to size_t
which, on 32 bit systems, can result in integer truncation.
In such a situation, it is possible that less than the minimum of 5
properties are parsed and processed, which will result in out of
boundary reads in init_decompression because the error check `if
(coder1->propertiesSize < 5)` still takes the uint64_t variable into
account.
Currently the `CMAKE_BUILD_TYPE` is being compared in a case-sensitive
way. It seems current CMake documentation [suggests treating this in a
case-insensitive manner
now-a-days](https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#case-sensitivity).
This being case-sensitive creates needless complexities in other
projects if they compile their own project with `cmake
-DCMAKE_BUILD_TYPE=release ..`, etc. In this specific case, libarchive
has a fatal error due to the lowercase `release`.
I'd honestly like to remove these comparisons entirely; as I'm not sure
if they're really needed or not if `libarchive` is only using the
Makefile or Ninja generators with CMake.
This PR changes the `CMAKE_BUILD_TYPE` comparison to be
case-insensitive, and leaves the rest alone.
This should also fix the following issue(s):
* https://github.com/libarchive/libarchive/issues/1792
Passwords for encryption must not be empty. Neither through command line
option nor through interactive input.
With this PR applied:
```
$ bsdtar --format zip --options zip:encryption -cf archive.zip input.txt
Enter passphrase: <press enter>
bsdtar: Encryption needs passphrase
```
Output with command line argument (unaffected by this PR):
```
$ bsdtar --format zip --options zip:encryption --passphrase '' -cf archive.zip input.txt
bsdtar: Empty passphrase is unacceptable
```
The outputs differ due to internal difference in handling the results.
It is still possible to supply a passphrase through command line
argument which cannot be entered interactively, i.e. $'\r\n'. See
https://github.com/libarchive/libarchive/pull/2115 for more details.
If libarchive is compiled on Windows without cygwin, strip \r and \n the
same way as it is done on POSIX systems.
Also, entering an empty password as "\r\n" should lead to an empty
string. Right now, the newlines are kept.
Proof of Concept:
1. Compile libarchive with Visual Studio
2. Create a password-protected ZIP file
```
PS> bsdtar.exe --format zip --options 'zip:encryption' -cf archive.zip input.txt
Enter passphrase: <press enter>
```
3. Extract ZIP file on Windows
```
PS> bsdtar.exe -xf archive.zip
Enter passphrase: <press enter>
```
4. Extract ZIP file on Linux
```
$ bsdtar -xf archive.zip
Enter passphrase: <press enter>
Enter passphrase:
```
As can be seen in step 4, it is impossible to extract the file on Linux
with interactive input, because \r and \n are stripped.
The only way to extract the content is through command line option
passphrase:
```
$ bsdtar -xf archive.zip --passphrase $'\r\n'
```
It's also true the other way around: Creating a ZIP file with an empty
password on Linux cannot be extracted interactively on Windows. Not
allowing empty passwords at all should be part of another PR. This one
is about unifying Windows and POSIX systems regarding newline handling.
These files are created when building libarchive within Visual Studio
Code, its CMake plugin and Visual Studio 2022 as compiler.
Without this change, Visual Studio Code shows hundreds of files in
GitLense plugin waiting to be commited.
Supplying a lot of "-" arguments to tools can lead to stack overflow due
to recursive *_getopt function calls.
Proof of Concept:
1. Compile libarchive with Visual Studio 2022
2. Call bsdtar with insane amount of arguments
```
PS> bsdtar.exe ("- "*10000).split(" ")
```
The event log shows that bsdtar.exe failed with `0xc00000fd` (stack
overflow).
If compiled with gcc, this does not happen by default because the code
is internally optimized to use this suggested loop instead. You have to
compile with CFLAGS="-O0" to provoke it with gcc as well.
Calling `__archive_mkstemp` can lead to access of an uninitialized
variable in `__archive_mktempx`, because `temp_name` is only initialized
if supplied `template` argument is `NULL`.
If `template` is not `NULL`, it is eventually compared with
`temp_name.s` anyway.
The fix is simple: Always initialize `temp_name`, which merely sets
values in the struct. No memory allocation occurs and the check leads to
the expected result.
How to reproduce:
1. Compile libarchive with Visual Studio 2022 and CMake's Debug profile
2. Run test `bsdtar_test_option_safe_writes`
3. A popup (Microsoft Visual C++ Runtime Library) appears, stating that
variable temp_name is being used without being initialized
`string_to_size` is only used in a code block conditionally compiled
with `#if HAVE_ZSTD_H && HAVE_ZSTD_compressStream`. If this block is not
compiled, GCC raises a warning with -Wunused-function.
configure scripts need to be runnable with a POSIX-compliant /bin/sh.
On many (but not all!) systems, /bin/sh is provided by Bash, so errors
like this aren't spotted. Notably Debian defaults to /bin/sh provided by
dash which doesn't tolerate such bashisms as '=='.
This retains compatibility with bash.
Signed-off-by: Sam James <sam@gentoo.org>
Follow-on to #1649: this just changes the name of the preprocessor macro
to use the standard pattern HAVE_<function name>
In particular: newer ZSTD implementations have a growing variety of
compression functions; the standard pattern will make it easier to
select among those someday.
This reworks PR #1613 (commit a902fcd4), addressing the same issues as that
earlier PR, but in a simpler and more portable fashion.
Co-authored-by: Martin Matuška <martin@matuska.org>
There appear to be RAR5 archives out in the wild that have blocks at the
end of files that don't produce any data. The code in libarchive has an
infinite loop that won't break until it processes a block that produces
data, which will end up reading past the end of the file if the last
block in the file produces no data.
This significantly changes how end-of-data markers are parsed.
In particular, the spec allows the end-of-data marker to have either
32-bit or 64-bit size values, and there is basically no indication which
is being used. (The spec mentions "Zip64 mode" in many places, but there
is no definitive way for a Zip reader to know whether the writer is
using this mode or not. My mis-reading of another part of the spec
caused me to believe that the Zip64 Extra Data field was such a marker,
but I've been patiently corrected. ;-)
So a Zip reader just has to guess: Try every possible end-of-data marker
format and accept it if any of the four possible forms is correct. In
libarchive's case, this required some non-trivial additional refactoring
to ensure that the CRC32, compressed size, and uncompressed size
statistics are always updated _before_ we need to look for an
end-of-data marker.
This generally follows the strategy outlined by Mark Adler for his
`sunzip` streaming unzip implementation.
While testing this, I played with pmqs/zipdetails which pointed out a
discrepancy in how libarchive writes the `UT` extra field. I folded a
fix for that in here as well.
Resolves#1834
TODO: It would be nice to augment the test suite with some static files
created by Java's implementation to verify that we can read those when
they hold entries of +/- 4GiB. The existing
`test_write_format_zip_large` uses an ad hoc RLE encoding trick to
exercise writing and reading back multi-gigabyte entries. I wonder if
that could be generalized to support deflate-compressed Zip data stored
in test files?
The new behavior of this policy makes check_include_file() and friends
use the CMAKE_REQUIRED_LIBRARIES variable before checking for headers.
This is used to check for cryptography algorithms provided by OpenSSL.
Signed-off-by: Collin Funk <collin.funk1@gmail.com>
For the most part, a good number of
failing tests are failing because they make the assumption that archive
contents are read as utf-8 by default, which is not true for Windows,
which assumes OEM code page by default.
Instead of just `min-frame-size` and `max-frame-size`, we now have four
separate options:
* `min-frame-in` delays the creation of a new frame on flush until the
uncompressed size of the current frame passes a certain threshold.
* `min-frame-out` delays the creation of a new frame on flush until the
compressed size of the current frame passes a certain threshold.
* `max-frame-in` forces the creation of a new frame as soon as possible
after the uncompressed size of the current frame reaches a certain
limit.
* `max-frame-out` forces the creation of a new frame as soon as possible
after the compressed size of the current frame reaches a certain limit.
We now also support `k`, `kB`, `M`, `MB`, `G` and `GB` suffixes for all
four options.
The old options are retained as aliases for the corresponding new
options.
The accepted way to specify multiple options is to list them all as one
comma-separated parameter. bsdtar would silently ignore all but the last
value, which can be very confusing. Print a warning in this scenario.
Currently, functions like `archive_entry_pathname_w` etc. fail on
Windows for .rar files that contain entries with Unicode filenames that
cannot be represented by the active codepage. This is because
`archive_mstring_update_utf8` first attempts to perform a UTF8->MBS
conversion before doing an MBS->WCS conversion. The first conversion (to
MBS) fails, which short-circuits to return failure. Later when we try
and read the string, `archive_mstring_get_wcs` will fail because it
_also_ tries to do a UTF8->MBS followed by an MBS->WCS conversion. The
conversion to MBS will of course fail again.
One possible workaround is to call `setlocale` with something like
`"en_US.utf8"`, however this is not feasible for some consumers.
This change fixes this issue by adding a "fallback" in
'archive_mstring_update_utf8' which will attempt to do a UTF8->WCS
conversion on Windows if the MBS conversion failed. This is not too
dissimilar from the implementation of `archive_mstring_copy_mbs_len_l`
which most - if not all - other archive formats seem to take, which will
by default call `archive_wstring_append_from_mbs_in_codepage` if the
passed in `archive_string_conv` object is non-null
Fixes#1971
An `if` without curly braces guards the following statement.
When `#ifndef` removes that statement, the `if` now guards whatever
statement comes next.
I have not yet tested the changes.
Introduced in https://github.com/libarchive/libarchive/pull/1790.
CC people from original PR: @jvoisin @mmatuska
On Windows, `archive_write_finish_entry()` does not restore time
correctly.
It truncates the last digit of original FILETIME which is stored in the
archive.
This commit fixes this behavior.
See libarchive#2049 for detailed explanation on this issue.
Doc fix to clarify that `--mac-metadata` is only supported for certain
formats. Hopefully just a short-term fix until support is added for
other formats. (See discussion on #2041)
The libarchive test harness always extracts these to the truncated
source filename, ignoring the name in the `begin` line. However, for
ease of experimentation, people will want to manually extract these with
`uudecode`, so the `begin` lines should have the correct filename.
The letter b stands for "from (b)eginning" and specifies that a
substitute expression should be matched from the beginning of the
string, regardless if and where a previous substitute expression
matched.
Example:
Transform filename from B-A to A-B and remove all underscores.
Attempt without option b:
bsdtar -cft -s '/\(.*\)-\(.*\)/\2-\1/gp' -s "/_//g" ab_c-d_ef
ab_c-d_ef >> d_ef-ab_c
With option b:
bsdtar -cft -s '/\(.*\)-\(.*\)/\2-\1/gp' -s "/_//gb" ab_c-d_ef
ab_c-d_ef >> def-abc
libarchive relies on its host program to call setlocale() on platforms
with nl_langinfo() present in order to correctly detect the user
selected locale. This ensures that bsdunzip does so.
Earlier MR https://github.com/libarchive/libarchive/pull/2000 forgot to
annotate some functions as __LA_NORETURN. While fixing that I've noticed
that the bsdcat.h header could use some fixes so I've snuck those in.
Kind of make sense to group in one PR, but can split people prefer so.
/cc @AtariDreams fyi
---------
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Since Xcode 1.5, macos ld has a -dead_strip flag which is roughly
equivalent to --gc-sections in GNU style linkers, let's use it.
Reference:
https://opensource.apple.com/source/cctools/cctools-622.5.1/RelNotes/CompilerTools.html
Results from running the following command, before and after this
change, formatted for easy comparison (the actual values will vary with
toolchain and library versions used):
```
size bsdtar bsdcpio bsdunzip bsdcat .libs/libarchive.13.dylib
__TEXT __DATA __OBJC others dec hex
655360 16384 0 4295196672 4295868416 1000dc000 bsdtar # before
638976 16384 0 4295180288 4295835648 1000d4000 bsdtar # after
638976 32768 0 4295196672 4295868416 1000dc000 bsdcpio # before
606208 32768 0 4295163904 4295802880 1000cc000 bsdcpio # after
147456 16384 0 4295065600 4295229440 100040000 bsdunzip # before
114688 16384 0 4295032832 4295163904 100030000 bsdunzip # after
131072 16384 0 4295065600 4295213056 10003c000 bsdcat # before
49152 16384 0 4295016448 4295081984 10001c000 bsdcat # after
638976 16384 0 229376 884736 d8000 .libs/libarchive.13.dylib # before
622592 16384 0 229376 868352 d4000 .libs/libarchive.13.dylib # after
```
xz filter returns incorrect eof if the end of an lzip member is at the
end of the read buffer (`state->stream.next_in`).
At next call of `xz_filter_read()`, following lzip members are never
read since `state->eof` is still flagged.
Fixes#2026.
* Option count is used later for the position of the file in the
arguments and otherwise it uses -- as the file.
---------
Signed-off-by: Alfred Wingate <parona@protonmail.com>
For `matrix`-strategy builds, GitHub actions will terminate all related
matrix legs when one of them fails.
Disabling `fail-fast` knocks this behavior off.
There is a discrepancy between the w and non-w filename APIs, wherein a
consumer of libarchive can open a multi-volume set with names in the
current locale (on Windows) but not with UTF-16 names.
This patch addresses that issue.
archive_read_open_filename_w delegates its work to
archive_read_open_filenames_w.
Fixes#1728
Tested passing on Windows. In the meantime, I will also test on Linux.
I am hoping that the build agents can help me determine FreeBSD and
macOS coverage.
The libarchive source tree is littered with `__FBSDID("$FreeBSD.*")` and
'$FreeBSD$' tags left over from extracting it from FreeBSD's Subversion
repo. They never made sense for a git repo as git doesn't expand them
and FreeBSD has now removed `$FreeBSD$` from most local source files so
these stand out.
In addition to `__FBSDID` I've removed `__RCSID` which was used once for
a `$NetBSD$` expansion. There might be more of a case to be made for
preserving this one as a diff-reduction measure, but it seems mostly
pointless.
This builds and tests pass except for
libarchive_test_read_disk_directory_traversals which failed on master as
well.
actions/checkout from 4.0.0 to 4.1.0r
github/codeql-action from 2.21.7 to 2.22.1
ossf/scorecard-action from 2.2.0 to 2.3.0
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
the function is marked as warn-unused-result, so by default in a Debug
build with cmake, when Werror is set, this fails to detect. do the same
for autotools.
When an ISO9660 archive contains hard links or sym links, the order of
the files in the output of 'bsdtar -tf filename' is not the natural
order.
With an extension to the key (while still supporting ISO files up to 2^48
bytes) the sorting order is guaranteed for ISO files that contain linked
files for up to 2^16 files in total.
Co-authored-by: Martin Matuska <martin@matuska.de>
Fixes#1972
This adds the Scorecard GitHub Action to monitor the project's
supply-chain security posture.
Signed-off-by: Pedro Kaj Kjellerup Nacht <pnacht@google.com>
Fixes#1959.
This PR hash-pins GitHub Actions used in workflows and sets up
dependabot to keep the Actions up-to-date.
I've configured dependabot to group all Actions together. So if it ever
discovers that multiple Actions have new versions, it'll only send a
single PR to bump them all together.
---------
Signed-off-by: Pedro Kaj Kjellerup Nacht <pnacht@google.com>
Fixes#1957.
This PR ensures all workflows run with minimal permissions, instead of
with `write-all` permissions. This will protect the project from
supply-chain attacks.
The change to codeql.yml is for consistency and future-proofing. Should
another job eventually be added to the workflow, it will run with just
`contents: read`.
Signed-off-by: Pedro Kaj Kjellerup Nacht <pnacht@google.com>
7z archives created on windows 7zip can lack unix permission info. In
this case, we need to translate the windows permissions into reasonable
unix equivalents.
Security vulnerabilities:
1. Heap overflow in url_encode() in archive_write_set_format_pax.c
2. NULL dereference in archive_write_pax_header_xattrs()
3. Another NULL dereference in archive_write_pax_header_xattrs()
4. NULL dereference in archive_write_pax_header_xattr()
The vulnerabilities can be triggered when writing pax archives
with extended attributes (SCHILY or LIBARCHIVE) by feeding attribute
names longer than INT_MAX or attribute names that fail to be encoded
properly.
Reported-by: Bahaa Naamneh of Crosspoint Labs
Add missing mkdir calls to `cat/test/list.h` and `unzip/test/list.h`
invocations, making them consistent with the other rules. Otherwise,
the build fails when configured with `--disable-dependency-tracking`,
as configure does not create the directories automatically then.
7-Zip 23.00 added a new ARM64 filter, which is also supported by recent
versions of liblzma. This PR adds support for this filter for both lzma
and non-lzma encoders.
This change is similar to what was done in
5da00ad75b.
Co-authored by: Greg Lewis <glewis@FreeBSD.org>
Co-authored by: Piotr Kubaj <pkubaj@anongoth.pl>
Obtained from: FreeBSD ports (9fcaa71e4675d12dc3c9a5895edd68610e03cbcd)
Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
There's a race condition with the umask() execution in multi-threaded
use of the libarchive.
It's the users responsibility to mutex archive_write_disk_header()
call.
Signed-off-by: Peter Kaestle <peter.kaestle@nokia.com>
CreateFileW is not allowed in Universal Windows Platform (UWP) builds but
CreateFile2 is available. We could just enable this code for UWP but
it's probably to use it going forward so the code is properly tested on
both sides.
https://cmake.org/cmake/help/v3.0/variable/BUILD_SHARED_LIBS.html
Usually it's built-in but make sure we have it on by default.
We don't need to rename the windows static library when it's the only one built.
This will also allow proper usage of the pkg-config file in this case. Otherwise
there is no way to select the static library that way.
CHECK_C_SOURCE_RUNS generates a build error when cross-compiling.
ZLIB_WINAPI is the only one tested. When the run test fails, it's tested again
with just a compilation.
No need to load the kernel library manually. It's always available. It's not
possible to load it in Universal Windows Platform (UWP) builds anyway.
No need to load the kernel library manually. It's always available. It's not
possible to load it in Universal Windows Platform (UWP) builds anyway.
Note that test_read_format_7zip_lzma2_arm passes, while
test_read_format_7zip_zstd_arm fails, I believe because
liblzma implements these filters internally for itself, unlike the
other compression libraries.
These two archives contain a linux C hello world executable, built for
gnueabihf. They were created like so (the first one using a fork of
7-zip with zstandard support):
7z a -t7z -m0=zstd -mf=ARM libarchive/test/test_read_format_7zip_zstd_arm.7z hw-gnueabihf
and
7z a -t7z -m0=lzma2 -mf=ARM libarchive/test/test_read_format_7zip_lzma2_arm.7z hw-gnueabihf
Zstandard support is not yet available in 7-Zip, though it is planned
for a future release:
https://sourceforge.net/p/sevenzip/feature-requests/1580/
The compression ID used here (4F71101) is copied from a popular 7-Zip
fork, which added ZSTD support back in 2017, and is mentioned in the
upstream 7-Zip ticket linked above:
https://github.com/mcmilk/7-Zip-zstd.git
Note that this does not add write support for Zstandard compression
in 7z archives.
Relates to #1656.
There is a popular 7-Zip fork with zstandard support, with releases as
far back as 2017:
https://github.com/mcmilk/7-Zip-zstd.git
Zstandard support is not yet available in 7-Zip, though it is planned
for a future release:
https://sourceforge.net/p/sevenzip/feature-requests/1580/
This change adds a couple of tests for reading 7-Zip archives which
use zstandard compression. They are expected to fail until support is
added in the following commit.
Relates to #1656.
Add error handling to the USE_READDIR_R code paths that set name_max
from struct statfs or statvfs; if the determined name_max == 0
then return an error.
Avoids a crash in tree_dir_next_posix() when the calculation of
dirent_size from name_max is too small for the memory allocated
for struct dirent.
This may fix Github issue #1149
This may fix NetBSD PR https://gnats.netbsd.org/56080
If using HAVE_STATVFS, set name_max from svfs.f_namelen
instead of sfs.f_namelen, to be consistent with the
rest of the function.
Noticed by code inspection.
libssl isn't required for libarchive. Remove libssl so the installed .pc
file doesn't cause consumers to subsequently overlink libssl into their
code.
This doesn't affect how the code is linked -- only consumers of the
pkg-config specified spec.
WARNS=6 on FreeBSD passes several CFLAGS that causes the previous code
to fail with `-Wincompatible-pointer-types-discards-qualifiers` when
compiled with clang.
This particular change adjusts the code to be
`-Wincompatible-pointer-types-discards-qualifiers` clean. This change
changes the calls to use OSSL_PARAM macro abbreviated calls, instead of
calling more verbose (and less documented) callers.
While here, also address a `mac` object leak if `ctx` cannot be
allocated cleanly by always free'ing `mac` after it's been attached to
`ctx`.
Co-authored-by: Pierre Pronchery <pierre@freebsdfoundation.org>
Sponsored by: The FreeBSD Foundation
Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>
Clang 16 introduced a warning about single bit bitfields in structs,
which is triggered by a few libarchive formats:
libarchive/archive_write_set_format_7zip.c:1541:13: error: implicit
truncation from 'int' to a one-bit wide bit-field changes value from 1
to -1 [-Werror,-Wsingle-bit-bitfield-constant-conversion]
file->dir = 1;
^ ~
This is because single bit bitfields only support values -1 and 0, if
they are signed.
For bitfields with two or more bits this can be intentional, but single
bit bitfields are typically used as booleans, so it is better to make
them unsigned.
Without this patch, `bsdcpio_test_option_c` failed after 2038-01-19 with
```
.../libarchive-3.6.2/cpio/test/test_option_c.c:143: Assertion failed: t >= now - 2
.../libarchive-3.6.2/cpio/test/test_option_c.c:169: Assertion failed: t >= now - 2
.../libarchive-3.6.2/cpio/test/test_option_c.c:205: Assertion failed: t >= now - 2
```
Background:
As part of my work on reproducible builds for openSUSE, I check that
software still gives identical build results in the future.
The usual offset is +16 years, because that is how long I expect some
software will be used in some places.
This showed up failing tests in our package build.
See https://reproducible-builds.org/ for why this matters.
When the `zstd:frame-per-file` option is specified, the zstd filter will start a new frame when flushed, i.e. for each file in the archive.
The `zstd:min-frame-size=N` option modifies the `zstd:frame-per-file` option in that it will not start a new frame unless the current one exceeds `N` bytes.
When the `zstd:max-frame-size=N` option is specified, the zstd filter will start a new frame any time the compressed size of the previous one exceeds `N` bytes.
These options decrease compression efficiency by a varying amount (depending on the exact composition of its contents) but render the tarball seekable, to a certain extent.
* Add a flush() method to write filters.
* Add an __archive_write_filters_flush() function which iterates over filters in order and invokes the flush method for any filter that has one and that has written data.
* Call __archive_write_filters_flush() before writing a new header.
Otherwise there are 32/64-bit pointer conversions going on. In Windows
since MSVC2005, time_t has been 64-bit. MinGW needs a hack to get 64-bit
time_t.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Hi,
Thanks for writing and maintaining libarchive!
What do you think about this trivial change that does not add "iconv" to
the pkg-config file's list of required packages unless it is actually
needed? On at least Debian GNU/Linux systems, the iconv(3) function is
part of the system C library and there is no Debian package that
installs an iconv.pc file, so I had to make this change to the Debian
package of libarchive.
Thanks again, and keep up the great work!
G'luck,
Peter
meson's cmake config file handling does not support mixing ${} and @@ in
the same file. Switch to @@ for consistency.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
The warnings occurs at least with CMake version 3.24. It is caused by
CMake trying to find the gcc library due to case sensitivity.
The warning message was:
```
CMake Warning (dev) at /usr/share/cmake-3.24/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
The package name passed to `find_package_handle_standard_args` (LIBGCC)
does not match the name of the calling package (LibGCC). This can lead to
problems in calling code that expects `find_package` result variables
(e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
build/cmake/FindLibGCC.cmake:17 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
CMakeLists.txt:1269 (FIND_PACKAGE)
This warning is for project developers. Use -Wno-dev to suppress it.
```
DONT_FAIL_ON_CRC_ERROR is already used in the rar5 support code, so let's make
use of it in another places as well. This should tremendously help with
fuzzing.
Commit 45c5008c4 requires all characters in an mtree file to be
printable, as determined by `isprint()`. This broke support for
reading mtree files with tab characters, which are valid and otherwise
supported by libarchive. Allow them. Add a unit test.
Fixes#1782
As libzstd typically comes with a pkgconfig file, use it.
Fixes compilation at least on OpenWrt where host paths were leaking into
the build.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
These headers are in conflict and only one is needed by
archive_read_disk_posix.c therefore include linux/fs.h if it exists
otherwise include sys/mount.h
It also helps compiling with glibc 2.36
where sys/mount.h conflicts with linux/mount.h see [1]
[1] https://sourceware.org/glibc/wiki/Release/2.36
The `size` attribute may contain a negative or too large value. Check
the range of the `entry_bytes_remaining` in `pax_attribute` the same way
as `header_common`. The test which is added passes both with and without
this change in a normal debug build. It is necessary to run with
`-fsanitize=undefined` to see that the undefined behavior is avoided.
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=48467
archive_read_data_into_fd passes a pointer to an uninitialized
variable as an output 'offset' argument into archive_read_data_block
function, and expects that this variable will always be initialized
inside of it.
Like this:
size_t size;
int64_t offset;
archive_read_data_block(a, &buf, &size, &offset);
/* some work with offset here */
But rar5 implementation of archive_read_data_block function leaves the
'offset' argument uninitialized in one code path (if file is
compressed and there are no uncompressed pending data blocks).
As a result, archive_read_data_info_fd function is using an
uninitialized variable as an initial offset of an output file. And in
most cases it causes an appending sparse block of a random size at the
beginning of the output file.
Also changed a few vsprintf to vsnprintf.
Most cases were trivial, one private function was changed to take the buffer length, one case required some fancy arithmetic.
Trying to use SHA256, SHA384 or SHA512 with mtree when linked against
Windows Crypto would result in silent failure. The call to
`CryptCreateHash` would fail with 0x80090008. The docs[1] say that
these algorithms require a different crypto provider, so let's make
that a parameter for `win_crypto_init` and choose at the call site along
with the algorithm.
[1] https://docs.microsoft.com/en-us/windows/win32/seccrypto/alg-id
Signed-off-by: David Macek <david.macek.0@gmail.com>
This adds Requires.private field in pkgconfig file. Using that field,
pkgconfig pulls all the private cflags or libs while static linking.
OpenSSL static libraries require some windows system libraies. Otherwise
static liking fails with libarchive.
The function returns 0 on error, which is seemingly very common with
OpenSSL 3.0 and rmd160.
Just error check the lot, so we don't get even more random failures with
future releases of OpenSSL.
Fixes#1549
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Rework function expand() to process integer passed by reference
and return an archive error code.
Fixes: 01a2d329df (support rar filters)
Reported-by: OSS-Fuzz #44547
Move the inclusion added by commit 90978db1 (windows: make sure we use
the right calling convention for libc, 2021-10-13, v3.6.0~39^2~1) to be
first. This is our convention in all other `.c` sources. It ensures
that our configured `_WIN32_WINNT` value is defined before including any
system headers.
Some specially crafted files declare invalid extraction parameters that
can confuse the RAR5 reader.
One of the arguments is the declared window size parameter that the
archive file can declare for each file stored in the archive. Some
crafted files declare window size equal to 0, which is clearly wrong.
This commit adds additional safety checks decreasing the tolerance of
the RAR5 format.
This commit also contains OSSFuzz sample #30459.
RAR5 reader uses several variables to manage the window buffer during
extraction: the buffer itself (`window_buf`), the current size of the
window buffer (`window_size`), and a helper variable (`window_mask`)
that is used to constrain read and write offsets to the window buffer.
Some specially crafted files can force the unpacker to update the
`window_mask` variable to a value that is out of sync with current
buffer size. If the `window_mask` will be bigger than the actual buffer
size, then an invalid access operation can happen (SIGSEGV).
This commit ensures that if the `window_size` and `window_mask` will be
changed, the window buffer will be reallocated to the proper size, so no
invalid memory operation should be possible.
This commit contains a test file from OSSFuzz #30442.
A few guiding principles:
* Each test source file includes ONLY "test.h" to make it easy
to create new tests.
* Each test suite has a "test.h" that includes "test_util/test_common.h"
to get access to all the common testing utility functions.
So "test_common.h" is then responsible for including
any smaller headers that declare specific pieces of
shared test functionality.
I've also pulled some test filtering logic that was _only_ used
in test_main.c into that file, and repurposed "test_utils.[ch]"
for common utility code. (Eventually, a lot of the assertion
helpers currently in "test_main.c" should probably be organized
into one or more source files of their own.)
Many tests use a loop calling rand() to fill buffers with test data. As
these calls cannot be inlined, this adds up to noticeable overhead:
For example, running on QEMU RISC-V the test_write_format_7zip_large_copy
test took ~22 seconds before and with this change it's ~17 seconds.
This change uses a simpler xorshift64 random number generator that can be
inlined into the loop filling the data buffer. By default the seed for this
RNG is rand(), but it can be overwritten by setting the TEST_RANDOM_SEED
environment variable.
For a native build the difference is much less noticeable, but it's still
measurable: test_write_format_7zip_large_copy takes 314.9 ms ± 3.9 ms
before and 227.8 ms ± 5.8 ms after (i.e. 38% faster for that test).
Added more range checks in the bit stream reading functions
(read_bits_16 and read_bits_32) in order to better guard against out of
memory reads.
This commit contains a test with OSSFuzz sample #30448.
ZSTD library can be compiled with the compressor disabled, which is
handy on space restricted systems as the compressor accounts for more
than two thirds of the library size.
Detect this case and use libzstd for the decompression only.
Compression will be done using zstd binary if it's available.
The lzip documentation specifies that the logarithm of the dictionary
base size may be in the range 12-29, and the lzip utility is quite
capable of creating an archive with a dictionary larger than 128M if
passed the appropriate -s command-line option.
If time_t is a signed 32-bit integer, then we cannot represent times
after 03:14:07 on 2038-01-19. Indicating an error if (Year > 2038) is
not sufficient; for safety, we need to bail if (Year >= 2038).
As the comment above this line notes, it would be better to check if
time_t is 32 bits first. And even if we didn't check for that, we could
use a much more complicated check:
((Year > 2038) || ((Year == 2038) && (Month > 1)) ||
((Year == 2038) && (Month == 1) && (Day >= 19)))
But I don't think the additional complexity is worth it; rejecting any
dates in 2038 eliminates fewer than 3 weeks of potentially-valid dates.
Reported by: Rasmus Villemoes
In b6b423f0 a fallback in tar to stdio was implemented. However, the check
for the tape device didn't interpret the correct value returned from
access(). Fix the check to implement the fallback to stdio properly.
Signed-off-by: Walter Lozano <walter.lozano@collabora.com>
Have `bsdtar` default to stdout if this system has no tape device. This uses an `access()` check to see if the default tape device (e.g., `/dev/tape` on FreeBSD) exists and will use stdout as the default if it doesn't exist. If the system does have a tape device, there is no change to the existing behavior.
For libarchive 4.0, we'll change the default behavior of `bsdtar`:
* The `TAPE` environment variable will still be honored at runtime
* The `_PATH_DEFTAPE` preprocessor macro will still be honored at build time
* But `_PATH_DEFTAPE` will no longer be set by libarchive's default build, with the effect that for most people, bsdtar will default to stdout if there is no `-f` option provided.
Since current tar defaults to tape devices that are rare nowadays add an
additional step to fallback to "-" if tape devices are not found.
This is a clean way to have a default to "-" on those systems that tape
devices are not present while keeping the current behavior for other
cases. Additionally prepare for future releases where this kind of defaults
will be dropped.
Signed-off-by: Walter Lozano <walter.lozano@collabora.com>
Analogue to the parent cmake commit, with linker flag detection.
The former two split the functions and data into separate sections
within the object file. Which makes it easier for the latter to properly
garbage collect and discard unused sections.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
The former two split the functions and data into separate sections
within the object file. Which makes it easier for the latter to properly
garbage collect and discard unused sections. For example
text data bss dec hex filename
208268 2056 4424 214748 346dc bsdcat -- before
93396 1304 4360 99060 182f4 bsdcat -- after
1059167 12112 24176 1095455 10b71f bsdcpio -- before
1002538 7320 23984 1033842 fc672 bsdcpio -- after
1093676 14248 6608 1114532 1101a4 bsdtar -- before
1062231 14176 6416 1082823 1085c7 bsdtar -- after
1097259 15032 6408 1118699 1111eb libarchive.so.18 -- before
1095675 14992 6216 1116883 110ad3 libarchive.so.18 -- after
Note:
This is enabled only with gcc/clang on non-Mac platforms. Ideally we'll
have a compile-time check, albeit that seems impossible with our ancient
cmake requirement.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
It's fairly common for people to use caf and xaf on Linux. The former in
itself being GNU tar specific - libarchive tar does not allow xa.
While it makes little sense to use xaf with libarchive tar, that is
implementation detail which gets in the way when trying to write trivial
tooling/scripts.
For the sake of compatibility, reduce the error to a warning and augment
the message itself. Making it clear that the option makes little sense.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Prior to version 3.3 cmake would always use -rdynamic. That in itself
causes all the internal symbols to be exported, increasing the binaries
by 5-10% and making it impossible for the compiler to reason, optimise
and discard unused code.
The -rdynamic is useful in two cases:
- having a third party module (say /usr/lib/foo/foobar.so) which is
underlinked and depends on symbols from the main binary - apps like
irssi, bash and zsh use that
- uses the glibc backtrace, which relies on dlopen/dlsym to fetch the
symbol data. Unwind is much better solution, since it replies on the
DWARF data
Our binaries do not use either of these - so drop the -rdynamic. The
autotools build doesn't use it either.
text data bss dec hex filename
229000 2120 4424 235544 39818 bsdcat -- before
208324 2120 4424 214868 34754 bsdcat -- after
1093939 12128 24176 1130243 113f03 bsdcpio -- before
1059181 12128 24176 1095485 10b73d bsdcpio -- after
1130091 14264 6608 1150963 118ff3 bsdtar -- before
1093690 14264 6608 1114562 1101c2 bsdtar -- after
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
The flags used across the two are identical, apart from -g.
There is no compelling reason, why we would omit -g for debug builds
with GCC, while using it with clang.
De-duplicate the sections.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
verify_sparse_file() assumes that every hole will be fully contained in only
one archive_read_data_block(). This is a reasonable assumption if the file is
indeed sparsely encoded in the archive because archive_read_data_block() will
just skip the hole and return the offset of the next data block.
However, if the file is not sparsely encoded in the archive, a hole consists of
a lot of zeroes that need to be read byte by byte. In this case, the archive
contains no information on where this block of zeroes ends and where actual
data begins. Therefore it can happen that a single archive_read_data_block()
contains both zeroes from a hole and actual data.
If this happens, assert(sparse->type == HOLE) fails. This assertion is
reasonable for sparsely encoded files because archive_read_data_block() will
never only read part of a hole (since it does not really "read" a hole at all,
it just returns a higher offset accounting for the size of the hole).
However, we want to start testing files with verify_sparse_file() that are
explicitly not sparsely encoded. In this case, the assertion does not
necessarily hold any more. Therefore we need to account for the case where the
overlapping block consists of data. To make sure the file contents are
correctly encoded in the archive, we need to test the contents of the data
block, like it is already done for blocks completely contained in the data read
by archive_read_data_block().
Note that this modification does not change the way sparsely encoded files are
verified, it just relaxes an edge case that cannot happen with sparsely encoded
files to make it possible to test any kind of file, whether sparsely encoded or
not.
Immediately after EVP_CIPHER_CTX_new() neither EVP_CIPHER_CTX_init()
nor EVP_CIHPER_CTX_reset() should be called: the purpose of the init
function is to initialize a context on the stack while reset clears
a used context for reuse. Neither situation is the case here.
Removing the code also fixes a potential NULL dereference because an
error of reset is not signaled to the caller. Fortunately reset doesn't
currently fail in this situation in current OpenSSL and LibreSSL.
The fixup list was erroneously assumed to be directories only.
Only in the case of critical file flags modification (e.g. SF_IMMUTABLE
on BSD systems), other file types (e.g. regular files or symbolic links)
may be added to the fixup list. We still need to verify that we are writing
to the correct file type, so compare the archive entry file type with
the file type of the file to be modified.
Fixes#1617
Sparse file information depends on the file system and can therefore be a
source of unreproducibility for the generated archives, e.g. if the same
content is compressed on a file system with and without sparse file support.
Add an option to suppress reading this information from disk entirely.
As before - move the dispatch/function pointer so a const data segment.
Separating it from the RW data section.
Note: the close function is _not_ optional - remove the NULL check.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Analogous to the earlier "can_seek" change. Drop the function pointer
in favour of a flag. Over the years, with over a dozen filters, no
filters actually implemented it.
If at a point in the future that changes, one can reinstate it.
Alternatively one could use a ARCHIVE_FILTER_NONE check.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Currently we the ::seek callback can be either NULL or client_seek_proxy.
Since we already explicitly use the latter, instead of ::seek just drop
the function pointer all together, in favour of a simple flag.
Alternatively one could use a ARCHIVE_FILTER_NONE check.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
There's a notable duplication across all the read bidder code.
Check the archive magic/state, set the bidder private data (to NULL in
all but one case), name and vtable.
Change the helper to do the actual registration, keeping things simpler
in the dozen+ filters. This also allows us to enforce the bidder ::bid
and ::init dispatch are non NULL. The final one ::free is optional.
NOTE: some of the bidders do _not_ set a name, which I suspect is a
pre-existing bug. I've left them as-is, but we might want to fix and
enforce that somehow.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
From over a dozen filters, only one instance has the vfunc... Always
returning ARCHIVE_OK.
Let's just remove the return type.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
There is no point in having stubs around. The caller already checks if
the function pointer is NULL before calling it.
With this in place only one implementation remains in filter_program.
Technically we can rework/remove that as well, although the solution
might not be so clean, so let's keep that for another day.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
None of the filters set ::options and based on git log, none of them
ever did despite that the API/infra was added over 10 years ago.
Let's just drop it, removing the majority of archive_set_filter_option()
which is essentially dead code.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Currently we have a constant dispatch/vtable that we populate at runtime
for unknown reason.
Remove the functions doing all the work and use a simple static const
struct archive_vtable.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Add a simple top-level .editorconfig file to manage common attributes
such as indentation style, trailing whitespace and newline at end of
file. The format is wide spread and has support for nearly every editor
out there - see https://editorconfig.org/ for more.
Majority of the project is C - which uses tabs, although there are some
CMake files using 2 space indent and shell scripts - predominantly using
4 space indent.
This makes it harder for casual contributors to butcher things :-)
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Remove the excessive fallthrough chain in parse_keyword(). Even though
it is in the else/error path, there is no point in comparing the key
another dozen (or more) times when we know it will fail.
Just use an early return (OK) or break respectively.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
In old tar formats directories are specified by appending a slash to
their names.
Directories may have sizes, but they do not have data blocks. Keep this
in mind and avoid skipping data blocks when encountering directories.
The size_t data type on 32 bit systems is smaller than int64_t. Check
the int64_t value before casting to size_t. If the value is too large
then stop operation instead of continuing operation with truncated
value.
The previous fix in b41daecb5 was incomplete. Fixup entries are
given the original path without calling cleanup_pathname().
To make sure we don't follow a symlink, we must strip trailing
slashes from the path.
The fixup entries are always directories. Make sure we try to modify
only directories by providing O_DIRECTORY to open() (if supported)
and if it fails to check directory via lstat().
Fixes#1566
When opening a file descriptor to set file flags on linux, ensure
no symbolic links are followed. This fixes the case when an archive
contains a directory entry followed by a symlink entry with the same
path. The fixup code would modify file flags of the symlink target.
On Linux ACLs on symbolic links are not supported.
We must avoid calling acl_set_file() on symbolic links as their
targets are modified instead.
While here, do not try to set default ACLs on non-directories.
Fixes#1565
Rename the flag "extracting_hardlink" to "checking_linkname" to
be more accurate about its use. If the variable is non-zero it
means that check_symlinks_fsobj() is called on the linkname
when a hardlink is going to be created.
On platforms that support the linkat(2) function we can safely
write hardlinks to symlinks as linkat(2) does not follow symlinks
by default.
Fixes#1044
GNU.sparse.numbytes and GNU.sparse.offset both have to be set before
gnu_add_sparse_entry can be called.
The GNU.sparse.numbytes parser checks for tar->sparse_numbytes.
This has to be tar->sparse_offset instead to work just like the
GNU.sparse.offset parser.
The variable last_digit_limit is negative since INT64_MIN itself is
negative as well. This means that the last digit after "limit" always
leads to maxval.
Turning last_digit_limit positive in itself is not sufficient because
it would lead to a signed integer overflow during shift operation.
If limit is reached and the last digit is last_digit_limit, the number
is at least maxval. The already existing if condition for even larger
(or smaller) values can be reused to prevent the last shift.
In my humble opinion it might make sense to reduce duplicated code and
keep it separated in a utility source file for shared use.
The archive_write_callback passed to archive_write_open() is documented as:
"each call to the write callback function should translate to a single write(2) system call.
On success, the write callback should return the number of bytes actually written"
And in most places, the code repeatedly calls the write callback, but when flushing
the buffer at close, the write callback was called once, assuming it would write everything.
This could result in a truncated archive.
A test is added to test short writes in different code paths.
Recent versions of GNU libc and GCC produce a warning on getcwd(0, PATH_MAX):
test_main.c: In function ‘get_refdir’:
test_main.c:3684:8: error: argument 1 is null but the corresponding size argument 2 value is 4096 [-Werror=nonnull]
3684 | pwd = getcwd(NULL, PATH_MAX);/* Solaris getcwd needs the size. */
This is because getcwd() is marked with the 'write_only (1, 2)' attribute.
Using the alternate getcwd(NULL, 0) path which is supported by GNU libc avoids this.
The available size returned from `__archive_read_ahead` can be larger
then required size. Substract by available size may underflow `skip`,
which will reach EOF too soon.
Rather than using the central directory offset as the file position directly, instead use the known location of the end-of-central-directory record and the size of the central directory to compute an offset. This allows us to efficiently handle Zip archives that have other data prepended to the file.
Use the size of the Central Directory and the offset of the EOCD to
calculate the real position.
This trick doesn't work for Zip64 as easily as we are not scanning
backwards to find the PK\x06\x06 entry.
Interestingly, it is never checked so it could be trying to parse
bad files.
Updated based on review
When the FreeBSD testsuite runs the libarchive tests it checks that stderr
is empty. Since #1382 this is no longer the case. This change restores
the behaviour of silencing bunzip2 stderr but doesn't bring back the
output text check.
Partially reverts 2e7aa5d9
Some of the test_write_format_7zip_large take longer than 10 minutes to
run on QEMU-RISC-V and therefore time out when run as part of the FreeBSD
test suite. This is even more noticeable when running on a CHERI-enabled
QEMU since the emulation of tagged memory makes it run slower.
On my local machine the impact is not as extreme, but
test_write_format_7zip_large_lzma1 takes 375ms when TEST_SLOW_HOST is set
and 8.2 seconds normally (i.e. 22 times slower).
The other alternative would be to skip these tests when not running on
real hardware, but it seems to me that compressing a smaller amount of
random data is preferable.
Adds a null check for the string parameter in archive_pathmatch
and archive_pathmatch_w to prevent a null pointer dereference.
Reported in github issue #1483.
Build fails on compiling xxhash.c having a fuction with "inline" specifier.
"inline" is a c99 keyword and c99 is not yet (fully) supported with MSVC toolset v120:
"The inline keyword is available only in C++. The __inline and __forceinline
keywords are available in both C and C++. For compatibility with previous versions,
_inline is a synonym for __inline."
(Source: http://msdn.microsoft.com/en-us/library/z8y1yy88.aspx)
This fix adds a patch that replaces "inline" with "__inline" in xxhash.c
Otherwise, the preprocessor is not detected and it leads to failure with
empty $CPP variable being used on la_TYPE_UID_T, leading to a cascade of
detection errors and redefinition of some types.
Creating a pointer by adding an offset to 0x1 is undefined behaviour and
results in an invalid pointer when running on CHERI systems. Use a
standards-compliant cast via uintptr_t instead.
This was found due to a crash while running the libarchive test suite on a
CHERI-RISC-V system.
The archive_write_open() function does not provide a free callback.
Freeing was done by the close callback. When the open callback fails,
the client filter is left in ARCHIVE_WRITE_FILTER_STATE_FATAL,
the close callback is not called and unfreed resources may be left behind.
Fixes#1456
UFS stores NFSv4 ACLs in the system extended attribute namespace. Per
libarchive maintainer, ignore those attributes.
Signed-off-by: Shawn Webb <shawn.webb@hardenedbsd.org>
In order to teach the packaging infrastructure how to support
HardenedBSD's method of exploit mitigation toggling, teach libarchive
how to handle the system filesystem extended attribute namespace.
Signed-off-by: Shawn Webb <shawn.webb@hardenedbsd.org>
The function I added only applies to FreeBSD. As such, if the function
declaration isn't ifdef'd out for other architectures, continuous
integration (CI) builds fail. Mitigate the failure by guarding the
function declaration with the proper preprocessor macro conditional.
Signed-off-by: Shawn Webb <shawn.webb@hardenedbsd.org>
In order to teach HardenedBSD's packaging infrastructure how to support
HardenedBSD's method of exploit mitigation toggling, teach libarchive
how to handle the system filesystem extended attribute namespace.
Signed-off-by: Shawn Webb <shawn.webb@hardenedbsd.org>
On test clean-up an 'rm -rf' will fail because 'lock' and 'lock/lock2'
directories have 0311 permissions. Change these permissions on exit so
that the directories can be deleted on success.
I was investigating libarchive test failures on FreeBSD and it turns out
we get a reproducible SIGBUS for test_archive_m5, etc. Debugging this shows
that libarchive and the tests disagree when it comes to the definition of
archive_md5_ctx: libarchive assumes it's the OpenSSL type whereas the test
use the libmd type. The latter is not necessarily aligned enough to store
a pointer (16 bytes for CHERI RISC-V), so we were crashing when storing
EVP_MD_CTX* to an 8-byte-aligned archive_md5_ctx.
To avoid problems like this in the future, add a new macro to the config
header and use that to ensure that all configuration check macros have
been defined in archive_digest_private.h.
This makes the function safer on linux since the file is warrantied to never
be visible on the filesystem, cannot be linked anywhere unless O_EXCL is
not specified and it is lost forever on any kind of program termination.
The error message is consistent with `archive_read_set_format`.
The absense of an error message here also means that the error message
in `archive_read_set_format` is actually never used.
Writer functions does not seem to have the same issue.
In VisualStudio 2019 error C4061 happens if you don't
have a case statement for every value of an enum.
The default condition is not enough.
This fix avoids error like this:
25>C:\sq5\LibArchive\libarchive\archive_read_support_format_warc.c(344,2): error C4061: enumerator 'WT_NONE' in switch of enum 'warc_type_t' is not explicitly handled by a case label [C:\sq5\LibArchive-build\libarchive\archive.vcxproj]
All the conversions already always go through the MBS form so
simply always convert to MBS before trying to convert to any other encoding
in the `archive_mstring_get_*` functions.
* Do this by calling `archive_mstring_get_mbs` to avoid duplicating code
* Add `struct archive*` parameter to `archive_mstring_get_mbs_l`
* Add test for set/get/update utf8 functions on entry
* Add test for encoding conversion.
Fix#931
Currently, the rdev field is set from archive_entry_dev, which is
the device number of the filesystem containing the device node, not
the device itself. It should instead use archive_entry_rdev.
Here's an example demonstrating the problem:
$ ls -l /dev/null
crw-rw-rw- 1 root root 1, 3 May 26 02:03 /dev/null
$ bsdtar --format=cpio -cf null.cpio /dev/null
bsdtar: Removing leading '/' from member names
$ bsdtar -tvf null.cpio
crw-rw-rw- 1 0 0 0,6 May 26 02:03 dev/null
some older and more obscure compilers (e.g. mipspro on irix) default to the strictest standard they know instead of the most permissive. since libarchive uses c99 features in several places throughout, but does not pass appropriate cflags to parse c99 code, it will fail to compile from the configure script on these systems. this change will have autoconf check if any flags are needed for the c compiler to accept c99 code, and add them to cflags.
Do not assert on specific output of bunzip2. This wrongly assumes
that all systems will be using the reference implementation of bzip2,
and it fails when an implementation with another output is used
(e.g. lbzip2).
Fixes#1380
There is a bug in the linked list implementation for passphrases. The insert to
head function does not account for the tail==head case and causes a leak. The
first entry into the list is lost when the second entry is added. The second
and beyond entries are are released properly, but the first is lost entirely.
The call to CloseHandle() is asymmetric and wrong.
The handle is created during the __achive_create_child() call in
__archive_write_program_open() aka the filter open callback.
While CloseHandle() is already called in the filter close callback aka
__archive_write_program_close().
Remove the incorrect call to CloseHandle() from the bidder free callback
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Move the Windows specific to the Windows version of the helper, making
the existing code shorter and simpler.
This means that the child "handle" type will vary across the two
platforms - something that we've already been doing with waitpid.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Change the function to return an int - ARCHIVE_OK or ARCHIVE_FAILED,
taking the child as an output argument.
This will allow us to simplify the existing code and have move platform
specifics in the platform files - posix and windows
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Follow the approach set by the crypto and use the exact same name for
the respective functions. This is perfectly fine, since only one
provider can be used (built) at given time.
As a result, we can kill off the excessively long ifdef chain, making
the more more manageable and shorter.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
This is the define set in archive_entry.h to use the right mode_t type per
platform. "mode_t" may not be available at all.
This is also how these functions are declared in archive_entry.h
To be used by the mtree reader, to populate the internal digest storage.
The function is deliberately not part of the user interface, since it's
interactions are not defined yet.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Allow applications to query the digests stored in the MTREE. Which they
can use at a later stage to perform integrity checks as needed.
Note: no digests are wired yet.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
This commits introduces the basic internal storage for digests. Those
will be used to expose a set/get API, which users can utilise for
integrity checking.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Factor our the md5/rmd160... digests struct, moving it to the private
header. This way we can reuse it within the mtree read code.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Print message if creation of temporary file fails.
When rename of temporary file fails, exit with ARCHIVE_FAILED.
Reword message on failed rename.
Revert d84ec3f and 85b9f66, call archive_set_error()w before unlinking.
The description of how hard links are handled in the 'new ASCII format' is wrong. Cpio stores the *first*, not *last* copy of the file with its entire contents, followed by entries with filesize set to zero for each subsequent entry that is hard linked. While creating the archive, how would it ever know it was processing the last one!
Fix `archive_string_append_from_wcs_in_codepage` to account for the
already-used portion of the buffer when computing the size of the
remaining buffer for `WideCharToMultiByte` output.
One of the fields in RAR5's base block structure is the size of the
header. Some invalid files declare a 0 header size setting, which can
confuse the unpacker. Minimum header size for RAR5 base blocks is 7
bytes (4 bytes for CRC, and 3 bytes for the rest), so block size of 0
bytes should be rejected at header parsing stage.
The fix adds an error condition if header size of 0 bytes is detected.
In this case, the unpacker will not attempt to unpack the file, as the
header is corrupted.
The commit also adds OSSFuzz #20459 sample to test further regressions
in this area.
The list of format and filter options in bsdtar(1) is incomplete.
Add reference to manual pages archive_write_set_options(3) and
archive_read_set_options(3) for a complete list of supported options.
Fixes#1323
In `archive_pack_dev.c` there is code checking the `HAVE_MAJOR` macro,
but it is not computed. Port the equivalent logic from
`archive_entry.c` to define the macro.
As Windows does not support atomic rename/_wrename
we have to unlink the file right before calling _wrename
leaving a short unsafe window where a different file with the
same name can be created and make the rename operation fail.
This flag changes the way that regular files are extracted:
Instead of removing existing files first and re-creating them in
order to replace their contents, a temporary file is created and
when writing to the temporary file is completed, the file is
rename(2)d to the final destination name.
This has the effect of presenting a consistent view of the file to
the system (either the file with the new contents or the file with
the old contents). Removing and overwriting the file has the
undesired side effect that the the system can either not see the
file at all (from the time it is being removed till the time it is
being re-created), or worse it can see partial file contents. This
is problematic when extracting system files (for example shared
libraries).
If the existing file that is going to be overwritten is a hard link,
for now we unlink it before calling rename(2). This can be done
correctly by creating a hardlink to a tmpnam(3) generated file
and then use rename(2), but that is fairly intrusive and requires
refactoring.
Fixes#1289
This has no but code correctness effect as cdeque_init() is static
and max_capacity_power_of_2 in our code is always 8192.
Found by LGTM.com code analysis
archive_read_support_format_rar5.c:
Bitfield int -> signed int
Archive_write_set_format_iso9660.c:
Bitfield int -> signed int
archive_write_set_format_xar.c:
Bitfield int -> signed int
archive_write_set_format_7zip.c:
Bitfield int -> signed int
archive_read_support_format_xar.c
Remove useless comparsion
archive_read_support_format_rar.c:
Fix invalid nested loop break.
Comment out dead code sections.
Simplify size comparsions of lensymbol and offsymbol.
archive_read_support_filter_uu.c:
Remove useless comparsions
archive_read_disk_posix.c:
Remove useless do-while-zero
Found by LGTM.com code analysis
When we grow the archive_string buffer, we have to make sure it fits
at least one maximum-sized multibyte character in the current locale
and the null character.
Fixes#1298
It appears that ZIPX files with type 14 stream ('lzma alone') can
contain LZMA_STREAM_END markers at the end of the stream. The ZIP reader
was displaying an "unknown error 1" status after encountering such
marker.
The fix handles such case, and the reader doesn't return error status
anymore. Thus it should be possible to unpack files that contain the
LZMA_STREAM_END marker at the end of the stream.
Fixes#1257
RAR5 archives can contain files that span across multiple .rar files. If the
archive contains a big file that doesn't fit to first .rar file, then this file
is continued in another .rar file.
In this case, the RAR compressor first emits the FILE base block for this big
file in the first .rar file. Then, it finishes first .rar file, and creates the
new .rar file. In this new file, it emits the continuation FILE block that
marks start of the continuation data for the rest of the huge file.
The problem was that the RAR5 reader didn't ignore the window size declaration
when parsing through the continuation FILE base block. The malicious file could
declare a different window size inside the continuation base block than was
declared in the primary FILE base block in the previous volume. The window size
from continuation block was applied, but the actual window buffer was not
reallocated. This resulted in a potential SIGSEGV error, since bounary checks
for accessing the window buffer were working incorrectly (the window size
variable didn't match the actual window buffer size).
The commit fixes the issue by ignoring the window size declaration in the
continuation FILE base block when switching volumes.
The commit also contains a test case and OSSFuzz sample #19509.
This allows us to control whether "SCHILY.xattr", "LIBARCHIVE.xattr" or
both headers (default) are written when storing extended attributes.
Document "hdrcharset" option for pax.
This ensures that filters may be opened and closed only once and
__archive_write_filter() may be called only on an open filter.
Refactor filter open code and move logic to archive_write.c
Fixes#351
This provides a defense-in-depth against programming errors due to the
partial state. Based on a report from Airbus Security - Vulnerability
Management.
The cal to mbrtowc() or mbtowc() should read up to mbs_length
bytes and not wcs_length. This avoids out-of-bounds reads.
mbrtowc() and mbtowc() return (size_t)-1 wit errno EILSEQ when
they encounter an invalid multibyte character and (size_t)-2 when
they they encounter an incomplete multibyte character. As we return
failure and all our callers error out it makes no sense to continue
parsing mbs.
As we allocate `len` wchars at the beginning and each wchar has
at least one byte, there will never be need to grow the buffer,
so the code can be left out. On the other hand, we are always
allocatng more memory than we need.
As long as wcs_length == mbs_length == len we can omit wcs_length.
We keep the old code commented if we decide to save memory and
use autoexpanding wcs_length in the future.
Fixes#1276
In the broken code, wchar_t was incorrectly assumed to be 16-bit.
This was not a portable way.
Fixed code uses archive_string and codepage information, instead of using archive_wstring.
The lastest lha format supports UNICODE filenames on its content, using extended headers(EXT_UTF16_FILENAME and EXT_UTF16_DIRECTORY).
However, currently libarchive ignores them.
This modification is to handle these extensions.
On architectures where ssize_t is 32 bits but file offsets are 64 bits
(such as 32-bit Linux with _FILE_OFFSET_BITS=64), the POSIX disk reader
would incorrectly skip large sparse regions due to a 32-bit integer
overflow in _archive_read_data_block(). This can result in the reader
failing with "Encountered out-of-order sparse blocks", since the
overflowed value is interpreted as a signed number and added to the
current offset.
The bytes variable was used to store the difference between two 64-bit
integers, but bytes is a ssize_t. Since this value of bytes was not
used after the block handling sparse offsets (it is always overwritten
in the block below), replace it with an int64_t sparse_bytes variable
that can always represent the difference without truncation.
Signed-off-by: Daniel Verkamp <dverkamp@chromium.org>
This can result in a heap overflow dependening on the buffer allocation.
Simplify the path length calculation and avoid binary operands.
Based on PR #1255 from Will Wagner
RAR5 archives can contain files compressed independently of each other,
and files that share a common window buffer, so files which are
compressed using 'solid' method. In the latter case, all files
are required to use the same window buffer, so window size should also
be the same.
OSSFuzz sample #15482 declares a different window size for multiple
solid files. RAR5 reader doesn't reallocate window buffer when
decompressing solid files, so it was possible to perform an
out-of-bounds read by declaring two solid files, where the second solid
file declared the window size parameter that was bigger than window size
used in first solid file.
This commit introduces additional checks to ensure all solid files are
using the same window size.
The commit also adds a test case using OSSFuzz sample #15482 to hunt
down regressions in the future.
Some other test cases had to be adjusted as well, because other OSSFuzz
samples were also declaring different window sizes for solid files. So
this commit has changed the error reporting for those invalid sample files.
Found with mandoc -Tlint; fixing the following messages:
WARNING: bad NAME section content
WARNING: missing comma before name
WARNING: new sentence, new line
WARNING: parenthesis in function name
WARNING: skipping no-space macro
WARNING: skipping paragraph macro
WARNING: unusual Xr order
WARNING: unusual Xr punctuation
STYLE: no blank before trailing delimiter
STYLE: possible typo in section name
STYLE: trailing delimiter
STYLE: whitespace at end of input line
For the meaning of the messages, see:
https://man.openbsd.org/mandoc#DIAGNOSTICS
Raise compression on the second test to level=20, and perform a
third at level=1. Expect the output archive sizes to line up
based on compression level. Reduces test susceptibility to small
output size variations from different libzstd releases.
We need to clean old entry data in archive_read_next_header2 in Windows
and Posix disk readers to ensure consistent results. One possible
failure mode: sparse data from the previous entry is carried over to
next non-sparse file entry, causing it to be mishandled.
If a file ends with a sparse "hole" that is larger than buffer supplied
to archive_read(), then archive_read() will return prematurely because
archive_read_data_block() will return ARHCIVE_EOF as there is no more
"real" data. We can fix that by not trying to refill data buffer until
we exhaust the hole range.
Fixes libarchive#1194
RAR5 uses filters in order to mutate data just before compression, to
achieve a better compression ratio. After decompression, this mutation
needs to be reversed by processing various filters that the compressor
uses.
One of such filters is an ARM executable file filter, which changes some
bytes in the input stream if the stream is recognized as an executable
file with ARM native code.
This commit fixes the situation when the decompressor using an ARM filter
was referencing a byte outside current window buffer. Such action is
invalid and can produce segmentation faults.
This commit also adds a test using OSSFuzz sample #15431.
The `window_mask` variable should be always in sync with the
`window_size` variable.
The commit fixes a bug in which there was one place that `window_size`
was modified, but `window_mask` wasn't updated. This was leading to a
SIGSEGV error, because by having wrong `window_mask`, RAR5 reader was
accessing the memory outsize current window buffer.
The commit also adds a test for this issue, together with OSSFuzz
sample #15278.
If read_data_compressed() returns ARCHIVE_FAILED, the caller is allowed
to continue with next archive headers. In addition to
rar->start_new_table=1 we need to set rar->ppmd_valid=0.
Reported by: OSS-Fuzz issue 15120
- the -p option does not restore owner by default.
- the -n option was listed twice
- file flags are called file attributes on Linux and are platform-specific
Fixes#1204
Lets ignore autotool generated files (.la .dirstamp .deps) everywhere
rather than hardcoded specific subdirs. We'll never add files with
those names to the source repo, so that should be OK.
We're already ignoring CMakeFiles/ everywhere (since the rule lacks
a leading / anchor), so we can delete the redundant paths.
Rather than hardcode every possible unittest and related files, add
globs that ignore all *_test related paths in the topdir. We won't
be adding paths like that to the source repo, so it should be OK.
The 0x7075 extension includes a crc of the filename that's in the CDE
to make sure that the UTF8 string is always up to date. If an older
tool updates the CDE but doesn't update the 0x7075 field, we want to
ignore the UTF8 string since it's stale.
Add Fedora_29_distcheck task with "make distcheck"
Add support for debug build with address sanitizer
Add missing free to tar/test/test_option_C_mtree.c
If read_data_compressed() returns ARCHIVE_FAILED, the caller is allowed
to continue with next archive headers. We need to set rar->start_new_table
after the ppmd7_context got freed, otherwise it won't be allocated again.
Reported by: OSS-Fuzz issue 2582
The reader was causing a SIGSEGV when the file has been declaring a
specific dictionary size. Dictionary sizes above 0xFFFFFFFF bytes are
overflowing size_t type on 32-bit builds. In case the file has been
declaring dictionary size of 0x100000000 (so, UINT_MAX+1), the
window_size variable effectively contained value of 0. Later, the memory
allocation function was skipping actual allocation of 0 bytes, but still
tried to unpack the data.
This commit limits the dictionary window size buffer to 64MB, so it
always fits in a size_t variable, and disallows a zero dictionary size
for files in the header processing stage.
One unit test had to be modified after this change.
RAR5 structure contains two places where a file can be marked as a
directory. First place is inside the file_flags field in FILE and
SERVICE base blocks, and the second place is inside file_attributes
bitfield also in the same base blocks.
The first directory flag was used to decide if the reader should
allocate any memory for the dictionary buffer needed to unpack the
files. Because if the file is actually a directory, then there should be
nothing to unpack, so if a file was marked as a directory here, the
reader did not allocate any dictionary buffer.
The second directory flag was used to indicate what file attributes
should be passed to the caller. So this second directory flag was used
as an actual indicator what the caller should do during archive
unpacking: should it treat it as a directory, or should it treat it as a
file.
Because of this situation, it was possible to declare a file as a
directory in the file_flags field, but do not declare it as a directory
in the second field, also adding a compressed stream to the FILE/SERVICE
base block. This situation was leading to a condition where the reader
was trying to use unallocated/already freed memory (because it did not
allocate a new dictionary buffer due to the directory flag set in
file_flags).
This commit fixes it so that the reader will check if it tries to
decompress a FILE/SERVICE block that has been declared as a directory in
the file_flags field. If the check will evaluate to true, it will return
an ARCHIVE_FAILED code, because it's not a valid action to take, and
shouldn't exist in valid archives at all.
Also added a unit test for this issue.
This should fix OSSFuzz issue #14574.
This commit also has influenced some of the other unit tests, because it
turned out the sample files used in other tests also did have
inconsistent directory flags in the file_flags and file_attributes
fields. So, some assertions in affected test cases have been changed to
be more relaxed, but still functional.
Whole libarchive uses tab characters to indent scopes. RAR5 reader and
RAR5 reader tests were using space characters for indentation.
Additionally ZIP reader tests was using space indentation only in
specific places, but most of the file used tab character for indent.
This commit converts space indentation characters to tabs.
It does not change anything in logic.
This commit fixes places where the window_mask variable, which is needed
to perform operations on the dictionary circular buffer, was casted to
an int variable.
In files that declare dictionary buffer size of 4GB, window_mask has a
value of 0xFFFFFFFF. If this value will be assigned to an int variable,
this will effectively make the variable to contain value of -1. This
means, that any cast to a 64-bit value will bit-extend the int variable
to 0xFFFFFFFFFFFFFFFF. This was happening during a read operation from
the dictionary. Such invalid window_mask variable was not guarding
against buffer underflow.
This commit should fix the OSSFuzz issue #14537.
The commit also contains a test case for this issue.
RAR5 reader did assume that the block contains full huffman table data.
In invalid files that declare existence of huffman tables, but also
declare too small block size to fit the huffman tables in, RAR5 reader
was interpreting memory beyond the allocated block.
The commit adds necessary buffer overflow checks and fails the huffman
table reading function in case truncated data will be detected.
The commit also provides a unit test for this case.
Should fix OSSFuzz issue #12817.
This commit fixes some undefined shift-left operations on types that do
not support such a big shift. Those invalid shift operations were
triggering on invalid files produced by fuzzing.
The commit also contains two unit tests that ensure such problems won't
arise in the future.
Fixes OSSFuzz cases #14490 and #14491.
RAR5 compresses its Huffman tables by using an algorithm similar to Run
Length Encoding. During uncompression of those tables, RAR5 reader
didn't perform enough checks to prevent from buffer overflow in some
cases.
This commit adds additional check that prevents from encountering a
buffer overflow error in some files.
The commit also adds a unit test to guard against regression of this
issue.
Fixes OSSFuzz case #12466.
This commit fixes a memory leak which is triggered by invalid files.
Sample test case that triggers the leak is provided by OSSFuzz #14470.
If the ZIPX file contanis an LZMA stream, and this stream is invalid,
the reader was allocating an LZMA decoding context which wasn't freed.
Later, when trying to unpack another LZMA stream, context was
re-initialized by allocating a new context and overwriting old pointers
to an unfreed memory, causing a memory leak.
After applying this commit, the LZMA stream context initialization
function will check if there is an non-freed previous context being in
use. If it exists, the reader will free the memory before allocating a
new LZMA unpacking context.
The commit also contains a test case with OSSFuzz sample #14470.
This commit adds support for the VERSION extra field appended to FILE
base block. This field allows to add version support for files inside
the archive. If the file name is 'abc' and its version is 15, libarchive
will unpack this file as 'abc;15'. Changing of file names is needed
because there can be multiple files inside the archive with the same
names and different versions. In order for the user to not be confused
which file is which, RAR5 reader changes the name.
Also this commit contains a unit test for VERSION extra field support.
Another change this commit introduces is ignoring of unknown extra
fields. Before applying the commit, RAR5 reader was failing to unpack
the file if an unknown field was encountered. But since the reader knows
the unknown field's size, it can skip it and ignore it, then proceed
with parsing the structure. After applying this commit, RAR5 reader will
skip and ignore unknown fields.
Unknown fields that are skipped include fields in FILE's extra header,
as well as unsupported REDIR types.
This commit fixes errors reported by ASan, as well as fixes runtime
behavior of RAR5 reader on OSSFuzz sample files:
#12999, #13029, #13144, #13478, #13490
Root cause for these changes is that merge_block() function was
sometimes called in a recursive way. But this function shouldn't be used
this way, because calling it recursively overwrites the global state
that is used by the function. So, the commit ensures the function will
not be called recursively.
There is also one fix that changes some tabs to spaces, because whole
file originally used space indentation.
support reading metadata from compressed files
In particular, this allows gzip + raw to return the filename stored in the gzip header rather than "data"
The raw format provides very little metadata. Allow filters to pass
back state that it knows about. With gzip, we know the original file
name, mtime, and file size. For now, we only pull out the first two
as those are available in the file header. The latter is in the file
trailer, so we'll have to add support for that later (if we can seek
the input).
Rar5 is implemented as an individual format and not a variant of rar
Makes the following functions usable with ARCHIVE_FORMAT_RAR_V5:
archive_read_set_format()
archive_read_support_format_by_code()
The E8E9 filter was accessing the window buffer with a direct memory
read. But since the window buffer is a circular buffer, some of its data
can span between the end of the buffer and beginning of the buffer. This
means that accessing the window buffer needs to be done always by a
reading function that is aware of the fact that the window buffer is
circular.
The commit changes direct memory read to the access through the
circular_memcpy() function.
This fixes some edge cases when the E8E9 filter data (4 bytes) is
spanned between the end of the window buffer and the beginning of the
buffer. This situation can happen in archives compressed with a small
dictionary size.
Testing uses only listing because extraction uses the same code
paths. Indirectly this covers also the new API call.
Some corner cases get special attention:
- archive where a file in a directory is present without the
directory
- the error when asking to extract a directory which is not
present
New functions:
archive_entry_symlink_type()
archive_entry_set_symlink_type()
Suppoted value constants:
AE_SYMLINK_TYPE_UNDEFINED
AE_SYMLINK_TYPE_FILE
AE_SYMLINK_TYPE_DIRECTORY
New extended tar header:
LIBARCHIVE.symlinktype
The function archive_entry_symlink_type() retrieves and the function
archive_entry_set_symlink_type() sets the symbolic link type of an archive
entry. The information about the symbolic link type is required to properly
restore symbolic links on Microsoft Windows. If the symlink type is set
to AE_SYMLINK_TYPE_FILE or AE_SYMLINK_TYPE_DIRECTORY and a tar archive
is written, an extended tar header LIBARCHIVE.symlinktype is stored with
the value "file" or "dir". When reading symbolic links on Windows, the
link type is automatically stored in the archive_entry structure.
On unix systems, the symlink type has no effect when reading or writing
symbolic links.
Sometimes it makes sense to extract or list a directory contained in
an archive without also doing the same for the content of the
directory, i.e. allowing -n (= --no-recursion) in combination with the
x and t modes.
bsdtar uses the match functionality in libarchive to track include
matches. A new libarchive API call
archive_match_set_inclusion_recursion() gets introduced to
influence the matching behavior, with the default behavior as before.
Non-recursive matching can be achieved by anchoring the path match at
both start and end. Asking for a directory which itself isn't in the
archive when in non-recursive mode is an error and handled by the
existing mechanism for tracking unused inclusion entries.
Treat targets ending with /. and /.. as directory symlinks
Explicitly test for file and directory symlinks
Improve debug output on test failure
Fix two memory allocations
If a platform lacks O_EXEC and we get a failure when starting the
traverse from within an unreadable directory, then don't score that as
a failure, since with the current code it can never succeed. But if
O_EXEC exists, the failure still counts.
1. Don't try to open ".." for reading as part of the process of
ascending out of an initially specified directory; it's wrong, and if
the directory is not readable it causes a spurious error.
2. If opening "." initially for reading fails, then open it for
execute instead, if O_EXEC exists. This avoids spurious and unhelpful
failures when the current directory is not readable.
Add test cases for the above.
At least the first of these issues is ancient; it was reported against
FreeBSD in 2014.
Use "undel" for FS_UNRM_FL file flag
Drop compat of UF_NOUNLINK and FS_UNRM_FL
Use "secdel" for FS_SECRM_FL and "journal-data" for FS_JOURNAL_DATA_FL
Fixes#1098
The crash happened on invalid files which declare more data than
actually are stored in the file.
This commit contains a fix that prevents PPMd8 decompressor from
crashing, as well as relevant tests containing files that were
triggering the crash.
Changed indent of some lines from tabs to spaces (because the whole file
is using space indenting) in RAR5 reader,
Removed some superfluous space characters from end of some lines in
RAR5 reader and ZIP reader.
file_new() sets file to NULL first.
when file_new() fails, file is set to NULL if it doesn't need to be freed
so, only free when need to. otherwise would deference a null pointer.
Found this from clang's analyzer
Fixes#1140
clang analyzer found this issue.
other archive_mstring_copy_* has the pattern:
```
if (xxx == NULL) {
aes->aes_set = 0;
return (0);
}
```
archive_mstring_copy_utf8() didn't follow that pattern, so if NULL is passed in, it will call strlen(NULL).
Noticed that archive_mstring_copy_wcs_len() doesn't follow the pattern either.
Fixes#1142
- if not descending, fail if tree_current_lstat() returns ENOENT
- fix the "File removed before we read it" error message if processing multiple files at a time.
Fixes#1082
The function “archive_entry_free” is implemented in the way that only
two functions are called which tolerate the passing of null pointers.
It is therefore not needed that a function caller repeats a corresponding check.
This issue was fixed by using the software “Coccinelle 1.0.7”.
Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
The process_head_file function was using memset() to clear the
archive_entry structure. The problem was that this structure could
contain pointers to allocated blocks of memory, and removing those
pointers with memset() resulted in a memory leak.
Switching it in favor of archive_entry_clear() effectively clears the
structure, but also releases any allocated memory blocks. This removes
the memory leak.
The commit also changes the way a temporary archive_entry instance is
being created when skipping a base block after block merge; instead of
directly creating a new instance on the stack, a constructor function
archive_entry_new() is used to ensure the new archive_entry instance is
not in an inconsistent state. This is needed because the fix described
in the first half of this commit message depends on the archive_entry
instance being in a consistent state due to the call of the
archive_entry_clear() function.
In multi-file RAR5 archives, if a block spans from one file to another,
the RAR5 reader merges both blocks into one, and feeds this merged block
to the decompressor function. The problem is that the block merge
function allocates the exact number of bytes for this block. This is
problematic because when trying to read the last byte from this new
block with bit reader functions, the bit reader functions will reference
few additional bytes right after the byte the caller is trying to read,
resulting in an out of bounds read.
The commit increases the allocation size for new merged block. This
ensures that bit reader functions will never perform any out of bounds
reads. Additional space is zeroed out to prevent errors from
instrumentation tools like ASan or Valgrind.
Fixes#1119
Fuzzing with CRCs disabled revealed that a call to get_uncompressed_data()
would sometimes fail to return at least 'minimum' bytes. This can cause
the crc32() invocation in header_bytes to read off into invalid memory.
A specially crafted archive can use this to cause a crash.
An ASAN trace is below, but ASAN is not required - an uninstrumented
binary will also crash.
==7719==ERROR: AddressSanitizer: SEGV on unknown address 0x631000040000 (pc 0x7fbdb3b3ec1d bp 0x7ffe77a51310 sp 0x7ffe77a51150 T0)
==7719==The signal is caused by a READ memory access.
#0 0x7fbdb3b3ec1c in crc32_z (/lib/x86_64-linux-gnu/libz.so.1+0x2c1c)
#1 0x84f5eb in header_bytes (/tmp/libarchive/bsdtar+0x84f5eb)
#2 0x856156 in read_Header (/tmp/libarchive/bsdtar+0x856156)
#3 0x84e134 in slurp_central_directory (/tmp/libarchive/bsdtar+0x84e134)
#4 0x849690 in archive_read_format_7zip_read_header (/tmp/libarchive/bsdtar+0x849690)
#5 0x5713b7 in _archive_read_next_header2 (/tmp/libarchive/bsdtar+0x5713b7)
#6 0x570e63 in _archive_read_next_header (/tmp/libarchive/bsdtar+0x570e63)
#7 0x6f08bd in archive_read_next_header (/tmp/libarchive/bsdtar+0x6f08bd)
#8 0x52373f in read_archive (/tmp/libarchive/bsdtar+0x52373f)
#9 0x5257be in tar_mode_x (/tmp/libarchive/bsdtar+0x5257be)
#10 0x51daeb in main (/tmp/libarchive/bsdtar+0x51daeb)
#11 0x7fbdb27cab96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
#12 0x41dd09 in _start (/tmp/libarchive/bsdtar+0x41dd09)
This was primarly done with afl and FairFuzz. Some early corpus entries
may have been generated by qsym.
A corrupted or malicious ISO9660 image can cause read_CE() to loop
forever.
read_CE() calls parse_rockridge(), expecting a Rockridge extension
to be read. However, parse_rockridge() is structured as a while
loop starting with a sanity check, and if the sanity check fails
before the loop has run, the function returns ARCHIVE_OK without
advancing the position in the file. This causes read_CE() to retry
indefinitely.
Make parse_rockridge() return ARCHIVE_WARN if it didn't read an
extension. As someone with no real knowledge of the format, this
seems more apt than ARCHIVE_FATAL, but both the call-sites escalate
it to a fatal error immediately anyway.
Found with a combination of AFL, afl-rb (FairFuzz) and qsym.
For every HAVE_LZMA_H check, include also a check for HAVE_LIBLZMA.
This was done in order to fix build issues in case where the 'lzma.h'
file does exist, but the SO library file doesn't exist.
This commit adds some support for extraction of '.zipx' files. Those
files are standard ZIP files that can contain files compressed with
different set of algorithms that standard '.zip' files use.
Support is still missing for Deflate64, JPEG and WavPack algorithms.
PKWare's APPNOTE.TXT simply documents the extra field
as a concatenation of <header><data> objects. It does
not explicitly allow nor prohibit additional padding.
Libarchive originally treated any extraneous data in the
extra field as an indication of corruption and issued an
error.
However, some Zip writers do appear to include additional zero
bytes in the extra field and expect them to be ignored.
This works if:
* We consider a field type of 0000 to be a padding field.
* In particular, 0000 0000 is then a valid field with type zero
and zero bytes of data.
* If there are fewer than 4 bytes left at the end of the extra
data (too few to be a valid extra field object), ignore it.
With the above conventions, zero padding can occur anywhere in
the extra field as long as the padding occurs in a multiple of
four bytes. Any number of bytes can occur at the end: groups
of four bytes will be skipped as valid empty fields; any final
group of less than four bytes will be simply ignored.
The test exercises all of the above options; the extra field data for
the file in this archive has:
* Four zero bytes at the beginning
* A valid UX field
* A pad field of type zero with a non-zero length
* An additional three final bytes
By verifying that the UX data was read correctly and that there
were no errors, we can be confident that all of the padding here
was handled correctly.
GCC8 tries to diagnose `snprintf()` overflows but isn't quite
smart enough for this case, so emits a false-positive warning.
Remember that `%12s` only specifies the minimum number of bytes. GCC8
conservatively assumes this might result in writing the full length of
`date2`. (Which will never be longer than 12 bytes, but GCC8
apparently can't reason about `strftime` format specifiers yet.)
Changing the specifier here to `%12.12s` explicitly truncates to 12
bytes and should help the compiler understand that this will never
overflow.
While I'm here, correct a minor typo in the previous line; it used
`sizeof(date)` instead of `sizeof(date2)`. (Both are the same
size, so this had no functional impact.)
The warc decoder only used read ahead, it wouldn't actually consume
data that had previously been printed. This means that if you specify
an invalid content length, it will just reprint the same data over
and over and over again until it hits the desired length.
This means that a WARC resource with e.g.
Content-Length: 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666665
but only a few hundred bytes of data, causes a quasi-infinite loop.
Consume data in subsequent calls to _warc_read.
Found with an AFL + afl-rb + qsym setup.
Currently, it is possible to create an archive that crashes bsdtar
with a malformed ACL:
Program received signal SIGSEGV, Segmentation fault.
archive_acl_from_text_l (acl=<optimised out>, text=0x7e2e92 "", want_type=<optimised out>, sc=<optimised out>) at libarchive/archive_acl.c:1726
1726 switch (*s) {
(gdb) p n
$1 = 1
(gdb) p field[n]
$2 = {start = 0x0, end = 0x0}
Stop this by checking that the length is not zero before beginning
the switch statement.
I am pretty sure this is the bug mentioned in the qsym paper [1],
and I was able to replicate it with a qsym + AFL + afl-rb setup.
[1] https://www.usenix.org/conference/usenixsecurity18/presentation/yun
Fuzzing uncovered some UAF and memory overrun bugs where a file in a
single file archive reported that it was split across multiple
volumes. This was caused by ppmd7 operations calling
rar_br_fillup. This would invoke rar_read_ahead, which would in some
situations invoke archive_read_format_rar_read_header. That would
check the new file name against the old file name, and if they didn't
match up it would free the ppmd7 buffer and allocate a new
one. However, because the ppmd7 decoder wasn't actually done with the
buffer, it would continue to used the freed buffer. Both reads and
writes to the freed region can be observed.
This is quite tricky to solve: once the buffer has been freed it is
too late, as the ppmd7 decoder functions almost universally assume
success - there's no way for ppmd_read to signal error, nor are there
good ways for functions like Range_Normalise to propagate them. So we
can't detect after the fact that we're in an invalid state - e.g. by
checking rar->cursor, we have to prevent ourselves from ever ending up
there. So, when we are in the dangerous part or rar_read_ahead that
assumes a valid split, we set a flag force read_header to either go
down the path for split files or bail. This means that the ppmd7
decoder keeps a valid buffer and just runs out of data.
Found with a combination of AFL, afl-rb and qsym.
new_size can be 0 with a malicious or corrupted RAR archive.
realloc(area, 0) is equivalent to free(area), so the region would
be free()d here and the free()d again in the cleanup function.
Found with a setup running AFL, afl-rb, and qsym.
The reader has assumed it's running on little-endian. The commit changes
direct memory reads to archive_le* function calls, which should allow
the reader to run on big-endian machines as well.
Changes were needed in the reader itself and in the file holding
reader's test cases.
The commit also removes 1 warning encountered when compiling under GCC
8 on PowerPC architecture.
Fixes#1097
This is to address the following configuration warning reported
when configuring LibArchive passing ZLIB_ROOT option:
CMake Warning (dev) at CMakeLists.txt:393 (FIND_PACKAGE):
Policy CMP0074 is not set: find_package uses <PackageName>_ROOT variables.
Run "cmake --help-policy CMP0074" for policy details. Use the cmake_policy
command to set the policy and suppress this warning.
CMake variable ZLIB_ROOT is set to:
/path/to/zlib-install
For compatibility, CMake is ignoring the variable.
This warning is for project developers. Use -Wno-dev to suppress it.
Co-authored-by: Matt McCormick <matt.mccormick@kitware.com>
During archiving, if some of the "delayed" hard link entries
happened to disappear on filesystem (or become unreadable) for
some reason (most probably race), the old code free()d the 'entry'
and continued with the loop; the next loop though dereferenced
'entry' and crashed the archiver.
Per report from Coverity.
This fixes this CMake warning with CMake 3.12 and newer:
CMake Warning (dev) at /usr/share/cmake/Modules/CheckIncludeFiles.cmake:110 (message):
Policy CMP0075 is not set: Include file check macros honor
CMAKE_REQUIRED_LIBRARIES. Run "cmake --help-policy CMP0075" for policy
details. Use the cmake_policy command to set the policy and suppress this
warning.
CMAKE_REQUIRED_LIBRARIES is set to:
/usr/lib/liblzma.so
For compatibility with CMake 3.11 and below this check is ignoring it.
Call Stack (most recent call first):
CMakeLists.txt:602 (CHECK_INCLUDE_FILES)
CMakeLists.txt:609 (LA_CHECK_INCLUDE_FILE)
Older gcc releases also warn when a local variable has the name of a global
function, like this output from gcc 4.6:
test_read_format_rar5.c:93:23: error: declaration of 'read' shadows a global declaration [-Werror=shadow]
- Fixed a bug during a block-by-block reading loop. Added a test that
checks for the existence of this bug.
- Fixed 2 unlimited loops encountered when unpacking corrupted data.
- Removed some 'maybe uninitialized' warnings.
Symbols added: HAVE_LIBB2, HAVE_BLAKE2_H
RAR5 reader sources expect those symbols to be defined when using system
libb2, but the symbols were missing. This commit fixes that.
identifiers for which there is both a struct and a function. MinGW uses
a #define for setting struct stat to be the right struct, so doing #undef
stat to clear the way for a #define for stat() the function
inadvertantly clobbers the selected stat struct. To avoid this, we
define la_stat() to _la_stat on Windows and stat() elsewhere, and then
use la_stat() instead of stat().
Some ZIP files (certain android AAR packages) include ZIP extra fields
that appear to be padded with 0s. This is technically incorrect, however
other ZIP implementations appear to deal with these type of technically
malformed files. It appears that these broken implementations may be
padding the field to end on a word-aligned boundary.
Instead of generating an error if the length of the extra field is less
than the minimum size (4 bytes), consume the 1 - 3 bytes while checking that
each byte is zero. If all bytes are zero, no error is raised.
The BLAKE2 source files are multi-licensed with the ability to choose
between CC0 1.0 Universal, OpenSSL or Apache 2.0 licenses. For libarchive
the CC0 1.0 Universal Public Domain Dedication should be acceptable.
The problem was a bad return value for the set_option() function. This
commit changes this return value to ARCHIVE_WARN, as there currently are
no options to be handled by the decompressor.
This is an initial implementation of a stream-oriented unpacker. Things
that should work:
- Extraction of any compression level (stored or compressed), file
enumeration, skipping through files,
- Support for any dictionary sizes,
- CRC32 and BLAKE2sp checksums,
- Solid archives,
- Multi-volume archives (part001, part002, etc),
- Solid multi-volume archives,
- DELTA, x86 and ARM filter support: other filters are not used
in version 5 of the format.
Things that are missing from this implementation:
- Encryption support,
- NTFS ACL permission settings,
- Parsing the Reed-Solomon recovery record.
This avoids easily triggerable super-linear performance on malformed
input. It is the first iteration towards better performance for
duplicate entries.
This is an initial implementation of a stream-oriented unpacker. Things
that should work:
- Extraction of any compression level (stored or compressed), file
enumeration, skipping through files,
- Support for any dictionary sizes,
- CRC32 and BLAKE2sp checksums,
- Solid archives,
- Multi-volume archives (part001, part002, etc),
- Solid multi-volume archives,
- DELTA, x86 and ARM filter support: other filters are not used
in version 5 of the format.
Things that are missing from this implementation:
- Encryption support,
- NTFS ACL permission settings,
- Parsing the Reed-Solomon recovery record.
This avoids easily triggerable super-linear performance on malformed
input. It is the first iteration towards better performance for
duplicate entries.
Remove non existent file and add missing file the list of libarchive
sources in Android.mk.
Signed-off-by: Anthony Brandon <anthony@amarulasolutions.com>
No overall change to the code path when root; when not root, ensure
created files are initially user writable if we need to do XATTRs or HFS
compression. Mode is fixed later to the desired final mode in this case.
Net of one extra syscall per file to fix the mode at the end only when
all of these are true:
* Not root
* Final mode is not owner-writable
* XATTRs or HFS compression are needed.
These changes make it unexpectedly pass the xattr test on freebsd.
For non-root users, set xattr first. For root users (who might be
setting security xattrs liable to clear with mode) set mode first.
Certainly needs testing on other platforms, but hits the cases I've
identified so far on Darwin.
Setting the mode on Darwin does not clear xattrs, but xattrs cannot be
set by a non-root user if the mode is not writable. Perform xattrs first
and then mode in _archive_write_disk_finish_entry() on Datwin. Darwin
ACLs are handled later.
Prior change handled --hfsCompression - specific xattrs, this catches
any other (encoded in the archive) attributes.
Some Windows zip archivers, such as PowerShell's Compress-Archive, use
backslash as the path separator in violation of the zip specification.
Info-Zip works around this by supporting this for archives that report
"version made by" of 0 (MS-DOS).
This change adds the same workaround to libarchive and adds a test of
this case using a zip file created with PowerShell.
When CreateFileW fails in next_entry(), it reads from errno to set the
archive error, but errno is not set by CreateFileW. Add a call to
la_dosmaperr to set errno properly first.
Enables unpacking multiple archives into a single directory whose permissions,
owner, and other attributes are pre-configured or otherwise determined ahead
of time by a single archive without the need to repeat the same attributes in
every archive, such as in package installation scenarios.
Signed-off-by: Paul Spangler <paul.spangler@ni.com>
If HAVE_COPYFILE_H is defined and allocating tar via calloc fails, tar
would be dereferenced before the tar == NULL check is done, causing a
NULL pointer dereference. Move the HAVE_COPYFILE_H block after the NULL
dereference check.
These callbacks are just indirections over the standard free & malloc
functions. Delete them and just call free & malloc directly.
This saves 672 .text bytes and 52 .data bytes on x86_64/Linux. Also
avoids needing writable data in the first place.
(1) Do not double free a for the stdin case. Reported by Sean Purcell in
PR #904.
(2) Do not query errors after archive_read_free either, the memory is
gone. Split operation into close and read, reporting errors from the
former.
The SDK for macOS 10.13 adds `futimens` and `utimensat` so our checks
for these symbols may pass. However, the symbols are not available at
runtime on older macOS versions. Instead on macOS we can check for
availability based on the deployment target version.
This follows convention establish by ".tgz", ".txz", etc.
The alias array was not sorted alphabetically (the only order seems to be grouping by decomposed extension), so the new element was just added to the end.
`pkgconf` supports a `Cflags.private` that simplifies static linking
for windows (mingw cross) builds. Users of `pkg-config` will be
unaffected by this as existing build scripts will already set the
`-DLIBARCHIVE_STATIC` as required.
A cpio "newc" archive with a namelength of "FFFFFFFF", if read on a
system with a 32-bit size_t, would result in namelength + name_pad
overflowing 32 bits and libarchive attempting to copy 2^32-1 bytes
from a 2-byte buffer, with appropriately hilarious results.
Check for this overflow and fail; there's no legitimate reason for a
cpio archive to contain a file with a name over 4 billion characters
in length.
Reported by: Eyal Itkin
Security: Corrupt archives can cause libarchive to crash on
32-bit platforms.
Sponsored by: Tarsnap Backup Inc.
The LZX decoder flattens the Huffman tree into a table, but wanted to
limit it to 1k elements to reduce L1 pressure. Since the logic for
filling was flawed and performance argument points more to a bad index
logic, remove the tree logic.
Reported-By: OSS-Fuzz issue 497
The original ustar header has a 100 character limit for the file name.
With the POSIX interchange format, an additional header attribute
contains the full path and libarchive tries to cut it to something
sensible. The GNU sparse file extension on the other hand also likes to
mangle the ustar header field to include a marker and contains yet
another attribute for the original path. If the name was truncated
earlier, this attribute would get the incorrect truncated name.
A bunch of constant variables/tables are missing const markings.
Adding them allows moving more data to the read only sections, and
ends up shrinking writable data a bit by 1k on x86_64/Linux. Not
much, but still nice.
Mac OS X changes:
- add support for extended file attributes via sys/xattr.h
- when extracting an archive entry that has mac_metadata and
mac_metadata is requested to be extracted, extended attributes
are restored only from mac_metadata.
- by default, extended attributes are stored both in mac_metadata and
SCHILY.xattr/LIBARCHIVE.xattr. This is subject to review and change.
To match behavior on other platforms, store extended attributes on
FreeBSD with extattr_set_link() if no fd is provided.
Detection of extended attributes support in configure stage has been
rewritten.
Added xattr platform test to libarchive and xattrs option test to bsdtar.
Richacls are interpreted as NFSv4 ACLs and stored in archive_acl (Richacl
flags and masks are not stored). Analog to mac_metadata, NFSv4 ACLs do not
get extracted when the extraction of extended attributes is enabled and the
"trusted.richacl" extended attribute is present.
RichACL masks and are calculated from file mode on extraction.
mac_metadata acl check has been moved in the code to be together with the
richacl check.
This makes the code much more readable and eases the addition of new
ACL implementatons in the future.
Additional changes:
- most of ACL detection is now done at configure stage
- configuration now reports what ACL was detected
- NFSv4 ACL tests now test INHERIT_ONLY and NO_PROPAGATE_INHERIT
This breaks a static dependency on libbz2. Before this,
any use of libarchive that ended up linking archive_util.c would
require libbz2. (And since archive_util.c contains a bunch of
very core functions, this is pretty much everybody.)
Now you only require libbz2 if you request library version information
or actually use some other feature that involves bzip2 support.
Note that most of the libraries provide version information via
compile-time macros -- libbz2 only provides such information via
a function call, which is why this issue only arises with libbz2.
In particular, the libbz2 version reports the version that we
are currently linked against; the other library versions report
the version we were compiled with (which may not be the same as
what we are currently using).
Rework the mtree_atol10 integer parser so it can parse
INT64_MIN without overflowing the intermediate value.
While here, make this function behave a little more predictably for
too-large input: It now always advances the pointer to the first
non-digit character.
Use mbr_uid_to_uuid() and mbr_gid_to_uuid() for backwards compatibility
instead. These functions have been introduced together with membership.h.
Fixes#882
Previously, a ZIP64 file needed to have a valid classic EOCD header.
This is not the case at least for archives generated by PHPZipStreamer.
This commit allows for a bit more than necessary: While PHPZipStreamer
sets the classic EOCD header to all -1's, the bidding function now
succeeds also if the EOCD header values have arbitrary values, as long
as the ZIP64 EOCD header seems plausible. This simplifies the logic and
should not do any harm in practice.
This fixes#869.
VS 2010 comes with the Win 7 SDK and its bcrypt.h does not define the
`BCRYPT_HASH_REUSABLE_FLAG` macro. Provide the definition locally if
it is not available.
This avoids wrtiting ACLs to tar files twice. Preferred method is
via copyfile(3) (tar option --mac-metadata). If user wants to create
archives with portable NFSv4 ACLs only (SCHILY.acl.ace header with no
GUID data) the --acls and --no-mac-metadata options must be specified
on the command line.
In the manual page be more specific which types of ACLs are stored
with --acls and --mac-metadata options.
Allow enabling and disabling reading or writing of:
Access Control Lists (--acls, --no-acls)
Extended file flags (--fflags, --no-fflags)
Extended attributes (--xattrs, --no-xattrs)
Mac OS X metadata in AppleDouble (--mac-metadata, --no-mac-metadata)
Make --disable-copyfile an alias to --no-mac-metadata.
Starting from glibc 2.25, those calls set errno to ENOENT
when the requested id is not found. So let's stop throwing
warning in this expected case.
Also rework the api of lookup_* functions so it is guaranteed that
lookup_name never returns NULL (unless ENOMEM).
@@ -7,7 +7,7 @@ command-line tools that use the libarchive library.
## Questions? Issues?
* http://www.libarchive.org is the home for ongoing
* https://www.libarchive.org is the home for ongoing
libarchive development, including documentation,
and links to the libarchive mailing lists.
* To report an issue, use the issue tracker at
@@ -23,6 +23,7 @@ This distribution bundle includes the following major components:
* **tar**: the 'bsdtar' program is a full-featured 'tar' implementation built on libarchive
* **cpio**: the 'bsdcpio' program is a different interface to essentially the same functionality
* **cat**: the 'bsdcat' program is a simple replacement tool for zcat, bzcat, xzcat, and such
* **unzip**: the 'bsdunzip' program is a simple replacement tool for Info-ZIP's unzip
* **examples**: Some small example programs that you may find useful.
* **examples/minitar**: a compact sample demonstrating use of libarchive.
* **contrib**: Various items sent to me by third parties; please contact the authors with any questions.
@@ -36,9 +37,13 @@ The top-level directory contains the following information files:
* **CMakeLists.txt** - input for "cmake" build tool, see INSTALL
* **configure** - configuration script, see INSTALL for details. If your copy of the source lacks a `configure` script, you can try to construct it by running the script in `build/autogen.sh` (or use `cmake`).
The following files in the top-level directory are used by the 'configure' script:
*`Makefile.am`, `aclocal.m4`, `configure.ac` - used to build this distribution, only needed by maintainers
*`Makefile.in`, `config.h.in` - templates used by configure script
The following files in the top-level directory are related to the 'configure' script and are only needed by maintainers:
*`configure.ac` - used (by autoconf) to build the configure script and related files
*`Makefile.am` - used (by automake) to generate Makefile.in
*`aclocal.m4` - auto-generated file (created by aclocal) used to build the configure script
*`Makefile.in` - auto-generated template (created by automake) used by the configure script to create Makefile
*`config.h.in` - auto-generated template (created by autoheader) used by the configure script to create config.h
## Documentation
@@ -70,7 +75,8 @@ know about any errors or omissions you find.
## Supported Formats
Currently, the library automatically detects and reads the following fomats:
Currently, the library automatically detects and reads the following formats:
* Old V7 tar archives
* POSIX ustar
* GNU tar format (including GNU long filenames, long link names, and sparse files)
@@ -78,19 +84,22 @@ Currently, the library automatically detects and reads the following fomats:
* POSIX pax interchange format
* POSIX octet-oriented cpio
* SVR4 ASCII cpio
* POSIX octet-oriented cpio
* Binary cpio (big-endian or little-endian)
* PWB binary cpio
* ISO9660 CD-ROM images (with optional Rockridge or Joliet extensions)
* ZIP archives (with uncompressed or "deflate" compressed entries, including support for encrypted Zip archives)
* ZIPX archives (with support for bzip2, zstd, ppmd8, lzma and xz compressed entries)
* GNU and BSD 'ar' archives
* 'mtree' format
* 7-Zip archives
* 7-Zip archives (including archives that use zstandard compression)
* Microsoft CAB format
* LHA and LZH archives
* RAR archives (with some limitations due to RAR's proprietary status)
* RAR and RAR 5.0 archives (with some limitations due to RAR's proprietary status)
* WARC archives
* XAR archives
The library also detects and handles any of the following before evaluating the archive:
* uuencoded files
* files with RPM wrapper
* gzip compression
@@ -99,8 +108,10 @@ The library also detects and handles any of the following before evaluating the
* lzma, lzip, and xz compression
* lz4 compression
* lzop compression
* zstandard compression
The library can create archives in any of the following formats:
* POSIX ustar
* POSIX pax interchange format
* "restricted" pax format, which will create ustar archives except for
@@ -109,22 +120,29 @@ The library can create archives in any of the following formats:
* Old V7 tar format
* POSIX octet-oriented cpio
* SVR4 "newc" cpio
* Binary cpio (little-endian)
* PWB binary cpio
* shar archives
* ZIP archives (with uncompressed or "deflate" compressed entries)
Loaded 100 of 1265 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Dustin L. Howett
Tobias Stoeckmann
Tim Kientzle
dependabot[bot]
Darren Carreras
Malachi Moody
Tobias Stoeckmann
subotac
Dan Nelson
Duncan Horn
datauwu
Peiwei Hu
Jaipaul Cheernam
Dag-Erling Smørgrav
Jin
0xmrniko
Dustin L. Howett
Kaif Khan
Daan De Meyer
Martin Matuska
R4mbb
data
datauwu
Pluto Yang
nvmb3r
cyphercodes
Luca Boccassi
Reyad Attiyat
GeorgH93
XananasX7
i1011
bililateral
isomorph-cyber
magnolia
00redbeer
yPin9andClaude Opus 4.8
Michalis Vasileiadis
Mostyn Bramley-Moore
François Degros
Vladimír Marek
KongQBin
SanjayR
Mark Johnston
Wei-Cheng Pan
br0nzu
Zhihan Zheng
Cris Jacob Maamor
Rudi Heitbaum
Tim Kientzle
ZX41R
Tyler Yankee
Jose Luis Duran
Asaf Meizner
Alejandro Perez
Alexander Lobakin
Benjamin Gilbert
Benoit Pierre
elhananhaenel
ShelpAm
LoboQ1ng
William Pursell
Georgios Patsakas
ZUENS2020
Grzegorz Antoniak
OwenSanzas
Sebastien Bacher
Constantinos Patsakis
Brad King
Vijay D'SilvaandCodeMender
Nagy Tibor
zhangjy1014
FooIbar
Enji Cooper
Josh Carp
AZero13
Leslie P. Polzer
Gabriel Barrantes
Dag-Erling Smørgrav
Alfred Neumayer
vcoxvco
ARJANEN Loïc Jean David
YoshiRulz
Chris McGee
Adam Ciarciński
Илья
Pat Brisbin
Eisuke Kawashima
Chris Brown
Marcin Mikula
Kraust
Alex James
Andrew Gunnerson
Will Fancher
Dan Yeaw
Hemant
Raman Shyshniou
Federico Giovanardi
Kyle Evans
Steve Lhomme
Zhaofeng Li
Heiko Becker
Nicholas Vinson
Marcus TillmannsandMartin Matuška
Moroshima
James Hilliard
Brian Campbell
Dmitry Ivankov
Graham Percival
zhongfly
Lukas Javorsky
Christoph Grüninger
Peter Kokot
nia
Nicholas Vinson
Martin Matuška
Peter Kästle
Stepan Tsepa
Jessica Clarke
Sam James
Silent
Stephane ChazelasandStephane Chazelas
Zeun
Julian Uy
Alexander Ziaee
MithicSpirit
nielash
unxed
vcoxvcoandvco
Emil Velikov
Christian Schmitz
Duncan Horn
Michał Górny
yyyy
наб
jet
terryniniandTimothy Lyanguzov
Grzegorz Antoniak
Sam Bingner
Fatima Qarni
alice
Mrmaxmeier
Sevan Janiyan
LukeSkyD
evan-swinney
Adrian Iain Lam
Sam James
jvreelanda
Spacefish
Ed Maste
Alexandr Reshetnikov
asolwaandasolawa
Collin Funk
Mark Johnston
Elvis Angelaccio
tnias
Matt Smith
Haelwenn Monnier
nooriro
grembo
Aaron Lindros
Klaus Holst Jacobsen
YAMASHINA Hio
Alfred Wingate
Jeffrey Walton
Brooks Davis
Roland ClobusandMartin Matuska
Pedro Kaj Kjellerup Nacht
tomaThomas
Jarred SumnerandJarred Sumner
Martin Matuska
Samuel Marks
Luke Rewega
Joshua RootandToby Peterson
Adrian Vovk
Wong Hoi Sing Edison
Mingye Wang
Sarah Gilmore
Pedro NachtandMartin Matuska
Albert Jin
Peter Kaestle
Kai
Luke Mewburn
Enji Cooper
Dimitry Andric
Xin "Russell" Liu
Po-Chuan Hsieh
Li kunyu
Bernhard M. Wiedemann
Bernhard M. Wiedemann
Rosen Penev
Peter Pentchev
TERESH1
lgtm-com[bot]andLGTM Migrator
Vladimir Kikhtenko
banjiuqingshan
Joris Clement
Julien Voisin
Erik Olofsson
Vincent Torri
Eric van Gyzen
Ewgeni Wolowik
John Reiser
Yuri Gribov
Khem Raj
Ben Wagner
obiwac
Joel Uckelman
Sergey Bobrenok
Sean McBride
Jan Starý
Zack Weger
tarsin
BogDan Vatra
Gaël PORTAY
Reshetnikov Alexandr
David Macek
Biswapriyo Nath
wangkerong
Christian Hesse
Michael Osipov
Ken Matsui
Mateusz Piotrowski
cielavenir
Jung-uk Kim
Younes El-karama
Jairo
Alexey Pelykh
Alex Richardson
Zdenek Zambersky
Petr Malat
Peter Pentchev
Walter Lozano
linear cannon
Todd Richmond
Charly C
Jonas Witschel
Theo Buehler
Ryan Libby
jiat75
AdamKorcz
JFranklin13
Joerg Sonnenberger
IohannRabeson
Alex Xu
Samanta Navarro
Russell Greene
Andy Brown
Adrian Ebeling
Owen W. Taylor
Christos Zoulas
Rolf Eike Beer
Ondrej Dubaj
jo620kix
Russell Mullens
Masalskaya, Anna
Tom Ivar Helbekkmo
Oliver Ford
Oleg Smirnov
Alexandre Janniaux
r0ptr
Anton Kochkov
Shawn Webb
Luis Henriques
lutianxiong
Cristian Rodríguez
Kirill Zhumarin
Yichao Yu
Steve Pieper
Marc Espie
Michael Forney
Gleb Fotengauer-Malinovskiy
goshhhy
Stefan Baranoff
Terrell Russell
David Capello
Eric Yu
Harry Mallon
Matúš Olekšák
Johannes 'josch' Schauer
Mikhail Novosyolov
Justin Brewer
Kamil Rytarowski
Ward Vandewege
Claybird
Pawel Jakub Dawidek
Krystian Wojcicki
Daniel Verkamp
Ingo Schwarze
FeRD (Frank Dana)
Maxime Le Moine
John Marino
Dmitry Torokhov
Mike Frysinger
Alexander Chaikovsky
Martin Matuska
Tomasz Mikolajewski
Ed Catmur
Patrick Ohly
Andrew Gierth
Pavel Raiskup
Patrick Cheng
János Illés
Eugene Grossbein
Ichiro
Markus Elfring
Arnadu Botella
Jakub Kulík
Daniel Axtens
Li-Wen Hsu
Gerald NDA
Jean-Christophe Fillion-RobinandMatt McCormick
Guido Schulz
Young_X
Joe Damato
John Starks
NamedKitten
Mike Ellery
Anthony Brandon
Kylie McClain
Eric Borisch
Pablo Hernandez-Cerdan
Don Meyer
Bernard Spil
Arshan Khanifar
Oleg Shparber
BenjaminTrapani
Omar Farooq
John Starks
Wei-Cheng Pan
Paul Spangler
ngie
Mike Frysinger
Jerome Duval
jrmarino
Tony Theodore
Colin Percival
Chris Roberts
Sean Purcell
Sean Purcell
Abhinav Upadhyay
Zhao Yongming
Phillip Berndt
codarrenvelvindron
J. Peter Mugaas
Jakub Wilk
tpg